AX86U Wireguard Blocking https

[lipangeng]

The device is ax86u and the original firmware is 3.0.0.4.388_21709.Using the WireGuard channel blocked https access and bothered me for two weeks.
1.TCP channels can be established, but HTTPS cannot be established. After Client hello is sent, no response is received. But the http protocol is completely normal.
2.Using tcpdump monitoring, we know that interface eth0 of 10.10.1.51 and interface br0 of ax86u can receive the returned content. Interface eth0 of 10.5.241.52 is not received.

10.5.32.67(curl) -> 10.5.241.52 (wireguard)-> Internet -> ax86u -> 10.10.1.51(Server)

Please help me. It's driving me crazy.

Pictures-----
https:

HTTP

 

[sfx2000]

1.TCP channels can be established, but HTTPS cannot be established. After Client hello is sent, no response is received. But the http protocol is completely normal.

Not a routing issue, because you have already contacted the host...

try forcing the fallback to tls 1.2...

curl -vvv --tlsv1.2 --tls-max 1.2 https://10.10.1.51

If this works, could be your curl build, or your openssl on either side, as curl calls openssl for ciphering/encryption

 

[lipangeng]

Not a routing issue, because you have already contacted the host...

try forcing the fallback to tls 1.2...

curl -vvv --tlsv1.2 --tls-max 1.2 https://10.10.1.51

If this works, could be your curl build, or your openssl on either side, as curl calls openssl for ciphering/encryption

Thanks for helping me, but it didn't work.

 

[lipangeng]

Server under ax86u, https access is normal.
All servers installed openssl/stable,stable-security,now 1.1.1n-0+deb11u3 amd64.
All server linux kernel versions are 5.10.0-19-amd64

 

[lipangeng]

I saved an iptables configuration using iptables-save. I'm not very proficient and I don't see anything particularly unusual.

 

[sfx2000]

Thanks for helping me, but it didn't work.

What does your server log say?

 

[sfx2000]

You might also find this handy to have...

GitHub - noxxi/p5-ssl-tools: various standalone perl scripts

various standalone perl scripts. Contribute to noxxi/p5-ssl-tools development by creating an account on GitHub.

github.com

The connection reset by peer is also a hint - the server is explicitly closing the connection

 

[lipangeng]

What does your server log say?

I set the debug log for ingress-nginx, Use curl -vvv -k --resolve dhubr.lipg.cn:443:10.10.1.51 https://dhubr.lipg.cn/v2/bjzoo/zoo-service/manifests/latest

Nginx:

2022/12/16 15:34:12 [debug] 27#27: *85611 accept: 10.5.32.67:35794 fd:14
2022/12/16 15:34:12 [debug] 27#27: *85611 event timer add: 14: 60000:1627112682
2022/12/16 15:34:12 [debug] 27#27: *85611 reusable connection: 1
2022/12/16 15:34:12 [debug] 27#27: *85611 epoll add event: fd:14 op:1 ev:80002001
2022/12/16 15:34:12 [debug] 27#27: accept() not ready (11: Resource temporarily unavailable)
2022/12/16 15:34:12 [debug] 27#27: *85611 http check ssl handshake
2022/12/16 15:34:12 [debug] 27#27: *85611 http recv(): 1
2022/12/16 15:34:12 [debug] 27#27: *85611 https ssl handshake: 0x16
2022/12/16 15:34:12 [debug] 27#27: *85611 tcp_nodelay
2022/12/16 15:34:12 [debug] 27#27: *85611 reusable connection: 0
2022/12/16 15:34:12 [debug] 27#27: *85611 SSL server name: "dhubr.lipg.cn"
2022/12/16 15:34:12 [debug] 27#27: *85611 SSL ALPN supported by client: h2
2022/12/16 15:34:12 [debug] 27#27: *85611 SSL ALPN supported by client: http/1.1
2022/12/16 15:34:12 [debug] 27#27: *85611 SSL ALPN selected: h2
2022/12/16 15:34:12 [debug] 27#27: *85611 ssl cert: connection reusable: 0
2022/12/16 15:34:12 [debug] 27#27: *85611 lua_certificate_by_lua: handler return value: 0, cert cb exit code: 1
2022/12/16 15:34:12 [debug] 27#27: *85611 SSL_do_handshake: -1
2022/12/16 15:34:12 [debug] 27#27: *85611 SSL_get_error: 2
2022/12/16 15:35:12 [debug] 27#27: *85611 event timer del: 14: 1627112682
2022/12/16 15:35:12 [debug] 27#27: *85611 SSL handshake handler: 0
2022/12/16 15:35:12 [info] 27#27: *85611 client timed out (110: Operation timed out) while SSL handshaking, client: 10.5.32.67, server: 0.0.0.0:443
2022/12/16 15:35:12 [debug] 27#27: *85611 close http connection: 14
2022/12/16 15:35:12 [debug] 27#27: *85611 reusable connection: 0
2022/12/16 15:35:12 [debug] 27#27: *85611 free: 00007F78D3494D60, unused: 56

 

[lipangeng]

You might also find this handy to have...

GitHub - noxxi/p5-ssl-tools: various standalone perl scripts

various standalone perl scripts. Contribute to noxxi/p5-ssl-tools development by creating an account on GitHub.

github.com

The connection reset by peer is also a hint - the server is explicitly closing the connection

It is normal without going through the WireGuard.I have no idea about this error.

 

[sfx2000]

just on a whim - try reducing maxMTU size on the router to 1360 - it could be that with WG encapsulation over the tunnel, the packets are getting fragmented and causing the timeout.

 

[lipangeng]

just on a whim - try reducing maxMTU size on the router to 1360 - it could be that with WG encapsulation over the tunnel, the packets are getting fragmented and causing the timeout.

mtu is amazing.
WireGuard interface default 1420, set to 1500 actually worked.
All the others are 1500, only WireGuard is 1420,But I don't understand why only https doesn't work?

br0       Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          inet addr:10.10.0.1  Bcast:10.10.1.255  Mask:255.255.254.0
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:32196830 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28232419 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:19731755817 (18.3 GiB)  TX bytes:8006752557 (7.4 GiB)

br1       Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C1 
          inet addr:192.168.101.1  Bcast:192.168.101.255  Mask:255.255.255.0
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:508 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1023 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:23396 (22.8 KiB)  TX bytes:62883 (61.4 KiB)

eth0      Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:20944572 errors:0 dropped:73391 overruns:0 frame:0
          TX packets:27641886 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:6227018390 (5.7 GiB)  TX bytes:20794913451 (19.3 GiB)

eth1      Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth1.501  Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth2      Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth2.501  Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth3      Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth3.501  Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth4      Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth4.501  Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth5      Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth5.501  Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth6      Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:158526 errors:0 dropped:1 overruns:0 frame:7124164
          TX packets:4203633 errors:38972 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21630439 (20.6 MiB)  TX bytes:1320090945 (1.2 GiB)
          Interrupt:48

eth6.0    Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:359708 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2062983 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:61196803 (58.3 MiB)  TX bytes:638131036 (608.5 MiB)

eth6.501  Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C0 
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:185373 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:10394619 (9.9 MiB)

eth7      Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C4 
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:105 errors:0 dropped:24 overruns:0 frame:0
          TX packets:4026034 errors:0 dropped:1323 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:13256 (12.9 KiB)  TX bytes:1268115458 (1.1 GiB)

eth7.0    Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C4 
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:91316 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1920009 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:19127914 (18.2 MiB)  TX bytes:625696998 (596.7 MiB)

eth7.501  Link encap:Ethernet  HWaddr 24:4B:FE:D4:02:C4 
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:185373 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:10394619 (9.9 MiB)

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING MULTICAST  MTU:65536  Metric:1
          RX packets:1633297 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1633297 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:447644135 (426.9 MiB)  TX bytes:447644135 (426.9 MiB)

lo:0      Link encap:Local Loopback 
          inet addr:127.0.1.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING MULTICAST  MTU:65536  Metric:1

spu_ds_dummy Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 
          UP RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

spu_us_dummy Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 
          UP RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

wgc4      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 
          inet addr:10.10.102.2  P-t-P:10.10.102.2  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:2587 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13248 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:239508 (233.8 KiB)  TX bytes:679948 (664.0 KiB)

wgc5      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 
          inet addr:10.10.100.2  P-t-P:10.10.100.2  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP  MTU:1420  Metric:1
          RX packets:5810774 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6114308 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1001911396 (955.4 MiB)  TX bytes:2355781040 (2.1 GiB)