Capturing inbound/outbound traffic that was (somewhat) split through two interfaces with tcpdump

[drmnsamoliu]

I have an ASUS RT AC-66U Wi-Fi router with stock firmware which works in router mode with ppp connection to ISP.

Recently when I was trying to capture network traffics on the router with tcpdump, I realized that most outbound traffic can only be captured on eth0 and most inbound traffic can only be captured on eth1.
I also observe that the outbound traffic have the router's public IP address (X.X.195.197) as src and inbound traffic have the local IP address of the device (192.168.2.154) who initialize the connection.

Following are some images from Wireshark
This is capture from eth0, we can see that there are two Client Hello with seemingly no Server Hello response:

This is capture from eth1, which has the two corresponding Server Hello response with no Client Hello:

Using the mergecap utility, I can get the full picture of TLS handshake, just with messed up src and dst:

From the information I can collect, the router has vlan setup because of the ppp connection:

I'm not familiar with how ppp and vlan works in low level so I need some clue about how and why this inbound/outbound separation on two interfaces occurred or is needed.

Also, another weird behavior is that capture using tcpdump -i any won't be able to capture most outbound traffic that will show up on eth0 either.
Where can I start looking into to at least be able to capture a "normal" LAN IP <-> Internet IP like capturing in access point mode or on a PC?

 

[ColinTaylor]

IIRC:

vlan1 is the router's physical LAN switch ports. eth1 and eth2 are the 2.4 and 5GHz radios respectively. These are all bridged together into br0 which is the "LAN interface".
eth0 is the WAN interface.

I see you also have vlan500 but I don't know what that is. It could be an IPTV vlan or part of using PPPoE. The PPPoE tunnel is usually called something like ppp0.

Be aware that if you have hardware acceleration enabled most of the traffic bypasses the network stack so you won't be able to capture it with tcpdump.

I've not used PPPoE on that router so I'm unsure of any particular quirks it might throw up.

 

[drmnsamoliu]

IIRC:

vlan1 is the router's physical LAN switch ports. eth1 and eth2 are the 2.4 and 5GHz radios respectively. These are all bridged together into br0 which is the "LAN interface".
eth0 is the WAN interface.

I see you also have vlan500 but I don't know what that is. It could be an IPTV vlan or part of using PPPoE. The PPPoE tunnel is usually called something like ppp0.

Be aware that if you have hardware acceleration enabled most of the traffic bypasses the network stack so you won't be able to capture it with tcpdump.

I've not used PPPoE on that router so I'm unsure of any particular quirks it might throw up.

Really appreciate for your knowledge!

Yes there is a ppp0 interface and if I understand correctly vlan500 is there because of selected ISP profile. The ISP requires vlan500 for internet connection.

About hardware acceleration, do you know where I can check to see if it's enabled?

 

[ColinTaylor]

About hardware acceleration, do you know where I can check to see if it's enabled?

I don't know in stock firmware. Try looking at LAN - Switch Control.

 

[ColinTaylor]

[LAN] How to set up Switch Control on ASUS Router? | Official Support | ASUS USA

 

[drmnsamoliu]

I don't know in stock firmware. Try looking at LAN - Switch Control.

Words can't show my gratitude Colin, it IS NAT acceleration affecting packet capture!
After disabling it I can capture the full TLS handshake now.

Now I'm going to look into how Broadcom implement their hardware acceleration because it looks kinda like a magic to me that packets can just slip through kernel but somehow still works fine.