RT-AX86U - Jumbo frames for 2.5GbE port.

[ColinTaylor]

So the switch inside pro version is not even passing MTU bigger than 1492 bytes to other clients connected to switch in pro version.

Maybe you can confirm with the following router command:

# ethswctl -c jumbo -p 9
JUMBO_PORT_MASK:0x010001FF
  GPHY_0 port accepts jumbo frames.
  GPHY_1 port accepts jumbo frames.
  GMII_1 port accepts jumbo frames.
  GMII_2 port accepts jumbo frames.
  GPON_SERDES port accepts jumbo frames.
  MOCA port accepts jumbo frames.
  USB port accepts jumbo frames.
  GPON port accepts jumbo frames.
  MIPS port accepts jumbo frames.
Success.

 

[Spc]

RT-AX86U Original version:

admin@RT-AX86U-5EC8:/tmp/home/root# ethswctl -c jumbo -p 9
JUMBO_PORT_MASK:0x010001FF
  GPHY_0 port accepts jumbo frames.
  GPHY_1 port accepts jumbo frames.
  GMII_1 port accepts jumbo frames.
  GMII_2 port accepts jumbo frames.
  GPON_SERDES port accepts jumbo frames.
  MOCA port accepts jumbo frames.
  USB port accepts jumbo frames.
  GPON port accepts jumbo frames.
  MIPS port accepts jumbo frames.
Success.

RT-AX86U PRO version:

admin@RT-AX86U_Pro-B120:/tmp/home/root# ethswctl -c jumbo -p 9
ioctl command return error!
Failure

 

[Spc]

If i try to force more then 1500 MTU it is rejected by driver itself.

 

[DJones]

LAN jumbo frames over the Ethernet switch is I think rated up to 16000 MTU so any jumbo frames within that will be passed through. I typically use 9014 or 9000, 9000 typically is what most windows clients will handle although certain Ethernet LAN pcie cards can be set higher up to 16000 MTU. It's better to keep all clients that are dealing with file transfers peer to peer at the same MTU's to avoid fragmentation. Just note that WAN MTU and LAN MTU, and WLAN MTU are different. Jumbo frames are for Ethernet only. WLAN will have a MTU of 1500 always so if your client passed through any wireless it's going to fragment it down to 1500 MTU even if your Node has jumbo frames enabled. And if your file transfers or general internet uses WAN then it will fragment down to 1500 or 1492 depending on internet type when leaving towards the internet.

Large jumbo frames are optimal for larger files, but not optimal for smaller files. Either way you only really have to worry about that if your transferring files peer to peer.

 

[Spc]

LAN jumbo frames over the Ethernet switch are rated up to 16000 MTU so any jumbo frames within that will be passed through. I typically use 9014 or 9000, 9000 typically is what most windows clients will handle although certain Ethernet LAN pcie cards can be set higher up to 16000 MTU. It's better to keep all clients that are dealing with file transfers peer to peer at the same MTU's to avoid fragmentation. Just note that WAN MTU and LAN MTU, and WLAN MTU are different. Jumbo frames are for Ethernet only. WLAN will have a MTU of 1500 always so if your client passed through any wireless it's going to fragment it down to 1500 MTU even if your Node has jumbo frames enabled. And if your file transfers or general internet uses WAN then it will fragment down to 1500 or 1492 depending on internet type when leaving towards the internet.

Large jumbo frames are optimal for larger files, but not optimal for smaller files. Either way you only really have to worry about that if your transfering files peer to peer.

Yea i am talking about local jumbo frames, ethernet switch only, not wan port or NAT translation.
WAN port and NAT translation need MTU of 1500 or 1510 max for vpn overhead.

 

[DJones]

Honestly I'd just set your clients to 9K MTU and forget about it. BUT if your using a dongle on the router your likely going to be capped at 1500 MTU. (I'm assuming your using the dongle on the computer not the router, but if your using it with the router and it's supported then this is likely the case. If your using it on the computer set it to 9K and forget. ) The Jumbo frames setting on the routers are for the built in LAN only. When your using a dongle with the router your likely subject to bridging which will always be set to the smallest MTU which will be your WAN.

 

[DJones]

Also for windows I would download TCP optimizer and set MTU to 9000 from there as well. https://www.speedguide.net/downloads.php

 

[Spc]

Honestly I'd just set your clients to 9K MTU and forget about it. BUT if your using a dongle on the router your likely going to be capped at 1500 MTU. (I'm assuming your using the dongle on the computer not the router, but if your using it with the router and it's supported then this is likely the case. If your using it on the computer set it to 9K and forget. ) The Jumbo frames setting on the routers are for the built in LAN only. When your using a dongle with the router your likely subject to bridging which will always be set to the smallest MTU which will be your WAN.

Yes i am talking about my local network, i know internet will have MTU of 1500 and for WAN "internet" more than 1500 is useless.
In my local network however I want to have 9k over RT-AX86U PRO switch, not wan port.

Tomorrow i'll test how much MTU can be forwarded via local switch on original version.
I still have some intel PRO MT server adapters laying around that have 16k jumbo frames.

I'll let you know if the limit is 9k or is it more as BCM84880 supports 18k:

BCM84880

The Broadcom® BCM84880 is a single-port 5GBASE-T/2.5GBASE-T/1000BASE-T/100BASE-TX Ethernet CMOS transceiver.

www.broadcom.com

ASUS RT-AX86U

For a list of all currently documented Broadcom chipsets with specifications, see Broadcom. AX5700 Dual Band WiFi 6 Gigabit Router Wireless-AX5700 Dual-band...

wikidevi.wi-cat.ru

 

[DJones]

If you have a NIC that supports MTU 18K then I think the hard limit is 65536 for forwarded packets, but I could be mistaken. I don't have higher then 16K NIC's but doesn't surprise me 5 or 10GBase would support higher.

 

[Spc]

I tested it using hardware from the same vendor, so we get same hardware / vendor across all 3 devices.
USB Network cards used: ASUS USB-C2500 2.5Gbit/s, jumbo frames enabled 16k

ping 10.0.0.223 -f -l 9674

Pinging 10.0.0.223 with 9674 bytes of data:
Reply from 10.0.0.223: bytes=9674 time=<1ms TTL=128
Reply from 10.0.0.223: bytes=9674 time=<1ms TTL=128
Reply from 10.0.0.223: bytes=9674 time=<1ms TTL=128
Reply from 10.0.0.223: bytes=9674 time=<1ms TTL=128

Ping statistics for 10.0.0.223:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

ping  10.0.0.223 -f -l 9675

Pinging 10.0.0.223 with 9675 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

So original version gets support up to 10k jumbo frames, 9674bytes + headers.

 

[Spc]

I've also tested LACP, LAN1+LAN2 in aggregation.

RT-AX86U (Original) passes through jumbo frames up to 10K in LACP 2Gbit/s interface.
RT-AX86U PRO stops at 1472 MTU in LACP 2Gbit/s interface.

So LACP is also affected by this MTU change.

 

[DJones]

Pinged my server which uses LACP to the router x2 1000Base-T at 9000MTU which passes at 8326 10GBase-T NIC set to 9014 and windows MTU set to 9000.

*Caveat PC's 10GBase-T goes to another 10GBase-T SFP+ switch which supports 16K MTU pass through then into the 2.5G port on my GT-AX11000 then LACP on LAN1+LAN2 of my router then into my server which is also set to 9000 MTU.

At least on mine it accepts larger then 1500 MTU through aggregation. I used to use a 5GBase-T USB dongle on my server, but for whatever reason it would occasionally cause the usb 3.0 pcie port to disconnect so I resorted back to using 2 of the 6 front 1000Base-T ports in LACP. Unfortunately samba's SMB direct or multichannel doesn't work unless your directly connected PC->Server which causes considerable slow down as LACP isn't utilized by Samba meaning LAN2 is just failover 99% of the time as my WAN port is 1000Base-T only.

 

[Spc]

So ASUS GT-AX11000 also accepts jumo frames.
Do you still have "Enable Jumbo Frames" in LAN Switch options ?

 

[DJones]

So ASUS GT-AX11000 also accepts jumo frames.
Do you still have "Enable Jumbo Frames" in LAN Switch options ?

Yes it’s enabled. It’s the non pro version so perhaps something has changed in the pro versions which usually have a newer kernel.

 

[Spc]

Yes it’s enabled. It’s the non pro version so perhaps something has changed in the pro versions which usually have a newer kernel.

Yes, new chipset, new cpu with different/new drivers that do not support jumbo frames ...

 

[DJones]

Baffling that jumbo frames wouldn’t be supported.

 

[Spc]

Baffling that jumbo frames wouldn’t be supported.

I agree.
It is very strange to not support jumbo frames at 2.5GbE - 10GbE speeds when $30 ASUS 2.5Gbit USB-C2500 dongle supports 16k

 

[DJones]

Jumbo frame ironically would be better performance for the cpu at those speeds as it would have to send less packets. It’s possible they might roll out jumbo frames later. But Asus isn’t really known for their software and it would be dependent on Broadcoms sdk drivers.

 

[Spc]

Jumbo frame ironically would be better performance for the cpu at those speeds as it would have to send less packets. It’s possible they might roll out jumbo frames later. But Asus isn’t really known for their software and it would be dependent on Broadcoms sdk drivers.

Yes, like you said it's all on broadcom SDK drivers this time, it's up to them to release jumbo frame support.