Forwarding Performance and Switching Capacity of RT-AC68U

bengalih

Regular Contributor
I've noticed that when I do large file copies between two wired systems (SystemA and SystemB) that my Wifi access to the internet is slowed.

These system are connected to LAN ports 1 and 2 respectively, which are GB ports. My file copy software reports speeds of about 108 MB/s which translates to about 865 mbps.
Count in the overhead of SMB traffic, and I am probably pretty close to saturating a 1 gbps connection.
However, this being a switch that traffic should be isolated between those two ports.
Looking at the router CPU/memory when the copy is going it does not appear to be near critical capacity.

But, when this copy is happening, all my Wifi traffic - even that straight from the Internet is slowed down horribly (i.e. a file that would normally take < 1 minute will take 2 hours).

If I introduce an external switch and connect these two system to that switch, then connect it into a port on the ASUS, I do not have this issue.

Is this a pure hardware limitation on the Asus?
 

L&LD

Part of the Furniture
What options, features and scripts are used? Which firmware? When was the last time a full reset to factory defaults performed?

With an over 8 year old device, the hardware, firmware, SDK level and possibly other factors are all suspect.

The possible suspects above are all reasons why a new(er), AX class router, always makes a network, as a whole, simply better. Even with no AX class devices currently in use.

If a full reset is performed after flashing the latest firmware (RMerlin, recommended), then doing a minimal and manual configuration doesn't fix this problem in your environment, a new router may be indicated as the 'fix' . Depending on what others input may produce for your issue.

Of course, with the external switch already working to fix this issue for now, no hardware upgrade is actually needed, today.
 

bengalih

Regular Contributor
What options, features and scripts are used? Which firmware? When was the last time a full reset to factory defaults performed?

With an over 8 year old device, the hardware, firmware, SDK level and possibly other factors are all suspect.

The possible suspects above are all reasons why a new(er), AX class router, always makes a network, as a whole, simply better. Even with no AX class devices currently in use.

If a full reset is performed after flashing the latest firmware (RMerlin, recommended), then doing a minimal and manual configuration doesn't fix this problem in your environment, a new router may be indicated as the 'fix' . Depending on what others input may produce for your issue.

Of course, with the external switch already working to fix this issue for now, no hardware upgrade is actually needed, today.

I'd rather not keep the switch there if unnecessary, which is why I am looking for an answer from others with similar devices whether or not they experience this problem (i.e. is it expected with this device). The device was reset when I updated to 386.3_2 which coincided with my flash drive failing and needed to be rebuilt.

I run skynet and have a lot of other configuration changes and scripts, but I'm not sure how relevant any of it is if I don't see the performance counters indicating an issue.

Upgrading is always a solution to better performance, but not something I plan to do at this point. However, if this is a problem that can be resolved, and not a hardware/design limitation of the router I would like to know.
 

tgl

Regular Contributor
The only way that 1Gb/s traffic between two LAN ports would be "free" is if the hardware designers had included dedicated hardware to make it so --- otherwise it's going to go through the router's main data transfer pathways and, evidently, clog those pretty thoroughly. It doesn't surprise me at all that ASUS didn't include such hardware, because they wouldn't see that use-case as being a primary use-case for an inexpensive wireless router. I wouldn't bet on their newer AX gear including it, either. If this is a primary use-case for you then putting that switch in front of the router to provide the dedicated pathway you need is exactly the right fix.
 

bengalih

Regular Contributor
The only way that 1Gb/s traffic between two LAN ports would be "free" is if the hardware designers had included dedicated hardware to make it so --- otherwise it's going to go through the router's main data transfer pathways and, evidently, clog those pretty thoroughly. It doesn't surprise me at all that ASUS didn't include such hardware, because they wouldn't see that use-case as being a primary use-case for an inexpensive wireless router. I wouldn't bet on their newer AX gear including it, either. If this is a primary use-case for you then putting that switch in front of the router to provide the dedicated pathway you need is exactly the right fix.

While I agree with the primary proposition - I'm not sure I buy the fact that it can't do it because it is "inexpensive". Even back when this model was being sold, it wasn't uncommon to be able to purchase a 5 port gigabit switch for less than half the cost of this router that would have a 10gbps switching capacity (5 ports - full duplex). While I am unable to test it currently, I would bet that if I plugged a SystemC and SystemD into LAN ports 3 & 4 I could possible get near gigabit speeds there while my other systems on 1 & 2 are also reaching those speeds.

This could be more of a packet forwarding rate issue and something that affects the WiFi specificially(?)

I have gig fiber internet, but am unable to reach full 1gb download speeds with the ASUS. On a good day I get in the mid to high 800s on wired clients and when I do I can see the CPU cores averaging in the low 90s. So there is clearly a CPU limitation that is preventing me from downloading at full GB when the WAN interface has to do the switching.

However, when transferring between LAN ports 1 & 2, I don't see CPU usage anywhere in this range which leads me to believe that it isn't a bottleneck. However the WAN speeds definitely decrease.

EDIT: It is possible on doing some more tests writing this that I was incorrect as to the severity of the WiFi speed decrease. I have experienced issues in the past when doing these long running file copies over LAN where my streaming media would buffer severely. This was to be expected since the streaming server was also involved in the file copy - so it would obviously suffer if it was already copying files near it's throughput capacity. Tonight however my daughter was having problems downloading some applications while on WiFi during my file copy. It appeared I was also experiencing this when I tried from my laptop. When I turned off the LAN file copy it appeared that the file transfer occurred with expected speed.
In doing some additional tests now however with the auxiliary switch removed, I seem to be able to download at normal speeds while the file transfer is occurring between the LAN ports.
So I may have jumped the gun with my accusations here. I'm going to keep the switch removed for a while and see if everything continues to look good.
 

tgl

Regular Contributor
I'm not buying the CPU load figures as being a relevant number. It's certain that there is some amount of switching fabric built into the router hardware, because otherwise the CPU would be saturated just trying to push bits from port X to port Y. So I think that "pushing bits" doesn't reflect into the CPU load report. My point is that it's unlikely that the switching fabric is a full NxN crossbar. That'd cost some money and it wouldn't produce numbers that ASUS could advertise. They probably made sure that they could pump bits at max rate between the WAN port and the radios, and maybe also between the WAN port and any one LAN port, and shaved a few bucks on cost otherwise.
 

heysoundude

Part of the Furniture
I can only assume you're currently running a swap on your entware device along with those scripts, so yeah, an AC68 still passing data at 865Mbps between wired clients is in itself impressive. But given how fast clients are and are becoming, AND with your Gigabit connection, it's time to put some more capable processing at the nexus of your network. Even an AC86 would be an upgrade, but an AX68 would be a better choice for future-proofness and not that much more money.
The kernel/code have improved..everything. the new AX machines are not insignificant improvements over your venerable AC model; you'll see an immediate difference...even when you turn the AC into an AP or mesh node.
 

bengalih

Regular Contributor
I can only assume you're currently running a swap on your entware device along with those scripts, so yeah, an AC68 still passing data at 865Mbps between wired clients is in itself impressive. But given how fast clients are and are becoming, AND with your Gigabit connection, it's time to put some more capable processing at the nexus of your network. Even an AC86 would be an upgrade, but an AX68 would be a better choice for future-proofness and not that much more money.
The kernel/code have improved..everything. the new AX machines are not insignificant improvements over your venerable AC model; you'll see an immediate difference...even when you turn the AC into an AP or mesh node.

Yes, running a swap although it doesn't really get utilized it is a requirement for Skynet IIRC.
Yes - upgrading would be beneficial, but it isn't a panacea, and TBH for the most part unnecessary in my situation.

It would be nice to get the full gigabit speeds of my fiber, but the 700-800 I normally get is well beyond what I need 99% of the time.
As per my update above, it appears that my LAN switching speeds are fine at gigabit speeds, which as I mention above I don't believe is that abnormal as even low consumer grade switches can usually handle full switching speeds on all ports.
My WiFi speeds could be better, but these are as much a problem with my clients, which at this point I am not ready to upgrade yet. Even then, the WiFi speeds are quite sufficient and the only time I wish they were quicker is when I do large file backups from my laptop to my servers.

So while upgrading everything would (obviously) improve everything, the cost outweigh the benefits for my usage. Even if the issue in the OP was accurate (which my EDIT in the previous post shows that maybe I was in error in measuring it to quickly), plunking in a $20 switch is a way more economical solution. Maybe near year end if the AX comes down in price I might consider it, but overall I'm still quite happy with what I am using.
 

drinkingbird

Senior Member
The only way that 1Gb/s traffic between two LAN ports would be "free" is if the hardware designers had included dedicated hardware to make it so --- otherwise it's going to go through the router's main data transfer pathways and, evidently, clog those pretty thoroughly. It doesn't surprise me at all that ASUS didn't include such hardware, because they wouldn't see that use-case as being a primary use-case for an inexpensive wireless router. I wouldn't bet on their newer AX gear including it, either. If this is a primary use-case for you then putting that switch in front of the router to provide the dedicated pathway you need is exactly the right fix.

The backplane would be the only real choke point and from my testing on an RT-AC1900 (pretty much the same as 68U, in some cases exactly the same) the backplane is plenty robust. A single cheap switching ASIC can handle 4G of switching throughput (5G if you include the WAN port) easily. These days, 16, 32, 48, 96G are all pretty common even in cheap switches). Enterprise switches are doing multiple terabits.
 

drinkingbird

Senior Member
I've noticed that when I do large file copies between two wired systems (SystemA and SystemB) that my Wifi access to the internet is slowed.

These system are connected to LAN ports 1 and 2 respectively, which are GB ports. My file copy software reports speeds of about 108 MB/s which translates to about 865 mbps.
Count in the overhead of SMB traffic, and I am probably pretty close to saturating a 1 gbps connection.
However, this being a switch that traffic should be isolated between those two ports.
Looking at the router CPU/memory when the copy is going it does not appear to be near critical capacity.

But, when this copy is happening, all my Wifi traffic - even that straight from the Internet is slowed down horribly (i.e. a file that would normally take < 1 minute will take 2 hours).

If I introduce an external switch and connect these two system to that switch, then connect it into a port on the ASUS, I do not have this issue.

Is this a pure hardware limitation on the Asus?

I just tested on my RT-AC1900, which is either the same as your 68U if you have one of the 1Ghz revisions, or slightly faster CPU if you have an older 800Mhz one, but CPU doesn't come into play here.

I have Aiprotection enabled, but no special firewall rules. I also have traffic monitoring enabled. Running Merlin 386.5 which I did a full reset and manual re-configure after installing.

Switch has CTF enabled and active, and also jumbo frame support enabled, however none of the PCs I was testing with had Jumbo frames enabled so that was not active. One doesn't support it, the other has it disabled, standard 1500 MTU.

LAN to LAN 1G file transfer was running at 110MB/sec (around 880mbit which is what you can expect with overhead at 1500 MTU). The two PCs were on port 2 and port 4 but that shouldn't matter, there is just one bridge with all 4 ports in it. I don't know the ASIC configuration but even if Port 1 and 2 shared one ASIC and 3 and 4 shared another it should not make any difference (if anything that would have just slowed my LAN performance). I guess you could test using the same ports I did and see if there is any difference, maybe WAN shares something with port 1, again, doubt it.

Ran a speedtest at the same time over AC wireless from a separate machine and got my full 300 down 350 up that my FIOS is capped at.

Are you sure you aren't running any custom EBTABLES that might interfere (one of your scripts or addons may have added something)? You would think if this was the case, CPU would be at 100% when you're seeing the issue but that can't necessarily be trusted 100%. Or maybe something has added some QOS that you aren't aware of that is interfering somehow (or maybe you configured it and forgot)?

As others have said, it could be time for a install of the latest FW and factory reset, followed by manual reconfigure. Test after enabling each of your addons/scripts and see if/when the issue starts to reoccur and that will point you in the right direction.

EDIT - For giggles I repeated the same test with the wired PCs on port 1 and 2. Same results. Actually a bit faster LAN transfer up around 910 to 920mbit but that is more likely due to the fact that I used the same large file for the test and the antivirus on both computers had already scanned it at that point. Or maybe ports 1 and 2 share an ASIC and 3 and 4 share another which would net a slight improvement when not splitting them up. But regardless, nothing like you're seeing.
 
Last edited:

L&LD

Part of the Furniture
Very likely, one or more of the scripts, features, and options you're using is causing this issue for you.

The most straightforward path to seeing if the hardware is capable, and, the combination you're using in your environment is/isn't compatible, is to do a full reset now.

As a quick test, I would do the steps below, at a minimum.
  • Backup both your JFFS and your router configuration via the GUI.
  • Download/verify the firmware you have running now and save it with the files above.
  • Safely remove your USB drive and do NOT re-insert it back into the router during these next few steps.
  • Physically remove the drive when it is fully unmounted and reboot the router.
  • Flash RMerlin 386.5_2 onto the router (be sure you also verify this download, if it is different from the firmware download above).
  • Fully reset the router using the following link for your model.
  • When the router has rebooted after the reset, perform the minimum necessary to gain access to the GUI.
  • Within the GUI, perform another reset, making sure to check the 'Initialize all settings...' box before clicking 'Restore'.
  • Again, perform the minimum necessary to gain access to the GUI after the router has rebooted.
  • Re-flash the same firmware version once more.
  • Perform the Hard Factory Reset once more.
  • Do not insert any USB devices. Do not customize the settings any more than needed. I suggest using a new/unique 8-character SSID, with a 16-character password for the two bands, with all characters being alphanumeric, with no spaces, punctuation, or smiley faces included.

Now, perform your testing.

If the hardware now supports your use case without choking, a full reset is indicated. Afterward, all your features/options and scripts are slowly introduced into the router one at a time and fully tested as you go (with a full reboot in between each change made). With this method, you will either find that fully resetting the router has made it behave with all your features/options and scripts enabled, or, you will find where it stumbles.

If the hardware isn't capable after testing as outlined above, you can simply flash the firmware version you are currently using, perform a full reset via the link above, and then import the saved backup config files. After inserting your USB drive that has the amtm scripts installed, and rebooting the router (maybe more than once), you should be back to where you are currently. At this point, I would be reconsidering spending on a current AX class router.

Edit: And a note that toggling settings on/off isn't the same thing as leaving them alone in the first place. If you do find yourself toggling settings on/off after a full reset, another is likely required for the fastest path back to a good/known state.
 
Last edited:

heysoundude

Part of the Furniture
Switch has CTF enabled and active, and also jumbo frame support enabled, however none of the PCs I was testing with had Jumbo frames enabled so that was not active. One doesn't support it, the other has it disabled, standard 1500 MTU.

LAN to LAN 1G file transfer was running at 110MB/sec (around 880mbit which is what you can expect with overhead at 1500 MTU). The two PCs were on port 2 and port 4 but that shouldn't matter, there is just one bridge with all 4 ports in it. I don't know the ASIC configuration but even if Port 1 and 2 shared one ASIC and 3 and 4 shared another it should not make any difference (if anything that would have just slowed my LAN performance). I guess you could test using the same ports I did and see if there is any difference, maybe WAN shares something with port 1, again, doubt it.

EDIT - For giggles I repeated the same test with the wired PCs on port 1 and 2. Same results. Actually a bit faster LAN transfer up around 910 to 920mbit but that is more likely due to the fact that I used the same large file for the test and the antivirus on both computers had already scanned it at that point. Or maybe ports 1 and 2 share an ASIC and 3 and 4 share another which would net a slight improvement when not splitting them up. But regardless, nothing like you're seeing.
I'm likely mistaken, but my recollection is that Jumbo frames is for IPv6-connected clients to negotiate optimal payloads among themselves and MTU is a v4 mechanism for reliability; I seem to recall reading something fairly authoritative about achieving close to network/connection speed/throughput ratings is highly dependant on getting these things right...bigger packets more often kind of thing lowering processing load on the router... I'll look it up. I'll post the link if I find it.

as for OP's $20 switch solution...it won't be - not until the rest of the clients are capable of the speeds of the switch. They need to work on finding the client with the slowest/least configurable network interface, and then upgrade it so something more agreeable with the rest of them on the network. It's dragging the rest down by imposing "speed limits" on them
 

ColinTaylor

Part of the Furniture
I'm likely mistaken, but my recollection is that Jumbo frames is for IPv6-connected clients to negotiate optimal payloads among themselves and MTU is a v4 mechanism for reliability; I seem to recall reading something fairly authoritative about achieving close to network/connection speed/throughput ratings is highly dependant on getting these things right...bigger packets more often kind of thing lowering processing load on the router... I'll look it up. I'll post the link if I find it.
You are mistaken. You've made this mistake before. But none of that is relevant to the OP's issue.
 

bengalih

Regular Contributor
I just tested on my RT-AC1900, which is either the same as your 68U if you have one of the 1Ghz revisions, or slightly faster CPU if you have an older 800Mhz one, but CPU doesn't come into play here.
...

Very likely, one or more of the scripts, features, and options you're using is causing this issue for you.

The most straightforward path to seeing if the hardware is capable, and, the combination you're using in your environment is/isn't compatible, is to do a full reset now.
...

Thanks for all the additional testing, and confirmation that the switch shouldn't have issues. I think everyone has missed the update I made here:


Where I state that I might have jumped the gun in reporting the slowed down wifi and it may have been external factors. So far it hasn't reoccurred in the 12 hours of file copying that has been going on (although my copy is only reporting closer to 40 MB/s now since it is copying lots of smaller files).

I do not think my router configuration is the culprit here and I don't believe a reset is necessary for this issue. However I may decide to do a reset anyway as I am interested in what I can achieve for my download bandwidth on my 1 gbps fiber connection with a totally fresh install with nothing else running.

@L&LD - I'm curious why you recommend a new SSID and a 16 character password as necessary? I thought about giving a new SSID for testing purposes just to ensure that no other clients connected and used bandwidth for my testing, but apart from just having a secure password, what is your reasoning for 16 character?

Also - to confirm, even when coming from 386.3_2 it is not recommended to restore an exported configuration? I have quite a bit of customization on my device and having to set all the GUI settings and NVRAM from scratch is laborious to document and reconfigure. For one, I would assume that recreating my VPN server settings from scratch will require me even to possibly have to distribute new client files all around. I know your procedure is meant to be "fool proof", but honestly I've never experienced an issue on firmware upgrade that a full reset fixed, and I will probably have to allocate several hours to do it based on your preferred method - which seems excessive every time a firmware update is delivered.
 

L&LD

Part of the Furniture
As I mentioned, how you implement any changes is up to you, but if you want to test the limits of the hardware, the steps outlined are the minimum I would be doing to ensure that is what you're testing.

Importing a saved backup config file may work. If it doesn't, you're back to square one. Better if issues surface immediately (then, you'll likely tie using the backup config file with the issues seen), worse if the same and/or other issues present themselves in the not too distant future.

A new and unique SSID (unique=never before used for any of your client devices, ever) ensures that any microcode they save for a specific connection isn't bringing its own set of gremlins.

A 16-character password used to be the limit for passwords (not the passphrase) and gives more than enough security too. Again - just using best practices here while testing (which of course I recommend for actual use too).

With the router in a good/known state as it should be after doing the above, it will give you a baseline against your actual configuration.

I have not seen any benefit to upgrading the router firmware, performing a full reset, then importing a saved backup config file made from the previous version, but I have seen many issues from doing so.

If it works for you, please continue to enjoy doing it as you've been. But a full reset with a minimal and manual configuration, along with a clean install of a newly formatted (NTFS, on a PC) USB drive for amtm + script use takes mere minutes anyway. And the long-term dependability of those networks is worth the few minutes extra spent.

I agree it is excessive every time a firmware update is delivered. But the changes/upgrade from v386.3_2 to 386.5_2 warrants it, if you have more than just a bare minimal install (which you do, it seems).
 

ColinTaylor

Part of the Furniture
Also - to confirm, even when coming from 386.3_2 it is not recommended to restore an exported configuration?
I assume you mean a "saved settings" file rather than some other sort of "exported" configuration. In which case no, you should only really restore from a saved file that was created using the same firmware version that you're currently running.

You could of course just update normally from 386.3_2 to 386.5_2 and then make a new "saved settings" file (which you should probably be doing anyway). You can then use this new file if you need to reset the current firmware.
 

drinkingbird

Senior Member
I'm likely mistaken, but my recollection is that Jumbo frames is for IPv6-connected clients to negotiate optimal payloads among themselves and MTU is a v4 mechanism for reliability; I seem to recall reading something fairly authoritative about achieving close to network/connection speed/throughput ratings is highly dependant on getting these things right...bigger packets more often kind of thing lowering processing load on the router... I'll look it up. I'll post the link if I find it.

as for OP's $20 switch solution...it won't be - not until the rest of the clients are capable of the speeds of the switch. They need to work on finding the client with the slowest/least configurable network interface, and then upgrade it so something more agreeable with the rest of them on the network. It's dragging the rest down by imposing "speed limits" on them

IPv4 and IPv6 have nothing to do with frames. Frames are Layer 2 and know nothing about IP address, only MAC address. Jumbo/Oversize packets (which only come into play when passing through a router or L3 device) are a bit more picky but I'm not aware of any difference between IPv4 and v6 for that. Jumbo frames definitely aren't anything to do with IPv6, been around since long before that. Larger frames and/or MTU isn't always better, but for LAN connections where you're doing large file transfers, they are typically beneficial. Most NAS will default to 9K frames for example.

Your switch comments are incorrect. You can mix and match all you want on a switch. Having a 100M client on a gig switch will not impact anything that isn't communicating with that client. It will not slow down two gig clients talking to each other. Heck you can have a 20 year old 10/Half NIC connected to a switch and it will not slow down other clients. I think you're thinking of Wifi where slower/older clients can slow everyone down (to a certain, sometimes severe extent, but it still isn't that everyone gets dragged down to the speed of the slowest client).
 

drinkingbird

Senior Member
Thanks for all the additional testing, and confirmation that the switch shouldn't have issues. I think everyone has missed the update I made here:


Where I state that I might have jumped the gun in reporting the slowed down wifi and it may have been external factors. So far it hasn't reoccurred in the 12 hours of file copying that has been going on (although my copy is only reporting closer to 40 MB/s now since it is copying lots of smaller files).

I do not think my router configuration is the culprit here and I don't believe a reset is necessary for this issue. However I may decide to do a reset anyway as I am interested in what I can achieve for my download bandwidth on my 1 gbps fiber connection with a totally fresh install with nothing else running.

@L&LD - I'm curious why you recommend a new SSID and a 16 character password as necessary? I thought about giving a new SSID for testing purposes just to ensure that no other clients connected and used bandwidth for my testing, but apart from just having a secure password, what is your reasoning for 16 character?

Also - to confirm, even when coming from 386.3_2 it is not recommended to restore an exported configuration? I have quite a bit of customization on my device and having to set all the GUI settings and NVRAM from scratch is laborious to document and reconfigure. For one, I would assume that recreating my VPN server settings from scratch will require me even to possibly have to distribute new client files all around. I know your procedure is meant to be "fool proof", but honestly I've never experienced an issue on firmware upgrade that a full reset fixed, and I will probably have to allocate several hours to do it based on your preferred method - which seems excessive every time a firmware update is delivered.

I'm not sure I agree with the whole new SSID thing, if you do a full reset and do not import a config file, not sure how anything would be left behind, but I guess who knows. I've never done it, been using the same SSID for years, but I do re-enter everything manually.

Honestly, I would never restore anything from a backup, especially if you're trying to troubleshoot an issue. Take screen shots and/or copy stuff to text files then re-enter/paste it back in manually. The only thing a backup is for is if you somehow get reset to factory defaults and need to restore your settings on the same firmware version. Even then, I'd prefer to re-enter it manually.

I guess if you have a ton of customizations are are staying on the same code base (386) it isn't as big of an issue, however did you do a full factory reset when going from 384 to 386? You may have some very old config stuff that is causing you issues and you're just making it worse every time you upgrade. As far as VPN stuff you should be able to export the cert and reimport it (no issue there, that's not config thats just a static file).

You don't have to do a full reset for every firmware upgrade. Many will do "dirty upgrades" when only going up 1 step within the same code base, unless the release notes specifically say a reset is needed. But if you haven't done a full factory reset and reconfigure in a long time, especially since 384, I think it may be time to set aside those couple hours.
 

tgl

Regular Contributor
I got curious about this topic so I thought I'd do a little actual testing, and I have both good and bad news. I don't have the OP's router model, so this doesn't directly answer his question, but I do have an XT8. All tests based on iperf3 results.

System under test: 2021-vintage ASUS XT8, running firmware 42095 in Access Point mode.

Test rig: 2021 MacBook Pro plugged into WAN port via UGreen 2.5G-capable ethernet-to-USB-C dongle, assorted other machines connected to LAN ports.

The good: it can sustain multiple 1G (well, 930-ish Mbps) transfers between different LAN ports, for example running WAN->LAN1 and LAN2->LAN3 transfers concurrently. It's full duplex too: I can run full-rate transfers in both directions between any two ports. There is detectable CPU load (around 20% of one core) per transfer, but nothing that would come close to saturating the CPU. Also, I couldn't detect any reproducible effect of LAN2<->LAN3 transfers on WAN<->wireless transfer rate. However, I'm only running this system on 80MHz bandwidth (too much radar activity around here) so the wireless transfers topped out at somewhere around 700Mbps using a 2x2 AX client. It'd be interesting to see results from a similar test by someone who can peg the wireless transfer rate.

The odd: There is a small amount of weird asymmetry. I can transfer between LAN2 and LAN3 at ~935Mbps either way, or both ways simultaneously. Transferring between WAN and LAN1, the same. Transferring LAN1 to LAN2 runs at a solid 940 Mbps, but the other direction only about 923. The same for LAN1 to LAN3 or WAN to LAN2/3. Yeah, it's a small difference, but it is absolutely reproducible and well above the inter-run variation. I guess there is something to @drinkingbird's upthread speculation that the port pairs might be getting managed by two separate chips.

The ugly: the allegedly 2.5G-capable WAN port CANNOT support two 1G streams simultaneously. I tried running two iperf3 sessions, one WAN->LAN2 and one WAN->LAN3. The transfer rate for each varied quite a lot (whereas all the foregoing tests showed rock-solid rates) and was only in the general area of 500Mbps each. In the reverse direction (concurrent LAN2 and LAN3 to WAN) each session ran at a pretty steady 466Mbps. This was sufficiently unexpected that I thought maybe something was wrong with my test rig ... but with the same gear plugged directly into a 2.5G ethernet switch, the MacBook and dongle don't have any trouble sustaining two concurrent full-rate iperf3 sessions with the same 1G-capable machines I used in the test. And both the MacBook and the XT8 reported their connection as running at 2500Mbps. So it's definitely the XT8 that is falling down.

I find this last result surprising enough that I wonder if anyone else can reproduce it. Unlike my (now proven wrong) theory that ASUS wouldn't care that much about LAN-port-to-LAN-port transfers, a WAN port that can't sustain the advertised speed seems to go right to the core of whatever performance claims they might wish to make.
 

drinkingbird

Senior Member
I got curious about this topic so I thought I'd do a little actual testing, and I have both good and bad news. I don't have the OP's router model, so this doesn't directly answer his question, but I do have an XT8. All tests based on iperf3 results.

System under test: 2021-vintage ASUS XT8, running firmware 42095 in Access Point mode.

Test rig: 2021 MacBook Pro plugged into WAN port via UGreen 2.5G-capable ethernet-to-USB-C dongle, assorted other machines connected to LAN ports.

The good: it can sustain multiple 1G (well, 930-ish Mbps) transfers between different LAN ports, for example running WAN->LAN1 and LAN2->LAN3 transfers concurrently. It's full duplex too: I can run full-rate transfers in both directions between any two ports. There is detectable CPU load (around 20% of one core) per transfer, but nothing that would come close to saturating the CPU. Also, I couldn't detect any reproducible effect of LAN2<->LAN3 transfers on WAN<->wireless transfer rate. However, I'm only running this system on 80MHz bandwidth (too much radar activity around here) so the wireless transfers topped out at somewhere around 700Mbps using a 2x2 AX client. It'd be interesting to see results from a similar test by someone who can peg the wireless transfer rate.

The odd: There is a small amount of weird asymmetry. I can transfer between LAN2 and LAN3 at ~935Mbps either way, or both ways simultaneously. Transferring between WAN and LAN1, the same. Transferring LAN1 to LAN2 runs at a solid 940 Mbps, but the other direction only about 923. The same for LAN1 to LAN3 or WAN to LAN2/3. Yeah, it's a small difference, but it is absolutely reproducible and well above the inter-run variation. I guess there is something to @drinkingbird's upthread speculation that the port pairs might be getting managed by two separate chips.

The ugly: the allegedly 2.5G-capable WAN port CANNOT support two 1G streams simultaneously. I tried running two iperf3 sessions, one WAN->LAN2 and one WAN->LAN3. The transfer rate for each varied quite a lot (whereas all the foregoing tests showed rock-solid rates) and was only in the general area of 500Mbps each. In the reverse direction (concurrent LAN2 and LAN3 to WAN) each session ran at a pretty steady 466Mbps. This was sufficiently unexpected that I thought maybe something was wrong with my test rig ... but with the same gear plugged directly into a 2.5G ethernet switch, the MacBook and dongle don't have any trouble sustaining two concurrent full-rate iperf3 sessions with the same 1G-capable machines I used in the test. And both the MacBook and the XT8 reported their connection as running at 2500Mbps. So it's definitely the XT8 that is falling down.

I find this last result surprising enough that I wonder if anyone else can reproduce it. Unlike my (now proven wrong) theory that ASUS wouldn't care that much about LAN-port-to-LAN-port transfers, a WAN port that can't sustain the advertised speed seems to go right to the core of whatever performance claims they might wish to make.

LAN to LAN should not cause any CPU utilization at all. It never passes to the CPU, unless something is totally designed wrong. When in AP mode, in theory WAN should just be another switch port but there may be some part that still involves the CPU, possibly through mistake in the code for AP mode or maybe some feature like bandwidth monitoring etc. I guess if you're going from 1G ports to 2.5G port it may utilize different bridge interfaces or something in which case some CPU involvement will be there.

My guess is the LAN ports have a 1G connection to the backplane and the WAN has a 2.5G (and WIFI also has multigig). So that backplane connection for the LAN ports is limiting your testing. The variation you're seeing in IPERF is exactly what you see when saturating a 1G connection with multiple streams (drops and retransmissions cause a lot of variation as each stream detects congestion, throttles back, then throttles up again, etc).

If we suspect that LAN 1+2 share one ASIC and LAN 3+4 share another (or possibly different lanes in the same ASIC) then I'd say try doing one stream from each pair of ports in hopes that they each have a 1G backplane connection, but looks like you did that with LAN 2+3 to WAN, so the total backplane connection for LAN appears to be 1G. It is possible this limitation is in the software (the virtual interface is set to 1G) but more likely hardware, so I guess they're just assuming that most people are using wireless and that's where they want the >1G speed. If you had a 4 lane wireless card at 80 mhz you could test and confirm that you can get over 1G on wireless (or if you could get 160Mhz working with 2 lane), I suspect you would.

Or try doing 1G from LAN and as much as you can from wireless and the sum should be >1G to the WAN port.

You could try SSH into the router and look at the interfaces, see if the LAN is in one bridge and the WAN/WIFI is in another. Moving the LAN into the WAN/WIFI bridge might bypass the limitation, but I don't see why they would be in different software bridges in AP mode, as that would require traffic to pass through the CPU.

They probably just focused on multigig wireless (WIFI to WAN) capability since that is what a lot of people are using these days. Less an less wired connections. There would be additional cost to implement multigig on all the LAN ports or even just the backplane connection. Probably only a matter of time before we see multigig on all ports on newer routers.
 
Last edited:

Similar threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top