2.5 gbe performance

_Jon_

Occasional Visitor
I am trying to test the performance of the 2.5 gbe network port on my RT-AX86U (running merlin). I have a 10 gbe NIC on my Windows 10 computer, which I have connected directly to the 2.5 gbe port using a brand new CAT6A cable. I can see in Windows that the link has synced at 2.5 gbe, and the router is showing this as well. I used ssh to connect to my router and run iperf3 on my computer and on the router. regardless of which device I use as the client or server, the maximum rate that I have seen in any of these tests is 1.26 gbps. Prior to my having purchased a 10 gbe NIC, the maximum speed was around 920 kbps. So, I can see that it is clearly better than 1 gbe, but I was expecting to see numbers closer to 2 gbps. Am I expecting too much? Any suggestions of what could be going on here? NIC drivers are the latest version as is merlin. I don't currently have any other 2.5 gbe devices that I can test so I don't know if the issue is on my router or my laptop. Thanks in advance.

Jon
 

Tech Junky

Very Senior Member
Well, I wouldn't expect anything less from Asus.

I run a custom network config I built from scratch and 5GE on both sides gives me 400MB/s of data transfer which is being limited by the spinners. If I added 2 more drives to the configuration I could exceed the 5GE and the network would become the bottleneck at that point.

Let's put things in simpler terms. I switched to 1gbps ISP plan and swapped in an Asus router w/ gig ports on it and it fell flat on its face at 1/4 to 1/3 of the speed it should have been able to hit. As soon as I ditched it for a R7800 the issue went away.

Moral of the story? Not all devices are created equal when it comes to performance.
 

ColinTaylor

Part of the Furniture
I am trying to test the performance of the 2.5 gbe network port on my RT-AX86U (running merlin). I have a 10 gbe NIC on my Windows 10 computer, which I have connected directly to the 2.5 gbe port using a brand new CAT6A cable. I can see in Windows that the link has synced at 2.5 gbe, and the router is showing this as well. I used ssh to connect to my router and run iperf3 on my computer and on the router. regardless of which device I use as the client or server, the maximum rate that I have seen in any of these tests is 1.26 gbps. Prior to my having purchased a 10 gbe NIC, the maximum speed was around 920 kbps. So, I can see that it is clearly better than 1 gbe, but I was expecting to see numbers closer to 2 gbps. Am I expecting too much? Any suggestions of what could be going on here? NIC drivers are the latest version as is merlin. I don't currently have any other 2.5 gbe devices that I can test so I don't know if the issue is on my router or my laptop. Thanks in advance.

Jon
By running iperf on the router itself the process starts to become CPU bound. I don't know what the limit is because I don't have any >1GbE devices to test it with. But if you're going LAN to LAN it's a switched connection and therefore doesn't use the router's CPU. So try and setup a test using simultaneous LAN to LAN connections. I suspect the aggregate throughput will be much better.
 

_Jon_

Occasional Visitor
am I understanding this correctly? are you saying that the asus router comes with a 2.5 gbe port but does not have enough horsepower to route data at 2 gbps? or are you saying that iperf is adding CPU overhead and reducing throughput? Unfortunately, I do not have another 2.5 gbe client to test on, and even if I did, any test that I ran would be evaluating the throughput of my 2.5 gbe switch not the router. my goal here is to be able to fully saturate 2 1 gb connections on the router, or to route WIFI traffic at full speed while 1 of the 1gbe connections is fully saturated. I do have a 2.5 gbe switch (and a second RT-AX86U for mesh) but I am trying to evaluate each component first. So, it sounds like the only way to test the 2.5 gbe port on the router would be to run iperf on my computer as the server (at 2.5 gbe) and then test from 2 or 3 different 1 gbe client devices simultaneously and see what the combined throughput is? Does that sound reasonable?
 

ColinTaylor

Part of the Furniture
am I understanding this correctly? are you saying that the asus router comes with a 2.5 gbe port but does not have enough horsepower to route data at 2 gbps? or are you saying that iperf is adding CPU overhead and reducing throughput?
Neither. LAN to LAN traffic is switched, doesn't use the router's CPU and is therefore fast. WAN to LAN (and vice versa) uses Broadcom's hardware acceleration features to achieve >1Gb throughput (I don't know the exact number) also by bypassing the router's CPU. What you were doing was Router<->LAN which uses the router's main CPU and may become a bottleneck at very high throughput. The latter is not a typical real-world scenario.

So, it sounds like the only way to test the 2.5 gbe port on the router would be to run iperf on my computer as the server (at 2.5 gbe) and then test from 2 or 3 different 1 gbe client devices simultaneously and see what the combined throughput is? Does that sound reasonable?
Yes, that's what I was suggesting.
 

Tech Junky

Very Senior Member
Consumer grade devices can't handle high performance networking.

LAN to LAN via the router should bypass the CPU for the most part using the ASIC's instead of the backplane.

iperf just generates traffic (packets) and is a point to point test. The WAN being 2.5GE should be able to handle ~2GE w/ overhead but, the additional software put into these boxes hobbles the throughput considerably. If you strip down all of the bloated software running on them you can achieve higher throughput w/o the convenience features being enabled.

Proper networking though instead of trying to combine 2 routers into a LAN solution doesn't yield performance. Especially when said HW doesn't perform well to begin with. When testing different routers I had better success with a R7800 but, also tested a Synology R2600 that worked well but was missing a key feature I needed for WIFI to work @ 160mhz.
 

Tech Junky

Very Senior Member
There's already a video out there showing the AX86U hitting near 2.5 gbps when using the 2.5 gbe port as a WAN port.
And is this real world or simulated in a LAB environment?

Are they using synthetic traffic at a certain packet size that's optimized for the router to handle at peak performance or is it mixed and sustained traffic you would see in an actual file transfer?

There are plenty of ways to make a turd look like a diamond for a short period of time. Take NVME drives for example boasting 7000mb/s but, what they don't show you is that cutting edge speed gets chopped in 1/2 or less when you surpass the first 15 seconds and exhaust the DRAM cache or the controller starts to heat up and throttle.
 

_Jon_

Occasional Visitor
Neither. LAN to LAN traffic is switched, doesn't use the router's CPU and is therefore fast. WAN to LAN (and vice versa) uses Broadcom's hardware acceleration features to achieve >1Gb throughput (I don't know the exact number) also by bypassing the router's CPU. What you were doing was Router<->LAN which uses the router's main CPU and may become a bottleneck at very high throughput. The latter is not a typical real-world scenario.


Yes, that's what I was suggesting.

Thank you for the advice. I can confirm that 2 devices can obtain ~950 mbps simultaneously, and when a third device is added they all drop. impossible to get an exact combined number with this method, but it is somewhere between 2 and 2.5 mbps so i am happy. next I will introduce my switch, rinse & repeat. just tryin to make sure that all of the pieces are working properly from the start. thanks again!
 

maxbraketorque

Very Senior Member
And is this real world or simulated in a LAB environment?

Are they using synthetic traffic at a certain packet size that's optimized for the router to handle at peak performance or is it mixed and sustained traffic you would see in an actual file transfer?

There are plenty of ways to make a turd look like a diamond for a short period of time. Take NVME drives for example boasting 7000mb/s but, what they don't show you is that cutting edge speed gets chopped in 1/2 or less when you surpass the first 15 seconds and exhaust the DRAM cache or the controller starts to heat up and throttle.

You got me, but here is the link. Looks like he knows what he's doing.
 

Tech Junky

Very Senior Member
Well, one is showing oenwrt and the other is ax86u

1643413847213.png


Looks like 2 PC's w/ the Router in the middle using a NIC + WIFI on one side.

1643414018107.png


Synthetic Lab testing is what this is. PC's on both sides acting like the internet.

Now, real world testing through a modem with 2.5GE / ISP plan to match would be more interesting to see the results. Knowing Asus he probably disabled everything one of the features to get those speeds.

The OS on the Asus lineup is the biggest bottleneck to contend with if you're looking to use those features IRL. A router is supposed to move packets not keep the kids from seeing porn, DPI, AV, and all the other crap they try to cram into the OS.

I mean I'm running my custom built PC / Linux router and can easily hit those speeds and higher but, when you look at the above picture you're only going to go as fast as your slowest link. If you have legacy BS sitting in the network it causes interrupts that slows everything down because the CPU needs to waste a cycle on the slower devices to let them have a turn in pushing traffic.

My setup would be:

AP <> PC <> Modem <> WWW

Since the Pc is the router the raw speed on the PC for testing purposes w/ VPN is :
1643414803925.png


1643414956540.png


So, to the CM which is a MB860 w/ 4 ports I have 2 Ethernet cables hooked up to it for LACP to get beyond the 1GE threshold. In the bottom capture the raw speed w/o VPN shows 1500 / 42. With VPN I've gotten just as high of a test w/ the right server selected on the VPN side.

Variables always come into play that give you mixed results for comparison.
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top