Ethernet backhaul speeds between BE88U and BE86U on 10G link

[BobbyFlobby]

Hello friends and foes,

Hopefully this hasnt been answered before, if it has, feel free to delete.

I have a Be88U and Be86U as a mesh node, both are connected to the 10G ports with ethernet backhaul set.

When i perform iperf tests between both routers, speed is not what i would expect at 2.7Gb or so so each test.
iperf3 -c IP -P4 -bidr

CPU doesnt appear to be maxing out either during the testing

I dont have QOS or Traffic Analyzer on and faily basic settings.

Do i need to adjust settings, or are these the expected speeds?

Thanks for any help/advice.

 

[jzchen]

I've read that AiProtection can also be a cause of slowdown. Do you have it on?

 

[BobbyFlobby]

I've read that AiProtection can also be a cause of slowdown. Do you have it on?

No, thats not on, game boost isnt either, and the only extra thing i have are some VPN settings but they are only enabled manually.

 

[degrub]

WHat do you get with -P10 ?

 

[BobbyFlobby]

WHat do you get with -P10 ?

I was getting less the more I did, i think with 10 was same or worse and 20 was really poor

 

[BobbyFlobby]

WHat do you get with -P10 ?

yep a bit less -p10

and -p20 getting worse.

 

[degrub]

No, thats not on, game boost isnt either, and the only extra thing i have are some VPN settings but they are only enabled manually.

did you also withdraw from the license and restart the router ? You may have to factory reset instead of just restart.
Just turning off the Ai P stuff doesn't get rid of the issue.

 

[user114514]

It's Broadcom's fault.

There is a bus bottleneck between the MAC and the CPU, but there is no bottleneck between the 10G PHY and the MAC. So as long as your traffic is fully hardware offloaded there is no bottleneck, but iperf3 on the router will let traffic go through the CPU anyway.

This is understandable because the performance of the cortex-A53 does not allow software PPPOE, NAT to break 1Gbps, and this larger bottleneck masks the bus bottleneck. For Broadcom, running iperf3 on BCM4916 is not a meaningful use case, and it's not worth the additional cost to increase the bus speed for it.

This issue also exists on older BCM4908-based platforms such as ax86u, where traffic is limited to 1.33Gbps when it can't be hardware offloaded, which makes the 2.7Gbps bottleneck you found look like it's not a coincidence that Broadcom just doubled BCM4916's bus speed.

 

[BobbyFlobby]

It's Broadcom's fault.

There is a bus bottleneck between the MAC and the CPU, but there is no bottleneck between the 10G PHY and the MAC. So as long as your traffic is fully hardware offloaded there is no bottleneck, but iperf3 on the router will let traffic go through the CPU anyway.

This is understandable because the performance of the cortex-A53 does not allow software PPPOE, NAT to break 1Gbps, and this larger bottleneck masks the bus bottleneck. For Broadcom, running iperf3 on BCM4916 is not a meaningful use case, and it's not worth the additional cost to increase the bus speed for it.

This issue also exists on older BCM4908-based platforms such as ax86u, where traffic is limited to 1.33Gbps when it can't be hardware offloaded, which makes the 2.7Gbps bottleneck you found look like it's not a coincidence that Broadcom just doubled BCM4916's bus speed.

Thanks for that explanation, greatly appreciated, and if you would allow me a follow up question.
1. Is there a way to test max speed bewteen the 2 routers?

 

[user114514]

Thanks for that explanation, greatly appreciated, and if you would allow me a follow up question.
1. Is there a way to test max speed bewteen the 2 routers?

You will need to place the iperf3 server and client on external devices such as NAS, PC. but considering that be86u only has a single 10GbE interface, you will need to test from four 2.5GbE interfaces simultaneously.