RT-N66U - Less performance than expected on high speed fibre connection

[rotor]

http://www.intel.com/content/www/us...oards/desktop-boards-d33217gke-dc3217iye.html

It has one Ethernet interface. The other model doesn't even have one specified in either the box or the motherboard itself.

Man I'm asleep at the wheel today... I saw the two HDMI ports and my mind saw two NICs. Sorry about the confusion.

 

[RMerlin]

PPPoE does add a fair amount of CPU overhead. That might be where the dual core architecture helps, as pppd can run on a separate thread and separate core than the rest.

I would consider running the PPPoE session on the modem, and see if it has an option that would allow you to put the Asus router into a DMZ. Not an ideal network setup, but it might give you the best of both worlds.

 

[rotor]

PPPoE does add a fair amount of CPU overhead. That might be where the dual core architecture helps, as pppd can run on a separate thread and separate core than the rest.

I would consider running the PPPoE session on the modem, and see if it has an option that would allow you to put the Asus router into a DMZ. Not an ideal network setup, but it might give you the best of both worlds.

I think DMZ is an option on the BT router. I will check this out. I was really hoping to have a single device. I guess it boils down to: do I really need the full 300 Mbps when 200 is so amazing anyway, or: is the marginal benefit of 200 -> 300 Mbps worth the additional latency and complexity of running the RT-N66U behind the BT router?

RMerlin, I was hoping you would know some secret trick for optimising the PPPd client, perhaps through IRQ coalescing...

 

[RMerlin]

I think DMZ is an option on the BT router. I will check this out. I was really hoping to have a single device. I guess it boils down to: do I really need the full 300 Mbps when 200 is so amazing anyway, or: is the marginal benefit of 200 -> 300 Mbps worth the additional latency and complexity of running the RT-N66U behind the BT router?

That is entirely up to you I guess. Doing such a setup wouldn't affect latency in any measurable way, it might cause NAT-related issues with some applications however. That will depend on the needs of your household.

RMerlin, I was hoping you would know some secret trick for optimising the PPPd client, perhaps through IRQ coalescing...

Sorry, not really a low-level kernel expert here.

 

[Advis]

rotor I'd love to be having the same problem as you are having right now. My street cabinet at home is still not activated for FTTC and I had to write to my local MP to beg for a fibre cabinet for my place of work.

Have you tried contacting ASUS support over this? I'm sure if you approached them they'd probably have a go at improving things despite the single core limitation. ASUSWRT still being a pretty new firmware project for them I bet they wouldn't like to lag behind a mere BT Home Hub.

 

[rotor]

rotor I'd love to be having the same problem as you are having right now. My street cabinet at home is still not activated for FTTC and I had to write to my local MP to beg for a fibre cabinet for my place of work.

Have you tried contacting ASUS support over this? I'm sure if you approached them they'd probably have a go at improving things despite the single core limitation. ASUSWRT still being a pretty new firmware project for them I bet they wouldn't like to lag behind a mere BT Home Hub.

I know, first world problem, right?!

I've emailed Asus, so let's see what they come back with.

 

[srinivasvaradaraj]

I remember some sirq cpu hog discussion on RedHat threads. If memory serves me right, there were some improvements on Linux kernel 2.6.29 and above. ASUS wrt unfortunately still uses 2.6.22 kernel. You could if you like grab their source and try to compile a custom kernel for yourself. However, some of the binary provided code may or may not work. Merlin might help you with some key pointers in this regard.

 

[RMerlin]

I remember some sirq cpu hog discussion on RedHat threads. If memory serves me right, there were some improvements on Linux kernel 2.6.29 and above. ASUS wrt unfortunately still uses 2.6.22 kernel. You could if you like grab their source and try to compile a custom kernel for yourself. However, some of the binary provided code may or may not work. Merlin might help you with some key pointers in this regard.

You can't recompile a vanilla kernel for the router. There's a lot of patches required for the Broadcom environment.

 

[rotor]

Just a quick update. I'm returning the RT-N66U -- it definitely isn't capable of handling 300 Mbps of PPPoE.

For future generations, here is the summary: I have a 300+ Mbps Internet connection, supplied over fibre (Fibre to the Premises), and requiring PPPoE. I purchased an Asus RT-N66U, thinking it would have ample power to support 300 Mbps, but unfortunately it appears that PPPoE is inefficiently implemented in the Linux kernel, and as a result CPU usage skyrockets to 100%, giving a maximum download speed of around 200 Mbps.

I looked at many options, engaged in discussions on several forums and IRC channels, and these are my findings:

- Asus RT-N66U running stock firmware (I tested two versions: 260, and beta 321). Maximum download speed of around 200 Mbps.

- The RT-N66U running Shibby. It was slower (around 180 Mbps).

- My existing Draytek 2820Vn (which served me well for 4+ years on ADSL), which maxes out at around 70 Mbps. It was never intended for these sorts of speeds. I am now using it as a wireless Access Point.

- I wouldn't mind another Draytek, but the 2130n (the fastest in its price bracket) is almost certainly not capable of achieving these speeds. Draytek Taiwan is reasonably sure it would max out at around 200 Mbps with PPPoE (from a an email someone on the Draytek forum sent to Draytek).

- A vendor did a test for me of a RouterBoard RB2011 which has a 600 MHz processor. Even overclocked to 750 MHz (this is a feature of the Mikrotik RouterOS) it was still only capable of 280 Mbps, and that was without any firewall rules (i.e. not apt for real-world use).

- A 4+ year-old Atom motherboard I have lying around (Intel D945GCLF2 with Atom 330, a dual-core 1.6GHz) running RouterOS. It breezed through the test: 310 Mbps at 20% CPU, including NAT and firewall rules enabled.

- Currently as my router I'm running RouterOS on a VM that runs on a Xeon E3-1220 (a quad-core 3.1 GHz) -- and of course it absolutely flies. CPU averages at 0.32%, and hitting the full 300 Mbps sends the VM CPU to a whopping 7.13% (and the VM only has 2 virtual CPUs out of the 4 physical cores available, so this is roughly equivalent to 3.5% of the processor's capability). For those of you about to point how insecure this is, I am passing through a NIC to the VM (using VT-d) for the WAN connection.

This really highlights to me how different processors handle different workloads. The MIPS-type processors used in routers may be good at some things, but are terrible at others; or maybe it is just that the Linux implementations in use aren't optimised. Hopefully as the broadband market continues to produce faster products, manufacturers will produce more powerful routers.

Another fascinating result is that the Atom motherboard idles at 24W, the Xeon idles at 30W, and the RT-N66U idles at 21W. So there is clearly no correlation between the amount of power consumed (at idle), and the raw processing power available in a CPU (I know that the wireless cards in the Asus require a few Watts to power as well). These are all measured using my UK equivalent of a kill-a-watt.

My current plan is to purchase an Intel DN2800MT motherboard (with an additional NIC), which idles at less than 10W.

I'd like to thank everyone for their input, and hopefully these test results will be useful for someone in the future.

 

[SoCalReviews]

I'm just curious what a custom built setup like one with the Intel Atom processor costs...for everything...more than $200 USD? The Asus routers are high end consumer grade and a great value for the price (currently around $150 USD for the RT-N66U). They feature more bells, whistles and customization capabilities than most people need for home or small business applications. Many users feel the performance of it as a dedicated wireless AP alone is worth the reasonable cost of these routers. However as you have shown there are some home and small business users who might benefit from higher performance wireless routers.

There are many other higher end solutions available if you are willing to pay for them but there are always the cost prohibitive factors that determine purchasing decisions. It would be interesting to know what the total final cost would be with your custom router and wireless setup...up to $500 USD? If you wanted good wireless performance then the RT-N66U and the RT-AC66U routers are great even when used as inexpensive dedicated APs. I am wondering wouldn't you want to keep the Asus in your custom setup for its wireless performance alone. Also, since the wireless performance setup is critical for many home and business users how much would be the entire custom router and wireless access point setup like the one you put together would be compared to simply purchasing a business professional grade wireless router from a company like Cisco, etc..

 

[rotor]

I'm just curious what a custom built setup like one with the Intel Atom processor costs...for everything...more than $200 USD? The Asus routers are high end consumer grade and a great value for the price (currently around $150 USD for the RT-N66U). They feature more bells, whistles and customization capabilities than most people need for home or small business applications. Many users feel the performance of it as a dedicated wireless AP alone is worth the reasonable cost of these routers. However as you have shown there are some home and small business users who might benefit from higher performance wireless routers.

There are many other higher end solutions available if you are willing to pay for them but there are always the cost prohibitive factors that determine purchasing decisions. It would be interesting to know what the total final cost would be with your custom router and wireless setup...up to $500 USD? If you wanted good wireless performance then the RT-N66U and the RT-AC66U routers are great even when used as inexpensive dedicated APs. I am wondering wouldn't you want to keep the Asus in your custom setup for its wireless performance alone. Also, since the wireless performance setup is critical for many home and business users how much would be the entire custom router and wireless access point setup like the one you put together would be compared to simply purchasing a business professional grade wireless router from a company like Cisco, etc..

From a quick visit to Newegg, LogicSupply and Crucial:

DN2800MT Motherboard $110
1 GB RAM $10
Case (M350) $40
AC Power Adapter $25
Riser card + IO shield (to allow the 2nd NIC) $17
Additional Intel Gigabit NIC $35
Intel 6300 Ultimate N 450 Mbps $35
mSATA 32GB $51 (this is a luxury, I would say just boot from a USB stick)

Total (excluding the mSATA) is $272 for a single Wireless N card, or $307 for dual Wireless N cards (for simultaneous 2.4 and 5.0 GHz support -- another luxury, there are relatively few 5.0 GHz devices out there yet).

This is a completely fanless solution that idles at around 10 W (maybe a bit more with the Wifi cards). The Asus idles at 21 W, and I calculate that the Atom is 5x - 7x more powerful than the Asus.

Note that I didn't spend any time finding the lowest price, I just knew which bits I needed, and quickly searched for them to have a rough idea. I'm in the UK but these prices are in USD and for the benefit of what I assume is the majority demographic of this forum. Of course, if you have any parts already (like a case, power supply or NIC floating around), then cost drops accordingly.

Versus the RT-N66U, at $170, which as you said, offers plenty of bells and whistles for a pretty good price, and covers the majority of needs for the majority of users, in a pretty luxurious way. It's a GREAT little router!

Commercial routers are almost an order of magnitude more expensive, and with no guarantee of performance (in a lot of cases they are a huge rip-off). You would also struggle to find one with e.g. dual-band 3x3 MIMO Wifi + gigabit + all the other features that retail routers provide. Then there's form-factor (usually they're not small), noise + heat + power-consumption, etc.

So of course I'm not advocating that the home build is for everyone; in fact, far from it. I could very well choose to live with the RT-N66U and be limited to "just" 200 Mbps. However, as a geek, I rise to the challenge of eking out every last drop of performance from my broadband, and if that requires me to come up with a kickass home-build, then that makes me happy because I have an excuse!

Note that I haven't made up my mind. It's still a lot of money for what should be a commodity device; but I do realise I am in the minority with my fast broadband.

Regarding wireless performance: I had the RT-N66U for three weeks, and to be honest, I didn't notice the difference. My iPhone 5 supports 5GHz, and my laptop has an Intel 5300 which is a pretty great little card that supports 3 streams on both bands (I benchmarked well over 200 Mbps to my Gigabit server on the 5 GHz band), and yet, in normal day to day use, there is absolutely no perceptual difference when compared to my current 2.4 GHz 4+ year-old Draytek router that is now serving as an AP. It's not like I'm using my wireless devices to copy multi-gigabyte files; so at this stage, for me 5 GHz wireless is cool, but nowhere near the top of the list of my requirements. I should point out that most of my home computing is done on a Core i7 PC on Gigabit Ethernet.

Thanks for your feedback, and I'd be interested to know how you feel re. homebuilt vs. RT-N66U now that you know it can be done for under $300.

 

[RMerlin]

I know this isn't your fault, but I would personally slap whoever at your ISP decided to put a 300 Mbits connection through a PPPoE session. In addition to the CPU load, you are wasting a LOT of that bandwidth just in PPPoE overhead. Any sane ISP would use straight DHCP at those kind of speed, not PPPoE.

Regarding power usage: keep in mind that the RT-N66U idle and full load power usage should be nearly the same.

 

[rotor]

I know this isn't your fault, but I would personally slap whoever at your ISP decided to put a 300 Mbits connection through a PPPoE session. In addition to the CPU load, you are wasting a LOT of that bandwidth just in PPPoE overhead. Any sane ISP would use straight DHCP at those kind of speed, not PPPoE.

I 100% agree. However, there is exactly ZERO chance of getting the ISP to change.

 

[SoCalReviews]

Thanks for your feedback, and I'd be interested to know how you feel re. homebuilt vs. RT-N66U now that you know it can be done for under $300.

Thank you for the cost breakdown. I might build one for the fun of it but the capabilities of the wireless part of your particular build is what concerns me. It seems to be the consensus opinion that the main reason the RT-N66U isn't working out for your application is the PPPoE processing overhead which the ISP provided router you already have could easily handle anyway. In this day and age most people are concerned with wireless performance since most gigabit Ethernet routers are more than capable for their wired Ethernet needs. If I was going to bother with a custom build router I think I would want the wireless to be also capable of three stream 802.11 ac instead of only N.

I wonder how much that cost compared to just getting the Asus RT-AC66U and whether there would be any noticeable performance difference between the Asus and the custom build on a 100Mbps down (DHCP) cable connection. My RT-N66U has already been shown to be capable of easily handling over thirty simultaneous connections with wireless connections on 2.4Ghz and 5Ghz. Your custom build sounds great and it will probably work well for you in the long run. But for the most cost effective solution if I had your 300Mbps down fiber connection (over PPPoE) with your ISP provided equipment then as I suggested a while back several times with my first posts I would have simply added a RT-AC66U (or RT-N66U) after the ISP provided router as an AP or second router using double NAT and been done with it.

ISP dual processor modem -$0
Asus RT-AC66U on sale - $179

 

[rotor]

Thank you for the cost breakdown. I might build one for the fun of it but the capabilities of the wireless part of your particular build is what concerns me. It seems to be the consensus opinion that the main reason the RT-N66U isn't working out for your application is the PPPoE processing overhead. In this day and age most people are concerned with wireless performance since most gigabit Ethernet routers are more than capable for their wired Ethernet needs. If I was going to bother with a custom build router I think I would want the wireless to be also capable of three stream 802.11 ac instead of only N. I wonder how much that cost compared to just getting the Asus RT-AC66U and whether there would be any noticeable performance difference between the Asus and the custom build on a 100Mbps down (DHCP) cable connection. My RT-N66U has already been shown to be capable of easily handling over thirty simultaneous connections with wireless connections on 2.4Ghz and 5Ghz.

The beauty of the home build is that you can replace the mini PCIe cards with whatever you want. 802.11ac seems completely pointless at the moment, as there are literally no clients for it. But in a couple of years, there will be plenty of 802.11ac mini PCIe cards to choose from, and it will be more appropriate as there will be actual use for it. Buying the Asus AC router right now you are being an early adopter, and as such you aren't getting a very good deal, plus the standard could change, newer chipsets may be more robust, more reliable, faster, etc.

The Asus routers are capable of 700+ Mbps of raw throughput -- it's only the PPPoE that puts a heavy load on the CPU. So if you have "only" 100 Mbps *and* you don't require PPPoE, then the Asus routers are a dead-simple answer to all your requirements. Building your own is of course a lot more fun!

My intention, if I end up building my own, will be to just have 2.4GHz (the motherboard I want has 2 x mini PCIe slots), and when AC becomes more commonplace, add it then.

 

[SoCalReviews]

I know this isn't your fault, but I would personally slap whoever at your ISP decided to put a 300 Mbits connection through a PPPoE session. In addition to the CPU load, you are wasting a LOT of that bandwidth just in PPPoE overhead. Any sane ISP would use straight DHCP at those kind of speed, not PPPoE.

Regarding power usage: keep in mind that the RT-N66U idle and full load power usage should be nearly the same.

Merlin, PPP is a very old technology going back to the days of dial-up internet. At the time it wasn't designed with modern day ultra high speed internet in mind. We know there is a hit taken for the extra processing required for PPPoE but I would be curious to know what you think the effective loss in total bandwidth is with PPPoE compared to DHCP....5%...10% or more?

This reference link seemed to explain some of the issues when using PPPoE...

http://revk.www.me.uk/2010/12/everything-you-wanted-to-know-about.html

 

[SoCalReviews]

The beauty of the home build is that you can replace the mini PCIe cards with whatever you want. 802.11ac seems completely pointless at the moment, as there are literally no clients for it. But in a couple of years, there will be plenty of 802.11ac mini PCIe cards to choose from, and it will be more appropriate as there will be actual use for it. Buying the Asus AC router right now you are being an early adopter, and as such you aren't getting a very good deal, plus the standard could change, newer chipsets may be more robust, more reliable, faster, etc.

The Asus routers are capable of 700+ Mbps of raw throughput -- it's only the PPPoE that puts a heavy load on the CPU. So if you have "only" 100 Mbps *and* you don't require PPPoE, then the Asus routers are a dead-simple answer to all your requirements. Building your own is of course a lot more fun!

My intention, if I end up building my own, will be to just have 2.4GHz (the motherboard I want has 2 x mini PCIe slots), and when AC becomes more commonplace, add it then.

I like having the home built router option. Right now I'm just not certain I want to pay the extra cost for it or the extra time required to set it up when I have so many other projects I can do to improve my home network. In another year the RT-AC66U cost might drop to only $100 USD and you may end up wanting to add it or another router model as an AP or second router anyway. Because it's designed specifically to efficiently handle connections on the 2.4Ghz and 5Ghz bandwidths with multiple guest accounts it would probably be more cost effective and work much better than if you tried to add multiple 802.11 ac PCIe cards to configure as APs in your custom Intel Atom home built router.

 

[RMerlin]

Merlin, PPP is a very old technology going back to the days of dial-up internet. At the time it wasn't designed with modern day ultra high speed internet in mind. We know there is a hit taken for the extra processing required for PPPoE but I would be curious to know what you think the effective loss in total bandwidth is with PPPoE compared to DHCP....5%...10% or more?

This reference link seemed to explain some of the issues when using PPPoE...

http://revk.www.me.uk/2010/12/everything-you-wanted-to-know-about.html

About 45-ish bytes per 1500 bytes packet (I don't recall the exact numbers, I only got the 1454 bytes per packet figure in my mind). Maybe that included other overhead caused by ATM, it's been years since I've looked into this I'm afraid.

 

[rotor]

It is 8 bytes per 1500-byte packet, or 0.5% -- i.e. nothing.

See the MTU/MRU section here:
http://en.wikipedia.org/wiki/Point-to-point_protocol_over_Ethernet

Its effect is in potentially causing packet fragmentation when 1500 byte packets have to be transmitted, and the router has to split them so they fit in the 1492 byte PPP packet.

 

[RMerlin]

It is 8 bytes per 1500-byte packet, or 0.5% -- i.e. nothing.

See the MTU/MRU section here:
http://en.wikipedia.org/wiki/Point-to-point_protocol_over_Ethernet

Its effect is in potentially causing packet fragmentation when 1500 byte packets have to be transmitted, and the router has to split them so they fit in the 1492 byte PPP packet.

I've definitely seen higher overhead figures mentioned back when I was using DSL myself, I just can't remember the actual explanation.