Best Router for Packet Priortization

Trip

Very Senior Member
Generally, if you want something that's simple to setup and just works, look for it to support fq_codel. The benefit of fq_codel is that the algorithm does all the work and distributes bandwidth fairly while distinguishing between small(ie gaming) packets, and large(ie plex) downloads to minimize latency and bufferbloat where needed. Pretty much all you need to do is set the max limits and you don't need to worry about priorities, etc.
To refine this, fq_codel creates per flow fairness, which doesn't necessarily equate to per host fairness, nor does it mean that any one connection will be guaranteed max performance, just fair (or better), based on whatever flows are active in-queue.
Support for fq_codel has very little purpose other than when you saturate your internet pipe. It just shows up on speedtest because that is what you are doing. In every day use it has little to no impact unless you have a very slow DSL internet pipe. And remember you can over saturate your internet pipe so badly your connection will time out when you run out of buffers even using fq_codel. So the use has a very narrow margin of use. Mainly speedtest.
Largely incorrect, and a vast dumbing-down of SQM in general.

fq_codel's purpose is to ensure flow fairness. That's it. Only when combined with a rate limiter such as HTB does it employ the de-bloating results so well attributed to its incarnation. Independent of that, flow fairness is still being applied via 1024 FIFO queues and is still smoothing traffic flow regardless of bandwidth utilization.

As for applicable use cases, "very slow DSL" is very much the prime example, but improvements can be seen just as well across other mediums, be they cable, fiber, etc. Innate latency may be lower on those mediums, minimizing the round-trip effects of de-bloating those links, yes, but as long as the link speed delta across interfaces remains larger than a 5x or so sum (ie. a 1Gb/1Gb/s LAN egressing to a typical 300/20 cable line or a 200/200 fiber line), you'll tend to see quite a improvement with fq_codel plus a rate limiter included. If you are simply lucky enough to just "buy more bandwidth", then that's a great out, but as I've mentioned before, a good mass of end-users globally are not in that situation. In the US, particularly, the national average of download/upload is nowhere near a value that couldn't at least be moderately to significantly helped by fairness-flow queuing, on any WAN medium, irrespective of whether the link is being saturated. fq_codel, in particular, builds its effectiveness on a gradient curve. It's not saturation based.

Perhaps delve into some bufferbloat docs and educate on where the benefits really lie, and how best to apply SQM in use-cases where it's clearly warranted, which, even today, number way beyond "niche".
 
Last edited:

coxhaus

Part of the Furniture
I use to work on circuits that were saturated for a year and more and some for months or more back in the old days. Cisco did a pretty good job back then in their enterprise routers. I kind of doubt consumer routers could tolerate this for that long.

I assume there is some kind of adjustment incase you do VOIP in terms of codel. I have not run VOIP and codel together. I have only used Cisco with VOIP.

I ran pfsense and I set up their codel and I saw no improvement while using it for over a year. I turned it on and off for a while.

I played around with low latency and speedtest which does not report as well. So I am not sure getting a good speedtest is the best for gamming.

Codel may be the best for fair sharing of the internet pipe when you fill it up but there may be other cases which need changes. So I don't think it is the end all for routers.
 
Last edited:

Trip

Very Senior Member
All due respect, cox, things have changed since then (15+ years ago I take it?) and there is a multiplied amount of edge compute power today, which has reached many an SMB and consumer router alike. Comparatively, I think you might be surprised how much one could tolerate, apples to apples.

Regarding accommodating for VoIP, or any specific traffic for that matter, I think you may not be understanding the concept here. Coordinated delay and fairness queuing does not class on traffic type; it classes on frame/packet size and time. There need not be any "adjustment" for any type of traffic... because there doesn't need to be. That's the beauty of it. The old-school, legacy classing approach goes out the window here (as it rightfully should), because it doesn't address the root cause of what codel/fq_codel were created to address: the bloat itself. Being front-of-the-line to a stacked up queue doesn't do you any good (old-school QoS).... a mechanism to prevent/control queue depth to begin with does (SQM). Make more sense now? If not, I'd urge you read More About Bufferbloat: What's Wrong With Simply Configuring QoS?

While you may have played around with pfSense and their implementation of codel (not fq_codel, I take it?), depending on when you did so, it was a far cry from being implemented properly (due to the porting process, pfSense's routing engine, and more), which could have very likely explained the results you saw, even if you had parameters tweaked to perfection.

The last point is one I actually agree on, which is of course that nothing alone is a silver bullet, and more high-quality, peer-reviewed research needs to be done to vet the applicability of SQM across varying use-cases, including WAN mediums, link speeds, etc.
 

Bensam123

Occasional Visitor
I own an XR500 router running the Netduma firmware and I personally wouldn't recommend it. It lacks too many necessary features and has way too many bugs. For instance, having IPv6 enabled on the router breaks their bufferbloat system and can sometimes lock up WAN entirely. Their GUI always reports wrong values and has glitches as well. Also, the firmware support is absolutely abysmal and so far we haven't gotten anything significant for well over a year. Just head on over to Netduma's forums and see for yourself all the owners complaining about their issues with the router and the company's lack of support.

And so, I too, once again, have set off on a quest to find the one router to rule them all. Sadly it seems that ALL consumer routers sold in stores have amazing hardware but really crap firmware to run it. Ubiquity Dream Machine and the Synology RT2600AC seem like they may have very good firmware running but I can't find any info anywhere about what types of QOS or SQM that they offer.

Recently I've even tried flashing OpenWrt on one of my routers in hopes of getting CAKE but immediately ran into stability issues right out of the box.
Untangle is pretty good although it is completely missing MAC cloning. I'm still using my 86u with Asus Merlin on it and the modified QoS list here. It's still not what I'm looking for, but better then pretty much all the other options till something that allows rearranging packets comes into play.

Kongs build of DDWRT used to be much better, but there was some fallout on their forums and he moved on to openwrt, which also doesn't have working mac cloning... Hilarious. The newer builds of DDWRT are very bug filled and have performance issues. I found this out through about 8 hours of troubleshooting before someone answered my question on their forums and I had to change a wireless switch option because I offended the router apparently.

OpenWRT is also a buggy mess as are a lot of open source firmwares that are hobby projects with the exception of works of love, DDWRT (used to) being one Merlin being another. You need to enjoy fixing broken things to make it work in your favor, which isn't one of my most enjoyable activities for a router.
 

coxhaus

Part of the Furniture
Untangle is pretty good although it is completely missing MAC cloning. I'm still using my 86u with Asus Merlin on it and the modified QoS list here. It's still not what I'm looking for, but better then pretty much all the other options till something that allows rearranging packets comes into play.
If you run Untangle as a transparent bridge behind a router which is one of the install options you can run a router which supports MAC cloning and still be able to run Untangle. You just can't do it with an all-in-one router as the name implies.
 

coxhaus

Part of the Furniture
So Trip I read your stuff and I still think it is the same. It is good fq_codel is making it into the Linux kernel. And maybe we will get support from the ISP side when their side ends up with buffer bloat. It sounds like it is auto Q management. It does not mention how many Qs. It really does not talk about UDP and TCP specifically. TCP will retransmit but UDP does not. So how granular it is I don't know. Maybe Q management is based on how sophisticated the hardware is? I still think it comes into play meaning you will see a difference when you are maxing out your internet pipe if you are using any kind of good router. Maybe there is such bad routing coding out there that you need it all the time on some routers? Cisco has been writing router code so many years I bet they use their own, long before this was available in the kernel. I guess there is another reason to get router code kernels up to levels that support this. I have been pushing this for a long time. I think the secret is in the Qs and how they are setup. Using balancing and gaming in the same sentence is kind of an oxymoron. Just because you backup 1 Q does not mean you are backing up all the Qs.

If this has got you all excited then you really to take a look at layer 3 switching. Routers are so much snappier when the L3 switch does all the local work like DHCP, local routing, etc. This you can see all the time not just when the internet pipe is full. Plus there is Q management in the L3 switches. The Cisco L3 switch go to 8 Qs. Well the Cisco L3 switch I setup for my daughter's VOIP IP phones. There is a thread on here some where.

PS
I found over on OpenWrt Smart Queue Management where it talks about UDP traffic still working well with a loaded internet pipe. So could it be that it is just UDP traffic getting priority over TCP traffic? This keeps your voice, DNS, video, and any UDP traffic running smoothly. Of course if you run too much 4K video you might slowdown your data, TCP traffic, to where it times out not just retransmits. It does compensate for ATM traffic vs Ethernet.
 
Last edited:

Trip

Very Senior Member
Good discussion here.

fq_codel uses 1024 FIFO queues by default, but can be adjusted from 1 to 65535 as desired (per RFC 8290, Sec. 1.3, paragraph 3). That RFC does reference TCP, mostly regarding transmit behavior and packet size. UDP is much less covered, indeed, but mentioned briefly in the mailing list (link) with Hoeiland-Joergensen and Collier-Brown commenting that it's a bit of an "it depends" scenario, with a limited amount of flow isolation able to be done with UDP (at best).

I'm a fan of working the above stuff in with local heavy lifting done on switches, for sure. The best of all worlds is probably a combo of both. L3 switching and classic QoS to get packets processed locally and ordered properly when delivered to the router (ie. first-in-queue, queues prioritized), then SQM at the WAN to optimize ingress/egress.
 

coxhaus

Part of the Furniture
Trip I guess you picked up on cake is better and faster than fd_codel?

I have no idea what they will use for 10 gig but at least the buffers are going to need to be bigger and faster. Can the kernel support this?
 
Last edited:

Trip

Very Senior Member
Trip I guess you picked on cake is better and faster than fd_codel?
Great question. Short answer is, mostly CAKE, but depends. For all its superiority -- per host fairness (not just per flow), ECN built-in, one-line deployment vs. having to create an fq_codel + shaper equivalent -- CAKE often uses more CPU per unit throughput of de-bloat, so for certain applications fq is actually more optimal, such as when you're CPU-limited with no beefier CPU option. Otherwise, though, CAKE is generally a preferential choice.
I have no idea what they will use for 10 gig but at least the buffers are going to need to be bigger and faster. Can the kernel support this?
I suspect the same dance will take place all over again: de-bloating several Gb/s or less from a 10Gb LAN to a 1Gb+ WAN -- just a 10x higher rate of exchange. Regarding what will be the bottleneck, presuming these items remain non-offloadable, I would think it would continue to be the consumer/SOHO edge device SoCs (ARM, etc.) primarily.

As for the Linux kernel being able to handle it, I presume it will, if not out-of-the-box, then with certain tweaks. If you read something like this Intel 10Gx base kernel driver spec (from almost two years ago now, by the way), buffers seem to be software-scalable via RAM allocation, and I'm sure similar capacities will be designed into lower-level SoCs, to where the operators (us) won't have to be worried about it, at least not by the time 10Gb fully commoditizes over the next few years.

On the LAN side (as you're familiar with the purchase of your SG350X stuff), 10Gb is there and ready; it's just a half to a whole order magnitude pricier than where it needs to be to start gaining mass adoption. I trust that will change over the next 2-5 years.
 
Last edited:

coxhaus

Part of the Furniture
I can tell you back in the old days we had to use ATM for video conferencing since we were all over the state of Texas and the tools were not there to use Ethernet. Times have changed.
 

Trip

Very Senior Member
Yep, formalized in RFC 8033 (Feb, 2017). PIE is more implementable at the hardware level -- aka modem code/silicon, whereas a codel-based solution would need to have a bit more underneath it to be roughly as effective. I wish Cisco had also supplied their gateways with at least a couple of these algorithms; hopefully they may appear in the next-gen of RVs/ISRs.

Here's a brief commentary on the overall comparative state of the algorithms (as of Jan 2020 at least) from Dave Taht on the Phoronix forums.
 

coxhaus

Part of the Furniture
Yep, formalized in RFC 8033 (Feb, 2017). I wish Cisco had also supplied their gateways with at least a couple of these algorithms; hopefully they may appear in the next-gen of RVs/ISRs.
Cisco gives you more than any consumer router. They give you the tools to set all kinds QOS like in big routers just not as robust. I have included a pic. You can build 3 types of traffic, priority. rate control, and low latency. Look at all the options on the lower left for menu options. You can even QOS the switch part. Way more flexible than these small routers except maybe Mikrotik.

Besides Codel is a moving target. First Codel then fd_codel, now cake. Cake is going to replace fd_codel soon. What is next. Better to have tools to build your own the way you want. Like I said I could break codel, fd_codel, and probably cake. I have not read up on cake. IF you overload one of these with enough UDP traffic it will break. This is an extreme case but if you are getting hammered then it could be done. With Cisco you can tune for that. Cisco is way more flexible. It is the engineering solution.

As far as I know Cisco IOS is not Linux only their small business networking devices. Which is a small part of Cisco. So I don't expect to see fd_codel when Cisco gives you big router tools so you can tune for any situation in a small business. Plus it will be easier to migrate to Cisco IOS pro network hardware in case your business grows.

I am running priority in my RV340 router and I have a feeling it is PIE. It handles buffer bloat. I guess it could be a form of codel but it can be tuned. If anybody knows speak up.

Here is what Cisco states in the RV340 router sales brochure for QoS.
"Traffic Classes, WAN Queuing, WAN Policing, WAN Bandwidth Management, Assign detailed QoS (Class of Service (CoS)/Differentiated Services Code Point (DSCP)/policies) settings per application or end device"
 

Attachments

Last edited:

Trip

Very Senior Member
Re- RV QoS "low latency" WAN queue option, I'd wager is just a stripped-down version of LLQ, which is strict priority queuing combined with CBWFQ (class-based weighted fair queuing), which still lacks any ability to de-bloat dynamically. A step in the right direction, but likely still more bloat-prone than other alternatives.

Also, IOS XE is Linux based, and is where most IOS platforms are moving to.
 
Last edited:

coxhaus

Part of the Furniture
Yes I think so. Here is a link. Scroll down to low latency. I can't find anything directly on the RV340 but I think this is close. Cisco upgraded QOS in the RV340 router firmware along the way. When Cisco finds new features they add them. The old RV340 router QOS is gone on my firmware and looks more like what I posted.

https://www.cisco.com/c/en/us/suppo...-quality-of-service-qos-queue-settings-o.html

I know nothing about IOS XE. I don't think it existed back when I worked. It seems to be IOS Linux based. I am not sure what devices run it.

If you are trying to get the lowest latency then you should not be concerned with buffer bloat.
I am no expert but fd_codel is trying to keep all the UDP packets running evenly and doing the best job they can balancing TCP. This does not give you the lowest latency but the best all around latency which is different. A problem with fd_codel is when it drops TCP packets it is dropping them at the front instead of the back which is hard to deal with.

PS
QOS Priority on the Cisco RV340 router is like Codel or fd_codel lowest latency is not. With lowest latency you are controlling the latency it is not automated. You are controlling what gets the lowest latency over everything else and I am sure there are penalties in terms of overall traffic like buffer bloat so you are changing priorities of data classes but you are giving the lowest latency to what ever class you decide over everything else. This is why I think lowest latency and buffer bloat don't go together. Codel is built around buffer bloat and the best overall latency.

Cisco is just giving you different ways to balance traffic, QOS, on a small internet pipe like priority, and low latency. Cisco also is giving you rate control for QOS but I have not played with it but assume it is based on the name.
 
Last edited:

Trip

Very Senior Member
If you are trying to get the lowest latency then you should not be concerned with buffer bloat.
I am no expert but fd_codel is trying to keep all the UDP packets running evenly and doing the best job they can balancing TCP. This does not give you the lowest latency but the best all around latency which is different. A problem with fd_codel is when it drops TCP packets it is dropping them at the front instead of the back which is hard to deal with.
I wonder if @dtaht might have a comment or two on all that?
 

Bensam123

Occasional Visitor
Since this thread is basically built around prioritizing low latency traffic, specifically gaming (however also very much VOIP), Cake is now available for newer Asus routers. Has anyone done extensive testing of cake vs fq_codel for gaming applications or voip for latency and consistency?

I personally have bought a R7800 and the cake implementation in DDWRT was less then stellar, generally performing worse then fq_codel. I bought it for a few different reasons including experimenting with voxel and openwrt after finding out Kong moved to it. As well as using Kongs original DDWRT which didn't include cake, but performed better then the newer versions by brain slayer.

Still not what I'm looking for, but another option.
 

Trip

Very Senior Member
Just because cake is available for newer Asus routers, doesn't mean it will run as well nor produce as good a result(s) versus Qualcomm running OpenWRT. Some objective testing would be appreciated by those who have both sets of hardware.

On that note, I'd put OpenWRT on the R7800 instead of DD-WRT for running Cake. You may find the results to be a bit better.
 

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top