CakeQOS CakeQoS-Merlin v2.0.0

[dave14305]

The next major feature to be added will be iptables rules to influence tin selection for upload traffic when using diffserv3 or higher. In general, that is easy enough to port from FlexQoS.

I’m less enamored with the idea of traffic rules for downloads using tc and skbedit because there’s a lot less flexibility (or maybe more complexity) than with iptables. The approach of using ipsets is interesting but requires that whole ipset and (optionally) dnsmasq layer to be managed.

I feel the most value will come from the upload traffic rules since most people have a more restrictive upload bandwidth than download bandwidth. And Cake doesn’t really require so much twiddling as you might think.

I’m soliciting ideas now for how this could all work, using the FlexQoS system as a starting point, albeit without the benefit of the TrendMicro classification and app awareness.

Rules would only be based on local and remote IPs and ports, and protocol. Instead of ”QoS Class” you would choose a DSCP value (e.g. af11, cs7, ef, etc.). You won‘t be able to directly choose a Tin because valid Tins could change if you switch from diffserv3 to diffserv4 or diffserv8. Choosing the DSCP value will map into the active diffserv option per the cake built-in mapping of DSCP to Tin. Granted, few people know what a DSCP value really means.

Time to get out pen and paper and start brainstorming,

 

[Kingp1n]

The next major feature to be added will be iptables rules to influence tin selection for upload traffic when using diffserv3 or higher. In general, that is easy enough to port from FlexQoS.

I’m less enamored with the idea of traffic rules for downloads using tc and skbedit because there’s a lot less flexibility (or maybe more complexity) than with iptables. The approach of using ipsets is interesting but requires that whole ipset and (optionally) dnsmasq layer to be managed.

I feel the most value will come from the upload traffic rules since most people have a more restrictive upload bandwidth than download bandwidth. And Cake doesn’t really require so much twiddling as you might think.

I’m soliciting ideas now for how this could all work, using the FlexQoS system as a starting point, albeit without the benefit of the TrendMicro classification and app awareness.

Rules would only be based on local and remote IPs and ports, and protocol. Instead of ”QoS Class” you would choose a DSCP value (e.g. af11, cs7, ef, etc.). You won‘t be able to directly choose a Tin because valid Tins could change if you switch from diffserv3 to diffserv4 or diffserv8. Choosing the DSCP value will map into the active diffserv option per the cake built-in mapping of DSCP to Tin. Granted, few people know what a DSCP value really means.

Time to get out pen and paper and start brainstorming,

@dave14305...this sounds like you will take CAKE to another level. Can't wait to see what's to come...

I did have a question in regards to future traffic rules on the upload. Would this mean similar traffic rules can be implemented (like Flex) for gaming and streaming within CAKE like identifying the IP for game consoles or streaming apps or am I way off on this?

Thanks for all your hard work on this!!! Much appreciated!

 

[dave14305]

I did have a question in regards to future traffic rules on the upload. Would this mean similar traffic rules can be implemented (like Flex) for gaming and streaming within CAKE like identifying the IP for game consoles or streaming apps or am I way off on this?

In theory yes, but you're still limited to few CAKE tins (Bulk, Best Effort, Video, Voice) so where would you tin Gaming? I dunno.

 

[Wade Coxon]

I feel the most value will come from the upload traffic rules since most people have a more restrictive upload bandwidth than download bandwidth. And Cake doesn’t really require so much twiddling as you might think.

I agree in part about your reasoning here, though I would suggest that you could achieve one big win on the download side by at the very least being able to classify Peer to Peer file transfers (torrent etc) to the bulk tin.
These are the only sorts of transfers that are really going to hammer a download link due to the large number of individual flows carrying high volumes of data.

Things like FTP and Web downloads can suck up a lot of bandwidth too, but since they will generally limit themselves to a single connection, the dual-srchost and dual-dsthost flow isolation will mostly mitigate those.

So, no need to be fancy on download. Dump torrents(etc) into bulk, and leave the rest in besteffort.

 

[Wade Coxon]

In theory yes, but you're still limited to few CAKE tins (Bulk, Best Effort, Video, Voice) so where would you tin Gaming? I dunno.

I think it is best to not overthink it with cake. Since cake on its own already does such a good job, then you only need "high priority", "low priority" and "everything else".

Leave high priority for things that need low latency. No need to break these down into sub-categories.
Network Control, Voice, Gaming, NTP

Low priority in the Bulk tin for anything that is large volume and cab happen in the background.
PtP, FTP

Then just let the rest fall into besteffort.

 

[dave14305]

I agree in part about your reasoning here, though I would suggest that you could achieve one big win on the download side by at the very least being able to classify Peer to Peer file transfers (torrent etc) to the bulk tin.
These are the only sorts of transfers that are really going to hammer a download link due to the large number of individual flows carrying high volumes of data.

Things like FTP and Web downloads can suck up a lot of bandwidth too, but since they will generally limit themselves to a single connection, the dual-srchost and dual-dsthost flow isolation will mostly mitigate those.

So, no need to be fancy on download. Dump torrents(etc) into bulk, and leave the rest in besteffort.

How would you identify torrent traffic within the confines of tc filter? Never used torrents so it’s a sincere question.

 

[Wade Coxon]

How would you identify torrent traffic within the confines of tc filter? Never used torrents so it’s a sincere question.

I don't know the inside-out's of tc filtering, but you could identify by TCP port 6881-6889 as the generally used ports.

 

[dave14305]

I don't know the inside-out's of tc filtering, but you could identify by TCP port 6881-6889 as the generally used ports.

You can experiment with this:

insmod em_cmp.ko
tc filter add dev ifb4eth0 parent 8155: protocol ip prio 30 basic match "cmp(u16 at 0 layer transport gt 6880) and cmp(u16 at 0 layer transport lt 6890)" action skbedit priority 8155:1

Ref: https://serverfault.com/a/544790
Replace 8155: with your own cake handle from ifb4eth0, and set download to diffserv3.

I found that changing download from besteffort to diffserv3 resulted in most of my incoming traffic getting put into bulk, but I’m on Comcast who seems to like CS1. So this is unusable for me.

 

[Wade Coxon]

You can experiment with this:

insmod em_cmp.ko
tc filter add dev ifb4eth0 parent 8155: protocol ip prio 30 basic match "cmp(u16 at 0 layer transport gt 6880) and cmp(u16 at 0 layer transport lt 6890)" action skbedit priority 8155:1

Ref: https://serverfault.com/a/544790
Replace 8155: with your own cake handle from ifb4eth0, and set download to diffserv3.

I found that changing download from besteffort to diffserv3 resulted in most of my incoming traffic getting put into bulk, but I’m on Comcast who seems to like CS1. So this is unusable for me.

Doesn't the wash setting take care of that upstream marking? (Or does the wash happen after tc filter takes effect this nullifying its usefulness?)

Anyhoo, thanks for the tip, however I just seem to get an error. After replacing parent with my current value of 8002.

RTNETLINK answers: No such file or directory
We have an error talking to the kernel

 

[dave14305]

Doesn't the wash setting take care of that upstream marking? (Or does the wash happen after tc filter takes effect this nullifying its usefulness?)

Anyhoo, thanks for the tip, however I just seem to get an error. After replacing parent with my current value of 8002.

RTNETLINK answers: No such file or directory
We have an error talking to the kernel

Did you replace both occurrences of 8155, and run the insmod?

Wash happens after cake has put the traffic in a tin. And tc filter happens before cake sees the traffic.

 

[Wade Coxon]

Did you replace both occurrences of 8155, and run the insmod?

Wash happens after cake has put the traffic in a tin. And tc filter happens before cake sees the traffic.

Ahh, complete comprehension error on my part. I had replaced both the parent numbers, but completely forgot the kernel module.
Command works now.....just need to experiment.

 

[ttgapers]

Rules would only be based on local and remote IPs and ports, and protocol. Instead of ”QoS Class” you would choose a DSCP value (e.g. af11, cs7, ef, etc.). You won‘t be able to directly choose a Tin because valid Tins could change if you switch from diffserv3 to diffserv4 or diffserv8. Choosing the DSCP value will map into the active diffserv option per the cake built-in mapping of DSCP to Tin. Granted, few people know what a DSCP value really means.

I think this is the best idea going forward to keep it simple. Upload is also usually the bottleneck....

 

[dave14305]

I think this is the best idea going forward to keep it simple. Upload is also usually the bottleneck....

I haven't found any good reference implementations of manipulating CAKE download traffic, except the OpenWRT implementations that rely on the custom connmark solution which isn't available on the older ASUS Linux kernels.

Even the smart SQM guys at OpenWRT admit that download classification is hard with CAKE.

I have to look back and see what conclusions were drawn when testing CAKE on the br0 interface instead of ifb4eth0.

 

[Wade Coxon]

I have to look back and see what conclusions were drawn when testing CAKE on the br0 interface instead of ifb4eth0.

Didn't that end up throttling LAN/WLAN traffic?

 

[dave14305]

Didn't that end up throttling LAN/WLAN traffic?

I do remember reading that. I don't personally see much LAN->WLAN traffic, so it was easily forgotten by me as a side-effect.

So what happened with the torrent rule? Did it even work at all?

 

[Wade Coxon]

So what happened with the torrent rule? Did it even work at all?

Sorry for not following up. I did have a quick experiment with it, and played with the port values, but nothing was making its way into the bulk tin.

Perhaps just for experimentation purposes, using FTP port 21 is an easier target to use for trials.

 

[dave14305]

Sorry for not following up. I did have a quick experiment with it, and played with the port values, but nothing was making its way into the bulk tin.

Perhaps just for experimentation purposes, using FTP port 21 is an easier target to use for trials.

Was the port range you gave the remote port (source) or the local port (destination)? The rule I posted assumed it was the remote source port for incoming download traffic.

 

[Audionut]

You can effectively use any port for bitorrent traffic, which means that the remote ports are determined by the settings chosen by the remote peer. If you are connected to 30 peers, each peer can be using its own unique port.

IMO, the local bitorrent port will need to be a fixed port number (somewhere 50,000+ is probably preferable) and have cake determine which bin to use based on the destination port of the incoming traffic.

Any port (source) > fixed port (local) > specified bin.


edit: If I change the parameters as such, would this work on the download? (xxxx being the cake handle and yyyyy being the local port)

tc filter add dev eth0 parent xxxx: protocol ip prio 30 basic match "cmp(u16 at 0 layer transport gt yyyyy) and cmp(u16 at 0 layer transport lt yyyyy)" action skbedit priority xxxx:1

 

[Audionut]

edit: If I change the parameters as such, would this work on the download? (xxxx being the cake handle and yyyyy being the local port)

tc filter add dev eth0 parent xxxx: protocol ip prio 30 basic match "cmp(u16 at 0 layer transport gt yyyyy) and cmp(u16 at 0 layer transport lt yyyyy)" action skbedit priority xxxx:1

Trying that throttled the torrent download speed considerably. There was certainly data being captured in the bulk tin. Not all data, but around 40% at a rough guess. This was on the upload also.

Applying the same as quoted, but to ifb4eth0, made no difference at all. The download was all going to the besteffort tin as usual.

edit: This command is defiantly capturing the upload and putting it into the bulk tin.

tc filter add dev eth0 parent 8001: protocol ip prio 30 basic match "cmp(u16 at 0 layer transport gt 63330) and cmp(u16 at 0 layer transport lt 63339)" action skbedit priority 8001:1

edit again: On the download, tried turning off wash, just in case, but it made no difference. Changing to diffserv4 made no difference.
Changing cake parameters in the GUI resets and changes the cake handles, requiring the commands to be entered again.

 

[dave14305]

Trying that throttled the torrent download speed considerably. There was certainly data being captured in the bulk tin. Not all data, but around 40% at a rough guess. This was on the upload also.

Applying the same as quoted, but to ifb4eth0, made no difference at all. The download was all going to the besteffort tin as usual.

edit: This command is defiantly capturing the upload and putting it into the bulk tin.

tc filter add dev eth0 parent 8001: protocol ip prio 30 basic match "cmp(u16 at 0 layer transport gt 63330) and cmp(u16 at 0 layer transport lt 63339)" action skbedit priority 8001:1

edit again: On the download, tried turning off wash, just in case, but it made no difference. Changing to diffserv4 made no difference.
Changing cake parameters in the GUI resets and changes the cake handles, requiring the commands to be entered again.

To specify the destination port, try with u16 at 2 instead of u16 at 0, still on ifb4eth0.