FlexQoS FlexQoS 1.0 - Flexible QoS Enhancement Script for Adaptive QoS

[JohnD5000]

I measure it by how hard it is to button my pants...oh, you mean bufferbloat...

I'm not too fanatical myself about bufferbloat. I run the occasional dslreports speedtest, but mostly if I notice a MS Teams or Skype call breaking up a lot during the workday, I revisit my bandwidth settings. I'm not a gamer, so I don't have any games or apps telling me I have bad ping times, etc.

As everyone will soon discover (if they haven't already)...I'm not really a QoS expert at all. I've just studied the living daylights out of the FreshJR_QOS script.

Well, you know more about QoS than me. Is bufferbloat an actual item that can be checked, or is more like a general feeling of whether something seems fast or slow? Seems like there should be a better way to check, but only thing I know how to check is speedtest and its dl,ul and ping response

 

[Vexira]

Well, you know more about QoS than me. Is bufferbloat an actual item that can be checked, or is more like a general feeling of whether something seems fast or slow? Seems like there should be a better way to check, but only thing I know how to check is speedtest and its dl,ul and ping response

It can technically be checked and also if you are a gamer or running latency sensitive applications you can feel it.

 

[ugandy]

It can technically be checked and also if you are a gamer or running latency sensitive applications you can feel it.

adaptive qos did nothing for bufferbloat for me. it did prevent packet loss on saturated lines.

 

[Vexira]

Fair enough. I never reached more than 750 connections on my own network to see how it behaved in real life. I just lifted it straight from Merlin's classification page. I've reverted that particular change for now with v0.8.1. Check for an update.

I'd like to think I can add this back using a session cookie to disable it on initial load, but let the user override it for the rest of the session. I'll need to plagiarize some of @Jack Yaz cookie code.

I've often wondered how the nf_contrack plays a role in that 750 and what the correlation is with QoS and it.

More of the relationship between nf_contrack expectation table and QoS.

 

[Vexira]

adaptive qos did nothing for bufferbloat for me. it did prevent packet loss on saturated lines.

Are you sure that you set it up correctly, also which router are you using?

I guess I'm lucky in that case because it worked for me, I've been experimenting with 90-95% bandwidth and the overhead value which depends on the value set will affect total bandwidth when you run a speed test.

 

[ugandy]

Are you sure that you set it up correctly, also which router are you using?

yes. ax88.
followed instructions. had to use 'automatic bandwidth' option because my UL speed swings during the day.

 

[JohnD5000]

So, I had FlexQoS loaded, a speedtest run on the modem (spdMerlin) results in around 235-240, but speedtest run on my laptop = 5 -6. Set bandwidth setting from manual (220) to automatic and speedtest returns to more normal number Set QoS off, again, more normal. Uninstall FlexQoS and use standard Merlin QoS and set manual (220) low speeds (around 4). Bandwidth setting to auto back to 235? What am I doing wrong? Speedtest is Mbps and Merlin setting is Mb/s. Do I need to adjust for that?

 

[Vexira]

yes. ax88.
followed instructions. had to use 'automatic bandwidth' option because my UL speed swings during the day.

Did you try spdmerlin the speed test script automatic bandwidth is not advisable.

 

[ugandy]

Did you try spdmerlin the speed test script automatic bandwidth is not advisable.

that's my problem. my upload line varies from 2Mbps to 20Mbps during workhours, making it impossible to pick a fixed limit for UL bandwidth. not flexqos fault

 

[Vexira]

that's my problem. my upload line varies from 2Mbps to 20Mbps during workhours, making it impossible to pick a fixed limit for UL bandwidth. not flexqos fault

Ahh I see yes well there is an automatic speed test script, spdmerlin Im planning to try it soon when I have space time to set it up.

 

[andresmorago]

hi
i just installed flexqos on my ac3100. i came from former freshjr script. so far, its running good. thanks @dave14305 for all the hard work.

I have a couple questions so far. sorry if they have been covered before.
*speedtest.net is going to net control packets. is this normal? how can i re-categorize it? When running a speedtest i dont see any connections tagged with net control category (red), just the pie and class data increasing.
*at the webgui i see a classification tab and the flexqos tab. is it normal to have the classification tab?
*what is the Added connection table counts mentioned in last changelog?
*here is my debug. am i good so far?

Spoiler

Debug:

Undf Prio: 2
Undf FlowID:
Classes Present: 8
Down Band: 151040
Up Band  : 9728
***********
Net Control = 1:10
Work-From-Home = 1:11
Gaming = 1:17
Others = 1:16
Web Surfing = 1:12
Streaming = 1:13
Downloads = 1:14
Defaults = 1:15
***********
Downrates -- 22656, 30208, 30208, 15104, 15104, 7552, 22656, 7552
Downceils -- 151040, 151040, 151040, 151040, 151040, 151040, 151040, 151040
Downbursts -- 7998b, 36793b, 17598b, 7998b, 6398b, 4799b, 3199b, 3199b
DownCbursts -- 188800b, 188800b, 188800b, 188800b, 188800b, 188800b, 188800b, 188800b
***********
Uprates -- 1459, 1945, 1945, 972, 486, 486, 1945, 486
Upceils -- 9728, 9728, 9728, 9728, 9728, 9728, 9728, 9728
Upbursts -- 3199b, 3199b, 3199b, 3199b, 3199b, 3199b, 3199b, 3199b
UpCbursts -- 11198b, 11198b, 11198b, 11198b, 11198b, 11198b, 11198b, 11198b
iptables settings: <>>udp>>500,4500>>3<>>udp>16384:16415>>>3<>>tcp>>119,563>>5<>>tcp>>80,443>08****>7<>>udp>>3478:3481>>3<10.0.0.2>>tcp>51416>>>2<10.0.0.2>>both>51415>>>5
iptables -D POSTROUTING -t mangle -o br0   -p udp  -m multiport  --sports 500,4500  -j MARK --set-mark 0x80060001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o br0   -p udp  -m multiport  --sports 500,4500  -j MARK --set-mark 0x80060001
iptables -D POSTROUTING -t mangle -o eth0   -p udp  -m multiport  --dports 500,4500  -j MARK --set-mark 0x40060001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o eth0   -p udp  -m multiport  --dports 500,4500  -j MARK --set-mark 0x40060001
iptables -D POSTROUTING -t mangle -o br0   -p udp  --dport 16384:16415   -j MARK --set-mark 0x80060001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o br0   -p udp  --dport 16384:16415   -j MARK --set-mark 0x80060001
iptables -D POSTROUTING -t mangle -o eth0   -p udp  --sport 16384:16415   -j MARK --set-mark 0x40060001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o eth0   -p udp  --sport 16384:16415   -j MARK --set-mark 0x40060001
iptables -D POSTROUTING -t mangle -o br0   -p tcp  -m multiport  --sports 119,563  -j MARK --set-mark 0x80030001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o br0   -p tcp  -m multiport  --sports 119,563  -j MARK --set-mark 0x80030001
iptables -D POSTROUTING -t mangle -o eth0   -p tcp  -m multiport  --dports 119,563  -j MARK --set-mark 0x40030001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o eth0   -p tcp  -m multiport  --dports 119,563  -j MARK --set-mark 0x40030001
iptables -D POSTROUTING -t mangle -o br0   -p tcp  -m multiport  --sports 80,443 -m mark --mark 0x80080000/0xc03f0000 -j MARK --set-mark 0x803f0001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o br0   -p tcp  -m multiport  --sports 80,443 -m mark --mark 0x80080000/0xc03f0000 -j MARK --set-mark 0x803f0001
iptables -D POSTROUTING -t mangle -o eth0   -p tcp  -m multiport  --dports 80,443 -m mark --mark 0x40080000/0xc03f0000 -j MARK --set-mark 0x403f0001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o eth0   -p tcp  -m multiport  --dports 80,443 -m mark --mark 0x40080000/0xc03f0000 -j MARK --set-mark 0x403f0001
iptables -D POSTROUTING -t mangle -o br0   -p udp   --sport 3478:3481  -j MARK --set-mark 0x80060001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o br0   -p udp   --sport 3478:3481  -j MARK --set-mark 0x80060001
iptables -D POSTROUTING -t mangle -o eth0   -p udp   --dport 3478:3481  -j MARK --set-mark 0x40060001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o eth0   -p udp   --dport 3478:3481  -j MARK --set-mark 0x40060001
iptables -D POSTROUTING -t mangle -o br0  -d 10.0.0.2  -p tcp  --dport 51416   -j MARK --set-mark 0x80040001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o br0  -d 10.0.0.2  -p tcp  --dport 51416   -j MARK --set-mark 0x80040001
iptables -D POSTROUTING -t mangle -o eth0  -s 10.0.0.2  -p tcp  --sport 51416   -j MARK --set-mark 0x40040001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o eth0  -s 10.0.0.2  -p tcp  --sport 51416   -j MARK --set-mark 0x40040001
iptables -D POSTROUTING -t mangle -o br0  -d 10.0.0.2  -p tcp  --dport 51415   -j MARK --set-mark 0x80030001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o br0  -d 10.0.0.2  -p tcp  --dport 51415   -j MARK --set-mark 0x80030001
iptables -D POSTROUTING -t mangle -o br0  -d 10.0.0.2  -p udp  --dport 51415   -j MARK --set-mark 0x80030001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o br0  -d 10.0.0.2  -p udp  --dport 51415   -j MARK --set-mark 0x80030001
iptables -D POSTROUTING -t mangle -o eth0  -s 10.0.0.2  -p tcp  --sport 51415   -j MARK --set-mark 0x40030001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o eth0  -s 10.0.0.2  -p tcp  --sport 51415   -j MARK --set-mark 0x40030001
iptables -D POSTROUTING -t mangle -o eth0  -s 10.0.0.2  -p udp  --sport 51415   -j MARK --set-mark 0x40030001 >/dev/null 2>&1
iptables -A POSTROUTING -t mangle -o eth0  -s 10.0.0.2  -p udp  --sport 51415   -j MARK --set-mark 0x40030001
appdb rules: <000000>6<00006B>6<0D0007>5<0D0086>5<0D00A0>5<12003F>4<13****>4<14****>4<1A****>5<0d0023>2<0d007e>2<0d00d1>2<0d0054>2
realtc filter del dev br0 parent 1: prio 2 > /dev/null 2>&1
realtc filter del dev eth0 parent 1: prio 2 > /dev/null 2>&1
realtc filter add dev br0 protocol all prio 2 u32 match mark 0x80000000 0xc000ffff flowid 1:16
realtc filter add dev eth0 protocol all prio 2 u32 match mark 0x40000000 0xc000ffff flowid 1:16
realtc filter add dev br0 protocol all prio 2 u32 match mark 0x8000006B 0xc03fffff flowid 1:16
realtc filter add dev eth0 protocol all prio 2 u32 match mark 0x4000006B 0xc03fffff flowid 1:16
realtc filter add dev br0 protocol all prio 15 u32 match mark 0x800D0007 0xc03fffff flowid 1:14
realtc filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0007 0xc03fffff flowid 1:14
realtc filter add dev br0 protocol all prio 15 u32 match mark 0x800D0086 0xc03fffff flowid 1:14
realtc filter add dev eth0 protocol all prio 15 u32 match mark 0x400D0086 0xc03fffff flowid 1:14
realtc filter add dev br0 protocol all prio 15 u32 match mark 0x800D00A0 0xc03fffff flowid 1:14
realtc filter add dev eth0 protocol all prio 15 u32 match mark 0x400D00A0 0xc03fffff flowid 1:14
realtc filter add dev br0 protocol all prio 20 u32 match mark 0x8012003F 0xc03fffff flowid 1:12
realtc filter add dev eth0 protocol all prio 20 u32 match mark 0x4012003F 0xc03fffff flowid 1:12
realtc filter del dev br0 parent 1: prio 22 > /dev/null 2>&1
realtc filter del dev eth0 parent 1: prio 22 > /dev/null 2>&1
realtc filter add dev br0 protocol all prio 22 u32 match mark 0x80130000 0xc03f0000 flowid 1:12
realtc filter add dev eth0 protocol all prio 22 u32 match mark 0x40130000 0xc03f0000 flowid 1:12
realtc filter del dev br0 parent 1: prio 23 > /dev/null 2>&1
realtc filter del dev eth0 parent 1: prio 23 > /dev/null 2>&1
realtc filter add dev br0 protocol all prio 23 u32 match mark 0x80140000 0xc03f0000 flowid 1:12
realtc filter add dev eth0 protocol all prio 23 u32 match mark 0x40140000 0xc03f0000 flowid 1:12
realtc filter add dev br0 protocol all prio 2 u32 match mark 0x801A0000 0xc03f0000 flowid 1:14
realtc filter add dev eth0 protocol all prio 2 u32 match mark 0x401A0000 0xc03f0000 flowid 1:14
realtc filter add dev br0 protocol all prio 15 u32 match mark 0x800d0023 0xc03fffff flowid 1:13
realtc filter add dev eth0 protocol all prio 15 u32 match mark 0x400d0023 0xc03fffff flowid 1:13
realtc filter add dev br0 protocol all prio 15 u32 match mark 0x800d007e 0xc03fffff flowid 1:13
realtc filter add dev eth0 protocol all prio 15 u32 match mark 0x400d007e 0xc03fffff flowid 1:13
realtc filter add dev br0 protocol all prio 15 u32 match mark 0x800d00d1 0xc03fffff flowid 1:13
realtc filter add dev eth0 protocol all prio 15 u32 match mark 0x400d00d1 0xc03fffff flowid 1:13
realtc filter add dev br0 protocol all prio 15 u32 match mark 0x800d0054 0xc03fffff flowid 1:13
realtc filter add dev eth0 protocol all prio 15 u32 match mark 0x400d0054 0xc03fffff flowid 1:13

FlexQoS v0.8.3 released 06/29/2020

thanks

 

[dave14305]

*speedtest.net is going to net control packets. is this normal? how can i re-categorize it? When running a speedtest i dont see any connections tagged with net control category (red), just the pie and class data increasing.

This is an indication that the router has reset the tc rules. Run the script with the -check parameter to see if it reapplies the rules (it will log in the syslog). This could also be an indication that my tc delay check isn’t detecting the right conditions and so it applies the tc rules too early and they get replaced.

*at the webgui i see a classification tab and the flexqos tab. is it normal to have the classification tab?

Yes, this is expected with using the new Merlin Addon API. I don’t need to replace the original Merlin tab, and I think it’s a useful diagnostic tool to have the original available to compare, at least in my beta period.

*what is the Added connection table counts mentioned in last changelog?

In the header where is says “Tracked connections” it now shows how many connections are below, and if you use a filter, how many are shown in the list.

 

[danielk]

Well, you know more about QoS than me. Is bufferbloat an actual item that can be checked, or is more like a general feeling of whether something seems fast or slow? Seems like there should be a better way to check, but only thing I know how to check is speedtest and its dl,ul and ping response

Bufferbloat is definitely a thing and you can measure it at dslreports.com, but not many will notice it unless you use your internet in a way where you are sensitive to latency issues (like games where ping matters a lot, or voip). There are even efforts being made like "Low latency Docsis" that specifically address latency like bufferbloat in cable/coax/hybridfiber networks. You can to some degree manage it yourself by ensuring you dont oversaturate your bandwidth, hence the ~90% cap on speeds with QoS. I'd suggest making an account at dslreports.com so you can see your historical results in comparison, and you can also add notes to each test so you can log what changes you made between the tests. If you go back a few pages, i have a post in this thread with pictures (under spoiler tags) that show you my results over 20+ tests. Though im not entirely sure how to enterpret those results, as the final grades dont always make sense when looking at the scores (or the bufferbloat gauge which sometimes jumps up high without that affecting the end results).

People seem to obsess with their max up/down speeds, which i can understand from a historical POV as im from the era of dial-up connections ("Dialuposaurus"), but with todays speeds i personally find those irrelevant. What i want (for my usage), is quality, not gigabit speeds. I want as few hops between nodes as possible, and more importantly excellent transmission times between all those nodes (low ping/latencies between each hop). Even if that line only was 1mbit up/down: as long as it traveled uncongested and at lightspeeds, i would prefer it over a sluggish gigabit and be patient with bigger downloads.

 

[JGrana]

I think I broke it when I tried to also detect WAN to LAN IPv6 connections and swap them, but since I currently have IPv6 enabled, I didn't test what would happen when the IPv6 prefix was blank (i.e. IPv6 disabled). So I'm guessing that it's swapping everything. I reverted that change for now. Pushed v0.8.2.

Somebody here lacks coding discipline...name rhymes with save14305...

Your codeing ability and discipline are fine. I used to manage a 200 person engineering team.
What you lack is simply in house software QA.
That's why many of us are here - your public QA team ;-)

 

[Huey11]

Hi, normally wait a moment before upgrading the firmware, but now applied it asap due to wanting to try your flexqos, thanks for this!

Question though: is it correct that there is a tab classification ánd a flexqos tab, basically showing the same info and only difference the customize button?
Or is there something wrong in my setup (asus rt-ac2900 384.18, with formerly freshqos installed)

 

[danielk]

Hi, normally wait a moment before upgrading the firmware, but now applied it asap due to wanting to try your flexqos, thanks for this!

Question though: is it correct that there is a tab classification ánd a flexqos tab, basically showing the same info and only difference the customize button?
Or is there something wrong in my setup (asus rt-ac2900 384.18, with formerly freshqos installed)
View attachment 24394

He literally just answered this few hours ago, 2 posts above. When kind people devote their time to freely produce stuff like FlexQoS, at least have the courtesy to try and not bog them down with questions already answered.

 

[danielk]

I will try that.. but from running a couple tests on my phone, it seems wifi traffic is bypassing the limits set. I will test some more, and report back here.

I tested this by setting my qos to 10% of my bandwidth, then ran speedtest.net and dslreports.com on a wifi-connected laptop, and my old android cellphone - both capped at the 1mbit ceiling perfectly on both up/down streams. On the 0.7 version for now.

 

[andresmorago]

This is an indication that the router has reset the tc rules. Run the script with the -check parameter to see if it reapplies the rules (it will log in the syslog). This could also be an indication that my tc delay check isn’t detecting the right conditions and so it applies the tc rules too early and they get replaced.

Yes, this is expected with using the new Merlin Addon API. I don’t need to replace the original Merlin tab, and I think it’s a useful diagnostic tool to have the original available to compare, at least in my beta period.

In the header where is says “Tracked connections” it now shows how many connections are below, and if you use a filter, how many are shown in the list.
View attachment 24391

Thanks so much Dave

 

[andresmorago]

Dave. I have a feature request for the tracked connections list.
Would it be possible to implement a link click to ipinfo.io or alienvault? That way when you click an IP address, it would Take you to a new website window And let you see IP address information.

Thanks for your consideration

 

[dave14305]

Dave. I have a feature request for the tracked connections list.
Would it be possible to implement a link click to ipinfo.io or alienvault? That way when you click an IP address, it would Take you to a new website window And let you see IP address information.

Thanks for your consideration

I can't recall if I read this or imagined it, but I seem to think there is a performance hit with rendering clickable links in a list of (up to) 500 connections that refresh every 3 seconds by default. There's already a lot of extra processing in displaying the list with the large number of possible rules to determine the Application class color and sorting by Class.

In general, how many of these IPs do you ever wonder about? I'm curious if anyone else would find this useful or not?