FlexQoS FlexQoS 1.2.4 - Flexible QoS Enhancement Script for Adaptive QoS

[dave14305]

@dave14305 just as an update a rooter reboot fixed the issue . I've been playing the streamer for hours today and no issues so far. The only thing that started to happen after the reboot I get frequent firewall restarts and the below is what I get in the syslog. Any idea what may be casing this?

Jan  2 15:33:58 RT-AC86U-4AB0 kernel: br0: received packet on wl0.1 with own address as source address
Jan  2 15:34:56 RT-AC86U-4AB0 dnsmasq-dhcp[27149]: DHCPREQUEST(br0) 192.168.1.79 80:2b:f9:1b:86:ef
Jan  2 15:34:56 RT-AC86U-4AB0 dnsmasq-dhcp[27149]: DHCPACK(br0) 192.168.1.79 80:2b:f9:1b:86:ef DESKTOP-V8J7HQR
Jan  2 15:35:26 RT-AC86U-4AB0 rc_service: amas_lib 1424:notify_rc restart_firewall
Jan  2 15:35:26 RT-AC86U-4AB0 kernel: 80:2B:F9:1B:86:EF already exist in UDB, can't add it
Jan  2 15:35:26 RT-AC86U-4AB0 custom_script: Running /jffs/scripts/service-event (args: restart firewall)
Jan  2 15:35:26 RT-AC86U-4AB0 nat: apply nat rules (/tmp/nat_rules_ppp0_eth0)
Jan  2 15:35:26 RT-AC86U-4AB0 custom_script: Running /jffs/scripts/firewall-start (args: ppp0)
Jan  2 15:35:26 RT-AC86U-4AB0 custom_script: Running /jffs/scripts/service-event-end (args: restart firewall)
Jan  2 15:35:26 RT-AC86U-4AB0 FlexQoS: /jffs/addons/flexqos/flexqos.sh (pid=11554) called in unattended mode with 1 args: -start
Jan  2 15:35:26 RT-AC86U-4AB0 FlexQoS: Applying iptables static rules
Jan  2 15:35:26 RT-AC86U-4AB0 FlexQoS: Applying iptables custom rules
Jan  2 15:35:27 RT-AC86U-4AB0 FlexQoS: Flushing conntrack table
Jan  2 15:35:27 RT-AC86U-4AB0 FlexQoS: No TC modifications necessary
Jan  2 15:35:34 RT-AC86U-4AB0 kernel: htb: htb qdisc 16: is non-work-conserving?

I’m glad things are running stable, but it’s an unsatisfying ending in terms of understanding the root cause. Your log shows amas_lib restarting the firewall, which is related to AiMesh.

 

[NGI]

Your log shows amas_lib restarting the firewall, which is related to AiMesh.

Thanks Dave that's a bit odd, I don't use AiMesh nor have I ever set it up since I only have a single router. Is there any way I can disable/kill this process since it is keep restarting the Firewall?

Update: After changing the Queue Discipline from fq_codel to sfq the frequent firewall restart stopped

 

[Kanji-San]

@dave14305 I finally updated to v1.1.0 and I love the logarithmic graphs
Occasionally the Automatically refresh data every ... setting is turned off because of too many tracked connections. Is it possible to increase this limit? I assume the more powerful models should be capable of.

 

[cloneman]

Very interesting initiative. Is there any interest in being able to classify groups of cloud-maintained ip addresses (ipset?) ? Dumping Steam CDN (downloads) and WIndows update servers into a lower tier to punish their horrid ingress breaking protocols would be great

 

[Kanji-San]

@dave14305 I noticed that sometimes there are no subdivisions on the Rate axis, specifically for high kb/s values:

Is this a feature or bug?

 

[dave14305]

@dave14305 I noticed that sometimes there are no subdivisions on the Rate axis, specifically for high kb/s values:

View attachment 29117

Is this a feature or bug?

It's probably a bug with a workaround I borrowed from another Chart.js user on Github to fix problems with the logarithmic scale. Sort of beyond my immediate comprehension what causes it, but it sometimes comes back on its own as new data gets graphed.

@dave14305 I finally updated to v1.1.0 and I love the logarithmic graphs
Occasionally the Automatically refresh data every ... setting is turned off because of too many tracked connections. Is it possible to increase this limit? I assume the more powerful models should be capable of.

It's a question of the httpd server processing so many connections to feed the array, and the browser drawing so many rows. Once it stops once, you can enable the refresh again and it should remember it for your current browser session. And if you have over 750 connectoins going, you really should filter the list anyway, since it will only display 500 entries. I don't develop for the fastest router; I develop for the slowest (since I started on the AC68U).

 

[dave14305]

Very interesting initiative. Is there any interest in being able to classify groups of cloud-maintained ip addresses (ipset?) ? Dumping Steam CDN (downloads) and WIndows update servers into a lower tier to punish their horrid ingress breaking protocols would be great

I've thought about ipset-based rules integrated with dnsmasq for ipset populating, but haven't really dug into it yet. I'm not fond of having expand the scope of FlexQoS to deal with dnsmasq. Who is maintaining the cloud-based ip addresses you're thinking about?

 

[rlj2]

@dave14305 This is on 386beta4 which has the fixed qos. I could always set a device > a category in iptables rules. When I do it now. Classification page shows correctly, Tracked connections on flexqos page shows correctly.
But the graph shows bandwidth usage like a rule was never created. (ie, if the laptop is playing netflix, its still going to video streaming). This worked before. Hopefully still will. That was a major thing for using flexqos for me.
Any help would be appreciated. I can give you whatever debugging you might need. EDIT: Nevermind its working on 386 on a AC86u, just not on a AX86u. maybe next version

 

[tmcb82]

I just want to make sure I fully understand and am setting up FlexQOS correctly:

Does the order of iptables Rules matter? If so, what is higher priority at the top or bottom?

And similar second question, does the order of AppDB Redirection Rules matter? Similarly, if so, what is higher priority at the top or bottom?

Thank you for any help! I really appreciate it!

 

[dave14305]

I just want to make sure I fully understand and am setting up FlexQOS correctly:

Does the order of iptables Rules matter? If so, what is higher priority at the top or bottom?

And similar second question, does the order of AppDB Redirection Rules matter? Similarly, if so, what is higher priority at the top or bottom?

Thank you for any help! I really appreciate it!

The bottom iptables rules take priority over the top iptables rules. The appDB rules don't matter, but the webui will sort wildcard rules to the bottom of the list to make it easier for me to interpret the rules during the connection table display.

 

[tmcb82]

The bottom iptables rules take priority over the top iptables rules. The appDB rules don't matter, but the webui will sort wildcard rules to the bottom of the list to make it easier for me to interpret the rules during the connection table display.

Thank you! Btw, per my other thread your script crushes Traditional QOS. I couldn’t get throughput above 130Mbps with it but with FlexQOS I can hit my set cap of 180Mbps (it’s 200Mbps service and I took 10% off for congestion).

What’s weird though is I have my upload set to 9Mbps (ISP set at 10 minus 10%) but I cannot hit any higher than 7.8Mbps with either Traditional or FlexQOS on but with them off I can get 10-11Mbps.

 

[dave14305]

Thank you! Btw, per my other thread your script crushes Traditional QOS. I couldn’t get throughput above 130Mbps with it but with FlexQOS I can hit my set cap of 180Mbps (it’s 200Mbps service and I took 10% off for congestion).

What’s weird though is I have my upload set to 9Mbps (ISP set at 10 minus 10%) but I cannot hit any higher than 7.8Mbps with either Traditional or FlexQOS on but with them off I can get 10-11Mbps.

Everyone may be shocked to learn that I have very unpredictable bandwidth and bufferbloat, despite all my efforts to learn the inner workings and intricacies of QoS. I have 400/25Mbit service from Comcast, but always assumed my AC68U was keeping me from reaching the full bandwidth with QoS, so when I upgraded to an 86U I thought I might be able to reach my shaped speeds with the faster processor. Nope.

I’ve been playing with OpenWRT on a Raspberry Pi and finally made the switchover this afternoon while the family was out, using the AC86U as an AP. Given all the specs of the Pi 4B, I expected to be able to easily shape 400 Mbit while running a modern Linux kernel (5.4.86). Nope. Maybe having the AC86U in the mix is a choke point?

I can reach speeds without QoS/SQM on either router (86U or Pi4B).

So most of my issues are likely out of my control. My modem is a Netgear CM500V and while Netgear rates it up to 600 Mbit, Comcast officially says it’s good up to 373 Mbit. But I can exceed that with QoS disabled, so I don’t think it’s the choke point yet.

The grass is always greener on the other side of the cable modem...

Now I have to wait for another “family is out” opportunity to switch back to the AC86U as the router.

 

[QuikSilver]

Now I have to wait for another “family is out” opportunity to switch back to the AC86U as the router.

I tell my kids the internet is down for maintenance and to go outside. Luckily my wife could care less for TV (streaming).

 

[Kingp1n]

Everyone may be shocked to learn that I have very unpredictable bandwidth and bufferbloat, despite all my efforts to learn the inner workings and intricacies of QoS. I have 400/25Mbit service from Comcast, but always assumed my AC68U was keeping me from reaching the full bandwidth with QoS, so when I upgraded to an 86U I thought I might be able to reach my shaped speeds with the faster processor. Nope.

I’ve been playing with OpenWRT on a Raspberry Pi and finally made the switchover this afternoon while the family was out, using the AC86U as an AP. Given all the specs of the Pi 4B, I expected to be able to easily shape 400 Mbit while running a modern Linux kernel (5.4.86). Nope. Maybe having the AC86U in the mix is a choke point?

I can reach speeds without QoS/SQM on either router (86U or Pi4B).

So most of my issues are likely out of my control. My modem is a Netgear CM500V and while Netgear rates it up to 600 Mbit, Comcast officially says it’s good up to 373 Mbit. But I can exceed that with QoS disabled, so I don’t think it’s the choke point yet.

The grass is always greener on the other side of the cable modem...

Now I have to wait for another “family is out” opportunity to switch back to the AC86U as the router.

@dave14305 how do your results compared when using cake or have you had the chance to test with cake (specifically talking about bufferbloat)?

 

[dave14305]

@dave14305 how do your results compared when using cake or have you had the chance to test with cake (specifically talking about bufferbloat)?

CAKE on OpenWRT was good for bufferbloat, but wasn't what I expected in terms of maximizing my bandwidth. I did think it was interesting that on OpenWRT, they change the default qdisc for all interfaces (even before QoS/SQM is enabled) to fq_codel (versus the usual default of pfifo_fast). I keep wondering if this might be applicable to ASUSWRT-Merlin at all, or if it would face the same limitations as Traditional QoS in relation to HW Acceleration.

I've cut back over to my AC86U as a router again and have given up my OpenWRT ambitions for now. I could have kept it running like it was, but I decided that if I were hit by a bus, my family wouldn't understand how our network is configured if it ever had a problem.

 

[dave14305]

I keep wondering if this might be applicable to ASUSWRT-Merlin at all, or if it would face the same limitations as Traditional QoS in relation to HW Acceleration.

Doesn't hurt to try.

tc qdisc replace dev eth0 root fq_codel limit 1024
tc qdisc replace dev br0 root fq_codel limit 1024

# tc -d -s qdisc
qdisc fq_codel 8001: dev eth0 root refcnt 2 limit 1024p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
 Sent 337088725 bytes 313106 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1514 drop_overlimit 0 new_flow_count 2 ecn_mark 0
  new_flows_len 0 old_flows_len 0
...snip eth1-eth6...
qdisc fq_codel 800a: dev br0 root refcnt 2 limit 1024p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
 Sent 1448768803 bytes 1023362 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1514 drop_overlimit 0 new_flow_count 7 ecn_mark 0
  new_flows_len 1 old_flows_len 1

 

[rlj2]

Everyone may be shocked to learn that I have very unpredictable bandwidth and bufferbloat, despite all my efforts to learn the inner workings and intricacies of QoS. I have 400/25Mbit service from Comcast, but always assumed my AC68U was keeping me from reaching the full bandwidth with QoS, so when I upgraded to an 86U I thought I might be able to reach my shaped speeds with the faster processor. Nope.

I’ve been playing with OpenWRT on a Raspberry Pi and finally made the switchover this afternoon while the family was out, using the AC86U as an AP. Given all the specs of the Pi 4B, I expected to be able to easily shape 400 Mbit while running a modern Linux kernel (5.4.86). Nope. Maybe having the AC86U in the mix is a choke point?

I can reach speeds without QoS/SQM on either router (86U or Pi4B).

So most of my issues are likely out of my control. My modem is a Netgear CM500V and while Netgear rates it up to 600 Mbit, Comcast officially says it’s good up to 373 Mbit. But I can exceed that with QoS disabled, so I don’t think it’s the choke point yet.

The grass is always greener on the other side of the cable modem...

Now I have to wait for another “family is out” opportunity to switch back to the AC86U as the router.

Its a pain trying to tweak and play with the family around . My wife only believes "the damn internet went down again" so many times

 

[BreakingDad]

Its a pain trying to tweak and play with the family around . My wife only believes "the damn internet went down again" so many times

I feel your pain, you have a 300$ router and want to spend 8 hours tweaking it and playing with your toy, and all you get is why have you broken the internet again, kid A was in the middle of game A, kid B was in the middle of game B, or why is there a message on the tv saying network disconnected.

 

[BreakingDad]

Can someone cleverer than me (I know thats a big ask) , come up with Flex QOS settings for Microsoft Teams priority please ?

Thanks

 

[rlj2]

Can someone cleverer than me (I know thats a big ask) , come up with Flex QOS settings for Microsoft Teams priority please ?

Thanks

Try setting a appdb rule with mark 060025 , then add it to the category you want.