[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[FreshJR]

Here is BETA7.

A giant change, is that fakeTC had an error all along that made it only apply sfq (regardless of the selection in WebUI -- this was NOT related to the error messages).
Now it works with codel/fq-codel again, so performance is back to how it was in firmware 369.
I also made some *hacky* changes to reduce the occurrence of kernel warnings.

It seems that while QOS is setting itself up, no one is allowed to make any nvram calls, or errors will soon follow.
AdaptiveQOS has to set finish settings itself up at an expected speed.
If it gets delayed or mismatched timing, it will complain a little bit and throw errors at you.

Changes to quantum have been made, so for some users, this little guy is back.

 kernel: HTB: quantum of class 10016 is big. Consider r2q change.

He is harmless, ignore him. That just means quantum is at its maxed value since you have high guaranteed bandwidth (He pops out at guaranteed rates of 16mbps+)

Good news is that these errors were purely cosmetic!
Bad news is that I hope ASUS doesn't lock down qos in the future to prevent any possible modifications, as these checks were not present before.

As a trade off of removing nvram calls, the application of user defined rates has been delayed to 100 seconds to give qos ample time to finish setting itself up.

Test it out, good luck, and Enjoy!

Oh there's also a snapchat rule. That little bandwidth monster got shifted down the ladder away from VOIP. Set it back to normal if it fits your use case!

Prerequisites:

You will need putty and pscp, as well as the script. Also you need to be on fw 382+

Spoiler

http://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html

You do not have to install these executables, they are portable. Move them, and the script, into a new folder called Putty on your C:\ drive.

It should look like this:

• C:\Putty\putty.exe
• C:\Putty\pscp.exe
• C:\Putty\FreshJR_QOS_beta7.txt
• C:\Putty\FreshJR_QOS_fakeTC_beta7.txt

Install/Update Guide:
1) turn OFF qos
2) In command prompt

cd C:\putty\
pscp -scp C:\Putty\FreshJR_QOS_beta7.txt admin@192.168.1.1:/jffs/scripts/FreshJR_QOS
pscp -scp C:\Putty\FreshJR_QOS_fakeTC_beta7.txt admin@192.168.1.1:/jffs/scripts/FreshJR_QOS_fakeTC

3) In putty

dos2unix /jffs/scripts/FreshJR_QOS
dos2unix /jffs/scripts/FreshJR_QOS_fakeTC
chmod 0755 /jffs/scripts/FreshJR_QOS
chmod 0755 /jffs/scripts/FreshJR_QOS_fakeTC
sed -i '/FreshJR_QOS/d' /jffs/scripts/firewall-start
/jffs/scripts/FreshJR_QOS install

4) turn ON qos


<< OUTDATED VERSION, NEWER RELEASE AVAILABLE >>

 

[chris.at]

Hi FreshJR,

updated to beta7, custom iptables upload rules won't get installed. These rules are placed in custom upload section:
iptables -D POSTROUTING -t mangle -o ppp0 -s 192.168.1.20/32 -j MARK --set-mark ${Net_mark}
iptables -A POSTROUTING -t mangle -o ppp0 -s 192.168.1.20/32 -j MARK --set-mark ${Net_mark}

Result:
admin@router:/jffs/scripts# iptables -t mangle -L
...
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
...

This worked with beta6.

Additional question: When using PPPoE eth0 is worthless, so is it ok to change all other upload rules with realtc... to ppp0 also?

Regards,
Chris

 

[Spydawg]

Does your scripts hold its install over a firmware update?
Or do I need to reinstall it after a firmware update?

 

[Sinner]

Does your scripts hold its install over a firmware update?
Or do I need to reinstall it after a firmware update?

I upgraded 384.4b1 - b2 last night and seemed to be working perfectly after as well in freshjr beta6. The freshjr beta7 i quickly installed this morning and noticed what appears to be all my rate settings shifted down one from where they should be. For example lets say i had “others” set at top priority with 50% bandwidth in the script. When i check the rates in putty its reporting prio 2 at that rate not prio 1. Netcontrol should be prio 0 and others should be prio 1 in this scenario.

Im out for the day but will look closer at it later. I aslo noticed the rates were all not quite right, seemingly scaling smaller towards prio 7 which might be related to the codel fix freshjr implemented i havo no idea on that as of yet.

 

[FreshJR]

Hi FreshJR,

updated to beta7, custom iptables upload rules won't get installed. These rules are placed in custom upload section:
iptables -D POSTROUTING -t mangle -o ppp0 -s 192.168.1.20/32 -j MARK --set-mark ${Net_mark}
iptables -A POSTROUTING -t mangle -o ppp0 -s 192.168.1.20/32 -j MARK --set-mark ${Net_mark}

Result:
admin@router:/jffs/scripts# iptables -t mangle -L
...
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
...

This worked with beta6.

Additional question: When using PPPoE eth0 is worthless, so is it ok to change all other upload rules with realtc... to ppp0 also?

Regards,
Chris

Wow nice. Thanks for this info.

Does stock QOS set up all upload rules as ppp0?

If so, my assumption/eth0 hard coded value is wrong and will need changing.

If you can generate a debug log it would help other users in your spot.

I will post instructions how to do so sook if you are willing to do so, as they have changed from the previous releases.

 

[RMerlin]

Wow nice. Thanks for this info.

Does stock QOS set up all upload rules as ppp0?

If so, my assumption/eth0 hard coded value is wrong and will need changing.

If you can generate a debug log it would help other users in your spot.

I will post instructions how to do so sook if you are willing to do so, as they have changed from the previous releases.

And it would mean that Asus changed it for a third time... Last I spoke with the Asus engineer working on QOS, eth0 was hard-coded for the Broadcom platform with AdaptiveQoS.

Sent from my P027 using Tapatalk

 

[charlie2alpha]

Lets not confuse the interface used for the tc rules and the actual logical interface iptables would use to manipulate packets. And anyway, the mangle table is completely ignored with H/W acceleration and Adaptive QoS, so those rules seem pointless to me.

 

[FreshJR]

Lets not confuse the interface used for the tc rules and the actual logical interface iptables would use to manipulate packets. And anyway, the mangle table is completely ignored with H/W acceleration and Adaptive QoS, so those rules seem pointless to me.

no the iptables rules do work. I have personally tested them!

I am in the process of testing them again, since I do not know why they are not applying for @chris.at as I have not changed anything in that area.

 

[chris.at]

I'm just asking if I assume right to change all from eth0 to ppp0, personally I don't know if there is a difference between interfaces used for iptables and tc. It's fact that a mangle rule which is installed with eth0 has no matches, installed with ppp0 matches count up. At least for iptables eth0 is bound to the modem with a local modem subnet ip and ppp0 is used for the PPPoE traffic with the official ip. When mangle table is not used with adaptive and hw-acceleration how it comes that the counter raises? If you tell me which output you need I'd love to provide it to you.

Edit:
I changed the content of both scripts back to beta6 and the same rules get installed. Back to beta7 and no rules in iptables.

Regarding the hard coded eth0 for tc rules: How can I check if they get installed to eth0 or ppp0 without FreshJR script? If it helps - there are matches in different classes up and down on qos statistics so it seems as if the tc part is working correctly with eth0 in it.

Edit 2:

I will post instructions how to do so sook if you are willing to do so, as they have changed from the previous releases.

Of course I'm willing to help

 

[FreshJR]

Just to make my findings public.

The BWDPI engine continually does checks to see if the QOS setup is applying properly.
With the script intercepting all tc commands, the commands incur a delay due to our post-processing.
With the latest version of this script, this delay has been minimized heavily.

--

Now as to whats happening under the hood.

When In the debug log it says "adaptive QOS: warming up", bwdpi is actually clearing any previous TC rules.
Some time shortly later, around the time it creates the first traffic class, it issues a qdisc show.

It seems that when it gets the results of this, it clears POSTROUTING in iptables (maybe other portions aswell).

As a hacky fix for the error messages, I figured we could delay sending the response of the qdisc show, to get more time to do our post processing.

I didn't realize it would clear iptables AFTER the delayed response gets returned. The delayed response occurs after our iptables rules have been applied.

--

This is unfortunate. I will probably get rid of the hacky workaround, and you will have to live with the occasional 1-2 error messages.

I will investigate a little further as to the script timing.

EDIT:

So kind of good news.

When you apply QOS, these two functions are executed "restart_qos;restart_firewall"
After the delayed qdisc command is returned, it does wipe iptables, but it also simulanously signifies the end of restart_qos.
That means other scripts within (firewall-start) that have iptables should not be effected.
It also means we can have the our iptables rules present in the script to be trigged by firewall-start aswell so qdisc show does not over write them.

Mar  6 15:38:34 admin: *** OUTPUT OF QDISC SHOW AFTER 300s SLEEP***
Mar  6 15:38:34 admin: qdisc show
    Mar  6 15:38:34 iptables: /tmp/filter.default
    Mar  6 15:38:34 iptables: /tmp/filter_ipv6.default
    Mar  6 15:38:34 miniupnpd[7160]: shutting down MiniUPnPd
    Mar  6 15:38:34 iptables: /tmp/nat_rules
    Mar  6 15:38:34 nat: apply nat rules (/tmp/nat_rules_eth0_eth0)
    Mar  6 15:38:34 iptables: /tmp/filter_rules
    Mar  6 15:38:34 iptables: -t mangle -F
    Mar  6 15:38:34 iptables: -t mangle -F
    Mar  6 15:38:34 iptables: -t mangle -N BWDPI_FILTER
    Mar  6 15:38:34 iptables: -t mangle -F BWDPI_FILTER
    Mar  6 15:38:34 iptables: -t mangle -A BWDPI_FILTER -i eth0 -p udp --sport 68 --dport 67 -j DROP
    Mar  6 15:38:34 iptables: -t mangle -A BWDPI_FILTER -i eth0 -p udp --sport 67 --dport 68 -j DROP
    Mar  6 15:38:34 iptables: -t mangle -A PREROUTING -i eth0 -p udp -j BWDPI_FILTER
    Mar  6 15:38:34 iptables: -t mangle -A FORWARD -o br0 -s 192.168.2.0/24 -d 192.168.2.0/24 -j MARK --set-mark 0x01/0x7
    Mar  6 15:38:34 iptables: -F
    Mar  6 15:38:34 iptables: -t mangle -F
    Mar  6 15:38:35 custom_script: Running /jffs/scripts/firewall-start (args: eth0)
    Mar  6 15:38:35 miniupnpd[25151]: HTTP listening on port 54844
    Mar  6 15:38:35 miniupnpd[25151]: Listening for NAT-PMP/PCP traffic on port 5351
    Mar  6 15:38:35 adaptive QOS: FreshJR started

 

[JayLEE]

Where do I find the script, and how to install it on to the router? I’m very new at this. Thank you.

 

[FreshJR]

Ok, everyone, here is BETA8.

The bugs found by users soon after BETA7 release have been squashed.

It is now fine tuned running like a rickety wagon pulled by an ageing horse.

Test it out, good luck, and Enjoy!
Maybe one day we can leave beta.

Prerequisites:

You will need putty and pscp, as well as the script. Also you need to be on fw 382+

Spoiler

http://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html

You do not have to install these executables, they are portable. Move them, and the script, into a new folder called Putty on your C:\ drive.

It should look like this:

• C:\Putty\putty.exe
• C:\Putty\pscp.exe
• C:\Putty\FreshJR_QOS_beta8.txt
• C:\Putty\FreshJR_QOS_fakeTC_beta8.txt

Install/Update Guide:
1) turn OFF qos
2) In command prompt

cd C:\putty\
pscp -scp C:\Putty\FreshJR_QOS_beta8.txt admin@192.168.1.1:/jffs/scripts/FreshJR_QOS
pscp -scp C:\Putty\FreshJR_QOS_fakeTC_beta8.txt admin@192.168.1.1:/jffs/scripts/FreshJR_QOS_fakeTC

3) In putty

dos2unix /jffs/scripts/FreshJR_QOS
dos2unix /jffs/scripts/FreshJR_QOS_fakeTC
chmod 0755 /jffs/scripts/FreshJR_QOS
chmod 0755 /jffs/scripts/FreshJR_QOS_fakeTC
sed -i '/FreshJR_QOS/d' /jffs/scripts/firewall-start
/jffs/scripts/FreshJR_QOS install

4) turn ON qos


I don't know if the download counts are accurate. You need BOTH files for this update!

<< CURRENT VERSION FOR FW v382+ >>

 

[JayLEE]

How do I get to the command prompt? The command prompt in Windows?

 

[FreshJR]

How do I get to the command prompt? The command prompt in Windows?

yes

 

[FreshJR]

Please add this configuration and these script to the Stock firmware

Script is not stable enough to go into stock firmware, as ASUS changes it too much between releases.

Image having to update your router 8 times, instead of the script 8 times.
It would also need a GUI built, etc.

 

[FreshJR]

But you turned the garbage into gold, how can they not do the same?

RMerlins intergration of fq-codel also helped a LOT.

But for your question. It's just too much maintenance and testing to keep up between releases for RMerlin.
I cannot tell you how many hours I have been staring at the system log to get everything working.

He does not work for ASUS.

Also maybe ASUS is not allowed to make any changes.
They did not create it, they licensed (rented) from another company TrendMicro.

Maybe the contract says no changes.

I just hope they do not try to lock it down in future releases for no apparent reason, besides giving us the finger.

 

[skeal]

This script is pretty freaking awesome man!

 

[Sinner]

does the new beta8 include the fq-codel instead of sfq corrections as well freshjr?

 

[FreshJR]

does the new beta8 include the fq-codel instead of sfq corrections as well freshjr?

Of course. That was a bug. It's fixed going forward.
We are not trying to get worse over time....

 

[skeal]

Just installed beta8 freaking awesome again man!!