[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[FreshJR]

@TicoMan

Game transferring is almost lowest priority (6th place)

Are you sure you have your QoS limits set correctly. Sounds like they could be set too high since it shouldn't choke streaming (unless streaming is in the last place).

Are testing after the 5 minute period? Is data is going into the grey “game transferring” label next to the pie chart.

EDIT:

Pushed 8.3 (very minor fixes per request)

-Fixed (press any key to continue) to actually detect any key instead of enter
-Fixed " [ " bad number message when choosing not to restart QoS (has no impact on expected functionality)

 

[sfortis]

Yup, I said backported the UI to partially display on older firmware’s

It is not possible to get the tracked connections portion working on older firmware’s simply by replacing the webpage.

To do so would require recompiling & installing a modified version of the older firmware so the WebUI could get access to an additional data source it requires.

(I didn’t bother hiding the unsupported elements unless you really want me too)

I don’t have the environment setup to compile from source nor do I have the router to test the results on, so it’s a dead end to provide a modified firmware from my end. Maybe you can ask RMerlin if he would be willing to provide one final update to your router.

yeap, i see, makes sense. bwdpi_conntrack function is missing on older firmwares. In any case, having your customizations on webUI is more than enough!

so two small suggestions:
1. you can totally remove the conntrack table UI component when firmware is <384.9
2. appdb search function is very strict, you can relax the wc on "grep -i "${1}" /tmp/bwdpi/bwdpi.app.db | wc -l )" -lt "5". Something like 10-15 will work better for common keywords like "DNS".

ps. you've done an amazing job!

ps2. Asus? are you listening? looking for talents??

 

[Vexira]

@FreshJR wow this new update, it's I'm beyond speechless at how well it's been running after I installed it, it's almost pure magic, all I can say is thank you very much I'm deeply grateful for this new update and Asus should make you head of the QoS department.

 

[FreshJR]

2. appdb search function is very strict, you can relax the wc on "grep -i "${1}" /tmp/bwdpi/bwdpi.app.db | wc -l )" -lt "5". Something like 10-15 will work better for common keywords like "DNS".

There used to be a performance reason for limiting results.

Previously when creating rules you had to hardcode the corresponding parameters into the script per AppDB rule:

-download mark
-upload mark
-prio


The issue was that prio had to pull the underlying TC structure per result and grep that data to calculate the recommended prio correctly.

TC Prio   : $(expr $(tc filter show dev br0 | grep "${cat_hex}0000" -B1 | tail -2 | cut -d " " -f7 | head -1) - 1)

The search results as to not run $(tc filter show dev br0 | grep ) too many times.

With the v8 update, the PRIO parameter is now calculated at script runtime so the old appdb search limit can be relaxed.

You can see the old actions of that code block via commented out code.

 

[TicoMan]

@TicoMan

Game transferring is almost lowest priority (6th place)

Are you sure you have your QoS limits set correctly. Sounds like they could be set too high since it shouldn't choke streaming (unless streaming is in the last place).

Are testing after the 5 minute period? Is data is going into the grey “game transferring” label next to the pie chart.

So my ISP speeds are 50/5, I set them up as 45/4.5 yet the Gaming Transferring sometimes reached the 50+Mb/s rates...

What I did right now was to set Game Downloads Max Allowed Bandwidth to 50%, which by obvious reasons gave the Streaming class space to breath... was that the right thing to do? that would mean that when no other traffic is going on the Game Download will not take full advantage of the available bandwidth right? Do you recommend trying something else?

 

[FreshJR]

@TicoMan bump it back to 100% and try lowering your QoS speeds from 45/4.5

If that doesn't work, you may have to edit the script and manually lower bursts.

If that works I may include it as a tweak in the next release.

 

[TicoMan]

Ok, so bumped back to 100%, limits set to 40/4.... Game Transfering still getting 90% of the bandwidth (36Mb/s rate)

.... if not a problem I will need some guidance with does bursts changes.

 

[FreshJR]

Ok, so bumped back to 100%, limits set to 40/4.... Game Transfering still getting 90% of the bandwidth (36Mb/s rate)

.... if not a problem I will need some guidance with does bursts changes.

Who is the game download provider. I will try and look into it.

Also grab the output of

 /jffs/scripts/FreshJR_QOS -debug

 

[TicoMan]

Who is the game download provider. I will try and look into it.

Shows as Game Transfering: Sony Playstation

 

[FreshJR]

Update: Manual install instructions && installers for users on stock AsusWRT have been posted

Update: v8.4 aswell (there was an additional bug report open on the github page I missed)
AppDB - 12003F was not functioning as expected

@TicoMan Are you sure the Sony Download was not actually going into Web Surfing which is above video streaming.
That is the only reason video streaming should/would be choking.
(You have to really observe the rates next to the pie charts for a complete picture).

You have many options to choose experiment with

1) If your game download is going into web surfing then to remove it from that section, as a very aggressive rule you can take all TCP traffic from remote ports 80,443 but only dealing with your game consoles CIDR range and stick it into "Game Transferring"

2) Or slightly less aggressive, you can take all TCP traffic from remote ports 80,443 with a current traffic mark of (13**** or 14****) but only dealing with your game consoles CIDR range and stick it into "Game Transferring".

Not sure which mark your game download was detected as. Do some more investigation if you need 13****, 14****, both, or simply the aggressive rule which swallows ALL the marks.

3) You can also bump the guaranteed rate higher for video streaming, so the new minimum guaranteed rate causes no buffering irreguardless what is happening elsewhere by taking some bandwidth away from other categories.

4) You can put Video Streaming above Web Surfing

There's lots of options at your disposal .... experiment

Typical traffic on ports and protocols was explained, and should be apparent when studying tracked connections. You then can spot more patterns with MARK for specific use cases.

--

Also if you ONLY have one gaming device active at a time.
You can simply stick all its traffic (Game + Game Downloads + ALL) into "Gaming" and place "Gaming" 2nd from bottom. Since it is only one device in the category, the console should be smart enough to PAUSE game transfers while it is actively playing online. Even though you placed it 2nd to the bottom, until it exceeds its guaranteed bandwidth it will still have first dibs on the network access.

--

@TicoMan

Report back, game transfers showing up in "Web Surfing" was your issue.
I can cook up another optional gaming rule to make more assumptions and stick MORE stuff into "Game Transfers"

 

[medwatt]

@FreshJR,
1. Where are the changes made to the WebUI saved ?
2. What happens if we want to redirect more than 4 traffic into different QOS container ?

 

[FreshJR]

@FreshJR,
1. Where are the changes made to the WebUI saved ?
2. What happens if we want to redirect more than 4 traffic into different QOS container ?

1) nvram
2) You cant. You have to make smarter device IP ranges to compact your rules into 4 slots. If you cant then you have to add extra rules directly into the script via hardcoded modifcations.

 

[medwatt]

1) nvram
2) You cant. You have to make smarter device IP ranges to compact your rules into 4 slots. If you cant then you have to add extra rules directly into the script via hardcoded modifcations.

Suppose I want to change remove/change one of the default rules that are under "AppDB Redirection (traffic control)". How can I go about doing that ?

 

[FreshJR]

Suppose I want to change remove/change one of the default rules that are under "AppDB Redirection (traffic control)". How can I go about doing that ?

Override it.

The rule precidence is as follows:

1) iptable4
2) iptable3
3) Iptable2
4) Iptable1
5) Iptable hardcoded
6) Appdb4
7) Appdb3
8) Appdb2
9) Appdb1
10) Appdb hardcoded

 

[medwatt]

Override it.

The rule precidence is as follows:

1) iptable4
2) iptable3
3) Iptable2
4) Iptable1
5) Iptable hardcoded
6) Appdb4
7) Appdb3
8) Appdb2
9) Appdb1
10) Appdb hardcoded

In other words, I create a duplicate rule ? That means I less slot to work with. Just out of curiosity, is there a limit to the number of slots you can give us via the webui ? It feels quite nicer to change things that way.

 

[FreshJR]

In other words, I create a duplicate rule ? That means I less slot to work with. Just out of curiosity, is there a limit to the number of slots you can give us via the webui ? It feels quite nicer to change things that way.

Yes we are at the limit. I repurposed two nvram variables from the feedback page. I don't want to steal anymore variables meant for other things.

 

[heysoundude]

Updating to the current (today’s release) breaks my Internet. I’ve rebooted my router a couple of times, but it hasn’t come back. Further, wifi calling is broken on my phone.
Merlin 384.9, ac86u
EDIT - a hard reboot got the GUI telling me there was an IP conflict and it changed the router's IP after uninstalling the script from terminal.

 

[Kingp1n]

@FreshJR, I upgraded to latest released, when clicking on the Classification tab I get the following screen below. Any way to fix this?

UPDATE: Disregard, As stated by Therion87, it's working now!!!

 

[Therion87]

Just wait. It seems to take a few minutes to get going.

 

[FreshJR]

@heysoundude you are having a separate problem unrelated to the script modifications. No modications are made to cause that sort of behavior.

@Kingp1n if you elected to restart QOS during the update it will still takes 5minutes to complete the process just as if you restarted it from the UI.