Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

Discussion in 'Asuswrt-Merlin' started by FreshJR, Jan 12, 2017.

  1. beboptrumpet

    beboptrumpet Occasional Visitor

    Joined:
    Nov 15, 2013
    Messages:
    41
    Location:
    Indianapolis, IN, USA
    IMG_1001.PNG IMG_0996.PNG Screenshot shows qos statistics whil playing cod-IW on Xbox 1.

    Definitely going to give this a try. I was looking at my qos statistics while i was playing Call of Duty on an Xbox one console--it is classifying all in game traffic as default. Just found out this was the case --there is a call of duty classification under bandwidth monitor...it shows up when I load the game, but disappears when I'm actually in a game since none of the traffic is correctly identified,

    Also, something of note. When I download actual games and add-one from the Xbox store, it shows as gaming traffic. I had an update yesterday going at 160mb on my 150/20 line that pretty much shut down all my other device's internet.

    I'm wondering if my trend micro engine has actuall updated... I see in the logs. Trend micro forward module 1.0.34
    IMG_0999.PNG
    However, when I go to the firmware update page I see 1.176?update 8/5/17
    IMG_1001.PNG
    I'm thinking I need to restore to factory then re-update
     
    Last edited: Aug 11, 2017
  2. RMerlin

    RMerlin Part of the Furniture

    Joined:
    Apr 14, 2012
    Messages:
    24,189
    Location:
    Canada
    One is the engine (the code part of the firmware), the other are the signature definition files, which get updated over the Internet.
     
  3. beboptrumpet

    beboptrumpet Occasional Visitor

    Joined:
    Nov 15, 2013
    Messages:
    41
    Location:
    Indianapolis, IN, USA
    Thanks for getting back. I did a factory reset and saw the same result.

    Adaptive qos is giving me a headache. The trend micro engine used to recognize & give priority to my favorite game, unfortionately that has all changed. When I first saw it i thought it might have been the culprit.

    Guess I'll get to work adding jrfresh's scrips.
     
  4. FreshJR

    FreshJR Regular Contributor

    Joined:
    Oct 8, 2016
    Messages:
    174
    It pretty easy to install. Takes like a minute.

    Ideally if you find that game downloads have their own port that doesn't overlap with game traffic, I would add a custom rule to send all that game download traffic To the file transfers category by port.
     
  5. beboptrumpet

    beboptrumpet Occasional Visitor

    Joined:
    Nov 15, 2013
    Messages:
    41
    Location:
    Indianapolis, IN, USA
    Xbox One System NAT port 3074 tcp&udp
    Black Ops3 was port 3074 for open NAT
    Infinite Warfare Xbox port to get open NAT 3076

    Open NAT can connect to all players, moderate can connect to open & moderate, & strict can connect only to open NAT players. Knowing that, I would imagine the game traffic is going to those ports.

    As you can see, in the pic, UpNp also forwards a few external ports in the 3000's to 3076 internal. These upnp port maps show up in system logs when I end up being host for a pub.

    IMG_1013.PNG
    I can think of a couple ways that might work to test & make sure.
    - A switch that allows for more detailed traffic monitoring. I think my cheap Netgear might work ....or set up a pic as a network bridge

    Id like to try it adding rules for those as a start. I should be able to tell if it worked by looking at the qos stats page. What would the mask be for ports 3074-3076?
     
    Last edited: Aug 12, 2017
  6. FreshJR

    FreshJR Regular Contributor

    Joined:
    Oct 8, 2016
    Messages:
    174
    Since 3074 is not divisible by 4 to get that exact range opened would take two rules.

    3074/fffe opens 3074 - 3075
    3076/ffff opens 3076

    Instead I would just recommend a slightly larger range to have the router perform less work.

    3072/fff8 would open 3072 - 3079

    The chart explaining this on the first page really shouldn't be that terrible to use.

    Here's a non valid example:
    **NON VALID MASK COMBINATION ** 3074/fffc opening 3074-3077 **NON VALID MASK COMBINATION**
    I included it to show explain why it would have no effect. It's simple, 3074 is not divisible by 4 so it's not a valid mask/mark combination.

    Also you do not have to buy a dedicated switch to view your requested information. You can simply install more advanced net monitoring apps into the router. The router in essence is like a full computer.

    Port forwards complicate things slightly since the router filters egress traffic. I explained these implications. Either way, if a rule is not possible, with the allotted bandwidth changes, the games shouldn't hiccup even if they aren't identified.
     
    Last edited: Aug 12, 2017
    beboptrumpet likes this.
  7. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    741
    Location:
    Australia
    i checked again you right ever cod except black ops 3 uses 3074, they changed it in a patch but it might be pc only.
     
  8. beboptrumpet

    beboptrumpet Occasional Visitor

    Joined:
    Nov 15, 2013
    Messages:
    41
    Location:
    Indianapolis, IN, USA
    Installed the scipt on 3.68-alpha2 on rt-ac87 Using fq_codel.

    My connection normally tests at 150/20 or better After a factory reset, freshJr script, & codel_fq, setting manual bandwidth limits (150/20), and a reboot. I'm getting somewhere in the neighborhood of 20/15. I tried both Speedtest & Dsl reports.
    I noticed on the qos statistics page that the download portion of the test goes to file transfer category, and the upload to web surfing. I have file transfer as my bottom category...I'm guessing my download results are related to that. Im thrilled with the consistency, and the blufferboat. I was seeing blufferboats in the 200's...now they stay below 10ms. Thanks JR. Looking forward to 2.0.

    Are the last 6 lines in the log a concern?

    What does checking the ATM checkbox next to WAN Packet overhead do?

    can I ssh and use nano to edit the values. If so, what would I need to type after I get into nano to bring it up?

    IMG_1014.PNG
    Also, I saw these entries in the log. Is this a concern?
     
    Last edited: Aug 13, 2017
  9. FreshJR

    FreshJR Regular Contributor

    Joined:
    Oct 8, 2016
    Messages:
    174
    I stand firm by the notion that Comcast customers do not have to account for WAN packet overhead. This is pre-modem overhead.
    I am pretty sure that Comcast customers are bandwidth limited on modem egress traffic not modem ingress traffic, which is where the overhead is present.

    Since Comcast limits at the modem, not before the modem, I do not think it is necessary to account for WAN overhead. Your ISP may vary. I just set rates under my advertised speeds until bufferbloat dropped.

    The non-work-converving is a normal message.
    I also have seen quantum message as well, but not recently.

    Either way, I did not change the quantum value from the one that ASUS supplied, so I guess its normal aswell.

    What do you mean you are only getting 20mbps/150mbps? Also you should set your QOS limits to like 80-95% of your max, not your max
     
    beboptrumpet likes this.
  10. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    741
    Location:
    Australia
    How would one go about changing the quantum, every time I see the system log I get a headache I understand if it's not an issue, I do vaguely remember reading something about it can be but I can't remember, also what is quantum for, other than a kernel error.
     
  11. wickedshrapnel

    wickedshrapnel New Around Here

    Joined:
    Aug 13, 2017
    Messages:
    4
    Short answer, here is how you fix it. They fixed it in Redhat. https://www.redhat.com/archives/libvir-list/2016-February/msg00125.html <- way above my pay grade.

    Long answer below

    What quantum it is.

    "It might be good time to touch concept of quantums now. In fact when more classes want to borrow bandwidth they are each given some number of bytes before serving other competing class. This number is called quantum. You should see that if several classes are competing for parent's bandwidth then they get it in proportion of their quantums. It is important to know that for precise operation quantums need to be as small as possible and larger than MTU.
    Normaly you don't need to specify quantums manualy as HTB chooses precomputed values. It computes classe's quantum (when you add or change it) as its rate divided by r2q global parameter. Its default value is 10 and because typical MTU is 1500 the default is good for rates from 15 kBps (120 kbit). For smaller minimal rates specify r2q 1 when creating qdisc - it is good from 12 kbit which should be enough. If you will need you can specify quantum manualy when adding or changing the class. You can avoid warnings in log if precomputed value would be bad. When you specify quantum on command line the r2q is ignored for that class."

    Source: http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm

    Basically I think the issue is this algorithm was written at a time when bandwidth speeds in the hundreds of megabytes or gigabit wasn't feasible and letting r2q calculate the quantum value freaks it out when that number is so high.

    If rate is less than 80 the quantum is small warning is provided.
    If rate is greater than 16000 quantum is big warning is provided

    Quantum is calculated as rate (in bytes) / r2q.
    r2q is by default 10

    For example, my download rate is set at 300Mb you take that (in bytes) divided by the default r2q of 10 and that equals 3,750,000 bytes for the quantum. The hard coded limit of quantum is 60,000 so it throws a warning.

    So in order to prevent these warnings for my 300Mb connection, my r2q needs to be 20,000. That will keep the quantum under 16,000 where it cries but I think it would need to be even a bigger number to get my quantum closer to my MTU of 1500. Changing the r2q value would be the easiest option I think since there are a lot of classes and that would suck to change all of them manually.

    There are a lot of classes, you can see them by running the following command
    Code:
    tc class show dev eth0
    The class in the log " HTB: quantum of class 10011 is big. Consider r2q change." will be shown in the command output truncated like "class htb 1:11" they substitute a : in place of the zeros.
     
    FreshJR and beboptrumpet like this.
  12. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    741
    Location:
    Australia
    Thanks for that explanation it makes sense now, I wonder if there's a way to get the script to automatically calculate r2q based on input of bandwidth values, I wonder if this is still an issue on the HSFC algorithm, shame we don't have it available as an option to see if it is better than HTB or doesn't have the same issues.
     
  13. wickedshrapnel

    wickedshrapnel New Around Here

    Joined:
    Aug 13, 2017
    Messages:
    4
    I was messing with pfSense for a while and HFSC is the best QoS method they have over there and everyone is begging for fq_codel there. https://forum.pfsense.org/index.php?topic=87931.0

    I guess fq_codel is the bee's knees as far as performance goes in QoS. I think this is something that would have to be fixed by ASUS rather than patched in a script. It is not breaking anything but I think it could be causing slight delays probably in milliseconds since it is not optimized. it's just more of an annoyance than anything. I like clean logs.
     
  14. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    741
    Location:
    Australia
    I've been reading the beast combination is FQ-Codel +HSFC, as in HSFC as packet scheduler and FQ-Codel as QOS discipline, read some ddwrt people saying that combo is amazing.
    Though I have to agree with you it should be up to asus to implement it, but they will only do it if they get a lot of requests, to do so.
    If the r2q was auto calculated we wouldn't see any of the r2q errors in the logs and it hope fully would become more accurate.
     
  15. wickedshrapnel

    wickedshrapnel New Around Here

    Joined:
    Aug 13, 2017
    Messages:
    4
    Cake is another one. It is built off fq_codel supposed to be an improvement of it. https://www.bufferbloat.net/projects/attachments/150817135028_cake-
    battlemesh-v8.pdf
     
  16. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    741
    Location:
    Australia
    I'm aware of cake, but apparently the current kernels are too old to support it RMerlin told me, or the full feature set of FQ-Codel sadly I wish asus would just port it since qos is closed source. Id love to have it if its possible.
    https://www.bufferbloat.net/projects/codel/wiki/Cake/
     
    Last edited: Aug 15, 2017
  17. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    741
    Location:
    Australia
    only those 2 show up in the log with the error I'm looking at the script o see where I can change the r 2q
    kernel: HTB: quantum of class 10012 is big. Consider r2q change.
    kernel: HTB: quantum of class 10013 is big. Consider r2q change
     
  18. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    741
    Location:
    Australia
    @FreshJR is there any way to implent what what @wickedshrapnel posted into the script in regards to modding the r2q values from what I read it affects accuracy of the shaper, you would know more about it than me to be honest, but I'm also curious as to what benefits could appear of it is fixed other than the annoying log messages. Is there any way to implement the commands from the links to the script.

    Fix and explanations:
    https://www.redhat.com/archives/libvir-list/2016-February/msg00125.html
    http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm
     
    Last edited: Aug 16, 2017
    beboptrumpet likes this.
  19. Arsenal

    Arsenal Occasional Visitor

    Joined:
    May 28, 2013
    Messages:
    11
  20. ThiagoCururu

    ThiagoCururu New Around Here

    Joined:
    Aug 17, 2017
    Messages:
    4
    Hello.

    I'm almost losing my sanity over this QoS.

    When I'm gaming and there's Netflix or YouTube traffic on my network, I get so many latency spikes that it makes impossible to play games that aren't flagged as Gaming on QoS. To be honest I don't even know if I have any game that is properly identified.
    To make things worse, all Steam traffic is considered Gaming, including downloads and updates.

    When I first saw this script I thought I would be saved, but unfortunately no.

    I followed that tutorial and got it running on my AC68U with firmware 380.67. Checked on System Log and it is working, but as you can see on screenshots below, it's all going downhill.

    I'm using a fiber connection with 50/25 Mb speeds (51.23 and 26.1 speeds at dslreports.com before that script. Bufferbloat with A and overall quality on D). Tested on automatic and manual bandwidth of 43/20 speeds. Tried sfq and fq_codel with no results. Overhead of 4, 16, 27 and 28 made no difference.

    I really don't know what else I can try. Any help?


    Game example: Path of Exile x64 without Steam. Lockstep and multithread DX11.
    latency.jpg
    latency2.jpg
    latency3.jpg
    QoS statistics.jpg
     

Share This Page