Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

Discussion in 'Asuswrt-Merlin' started by FreshJR, Jan 12, 2017.

  1. cybrnook

    cybrnook Senior Member

    Joined:
    Jan 8, 2014
    Messages:
    444
  2. cybrnook

    cybrnook Senior Member

    Joined:
    Jan 8, 2014
    Messages:
    444
  3. cybrnook

    cybrnook Senior Member

    Joined:
    Jan 8, 2014
    Messages:
    444
    stoopid.... maybe it thinks we are trying to do some php/sql haxors or something. Thinks we are attacking the forum.
     
  4. RMerlin

    RMerlin Part of the Furniture

    Joined:
    Apr 14, 2012
    Messages:
    24,604
    Location:
    Canada
    Feel free to talk to Cloudflare if you think this isn't necessary. They're the ones providing that security layer, not SNB, neither Tim.

    With script kiddies frequently attacking this site, Tim is forced to increase the security level on Cloudflare's end.
     
  5. MarCoMLXXV

    MarCoMLXXV Guest

    Haven't had these issues with several other forum platforms, as I have been moderating for quite some years (>15). Either things have changed badly in the last year (when I decided to quit moderating as I have my hands full with my own kid) or this forum is definitely running on the wrong platform.

    Too bad it enforces such annoying measures on legit users trying to help one another. This will make you laugh: the first time I got the 'You're blocked message', I sent a mail to the moderator(s?) with what I had done and my intentions to support another member. Stayed away for a couple of days, waiting for a reply and of course, to be unbanned. Then I realised I would never get a reply and I wasn't actually banned... Boy, I can be a major idot at times...

    @FreshJR, to get back on topic: I noticed there's a tc package in entware-ng as well. Does it offer any advantages over the stock tc provided in BusyBox, which your script uses when there's no realtc present?
     
  6. MarCoMLXXV

    MarCoMLXXV Guest

    It just keep getting better and better :)

    Bandwidth set @ 15/150 in WebUi, WAN packet overhead set to 0. Custom priorities, values in script:

    Code:
        ####################  Variables Setup #####################
    
        #Percent of download speed guaranteed per QOS catagory, change below as desired (sum should equal 100)
            NetControl_DownBandPercent=5           
            VoIP_DownBandPercent=5                       
            Gaming_DownBandPercent=10                   
            Others_DownBandPercent=10                  
            WebSurfing_DownBandPercent=20              
            Video_DownBandPercent=25                   
            FileTransfer_DownBandPercent=20               
            Default_DownBandPercent=5                   
     
        #Percent of upload speed guaranteed per QOS catagory, change below as desired (sum should equal 100)
            NetControl_UpBandPercent=5                  
            VoIP_UpBandPercent=5                       
            Gaming_UpBandPercent=5                       
            Others_UpBandPercent=30                       
            WebSurfing_UpBandPercent=25                   
            Video_UpBandPercent=10                       
            FileTransfer_UpBandPercent=15               
            Default_UpBandPercent=5                       
    
    
    Results:

    [​IMG]

    Bufferbloat down: 22 / up: 3

    Any recommendations to the variables setup above? No VoIP here, and little to no gaming (nothing causing a network load anyway or where a low latency is of importance). Mostly websurfing, file transferring and video (YT and Vod). Would like to setup OwnCloud sometime soon, as I'm tired of paying fees while have 15Mbit upload available and 5 Terabyte (over three NAS'es) lying around here, catching dust.

    Another question to @FreshJR: as you changed redirection of unrecognized traffic from Default to Others, can I put Default_DownBandPercent's value to 0 or 1 or will it only work with multiples of 5?
     
    Last edited by a moderator: Sep 8, 2017
  7. Lacrocious

    Lacrocious New Around Here

    Joined:
    Aug 16, 2015
    Messages:
    8
    @FreshJR - Thank you for your help. I am all set. I upgraded to 1.92 and plugged in my additional $(tc ) rules for BackBlaze destinations - setting them to Default. I can get full speed uploads until pretty much anything else comes along. This is great. It will let me easily move from CrashPlan to BackBlaze B2 w/ARQ client. CrashPlan never published their IP ranges and they changed them periodically - so it was hard to control uploads with CrashPlan. Your script would have helped, but it would have been more work to maintain servers over time.

    Thank you again.
     
  8. FreshJR

    FreshJR Regular Contributor

    Joined:
    Oct 8, 2016
    Messages:
    174
    @MarCoMLXXV
    I wouldn't set it to 0.
    You can set it to 1, it doesn't have to be multiples of 5.

    Also that bandwidth is simply reserved. When defaults is not using that bandwidth, the bandwidth will be available for other catagories. Bandwidth is never wasted and held unused.

    As for your results. I think it's just regular network variance as to what's effecting your scores every so often.

    @Lacrocious

    Don't say your old backup service was unworkable. I bet you they had a unique dscp packet mark we could of tapped. Glad everything is working

    @RMerlin

    Love the forum software. It's super quick + full featured on mobile and scales well to the screen. I hate whatever forum software makes that enchanced mobile view with infinite scrolling. It scales terribly and is slow. Just playful banther pointing out unique quirks. Either way it's super fast.
     
    MarCoMLXXV likes this.
  9. MarCoMLXXV

    MarCoMLXXV Guest

    Thanks for your reply @FreshJR.

    If you have the time for it, could you please answer the question above? Probably missed it while I was spilling my guts about the forum software :D If you answered it before, could you please point me to the post?

    Thanks in advance :)
    Marco
     
  10. charlie2alpha

    charlie2alpha Senior Member

    Joined:
    Nov 15, 2014
    Messages:
    353
    Location:
    Greece
    This script looks interesting. But I do have a question, is it possible to create a custom rule that combines source mac and a source port/port range? I did look at the script comments, all the iptables rules go to the postrouting chain and I wonder if that actually works.
     
  11. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    855
    Location:
    Australia
    I adjusted the map to be the same as the old one.
     
  12. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    855
    Location:
    Australia
    I made sure all he permissions matched the ones when I uploaded via putty. It works perfectly fine and I can screen shot it.
     
  13. MarCoMLXXV

    MarCoMLXXV Guest

    Oh, my dear @Vexira, no need for for screenshots. Not questioning you at all, and I apologize if it came out wrong (English is not my native language, so sometimes things just unintended come out the wrong way). I believe you on your (dark) blue eyes (Dutch saying, without the 'Dark'-part, that's just related to your awesome avatar). Did you need the -p option? Or are you running a unix-based system and set permissions prior to uploading?
     
  14. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    855
    Location:
    Australia
    That's OK I just wanted to help, gui based on Windows, it didn't I didn't read it properly I was offering screenshots for those who wanted to use the same method as I did.
     
    Last edited: Sep 9, 2017
    MarCoMLXXV likes this.
  15. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    855
    Location:
    Australia
    Thank you for the compliment, my eyes are brown in real life, also I understand my parents are not native English speakers.
     
    MarCoMLXXV likes this.
  16. jpclarke

    jpclarke Regular Contributor

    Joined:
    Nov 6, 2015
    Messages:
    89
    Working great for me. Trying to use bc to do the calculations instead of losing the decimal part of the up and down bandwidth.
     
    Last edited: Sep 9, 2017
  17. Lacrocious

    Lacrocious New Around Here

    Joined:
    Aug 16, 2015
    Messages:
    8
    After reading Vexira's info, I found the -p (Preserve File Attributes) option to pscp and tried it while watching the permissions via Putty. It would be nice not to have to chmod every upload while testing. Interestingly enough - If I use the -p option, it resets the permissions. If I don't use the -p option, pscp doesn't reset permissions. Once they are set after initial upload they seem to stay in place. Odd - different than I expected.
     
  18. Lacrocious

    Lacrocious New Around Here

    Joined:
    Aug 16, 2015
    Messages:
    8
    Yes - CrashPlan does allow you to set a DSCP packet mark - so in theory I could have had QOS control. In practice, some of my initial investigations gave info that Windows 7 clients didn't allow setting DSCP by apps - overwriting it. I never actually sniffed the packets to check it out, but then again I didn't have a router with options to properly handle it. I tried Tomato on an old router, and could get QOS working, but that router passed away and I found Asus. Could I control it now? Sure - with the controls your script provides. So putting blame appropriately, it wasn't CrashPlan, but the routers.
     
  19. MarCoMLXXV

    MarCoMLXXV Guest

    That's the opposite of what you'd expect indeed. Please let me know if I have to change something in my post, the info there I gathered from the man pages and some other (Linux) sources, but as I use a different way myself (opening the file through sftp (that's how the routers' filesystem is mounted on my linux laptop - it's 'just' another folder), copy new version locally from Pastebin, pasting it into the opened file while replacing it's content, saving it - which basically uploads the edited file back to the router and the I edit the remote file with nano through ssh, I never have issues with permissions or changes of ownerships or groups.
     
  20. Lacrocious

    Lacrocious New Around Here

    Joined:
    Aug 16, 2015
    Messages:
    8
    I did some testing and can confirm using pscp without using -p (preserve file attributes):
    • The first time you copy a file, it gets default permissions of 644.
    • If the file exists, it preserves the current permissions.
    Per this Putty Wish List entry, you can see that the default is to preserve permissions.

    If you section 5.2.2.2 of the Putty Doc states: "By default, files copied with PSCP are timestamped with the date and time they were copied. The -p option preserves the original timestamp on copied files.". I can see this as well.

    Doing a "pscp", the date/timestamp is Now; with "pscp -p" the date/timestamp is the date/time of the file on the source device. This means you can pscp -p a file, overwriting an existing destination file with a source file dated older than that existing destination file. The router file system must think that this is considered a "new" file and sets the permissions appropriately back to 644 (per the Putty Wish List entry above).

    I don't have anything to backup the theory of overwriting an existing file with a file with an earlier date is considered "new", but that is what it appears to be doing. The "pscp -p" doesn't explicitly do anything with permissions, it only affects file date/time.
     

Share This Page