1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

Discussion in 'Asuswrt-Merlin' started by FreshJR, Jan 12, 2017.

  1. gmangt4

    gmangt4 Occasional Visitor

    Joined:
    May 14, 2017
    Messages:
    21
    Activision say these ports are used for Blops 3 on PS4

    • TCP: 80, 443, 1935, 3480
    • UDP:
    • TCP and UDP: 3478-3479, 3074, 3075
    Hope that helps and is not oo off topic
     
  2. Please support SNBForums! Just click on this link before you buy something from Amazon and we'll get a small commission on anything you buy. Thanks!
  3. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    1,286
    Location:
    Australia
    The other console not having open nat means that, the client is using or requesting port 3074 but not receiving it and the router is not preforming port translation correctly, if you observe upnp logs with two sets of eg bops 3 open you will see what I mean one will get the primary port the other will get primary on inbound and a different on external, if I remember correctly, there's a good thread here called multiple xbox ones, I posted a few things, though I will hopefully get my guide on open nat written to see if I can't help anyone. Also cod on pc is infested like many other pc shooters with cheaters thus killing the player base, also the game is killed by supply drops, I do play a lot of xbox, but the sad thing is my head set is not fully supported by the console. I find it it easier to get a match in Overwatch than in cod on pc.
    Though Merlin squash most of the major upnp issues, there's are still a few yet to be fixed but multi console support works for the most part thanks to Merlins hard work.
    Also ill assume that you have been watching the upnp log for ports.
     
    Last edited: Jul 25, 2017
  4. lilstone87

    lilstone87 Senior Member

    Joined:
    Dec 13, 2012
    Messages:
    256
    @Vexira @FreshJR
    Actually I did still have a older router of mine laying around, which supported Tomato firmware. So I loaded it on it, and ran my PS4 threw it. Which if you have used Tomato before, you know the QoS system actually has a section in which it shows all active addresses, and data passing threw different ports. So last couple days, I have been monitoring my PS4 while using it that way.

    So I was able to see which port's were active, and which of them was passing active data while certain things were in use... Like using party chat, and playing both Call of Duty Black Ops 3, and Battlefield Hardline online multiplayer. Between the three things I used. Port 9307 was used for party chat primarily, I do think I once seen port 9305 used while I was in a party chat with 7 people. However 99.9% of the time port 9307 was used for most people I was connected to in party chat. Which party chat on PS4 is setup P2P, not any kind of server hosting the chat.

    Also like I mentioned while playing Battlefield Hardline, while in a game/lobby port 3659 is used to pass gaming data. I know these game list multiple ports they use, and they might use them to log certain stuff like in game voip, if you're not using party chat, and statistic data for the matches you play. However most these games at least for console games I have played.. Which I have been a console gamer online for a long time now. Strictly uses one port to pass active gaming packet's while in a match. Battlefield Hardline is port 3659, Black Ops 3 is port 3074, and likely all the COD games on PS4 use port 3074 to pass in match gaming packet's. Which I know a lot of people still actively play Call of Duty on console, but for me the last couple games have went downhill for me. So I haven't played COD as much myself... But hope the new COD WWII brings some enjoyment back to COD series, with things being boots on the ground again.
     
  5. lilstone87

    lilstone87 Senior Member

    Joined:
    Dec 13, 2012
    Messages:
    256
    @FreshJR
    Well I turned on my PS4 a little bit ago, and one of my games started downloading a patch, and after it finished. I decided to take a screenshot of Tomato's QoS View Detail's page. Just to show you what the PS4 has open, in mostly a idle state.. As I just powered it on, and let it sit till after the patch download was done. No game started, or anything. So I will attach a screenshot in this post, which you might be able to use, to see what the PS4 is accessing, before any games are involved.

    I know I mentioned seeing the PS4 use between 49k to 65k port range. However them port's are listed as source ports for the PS4, yet for the destination port(Sony's servers, and such.). The destination port is a mixture of port 80/443.
     

    Attached Files:

  6. lilstone87

    lilstone87 Senior Member

    Joined:
    Dec 13, 2012
    Messages:
    256
    Also here's a second screenshot of my PS4 active connection from just a few minutes ago. This is after like a hour of me playing Battlefield Hardline multiplayer. I had tomato set to show active connection's with the most used data at the top. You can see Hardline was at the top using source port of 3659.
     

    Attached Files:

  7. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    1,286
    Location:
    Australia
    ahh i see also long as the port number is on both inbound and outbound that the main port number the game client uses. Though it appears that on pc infinte warfare uses 3074 like the rest of the games, still to this day i dont understand why trearch changed the black ops 3 pc port from 3074 to 27017 it made all the port forward guides for the pc version invalid. But still props for your efforts, im starting to belove that some games may use slightly diffrent ports on pc compared to consoles.
     
  8. FreshJR

    FreshJR Very Senior Member

    Joined:
    Oct 8, 2016
    Messages:
    747
    That's a damn nice traffic analyzer.

    We have nice one too, with a a nicer interface I may add, but instead it only sorts by device then application name. Wish we could append port and ip usuage into our traffic statistics. Most likely this was all intended to be automatic to be all handled with definitions so we wouldn't have to mess with and have it be confusing for many customers. Remember, we are working outside the box here.

    Either way, without a more advanced analyzer, doesn't Sony or the game developers themselves publish this information online?

    Check out Apple, they did documentation correctly, https://support.apple.com/en-us/HT202944

    Even T-Mobile has this information published on their website.

    Looking up Linux traffic analyzers, many nice ones are available to be installed. You can also use wireshark from pc to sniff all traffic. Many options
     
    Last edited: Jul 26, 2017
  9. MarCoMLXXV

    MarCoMLXXV Guest

    Hi @FreshJR , first of all thanks for your effort to write and maintain your script. I installed it a few days ago and everything worked flawless. Yesterday I did a speedtest and the results were much lower than usual, so I thought of your script. I noticed the speedtest was registered under File Transferring, so I edited the variables so that File Transfers was given more bandwidth and lowered some other. Checked that for both upload and download the percentages were still 100% in total each. Rebooted my router, did the same speedtest, results were still unsatisfying. I checked syslog and saw a huge amount of log lines like below:

    Code:
    Jul 28 19:52:34 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:52:34 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:52:34 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:52:34 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:52:34 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:52:34 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:52:34 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:52:34 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:52:34 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:52:34 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:53:04 dMP17 kernel: net_ratelimit: 79 callbacks suppressed
    Jul 28 19:53:04 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:53:04 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:53:04 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:53:04 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:53:05 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:53:05 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:53:05 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:53:05 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:53:05 dMP17 kernel: TCP: time wait bucket table overflow
    Jul 28 19:53:05 dMP17 kernel: TCP: time wait bucket table overflow
    There were literally thousand of these lines, which I've never seen before in syslog. Then, a short time later, I noticed:

    Code:
    Jul 28 19:54:34 dMP17 marco: Adaptive QOS: Modification Script Started
    Jul 28 19:55:06 dMP17 marco: Adaptive QOS: No change required for Unidentified Traffic Container or Custom Rules
    Jul 28 19:55:07 dMP17 marco: Adaptive QOS: Changing minimum alloted bandwidth per QOS category to user defined percentages
    Jul 28 19:55:38 dMP17 marco: Adaptive QOS: Modification Script Started
    Jul 28 19:56:01 dMP17 marco: Adaptive QOS: Modification Script Started
    Jul 28 19:56:13 dMP17 kernel: SysRq : Emergency Sync <----------
    Jul 28 19:56:13 dMP17 kernel: Emergency Sync complete <---------
    Jul 28 19:56:15 dMP17 kernel: mod epilog takes 0 jiffies
    Jul 28 19:56:15 dMP17 kernel: IDPfw: Exit IDPfw
    Jul 28 19:56:15 dMP17 kernel: Stop the IPS/AppID engine...
    Jul 28 19:56:16 dMP17 kernel: IDPfw: Exit chrdev /dev/idpfw with major 191
    Jul 28 19:56:16 dMP17 kernel: [udb_exit:959] Free udb at cde1f000
    Jul 28 19:56:16 dMP17 kernel: [udb_exit:960] Free app at d2db3000
    Jul 28 19:56:16 dMP17 kernel: [udb_exit:961] Free patrol table at d2dd6000
    Jul 28 19:56:16 dMP17 custom_script: Running /jffs/scripts/services-stop
    Jul 28 19:56:16 dMP17 WEBDAV_Server: daemon is stopped
    Jul 28 19:56:16 dMP17 NAT_Tunnel: AAE Service is stopped
    Jul 28 19:56:16 dMP17 NAT_Tunnel: AAE Service is stopped
    Jul 28 19:56:16 dMP17 iTunes: daemon is stopped
    Jul 28 19:56:16 dMP17 FTP_Server: daemon is stopped
    Jul 28 19:56:17 dMP17 pixelserv: 3147 uts, 0 log, 211 req, 432 avg, 538 rmx, 153 tav, 9997 tmx, 193 slh, 2 slm, 0 sle, 9 slu, 1 nfe, 0 gif, 0 ico, 6 txt, 0 jpg, 0 png, 0 swf, 0 sta, 0 stt, 0 ufe, 0 rdr, 0 nou, 0 pth, 0 204, 0 pst, 0 hed, 0 bad, 0 err, 0 tmo, 204 cls
    Jul 28 19:56:17 dMP17 pixelserv: exit on SIGTERM
    Jul 28 19:56:18 dMP17 Samba_Server: smb daemon is stopped
    Jul 28 19:56:18 dMP17 kernel: gro disabled
    Jul 28 19:56:18 dMP17 Timemachine: daemon is stopped
    Jul 28 19:56:18 dMP17 disk_monitor: Finish
    Jul 28 19:56:19 dMP17 ntpd: ntpd exiting on signal 15 (Terminated)
    My initial changes (before things went wrong) were:

    Code:
        ####################  Variables Setup #####################
    
        #Percent of download speed guaranteed per QOS catagory, change below as desired (sum should equal 100)
            NetControl_DownBandPercent=5                    #This value can be adjust as desired
            VoIP_DownBandPercent=5                            #This value can be adjust as desired
            Gaming_DownBandPercent=15                        #This value can be adjust as desired
            Others_DownBandPercent=10                        #This value can be adjust as desired        #Note: New destination for all unidentified traffic per script default
            WebSurfing_DownBandPercent=20                    #This value can be adjust as desired
            Video_DownBandPercent=35                        #This value can be adjust as desired
            FileTransfer_DownBandPercent=5                    #This value can be adjust as desired
            Default_DownBandPercent=5                        #This value can be adjust as desired        #Note: Original destination all for unidentified traffic, no traffic should flow here
      
        #Percent of upload speed guaranteed per QOS catagory, change below as desired (sum should equal 100)
            NetControl_UpBandPercent=5                        #This value can be adjust as desired
            VoIP_UpBandPercent=5                            #This value can be adjust as desired
            Gaming_UpBandPercent=15                            #This value can be adjust as desired
            Others_UpBandPercent=30                            #This value can be adjust as desired        #Note: New destination for all unidentified traffic per script default
            WebSurfing_UpBandPercent=15                        #This value can be adjust as desired
            Video_UpBandPercent=20                            #This value can be adjust as desired
            FileTransfer_UpBandPercent=5                    #This value can be adjust as desired
            Default_UpBandPercent=5                            #This value can be adjust as desired        #Note: Original destination all for unidentified traffic, no traffic should flow here
    
    and after the changes I made, it looked like:

    Code:
    ####################  Variables Setup #####################
    
        #Percent of download speed guaranteed per QOS catagory, change below as desired (sum should equal 100)
            NetControl_DownBandPercent=5                    #This value can be adjust as desired
            VoIP_DownBandPercent=5                            #This value can be adjust as desired
            Gaming_DownBandPercent=15                        #This value can be adjust as desired
            Others_DownBandPercent=5                        #This value can be adjust as desired        #Note: New destination for all unidentified traffic per script default
            WebSurfing_DownBandPercent=20                    #This value can be adjust as desired
            Video_DownBandPercent=30                        #This value can be adjust as desired
            FileTransfer_DownBandPercent=15                    #This value can be adjust as desired
            Default_DownBandPercent=5                        #This value can be adjust as desired        #Note: Original destination all for unidentified traffic, no traffic should flow here
      
        #Percent of upload speed guaranteed per QOS catagory, change below as desired (sum should equal 100)
            NetControl_UpBandPercent=5                        #This value can be adjust as desired
            VoIP_UpBandPercent=5                            #This value can be adjust as desired
            Gaming_UpBandPercent=15                            #This value can be adjust as desired
            Others_UpBandPercent=25                            #This value can be adjust as desired        #Note: New destination for all unidentified traffic per script default
            WebSurfing_UpBandPercent=15                        #This value can be adjust as desired
            Video_UpBandPercent=15                            #This value can be adjust as desired
            FileTransfer_UpBandPercent=15                    #This value can be adjust as desired
            Default_UpBandPercent=5                            #This value can be adjust as desired        #Note: Original destination all for unidentified traffic, no traffic should flow here
    
    When I saw the kernel doing an emergency sync, I decided to remove your script for now. No more lines as quoted above anymore, and through the night it remained stable and syslog hasn't showed any unknown errors anymore.

    Any suggestions to what has caused this? Thanks in advance.
     
    Last edited by a moderator: Jul 29, 2017
  10. FreshJR

    FreshJR Very Senior Member

    Joined:
    Oct 8, 2016
    Messages:
    747
    @MarCoMLXXV send me a copy of the exact file you uploaded to the router.

    I haven't run into the issue myself
     
  11. MarCoMLXXV

    MarCoMLXXV Guest

    Thanks for your reply. I've posted mine here: https://pastebin.com/WRR0dGsU

    The only thing I changed were the values in the section 'variables setup', the rest of the script remained untouched. Maybe a wrong distribution of the values across containers?
     
    Last edited by a moderator: Jul 30, 2017
  12. FreshJR

    FreshJR Very Senior Member

    Joined:
    Oct 8, 2016
    Messages:
    747
    I need the actual text file, not the pastebin, PM it to me.

    When I downloaded ur pastebin, it had the WRONG return line character. If that indeed is the case with the file you uploaded to the router, its not compatible and will lead to weird problems. I warned about this in the initial post.

    Good on not adding spaces after the = operator. That is because variable assignment is very picky aswell in shell scripts. Everything else was unchanged as you said.
     
    Last edited: Jul 30, 2017
  13. MarCoMLXXV

    MarCoMLXXV Guest

    I don't see a way to attach a file through PM? There's no 'Upload file' button in Conversations? Or am I missing something.

    As for the line endings. I always edit in NotePad++ and make sure it's EOLs are converted to UNIX (LF) before I upload anything to my router. Just checked, it's definitely UNIX LF. Got a nine year old around here so messing up my router means all hell will break loose here :confused:

    Does this help: https://www.dropbox.com/sh/e3v7z22h1l1r3gq/AACyF2pFJs5WzayI6nEdkLtFa?dl=0 ? I've attached it to this post as well.

    I don't get why Pastebin would convert the line feeds, though.

    Strange thing is, it worked perfectly at first, untill I made some changes to the variables, then things went down the hill... Even after a reboot, the errors were still flooding my logs.
     

    Attached Files:

  14. Nigel Jones

    Nigel Jones Regular Contributor

    Joined:
    May 23, 2017
    Messages:
    100
    I've noticed on a few occasions now that I see my speedtest.net result a little slow (150-160 Mbps) on my PC compared to usual (215-220 Mbps).

    Few systems are connected. I see little data flow. I disable QOS -> full speed. I reenable QOS (which brings in the adjustments) -> also full speed.

    Any ideas? This could be a merlin QOS issue, or an issue with these rules, perhaps a memory leak in the kernel too - any tips on debugging welcome?
    If it continues to occur I'll first try disabling the mods here...

    AC-3200 running .67 & script 1.9
     
  15. Uthall

    Uthall Occasional Visitor

    Joined:
    Jul 26, 2017
    Messages:
    42
    Hello,

    I want to add port 25461 into the script and have it classified as "Video and Audio Streaming"

    How do i do that?

    Thanks
    Jeff

    EDIT: Its OK, i think i worked it out
     
    Last edited: Jul 31, 2017
  16. MarioCaires

    MarioCaires Regular Contributor

    Joined:
    May 13, 2015
    Messages:
    137
    Location:
    London
    Ahahah... I think I have the same "Streaming" service as you! However I found that the traffic is already correctly identified even w/o the custom rule. ;)
     
  17. Uthall

    Uthall Occasional Visitor

    Joined:
    Jul 26, 2017
    Messages:
    42
    Lol....

    Well i wish mine was, what version of the Merlin firmware are you using, and i take it your not using the custom script?
     
  18. MarioCaires

    MarioCaires Regular Contributor

    Joined:
    May 13, 2015
    Messages:
    137
    Location:
    London
    Merlin's version is in my signature and I'm using the script with some custom rules, mostly for torrent traffic.
     
  19. Uthall

    Uthall Occasional Visitor

    Joined:
    Jul 26, 2017
    Messages:
    42
    Question, the variables in the script don't match the names in the QOS Statistics window

    eg.
    script = "Streaming"
    QOS Statistics = "Video and Audio Streaming"

    Does that matter?
     
  20. MarioCaires

    MarioCaires Regular Contributor

    Joined:
    May 13, 2015
    Messages:
    137
    Location:
    London
    Doesn't matter, they're the same, just named differently by FreshJR.
     
  21. FreshJR

    FreshJR Very Senior Member

    Joined:
    Oct 8, 2016
    Messages:
    747
    Defiantly different names than webUI. Spaces in variables are a no-no. Valid names are the following (Case Sensitive)

    VOIP
    Gaming
    Others
    Web
    Downloads
    Default
     
Please support SNBForums! Just click on this link before you buy something from Amazon and we'll get a small commission on anything you buy. Thanks!