1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

FreshJR QoS script w/bwdpi.app.db v. 2.172

Discussion in 'Asuswrt-Merlin' started by ututo01, May 24, 2020.

  1. ututo01

    ututo01 Occasional Visitor

    Joined:
    May 24, 2020
    Messages:
    14
    My name is Daniel Olivera. Live in Buenos Aires.
    This is my first post in this forum

    I made an adaptation of FreshJR QoS adaptive script to support new applications optimization in bwdpi app database from new stock firmware version 3.0.0.4.384.81844

    I`m using Merlin 384.17 and bwdpi.* database, new signature version 2.172 and made modifications in FreshJR script to view "Work from Home" and "Learn from Home" new categories availables in stock firmware 384.8xxxx. Bwdpi database was extracted from stock firmware and copied to Merlin firmware using HDD connected in USB port.

    QoS adaptive is manual configured for speed and using Fq-CoDel with overhead in 19 (Bridget/IPoE option)
    I have 300/300 Mb FTTH by ISP Movistar in Buenos Aires, Argentina with RT-AC5300 router

    Some services like a local supermarket called COTO, Jitsi (web videocall), static content from Facebook and Instagram was added using legacy iptables mark style in FreshJR script

    Also made linux kernel modifications in some parameters.
    Some pictures and speed test video from my phone in following url (OneDrive)

    https://photos.app.goo.gl/TeezXS2Qrt3ukX9u9


    Best to all.
     
    Last edited: May 24, 2020
    Vexira and Kingp1n like this.
  2. QuikSilver

    QuikSilver Very Senior Member

    Joined:
    Jan 30, 2019
    Messages:
    684
    Location:
    BFE
    The pictures look great, but I'm not sure I follow the point of this thread. It seems you updated the old FreshJr script to include the new QOS categories but I'm not sure I understand the "bwdpi" reference. Do you care to explain in more detail how you did so others could update theirs or follow along as well. I'm really interested in this as other are as well due to the fact that .18 changes how FreshJR's script works and without an updated script some like me will choose to hold off for now.
     
    Vexira and Kingp1n like this.
  3. Kingp1n

    Kingp1n Very Senior Member

    Joined:
    Feb 27, 2018
    Messages:
    616
    c
    Can you upload/share your script here? Thanks!
     
    Vexira and QuikSilver like this.
  4. maghuro

    maghuro Senior Member

    Joined:
    Feb 19, 2020
    Messages:
    445
    Location:
    Portugal
    Vexira and QuikSilver like this.
  5. ututo01

    ututo01 Occasional Visitor

    Joined:
    May 24, 2020
    Messages:
    14
    Hi.

    Bwdpi is located in /tmp/bwdpi directory in router filesystem. This set of files have ids for programs and category assigned for each in QoS.
    I´m using FreshJR last version (8.8) installed using amtm command.
    First I flash my rt-ac5300 router with stock firmware. Then copy to an external HDD all directory bwdpi (/tmp/bwdpi)
    Re flash using Merlin 384.17 firmware. Using phone app, set QoS adaptive and other like https access, remote access, etc.
    Using web router access, set in QoS for use Fq-Codel, manual speed settings for download and upload and packet overhead in 19 for my FTTH Bridged/IPoE connection.
    Then update signature in Administration->Firmware and set SSH access for local only. Obvious set jffs filesystem.
    Accessing using ssh, set amtm and install FreshJR in last version.
    Copy from HDD stock "bwdpi" databases (only files bwdpi.*) to /tmp/bwdpi directory

    Then modify "firmware-restart" to kill "dcd" program (see picture about this) and set bwdpi_dpi_ver nvram variable
    Modifications in FreshJR was some hard. Some local services was address as iptables mark legacy method.

    Stock firmware show one new category called "Learn from Home" in QoS (app and web access).
    FreshJR original web ".asp" page show like "Net Control" between "Web Surfing" and "Voip..."
    This is because new bwdpi categories have one new "rulelist" not present in array of categories in FreshJR original www status page.
    This new "rulelist" was added manually and fix this issue, showing all categories (like photos shown).

    Original (in .asp web page for FreshJR Classification)
    bwdpi_app_rulelist = "9,20<8<4<0,5,6,15,17<13,24<1,3,14<7,10,11,21,23<<"

    New rulelist (Learn for Home)
    bwdpi_app_rulelist = "9,20<8<4<0,5,6,15,17<4,13<13,24<1,3,14<7,10,11,21,23<<"

    NOTE: "<4.13" was added for new category "Learn from Home"

    Then add array values and name for show in status web page.


    Best
     
    Last edited: May 24, 2020
    Vexira and QuikSilver like this.
  6. ututo01

    ututo01 Occasional Visitor

    Joined:
    May 24, 2020
    Messages:
    14
    Hi.

    Here all files modified by me in OneDrive.

    https://1drv.ms/u/s!ApLVLOxftzQxgYQV0B4Hd0_Ukj495Q?e=s1Ob4a

    Files:

    FreshJR
    firewall-restart (created originally by amtm when install FreshJR script)
    www_FreshJR_QoS_Stats.asp (web status page for QoS and FreshJR)


    Best
     
    Last edited: May 24, 2020
    Vexira, QuikSilver and Kingp1n like this.
  7. dave14305

    dave14305 Part of the Furniture

    Joined:
    May 19, 2018
    Messages:
    3,401
    Location:
    USA
    People should realize you have many hard coded custom rules in your script, so it might not be good for them to blindly copy it.

    I have been studying the FreshJR script for a few weeks now, and had been working on enhancements. It’s too soon for me to have a 384.18 build for an AC68U, but I will try to make the necessary adjustments after one is available. If you have confidence in your changes, you could submit a pull request on Github.

    https://github.com/dave14305/FreshJR_QOS
     
  8. ututo01

    ututo01 Occasional Visitor

    Joined:
    May 24, 2020
    Messages:
    14
    Hi.

    Most important is not a personal changes about personal services using FreshJR.
    Some of these are in legacy method to set correct flows for qos using IPs and ports of services.

    Example is my supermarket for online sale (called COTO), static content of Instagram or Facebook for speed up in phone and web access, my home security cameras, Google Classrom, New Facebook Rooms service (Instagram, Messenger Whatsapp), Jitsi Web Videocall service, Adguard DNS (port 5443 marked like Net_mark_XXXX) and Google APIs for my Google Home appliances marked like Net_mark for speed up responses.

    Best part is use new categories to show new programs in 384.17 version of firmware using stock firmware database and show this categories fine in "asp" status page.
    If you use 384.17 filesystem, database and categories are different in stock most new firmware version.

    Now I can see fine "Zoom", "TED", "Whatsapp". "Amazon", Dailymotion", "Google Play", "LetsEncrypt" and others in status page and set (legacy as previously explained) others new services to start using asap in better QoS optimization.

    Please see pictures in my first post.
    My only motivation was set my router at top of my connection speed

    Please see one difference like example
    Original (in .asp web page for FreshJR Classification)
    bwdpi_app_rulelist = "9,20<8<4<0,5,6,15,17<13,24<1,3,14<7,10,11,21,23<<"

    New rulelist (Learn for Home)
    bwdpi_app_rulelist = "9,20<8<4<0,5,6,15,17<4,13<13,24<1,3,14<7,10,11,21,23<<"

    Diference in new bwdpi database: <4.13 is new

    This work was for don´t wait for new release and have this advantages now.


    Best
     
    Last edited: May 24, 2020
  9. joe scian

    joe scian Senior Member

    Joined:
    Apr 22, 2018
    Messages:
    484
    I cannot access
    https://photos.app.goo.gl/TeezXS2Qrt3ukX9u9

    any chance you could host this elsewhere please !!
     
  10. Martineau

    Martineau Part of the Furniture

    Joined:
    Jul 8, 2012
    Messages:
    3,392
    Location:
    UK
    Have you tried whitelisting 'goo.gl'?

    e.g. I had to......using 'unbound_manager' for Ad block
    Code:
    e  = Exit Script [?]
    
    A:Option ==> ew
    
     
    QuikSilver and joe scian like this.
  11. ututo01

    ututo01 Occasional Visitor

    Joined:
    May 24, 2020
    Messages:
    14
    Vexira and joe scian like this.
  12. joe scian

    joe scian Senior Member

    Joined:
    Apr 22, 2018
    Messages:
    484
    Yes- just tried that - thanks its working Martineau
     
  13. joe scian

    joe scian Senior Member

    Joined:
    Apr 22, 2018
    Messages:
    484
    Hi Martineau

    does whitelisting actually change adservers from this
    Code:
    local-zone: "goo.gl" always_nxdomain
    to this?
    Code:
    #local-zone: "goo.gl" always_nxdomain
    Because this is the only way that whitelisting actually works for me. I can add goo.gl to permlist and I can add goo.gl to allowhost
    but neither work until I actually comment out
    Code:
    #local-zone: "goo.gl" always_nxdomain
    in adservers and then rl
     
  14. ututo01

    ututo01 Occasional Visitor

    Joined:
    May 24, 2020
    Messages:
    14
    Hi

    Steps to install all files and bwdpi.

    - upgrade or install last version of app en phone (ios, android)
    - flash router (I test only on RT-AC5300) with Merlin 384.17
    - Set "Factory reset" with physical button on router
    - Configure ISP, wireless and default user in app or web access
    - Configure using app adaptive QoS using by "Game" (default option of adaptive QoS). Use app because change nvram variable called "
    bwdpi_app_rulelist" with new category definition.
    - Set SSH access for lan only
    - Ser jffs scripts in System page on router web access
    - Upgrade "signature" in firmware upgrade page on router web access
    - Set QoS in web access to router to use "Fq-CoDel" algorithm and set download an upload speed connection as "manual"
    - Ser "packet overhead" if know, use preconfigured parameter o leave in zero.
    - Access using ssh to router console
    - Use amtm to install FreshJR script
    - Replace FreshJR_QOS file with modified (https://1drv.ms/u/s!ApLVLOxftzQxgYQV0B4Hd0_Ukj495Q?e=s1Ob4a) in /jffs/scripts
    - Same replace with "firewall-restart" and "www_FreshJR_QoS_Stats.asp" file. Both in /jffs/scripts directory on filesystem router
    - Download "bwdpi.zip" file from https://1drv.ms/u/s!ApLVLOxftzQxgYQV0B4Hd0_Ukj495Q?e=s1Ob4a end decompress
    - Copy only files "bwdpi.*" to directory /tmp/bwdpi located un router filesystem
    - Restart QoS in console on router with following commands: servie firewall_restart

    Now you can use modified FreshJR qos script and new bwdpi database of programs.
    Again, all my photos with details are here: https://photos.app.goo.gl/TeezXS2Qrt3ukX9u9
    Including details of new apps like ZOOM or TED by example.
    FreshJR script modified have another optimizations using legacy iptables mark packets for Facebook Rooms, static content of Facebook and Instagram, Google Classroom, Google APIs for my Google Home environment, Adguard DNS and others.
    Legacy method is not register in status page. But you will see Zoom, Ted, Whatsapp as new apps optimized for QoS.

    Best.
     
    QuikSilver, Kingp1n and joe scian like this.
  15. joe scian

    joe scian Senior Member

    Joined:
    Apr 22, 2018
    Messages:
    484
    Having completed above steps I get 2 new additions to class categories - "work from home" and "game transferring". I do not have "learn from home"
     
  16. ututo01

    ututo01 Occasional Visitor

    Joined:
    May 24, 2020
    Messages:
    14
    In console, you can see nvram variable with new category

    [email protected]:/tmp/home/root# nvram show | grep bwdpi_app_rulelist
    bwdpi_app_rulelist=9,20<8<4<0,5,6,15,17<4,13<13,24<1,3,14<7,10,11,21,23<game
    size: 79617 bytes (51455 left)
    [email protected]:/tmp/home/root#

    If you see "<4,13" in this option new category was added to your system

    In Classification page (http://router.asus.com/QoS_Stats.asp) can see in "pie" explanation at right new categories.
    That is only visualization.

    When download from my OneDrive "FressJR" script and copy to /jffs/script need replace original files.
    Please remove ".txt" in FreshJR script and www_FreshJR_QoS_Stats.asp from downloaded files to replace original FreshJR files or backup originals if you want.
    This extension added is only prevention.


    Best
     
    Kingp1n likes this.
  17. ututo01

    ututo01 Occasional Visitor

    Joined:
    May 24, 2020
    Messages:
    14
    New category can see like this: https://photos.app.goo.gl/tWK2CTG4AEaCMJwx9
    in nvram variable called bwdpi_app_rulelist can si as secuence "<4,13"


    Best.
     
  18. Kingp1n

    Kingp1n Very Senior Member

    Joined:
    Feb 27, 2018
    Messages:
    616
    It seems you have assigned IP addresses for some of the applications i.e. TED, WhatsApp, JITSI, etc.

    I wont use these rules so I put the # sign to prevent them from working at this time.
     
    QuikSilver likes this.
  19. ututo01

    ututo01 Occasional Visitor

    Joined:
    May 24, 2020
    Messages:
    14
    Ok. Thats rules are only "legacy" method.
    This rules don´t show in Classification page.

    Best.
     
  20. Milan

    Milan Regular Contributor

    Joined:
    Nov 21, 2016
    Messages:
    162
    is there a way to overwrite *so lib on a running system without flashing stock fw ?