1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

Diversion - the Router Ad-Blocker

Discussion in 'Asuswrt-Merlin' started by thelonelycoder, Aug 25, 2018.

  1. dave14305

    dave14305 Part of the Furniture

    Joined:
    May 19, 2018
    Messages:
    2,405
    Location:
    USA
    That would add many hosts beyond your chosen blocklist.
     
  2. eclp

    eclp Senior Member

    Joined:
    May 15, 2016
    Messages:
    303
    Is it possible that Diversion uses only a limited amount of RAM available in the device (I know that Linux handles memory differently than, for example, Windows)?

    :)
     
  3. dave14305

    dave14305 Part of the Furniture

    Joined:
    May 19, 2018
    Messages:
    2,405
    Location:
    USA
    Your dnsmasq memory usage is directly related to the size of your blocking list in Diversion.
     
  4. Diamond67

    Diamond67 Senior Member

    Joined:
    Jul 23, 2015
    Messages:
    211
    I don't know what Unbound is so my answer is no.
     
    dave14305 likes this.
  5. eclp

    eclp Senior Member

    Joined:
    May 15, 2016
    Messages:
    303
    Previously, I used Diversion & Pixelserv only with default settings. After a certain time, probably several days, the RAM ran full and the device eventually crashed or the GUI was no longer accessible. I observed this behavior for at least half a year before I decided not to use the script for the time being. Since then, there has been no such behavior described above. If possible, I would therefore like to limit the use of RAM and reuse Diversion.

    :)
     
  6. dave14305

    dave14305 Part of the Furniture

    Joined:
    May 19, 2018
    Messages:
    2,405
    Location:
    USA
    I stopped using Pixelserv a week or so ago after I noticed 184 pixelserv-tls processes (really threads) stemming from a blocked domain triggered while my Termius iOS ssh client was running. It never let up. So some apps may be persistent and repeatedly try to access a blocked domain rapidly in succession, spawning many pixelserv-tls threads visible in htop. I believe I read before that Instagram was another such app.

    Otherwise, Diversion itself is very memory friendly if using anything smaller than the Large blocking list. Plus hosts are also of questionable value for me personally and add a lot of hosts to the memory footprint.

    Start small, start Lite and observe.
     
  7. Makaveli

    Makaveli Very Senior Member

    Joined:
    Nov 4, 2016
    Messages:
    676
    Location:
    Canada
    Hmm this is something I will keep an eye out for.

    I'm using the Standard blocking list and do use instagram.
     
  8. juched

    juched Regular Contributor

    Joined:
    Jan 14, 2018
    Messages:
    167
    Location:
    Canada
    I too have seen, and regularly do see my pixelserv-tls thread count go up over 100, and the memory grows too. Then it stops and works it way down to 2 or 3 threads. No idea what is causing this yet.
     
  9. juched

    juched Regular Contributor

    Joined:
    Jan 14, 2018
    Messages:
    167
    Location:
    Canada
    @thelonelycoder quick question for you. I noticed that in your rotate-logs script you still maintain dnsmasq.log2 (weekly) and dnsmasq.log1 (yesterday) even if weekly stats is turned off. Curious what else you use the dnsmasq.log1/2 for? Seems like a waste to keep around unless other parts are using it, and it is extra disk and computation.

    personally my dnsmasq logs grow very quickly, so I am turning off stats generation. It took over 11 minutes last night to generate the stats, and that is on my RT-AX88U which is quite fast.

    Thank you for your time.
     
  10. Butterfly Bones

    Butterfly Bones Very Senior Member

    Joined:
    Apr 10, 2017
    Messages:
    1,272
    Location:
    USA
    That was explained by kvic in the pixelserv-tls thread back in the earlier active days of development. That is by design when you get say 15-20 people all web browsing and all those ads hitting so pixelserv-tls increases the number requests per service thread and then the number of active service threads. When / if activity slows, the number of service threads will decrease.

    It all shows in the pixelserv stats.
    Code:
    pixelserv-tls 2.3.1 (compiled: Jan 31 2020 13:27:14 flags: tfo tls1_3) options: 192.168.1.2
    
    uts 0d 00:44 process uptime
    log 1 critical (0) error (1) warning (2) notice (3) info (4) debug (5)
    kcc 1 number of active service threads
    kmx 1 maximum number of service threads
    kvg 0.00 average number of requests per service thread
    krq 0 max number of requests by one service thread
    
     
    Makaveli and dave14305 like this.
  11. thelonelycoder

    thelonelycoder Part of the Furniture

    Joined:
    Jan 23, 2014
    Messages:
    6,464
    Location:
    Switzerland
    Logging is enabled.
     
  12. juched

    juched Regular Contributor

    Joined:
    Jan 14, 2018
    Messages:
    167
    Location:
    Canada
    Gotcha. Thanks.
     
  13. GregS

    GregS Occasional Visitor

    Joined:
    Apr 3, 2015
    Messages:
    36
    I'm seeing an odd issue on my RT-AX88U, after awhile no DNS requests get resolved. nslookup from any client on the network will time out, following the dnsmasq log shows no queries. If I disable diversion then DNS resolution starts working again, enable and it stops again. A reboot will resolve the issue so dns will work while diversion is enabled; but the issue will return days later.
    Any ideas what could be wrong? Or what I should check on the next time this happens. I did force a diversion update during the last issue but that didn't help.

    For background I have been using Diversion for a few years on an AC68U and only recently upgraded to an AX88U as repeated requests to pixelserv from my ShieldTV kept overloading the router. I'm using the Standard+ blocking list. I'm using v384.15_0 of Merlin's firmware.
     
  14. L&LD

    L&LD Part of the Furniture

    Joined:
    Dec 9, 2013
    Messages:
    11,437
    @GregS, did you reuse your USB drive that was running Diversion on the RT-AC68U to the RT-AX88U?

    If you did, I would recommend formatting the USB drive on a PC and 'Format JFFS partition on next reboot' to completely wipe out your previous configuration. Make sure to save anything of importance on the JFFS partition first, if needed.

    Then, use amtm to format the USB drive to Ext4 w/journaling and create a swap file. Now, do a clean install of the scripts you use, beginning with Diversion.
     
  15. dave14305

    dave14305 Part of the Furniture

    Joined:
    May 19, 2018
    Messages:
    2,405
    Location:
    USA
    You will be OK to continue. The install function is catching 44300 instead of just 443 (partial match).
     
  16. ugandy

    ugandy Occasional Visitor

    Joined:
    Sep 6, 2019
    Messages:
    13
    thanks. upnp media server was on.
     
    donbru likes this.
  17. bitmonster

    bitmonster Senior Member

    Joined:
    Sep 26, 2018
    Messages:
    267
    I'm not sure if this is the right place to ask - but is it possible to implement a Murdoch Media blocklist? I am just disgusted by their behaviour and want all Murdoch sites blocked so I can do my bit to starve this parasite of clicks and revenue. Diversion blocklists would be the perfect way to do it, as I also have my phone pretty much permanently VPN'd through home so it would protect my phone from this disease too.

    https://iview.abc.net.au/show/media-watch/series/0/video/FA1935H008S00
     
  18. maghuro

    maghuro Regular Contributor

    Joined:
    Feb 19, 2020
    Messages:
    144
    Location:
    Portugal
    AFAIK you can create your own blacklist on top of any list you're using.

    Just add the domain(s) you want to block after installing diversion
     
  19. jrmwvu04

    jrmwvu04 Very Senior Member

    Joined:
    Mar 29, 2016
    Messages:
    603
    Location:
    United States
    It likes to ping e.crashlyrics.com and doesn’t accept no for an answer. Not that uncommon in iOS apps in my experience
     
  20. Butterfly Bones

    Butterfly Bones Very Senior Member

    Joined:
    Apr 10, 2017
    Messages:
    1,272
    Location:
    USA
    Back during pixelserv active development, kvic gave an alternate way to handle those pesky carrier domains. I've run this way since those early days.
    Code:
    # /jffs/configs/hosts.add
    
    0.0.0.0 t.appflyer.com
    0.0.0.0 x.flyme.com
    0.0.0.0 e.crashlytics.com
    0.0.0.0 app-measurement.com
    0.0.0.0 gateway-carry.icloud.com                 
    0.0.0.0 e12930.ksd.akamaiedge.net                 
    0.0.0.0 ssl.google-analytics.com
    0.0.0.0 mesu.g.aaplimg.com                       
    0.0.0.0 gsp64-ssl.ls-apple.com.akadns.net         
    0.0.0.0 p3-buy.itunes-apple.com.akadns.net       
    0.0.0.0 e673.dsce9.akamaiedge.net                 
    0.0.0.0 settings.crashlytics.com
    0.0.0.0 reports.crashlytics.com
    0.0.0.0 onesignal.com
    0.0.0.0 data.flurry.com
    0.0.0.0 api.branch.io
    0.0.0.0 www.google-analytics.com
    0.0.0.0 adservice.google.com
    0.0.0.0 ad.doubleclick.net
    0.0.0.0 ssl.google-analytics.com
    0.0.0.0 securepubads.g.doubleclick.net
    0.0.0.0 ib.adnxs.com
    0.0.0.0 sb.scorecardresearch.com
    0.0.0.0 js-agent.newrelic.com
    0.0.0.0 www.googleadservices.com
    0.0.0.0 ads.servebom.com
    0.0.0.0 api.amplitude.com
    
     
    L&LD, QuikSilver and Marin like this.