1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

pixelserv - A Better One-pixel Webserver for Adblock

Discussion in 'Asuswrt-Merlin' started by kvic, Jul 28, 2015.

  1. FalconB

    FalconB Occasional Visitor

    Joined:
    Apr 20, 2017
    Messages:
    29
    Sorry if this has been asked before but I can't find an answer. So, I've used Pixelserv-tls along AB-Solution for quite a long time now, and it's working great. However, I see questions and discussions from time to time regarding certificates and how one should import it to the web-browser. What's up with that? Ads are currently beeing blocked for me, so what benefit do I get if I import the certificate?
     
    Last edited: Dec 14, 2017
  2. Please support SNBForums! Just click on this link before you buy something from Amazon and we'll get a small commission on anything you buy. Thanks!
  3. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,059
    Location:
    Hong Kong
    @FalconB

    See if the FAQ provides additional info on what's pixelserv-tls and how it works. Importing the CA certificate on clients allows graceful communication between clients and pixelserv-tls server for HTTPS ad requests which are trending upward everyday. You may also want to read this blog post: pixelserv-tls More is Less.
     
    SMS786 and FalconB like this.
  4. Xentrk

    Xentrk Very Senior Member

    Joined:
    Jul 21, 2016
    Messages:
    1,374
    Location:
    Thailand
    I will keep you posted. I run another host file ad blocker solution I found on the web. I will update the post with the link when I get home. It is not as comprehensive as ABS. But it does the job.
     
  5. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,059
    Location:
    Hong Kong
    @Xentrk @Lost Dog

    I know some users have been compiling pixelserv-tls for LEDE/OpenWRT. You can make a request to OpenWRT to add pixelserv-tls to their packages. You may also try to contact the person in this ticket. See if he can share his build or create one compatible for your travel router. I can also link his work to the supported platforms page.

    On the other hand, there is a OpenWRT package known as "adblock" which does a similar job of preparing blocked host files for DNSmasq. It seems doing a good job on performance.
     
  6. Xentrk

    Xentrk Very Senior Member

    Joined:
    Jul 21, 2016
    Messages:
    1,374
    Location:
    Thailand
    Thanks @kvic,

    I did some research on the adblock package. The prerequisite is LEDE project, tested with latest stable release (LEDE 17.01) and with current LEDE snapshot. I'd have to flash LEDE and it looked like more effort than I wanted to put into it for the little I use the travel router.

    I will do as you suggest in regard to pixelserv. I just came across this one https://github.com/jjack/openwrt-adblock and has a link to a 1 pixel gif.

    The link to the site I got the code from is here. https://paul.is-a-geek.org/2015/06/dns-based-adblock-using-openwrt-opendns-and-dnsmasq/. But the site is not working to an invalid certificate.
    #!/bin/sh
    #Block ads, malware, etc.
    logger -t "adblock" -s 'Starting adblock setup...'

    #Delete the old adblock_hosts to make room for the updates
    rm /tmp/adblock_hosts

    logger -t "adblock" -s 'Downloading hosts lists...'
    wget -qO- "http://winhelp2002.mvps.org/hosts.txt" | awk '/^0.0.0.0/' > /tmp/block.build.list
    wget -qO- "http://www.malwaredomainlist.com/hostslist/hosts.txt" | awk '{sub(/^127.0.0.1/, "0.0.0.0")} /^0.0.0.0/' >> /tmp/block.build.list
    wget -qO- "http://hosts-file.net/ad_servers.txt" | awk '{sub(/^127.0.0.1/, "0.0.0.0")} /^0.0.0.0/' >> /tmp/block.build.list
    wget -qO- "http://adaway.org/hosts.txt" | awk '{sub(/^127.0.0.1/, "0.0.0.0")} /^0.0.0.0/' >> /tmp/block.build.list

    if [ -s "/etc/black.list" ]
    then
    logger -t "adblock" -s 'Adding blacklist...'
    awk '/^[^#]/ { print "0.0.0.0",$1 }' /etc/black.list >> /tmp/block.build.list
    fi

    logger -t "adblock" -s 'Sorting lists...'
    awk '{sub(/\r$/,"");print $1,$2}' /tmp/block.build.list|sort -u > /tmp/block.build.before

    if [ -s "/etc/white.list" ]
    then
    #Filter the blacklist, supressing whitelist matches
    # This is relatively slow =-(
    logger -t "adblock" -s 'Filtering white list...'
    awk '/^[^#]/ {sub(/\r$/,"");print $1}' /etc/white.list | grep -vf - /tmp/block.build.before > /tmp/adblock_hosts
    else
    cat /tmp/block.build.before > /tmp/adblock_hosts
    fi

    logger -t "adblock" -s 'Cleaning up...'

    #Delete files used to build list to free up the limited space
    rm -f /tmp/block.build.before
    rm -f /tmp/block.build.list

    logger -t "adblock" -s 'Restarting dnsmasq...'
    /etc/init.d/dnsmasq restart

    logger -t "adblock" -s 'Finished adblock setup'

    exit 0
    I have something similar on a DD-WRT router that has a few more lists that I plan to test with if it tests out okay. Very similar but with the same amount of host files that AB-Solution has.
     
    kvic likes this.
  7. FalconB

    FalconB Occasional Visitor

    Joined:
    Apr 20, 2017
    Messages:
    29
    Thanks for the links, had read some of it before, but still, I don't get it. Sorry for going on about this, but what do you mean by "Importing the CA certificate on clients allows graceful communication between clients and pixelserv-tls server"? What is the purpose of the certificate and what are the benefits of importing it into my web-browser? If someone could explain it in really simple terms I would really appreciate it. Please feel free to PM me if you don't want to clutter the thread with this. Again, sorry for beeing thick in the head and thanks for putting your effort into this, it's great work!
     
  8. thelonelycoder

    thelonelycoder Part of the Furniture

    Joined:
    Jan 23, 2014
    Messages:
    4,176
    Location:
    Switzerland
    Operating systems and browsers have a list of trusted certificates.
    Apps and browsers check that list first before communicating with a server running on the secure protocol, https is one of them. If no valid certificate is found, communication is aborted.

    Pixelserv uses a self-signed certificate which is not in that list. And since pixelserv-tls is just another server running on https from the viewpoint of apps or browsers, communication with it is refused.
    Unless you import the pixelserv certificate and with that set it as trusted in the list.
     
    SMS786, Xentrk and FalconB like this.
  9. FalconB

    FalconB Occasional Visitor

    Joined:
    Apr 20, 2017
    Messages:
    29
    Ok, that cleared some things up. So if I am understanding it correctly: Since I have not imported the certificate to my browser, that's the reason for some sites/links beeing blocked (for example some of the google shopping links), right? And if I were to import the certificate, those site would load (but still without the ads), correct?
     
  10. thelonelycoder

    thelonelycoder Part of the Furniture

    Joined:
    Jan 23, 2014
    Messages:
    4,176
    Location:
    Switzerland
    That's the gist of it.
    When you visit a website the page is dowloaded which may contain links to other sources. The browser/app then requests that content. If the source is blocked in AB-Solution the request either dead ends at or is answered with a pixel sized image by pixelserv. The app/browser trusts the response from pixelserv if you have the certificate imported.
     
    SMS786, heysoundude and FalconB like this.
  11. FalconB

    FalconB Occasional Visitor

    Joined:
    Apr 20, 2017
    Messages:
    29
    Well, thank you very much for straighten this out for me!
     
  12. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,059
    Location:
    Hong Kong
    @Xentrk

    I believe the "adblock" description meant to say it tested to work on the latest OpenWRT. Not that it depends on anything specific to LEDE. Note that LEDE and OpenWRT merged already this month, and "adblock" package (which is a bash script) started on OpenWRT when LEDE didn't exist yet.

    I would think it shall work in any recent versions of OpenWRT/LEDE. The script you posted will do the job too. So in the end, it's personal preference really but I've to praise the author of picking NXDOMAIN. Pls read the speed benchmark of pixelserv-tls against NXDOMAIN.
     
  13. Xentrk

    Xentrk Very Senior Member

    Joined:
    Jul 21, 2016
    Messages:
    1,374
    Location:
    Thailand
    Very kewl! Thanks for the update. When I last researched this, I learned there was a plan to merge the two together. If I recall, LEDE was forked from OpenWRT? I will get with Gli support to see if a new OS version is available or in the works. I will let you know.
     
  14. snixel

    snixel Occasional Visitor

    Joined:
    Dec 13, 2017
    Messages:
    22
    I don't know if it's asked before, but I'm currently using version: v35.HZ12.Kk compiled: Sep 25 2017 that came with AB-Solution. Can i just update with
    sh -c "$(wget -qO - https://kazoo.ga/pixelserv-tls/install-beta.sh)" to v2.0.1 or do I wait for an update of ab-solution?
     
  15. thelonelycoder

    thelonelycoder Part of the Furniture

    Joined:
    Jan 23, 2014
    Messages:
    4,176
    Location:
    Switzerland
    Makaveli likes this.
  16. elorimer

    elorimer Senior Member

    Joined:
    Dec 16, 2013
    Messages:
    430
    I haven't lost sight of this. Post 608 has 58 likes. Are we there yet?
     
  17. froggy666uk

    froggy666uk New Around Here

    Joined:
    May 20, 2013
    Messages:
    7
    Was wondering the same myself!

    Sent from my Nexus 5X using Tapatalk
     
  18. snixel

    snixel Occasional Visitor

    Joined:
    Dec 13, 2017
    Messages:
    22
  19. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,059
    Location:
    Hong Kong
    I'm extending my offer to Easter Saturday so that ppl get a bit more time to work on it :)

    Happy holidays!
     
  20. snixel

    snixel Occasional Visitor

    Joined:
    Dec 13, 2017
    Messages:
    22
    mayb a dumb question, but how do get the CA certs of the router or can I enable SFTP somewhere? The only way I now see it is to power off the router and put the usb key in a linux box.

    Thanks and happy holidays!
     
  21. Adamm

    Adamm Very Senior Member

    Joined:
    Mar 26, 2013
    Messages:
    1,515
    You can access the files via network on a windows device. I also believe these files are backed up with AB-Solution if you use that in conjunction with pixelserv
     
    SMS786 likes this.
Please support SNBForums! Just click on this link before you buy something from Amazon and we'll get a small commission on anything you buy. Thanks!