Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

Privacy Filter (Another IPSET Script)

Discussion in 'Asuswrt-Merlin' started by swetoast, Jan 11, 2017.

  1. swetoast

    swetoast Guest

    Joined:
    Apr 12, 2016
    Messages:
    804
    Hi Everyone, I'm back with another IPSET for your firewall this time around its for blocking Telemetry and some Android Rootkit along with Shodan.io Scanners. For official installatins instructions and more information please consult the wiki about the information.

    NOTE: for all users running iblocklist-loader.sh will be ineligible for support since redhat has incorporated telemetry block along with shodan.io block this makes privacy-filter incompatible so either run his script or my script dont run both will add incompatible check to debugtool.

    To see if the firewall rule is working use the following command:

    ipv4
    Code:
    iptables -L -v | grep "privacy-filter_ipv4" | awk '{print "Privacy Filter (ipv4) Blocked: " $1 " packets", $2 " is the size of the transmission"}'
    ipv6
    Code:
    ip6tables -L -v | grep "privacy-filter_ipv6" | awk '{print "Privacy Filter (ipv6) Blocked: " $1 " packets", $2 " is the size of the transmission"}'
    How to test if an ip is on the list before reporting that this filter is blocking unwanted sources.

    Code:
    ipset test [set] [ip]
    
    example:
    ipset test privacy-filter_ipv4 43.2.5.45
    and to investigate what entry is causing it, well that takes time but remove 10 sources at the time and re-run the script then narrow it down to the offending source.
     
    Last edited: May 4, 2017
    eclp, wallyg8r, bayern1975 and 2 others like this.
  2. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    485
    I don't quite get this line ..... maybe I'm reading it wrong. For me it says
    " if the dnsmasq.conf.add file doesn't exist OR privacy-filter doesn't turn up in a search of dnsmasq.conf.add" then... the next bit. Is that what is meant to test...if so isn't it going to be true all the time?
     
  3. swetoast

    swetoast Guest

    Joined:
    Apr 12, 2016
    Messages:
    804
    it checks if the file is there then it adds to the file thats basically what it does :) then it adds the content to the file and sets it to 127.0.0.1
     
  4. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    485
    I dont understand why you do the file check and in the next line delete it anyway.
    Then you create it again when you append all the server entries.
    Code:
    rm -f $dnsmasq_cfg
        for i in `cat $path/privacy-filter.list`;
    do
        echo "server=/$i/127.0.0.1#1919" >> $dnsmasq_cfg
    done
    Dont get me wrong...not being critical at all...just trying to understand the code.. told you i was dumb :)
     
  5. swetoast

    swetoast Guest

    Joined:
    Apr 12, 2016
    Messages:
    804
    it might be tricky but im sure you will figure it out :)
     
  6. Nutz2U2

    Nutz2U2 Occasional Visitor

    Joined:
    Nov 5, 2016
    Messages:
    24
    Hi to Everyone,

    Wish to thank ALL the boys and girls who made this script workable.

    Works like a charm ;).
     
    swetoast likes this.
  7. Xentrk

    Xentrk Very Senior Member

    Joined:
    Jul 21, 2016
    Messages:
    947
    Location:
    Chiang Mai, Thailand
    Hi @swetoast,

    For preliminary testing purposes, I copied your script to a file called privacy-filter. When running it, I get two error messages:

    Code:
    [email protected]:/jffs/scripts# ./privacy-filter
    cat: can't open '/opt/var/cache/privacy-filter/privacy-filter.list': No such file or directory
    
    Done.
    cat: can't open '/opt/var/cache/privacy-filter/privacy-filter.txt': No such file or directory
    If I create the privacy-filter directory and privacy.list and .txt files, the error message goes away. But I have no values in the .list or .txt file. A dnsmasq.conf.add file does not get created. I need your help in getting this to work. Thanks again for your help!

    EDIT:
    Code:
    lsmod | grep "xt_set"
    xt_set                  2964  4
    ip_set                 18113  3 xt_set,ip_set_hash_ip,ip_set_hash_net
    The Done. response appears to come from this command:
    Code:
    service restart_dnsmasq
     
    Last edited: Jan 13, 2017
  8. visortgw

    visortgw Regular Contributor

    Joined:
    Jun 18, 2015
    Messages:
    133
    Post #1 shows you how to initially populate the privacy.list and .txt files.
     
  9. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    485
    The dnsmasq.conf.add file gets deleted every time you run the script and it only gets recreated by the cat command if there is actually something to append.
    Code:
    rm -f $dnsmasq_cfg
        for i in `cat $path/privacy-filter.list`;
    do
        echo "server=/$i/127.0.0.1#1919" >> $dnsmasq_cfg
    done
    as @visortgw mentions, the privacy-filter.list file will have to be populated with at least one domain.
     
  10. swetoast

    swetoast Guest

    Joined:
    Apr 12, 2016
    Messages:
    804
    yes so that the firmware knows that its supposed to add it to then config, thats why.
     
  11. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    485
    I know I'm not the sharpest tool in the shed when it comes to code...but this bit of the script just doesn't make sense to me.
    The dnsmasq.conf.add file will be checked for the string "privacy-filter" with grep and the statement will be true if nothing is found. The dnsmasq.conf.add file should contain a load of lines with server=/some.host.com/127.0.0.1#1919 type entries..... there will never be a time where grep will find "privacy-filter" so it will always be true...... what am i missing? is "" not interpreted in the way i think?
     
  12. swetoast

    swetoast Guest

    Joined:
    Apr 12, 2016
    Messages:
    804
    yepp thats a typo gonna fix it that should read "server=.*" then it should be -e for regexp still its not a major flaw :p sloppy on my part hence why i have the community to correct me :D

    been awhile since i wrote that script just updated with my ipset checking system think my plan was to have a commented line in the dnsmasq.add file
     
  13. Xentrk

    Xentrk Very Senior Member

    Joined:
    Jul 21, 2016
    Messages:
    947
    Location:
    Chiang Mai, Thailand
    Doh! I re-read it and see I have to create the files and save it with the data as you state. For some reason, I thought it was similar to the malware-filter script and was going to populate those files for me.

    Many Thanks! Sending you my gratitude from the Land of Smiles.
     
  14. swetoast

    swetoast Guest

    Joined:
    Apr 12, 2016
    Messages:
    804
    yeah this one doesn't populate on its own
     
  15. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    485
    would it make sense to check for each entry in the dnsmasq.conf.add file and only append it if it doesn't exist... would save you having to delete and recreate it every time to avoid duplicates.. perhaps throw something like this in your "for i in" loop
    Code:
    [ "$(grep $i $dnsmasq_cfg)" = "" ]
    Would make it safer if some other script had written a dnsmasq.conf.add too (play nice with others policy)
     
    Last edited: Jan 14, 2017
  16. swetoast

    swetoast Guest

    Joined:
    Apr 12, 2016
    Messages:
    804
    Revision 2 posted in opening post

    Changes are proper detection if privacy filter is already loaded in dnsmasq

    So what it does now is to check if there is an add file then it checks if there is a server= present in the real dnsmasq if there isnt then it adds privacy filter to the dnsmasq file.

    Edit: seems i found a bug with grep with this revision
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=617183

    also getting this message so ill see if there is another way to do it.
     
    Last edited: Jan 26, 2017
  17. martinr

    martinr Very Senior Member

    Joined:
    Nov 27, 2014
    Messages:
    1,002
    Location:
    United Kingdom
    You seem pretty sharp to me.
     
  18. swetoast

    swetoast Guest

    Joined:
    Apr 12, 2016
    Messages:
    804
    Well i dont mind tomsk input, keeps me on my game :)
     
    Last edited: Jan 29, 2017
  19. swetoast

    swetoast Guest

    Joined:
    Apr 12, 2016
    Messages:
    804
    bump to revision 3 to get around the grep bug added a bell sign to dnsmasq with the title #privacy-filter that the script checks for then adds if its not added
     
  20. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    485
    would it be better to do the alterations to dnsmasq.conf with a .postconf script?
    not sure this would even work.. but alter the top of your script to something like this...

    Code:
    dnsmasq_cfg=/jffs/scripts/dnsmasq.postconf
    
    if [ ! -f $dnsmasq_cfg ]; then
                    for i in `cat $path/privacy-filter.list`
                    do echo "pc_append server=/$i/127.0.0.1#1919" >> $dnsmasq_cfg; done
                    service restart_dnsmasq
                    logger -s -t privacy-filter was added to $dnsmasq_cfg; fi
                   
            else    for i in `cat $path/privacy-filter.list`
            if      [ "$(grep $i $dnsmasq_cfg)" = "" ]
            then    do echo "pc_append server=/$i/127.0.0.1#1919" >> $dnsmasq_cfg; done
                    service restart_dnsmasq
                    logger -s -t privacy-filter was added to $dnsmasq_cfg
            fi
    fi
     

Share This Page