Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

Privacy Filter (Another IPSET Script)

Discussion in 'Asuswrt-Merlin' started by swetoast, Jan 11, 2017.

  1. bayern1975

    bayern1975 Senior Member

    Joined:
    Sep 22, 2015
    Messages:
    439
    i have no idea what could be wrong.....i will remove this script temporarily.....
    EDIT: strange, some hostnames not pinging, some pinging and some redirecting to computer....
    Code:
    ASUSWRT-Merlin RT-AC3200 380.65-0 Fri Feb  3 05:20:08 UTC 2017
    [email protected]:/tmp/home/root# ping -c 5 216.117.2.180
    PING 216.117.2.180 (216.117.2.180): 56 data bytes
    
    --- 216.117.2.180 ping statistics ---
    5 packets transmitted, 0 packets received, 100% packet loss
    [email protected]:/tmp/home/root# ping -c 5 statsfe2.update.microsoft.com.aka
    dns.net
    PING statsfe2.update.microsoft.com.akadns.net (65.52.108.153): 56 data bytes
    
    --- statsfe2.update.microsoft.com.akadns.net ping statistics ---
    5 packets transmitted, 0 packets received, 100% packet loss
    [email protected]:/tmp/home/root# ping -c 5 a-0003.a-msedge.net
    PING a-0003.a-msedge.net (204.79.197.203): 56 data bytes
    64 bytes from 204.79.197.203: seq=0 ttl=123 time=34.647 ms
    64 bytes from 204.79.197.203: seq=1 ttl=123 time=35.167 ms
    64 bytes from 204.79.197.203: seq=2 ttl=123 time=34.404 ms
    64 bytes from 204.79.197.203: seq=3 ttl=123 time=31.159 ms
    64 bytes from 204.79.197.203: seq=4 ttl=123 time=32.140 ms
    
    --- a-0003.a-msedge.net ping statistics ---
    5 packets transmitted, 5 packets received, 0% packet loss
    round-trip min/avg/max = 31.159/33.503/35.167 ms
    [email protected]:/tmp/home/root# ping -c 5 a-0002.a-msedge.net
    PING a-0002.a-msedge.net (204.79.197.201): 56 data bytes
    64 bytes from 204.79.197.201: seq=0 ttl=123 time=36.000 ms
    64 bytes from 204.79.197.201: seq=1 ttl=123 time=32.396 ms
    64 bytes from 204.79.197.201: seq=2 ttl=123 time=34.940 ms
    64 bytes from 204.79.197.201: seq=3 ttl=123 time=34.144 ms
    64 bytes from 204.79.197.201: seq=4 ttl=123 time=34.736 ms
    
    --- a-0002.a-msedge.net ping statistics ---
    5 packets transmitted, 5 packets received, 0% packet loss
    round-trip min/avg/max = 32.396/34.443/36.000 ms
    [email protected]:/tmp/home/root# ping -c 5 a-0004.a-msedge.net
    PING a-0004.a-msedge.net (204.79.197.206): 56 data bytes
    64 bytes from 204.79.197.206: seq=0 ttl=122 time=33.530 ms
    64 bytes from 204.79.197.206: seq=1 ttl=122 time=32.978 ms
    64 bytes from 204.79.197.206: seq=2 ttl=122 time=32.391 ms
    64 bytes from 204.79.197.206: seq=3 ttl=122 time=30.240 ms
    64 bytes from 204.79.197.206: seq=4 ttl=122 time=33.552 ms
    
    --- a-0004.a-msedge.net ping statistics ---
    5 packets transmitted, 5 packets received, 0% packet loss
    round-trip min/avg/max = 30.240/32.538/33.552 ms
    [email protected]:/tmp/home/root# ping -c 5 a.ads1.msn.com
    PING a.ads1.msn.com (0.0.0.0): 56 data bytes
    64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.178 ms
    64 bytes from 127.0.0.1: seq=1 ttl=64 time=0.104 ms
    64 bytes from 127.0.0.1: seq=2 ttl=64 time=0.125 ms
    64 bytes from 127.0.0.1: seq=3 ttl=64 time=0.123 ms
    64 bytes from 127.0.0.1: seq=4 ttl=64 time=0.109 ms
    
    --- a.ads1.msn.com ping statistics ---
    5 packets transmitted, 5 packets received, 0% packet loss
    round-trip min/avg/max = 0.104/0.127/0.178 ms
    [email protected]:/tmp/home/root# ping -c 5 choice.microsoft.com
    PING choice.microsoft.com (0.0.0.0): 56 data bytes
    64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.199 ms
    64 bytes from 127.0.0.1: seq=1 ttl=64 time=0.116 ms
    64 bytes from 127.0.0.1: seq=2 ttl=64 time=0.124 ms
    64 bytes from 127.0.0.1: seq=3 ttl=64 time=0.120 ms
    64 bytes from 127.0.0.1: seq=4 ttl=64 time=0.116 ms
    
    --- choice.microsoft.com ping statistics ---
    5 packets transmitted, 5 packets received, 0% packet loss
    round-trip min/avg/max = 0.116/0.135/0.199 ms
    [email protected]:/tmp/home/root# ping -c 5 telecommand.telemetry.microsoft.c
    om
    PING telecommand.telemetry.microsoft.com (0.0.0.0): 56 data bytes
    64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.160 ms
    64 bytes from 127.0.0.1: seq=1 ttl=64 time=0.113 ms
    64 bytes from 127.0.0.1: seq=2 ttl=64 time=0.111 ms
    64 bytes from 127.0.0.1: seq=3 ttl=64 time=0.126 ms
    64 bytes from 127.0.0.1: seq=4 ttl=64 time=0.120 ms
    
    --- telecommand.telemetry.microsoft.com ping statistics ---
    5 packets transmitted, 5 packets received, 0% packet loss
    round-trip min/avg/max = 0.111/0.126/0.160 ms
    these hostnames allways can ping.....
    Code:
    a-0002.a-msedge.net
    a-0003.a-msedge.net
    a-0004.a-msedge.net
    a-0005.a-msedge.net
    a-0006.a-msedge.net
    a-0007.a-msedge.net
    a-0008.a-msedge.net
    a-0009.a-msedge.net
     
    Last edited: Feb 11, 2017
  2. swetoast

    swetoast Very Senior Member

    Joined:
    Apr 12, 2016
    Messages:
    527
    said it several times over check iptables instead pinging often responds since there can be redirects etc. @bayern1975 if your so insecure over stuff maybe its not a good idea to run advanced scripts on your router maybe begin with learning about linux in the first place.
     
    Last edited: Feb 12, 2017
  3. bayern1975

    bayern1975 Senior Member

    Joined:
    Sep 22, 2015
    Messages:
    439
    @swetoast, I didn't say nothing bad over you, scripts or other authors....but I can' t understand why most hostnames can pinging if they should be blocked....
     
  4. swetoast

    swetoast Very Senior Member

    Joined:
    Apr 12, 2016
    Messages:
    527
    where is what i dont like i said it over and over again "did you check using iptables and see if packet count went up after your test" my guess is no. You probably just pinged it it responded and there for you claim its not working.

    Code:
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination         
     3215  152K REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set privacy-filter_ipv4 src,dst reject-with icmp-port-unreachable
    spoiler it works.
     
  5. swetoast

    swetoast Very Senior Member

    Joined:
    Apr 12, 2016
    Messages:
    527
    Bump to revision 10

    Changelog:
    • Path fixes by Tomsk (huge tnx)
     
  6. Cedarhillguy

    Cedarhillguy New Around Here

    Joined:
    Jan 15, 2017
    Messages:
    7
    Version 10 fails for me with an error that "/opt/bin/xargs : not found". Entware isn't installed on my router .

    It appears, in this ipv4_block line of code, that it checks if "/opt/bin/xargs" exists and if it doesn't (-z option) it then it attempts on next line to run from that non-existent path.

    Code:
            if [ -z "$(which /opt/bin/xargs)" ]
                then cat $path/privacy-filter.list | /opt/bin/xargs -n 5 -I {} sh -c "traceroute -4 {} | head -1 >> "$path/privacy-filter_ipv4.tmplist1""
    
    Suggest changing the second line to:

    Code:
                then cat $path/privacy-filter.list | xargs -n 5 -I {} sh -c "traceroute -4 {} | head -1 >> "$path/privacy-filter_ipv4.tmplist1""
    
     
  7. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    471
    Oops you're right haha.... actually you only have to test the xargs path in the if [ -z "$(which /opt/bin/xargs)" ] bit. If entware is running it will point the xargs call to the right version anyway. You can replace /opt/bin/xargs with a straight forward xargs elsewhere in the script. The purpose of the test is just to remove the -P10 switch for the busybox (router) version as it doesn't work. Try xargs --version to see.
     
    Last edited: Feb 12, 2017
  8. swetoast

    swetoast Very Senior Member

    Joined:
    Apr 12, 2016
    Messages:
    527
    fixed on wiki and op
     
  9. PeterR

    PeterR New Around Here

    Joined:
    May 29, 2013
    Messages:
    3
    Is there anyone else who finds Skype is blocked when the filters are active?
     
  10. swetoast

    swetoast Very Senior Member

    Joined:
    Apr 12, 2016
    Messages:
    527
    cause skype uses the same domains as in the list perhaps ? so here is how to resolve that if you want that app

    find out which domain it is, is another issue so lets dig it out and remove it from the blocklist
     
  11. swetoast

    swetoast Very Senior Member

    Joined:
    Apr 12, 2016
    Messages:
    527
    PeterR likes this.
  12. bayern1975

    bayern1975 Senior Member

    Joined:
    Sep 22, 2015
    Messages:
    439
    30 hours router online and check over putty my privacy and still zeroes?
    Code:
    ASUSWRT-Merlin RT-AC3200 380.65-0 Fri Feb  3 05:20:08 UTC 2017
    [email protected]:/tmp/home/root# iptables -L FORWARD -v
    
    
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination         
        0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set privacy-filter_ipv4 src,dst reject-with icmp-port-unreachable
     
  13. swetoast

    swetoast Very Senior Member

    Joined:
    Apr 12, 2016
    Messages:
    527
    if the traffic isnt there then it isnt there the rule is certainly there but i cant fix YOUR router and i dont know HOW you have set it up its working for 99% of the other people its always YOU that have the issues if we start looking at my other threads your the common denominator..

    im simply putting it blunt, tired of helping you. The only advice i can give you is learn linux and learn how stuff works so that you know whats wrong.. and how to setup things proper.

    Code:
    13158  623K REJECT     all  --  any    any     anywhere             anywhere             match-set privacy-filter_ipv4 src,dst reject-with icmp-port-unreachable
     
  14. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    471
    Check that the ipset is created and populated with IP addresses
    Code:
    ipset -L privacy-filter_ipv4
    or
    ipset -L privacy-filter_ipv6
     
  15. bayern1975

    bayern1975 Senior Member

    Joined:
    Sep 22, 2015
    Messages:
    439
    i got his when put this in terminal.....
    Code:
    ASUSWRT-Merlin RT-AC3200 380.65-0 Fri Feb  3 05:20:08 UTC 2017
    [email protected]:/tmp/home/root# ipset -L privacy-filter_ipv4
    Name: privacy-filter_ipv4
    Type: hash:ip
    Revision: 0
    Header: family inet hashsize 1024 maxelem 65536
    Size in memory: 8264
    References: 1
    Members:
    104.131.0.69
    [email protected]:/tmp/home/root# ipset -L privacy-filter_ipv6
    ipset v6.29: The set with the given name does not exist
     
  16. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    471
    There is only one IP in the hash set.... thats why all your probes are getting past the iptables rule.
    You must have some other filter which is preventing the traceroute from resolving the IP for the other domains.
     
  17. bayern1975

    bayern1975 Senior Member

    Joined:
    Sep 22, 2015
    Messages:
    439
    i have just this ipset privacy script and AB-Solution script....i tested without AB-Solution but is same results.....
     
  18. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    471
    Here is mine...see the difference. And that is not a full set either as my AB-solution host file is blocking some of them.
    Code:
    [email protected]:/tmp/home/root# ipset -L privacy-filter_ipv4
    Name: privacy-filter_ipv4
    Type: hash:ip
    Revision: 0
    Header: family inet hashsize 1024 maxelem 65536
    Size in memory: 9000
    References: 1
    Members:
    82.221.105.7
    204.79.197.210
    71.6.158.166
    82.221.105.6
    134.170.115.60
    64.4.54.22
    134.170.188.248
    104.131.0.69
    23.36.69.246
    216.117.2.180
    131.253.14.76
    204.79.197.208
    188.138.9.50
    198.20.70.114
    157.56.96.58
    65.55.130.50
    207.68.166.254
    204.79.197.209
    184.25.204.97
    71.6.135.131
    198.20.99.130
    93.184.215.201
    104.16.51.93
    204.79.197.203
    195.22.26.248
    198.20.69.74
    157.55.129.21
    157.58.249.57
    204.79.197.206
    204.79.197.211
    23.38.232.12
    204.79.197.201
    66.240.192.138
    93.120.27.62
    198.20.69.98
    71.6.167.142
    65.52.108.74
    85.25.43.94
    71.6.165.200
    66.240.236.119
    209.126.110.38
    204.79.197.204
    134.170.58.125
    85.25.103.50
    114.80.68.223
    204.79.197.200
    157.56.57.5
     
  19. tomsk

    tomsk Senior Member

    Joined:
    Sep 3, 2016
    Messages:
    471
    If you select the Large filter with AB-solution it blocks A LOT.... maybe all those IPs even.
    You can turn AB off from the UI using the [a] option... then run the privacy filter again...it will rebuild the ipset...then test how many IP it contains.
     
  20. bayern1975

    bayern1975 Senior Member

    Joined:
    Sep 22, 2015
    Messages:
    439
    tested without ab-solution...still get just one IP in blocked list when I check with ipset -L privacy-filter_ipv4....I realy can't find where and what is wrong in my router.....
     

Share This Page