Privacy Filter (Another IPSET Script)

[Xentrk]

I updated to the WIP version 20 and rebooted the router. The following is from log file:

Apr 20 18:56:52 system: Privacy Filter (ipv4) loaded 0 unique ip addresses that will be rejected from contacting your router.
Apr 20 18:56:52 system: Privacy Filter (ipv4) loaded 0 unique ip addresses that will be rejected from contacting your router.
Apr 20 18:56:52 system: Privacy Filter (ipv4) loaded 0 unique ip addresses that will be rejected from contacting your router.

I reverted back to version 19 and rebooted the router:

Apr 20 19:16:43 system: Privacy Filter (ipv4) loaded 51 unique ip addresses that will be rejected from contacting your router.
Apr 20 19:16:43 system: Privacy Filter (ipv4) loaded 51 unique ip addresses that will be rejected from contacting your router.
Apr 20 19:16:43 system: Privacy Filter (ipv4) loaded 51 unique ip addresses that will be rejected from contacting your router.

Thinking WIP version reporting loading of 0 ip addresses was an error, I updated privacy-filter to version 20 and rebooted:

Apr 20 20:16:43 system: Privacy Filter (ipv4) loaded 0 unique ip addresses that will be rejected from contacting your router.
Apr 20 20:16:43 system: Privacy Filter (ipv4) loaded 0 unique ip addresses that will be rejected from contacting your router.
Apr 20 20:16:43 system: Privacy Filter (ipv4) loaded 0 unique ip addresses that will be rejected from contacting your router.

Seeing that nothing got loaded, I ran version 19 from the command line:

admin@RT-AC88U:/jffs/scripts# ./privacy-filter
nslookup: can't resolve 's.gateway.messenger.live.com'
system: Privacy Filter (ipv4) loaded 59 unique ip addresses that will be rejected from contacting your router.

 

[swetoast]

mind just running this part

#!/bin/sh
cat /jffs/privacy-filter.list | \
xargs -n 5 -I {} sh -c "nslookup {}" | \
grep -i "Address" | \
awk '{print $3}'

really interested in the content of the printout

 

[FalconB]

Hi everyone, first post here!

I'm currently running both "AB-Solution" and "Privacy Filter" on my router. Both work fine, great work and a big thanks to all who contributed! However, I have a question, that may show my lack of understanding about all of this, but still:

• "AB-Solution" blocks INCOMING ads from beeing sent FROM Internet TO all devices behind my router
• "Privacy Filter" blocks OUTGOING telemetry data from beeing sent FROM devices behind my router TO the Internet

So far so good. Now, when "Privacy Filter" updates the list of blocked sites/IP-addresses, some addresses will be put in the priavcy filter list, but some sites/addresses will not, due to "AB-Solution" blocking them. So, if "AB-Solution" is blocking INCOMING traffic and "Privacy Filter" blocks OUTGOING traffic, doesn't that mean that:

1. Telemetry-sites will not be added to the privacy filter list because the INCOMING traffic (in respons to nslookup from "Privacy Filter") will be blocked by "AB-Solution"
2. Telemetry-data will still be able to be sent OUT of the home-network since the receiving sites are not in the privacy filter list

...or am I missing something?

The number of sites blocked by "Privacy Filter" just about doubles when I run the script with "AB-Solution" off, which concurs with (1) above.

The solution would be to disable "AB-Solution" while updating "Privacy Filter".



@swetoast: Bra skit du har knåpat ihop ;-)

 

[swetoast]

Will link your lovely tutorial on the wiki as a troubleshooter, tackar så mycket

 

[Xentrk]

mind just running this part

#!/bin/sh
cat /jffs/privacy-filter.list | \
xargs -n 5 -I {} sh -c "nslookup {}" | \
grep -i "Address" | \
awk '{print $3}'

really interested in the content of the printout

Sent you the output in a PM as there are output that should not be seen by the general public

 

[thelonelycoder]

Hi everyone, first post here!

I'm currently running both "AB-Solution" and "Privacy Filter" on my router. Both work fine, great work and a big thanks to all who contributed! However, I have a question, that may show my lack of understanding about all of this, but still:

• "AB-Solution" blocks INCOMING ads from beeing sent FROM Internet TO all devices behind my router
• "Privacy Filter" blocks OUTGOING telemetry data from beeing sent FROM devices behind my router TO the Internet

So far so good. Now, when "Privacy Filter" updates the list of blocked sites/IP-addresses, some addresses will be put in the priavcy filter list, but some sites/addresses will not, due to "AB-Solution" blocking them. So, if "AB-Solution" is blocking INCOMING traffic and "Privacy Filter" blocks OUTGOING traffic, doesn't that mean that:

1. Telemetry-sites will not be added to the privacy filter list because the INCOMING traffic (in respons to nslookup from "Privacy Filter") will be blocked by "AB-Solution"
2. Telemetry-data will still be able to be sent OUT of the home-network since the receiving sites are not in the privacy filter list

...or am I missing something?

The number of sites blocked by "Privacy Filter" just about doubles when I run the script with "AB-Solution" off, which concurs with (1) above.

The solution would be to disable "AB-Solution" while updating "Privacy Filter".



@swetoast: Bra skit du har knåpat ihop ;-)

AB-Solution does not block incoming traffic.
If it were so, you would not need the privacy filter as I could simply use the same filter lists and put @swetoast's great work out of business...

AB simply tells dnsmasq to direct queries for blocked domains FROM clients to an internal IP address (0.0.0.0 or the pixelserv-tls IP) instead of the real domain's IP address.
So, if any of the domains in the privacy filter also happen to be in the blocking file or blacklist of AB, then these domains are already blocked from reaching the real server.
Preventing them from sending telemetry data to their greedy servers.
The Privacy filter and AB complement each other, some overlap is expected but it has no effect on the overall outcome.
In cases where they do overlap, your devices are still prevented from sending telemetry data and you will not see ads.

 

[swetoast]

i know how it works and how it redirects traffic and thats why the filter is getting less hits, its already covered.

https://www.snbforums.com/threads/privacy-filter-another-ipset-script.36801/page-14#post-319171

 

[thelonelycoder]

i know how it works and how it redirects traffic and thats why the filter is getting less hits, its already covered thank you

https://www.snbforums.com/threads/privacy-filter-another-ipset-script.36801/page-14#post-319171

I did not reply to you but the user so they understand it.
It would be good you use the reply or quote functions so people know what your post is referring to.

 

[swetoast]

@Xentrk Looked over your output and much is "Redirected" by AB-Solutions and the output looks correct so i really dont see a flaw just lets hits due to so many being redirected.

 

[thelonelycoder]

@Xentrk Looked over your output and much is "Redirected" by AB-Solutions and the output looks correct so i really dont see a flaw just lets hits due to so many being redirected.

I can see a problem in the mentioned scenario by @FalconB in that it prevents from populating your privacy filters with additional blocking IP's if AB blocks the domain from being resolved.
Say, in the case of Microsoft telemetry IP/domains. But it would be the same for your ad-blocker script. Do you think it is worthwile to look into that for the benefit of your Privacy Filter?

 

[swetoast]

My blocker doesnt block telemetry

Apr 21 00:01:25 system: Privacy Filter (ipv4) loaded 118 unique ip addresses that will be rejected from contacting your router.

made sure of that since i want to keep the two separated.

 

[thelonelycoder]

My blocker doesnt block telemetry

OK then.

 

[swetoast]

@thelonelycoder But its easily solved, we both acknowledge that if AB-Solutions and Privacy filter is installed my filter gets lets hits due to some domains being redirected, if we put this in FAQ or something the users atleast knows why it happens then i can link falconb post on the wiki as a howto

 

[Xentrk]

@Xentrk Looked over your output and much is "Redirected" by AB-Solutions and the output looks correct so i really dont see a flaw just lets hits due to so many being redirected.

Many thanks @swetoast. Thanks for taking the time to look into it.

 

[swetoast]

np Xentrk glad to do it

 

[swetoast]

https://github.com/RMerl/asuswrt-merlin/wiki/Ipset-script-installation-instructions#privacy-filter

added updated info regarding AB-Solution and Privacy-Filter

 

[thelonelycoder]

https://github.com/RMerl/asuswrt-merlin/wiki/Ipset-script-installation-instructions#privacy-filter

added updated info regarding AB-Solution and Privacy-Filter

I really don't like you link to that post as the assumptions are wrong what AB or any ad-blocker such as yours does. This does not help the user.
I believe it's better you just write the solution directly into the wiki:
Note: For users using an ad-blocking script such as uBlockr or AB-Solution, you may have to disable ad-blocking while updating the privacy rules.

 

[swetoast]

ublockr doesnt block telemetry still, but i can modify so it says that users can disable your adblocker while updating privacy-filter

 

[swetoast]

Updated wiki again

 

[thelonelycoder]

These are just domains that either have been deactivated or that AB-Solutions or uBlockr is redirecting traffic towards its own domain this will make the number of blocked domains smaller in some cases this is perfectly normal.

Minor correction: It's AB-Solution, no s at the end.
Thanks.