Unbound and Pixelserv-Tls

[Deleted member 62525]

Experimenting redirecting Unbound adBlocking to Pixelserv-tls. I have installed and configured Pixelserv-tls, generated certificates and make it startup on boot. Verified it is working in log and running servstats in the browser. This part is done.

Can anyone suggest what changes I have to make to Unbound.conf and Unbound adblocker to finish it?
I have manually (just for testing purposes replace always_nxdomain string with pixelserv IP address in /opt/var/lib/unbound/adblock/adservers file. When I restarted Unbound it is complaining with some errors about this change. I assume I have to make some adjustments to unbound.conf.

FYI - This is a Pixelserv-Tls install only, no Diversion.

 

[dave14305]

You can't just replace always_nxdomain with an IP address. It's a different format altogether to redirect to an IP. I would recommend testing with just local-data statements in the include file.

local-data: "-sso.anbtr.com. 0 A 192.168.1.2"
local-data: "0-07.ru. 0 A 192.168.1.2"
local-data: "0-day.us. 0 A 192.168.1.2"
local-data: "0.01.2.13.3.sydneypropertyinvestors.com. 0 A 192.168.1.2"
local-data: "0.01.2.23.3.sydneypropertyinvestors.com. 0 A 192.168.1.2"

Or the long way (with local-zone plus local-data) is documented here: https://deadc0de.re/articles/unbound-blocking-ads.html

 

[Deleted member 62525]

You can't just replace always_nxdomain with an IP address. It's a different format altogether to redirect to an IP. I would recommend testing with just local-data statements in the include file.

local-data: "-sso.anbtr.com. 0 A 192.168.1.2"
local-data: "0-07.ru. 0 A 192.168.1.2"
local-data: "0-day.us. 0 A 192.168.1.2"
local-data: "0.01.2.13.3.sydneypropertyinvestors.com. 0 A 192.168.1.2"
local-data: "0.01.2.23.3.sydneypropertyinvestors.com. 0 A 192.168.1.2"

Or the long way (with local-zone plus local-data) is documented here: https://deadc0de.re/articles/unbound-blocking-ads.html

Thanks @dave14305. I found this nifty awk code that sort of works for me. Same link you provided. I am not an awk expert so I am struggling a little bit with this.

cat adservers | grep 'always_nxdomain' | awk '{print "local-zone: "$2" redirect\nlocal-data: "$2" A 192.168.50.190"}' > ads.conf

The output is little off since I need to move double quotes (") under local-data and put it at the end of the line. Can you help and suggest the fix?

local-zone: "007angels.com" redirect
local-data: "007angels.com" A 192.168.50.190
local-zone: "008.free-counter.co.uk" redirect
local-data: "008.free-counter.co.uk" A 192.168.50.190
local-zone: "008.free-counters.co.uk" redirect

 

[dave14305]

cat adservers | grep 'always_nxdomain' | awk '{print "local-zone: "$2" redirect\nlocal-data: \""$2" A 192.168.50.190\""}' > ads.conf

I would consider adding a zero before the A to enforce a 0 second time-to-live, similar to how dnsmasq handles local hosts. Makes whitelisting easier if you don't have to wait for TTL to expire.

 

[Deleted member 62525]

Yes, modified the command and added TTL. Final code below. Created cru line to invoke the code 15 minutes after unbound adblocker executed.
This is not the cleanest code for my my experimenting over next few days it should work. I did not want to modify existing Unbound code so this basically convert Unbounbd adservers file to a compliant file to be used by Pixelserv. Have fun anyone that want to try it.

#!/bin/sh

SDIR="/opt/var/lib/unbound/adblock/"
PIXELIP="192.168.1.100"

logger -s -t "($(basename $0))" $$ "Converting nxdomain to Pixelserv format."
cat $SDIR/adservers | grep 'always_nxdomain' | tr -d \" | awk '{print "local-data: \""$2" 0 A $PIXELIP \""}' > $SDIR/ads.conf
cp $SDIR/ads.conf $SDIR/adservers
logger -s -t "($(basename $0))" $$ "Unbound adblocker adservers file has been updated."
unbound-control reload

 

[Martineau]

Yes, modified the command and added TTL. Final code below. Created cru line to invoke the code 15 minutes after unbound adblocker executed.
This is not the cleanest code for my my experimenting over next few days it should work. I did not want to modify existing Unbound code so this basically convert Unbounbd adservers file to a compliant file to be used by Pixelserv. Have fun anyone that want to try it.

#!/bin/sh

SDIR="/opt/var/lib/unbound/adblock/"
PIXELIP="192.168.1.100"

logger -s -t "($(basename $0))" $$ "Converting nxdomain to Pixelserv format."
cat $SDIR/adservers | grep 'always_nxdomain' | tr -d \" | awk '{print "local-data: \""$2" 0 A $PIXELIP \""}' > $SDIR/ads.conf
cp $SDIR/ads.conf $SDIR/adservers
logger -s -t "($(basename $0))" $$ "Unbound adblocker adservers file has been updated."
unbound-control reload

IP=192.168.1.2; awk -F'"' -v pixelservip=${IP} '{print "local-data: \""$2" 0 A "pixelservip"\""}'  /opt/var/lib/unbound/adblock/adservers > /opt/var/lib/unbound/adblock/pixelserv

cp /opt/var/lib/unbound/adblock/pixelserv /opt/var/lib/unbound/adblock/adservers;unbound-control reload

FYI,

1. You have neglected to inform users that they should also remove the daily AD Block cron job, otherwise the pixelserv-tls format will be reverted.

2. Furthermore, unless you expect novice users to also have Diversion running concurrently, you should post your custom version of /init.d/S80pixelserv-tls ?

i.e. you can't use the /init.d/S80pixelserv-tls script installed by Diversion.
(Despite politely asking that it is modified to accommodate both Diversion and unbound it was ignored.)

 

[juched]

IP=192.168.1.2; awk -F'"' -v pixelservip=${IP} '{print "local-data: \""$2" 0 A "pixelservip"\""}'  /opt/var/lib/unbound/adblock/adservers > /opt/var/lib/unbound/adblock/pixelserv

cp /opt/var/lib/unbound/adblock/pixelserv /opt/var/lib/unbound/adblock/adservers;unbound-control reload

FYI,

1. You have neglected to inform users that they should also remove the daily AD Block cron job, otherwise the pixelserv-tls format will be reverted.

2. Furthermore, unless you expect novice users to also have Diversion running concurrently, you should post your custom version of /init.d/S80pixelserv-tls ?

i.e. you can't use the /init.d/S80pixelserv-tls script installed by Diversion.
(Despite politely asking that it is modified to accommodate both Diversion and unbound it was ignored.)

They added a cru command to run 15 minutes after, so it daily converts back.

This could be supported, but I don't see the advantage yet. I just switched back for the last 2 days from diversion/pixelserv to unbound with nx_domain, and I don't notice much of a difference. Even Kvic's testing showed nx_domain acting about the same for performance.

Interested to hear from others.

 

[Deleted member 62525]

IP=192.168.1.2; awk -F'"' -v pixelservip=${IP} '{print "local-data: \""$2" 0 A "pixelservip"\""}'  /opt/var/lib/unbound/adblock/adservers > /opt/var/lib/unbound/adblock/pixelserv

cp /opt/var/lib/unbound/adblock/pixelserv /opt/var/lib/unbound/adblock/adservers;unbound-control reload

FYI,

1. You have neglected to inform users that they should also remove the daily AD Block cron job, otherwise the pixelserv-tls format will be reverted.

2. Furthermore, unless you expect novice users to also have Diversion running concurrently, you should post your custom version of /init.d/S80pixelserv-tls ?

i.e. you can't use the /init.d/S80pixelserv-tls script installed by Diversion.
(Despite politely asking that it is modified to accommodate both Diversion and unbound it was ignored.)

This is not by all means a code I would recommend to be ready and go. Its only experimental and assuming that user has installed pixelserv-tls and configured it. I am staying home as many of us do these days working remote so in a spare time decided to test it and see how it performs vs pure Unbound. So far after few hours I don't really see a major differences at least with my browsing habits. Performance wise it on par with Unbound, hard to tell. Maybe a little faster with only Unbound, in my opinion.

Observations and Results:

I started with a simple setup. Unbound installed and running, no Diversion installed. Installed Pixelserv-TLS package from entware and configured it including creating proper certificate. Then, I converted Unbound adblocker adservers file to redirect to the Pixelserv. Rebooted the router, confirmed in the log that pixelserv started and running. During my testing observing servstats page to make sure things are blocking and count is increasing.

After few hours of testing and some 50 or more random and common pages to invoke, I have to say that I will be staying with Unbound blocking. I found performance is little faster with Unbound blocking and having only Unbound installed the maintenance is simpler and easier to do. As for aesthetics where ads are replaced by a single pixel with Pixelserv-tls vs Unbound leaving a larger area, this is for discussion. Some web pages even with Pixelserv leave large areas blank and in all honesty that does not bother me regardless if its Unbound or Pixelserv.

 

[SomeWhereOverTheRainBow]

This is not by all means a code I would recommend to be ready and go. Its only experimental and assuming that user has installed pixelserv-tls and configured it. I am staying home as many of us do these days working remote so in a spare time decided to test it and see how it performs vs pure Unbound. So far after few hours I don't really see a major differences at least with my browsing habits. Performance wise it on par with Unbound, hard to tell. Maybe a little faster with only Unbound, in my opinion.

Observations and Results:

I started with a simple setup. Unbound installed and running, no Diversion installed. Installed Pixelserv-TLS package from entware and configured it including creating proper certificate. Then, I converted Unbound adblocker adservers file to redirect to the Pixelserv. Rebooted the router, confirmed in the log that pixelserv started and running. During my testing observing servstats page to make sure things are blocking and count is increasing.

After few hours of testing and some 50 or more random and common pages to invoke, I have to say that I will be staying with Unbound blocking. I found performance is little faster with Unbound blocking and having only Unbound installed the maintenance is simpler and easier to do. As for aesthetics where ads are replaced by a single pixel with Pixelserv-tls vs Unbound leaving a larger area, this is for discussion. Some web pages even with Pixelserv leave large areas blank and in all honesty that does not bother me regardless if its Unbound or Pixelserv.

Note: I will say unbound does a decent job for blocking a small amount of ads ( or several smaller split up list for those blocking more than 50,000 ads.), but it is not generous with the amount of memory it needs to do such.
I have found DNSMASQ+Pixelserv-tls (diversion) provides the overall best performance for adblocking, and followed by a close second is pihole or adguard home. Unbound is great in terms of what it offers with cache, recursion, security, and privacy options. It is a memory guzzler as far as adblocking is concerned, but can do a decent job for a smaller load adblocking.

 

[Deleted member 62525]

Note: I will say unbound does a decent job for blocking a small amount of ads ( or several smaller split up list for those blocking more than 50,000 ads.), but it is not generous with the amount of memory it needs to do such.
I have found DNSMASQ+Pixelserv-tls (diversion) provides the overall best performance for adblocking, and followed by a close second is pihole or adguard home. Unbound is great in terms of what it offers with cache, recursion, security, and privacy options. It is a memory guzzler as far as adblocking is concerned, but can do a decent job for a smaller load adblocking.

Agree with you to some extend. Sure my test was not a technical enough skipping things such as measuring memory consumption or CPU etc. Over past 2 year I was running Diversion and Pixel before running Unbound. On my small home network and for average user Unbound will be more than enough. There are just many variables to consider apart from memory, CPU, and other processes you are running on your router. Your connection speed, number of users and browsing habits. If you use VPN like I do for example VPN provider does some ads blocking already. In my case at least and limited users, using RT86U router, and limited apps, Unbound works really good, CPU very low, mem is at 72% and steady.

 

[juched]

Agree with you to some extend. Sure my test was not a technical enough skipping things such as measuring memory consumption or CPU etc. Over past 2 year I was running Diversion and Pixel before running Unbound. On my small home network and for average user Unbound will be more than enough. There are just many variables to consider apart from memory, CPU, and other processes you are running on your router. Your connection speed, number of users and browsing habits. If you use VPN like I do for example VPN provider does some ads blocking already. In my case at least and limited users, using RT86U router, and limited apps, Unbound works really good, CPU very low, mem is at 72% and steady.

I do agree that the memory is much more steady using just unbound for as blocking, not that memory consumption on Linux is bad, it uses memory fully to cache so not a problem. But with diversion and pixelserv-tls (I suspect more the pixelserv part) it generates activity on my system which does cause that cache memory use to build up. Unbound alone does not.