AB-Solution - The Ad Blocking Solution

[IsaacFL]

Just a question out of curiosity. Ab-Solution is working great for me.

Does the pixelserv work with ipv6 addresses?

I know that Ab-Solution gives the :: address but not sure how pixelserv works.

I notice that when I do the "2. filtered by blocked domains" on the follow log, it only seems to only be looking for the ipv4 address of the pixelserv and during setup, there is no questions about ipv6 address.

Could an ad site use an ipv6 address to get around pixelserv?

 

[thelonelycoder]

Does the pixelserv work with ipv6 addresses?
I know that Ab-Solution gives the :: address but not sure how pixelserv works.

No, pixelserv only works with IPv4 addresses, meaning only those go through pixelserv-tls.
Blocked IPv6 addresses get blocked the traditional way by sending them to the IPv6 null address which is ::.
:: is the IPv6 equivalent to the IPv4 0.0.0.0 address, both are non-routing addresses.

I notice that when I do the "2. filtered by blocked domains" on the follow log, it only seems to only be looking for the ipv4 address of the pixelserv and during setup, there is no questions about ipv6 address.

This is a coding error, carried over from earlier versions of AB-Solution. Thanks for catching that one
It should have been changed to the same strings I look for when counting ads or compile the stats. It will be included in the upcoming v3.9.2.

The IPv6 support is auto-enabled in AB-Solution, there is no interaction necessary. If you disable/enable it, the blocking file gets auto-converted to either include or exclude it next time the blocking file is updated or you start the AB UI.

Could an ad site use an ipv6 address to get around pixelserv?

As IPv6 addresses don't go through pixelserv it does indeed go around it, but they are "dead-ended" in :: and are blocked that way.

 

[Andy1932]

I just set up my RT-AC86U. I'm getting these errors when starting and exiting AB Solution:

exiting AB-Solution
 checking installation state
ls: /tmp/var/spool/cron/crontabs: No such file or directory
grep: /tmp/var/spool/cron/crontabs/admin: No such file or directory
ls: /tmp/var/spool/cron/crontabs: No such file or directory
grep: /tmp/var/spool/cron/crontabs/admin: No such file or directory
grep: /tmp/var/spool/cron/crontabs/admin: No such file or directory
grep: /tmp/var/spool/cron/crontabs/admin: No such file or directory
 all good, shutting down UI

 

[hvluu]

Just installed newest version (3.9.1) and this is the error I've got

checking AB-Solution installation


checking /jffs/scripts/dnsmasq.postconf


jffs file check, passed:

/jffs/scripts/dnsmasq.postconf

file found

content looks good


checking /jffs/scripts/post-mount


jffs file check, passed:

/jffs/scripts/post-mount

file found

content looks good


checking ab_dnsmasq_postconf.sh

file found

content looks good


checking white and black list

files found, no further checks


checking blocking_file

- 100%[===================>] 239.48K 242KB/s in 1.0s

- 100%[===================>] 5.73M 1.81MB/s in 3.3s

- 100%[===================>] 22.03K 87.2KB/s in 0.3s

- 100%[===================>] 170.08K 314KB/s in 0.5s

- 100%[===================>] 332.79K 518KB/s in 0.6s

- [ <=> ] 1.30M 427KB/s in 3.1s

sort: out of memory

cat: can't open '/tmp/mnt/New_Volume/adblocking/blocking_file': No such file or directory

blocking_file file not found

grep: /tmp/mnt/New_Volume/adblocking/blocking_file: No such file or directory


checking ab-solution.cfg

file found

content looks good


checking update-hosts.add

file found

content looks good


checking rotate-logs.add

file found

content looks good


checking cron job blocking file update

ls: /tmp/var/spool/cron/crontabs: No such file or directory

grep: /tmp/var/spool/cron/crontabs/hungluu: No such file or directory


checking cron job log rotate

ls: /tmp/var/spool/cron/crontabs: No such file or directory

grep: /tmp/var/spool/cron/crontabs/hungluu: No such file or directory


checking symlink state

file found

content looks good


checking shell history state

file found

content looks good


checking /etc/dnsmasq.conf

file found

adblocking state looks good

logging state looks good


Errors found.

Read the check's output for clues.


Hit [Enter] to return to Menu

Edit: forgot to mention I'm running 382.1 beta 3 on AC86U

 

[thelonelycoder]

@Andy1932 and @hvluu the cron file errors will be fixed in the upcoming AB-Solution 3.9.2

Just installed newest version (3.9.1) and this is the error I've got

- 100%[===================>] 332.79K 518KB/s in 0.6s

- [ <=> ] 1.30M 427KB/s in 3.1s

sort: out of memory

cat: can't open '/tmp/mnt/New_Volume/adblocking/blocking_file': No such file or directory

Your router runs out of RAM.
Select a smaller blocking file or investigate if other processes use more memory than usual.

 

[.TT.]

@thelonelycoder Anyway to disable ipv6 blocking in ab-solution? (hidden option?)
I only use ipv6 for vpn and its not filtered anyway.

 

[thelonelycoder]

@thelonelycoder Anyway to disable ipv6 blocking in ab-solution? (hidden option?)
I only use ipv6 for vpn and its not filtered anyway.

I'm not planning to do that.
But as a workaround you can manually edit ab-solution.sh to fool AB to think IPv6 is disabled.

Exit the AB UI to do this.
Around line 470 in ab-solution.sh edit this from:

ipv6_service=$(nvram get ipv6_service)

to hard-coded disabled:

ipv6_service=disabled

This will disable IPv6 within (and only in) AB-Solution completely.
Then start the AB UI, the blocking file will be converted during startup, a message is shown that it does so.
Note that you'd have to edit that line every time ab-solution.sh is re-downloaded from the server.

 

[.TT.]

I'm not planning to do that.
But as a workaround you can manually edit ab-solution.sh to fool AB to think IPv6 is disabled.
Exit the AB UI to do this.
Around line 470 in ab-solution.sh edit this from:

ipv6_service=$(nvram get ipv6_service)

to hard-coded disabled:

ipv6_service=disabled

This will disable IPv6 withing (and only in) AB-Solution completely.
Then start the AB UI, the blocking file will be converted during startup, a message is shown that it does so.
Note that you'd have to edit that line every time ab-solution.sh is re-downloaded from the server.

Thanks!

 

[thelonelycoder]

AB-Solution 3.9.2 is now available

Use cu to update to this latest version.

Changelog v3.9.2:
All files: Bump version number to 3.9.2(.x)

Compatibility changes for Asuswrt-Merlin 382.x, bug-fixes for both 380.x and 382.x:
- rc: Correct AiProtection state detection
- f: Blocked IPv6 domains are now shown
- cj, AB-Solution UI start: Crontab file error fix
- ps: Additional pixelserv IP address checks when installing
- ps: Logging setting is now moved to 'Add or remove pixelserv switches' (necessary for the upcoming pixelserv-tls version Kl)
- shared-AB-whitelist: File is now auto-generated during AB-Solution UI start if missing

 

[elorimer]

Two suggestions for TLC to consider:
1. As part of the entware install, offer creation of a swap file. I know this duplicates Adamm's script.
2. As part of adding Whitelist entries, offer the ability to insert a #comment. Some of the things I've added along the way are kind of obscure--there are two entries that you have to make to play Starz content, for example.

So this:

#The following is necessary to play Starz
api.mixpanel.com

Is better than this:

api.mixpanel.com

I know I will forget in two weeks.

 

[jrmwvu04]

1. As part of the entware install, offer creation of a swap file

Unless something changes (which is quite possible), you might want to have this going forward for pixelserv. Version Kl has been running the system low on memory when it gets hit hard, and that’s with a relatively conservative block list.

 

[elorimer]

Unless something changes (which is quite possible), you might want to have this going forward for pixelserv. Version Kl has been running the system low on memory when it gets hit hard, and that’s with a relatively conservative block list.

I know you've had problems with memory and Kl and needed a swapfile, but I'm not remotely close to having a problem on a 56U:

             total       used       free     shared    buffers     cached
Mem:        255708     125756     129952          0       1216      21792
-/+ buffers/cache:     102748     152960
Swap:       261620          0     261620

 

[jrmwvu04]

but I'm not remotely close to having a problem on a 56U:

Mine looks that was as well 99.9% of the time. The issue arises in fringe cases where pixelserv gets hit hard at once. At any rate, I believe that @kvic is considering some tweaking to be done to Kl before it goes final to address this. Which is why I say, “unless something changes”. The point is, I agree with your suggestion about an option to configure swap for AB.

 

[abescalamis]

My RT-AC88U shows to use 45% of the memory all the time, is that normal? I have AB-solution with pixelserv-tls up and running and using the large host file list

 

[thelonelycoder]

My RT-AC88U shows to use 45% of the memory all the time, is that normal? I have AB-solution with pixelserv-tls up and running and using the large host file list

45% looks normal with the large blocking file. Dnsmasq loads the blocking file and the blacklist into memory.

 

[thelonelycoder]

Two suggestions for TLC to consider:
1. As part of the entware install, offer creation of a swap file. I know this duplicates Adamm's script.
2. As part of adding Whitelist entries, offer the ability to insert a #comment. Some of the things I've added along the way are kind of obscure--there are two entries that you have to make to play Starz content, for example.

Thanks for the suggestions.
I'll consider these for a future update, the last was just missed by a few hours.
I was not aware of @Adamm building in the creation and management of a swap file into Skynet.
This is a recent problem, introduced with (and apparently only with) the RT-AC86U on the 382.x firmware.
Other devices, be they on 380.x or 382.x, do not appear to be effected by high RAM usage during processing of scripts.
In my opinion I consider this temporary until the reason for this is fixed on that device.

The comments in the white and blacklist however are very likely to be built in.

 

[ggkthx]

Does AB already block/prevent cryptomining? If not, is it possible to add hosts to do so? Would be good if the amalgamated list could include it.

 

[thelonelycoder]

Does AB already block/prevent cryptomining? If not, is it possible to add hosts to do so? Would be good if the amalgamated list could include it.

When I checked about two weeks ago the relevant hosts were included in three hosts files AB uses.
One of them is pgl.yoyo.org which is part of every blocking file type you can select.
All devices behind AB-Solution are safe, and have been for a while.

 

[Twiglets]

TLC,
I am reposting the latest version of the changes I made to 'functions.add' to give a more colourful and useful (to me) listing for 'Follow the logfile' Option 1.
They seem to have been lost a few changes ago in 'AB-Solution'.
If you still feel the change is useful please update the 'functions.add' as per below.

Sample screencap attached.

Green = Original DNS Query.
Light Blue = Data from DNS Cache (via dnsmasq).
Yellow = Forwarded to DNS Server
White = DNS Reply data from Query.

[Please ignore the 'validation' lines in screencap, as I am using DNSSEC ..... badly !!! ]


Changes:
=======
In functions.add change lines 1098-1111 to:

# part of # [Ff] follow the logfile (advanced)
tail_logfile(){
trap : INT
if [ $1 == standard ];then
echo -e " --> following the logfile now (tail -F)"
echo -e " $ansi_red blocked domains are color marked$nc:\n"
tail -F $abSolutionPath/$logsDir/dnsmasq.log | \
while read line;do
if echo "$line" | grep -q "is $custom_ipV4";then
echo -e "$ansi_red$line$nc"
else
if echo "$line" | grep -q "127.0.0.1"; then
echo -e "$ansi_yellow$line$nc"
else
if echo "$line" | grep -q ": query"; then
echo -e "$ansi_green$line$nc"
else
if echo "$line" | grep -q ": cached "; then
echo -e "$ansi_lt_blue$line$nc"
else
echo "$line"
fi
fi
fi
fi
done

Add to top of file the following to define colours used:

# Set colours for use in 'Follow the Logfile' Option 1
ansi_red="\033[1;31m";
ansi_green="\033[1;32m";
ansi_yellow="\033[1;33m";
ansi_lt_blue="\033[1;36m";

That is all.

Sample Screencap:

Sorry seem to have lost indent in the listing.
Appears in the edit window but not in the posting. ???!!!

 

[thelonelycoder]

TLC,
I am reposting the latest version of the changes I made to 'functions.add' to give a more colourful and useful (to me) listing for 'Follow the logfile' Option 1.
They seem to have been lost a few changes ago in 'AB-Solution'.
If you still feel the change is useful please update the 'functions.add' as per below.

Sample screencap attached.

Green = Original DNS Query.
Light Blue = Data from DNS Cache (via dnsmasq).
Yellow = Forwarded to DNS Server
White = DNS Reply data from Query.

[Please ignore the 'validation' lines in screencap, as I am using DNSSEC ..... badly !!! ]


Changes:
=======
In functions.add change lines 1098-1111 to:

# part of # [Ff] follow the logfile (advanced)
tail_logfile(){
trap : INT
if [ $1 == standard ];then
echo -e " --> following the logfile now (tail -F)"
echo -e " $ansi_red blocked domains are color marked$nc:\n"
tail -F $abSolutionPath/$logsDir/dnsmasq.log | \
while read line;do
if echo "$line" | grep -q "is $custom_ipV4";then
echo -e "$ansi_red$line$nc"
else
if echo "$line" | grep -q "127.0.0.1"; then
echo -e "$ansi_yellow$line$nc"
else
if echo "$line" | grep -q ": query"; then
echo -e "$ansi_green$line$nc"
else
if echo "$line" | grep -q ": cached "; then
echo -e "$ansi_lt_blue$line$nc"
else
echo "$line"
fi
fi
fi
fi
done

Add to top of file the following to define colours used:

# Set colours for use in 'Follow the Logfile' Option 1
ansi_red="\033[1;31m";
ansi_green="\033[1;32m";
ansi_yellow="\033[1;33m";
ansi_lt_blue="\033[1;36m";

That is all.

Sample Screencap:

View attachment 10886

Sorry seem to have lost indent in the listing.
Appears in the edit window but not in the posting. ???!!!

Your original proposal only has the blocked domains colored, that is still built in:
https://www.snbforums.com/threads/a...ing-solution-v3-9-2.37511/page-21#post-319223

But this looks much better!
I've tidied up the if clauses (you can use elif instead of else if ... fi) and I'm going to also use the theme colors since they appear best on the users terminal. But since the themes only have three text colors defined, I will have to add another one to it.
Also, your code does not work for IPv6 domains, use "$blockingFile\|blacklist.txt" instead of "is $custom_ipV4".

I'll include it in the next update.