Diversion Diversion - the Router Ad-Blocker

[thelonelycoder]

Hi, Unfortunately I cannot use Diversion with my Tomato Router software.
But the pixelserv-tls runs on Tomato as well.

Diversion creates a pretty good blocking list.
Mine is now 1 year old and I would like to have a current one without Asus Merlin to flash on the router first.

Maybe someone here can help me and provide me with the current blocking list?
Thanks.

Which one of the predefined blocking lists do you want compiled?
https://diversion.ch/diversion/use/blocking-file-hosts-files.html
And what blocking IP is pixelserv-tls listening on?

 

[Spiker]

@thelonelycoder
Thanks for the link. The information there is exactly what I was looking for.

My old one is about 865 kb, then I guess I use the standard list.
With this Information I can then almost create the list by myself...
Is there a trick how I can convert the list so that several addresses are in one row ?

like this ->
192.168.123.60 diversion-adblocking-ip.address 0.nextyourcontent.com 0.r.msn.com 000.0x1f4b0.com 000.gaysexe.free.fr 000free.us

 

[thelonelycoder]

@thelonelycoder
Thanks for the link. The information there is exactly what I was looking for.

My old one is about 865 kb, then I guess I use the standard list.
With this Information I can then almost create the list by myself...
Is there a trick how I can convert the list so that several addresses are in one row ?

like this ->
192.168.123.60 diversion-adblocking-ip.address 0.nextyourcontent.com 0.r.msn.com 000.0x1f4b0.com 000.gaysexe.free.fr 000free.us

I don't know how good you are with scripts, but this is roughly what Diversion does. Edit as needed.
And with that we are back to the copy/paste age of Adblocking with a combined hosts file

# pixelserv-tls IP
blockingIP=192.168.123.60

# Must not be over 25!
domainsPerLine=20

# Hosts file raw URL
hostsFile=https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

# Full path including filename to blocking list
blockinglist=/tmp/mnt/8GB/blockinglist

# Do not change below
curl_dl(){ curl --insecure --location --connect-timeout 10 --retry 3 -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:53.0) Gecko/20100101 Firefox/53.0" --progress-bar "$@";}
curl_dl "$hostsFile" | grep "^[^#]" \
| sed -e "s/^[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/X/g" \
| grep -w "^X" | awk '{print " "$2}' | grep -E '[[:alnum:]]+[.][[:alnum:]_.-]+' \
| awk '!/ [0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$/' | awk '!/[:?\/;]/' \
| awk '{if(NR%'"$domainsPerLine"'==0){a=a""$0;print b a" "; a=""}else a=a""$0}END{if(a)print b a" "}' b="$blockingIP" \
> "${blockinglist}"

 

[Spiker]

I don't know how good you are with scripts, but this is roughly what Diversion does. Edit as needed.
And with that we are back to the copy/paste age of Adblocking with a combined hosts file

# pixelserv-tls IP
blockingIP=192.168.123.60

# Must not be over 25!
domainsPerLine=20

# Hosts file raw URL
hostsFile=https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

# Full path including filename to blocking list
blockinglist=/tmp/mnt/8GB/blockinglist

# Do not change below
curl_dl(){ curl --insecure --location --connect-timeout 10 --retry 3 -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:53.0) Gecko/20100101 Firefox/53.0" --progress-bar "$@";}
curl_dl "$hostsFile" | grep "^[^#]" \
| sed -e "s/^[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/X/g" \
| grep -w "^X" | awk '{print " "$2}' | grep -E '[[:alnum:]]+[.][[:alnum:]_.-]+' \
| awk '!/ [0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$/' | awk '!/[:?\/;]/' \
| awk '{if(NR%'"$domainsPerLine"'==0){a=a""$0;print b a" "; a=""}else a=a""$0}END{if(a)print b a" "}' b="$blockingIP" \
> "${blockinglist}"

Works very well, thank you very much !
I can program very little but modify well and the rest is logic, there is enough of it ;-)

Right that is one step back.
I would also like to switch to Asus Merlin,
then I could just take advantage of Diversion.
That would also be the better solution!

But I get with the delivered Entware (maintained by zyxmon & ryzhovau)
only the current Freeradius 3 installed.


I need this for the Wlan authentication with certificates (WPA2-Enterprise EAP-TLS)


Certificates and Config for Freeradius 2 are working

Strange but the same config does not with Freeradius 3

I did try and try but I gave up after 8 days :-(

so I will stay with FreshTomato until I found a Solution for Freeradius 3

 

[Ninko]

Completely new to Diversion, but have a quick question.
I've seen in a few screenshots Diversion has GUI tab, but one doesn't show up on my install, is there meant to be one?

Thanks

 

[Butterfly Bones]

Completely new to Diversion, but have a quick question.
I've seen in a few screenshots Diversion has GUI tab, but one doesn't show up on my install, is there meant to be one?

Thanks

Are you using Diversion Standard or Lite?
If on Standard, hit d
d Diversion Standard
then 10
10. Enable Diversion WebUI beta feature!

 

[bluepoint]

Are you using Diversion Standard or Lite?
If on Standard, hit d
d Diversion Standard
then 10
10. Enable Diversion WebUI beta feature!

These options are also available in the lite version.

 

[thelonelycoder]

Completely new to Diversion, but have a quick question.
I've seen in a few screenshots Diversion has GUI tab, but one doesn't show up on my install, is there meant to be one?

Thanks

The limited WebUI beta is available in d.
Read this also: https://www.snbforums.com/threads/diversion-the-router-ad-blocker.48538/page-326#post-589539

 

[thelonelycoder]

Are you using Diversion Standard or Lite?
If on Standard, hit d
d Diversion Standard
then 10
10. Enable Diversion WebUI beta feature!

The limited WebUI beta is available for all Diversion Editions.

 

[Ninko]

That did it, thanks guys!

 

[Klesk Reaver]

Recommended read/write speeds for flash drives to use with Diversion? USB 2.0, USB 3.0? does it matter?

Should I only care about a swapfile if the RAM in my router fills up? or is it just good to have one anyway?

 

[thelonelycoder]

Recommended read/write speeds for flash drives to use with Diversion? USB 2.0, USB 3.0? does it matter?

Should I only care about a swapfile if the RAM in my router fills up? or is it just good to have one anyway?

Get a fast device if you're planning on installing uiDivStats.
A swap file is highly recommended in any case.

 

[Ninko]

Another quick question, what's the benefits to using the Alternate blocking list option?

Thanks

 

[alekos]

I have a weird issue that started today with Amazon App (Canada), I started getting the error on the app - I've included the 5 main Amazon domains to the whitelist a few months ago and it's been great until today, even tho I have whitelisted
's.amazon-adsystem.com'
It shows up blocked on the IP Block log along with the app not working. All other Whitelisted domains are working fine.
Is their something I've missed?

 

[jsbeddow]

I have a weird issue that started today with Amazon App (Canada), I started getting the error on the app - I've included the 5 main Amazon domains to the whitelist a few months ago and it's been great until today, even tho I have whitelisted
's.amazon-adsystem.com'
It shows up blocked on the IP Block log along with the app not working. All other Whitelisted domains are working fine.
Is their something I've missed?

This is probably off topic, but as someone who resists installing yet another "app" (especially when it just for a site that can be accessed through a browser), I am always amazed at the number of times this Amazon app issue crops up. What the H is the appeal of having an app for Amazon (other than their one time gift offer of $10 the first time you use it)???
Sorry, rant over...

Edit: I imagine that the app is designed to "alert you" of daily deals through notifications...so it is not surprising that an advertisement driven app is somewhat disabled or disrupted by any means to block ads, which is what Diversion is fundamentaly meant to do.

 

[alekos]

This is probably off topic, but as someone who resists installing yet another "app" (especially when it just for a site that can be accessed through a browser), I am always amazed at the number of times this Amazon app issue crops up. What the H is the appeal of having an app for Amazon (other than their one time gift offer of $10 the first time you use it)???
Sorry, rant over...

Edit: I imagine that the app is designed to "alert you" of daily deals through notifications...so it is not surprising that an advertisement driven app is somewhat disabled or disrupted by any means to block ads, which is what Diversion is fundamentaly meant to do.

You're right. I don't have a lot of apps installed actually. And the Amazon app, I find it just a lot faster and easier than accessing it through the web. I have disabled notifications on all but 3 apps (I like to have minimal apps, notifications etc on my phone, and nothing accesses my location except maps when I'm in my car).
I guess it was the Energized lists that was using, I reverted back to one of the defualt lists (Large) and everything works now. I had everything working perfectly for weeks, nothing changed, that's why I posted...
I hate posting questions when I have issues. I didn't want to post, really, especially for an Amazon App! (And I hate posting when the issue isn't even with Diversion, but something I changed/added ... just seemed so weird that a domain that is in a WL is being blocked, but I forget that there are associated domains also.,,.<*edit

 

[CriticJay]

Despite having correctly installed certificates, I still have a slu to slh ration of about 2.5:1 Running the pixelserv log at 2 I can identify the problem sites from 'handshake failed: unknown cert.'

Rather than disabling pixelserv completely, I have added the problem sites to hosts.add and the whitelist. I have also seen that instead of using hosts.add, I could add these sites via dnsmasq.conf.add.

Is there any reason to prefer one method over the other?

How do I modify hosts.add to include IPv6?

e.g. I have mobile.pipe.aria.microsoft.com in my whitelist and the line '0.0.0.0 mobile.pipe.aria.microsoft.com' in hosts.add

where the IPv4 query [A] works as expected and returns 0.0.0.0

but ipV6 [AAAA] returns the 'correct' address 2620:119:53::53

Also there a way to directly edit the whitelist (e.g. via ssh) other than by having a hosted whitelist?

I'm trying to do this (what you did) because the Amazon Android app on my phone doesn't seem to like "ad.doubleclick.net" going to a pixelserv-tls IP. I'm trying to make that domain go to the 0.0.0.0 black hole by creating a hosts.add like you did.

Interesting result: pinging that domain ON THE ROUTER shows it getting resolved to 0.0.0.0

ping that domain on my WIndows laptop after a fresh reboot still shows it getting resolved to the Pixelserv internal IP!

any ideas?

 

[CriticJay]

I have a weird issue that started today with Amazon App (Canada), I started getting the error on the app - I've included the 5 main Amazon domains to the whitelist a few months ago and it's been great until today, even tho I have whitelisted
's.amazon-adsystem.com'
It shows up blocked on the IP Block log along with the app not working. All other Whitelisted domains are working fine.
Is their something I've missed?

i had the same issue as you and i did troubleshooting using the "follow" option in diversion - that's what you are supposed to do, to determine which blocked host is causing the problem

i determined the problem host was "ad.doubleclick.net". if you whitelist that, problem goes away.

i'm trying to get that host to resolve to a DNS BL (0.0.0.0) instead, because obviously a lot of regular ads come from that host as well and I'd rather not whitelist it across the board ...

(typically problems with the amazon shopping app is due to blocked hosts on Pixelserv-TLS)

 

[alekos]

i had the same issue as you and i did troubleshooting using the "follow" option in diversion - that's what you are supposed to do, to determine which blocked host is causing the problem

i determined the problem host was "ad.doubleclick.net". if you whitelist that, problem goes away.

i'm trying to get that host to resolve to a DNS BL (0.0.0.0) instead, because obviously a lot of regular ads come from that host as well and I'd rather not whitelist it across the board ...

(typically problems with the amazon shopping app is due to blocked hosts on Pixelserv-TLS)

I guess I wasn't really clear, but ofcourse I followed the "follow" option, and today that domain appeared... and it's been fine for many many weeks. 'doubleclick' wasn't appearing as blocked on my phone's ip (using follow command)

 

[CriticJay]

I guess I wasn't really clear, but ofcourse I followed the "follow" option, and today that domain appeared... and it's been fine for many many weeks. 'doubleclick' wasn't appearing as blocked on my phone's ip (using follow command)

For me once I whitelisted the doubleclick domain, the amazon app on android started working again. today was the first time i had this issue on my phone (although ive seen people here complain about it before).

however, i dont want to whitelist doubleclick (because it's a top 10 Ad server) - instead i think if doubleclick can get sent to a blackhole, but all other blocklist sites get sent to the pixelserv-TLS IP -- that's what I'd like to try.