Diversion Diversion - the Router Ad-Blocker

[KGB7]

Sometimes the websites themselves change how they deliver ads. Or the public blocking lists change.

What you’re doing isn’t working so gather some facts from the logs.

Unfortunately, Diversion website has limited information and not user friendly. Even though ive been pocking around Diversion settings, I can figure out a way to look at an extensive log.

 

[KGB7]

Apologies for multiple posts in the row, I dont see an option to multi-quote posts.

I reverted back to firmware 384.16 on Asus ac68u. No improvement at this time.

One more thing that I have left to test, is install Diversion using: curl -Os https://diversion.ch/install && sh install. Because I have been installing Diversion from AMTM and thats when I think the issue has started. Its the only thing that I can rule out at this time.

One last question for tonight. Diversion recommends to format USB thumb drive as ext2. Any reason I should not be using ext3 or ext4? I read up on the differences between the 3, but is there a specific reason when it comes to Diversion it self, why I should use ext2? I think in the past I formatted the thumb drive with ext4, because its newer format.

Im really trying to figure this out, and for the first time in a long time, I am stumped.


Thanks again everyone for the help.

 

[thelonelycoder]

One more thing that I have left to test, is install Diversion using: curl -Os https://diversion.ch/install && sh install. Because I have been installing Diversion from AMTM and thats when I think the issue has started. Its the only thing that I can rule out at this time.

amtm runs the exact same command as the one posted on my website.

 

[thelonelycoder]

Doesn't say much.

May 5 01:12:03 dnsmasq[5259]: query[A] keyvalueservice.fe.apple-dns.net from 192.160.100.13
May 5 01:12:03 dnsmasq[5259]: forwarded keyvalueservice.fe.apple-dns.net to 192.168.100.1
May 5 01:12:03 dnsmasq[5259]: reply keyvalueservice.fe.apple-dns.net is 17.248.143.124
May 5 01:12:03 dnsmasq[5259]: reply keyvalueservice.fe.apple-dns.net is 17.248.134.154
May 5 01:12:03 dnsmasq[5259]: reply keyvalueservice.fe.apple-dns.net is 17.248.134.168
May 5 01:12:03 dnsmasq[5259]: reply keyvalueservice.fe.apple-dns.net is 17.248.134.176
May 5 01:12:03 dnsmasq[5259]: reply keyvalueservice.fe.apple-dns.net is 17.248.134.203
May 5 01:12:03 dnsmasq[5259]: reply keyvalueservice.fe.apple-dns.net is 17.248.134.208
May 5 01:12:03 dnsmasq[5259]: reply keyvalueservice.fe.apple-dns.net is 17.248.134.211

Is that exact domain blocked in Diversion? It's not blocked with the large blocking list in use.

 

[thelonelycoder]

Unfortunately, Diversion website has limited information and not user friendly. Even though ive been pocking around Diversion settings, I can figure out a way to look at an extensive log.

Enter f in Diversion, it'll give a few options. Select 3. Blocked domains then browse a page with ads, it should show what's blocked.

 

[KGB7]

Enter f in Diversion, it'll give a few options. Select 3. Blocked domains then browse a page with ads, it should show what's blocked.

i This follows the Dnsmasq log file (tail -F)

1. Unfiltered log
2. Unfiltered log extra highlighted
3. Blocked domains
4. Blocked domains, by device IP
5. Term or IP

Enter selection [1-5 e=Exit] 3

i Press Ctrl-C to exit

04:35:21 blocked by blockinglist sdk.split.io
04:35:41 blocked by blockinglist googleads.g.doubleclick.net
04:35:43 blocked by blockinglist static.doubleclick.net
04:36:21 blocked by blockinglist events.split.io
04:36:21 blocked by blockinglist sdk.split.io
04:37:05 blocked by blockinglist clksite.com
04:37:22 blocked by blockinglist sdk.split.io
04:37:42 blocked by blockinglist googleads.g.doubleclick.net
04:38:22 blocked by blockinglist sdk.split.io
04:39:22 blocked by blockinglist sdk.split.io
04:39:48 blocked by blockinglist googleads.g.doubleclick.net
----------------------------------------------------------------------------

This is what I get when I have 3 tabs open to this site, to YouTube site and to 3rd site.

 

[KGB7]

Is that exact domain blocked in Diversion? It's not blocked with the large blocking list in use.

i This follows the Dnsmasq log file (tail -F)

1. Unfiltered log
2. Unfiltered log extra highlighted
3. Blocked domains
4. Blocked domains, by device IP
5. Term or IP

Enter selection [1-5 e=Exit] 2

i Press Ctrl-C to exit

May 5 04:41:22 dnsmasq[5259]: query[A] sdk.split.io from 192.160.100.13
May 5 04:41:22 dnsmasq[5259]: blocked by blockinglist sdk.split.io is 192.160.100.4
May 5 04:41:36 dnsmasq[5259]: query[A] time-macos.apple.com from 192.160.100.13
May 5 04:41:36 dnsmasq[5259]: forwarded time-macos.apple.com to 192.168.100.1
May 5 04:41:36 dnsmasq[5259]: reply time-macos.apple.com is <CNAME>
May 5 04:41:36 dnsmasq[5259]: reply time-osx.g.aaplimg.com is 17.253.20.253
May 5 04:41:36 dnsmasq[5259]: reply time-osx.g.aaplimg.com is 17.253.14.125
May 5 04:41:36 dnsmasq[5259]: reply time-osx.g.aaplimg.com is 17.253.14.251
May 5 04:41:36 dnsmasq[5259]: reply time-osx.g.aaplimg.com is 17.253.14.253
May 5 04:41:36 dnsmasq[5259]: reply time-osx.g.aaplimg.com is 17.253.20.125
May 5 04:42:20 dnsmasq[5259]: query[PTR] lb._dns-sd._udp.0.100.160.192.in-addr.arpa from 192.160.100.13
May 5 04:42:20 dnsmasq[5259]: forwarded lb._dns-sd._udp.0.100.160.192.in-addr.arpa to 192.168.100.1
May 5 04:42:22 dnsmasq[5259]: query[A] sdk.split.io from 192.160.100.13
May 5 04:42:22 dnsmasq[5259]: blocked by blockinglist sdk.split.io is 192.160.100.4
May 5 04:42:50 dnsmasq[5259]: query[A] gspe35-ssl.ls.apple.com from 192.160.100.13
May 5 04:42:50 dnsmasq[5259]: forwarded gspe35-ssl.ls.apple.com to 192.168.100.1
May 5 04:42:50 dnsmasq[5259]: reply gspe35-ssl.ls.apple.com is <CNAME>
May 5 04:42:50 dnsmasq[5259]: reply gspe35-ssl.ls-apple.com.akadns.net is <CNAME>
May 5 04:42:50 dnsmasq[5259]: reply gspe35-ssl.ls.apple.com.edgekey.net is <CNAME>
May 5 04:42:50 dnsmasq[5259]: reply e6987.e9.akamaiedge.net is 184.24.151.149
May 5 04:43:00 dnsmasq[5259]: query[A] safebrowsing.googleapis.com from 192.160.100.13
May 5 04:43:00 dnsmasq[5259]: forwarded safebrowsing.googleapis.com to 192.168.100.1
May 5 04:43:00 dnsmasq[5259]: reply safebrowsing.googleapis.com is 64.233.185.95
May 5 04:43:22 dnsmasq[5259]: query[A] sdk.split.io from 192.160.100.13
May 5 04:43:22 dnsmasq[5259]: blocked by blockinglist sdk.split.io is 192.160.100.4
May 5 04:44:23 dnsmasq[5259]: query[A] sdk.split.io from 192.160.100.13
May 5 04:44:23 dnsmasq[5259]: blocked by blockinglist sdk.split.io is 192.160.100.4
May 5 04:45:23 dnsmasq[5259]: query[A] sdk.split.io from 192.160.100.13
May 5 04:45:23 dnsmasq[5259]: blocked by blockinglist sdk.split.io is 192.160.100.4
May 5 04:45:37 dnsmasq[5259]: query[A] init.push.apple.com from 192.160.100.13
May 5 04:45:37 dnsmasq[5259]: forwarded init.push.apple.com to 192.168.100.1
May 5 04:45:37 dnsmasq[5259]: reply init.push.apple.com is <CNAME>
May 5 04:45:37 dnsmasq[5259]: reply init.push-apple.com.akadns.net is 17.188.164.220
May 5 04:45:37 dnsmasq[5259]: reply init.push-apple.com.akadns.net is 17.188.166.108
May 5 04:45:37 dnsmasq[5259]: reply init.push-apple.com.akadns.net is 17.188.166.160
May 5 04:45:37 dnsmasq[5259]: reply init.push-apple.com.akadns.net is 17.188.167.215
May 5 04:45:37 dnsmasq[5259]: reply init.push-apple.com.akadns.net is 17.188.161.93
May 5 04:45:37 dnsmasq[5259]: reply init.push-apple.com.akadns.net is 17.188.161.184
May 5 04:45:37 dnsmasq[5259]: reply init.push-apple.com.akadns.net is 17.188.163.28
May 5 04:45:37 dnsmasq[5259]: reply init.push-apple.com.akadns.net is 17.188.163.211

---------------------------------------------------------------------------------------------------

 

[dave14305]

May 5 04:43:00 dnsmasq[5259]: forwarded safebrowsing.googleapis.com to 192.168.100.1

What is 192.168.100.1? It’s unusual for an upstream DNS server from your WAN settings.

 

[JaimeZX]

Do you see overlay ads in YouTube videos??
Thanks for the suggestion.

Yes, we're not aware of any way to get rid of them. Not all videos have ads though. Depends on who posts the video and what their settings are, IIRC.

 

[JaimeZX]

Does Skynet work?
Is your external drive mounted? (Does it show up as mounted in the GUI? What about fdisk -l ?)

 

[bluepoint]

Cleared all caches and dns. Flushed everything, using terminal commands.
In the past, this tactic did work, but its not working for some reason at this time.

All websites did require manually entering credentials; emails, banking, medical, etc etc. So I think I cleared everything out.

The ad overlay on top of YouTube video is something I have never seen before. It could be new and just been implemented by Youtube. But last week I did not see the overlay ads on top of videos. I attached image in my earlier post.

Do you see overlay ads in YouTube videos??

Thanks for the suggestion.

Yes, overlay ads in youtube are normal, it depends on the video you're watching, sometimes they have sometimes they don't. Diversion does not block these kind of ads in youtube. Sometimes you'll see blank placeholder, when this happens it means Diversion blocked the content but not the placeholder and to get it out of the way click the x. If you want to watch youtube ads free, use a browser extension like uBlock Origin, it works.

 

[HairyA00]

Yes, overlay ads in youtube are normal, it depends on the video you're watching, sometimes they have sometimes they don't. Diversion does not block these kind of ads in youtube. Sometimes you'll see blank placeholder, when this happens it means Diversion blocked the content but not the placeholder and to get it out of the way click the x. If you want to watch youtube ads free, use a browser extension like uBlock Origin, it works.

Huge believer in this. Ad-blocking is an approach that is multi-tiered. There is no one size fits all, but in combination, your experience gets better and better. uBlock Origin is absolutely a no-brainer. There are several others that will enhance your experience as well.

Sinkholes are great for IoT devices that cannot have content blockers installed on them. They are also great for mobile devices, blocking known telemetry domains, etc. Yucky stuff that you can set and forget. I always forget on the rare occasion that I Chromecast something or watch a video on my phone that YouTube even has ads; most of my YouTube usage is on my computer with a combination of extensions.

I don't want to hijack this convo with extensions to install... but here are my must-haves (I do not have FB, but if you are still using it, FB Purity makes FB actually usable):
HTTPS Everywhere, uBlock Origin, Location Guard, F.B. Purity, Facebook Container (Firefox only).

On Android, there are NewPipe and YouTube Vanced also available.

Diversion, or any DNS sinkhole for that matter, cannot block YouTube ads. They serve their ads from the same domains as their content. Sinkholes see the domain request only but none of the traffic that follows. Content blockers see the entire URL and all the html and Javascript that loads, and can act based on that.

 

[KGB7]

What is 192.168.100.1? It’s unusual for an upstream DNS server from your WAN settings.

That is a dedicated HotSpot connected to the router via usb. And its my main source of internet.
That is not an actual IP from IPS it self. That IP is something I set manually in the HotSpot itself, because hotspot has an internal router function.

 

[KGB7]

Yes, overlay ads in youtube are normal, it depends on the video you're watching, sometimes they have sometimes they don't. Diversion does not block these kind of ads in youtube. Sometimes you'll see blank placeholder, when this happens it means Diversion blocked the content but not the placeholder and to get it out of the way click the x. If you want to watch youtube ads free, use a browser extension like uBlock Origin, it works.

Thanks for clarification.

Im using AdGuard, I have all filters enabled and set to level 11.
Something have has changed, im not sure what. It feels like I have malware on my Mac, but ive scanned it multiple times and I havent installed any software in the past month.

 

[KGB7]

I got my Dell laptop out of the closet that is running Windows 8.1. Its connected to my wifi router ac68u that has Diversion running on it.
It has latest FireFox, Internet Explorer and Microsoft Edge.
No adblockers installed. Everything is vanilla.
.
.
.

No ads, no ad pop ups. Empty Overlay ads in YouTube videos are still there. But its still a big improvement.
Every browser and website is add free because of Diversion on Windows Laptop. But I rarely use the laptop in rare cases.

Something, I dont know what, is causing issues on my Mac laptop. Looks like i'll have to format it and start from scratch.
I'll remove AdGuard as a final test to see if it makes a difference.
A week ago everything was perfect and I did not make a single change and I did not install any new software.
But something did change and I cant figure it out. Ive been at this for 3 days now.

Thanks for the help everyone.

If any of you are Mac experts, please chime in so I dont have to reinstall the OS.

 

[KGB7]

Huge believer in this. Ad-blocking is an approach that is multi-tiered. There is no one size fits all, but in combination, your experience gets better and better. uBlock Origin is absolutely a no-brainer. There are several others that will enhance your experience as well.

Sinkholes are great for IoT devices that cannot have content blockers installed on them. They are also great for mobile devices, blocking known telemetry domains, etc. Yucky stuff that you can set and forget. I always forget on the rare occasion that I Chromecast something or watch a video on my phone that YouTube even has ads; most of my YouTube usage is on my computer with a combination of extensions.

I don't want to hijack this convo with extensions to install... but here are my must-haves (I do not have FB, but if you are still using it, FB Purity makes FB actually usable):
HTTPS Everywhere, uBlock Origin, uBlock Origin Extra (Chrome only), Location Guard, F.B. Purity, Facebook Container (Firefox only).
For even more anti-adblock blocking, you can also configure Nano Defender: https://jspenguin2017.github.io/uBlockProtector/#extra-installation-steps-for-ublock-origin

On Android, there are NewPipe and YouTube Vanced also available.

Diversion, or any DNS sinkhole for that matter, cannot block YouTube ads. They serve their ads from the same domains as their content. Sinkholes see the domain request only but none of the traffic that follows. Content blockers see the entire URL and all the html and Javascript that loads, and can act based on that:
https://diversion.ch/faq-reader/can-diversion-block-youtube-ads.html
https://discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253

This is a great intel.

I was a windows user since Win 3.1. In the last 3 years I slowly merged in to Apple Ecosystem on a budget. And I do love Apple Ecosystem. I wont go in to details why so not to highjack the thread.
But the ad blockers on Safari just aren't that good on iOS or Mac OS. I dont use Chrome or Firefox anymore, because of Safari ecosystem.
I have AdGuard on all of my 3 Apple devices, that were highly recommended by many website reviews, but Diversion does block ads far better in my experience then any top 3 ad blockers on MacOS and on iOS.

If ublock, 1blocker, Origin, Adblocker, Adblockplus, etc etc etc. Were useful on MacOS and on iOS, we would not need Diversion.
Wipr is great, but has no white list. Not much of community and limited support.

Back on topic.

 

[L&LD]

@KGB7 try using Edge chromium on the MAC.

 

[majnu]

Does anyone remember the name of the application where I can access my router's file directory via a user friendly interface?

iirc it used to be mentioned on the diversion requirements showing the path to the whitelist or maybe it was mentioned some place else but I cannot find it anymore.

I wish to use this approach as I have a whitelist text file I want to paste into the relevant directory.

Thanks

 

[tomsk]

Does anyone remember the name of the application where I can access my router's file directory via a user friendly interface?

iirc it used to be mentioned on the diversion requirements showing the path to the whitelist or maybe it was mentioned some place else but I cannot find it anymore.

I wish to use this approach as I have a whitelist text file I want to paste into the relevant directory.

Thanks

The post you linked to is about hosted whitelists. You just need to give diversion the address of the hosted list and it will will replace the current whitelist. If you have a text file of domains you want to ad, then just edit the list in nano. Don't forget to process the list through el in diversion afterwards

nano /opt/share/diversion/list/whitelist

I use midnight commander to root around files in the router

tOmsK@RT-AC68U-4690:/tmp/home/root# opkg list mc
mc - 4.8.23-2 - GNU Midnight Commander is a visual file manager.
 It's a feature rich full-screen text mode application that allows you to copy,
 move and delete files and whole directory trees, search for files and run commands in the subshell.
 Internal viewer and editor are included as well.

opkg install mc

 

[KGB7]

@KGB7 try using Edge chromium on the MAC.

No can do. It would stray away from apple ecosystem.
Let me explain.
If I log in to a bank account or in to my medical records at Medstar using Safari on Mac. The login information will be seamlessly shared across all my Apple devices via cloud as long as I use Safari. So if I want to log in to my bank on my iPhone for the first time, the login info will be shared from laptop to phone with one touch.
If I change bank password on iPhone, Mac will get the updated password. If I change routers password on iPad, ...Mac and iPhone will get new password under a minute.
WiFi passwords are also shared across all devices through cloud.
Edge, Chrome, Firefox, etc cant do that, because they have no ecosystem . Thats why I chose apple ecosystem. I love simplicity even though I was a dedicated Windows and Android user for many many years.
Even 7 years ago, connecting a network printer to a Mac was soooooo simple. It took less than 15sec.

But we can discuss the details in another post if you would like.