Diversion Diversion - the Router Ad-Blocker

[thelonelycoder]

I've updated pixelserver using diversion, from version 2.1.1 to 2.1.2, but now im getting this:

Sep 12 02:03:24 pixelserv-tls[12611]: Listening on :*:443
Sep 12 02:03:24 pixelserv-tls[12611]: Abort: Address already in use - :*:80

Entware or whatever updated pixelserv-tls must have overwritten the /opt/etc/init.d/S80pixelserv-tls.
Diversion uses a customized version of it, optimized with settings for your router. A Re-Install in d fixes that.

 

[thelonelycoder]

Interesting! I didn't have a warning message about pixelserv-tls not running. What router model and firmware version are you using?

I'm doing a test of $(pidof pixelserv-tls) to check. Maybe one instance was running?

 

[M@rco]

I'll look into it and likely simple to fix. Those DHCPREQUEST(br0) were filtered out in AB-Solution stats, so they should also not be included in Diversion. In c Diversion stats, did you set "Filter local client names" to on? This has immediate affect on the stats if you run it.
Also, try setting "domain-needed" in the Dnsmasq settings in ds. This, however, will only be fully effective with a new set of dnsmasq.log files which is after the stats and the blocking file update have run through the weekly cron job scheduler.

Thanks for looking into it. I didn't have 'Filter local client names' switched on in c 2 4 (but I have now). I switched 'domain-needed' to on in ds 5 1 and reschuled the weekly stats to run tonight. Will report back.

 

[Xentrk]

I'm doing a test of $(pidof pixelserv-tls) to check. Maybe one instance was running?

Thanks for looking into it.

I think it was the code snip from config-webgui that detected pixelserv-tls was not running last time I had a similar issue.

 [[ "$(/bin/ps | grep -v grep | grep -o pixelserv-tls)" = ""  || ! -x /opt/bin/pixelserv-tls ]] && \
      echo "You do not appear to have pixelserv-tls installed or running." &&  \
        exit 1

I will try to duplicate the concern at one of the sites I support tomorrow. I'll check to see if Diversion catches the error in addition to the code snip above. I'll let you know what I find out.

 

[Jack Yaz]

Before I go log diving, has anyone else experienced issues with Google Play Store downloads being blocked on the Standard blocking file?

 

[skeal]

Before I go log diving, has anyone else experienced issues with Google Play Store downloads being blocked on the Standard blocking file?

My updates downloaded with zero issues last night and I use the same blocking file.

 

[Jack Yaz]

My updates downloaded with zero issues last night and I use the same blocking file.

OK thanks, I'll turn my attentions to other areas, could be AiProtection acting up again

 

[thelonelycoder]

Thanks for looking into it. I didn't have 'Filter local client names' switched on in c 2 4 (but I have now). I switched 'domain-needed' to on in ds 5 1 and reschuled the weekly stats to run tonight. Will report back.

I see now where the DHCPREQUEST(br0) are coming from.
I have LAN/DHCP Server/Hide DHCP/RA queries set to yes on all my routers.

 

[M@rco]

I see now where the DHCPREQUEST(br0) are coming from.
I have LAN/DHCP Server/Hide DHCP/RA queries set to yes on all my routers.

I used to supress them too in syslog, maybe that's why I've never noticed it before. For debugging I enabled them recently, prior to installing Diversion (if I recall correctly). Is it possible to filter these out of Diversion stats?

 

[thelonelycoder]

Before I go log diving, has anyone else experienced issues with Google Play Store downloads being blocked on the Standard blocking file?

This one I did not mention in the Diversion 4.0.1 update change log, I simply forgot about it but it seemed to make sense at the time I did the change locally.

Due to the restirctive blocking, I have removed https://raw.githubusercontent.com/S...lternates/fakenews-gambling-porn-social/hosts
and replaced it with the less intrusive
https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-gambling-porn/hosts

This change only affects new installations or if you change or reapply one of the four blocking file types.
It would make sense if Google Play was included in the fakenews-gambling-porn-social list but not in fakenews-gambling-porn. Give it a try by re-selecting your blocking file type then run a manual update.

 

[thelonelycoder]

I used to supress them too in syslog, maybe that's why I've never noticed it before. For debugging I enabled them recently, prior to installing Diversion (if I recall correctly). Is it possible to filter these out of Diversion stats?

That would be handy for some to see these requests in the log for other reasons. Just not in the stats.
I'll see what I can do, should not be too hard.

 

[M@rco]

That would be handy for some to see these requests in the log for other reasons. Just not in the stats.
I'll see what I can do, should not be too hard.

If this was already the case in AB-Solution, I'm actually surprised nobody ever reported it before... I'll hide the dhcp messages for now. If (and when) you've had time to solve it in a future release, I'll enable them again to test to see if it's solved. Thanks again!

 

[Jack Yaz]

This one I did not mention in the Diversion 4.0.1 update change log, I simply forgot about it but it seemed to make sense at the time I did the change locally.

Due to the restirctive blocking, I have removed https://raw.githubusercontent.com/S...lternates/fakenews-gambling-porn-social/hosts
and replaced it with the less intrusive
https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-gambling-porn/hosts

This change only affects new installations or if you change or reapply one of the four blocking file types.
It would make sense if Google Play was included in the fakenews-gambling-porn-social list but not in fakenews-gambling-porn. Give it a try by re-selecting your blocking file type then run a manual update.

Thanks, but it doesn't work even with Diversion disabled and DNS caches flushed. Oddly, it works on WiFi but not when on VPN to the local network. Firewall not reporting any blocks, so colour me confused

 

[thelonelycoder]

Thanks, but it doesn't work even with Diversion disabled and DNS caches flushed. Oddly, it works on WiFi but not when on VPN to the local network. Firewall not reporting any blocks, so colour me confused

Disable ad-blocking, with logging enabled. Then use f and do what you do. Maybe the dnsmasq log reveals useful info.

 

[Jack Yaz]

Disable ad-blocking, with logging enabled. Then use f and do what you do. Maybe the dnsmasq log reveals useful info.

Literally everything works (seemingly) except the download of an app itself. It must be something with my VPN Server config, but I'm stumped as to what would specifically break app downloads (this is off-topic now I'm aware!)

 

[thelonelycoder]

Literally everything works (seemingly) except the download of an app itself. It must be something with my VPN Server config, but I'm stumped as to what would specifically break app downloads (this is off-topic now I'm aware!)

We are all ears, let us know what was causing it.
I'd like to go hiking, but that is off topic.

 

[M@rco]

Literally everything works (seemingly) except the download of an app itself. It must be something with my VPN Server config, but I'm stumped as to what would specifically break app downloads (this is off-topic now I'm aware!)

Before you're about to check your entire config, this reminds of an old issue (back in the ol' days when I was still using Android ) when apps suddenly failed to download. I was always busy flashing custom ROMs, and had issues like this on several occassions. Until I discovered it was an issue which apparently even on stock firmware occured. Suggestions like clearing the Play Store cache, logging out and back into your account or even switching to 4G and back to WiFi mostly solved the issue. But that's quite some time (and three iPhones) ago so you might want do some Google-Fu to see if something like this still incidently occurs, before you go down to basement to inspect your VPN tunnel from the inside...

And now I have to find a way to get rid of the image of @thelonelycoder hiking in the most horrific outfit known to men

...but that's off-topic

 

[skeal]

Before you're about to check your entire config, this reminds of an old issue (back in the ol' days when I was still using Android ) when apps suddenly failed to download. I was always busy flashing custom ROMs, and had issues like this on several occassions. Until I discovered it was an issue which apparently even on stock firmware occured. Suggestions like clearing the Play Store cache, logging out and back into your account or even switching to 4G and back to WiFi mostly solved the issue. But that's quite some time (and three iPhones) ago so you might want do some Google-Fu to see if something like this still incidently occurs, before you go down to basement to inspect your VPN tunnel from the inside...

And now I have to find a way to get rid of the image of @thelonelycoder hiking in the most horrific outfit known to men

...but that's off-topic

Nice!!

 

[truglodite]

...
Your custom scripts are now safe, Diversion does not touch them.

Confirmed, after updating from 4.0.0 to 4.0.1 my scripts are still there. Thanks!

 

[Xentrk]

Thanks for looking into it.

I think it was the code snip from config-webgui that detected pixelserv-tls was not running last time I had a similar issue.

 [[ "$(/bin/ps | grep -v grep | grep -o pixelserv-tls)" = ""  || ! -x /opt/bin/pixelserv-tls ]] && \
      echo "You do not appear to have pixelserv-tls installed or running." &&  \
        exit 1

I will try to duplicate the concern at one of the sites I support tomorrow. I'll check to see if Diversion catches the error in addition to the code snip above. I'll let you know what I find out.

I see the problem now. I was still on version 4.0. That is why I did not see an error message about pixelserv-tls not running. Version 4.0.1 captured the pixelserv-tls error:

A reinstall of Diversion is required to fix the issue.

After upgrade to pixelserv-tls 2.1.2:

Sep 13 05:34:32 RT-AC88U-9E18 pixelserv-tls[16719]: pixelserv-tls 2.1.2 (compiled: Sep  8 2018 20:33:38) options: <none>
Sep 13 05:34:35 RT-AC88U-9E18 pixelserv-tls[16719]: Listening on :*:443
Sep 13 05:34:35 RT-AC88U-9E18 pixelserv-tls[16719]: Abort: Address already in use - :*:80

After reinstall of Diversion:

Sep 13 12:37:34 RT-AC88U-9E18 pixelserv-tls[17882]: pixelserv-tls 2.1.2 (compiled: Sep  8 2018 20:33:38) options: 192.168.1.2
Sep 13 12:37:34 RT-AC88U-9E18 pixelserv-tls[17882]: Listening on :192.168.1.2:443
Sep 13 12:37:34 RT-AC88U-9E18 pixelserv-tls[17882]: Listening on :192.168.1.2:80