Diversion Diversion - the Router Ad-Blocker

[L&LD]

Of course, but no placebo here. Fixed control channels, fixed bandwidth, and enough time for the surrounding AP's to 'automatically' ignore the channels I choose do make a significant difference.

Not always, but that also usually means that I simply chose a wrong control channel (at first) for the RF environment the router is operating in.

 

[archiel]

Despite having correctly installed certificates, I still have a slu to slh ration of about 2.5:1 Running the pixelserv log at 2 I can identify the problem sites from 'handshake failed: unknown cert.'

Rather than disabling pixelserv completely, I have added the problem sites to hosts.add and the whitelist. I have also seen that instead of using hosts.add, I could add these sites via dnsmasq.conf.add.

Is there any reason to prefer one method over the other?

How do I modify hosts.add to include IPv6?

e.g. I have mobile.pipe.aria.microsoft.com in my whitelist and the line '0.0.0.0 mobile.pipe.aria.microsoft.com' in hosts.add

where the IPv4 query [A] works as expected and returns 0.0.0.0

but ipV6 [AAAA] returns the 'correct' address 2620:119:53::53

Also there a way to directly edit the whitelist (e.g. via ssh) other than by having a hosted whitelist?

 

[dave14305]

I have also seen that instead of using hosts.add, I could add these sites via dnsmasq.conf.add.

Is there any reason to prefer one method over the other?

The benefit of hosts.add is that you don’t have to remember the dnsmasq syntax.

How do I modify hosts.add to include IPv6?

0.0.0.0   mobile.pipe.aria.microsoft.com
::  mobile.pipe.aria.microsoft.com

Also there a way to directly edit the whitelist (e.g. via ssh) other than by having a hosted whitelist?

You can vi or nano /opt/share/diversion/list/whitelist and then run diversion to sort and process the updated whitelist. Not an encouraged method, but technically possible if you know how to navigate vi or nano.

 

[archiel]

You can vi or nano /opt/share/diversion/list/whitelist and then run diversion to sort and process the updated whitelist. Not an encouraged method, but technically possible if you know how to navigate vi or nano.

I am not particularly good with vi or nano, so I copy files to my windows desktop (in WinSCP), use Notepad++ as my external editor (still in WinSCP) and then copy back. Permissions are retained (as it is a copy) so no need to reset. Not very elegant, but much easier for me than remembering the syntax in vi / nano

 

[thelonelycoder]

I am not particularly good with vi or nano, so I copy files to my windows desktop (in WinSCP), use Notepad++ as my external editor (still in WinSCP) and then copy back. Permissions are retained (as it is a copy) so no need to reset. Not very elegant, but much easier for me than remembering the syntax in vi / nano

You can set WinSCP to open files in Notepad++.
That's how I work all the time while developing. Though to make it even simpler, I have a Samba root directory share enabled for my devlopment routers.

 

[thelonelycoder]

Had a number of strange problems on the router this morning, and now I notice diversion is no longer counting ads blocked; and the counter was reset to zero; how can i reverse this?

Corruption, failed writes, interrupted count, bad USB device.

 

[dugaduga]

Corruption, failed writes, interrupted count, bad USB device.

is there a file i can manually edit and set total ad count blocked? & where is dnsmasq log hiding on the router? (update, found it under var/log, winscp search couldn't find it for whatever reason

 

[thelonelycoder]

is there a file i can manually edit and set total ad count blocked? & where is dnsmasq log hiding on the router?

/opt/share/diversion/.conf/diversion.conf
Change adsBlocked= and adsBlockedAlt= when the alternate blocking file is enabled. Write only numbers, the segmentation is done in the UI.

/opt/var/log/dnsmasq.log* where log is the current, log1 the previous day's log and log3 the weeks accumulated log file.

 

[dugaduga]

Corruption, failed writes, interrupted count, bad USB device.

Thank you thelonelycoder

 

[dugaduga]

Whats up with this?

 May 23 16:43:27 dnsmasq[733]: bad address at /opt/share/diversion/list/blacklist line 79
May 23 16:43:27 dnsmasq[733]: bad address at /opt/share/diversion/list/blacklist line 80
May 23 16:43:27 dnsmasq[733]: bad address at /opt/share/diversion/list/blacklist line 81
May 23 16:43:27 dnsmasq[733]: bad address at /opt/share/diversion/list/blacklist line 82
May 23 16:43:27 dnsmasq[733]: bad address at /opt/share/diversion/list/blacklist line 83

Blacklist, lines 79-83

clientsdk.luminatinet.com
clientsdk.lum-sdk.io
perr.l-agent.me
perr.l-err.biz
perr.lum-sdk.io

If I add the entire domain list of luminati's CDN's, they all show up as bad addresses in dnsmasq. So I removed the huge list and created a WC to scale down the log. The 5 I left behind still cause the problem. No other domain creates this problem. I'm stumped.

 

[dugaduga]

@thelonelycoder is manually changing the yt ip in diversion.conf and restarting diversion enough to make it work properly? I notice in wireshark streaming youtube videos are still using the same IP diversion had updated after I had lost previous my settings

 

[dave14305]

Whats up with this?

 May 23 16:43:27 dnsmasq[733]: bad address at /opt/share/diversion/list/blacklist line 79
May 23 16:43:27 dnsmasq[733]: bad address at /opt/share/diversion/list/blacklist line 80
May 23 16:43:27 dnsmasq[733]: bad address at /opt/share/diversion/list/blacklist line 81
May 23 16:43:27 dnsmasq[733]: bad address at /opt/share/diversion/list/blacklist line 82
May 23 16:43:27 dnsmasq[733]: bad address at /opt/share/diversion/list/blacklist line 83

Blacklist, lines 79-83

clientsdk.luminatinet.com
clientsdk.lum-sdk.io
perr.l-agent.me
perr.l-err.biz
perr.lum-sdk.io

If I add the entire domain list of luminati's CDN's, they all show up as bad addresses in dnsmasq. So I removed the huge list and created a WC to scale down the log. The 5 I left behind still cause the problem. No other domain creates this problem. I'm stumped.

If you added them through Diversion they should be preceded automatically by the blocking IP.

 

[dugaduga]

If you added them through Diversion they should be preceded automatically by the blocking IP.

face palm, amazed you caught that one, you are pure genius, thanks dave

 

[V@no]

For these who want see the computer IP along side with blocked domain in F -> 3 (Blocked domains) in /mnt/Diversion/entware/share/diversion/file/functions.div find first instance of (line 2794):

printf "%-43s%s\\n" "${GRAY} $( echo $line | awk '{print $3, "blocked by " $(NF-3)}' | sed 's|/opt/share/diversion/list/||;s|config|wc-blacklist|')${NC} " "${RED}$( echo $line | awk '{print $(NF-2)}')${NC}"

Replace it with:

printf "%-43s%-23s%s\\n" "${GRAY} $( echo $line | awk '{print $3, "blocked by " $(NF-3)}' | sed 's|/opt/share/diversion/list/||;s|config|wc-blacklist|')${NC}" "${BLUE}$( echo $line | awk '{print $(NF-4)}' | sed 's/\/[0-9]*$//')${NC}" "${RED}$( echo $line | awk '{print $(NF-2)}')${NC}"

P.S.
Just curious. What are $(NF-x) variables are? In this line I could use $6 instead of $(NF-4) with the same result...

 

[thelonelycoder]

P.S.
Just curious. What are ${NF-x} variables are? In this line I could use $6 instead of ${NF-4} with the same result...

Your code example only works if log-queries=extra is enabled in ds, which adds the requester IP address to the log line.
With this setting, the number of fields changes in the dnsmasq.log file, hence the $(NF-x) number of field selector. Your plain field selector $6 will not show the desired field with this ds setting set to off.

 

[V@no]

Thanks.
Computer IP seems to be shown on previous line, replacing this section:

       tail -F "${LOG_DIR}/dnsmasq.$log" | \
       while read line; do
           if echo "$line" | /opt/bin/grep -q "list/blockinglist\|list/blacklist\| config .* is $blockingIP"; then
               printf "%-43s%s\\n" "${GRAY} $( echo $line | awk '{print $3, "blocked by " $(NF-3)}' | sed 's|/opt/share/diversion/list/||;s|config|wc-blacklist|')${NC} " "${RED}$( echo $line | awk '{print $(NF-2)}')${NC}"
           fi
       done

With this:

       prevLine=''
       tail -F "${LOG_DIR}/dnsmasq.$log" | \
       while read line; do
           if echo "$line" | /opt/bin/grep -q "list/blockinglist\|list/blacklist\| config .* is $blockingIP"; then
               printf "%-43s%-23s%s\\n" "${GRAY} $( echo $line | awk '{print $3, "blocked by " $(NF-3)}' | sed 's|/opt/share/diversion/list/||;s|config|wc-blacklist|')${NC}" "${BLUE}$( echo $prevLine | awk '{print $NF}')${NC}" "${RED}$( echo $line | awk '{print $(NF-2)}')${NC}"
           fi
           prevLine=$line
       done

Seems to work with log-queries=extra on and off, unless there is another setting that can disable that too..

[EDIT]
Here are modified functions.div and install.div files with the following modifications:

• edit *list is faster now, it doesn't show list's content until user requested it via new option #8 which significantly speeds up the process with a long list
  
• most yes/no questions have "default" command marked with brackets: [1=[Yes] 2=No] which will be automatically selected when hit enter key without any input
• Set domain active/inactive function now shows the domain itself and allows exit without changing the line
• computer ip now shows in blocked domains log (follow dnsmasq.log -> 3)
• removed redundant prompt at Change pixelserv-tls IP address
• enter IP address function accepts second parameter for default IP which will be used if user hit enter without any input

 

[sanmibuh]

Hello, I have a problem when Im installing diversion with pixelserv-tls.
I have an "uncommon" local net configuration with AC86U router on 10.0.0.1 with net mask 255.255.0.0
I have my servers with reserved ips 10.0.0.x
My personal devices with reserved ips 10.0.1.x
My wife devices with reserved ips 10.0.2.x
My childs devices with reserved ips 10.0.3.x
DHCP devices by default on 10.0.4.x

So whe i was installing, I cannot set pixelserv-tls on 10.0.0.x (free) ip: "10.0.0.10 is not a router address"
I solve the problem configurinng the DHCP, instead of from 10.0.4.1, with 10.0.0.10.

Today I see that Amazon android app is not working. I want to see the log from my phone (10.0.1.1), but again the problem: 10.0.1.1 is not a router address.

I think the problem is that Diversion is assuming 255.255.255.0 mask. Why this limitation?

 

[adampk17]

Hello, I have a problem when Im installing diversion with pixelserv-tls.
I have an "uncommon" local net configuration with AC86U router on 10.0.0.1 with net mask 255.255.0.0
I have my servers with reserved ips 10.0.0.x
My personal devices with reserved ips 10.0.1.x
My wife devices with reserved ips 10.0.2.x
My childs devices with reserved ips 10.0.3.x
DHCP devices by default on 10.0.4.x

So whe i was installing, I cannot set pixelserv-tls on 10.0.0.x (free) ip: "10.0.0.10 is not a router address"
I solve the problem configurinng the DHCP, instead of from 10.0.4.1, with 10.0.0.10.

Today I see that Amazon android app is not working. I want to see the log from my phone (10.0.1.1), but again the problem: 10.0.1.1 is not a router address.

I think the problem is that Diversion is assuming 255.255.255.0 mask. Why this limitation?

I mean this with the absolute most respect possible but I think your IP scheme needs more thought. What you are doing is not how you IP scheme a home network even with as much control and separation as you seem to want between devices. Your IP scheme may be pleasing to the eye but it covers a crazy amount of ground for a home network and your subnet mask is off the chart. You have room for 65,534 hosts in there.

 

[sanmibuh]

I mean this with the absolute most respect possible but I think your IP scheme needs more thought. What you are doing is not how you IP scheme a home network even with as much control and separation as you seem to want between devices. Your IP scheme may be pleasing to the eye but it covers a crazy amount of ground for a home network and your subnet mask is off the chart. You have room for 65,534 hosts in there.

I know there is no real separation. All the devices are in the same subnet

Anyway, there is no performance difference, so I'm using this schema to organize them. Really in most home nets, c class subnets are too big. It is not normal having 254 devices.

But the the problem is that diversión asumes c class subnet from DHCP initial ip.

If this is a limitation, I could change my subnet.

Enviado desde mi CLT-L09 mediante Tapatalk

 

[Svenskmat]

Try whitelisting those i put above ^^^^

you will need to reprocess your whitelist , and refresh your DNS before testing.

How do you "refresh your DNS before testing on Android"

Thanks