Unbound unbound_manager (Manager/Installer utility for unbound - Recursive DNS Server)

juched · May 8, 2020

I've uploaded v3.11Beta on GitHub dev branch for those that want to try it

Code:

e  = Exit Script [?]

A:Option ==> uf dev

A:Option ==> adblock

View attachment 23327

Having visited YouTube, rather than wait 5 mins ('cos you are too excited!

), you can force the scanning.....

Code:

A:Option ==> adblock update

Updating Ads and Tracker Blocking.....
                        
 _____   _ _   _         _
|  _  |_| | |_| |___ ___| |_
|     | . | . | | . |  _| '_|
|__|__|___|___|_|___|___|_,_|
(gen_adblock.sh): 32637 @juched - v1.0.6 - Thanks to @SomeWhereOverTheRainBow

Removing possible temporary files..
Downloading list(s) from block site(s) configured...
Attempting to Download 1 of 3 from https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts.
######################################################################## 100.0%
Attempting to Download 4 of 3 from https://raw.githubusercontent.com/llacb47/mischosts/master/tiktok-hosts.
######################################################################## 100.0%
Attempting to Download 7 of 3 from https://raw.githubusercontent.com/mitchellkrogza/The-Big-List-of-Hacked-Malware-Web-Sites/master/hacked-domains.list.
######################################################################## 100.0%
Downloading list(s) from allow site(s) configured...
Adding user requested hosts to list...
Removing user requested hosts from list...
Removing required hosts from list...
Removing unnecessary formatting from the domain list...
Generating Unbound adservers file...
(gen_adblock.sh): 32637 Number of adblocked hosts: 63968
Generating Unbound unload/load lists...
Loading/Unload Unbound local-zones to take effect...
removed 63968 zones
added 63968 zones
Removing temporary files...
Adblock update complete!
Updating YouTube Video Ad Blocking.....
Y88b   d88P 88888888888     d8888      888 888888b.   888                   888
 Y88b d88P      888        d88888      888 888  "88b  888                   888
  Y88o88P       888       d88P888      888 888  .88P  888                   888
   Y888P        888      d88P 888  .d88888 8888888K.  888  .d88b.   .d8888b 888  888
    888         888     d88P  888 d88" 888 888  "Y88b 888 d88""88b d88P"    888 .88P
    888         888    d88P   888 888  888 888    888 888 888  888 888      888888K
    888         888   d8888888888 Y88b 888 888   d88P 888 Y88..88P Y88b.    888 "88b
    888         888  d88P     888  "Y88888 8888888P"  888  "Y88P"   "Y8888P 888  888
## by @juched - dynamically block YT ads - v1.0                
gen_ytadblock.sh
Forcing to use YT IP
Generating Unbound yt adblock list...
(gen_ytadblock.sh): 388 Number of yt adblocked domains: 4
Loading/Unload Unbound local-data to take effect...
added 4 datas
All done updating YT hosts!

Ok,I need to fix the empty file detection. Your log shows that ipYTforce file didn’t get populated correctly since you didn’t have any YouTube hits in your cache when first run and likely the file is empty.

Can you erase that file manually and run it again? It should write out an IP on screen.

--- edit ---

Pushed fix to dev branch. Had a wrong bracket so file detection didn't work. run again and it should pick a force IP if needed, even if file exists and is empty. V1.1.

juched · May 8, 2020

tomsk said:
I understand that your script creates a list of domains and loads them as local data...... but i don't understand how unbound knows to block them. For the ad blocking you have a local zone and for each match you tell what to do with it e.g

Code:

local-zone: "017gj.com" always_nxdomain local-zone: "01apple.com" always_nxdomain local-zone: "01mspmd5yalky8.com" always_nxdomain

care to explain in simple terms? just trying to understand the mechanism ....... thanks

It doesn't block them, but forces all fo them to use the same IP. From what I read on the original thread for pi-hole users, there seems to be a correlation to changing domains and a new IP to new ads showing up. Now, only YT really knows, but I would guess that the "show ad profile curve" (as they called in that thread) is somehow based to the machine you are using, so to slow down the ads over time. But when the domain/IP changes, it starts up again (likely as YT wants to it), but by forcing to use the same IP here, you stick to one server so ads basically stop.

Just a guess, based on results seen, but not based on any knowledge of how they designed it.

I am also a bit concerned if I get stuck on 1 IP, what happens when that machine goes down or maintenance.... this is why it is experimental, just following what that thread suggested. Lets see. easy to turn off.

Martineau · May 8, 2020

juched said:
Ok,I need to fix the empty file detection. Your log shows that ipYTforce file didn’t get populated correctly since you didn’t have any YouTube hits in your cache when first run and likely the file is empty.

Can you erase that file manually and run it again? It should write out an IP on screen.

--- edit ---

Pushed fix to dev branch. Had a wrong bracket so file detection didn't work. run again and it should pick a force IP if needed, even if file exists and is empty. V1.1.

I would have thought you would have seen the highlighted error back in my post #

Code:

A:Option ==> adblock

Option Auto Reply 'y'    Installing Ads and Tracker and YouTube Video Ad Blocking.....
    adblock/gen_adblock.sh downloaded successfully
    adblock/permlist downloaded successfully
    adblock/gen_ytadblock.sh downloaded successfully Github 'dev/development' branch
Custom '/opt/share/unbound/configs/blocksites' already exists - 'adblock/blocksites' download skipped
Custom '/opt/share/unbound/configs/allowsites' already exists - 'adblock/allowsites' download skipped
Custom '/opt/share/unbound/configs/blockhost' already exists - 'adblock/blockhost' download skipped
Custom '/opt/share/unbound/configs/allowhost' already exists - 'adblock/allowhost' download skipped
Adding Ad and Tracker blocker (Ad Block)'include: /opt/var/lib/unbound/adblock/adservers'
Creating Daily cron job for YouTube Ad Tracker update
Creating Daily cron job for Ad and Tracker update
Executing '/opt/var/lib/unbound/adblock/gen_ytadblock.sh'.....

Y88b   d88P 88888888888     d8888      888 888888b.   888                   888
 Y88b d88P      888        d88888      888 888  "88b  888                   888
  Y88o88P       888       d88P888      888 888  .88P  888                   888
   Y888P        888      d88P 888  .d88888 8888888K.  888  .d88b.   .d8888b 888  888
    888         888     d88P  888 d88" 888 888  "Y88b 888 d88""88b d88P"    888 .88P
    888         888    d88P   888 888  888 888    888 888 888  888 888      888888K
    888         888   d8888888888 Y88b 888 888   d88P 888 Y88..88P Y88b.    888 "88b
    888         888  d88P     888  "Y88888 8888888P"  888  "Y88P"   "Y8888P 888  888
## by @juched - dynamically block YT ads - v1.1                   

gen_ytadblock.sh
No stored IP in file /opt/share/unbound/configs/ipytforce, checking cache for an ip...
Forcing to use YT IP 62.24.208.16
Generating Unbound yt adblock list...
(gen_ytadblock.sh): 6481 Number of yt adblocked domains: 3

Loading/Unload Unbound local-data to take effect...
added 3 datas
All done updating YT hosts!

glehel · May 8, 2020

dnsmasq disable error message
what a problem?
[1588942988] unbound-checkconf[3385:0] error: error parsing local-data at 2 '.AC86U. IN A 192.168.1.106': Empty label
[1588942988] unbound-checkconf[3385:0] error: Bad local-data RR .AC86U. IN A 192.168.1.106
[1588942988] unbound-checkconf[3385:0] fatal error: failed local-zone, local-data configuration

Milan · May 8, 2020

glehel said:
dnsmasq disable error message
what a problem?
[1588942988] unbound-checkconf[3385:0] error: error parsing local-data at 2 '.AC86U. IN A 192.168.1.106': Empty label
[1588942988] unbound-checkconf[3385:0] error: Bad local-data RR .AC86U. IN A 192.168.1.106
[1588942988] unbound-checkconf[3385:0] fatal error: failed local-zone, local-data configuration

you need to populate domain name in LAN GUI - had same problem.

Martineau · May 8, 2020

glehel said:
dnsmasq disable error message
what a problem?
[1588942988] unbound-checkconf[3385:0] error: error parsing local-data at 2 '.AC86U. IN A 192.168.1.106': Empty label
[1588942988] unbound-checkconf[3385:0] error: Bad local-data RR .AC86U. IN A 192.168.1.106
[1588942988] unbound-checkconf[3385:0] fatal error: failed local-zone, local-data configuration

- Strange this was already patched with this Hotfix commit

Code:

Warning: Cannot replicate dnsmasq's local hosts; Blank router domain name; see $HTTP_TYPE://$(nvram get lan_ipaddr):$HTTP_PORT/Advanced_LAN_Content.asp LAN->LAN-IP $HARDWARE_MODEL's Domain Name

glehel · May 8, 2020

the domain name is filled out

juched · May 8, 2020

Martineau said:

I would have thought you would have seen the highlighted error back in my post #

Code:

A:Option ==> adblock

Option Auto Reply 'y'    Installing Ads and Tracker and YouTube Video Ad Blocking.....
    adblock/gen_adblock.sh downloaded successfully
    adblock/permlist downloaded successfully
    adblock/gen_ytadblock.sh downloaded successfully Github 'dev/development' branch
Custom '/opt/share/unbound/configs/blocksites' already exists - 'adblock/blocksites' download skipped
Custom '/opt/share/unbound/configs/allowsites' already exists - 'adblock/allowsites' download skipped
Custom '/opt/share/unbound/configs/blockhost' already exists - 'adblock/blockhost' download skipped
Custom '/opt/share/unbound/configs/allowhost' already exists - 'adblock/allowhost' download skipped
Adding Ad and Tracker blocker (Ad Block)'include: /opt/var/lib/unbound/adblock/adservers'
Creating Daily cron job for YouTube Ad Tracker update
Creating Daily cron job for Ad and Tracker update
Executing '/opt/var/lib/unbound/adblock/gen_ytadblock.sh'.....

Y88b   d88P 88888888888     d8888      888 888888b.   888                   888
 Y88b d88P      888        d88888      888 888  "88b  888                   888
  Y88o88P       888       d88P888      888 888  .88P  888                   888
   Y888P        888      d88P 888  .d88888 8888888K.  888  .d88b.   .d8888b 888  888
    888         888     d88P  888 d88" 888 888  "Y88b 888 d88""88b d88P"    888 .88P
    888         888    d88P   888 888  888 888    888 888 888  888 888      888888K
    888         888   d8888888888 Y88b 888 888   d88P 888 Y88..88P Y88b.    888 "88b
    888         888  d88P     888  "Y88888 8888888P"  888  "Y88P"   "Y8888P 888  888
## by @juched - dynamically block YT ads - v1.1                  

gen_ytadblock.sh
No stored IP in file /opt/share/unbound/configs/ipytforce, checking cache for an ip...
Forcing to use YT IP 62.24.208.16
Generating Unbound yt adblock list...
(gen_ytadblock.sh): 6481 Number of yt adblocked domains: 3

Loading/Unload Unbound local-data to take effect...
added 3 datas
All done updating YT hosts!

yes, I did.

Martineau · May 8, 2020

glehel said:
the domain name is filled out

So it's probably the '/opt/share/unbound/configs/unbound.conf.localhosts' than contains an invalid entry for the .106 device ?

If unbound won't start, then delete the 2 lines in 'unbound.conf'

Code:

server:
include: "/opt/share/unbound/configs/unbound.conf.localhosts"           # Custom server directives

or

Code:

true > "/opt/share/unbound/configs/unbound.conf.localhosts"

I suspect there is an issue with the NVRAM variables

Code:

nvram get dhcp_staticlist

nvram get dhcp_hostnames

i.e. 'unbound_manager' tries for each matching pair ' MAC Addresss>IP Address' found in 'dhcp_staticlist' to lookup the matching MAC Address in 'dhcp_hostnames' to resolve the name.

Without having a copy of both your NVRAM variables I'm not sure if the format is different or perhaps you manually create entries in '/etc/dnsmasq.conf ' via '/jffs/configs/dnsmasq.conf.add'?

EDIT: Uploaded v3.11Beta 3 to Github dev branch

Code:

e  = Exit Script [?]

A:Option ==> uf dev

i.e. Ignore mismatch (no name) MAC entries when generating 'unbound.conf.localhosts'

juched · May 8, 2020

Martineau said:
i.e. 'unbound_manager' tries for each matching pair ' MAC Addresss>IP Address' found in 'dhcp_staticlist' to lookup the matching MAC Address in 'dhcp_hostnames' to resolve the name.

I moved a while ago to using dnsmasq.conf.add to specify my DHCP reservations, and also provide an included name. Could unbound_manager support importing those as well?

Code:

dhcp-host=XX:XX:XX:XX:XX:XX,192.168.x.X,Hostname

Martineau · May 8, 2020

juched said:
I moved a while ago to using dnsmasq.conf.add to specify my DHCP reservations, and also provide an included name. Could unbound_manager support importing those as well?

Code:

dhcp-host=XX:XX:XX:XX:XX:XX,192.168.x.X,Hostname

Yeah most would be using NVRAM, but I suppose I should ONLY use '/etc/dnsmasq.conf' as they end up there anyway.

glehel · May 8, 2020

Warning: 00:00:EE:3A:A8:63 (192.168.1.101) not found in 'nvram show dhcp_hostnames'
Warning: 0E:06:BB:85:08

F (192.168.1.102) not found in 'nvram show dhcp_hostnames'
Warning: 3E:FF:2D:4F:5F:FC (192.168.1.103) not found in 'nvram show dhcp_hostnames'

etc/dnsmasq.conf not show hostname
hostname found etc/hosts.dnsmasq and jffs/nvram

tomsk · May 8, 2020

juched said:
It doesn't block them, but forces all fo them to use the same IP. From what I read on the original thread for pi-hole users, there seems to be a correlation to changing domains and a new IP to new ads showing up. Now, only YT really knows, but I would guess that the "show ad profile curve" (as they called in that thread) is somehow based to the machine you are using, so to slow down the ads over time. But when the domain/IP changes, it starts up again (likely as YT wants to it), but by forcing to use the same IP here, you stick to one server so ads basically stop.

Just a guess, based on results seen, but not based on any knowledge of how they designed it.

I am also a bit concerned if I get stuck on 1 IP, what happens when that machine goes down or maintenance.... this is why it is experimental, just following what that thread suggested. Lets see. easy to turn off.

Code:

r1.sn-4wg7ln7d.googlevideo.com. IN A 74.125.167.121
r1.sn-4wg7ln7k.googlevideo.com. IN A 74.125.167.121
r1.sn-hgn7rn7y.googlevideo.com. IN A 74.125.167.121
r1.sn-hgn7rne7.googlevideo.com. IN A 74.125.167.121
r2.sn-4wg7ln76.googlevideo.com. IN A 74.125.167.121
r2.sn-4wg7ln7d.googlevideo.com. IN A 74.125.167.121
r2.sn-4wg7ln7k.googlevideo.com. IN A 74.125.167.121

Ok i get the idea now .... now the script is fixed its adding the same IP for every record

juched · May 8, 2020

Martineau said:
I've uploaded v3.11Beta on GitHub dev branch for those that want to try it

I see code to cleanup the cron job inside the uninstall, but no code to add it in the first place. Without this will not be applied after restart, and will loose the ability to discover new YT domains.

Code:

            [ ! -f /jffs/scripts/services-start ] && { echo "#!/bin/sh" > $FN; chmod +x $FN; }
            if [ -z "$(grep -E "gen_adblock" /jffs/scripts/services-start | grep -v "^#")" ];then
                $(Smart_LineInsert "$FN" "$(echo -e "cru a adblock \"0 5 * * *\" ${CONFIG_DIR}adblock/gen_adblock.sh\t# unbound_manager")" )  # v1.13
                $(Smart_LineInsert "$FN" "$(echo -e "cru a ytadblock \"*/5 * * * *\" ${CONFIG_DIR}adblock/gen_ytadblock.sh\t# unbound_manager")" )
            fi

Also, a couple lines up you are missing one portion of the cron, needs 4 *.
cru a ytadblock "*/5 * * * *" ${CONFIG_DIR}adblock/gen_ytadblock.sh

ugandy · May 8, 2020

sorry if I missed it, but if i enable adblock in unbound, what block list(s) is used?
thanks

tomsk · May 8, 2020

ugandy said:
sorry if I missed it, but if i enable adblock in unbound, what block list(s) is used?
thanks

It downloads the Steven black master list
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

ugandy · May 8, 2020

tomsk said:
It downloads the Steven black master list
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

ah. so we get the same protection as diversion, with a simplified setup. but no pixelserv

tomsk · May 8, 2020

ugandy said:
ah. so we get the same protection as diversion, with a simplified setup. nice!

Yes fairly close depending on that else you might have bolted onto diversion

Code:

A:Option ==> ad

Analysed Diversion file: 'blockinglist'     Type=pixelserv, (Adblock Domains=55204) would add 620 entries
Analysed Diversion file: 'blacklist'     Type=pixelserv, (Adblock Domains=55204) would add 2 entries
Analysed Diversion file: 'whitelist'     Type=URL, (Adblock URLs=19) would add 22 entries

I guess the ultimate if you wanted identical lists would to be to import them from diversion... the code is in the script to do the dnsmasq to unbound type lists conversion already ( the steven black one downloads in dnsmasq form and needs conversion)

ARKASHA · May 8, 2020

Martineau said:
I don't understand this statement

unbound+dnsmasq combination is proven to be very stable and reliable. Can you explain why you say 'dnsmasq disable' fixed a non-working unbound?

NOTE: Whilst using unbound as the primary DNS server for your LAN (bypassing dnsmasq) does work, it is still an experimental feature. Use at your own risk

After upgrading to 384.17 adjective installing unbound, although rebooting multiple times I verified 0 hits in unbound statistics, more ipleak. net showed the Dns' from Quad9 and not my provider IP. So I disabled dnsmasq and now it works (with Adblock)

Inviato dal mio ONEPLUS A6003 utilizzando Tapatalk

ttgapers · May 8, 2020

QQ: If I use Unbound exclusively (bypass dnsmasq), is it safe/recommended to change the following from Tools>Other Settings to YES?

Wan: Use local caching DNS server as system resolver (default: No)

Looks like that would ensure ALL DNS requests are processed by Unbound. Am I correct in this line of thinking?

Any impacts to be aware of?

Thread starter	Title	Forum	Replies	Date
U	Unbound Help setting up Unbound.	Asuswrt-Merlin AddOns	2	Apr 4, 2024
B	Knot Resolver vs unbound?	Asuswrt-Merlin AddOns	1	Mar 17, 2024
V	unbound blocking apple private relay.	Asuswrt-Merlin AddOns	1	Mar 16, 2024
B	Unbound unbound/WAN settings	Asuswrt-Merlin AddOns	31	Mar 5, 2024
	Solved Latest entware update borked up Unbound	Asuswrt-Merlin AddOns	25	Feb 24, 2024
Q	Problems unbound reverse proxy	Asuswrt-Merlin AddOns	2	Jan 14, 2024
M	Unbound Show local names in unbound log requests	Asuswrt-Merlin AddOns	5	Nov 14, 2023
J	Unbound Configure unbound with multiple VPNs	Asuswrt-Merlin AddOns	7	Nov 13, 2023
I	Unbound [SOLVED] Unbound not resolving anything	Asuswrt-Merlin AddOns	40	Oct 19, 2023
S	Unbound Unbound graphical statistics tab doesn't update all groups	Asuswrt-Merlin AddOns	4	Oct 3, 2023

Unbound unbound_manager (Manager/Installer utility for unbound - Recursive DNS Server)

Very Senior Member

Very Senior Member

Part of the Furniture

Regular Contributor

Senior Member

Part of the Furniture

Regular Contributor

Very Senior Member

Part of the Furniture

Very Senior Member

Part of the Furniture

Regular Contributor

Very Senior Member

Very Senior Member

Very Senior Member

Very Senior Member

Very Senior Member

Very Senior Member

Occasional Visitor

Senior Member

Similar threads

Similar threads

Sign Up For SNBForums Daily Digest