What's new

Unbound unbound_manager (Manager/Installer utility for unbound - Recursive DNS Server)

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

Not for me. I get the following
"unbound NOT installed! or 'extended-stats:' template NOT defined in 'unbound.conf'?
As per the v2.14 release notes, you need 'unbound.conf' v1.06
 
AKA, use the “i” command to update unbound.conf.


@Martineau, can you confirm if the stunning, adblock and stats scripts are updated via “i”?
Yes.

Clearly v2.13/v2.14 demonstrates that it is possible to implement optional features without the need to retrieve ALL files, but currently it allows the script to pre-empt/recover from any potential corruption due to missing files/disk errors.

NOTE: Apparently unbound v1.10.x has been released, so when it is available on Entware it will be automatically installed. (Not sure if anyone has manually installed it?)

This may not be the best design, but the only exception will be in the next release (v2.15) where it would be inappropriate to retrieve ALL of your Ad Block files if they have been customised.

e.g.
Code:
Option Auto Reply 'y' Installing Ads and Tracker Blocking.....
 adblock/gen_adblock.sh downloaded successfully
Custom '/opt/share/unbound/configs/sites' already exists - 'adblock/sites' download skipped
Custom '/opt/share/unbound/configs/blockhost' already exists - 'adblock/blockhost' download skipped
Custom '/opt/share/unbound/configs/permlist' already exists - 'adblock/permlist' download skipped
Executing '/opt/var/lib/unbound/adblock/gen_adblock.sh'.....
Removing possible temporary files..
Processsing hosts file @ https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
######################################################################## 100.0%
Combining User Custom block host...
Edit User Custon list of allowed domains...
Removing duplicate formatting from the domain list...
51369 domains compiled
Generating Unbound adlist.....
Removing temporary files...
Restarting Unbound DNS server...
/opt/share/unbound/configs/cache.tmp
error: SSL handshake failed
547599556624:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915:
 Shutting down unbound...              done.
 Starting unbound...              done.
Adding Ad and Tracker 'include: /opt/var/lib/unbound/adblock/adservers'
Creating Daily cron job for Ad and Tracker update
 Shutting down unbound...              done.
 Starting unbound...              done.
The problem is, what needs to happen if you provide say a truly 'must-have' 'permlist' ?
 
Last edited:
Does the adblock feature of this script work separately/differently from diversion?
 
Does the adblock feature of this script work separately/differently from diversion?

From the Q&A in Post 4, this may go some way towards answering your question:

Q. Can I run unbound+dnsmasq+diversion together?
A. Yes. However, unbound+Ad Block+diversion is NOT recommended, simply because Ad Block and diversionessentially perform the same function so a duplication of effort is wasteful. Also, the domains must be stored in memory, so if you have both Ad Block and diversioninstalled (issue the 'ad'command to see how manyentries are in use) one set maysimply not be referenced butstill occupies memory.”
 
From the Q&A in Post 4, this may go some way towards answering your question:

Q. Can I run unbound+dnsmasq+diversion together?
A. Yes. However, unbound+Ad Block+diversion is NOT recommended, simply because Ad Block and diversionessentially perform the same function so a duplication of effort is wasteful. Also, the domains must be stored in memory, so if you have both Ad Block and diversioninstalled (issue the 'ad'command to see how manyentries are in use) one set maysimply not be referenced butstill occupies memory.”

A possible warning. With a very large Adblock list the unbound solution may not work. If you get errors where unbound won’t load when trying a new domain or host list then revert and run the gen_adblock.sh again. I was testing with the osid dbl list and it generates a 52mb adserver file which cannot be loaded by unbound.conf.
 
Yes.

Clearly v2.13/v2.14 demonstrates that it is possible to implement optional features without the need to retrieve ALL files, but currently it allows the script to pre-empt/recover from any potential corruption due to missing files/disk errors.

NOTE: Apparently unbound v1.10.x has been released, so when it is available on Entware it will be automatically installed. (Not sure if anyone has manually installed it?)

This may not be the best design, but the only exception will be in the next release (v2.15) where it would be inappropriate to retrieve ALL of your Ad Block files if they have been customised.

e.g.
Code:
Option Auto Reply 'y' Installing Ads and Tracker Blocking.....
 adblock/gen_adblock.sh downloaded successfully
Custom '/opt/share/unbound/configs/sites' already exists - 'adblock/sites' download skipped
Custom '/opt/share/unbound/configs/blockhost' already exists - 'adblock/blockhost' download skipped
Custom '/opt/share/unbound/configs/permlist' already exists - 'adblock/permlist' download skipped
Executing '/opt/var/lib/unbound/adblock/gen_adblock.sh'.....
Removing possible temporary files..
Processsing hosts file @ https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
######################################################################## 100.0%
Combining User Custom block host...
Edit User Custon list of allowed domains...
Removing duplicate formatting from the domain list...
51369 domains compiled
Generating Unbound adlist.....
Removing temporary files...
Restarting Unbound DNS server...
/opt/share/unbound/configs/cache.tmp
error: SSL handshake failed
547599556624:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915:
 Shutting down unbound...              done.
 Starting unbound...              done.
Adding Ad and Tracker 'include: /opt/var/lib/unbound/adblock/adservers'
Creating Daily cron job for Ad and Tracker update
 Shutting down unbound...              done.
 Starting unbound...              done.
The problem is, what needs to happen if you provide say a truly 'must-have' 'permlist' ?

Good point.

I could make permlist the one we update with a fixed list of allowable sites and keep it in the /opt/var/lib/unbound/adblock folder. Then could add a new allowhost file (along with blockhost and sites files) which would live in /opt/share/unbound/configs which would only be download if missing.

If I prepare that before you switch to my GitHub for download it would be a single time people would have re-enter their settings.


How hard is it to add in the file edit options for sites/allowhost/blockhost files from the menu like you do for unbound.conf? Easy way to edit the correct files.

I am going to update permlist as I see diversion has done an update so it matches.
 
the 'pauses' between displaying the menu items are due to the random delays when using the unbound-control utility (between 0.75-2.00 secs per call), so simply checking if unbound is ACTIVE, and if FULL logging (query/reply ) is enabled can take 5 or 6 seconds
Do you get better responsiveness if you set "control-use-cert: no" in unbound.conf under remote-control:? I get very fast response without using the certs, and there's not much point of securing unbound-control if the only allowed access is within the router itself.
Code:
remote-control:
  control-enable: yes
  control-use-cert: no
  control-interface: 127.0.0.1
 
Do you get better responsiveness if you set "control-use-cert: no" in unbound.conf under remote-control:? I get very fast response without using the certs, and there's not much point of securing unbound-control if the only allowed access is within the router itself.
Code:
remote-control:
  control-enable: yes
  control-use-cert: no
  control-interface: 127.0.0.1
Tangible improvement :D - so I have added 'fastmenu [disable]' command to unbound_manager (unreleased v2.15)
 
I could make permlist the one we update with a fixed list of allowable sites and keep it in the /opt/var/lib/unbound/adblock folder. Then could add a new allowhost file (along with blockhost and sites files) which would live in /opt/share/unbound/configs which would only be download if missing.

If I prepare that before you switch to my GitHub for download it would be a single time people would have re-enter their settings.
OK - Let me know when you are ready.
How hard is it to add in the file edit options for sites/allowhost/blockhost files from the menu?
24 man-hours later .... :p

Added the Ad Block file edit command (unreleased v2.15) with logical descriptive names (can be changed later if inappropriate)

e.g. 'ew = Edit Ad Block Whitelist (eb=Blacklist; ec=Config; el {Ad Block file})'

...under Advanced Tools
Code:
e  = Exit Script

A:Option ==> 3

unbound (pid 3080) is running... uptime: 0 Days, 00:22:44 version: 1.9.6 # rgnldo Github Version=v1.06 Martineau update (Date Loaded by unbound_manager Mon Mar 2 13:09:29 GMT 2020)

u = Push to Github PENDING for (Major) unbound_manager UPDATE v2.15 >>>> v2.14

i  = Update unbound Installation ('/opt/var/lib/unbound/')      l  = Show unbound log entries (lo=Enable Logging)
z  = Remove unbound/unbound_manager Installation                v  = View ('/opt/var/lib/unbound/') unbound Configuration (vx=Edit)
x  = Stop unbound                                               vb = Backup current (/opt/var/lib/unbound/unbound.conf) Configuration
                                                                rl = Reload Configuration (Doesn't halt unbound) e.g. 'rl test1[.conf]' (Recovery use 'rl reset/user')
?  = About Configuration                                        oq = Query unbound Configuration option e.g 'oq verbosity' (ox=Set) e.g. 'ox log-queries yes'
sd = Show dnsmasq Statistics/Cache Size                         s  = Show unbound Extended statistics (s=Summary Totals; sa=All; s-=Disable Extended Stats)
                                                                fastmenu = Disable SLOW unbound-control LAN SSL cert validation
scribe = Enable scribe (syslog-ng) unbound logging              ad = Analyse Diversion White/Black lists ([ file_name [type=adblock] ])
                                                                ew = Edit Ad Block Whitelist (eb=Blacklist; ec=Config; el {Ad Block file})
dumpcache = Manually use restorecache after REBOOT              ca = Cache Size Optimisation  ([ 'reset' ])
dig = {domain} Show dig info e.g. dig qnamemintest.internet.nl  lookup = {domain} Show the name servers used for domain e.g. lookup asciiart.eu
dnsinfo = {dns} Show DNS Server e.g. dnsinfo                    dnssec = {url} Show DNSSEC Validation Chain e.g. dnssec www.snbforums.com
links = Show list of external URL links

e  = Exit Script

[Enter] Leave Advanced Tools Menu
 
Last edited:
A possible warning. With a very large Adblock list the unbound solution may not work. If you get errors where unbound won’t load when trying a new domain or host list then revert and run the gen_adblock.sh again. I was testing with the osid dbl list and it generates a 52mb adserver file which cannot be loaded by unbound.conf.
I had this problem with testing but issue went away when I split it and listed it as two separatist instead of one list,for any one who has to go this route.
 
OK - Let me know when you are ready.

24 man-hours later .... :p

Added the Ad Block file edit command (unreleased v2.15) with logical descriptive names (can be changed later if inappropriate)

e.g. 'ew = Edit Ad Block Whitelist (eb=Blacklist; ec=Config; el {Ad Block file})'

...under Advanced Tools
Code:
e  = Exit Script

A:Option ==> 3

unbound (pid 3080) is running... uptime: 0 Days, 00:22:44 version: 1.9.6 # rgnldo Github Version=v1.06 Martineau update (Date Loaded by unbound_manager Mon Mar 2 13:09:29 GMT 2020)

u = Push to Github PENDING for (Major) unbound_manager UPDATE v2.15 >>>> v2.14

i  = Update unbound Installation ('/opt/var/lib/unbound/')      l  = Show unbound log entries (lo=Enable Logging)
z  = Remove unbound/unbound_manager Installation                v  = View ('/opt/var/lib/unbound/') unbound Configuration (vx=Edit)
x  = Stop unbound                                               vb = Backup current (/opt/var/lib/unbound/unbound.conf) Configuration
                                                                rl = Reload Configuration (Doesn't halt unbound) e.g. 'rl test1[.conf]' (Recovery use 'rl reset/user')
?  = About Configuration                                        oq = Query unbound Configuration option e.g 'oq verbosity' (ox=Set) e.g. 'ox log-queries yes'
sd = Show dnsmasq Statistics/Cache Size                         s  = Show unbound Extended statistics (s=Summary Totals; sa=All; s-=Disable Extended Stats)
                                                                fastmenu = Disable SLOW unbound-control LAN SSL cert validation
scribe = Enable scribe (syslog-ng) unbound logging              ad = Analyse Diversion White/Black lists ([ file_name [type=adblock] ])
                                                                ew = Edit Ad Block Whitelist (eb=Blacklist; ec=Config; el {Ad Block file})
dumpcache = Manually use restorecache after REBOOT              ca = Cache Size Optimisation  ([ 'reset' ])
dig = {domain} Show dig info e.g. dig qnamemintest.internet.nl  lookup = {domain} Show the name servers used for domain e.g. lookup asciiart.eu
dnsinfo = {dns} Show DNS Server e.g. dnsinfo                    dnssec = {url} Show DNSSEC Validation Chain e.g. dnssec www.snbforums.com
links = Show list of external URL links

e  = Exit Script

[Enter] Leave Advanced Tools Menu

Thank you! It is hard not to think about the next change and the next tweak once you start. Focusing on next update to graphing.
 
The GUI tab has received graphs!

3 new graph types. You should be able to type:
Code:
A:Option ==> sgui <enter>
To update to the latest version (v1.1.0)

Graphs include a running line graph of your cache hit percentage (aka, how well are you using your cache).
upload_2020-3-3_2-54-3.png


Second graph covers a histogram (extended stats needs to be enabled) for how quickly unbound is responding to requests:
upload_2020-3-3_2-56-51.png


Third graph shows what answer codes have been returned, good to see how many NOERROR and how many NXDOMAIN if you have adblock enabled:
upload_2020-3-3_2-57-49.png



Of course you still have the text stats:
upload_2020-3-3_2-58-12.png


I hope this works for people. It will require at least one other graph tab installed to get the shared-jy folder and files in place. @Jack Yaz any recommendations on how to best install those?
 

Attachments

  • upload_2020-3-3_2-56-7.png
    upload_2020-3-3_2-56-7.png
    225.4 KB · Views: 150
Last edited:
The GUI tab has recived graphs!

3 new graph types. You should be able to type:
Code:
A:Option ==> sgui <enter>
To update to the latest version (v1.1.0)

Graphs include a running line graph of your cache hit percentage (aka, how well are you using your cache).
View attachment 21723

Second graph covers a histogram (extended stats needs to be enabled) for how quickly unbound is responding to requests:
View attachment 21725

Third graph shows what answer codes have been returned, good to see how many NOERROR and how many NXDOMAIN if you have adblock enabled:
View attachment 21726


Of course you still have the text stats:
View attachment 21727

I hope this works for people.
oooooo gimme gimme. :p
Great job!
 
Cool!
Can you please elaborate the second graph?
What is the x and y axis representing?
 
Cool!
Can you please elaborate the second graph? What is the x and y axis represent?

This graph is essentially the built in histogram stats unbound has under extended stats. It shows the number of DNS requests which returned in that time range. So each bar is a bucket of a time range (ie. 32ms to 64ms) and the count is how many answers were returned in that range.

Good to see how your personal DNS server is performing.
 
Works and looks great on my RT-86U with firmware 384.16 alpha1

Many thanks [emoji4]


Sent from my iPad using Tapatalk
 
The GUI tab has received graphs!

3 new graph types. You should be able to type:
Code:
A:Option ==> sgui <enter>
To update to the latest version (v1.1.0)

Graphs include a running line graph of your cache hit percentage (aka, how well are you using your cache).
View attachment 21723

Second graph covers a histogram (extended stats needs to be enabled) for how quickly unbound is responding to requests:
View attachment 21725

Third graph shows what answer codes have been returned, good to see how many NOERROR and how many NXDOMAIN if you have adblock enabled:
View attachment 21726


Of course you still have the text stats:
View attachment 21727

I hope this works for people. It will require at least one other graph tab installed to get the shared-jy folder and files in place. @Jack Yaz any recommendations on how to best install those?
Since you're already practically re-using my code verbatim :p, I'm sure you can borrow the update_file, create_dirs and create_symlinks to maintain the shared-jy. Do note if you do want to use shared-jy it is imperative you add any additional files I may choose to add, otherwise your implementation could break any/all of my scripts.
 
Last edited:

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top