Unbound unbound_manager (Manager/Installer utility for unbound - Recursive DNS Server)

[Treadler]

[ "x$var2" == "x" ]

or [ -z "$2"] may bet better for
if [ "$2" != "" ]

and i don't understand why "$1" isn't getting passed off properly.


"$1" is representing host or domains or whitelist whatever you have defined it as.
"$1" "$2"
hosts url
domains url
whitelist-domains url

in the sites file

Im not seeing any error messages, everything looks fine, but, anything I’ve changed in the “sites” folder is removed & everything returned to the default.

 

[juched]

@Jack Yaz @Martineau I have submitted a pull request to add the base functionality and text based stats for Unbound WebUI.

@Jack Yaz I heavily sourced code and inspiration from you... hope that is ok?
https://github.com/jackyaz/Unbound-Asuswrt-Merlin/pull/6

@Martineau I noticed that when gen_adblock.sh is run, the cache is destroyed. This means that for people who use adblock that their build of of cache is gone. Any suggestions on how we can avoid this? Also noticed that on adblock restarting unbound that extended statistics isn't on anymore.

Thank you. Next to see about a couple graph ideas. That may take some time.

 

[Jack Yaz]

@Jack Yaz @Martineau I have submitted a pull request to add the base functionality and text based stats for Unbound WebUI.

@Jack Yaz I heavily sourced code and inspiration from you... hope that is ok?
https://github.com/jackyaz/Unbound-Asuswrt-Merlin/pull/6


View attachment 21680

@Martineau I noticed that when gen_adblock.sh is run, the cache is destroyed. This means that for people who use adblock that their build of of cache is gone. Any suggestions on how we can avoid this? Also noticed that on adblock restarting unbound that extended statistics isn't on anymore.

Thank you. Next to see about a couple graph ideas. That may take some time.

I have several suggestions for improvements, which I will comment on the Github PR.

 

[Martineau]

@Martineauwhen gen_adblock.sh is run, the cache is destroyed. This means that for people who use adblock that their build of of cache is gone.

Any suggestions on how we can avoid this?

If unbound is UP, then save the cache and after the unbound restart issue the cache restore request?

e.g. unbound_manager uses the following unbound restart save/restore logic so you could replicate the code into 'gen_adblock.sh'

Alternatively I could add an additional unbound_manager option similar to the existing 'install' / 'recovery' commandline directives

e.g.

unbound_manager restart

so you would simply call unbound_manager to manage the cache and initiate the unbound restart?

Also noticed that on adblock restarting unbound that extended statistics isn't on anymore.

The original idea was to dynamically enable unbound-control directives that are, as stated in the documentation:

detrimental to the performance of unbound.

However, given the obsession with GUI stats/graphs, I can push a modified unbound_manager/ 'unbound.conf' to allow the setting of 'extended-stats: yes' to be retained over unbound restarts.

P.S. Not sure if the forum consensus is that 'extended-stats: yes' should now always be enabled by default?

 

[juched]

If unbound is UP, then save the cache and after the unbound restart issue the cache restore request?

e.g. unbound_manager uses the following unbound restart save/restore logic so you could replicate the code into 'gen_adblock.sh'

Alternatively I could add an additional unbound_manager option similar to the existing 'install' / 'recovery' commandline directives

e.g.

unbound_manager restart

so you would simply call unbound_manager to manage the cache and initiate the unbound restart?

The original idea was to dynamically enable unbound-control directives that are, as stated in the documentation:

View attachment 21681

detrimental to the performance of unbound.

However, given the obsession with GUI stats/graphs, I can push a modified unbound_manager/ 'unbound.conf' to allow the setting of 'extended-stats: yes' to be retained over unbound restarts.

P.S. Not sure if the forum consensus is that 'extended-stats: yes' should now always be enabled by default?

A command line method means I don’t need to keep the script updated when unbound_manager improves in the future. So we can restart without loosing cache would be excellent.

On the second point, defaulting to extended stats off is likely fine, but if it could stick once you turn it on that would be great.

Not sure the performance is hugely impacted. Didn’t read that anywhere as the stats are in memory. Logging definitely th warn over and over it has a big impact on speed. This is what makes it unlikely to ever have stats like diversion does with dnsmasq logs enabled.

 

[Martineau]

Ok, here’s where I approach (once again), showing my lack of knowledge.

I have gone to the relevant folder (sites) using winscp, & amended & saved the existing (Steven Black) list.
Then run ‘i’ to update Unbound, & my change gets wiped & put back to the default.
So, 2nd attempt, left the default list there, & added the additional Steven Black list I would like to use.
Run ‘i’, everything put back to default.
So, both my amendments & changes were ignored.

What glaringly obvious step am I missing to alter my ad blocking list?

Perhaps @juched should move the 'sites' file to '/opt/share/unbound/configs' ?

 

[rgnldo]

Not sure the performance is hugely impacted. Didn’t read that anywhere as the stats are in memory. Logging definitely th warn over and over it has a big impact on speed. This is what makes it unlikely to ever have stats like diversion does with dnsmasq logs enabled.

If the goal is security and privacy, is long or reused caching interesting? There is a high chance of generating errors.

Bigger isn't always better

 

[rgnldo]

I usually organize these routines, via cron:

#!/bin/sh
echo "Clean cache NODATA and crash zones..."
unbound-control flush_negative
unbound-control flush_bogus

 

[rgnldo]

It is a matter of choices. Choosing to manage statistics via unbound-control, just disable the syslog. Performance even improves, better for the unbound.

verbosity: 0
statistics-interval: 0
extended-statistics: yes
statistics-cumulative: no

 

[juched]

Perhaps @juched should move the 'sites' file to '/opt/share/unbound/configs' ?

Good idea. I added that change to the latest pull request.

My thought was that if you like this could be added as another installable option to the unbound_manager.sh script. If the user wants WebGUI they pick option 6 during install.

My script doesn't do any download or update, etc. That is something your script does already. All you would need to do is download to /jffs/addons/unbound and run unbound_stats.sh install.

You also would need to download the asp file. Just those two.

 

[Treadler]

If unbound is UP, then save the cache and after the unbound restart issue the cache restore request?

e.g. unbound_manager uses the following unbound restart save/restore logic so you could replicate the code into 'gen_adblock.sh'

Alternatively I could add an additional unbound_manager option similar to the existing 'install' / 'recovery' commandline directives

e.g.

unbound_manager restart

so you would simply call unbound_manager to manage the cache and initiate the unbound restart?

The original idea was to dynamically enable unbound-control directives that are, as stated in the documentation:

View attachment 21681

detrimental to the performance of unbound.

However, given the obsession with GUI stats/graphs, I can push a modified unbound_manager/ 'unbound.conf' to allow the setting of 'extended-stats: yes' to be retained over unbound restarts.

P.S. Not sure if the forum consensus is that 'extended-stats: yes' should now always be enabled by default?

Re stats, my pick would be just to leave them at the default.
Those that want the extra info can easily opt for it.

Whatever is manually selected then surviving a restart makes sense to me.....

 

[JSewell]

If the goal is security and privacy, is long or reused caching interesting? There is a high chance of generating errors.

No, there isn't a high chance of generating errors with a reloaded cache under the circumstances described. The point of setting 'prefetch: yes' is that unbound keeps the cache valid in advance of future requests from clients.

Problems may occur if you reload a cache saved a while ago or if you have increased the value of 'cache-min-ttl' to a significant degree.

The privacy aspect, though, might be a concern for some. If the saved cache is deleted after the restart, then it's probably not an issue.

Bigger isn't always better

Which is why 'extended-statistics: yes' is useful, as it shows the actual size of the caches being used. You can then tweak your config as appropriate.

 

[rgnldo]

No, there isn't a high chance of generating errors with a reloaded cache under the circumstances described. The point of setting 'prefetch: yes' is that unbound keeps the cache valid in advance of future requests from clients.

Problems may occur if you reload a cache saved a while ago or if you have increased the value of 'cache-min-ttl' to a significant degree.

The privacy aspect, though, might be a concern for some. If the saved cache is deleted after the restart, then it's probably not an issue.



Which is why 'extended-statistics: yes' is useful, as it shows the actual size of the caches being used. You can then tweak your config as appropriate.

Smoothly. If you check, in the default unbound.conf, I worried about the cache lifetime. Consider that the cache should be understood as a temporary mechanism, not as an end. DNS cache covers not only Unbound, clients on the LAN. If you want to avoid DNS poisoning and other typical DNS server problems, avoid reusing DNS cache.

 

[Martineau]

If the saved cache is deleted after the restart, then it's probably not an issue.

unbound_manager deletes the cache file immediately it is restored, so for a 'rs' (unbound restart) command the cache file exists for a second or so.

However the recently feature to allow manually preserving the cache over a REBOOT, potentially could mean that the file is exposed or may even be completely out-of-date at the point of restore.

 

[juched]

Perhaps @juched should move the 'sites' file to '/opt/share/unbound/configs' ?

Perhaps permlist and blacklist too?

 

[Deleted member 62525]

Perhaps @juched should move the 'sites' file to '/opt/share/unbound/configs' ?

Would it be more standardized if we align with other extensions like Skynet and use jffs/scripts and jffs/config directories?

 

[Martineau]

Would it be more standardized if we align with other extensions like Skynet and use jffs/scripts and jffs/config directories?

No

 

[^Tripper^]

The original idea was to dynamically enable unbound-control directives that are, as stated in the documentation:

View attachment 21681

detrimental to the performance of unbound.

I realise you’re referring to ‘extended-statistics’, but should enabling standard logging (“lo” in advanced tools) have any affect in performance?

I’ve had it turned on and all seemed fine but just as a test I tried disabling it. Wow.

 

[juched]

I realise you’re referring to ‘extended-statistics’, but should enabling standard logging (“lo” in advanced tools) have any affect in performance?

I’ve had it turned on and all seemed fine but just as a test I tried disabling it. Wow.

Logging has a huge impact. Needs to write to the USB continually to slow things down.

Statistics and extended statistics are in memory tracked which is much faster, and they still warn about it.

From the unbound documentation:

Default is no.  Note that  it takes time to print these lines which makes the server (signifi-cantly) slower.

 

[Martineau]

I've uploaded v2.14 and unbound.conf v1.06

 Version=2.14
 Github      md5=dba05bcfbb60cf9de950a48a508eb7b9

Use of the 'i = Update unbound Installation' may be **REQUIRED** (see CHANGE log)

CHANGE: 's+' command to dynamically enable/disable 'extended-statistics:' (required for the new 'sgui' menu option) is now also saved in 'unbound.conf' ('unbound.conf' v1.06 **REQUIRED**)
NEW: 'sgui [uninstall]' menu command installs/uninstalls @juched's Router GUI TAB to graphically display unbound statistics

rs = Restart (or Start) unbound (use 'rs nocache' to flush cache) s  = Show unbound Extended statistics (s=Summary Totals; sa=All; sgui=Install GUI TAB; s-=Disable Extended Stats)
e  = Exit Script

A:Option ==> sgui

Do you want to add router GUI TAB to display stats?

    Reply 'y' or press [Enter]  to skip
y

 Installing @juched's GUI TAB to display unbound stats.....
 unbound_stats.sh downloaded successfully
 unboundstats_www.asp downloaded successfully

 Mounting Unbound_Stats.sh WebUI page as userX.asp

 Router Configuration recommended pre-reqs status:
 [✔] Swapfile=1048572 kB
 [✔] DNS Filter=ON
 [✔] DNS Filter=ROUTER
 [✖] Warning WAN: Use local caching DNS server as system resolver=YES    see http://192.168.1.1:80/Tools_OtherSettings.asp ->Advanced Tweaks and Hacks
 [✔] Entware NTP server is running
 [✔] Enable DNS Rebind protection=NO
 [✔] Enable DNSSEC support=NO

 Options:
 [✔] unbound Logging
 [✔] unbound CPU/Memory Performance tweaks
 [✔] Router GUI statistics TAB installed

NEW: Add commandline option to restart unbound i.e. 'unbound_manager restart'

@juched unbound_manager will retrieve the GUI TAB files from your Github, so you no longer need to tediously create unnecessary external Pull-requests.

If you agree, I will modify unbound_manager to also retrieve the Ad Block files from your repository?