Unbound - Authoritative Recursive Caching DNS Server

[rgnldo]

Furthermore your attitude to other more learned forum members is appalling, particularly over the POST-CMD debacle, and have continued to ignore my suggestions to make the maintenance of the script easier such as version control on you 'unbound.conf'

PM message is PM. You ask to discuss the post is normal. But leaking without consent is disrespectful. Several PM messages were with me with suggestions. At no time did I leak PM from the conversations. But anyway, I understand your position. It is not feasible to work together.

 

[Martineau]

PM message is PM. You ask to discuss the post is normal. But leaking without consent is disrespectful. Several PM messages were with me with suggestions. At no time did I leak PM from the conversations. But anyway, I understand your position. It is not feasible to work together.

The leaked PM in question contained no personal in-confidence information, and was a request to undo hours of coding on your whim
Disclosure was deemed acceptable as this was to formally document the development cycle of the script in the thread (where it should rightly be with all of the other formal requests) and to substantiate the reason for the enforced version rewrite.

We are in different timezones and as a preference I do not subscribe to any forum threads for email notifications - if I miss a post then so be it.

You have no idea of my lifestyle i.e. random shift work/sleep patterns, so continuing to spam my phone with your PMs is more than disrespectful - wouldn't you agree?

I await to be banned from the forum for a breach of the rules.

 

[rgnldo]

Didn't you notice the various script installation errors? But anyway, I am for education. I recognize your efforts. In the same way that I invited you to this project in a PM message, I thank you here for all your efforts. Feel free to continue development. Follow the project, but without me.

 

[rgnldo]

personal in-confidence information

I have several private messages asking. None leaked. Let us set this aside. You are an excellent dev. I appreciate your efforts.

 

[Martineau]

With John’s fork, there are some dnsmasq.conf differences I added to dnsmasq.postconf (or unbound.postconf in your repo):

pc_delete "resolv-file" "$CONFIG"
pc_append "no-resolv" "$CONFIG"

John doesn’t use a server-file= parameter, nor the no-resolv that Merlin does.
If you can add those if uname -o = “ASUSWRT-Merlin-LTS” then it should work fine with both versions.

My memory was manageable with only the Steven Black list. If I added the Plus Hosts, unbound would take more than 382 MB RAM and bring things to a big slowdown, swapping. Interested in others’ experiences.

It was also important to remove the diversion addn-hosts conserve memory if using unbound Adblock, and disable DNS Rebind protection if using a private IP for Pixelserv:

pc_delete "addn-hosts=/opt/share/diversion/list/" "$CONFIG"
pc_delete "stop-dns-rebind" "$CONFIG"

EDIT: Also, John's fork has included haveged since 25E1j9527, so no need to install haveged on John's fork.

I have tried to interpret/incorporate a couple of your LTS-tweaks in v1.26, but will need to think about managing the 'addn-hosts' for a later release.

FYI, unbound_manager will report the Warning if both unbound Ad Block and Diversion are ACTIVE

e  = Exit Script

Option ==> ?

    Version=1.25

<snip>

    Options:

    [✔] unbound Logging
    [✔] Ad and Tracker Blocking (No. of Adblock domains=51519,Blocked Hosts=832,Whitelist=21, - Warning Diversion is also ACTIVE)

    Memory/Cache:

    'key-cache-size:' 62307328 (59.42m)
    'msg-cache-size:' 62307328 (59.42m)
    'rrset-cache-size:' 83076437 (79.23m)

However, when the user sees the above warning message and decides to terminate Diversion (rather than remove unbound Ad Block) I do find it rather unfortunate that the user cannot then manually restart S80pixelserv-tls for testing with unbound.

Subsequently,as a convenience (when requested), unbound_manager will alter '/init.d/S80pixelserv-tls'

if [ "$DIVERSION_STATUS" = "enabled" ] && [ "$psState" = "on" ]; then

to

if [ -n "$(pidof unbound)" ] || { [ "$DIVERSION_STATUS" = "enabled" ] && [ "$psState" = "on" ]; };then    # unbound_manager

so the user can now ALWAYS start pixelserv if either unbound or Diversion is UP (FYI @thelonelycoder )

 

[dave14305]

I have tried to interpret/incorporate a couple of your LTS-tweaks in v1.26, but will need to think about managing the 'addn-hosts' for a later release.

Great! I will test it this evening when the family is out of the house.

However, when the user sees the above warning message and decides to terminate Diversion (rather than remove unbound Ad Block) I do find it rather unfortunate that the user cannot then manually restart S80pixelserv-tls for testing with unbound.

This is why I resorted to just removing the addn-hosts lines dynamically in the postconf. Pixelserv will still start, and Diversion will automatically be active if I disable Unbound and restart dnsmasq. I also considered an S81pixelserv-tls that checks if pixelserv-tls is already running before starting it. So it leaves thelonelycoder's S80 file alone and still let's Pixelserv-tls startup if Diversion is manually disabled.

 

[dave14305]

@Martineau Some additional observations after running the manager script (I haven't tried a fresh install until tonight):

• Error message about DNS Local Cache due to code error [ $(nvram get nvram get dns_local_cache) != "0" ] --- Also, john's fork doesn't have this nvram variable or feature anyway -- router always uses dnsmasq.
  
• NTP check on John's fork should check nvram ntpd_server=1 or 0. ntpd_enable doesn't exist.

        Router Configuration recommended pre-reqs status:

        [✔] Swapfile=262140 kB
        [✔] DNS Filter=ON
        [✔] DNS Filter=ROUTER
[: 0: unknown operand
        [✔] WAN: Use local caching DNS server as system resolver=NO
[: 0: unknown operand
        [✔] Enable local NTP server=YES

 

[Martineau]

@Martineau Some additional observations after running the manager script (I haven't tried a fresh install until tonight):

• Error message about DNS Local Cache due to code error [ $(nvram get nvram get dns_local_cache) != "0" ] --- Also, john's fork doesn't have this nvram variable or feature anyway -- router always uses dnsmasq.
  
• NTP check on John's fork should check nvram ntpd_server=1 or 0. ntpd_enable doesn't exist.

        Router Configuration recommended pre-reqs status:

        [✔] Swapfile=262140 kB
        [✔] DNS Filter=ON
        [✔] DNS Filter=ROUTER
[: 0: unknown operand
        [✔] WAN: Use local caching DNS server as system resolver=NO
[: 0: unknown operand
        [✔] Enable local NTP server=YES

Thanks, pushed hotfix to GitHub, it usually takes a few mins for the new MD5 to be published?....not sure why, so issue

e  = Exit Script

Option ==> uf

 

[Kingp1n]

I hope we can put the miscommunication set aside so we can continue to make unbound better. The more contributors in this helps the whole community vs just 1 person dng everything. Look forward for @rgnldo and @Martineau continue to collabo with no hard feelings. Appreciate all you guys do!!!

 

[thelonelycoder]

I have tried to interpret/incorporate a couple of your LTS-tweaks in v1.26, but will need to think about managing the 'addn-hosts' for a later release.

FYI, unbound_manager will report the Warning if both unbound Ad Block and Diversion are ACTIVE

e  = Exit Script

Option ==> ?

    Version=1.25

<snip>

    Options:

    [✔] unbound Logging
    [✔] Ad and Tracker Blocking (No. of Adblock domains=51519,Blocked Hosts=832,Whitelist=21, - Warning Diversion is also ACTIVE)

    Memory/Cache:

    'key-cache-size:' 62307328 (59.42m)
    'msg-cache-size:' 62307328 (59.42m)
    'rrset-cache-size:' 83076437 (79.23m)

However, when the user sees the above warning message and decides to terminate Diversion (rather than remove unbound Ad Block) I do find it rather unfortunate that the user cannot then manually restart S80pixelserv-tls for testing with unbound.

Subsequently,as a convenience (when requested), unbound_manager will alter '/init.d/S80pixelserv-tls'

if [ "$DIVERSION_STATUS" = "enabled" ] && [ "$psState" = "on" ]; then

to

if [ -n "$(pidof unbound)" ] || { [ "$DIVERSION_STATUS" = "enabled" ] && [ "$psState" = "on" ]; };then    # unbound_manager

so the user can now ALWAYS start pixelserv if either unbound or Diversion is UP (FYI @thelonelycoder )

Note that
- Diversion replaces that file with each update.
- Diversion and amtm replace that file when a pixelserv-tls version change is detected when updating Entware packages with ep.

 

[Martineau]

Note that
- Diversion replaces that file with each update.
- Diversion and amtm replace that file when a pixelserv-tls version change is detected when updating Entware packages with ep.

Yes understood, but the heads-up was to gauge your willingness to ensure that whenever Diversion/amtm updates the file that the resulting script is capable of starting pixelserv if either unbound or Diversion is UP.

Alternatively provide a menu option in amtm to install pixelserv as a separate utility.

 

[thelonelycoder]

<Start public rant>
Would you two guys communicate things before implementing it? I've been around for a while on this board and have contributed and collaborated a lot but never seen this.
I PM'd you both to discuss amtm compatibility - it ended with you two arguing.
Now I see it's a matter of personalities and I'm left in between - clueless. And then get asked to do either one thing or the other OR?
For the time being I have no time for this thread's project. We'll talk again when things between @rgnldo and @Martineau are sorted out AND things that concern my scripts are properly communicated AND I have time and AM willing to look into it.
</End public rant>

 

[Martineau]

<Start public rant>
Would you two guys communicate things before implementing it? I've been around for a while on this board and have contributed and collaborated a lot but never seen this.
I PM'd you both to discuss amtm compatibility - it ended with you two arguing.
Now I see it's a matter of personalities and I'm left in between - clueless. And then get asked to do either one thing or the other OR?
For the time being I have no time for this thread's project. We'll talk again when things between @rgnldo and @Martineau are sorted out AND things that concern my scripts are properly communicated AND I have time and AM willing to look into it.
</End public rant>

I'm happy to have unbound manager added to amtm.

 

[thelonelycoder]

I'm happy to have unbound manager added to amtm.

So am I, you know that already, but not under these circumstances.

 

[Martineau]

So am I, you know that already, but not under these circumstances.

What circumstances ?
I wrote and host the utilty on my GitHub, so if I give my consent (like I always have) for inclusion in amtm then I don't see any conflict.

 

[elorimer]

I know I shouldn't, but I will.

@Martineau and @dave14305 have taken this a lot farther, a lot faster, than it would have gone on its own steam. They have also been a lot less delphic so I have understood a little better the 40 pages of this thread; I would have abandoned it around page 5.

Waiting until the dust settles and I can understand better how this and/or the other DNS efforts floating around would be better than/improve my Diversion/Pixelserv setup.

 

[Kingp1n]

Unbound sure has come a long way. Hope to see it included inside amtm in the near future.

 

[Martineau]

I know I shouldn't, but I will.

@Martineau and @dave14305 have taken this a lot farther, a lot faster, than it would have gone on its own steam. They have also been a lot less delphic so I have understood a little better the 40 pages of this thread; I would have abandoned it around page 5.

Waiting until the dust settles and I can understand better how this and/or the other DNS efforts floating around would be better than/improve my Diversion/Pixelserv setup.

I understand your dilemma

i.e. is the mature/well understood combination

dnsmasq+diversion+pixelserv

enough for most users? or do any of the other proposed combinations

unbound+adblock+pixelserv

or

unbound+diversion+pixelserv

provide better features/reliability/performance?

i.e. currently if you have need for the DNS stats (GUI or otherwise) then clearly diversion is a MUST-HAVE

If you need true Blacklist/Whitelist wildcards then does unbound+Ad Block offer more flexibility?

Is dnsmasq more reliable? - I recall dnsmasq lookup glitches with DoT in its infancy i.e. wait a second then retry and dnsmasq may kick itself back to life!.
I can honestly say this has never happened (yet) when using unbound but I mention this only as a datapoint not as a biased endorsement of unbound.

I know I shouldn't but.....

There have been many many 'personality' clashes in the forums (strange @thelonelycoder can't recall any) but encouragingly, all of the antagonists (despite an uneasy truce) still remain as valued contributors for the benefit of the forums,

P.S. For the record, rightly or wrongly I have accommodated every @rgnldo design request into the script.

My concern is that I don't recall having to tweak the number of thread and cache memory for dnsmasq.

Is the implication that unbound cannot compete with dnsmasq on a level playing field without increasing the number of CPUs or cache memory?

i.e. unbound suggested cache tweak goes from 8+8+16(MB) to (on my RT-AC68U) 59.42+59.42+79.23(MB) over 600% increase, yet when queried by another forum member as to how the algorithm was selected there wasn't a substantiating reply?

I have simply endeavored to keep @rgnldo 'honest' - after all he invited me to write the script (or did hundreds decline before I was reluctantly offered the gig?) and if he chooses to no longer collaborate directly with me, then that is his personal prerogative, but that doesn't mean it is impossible for him to continue to participate in the forum to assist other members.

I consider the matter closed and sincerely hope @rgnldo will reconsider his decision.

 

[L&LD]

I was saddened and a little shocked to see this misunderstanding boil over in the last couple of days (to us outsiders). But I am very happy to see a cooperative spirit persist above it all.

I can only begin to understand how frustrating it must be to decipher and translate words and meanings from varied people and backgrounds, languages and cultures and turn that into useful code.

Yet, this seems to be happening.

I am again very grateful to be in such company. Even when tempers and personalities clash, as they will, the goal remains in sight and the good work is further improved upon.

I can't say I totally understand at a deep level what is going on here (code-wise), but I do know that unbound_manager.sh has made my RT-AX88U significantly faster for consuming the web. Even with a 1Gbps up/down (symmetrical) connection, the improvement is jaw-dropping.

Having these options and support on RMerlin powered Asus routers put all other consumer options (and many 'prosumer' and entry-level 'enterprise' options) to shame.

Again, many thanks to all developers, scriptwriters, and alpha and beta testers that push the limits of these scripts enough to break them (thus, making them bullet-proof for mere mortals), and making our routers and networks not only more performant but also more reliable, stable and secure too.

 

[Kingp1n]

I was saddened and a little shocked to see this misunderstanding boil over in the last couple of days (to us outsiders). But I am very happy to see a cooperative spirit persist above it all.

I can only begin to understand how frustrating it must be to decipher and translate words and meanings from varied people and backgrounds, languages and cultures and turn that into useful code.

Yet, this seems to be happening.

I am again very grateful to be in such company. Even when tempers and personalities clash, as they will, the goal remains in sight and the good work is further improved upon.

I can't say I totally understand at a deep level what is going on here (code-wise), but I do know that unbound_manager.sh has made my RT-AX88U significantly faster for consuming the web. Even with a 1Gbps up/down (symmetrical) connection, the improvement is jaw-dropping.

Having these options and support on RMerlin powered Asus routers put all other consumer options (and many 'prosumer' and entry-level 'enterprise' options) to shame.

Again, many thanks to all developers, scriptwriters, and alpha and beta testers that push the limits of these scripts enough to break them (thus, making them bullet-proof for mere mortals), and making our routers and networks not only more performant but also more reliable, stable and secure too.

Great points...not only have my speeds improved, but it seems my CPU is working less!