Stubby-Installer-Asuswrt-Merlin

unknownz

Occasional Visitor
Hey @DonnyJohnny what's the best command line test, for end to end DoT and DNSSEC negotiation? We need a unified agreed upon test that is thorough. It is obvious we cannot rely on these test sites as different configurations breaks the site. Any ideas what we could use?
for DNSSEC verification, dig command will be good for testing. assuming im doing the test on the router itself,
Code:
[email protected]:/tmp/home/root# dig +dnssec +multi cloudflare.com

; <<>> DiG 9.11.3 <<>> +dnssec +multi cloudflare.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38901
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;cloudflare.com.                IN A

;; ANSWER SECTION:
cloudflare.com.         275 IN A 198.41.214.162
cloudflare.com.         275 IN A 198.41.215.162
cloudflare.com.         275 IN RRSIG A 13 2 600 (
                                20181026063706 20181024043706 35273 cloudflare.com.
                                iDLoJoJ8R0Dhd0fTHvcLq4KwWrjhMpocHJQMJC4DUIKZ
                                anbhHcqrorBhgRUmNM8cM1B8vfupMxD+34LDqv9PMg== )

;; Query time: 20 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Oct 25 13:42:41 +08 2018
;; MSG SIZE  rcvd: 185
the AD flag in the result is indication whether DNSSEC is in place

in the following test where i specify my ISP DNS server as the resolver using @xxx.xxx.xxx, the AD flag is not there as it does not support DNSSEC
Code:
[email protected]:/tmp/home/root# dig +dnssec +multi cloudflare.com @172.17.5.36

; <<>> DiG 9.11.3 <<>> +dnssec +multi cloudflare.com @172.17.5.36
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23373
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;cloudflare.com.                IN A

;; ANSWER SECTION:
cloudflare.com.         361 IN A 198.41.214.162
cloudflare.com.         361 IN A 198.41.215.162

;; Query time: 4 msec
;; SERVER: 172.17.5.36#53(172.17.5.36)
;; WHEN: Thu Oct 25 13:46:20 +08 2018
;; MSG SIZE  rcvd: 75
for DoT verification, i guess the best way will be to monitor stubby log using stubby -l when doing resolves using dig or nslookup commands.
alternatively, u can also consider to monitor dnsmasq log to see how the resolves has been carried out depending in your setup e.g. dnsmasq -> stubby -> cloudflare via TLS

[edit]
doing a closer monitor of the dnsmasq log when visiting cloudflare's help (https://1.1.1.1/help) page, it seems that the page is doing resolves to this domain, a28c0a2b-bf7a-436e-bec8-532ae918078c.is-dot.cloudflareresolve.com that's causing the test to fail when DNSSEC is enabled
 
Last edited:

Xentrk

Part of the Furniture
how to you reinstall the ca-certificates?
There is another certificate package I have installed called ca-bundle that may be a factor.

Install:
Code:
opkg install ca-bundle
 

DonnyJohnny

Very Senior Member
Fantastic work on this script, everything working as expected on my RT-AC68U. Passing all tests at https://www.cloudflare.com/ssl/encrypted-sni/ in Firefox Developer Edition after setting network.security.esni.enabled = true and network.trr.mode = 2.

Being able to use ESNI mean you are not using stubby but via Firefox built in DOH specified via trr.

When u use Firefox built in DOH, Diversion/pixelserv-tls will not work as Firefox built-in DOH will intercept the dns request.
 

DonnyJohnny

Very Senior Member
With the changes on Firefox, I now get a pass for the DoH test
View attachment 14895
That mean dns leak as dns request is done via 2 resolvers. One via stubby (with diversion/pixelserv-tls if installed) and one via Firefox built-in DOH (bypass diversion/pixelserv-tls).
This mean if u may see ads sometime even if they are blocked by diversion.
 

Jack Yaz

Part of the Furniture
There is another certificate package I have installed called ca-bundle that may be a factor.

Install:
Code:
opkg install ca-bundle
ca-bundle installs the ca-certificates.crt file which you then reference in stubby.yml

ca-certificates installs the loose certificate files (minus ca-certificates.crt), and no line in stubby.yml should be required
 

bbunge

Very Senior Member
I have disabled the tls_ca_file in my stubby.yml. I have also enabled stubby to use DNSSEC by adding:
Code:
dnssec_return_status: GETDNS_EXTENSION_TRUE
I am not using IPV6 so I have removed the IPV6 entries in stubby.yml and /jffs/configs/dnsmasq.conf.add
I am using the two Cloudflare resolvers with round_robin enabled. I have tested Quad9 and Clean Browsing with limited success. Actually with Quad9 enabled the router would not time sync on reboot, even with an IP in NTP Time.
Here is my stubby.yml:
Code:
# stubby.yml configuration file created by Xentrk
# version 1.0.0
# tls_ca_file: "/rom/etc/ssl/certs/ca-certificates.crt"
resolution_type: GETDNS_RESOLUTION_STUB
dns_transport_list:
  - GETDNS_TRANSPORT_TLS
tls_authentication: GETDNS_AUTHENTICATION_REQUIRED
dnssec_return_status: GETDNS_EXTENSION_TRUE
tls_query_padding_blocksize: 128
edns_client_subnet_private : 1
round_robin_upstreams: 1
idle_timeout: 2000
tls_connection_retries: 5
tls_backoff_time: 900
timeout: 2000
appdata_dir: "/opt/var/cache/stubby"
listen_addresses:
  - [email protected]

upstream_recursive_servers:
# Cloudflare Primary IPv4
  - address_data: 1.1.1.1
    tls_auth_name: "cloudflare-dns.com"
# Cloudflare Secondary IPv4
  - address_data: 1.0.0.1
    tls_auth_name: "cloudflare-dns.com"
Yes, stubby has downloaded the files in /opt/var/cache/stubby and refreshes root.key regularly. I am please with the way this is running and feel confident that DoT and DNSSEC is working as intended.
On another note I've discovered that the stubby in John's fork does not use DNSSEC and I've not been able to prove it is as secure as this stubby add on.
 

DonnyJohnny

Very Senior Member
I have disabled the tls_ca_file in my stubby.yml. I have also enabled stubby to use DNSSEC by adding:
Code:
dnssec_return_status: GETDNS_EXTENSION_TRUE
I am not using IPV6 so I have removed the IPV6 entries in stubby.yml and /jffs/configs/dnsmasq.conf.add
I am using the two Cloudflare resolvers with round_robin enabled. I have tested Quad9 and Clean Browsing with limited success. Actually with Quad9 enabled the router would not time sync on reboot, even with an IP in NTP Time.
Here is my stubby.yml:
Code:
# stubby.yml configuration file created by Xentrk
# version 1.0.0
# tls_ca_file: "/rom/etc/ssl/certs/ca-certificates.crt"
resolution_type: GETDNS_RESOLUTION_STUB
dns_transport_list:
  - GETDNS_TRANSPORT_TLS
tls_authentication: GETDNS_AUTHENTICATION_REQUIRED
dnssec_return_status: GETDNS_EXTENSION_TRUE
tls_query_padding_blocksize: 128
edns_client_subnet_private : 1
round_robin_upstreams: 1
idle_timeout: 2000
tls_connection_retries: 5
tls_backoff_time: 900
timeout: 2000
appdata_dir: "/opt/var/cache/stubby"
listen_addresses:
  - [email protected]

upstream_recursive_servers:
# Cloudflare Primary IPv4
  - address_data: 1.1.1.1
    tls_auth_name: "cloudflare-dns.com"
# Cloudflare Secondary IPv4
  - address_data: 1.0.0.1
    tls_auth_name: "cloudflare-dns.com"
Yes, stubby has downloaded the files in /opt/var/cache/stubby and refreshes root.key regularly. I am please with the way this is running and feel confident that DoT and DNSSEC is working as intended.
On another note I've discovered that the stubby in John's fork does not use DNSSEC and I've not been able to prove it is as secure as this stubby add on.
For your info, 1.0.0.1 do not have DNSSEC. Only 1.1.1.1.
If I am not wrong, the DNSSEC extension in stubby don’t work well with cloudflare or Quad9. Maybe it is me only. Do your own testing.

To test, you need to install dig via entware
opkg update
opkg install bind-dig

Then follow the instruction here to test dnssec
https://docs.menandmice.com/display/MM/How+to+test+DNSSEC+validation

If u see AD flag mean dnssec validation is working.

U can test with stubby dnssec extension and off it then turn on router own dnssec validation. Under LAN, enable dnssec.
If experiment with setting strict dnssec validation to YES and NO.

Also check https://1.1.1.1/help to see if cloudflare DOT is working properly.
 

skeal

Part of the Furniture
I have disabled the tls_ca_file in my stubby.yml. I have also enabled stubby to use DNSSEC by adding:
Code:
dnssec_return_status: GETDNS_EXTENSION_TRUE
I am not using IPV6 so I have removed the IPV6 entries in stubby.yml and /jffs/configs/dnsmasq.conf.add
I am using the two Cloudflare resolvers with round_robin enabled. I have tested Quad9 and Clean Browsing with limited success. Actually with Quad9 enabled the router would not time sync on reboot, even with an IP in NTP Time.
Here is my stubby.yml:
Code:
# stubby.yml configuration file created by Xentrk
# version 1.0.0
# tls_ca_file: "/rom/etc/ssl/certs/ca-certificates.crt"
resolution_type: GETDNS_RESOLUTION_STUB
dns_transport_list:
  - GETDNS_TRANSPORT_TLS
tls_authentication: GETDNS_AUTHENTICATION_REQUIRED
dnssec_return_status: GETDNS_EXTENSION_TRUE
tls_query_padding_blocksize: 128
edns_client_subnet_private : 1
round_robin_upstreams: 1
idle_timeout: 2000
tls_connection_retries: 5
tls_backoff_time: 900
timeout: 2000
appdata_dir: "/opt/var/cache/stubby"
listen_addresses:
  - [email protected]

upstream_recursive_servers:
# Cloudflare Primary IPv4
  - address_data: 1.1.1.1
    tls_auth_name: "cloudflare-dns.com"
# Cloudflare Secondary IPv4
  - address_data: 1.0.0.1
    tls_auth_name: "cloudflare-dns.com"
Yes, stubby has downloaded the files in /opt/var/cache/stubby and refreshes root.key regularly. I am please with the way this is running and feel confident that DoT and DNSSEC is working as intended.
On another note I've discovered that the stubby in John's fork does not use DNSSEC and I've not been able to prove it is as secure as this stubby add on.
Thanks for sharing your config. Can you tell me what commands and or websites you used to prove this out? This looks really awesome!! TIA:)
 

bbunge

Very Senior Member
Thanks for sharing your config. Can you tell me what commands and or websites you used to prove this out? This looks really awesome!! TIA:)
ah yeah 1.0.0.1 does do dnssec. it is just an alt resolver for cloud flare


Here is a list of upstream recursive servers that support DoT and DNSSEC
Code:
    upstream_recursive_servers:
    # IPv4 and IPV6 addresses
    # # Cloudflare servers
      - address_data: 1.1.1.1
        tls_auth_name: "cloudflare-dns.com"

      - address_data: 2606:4700:4700::1111
        tls_auth_name: "cloudflare-dns.com"
                  
    # # Cloudflare Alt servers
      - address_data: 1.0.0.1 
        tls_auth_name: "cloudflare-dns.com"

      - address_data: 2606:4700:4700::1001
        tls_auth_name: "cloudflare-dns.com"

    # # Quad9 Secure servers
      - address_data: 9.9.9.9
        tls_auth_name: "dns.quad9.net"
                  
      - address_data: 2620:fe::fe
        tls_auth_name: "dns.quad9.net"
                  
    # # Quad9 Secure Alt servers
      - address_data: 149.112.112.112
        tls_auth_name: "dns.quad9.net"

      - address_data: 2620:fe::9
        tls_auth_name: "dns.quad9.net"

    # # Cleanbrowsing-Security servers
      - address_data: 185.228.168.9
        tls_auth_name: "security-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:1::2
        tls_auth_name: "security-filter-dns.cleanbrowsing.org"

    # # Cleanbrowsing-Security Alt servers
      - address_data: 185.228.169.9
        tls_auth_name: "security-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:2::2
        tls_auth_name: "security-filter-dns.cleanbrowsing.org"

    # # Cleanbrowsing-Family servers
      - address_data: 185.228.168.168
        tls_auth_name: "family-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:1::
        tls_auth_name: "family-filter-dns.cleanbrowsing.org"
                  
    # # Cleanbrowsing-Family Alt servers
      - address_data: 185.228.168.169
        tls_auth_name: "family-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:2
        tls_auth_name: "family-filter-dns.cleanbrowsing.org"

    # # Cleanbrowsing-Adult servers
      - address_data: 185.228.168.10
        tls_auth_name: "adult-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:1::1
        tls_auth_name: "adult-filter-dns.cleanbrowsing.org"
                  
    # # Cleanbrowsing-Adult Alt servers
      - address_data: 185.228.168.11
        tls_auth_name: "adult-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:2::1::
        tls_auth_name: "adult-filter-dns.cleanbrowsing.org"
 
Last edited:

DonnyJohnny

Very Senior Member
ah yeah 1.0.0.1 does do dnssec. it is just an alt resolver for cloud flare


Here is a list of upstream recursive servers that support DoT and DNSSEC
Code:
    upstream_recursive_servers:
    # IPv4 and IPV6 addresses
    # # Cloudflare servers
      - address_data: 1.1.1.1
        tls_auth_name: "cloudflare-dns.com"

      - address_data: 2606:4700:4700::1111
        tls_auth_name: "cloudflare-dns.com"
               
    # # Cloudflare Alt servers
      - address_data: 1.0.0.1
        tls_auth_name: "cloudflare-dns.com"

      - address_data: 2606:4700:4700::1001
        tls_auth_name: "cloudflare-dns.com"

    # # Quad9 Secure servers
      - address_data: 9.9.9.9
        tls_auth_name: "dns.quad9.net"
               
      - address_data: 2620:fe::fe
        tls_auth_name: "dns.quad9.net"
               
    # # Quad9 Secure Alt servers
      - address_data: 149.112.112.112
        tls_auth_name: "dns.quad9.net"

      - address_data: 2620:fe::9
        tls_auth_name: "dns.quad9.net"

    # # Cleanbrowsing-Security servers
      - address_data: 185.228.168.9
        tls_auth_name: "security-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:1::2
        tls_auth_name: "security-filter-dns.cleanbrowsing.org"

    # # Cleanbrowsing-Security Alt servers
      - address_data: 185.228.169.9
        tls_auth_name: "security-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:2::2
        tls_auth_name: "security-filter-dns.cleanbrowsing.org"

    # # Cleanbrowsing-Family servers
      - address_data: 185.228.168.168
        tls_auth_name: "family-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:1::
        tls_auth_name: "family-filter-dns.cleanbrowsing.org"
               
    # # Cleanbrowsing-Family Alt servers
      - address_data: 185.228.168.169
        tls_auth_name: "family-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:2
        tls_auth_name: "family-filter-dns.cleanbrowsing.org"

    # # Cleanbrowsing-Adult servers
      - address_data: 185.228.168.10
        tls_auth_name: "adult-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:1::1
        tls_auth_name: "adult-filter-dns.cleanbrowsing.org"
               
    # # Cleanbrowsing-Adult Alt servers
      - address_data: 185.228.168.11
        tls_auth_name: "adult-filter-dns.cleanbrowsing.org"

      - address_data: 2a0d:2a00:2::1::
        tls_auth_name: "adult-filter-dns.cleanbrowsing.org"
Sorry. You are right.
I mixed up cloudflare and quad9.
Quad9 9.9.9.9 is dnssec and 9.9.9.10 is not.

By the way, did u get the ad flag with the dig command?
And is the https://1.1.1.1/help reflect correctly?
 

bbunge

Very Senior Member
For your info, 1.0.0.1 do not have DNSSEC. Only 1.1.1.1.
If I am not wrong, the DNSSEC extension in stubby don’t work well with cloudflare or Quad9. Maybe it is me only. Do your own testing.

To test, you need to install dig via entware
opkg update
opkg install bind-dig

Then follow the instruction here to test dnssec
https://docs.menandmice.com/display/MM/How+to+test+DNSSEC+validation

If u see AD flag mean dnssec validation is working.

U can test with stubby dnssec extension and off it then turn on router own dnssec validation. Under LAN, enable dnssec.
If experiment with setting strict dnssec validation to YES and NO.

Also check https://1.1.1.1/help to see if cloudflare DOT is working properly.
from what I'm reading at that web site dig only validates the server you are testing. I see nothing that it tests your connection. "After installing and configuring a DNSSEC validating secure DNS server, the administrator should test that" tells me that the server is being tested not the client. This is nothing new as all the other "DNSSEC test" sites just test the server.
 

DonnyJohnny

Very Senior Member
from what I'm reading at that web site dig only validates the server you are testing. I see nothing that it tests your connection. "After installing and configuring a DNSSEC validating secure DNS server, the administrator should test that" tells me that the server is being tested not the client. This is nothing new as all the other "DNSSEC test" sites just test the server.
Suppose for dnssec to work, stubby need to validate the connection by setting the dnssec extension.
So the dig function and ad flag is to prove that the queries is validated (secured).
Question here is cloudflare seen to fail if it is set to do strict validation.
And once again, have you tested the dig function with the dnssec extension enabled? And what is the status in https://1.1.1.1/help with the extension enabled.
 

bbunge

Very Senior Member
Suppose for dnssec to work, stubby need to validate the connection by setting the dnssec extension.
So the dig function and ad flag is to prove that the queries is validated (secured).
Question here is cloudflare seen to fail if it is set to do strict validation.
And once again, have you tested the dig function with the dnssec extension enabled? And what is the status in https://1.1.1.1/help with the extension enabled.
I removed stubby and am back to plain (if you can call Merlin plain) Merlin with LAN/DNSSEC enabled with strict unsigned checked. 1.1.1.1/help shows no DoT/DoH (to be expected).
dig 1.1.1.1 +dnssec +multi shows:
Code:
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1
 

DonnyJohnny

Very Senior Member
I removed stubby and am back to plain (if you can call Merlin plain) Merlin with LAN/DNSSEC enabled with strict unsigned checked. 1.1.1.1/help shows no DoT/DoH (to be expected).
dig 1.1.1.1 +dnssec +multi shows:
Code:
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1
Ok... that’s correct result.

The cloudflare failed is more pertaining to stubby/dnscrypt-proxy over DOT/DoH not working if strict validation is enabled.
 

bbunge

Very Senior Member
Installed stubby again. No mod to stubby.yml
1.1.1.1/help DoT - Yes
Enabled LAN/DNSEC. 1.1.1.1/help DoT - No AD flag in dig
Disabled above. Enabled DNSSEC in stubby. /opt/var/cache/stubby populated with downloaded files from ICAN. 1.1.1.1/help - DoT - No no AD flag in dig

Does this possibly mean that dig is getting its values from dnsmasq and that stubby is doing DNSSEC and that dig can't check it?

Family wants to use the internet tonight so my testing for the day is over...
 

bbunge

Very Senior Member
For those of you with an IOS device there is an ISC Dig app that you can use to check DNSSEC. It showed me the same results as bind-dig.

Sent from my SM-T380 using Tapatalk
 

Treadler

Very Senior Member
For those of you with an IOS device there is an ISC Dig app that you can use to check DNSSEC. It showed me the same results as bind-dig.

Sent from my SM-T380 using Tapatalk
Well done, nice find!
Thank you.
 

Xentrk

Part of the Furniture
Enabling dnssec_return_status: GETDNS_EXTENSION_TRUE i lost connection !?
Update: I was able to duplicate the issue when I had DNSSEC turned on in the Web Gui and dnssec_return_status: GETDNS_EXTENSION_TRUE in stubby.yml.

Stubby uses the getdns Zero configuration DNSSEC feature. The test team has been having inconsistent results using dnssec_return_status: GETDNS_EXTENSION_TRUE. I have been testing with it using Cloudflare 1.1.1.1.

When I first started testing last month, the system would generate root anchor files in /opt/var/cache/stubby. Has not happened in the last 25 days or so though. But two of the testers see root anchor files. This is when testing the same config. I did save a copy of the files which I've used in some recent test cases. Here is a summary of my results the other day.

Note: The site https://www.cloudflare.com/ssl/encrypted-sni/ is DNSSEC supported whereas the Cloudflare test page, https://1.1.1.1/help, does not support DNSSEC.

Code:
=====
TEST 1
#tls_ca_file: "/rom/etc/ssl/certs/ca-certificates.crt"
dnssec_return_status: GETDNS_EXTENSION_FALSE
NO root anchors in /opt/var/cache/stubby
Results:
1. DNSSEC test sites work
2. Cloudflare test page works
3. https://www.cloudflare.com/ssl/encrypted-sni/
    yes - SECURE DNS
    yes - DNSSEC
4. stubby -l: STUBBY: DNSSEC Validation is OFF
 
TEST 2
tls_ca_file: "/rom/etc/ssl/certs/ca-certificates.crt"
dnssec_return_status: GETDNS_EXTENSION_TRUE
NO root anchors in /opt/var/cache/stubby
Results:
1. DNSSEC test sites DO NOT work
2. Cloudflare test page works
3. https://www.cloudflare.com/ssl/encrypted-sni/
    yes - SECURE DNS
    NO - DNSSEC
4. stubby -l: STUBBY: DNSSEC Validation is ON
 
TEST 3
#tls_ca_file: "/rom/etc/ssl/certs/ca-certificates.crt"
dnssec_return_status: GETDNS_EXTENSION_TRUE
root anchors in /opt/var/cache/stubby
Results:
1. DNSSEC test sites work
2. Cloudflare test page DOES NOT work
3. https://www.cloudflare.com/ssl/encrypted-sni/
    No - SECURE DNS
    yes - DNSSEC
4. stubby -l: STUBBY: DNSSEC Validation is ON
 
TEST 4
tls_ca_file: "/rom/etc/ssl/certs/ca-certificates.crt"
dnssec_return_status: GETDNS_EXTENSION_FALSE
root anchor files in /opt/var/cache/stubby
Results:
1. DNSSEC test sites work
2. Cloudflare test page works
3. https://www.cloudflare.com/ssl/encrypted-sni/
    Yes - SECURE DNS
    yes - DNSSEC
4. stubby -l: STUBBY: DNSSEC Validation is OFF
I did create an issue earlier with the Stubby devs. May need to re-open. They linked me to an issue where Stubby is not pulling the trust anchors. Just noticed there was an update 3 days ago where they acknowledge the need to add better debugging for failure mode.

Stubby is still considered early development or experimental. Jumping on the bus early may come with some risks.

There are others having mixed results with Stubby + DNSSEC on the OpenWRT Stubby thread. Some see anchor files and some don't.
 
Last edited:

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top