Problems with dnsmasq scripts and manual configuration

[CornfieldWin]

After the policy forbidding system directory names in posted content was explained, here is the intent of the OP:

OK, let's be real clear and specific:

1. dnsmasq.postconf is being repeatedly invoked nine times during reboot. Not maybe. Definitely. As seen in the 9 times by updated dnsmasq.conf.add content. Obviously appending nine times is a bad idea. I have no idea (not big deal in itself!) why. No claim as to the mechanism, only the result. It does not seem compliant with expected behavior as documented by the authoritative Wiki description (ref: user script descriptions) which calls for a single invocation per reboot/service restart. This behavior is 100% repeatable.

2. hosts.postconfs shows in syslog as having been launched (exact entry available) but the script's touch file in jffs does not appear upon reboot and the dynamic system hosts file in etc is not appended at all despite both working when tested in place in the correct jffs location by SSH. This behavior is 100% repeatable.

3. dnsmasq.conf.add as correctly configured manually rather than by script behaves as specified with the contents winding up appended to dnsmasq.conf in the system etc directory. But the addn-hosts dnsmasq directive does not come into effect supplementing the system hosts file. The additional host names added are not resolved. This behavior is 100% repeatable.

Again no claim as to cause or responsibility but simply seeking the fix to get around these problems.

 

[dave14305]

You can’t post certain system paths related to etc or hosts. The forum will block it.

 

[dave14305]

As usual, no idea what you’re talking about. Upload stuff to pastebin and post a link here.

 

[CornfieldWin]

Yeah, share that idea part, the directories issue was a surprise and enough said.

OK, let's be real clear and specific:

1. dnsmasq.postconf is being repeatedly invoked nine times during reboot. Not maybe. Definitely. As seen in the 9 times by updated dnsmasq.conf.add content. Obviously appending nine times is a bad idea. I have no idea (not big deal in itself!) why. No claim as to the mechanism, only the result. It does not seem compliant with expected behavior as documented by the authoritative Wiki description (ref: user script descriptions) which calls for a single invocation per reboot/service restart. This behavior is 100% repeatable.

2. hosts.postconfs shows in syslog as having been launched (exact entry available) but the script's touch file in jffs does not appear upon reboot and the dynamic system hosts file in etc is not appended at all despite both working when tested in place in the correct jffs location by SSH. This behavior is 100% repeatable.

3. dnsmasq.conf.add as correctly configured manually rather than by script behaves as specified with the contents winding up appended to dnsmasq.conf in the system etc directory. But the addn-hosts dnsmasq directive does not come into effect supplementing the system host file. The additional host names added are not resolved. This behavior is 100% repeatable.

Again no claim as to cause or responsibility but simply seeking the fix to get around these problems.

 

[dave14305]

Perhaps you're not understanding how the dnsmasq custom configs and scripts works when dnsmasq is started.

1. The firmware deletes /etc/dnsmasq.conf if it exists.
2. The firmware generates the initial /etc/dnsmasq.conf based on defaults and nvram settings.
3. /jffs/configs/dnsmasq.conf.add is appended to /etc/dnsmasq.conf
4. If present, /jffs/configs/dnsmasq.conf overwrites /etc/dnsmasq.conf
5. /jffs/scripts/dnsmasq.postconf is run, being passed the /etc/dnsmasq.conf path as a parameter for the postconf script to act upon.

There's no opportunity for dnsmasq.postconf to run multiple times on the same file unless your script is poorly written somehow. dnsmasq may restart multiple times during boot as the WAN flaps up and down, or other conditions happen. dnsmasq.conf.add is completely independent from dnsmasq.postconf.

The same sequence happens for hosts as well. Your system log will tell you if the firmware finds any permissions faults with your scripts. But you seem to refuse to post any evidence of your problems, so everyone is left guessing. You're a real pain in the a$$, but who isn't?

 

[dave14305]

1. dnsmasq.postconf is being repeatedly invoked nine times during reboot. Not maybe. Definitely. As seen in the 9 times by updated dnsmasq.conf.add content.

Taking you literally, it sounds like you are using dnsmasq.postconf to populate dnsmasq.conf.add. This is not how it's meant to work, if this is indeed what your script does.

 

[CornfieldWin]

Perhaps you're not understanding how the dnsmasq custom configs and scripts works when dnsmasq is started.

1. The firmware deletes /etc/dnsmasq.conf if it exists.
2. The firmware generates the initial /etc/dnsmasq.conf based on defaults and nvram settings.
3. /jffs/configs/dnsmasq.conf.add is appended to /etc/dnsmasq.conf
4. If present, /jffs/configs/dnsmasq.conf overwrites /etc/dnsmasq.conf
5. /jffs/scripts/dnsmasq.postconf is run, being passed the /etc/dnsmasq.conf path as a parameter for the postconf script to act upon.

There's no opportunity for dnsmasq.postconf to run multiple times on the same file unless your script is poorly written somehow. dnsmasq may restart multiple times during boot as the WAN flaps up and down, or other conditions happen. dnsmasq.conf.add is completely independent from dnsmasq.postconf.

The same sequence happens for hosts as well. Your system log will tell you if the firmware finds any permissions faults with your scripts. But you seem to refuse to post any evidence of your problems, so everyone is left guessing. You're a real pain in the a$$, but who isn't?

I will take that as a backhanded compliment. Thank you for explaining the documentation. The problem is precisely and completely specified behaviorally. Still need to know where and how things go off track. Here's a bunch of data elements to go with it:

In jffs custom configs:
-rw-rw-rw- 1 windsong root 237 Sep 22 22:01 dnsmasq-2.conf.add # Confidential network information obfuscated
local=/____.home.arpa/ # Using RFC 8375 for local TLD
domain="____.home.arpa"
-rw-rw-rw- 1 windsong root 124 Sep 23 01:11 dnsmasq.addn-hosts.add # Confidential network information obfuscated here
*.*.52.1 ____.home.arpa ____
*.*.53.1 ____.home.arpa ____
-rw-rw-rw- 1 windsong root 224 Sep 22 23:57 dnsmasq.conf.add # Confidential network information obfuscated here
addn-hosts=/jffs/configs/dnsmasq.addn-hosts.add
server=/____.home.arpa/*.*.52.1
server=/____.home.arpa/*.*.53.
-rwxrwxrwx 1 windsong root 515 Sep 22 14:37 dnsmasq.localdomains.conf # Executable to work with source (.) command
# Confidential obfuscation and confirmed to work in scripts, see scripts below
DNSMASQDOMAINS='home.arpa *.*.1.1 ------ *.*.52.1 ----- *.*.53.1 -----'

In jffs custom scripts:
-rwxrwxrwx 1 windsong root 1463 Sep 22 21:34 dnsmasq.postconf # DOS notation used here to avoid conflict with posting rules
#\bin\sh
CONFIG=$1
CONFIGLAND=\jffs\configs"
SCRIPTLAND="\jffs\scripts"
DNSMASQCONF="$CONFIGLAND\dnsmasq.conf"
TARGET="$DNSMASQCONF.add"
LOCALDNSCONFIGS="$CONFIGLAND\dnsmasq.localdomains.conf"
source $LOCALDNSCONFIGS # using pre-Posix style
# $CONFIG script to modify (append to) and $TARGET which ultimately is meant to append to $DNSMASQCONF when starting the dnsmasq service
# Shell source global (immutable) for use here and other dnsmasq-SDN.conf.add, and hosts.postconf scripts:
# Format: Main_LAN_Domain SDN_1_DNS_IP SDN_1_Hostname SDN_2_DNS_IP SDN_2_Hostname … the separator is " " (empty space)DNSMASQ
# Note: additional configurations must match consistently with external configured $CONFIG, $TARGET, hosts file, and GUI definitions
# done this way to avoid strange start and restart scenarios after hopefully a full reconfiguration from scratch by DNSMASQ and possibly YAZDhcp

echo $DNSMASQDOMAINS | awk '{for (i=4; i<NF; i=i+2) print "server=/" $(i+1) "." $1 "/" $i}' >>$TARGET
exit 0
-rwxrwxrwx 1 windsong root 1259 Sep 22 23:03 hosts.postconf
#\bin\sh
CONFIG=$1
CONFIGLAND=\jffs\configs"
SCRIPTLAND="\jffs\scripts"
TARGET="\etc\hosts" # really \tmp\etc\hosts
LOCALDNSCONFIGS="$CONFIGLAND/dnsmasq.localdomains.conf" #Confidential configuration information formatted as below
. "$LOCALDNSCONFIGS" #Brings in DNSMASQDOMAINS (POSIX style source)
touch \jffs\hostpost.flag #Debugging only

echo $DNSMASQDOMAINS | awk '{for (i=4; i<NF; i=i+2) print $i " " $(i+1) "." $1 " " $(i+1)}' >>$TARGET
exit 0
Syslog with all references to DNSMASQ # (??? Timestamp, maybe NTP issue that cleared up)
Dec 31 19:00:40 custom_script: Running hosts.postconf (args:hosts) # System directories removed to prevent posting conflict
Dec 31 19:00:40 custom_config: Appending content of dnsmasq.conf.add.
Dec 31 19:00:40 wlceventd: wlceventd_proc_event(685): eth6: Auth 74:FE:CE:A5:E7:B3, status: Successful (0), rssi:0
Dec 31 19:00:40 dnsmasq[2717]: failed to create listening socket for 192.168.52.1: Cannot assign requested address
Dec 31 19:00:40 dnsmasq[2717]: FAILED to start up
Dec 31 19:01:11 custom_script: Running hosts.postconf (args: hosts)
Dec 31 19:01:11 custom_config: Appending content of dnsmasq.conf.add.
Dec 31 19:01:39 rc_service: udhcpc_wan 4850:notify_rc start_dnsmasq 255
Dec 31 19:01:43 custom_script: Running hosts.postconf (args: hosts)
Dec 31 19:01:43 custom_config: Appending content of dnsmasq.conf.add.
Dec 31 19:01:45 custom_config: Appending content of dnsmasq.conf.add.
Dec 31 19:01:47 custom_script: Running hosts.postconf (args: hosts)
Dec 31 19:01:47 custom_config: Appending content of dnsmasq.conf.add.
Sep 22 22:06:54 custom_script: Running hosts.postconf (args: hosts)
Sep 22 22:06:54 custom_config: Appending content of dnsmasq.conf.add.
Sep 22 22:06:54 custom_config: Appending content of \dnsmasq-2.conf.add.
Sep 22 22:07:05 <same as above>
Sep 22 22:07:08 <same as above>
Sep 22 22:07:20 <same as above>

 

[CornfieldWin]

Taking you literally, it sounds like you are using dnsmasq.postconf to populate dnsmasq.conf.add. This is not how it's meant to work, if this is indeed what your script does.

Now we are getting somewhere. Yes, that is the sequence. I had wondered if this should be done another way once at boot time. Could the multiple repetitions be related to that? Suggestions welcome. Then used pure data configuration in jffs configs just in case. That still leaves the hosts.postconf and addn-hosts issues which are critical to the scheme.

 

[dave14305]

echo $DNSMASQDOMAINS | awk '{for (i=4; i<NF; i=i+2) print "server=/" $(i+1) "." $1 "/" $i}' >>$TARGET

Just change TARGET to CONFIG ($1 which is /etc/dnsmasq.conf passed via parameter). Same for the hosts.postconf.

echo $DNSMASQDOMAINS | awk '{for (i=4; i<NF; i=i+2) print "server=/" $(i+1) "." $1 "/" $i}' >>$CONFIG

It might be easier if you just post an example of the desired final output in dnsmasq.conf, and we can do away with a lot of those files and variables.

 

[CornfieldWin]

Just change TARGET to CONFIG ($1 which is /etc/dnsmasq.conf passed via parameter). Same for the hosts.postconf.

echo $DNSMASQDOMAINS | awk '{for (i=4; i<NF; i=i+2) print "server=/" $(i+1) "." $1 "/" $i}' >>$CONFIG

It might be easier if you just post an example of the desired final output in dnsmasq.conf, and we can do away with a lot of those files and variables.

It is posted as requested and not relevant to the stated problem. The data part is just fine as confirmed by examining all final file contents. The Issue is why the hosts.postconf does not fire to do its job as (mis?)understood to be documented and why the addn-hosts in the final dnsmasq.conf file does not work either. That means that the Vlans DNS servers cannot be accessed by name form the main LAN.

As done by hand in jffs configs:

-rw-rw-rw- 1 windsong root 224 Sep 22 23:57 dnsmasq.conf.add # Confidential network information obfuscated here
addn-hosts=/jffs/configs/dnsmasq.addn-hosts.add
server=/____.home.arpa/*.*.52.1
server=/____.home.arpa/*.*.53.1

-rw-rw-rw- 1 windsong root 124 Sep 23 01:11 dnsmasq.addn-hosts.add # Confidential network information obfuscated here
*.*.52.1 ____.home.arpa ____
*.*.53.1 ____.home.arpa ____

Finally the less pressing problem with dnsmasq.conf as alluded has something to do with updating dnsmasq.conf.add at the wrong point in the boot/restart cycle. Please explain the correct time and where to do it by script.

Lastly, maintaining consistency across scripts motivates having the source configuration variable which can be edited in one place and which was requested to see. The problems do NOT originate from that as explained. It is a behavioral, not a data problem. Things are happening at the wrong time and place (dnsmasq.postconf?) plus not happening when names should resolve (hosts.postconf and addn-hosts).

 

[dave14305]

If the information you want to add to dnsmasq.conf is static and not going to change, just put it in the dnsmasq.conf.add file and delete the dnsmasq.postconf. postconf is generally only needed if you want to delete or modify a line from the original conf, or if your changes are dynamic in nature requiring actual scripting. If all you want to do is append configuration lines, just use dnsmasq.conf.add.

 

[dave14305]

The Issue is why the hosts.postconf does not fire to do its job

Your system log excerpt shows hosts.postconf being executed. Hosts will get recreated every time dnsmasq restarts. What isn't it doing or what is still missing when you look at the final hosts file? What if you run /jffs/scripts/hosts.postconf /etc/hotsts manually (deliberately misspelled hosts for forum posting reasons)?

What I don't see is any messages from dnsmasq itself. Do you have any options to redirect dnsmasq log-facility to a file or some other adblocking script? Why is the instance at 192.168.52.1 unable to startup? What is already listening on that IP and port 53?

 

[CornfieldWin]

If the information you want to add to dnsmasq.conf is static and not going to change, just put it in the dnsmasq.conf.add file and delete the dnsmasq.postconf. postconf is generally only needed if you want to delete or modify a line from the original conf, or if your changes are dynamic in nature requiring actual scripting. If all you want to do is append configuration lines, just use dnsmasq.c

 

[CornfieldWin]

1. Answered: Regenerate the system/firmware config files manually as needed only to maintain Vlan and main LAN DNS consistency as described in the script if and only if

• Applicable change in GUI configuration of main LAN or Vlan related DNS values (add, delete or modify network name or address), (inclusive) or
• A desired change regarding which Vlans are to share DNS (or not) with the main LAN
• This happens independent of boot or service restart but after an update perform a boot or service restart to put the updates into service.

Thus the dnsmasq.postconf script should not be in jss scripts (best to rename) but executed manually like a make file to maintain consistency of DNS spanning features not in the GUI. Might make a simple and powerful extension in the Guest Pro GUI like the asuswrt-Merlin unique iptables check box but that is out of scope.

2. Problem: Why doesn't hosts.postconf work as described upon boot like it does as reported when run manually in place from SSH?

3. Problem: Why doesn't addn-host work? Those are basic firmware and dnsmasq services. They are essential to the dnsmasq to dnsmasq DNS Vlan scheme described elsewhere if it is to work at all.

So the side issue (1) has been settled but not the essential problems (2) and (3).

Note: it is beyond the scope of my knowledge why the syslog messages appear other than to note that dnsmasq instances run on every single TCP and UDP DNS related port. Here are the currently running processes with dnsmasq in their name:
11515 nobody 2792 S dnsmasq --log-async
11516 windsong 2660 S dnsmasq --log-async
11523 nobody 2660 S dnsmasq -C \etc\dnsmasq-1.conf --log-async
11524 windsong 2660 S dnsmasq -C \etc\dnsmasq-1.conf --log-async
11530 nobody 2660 S dnsmasq -C \etc\dnsmasq-2.conf --log-async
11531 windsong 2660 S dnsmasq -C \etc\dnsmasq-2.conf --log-async

I put nothing on any DNS port but the Guest Pro Portal (SND=1, Vlan=52 by default) is broken. It puts up the splash page but will not verify a correct access code solidly returning a fail message even if there is no access code configured. That problem is (4) for future debugging.

 

[CornfieldWin]

Your system log excerpt shows hosts.postconf being executed. Hosts will get recreated every time dnsmasq restarts. What isn't it doing or what is still missing when you look at the final hosts file? What if you run /jffs/scri

pts/hosts.postconf /etc/hotsts manually (deliberately misspelled hosts for forum posting reasons)?

What I don't see is any messages from dnsmasq itself. Do you have any options to redirect dnsmasq log-facility to a file or some other adblocking script? Why is the instance at 192.168.52.1 unable to startup? What is already listening on that IP and port 53?

Just change TARGET to CONFIG ($1 which is /etc/dnsmasq.conf passed via parameter). Same for the hosts.postconf.

echo $DNSMASQDOMAINS | awk '{for (i=4; i<NF; i=i+2) print "server=/" $(i+1) "." $1 "/" $i}' >>$CONFIG

It might be easier if you just post an example of the desired final output in dnsmasq.conf, and we can do away with a lot of those files and variables.

 

[CornfieldWin]

Same difference in this case - if only hosts.postconf would just run at boot time, which so far it has not. The use of shell variables as symbolic names is intentional for reasons of good coding practice, clarity and consistency across scripts to enforce configuration relationships, and parallel to use of global variables in header files.

As reported \etc\hosts [how did you insert system file names into a post?] never updates and the jffs touch file as mentioned in asuswrt_Merlin scripting never appears.

 

[dave14305]

Dec 31 19:00:40 custom_script: Running hosts.postconf (args:hosts) # System directories removed to prevent posting conflict
Dec 31 19:01:11 custom_script: Running hosts.postconf (args: hosts)
Dec 31 19:01:43 custom_script: Running hosts.postconf (args: hosts)
Dec 31 19:01:47 custom_script: Running hosts.postconf (args: hosts)
Sep 22 22:06:54 custom_script: Running hosts.postconf (args: hosts)

You keep saying it doesn't run at boot time, but it clearly did run based on your logs. Dec 31 is during boot up before NTP syncs.

The use of shell variables as symbolic names is intentional for reasons of good coding practice, clarity and consistency across scripts to enforce configuration relationships, and parallel to use of global variables in headers files.

That's really paying off during troubleshooting. All that "clarity".

 

[CornfieldWin]

You keep saying it doesn't run at boot time, but it clearly did run based on your logs. Dec 31 is during boot up before NTP syncs.

That's really paying off during troubleshooting. All that "clarity".

Yes and no. Syslog says it ran if you trust the log message. But there is no sign that it actually spawned and executed. The hosts file did not change and the touch file in jffs did not appear as did from SSH. So did it actually execute or something intended to launch it but didn't. Frustrating, I see no conclusive sign of what happened. Perhaps there is an owner or user group complication that is less than obvious given the permission a+rx as indicated in the guidelines.

 

[dave14305]

But there is no sign that it actually spawned and executed.

There must be an error in your script. Hard to tell because the version you posted earlier has blantant errors with the shebang and some variable quoting.