What's new

wgclient-start with iptables nat rules after router reboot

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

zjarkov

New Around Here
I have a wgclient-start (and its wgclient-stop counterpart) with some iptables rules to modify a nat. It works as expected when enabling/disabling wg client. However, after router (RT-AX86U with 388.2_2) reboot, all iptables rules set by the script are missing though the script has been obviously launched - I can see traces in syslog and also has put touch /tmp/wgclient$1-start into wgclient-start for sure.
Does anybody know some workaround or how can I trace what is wrong?
 
My guess... the firewall gets reloaded multiple times on startup, what Im thinking is wgclient-start is run at boot and later on during boot the firewall gets reloaded erasing your rules... check in syslog for May 4 23:05:11 MyROUTERname rc_service: waitting "restart_firewall" via ... appearing after wgclient-start has run
 
Last edited:
My guess... the firewall gets reloaded multiple times on startup, what Im thinking is wgclient-start is run at boot and later on during boot the firewall gets reloaded erasing your rules... check in syslog for May 4 23:05:11 MyROUTERname rc_service: waitting "restart_firewall" via ... appearing after wgclient-start has run
Thrilled for a while, but this doesn't seems to be the case :-(

Code:
@router:/tmp/home/root# grep -ni restart_firewall /tmp/syslog.log*
/tmp/syslog.log:428:May  5 07:05:10 rc_service: service 1511:notify_rc restart_firewall
/tmp/syslog.log:541:May  5 07:05:13 rc_service: waitting "restart_firewall" via  ...
@router:/tmp/home/root# grep -ni wgclient /tmp/syslog.log*
/tmp/syslog.log:591:Nov 25 02:19:59 custom_script: Running /jffs/scripts/wgclient-start (args: 1)
 
:confused: bummer, was a wild a** guess.

Someone else might have a better idea.
 
Thrilled for a while, but this doesn't seems to be the case :-(

Code:
@router:/tmp/home/root# grep -ni restart_firewall /tmp/syslog.log*
/tmp/syslog.log:428:May  5 07:05:10 rc_service: service 1511:notify_rc restart_firewall
/tmp/syslog.log:541:May  5 07:05:13 rc_service: waitting "restart_firewall" via  ...
@router:/tmp/home/root# grep -ni wgclient /tmp/syslog.log*
/tmp/syslog.log:591:Nov 25 02:19:59 custom_script: Running /jffs/scripts/wgclient-start (args: 1)
I think @Maverickcdn is correct anyway. I see 3 different ways of solving this
1. Put your nat rules in nat-start instead.
2. Restarting wg peers in nat-start
3. Fw uses /etc/wg/fw_wgc1_nat.sh to store these rules, you could pipe them there. This would be elegant but may require some tinkering, but I could help you.
 
:confused: bummer, was a wild a** guess.

Someone else might have a better idea.
Hi, your guess is correct at the end, just traces in syslog are not as originally expected, thx for a tip.

I let log file to be polluted from nat-start and wgclient-start and (at least) in my case the nat-start is called 3 times during boot when the last call is done approx 13s after WG start script.
 
Last edited:
I think @Maverickcdn is correct anyway. I see 3 different ways of solving this
1. Put your nat rules in nat-start instead.
2. Restarting wg peers in nat-start
3. Fw uses /etc/wg/fw_wgc1_nat.sh to store these rules, you could pipe them there. This would be elegant but may require some tinkering, but I could help you.
Thank you for the options.
I have a few questions to 3)
  • Does it persist between FW upgrades?
  • Under what condition it is (re)generated? I presume I could loose customization from time to time.
 
Last edited:
Does it persist between FW upgrades?
This is the tinkering needed, from the date mark on my file I would say it's created when you hit apply in the wg client gui section (so, when you create or change something). I wouldn't think it's changed at fw update but I don't know. I haven't updated since I created mine.

Under what condition it is (re)generated? I presume I could loose customization from time to time.
I don't know if there are any other reasons for recreation.

So, to be safe, maybe your wgclient-start should check some #tag you put there and if it does not exist you need to re-add them. Wgclient-start probably also need to execute them.
 
Last edited:
At the end I have used nat-start, as suggested by @ZebMcKayhan, to check for running wg clients and rerun wgclient-start which has been made safe to reload/rerun - avoiding adding duplicate nat rules.
 
Last edited:
Would you mindre sharing your nat-start code part that does this so others could benefit?
Sure, anyway, I'll be thankful if you find any problem or possible vulnerability...
Here you go:

nat-start
Bash:
#!/bin/sh

# cycle all running WireGuard clients
for wgc in `ls /tmp/wgclient-enabled_* 2>/dev/null`; do
    # reload/re-run the `wgclient-start X`
    /jffs/scripts/wgclient-start ${wgc/*_/}
done

wgclient-start
Bash:
#!/bin/sh

# set NAT rules for WireGuard client
/jffs/scripts/wgclient-natrules $1 I
# mark that WG client start script has been run
touch /tmp/wgclient-enabled_$1

wgclient-stop
Bash:
#!/bin/sh

# remove NAT rules for WireGuard client
/jffs/scripts/wgclient-natrules $1 D
# remove mark that WG client start script has been run
rm -f /tmp/wgclient-enabled_$1

wgclient-natrules
Bash:
#!/bin/sh

USAGE=$(cat << EOF
USAGE:
    `basename $0` WGCLIENT_NUM [OPERATION]

        WGCLIENT_NUM    WireGuard client number, the same as passed
                        to 'wgclient-start' and 'wgclient-stop' scripts.
        OPERATION       Operation to be performed, valid values are 'I'
                        (insert) or 'D' (delete). Optional, default 'I'.
EOF
)

# check input arguments
OP="${2:-I}"
[ 'I' == $OP -o 'D' == $OP ] || { echo -e "Invalid OPERATION '$OP'\n$USAGE" >&2; exit 1; }

# helper to process the nat rule
natrule()
{
    if [ 'I' == $OP ]; then
        iptables -t nat -C $@ 2>/dev/null || iptables -t nat -I $@
    else
        iptables -t nat -D $@
    fi
}

# rules for WireGuard client 1
if [ 1 -eq $1 ]; then
    # manage custom NAT rules here
    natrule PREROUTING -i wgc$1 -p tcp --dport 55443 -s 10.8.50.0/28 -j DNAT --to-destination 192.168.0.15:443
    natrule POSTROUTING -o br0 -p tcp -s 10.8.50.0/28 -d 192.168.0.15 --dport 443 -j MASQUERADE

    # ...
fi
 
Last edited:
Sure, anyway, I'll be thankful if you find any problem or possible vulnerability...
wgclient-natrules
Bash:
# check input arguments
[ 'I' == $OP -o 'D' == $OP ] || { echo -e "Invalid OPERATION '$OP'\n$USAGE"
Missing a closing brace } on your echo or its an erroneous {

Otherwise glad we were able to lead you in the right direction!
 
@zjarkov Thanks for sharing! Nice way to setup your rules in a single line like this!

I couldnt keep my hands off, so I borrowed your rules and made this, mostly because I'm curious if it works. It does the job but I don't have the ability to test against the firmware right now:

wgclient-start
Code:
#!/bin/sh

file="/etc/wg/fw_wgc1_nat.sh"

cat <<EOF | while read -r NAT_RULE;
iptables -t nat -I PREROUTING -i wgc1 -p tcp --dport 55443 -s 10.8.50.0/28 -j DNAT --to-destination 192.168.0.15:443
iptables -t nat -I POSTROUTING -o br0 -p tcp -s 10.8.50.0/28 -d 192.168.0.15 --dport 443 -j MASQUERADE
EOF

do
if [ -z "$(cat "$file" | grep "$NAT_RULE")" ];then #if rule is not found in files
   echo $NAT_RULE >> $file #append rule to file
fi
   eval $NAT_RULE #Execute rule
done

wgclient-stop
Code:
#!/bin/sh
iptables -t nat -D PREROUTING -i wgc1 -p tcp --dport 55443 -s 10.8.50.0/28 -j DNAT --to-destination 192.168.0.15:443 
iptables -t nat -D POSTROUTING -o br0 -p tcp -s 10.8.50.0/28 -d 192.168.0.15 --dport 443 -j MASQUERADE

I'm not a coder so feel free to criticize.


It will look if the rules are in the fw file and add them if they are not there. Then execute them.
but if you change the rules it will not delete any entries in the fw file and I don't know when this file is recreated, possibly never, so one needs to check this when updating the rules.
 
@zjarkov Thanks for sharing! Nice way to setup your rules in a single line like this!

I couldnt keep my hands off, so I borrowed your rules and made this, mostly because I'm curious if it works. It does the job but I don't have the ability to test against the firmware right now:

wgclient-start
Code:
#!/bin/sh

file="/etc/wg/fw_wgc1_nat.sh"

cat <<EOF | while read -r NAT_RULE;
iptables -t nat -I PREROUTING -i wgc1 -p tcp --dport 55443 -s 10.8.50.0/28 -j DNAT --to-destination 192.168.0.15:443
iptables -t nat -I POSTROUTING -o br0 -p tcp -s 10.8.50.0/28 -d 192.168.0.15 --dport 443 -j MASQUERADE
EOF

do
if [ -z "$(cat "$file" | grep "$NAT_RULE")" ];then #if rule is not found in files
   echo $NAT_RULE >> $file #append rule to file
fi
   eval $NAT_RULE #Execute rule
done

wgclient-stop
Code:
#!/bin/sh
iptables -t nat -D PREROUTING -i wgc1 -p tcp --dport 55443 -s 10.8.50.0/28 -j DNAT --to-destination 192.168.0.15:443 
iptables -t nat -D POSTROUTING -o br0 -p tcp -s 10.8.50.0/28 -d 192.168.0.15 --dport 443 -j MASQUERADE

I'm not a coder so feel free to criticize.


It will look if the rules are in the fw file and add them if they are not there. Then execute them.
but if you change the rules it will not delete any entries in the fw file and I don't know when this file is recreated, possibly never, so one needs to check this when updating the rules.
Didn't try it yet but in general that could work with some adjustments.
At first you can do the test more elegant way if grep -q "$NAT_RULE" $file; then and as second you should remove particular rules from /etc/wg/fw_wgc1_nat.sh in wgclient-stop, otherwise, when you stop/disable the client, the rules will persist and 'll be applied after router reboot.
 
At first you can do the test more elegant way if grep -q "$NAT_RULE" $file; then
thanks, Ill make some quick test this evening.

second you should remove particular rules from /etc/wg/fw_wgc1_nat.sh in wgclient-stop, otherwise, when you stop/disable the client, the rules will persist and 'll be applied after router reboot.
I thought this was taken care of in the fw part handling nat events, but I just checked the code and it doesnt. I'm not sure, but it looks like the fw might be changing -I to -D in this file using sed when the peer is stopped, but that would apply to our rules as well. it then looks like the file is deleted after it has been executed. maybe this takes care of both removing our rules and preventing further execution:
snprintf(path, sizeof(path), "%s/fw_%s_nat.sh", WG_DIR_CONF, ifname);
if(f_exists(path)) {
eval("sed", "-i", "s/-I/-D/", path);
eval("sed", "-i", "s/-A/-D/", path);
eval(path);
unlink(path);

the file would need to be re-created when peer is started, but wonder what happens during reboot...

maybe you are better at reading the fw code than me: https://github.com/RMerl/asuswrt-merlin.ng/blob/master/release/src/router/rc/wireguard.c
 
Didn't try it yet but in general that could work with some adjustments.
At first you can do the test more elegant way if grep -q "$NAT_RULE" $file; then and as second you should remove particular rules from /etc/wg/fw_wgc1_nat.sh in wgclient-stop, otherwise, when you stop/disable the client, the rules will persist and 'll be applied after router reboot.
I can confirm that firmware removes ALL rules added to this file, so it's not needed to remove any rules in wgclient-stop. The file is also removed when the client is stopped. I have not checked reboot yet or if all is executed OK at nat-flush event.

I changed the grep statement according to your suggestion and added to only apply on the right peer start. This works for me:
wgclient-start
Code:
#!/bin/sh

if [ "$1" -eq "2" ]; then # only for wgc2
   file="/etc/wg/fw_wgc2_nat.sh"
cat <<EOF | while read -r NAT_RULE;
iptables -t nat -I PREROUTING -i wgc2 -p tcp --dport 55443 -s 10.8.50.0/28 -j DNAT --to-destination 192.168.0.15:443
iptables -t nat -I POSTROUTING -o br0 -p tcp -s 10.8.50.0/28 -d 192.168.0.15 --dport 443 -j MASQUERADE
EOF
   do
      if ! grep -q "$NAT_RULE" $file; then
         echo $NAT_RULE >> $file
      fi
         eval $NAT_RULE
   done
fi

wgclient-stop
not needed

I will test reboots tomorrow but I don't expect there to be any issues.
 
Last edited:
wgclient-start
Code:
#!/bin/sh

if [ "$1" -eq "2" ]; then # only for wgc2
   file="/etc/wg/fw_wgc2_nat.sh"
cat <<EOF | while read -r NAT_RULE;
iptables -t nat -I PREROUTING -i wgc2 -p tcp --dport 55443 -s 10.8.50.0/28 -j DNAT --to-destination 192.168.0.15:443
iptables -t nat -D POSTROUTING -o br0 -p tcp -s 10.8.50.0/28 -d 192.168.0.15 --dport 443 -j MASQUERADE
EOF
   do
      if ! grep -q "$NAT_RULE" $file; then
         echo $NAT_RULE >> $file
      fi
         eval $NAT_RULE
   done
fi

wgclient-stop
not needed
This actually could be nice all-in-one script :)

Questions:
  1. Are /etc/wg/* files anyhow documented and stable (naming, existence, its purpose, whatever...)? I am a "normal" user who usually fiddle only with official APIs to avoid future problems (unannounced change of background stuff).
  2. I might be wrong but IMHO this way it is rather wgclient%d.postconf than wgclient-start script. Of course it would need a slight modification to be truly postconf.
 
Are /etc/wg/* files anyhow documented and stable (naming, existence, its purpose, whatever...)? I am a "normal" user who usually fiddle only with official APIs to avoid future problems (unannounced change of background stuff).
No, they are not official in that sense. I guess a major change to the code could change their names or behaviour. Possibly the 3.0.0.6 release would render this useless (together with many add-ons).


I might be wrong but IMHO this way it is rather wgclient%d.postconf than wgclient-start script. Of course it would need a slight modification to be truly postconf.
Well, in the sense as we altering config files, but wg postconf is executed after /etc/wg/wgcX.conf is created and before it is sent to the interface. Our nat file wouldn't exist then.
 
This actually could be nice all-in-one script
I could confirm that it does the job through ordinary reboot. I wouldn't expect accidental reboot to be any difference.
And, as you say, nat rules gets flushed after wgc start and the rules are still there so it will be applied during nat-flush events as well.

Of course the use of this file is unofficial, so maybe this is not the way to go, but I leave it here for anyone to use if they wish. I wouldn't expect this to change in the coming minor releases but who knows for the next major release.
 

Similar threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top