What's new

iptables, ipsets and VPN 'kill switch'

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

A

andrew2018

Guest
Hi everyone, I've tried everything I can think of but can't find a solution to a routing problem. I have a RT-AC68U (a genuine one running Merlin 384.6) with two separate VPN connections. I'm using scripts (thanks to Martineau and others) that set up policy-based routing using ipsets.

It all works great except when one of the VPN connections drops or I take it down manually. Traffic that I want to go out only via that VPN connection starts going out via the WAN connection instead and keeps doing that even when the VPN connection comes back up.

I've been testing it by putting a couple of useful web sites like dnsleaktest.com and whatismyipaddress.com in different ipsets so I can tell whether they are seeing my traffic through a VPN or the WAN.

Can someone see what I'm missing please? Thanks for any help.

nat-start script
Code:
#!/bin/sh

ip rule del prio 9990
ip rule add fwmark 0x7000 table main prio 9990
ip rule del fwmark 0x1000
ip rule add fwmark 0x1000 table ovpnc1 prio 9991
ip rule del fwmark 0x2000
ip rule add fwmark 0x2000 table ovpnc2 prio 9992

# WAN (traffic that must be forced to go via ISP)
iptables -v -t mangle -D PREROUTING -i br0 -p all -m set --match-set WAN0 dst,dst -j MARK --set-mark 0x7000/0x7000
iptables -v -t mangle -A PREROUTING -i br0 -p all -m set --match-set WAN0 dst,dst -j MARK --set-mark 0x7000/0x7000

# VPN 1 (VPN client 1)
iptables -v -t mangle -D PREROUTING -i br0 -p all -m set --match-set VPN1 dst,dst -j MARK --set-mark 0x1000/0x1000
iptables -v -t mangle -A PREROUTING -i br0 -p all -m set --match-set VPN1 dst,dst -j MARK --set-mark 0x1000/0x1000

# VPN 2 (VPN client 2)
iptables -v -t mangle -D PREROUTING -i br0 -p all -m set --match-set VPN2 dst,dst -j MARK --set-mark 0x2000/0x2000
iptables -v -t mangle -A PREROUTING -i br0 -p all -m set --match-set VPN2 dst,dst -j MARK --set-mark 0x2000/0x2000

services-start script
Code:
#!/bin/sh

modprobe xt_set

ipset create WAN0 list:set
ipset create VPN1 list:set
ipset create VPN2 list:set

ipset create WAN0-novpn                 hash:net family inet hashsize 1024 maxelem 65536
ipset add WAN0 WAN0-novpn

ipset create VPN1-group1            hash:net family inet hashsize 1024 maxelem 65536
ipset create VPN1-group2          hash:net family inet hashsize 1024 maxelem 65536
ipset create VPN1-group3           hash:net family inet hashsize 1024 maxelem 65536
ipset add VPN1 VPN1-group1
ipset add VPN1 VPN1-group2
ipset add VPN1 VPN1-group3

ipset create VPN2-group1            hash:net family inet hashsize 1024 maxelem 65536
ipset create VPN2-group2          hash:net family inet hashsize 1024 maxelem 65536
ipset create VPN2-group3           hash:net family inet hashsize 1024 maxelem 65536
ipset add VPN2 VPN2-group1
ipset add VPN2 VPN2-group2
ipset add VPN2 VPN2-group3


When VPN connections come up, I also run a script that restarts dnsmasq.

dnsmasq.conf.add
Code:
# Never forward plain names (without a dot or domain part)
domain-needed

# WAN0 - No VPN
# Set up ipsets and direct DNS traffic to the DNS server of the WAN provider
ipset=/site1.com/site2.com/WAN0-novpn
server=/site1.com/site2.com/NNN.NNN.NNN.NNN

# VPN1
# Set up ipsets and direct DNS traffic to the DNS server of the VPN provider
ipset=/site3.com/site4.com/VPN1-group1
server=/site3.com/site4.com/MMM.MMM.MMM.MMM

and so on for the other ipsets
 
It all works great except when one of the VPN connections drops or I take it down manually. Traffic that I want to go out only via that VPN connection starts going out via the WAN connection instead and keeps doing that even when the VPN connection comes back up.
This may help with the issue of traffic traversing the WAN if the VPN goes down.

From the Policy based routing Wiki...

Another setting exposed when enabling Policy routing is to prevent your routed clients from accessing the Internet if the VPN tunnel goes down. To do so, enable "Block routed clients if tunnel goes down".
 
Thank you. I should have said that I've got the "Block routed clients if tunnel goes down" box ticked for both the VPN clients.
 
I plan to publish my selective routing scripts on github soon. They may be able to satisfy your use case. Here are some snips. Note I list the table as a number rather than using ovpnc1, etc. Individual clients and website are routed using the ip rule command. Streaming media IPv4 addresses are matched to a fwmark and iptables routes accordingly.

Code:
create_fwmarks () {
# WAN
    ip rule del fwmark 0x7000/0x7000 > /dev/null 2>&1
    ip rule add fwmark 0x7000/0x7000 table 254 prio 4000
    
#VPN Client 1
    ip rule del fwmark 0x1000/0x1000 > /dev/null 2>&1
    ip rule add fwmark 0x1000/0x1000 table 111 prio 5010


#VPN Client 2
    ip rule del fwmark 0x2000/0x2000 > /dev/null 2>&1
    ip rule add fwmark 0x2000/0x2000 table 112 prio 5020
<snip>

Code:
 iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set LAN_GW src,dst -j MARK --set-mark 0x7000/0x7000 > /dev/null 2>&1
    iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set LAN_GW src,dst -j MARK --set-mark 0x7000/0x7000

Code:
# Routing rules for OVPNC1 traffic go here. Start with prio 4101 and increment by 1
# Route https://www.whatismyip.com/ traffic to OVPNC1
    ip rule del to 104.27.192.92 table 111 > /dev/null 2>&1
    ip rule add to 104.27.192.92 table 111 prio 4101
    ip rule del to 104.27.193.92 table 111 > /dev/null 2>&1
    ip rule add to 104.27.193.92 table 111 prio 4101
<snip>

Code:
 ip rule del from 192.168.1.152 table 111 > /dev/null 2>&1
 ip rule add from 192.168.1.152 table 111 prior 9901
 
I've found a way (don't know if it's the best way) to prevent traffic that should go via a VPN connection from being sent via the WAN connection if the VPN goes down. I added a few lines to nat-start:

Code:
# Prevent packets matching ipsets from going out via an interface that doesn't match their set
iptables -v -t mangle -D POSTROUTING ! -o tun11 -p all -m set --match-set VPN1 dst,dst -j DROP                   
iptables -v -t mangle -A POSTROUTING ! -o tun11 -p all -m set --match-set VPN1 dst,dst -j DROP                   
iptables -v -t mangle -D POSTROUTING ! -o tun12 -p all -m set --match-set VPN2 dst,dst -j DROP                   
iptables -v -t mangle -A POSTROUTING ! -o tun12 -p all -m set --match-set VPN2 dst,dst -j DROP

Is this ok or is there another way? Thanks.
 
I've found a way (don't know if it's the best way) to prevent traffic that should go via a VPN connection from being sent via the WAN connection if the VPN goes down. I added a few lines to nat-start:

Code:
# Prevent packets matching ipsets from going out via an interface that doesn't match their set
iptables -v -t mangle -D POSTROUTING ! -o tun11 -p all -m set --match-set VPN1 dst,dst -j DROP                  
iptables -v -t mangle -A POSTROUTING ! -o tun11 -p all -m set --match-set VPN1 dst,dst -j DROP                  
iptables -v -t mangle -D POSTROUTING ! -o tun12 -p all -m set --match-set VPN2 dst,dst -j DROP                  
iptables -v -t mangle -A POSTROUTING ! -o tun12 -p all -m set --match-set VPN2 dst,dst -j DROP

Is this ok or is there another way? Thanks.
Most people use the built in kill switch setting on the gui that appears when you enable Policy Rules or Policy Rules (Strict)

https://github.com/RMerl/asuswrt-merlin/wiki/Policy-based-routing
 
Thanks. Why I’m doing it this way is because I have IPsets that route traffic to particular domains via particular VPN clients. None of my devices are permanently assigned to a particular VPN client, so the built-in approach doesn’t work.
 
Last edited by a moderator:
Why I’m doing it this way is because I have IPsets that route traffic to particular domains via particular VPN clients.

None of my devices are permanently assigned to a particular VPN client, so the built-in approach doesn’t work.

Not sure if I fully understand your explanation to not use the built-in GUI kill switch capability? o_O

However, your solution is perfectly valid (although I personally use the '-t filter FORWARD' chain), but if you are truly paranoid about a WAN leak when accessing your VPN only domains, then perhaps you should modify your nat-start script to check if your custom fail-safe WAN blocking rule(s) already exists to eliminate the need to delete/re-add the rule(s). ;)
 
Last edited:
Not sure if I fully understand your explanation to not use the built-in GUI kill switch capability? o_O

Thanks. I have checked the box in the UI to block traffic if each VPN goes down but that seems to have no effect.

My theory is that it’s because I don’t have any entries in the (user interface) address list for each VPN client. That is, no clients are specified to be permanently routed via a VPN. As far as I can tell that means the router doesn’t have any ‘kill switch’ functionality in my case. If a VPN connection goes down, the router doesn’t know there are any clients to block because no clients have been specified to route via that VPN (in the user interface).

That’s why I found I needed to add the rules I listed earlier. I’m interested to understand this better from those with much more experience than I have in this.
 
Thanks. I have checked the box in the UI to block traffic if each VPN goes down but that seems to have no effect.

My theory is that it’s because I don’t have any entries in the (user interface) address list for each VPN client. That is, no clients are specified to be permanently routed via a VPN. As far as I can tell that means the router doesn’t have any ‘kill switch’ functionality in my case. If a VPN connection goes down, the router doesn’t know there are any clients to block because no clients have been specified to route via that VPN (in the user interface).

That’s why I found I needed to add the rules I listed earlier. I’m interested to understand this better from those with much more experience than I have in this.

D'oh :rolleyes: - you are correct.

For the Selective routing of Ports/MACs and IPSETs, I have always created a 'dummy' VPN Client entry since I require the ability to exploit the 'Accept DNS Configuration=EXCLUSIVE' option that only creates the appropriate DNSVPNX chains if the table isn't empty......never occurred to me that the GUI entries also influences the kill switch :rolleyes:

e.g. My local LAN uses 10.xxx.xxx.xxx, so for VPN Client 1 I enter:
Code:
DummyVPN1   172.16.1.1   0.0.0.0   VPN

If you create the dummy entry, then the 'prohibit' kill switch will be enabled in the appropriate ovpncX routing table.

NOTE: You need to be aware of the limitations of setting the kill switch for multiple VPN Clients; if you intend to dynamically switch/share the IPSETs between active VPN clients.
 
Last edited:
D'oh :rolleyes: - you are correct.

For the Selective routing of Ports/MACs and IPSETs, I have always created a 'dummy' VPN Client entry since I require the ability to exploit the 'Accept DNS Configuration=EXCLUSIVE' option that only creates the appropriate DNSVPNX chains if the table isn't empty......never occurred to me that the GUI entries also influences the kill switch :rolleyes:

e.g. My local LAN uses 10.xxx.xxx.xxx, so for VPN Client 1 I enter:
Code:
DummyVPN1   172.16.1.1   0.0.0.0   VPN

If you create the dummy entry, then the 'prohibit' kill switch will be enabled in the appropriate ovpncX routing table.

NOTE: You need to be aware of the limitations of setting the kill switch for multiple VPN Clients; if you intend to dynamically switch/share the IPSETs between active VPN clients.

Very interesting.

I use Accept DNS Configuration = Strict since Exclusive bypasses dnsmasq. I need to use dnsmasq for ad blocking. I have to add an entry dhcp-option DNS some.dns.ip.address in the Custom Config section, e.g dhcp-option DNS 1.1.1.1 to avoid routing issues. The downside is DNS leaks. But it has not created an issue for my use case.

My selective routing scripts were using the ip rule add from 192.168.1.152 table 111 prior 6000 syntax to add clients to the correct interface. But in my testing, I realized I was bypassing the DNSVPNX chain which would impact users who want to use Accept DNS Configuration = Exclusive. Not an issue if using Strict. I spent two days trying a modified updown.sh script to duplicate what Asuswrt-Merlin was doing. I encountered issues. So, I changed directions and decided to load the client rules in the web gui instead.

I have a file containing lan clients I load from static dhcp nvram variable. It creates a file like this:
Code:
1 192.168.1.10 my_laptop
2 192.168.1.11 my_streaming_media_device
<snip>
I can edit the file and change the first number to represent the iface, e.g. 0=WAN, 1=OVPNC1, etc..

I then use this file as input to load the clients to the nvram variable vpn_client1_clientlist.

Code:
create_lan_client_routes () {
# Implement routing rules for LAN clients configured in /jffs/configs/x3mRouting_lan_client_rules

    old_IFS="$IFS"
    IFS=" "
    DEST="0.0.0.0"
    VPN="VPN"
    RECORD_BEGIN="<"
    FIELD_SEPARATOR=">"
 
    while IFS=" " read -r OVPNC IP DESCRIPTION
    do
        if [ $(echo "$OVPNC" | cut -c 1) != "#" ]; then
            if [ "$OVPNC" -ge "0" ]; then
                case "$OVPNC" in
                    1) printf '%s%s%s%s%s%s%s%s\n' "$RECORD_BEGIN" "$DESCRIPTION" "$FIELD_SEPARATOR" "$IP" "$FIELD_SEPARATOR" "$DEST" "$FIELD_SEPARATOR" "$VPN" >> /tmp/ovpnc1.$$ ;;
                    2) printf '%s%s%s%s%s%s%s%s\n' "$RECORD_BEGIN" "$DESCRIPTION" "$FIELD_SEPARATOR" "$IP" "$FIELD_SEPARATOR" "$DEST" "$FIELD_SEPARATOR" "$VPN" >> /tmp/ovpnc2.$$ ;;
                    3) printf '%s%s%s%s%s%s%s%s\n' "$RECORD_BEGIN" "$DESCRIPTION" "$FIELD_SEPARATOR" "$IP" "$FIELD_SEPARATOR" "$DEST" "$FIELD_SEPARATOR" "$VPN" >> /tmp/ovpnc3.$$ ;;
                    4) printf '%s%s%s%s%s%s%s%s\n' "$RECORD_BEGIN" "$DESCRIPTION" "$FIELD_SEPARATOR" "$IP" "$FIELD_SEPARATOR" "$DEST" "$FIELD_SEPARATOR" "$VPN" >> /tmp/ovpnc4.$$ ;;
                    5) printf '%s%s%s%s%s%s%s%s\n' "$RECORD_BEGIN" "$DESCRIPTION" "$FIELD_SEPARATOR" "$IP" "$FIELD_SEPARATOR" "$DEST" "$FIELD_SEPARATOR" "$VPN" >> /tmp/ovpnc5.$$ ;;
                esac
            fi
        fi
    done < /jffs/configs/x3mRouting_lan_client_rules
     IFS=$old_IFS

# route OVPNC1 clients
    if [ -s "/tmp/ovpnc1.$$" ]; then
            awk '{ print }' ORS='' < "/tmp/ovpnc1.$$" > /tmp/ovpnc
            nvram set vpn_client1_clientlist=$(cat /tmp/ovpnc)
            nvram commit
            service restart_vpnclient1 &
    fi

<snip>
I get the concern with the kill switch. However, in my 2.5 years of using my provider, I've never had my OpenVPN connection go down. I wonder how prevalent the issue of having the OpenVPN connection dropping is with other VPN providers?
 
Last edited:
I have 'strict' checked for all VPN clients but because no devices are permanently assigned to a VPN client they all use dnsmasq and Ab-solution seems to work fine.

Coming back to the config I posted earlier I'll look into martineau's suggestion of using the '-t filter FORWARD' chain instead of the POSTROUTING chain. I thought POSTROUTING might be a safer bet because it's the last possible stage.

Is there any important reason why not to use the delete/add combo with the rules in nat-start? Is it for efficiency or something? Seems faster than checking to see if the rule already exists. An error in deleting a nonexistent rule doesn't really matter does it?

My VPNs are pretty reliable and don't go down often but if the WAN goes down it takes the VPNs with it and they don't always come back up together. Is there any reason why you wouldn't want to have rules that prevent VPN traffic going out the WAN port just in case?
 
Is there any important reason why not to use the delete/add combo with the rules in nat-start?
Can you afford to take the risk that a client device initiates a request during the admittedly brief 'delete' window and consequently leaks via the WAN? - given that nat-start can be 'randomly' executed even if the WAN/VPN are actually both UP and connected.
Seems faster than checking to see if the rule already exists.
Hmm cutting corners when it comes to security always ends badly.
However, implementing the kill rule as quickly as possibly is paramount (preferably before the WAN is UP) and could possibly be improved by say immediately inserting a new iptables kill rule (regardless) at the top of the chain, then removing any duplicates in the chain at your leisure.
An error in deleting a nonexistent rule doesn't really matter does it?
No, but deleting an existing fully functional kill switch rule then failing to check that the replacement rule was successfully recreated is going to be potentially disasterous.
i.e. few scripts explicitly check that the iptables kill rule was successfully created (post delete).

Now whilst the risk of either of the above is extremely low, placing primary importance on a VPN Kill switch rightly or wrongly would suggest that your activities could be inferred as dubious.;)

I'm sure that is not the case :D but surely you would wish to confidently eliminate any embarrassing exposure? and I was just trying to make you aware of ways to improve your iptables based kill switch implementation.
 
Good reason thanks. I just want to control which sites are accessed via VPN and which via WAN but none of it is critical. Not sure how to do something like you described but will think about it.
 
I just want to control which sites are accessed via VPN and which via WAN but none of it is critical.
Is it just websites you want to selectively route? I wrote a script that automates the process. All you have to do is list the iface and website name in a config file. It does a drill on the site name, parses the ipv4 address, and issues the ip rule command to the ipv4 address to the iface.
 
I have a file containing lan clients I load from static dhcp nvram variable. It creates a file like this:
I can edit the file and change the first number to represent the iface, e.g. 0=WAN, 1=OVPNC1, etc..
I then use this file as input to load the clients to the nvram variable vpn_client1_clientlist.
But in my testing, I realized I was bypassing the DNSVPNX chain which would impact users who want to use Accept DNS Configuration = Exclusive. Not an issue if using Strict.
I spent two days trying a modified updown.sh script to duplicate what Asuswrt-Merlin was doing.
I encountered issues.
So, I changed directions and decided to load the client rules in the web gui instead.
Not sure what issues you encountered?

For firmware v384.xx, NVRAM is a scarce resource and there are now size limits imposed which makes it necessary for some users to import settings such as a large number of Port Forwarding rules etc. from an external file.

So I patched the firmware scripts vpnrouting.sh and updown.sh in order to treat the VPN Policy rules Description field as an additional Custom Configuration panel.

e.g. Instruct the firmware to generate the VPN Client 1 RPDB rules from file '/jffs/configs/ovpnc1.nvram' rather than from NVRAM variable 'vpn_client1_clientlist'

upload_2018-8-15_19-11-19.png



Consequently, on the plus side, if there are say 50 RPDB entries in '/jffs/configs/ovpnc1.nvram', that is an NVRAM saving of almost 1500 bytes!, but the downside is that you are no longer able to review the Selective routing rules via the GUI, but will need to interrogate the active RPDB rules using the command 'ip rules'
 
Not sure what issues you encountered?
Hi @Martineau,

The issue is when using the ip rule command to route a client device to the VPN tunnel where Accept DNS Configuration = Exclusive, the DNSVPNx chain is not updated with the client device and DNS Leaks.

For example, in OVPNC2, if Accept DNS Configuration = Exclusive, and I add client device 192.168.22.152 to the Web GUI or, using a script to populate nvram variable vpn_client2_clientlist, I see the DNSVPN2 in the PREROUTING chain.

Code:
# iptables -nvL PREROUTING -t nat --line

Chain PREROUTING (policy ACCEPT 28 packets, 2568 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 DNSVPN2    tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53
2       12   756 DNSVPN2    udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:53

And, the client device is listed in the DNSVPN2 Chain and DNS does not leak.
Code:
# iptables --line -t nat -nvL DNSVPN2

Chain DNSVPN2 (2 references)
num   pkts bytes target     prot opt in     out     source               destination
1       12   756 DNAT       all  --  *      *       192.168.22.152       0.0.0.0/0            to:104.223.91.194
2        0     0 DNAT       all  --  *      *       192.168.22.155       0.0.0.0/0            to:104.223.91.194
I then remove the 192.168.22.152 from OPVPNC2 and add it OVPNC3 where Accept DNS Configuration = Exclusive using the ip rule command:
Code:
ip rule add from 192.168.22.152 table 113 prio 6000# route laptop to OVPNC3

But when I check the DNSVPN3 Chain, the client device 192.168.22.152 is not listed and DNS Leaks.
Code:
# iptables --line -t nat -nvL DNSVPN3

Chain DNSVPN3 (2 references)
num   pkts bytes target     prot opt in     out     source               destination
As a result, I felt I needed to change my selective routing script before posting on github to route the clients by updating the nvram variable vpn_clientX_clientlist rather than using the ip rule command so I don't deviate from the behavior of the firmware. I use Accept DNS Configuration = Strict since Exclusive bypasses dnsmasq, which prevents AB-Solution from working.

John's Fork and Merlin firmware has different behavior with Exclusive.

AB-Solution is the ad blocking solution for Asus routers using Asuswrt-Merin firmware. AB-Solution requires DNSmasq to work properly. With Asuswrt-Merlin firmware, OpenVPN clients use the VPN tunnel’s DNS. As a result, AB-Solution will not work for LAN clients connected to the VPN tunnel when using Policy Rules since DNSmasq is by-passed. AB-Solution will still work for devices connected to the WAN though.

John9547 LTS fork has implemented DNS differently than Asuswrt-Merlin. The DNS rules are reversed. With Accept DNS Configuration set to Exclusive, the VPN clients will use DNSmasq and AB-Solution will work. There is also a check box on how you want to handle the WAN clients. If you leave it unchecked, the WAN clients will also use the VPN DNS servers (but not the tunnel) and they can use AB-Solution. If you check the box, the WAN client requests are sent directly to the WAN DNS servers and AB-Solution will not be available.
 
Last edited:
I then remove the 192.168.22.152 from OPVPNC2 and add it OVPNC3 where Accept DNS Configuration = Exclusive using the ip rule command:
Code:
ip rule add from 192.168.22.152 table 113 prio 6000# route laptop to OVPNC3
But when I check the DNSVPN3 Chain, the client device 192.168.22.152 is not listed and DNS Leaks.
Since you are manually managing the creation of the RPDB rule, surely you would expect to also manually manage the DNS redirect? i.e. (crudely) issue (for the RMerlin frmware)
Code:
iptables -t nat -A DNSVPN3 -s 192.168.22.152 -j DNAT --to-destination $VPNDNS_IP 2>/dev/nul
P.S. So for john957's LTS fork you don't have to modify the DNSVPNX chain (if it exists) except for WAN entries?

But thanks for the explanation, and upon review it is no doubt easier to create the NVRAM variable in order to accommodate the GUI nuances of RMerlin's vs. john9527's interpretation of the meaning of 'Accept DNS Configuration=EXLUSIVE'.

My point was to illustrate that it is 'fairly' trivial to automate the inclusion of your '/tmp/ovpnc1.$$' rules pre-empting the need to actually modify the NVRAM 'clientlist' variable.

i.e. Patch both scripts vpnrouting.sh and updown.sh

vpnrouting.sh
Code:
VPN_IP_LIST=$(nvram get vpn_client${VPN_UNIT}_clientlist)$(nvram get vpn_client{VPN_UNIT}_clientlist1)$(nvram get vpn_client{VPN_UNIT}_clientlist2)$(nvram get vpn_client{VPN_UNIT}_clientlist3)$(nvram get vpn_client{VPN_UNIT}_clientlist4)$(nvram get vpn_client{VPN_UNIT}_clientlist5)
effectively becomes
Code:
VPN_IP_LIST=$(nvram get vpn_client${VPN_UNIT}_clientlist)$(nvram get vpn_client{VPN_UNIT}_clientlist1)$(nvram get vpn_client{VPN_UNIT}_clientlist2)$(nvram get vpn_client{VPN_UNIT}_clientlist3)$(nvram get vpn_client{VPN_UNIT}_clientlist4)$(nvram get vpn_client{VPN_UNIT}_clientlist5)
VPN_IP_LIST=$VPN_IP_LIST""$(cat "/tmp/ovpnc${$VPN_UNIT}.$$")
and
updown.sh
Code:
VPN_IP_LIST=$(nvram get vpn_client$(echo $instance)_clientlist)
effectively becomes
Code:
VPN_IP_LIST=$(nvram get vpn_client$(echo $instance)_clientlist)
VPN_IP_LIST=$VPN_IP_LIST""$(cat "/tmp/ovpnc${instance}.$$")

FYI, With a simple GUI tweak (and modifying vpnrouting.sh)

upload_2019-3-12_11-47-34.png



shows you can also now allow GUI specification of IPSETs to be Selectively routed!

Code:
Aug 16 12:22:22 RT-AC68U daemon.notice ovpn-client1[2517]: updown.sh tun11 1500 1552 xxx.vpn.isp.xxx 255.255.255.0 init

RT-AC68U user.warn (updown.sh): 2625 v384.07 Patched by Martineau [tun11 1500 1552 xxx.vpn.isp.xxx 255.255.255.0 init]

RT-AC68U user.warn (updown.sh): 2625 Martineau Appending VPN Clients from '/jffs/configs/ovpnc1.nvram'

RT-AC68U user.warn openvpn-updown: Forcing 172.16.1.0 to use DNS server xxx.vpn.dns.xxx
RT-AC68U user.warn openvpn-updown: Forcing 172.16.1.1 to use DNS server xxx.vpn.dns.xxx
RT-AC68U user.warn openvpn-updown: Excluding 172.16.1.100 from forced DNS routing
RT-AC68U user.warn openvpn-updown: Forcing 172.168.1.200 to use DNS server xxx.vpn.dns.xxx
RT-AC68U user.warn openvpn-updown: Forcing 172.16.100.1 to use DNS server xxx.vpn.dns.xxx
RT-AC68U user.warn openvpn-updown: Forcing 172.16.100.2 to use DNS server xxx.vpn.dns.xxx
RT-AC68U user.warn openvpn-updown: Forcing 172.16.100.3 to use DNS server xxx.vpn.dns.xxx
RT-AC68U kern.notice rc_service: service 2829:notify_rc updateresolv
RT-AC68U user.warn custom_script: Running /jffs/scripts/openvpn-event (args: tun11 1500 1552 xxx.vpn.isp.xxx 255.255.255.0 init)
RT-AC68U user.warn openvpn-event[2834]: User openvpn-event running
RT-AC68U user.warn openvpn-event[2834]:      Script executing.. for event: vpnclient1-up
RT-AC68U user.warn (vpnclient1-up): 2853 User Processing 'up' (tun11) via xxx.vpn.isp.xxx args = [tun11 1500 1552 xxx.vpn.isp.xxx 255.255.255.0 init]
RT-AC68U user.warn (vpnclient1-up): 2853 Added cru #GetVPN1_WANIP# every hh:01
RT-AC68U user.warn (VPN_SyslogMonitor.sh): 2885 VPN Syslog Event Monitor starting.....
RT-AC68U user.warn (vpnclient1-up): 2853 Acquiring lock semaphore '/tmp/WiFiVPN.sh-flock'
RT-AC68U user.warn (vpnclient1-up): 2853 Lock semaphore acquired '/tmp/WiFiVPN.sh-flock'
RT-AC68U user.warn (vpnclient1-up): 2853 Updated WiFi VPN map configuration '/jffs/configs/WiFiVPN_map' Client: '1 wl0.3 # HMA New York (connected Thu Aug 16 12:22:25 DST 2018)'
RT-AC68U user.warn (vpnclient1-up): 2853 Lock semaphore released '/tmp/WiFiVPN.sh-flock'
RT-AC68U user.warn (vpnclient1-up): 2853 User Processing Complete.
RT-AC68U daemon.notice ovpn-client1[2517]: /usr/sbin/ip route add xxx.xxx.xxx.xxx/32 via xxx.wan.isp.xxx
RT-AC68U daemon.notice ovpn-client1[2517]: /usr/sbin/ip route add 0.0.0.0/1 via xxx.vpn.gw.xxx
RT-AC68U daemon.notice ovpn-client1[2517]: /usr/sbin/ip route add 128.0.0.0/1 via xxx.vpn.gw.xxx

RT-AC68U user.warn (vpnrouting.sh): 2955 v384.07 Patched by Martineau []

RT-AC68U user.warn (vpnrouting.sh): 2955 Martineau Appending VPN Clients from '/jffs/configs/ovpnc1.nvram'

RT-AC68U user.warn openvpn-routing: Configuring policy rules for client 1

RT-AC68U user.warn (vpnrouting.sh): 2955 Martineau Deleting routes 0.0.0.0/1 and 128.0.0.0/1 from table main (254)

RT-AC68U user.warn (vpnrouting.sh): 27422 Martineau Selectively routing IPSET 'NETFLIX'.....
RT-AC68U user.warn (vpnrouting.sh): 27422 Martineau Selective IPSET 'NETFLIX' routing fwmark (0x7000) via WAN ENABLED

RT-AC68U user.warn (vpnrouting.sh): 27422 Martineau Selectively routing IPSET 'VPN1IPs'.....
RT-AC68U user.warn (vpnrouting.sh): 27422 Martineau Selective IPSET 'VPN1IPs' routing fwmark (0x1000) via VPN Client 1 ('HMA New York') ENABLED

RT-AC68U user.warn (vpnrouting.sh): 27422 Martineau Adding VPN1 RPDB fwmark rule 0x1000/0x1000 prio 10100

RT-AC68U daemon.notice ovpn-client1[25602]: Initialization Sequence Completed
 
Last edited:
This conversation is very timely. About a week ago, I was testing my scripts and I realized I was not honoring the DNS iptable rules when Accept DNS Configuration=Exclusive when using the “ip rule add from xxx.xxx.xxx.xxx table 11x” command to assign LAN clients to a tunnel. This applies to the routing of IPSET lists as well.

e.g. Instruct the firmware to generate the VPN Client 1 RPDB rules from file '/jffs/configs/ovpnc1.nvram' rather than from NVRAM variable 'vpn_client1_clientlist'

I am using a similar technique. I wrote a script to write dhcp static assignments to the file /jffs/configs/lan_client_rules. I place a number in the first column to define the interface I want them assigned to. The main routing script then uses the ip rules command to assign the LAN clients to the correct interface according the configuration file /jffs/configs/lan_client_rules and assigns a prior number. So much easier than typing the ip addresses and descriptions in the web gui.

Thank you very much for the recommendations!!

Sounds like I was on the correct path with my first attempt of manipulating updown.sh by attempting to manipulate the DNSVPNx chain for LAN clients inside the /jffs/scripts/openvpn-event. I did not fully understand the dynamics of vpnrouting.sh and updown.sh a week ago. Nor, how to implement the override of /usr/sbin/vpnrouting.sh and /usr/sbin/updown.sh using the

Code:
route-up /jffs/scripts/vpnrouting.sh

command inside the Custom Config section until I went back and reviewed your posts on the topic yesterday and in other threads, and walking thru the code.

I ran your modified Matineau Hacks vpnrouting.sh by placing

Code:
route-up /jffs/scripts/vpnrouting.sh
in the custom config section saw that it took precedence over /usr/sbin/vpnrouting.sh.

According to this post
updown.sh is called on --up and --down, and it also calls openvpn-event.

And in the same thread,
updown.sh is what the firmware uses internally, you cannot replace it. That script will call openvpn-event, which is the one you will need to put your own code in it.

Based on this, /usr/sbin/updown.sh overrides can be done by placing my required updown.sh updates in /jffs/scripts/openvpn-event. Correct??

When trying to understand scripts written by others, I like to run the script from the command line with set -x option enabled so I can see the code execute in real time. I tried calling /jffs/scripts/vpnrouting.sh from the command line and passing the parms. It did not work. The only solution I can think of is to add more logger messages to output the information I need to the system log for testing purposes. Any suggestions?

I have been working with 384.5 and upgraded to 384.7 Alpha 1 this afternoon. On both versions, vpnrouting.sh is logging several error messages when trying to delete routes:


ERROR: Linux route delete command failed: external program exited with error status: 2


Code:
Aug 17 14:37:13 ovpn-client3[12187]: /usr/sbin/ip link set dev tun13 up mtu 1500
Aug 17 14:37:13 ovpn-client3[12187]: /usr/sbin/ip addr add dev tun13 local 10.24.0.6 peer 10.24.0.5
Aug 17 14:37:15 ovpn-client3[12187]: /usr/sbin/ip route add xx.1xx.xxx.198/32 via 180.183.144.1
Aug 17 14:37:15 ovpn-client3[12187]: /usr/sbin/ip route add 0.0.0.0/1 via 10.24.0.5
Aug 17 14:37:15 ovpn-client3[12187]: /usr/sbin/ip route add 128.0.0.0/1 via 10.24.0.5
Aug 17 14:37:15 ovpn-client3[12187]: /usr/sbin/ip route add 10.24.0.1/32 via 10.24.0.5
Aug 17 14:37:16 ovpn-client3[12187]: Initialization Sequence Completed
Aug 17 14:38:29 ovpn-client3[12187]: event_wait : Interrupted system call (code=4)
Aug 17 14:38:29 ovpn-client3[12187]: vpnrouting.sh tun13 1500 1585 10.24.0.6 10.24.0.5 init
Aug 17 14:38:29 ovpn-client3[12187]: /usr/sbin/ip route del 10.24.0.1/32
Aug 17 14:38:29 ovpn-client3[12187]: ERROR: Linux route delete command failed: external program exited with error status: 2
Aug 17 14:38:29 ovpn-client3[12187]: /usr/sbin/ip route del xx.1xx.xxx.198/32
Aug 17 14:38:29 ovpn-client3[12187]: /usr/sbin/ip route del 0.0.0.0/1
Aug 17 14:38:29 ovpn-client3[12187]: ERROR: Linux route delete command failed: external program exited with error status: 2
Aug 17 14:38:29 ovpn-client3[12187]: /usr/sbin/ip route del 128.0.0.0/1
Aug 17 14:38:29 ovpn-client3[12187]: ERROR: Linux route delete command failed: external program exited with error status: 2
Aug 17 14:38:29 ovpn-client3[12187]: Closing TUN/TAP interface
Aug 17 14:38:29 ovpn-client3[12187]: /usr/sbin/ip addr del dev tun13 local 10.24.0.6 peer 10.24.0.5
Aug 17 14:38:29 ovpn-client3[12187]: SIGTERM[hard,] received, process exiting
I have not had time yet to investigate the errors yet.

At first, I did not understand what the purpose of the DNSVPNx dummy entry was until I did another search and found your quote
To fix this exposure, I always create a dummy entry using a non-existent LAN address to ensure the required DNSVPNx chain is always created/available.

So, that makes sense. I tested with it and see the impact.

Where I still have questions is with managing DNS for ipsets that I route thru the tunnel via the command below when Accept DNS Configuration=Exclusive:

Code:
iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set x3mRouting_BBC src,dst -j MARK --set-mark "$FWMARK" > /dev/null 2>&1

For clients listed in the web gui, the syntax generated by the firmware is
Code:
usr/sbin/iptables -t nat -A DNSVPN3 -s 192.168.22.15 -j DNAT --to-destination 10.9.0.1

Do I need a similar entry for the ipset list? So far, I have been unsuccessful in my attempts at the correct iptables syntax to route an ipset list to 10.9.0.1 in the DNVPN3 Chain.

Is the _IPSET list entry in the web gui a dummy description or the actual IPSET list name?
Is the IP address you are using for the IPSET list entry in the Web GUI also non reserved and assigned dhcp entry?

Thanks again for all of your help.
 
Last edited:

Support SNBForums w/ Amazon

If you'd like to support SNBForums, just use this link and buy anything on Amazon. Thanks!

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top