RT-AC86U Policy Based Routing through VPN conundrum

TorchedPoseidon

New Around Here
I have issues trying to route traffic from tcp port 51413. I have the feeling it's something simple that's staring me in the face, but can't for the life of me find what I'm missing so I was hoping for some extra eyes to help me out.

The goal: Have only traffic (inbound and outbound) from port 51413 (originating from ip 10.0.0.31) go through the VPN tunnel and let all other traffic go through WAN. Preferably with a kill switch that prevents any traffic from port 51413 going through WAN at all times.
Additionally I'm running a PiHole from this source IP so I need to make sure that there's no interference in terms of access to DNS servers by the PiHole service, which runs through port 53 if I'm not mistaken. That takes priority over the risk of DNS leaks so if the DNS configuration needs to be changed to something less strict that's ok.

The way I've been trying to accomplish this (thanks to documentation of members like @Martineau and @john9527) is as follow:

FW: Merlin 384.13_0
The relevant client settings (Client1)
Automatic start at boot time: Yes
Interface Type: TUN
Protocol: UDP
Accept DNS Configuration: Exclusive
Create NAT on tunnel: Yes
Inbound Firewall: Allow
Compression: None
Force Internet traffic through tunnel: Policy Rules (strict); No policy rules set up through GUI
Block routed clients if tunnel goes down: Yes

Content of /jffs/scripts/nat-start:

Code:
#!/bin/sh

sleep 10  # During the boot process nat-start may run multiple times so this is required

# Ensure duplicate rules are not created
for VPN_ID in 0 1 2 3 4 5
   do
      ip rule del prio 999$VPN_ID  2>/dev/null
   done

# Create the RPDB rule for WAN
ip rule add from 0/0 fwmark "0x8000/0x8000" table main   prio 9990        # WAN   fwmark
Which is effectively creates a reset state and adds an RPDB rule for WAN (didn't know of a more elegant place to put it) mirroring the placement of the RPDB rules for the individual VPN clients in their respective scripts.

Content of /jffs/scripts/openvpn-event:

Code:
#!/bin/sh


scr_name="$(basename $0)[$$]"


case "$1" in

    "tun11")

        vpn_name="client1"

        ;;

    "tun12")

        vpn_name="client2"

        ;;

    "tun13")

        vpn_name="client3"

        ;;

    "tun14")

        vpn_name="client4"

        ;;

    "tun15")

        vpn_name="client5"

        ;;

    "tun21")

        vpn_name="server1"

        ;;

    "tun22")

        vpn_name="server2"

        ;;

    *)

        vpn_name=""

        ;;

esac


# Call appropriate script based on script_type

vpn_script_name="vpn$vpn_name-$script_type"


# Check script state/use nvram to save last script run

vpn_script_state=$(nvram get vpn_script_state)

nvram set vpn_script_state="$vpn_script_name"

if [ "$vpn_script_name" = "$vpn_script_state" ]; then

    echo "VPN script" $vpn_script_name "already run" | logger -t "$scr_name"

    exit 0

fi


if [[ -f "/jffs/scripts/$vpn_script_name" ]] ; then

    sh /jffs/scripts/$vpn_script_name $*

else

    echo "Script not defined for event: "$vpn_script_name | logger -t $scr_name

    exit 0

fi


exit 0
Content of /jffs/scripts/vpnclient1-route-up

Code:
#!/bin/sh



ip rule add from 0/0 fwmark 0x1000/0x1000 table ovpnc1 prio 9993             # VPN 1 fwmark


iptables -t mangle -A PREROUTING -i br0 -m iprange --src-range 10.0.0.31 -p tcp -m multiport --sport 51413 -j MARK --set-mark 0x1000/0x1000

iptables -t mangle -A PREROUTING -i br0 -m iprange --src-range 10.0.0.31 -p tcp -m multiport --dport 51413 -j MARK --set-mark 0x1000/0x1000
And bonus contents (for the sake of good practice) of /jffs/scripts/vpnclient1-route-down:

Code:
#!/bin/sh


ip rule del from 0/0 fwmark 0x1000/0x1000 table ovpnc1 prio 9993             # VPN 1 fwmark


iptables -t mangle -D PREROUTING -i br0 -m iprange --src-range 10.0.0.31 -p tcp -m multiport --sport 51413 -j MARK --set-mark 0x1000/0x1000

iptables -t mangle -D PREROUTING -i br0 -m iprange --src-range 10.0.0.31 -p tcp -m multiport --dport 51413 -j MARK --set-mark 0x1000/0x1000
With the other vpnclientX-route-up/down scripts simply containing fwmarks
Scripts are set as executable

Sadly this doesn't seem to work as per https://ipleak.net

Logs do mention something about the scripts:
Code:
Dec  5 10:34:33 custom_script: Running /jffs/scripts/openvpn-event (args: tun11 1500 1553 10.200.0.66 10.200.0.65 init)
Dec  5 10:34:33 openvpn-event[2449]: Script not defined for event: vpnclient1-up
However renaming vpnclient1-route-up to just vpnclient1-up doesn't change anything.

Some (semi-)relevant diagnostic output:

Output of ip route show table 111:
Code:
10.0.0.0/24 dev br0  proto kernel  scope link  src 10.0.0.1

10.200.0.65 dev tun11  proto kernel  scope link  src 10.200.0.66

Output of iptables --list:
Code:
Chain INPUT (policy ACCEPT)

target     prot opt source               destination        

DROP       icmp --  anywhere             anywhere             icmp echo-request

ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED

DROP       all  --  anywhere             anywhere             state INVALID

PTCSRVWAN  all  --  anywhere             anywhere           

PTCSRVLAN  all  --  anywhere             anywhere           

ACCEPT     all  --  anywhere             anywhere             state NEW

ACCEPT     all  --  anywhere             anywhere             state NEW

OVPN       all  --  anywhere             anywhere             state NEW

ACCEPT     udp  --  anywhere             anywhere             udp spt:bootps dpt:bootpc

INPUT_ICMP  icmp --  anywhere             anywhere           

DROP       all  --  anywhere             anywhere           


Chain FORWARD (policy DROP)

target     prot opt source               destination        

ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED

other2wan  all  --  anywhere             anywhere           

ACCEPT     all  --  anywhere             anywhere           

DROP       all  --  anywhere             anywhere             state INVALID

NSFW       all  --  anywhere             anywhere           

ACCEPT     all  --  anywhere             anywhere           

ACCEPT     all  --  anywhere             anywhere             ctstate DNAT

OVPN       all  --  anywhere             anywhere             state NEW

DROP       all  --  anywhere             anywhere           


Chain OUTPUT (policy ACCEPT)

target     prot opt source               destination        


Chain ACCESS_RESTRICTION (0 references)

target     prot opt source               destination        


Chain DNSFILTER_DOT (0 references)

target     prot opt source               destination        


Chain FUPNP (0 references)

target     prot opt source               destination        

ACCEPT     tcp  --  anywhere             10.0.0.31            tcp dpt:51413


Chain INPUT_ICMP (1 references)

target     prot opt source               destination        

RETURN     icmp --  anywhere             anywhere             icmp echo-request

RETURN     icmp --  anywhere             anywhere             icmp timestamp-request

ACCEPT     icmp --  anywhere             anywhere           


Chain NSFW (1 references)

target     prot opt source               destination        


Chain OVPN (2 references)

target     prot opt source               destination        

ACCEPT     all  --  anywhere             anywhere           


Chain PControls (0 references)

target     prot opt source               destination        

ACCEPT     all  --  anywhere             anywhere           


Chain PTCSRVLAN (1 references)

target     prot opt source               destination        


Chain PTCSRVWAN (1 references)

target     prot opt source               destination        


Chain SECURITY (0 references)

target     prot opt source               destination        

RETURN     tcp  --  anywhere             anywhere             tcpflags: FIN,SYN,RST,ACK/SYN limit: avg 1/sec burst 5

DROP       tcp  --  anywhere             anywhere             tcpflags: FIN,SYN,RST,ACK/SYN

RETURN     tcp  --  anywhere             anywhere             tcpflags: FIN,SYN,RST,ACK/RST limit: avg 1/sec burst 5

DROP       tcp  --  anywhere             anywhere             tcpflags: FIN,SYN,RST,ACK/RST

RETURN     icmp --  anywhere             anywhere             icmp echo-request limit: avg 1/sec burst 5

DROP       icmp --  anywhere             anywhere             icmp echo-request

RETURN     all  --  anywhere             anywhere           


Chain default_block (0 references)

target     prot opt source               destination        


Chain logaccept (0 references)

target     prot opt source               destination        

LOG        all  --  anywhere             anywhere             state NEW LOG level warning tcp-sequence tcp-options ip-options prefix "ACCEPT "

ACCEPT     all  --  anywhere             anywhere           


Chain logdrop (0 references)

target     prot opt source               destination        

LOG        all  --  anywhere             anywhere             state NEW LOG level warning tcp-sequence tcp-options ip-options prefix "DROP "

DROP       all  --  anywhere             anywhere           


Chain other2wan (1 references)

target     prot opt source               destination        

RETURN     all  --  anywhere             anywhere           

DROP       all  --  anywhere             anywhere
I hit the character count limit for this post so I'll continue below
 
Last edited:

TorchedPoseidon

New Around Here
Pt. 2: Last couple of outputs

Output of iptables --list-rules:

Code:
-P INPUT ACCEPT

-P FORWARD DROP

-P OUTPUT ACCEPT

-N ACCESS_RESTRICTION

-N DNSFILTER_DOT

-N FUPNP

-N INPUT_ICMP

-N NSFW

-N OVPN

-N PControls

-N PTCSRVLAN

-N PTCSRVWAN

-N SECURITY

-N default_block

-N logaccept

-N logdrop

-N other2wan

-A INPUT -i eth0 -p icmp -m icmp --icmp-type 8 -j DROP

-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

-A INPUT -m state --state INVALID -j DROP

-A INPUT ! -i br0 -j PTCSRVWAN

-A INPUT -i br0 -j PTCSRVLAN

-A INPUT -i br0 -m state --state NEW -j ACCEPT

-A INPUT -i lo -m state --state NEW -j ACCEPT

-A INPUT -m state --state NEW -j OVPN

-A INPUT -p udp -m udp --sport 67 --dport 68 -j ACCEPT

-A INPUT -p icmp -j INPUT_ICMP

-A INPUT -j DROP

-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT

-A FORWARD ! -i br0 -o eth0 -j other2wan

-A FORWARD -i br0 -o br0 -j ACCEPT

-A FORWARD -m state --state INVALID -j DROP

-A FORWARD -j NSFW

-A FORWARD -i br0 -j ACCEPT

-A FORWARD -m conntrack --ctstate DNAT -j ACCEPT

-A FORWARD -m state --state NEW -j OVPN

-A FORWARD -j DROP

-A FUPNP -d 10.0.0.31/32 -p tcp -m tcp --dport 51413 -j ACCEPT

-A INPUT_ICMP -p icmp -m icmp --icmp-type 8 -j RETURN

-A INPUT_ICMP -p icmp -m icmp --icmp-type 13 -j RETURN

-A INPUT_ICMP -p icmp -j ACCEPT

-A OVPN -i tun11 -j ACCEPT

-A PControls -j ACCEPT

-A SECURITY -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m limit --limit 1/sec -j RETURN

-A SECURITY -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -j DROP

-A SECURITY -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK RST -m limit --limit 1/sec -j RETURN

-A SECURITY -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK RST -j DROP

-A SECURITY -p icmp -m icmp --icmp-type 8 -m limit --limit 1/sec -j RETURN

-A SECURITY -p icmp -m icmp --icmp-type 8 -j DROP

-A SECURITY -j RETURN

-A logaccept -m state --state NEW -j LOG --log-prefix "ACCEPT " --log-tcp-sequence --log-tcp-options --log-ip-options

-A logaccept -j ACCEPT

-A logdrop -m state --state NEW -j LOG --log-prefix "DROP " --log-tcp-sequence --log-tcp-options --log-ip-options

-A logdrop -j DROP

-A other2wan -i tun+ -j RETURN

-A other2wan -j DROP
Output of iptables -nvL --line -t mangle:
Code:
Chain PREROUTING (policy ACCEPT 630K packets, 139M bytes)

num   pkts bytes target     prot opt in     out     source               destination         

1       29  2412 MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7

2    19948 3921K BWDPI_FILTER  udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0           

3      111  9047 MARK       tcp  --  br0    *       0.0.0.0/0            0.0.0.0/0            source IP range 10.0.0.31-10.0.0.31 multiport sports 51413 MARK or 0x1000

4      106  7599 MARK       tcp  --  br0    *       0.0.0.0/0            0.0.0.0/0            source IP range 10.0.0.31-10.0.0.31 multiport dports 51413 MARK or 0x1000


Chain INPUT (policy ACCEPT 428K packets, 75M bytes)

num   pkts bytes target     prot opt in     out     source               destination         


Chain FORWARD (policy ACCEPT 202K packets, 64M bytes)

num   pkts bytes target     prot opt in     out     source               destination         


Chain OUTPUT (policy ACCEPT 394K packets, 113M bytes)

num   pkts bytes target     prot opt in     out     source               destination         


Chain POSTROUTING (policy ACCEPT 596K packets, 177M bytes)

num   pkts bytes target     prot opt in     out     source               destination         

1       43  4286 MARK       all  --  *      br0     0.0.0.0/0            0.0.0.0/0            mark match 0x40000000/0xc0000000 MARK xset 0x80000000/0xc0000000


Chain BWDPI_FILTER (1 references)

num   pkts bytes target     prot opt in     out     source               destination         

1        0     0 DROP       udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            udp spt:68 dpt:67

2        0     0 DROP       udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            udp spt:67 dpt:68
I'm pretty much stuck so any hint to point me in the right direction is much appreciated.
 

Martineau

Part of the Furniture
Code:
Accept DNS Configuration: Exclusive
Force Internet traffic through tunnel: Policy Rules (strict); No policy rules set up through GUI
The above won't work...

You need to have at least one entry in the Selective Routing table

e.g. use an IP that doesn't exist on your LAN

upload_2019-12-5_21-23-27.png

for the DNSVPNx chains to be created, and at the moment there is no default route in your VPN Client 1 routing table 111.
Code:
ip route show table 111 | grep -E "^default"
Also, if your VPN ISP doesn't allow inbound unsolicited traffic, then even with changing the VPN Client tunnel default to 'allow'

upload_2019-12-5_21-39-14.png


it won't work.

P.S. You'd get more on a page if you didn't double space your details! ;)
 
Last edited:

TorchedPoseidon

New Around Here
;)
The above won't work...

You need to have at least one entry in the Selective Routing table

e.g. use an IP that doesn't exist on your LAN


for the DNSVPNx chains to be created, and at the moment there is no default route in your VPN Client 1 routing table 111.
Thanks, added it and it seems there now is a default route:

Code:
default via 10.200.0.17 dev tun11
Also, if your VPN ISP doesn't allow inbound unsolicited traffic, then it won't work.
Since it's not unsolicited it should work. In either case, thus far the testing has been on outbound traffic for… surprise, surprise, the transmission-daemon running on the pi.

P.S. You'd get more on a page if you didn't double space your details!
Sorry about that, I lazily copy pasted from the GUI and deleted the irrelevant parts.

Currently there's still an ip leak although it looks a bit more promising:
Code:
iptables -nvL --line -t mangle

Chain PREROUTING (policy ACCEPT 30486 packets, 6508K bytes)

num   pkts bytes target     prot opt in     out     source               destination        

1       49  4184 MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7

2       78  6596 MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7

3    24607 4948K BWDPI_FILTER  udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0          

4      120  9802 MARK       tcp  --  br0    *       0.0.0.0/0            0.0.0.0/0            source IP range 10.0.0.31-10.0.0.31 multiport sports 51413 MARK or 0x1000

5      156 10870 MARK       tcp  --  br0    *       0.0.0.0/0            0.0.0.0/0            source IP range 10.0.0.31-10.0.0.31 multiport dports 51413 MARK or 0x1000

6        0     0 MARK       tcp  --  br0    *       0.0.0.0/0            0.0.0.0/0            source IP range 10.0.0.31-10.0.0.31 multiport sports 51413 MARK or 0x1000

7        5   300 MARK       tcp  --  br0    *       0.0.0.0/0            0.0.0.0/0            source IP range 10.0.0.31-10.0.0.31 multiport dports 51413 MARK or 0x1000


Chain INPUT (policy ACCEPT 19961 packets, 3413K bytes)

num   pkts bytes target     prot opt in     out     source               destination        


Chain FORWARD (policy ACCEPT 10513 packets, 3094K bytes)

num   pkts bytes target     prot opt in     out     source               destination        


Chain OUTPUT (policy ACCEPT 18527 packets, 4211K bytes)

num   pkts bytes target     prot opt in     out     source               destination        


Chain POSTROUTING (policy ACCEPT 29019 packets, 7304K bytes)

num   pkts bytes target     prot opt in     out     source               destination        

1        0     0 MARK       all  --  *      br0     0.0.0.0/0            0.0.0.0/0            mark match 0x40000000/0xc0000000 MARK xset 0x80000000/0xc0000000


Chain BWDPI_FILTER (1 references)

num   pkts bytes target     prot opt in     out     source               destination        

1        0     0 DROP       udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            udp spt:68 dpt:67

2        5  1648 DROP       udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            udp spt:67 dpt:68
I'm going to reboot the router and see if that will bring salvation, I've noticed the VPN client can be rather fickle at times pre-reboot.

Edit: Alas, it still leaks
 
Last edited:

Martineau

Part of the Furniture
Thanks, added it and it seems there now is a default route:

Code:
default via 10.200.0.17 dev tun11
Since it's not unsolicited it should work. In either case, thus far the testing has been on outbound traffic for… surprise, surprise, the transmission-daemon running on the pi.

Edit: Alas, it still leaks
Still double spacing? - Which SSH client are you using?

For VPN Clients that must use the EXCLUSIVE VPN ISP DNS, then they will have an entry in the appropriate DNSVPNx chain
Code:
iptables --line -t nat -nvL DNSVPN1
However, with DoT etc., using the VPN ISP DNS to prevent DNS leaks to your ISP is no longer a concern.
(DoT hides the DNS request from both parties and provides other secure features)

Obviously if you are using a bespoke VPN specific service that only the VPN ISP's DNS can resolve then you would have to use it.

Anyway, if you want the RaspPi to use the WAN for all traffic except the Transmission Port but use the EXCLUSIVE VPN ISP DNS then that can be implemented with a bit of trickery.

Same for the KILL-Switch, but if you can't figure it out yourself then someone will jump in.
 

TorchedPoseidon

New Around Here
Still double spacing? - Which SSH client are you using?
Just Apple's plain old stock Terminal that comes with zsh shell these days. Spacing seems fine on my end, are you sure your browser isn't messing around?


Anyways

However, with DoT etc., using the VPN ISP DNS to prevent DNS leaks to your ISP is no longer a concern.
(DoT hides the DNS request from both parties and provides other secure features)
Fair point, hadn't taken that into account.

Anyway, if you want the RaspPi to use the WAN for all traffic except the Transmission Port but use the EXCLUSIVE VPN ISP DNS then that can be implemented with a bit of trickery.

Same for the KILL-Switch, but if you can't figure it out yourself then someone will jump in.
At this stage those are 'nice to have' for a later stage as I'm still struggling to get traffic from the transmission port routed through the VPN even without the DNS and kill switch bells and whistles.

Could you think of a reason why, with the current implementations, traffic through 51413 still seems to reach https://ipleak.net through WAN?
 

Martineau

Part of the Furniture
Spacing seems fine on my end, are you sure your browser isn't messing around?
Adamant, all browsers I've tried render your text the same :rolleyes:
Could you think of a reason why, with the current implementations, traffic through 51413 still seems to reach https://ipleak.net through WAN?
Because the RaspPi makes the DNS request on behalf of Transmission, and since you haven't specifically redirected the RaspPi to use the EXCLUSIVE VPN DNS via the VPN it must (by default) use the WAN.

NOTE: You can influence which DNS server is used for say a site required by Transmission by specific dnsmasq 'server=' directives.
I'm still struggling to get traffic from the transmission port routed through the VPN
I'm assuming that the Selective Port 51413 traffic is successfully going out via the VPN, but perhaps no inbound connections are seen?

You can test if the Selective Port Routing is working
Try visiting ipleak.net, from say the RaspPi (10.0.0.31) or your Apple Mac (replacing 10.0.0.31 as appropriate) then issue
Code:
iptables -t mangle -A PREROUTING -i br0 --src 10.0.0.31 -p tcp -m multiport --dport 80,443 -j MARK --set-mark 0x1000/0x1000
to route the RaspPi's web traffic via VPN Client 1 and retry. The end-point should have changed - easier to confirm if VPN is in another country.;) Use the following to remove the Selective Port 80/443 Routing
Code:
iptables -t mangle -D PREROUTING -i br0 --src 10.0.0.31 -p tcp -m multiport --dport 80,443 -j MARK --set-mark 0x1000/0x1000
Applications are difficult to Selectively Route but since applications use ports, then there is a chance it will work, providing the ports they use are unique, i.e. Firefox/Chrome are apps, but they both use Port 80, so if you are running them on the RaspPi, the router cannot tell which app just requested the Port 80 outbound request.

Anyway, an old trick when running Transmission on the router itself is to use an alias rather than the router's IP.
i.e. You could try spoofing Transmission.
Code:
ifconfig br0:trans xxx.xxx.xxx.xxx up   - where xxx.xxx.xxx.xxx is outside your DHCP pool
 
Bind Transmission to xxx.xxx.xxx.xxx by changing settings on the RaspPi

"bind-address-ipv4": "xxx.xxx.xxx.xxx"
Then you can simply redirect ALL Transmission traffic via the VPN simply by specifying the Alias br0 IP address xxx.xxx.xxx.xxx in the Policy rules GUI.
kill switch bells and whistles.
You can enable the KILL-Switch in the VPN Client GUI

And optionally old-skool KILL-Switch - Only required for WAN outbound
Code:
iptables -I FORWARD -i br0 ! -o tun1+ -s 10.0.0.31 -p tcp -m multiport --sport 51413 -j DROP
but just in case a Port Forward was inadvertently opened....
Code:
iptables -I FORWARD -i $(nvram get wan0_ifname) -d 10.0.0.31 -p tcp -m multiport --dport 51413  -j DROP
 
Last edited:

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top