RT-AC3200 380.69 Dual Wan and Plex Media Server

[Astuni]

Hello there,
i have an Asus RT-AC3200 flashed with merlin 380.69.

I have 2 separate connections with 2 isp's, one is 200/20 (let's call it WAN1) and the other one is 100/34, (WAN2).

I have successfully setup the Dual Wan using load balance mode (2:1) and tested that i can effectively saturate both connections by doing some heavy iperf's to public servers.

Now the tricky part.

I have a Plex Media Server that i want to setup to have both outgoing and incoming connections routed through WAN2 since it has more upload bandwidth.

To accomplish this i need to force the PMS to announce itself to plex.tv using the WAN2 public ip, so i thought i could setup static routes in dual wan page forcing the PMS lan address to all the ip's of Plex.tv (did a nslookup for them) using WAN2 connection. I've setup that and it works, the PMS goes live and announce itself using WAN2 public ip only.

Now to have users actually connect to the PMS i need to manually forward the public port he uses to my local port inside the lan, so i did a simple port forward rule like this:

After applying this what happens is very strange (at least for my understanding), the port 17592 gets intermittently forwarded on either wan public ip, it switches from one wan ip to the other, one second i can connect to it using wan1 public ip, 5 seconds later it gets forwarded on the other one. This causes the PMS to be unreachable at intervals.

Is there a way to force a port forward rule to be valid only for one of the two wan interfaces?

More generally, is there a way to tell the router that specific lan ip's needs to have all their traffic routed to one and only wan interface, either is outgoing connections or incoming, port forwarded, ones?

I hope i was clear enough, and thanks to everyone that will help me out!

 

[Martineau]

Hello there,
i have an Asus RT-AC3200 flashed with merlin 380.69.

I have 2 separate connections with 2 isp's, one is 200/20 (let's call it WAN1) and the other one is 100/34, (WAN2).

I have successfully setup the Dual Wan using load balance mode (2:1) and tested that i can effectively saturate both connections by doing some heavy iperf's to public servers.

Now the tricky part.

I have a Plex Media Server that i want to setup to have both outgoing and incoming connections routed through WAN2 since it has more upload bandwidth.

To accomplish this i need to force the PMS to announce itself to plex.tv using the WAN2 public ip, so i thought i could setup static routes in dual wan page forcing the PMS lan address to all the ip's of Plex.tv (did a nslookup for them) using WAN2 connection. I've setup that and it works, the PMS goes live and announce itself using WAN2 public ip only.

Now to have users actually connect to the PMS i need to manually forward the public port he uses to my local port inside the lan, so i did a simple port forward rule like this:

After applying this what happens is very strange (at least for my understanding), the port 17592 gets intermittently forwarded on either wan public ip, it switches from one wan ip to the other, one second i can connect to it using wan1 public ip, 5 seconds later it gets forwarded on the other one. This causes the PMS to be unreachable at intervals.

Is there a way to force a port forward rule to be valid only for one of the two wan interfaces?

More generally, is there a way to tell the router that specific lan ip's needs to have all their traffic routed to one and only wan interface, either is outgoing connections or incoming, port forwarded, ones?

I hope i was clear enough, and thanks to everyone that will help me out!

In response to your PM......

It has been a long time since I played with Dual-WAN, but I recall Asus originally defines two RPDB tables 100 & 200?

So to force LAN device affinity to a particular WAN interface, the GUI allows defining affinity 'WAN interfaces' which uses these RPDB rule tables.

However for Selective Port Routing (as per VPNs) we should still be able to use simple old-skool fwmark tagging (rather than delve into the complex contemporary Load-Balancing CONNMARK rules)

You could try creating the following script....or copy'n'paste the 4 commands directly into the SSH command line

#!/bin/sh

# Try and Tag the Plex port traffic to bind to a specific WAN.....
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 32400 -j MARK --set-mark 0x90000000/0xf0000000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 32400 -j MARK --set-mark 0x90000000/0xf0000000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -d plex.tv                        -j MARK --set-mark 0x90000000/0xf0000000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -d plex.tv                        -j MARK --set-mark 0x90000000/0xf0000000

In theory Plex traffic should now always bind to a single WAN.

At this point if we have bound Plex traffic to the slower '200/20' WAN (rather than the more appropriate faster '100/34' WAN) we can fix it later, but at the moment we need to see if this Plex port affinity actually works.

If it doesn't work, then issue the following commands and post the output, remembering to obfuscate/redact your WAN IPs before posting!
​

# Show the Dual-WAN routes
ip route show table 100
ip route show table 200

# and the RBPD fwmark/CONNTRACK rules
ip rule

# Show what rules we have added
iptables -nvL  --line -t mangle

 

[Astuni]

Thanks for your reply Eric,
i have added the iptables rules and i think they haven't worked because my plex is still announcing itself to plex.tv using randomly one of the two public wan ip's, however from my little understanding, i think i didn't explained well what i am trying to do probably because i have omitted some details from my first post.

First things first more details about the single wan connections:

WAN1 "the <slower> upload one" is PPPOE, and it gets a dynamic public ip from the isp.
WAN2 "the <faster> upload one" has a slightly complex setup because the isp router has a static ip on wan side and releases a local lan ip to my asus lan1 port set as the secondary wan, hence the "double nat" where i have to setup the port forward from 17592 on isp modem to 17592 on my asus then additionally nat it from 17592 to 32400 on my server.

There are 2 separate issues that i want to overcome:

My Plex Media Server machine is 172.16.0.100, static ip.
The goals are the following:

• I want to define a rule that forces that lan ip address to connect to plex.tv addresses only via wan2, that's because when i turn on plex remote access the PMS needs to tell plex.tv what is my public ip and which port on that public ip is listening for incoming connections, i believe this is done via regular http/https by the program itself, so far i have accomplished this in "DUAL WAN" page using routing rules and setting up a rule with source 172.16.0.100 and destination all forcing it through wan2, the goal is however to force this ONLY for plex.tv addresses and let the rest of the server connections to freely load balance between the two wan's.
• The public port plex announces is 17592 which is correctly forwarded from the wan2 modem to my asus, then my asus router translates it to 172.16.0.100:32400 which is the actual port my server is listening. The behaviour i have here, no matter how i setup the port forwarding on the asus gui, is that the forward is either bounced on the two wan interfaces so i can intermittently access my server by pointing either at wan1_public_ip:17592 or wan2_public_ip:17592, while i want to have it fixed on wan2 public address.

Hope its "slightly" clearer now, thanks for your help, ive been fighting with this for some days now and frustration is kicking in, but i love to learn so thanks for the help you are giving!

Here's the output of the commands you wanted, hope they can help:

admin@RT-AC3200-1030:/tmp/home/root# ip route show table 100
192.168.100.1 dev ppp0  proto kernel  scope link
172.16.0.0/24 dev br0  proto kernel  scope link  src 172.16.0.1
192.168.1.0/24 dev vlan3  proto kernel  scope link  src 192.168.1.2
127.0.0.0/8 dev lo  scope link
default via 192.168.100.1 dev ppp0
admin@RT-AC3200-1030:/tmp/home/root# ip route show table 200
192.168.100.1 dev ppp0  proto kernel  scope link
192.168.1.254 dev vlan3  proto kernel  scope link
172.16.0.0/24 dev br0  proto kernel  scope link  src 172.16.0.1
192.168.1.0/24 dev vlan3  proto kernel  scope link  src 192.168.1.2
127.0.0.0/8 dev lo  scope link
default via 192.168.1.254 dev vlan3
admin@RT-AC3200-1030:/tmp/home/root#
admin@RT-AC3200-1030:/tmp/home/root# ip rule
0:      from all lookup local
20200:  from <WAN1_IP> lookup wan0
20200:  from <WAN2_IP> lookup wan1
20200:  from 85.37.17.16 lookup wan0
20200:  from 85.38.28.68 lookup wan0
20200:  from 8.8.8.8 lookup wan1
20200:  from 8.8.4.4 lookup wan1
20400:  from all to 192.168.100.1 lookup wan0
20400:  from all to 192.168.1.254 lookup wan1
20400:  from all to 85.37.17.16 lookup wan0
20400:  from all to 85.38.28.68 lookup wan0
20400:  from all to 8.8.8.8 lookup wan1
20400:  from all to 8.8.4.4 lookup wan1
32766:  from all lookup main
32767:  from all lookup default
admin@RT-AC3200-1030:/tmp/home/root#
admin@RT-AC3200-1030:/tmp/home/root# iptables -nvL  --line -t mangle
Chain PREROUTING (policy ACCEPT 4538 packets, 1138K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1       76 39374 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport sports 32400 MARK xset 0x90000000/0xf0000000
2        0     0 MARK       all  --  br0    *       172.16.0.100         54.77.197.74         MARK xset 0x90000000/0xf0000000
3        0     0 MARK       all  --  br0    *       172.16.0.100         52.31.73.202         MARK xset 0x90000000/0xf0000000
4        0     0 MARK       all  --  br0    *       172.16.0.100         34.248.236.84        MARK xset 0x90000000/0xf0000000
5      166 57684 MARK       all  --  br0    *       172.16.0.100         52.17.222.85         MARK xset 0x90000000/0xf0000000
6        0     0 MARK       all  --  br0    *       172.16.0.100         34.252.160.54        MARK xset 0x90000000/0xf0000000
7      107 35082 MARK       all  --  br0    *       172.16.0.100         54.171.208.164       MARK xset 0x90000000/0xf0000000
8        0     0 MARK       all  --  br0    *       172.16.0.100         52.30.134.235        MARK xset 0x90000000/0xf0000000
9        0     0 MARK       all  --  br0    *       172.16.0.100         52.31.227.227        MARK xset 0x90000000/0xf0000000

Chain INPUT (policy ACCEPT 2310 packets, 424K bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 2166 packets, 710K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1      709 5029K MARK       all  --  *      br0     172.16.0.0/24        172.16.0.0/24        MARK xset 0x1/0x7

Chain OUTPUT (policy ACCEPT 2506 packets, 794K bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 4674 packets, 1508K bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain BWDPI_FILTER (0 references)
num   pkts bytes target     prot opt in     out     source               destination
admin@RT-AC3200-1030:/tmp/home/root#

 

[Martineau]

Thanks for your reply Eric,
i have added the iptables rules and i think they haven't worked because my plex is still announcing itself to plex.tv using randomly one of the two public wan ip's, however from my little understanding, i think i didn't explained well what i am trying to do probably because i have omitted some details from my first post.

First things first more details about the single wan connections:

WAN1 "the <slower> upload one" is PPPOE, and it gets a dynamic public ip from the isp.
WAN2 "the <faster> upload one" has a slightly complex setup because the isp router has a static ip on wan side and releases a local lan ip to my asus lan1 port set as the secondary wan, hence the "double nat" where i have to setup the port forward from 17592 on isp modem to 17592 on my asus then additionally nat it from 17592 to 32400 on my server.

There are 2 separate issues that i want to overcome:

My Plex Media Server machine is 172.16.0.100, static ip.
The goals are the following:

• I want to define a rule that forces that lan ip address to connect to plex.tv addresses only via wan2, that's because when i turn on plex remote access the PMS needs to tell plex.tv what is my public ip and which port on that public ip is listening for incoming connections, i believe this is done via regular http/https by the program itself, so far i have accomplished this in "DUAL WAN" page using routing rules and setting up a rule with source 172.16.0.100 and destination all forcing it through wan2, the goal is however to force this ONLY for plex.tv addresses and let the rest of the server connections to freely load balance between the two wan's.
• The public port plex announces is 17592 which is correctly forwarded from the wan2 modem to my asus, then my asus router translates it to 172.16.0.100:32400 which is the actual port my server is listening. The behaviour i have here, no matter how i setup the port forwarding on the asus gui, is that the forward is either bounced on the two wan interfaces so i can intermittently access my server by pointing either at wan1_public_ip:17592 or wan2_public_ip:17592, while i want to have it fixed on wan2 public address.

Hope its "slightly" clearer now, thanks for your help, ive been fighting with this for some days now and frustration is kicking in, but i love to learn so thanks for the help you are giving!

Here's the output of the commands you wanted, hope they can help:

admin@RT-AC3200-1030:/tmp/home/root# ip route show table 100
192.168.100.1 dev ppp0  proto kernel  scope link
172.16.0.0/24 dev br0  proto kernel  scope link  src 172.16.0.1
192.168.1.0/24 dev vlan3  proto kernel  scope link  src 192.168.1.2
127.0.0.0/8 dev lo  scope link
default via 192.168.100.1 dev ppp0
admin@RT-AC3200-1030:/tmp/home/root# ip route show table 200
192.168.100.1 dev ppp0  proto kernel  scope link
192.168.1.254 dev vlan3  proto kernel  scope link
172.16.0.0/24 dev br0  proto kernel  scope link  src 172.16.0.1
192.168.1.0/24 dev vlan3  proto kernel  scope link  src 192.168.1.2
127.0.0.0/8 dev lo  scope link
default via 192.168.1.254 dev vlan3
admin@RT-AC3200-1030:/tmp/home/root#
admin@RT-AC3200-1030:/tmp/home/root# ip rule
0:      from all lookup local
20200:  from <WAN1_IP> lookup wan0
20200:  from <WAN2_IP> lookup wan1
20200:  from 85.37.17.16 lookup wan0
20200:  from 85.38.28.68 lookup wan0
20200:  from 8.8.8.8 lookup wan1
20200:  from 8.8.4.4 lookup wan1
20400:  from all to 192.168.100.1 lookup wan0
20400:  from all to 192.168.1.254 lookup wan1
20400:  from all to 85.37.17.16 lookup wan0
20400:  from all to 85.38.28.68 lookup wan0
20400:  from all to 8.8.8.8 lookup wan1
20400:  from all to 8.8.4.4 lookup wan1
32766:  from all lookup main
32767:  from all lookup default
admin@RT-AC3200-1030:/tmp/home/root#
admin@RT-AC3200-1030:/tmp/home/root# iptables -nvL  --line -t mangle
Chain PREROUTING (policy ACCEPT 4538 packets, 1138K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1       76 39374 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport sports 32400 MARK xset 0x90000000/0xf0000000
2        0     0 MARK       all  --  br0    *       172.16.0.100         54.77.197.74         MARK xset 0x90000000/0xf0000000
3        0     0 MARK       all  --  br0    *       172.16.0.100         52.31.73.202         MARK xset 0x90000000/0xf0000000
4        0     0 MARK       all  --  br0    *       172.16.0.100         34.248.236.84        MARK xset 0x90000000/0xf0000000
5      166 57684 MARK       all  --  br0    *       172.16.0.100         52.17.222.85         MARK xset 0x90000000/0xf0000000
6        0     0 MARK       all  --  br0    *       172.16.0.100         34.252.160.54        MARK xset 0x90000000/0xf0000000
7      107 35082 MARK       all  --  br0    *       172.16.0.100         54.171.208.164       MARK xset 0x90000000/0xf0000000
8        0     0 MARK       all  --  br0    *       172.16.0.100         52.30.134.235        MARK xset 0x90000000/0xf0000000
9        0     0 MARK       all  --  br0    *       172.16.0.100         52.31.227.227        MARK xset 0x90000000/0xf0000000

Chain INPUT (policy ACCEPT 2310 packets, 424K bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 2166 packets, 710K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1      709 5029K MARK       all  --  *      br0     172.16.0.0/24        172.16.0.0/24        MARK xset 0x1/0x7

Chain OUTPUT (policy ACCEPT 2506 packets, 794K bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 4674 packets, 1508K bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain BWDPI_FILTER (0 references)
num   pkts bytes target     prot opt in     out     source               destination
admin@RT-AC3200-1030:/tmp/home/root#

i think i didn't explained well what i am trying to do probably because i have omitted some details from my first post.

Normally this would be sufficient reason for me to walk-away, and let others address your issue.

However, I'll continue for a little while longer........so, fortunately it appears the new specific 32400 port rule is indeed firing:

Chain PREROUTING (policy ACCEPT 4538 packets, 1138K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1       76 39374 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport sports multiport sports 32400 MARK xset 0x90000000/0xf0000000

but the rule has been '-I' inserted rather than '-A' appended to the '-t mangle' PREROUTING chain!!?? .... or was the chain empty when you manually added the rule?

NOTE: ALL fwmark rules are non-terminating in the processing order, so last matching rule will apply - hence rule order is crucially important.!

Whilst the Dual-WAN fwmark tag value '0x90000000/0xf0000000' appears to be used by other rules in the '-t mangle' PREROUTING chain, suspiciously, there is no matching RPDB rule.

So you could try manually issuing

ip rule add fwmark 0x90000000/0xf0000000 table wan1 prio 100
ip route flush cache
ip rule

or perhaps it may be prudent to use an undefined custom fwmark

I suggest you start clean, so reboot then issue the following

ip rule del fwmark 0x7000/0x7000 2> /dev/null
ip rule add fwmark 0x7000/0x7000 table wan1 prio 100
ip route flush cache

iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 32400 -j MARK --set-mark 0x7000/0x7000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 32400 -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -d plex.tv                        -j MARK --set-mark 0x7000/0x7000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -d plex.tv                        -j MARK --set-mark 0x7000/0x7000

ip rule

iptables -nvL --line -t mangle

EDIT: Whoops Didn't read the title [Which Router/Firmware are you using?]

 

[Astuni]

Normally this would be sufficient reason for me to walk-away, and let others address your issue.

Thank you for not having done it

but the rule has been '-I' inserted rather than '-A' appended to the '-t mangle' PREROUTING chain!!?? .... or was the chain empty when you manually added the rule?

Yep it was empty, all those ip's are all the ip's plex.tv resolves, i just issued your commands on the first post.

P.S.Which Router/Firmware are you using?

RT-AC3200 on 380.69

I suggest you start clean, so reboot then issue the following

ip rule del fwmark 0x7000/0x7000 2> /dev/null
ip rule add fwmark 0x7000/0x7000 table wan1 prio 100
ip route flush cache

iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 32400 -j MARK --set-mark 0x7000/0x7000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 32400 -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -d plex.tv -j MARK --set-mark 0x7000/0x7000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -d plex.tv -j MARK --set-mark 0x7000/0x7000

ip rule

iptables -nvL --line -t mangle

Ok, i have done these, and i think now it's really working as it should as i am refreshing continuously my wan2_ip:17592 and it always lands to the server without switching to the other wan address, also the PMS always goes on on that ip address too so far.
So i think your rules solved the problem.
Really, i've been working in IT since like 20 years but never had the need to dig deep into iptables and generally linux network handling and this was so frustrating, so i really can't thank you enough for your precious help, and i think i have just found my next thing to study and master!

One last bit of information, to make these rules survive across a reboot, supposing i have already formatted the jffs partition and enabled script execution, where i have to put the script to make sure it gets executed on each reboot?

Thanks again!

 

[Martineau]

I think now it's really working as it should as i am refreshing continuously my wan2_ip:17592 and it always lands to the server without switching to the other wan address, also the PMS always goes on on that ip address too so far.

So i think your rules solved the problem.

Asus has changed the way the Dual-WAN Load-Balancing is implemented in Firmware v382.xx, however glad it's working for you.

As a favour, could you please post the output of the following again?

ip rule
iptables -nvL --line -t mangle

One last bit of information, to make these rules survive across a reboot, supposing i have already formatted the jffs partition and enabled script execution, where i have to put the script to make sure it gets executed on each reboot?

JFFS content such as Scripts files in '/jffs/scripts' and Config files in '/jffs/configs' etc. survives a reboot and, in most cases, even firmware upgrades.

I recommend @john9527's NVRAM Save Restore utility or you should use the backup utilities on the 'Administration->Restore/Save/Upload' Tab regularly and always prior to you performing a firmware upgrade or a manual /jffs format - intentionally or not!.

I would put the custom fwmark commands in /jffs/scripts/firewall-start

 

[Astuni]

As a favour, could you please post the output of the following again?

Here you go:

admin@RT-AC3200-1030:/tmp/home/root# ip rule
0:      from all lookup local
100:    from all fwmark 0x7000/0x7000 lookup wan1
20100:  from 172.16.0.13 lookup wan0
20200:  from <WAN1_IP> lookup wan0
20200:  from <WAN2_IP> lookup wan1
20200:  from 85.37.17.16 lookup wan0
20200:  from 85.38.28.68 lookup wan0
20200:  from 8.8.8.8 lookup wan1
20200:  from 8.8.4.4 lookup wan1
20400:  from all to 192.168.100.1 lookup wan0
20400:  from all to 192.168.1.254 lookup wan1
20400:  from all to 85.37.17.16 lookup wan0
20400:  from all to 85.38.28.68 lookup wan0
20400:  from all to 8.8.8.8 lookup wan1
20400:  from all to 8.8.4.4 lookup wan1
32766:  from all lookup main
32767:  from all lookup default
admin@RT-AC3200-1030:/tmp/home/root# iptables -nvL --line -t mangle
Chain PREROUTING (policy ACCEPT 237 packets, 58100 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1       15 13811 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport sports 32400 MARK or 0x7000
2        0     0 MARK       all  --  br0    *       172.16.0.100         54.77.197.74         MARK or 0x7000
3        0     0 MARK       all  --  br0    *       172.16.0.100         52.17.222.85         MARK or 0x7000
4        0     0 MARK       all  --  br0    *       172.16.0.100         52.31.73.202         MARK or 0x7000
5        0     0 MARK       all  --  br0    *       172.16.0.100         52.214.33.245        MARK or 0x7000
6        0     0 MARK       all  --  br0    *       172.16.0.100         54.171.208.164       MARK or 0x7000
7        0     0 MARK       all  --  br0    *       172.16.0.100         34.252.160.54        MARK or 0x7000
8        0     0 MARK       all  --  br0    *       172.16.0.100         52.30.134.235        MARK or 0x7000
9        0     0 MARK       all  --  br0    *       172.16.0.100         34.248.236.84        MARK or 0x7000

Chain INPUT (policy ACCEPT 88 packets, 14496 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 142 packets, 43035 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 MARK       all  --  *      br0     172.16.0.0/24        172.16.0.0/24        MARK xset 0x1/0x7

Chain OUTPUT (policy ACCEPT 88 packets, 21293 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 251 packets, 70178 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain BWDPI_FILTER (0 references)
num   pkts bytes target     prot opt in     out     source               destination

I recommend @john9527's NVRAM Save Restore utility or you should use the backup utilities on the 'Administration->Restore/Save/Upload' Tab regularly and always prior to you performing a firmware upgrade or a manual /jffs format - intentionally or not!.

I already use it with much joy!

I would put the custom fwmark commands in /jffs/scripts/firewall-start

Done and rebooted and everything came up nicely!

Thank you so much!

 

[Martineau]

Here you go:

admin@RT-AC3200-1030:/tmp/home/root# ip rule
0:      from all lookup local
100:    from all fwmark 0x7000/0x7000 lookup wan1
20100:  from 172.16.0.13 lookup wan0
20200:  from <WAN1_IP> lookup wan0
<snip>

Many thanks, I was definitely confused when I saw the rogue '0x90000000/0xf0000000' fwmarks... I hadn't allowed for the fact that you would 'do your own thing'!

So I need to be aware that whilst prio 100 is probably overkill for your current v380.90 RPDB table (prio 9990 is what I normally use/promote for the '0x7000/0x7000' wan0 rule), under v382.xx Asus has stupidly decided to severely reduce the number of unique high-priority end-user rules unless we now assign multiple rules to the same priority.

e.g. if/when you migrate to v382.xx then your current RPDB table will probably look like this,

ip rule

0:    from all lookup local
100:  from all fwmark 0x7000/0x7000 lookup wan1
100:  from 172.16.0.13 lookup wan0
150:  from all fwmark 0x80000000/0xf0000000 lookup wan0
150:  from all fwmark 0x90000000/0xf0000000 lookup wan1
200:  from <WAN1_IP> lookup wan0
200:  from <WAN2_IP> lookup wan1
200:  from 85.37.17.16 lookup wan0
200:  from 85.38.28.68 lookup wan0
200:  from 8.8.8.8 lookup wan1
200:  from 8.8.4.4 lookup wan1
400:  from all to 192.168.100.1 lookup wan0
400:  from all to 192.168.1.254 lookup wan1
400:  from all to 85.37.17.16 lookup wan0
400:  from all to 85.38.28.68 lookup wan0
400:  from all to 8.8.8.8 lookup wan1
400:  from all to 8.8.4.4 lookup wan1
32766:  from all lookup main
32767:  from all lookup default

and the custom fwmark '0x7000/0x7000' will be obsolete as we can simply use the Asus defined Load-Balancing fwmark '0x90000000/0xf0000000 'instead.

P.S. Thanks for the positive feedback.

 

[Astuni]

and the custom fwmark '0x7000/0x7000' will be obsolete as we can simply use the Asus defined Load-Balancing fwmark '0x90000000/0xf0000000 'instead.

so, if i understood correctly, i would just use the iptables rules with those asus defined fwmarks for wan1 without setting the custom fwmarks with the ip rule add commands, right?

 

[Martineau]

so, if i understood correctly, i would just use the iptables rules with those asus defined fwmarks for wan1 without setting the custom fwmarks with the ip rule add commands, right?

Ideally yes.

With more and more of the firmware becoming closed-source, there is no guarantee that Asus would not in the future use our chosen fwmark ( '0x7000/0x7000') for other purposes, so potentially we could cause unexpected issues by inappropriately setting this fwmark.

Obviously Asus may also decide to change their current Dual-WAN Load-Balancing ('0x80000000/0xf0000000' and '0x90000000/0xf0000000') fwmarks but hopefully it should be fairly easy in firmwares > v382.xx to spot the change?

 

[Astuni]

Just another question, is there a way to implement some sort of session persistence?

I have found that some sites that require you to keep your connection steady through the whole session (home banking and such) have some sort of problems if during session the router switches to the other wan.

EDIT: or, better put: Can i setup a selective routing on that 172.16.0.100 host to make it go through the wan 2 only for the rules we've setup so far (sport 32400 and plex.tv) and use wan1 for everything else?

 

[Xentrk]

Many thanks, I was definitely confused when I saw the rogue '0x90000000/0xf0000000' fwmarks... I hadn't allowed for the fact that you would 'do your own thing'!

So I need to be aware that whilst prio 100 is probably overkill for your current v380.90 RPDB table (prio 9990 is what I normally use/promote for the '0x7000/0x7000' wan0 rule), under v382.xx Asus has stupidly decided to severely reduce the number of unique high-priority end-user rules unless we now assign multiple rules to the same priority.

e.g. if/when you migrate to v382.xx then your current RPDB table will probably look like this,

ip rule

0:    from all lookup local
100:  from all fwmark 0x7000/0x7000 lookup wan1
100:  from 172.16.0.13 lookup wan0
150:  from all fwmark 0x80000000/0xf0000000 lookup wan0
150:  from all fwmark 0x90000000/0xf0000000 lookup wan1
200:  from <WAN1_IP> lookup wan0
200:  from <WAN2_IP> lookup wan1
200:  from 85.37.17.16 lookup wan0
200:  from 85.38.28.68 lookup wan0
200:  from 8.8.8.8 lookup wan1
200:  from 8.8.4.4 lookup wan1
400:  from all to 192.168.100.1 lookup wan0
400:  from all to 192.168.1.254 lookup wan1
400:  from all to 85.37.17.16 lookup wan0
400:  from all to 85.38.28.68 lookup wan0
400:  from all to 8.8.8.8 lookup wan1
400:  from all to 8.8.4.4 lookup wan1
32766:  from all lookup main
32767:  from all lookup default

and the custom fwmark '0x7000/0x7000' will be obsolete as we can simply use the Asus defined Load-Balancing fwmark '0x90000000/0xf0000000 'instead.

P.S. Thanks for the positive feedback.

Thanks for the heads up. Looks like I need to plan for some modifications to my selective routing scripts when I cut over.

 

[Astuni]

Hello Martineau, i hope you're well.

In the last few days i've looked into this trying to learn how to deal with custom iptables rules and fwmarks, all my goals are pretty much reached but i'm still struggling with a rule and i would be grateful if you could look into this to see if i made some mistakes.

My actual /jffs/scripts/firewall-start script looks like this:

#!/bin/sh
###  Adding fwmark 0x7000 for wan0 and 0x8000 for wan1  ###
ip rule del fwmark 0x7000/0x7000 2> /dev/null
ip rule add fwmark 0x7000/0x7000 table wan0 prio 100
ip rule del fwmark 0x8000/0x8000 2> /dev/null
ip rule add fwmark 0x8000/0x8000 table wan1 prio 100
ip route flush cache

###  adding rules to send standard web traffic through wan0  ###
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --dport 80,443 -j MARK --set-mark 0x7000/0x7000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --dport 80,443 -j MARK --set-mark 0x7000/0x7000

###  adding rules to make ftp server listen on connections incoming from wan0  ###
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 990,60035 -j MARK --set-mark 0x7000/0x7000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 990,60035 -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 60000:60034 -j MARK --set-mark 0x7000/0x7000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 60000:60034 -j MARK --set-mark 0x7000/0x7000

###  adding rules to make plex media server incoming connections and outgoing connections to public external port going through wan1 ###
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 32400 -j MARK --set-mark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 32400 -j MARK --set-mark 0x8000/0x8000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --dport 17592 -j MARK --set-mark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --dport 17592 -j MARK --set-mark 0x8000/0x8000

###  adding rules to make plex media server reach plex.tv addresses always through wan1, to announce the correct wan1 ip  ###
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -d plex.tv -j MARK --set-mark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -d plex.tv -j MARK --set-mark 0x8000/0x8000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -d app.plex.tv -j MARK --set-mark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -d app.plex.tv -j MARK --set-mark 0x8000/0x8000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -d my.plexapp.com -j MARK --set-mark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -d my.plexapp.com -j MARK --set-mark 0x8000/0x8000

Basically everything but the last group of rules works, in detail:
- My ftp server is connectable through wan0 ip only;
- All my server standard web traffic (http/https) gets sent through wan0;
- The Plex Media server is reachable through is public port 17592 on wan1 ip only and gets correctly forwarded to lan address ip on port 32400, however (and here's the last groups of rules), if the first rule (the one that forwards all http/https traffic through the wan0) is added, the following rules:

###  adding rules to make plex media server reach plex.tv addresses always through wan1, to announce the correct wan1 ip  ###
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -d plex.tv -j MARK --set-mark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -d plex.tv -j MARK --set-mark 0x8000/0x8000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -d app.plex.tv -j MARK --set-mark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -d app.plex.tv -j MARK --set-mark 0x8000/0x8000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -d my.plexapp.com -j MARK --set-mark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -d my.plexapp.com -j MARK --set-mark 0x8000/0x8000

gets ignored, so my plex media server gets always announced using wan0 public ip which is not what i desire.

Note that if i comment out the http/https rule and start the router, the plex media server correctly announces wan1 ip address, then i can add that rule manually, and since the pms is already started, everything will work as expected until i either reboot the router or the server, which could be fine but its a bit annoying.
You said in one of your previous posts that:

NOTE: ALL fwmark rules are non-terminating in the processing order, so last matching rule will apply - hence rule order is crucially important.!

So i thought that if i put that http/https rule FIRST, then it will match also the following plex.tv specific rules "overriding" the wider one but that's not the case, even if i put that http/https rule last the behaviour doesn't change.

Is there a way i can setup it so all the http/https traffic, EXCEPT that directed to these plex.tv hosts which needs to go through wan1, goes through wan0?

I am posting output for ip rule and iptables -nvL -t mangle --line of the actual setup:

admin@RT-AC3200-1030:/tmp/home/root# ip rule
0:      from all lookup local
100:    from all fwmark 0x7000/0x7000 lookup wan0
100:    from all fwmark 0x8000/0x8000 lookup wan1
20100:  from 172.16.0.13 lookup wan0
20200:  from <wan0_ip> lookup wan0
20200:  from <wan1_ip> lookup wan1
20200:  from 85.37.17.16 lookup wan0
20200:  from 85.38.28.68 lookup wan0
20200:  from 8.8.8.8 lookup wan1
20200:  from 8.8.4.4 lookup wan1
20400:  from all to 192.168.100.1 lookup wan0
20400:  from all to 192.168.1.254 lookup wan1
20400:  from all to 85.37.17.16 lookup wan0
20400:  from all to 85.38.28.68 lookup wan0
20400:  from all to 8.8.8.8 lookup wan1
20400:  from all to 8.8.4.4 lookup wan1
32766:  from all lookup main
32767:  from all lookup default



admin@RT-AC3200-1030:/tmp/home/root# iptables -nvL --line -t mangle
Chain PREROUTING (policy ACCEPT 199M packets, 212G bytes)
num   pkts bytes target     prot opt in     out     source               destination
1      29M 1217M MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport dports 80,443 MARK or 0x7000
2      766  120K MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport sports 990,60035 MARK or 0x7000
3      11M   75G MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport sports 60000:60034 MARK or 0x7000
4     158K 1563M MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport sports 32400 MARK or 0x8000
5      109 23039 MARK       all  --  br0    *       172.16.0.100         54.77.197.74         MARK or 0x8000
6      145 24833 MARK       all  --  br0    *       172.16.0.100         34.252.160.54        MARK or 0x8000
7      274 44069 MARK       all  --  br0    *       172.16.0.100         52.214.33.245        MARK or 0x8000
8      107 22409 MARK       all  --  br0    *       172.16.0.100         52.17.222.85         MARK or 0x8000
9      428  106K MARK       all  --  br0    *       172.16.0.100         34.248.236.84        MARK or 0x8000
10     477 92725 MARK       all  --  br0    *       172.16.0.100         52.31.227.227        MARK or 0x8000
11     834  182K MARK       all  --  br0    *       172.16.0.100         52.30.134.235        MARK or 0x8000
12      86 19965 MARK       all  --  br0    *       172.16.0.100         52.31.73.202         MARK or 0x8000
13     296 53491 MARK       all  --  br0    *       172.16.0.100         52.50.213.122        MARK or 0x8000
14     345 56283 MARK       all  --  br0    *       172.16.0.100         54.77.136.139        MARK or 0x8000
15       0     0 MARK       all  --  br0    *       172.16.0.100         52.48.167.201        MARK or 0x8000
16       0     0 MARK       all  --  br0    *       172.16.0.100         34.250.110.202       MARK or 0x8000
17       0     0 MARK       all  --  br0    *       172.16.0.100         54.72.125.167        MARK or 0x8000
18    3618  185K MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport dports 17592 MARK or 0x8000

Chain INPUT (policy ACCEPT 132K packets, 26M bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 199M packets, 212G bytes)
num   pkts bytes target     prot opt in     out     source               destination
1      125 26833 MARK       all  --  *      br0     172.16.0.0/24        172.16.0.0/24        MARK xset 0x1/0x7

Chain OUTPUT (policy ACCEPT 185K packets, 51M bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 199M packets, 212G bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain BWDPI_FILTER (0 references)
num   pkts bytes target     prot opt in     out     source               destination

Hope to get some light from you!

Thanks for your help.

 

[Martineau]

Hello Martineau, i hope you're well.

In the last few days i've looked into this trying to learn how to deal with custom iptables rules and fwmarks, all my goals are pretty much reached but i'm still struggling with a rule and i would be grateful if you could look into this to see if i made some mistakes.

May I again politely remind you that I still reserve the right to particpate/respond to forum posts at my discretion - time permitting etc.

i.e. I am under no obligation to provide dedicated support for your issues either submitted to me privately via PMs or addressed directly solely to me by public forum posts such as this.

Is there a way i can setup it so all the http/https traffic, EXCEPT that directed to these plex.tv hosts which needs to go through wan1, goes through wan0?

YES.

HINT: I suggest your single HTTP/HTTPS rule should be replaced by two rules to eliminate ambiguity.

My proposed solution to be added later if I have time to write it up this evening.....unless someone else jumps in first!

EDIT: Update 4hrs later

OK just to recap.....

You state there is nothing actually wrong with your Selective HTTP/HTTPS Port tagging rule, but unfortunately it is seemingly installed too early in the boot process, so simply delaying the rule creation could be a quick-fix solution?

However, I know from personal experience that timing issues are notoriously difficult to resolve reliably.

So, I propose implementing the flexible IPSET technique solution (rather than the use of custom '-t mangle' fwmark chains or the use of additional explicit inline iptables '-j RETURN' rules etc.)

First, create an IPSET containing all of the target PLEX host IPs:
e.g.

ipset create PLEX hash:ip comment

ipset add PLEX 54.77.197.74  comment "plex.tv"
<etc.>
ipset add PLEX 52.50.213.122 comment "app.plex.tv"

Check its contents

ipset list PLEX

Now insert at the top of your block of custom fwmark tagging rules these two mutually exclusive rules:

# ALL HTTP/HTTPS traffic from 172.16.0.100 is routed via WAN0, EXCEPT to PLEX.TV hosts which MUST be routed via WAN1
iptables -t mangle -A PREROUTING -s 172.16.0.100/32 -i br0 -p tcp -m set   --match-set PLEX dst -m multiport --dports 80,443 -j MARK --set-xmark 0x8000/0x8000
iptables -t mangle -A PREROUTING -s 172.16.0.100/32 -i br0 -p tcp -m set ! --match-set PLEX dst -m multiport --dports 80,443 -j MARK --set-xmark 0x7000/0x7000

# 1st rule means: if (172.16.0.100 sends a request to a matching     IPSET defined PLEX host) && (dest port is either HTTP or HTTPS); then tag it via WAN1
# 2nd rule means: if (172.16.0.100 sends a request to a non-matching IPSET defined PLEX host) && (dest port is either HTTP or HTTPS); then tag it via WAN0

producing this (much tidier and unambiguous) fwmark tagging chain:
e.g.

iptables -nvL --line -t mangle

Chain PREROUTING (policy ACCEPT 199M packets, 212G bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            match-set PLEX dst multiport dports 80,443 MARK or 0x8000
2        0     0 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            ! match-set PLEX dst multiport dports 80,443 MARK or 0x7000
3      766  120K MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport sports 990,60035 MARK or 0x7000
4      11M   75G MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport sports 60000:60034 MARK or 0x7000
5     158K 1563M MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport sports 32400 MARK or 0x8000
6     3618  185K MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            multiport dports 17592 MARK or 0x8000

which will hopefully achieve your desired Selective HTTP/HTTPS Dual-WAN routing.

P.S. If it doesn't work, it's been a long day, then you may need to move the two rules to the end of the chain, or explicitly ensure that ALL rules are unique - such that if a matching rule in the chain has set the fwmark tag, then the fwmark tag cannot be altered by any rule further down in the chain using an alternative technique.

 

[Astuni]

May I again politely remind you that I still reserve the right to particpate/respond to forum posts at my discretion - time permitting etc.

i.e. I am under no obligation to provide dedicated support for your issues either submitted to me privately via PMs or addressed directly solely to me by public forum posts such as this.

Ok, well, I’ve just replied to the thread quoting one of your earlier proposed solutions, maybe that “Hello Martineau” incipit sounded to you too personal like I am pretending a reply or direct support from you once again but it was just “good manners” and a polite way to follow up to a thread in which you were absolutely helpful and kind in helping me out, probably the whole thing started in a wrong way and it was absolutely my fault I’m sorry again if that sounded to you like this. And under no circumstances you are forced to help me again if you haven’t the time/will to do it.

Thanks for your help so far and for the hints you may want to suggest if you can.

 

[Martineau]

Thanks for your help so far and for the hints you may want to suggest if you can.

Proposed solution uploaded

 

[Astuni]

So, I propose implementing the flexible IPSET technique solution (rather than the use of custom '-t mangle' fwmark chains or the use of additional explicit inline iptables '-j RETURN' rules etc.)

First, create an IPSET containing all of the target PLEX host IPs:
e.g.

ipset create PLEX hash:ip comment

ipset add PLEX 54.77.197.74 comment "plex.tv"
<etc.>
ipset add PLEX 52.50.213.122 comment "app.plex.tv"

Check its contents

ipset list PLEX

Now insert at the top of your block of custom fwmark tagging rules these two mutually exclusive rules:

# ALL HTTP/HTTPS traffic from 172.16.0.100 is routed via WAN0, EXCEPT to PLEX.TV hosts which MUST be routed via WAN1
iptables -t mangle -A PREROUTING -s 172.16.0.100/32 -i br0 -p tcp -m set --match-set PLEX dst -m multiport --dports 80,443 -j MARK --set-xmark 0x8000/0x8000
iptables -t mangle -A PREROUTING -s 172.16.0.100/32 -i br0 -p tcp -m set ! --match-set PLEX dst -m multiport --dports 80,443 -j MARK --set-xmark 0x7000/0x7000

# 1st rule means: if (172.16.0.100 sends a request to a matching IPSET defined PLEX host) && (dest port is either HTTP or HTTPS); then tag it via WAN1
# 2nd rule means: if (172.16.0.100 sends a request to a non-matching IPSET defined PLEX host) && (dest port is either HTTP or HTTPS); then tag it via WAN0

producing this (much tidier and unambiguous) fwmark tagging chain:
e.g.

iptables -nvL --line -t mangle

Chain PREROUTING (policy ACCEPT 199M packets, 212G bytes)
num pkts bytes target prot opt in out source destination
1 0 0 MARK tcp -- br0 * 172.16.0.100 0.0.0.0/0 match-set PLEX dst multiport dports 80,443 MARK or 0x8000
2 0 0 MARK tcp -- br0 * 172.16.0.100 0.0.0.0/0 ! match-set PLEX dst multiport dports 80,443 MARK or 0x7000
3 766 120K MARK tcp -- br0 * 172.16.0.100 0.0.0.0/0 multiport sports 990,60035 MARK or 0x7000
4 11M 75G MARK tcp -- br0 * 172.16.0.100 0.0.0.0/0 multiport sports 60000:60034 MARK or 0x7000
5 158K 1563M MARK tcp -- br0 * 172.16.0.100 0.0.0.0/0 multiport sports 32400 MARK or 0x8000
6 3618 185K MARK tcp -- br0 * 172.16.0.100 0.0.0.0/0 multiport dports 17592 MARK or 0x8000

which will hopefully achieve your desired Selective HTTP/HTTPS Dual-WAN routing.

P.S. If it doesn't work, it's been a long day, then you may need to move the two rules to the end of the chain, or

- such that if a matching rule in the chain has set the fwmark tag, then the fwmark tag cannot be altered by any rule further down in the chain using an alternative technique.

Thanks a lot, i've elaborated on your proposed solution to overcome a couple of things, like not having ipset modules loaded by default on iptables so i had to modprobe them, also, stealing your proposed solution from elsewhere in this forum (that's truly dedication, i'm now your biggest fan ), i managed to dynamically populate ipset with the plex ip addresses using dnsmasq, i will need to periodically flush the ipset and invoke a couple of nslookups to refresh ip's (if they change on plex side) but overall now everything works and you were absolutely right in saying that the key of everything was:

explicitly ensure that ALL rules are unique - such that if a matching rule in the chain has set the fwmark tag, then the fwmark tag cannot be altered by any rule further down in the chain using an alternative technique.

Now for the actual scripts.
i have created a /jffs/configs/dnsmasq.conf.add with the following content:

ipset=/plex.tv/app.plex.tv/my.plexapp.com/Plex

then edited my /jffs/scripts/firewall-start with this content:

#!/bin/sh
###  Tag 0x7000 for wan0 and 0x8000 for wan 1  ###
ip rule del fwmark 0x7000/0x7000 2> /dev/null
ip rule add fwmark 0x7000/0x7000 table wan0 prio 100
ip rule del fwmark 0x8000/0x8000 2> /dev/null
ip rule add fwmark 0x8000/0x8000 table wan1 prio 100
ip route flush cache

###  creation of an hash:ip ipset for Plex addresses  ###
ipset create Plex hash:ip

###  invoking a couple of nslookups so dnsmasq will do his work in populating the ipset we've just created  ###
for host in plex.tv my.plexapp.com app.plex.tv; do
    nslookup $host > /dev/null 2>&1
    sleep 5s
    nslookup $host > /dev/null 2>&1
done;

###  modprobing for iptables ipset modules  ###
for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set; do
    modprobe $module
done;

###  adding rules for ftp server tagging them on wan 0  ###
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 990,60035 -j MARK --set-mark 0x7000/0x7000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 990,60035 -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 60000:60034 -j MARK --set-mark 0x7000/0x7000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 60000:60034 -j MARK --set-mark 0x7000/0x7000

###  adding rules for local and public plex media server ports tagging them for wan1 ###
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 32400 -j MARK --set-mark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --sport 32400 -j MARK --set-mark 0x8000/0x8000
iptables -t mangle -D PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --dport 17592 -j MARK --set-mark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -i br0 --src 172.16.0.100 -p tcp -m multiport --dport 17592 -j MARK --set-mark 0x8000/0x8000

###  adding rules for routing all http/https traffic through wan0 except plex.tv, my.plexapp.com and app.plex.tv  ###
iptables -t mangle -D PREROUTING -s 172.16.0.100/32 -i br0 -p tcp -m set   --match-set Plex dst -m multiport --dports 80,443 -j MARK --set-xmark 0x8000/0x8000 2> /dev/null
iptables -t mangle -A PREROUTING -s 172.16.0.100/32 -i br0 -p tcp -m set   --match-set Plex dst -m multiport --dports 80,443 -j MARK --set-xmark 0x8000/0x8000
iptables -t mangle -D PREROUTING -s 172.16.0.100/32 -i br0 -p tcp -m set ! --match-set Plex dst -m multiport --dports 80,443 -j MARK --set-xmark 0x7000/0x7000 2> /dev/null
iptables -t mangle -A PREROUTING -s 172.16.0.100/32 -i br0 -p tcp -m set ! --match-set Plex dst -m multiport --dports 80,443 -j MARK --set-xmark 0x7000/0x7000

Do you see any potential issue? I think i just miss to make a little script to cron periodic flushes and nslookups to repopulate the iplist and im done, right?

So far everything works as expected, also i learned a lot from this and thanks again for your patience and availability!

 

[Martineau]

....also stealing your proposed solution from elsewhere in this forum

No problem , although if you do publish scripts re-using large chunks of my code it would be appreciated if you included a small reference to my (helpful) contribution.

....not having ipset modules loaded by default on iptables so i had to modprobe them,

Abject apologies ...I'd long forgotten about the need for some users to explicitly load the appropriate IPSET v6.x related modules. I personally prefer to use nat-start (rather than firewall-start) for my fwmark tagging rules, and since nat-start may be called (randomly by the firmware) multiple times I perform the one-off loading of the IPSET modules in init-start.

I think i just miss to make a little script to cron periodic flushes and nslookups to repopulate the iplist and im done, right?

Lack of housekeeping/management of obsolete entries in IPSETs probably isn't going to cause you an issue assuming we are talking fewer than 10 member entries rather than 1000s etc.
So you could periodically (or during a controlled shutdown from services-stop) save the current PLEX IPSET to /mnt/xxx and modify post-mount to restore it during the boot process.

Do you see any potential issue? So far everything works as expected

No it looks good - since it apparently works to your satisfaction.

Whilst it is probably overkill for your limited number of ports, you could obviously create an IPSET for the FTP/Plex ports and then reference it to reduce the number of tagging rules even further?, but for the HTTP/HTTPS ports it makes for easier reading of the tagging chain to have them explicitly on view.

I'm pleased that you proved my assumption that only two Plex host related rules are needed, but the other combinations should adequately cover all your requirements (even if they are no longer unique!)
e.g. The six available combinations......

iptables -nvL --line -t mangle
Chain PREROUTING (policy ACCEPT 727K packets, 124M bytes)
num   pkts bytes target     prot opt in     out     source               destination       
1        0     0 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            match-set PLEX dst  MARK or 0x?000
2        0     0 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            ! match-set PLEX dst  MARK or 0x?000
3        0     0 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            match-set PLEX dst multiport dports  !80,443 MARK or 0x?000
4        0     0 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            match-set PLEX dst multiport dports 80,443 MARK or 0x?000
5        0     0 MARK       tcp  --  br0    *       172.16.0.100         0.0.0.0/0            ! match-set PLEX dst multiport dports 80,443 MARK or 0x?000
6        0     0 RETURN     tcp  --  br0    *       172.16.0.100         0.0.0.0/0            ! match-set PLEX dst multiport dports !80,443[CODE]

 

[Astuni]

Lack of housekeeping/management of obsolete entries in IPSETs probably isn't going to cause you an issue assuming we are talking fewer than 10 member entries rather than 1000s etc.
So you could periodically (or during a controlled shutdown from services-stop) save the current PLEX IPSET to /mnt/xxx and modify post-mount to restore it during the boot process.

What i did, honestly in a quick and dirty way maybe, was creating a /jffs/script/refresh_ipset.sh with the following content:

#!/bin/sh
ipset flush Plex > /dev/null 2>&1

for host in plex.tv my.plexapp.com app.plex.tv; do
    nslookup $host > /dev/null 2>&1
    sleep 5s
    nslookup $host > /dev/null 2>&1
done;

then in /jffs/scripts/init-start i've put:

#!/bin/sh
cru a ScheduledIpsetRefresh "0 4 * * * /jffs/scripts/refresh_ipset.sh"

To cron the flush and refresh each night at 4.

I did some random tests and it flushes and repopulates the ipset correctly, i guess it's still a valid solution?

I'm getting more and more into this so now that i have found a working setup i will experiment the other hints you proposed to learn.

Can't thank you enough again!