Menu
What's new
By:
Filters Better Search

Selective Routing for Netflix

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

Xentrk said:
Why is that? Curious minds want to know. Conflict with use of fwmarks/bitmasks vs. firmware?
Click to expand...
Fwmark '0x7000' seemingly doesn't work for the WAN on (some/all?) HND models but simply using a single bitmask such as '0x8000' does.

Since reordering the masks, and relegating the multi-bit masks i.e. '0x5000' and '0x7000' then for anyone that needs to use VPN Clients 4/5 may be out of luck, unless I find time to test it.
 
Martineau said:
so if you have an IPv6 WAN ISP, you have to disable IPv6 on the VPN ISP's tunnel.
Click to expand...

Care to elaborate on the exact method you are referring to? Documentation for the topic is very much lacking and all the methods I've tried so far result in IPv6 continuing to leak.

Ideally the solution would be to have dnsmasq only return IPv4 records for the same select domains so prioritization never kicks in.
 
Last edited:
Martineau said:
Turning on VPN disables native ipv6
Naitive IPV6 And VPN.
Click to expand...

Unfortunately the issue is more to-do with dnsmasq in this situation (I rushed my first post and overlooked this fact). The way we are routing traffic is based on the contents of an IPSet which dnsmasq dynamically fills.

Code: 
echo "ipset=/netflix.com/nflxvideo.net/nflxso.net/nflxext.com/nflximg.net/VPNFlix-Netflix #VPNFlix" >> /jffs/configs/dnsmasq.conf.add

Which is being filled as expected, but only with IPv4 addresses;

kFzpJbG.png


As IPv6 is configured to take priority during DNS lookups, we never hit any IP's in our IPSet (thus traffic is never being redirected over the VPN), instead we hit their IPv6 counterparts;

Code: 
iptables -A PREROUTING -t mangle -m set --match-set VPNFlix-Master dst -j MARK --set-mark "$FWMARK_OVPNC1" 2>/dev/null

99XODmu.png





With that being said, the only real solution I can think of is finding a way to force these select domains to return their IPv4 addresses only at a dnsmasq level. Weather that is possible or not without nuking IPv6 entirely is the big question.
 
Adamm said:
Unfortunately the issue is more to-do with dnsmasq in this situation (I rushed my first post and overlooked this fact). The way we are routing traffic is based on the contents of an IPSet which dnsmasq dynamically fills.

Which is being filled as expected, but only with IPv4 addresses
Click to expand...

I suppose you would need to use the same method you use in your script to collate/populate IPv6 IPSETs?

However, dnsmasq does support collating both IPv4 and IPv6 IP addresses! :eek: but I guess no-one with an IPv6 ISP (my ISP is IPv4 only) who wishes to Selectively Route Netflix via the IPv6 WAN has found this to be an issue until now?

However, if dnsmasq on the router hasn't been configured to automatically populate an IPSET with IPv6 addresses (or the feature isn't available on MIPs routers), then the alternative is to use the ASNUM scraping method at regular intervals:

e.g. for Netflix
Code: 
./IPSubnet_fromAS.sh AS2906

 (IPSubnet_fromAS.sh) v1.02 Extracting netflix.com IP subnets from https://ipinfo.io/AS2906  Netflix Streaming Services Inc.

IPSET AS2906 and AS2906-v6 created.
 
IPSET AS2906==>
23.246.0.0/18   23.246.2.0/24   23.246.3.0/24   23.246.6.0/24   23.246.7.0/24  23.246.10.0/24  23.246.11.0/24  23.246.14.0/24
<snip>

IPSET AS2906 Total:  148
 
IPSET AS2906-v6==>
2a00:86c0:1000::/48 2a00:86c0:1001::/48 2a00:86c0:1002::/48 2a00:86c0:1003::/48 2a00:86c0:1008::/48 2a00:86c0:1009::/48 2a00:86c0:100::/48
2a00:86c0:1012::/48   2a00:86c0:1013::/48 2a00:86c0:1014::/48 2a00:86c0:1015::/48 2a00:86c0:1016::/48 2a00:86c0:1017::/48 2a00:86c0:1018::/48 2a00:86c0:1021::/48
2a00:86c0:1022::/48 2a00:86c0:1023::/48 2a00:86c0:1026::/48 2a00:86c0:1027::/48 2a00:86c0:1034::/48 2a00:86c0:1044::/48 2a00:86c0:1045::/48
2a00:86c0:1046::/48 2a00:86c0:1047::/48 2a00:86c0:1048::/48 2a00:86c0:1049::/48 2a00:86c0:1050::/48 2a00:86c0:1051::/48 2a00:86c0:1052::/48 2a00:86c0:1054::/48 2a00:86c0:1055::/48 2a00:86c0:1056::/48
2a00:86c0:1057::/48 2a00:86c0:1058::/48 2a00:86c0:1059::/48 2a00:86c0:108::/48 2a00:86c0:109::/48 2a00:86c0:110::/48 2a00:86c0:111::/48 2a00:86c0:112::/48    2a00:86c0:113::/48 2a00:86c0:114::/48
2a00:86c0:115::/48 2a00:86c0:120::/48 2a00:86c0:121::/48 2a00:86c0:122::/48 2a00:86c0:124::/48 2a00:86c0:125::/48    2a00:86c0:126::/48 2a00:86c0:127::/48 2a00:86c0:2002::/48 2a00:86c0:2004::/48
2a00:86c0:2005::/48 2a00:86c0:2006::/48 2a00:86c0:2007::/48 2a00:86c0:2010::/48   2a00:86c0:2011::/48 2a00:86c0:2012::/48 2a00:86c0:2013::/48 2a00:86c0:2014::/48 2a00:86c0:2015::/48 2a00:86c0:2016::/48 2a00:86c0:2017::/48
2a00:86c0:2018::/48   2a00:86c0:2019::/48 2a00:86c0:2020::/48 2a00:86c0:2021::/48 2a00:86c0:2022::/48 2a00:86c0:2023::/48 2a00:86c0:2036::/48 2a00:86c0:2037::/48 2a00:86c0:2044::/48   2a00:86c0:2045::/48
2a00:86c0:2048::/48 2a00:86c0:2049::/48 2a00:86c0:2050::/48 2a00:86c0:2051::/48 2a00:86c0:2056::/48 2a00:86c0:2058::/48 2a00:86c0:2059::/48   2a00:86c0:2060::/48 2a00:86c0:2062::/48 2a00:86c0:2063::/48
2a00:86c0:2064::/48 2a00:86c0:2065::/48 2a00:86c0:2068::/48 2a00:86c0:2070::/48 2a00:86c0:2071::/48   2a00:86c0:2072::/48 2a00:86c0:2073::/48 2a00:86c0:2074::/48 2a00:86c0:2075::/48 2a00:86c0:2078::/48 2a00:86c0:2079::/48
2a00:86c0:2080::/48 2a00:86c0:2082::/48   2a00:86c0:2083::/48 2a00:86c0:2088::/48 2a00:86c0:2089::/48 2a00:86c0:21::/48 2a00:86c0:39bc::/48 2a00:86c0:39bd::/48 2a00:86c0:4::/48 2a00:86c0:5::/48
2a00:86c0:600::/48 2a00:86c0:601::/48 2a00:86c0:98::/48 2a00:86c0:99::/48  2a00:86c0::/32 2a00:86c0:ff0a::/48 2607:fb10:5020::/44 2607:fb10:5021::/48 2607:fb10:7050::/44 2607:fb10:7051::/48 2607:fb10:7071::/48 2607:fb10:7090::/44
2607:fb10:70b0::/44 2607:fb10:70c0::/44 2607:fb10:70d0::/44 2607:fb10:70d1::/48 2607:fb10:70e0::/44 2607:fb10:70e1::/48 2607:fb10:70e2::/48  2607:fb10::/32 2620:10c:7000::/44 2620:10c:7008::/48 2620:10c:700d::/48
 
IPSET AS2906-v6 Total:  127

and include both IPv4 and IPv6 IPSETs in a master IPSET list when IPv6 VPNs are supported.

2019-04-02_12-52-22.png


Perhaps @Xentrk should plan to incorporate IPv6 in the case where the Selective Routing is required to bypass the VPN and use the IPv6 WAN?
 
Last edited:
@Martineau
Support for IPv6 was not in the original project scope. I can easily add the functionality in the follow-up release after getting the IPv4 project scope completed.

@Adamm,
Slightly different syntax to create IPv6 IPSET lists. Must specify family inet6. IPv6 IPSET list example:

Code: 
ipset create setipv6 hash:ip family inet6
ipset add setipv6 2001:2001:2001::
ipset add setipv6 2001:2001:abcd::
ipset add setipv6 2606:4700:4700::1111

The option family { inet | inet6 } defines the protocol family of the IP addresses to be stored in the set. By default it is inet (IPv4).

http://ipset.netfilter.org/ipset.man.html
 
Martineau said:
However, dnsmasq does support collating both IPv4 and IPv6 IP addresses! :eek: but I guess no-one with an IPv6 ISP (my ISP is IPv4 only) who wishes to Selectively Route Netflix via the IPv6 WAN has found this to be an issue until now?
Click to expand...

The IPSet populating was a bit of a derp on my end initially (I forgot that IPv6 IPSets need the 'family inet6' pram), even so without a IPv6 supported VPN we just run into the same issues;

3Tauagk.png


Ry9vv4w.png



Taking this into consideration, the only way I can see any kind of "IPv6 support/leak protection" going forward is to manipulate dnsmasq to only lookup IPv4 addresses for those select domains..


Xentrk said:
Support for IPv6 was not in the original project scope. I can easily add the functionality in the follow-up release after getting the IPv4 project scope completed.
Click to expand...

Unless I'm overlooking something, wouldn't you run into the same issue?
 
Last edited:
Adamm said:
The IPSet populating was a bit of a derp on my end initially (I forgot that IPv6 IPSets need the 'family inet6' pram), even so without a IPv6 supported VPN we just run into the same issues;
Taking this into consideration, the only way I can see any kind of "IPv6 support/leak protection" going forward is to manipulate dnsmasq to only lookup IPv4 addresses for those select domains..
Click to expand...
Really, you mean like we do already - together with the IPv6 route ignore?
Adamm said:
Xentrk said:
Support for IPv6 was not in the original project scope. I can easily add the functionality in the follow-up release after getting the IPv4 project scope completed.
Click to expand...

Unless I'm overlooking something, wouldn't you run into the same issue?
Click to expand...

As per post #1

upload_2019-4-2_16-4-17.png


and @Xentrk fully understood my post...

upload_2019-4-2_16-4-44.png
 
Martineau said:
Really, you mean like we do already - together with the IPv6 route ignore?
Click to expand...

When using the custom directives you referenced all my clients loose IPv6 connectivity. Am I misunderstanding something here?
 
So I found a solution to IPv6 essentially leaking and bypassing selective routing rules. As I originally hypothesized the answer was manipulating dnsmasq to not return AAAA records on select domains. In the case of netflix, to bypass their geo-blocking this is only required on the main domain.


Add to /jffs/configs/dnsmasq.conf.add

Code: 
server=/netflix.com/127.0.0.1#5453 #VPNFlix
address=/netflix.com/:: #VPNFlix

Before:

Code: 
C:\Users\Adamm>nslookup netflix.com
Server:  UnKnown
Address:  2406:****:***:****::1

Non-authoritative answer:
Name:    netflix.com
Addresses:  2620:108:700f::342a:e4ed
          2620:108:700f::3645:effd
          2620:108:700f::3646:4946
          2620:108:700f::3647:6f22
          2620:108:700f::22d0:15cc
          2620:108:700f::23a0:b494
          2620:108:700f::23a6:44b7
          2620:108:700f::340b:18c1
          52.25.53.155
          52.26.111.43
          52.40.54.78
          52.41.20.47
          52.41.193.16
          54.69.16.110
          54.186.9.111
          35.164.65.152


After;

Code: 
C:\Users\Adamm>nslookup netflix.com
Server:  UnKnown
Address:  2406:****:***:****::1

Non-authoritative answer:
Name:    netflix.com
Addresses:  ::
          54.69.239.253
          54.70.73.70
          54.71.111.34
          34.209.185.102
          35.160.180.148
          35.166.68.183
          52.11.24.193
          52.42.228.237
 
Adamm said:
So I found a solution to IPv6 essentially leaking and bypassing selective routing rules. As I originally hypothesized the answer was manipulating dnsmasq to not return AAAA records on select domains. In the case of netflix, to bypass their geo-blocking this is only required on the main domain.


Add to /jffs/configs/dnsmasq.conf.add

Code: 
server=/netflix.com/127.0.0.1#5453 #VPNFlix
address=/netflix.com/:: #VPNFlix

Before:

Code: 
C:\Users\Adamm>nslookup netflix.com
Server:  UnKnown
Address:  2406:****:***:****::1

Non-authoritative answer:
Name:    netflix.com
Addresses:  2620:108:700f::342a:e4ed
          2620:108:700f::3645:effd
          2620:108:700f::3646:4946
          2620:108:700f::3647:6f22
          2620:108:700f::22d0:15cc
          2620:108:700f::23a0:b494
          2620:108:700f::23a6:44b7
          2620:108:700f::340b:18c1
          52.25.53.155
          52.26.111.43
          52.40.54.78
          52.41.20.47
          52.41.193.16
          54.69.16.110
          54.186.9.111
          35.164.65.152


After;

Code: 
C:\Users\Adamm>nslookup netflix.com
Server:  UnKnown
Address:  2406:****:***:****::1

Non-authoritative answer:
Name:    netflix.com
Addresses:  ::
          54.69.239.253
          54.70.73.70
          54.71.111.34
          34.209.185.102
          35.160.180.148
          35.166.68.183
          52.11.24.193
          52.42.228.237
Click to expand...
Very interesting solution. Thanks for posting the fix.

Those IPv4 and IPv6 addresses belong to Amazon AS16509:

https://bgp.he.net/AS16509#_prefixes
https://bgp.he.net/AS16509#_prefixes6

We all know by now that NF hosts on Amazon server. I could never get the selective routing to work on the netflix-vpn-bypass project unless I also specified amazonaws.com in the IPSET entry in dnsmasq.conf.add. Let me know if you are able to obtain your objective without having to add the entry.

I recently did a mock up on the Ubuntu VM on my laptop to add IPv6 functionality. Since I'm on the road, I can't test it on the router at this time. But even if I could, I don't have ISP support for IPv6 and I would never be able to spot an issue like this.
 
Last edited:
Xentrk said:
We all know by now that NF hosts on Amazon server. I could never get the selective routing to work on the netflix-vpn-bypass project unless I also specified amazonaws.com in the IPSET entry in dnsmasq.conf.add. Let me know if you are able to obtain your objective without having to add the entry.
Click to expand...

I haven't found the need to use ASN's at all, dnsmasq generating the ipset with the following domains is sufficient from my testing.

Code: 
echo "ipset=/netflix.com/nflxvideo.net/nflxso.net/nflxext.com/nflximg.net/VPNFlix-Netflix #VPNFlix" >> /jffs/configs/dnsmasq.conf.add
 
Adamm said:
I haven't found the need to use ASN's at all, dnsmasq generating the ipset with the following domains is sufficient from my testing.

Code: 
echo "ipset=/netflix.com/nflxvideo.net/nflxso.net/nflxext.com/nflximg.net/VPNFlix-Netflix #VPNFlix" >> /jffs/configs/dnsmasq.conf.add
Click to expand...

So @Adamm, do you use your own script for this selective routing then or have you simply modified @Xentrk’s script? Would you mind sharing the steps on how you got this to work on your AX88U? I would be interested to try it in mine. Thank you!


Sent from my iPhone using Tapatalk
 
Marin said:
So @Adamm, do you use your own script for this selective routing then or have you simply modified @Xentrk’s script? Would you mind sharing the steps on how you got this to work on your AX88U? I would be interested to try it in mine. Thank you!


Sent from my iPhone using Tapatalk
Click to expand...

My use case is a-little different, because I'm outside the USA I use it to bypass geo-blocking and only filter select traffic through the VPN.

Xentrk does the opposite, he puts all traffic through the VPN besides select traffic which goes through WAN.
 
Adamm said:
I haven't found the need to use ASN's at all, dnsmasq generating the ipset with the following domains is sufficient from my testing.

Code: 
echo "ipset=/netflix.com/nflxvideo.net/nflxso.net/nflxext.com/nflximg.net/VPNFlix-Netflix #VPNFlix" >> /jffs/configs/dnsmasq.conf.add
Click to expand...
What I meant is when using the ipset method for Netflix in dnsmasq, I also have to add the amazonaws.com domain with the other Netflix domain names.

For most of my selective routing needs, I was able to convert many streaming services to use ASN and it worked fine when watching from a Roku or Fire TV. But when I went to watch in a browser or iPad app, I had issues. The fix was to combine the ASN method with the IPSET method in dnsmasq.conf.add. Hulu was one example where this occurred. BBC was another. Most of the time, it involved looking at the web site source code for .net and .com and adding those domains to the ipset directive in dnsmasq.conf.add.

I would create one ipset list for the ASN and one for the dnsmasq entry and a separate iptables entry for each. This way, I could see the number of packets traversing each iptable entry in the chain.

Placing the ipset list iptables rule specified in dnsmasq below the iptables rule for the ASN ipset list gave me some indication of how many packets or IP address the ASN list didn’t catch.
 
Last edited:
Marin said:
So @Adamm, do you use your own script for this selective routing then or have you simply modified @Xentrk’s script? Would you mind sharing the steps on how you got this to work on your AX88U? I would be interested to try it in mine. Thank you!


Sent from my iPhone using Tapatalk
Click to expand...
I use 3 vpn clients for routing streaming media. I also have rules for the individual client devices. Most clients use a shared VPN server. But two family members prefer to use the WAN. :eek:

Very close to publishing the scripts/project on GitHub. Stay tuned.
 
Xentrk said:
What I meant is when using the ipset method for Netflix in dnsmasq, I also have to add the amazonaws.com domain with the other Netflix domain names.

For most of my selective routing needs, I was able to convert many streaming services to use ASN and it worked fine when watching from a Roku or Fire TV. But when I went to watch in a browser or iPad app, I had issues.
Click to expand...

From my testing I found that tunneling Amazon's whole CDN isn't necessary for Netflix, I can't speak for Hulu or BBC. But with 300ms latency from Australia > America I try to avoid any unnecessary VPN traffic. Amazon AWS is massive and not just Netflix content, is there a certain Netflix feature that is geo-restricted without it?
 
Adamm said:
My use case is a-little different, because I'm outside the USA I use it to bypass geo-blocking and only filter select traffic through the VPN.

Xentrk does the opposite, he puts all traffic through the VPN besides select traffic which goes through WAN.
Click to expand...

I see. Thank you!!


Sent from my iPhone using Tapatalk
 
Adamm said:
From my testing I found that tunneling Amazon's whole CDN isn't necessary for Netflix, I can't speak for Hulu or BBC. But with 300ms latency from Australia > America I try to avoid any unnecessary VPN traffic. Amazon AWS is massive and not just Netflix content, is there a certain Netflix feature that is geo-restricted without it?
Click to expand...
I can't recall the error message I received. I don't think it was the proxy error though. I think it was a general technical difficulties error message. I will test when I get home next week. It will be good for me to note the error and add it to the README.md.

Using the technique in IPSET_Netflix.sh, I only fetch Amazon US region domains:
Code: 
# Download Amazon AWS json file
download_AMAZONAWS () {
    wget https://ip-ranges.amazonaws.com/ip-ranges.json -O /opt/tmp/ip-ranges.json

    for REGION in us-east-1 us-east-2 us-west-1 us-west-2
        do
            jq '.prefixes[] | select(.region=='\"$REGION\"') | .ip_prefix' < "$FILE_DIR/ip-ranges.json" | sed 's/"//g' | sort -u >> "$FILE_DIR/x3mRouting_AMAZONAWS"
        done
    rm -rf /opt/tmp/ip-ranges.json
}
https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html

As a result, it also routes Amazon Prime traffic. But they are another service that blocks VPN providers so it ended up being a feature I needed and stumbled upon by accident when researching how to route Netflix.
 
Last edited:
Any updates on your update? I've royally badgered my set up after diversion ad block update and thinking I could mess with IPSET directly.

weirdly some domains respect the diversion route and some go through the VPN

Update: Sorted it using new script on github.
 
Last edited:
You must log in or register to reply here.

Similar threads

themaidenmaniac
x3mrouting selective routing help for a noobie
Replies
2
Views
556
lomax
L
R
Domain VPN Routing
2
Replies
24
Views
2K
Ranger802004
R
S
Wireguard VPN change routes ip
Replies
0
Views
199
S75
S
mattmcspirit
Wireguard Selective Routing - Cannot isolate to a single client
Replies
2
Views
634
mattmcspirit
mattmcspirit
dougm
[solved] PFSense+OpenVPN: Problems Routing Specific VLAN traffic out VPN
Replies
1
Views
2K
dougm
dougm
Similar threads
Thread starter Title Forum Replies Date
H Routing wireguard VPN 0
dougm [solved] PFSense+OpenVPN: Problems Routing Specific VLAN traffic out VPN VPN 1

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!

Members online

Total: 905 (members: 24, guests: 881)
Top