Selective Routing for Netflix

[Xentrk]

Asus RT-AC5300 on Merlin WRT 380.69
The victory was shortlived - the AWS rules do seem to be allowing Prime Video playback but only on my desktop, when I used the app on my Shield TV I get blocked for using a VPN.

Netflix doesn't seem to be working either but I'm guessing that's because your script targets US servers/IPs? Any suggestions welcome

I noticed in the output of iptables -nvL PREROUTING -t mangle --line, it is reporting a fwmark of 0x8000 instead of 0x7000 that is used in the code.

Chain PREROUTING (policy ACCEPT 42537 packets, 5726K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 MARK       all  --  !eth0  *       0.0.0.0/0            192.168.0.7          MARK or 0x8000

When I tested the script, NF bypassed the VPN and I ended up with the Thai version of Netflix. I could tell as when I typed netflix.com, it resolved to netflix.com/th-en. Please double check the fwmark. I made a change to the OP

ip rule add from 0/0 fwmark 0x7000 table main prio 9990

changed to

ip rule add from 0/0 fwmark 0x7000/0x7000 table main prio 9990

 

[MrFahrenheit]

I noticed in the output of iptables -nvL PREROUTING -t mangle --line, it is reporting a fwmark of 0x8000 instead of 0x7000 that is used in the code.

Chain PREROUTING (policy ACCEPT 42537 packets, 5726K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 MARK       all  --  !eth0  *       0.0.0.0/0            192.168.0.7          MARK or 0x8000

When I tested the script, NF bypassed the VPN and I ended up with the Thai version of Netflix. I could tell as when I typed netflix.com, it resolved to netflix.com/th-en. Please double check the fwmark. I made a change to the OP

ip rule add from 0/0 fwmark 0x7000 table main prio 9990

changed to

ip rule add from 0/0 fwmark 0x7000/0x7000 table main prio 9990

Thanks, the output was from before I got the script to run. I've updated it including the line that you mention you've changed in the OP and now it looks like this:

Chain PREROUTING (policy ACCEPT 1285 packets, 588K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    81721   73M MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
2        0     0 MARK       tcp  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX dst,dst MARK or 0x7000

Just tried Netflix on browser and on the app and both still come up with the proxy/unblocker message

 

[Xentrk]

Thanks, the output was from before I got the script to run. I've updated it including the line that you mention you've changed in the OP and now it looks like this:

Chain PREROUTING (policy ACCEPT 1285 packets, 588K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    81721   73M MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
2        0     0 MARK       tcp  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX dst,dst MARK or 0x7000

Just tried Netflix on browser and on the app and both still come up with the proxy/unblocker message

How many VPN Clients do you have active?

Also, can you verify that you have Redirect Internet Traffic is set to Policy Rules or Policy Rules (Strict). When you issue the ip rule command, it should list the IP addresses of the clients you have defined in the OpenVPN Client web gui page. I don't see any clients listed when you provided the output in the previous post.

Try changing the setting Block routed clients if tunnel goes down to No

 

[steef84]

I see some of the domains belong to Akamai. I did a nslookup on their domain names against two sites I use. I got referred to Akamai. But could not find a match against the IPv4 address. To handle this, I do an nslookup on the the Akamai domain names you supplied and also included the AS numbers assigned to Akamai - As35944 and AS20940. I am not sure how doing do may impact your other streaming media sources though. I suggest first commenting out the sections that perform the nslookup for the Akamai AS Numbers, then uncomment and retest if it still does not work. If none of this works for you, I may want you to include a pic of your OpenVPN Web Gui screen just to make sure there is not some setting there that is causing the issue. If that does not work, I can try my original suggestion of filtering just for the EU IPv4 addresses.

#!/bin/sh
set -xo
####################################################################################################
#
# Author: Xentrk
# Dec-20-2017 Beta V 1.4 Added lookup for four CDN Akamai Technology domains.
#
#####################################################################################################
# Script Description:
#
# The purpose of this script is for selective routing of Netflix traffic using
# Autonomous System Numbers (ASNs). ASNs are assigned to entities such as Internet
# Service Providers and other large organizations that control blocks of IP addresses.
#
# Netflix and other services that use Amazon AWS servers are blocking VPN's.
#
# This script will
#   1. Create shared whitelist entry for ipinfo.io in /jffs/shared-SelectiveRouting-whitelist for use by AB-Solution and Skynet.
#      Otherwise, ipinfo.io may be blocked and the script will not work.
#     2. Obtain the IPv4 addresses used by Netflix and Amazon AWS USA from ipinfo.io.
#      IPv6 addresses are excluded in this version.
#   3. Create the IPSET list NETFLIX
#   4. Add the IPv4 address to the IPSET list NETFLIX
#   5. Route IPv4 addresses in IPSET list NETFLIX to WAN interface.
#
# Note 1: IPSET syntax differs between version 6 and 4.5
#             Syntax for ipset v6
#                ipset create WAN0 list:set
#                ipset add WAN0 setlist (e.g. SPEEDTEST)
#             for routers running ipset v4.5 (ipset -V)
#                ipset -N WAN0 setlist (e.g. SPEEDTEST)
#
# Note 2: In the event one needs to use IPv6 in the future, the syntax is: ipset -N NETFLIX-v6 hash:net family ipv6
#
# Note 3: Troubleshooting
#
#             You can use these sites for AS validation and troubleshooting to lookup ASNs:
#
#               https://bgp.he.net/AS16509 (Click on the prefixes tab to view IP addresses)
#               http://ipinfo.io/AS2906
#      
# Note 4: Required OpenVPN Client Settings
#
#         - Redirect Internet Traffic = Policy Rules or Policy Rules (Strict)
#         - Others?
#
#######################################################################
logger -t "($(basename $0))" $$ Starting IPSET_Netflix_2.sh..." $0${*:+ $*}."

# check shared-SelectiveRouting-whitelist so ipinfo.io is not blocked
# by AB-Solution and Skynet

echo -e "\n checking shared-SelectiveRouting-whitelist"
if [ -s "/jffs/shared-SelectiveRouting-whitelist" ];then
  echo " file found, no further checks"
else
  echo "generating missing file"
  echo "ipinfo.io" > /jffs/shared-SelectiveRouting-whitelist
fi

ipset create NETFLIX hash:net family inet hashsize 1024 maxelem 65536
ipset create AMAZONAWS hash:net family inet hashsize 1024 maxelem 65536
ipset create AKAMAI hash:net family inet hashsize 1024 maxelem 65536

# Pull all IPv4s listed for Netflix USA - AS2906
netsv4=`curl http://ipinfo.io/AS2906 2>/dev/null | grep -E "a href.*2906\/" | grep -v ":" |sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
for net in $netsv4
do
  ipset add NETFLIX $net
done
unset netsv4

# Pull all IPv4s listed for Amazon AWS - AS16509
netsv4=`curl http://ipinfo.io/AS16509 2>/dev/null | grep -E "a href.*16509\/" | grep -v ":" | sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
for net in $netsv4
do
  ipset add AMAZONAWS $net
done
unset netsv4

for domain_name in cdn-0.nflximg.com occ-0-2433-2430.1.nflxso.net occ-1-2433-2430.1.nflxso.net occ-2-2433-2430.1.nflxso.net
    do
      echo "domain name:" $domain_name
      for ip in $(nslookup $domain_name | awk '/^Name:/,0{if (/^Addr/)print $3}' | grep -v ":"); do
        echo "ip address is:" $ip
        ipset add AKAMAI $ip
      done
    done
unset ip
unset domain_name

# Pull all IPv4s listed for Akamai - 35994
netsv4=`curl http://ipinfo.io/AS35994 2>/dev/null | grep -E "a href.*35994\/" | grep -v ":" |sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
for net in $netsv4
do
  ipset add AKAMAI $net
done
unset netsv4

# Pull all IPv4s listed for Akamai - 20940
netsv4=`curl http://ipinfo.io/AS20940 2>/dev/null | grep -E "a href.*20940\/" | grep -v ":" |sed 's/^.*\">//; s/<.*//; /^\s*$/d'`
for net in $netsv4
do
  ipset add AKAMAI $net
done
unset netsv4

###########################################################
#Create table to contain items added automatically by wan #
###########################################################
ip rule del prio 9990
ip rule add from 0/0 fwmark 0x7000 table main prio 9990

iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst -j MARK --set-mark 0x7000/0x7000

iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set AMAZONAWS dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set AMAZONAWS dst -j MARK --set-mark 0x7000/0x7000

iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set AKAMAI dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set AKAMAI dst -j MARK --set-mark 0x7000/0x7000

logger -t "($(basename $0))" $$ Ending IPSET_Netflix_2.sh..." $0${*:+ $*}."

I tried the suggested script. Commented out the AS lines, refired the script, and then Netflix starts complaining. After uncommenting no joy. Below screenshots f

 

[Xentrk]

I tried the suggested script. Commented out the AS lines, refired the script, and then Netflix starts complaining. After uncommenting no joy. Below screenshots

Change the setting Block routed clients if Tunnel goes down to No. I think the Yes setting is preventing the NF traffic from traversing to the WAN. The other difference with my setting is Accept DNS Configuration. I have it set to exclusive so I use the DNS servers provided by the VPN server and prevent DNS leaks. Please try those changes and let me know. I also tested with Redirect Internet Traffic to Policy Rules (Strict).

 

[steef84]

Today I made a shot again. I changed the setting Block routed clients if tunnels go down to no and Accept DNS Configuration to exclusive.
I tried to script you supplied with the Akamai ASN.
1) First rebooted router.
2) Commented out the Akamai stuff. No joy .
3) uncommented the Akamai stuff. Still no joy.

The I read about you changed the OP, with the changed ip rule add from 0/0 fwmark 0x7000/0x7000 table main prio 9990 code. So I changed to script with this code. Don't know if its good but still

ip rule del prio 9990
ip rule add from 0/0 fwmark 0x7000/0x7000 table main prio 9990
iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst,dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst,dst -j MARK --set-mark 0x7000/0x7000

ip rule del prio 9990
ip rule add from 0/0 fwmark 0x7000/0x7000 table main prio 9990
iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set AKAMAI dst,dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set AKAMAI dst,dst -j MARK --set-mark 0x7000/0x7000

Then did the steps again. And NF welcomes me with the nice unblocker message again. Is there more I can try for now? I might provide you a config from my provider for testing purposes? Also: I like the setting to block the client if tunnels goes down. Is there a way to let this settings enabled?

 

[Xentrk]

Today I made a shot again. I changed the setting Block routed clients if tunnels go down to no and Accept DNS Configuration to exclusive.
I tried to script you supplied with the Akamai ASN.
1) First rebooted router.
2) Commented out the Akamai stuff. No joy .
3) uncommented the Akamai stuff. Still no joy.

The I read about you changed the OP, with the changed ip rule add from 0/0 fwmark 0x7000/0x7000 table main prio 9990 code. So I changed to script with this code. Don't know if its good but still

ip rule del prio 9990
ip rule add from 0/0 fwmark 0x7000/0x7000 table main prio 9990
iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst,dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst,dst -j MARK --set-mark 0x7000/0x7000

ip rule del prio 9990
ip rule add from 0/0 fwmark 0x7000/0x7000 table main prio 9990
iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set AKAMAI dst,dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set AKAMAI dst,dst -j MARK --set-mark 0x7000/0x7000

Then did the steps again. And NF welcomes me with the nice unblocker message again. Is there more I can try for now? I might provide you a config from my provider for testing purposes? Also: I like the setting to block the client if tunnels goes down. Is there a way to let this settings enabled?

Can you provide the output of these commands again?

iptables -nvL PREROUTING -t mangle --line
ip rule
ip route show | grep tun
cat /rom/etc/iproute2/rt_tables

I don't see the iptables chain for the WAN fwmark the script creates in your output for the iptables -nvL PREROUTING -t mangle --line command.

Do you have two vpn clients running? Does it also use a Policy Rules?

 

[RevNelson]

Thank you for this great script, Xentrk. When I run this, I keep getting the following error:

iptables: No chain/target/match by that name.

I have Policy Rules (Strict) enabled and can see it in effect when I run ip rule. Additionally, I can confirm the ipset is created correct by using ipset list. Unfortunately, when I run iptables -nvL PREROUTING -t mangle --line I only see the following:

admin@RT-AC5300-0000:/jffs/scripts# iptables -nvL PREROUTING -t mangle --line
Chain PREROUTING (policy ACCEPT 88989 packets, 82M bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    30629 4779K MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
2    34320 9322K BWDPI_FILTER  udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0

Any pointers to help me correct this? I'm on an ASUS AC5300 running Merlin 380.69.

 

[Xentrk]

Thank you for this great script, Xentrk. When I run this, I keep getting the following error:

iptables: No chain/target/match by that name.

I have Policy Rules (Strict) enabled and can see it in effect when I run ip rule. Additionally, I can confirm the ipset is created correct by using ipset list. Unfortunately, when I run iptables -nvL PREROUTING -t mangle --line I only see the following:

admin@RT-AC5300-0000:/jffs/scripts# iptables -nvL PREROUTING -t mangle --line
Chain PREROUTING (policy ACCEPT 88989 packets, 82M bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    30629 4779K MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
2    34320 9322K BWDPI_FILTER  udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0

Any pointers to help me correct this? I'm on an ASUS AC5300 running Merlin 380.69.

I hope the AC5300 does not have the same iptables issue as reported by several AC68U owners in the Selective Routing thread. For reasons unknown, the iptables command using the append (-A) appears to not be working. You will get an error when running for the first time on the first command with the -D (Delete) in it as the chain does not exist.

iptables -t mangle -D PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst,dst -j MARK --set-mark 0x7000/0x7000
iptables -t mangle -A PREROUTING -i br0 -p tcp -m set --match-set NETFLIX dst,dst -j MARK --set-mark 0x7000/0x7000

Turn on verbose output by removing the pound sign on the line

# set -xo

See if you get the message below for both iptables commands.

iptables: No chain/target/match by that name.

Let me know what happens and include the ouput of these commands:
iptables -nvL PREROUTING -t mangle --line
ip rule
ip route show | grep tun
cat /rom/etc/iproute2/rt_tables

Do you use Quality of Service (QOS)? If so, does turning it off have any impact? QOS uses 0X8000 fwmark. We should be okay. But something may have changed recently?

 

[JJQuin]

Thanks for the script. I've had PIA for the last 6 months but my wife was complaining she couldn't watch Netflix. I purchased NordVPN and it works great for Netflix and Hulu but now some websites come up really slow. I have NordVPN as my primary VPN and PIA setup as a backup (though this is still in testing - https://www.snbforums.com/threads/killswitch-move-to-next-vpn-client-in-line.43912).

I was wondering if it would be possible to change the script so instead of using the WAN interface it could use a different OpenVPN client. I would love to switch the VPNs around so PIA was the primary and NordVPN was only used for Netflix and as a backup. It looks like it would be a change to the iptables rules but I'm not familiar enough with iptables to make the change. I currently have NordVPN on client 3 and PIA on client 2. Any suggestions?

Thanks

 

[Xentrk]

Thanks for the script. I've had PIA for the last 6 months but my wife was complaining she couldn't watch Netflix. I purchased NordVPN and it works great for Netflix and Hulu but now some websites come up really slow. I have NordVPN as my primary VPN and PIA setup as a backup (though this is still in testing - https://www.snbforums.com/threads/killswitch-move-to-next-vpn-client-in-line.43912).

I was wondering if it would be possible to change the script so instead of using the WAN interface it could use a different OpenVPN client. I would love to switch the VPNs around so PIA was the primary and NordVPN was only used for Netflix and as a backup. It looks like it would be a change to the iptables rules but I'm not familiar enough with iptables to make the change. I currently have NordVPN on client 3 and PIA on client 2. Any suggestions?

Thanks

My selective routing script routes does exactly what you want. I have been makings some modifications this past week. Give me a day or two to polish it up. I have more improvements in the works. But I would like to see how it works on the AC86U. I'll send you a PM when ready along with a link to pastebin where you can download the code.

What encryption level are you using? Most people have been reporting really good OpenVPN performance with the AC86U. With OpenVPN 2.4, the GCM ciphers should help out. Do you have acceleration turned on? Or is it the WAN speed you get from your ISP?

 

[JJQuin]

Great, I'll be happily waiting for your PM! My max speed internet speed with my provider is about 118mpbs. With both VPNs I'm betting 108-114mbps. I'm using QoS so hardware acceleration is disabled.

The reason I chose the RT-AC86U was because of the aes-ni hardware encrypt/decrypt in the processor. According to Restore Privacy website (one of the few VPN review/info websites I've found that doesn't seem like an advertisement for the most popular VPNs) only the RT-AC86U and ROG GT-3200 have aes-ni. The GT-3200 was way to expensive, has a quad-core processor and doesn't work with Merlin so it was an easy decision.

According to what I understand a dual core processor is better for OpenVPN since it only uses one core to encrypt/decrypt.

RT-AC86U - 1800 dual core - each core 900Mhz
GT-AC5300 - 1800 quad core - each core only 450Mhz

You probably knew this already but hopefully it will help someone just starting out XD.

https://restoreprivacy.com/vpn-routers/

 

[JJQuin]

Hey Xentrik, so I have the script (beta 1.3) running and I can watch netflix on PIA (awesome!). But I have a minor issue. The scripts won't seem to run through nat-start. I have to manually ssh in and run the script after each reboot . I checked my nat-start and the script and they match what you have entered in this thread. I made the script executable, but still no luck. I checked the log and the nat-start is running but the IPSET-Netflix.sh is not. I confirmed the script name is the same in nat-start and /jffs/scripts. I also used notepad++ to make sure the script has the correct EOL.

#!/bin/sh
sh /jffs/scripts/IPSET_Netflix.sh

Contents of /jffs/scripts:

-rwxr-xr-x    1 xxx root          3731 Jan 22 15:45 IPSET_Netflix.sh
-rwxr-xr-x    1 xxx root         18818 Jan 21 00:57 amtm
-rw-rw-rw-    1 xxx root            46 Jan 22 16:45 nat-start
-rw-rw-rw-    1 xxx root            70 Jan 21 01:01 post-mount

From my log you can see nat-start but I don't see IPSET_Netflix.sh until I manually run it.

Jan 22 16:46:43 custom_script: Running /jffs/scripts/nat-start

I ran ip rule, ipset -L NETFLIX and iptables -nvL PREROUTING -t mangle --line before and after running the script manually after each reboot and it's obvious that the script is simply not running or getting stopped before it starts to create the NETFLIX ipset. I even deleted shared-SelectiveRouting, rebooted and manually ran sh nat-start. The shared-SelectiveRouting file was not created until I manually ran the script but running sh-nat-start worked perfectly. Not sure why it won't run correctly during reboot.

I'm on a RT-AC86U with 382.2_beta3 (official) installed. I currently don't have either AB-Solution or Skynet installed (was having issues with VPN blocking warframe servers and needed to confirm it what the cause was, haven't reinstalled yet).

Any help I would appreciate it!

Thanks

 

[Xentrk]

Hey Xentrik, so I have the script (beta 1.3) running and I can watch netflix on PIA (awesome!). But I have a minor issue. The scripts won't seem to run through nat-start. I have to manually ssh in and run the script after each reboot . I checked my nat-start and the script and they match what you have entered in this thread. I made the script executable, but still no luck. I checked the log and the nat-start is running but the IPSET-Netflix.sh is not. I confirmed the script name is the same in nat-start and /jffs/scripts. I also used notepad++ to make sure the script has the correct EOL.

#!/bin/sh
sh /jffs/scripts/IPSET_Netflix.sh

Contents of /jffs/scripts:

-rwxr-xr-x    1 xxx root          3731 Jan 22 15:45 IPSET_Netflix.sh
-rwxr-xr-x    1 xxx root         18818 Jan 21 00:57 amtm
-rw-rw-rw-    1 xxx root            46 Jan 22 16:45 nat-start
-rw-rw-rw-    1 xxx root            70 Jan 21 01:01 post-mount

From my log you can see nat-start but I don't see IPSET_Netflix.sh until I manually run it.

Jan 22 16:46:43 custom_script: Running /jffs/scripts/nat-start

I ran ip rule, ipset -L NETFLIX and iptables -nvL PREROUTING -t mangle --line before and after running the script manually after each reboot and it's obvious that the script is simply not running or getting stopped before it starts to create the NETFLIX ipset. I even deleted shared-SelectiveRouting, rebooted and manually ran sh nat-start. The shared-SelectiveRouting file was not created until I manually ran the script but running sh-nat-start worked perfectly. Not sure why it won't run correctly during reboot.

I'm on a RT-AC86U with 382.2_beta3 (official) installed. I currently don't have either AB-Solution or Skynet installed (was having issues with VPN blocking warframe servers and needed to confirm it what the cause was, haven't reinstalled yet).

Any help I would appreciate it!

Thanks

The executable bit is not on nat-start. Try chmod 755 nat-start. Then, run nat-start from the command line to see if it runs. For a final test, reboot or bounce the wan or one of the vpn clients and check to see if it ran. You can search for nat-start in the System Log.

 

[JJQuin]

Yea that was the problem. Figured since it was showing as started in the system log it was fine. Oh well still learning!

Thanks!

 

[JJQuin]

Anyone else having trouble with this script this morning? I noticed that my Netflix ipset was not populating.

When I run the curl command from the bash prompt i get this:

curl http://ipinfo.io/AS16509
<HTML><HEAD><TITLE>404 Not Found</TITLE></HEAD>
<BODY BGCOLOR="#cc9999"><H4>404 Not Found</H4>
File not found.
</BODY></HTML>

Been using this script for a couple weeks now with no issues. I added ipinfo.io to the shared whitelist but no luck. Anyone else seen this?

 

[JJQuin]

I reinstalled AB-Solution and pixelsrv after a reset. I made sure ipinfo.io was in the shared whitelist, but for some reason that failed. When I did an nslookup on ipinfo.io it gave my pixelsrv address. I used the AB-Solution menu to manually add it and that took care of the issue.

 

[T. Utech]

Xentrk,

Thank you very much for your excellent selective routing script. I am using it on my Asus RT-AC68U running 3.0.0.4.374.43_30E3j9527 of John's Merlin LTS fork. The script that I'm using comes from post 16 of this thread, beta 1.4 with the Akamai addition. I'm wondering if you figured out how to parse the JSON list of Amazon IPs as you mentioned in post 4 of this thread?

 

[Xentrk]

Xentrk,

Thank you very much for your excellent selective routing script. I am using it on my Asus RT-AC68U running 3.0.0.4.374.43_30E3j9527 of John's Merlin LTS fork. The script that I'm using comes from post 16 of this thread, beta 1.4 with the Akamai addition. I'm wondering if you figured out how to parse the JSON list of Amazon IPs as you mentioned in post 4 of this thread?

I did figure how to parse the JSON file. There is an entware package I had to install. I need to wrap up a few things. I will then post the instructions along with an new version.

 

[Xentrk]

This script will extract the IPv4 addresses for ALL Amazon AWS regions using the json file format (provided by Amazon AWS) and load them into the IPSET list AMAZONAWS. Entware package "jq" is required (opkg install jq) for the script to work.

#!/bin/sh
####################################################################################################
# get_aws.sh
# Written By: Xentrk
# Version 1.0
#
# Description:
# This script will extract all Amazon AWS locations and load them into the IPSET list AMAZONAWS
#
# Requirements: Entware Package jq
# jq Official Site: https://stedolan.github.io/jq/
#
# Amazon AWS JSON File Information (Reference Example 3. Get all IP addresses):
#  https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html
#
# Use this example to extract information for a specific region:
#   jq  '.prefixes[] | select(.region=="us-east-1")' < ip-ranges.json
#   Valid regions are:
#       ap-northeast-1 | ap-northeast-2 | ap-south-1 | ap-southeast-1 | ap-southeast-2 |
#       ca-central-1 | cn-north-1 | eu-central-1 | eu-west-1 | eu-west-2 | eu-west-3 |
#       sa-east-1 | us-east-1 | us-east-2 | us-gov-west-1 | us-west-1 | us-west-2 | GLOBAL
#
####################################################################################################
# Uncomment the line below for debugging
# set -xo

logger -t "($(basename $0))" $$ Starting" $0${*:+ $*}."

# Download Amazon AWS json file
wget https://ip-ranges.amazonaws.com/ip-ranges.json -O /jffs/scripts/ip-ranges.json

# Create IPSET lists
ipset create AMAZONAWS hash:net family inet hashsize 1024 maxelem 65536

#Pull all IPv4s listed for Amazon AWS

for IPv4 in `jq -r '.prefixes | .[].ip_prefix' < ip-ranges.json`
do
  ipset add AMAZONAWS $IPv4
done
unset IPv4

logger -t "($(basename $0))" $$ Ending..." $0${*:+ $*}."

I spent some time going thru the manuals trying to figure out the syntax for specifying one or more regions and have it return just the IPv4 addresses for the regions specified. The sample below returns more information than I want. But it does demo the use of the comparison operator. I'll take a look at it again tomorrow. json files and jq parsers are all new to me. If anyone else wants to collaborate on this, I would be grateful.

$ jq  '.prefixes[] | select(.region=="us-east-1")' < ipranges.json

{
  "ip_prefix": "23.20.0.0/14",
  "region": "us-east-1",
  "service": "AMAZON"
},
{
  "ip_prefix": "50.16.0.0/15",
  "region": "us-east-1",
  "service": "AMAZON"
},
{
  "ip_prefix": "50.19.0.0/16",
  "region": "us-east-1",
  "service": "AMAZON"
},

References:
1. jq Official Site: https://stedolan.github.io/jq/
2. Amazon AWS JSON File Information - https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html