1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

x3mRouting ~ Selective Routing for Asuswrt-Merlin Firmware

Discussion in 'Asuswrt-Merlin' started by Xentrk, Jul 28, 2019.

  1. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    Selectively route LAN clients, website or streaming media traffic over the WAN or OpenVPN client interfaces on Asuswrt-Merlin firmware using IPSET lists.

    Project Development
    I used Amazon Prime, BBC, CBS All Access, Hulu, Netflix and Sling streaming media services in developing the project and include them in the usage examples on the x3mRouting GitHub repository.

    Introduction
    The features of x3mRouting include three selective routing methods to select from:

    1. x3mRouting for LAN Clients Method

    An alternative approach to automate and easily assign LAN clients to a WAN or OpenVPN Client interface using a list of DHCP Static Lease clients to assign interfaces. This method eliminates the need to enter the LAN Client information and IP addresses in the OpenVPN Client Screen.

    Code:
    #########################################################
    # Assign the interface for each LAN client by entering  #
    # the appropriate interface number in the first column  #
    # 0 = WAN                                               #
    # 1 = OVPNC1                                            #
    # 2 = OVPNC2                                            #
    # 3 = OVPNC3                                            #
    # 4 = OVPNC4                                            #
    # 5 = OVPNC5                                            #
    #########################################################
    0 192.168.1.150 SamsungTV
    1 192.168.1.151 Samsung-Phone
    2 192.168.1.152 Asus-Laptop
    2 192.168.1.153 iPad
    1 192.168.1.154 Lenovo-Laptop
    

    2. x3mRouting OpenVPN Client Screen & IPSET Shell Script Method

    Provides the ability to create IPSET lists using shell scripts and selectively route the IPSET lists by entering the IPSET name in a modified OpenVPN Client Screen. This method can’t be used to route IPSET lists to the WAN interface. Use the x3mRouting IPSET Shell Script Method below if you have this requirement.

    @Martineau coded the revisions to the OpenVPN Client screen as a proof of concept on how the Policy Rules section could be modified to incorporate the selective routing of IPSET lists. I greatly appreciate his generosity in providing the modified code and allowing me to include it in the project.

    upload_2019-7-28_13-43-17.png

    3. x3mRouting IPSET Shell Script Method


    Provides the ability to create and selectively route IPSET lists using shell scripts. If you are a person who likes to flash alpha and beta software releases and perform firmware updates once they become available, this is the method for you. No modifications to the firmware source code are used in this method.

    Detailed descriptions and usage examples of each method are available on the x3mRouting GitHub repository.

    Requirements
    1. An Asus router with Asuswrt-Merlin firmware installed.
    2. A USB drive with entware installed. Entware can be installed using amtm - the SNBForum Asuswrt-Merlin Terminal Menu
    3. Policy Rules (Strict) or Policy Rules enabled on the OpenVPN Client screen.
    Installation
    You can install x3mRouting from an SSH command line or using AMTM.

    To install from the SSH command line, copy and paste the command below into an SSH session:

    Code:
    /usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/Xentrk/x3mRouting/master/x3mRouting" -o "/opt/bin/x3mRouting" && chmod 755 /opt/bin/x3mRouting && x3mRouting
    This command will download and install the installation menu x3mRouting to the /opt/bin directory. To access the installation menu, type the command x3mRouting on a command line. The installation script is a menu with options to install the three methods described and options to update or remove the repository. The x3mRouting programs are installed in the /jffs/scripts/x3mRouting directory.

    upload_2019-7-28_13-44-46.png

    Acknowledgements
    Please see the GitHub repository for all of the Acknowledgements

    I especially want to thank Martineau, who has been very generous in sharing his OpenVPN and Selective Routing expertise with me and others on the forum. This project was only made possible through his support and collaboration.
     
    Last edited: Aug 4, 2019
  2. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    Reserved
     
  3. Jack Yaz

    Jack Yaz Part of the Furniture

    Joined:
    Apr 20, 2017
    Messages:
    2,940
    It looks like a lot of files are needed - are they loose in /jffs/scripts or do you create your own directory?

    YazFi users - I don't anticipate any compatibility issues, but if you run into anything drop me a line and I'll work on it ASAP.
     
    QuikSilver and Xentrk like this.
  4. thelonelycoder

    thelonelycoder Part of the Furniture

    Joined:
    Jan 23, 2014
    Messages:
    6,454
    Location:
    Switzerland
    Everything in /jffs/ goes into the /x3mRouting folder. The main x3mRouting file is /opt/bin/x3mRouting.
     
  5. JFlynnXYZ

    JFlynnXYZ New Around Here

    Joined:
    Jul 28, 2019
    Messages:
    2
    Hey guys, I seem to be having some issues with my setup using this. I have a feeling I'm making a dumb mistake but have tried two days to get this to work with no luck. I have my vpn setup and am trying to get access to Netflix by using this routing method. I'm having no luck with it.

    I've tried the both the Netflix methods (originally doing it using netflix-vpn-bypass)
    * load_ASN_ipset.sh NETFLIX AS2906
    * load_DNSMASQ_ipset.sh NETFLIX amazonaws.com,netflix.com,nflxext.com,nflximg.net,nflxso.net,nflxvideo.net

    I've attached a log of the connection I made with dnsmasq --log-async --log-queries.

    Any help is much appreciated :)
     

    Attached Files:

  6. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    The entries in the dnsmasq.log file indicate that the IP addresses of the Netflix domain names are getting added to the IPSET list NETFLIX. You can verify this using the command:

    Code:
    ipset -L NETFLIX
    or the command
    Code:
    liststats
    You are using Method 2 of the scripts. With this method, the IPSET is created but not the routing. For the routing to work, you have to add the IPSET list name to the OpenVPN Client Screen. The screen only allows routing through the VPN interface and does not have the capability to route thru the WAN interface. For my use case, I have a VPN provider that provides me with a Private VPN Address to get around their VPN blocks. But most people use shared VPN servers that Netflix blocks. In this case, you need to route Netflix to the WAN interface using Method 3. With Method 3, the syntax is similar to Method 2. The difference is the name of the script and adding the interface name to the list of parameters you pass to the script.

    With Netflix, there are two options that will work. You can use the ASN or the DNSMASQ methods:

    ASN
    Code:
    sh /jffs/scripts/x3mRouting/load_ASN_ipset_iface.sh 0 NETFLIX AS2906
    DNSMASQ
    Code:
    sh load_DNSMASQ_ipset_iface.sh 0 NETFLIX amazonaws.com,netflix.com,nflxext.com,nflximg.net,nflxso.net,nflxvideo.net
    To check that the routing rules are in affect, type the command
    Code:
    iptables -nvL PREROUTING -t mangle --line
    Result
    Code:
    hain PREROUTING (policy ACCEPT 5808K packets, 6404M bytes)
    num   pkts bytes target     prot opt in     out     source               destination
    1        1    60 MARK       all  --  tun13  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    2     661K  863M MARK       all  --  tun15  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    3        1    60 MARK       all  --  tun14  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    4    76880   70M MARK       all  --  tun12  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    5    2030K 2737M MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    6        0     0 MARK       all  --  tun21  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    7        0     0 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX dst MARK set 0x1000
    8    1067K   60M MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set HULU_WEB dst MARK set 0x1000
    9    33488 6945K MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set AMAZON dst MARK set 0x1000
    10    129K 9898K MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set MOVETV dst MARK set 0x3000
    11   27284 5635K MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set CBS_WEB dst MARK set 0x3000
    12       0     0 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set BBC_WEB dst MARK set 0x4000
    Also, make sure Policy Rules (Strict) or Policy Rules are enabled on the OpenVPN Client Screen.
     
    Last edited: Jul 28, 2019
    Wisiwyg, royarcher, Torson and 2 others like this.
  7. JFlynnXYZ

    JFlynnXYZ New Around Here

    Joined:
    Jul 28, 2019
    Messages:
    2
    Perfect! I misunderstood some of the meaning of the scripts (very new to the whole router game) and simply ran the option 3 DNSMASQ code as you described and now it works perfectly!

    Do you have some kind of tip-jar or someway I can say thanks? :) Excellent work on the scripts, from all the people involved!

    Also, I fixed a bug (just a naming error with one of the functions) and did a pull request on GitHub https://github.com/Xentrk/x3mRouting/pull/4
     
    Kingp1n likes this.
  8. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    I merged the update. Thanks for letting me know about the typo.

    No tip jar at the moment. But I appreciate the offer. A lot of work and effort went into the project. Thanks for the feedback and grateful you got it working.
     
    royarcher, QuikSilver, L&LD and 2 others like this.
  9. Torson

    Torson Regular Contributor

    Joined:
    Aug 3, 2018
    Messages:
    67
    Hi, that's exactly what I was looking for ...

    My initial use case is just like the one detailed in post #6 - using a shared VPN provider, one device is routed through the VPN client (Policy Rules are enabled) to all destinations. Using method 3 I ran:
    Code:
    sh /jffs/scripts/x3mRouting/load_ASN_ipset_iface.sh 0 NETFLIX AS2906
    Netflix comes up with the usual VPN related error. A check if the routing rules are in effect yields:
    Code:
    Chain PREROUTING (policy ACCEPT 97647 packets, 122M bytes)
    num   pkts bytes target     prot opt in     out     source               destination
    1     2018 1942K MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    2    11860 1111K MARK       all  --  tun21  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    3     488K   95M BWDPI_FILTER  udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0
    4     5209  350K MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX dst MARK or 0x8000
    
    Does QoS get in the way of selective re-routing or any other issue?

    ___________________Edit____later _____in_____the_____day___________

    I turned QoS off, and the routing rules look like this:
    Code:
    Chain PREROUTING (policy ACCEPT 751 packets, 90157 bytes)
    num   pkts bytes target     prot opt in     out     source               destination
    1        7   968 MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    2        0     0 MARK       all  --  tun21  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    3    64714 3794K MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX dst MARK or 0x8000
    
    Still no dice... Netflix complains about a VPN or Proxy. Is the TV too smart?

    ***Running 384.13 Stable on RT-AC66U_B AiMesh mode with a similar model node + amtm, diversion & skynet latest***
     
    Last edited: Aug 1, 2019
  10. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    The routing iptables rules look okay to me. I see packets are traversing the iptables chain. Check the IPSET list has values using the command liststats or ipset -L NETFLIX.

    Try the DNSMASQ method and/or trying combining the ASN with the DNSMASQ method, which means running both scripts.

    One forum member had to combine the ASN and DNSMASQ method in the first iteration of x3mRouting called netflix-vpn-bypass project. I suspect it has to do with Content Delivery Networks (CDN) caching content to be geographically closer to the streaming location.

    DNSMASQ Method for Netflix
    Code:
    sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset_iface.sh 0 NETFLIX netflix.com,nflxext.com,nflximg.net,nflxso.net
    If you still have issues, try adding amazonaws.com to the list of domains. Netflix hosts on Amazon servers. Similarly, you can try the ASN method for Amazon:

    Code:
    sh /jffs/scripts/x3mRouting/load_AMAZON_ipset_iface.sh 0 AMAZON-US US
    I think one of the testers in EU ended up specifying the GLOBAL region. US works for me.
     
    Last edited: Aug 1, 2019
    Torson likes this.
  11. Torson

    Torson Regular Contributor

    Joined:
    Aug 3, 2018
    Messages:
    67
    Thank you for the reply... I wish it was that simple :)
    Actually I had to run both, DNSMASQ and ASN methods for Netflix (amazonws.com is in the list of domains.)

    As for ipset -L NETFLIX:
    Code:
    ipset -L NETFLIX
    Name: NETFLIX
    Type: hash:net
    Revision: 6
    Header: family inet hashsize 1024 maxelem 65536
    Size in memory: 8324
    References: 1
    Number of entries: 152
    Members:
    ...followed by 152 IP ranges
    and liststats:
    Code:
    AMAZON - 43
    NETFLIX - 152
    SkySports - 1
    Skynet-Blacklist - 154594
    Skynet-BlockedRanges - 1688
    Skynet-IOT - 0
    Skynet-Master - 2
    Skynet-Whitelist - 5067
    
    For Amazon and Netflix it's a SmartTV IP that I have in the rules for routing section.

    I have Amazon GLOBAL with 43 ranges.
    Please note that also SkySports does not route to VPN client 2:
    Code:
     sh load_DNSMASQ_ipset_iface.sh 2 SkySports www.skysports.com
    For SkySports I use a PC, which works just fine if I add it to the rules for routing in the router GUI for VPN client 2 but does not selectively route through interface 2 on the router.

    Is there anything else that I'm missing in the big picture?
     
    Last edited: Aug 1, 2019
  12. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    Is Netflix working for you now?

    I have Netflix working on FireTV, Roku, Nvidia Shield. I also tested using the iOS streaming media apps and streaming from a browser. My current set up is:

    Code:
    sh /jffs/scripts/x3mRouting/load_AMAZON_ipset_iface.sh 1 AMAZON_US US
    
    sh /jffs/scripts/x3mRouting/load_ASN_ipset_iface.sh 1 NETFLIX AS2906
    
    sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset_iface.sh 3 BBC_WEB bbc.co.uk,bbc.com,bbc.gscontxt.net,bbci.co.uk,bbctvapps.co.uk,ssl-bbcsmarttv.2cnt.net,llnwd.net
    sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset_iface.sh 5 CBS_WEB cbs.com,cbsnews.com,cbssports.com,cbsaavideo.com,omtrdc.net,akamaihd.net,irdeto.com,cbsi.com,cbsig.net
    sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset_iface.sh 1 HULU_WEB hulu.com,hulustream.com,akamaihd.net
    sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset_iface.sh 5 MOVETV movetv.com
    If you still have issues with Netflix, the next thing we can do is check the Routing Policy Database (RPDB) priorities. Please type the command ip rule at the command line and post results.

    For skysports.com, you can drop the "www" and just use skysports.com. Also, there are other domains you need to include.

    For example,
    Code:
    sh load_DNSMASQ_ipset_iface.sh 0 SKY sky.com autoscan
    Will get additional domains:

    Code:
    ipset=/akadns.net/edgekey.net/edgesuite.net/epgsky.com/sky.com/SKY from 'a674.hsar.cdn.sky.com.edgesuite.net/adm.sky.com/assets.sky.com/assets.sky.com-secure.edgekey.net/awk.epgsky.com' etc
    Similarly, you can try the ASN method for sky.com using AS5607

    Code:
    # nslookup skysports.com
    Server:    127.0.0.1
    Address 1: 127.0.0.1 localhost.localdomain
    
    Name:      skysports.com
    Address 1: 90.216.129.62
    
    # whob 90.216.129.62
    IP: 90.216.129.62
    Origin-AS: 5607
    Prefix: 90.192.0.0/11
    AS-Path: 18106 2914 5607
    AS-Org-Name: BSKYB-BROADBAND-AS
    Org-Name: Sky UK Limited
    Net-Name: BSKYB-BROADBAND
    Cache-Date: 1564643813
    Latitude: 51.508530
    Longitude: -0.125740
    City: London
    Region: England
    Country: United Kingdom
    Country-Code: GB
    
    Or, mine dnsmasq to get the top level domain names using the method described on the netflix-vpn-bypass project. The other technique I've used is to go to the website, right click on the page and select "View Source Code" form the menu. I then search on ".com" and ".net" or "href" to see what domains the site is using. For example, skysports.com website also has references to sky.com, skybet.com, akamai.net and 365dm.com.
     
    Last edited: Aug 1, 2019
    Torson likes this.
  13. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    @Torson

    Here are some sky.com domain names posted by @Martineau on the post which may be of some help. I took the raw list and processed it using the script getdomainnames.sh.

    Code:
    analytics.faw.sky.com
    awk.epgsky.com
    config.ott.sky.com
    config.skyq-b.interactive.sky.com
    desktop.client.ott.sky.com
    entity.search.sky.com
    images.metadata.sky.com
    init.sky.com
    metrics.sky.com
    p.sky.com
    qgo-desktop-sgw.skyanywhere.com
    static.skyq-b.interactive.sky.com
    uk.imageservice.sky.com
    Alternatively, you can try the ASN method: https://bgp.he.net/AS5607
     
    Last edited: Aug 2, 2019
    Torson likes this.
  14. Torson

    Torson Regular Contributor

    Joined:
    Aug 3, 2018
    Messages:
    67
    @Xentrk, thank you for the patience and guidance. I made some progress - I now have Netflix and Amazon running smoothly - tested briefly several movies/tv shows some I previously watched, some new; they all play well, no surprises there.
    It's all a bit convoluted, but that's how it seems to work North of the border where Rogers is the cable king and I'm spending some time here.
    As for Skysports, it starts playing the clips then it comes up with the dreaded geolocation thing... Actually I picked up a lot of .com and .net entries from the pages' source code. However, something is still missing - I'll keep digging.
    Also, I whitelisted some sites that were blocked by Diversion, but eventually I disabled it altogether while testing.

    Here is my current setup:
    Code:
    #!/bin/sh
    sh /jffs/scripts/x3mRouting/load_ASN_ipset_iface.sh 0 NETFLIX-812 AS812
    sh /jffs/scripts/x3mRouting/load_ASN_ipset_iface.sh 0 NETFLIX-14618 AS14618
    sh /jffs/scripts/x3mRouting/load_ASN_ipset_iface.sh 0 NETFLIX-394406 AS394406
    sh /jffs/scripts/x3mRouting/load_ASN_ipset_iface.sh 0 NETFLIX-2906 AS2906
    sh /jffs/scripts/x3mRouting/load_ASN_ipset_iface.sh 0 AMAZON-812 AS812
    sh /jffs/scripts/x3mRouting/load_ASN_ipset_iface.sh 0 AMAZON-16509 AS16509
    sh /jffs/scripts/x3mRouting/load_DNSMASQ_ipset_iface.sh 2 SkySports e0.365dm.com,e1.365dm.com,e2.365dm.com,fresh8.co,annex.fresh8.co,cdn-gs.fresh8.co,player.ooyala.com,static.chartbeat.com,assets.adobedtm.com,img.skysports.com,video.skysports.com,cdn.livefyre.com,app-cdn.spot.im,cdn.teads.tv,services.babator.com,eu-services.babator.com,widgets.oddschecker.com,pagead2.googlesyndication.com,metrics-api.librato.com,cloud-static.storage.googleapis.com,analytics.ooyala.com,l.ooyala.com,a248.e.akamai.net,ads.pubmatic.com,bskyb.demdex.net,config.seedtag.com,cs.seedtag.com,csync.smartadserver.com,dynamic-cdn.spot.im,fonts.googleapis.com,imasdk.googleapis.com,player.h-cdn.com,player2.h-cdn.com,recirculation.spot.im,secure-assets.rubiconproject.com,sync.richaudience.com,us-u.openx.net
    Code:
    /tmp/mnt/asus/conf# iptables -nvL PREROUTING -t mangle --line
    Chain PREROUTING (policy ACCEPT 18537 packets, 7329K bytes)
    num   pkts bytes target     prot opt in     out     source               destination
    1        0     0 MARK       all  --  tun21  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    2      409  357K MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    3       33  2563 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX-812 dst MARK or 0x8000
    4       84 21377 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX-14618 dst MARK or 0x8000
    5        0     0 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX-394406 dst MARK or 0x8000
    6        0     0 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX-2906 dst MARK or 0x8000
    7       32  2522 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set AMAZON-812 dst MARK or 0x8000
    8       84 10551 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set AMAZON-16509 dst MARK or 0x8000
    9       42  6706 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set SkySports dst MARK or 0x2000
    
    I tried to add an ASN Skysports rule, but didn't make any difference.
    The following part looks like an over sized hammer hitting a small nail, but the router is happy and everything else works like a charm (AiMesh included.)

    Code:
     liststats
    AMAZON-16509 - 3063
    AMAZON-812 - 574
    NETFLIX-14618 - 188
    NETFLIX-2906 - 152
    NETFLIX-394406 - 2
    NETFLIX-812 - 574
    SkySports - 152
    SkySports1 - 0
    Skynet-Blacklist - 152854
    Skynet-BlockedRanges - 1638
    Skynet-IOT - 0
    Skynet-Master - 2
    Skynet-Whitelist - 5015
    
    ..and finally
    Code:
     ip rule
    0:      from all lookup local
    9990:   from all fwmark 0x8000/0x8000 lookup main
    9994:   from all fwmark 0x2000/0x2000 lookup ovpnc2
    10101:  from 192.168.1.238 lookup ovpnc1
    32766:  from all lookup main
    32767:  from all lookup default
    
    ...now it would be nice to have the ability to concatenate the ASN statements based on ipset.

    Thanks again, and let me know if you have any ideas for Skysports and/or comments on the working stuff.
     
    Last edited: Aug 3, 2019
  15. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    The only ideas on Skysports and sky is to try AS5607 , the domain names I posted above and/or mining dnsmasq.log for the domain names gathered from surfing the website. I did some web searching and found the site below may be useful to find the top level domain names for skysports.com and sky.com.

    https://www.yougetsignal.com/tools/web-sites-on-web-server/


    upload_2019-8-3_15-27-19.png
    upload_2019-8-3_15-28-42.png

    You can use the domain names listed above in the dnsmasq method. Try it first without the amazon.com domain.

    For Netflix, I don't see any packets traversing the chain for these two IPSET lists:

    Code:
    5        0     0 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX-394406 dst MARK or 0x8000
    6        0     0 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX-2906 dst MARK or 0x8000
    That is one of the reasons for having separate routing rule per IPSET list as it helps in analysis and finding out if packets are traversing the routing rules. Routing rules are processed in the order listed. The traffic may be matching first on chains 3 and 4 and not reaching chains 5 and 6.

    Code:
    3       33  2563 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX-812 dst MARK or 0x8000
    4       84 21377 MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX-14618 dst MARK or 0x8000
    You can eliminate one of these lists are they are duplicates and the routing rules are the same:

    AMAZON-812
    NETFLIX-812

    I don't understand why AS812 is needed unless one is living in Canada.

    The AMAZON-16509 and NETFLIX-14618 list should be included in the Amazon method using the US region:

    Code:
    sh /jffs/scripts/x3mRouting/load_AMAZON_ipset_iface.sh 2 AMAZON-US US
    The other setting I recommend is to put the Router's IP address in the OpenVPN Client Screen and route it to the WAN iface. For my use case, I find the entry for the router is required in the OpenVPN Client 1 Screen when using more than one OpenVPN Client instance and have multiple rules routing traffic through more than one VPN interface and other traffic through the WAN interface.
     
    Torson likes this.
  16. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    Attention: x3mRouting for LAN Clients Method 1 users

    The program x3mRouting_client_config.sh that generates the file
    /jffs/configs/x3mRouting_client_rules (example below) has been updated for Asuswrt-Merlin 384.13.

    /jffs/configs/x3mRouting_client_rules
    Code:
    #########################################################
    # Assign the interface for each LAN client by entering  #
    # the appropriate interface number in the first column  #
    # 0 = WAN                                               #
    # 1 = OVPNC1                                            #
    # 2 = OVPNC2                                            #
    # 3 = OVPNC3                                            #
    # 4 = OVPNC4                                            #
    # 5 = OVPNC5                                            #
    #########################################################
    0 192.168.1.150 SamsungTV
    1 192.168.1.151 Samsung-Phone
    2 192.168.1.152 Asus-Laptop
    2 192.168.1.153 iPad
    1 192.168.1.154 Lenovo-Laptop
    To update, type x3mRouting at the command line and select option 5 from the menu to update:

    [5] = Check for updates to existing x3mRouting installation
     
    Torson, Kingp1n and andresmorago like this.
  17. CMc

    CMc Occasional Visitor

    Joined:
    Dec 1, 2018
    Messages:
    18
    Coming from the old script, is there a method (or actions) you recommend to 'upgrade' to your new script, to avoid conflicts etc? (or import existing settings).
     
  18. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    You shouldn't have to take any actions. You can always backup the /jffs partition or the /jffs/scripts/x3mRouting directory as an extra safety precaution. I sometimes make a backup directory of an existing installation during testing of changes so I can easily recover if need be.

    Code:
    mkdir /jffs/scripts/x3mRouting-bkup
    cp /jffs/scripts/x3mRouting/*.* /jffs/scrits/x3mRouting-bkup
    The script mentioned above is probably a script that is executed one time during the initial set-up and not run again, unless one adds more clients to static DHCP lease reservations and wants to create an updated list containing the newly added clients. If there is an issue, I could give you a command to download a prior version of the script on GitHub. This script does make a backup of /jffs/configs /x3mRouting_client_rules if one exists. So, if you did add new clients and need to generate a new file, you can copy/paste the contents from the backup file to the new file. Then, you only have to enter the interface assignments for the new clients.
     
  19. Torson

    Torson Regular Contributor

    Joined:
    Aug 3, 2018
    Messages:
    67
    Getting close to a fully functional system - all components work, but not together:

    1. Netflix and Amazon work properly when the router's IP address is added to the OVPN Client 1 (along with the TV's IP) and routed to the WAN interface. However, Skysports is not working.

    Code:
    Chain PREROUTING (policy ACCEPT 3834 packets, 1143K bytes)
    num   pkts bytes target     prot opt in     out     source               destination
    1      208  122K MARK       all  --  tun12  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    2     2461 1290K MARK       all  --  tun11  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    3        0     0 MARK       all  --  tun21  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x1/0x7
    4    41417 2447K MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set ROGERS_812 dst MARK or 0x8000
    5     4461 2250K MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set NETFLIX_14618 dst MARK or 0x8000
    6    52088   74M MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set AMAZON_16509 dst MARK or 0x8000
    7     3589  980K MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set SSp_rev dst MARK or 0x2000
    8    16158 2161K MARK       all  --  br0    *       0.0.0.0/0            0.0.0.0/0            match-set SkySports_masq dst MARK or 0x2000
    
    Code:
    0:      from all lookup local
    9990:   from all fwmark 0x8000/0x8000 lookup main
    9994:   from all fwmark 0x2000/0x2000 lookup ovpnc2
    10001:  from 192.168.1.1 lookup main
    10101:  from 192.168.1.238 lookup ovpnc1
    32766:  from all lookup main
    32767:  from all lookup default
    
    Code:
    AMAZON_16509 - 3064
    NETFLIX_14618 - 188
    ROGERS_812 - 574
    SSp_rev - 2
    SkySports_masq - 232
    
    2. Skysports works when removing the router's WAN interface from the OVPN Client 1. Netflix and Amazon do not work anymore.
    That's the only difference:
    Code:
    0:      from all lookup local
    9990:   from all fwmark 0x8000/0x8000 lookup main
    9994:   from all fwmark 0x2000/0x2000 lookup ovpnc2
    10101:  from 192.168.1.238 lookup ovpnc1
    32766:  from all lookup main
    32767:  from all lookup default
    
    Which would be the middle way where all 3 work together?
     
  20. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,748
    Location:
    The Land of Smiles
    Good news is you are closer now. But the results are a unexpected so I don't have an easy answer at the moment :confused:

    Two things are going on with selective routing.

    The first item is to create the IPSET lists for the traffic we want to route. Go back to the first setting in your post. Then, add AS2906 for Netflix as that one has been the go to ASN for Netflix for several years now for myself and others.
    Code:
    sh load_ASN_ipset_iface.sh 0 NETFLIX-2906 AS2906
    Test and see what happens.

    How are you creating the IPSET lists for Sky? Did you try including AS5607? I wonder if the IPSET lists for sky.com still need some additional entries. If so, then the proper matching is not occurring which may create issues. So if you can add AS5607, that will help eliminate that as a concern.

    Traffic is matched according to the RPDB priorities. The lower number has a higher priority.
    • Rogers, Netflix and Amazon match prio 9990 and route to the WAN.
    • Sky matches prio 9994 and routes to ovpnc2.
    • 192.168.1.238 matches prio 10101 and routes to ovpnc1.
    You can manually change the RPDB database. If you still have issues after you have entered the routers IP address to the OVPNC1 screen and route it to the WAN iface, remove the priority rule:
    Code:
    ip rule del prio 10001
    
    Then, add it back as the first priority above the fwmark/bitmask rules and see what happens:
    Code:
    ip rule add from 192.168.1.1 table main prio 9000
     
    Torson likes this.