Thank alot for the info. I'll try the setup later today. Last question I used Comcast and use their Stream app which allows you to see live TV content from your own home thru WIFI thru any device. However since I have PIA VPN set up to a server in the west coast, the stream app tells me to log on to my home network even though I am so it gets blocked. What would be the rule or how can I determine to add stream to bypass VPN?
Selective Routing for Netflix
- Thread starter Xentrk
- Start date
Xentrk
Part of the Furniture
You may have to mine dnsmasq log file to determine the domain names the Xfinity Stream app uses. To determine the domain names, install the script getdomainnames.sh to /jffs/scripts/getdomainnames.sh.
Code:
#!/bin/sh
# This script will format the output of tail -f dnsmasq.log > logfile
# where logfile is the output of tail -f dnsmasq.log
# 1. extract records whose contents contain the word "query" and the IP address of the
# client that is generating the taffic.
# 2. output only the domain names
# 3. sort file for unique contents to eliminate duplicates
# 4. save to $1_domains
#
# Parameters Passed
# $1 = provide the name of the source file when running the script
# e.g. ./getdomainnames.sh logfile IPv4_Address
# $2 = The IP address of the LAN client that generated the query in dnsmasq
#
# Uncomment the line below for debugging
#set -x
source_file=/opt/var/log/$1
output_file="${source_file}_domains"
IP=$2
egrep -w 'query|"$IP"' "$source_file" | awk '{ print $6 }' | sort -u > "$output_file"Navigate to the dnsmasq log file directory: cd /opt/var/log
Turn off the OpenVPN Client so all of your network traffic will traverse thru the WAN. Navigate to the dnsmasq log file directory /opt/var/log. Type the command to start capturing domains used by Xfinity Stream app:
Code:
tail -f dnsmasq.log > StreamNow, go to the device you are watching Stream from. If you are streaming from your PC or laptop, close out other applications to minimize collecting domain names for non-Netflix traffic. Watch several videos for a few minutes each to generate traffic and log entries to dnsmasq.log. Select all of the menu options.
When done generating Stream traffic, press ctrl-C to stop logging to the /opt/var/log/Stream file. Run the getdomainnames.sh script, passing the file name and IP address of the device you were watching Stream from. For example:
Code:
sh /jffs/scripts/getdomainnames.sh Stream 192.168.1.20This will create a file called Stream_domains in the /opt/var/log directory. Open the file in an editor to view the domains names collected when watching Stream. The next step is to desk check the file for domains not related to Stream. These are domains generated by other applications on the LAN client you streamed from.
Do not use the fully qualified domain name. For example, the domain occ-0-1077-1062.1.nflxso.net would be entered as nflxso.net; Likewise, www.netflix.com would be entered as netflix.com. Now that you have the top level domains, you can try to run the script and pass the top level domain names as the parameters per the usage instructions.
The nslookup <domain_name> command is useful in looking up IPv4 addresses associated with a domain. Once you have the IPv4 address, you can use the whob <IPv4 address> command to display more information about the domain to confirm if it is associated with Stream. whob is an entware package. Install using the command opkg install whob
Here is an example from Netflix:
Code:
# nslookup occ-0-1077-1062.1.nflxso.net
Server: 127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain
Name: occ-0-1077-1062.1.nflxso.net
Address 1: 2a00:86c0:600:96::138 ipv6_1.lagg0.c009.lax004.ix.nflxvideo.net
Address 3: 198.38.96.132 ipv4_1.lagg0.c003.lax004.ix.nflxvideo.net
# whob 198.38.96.147
IP: 198.38.96.147
Origin-AS: 2906
Prefix: 198.38.96.0/24
AS-Path: 18106 4657 6762 2906
AS-Org-Name: Netflix Streaming Services Inc.
Org-Name: Netflix Streaming Services Inc.
Net-Name: SSI-CDN-2
Cache-Date: 1536245423
Latitude: 39.738008
Longitude: -75.550353
City: Wilmington
Region: Delaware
Country: United States
Country-Code: USIf possible, you can try to use the ASN script if you find that all of the domains used by the Stream app resolved to the same ASN.
Xentrk
Part of the Furniture
i appreciate all the help...so Netflix, Hulu work flawlessly. i really do appreciate all the assistance. I'm trying to setup comcast gateway but don't understand the whole map /8 or 16 address. I looked at the other link you posted and I see the inputs below when searching "xfinity"....How do I map the /14 or /16 address when looking at the info below?
search "xfinity" from link:
https://gist.github.com/peeomid/d6a6d3ae48d8bb94051181870a967334
these are some of the lines found:
"id": 150,
"source": "xfinity",
"display_name": "Xfinity",
"type": "free",
"info": "http:\/\/xfinitytv.comcast.net\/",
"ios_app": "http:\/\/itunes.apple.com\/app\/xfinity-tv-go\/id552293383",
"android_app": "https:\/\/play.google.com\/store\/apps\/details?id=com.xfinity.playnow"
},
{
"id": 26,
"source": "xfinity_tveverywhere",
"display_name": "Xfinity",
"type": "tv_everywhere",
"info": "http:\/\/xfinitytv.comcast.net\/",
"ios_app": "http:\/\/itunes.apple.com\/app\/xfinity-tv-go\/id552293383",
"android_app": "https:\/\/play.google.com\/store\/apps\/details?id=com.xfinity.playnow"
},
{
"id": 222,
"source": "xfinity_purchase",
"display_name": "Xfinity",
"type": "purchase",
"info": "http:\/\/xfinitytv.comcast.net\/",
"ios_app": "http:\/\/itunes.apple.com\/app\/xfinity-tv-go\/id552293383",
"android_app": "https:\/\/play.google.com\/store\/apps\/details?id=com.xfinity.playnow"
},
{
Last edited:
Xentrk
Part of the Furniture
You can ask Xfinity what your gateway address is. Or, look at what they assign you. The command nvram show | grep gateway will show you the address. Say it returns a value like 123.25.30.254. You can then try entering 123.25.30.0/16 or 123.25.30.0/8 in the Policy Routing section of the OpenVPN Policy screen and route to the WAN interface.
Alternatively, you can go to https://bgp.he.net/ and type the name of the gateway IP address in the search bar and see what ASN the IP addresses is a member of. Then, use the ASN script to route the AS number to the WAN iface.
The "xfinity" from link was just used to provide some clues as to what to look for when mining the domain names using the getdomainnames.sh script. I primarily use the getdomainnames.sh script to find the domain names for the selective routing of streaming sites. The other technique I uses is to go to the website, right click on the page and select the View Source code option. I then search for .com and .net to see what domains they use. Reminder that you only need to use the top level domain name for the ipset method inside of dnsmasq to work. For a domain like xfinitytv.comcast.net, you only need to specify comcast.net as a parameter using the DNSMASQ script.
Last edited:
Xentrk
Part of the Furniture
I should have added to the above is that you can verify the gateway ip address/CIDR notation by using a subnet calculator.
I'll try this route 1st and hope this works...I'm out of town now will report back soon. Thanks again
UPDATE 1: Got it working. I used the ASN script as you mentioned in your earlier post and added the ASN command (sh /jffs/scripts/x3mRouting/load_ASN_ipset_iface.sh 0 XFINITY AS7922) to the nat-start and it started working. Thanks again for everything and your amazing script!!!!
Last edited:
Xentrk
Part of the Furniture
I am very happy x3mRouting worked for you and thank you for reporting back!! I made some updates to the scripts over the weekend to make sure the project is POSIX compliant.
You can update you current installation by running the installation menu. You will see an option #=[8] Update install_x3mRouting.sh. Then, choose option # [5] = Check for updates to existing x3mRouting installation to update the scripts.
Thank you for participating in the "soft" launch of the x3mRouting project. Once 384.12 goes live, I'll create a new support thread for the project.
Xentrk, i noticed when I used the AS7922 (Comcast) command in nat-start...my speeds drop significantly, my daily speeds average between 225-260mbps DL and 9 UL. However as soon as I input the command , reboot and wait the 5 minutes, my DL speeds drop under 100 Mbps and range from 75 -100. Any ideas why my speeds drop so much? As soon as remove it my speeds go back up to normal.
Update: I tried using AS7016 instead and it seems this has fixed the issue. We'll continue to monitor...
Update: I tried using AS7016 instead and it seems this has fixed the issue. We'll continue to monitor...
Last edited:
Xentrk
Part of the Furniture
I don't understand the reason for the slow down, especially if you are routing the Comcast gateway to the WAN. Is the client device you are testing speed with routed thru the VPN tunnel or the WAN interface? It is normal for downloads speeds to drop when using OpenVPN.
Did you make a typo? AS7916 doesn't belong to Comcast. Here is the list of AS numbers belonging to Comcast.
The device is currently being routed thru VPN. When I used the command (nvram show | grep gateway ) thru SSH, I received the address 73.175.128.1. When I go to the bge.he.net and input this address, that's when I see both 7922 and 7016. I was maybe thinking one is for strictly for cable tv and the other one is for internet but thats just a guess. I'll continue to monitor.
Last edited:
Xentrk
Part of the Furniture
I did a search on the IP address but replaced the 1 at the end with a zero:
The AS7922 probably appears because it is close match. You can also try just entering the 73.175.0.0/16 in the Policy Rule section of the OpenVPN Client screen and route it to the WAN interface. For example:
Comcast 0.0.0.0 73.175.0.0/16 WAN
Xentrk
Part of the Furniture
Right now, you have to access the menu using the command
sh /jffs/scripts/install_x3mRouting.sh
or, if you are in the /jffs/scripts directory: ./install_x3mRouting.sh
A short cut is the last item I want to incorporate before a formal go-live. Currently pondering the best way to implement. I may need to rename the project directory or move it to /opt/share from /jffs/scripts.
Since the project repository is the directory named /jffs/scrits/x3mRouting, I can't use x3mRouting for the short cut name. Linux doesn't like having a file with the same name as a directory.
Marin
Very Senior Member
Xentrk
Part of the Furniture
I just tested a proof of concept on how I can get it all to work. I will rename the install_x3mRouting.sh script to x3mRouting and place it in /opt/bin rather than /jffs/scripts. This appears to be the method used by Diversion written by @thelonelycoder . Eliminates the need for a symbolic link and still allows one to navigate to the project directory x3mRouting when inside of /jffs/scripts.
Xentrk
Part of the Furniture
@thelonelycoder, if I implement the solution above, will a firmware update wipeout the file I place in /opt/bin?
thelonelycoder
Part of the Furniture
I've read of the /jffs/ partition wipe out during firmware updates. But hat might have only been the case when the size of the partition intentionally changed or something was done to it to that effect.
Any attached USB devices are unaffected by firmware changes, exemting the rare corruption cases or device failures.
Placing the Diversion main script file directly into /opt/bin/ has been a wise decision when I laid the groundwork for Diversion at the time.
Xentrk
Part of the Furniture
Thank you. I will go ahead and make the change. I finished all of the posix issues this past week and all of my has been going good. This as the last item I wanted to take care of before formal go-live.
Xentrk
Part of the Furniture
@Marin,
I updated the repo so the menu x3mRouting is installed in the /opt/bin/ directory. The menu can now be accessed by typing the command x3mRouting at any location in the file system. You can download the new menu using the command:
Manually remove the old menu:
The README.md has been updated to reflect the change. No other programs were updated.
Thank you for the comments on the project!
I updated the repo so the menu x3mRouting is installed in the /opt/bin/ directory. The menu can now be accessed by typing the command x3mRouting at any location in the file system. You can download the new menu using the command:
Code:
/usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/Xentrk/x3mRouting/master/x3mRouting" -o "/opt/bin/x3mRouting" && chmod 755 /opt/bin/x3mRouting && x3mRoutingManually remove the old menu:
Code:
rm /jffs/scripts/install_x3mRouting.shThe README.md has been updated to reflect the change. No other programs were updated.
Thank you for the comments on the project!
Similar threads
Similar threads
| Thread starter | Title | Forum | Replies | Date |
|---|---|---|---|---|
| H | Routing wireguard | VPN | 0 | |
|
[solved] PFSense+OpenVPN: Problems Routing Specific VLAN traffic out VPN | VPN | 1 |
Similar threads
-
-
[solved] PFSense+OpenVPN: Problems Routing Specific VLAN traffic out VPN
- Started by dougm
- Replies: 1
Latest threads
-
RT-AX88U Pro and RT-AX86U Pro vs Dual GT-AXE11000 Setup- Started by Mxlt
- Replies: 1
-
-
-
Hyperotpic router replacement - will it solve my problem with connection dropping!
- Started by ACME NoLiFe
- Replies: 2
-
Sign Up For SNBForums Daily Digest
Get an update of what's new every day delivered to your mailbox. Sign up here!