What's new

[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

Status
Not open for further replies.
Howdy.
I just notice my readynas NAS is uploading to google drive (its syncing process) as upload/download in the websurfing section.
I use this to push up my local CCTV IP TV recordings from an FTO share on the NAS that the cam's upload to.
I have no idea how I can change that type of traffic but I could add an IP address override for its uploads ?
Would that be the best way of doing it ?
Thanks
 
Howdy.
I just notice my readynas NAS is uploading to google drive (its syncing process) as upload/download in the websurfing section.
I use this to push up my local CCTV IP TV recordings from an FTO share on the NAS that the cam's upload to.
I have no idea how I can change that type of traffic but I could add an IP address override for its uploads ?
Would that be the best way of doing it ?
Thanks

Custom rule that sends all upload traffic from LAN device at IP 192.168.2.XXX -> FileTransfer Category.

See rule templates.
 
Thanks fresh for the quick reply.
I had seen this rule I could copy/edit in your sampes - realtc filter add dev eth0 protocol all prio $1 u32 match ip src 192.168.2.22/32 flowid ${VOIP}
But what threw me was the bit in the sample for this template rule 2 that said 'not working/use iptables alernative'.
The iptables then has this :
iptables -D POSTROUTING -t mangle -o eth0 -s 192.168.2.123/32 -j MARK --set-mark ${VOIP_mark} &> /dev/null
iptables -A POSTROUTING -t mangle -o eth0 -s 192.168.2.123/32 -j MARK --set-mark ${VOIP_mark}

FYI, my ip address for this readynas is 192.168.1.5, and I would want it part of the ${Others}
 
But what threw me was the bit in the sample for this template rule 2 that said 'not working/use iptables alernative'.

Correct.

The upload rule will NOT be following the typical tc rule format you would expect it too.
This is since if you applied the TC rule in the typical format, it would not achieve what you expect it too.

As a result I supplied an alternative iptables to achieve what is actually needed.

The following iptable rule should work.

Code:
iptables -D POSTROUTING -t mangle -o eth0 -s 192.168.1.5/32 -j MARK --set-mark ${Others_mark} &> /dev/null
iptables -A POSTROUTING -t mangle -o eth0 -s 192.168.1.5/32 -j MARK --set-mark ${Others_mark}

Be sure to place it in the iptables section.
 
Last edited:
Nice 1 - thanks for that.

edit - hhhmmm - did not work, do I need to restart the router, or the syncing process itself ? For this to take affect ?
 
Last edited:
did not work, do I need to restart the router, or the syncing process itself ?

PM me the output of

Code:
iptables-save -t mangle

For changes to take effect you have to

1) Upload the modified script
2) Restart QOS (just click Apply)
3) If the FileTransfer was started BEFORE QOS was start/restarted, then yes, the file transfer has to completely stop and be restarted.

QOS rules are only parsed on initial connection <<NOT ON EXISTING CONNECTIONS>>.

Pausing wont work either. Connection has to be fully restarted.
 
Last edited:
Hi, This is the output - I imagine the file transfer sync thing needs to be restarted then - if the below is correct.

admin@RT-AC68U-AC38:/tmp/home/root# iptables-save -t mangle
# Generated by iptables-save v1.4.15 on Thu Jun 21 13:37:31 2018
*mangle
:pREROUTING ACCEPT [953999:813021766]
:INPUT ACCEPT [18483:2940090]
:FORWARD ACCEPT [935341:810069822]
:OUTPUT ACCEPT [12076:3242260]
:pOSTROUTING ACCEPT [947433:813323388]
:BWDPI_FILTER - [0:0]
-A POSTROUTING -s 192.168.1.36/30 -o eth0 -m mark --mark 0x40000000/0x4000ffff -j MARK --set-xmark 0x40080001/0xffffffff
-A POSTROUTING -s 192.168.1.5/32 -o eth0 -j MARK --set-xmark 0x400a0001/0xffffffff
COMMIT
# Completed on Thu Jun 21 13:37:31 2018
admin@RT-AC68U-AC38:/tmp/home/root#
 
Hi, This is the output - I imagine the file transfer sync thing needs to be restarted then - if the below is correct.

The output is correct.

Just to confirm, when you see the following line in syslog

Code:
custom_script: Running /jffs/scripts/firewall-start (args: eth0)

it says

Code:
args:eth0

instead of

Code:
args:ppp0 or args:vlanXXX
 
no worries. I will get it and wack in my mods and re-do. Ta.

edit - qq - I downloaded it, do I rename it to the usual name on upload ?
 
no worries. I will get it and wack in my mods and re-do. Ta.

edit - qq - I downloaded it, do I rename it to the usual name on upload ?

Yes it will be called FreshJR_QOS

If you forgot to uninstall the fast version before uploading this alternate version, you will have to restart the router after running the install command on the alternative version script so the remnants of the Fast version are disabled.
 
Hi,

I did do the -uninstall.
Last question, I make no changes to the one called FreshJR_QOS_fast_v1_fakeTC ?

My personal instructions I have amended like so :

cd to :\ProgramFiles\Putty
pscp -scp C:\Users\Vaise\Downloads\FreshJR_QOS_v5_ALTERNATIVE_WAN_INTERFACE.txt admin@192.168.1.1:/jffs/scripts/FreshJR_QOS
pscp -scp C:\Users\Vaise\Downloads\FreshJR_QOS_fast_v1_fakeTC.txt admin@192.168.1.1:/jffs/scripts/FreshJR_QOS_fakeTC
To Install :
Use router admin to TURN OFF QOS
Start up putty and connect to the router as admin
dos2unix /jffs/scripts/FreshJR_QOS
dos2unix /jffs/scripts/FreshJR_QOS_fakeTC
sh /jffs/scripts/FreshJR_QOS -install
Use router admin to TURN ON QOS
 
I always wondered, if you use skype, the ports is says to open are thus :
  • 443/TCP.
  • 3478-3481/UDP.
  • 49152-65535/UDP + TCP.
How does that relate to the config file - duplicate the wifi lines for 3478:3481 ?
 
Last edited:
PORTS EXPLAINED
(Difference between Local/Remote Ports + Everything Else)


Every connection has two set ports.

The REMOTE port on the (WAN) server.
The LOCAL port on your (LAN) device.

In TC/Iptables these are distinguished by source/destination tags.

For Download traffic
sport = source port = port on the remote (WAN) server
dport = destination port = port of your local (LAN) device
(br0 interface is defined to catch only download traffic).​

For Upload traffic
sport = source port = port of your local (LAN) device
dport = destination port = port on the remote (WAN) server
(eth0 interface is defined to catch only upload traffic).
The remote and local ports are generally NOT the same.

---

Let's look at three examples.

A) VoIP
1) The (remote) VoIP server is typically hosted on port 4500
2) Your (local) device typically receives traffic on port 4500
**Android Wifi-Calling does NOT follow this pattern**

VOIP.png

B) Website
1) The (remote) website server is typically hosted on port 80 or 443.
2) Your (local) device will receive traffic on a random port in the range between 49152 - 65535.

This is called the ephemeral/dynamic port range.

Websites and applications using ephemeral port range will be assigned a random port on on every visit.

I also hope this made it easy to see as to why you should NOT create QOS rules (or even port forwards) EVER for ports 80 / 443 OR the ephemeral 49152-65535 range.

Website.png

C) Torrent
1) The (remote) peer will receive/send data from a port they have defined. This is not known beforehand.
2) Your (local) device will receive/send data from a port you have defined.

torrent.png

With this in mind, your custom port rules can filter by either.

1) the remote port from the (WAN) server.
or
2) the local port on your (LAN) device.
If you mix up the designations, don't be surprised if the rule is not catching any traffic.



---

One last bit of important information, TCP vs UDP.
What is the difference?

TCP - used for to transfer information WITHOUT DATA LOSS (most used protocol)
eg. receiving device sends ACKnowledgement of received/uncorrupted data
UDP - used for REAL TIME TRANSFER of ping sensitive information (data corruption possible)
eg. when transferring VoIP/Gaming packets there is no time to wait for an ACKnowledgement

UDP skips the ACKnowledgement protocol and will keep sending new data as it occurs without re-transmitting missing or corrupted data​

Why is this important?

TC rules filter on both UDP & TCP protocols without distinction.

Iptables need to have either the UDP or TCP protocol explicitly defined
(a wrong choice will make rule not work).

Knowledge of UDP / TCP difference can help narrow down between integrity sensitive or ping sensitive data​

---

Awesome, now that I am a port master, how do I find out what ports are being used?
Method 1) On the local device itself

You can check all port usage via the following:

Go to Task Manager -> Performance -> Resource Monitor -> Network.

Look for your executable in both

-TCP Connections (Active TCP connections show up here)
&&
-Listening Ports (Active UDP connections show up here + opened TCP connections)


--Information under TCP connections.
Local & Remote TCP ports defined here per application.

--Information under Listening Ports
Listening Ports & that ports Protocol are defined here.
Note: Listening Port means LOCAL port
Method 2) On the router

Network Tools --> Netstat --> Netstat-nat --> By Source IP

Local IP/Ports are on the left
Remote IP/Ports are on the right
Method 3) Look it up online​
--

No more port question will ever be answered.
It takes me 30 minutes to type out the same thing on EVERY question.
This is your one shot go to with pictures.
This information is also readily available online via search or even earlier in this thread.
The remote/local port designations are also explained in the comments of the template rules.
 
Last edited:
Last question, I make no changes to the one called FreshJR_QOS_fast_v1_fakeTC ?

fakeTC is not needed for the alternate version as it is uses the compatible version of the script as it's framework.

As such this line isn't needed.

Code:
pscp -scp C:\Users\Vaise\Downloads\FreshJR_QOS_fast_v1_fakeTC.txt admin@192.168.1.1:/jffs/scripts/FreshJR_QOS_fakeTC

But it doesn't hurt that an unused file is on the router, so you can leave it or delete it.
 
That new rule to send game updates to default works awesome. All my Steamm transactions were handled this way. Right on @FreshJR awesome work. Thanks again for the help!!
 
That new rule to send game updates to default works awesome. All my Steamm transactions were handled this way. Right on @FreshJR awesome work. Thanks again for the help!!

Unfortunately this wont always be the case depending on ISP.

I found that my local ISP caches **most** steam content. This means that my steam downloads actually originate from my local ISP's CDN instead of the steam hosted servers.

The issue is that my local ISP's cdn is https encrypted so QOS sticks it into WebSurfing.
(I think QOS identifies Valve servers into "gaming" via their CIDR IP range, which then get redirected from "gaming -> defaults" via my rule).

It may work for others as different ISP's may not have a CDN or they may not have an https encrypted CDN, but if the traffic not not originally picked up by QOS as "gaming", you are out of luck.

I've had good luck with PS4 downloads originating from SONY.
I am glad steam is working like that on your end.
 
Last edited:
hey @FreshJR im workin on a custom rule for someone here and seems you've redone your 1st 2 posts. Great job on that but im having some confusion most likely due to not messing with this for sometime. I'm trying to make a match mark rule for a game for someone. the game is wow. seems the database is classifying it as gaming as it should I guess but would like it moved into voip. Heres what ive got so far:

cat /tmp/bwdpi/bwdpi.app.db | grep -i "WOW Game"

gets me this
8,22,0,WOW Game

ive got this setup so far but I believe the bolded section im having trouble with:
${tc} filter add dev br0 protocol all prio 6 u32 match mark 0x80000016 0xc03f00ff flowid ${VOIP} #WOW Live

any suggestions? The hex stuff keeps confusing me
 
Status
Not open for further replies.

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top