FlexQoS FlexQoS 1.2.5 - Flexible QoS Enhancement Script for Adaptive QoS

[dave14305]

Anything I should be concern at all for my specific setup? I know using a full time VPN plus the additional scripts makes everything a little complicated with A.QoS.

Only be concerned if you reach the point when you no longer understand all the moving parts of your network.

I like experimenting with lots of things, but at the end of the day I prefer simplicity.

 

[sentinelvdx]

Don’t use the internet so much. The tracked connections are just a reflection of the activity on your network. You can reenable Refresh if you want to.

What do you think was happening at the time?

What?! Haha but that's why I use QoS, to don't worry when it's saturated.

I started looking and found several game clients downloading updates at the same time, plus my NAS uploading data to the cloud.
-Steam
-GoG
-Windows Updates
-My TV Streaming

After all finished doing their stuff all went back to normal

 

[geobernd]

Just wanted to add another word of praise:
I am switching my online backups from Crashplan to Backblaze (due to Crashplan's new 30 day only retention policy on deleted files).
Took me less than 5 minutes to add Backblaze's IP ranges to FlexQoS and classify so that all my other activities are not impeded. Now uploading at full throttle for a couple of weeks (thanks to only 18mbit/s upload) and if I didn't have a network traffic widget open I wouldn't even know there was any activity:
All other internet activity an any device - video chat, VoIP etc... - runs as if the whole connection was Idle....
Fantastic!!!

 

[dave14305]

Just wanted to add another word of praise:
I am switching my online backups from Crashplan to Backblaze (due to Crashplan's new 30 day only retention policy on deleted files).
Took me less than 5 minutes to add Backblaze's IP ranges to FlexQoS and classify so that all my other activities are not impeded. Now uploading at full throttle for a couple of weeks (thanks to only 18mbit/s upload) and if I didn't have a network traffic widget open I wouldn't even know there was any activity:
All other internet activity an any device - video chat, VoIP etc... - runs as if the whole connection was Idle....
Fantastic!!!

What range did you use? I use Backblaze B2 storage with Duplicati but haven’t setup a rule for it.

Another fun thing to watch is the FlexQoS Upload graph chugging away on File Transfers until you create higher priority traffic, then watch the big upload get throttled until things settle down again.

 

[geobernd]

I am using Backblaze Personal and added all 4 current IP ranges (probably only need the US West Coast ones but I had enough unused IP Table entries)... B2 probably uses different servers...

I actually gave it 'Others' so my legitimate File Transfers that I run every once in a while get higher priority...
And indeed I was watching the Upload graph to make sure it all works and my other transfer get higher priority etc... Networking can be fun

 

[dave14305]

I am using Backblaze Personal and added all 4 current IP ranges (probably only need the US West Coast ones but I had enough unused IP Table entries)... B2 probably uses different servers...

View attachment 36408

I actually gave it 'Others' so my legitimate File Transfers that I run every once in a while get higher priority...
And indeed I was watching the Upload graph to make sure it all works and my other transfer get higher priority etc... Networking can be fun

This is a really good case for using ipsets for the remote IPs, but there would not be an easy way to know what was contained in the ipset when processing the rules against the tracked connections in the GUI.

 

[geobernd]

This is a really good case for using ipsets for the remote IPs, but there would not be an easy way to know what was contained in the ipset when processing the rules against the tracked connections in the GUI.

I used the list of IPs from here - hopefully it won't catch much that is not Backblaze...
https://help.backblaze.com/hc/en-us/articles/217664588-Backblaze-IP-Address-List
From description it actually looks like it also covers the B2 servers....

 

[maghuro]

Something odd is happening.

I purposedly put Speedtest.net as Net Control and whenever I run a speedtest, it counts both as Net Control and Web Surfing.

The same tested with fast.com, it counts both as Streaming and WebSurfing.

Using the last "beta", is it possible that the ipv6 commit broke something?

https://pastebin.com/TLu2qe8g (Sorry, 10000 char limit)

Edit: I just noticed the following - Both fast.com and speedtest.net gave me around 850mbps down. As I can see, half of it (~400mbps) went to Web Surfing and the other half to Streaming (or NCP in the case)

 

[dave14305]

Something odd is happening.

I purposedly put Speedtest.net as Net Control and whenever I run a speedtest, it counts both as Net Control and Web Surfing.

The same tested with fast.com, it counts both as Streaming and WebSurfing.

Using the last "beta", is it possible that the ipv6 commit broke something?

https://pastebin.com/TLu2qe8g (Sorry, 10000 char limit)

Edit: I just noticed the following - Both fast.com and speedtest.net gave me around 850mbps down. As I can see, half of it (~400mbps) went to Web Surfing and the other half to Streaming (or NCP in the case)

View attachment 36414

Speedtests usually require multiple connections to load up a high-speed connection. It could be that only half the connections were properly classified by Adaptive QoS. You’d have to see what connections were listed under Tracked Connections for that device as Web Surfing while the speedtest was running. Usually on port 8080 or something close to that.

 

[dave14305]

I purposedly put Speedtest.net as Net Control and whenever I run a speedtest, it counts both as Net Control and Web Surfing.

I ran a speedtest.net test tonight and none of the connections were classified as Speedtest.net. I had Skype, Untracked, and HTTP (all on port 8080).

 

[maghuro]

S

I ran a speedtest.net test tonight and none of the connections were classified as Speedtest.net. I had Skype, Untracked, and HTTP (all on port 8080).

Something wrong with the last signature update?

 

[maghuro]

Here it goes.
Same IP, same exit port, different internal port.

Different classifications - HTTP and Speedtest.net.

Signatures?

 

[Kingp1n]

@dave14305

Just wanted to say, THANK YOU for all you do...the latest development version is working flawlessly even with my sucky comcast speeds (paying for 50/5 mbps) & all while using a VPN.

It's funny/crazy that I tested comcast's 400/10 mbps (also tested 200/5, 100/5) speeds and realized all you need is the proper router & script to route internet traffic accordingly. I didn't notice any difference and finally sticked with 50/5 and everything (I mean everthing) runs smoothly to include some 1080p/4K streaming content, a couple wired & wireless PS4/Xbox gaming devices playing multi-player online games with no lag.

I appreciate all you've done with FlexQoS.

 

[albinoman887]

is the bandwith not being messured when using a vpn still a thing ?

 

[BikeHelmet]

I think I stumbled across a bug. I rearranged my priority levels to better fit my network:

However, I then discovered that Learn-From-Home (Green) was using all the bandwidth. I dug into it, and... Disney+, YouTube, Netflix, FaceBook (videos on wall) were all in the Green category - Learn-From-Home - which by default is Video and Audio Streaming. I manually created rules to shuffle them into the proper purple category, which has vastly improved the internet experience, but before that it got bogged down and choked out websites from loading.

I use the Streaming category for latency non-sensitive streaming, and Work-From-Home for latency-sensitive streaming. YouTube and Netflix have 60+ second buffers, so a little bit of ping doesn't matter. Zoom calls though, or Discord, or VOIP - very important to be in the right category.

Just now I spotted regular websites in there too...

I am wondering if I should just reset FlexQOS and reconfigure, leaving the order of everything default? I assumed that if I shuffled priorities of Streaming lower, that the categorized stuff would go lower, not drop into the category that takes its place. It appears I'll have to rethink my %'s, as I had streaming stuff towards the bottom with high minimum bandwidth guarantees, and other stuff towards the top with lower guarantees. But if nothing can be moved without adding to the limited 32 rule AppDB area, my current arrangement isn't going to do the trick.

Any thoughts on how I should proceed?

Cheers,

 

[dave14305]

Any thoughts on how I should proceed?

Don’t think of Learn-From-Home as a useful category. Think of it like the human appendix — doesn’t serve a useful purpose anymore, but can sometimes lead to trouble, and should be removed.

Learn-From-Home consists of all Streaming and some Web Surfing. So your experience is 100% expected when Learn-From-Home is higher priority than Streaming and Web Surfing.

I have Learn-From-Home at the bottom and I don’t think about it at all.

 

[BikeHelmet]

Alright, so that's the expected behaviour - I'm curious though, what exactly ends up in Video and Audio Streaming if all the common video and audio streaming is in Learn From Home? Zoom and VOIP are in Work From Home... Streaming does very little? This is just a quirk of how QOS/TrendMicro do it, right? Got to learn their categorisation quirks and roll with it? It almost seems that the way they use categories, we could get away with only having ~5 buckets instead of 8.

I will move Learn From Home into the same area as File Transfers, now that I better understand what is in it. (All the high bandwidth stuff!)

I noticed you have Web Surfing under Others - is there a reason for that? What are your minimums/maximums %'s for each category? Back when I did Tomato QOS, I used to have Web Surfing (under 1024KB for web ports) very high up, then after it exceeded that it dropped into the category right above video streams. (YouTube, Netflix, etc.) That way downloads quickly got deprioritized without interfering with maximum website responsiveness. This was a good compromise considering that many other things use those ports - Steam game downloads, some XBox stuff, and even some game traffic itself. (Talk about annoying.) As such, it can't be deprioritized to the point that it's in the trash heap, or you end up with problems.

With that in mind, I'd like to understand why you have Web Surfing where it is, with Others above it.

Cheers,

 

[dave14305]

what exactly ends up in Video and Audio Streaming if all the common video and audio streaming is in Learn From Home?

Nothing will, unless you create iptables rules to force traffic to Streaming. All the traffic will end up in the higher priority bucket and nothing will reach Streaming.

This is just a quirk of how QOS/TrendMicro do it, right? Got to learn their categorisation quirks and roll with it? It almost seems that the way they use categories, we could get away with only having ~5 buckets instead of 8.

Yes, it’s been a head scratcher ever since they announced the new categories. Essentially, Learn-From-Home = Streaming + ( Web Surfing - Social Networks ). So all that would end up in Web Surfing would be social media traffic.

I only need 6 of 8 categories for my usage patterns.

I noticed you have Web Surfing under Others - is there a reason for that?

Some of my work VPN traffic ends up Untracked, so I want it higher than general web surfing, but lower than traffic like Teams or streaming. I take the approach that if it’s not general web surfing or file transfers, it gets higher priority. Others is not a high-volume category for me, so it’s not going to impact the lower priority categories too much.

 

[BikeHelmet]

Well, I am officially confused. I had created some rules sending Disney+, YouTube, YouTube Kids, Netflix, Snapchat and a few others into Streaming. I deleted them afterwards and moved Learn-From-Home to the bottom... only kept a couple rules like Snapchat and TikTok. Yet... they're still classifying as Streaming rather than Learn-From-Home, now? I feel like I'm missing some nuance or piece of info that will help me understand this stuff? The rules are gone - why did they stay in the specified categories?

 

[dave14305]

Well, I am officially confused. I had created some rules sending Disney+, YouTube, YouTube Kids, Netflix, Snapchat and a few others into Streaming. I deleted them afterwards and moved Learn-From-Home to the bottom... only kept a couple rules like Snapchat and TikTok. Yet... they're still classifying as Streaming rather than Learn-From-Home, now? I feel like I'm missing some nuance or piece of info that will help me understand this stuff? The rules are gone - why did they stay in the specified categories?

Any traffic previously landing in Learn-From-Home (when it was higher priority) will now be “claimed” first by Web Surfing or Streaming since both are evaluated first in the priority rules. Learn-From-Home is inert once it is lower priority than those other 2 categories.

I am confused by your screenshot showing Snapchat in Streaming from an AppDB rule (denoted by the ~), but your AppDB screenshot shows no such rule.

And the 2 rules moving TikTok and Crunchyroll would now be superfluous since they are in Streaming by default.