[Release] FreshJR Adaptive QOS (Improvements / Custom Rules / and Inner workings)

[dave14305]

Hmm, interesting question. There was a period where AT&T screwed up a firmware update that dropped DMZ+ users' bandwidth by ~40% and people were switching to a different modem. Presumably I could look up that modem and buy one off ebay or Amazon.

I've been assuming the DHCP lease expiration is triggering this loop, so assuming it's not a failing modem, I'm not sure a different type of modem would fix the problem. I'll add it to the hail mary list once I've exhausted any other options...

This thread at AT&T suggests it's a function of the DMZ+ mode on the modem. The ASUS firewall will always restart once the WAN is reconnected in case the WAN IP changed. Hard to predict when QoS will reset without an obvious "restart_qos" appearing in the system log. With a 10 minute lease, it should try to renew after 5 minutes.

For further logging, I suggest creating empty scripts /jffs/scripts/wan-event and /jffs/scripts/dhcpc-event (and chmod 755 to make them executable). You will see more log entries when the WAN changes and WAN dhcp events happen.

 

[dave14305]

I'm a long time user of Adaptive QoS on my RRT-AC68U and use FreshJRs script. I have a number of devices on my network that connect to an internet host and constantly send data - CCTV camera, ADSB Receiver etc. and I've noticed that these are often not shown in the QoS Monitoring and I believe aren't being included in bandwidth management. I think I understand why:

When the router starts up these devices connect prior to QoS getting properly started. With FreshJRs script this is even further complicated as his script has to wait until QoS is fully up and running before applying the changes some minutes later. Additionally my router reboots every night because I suffer the memory leak from Trend Micro's AI Protection code so I need to try to automate a fix.

I'm guessing that if I can find a way to reset all active connections on the router after FreshJRs changes have applied this would cause the re-established connections to be properly shaped by QoS.

The nuclear option would be to run "service restart_wireless" if all the devices are wireless. If they are also wired, you would also have to restart the WAN as well with restart_net which would restart the firewall and QoS again and put you back in the same situation (not helpful).

 

[Jack Yaz]

The nuclear option would be to run "service restart_wireless" if all the devices are wireless. If they are also wired, you would also have to restart the WAN as well with restart_net which would restart the firewall and QoS again and put you back in the same situation (not helpful).

You could haul the radio down

wl -i eth5 radio off
sleep 10
wl -i eth5 radio on

 

[solstyce]

For further logging, I suggest creating empty scripts /jffs/scripts/wan-event and /jffs/scripts/dhcpc-event (and chmod 755 to make them executable). You will see more log entries when the WAN changes and WAN dhcp events happen.

Thanks, I'll give that a try.

Further reading makes me wonder if the AC68U firewall might be blocking the DHCP renewals requests, causing the connection reset and QoS reset.

I tried

iptables -I INPUT -p UDP  --dport 68 --sport 67 -j ACCEPT

but that didn't seem to do anything.

Can anyone tell me how I could check my firewall log to see if this is what's happening? Or comment on if this iptables command should work? My network knowledge is old-school, pre-Linux, pre-iptables and I'd expect I'm doing something wrong.

 

[dave14305]

Thanks, I'll give that a try.

Further reading makes me wonder if the AC68U firewall might be blocking the DHCP renewals requests, causing the connection reset and QoS reset.

I tried

iptables -I INPUT -p UDP  --dport 68 --sport 67 -j ACCEPT

but that didn't seem to do anything.

Can anyone tell me how I could check my firewall log to see if this is what's happening? Or comment on if this iptables command should work? My network knowledge is old-school, pre-Linux, pre-iptables and I'd expect I'm doing something wrong.

I'd be cautious of a 9 year old forum post, but it's an interesting theory. You might get better visibility in a new thread in this subforum, hopefully catching the attention of smart guys like @ColinTaylor.

You can checkout the existing rules and hits using these commands:

iptables -t mangle -nvL BWDPI_FILTER
iptables -nvL INPUT

These will show any hit counts associated with the existing rules below:

*mangle
-A BWDPI_FILTER -i eth0 -p udp -m udp --sport 68 --dport 67 -j DROP
-A BWDPI_FILTER -i eth0 -p udp -m udp --sport 67 --dport 68 -j DROP
*filter
-A INPUT -p udp -m udp --sport 67 --dport 68 -j ACCEPT

 

[solstyce]

I'd be cautious of a 9 year old forum post, but it's an interesting theory. You might get better visibility in a new thread in this subforum, hopefully catching the attention of smart guys like @ColinTaylor.

You can checkout the existing rules and hits using these commands:

iptables -t mangle -nvL BWDPI_FILTER
iptables -nvL INPUT

These will show any hit counts associated with the existing rules below:

*mangle
-A BWDPI_FILTER -i eth0 -p udp -m udp --sport 68 --dport 67 -j DROP
-A BWDPI_FILTER -i eth0 -p udp -m udp --sport 67 --dport 68 -j DROP
*filter
-A INPUT -p udp -m udp --sport 67 --dport 68 -j ACCEPT

Thanks, Dave! I'll give those a try.

I rebooted after creating the additional custom scripts and opening DHCP, but still have the same problem. Cloudflare still blocking my syslog, so representative sample here - https://pastebin.com/kXqgdiZG

 

[dave14305]

Thanks, Dave! I'll give those a try.

I rebooted after creating the additional custom scripts and opening DHCP, but still have the same problem. Cloudflare still blocking my syslog, so representative sample here - https://pastebin.com/kXqgdiZG

Also see this old thread, page 11: https://www.snbforums.com/threads/is-this-a-bug-on-asus-firmware-wan-disconnections.8433/page-11

Only helpful to understand the history of the existing rules meant to account for these odd DHCP issues.

 

[solstyce]

Also see this old thread, page 11: https://www.snbforums.com/threads/is-this-a-bug-on-asus-firmware-wan-disconnections.8433/page-11

Only helpful to understand the history of the existing rules meant to account for these odd DHCP issues.

Excellent, thanks! Working through it now.

Interestingly, I'm not seeing any traffic through my attempt to open DHCP ports via my own rule. But putting in those dummy custom scripts helped. With FreshJR's QoS on, I get 10 minute resets. With it turned off, I see "custom_script: Running /jffs/scripts/dhcpc-event (args: renew)" every 5 minutes and no drop in connection or firewall-restart execution.

So the renewal works fine without FreshJR's Q0S. That makes me think there's something about the QoS rules added by the script that is blocking the renewal from completing.

 

[dave14305]

Excellent, thanks! Working through it now.

Interestingly, I'm not seeing any traffic through my attempt to open DHCP ports via my own rule. But putting in those dummy custom scripts helped. With FreshJR's QoS on, I get 10 minute resets. With it turned off, I see "custom_script: Running /jffs/scripts/dhcpc-event (args: renew)" every 5 minutes and no drop in connection or firewall-restart execution.

So the renewal works fine without FreshJR's Q0S. That makes me think there's something about the QoS rules added by the script that is blocking the renewal from completing.

If you keep ASUS Adaptive QoS enabled, but FreshJR_QOS disabled, the behavior is normal?

If so, the only possible thing I can think of based on my analysis of the script these past couple weeks is the rule that marks any outbound traffic originating from the router as Downloads.
As a test, you could run these commands in SSH to replace the rule with one that excludes 67 and 68/udp:

iptables -D OUTPUT -t mangle -o eth0 -p udp -m multiport ! --dports 53,123 -j MARK --set-mark 0x40030001/0xffffffff
iptables -I OUTPUT 1 -t mangle -o eth0 -p udp -m multiport ! --dports 53,67,68,123 -j MARK --set-mark 0x40030001/0xffffffff

If everything goes to hell, you can get back by just restarting the firewall with "service restart_firewall".

 

[Deleted member 62525]

How do you guys setup Adoptive QoS download and upload limits when your local lan is split between some devices having direct WAN access and some with VPN? Do you first measure speed with VPN off or VPN on?

 

[L&LD]

The VPN speed will more than likely be below the WAN speed so no need to worry about that aspect in most cases.

I like to start with just the up or down value at 70% of the maximum value I'm adjusting (up or down ISP paid for speeds) and move in 10% increments to quickly narrow it down to what it prefers.

I then do a final tweak when both up and down are at their indicated best settings and test at finer increments as one will affect the other. 5% increments are usually sufficient for faster ISP providers (100Mbps up/down symmetrical or better). For slower connections (particularly the 'up' value), I go down to the single Mbps 'tweaks' range.

 

[Deleted member 62525]

The VPN speed will more than likely be below the WAN speed so no need to worry about that aspect in most cases.

I like to start with just the up or down value at 70% of the maximum value I'm adjusting (up or down ISP paid for speeds) and move in 10% increments to quickly narrow it down to what it prefers.

I then do a final tweak when both up and down are at their indicated best settings and test at finer increments as one will affect the other. 5% increments are usually sufficient for faster ISP providers (100Mbps up/down symmetrical or better). For slower connections (particularly the 'up' value), I go down to the single Mbps 'tweaks' range.

Thanks, that is how I set it up and it works really good for both WAN and VPN clients. Just wanted to make sure I did it right.

 

[dave14305]

If you keep ASUS Adaptive QoS enabled, but FreshJR_QOS disabled, the behavior is normal?

If so, the only possible thing I can think of based on my analysis of the script these past couple weeks is the rule that marks any outbound traffic originating from the router as Downloads.
As a test, you could run these commands in SSH to replace the rule with one that excludes 67 and 68/udp:

iptables -D OUTPUT -t mangle -o eth0 -p udp -m multiport ! --dports 53,123 -j MARK --set-mark 0x40030001/0xffffffff
iptables -I OUTPUT 1 -t mangle -o eth0 -p udp -m multiport ! --dports 53,67,68,123 -j MARK --set-mark 0x40030001/0xffffffff

If everything goes to hell, you can get back by just restarting the firewall with "service restart_firewall".

@solstyce Just wondering if you've tried this yet since you've also created a new thread.

 

[solstyce]

I use Comcast but I ended up buying my own modem. Are you able to use your own modem with ATT?

Called Sonic, and ATT only allow their own ATT equipment on this plan. Not to say I couldn't buy an aftermarket gateway and plug it in to see what happens... Thanks to dave14305's help it appears to be related to the firewall since renewals happen when QoS is off but fail when it's on. But I won't rule out hardware problems until we figure it all out. :fingers-crossed:

 

[solstyce]

If you keep ASUS Adaptive QoS enabled, but FreshJR_QOS disabled, the behavior is normal?

Hmm, good question. Disabled via "FreshJRQoS -disable" (but didn't reboot due to family on the net but did do "service restart_firewall") and then started Asus Adaptive QoS, and it has the same problem - WAN dropped and reconnected after 10 minutes. So I guess that means it's not a FreshJR issue but some other QoS problem. Thanks to the empty scripts, the calls being made are clearly disabling the WAN due to dhcp expiration.

Still no joy with Cloudflare, here's a section of the log with Asus Adaptive QoS on pastebin: https://pastebin.com/0Ecfdye6

I
If so, the only possible thing I can think of based on my analysis of the script these past couple weeks is the rule that marks any outbound traffic originating from the router as Downloads.
As a test, you could run these commands in SSH to replace the rule with one that excludes 67 and 68/udp:

iptables -D OUTPUT -t mangle -o eth0 -p udp -m multiport ! --dports 53,123 -j MARK --set-mark 0x40030001/0xffffffff
iptables -I OUTPUT 1 -t mangle -o eth0 -p udp -m multiport ! --dports 53,67,68,123 -j MARK --set-mark 0x40030001/0xffffffff

If everything goes to hell, you can get back by just restarting the firewall with "service restart_firewall".

Pardon my ignorance, but doesn't this just mark the DHCP (67,68) along with DNS and NTP traffic with the classification for tc to control? Oh, wait, I missed the bang, I'm guessing that excludes those ports.

Ok, started FreshJR QoS back up, waited 5 mins for the rules to apply, and then ran these commands. Still failed! *sigh* Log at https://pastebin.com/UFX4w7qp The one difference I notice is that it gave

May  6 18:59:41 WAN_Connection: ISP's DHCP did not function properly.

which I usually only see once on reboot when DHCP is first initializing. A clue?

 

[dave14305]

Hmm, good question. Disabled via "FreshJRQoS -disable" (but didn't reboot due to family on the net but did do "service restart_firewall") and then started Asus Adaptive QoS, and it has the same problem - WAN dropped and reconnected after 10 minutes. So I guess that means it's not a FreshJR issue but some other QoS problem. Thanks to the empty scripts, the calls being made are clearly disabling the WAN due to dhcp expiration.

Still no joy with Cloudflare, here's a section of the log with Asus Adaptive QoS on pastebin: https://pastebin.com/0Ecfdye6



Pardon my ignorance, but doesn't this just mark the DHCP (67,68) along with DNS and NTP traffic with the classification for tc to control? Oh, wait, I missed the bang, I'm guessing that excludes those ports.

Ok, started FreshJR QoS back up, waited 5 mins for the rules to apply, and then ran these commands. Still failed! *sigh* Log at https://pastebin.com/UFX4w7qp The one difference I notice is that it gave

May  6 18:59:41 WAN_Connection: ISP's DHCP did not function properly.

which I usually only see once on reboot when DHCP is first initializing. A clue?

How about disabling QoS and posting a log showing happy DHCP renewing for 60 minutes. Maybe there’s a clue in the absence of failure.

BTW Cloudflare will block posts that contain /etc when concatenated with hosts.

 

[solstyce]

@solstyce Just wondering if you've tried this yet since you've also created a new thread.

Just did, sorry for the delayed response, crazy days.

Also I apologise if I offended you with the new thread. New here, and don't know the protocol (pun intended)... Looking to increase my understanding of ipables so I can contribute more than my ignorance and my gratitude to the investigation

 

[dave14305]

Also I apologise if I offended you with the new thread.

No offense taken since I suggested the new thread earlier today. I was just unclear if the new thread had all the latest “findings” from this thread or not.

 

[maxbraketorque]

Just did, sorry for the delayed response, crazy days.

Also I apologise if I offended you with the new thread. New here, and don't know the protocol (pun intended)... Looking to increase my understanding of ipables so I can contribute more than my ignorance and my gratitude to the investigation

Will bypassing the gateway entirely resolve this issue?

https://www.dslreports.com/forum/r32491796-bypass-att-pace-gateway

 

[maxbraketorque]

Will bypassing the gateway entirely resolve this issue?

https://www.dslreports.com/forum/r32491796-bypass-att-pace-gateway

There are quite a few threads on this topic:

https://www.dslreports.com/forum/r32512782-AT-T-Fiber-Dumb-Switch-Bypass

https://www.dslreports.com/forum/r32524555-AT-T-Fiber-Automating-the-dumb-switch-bypass

And this:

https://github.com/bypassrg/att