Skynet Skynet - Router Firewall & Security Enhancements

[.TT.]

@Adamm v5.4.1 gives the following error when I run the banmalware option.

Removing Previous Malware Bans  [0s]
Downloading filter.list         [0s]
Whitelisting Shared Domains     /jffs/scripts/firewall: line 2034: can't fork

I'm currently only blocking 7 IPs

Same for me after update.

 

[Adamm]

@Adamm v5.4.1 gives the following error when I run the banmalware option.

Removing Previous Malware Bans  [0s]
Downloading filter.list         [0s]
Whitelisting Shared Domains     /jffs/scripts/firewall: line 2034: can't fork

I'm currently only blocking 7 IPs

From what I read this indicates a maximum process limitation, not something I can reproduce on my end though so it must be unique to the AC86U. Try update to v5.4.2

If its still causing issues, I'll have to see if I can manually increase this limit (and find out what it currently sits at).

 

[Adamm]

@Adamm v5.4.1 gives the following error when I run the banmalware option.

Removing Previous Malware Bans  [0s]
Downloading filter.list         [0s]
Whitelisting Shared Domains     /jffs/scripts/firewall: line 2034: can't fork

I'm currently only blocking 7 IPs

Same for me after update.

Could you both please also post the output of;

ulimit -Ha

 

[.TT.]

Tried a fresh install and it fails.
Checking Skynet IPTable... [Failed]
running ban malware:
Removing Previous Malware Bans [0s]
Downloading filter.list [0s]
Whitelisting Shared Domains /jffs/scripts/firewall: line 2035: can't fork

Spoiler: ulimit -Ha

-f: file size (blocks) unlimited
-t: cpu time (seconds) unlimited
-d: data seg size (kb) unlimited
-s: stack size (kb) unlimited
-c: core file size (blocks) 0
-m: resident set size (kb) unlimited
-l: locked memory (kb) 64
-p: processes 1719
-n: file descriptors 4096
-v: address space (kb) unlimited
-w: locks unlimited
-e: scheduling priority 0
-r: real-time priority 0

 

[Adamm]

Tried a fresh install and it fails.
Checking Skynet IPTable... [Failed]
running ban malware:
Removing Previous Malware Bans [0s]
Downloading filter.list [0s]
Whitelisting Shared Domains /jffs/scripts/firewall: line 2035: can't fork

Spoiler: ulimit -Ha

-f: file size (blocks) unlimited
-t: cpu time (seconds) unlimited
-d: data seg size (kb) unlimited
-s: stack size (kb) unlimited
-c: core file size (blocks) 0
-m: resident set size (kb) unlimited
-l: locked memory (kb) 64
-p: processes 1719
-n: file descriptors 4096
-v: address space (kb) unlimited
-w: locks unlimited
-e: scheduling priority 0
-r: real-time priority 0

You can ignore that failed message, above it there was probably a notice that there was a locked process meaning Skynet hadn't fully booted up. This takes 20-40 seconds.

As for the error, the process limit is slightly lower on the AC86U. Try force update and see if the patch I just uploaded makes a difference, I increased this number to 3000.

sh /jffs/scripts/firewall update -f

 

[.TT.]

Still no luck with banmalware.
Whitelisting Shared Domains /jffs/scripts/firewall: line 2036: can't fork

 

[SeaConn]

Could you both please also post the output of;

ulimit -Ha

Updated to 5.4.2.

Here ya go:

User0@RT-AC86U-98E0:/tmp/home/root# ulimit -Ha
-f: file size (blocks)             unlimited
-t: cpu time (seconds)             unlimited
-d: data seg size (kb)             unlimited
-s: stack size (kb)                unlimited
-c: core file size (blocks)        0
-m: resident set size (kb)         unlimited
-l: locked memory (kb)             64
-p: processes                      1719
-n: file descriptors               4096
-v: address space (kb)             unlimited
-w: locks                          unlimited
-e: scheduling priority            0
-r: real-time priority             0

 

[SeaConn]

Try force update and see if the patch I just uploaded makes a difference

Made no difference. I get the same error message as before.

 

[Jan Adelsson]

Is this normal?

 

[SeaConn]

Is this normal?

That's normal behavior if you haven't ran the number [3] Banmalware option yet.

Run option three and it should report that it's banning about three or four hundred thousand IPs next time you load up the menu.

 

[SeaConn]

Try force update and see if the patch I just uploaded makes a difference, I increased this number to 3000.

Just ran "Banmalware" option and got this:

Select Filter List:
[1] --> Default
[2] --> Custom

[1-2]: 1

Saving Changes                  [1s]
Removing Previous Malware Bans  [0s]
Downloading filter.list         [0s]
Whitelisting Shared Domains     [0s]
Consolidating Blacklist         [8s]
Filtering IPv4 Addresses        [1s]
Filtering IPv4 Ranges           [0s]
Applying Blacklists             [2s]

For False Positive Website Bans Use; ( sh /jffs/scripts/firewall whitelist domain URL )

Skynet: [Complete] 155911 IPs / 3066 Ranges Banned. 8163 New IPs / 0 New Ranges Banned.  Inbound /  Outbound Connections Blocked! [18s]

 

[skeal]

Looks good!

 

[Adamm]

Just ran "Banmalware" option and got this:

Interesting that the fix worked for you and not @.TT. ... Maybe when he updated it hadn't propagated on githubs servers yet so he only got the old version still?

Also that run time is damn fast on the AC86U 12s vs 40s on the AC68U

@.TT. try run the force update command again and see if it works.

sh /jffs/scripts/firewall update -f

 

[SeaConn]

Maybe when he updated it hadn't propagated on githubs servers yet so he only got the old version still?

Probably not the reason. It didn't work for me the first time, either.

I just ran Banmalware, and got the fork error again.

Select Filter List:
[1] --> Default
[2] --> Custom

[1-2]: 1                          

Saving Changes                  [2s]
Removing Previous Malware Bans  [2s]
Downloading filter.list         [0s]
Whitelisting Shared Domains     /jffs/scripts/firewall: line 2036: can't fork

But when I run it a second time, I get this:

Select Filter List:
[1] --> Default
[2] --> Custom

[1-2]: 1

Saving Changes                  [0s]
Removing Previous Malware Bans  [0s]
Downloading filter.list         [0s]
Whitelisting Shared Domains     [0s]
Consolidating Blacklist         [10s]
Filtering IPv4 Addresses        [1s]
Filtering IPv4 Ranges           [0s]
Applying Blacklists             [1s]

For False Positive Website Bans Use; ( sh /jffs/scripts/firewall whitelist domain URL )

Skynet: [Complete] 101350 IPs / 2604 Ranges Banned. -54561 New IPs / -462 New Ranges Banned.  Inbound /  Outbound Connections Blocked! [27s]

 

[Adamm]

@RMerlin Any ideas on this one? I can't find the specific error online, but those types of errors seem to point to process/resource limitations. I see a post from 2016 where you mention this could be a ram limitation, but considering the AC86U has twice that of a AC68U I doubt its running out. At its peak the script only increases ram usage by 20MB for around 2 seconds.

With that being said, this error is unique to the AC86U, all other models have no problem running the function. The code specifically causing the issue on this model is;

        cd /tmp/skynet || exit 1
        while IFS= read -r "domain"; do
            /usr/sbin/curl -fs "$domain" -O &
        done < /jffs/shared-Skynet-whitelist
        wait

Which basically just downloads a list of 32 small files in parallel. Is there some sort of limit possibly that was increased in the old codebase?

 

[RMerlin]

No idea, sorry.

There's just too much that got changed with this new platform, and I don't fully understand even half of it yet. Sometimes it causes weird issues such as the iptables one that I fixed earlier this week (I actually sent that fix upstream to Asus since it seemed potentially serious). That was was caused by GCC > 4.7.

I know that there's a problem related to the toolchain and forking that affected Asus's own code as well (the router would suddenly fail to fork() anything, it was a large part of what made the GT-AC5300 half-unusable during its first few months on the market). All they told me is there was a bug in the BCM toolchain, and they had to work their way around it. I was unable to obtain any further details when I asked them. Maybe the same issue is also affecting you, or it could be something different. The symptoms do look similar (failure to fork() ).

 

[Adamm]

No idea, sorry.

There's just too much that got changed with this new platform, and I don't fully understand even half of it yet. Sometimes it causes weird issues such as the iptables one that I fixed earlier this week (I actually sent that fix upstream to Asus since it seemed potentially serious). That was was caused by GCC > 4.7.

I know that there's a problem related to the toolchain and forking that affected Asus's own code as well (the router would suddenly fail to fork() anything, it was a large part of what made the GT-AC5300 half-unusable during its first few months on the market). All they told me is there was a bug in the BCM toolchain, and they had to work their way around it. I was unable to obtain any further details when I asked them. Maybe the same issue is also affecting you, or it could be something different. The symptoms do look similar (failure to fork() ).

That's a bummer. I'll disable the parallel downloads for the time being on the AC86U, not sure if it affects other models on the .382 codebase but if it does I will disable it for them all. Hopefully Asus find the cause in the next few GPL drops (or document how to avoid it).

@.TT. @SeaConn this change is live in v5.4.3. Unfortunately it will be slightly slower on the "Consolidating Blacklist" step, but at this time its unavoidable. Thanks again to you guys for helping me work the bugs out of the new platform. Hopefully I make some progress in the near future contacting Asus.

 

[SeaConn]

@.TT. @SeaConn this change is live in v5.4.3.

I forced the update to v5.4.3. Option 3. Banmalware yields the following fork error.

Select Filter List:
[1] --> Default
[2] --> Custom

[1-2]: 1

Saving Changes                  [0s]
Removing Previous Malware Bans  [0s]
Downloading filter.list         [0s]
Whitelisting Shared Domains     /jffs/scripts/firewall: line 2046: can't fork

Dang. Second try yields this:

Select Filter List:
[1] --> Default
[2] --> Custom

[1-2]: 1

Saving Changes                  [0s]
Removing Previous Malware Bans  [0s]
Downloading filter.list         [1s]
Whitelisting Shared Domains     /jffs/scripts/firewall: line 2046: echo: Cannot allocate memory
Consolidating Blacklist         [8s]
Filtering IPv4 Addresses        [1s]
Filtering IPv4 Ranges           [0s]
Applying Blacklists             [2s]

For False Positive Website Bans Use; ( sh /jffs/scripts/firewall whitelist domain URL )

Skynet: [Complete] 99362 IPs / 3085 Ranges Banned. 99362 New IPs / 3085 New Ranges Banned.  Inbound /  Outbound Connections Blocked! [18s]

So the first time I run it, it says it can't fork. Then the second time it says it can't allocate memory.

 

[Adamm]

So the first time I run it, it says it can't fork. Then the second time it says it can't allocate memory.

This is really strange, especially now the issue is occurring during single download. Seems like its curl in general causing the issue?

Perhaps updating to 7.56.1 would resolve the issue @RMerlin

EDIT; It actually looks like RMerlin ported over the curl 7.54.1 from the AM380 codebase, as Asus have curl-7.21.7 by default which is even older again. Maybe that's where the issue lies

 

[SeaConn]

Seems like its curl in general causing the issue?

Maybe we can change "curl -fs" to "curl -sS" so we can see if it's throwing an error?