Skynet Skynet - Router Firewall & Security Enhancements

Adamm · Nov 7, 2017

2992 said:
I think what happens is that Banmalware doesn't run anymore right after reboot, as before this Skynet version, I could see in the log the Skynet message every hour (assuming the router rebooted 5am, the skynet was starting fine and the message was every hour, 6am, 7am, etc), but now, after rooboot, I cannot see that message anymore, for example, the router reboots at 5, then there is no message anymore until I manually run the Banmalware. After that, the message is back there every hour... until next reboot.

Nothing in the startup or save functions have been altered in months, so I think we need to look outside the box.

2992 said:
Anyways, this is 10 minutes after router reboot:

It seems to me that this issue may be specific to the (closed source) XWRT firmware. The fact Skynet isn't initiated for 10 minutes after a reboot throws up some red flags (this happens almost immediately on Asus devices)

Some other possibilities may be that the USB you have Skynet installed to (or possibly another script?) is deleting the Skynet files located in "/tmp/mnt/abs/skynet" during the boot process. Maybe the USB is corrupt or something of that nature, hard to say.

But with the output you posted above, the save function is working as expected and saving changes to "/tmp/mnt/abs/skynet/scripts/ipset.txt". With that being said, either this file is being deleted during boot or the ipset restore function isn't working as expected. Considering the way banmalware functions using this and your output above shows it working, the latter is much less likely.

I suggest the following;

1). Uninstall then reinstall Skynet
2). When reinstalling Skynet, try install it to JFFS and see if the issue persists. That way we can rule out USB related issues.

2992 · Nov 7, 2017

Adamm said:
Nothing in the startup or save functions have been altered in months, so I think we need to look outside the box.

It seems to me that this issue may be specific to the (closed source) XWRT firmware. The fact Skynet isn't initiated for 10 minutes after a reboot throws up some red flags (this happens almost immediately on Asus devices)

Some other possibilities may be that the USB you have Skynet installed to (or possibly another script?) is deleting the Skynet files located in "/tmp/mnt/abs/skynet" during the boot process. Maybe the USB is corrupt or something of that nature, hard to say.

But with the output you posted above, the save function is working as expected and saving changes to "/tmp/mnt/abs/skynet/scripts/ipset.txt". With that being said, either this file is being deleted during boot or the ipset restore function isn't working as expected. Considering the way banmalware functions using this and your output above shows it working, the latter is much less likely.

I suggest the following;

1). Uninstall then reinstall Skynet
2). When reinstalling Skynet, try install it to JFFS and see if the issue persists. That way we can rule out USB related issues.

Could be because of XWRT, but I personally don't think so, as all the previous Skynet versions ran fine, and I haven't changed or modified anything since the first firmware install (few months ago).
I will re-install Skynet as you wrote, later today or tomorrow (limited free time here, factor in the wife...), so we can factor out the USB, but again, I personally think the USB is fine, as ABSolution is running perfectly fine from the same USB.

PS: is there any way I can install a previous version of Skynet?..

(PPS: I incline to believe something is not running fine on the Skynet version I have now here, factoring in the Dropbox connectivity issue I've wrote in my previous post. Maybe something is corrupted on it, or I do not know. I will re-instal it again, then let's see what gives.)

.TT. · Nov 7, 2017

@Adamm
I am seeing some new warnings in the latest version.

Downloading filter.list [0s]
Whitelisting Shared Domains Consolidating Blacklist [7s]
Saving Changes [2s]
Removing Previous Malware Bans ipset v6.32: Missing mandatory argument of optio n `comment'
Try `ipset help' for more information.
Filtering IPv4 Addresses [2s]
Filtering IPv4 Ranges [0s]
Applying Blacklists ipset v6.32: Missing mandatory argument of optio n `comment'
Try `ipset help' for more information.

For False Positive Website Bans Use; ( sh /jffs/scripts/firewall whitelist domai n URL )

Skynet: [Complete] 150746 IPs / 3369 Ranges Banned. 6658 New IPs / 201 New Range s Banned. 13 Inbound / 29 Outbound Connections Blocked! [133s]

Function is still ok =)

Adamm · Nov 7, 2017

.TT. said:
@Adamm
I am seeing some new warnings in the latest version.

Downloading filter.list [0s]
Whitelisting Shared Domains Consolidating Blacklist [7s]
Saving Changes [2s]
Removing Previous Malware Bans ipset v6.32: Missing mandatory argument of optio n `comment'
Try `ipset help' for more information.
Filtering IPv4 Addresses [2s]
Filtering IPv4 Ranges [0s]
Applying Blacklists ipset v6.32: Missing mandatory argument of optio n `comment'
Try `ipset help' for more information.

For False Positive Website Bans Use; ( sh /jffs/scripts/firewall whitelist domai n URL )

Skynet: [Complete] 150746 IPs / 3369 Ranges Banned. 6658 New IPs / 201 New Range s Banned. 13 Inbound / 29 Outbound Connections Blocked! [133s]

Function is still ok =)

Well this is definitely the week of unusual errors. Somehow an entry in your ipset.txt file has the comment arg specified but no comment, not quite sure how that happened but to track it down issue the following command (and paste the result here and I can give you a followup command to remove it)

Code:

grep "^add .* comment$" /tmp/mnt/Asus/skynet/ipset.txt

Also like always make sure you're running the latest version.

Edit;

Assuming the command above didn't output anything too important, you can fix it via;

Code:

sh /jffs/scripts/firewall disable

grep "^add .* comment$" /tmp/mnt/Asus/skynet/ipset.txt | while read -r badipset; do sed -i "\\~$badipset~d" "/tmp/mnt/Asus/skynet/ipset.txt"; done

sh /jffs/scripts/firewall restart

.TT. · Nov 7, 2017

All good again after running:

Code:

sh /jffs/scripts/firewall disable

grep "^add .* comment$" /tmp/mnt/Asus/skynet/ipset.txt | while read -r badipset; do sed -i "\\~$badipset~d" "/tmp/mnt/Asus/skynet/ipset.txt"; done

sh /jffs/scripts/firewall restart

Thanks =)

.TT. · Nov 7, 2017

Not sure if it is possible but i would love to see all IPs detected/blocked by AI-protection (two-way ips) included in auto ban as well. Any way Skynet can read this data?

Protik · Nov 7, 2017

So after a manual reboot, the Skynet log entries are,

Code:

Nov  7 09:37:34 Skynet: [INFO] Startup Initiated... ( debug banmalware autoupdate usb=/tmp/mnt/pdas001 )
Nov  7 09:37:43 Skynet: [INFO] Updating VPN Whitelist...
Nov  7 09:37:52 Skynet: [Complete] 42208 IPs / 0 Ranges Banned. 42208 New IPs / 0 New Ranges Banned.  Inbound /  Outbound Connections Blocked! [10s]
Nov  7 09:38:02 Skynet: [INFO] Lock File Detected (start debug banmalware autoupdate usb=/tmp/mnt/pdas001) (pid=1153) - Exiting
Nov  7 09:38:09 Skynet: [Complete] 42208 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [36s]

Then-

Code:

wc -l /tmp/mnt/pdas001/skynet/scripts/ipset.txt
42261 /tmp/mnt/pdas001/skynet/scripts/ipset.txt

And-

Code:

{ ipset save Whitelist; ipset save Blacklist; ipset save BlockedRanges; ipset save Skynet; } | wc -l
42261

Then I ran banmalware-

Code:

admin@RT-AC68U-DF28:/tmp/home/root# /jffs/scripts/firewall
#!/bin/sh
#############################################################################################################
#                               _____ _                     _           _____                               #
#                              / ____| |                   | |         | ____|                              #
#                             | (___ | | ___   _ _ __   ___| |_  __   _| |__                                #
#                              \___ \| |/ / | | | '_ \ / _ \ __| \ \ / /___ \                               #
#                              ____) |   <| |_| | | | |  __/ |_   \ V / ___) |                              #
#                             |_____/|_|\_\\__, |_| |_|\___|\__|   \_/ |____/                               #
#                                           __/ |                                                           #
#                                          |___/                                                            #
#                                                                                                           #
## - 5/11/2017 -                   Asus Firewall Addition By Adamm v5.4.9                                   #
##                                 https://github.com/Adamm00/IPSet_ASUS                                    #
#############################################################################################################


Router Model; RT-AC68U
Skynet Version; v5.4.9 (5/11/2017)
iptables v1.4.14 - (eth0 @ 192.168.2.1)
ipset v6.32, protocol version: 6
FW Version; 380.68_4 (Oct 4 2017) (2.6.36.4brcmarm)
Install Dir; /tmp/mnt/pdas001/skynet (1.4G / 1.8G Space Available)
SWAP File; /tmp/mnt/pdas001/ptkswap.swp (256.3M)
Boot Args; /jffs/scripts/firewall start debug banmalware autoupdate usb=/tmp/mnt/pdas001

42208 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 4 Inbound / 0 Outbound Connections Blocked!

Select Menu Option:
[1]  --> Unban
[2]  --> Ban
[3]  --> Banmalware
[4]  --> Whitelist
[5]  --> Import IP List
[6]  --> Deport IP List
[7]  --> Save
[8]  --> Restart Skynet
[9]  --> Temporarily Disable Skynet
[10] --> Update Skynet
[11] --> Debug Options
[12] --> Stats
[13] --> Install Skynet / Change Boot Options
[14] --> Uninstall

[e]  --> Exit Menu

[1-13]: 3

Select Filter List:
[1] --> Default
[2] --> Custom

[1-2]: 1

Downloading filter.list         [0s]
Whitelisting Shared Domains     [2s]
Consolidating Blacklist         [14s]
Saving Changes                  [2s]
Removing Previous Malware Bans  [0s]
Filtering IPv4 Addresses        [6s]
Filtering IPv4 Ranges           [1s]
Applying Blacklists             [13s]

For False Positive Website Bans Use; ( sh /jffs/scripts/firewall whitelist domain URL )

Skynet: [Complete] 151306 IPs / 3345 Ranges Banned. 109098 New IPs / 3345 New Ranges Banned. 4 Inbound / 0 Outbound Connections Blocked! [47s]

Now-

Code:

wc -l /tmp/mnt/pdas001/skynet/scripts/ipset.txt
154711 /tmp/mnt/pdas001/skynet/scripts/ipset.txt

And-

Code:

{ ipset save Whitelist; ipset save Blacklist; ipset save BlockedRanges; ipset save Skynet; } | wc -l
154711

TheUntouchable · Nov 7, 2017

Just a request:
Is it possible to add a new option "blocked per device" to Stats->Search? I have seen that there is now a point under Stats for blocked devices, which would be a great addition to it

2992 · Nov 7, 2017

2992 said:
Could be because of XWRT, but I personally don't think so, as all the previous Skynet versions ran fine, and I haven't changed or modified anything since the first firmware install (few months ago).
I will re-install Skynet as you wrote, later today or tomorrow (limited free time here, factor in the wife...), so we can factor out the USB, but again, I personally think the USB is fine, as ABSolution is running perfectly fine from the same USB.

PS: is there any way I can install a previous version of Skynet?..

(PPS: I incline to believe something is not running fine on the Skynet version I have now here, factoring in the Dropbox connectivity issue I've wrote in my previous post. Maybe something is corrupted on it, or I do not know. I will re-instal it again, then let's see what gives.)

@Adamm, so, I've managed to install it on JFFS, and it works ok. Did couple of reboots and it was looking ok.
Then I've uninstalled it from JFFS, and installed it back on USBDrive. Rebooted once, and it was fine. Then rebooted again and again, and sometimes it shows "0" IPs, someother times it showing different values, i.e. 86472 IPs, then ran Banmalware and it shows 111462, then run again Banmalware and it (finally) shows 149746.

BUT, now I've got alot of these messages in the router's log:
kernel: EXT2-fs (sda1): error: ext2_free_blocks: bit already cleared for block 386070
Does that means the USB Drive is dying, or?... (EDIT: it didn't show up again since that one time.)
What if I will always keep Skynet on JFFS?

PS: I've noticed Skynet's version is v5.5.0 now.
.
.
.
and then at first hour fix:

Code:

21:00:01 Skynet: [Complete] 0 IPs / 0 Ranges Banned. -84087 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [1s]

skeal · Nov 7, 2017

2992 said:
@Adamm, so, I've managed to install it on JFFS, and it works ok. Did couple of reboots and it was looking ok.
Then I've uninstalled it from JFFS, and installed it back on USBDrive. Rebooted once, and it was fine. Then rebooted again and again, and sometimes it shows "0" IPs, someother times it showing different values, i.e. 86472 IPs, then ran Banmalware and it shows 111462, then run again Banmalware and it (finally) shows 149746.

BUT, now I've got alot of these messages in the router's log:
kernel: EXT2-fs (sda1): error: ext2_free_blocks: bit already cleared for block 386070
Does that means the USB Drive is dying, or?... (EDIT: it didn't show up again since that one time.)
What if I will always keep Skynet on JFFS?

PS: I've noticed Skynet's version is v5.5.0 now.
.
.
.
and then at first hour fix:

Code:

21:00:01 Skynet: [Complete] 0 IPs / 0 Ranges Banned. -84087 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [1s]

Do an error check and try it again. You can scan at boot with one of the scripts on github. Realise that when the system reboots it doesn't have a way to unmount the drives gracefully. If you have a swap on your drive you are even more at risk of a corruption from a bad shut down or a write function running when shut down takes place. It just shuts down anything can happen. Also i have noticed that ext4 file system is the most durable file system.

Adamm · Nov 7, 2017

.TT. said:
All good again after running:

Sweet!

.TT. said:
Not sure if it is possible but i would love to see all IPs detected/blocked by AI-protection (two-way ips) included in auto ban as well. Any way Skynet can read this data?

The AI-Protect code is all closed source and the data isn't available unfortunately.

TheUntouchable said:
Is it possible to add a new option "blocked per device" to Stats->Search? I have seen that there is now a point under Stats for blocked devices, which would be a great addition to it

That was the plan, will implement it in the near future.

2992 said:
BUT, now I've got alot of these messages in the router's log:
kernel: EXT2-fs (sda1): error: ext2_free_blocks: bit already cleared for block 386070
Does that means the USB Drive is dying, or?..

With the situation you described, the USB is definitely dying. Time to splurge on a new one.

Protik said:
So after a manual reboot, the Skynet log entries are,

Kinda sounds like this also may be a USB related issue. Try reinstall to JFFS and see if you have the same issues upon reboots. If you don't might be time to also get a new USB/

2992 · Nov 8, 2017

Adamm said:
With the situation you described, the USB is definitely dying. Time to splurge on a new one.

Kinda sounds like this also may be a USB related issue. Try reinstall to JFFS and see if you have the same issues upon reboots. If you don't might be time to also get a new USB/

I will try another USB stick, meanwhile, I'll re-install it on JFFS.

Anyways, I've tried again this scenario: I've set the router to reboot at 5:00, and here are the messages at around 5 and 6:

Code:

Nov  8 05:01:59 Skynet: [INFO] Startup Initiated... ( banmalware autoupdate usb=/tmp/mnt/abs )
*some other messages here*
Nov  8 05:01:59 Skynet: [INFO] Setting Up Skynet...
*some other messages here*
Nov  8 05:02:20 Skynet: [Complete] 150481 IPs / 3368 Ranges Banned. 150481 New IPs / 3368 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [22s]

That is looking ok, but after one hour...

Code:

Nov  8 06:00:01 Skynet: [Complete] 0 IPs / 0 Ranges Banned. -150481 New IPs / -3368 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [1s]

Adamm · Nov 8, 2017

2992 said:
Anyways, I've tried again this scenario: I've set the router to reboot at 5:00, and here are the messages at around 5 and 6:

The "Setting Up Skynet" message was something I specifically added for your issue. Whats basically happening is, due to the issues on the USB, the system isn't properly detecting the file so its setting up the IPSets as new. The message at "05:02:20" is actually inaccurate, the counter at this point should be 0 but there was a minor coding error which didn't update this figure, but this is corrected the following hour as you can see.

TL;DR your USB is dying.

Adamm · Nov 8, 2017

TheUntouchable said:
Just a request:
Is it possible to add a new option "blocked per device" to Stats->Search? I have seen that there is now a point under Stats for blocked devices, which would be a great addition to it

This has now been added in v5.5.1

To use it;

Code:

sh /jffs/scripts/firewall stats search device xxx.xxx.xxx.xxx

.TT. · Nov 8, 2017

Adamm said:
Sweet!

The AI-Protect code is all closed source and the data isn't available unfortunately.

ok, i had a feeling this would be the case.
Thanks =)

2992 · Nov 8, 2017

Adamm said:
The "Setting Up Skynet" message was something I specifically added for your issue. Whats basically happening is, due to the issues on the USB, the system isn't properly detecting the file so its setting up the IPSets as new. The message at "05:02:20" is actually inaccurate, the counter at this point should be 0 but there was a minor coding error which didn't update this figure, but this is corrected the following hour as you can see.

TL;DR your USB is dying.

It could be, but I have just re-installed everything on a different/new USBDrive, and:

Code:

Nov  8 13:36:07 Skynet: [INFO] Setting Up Skynet...
Nov  8 13:36:27 Skynet: [Complete] 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [21s]
Nov  8 14:00:01 Skynet: [Complete] 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [1s]
Nov  8 15:00:01 Skynet: [Complete] 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [1s]
Nov  8 16:00:01 Skynet: [Complete] 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [1s]
Nov  8 17:00:01 Skynet: [Complete] 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [1s]
Nov  8 18:00:01 Skynet: [Complete] 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [1s]
Nov  8 19:00:02 Skynet: [Complete] 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [0s]

Could it be that this USBDrive is also not ok?...
I've later performed a reboot, and:

Code:

Nov  8 19:41:01 Skynet: [Complete] 159778 IPs / 3371 Ranges Banned. 159778 New IPs / 3371 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [24s]
Nov  8 20:00:01 Skynet: [Complete] 0 IPs / 0 Ranges Banned. -159778 New IPs / -3371 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [1s]

thelonelycoder · Nov 8, 2017

2992 said:
Could it be that this USBDrive is also not ok?...

You would see a lot of errors in the Syslog for your USB device, especially since you have AB-Solution installed and logging enabled.

2992 · Nov 8, 2017

thelonelycoder said:
You would see a lot of errors in the Syslog for your USB device, especially since you have AB-Solution installed and logging enabled.

I can see/identify no errors, neither on the previous USBDrive (except once) nor on the new USBDrive. :|
And ABS works perfectly fine.

Protik · Nov 8, 2017

2992 said:
I can see/identify no errors, neither on the previous USBDrive (except once) nor on the new USBDrive. :|
And ABS works perfectly fine.

Same here. Ab-solution works fine in the same USB where Skynet is having issues. But one thing is, it seems my USB have a FAT16 partition somewhere. The fdisk shows,

Code:

admin@RT-AC68U-DF28:/tmp/home/root# fdisk -l /dev/sda

Disk /dev/sda: 2055 MB, 2055208960 bytes
242 heads, 63 sectors/track, 263 cylinders
Units = cylinders of 15246 * 512 = 7805952 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sda1               1         256     1945584   6 FAT16
Partition 1 has different physical/logical beginnings (non-Linux?):
     phys=(0, 0, 32) logical=(0, 0, 33)
Partition 1 has different physical/logical endings:
     phys=(768, 241, 63) logical=(255, 55, 5)

Can you check if that's the case for you too?

thelonelycoder · Nov 8, 2017

2992 said:
I can see/identify no errors, neither on the previous USBDrive (except once) nor on the new USBDrive. :|
And ABS works perfectly fine.

Then it seems it's related to a single or multiple corrupt files.
Since both Skynet and AB write frequently to USB devices, a sudden loss of power might cause corrupted files.
There's not much one can do about it but find and fix it.

Thread starter	Title	Forum	Replies	Date
S	Skynet Skynet showing router itself making calls to China?	Asuswrt-Merlin AddOns	26	Monday at 8:33 PM
J	Skynet Skynet - Blocked outbounds coming from the router itself?	Asuswrt-Merlin AddOns	12	May 9, 2024
O	Skynet Installing Skynet causes router to crash and reboot	Asuswrt-Merlin AddOns	26	Feb 17, 2024
	Diversion Diversion & Skynet Missing on GUI after Router Reboot.	Asuswrt-Merlin AddOns	7	Aug 21, 2023
C	Skynet Statistics Monitoring From - N/A Log Size - N/A	Asuswrt-Merlin AddOns	1	Friday at 12:51 AM
R	Skynet How to see which domains get blocked by Skynet?	Asuswrt-Merlin AddOns	3	May 3, 2024
	Blocking “RO” country code in Skynet blocks Proton VPN UK connections	Asuswrt-Merlin AddOns	2	Apr 28, 2024
P	Any usefulness for Skynet + pfsense/opnsense in transparent bridge mode?	Asuswrt-Merlin AddOns	12	Apr 22, 2024
	Solved Skynet running slow with install/update issues from AMTM?	Asuswrt-Merlin AddOns	10	Mar 21, 2024
B	Skynet Source LAN IP for outbound IP blocked by SkyNet	Asuswrt-Merlin AddOns	8	Feb 26, 2024

Skynet Skynet - Router Firewall & Security Enhancements

Part of the Furniture

Regular Contributor

Regular Contributor

Part of the Furniture

Regular Contributor

Regular Contributor

Senior Member

Regular Contributor

Regular Contributor

Part of the Furniture

Part of the Furniture

Regular Contributor

Part of the Furniture

Part of the Furniture

Regular Contributor

Regular Contributor

Part of the Furniture

Regular Contributor

Senior Member

Part of the Furniture

Similar threads

Similar threads

Sign Up For SNBForums Daily Digest