Skynet Skynet - Router Firewall & Security Enhancements

[RMerlin]

Thanks. Also for reference I was implying that for whatever reason even though HND's BusyBox is older, the date binary seems to have more functionality (it was stripped in 1.25.0 for whatever reason with no documentation). Regardless a unified version would be great.

Could be a different configuration. Busybox allows you to enable/disable certain features or command switches at build time, for people trying to better streamline their bb size. I'll take a look to see if maybe HND's config_base might be different.

 

[RMerlin]

Hm, both instances are compiled with the same options:

Could it be caused rather by the difference between uclibc (ARM) and glibc (HND)?

I can't see any change specific to the date command in the 1.25 and 1.25.1 changelog either.

 

[Adamm]

Could it be caused rather by the difference between uclibc (ARM) and glibc (HND)?

I would assume its to-do with busybox as its documented date functionality that's missing from the ARM build.

http://man7.org/linux/man-pages/man1/date.1.html

       By default, date pads numeric fields with zeroes.  The following
       optional flags may follow '%':

       -      (hyphen) do not pad the field

       _      (underscore) pad with spaces

       0      (zero) pad with zeros

       ^      use upper case if possible

       #      use opposite case if possible

Although, I didn't exactly see any commits in their git repo from 24 October 2015 - 7 October 2016 (release dates for 1.24.1 / 1.25.1) that would indicate any functionality change.

 

[RMerlin]

I would assume its to-do with busybox as its documented date functionality that's missing from the ARM build.

http://man7.org/linux/man-pages/man1/date.1.html

       By default, date pads numeric fields with zeroes.  The following
       optional flags may follow '%':

       -      (hyphen) do not pad the field

       _      (underscore) pad with spaces

       0      (zero) pad with zeros

       ^      use upper case if possible

       #      use opposite case if possible

Although, I didn't exactly see any commits in their git repo from 24 October 2015 - 7 October 2016 (release dates for 1.24.1 / 1.25.1) that would indicate any functionality change.

It depends on how they implement the formatting. If they rely on a libc function, then it could be down to a difference between uclibc and glibc, in which case upgrading busybox won't change anything.

 

[Butterfly Bones]

This is interesting. Very aggressive auto bans today. This is my entire month of autobans - 10 of 18 today alone.

Feb  4 14:08:07 kernel: [BLOCKED - NEW BAN]
Feb  4 16:14:43 kernel: [BLOCKED - NEW BAN]
Feb  4 16:14:43 kernel: [BLOCKED - NEW BAN]
Feb  4 16:15:17 kernel: [BLOCKED - NEW BAN]
Feb 10 06:49:16 kernel: [BLOCKED - NEW BAN]
Feb 11 08:35:44 kernel: [BLOCKED - NEW BAN]
Feb 13 21:09:11 kernel: [BLOCKED - NEW BAN]
Feb 20 09:44:20 kernel: [BLOCKED - NEW BAN]
Feb 24 09:42:05 kernel: [BLOCKED - NEW BAN]
Feb 28 12:42:19 kernel: [BLOCKED - NEW BAN]
Feb 28 12:47:42 kernel: [BLOCKED - NEW BAN]
Feb 28 12:50:07 kernel: [BLOCKED - NEW BAN]
Feb 28 12:50:45 kernel: [BLOCKED - NEW BAN]
Feb 28 12:52:02 kernel: [BLOCKED - NEW BAN]
Feb 28 12:52:40 kernel: [BLOCKED - NEW BAN]
Feb 28 13:08:19 kernel: [BLOCKED - NEW BAN]
Feb 28 13:26:23 kernel: [BLOCKED - NEW BAN]
Feb 28 16:30:30 kernel: [BLOCKED - NEW BAN]
Feb 28 16:39:31 kernel: [BLOCKED - NEW BAN]

This is the change of Skynet updates this morning.

Feb 28 02:25:29 Skynet: [Complete] 81383 IPs / 1797 Ranges Banned. -44653 New IPs / -11 New Ranges Banned.

I guess since so many (44000+) were cleared this morning, that now they start getting banned again. This is not a problem, just an observation. The daily changes of adding and subtracting IPs on a daily basis fascinates me.

edit - After more investigation using the Stats > Search, all 10 of todays autobans are from China, three different providers using OTX info.

 

[Adamm]

I guess since so many (44000+) were cleared this morning, that now they start getting banned again. This is not a problem, just an observation. The daily changes of adding and subtracting IPs on a daily basis fascinates me.

Yeah I guess one of the lists had a significant cleanup which is a good thing.

edit - After more investigation using the Stats > Search, all 10 of todays autobans are from China, three different providers using OTX info.

Not a huge surprise, asian and middle eastern countries seem to be a significant portion attack sources. But considering the countries population it's not so hard to believe.

 

[XIII]

DNS was not working at home this morning. Turned out SkyNet was blocking my DNS solver... (Quad9; 9.9.9.9)

I could whitelist it on two routers (and DNS started working again), but on the third router I could not whitelist it, because it was already present (but does not show up if I list manually whitelisted IP's):

ipset v6.32: Element cannot be added to the set: it's already added

Yet, all 3 routers use the same configuration...

 

[Adamm]

DNS was not working at home this morning. Turned out SkyNet was blocking my DNS solver... (Quad9; 9.9.9.9)

I could whitelist it on two routers (and DNS started working again), but on the third router I could not whitelist it, because it was already present (but does not show up if I list manually whitelisted IP's):

ipset v6.32: Element cannot be added to the set: it's already added

Yet, all 3 routers use the same configuration...

Any DNS entries should be automatically whitelisted. I also don't see why 9.9.9.9 would have been blocked as its not present on any malware list.

 

[XIII]

Any DNS entries should be automatically whitelisted. I also don't see why 9.9.9.9 would have been blocked as its not present on any malware list.

I don't understand either. DNS was not working though. Worked when I temporarily disabled SkyNet. Did not work again when I re-enabled SkyNet. Worked after I whitelisted 9.9.9.9.

Unfortunately I did not have time to investigate this further.

Is there a way to force add something (IP/domain) to the whitelist? (for the third router)

 

[Adamm]

I don't understand either. DNS was not working though. Worked when I temporarily disabled SkyNet. Did not work again when I re-enabled SkyNet. Worked after I whitelisted 9.9.9.9.

Unfortunately I did not have time to investigate this further.

Next time use the following command to see if/why the IP is actually banned;

sh /jffs/scripts/firewall stats search ip xxx.xxx.xxx.xxx

Is there a way to force add something (IP/domain) to the whitelist? (for the third router)

I assume you are getting the following error, because as the error suggests the entry is already added (probably automatically if its DNS)

ipset v6.32: Element cannot be added to the set: it's already added

 

[DonnyJohnny]

Next time use the following command to see if/why the IP is actually banned;

sh /jffs/scripts/firewall stats search ip xxx.xxx.xxx.xxx

I assume you are getting the following error, because as the error suggests the entry is already added (probably automatically if its DNS)

ipset v6.32: Element cannot be added to the set: it's already added

There was a time where 8.8.8.8 and 9.9.9.9 is blocked. I have since whitelist them.

 

[vibroverbus]

Hey Adam, little thing but, do you think you might consider changing your firewall-start install/update lines from a 'sed d' followed by echo >> firewall-start, to a single sed s or sed c to just replace the line in situ, in a more tidy fashion? It'd just a little nicer to my other entries in the script, and means any comments I put in there stay in the right places and order vs your entry... (same for firewall-stop for that matter).

(Of course you're already vastly better than the Entware3 install routine which just nukes the services-start script assuming you couldn't possible give a crap about what might be in there now... That was fun to discover... )

 

[Ubimo]

Why is that? So many minus IPs?

Before update:
Mar 3 01:00:04 Skynet: [Complete] 83235 IPs / 1726 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 7960 Inbound / 3837 Outbound Connections Blocked! [4s]
After update:
Mar 3 02:00:01 admin: AB-Solution added entries via ab_dnsmasq_postconf.sh
Mar 3 02:00:01 admin: AB-Solution linked ab_dnsmasq_postconf.sh via /jffs/scripts/dnsmasq.postconf
Mar 3 02:00:03 Skynet: [INFO] Lock File Detected (save) (pid=24477) - Exiting
Mar 3 02:00:06 Skynet: [Complete] 83235 IPs / 1726 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 8223 Inbound / 3837 Outbound Connections Blocked! [6s]
Mar 3 02:00:46 admin: AB-Solution: file 3 download failed or file is not hosts file, using backup file
Mar 3 02:04:32 admin: AB-Solution updated blocking file, 771248 domains are now blocked
Mar 3 02:04:35 admin: AB-Solution counted ads before log files reset (triggered by update-hosts.add)
Mar 3 02:04:35 admin: AB-Solution blocked 57,148 total 0 week 0 new ads
Mar 3 02:04:35 admin: AB-Solution rotated dnsmasq log files
Mar 3 02:25:49 Skynet: [Complete] 202 IPs / 13 Ranges Banned. -83033 New IPs / -1713 New Ranges Banned. 8347 Inbound / 3837 Outbound Connections Blocked! [49s]
Mar 3 03:00:01 Skynet: [Complete] 202 IPs / 13 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 8347 Inbound / 3837 Outbound Connections Blocked! [1s]

Edit:
After updating ab-solution and skynet its back to normal:
Mar 3 13:26:25 Skynet: [Complete] 87837 IPs / 1789 Ranges Banned. 87635 New IPs / 1776 New Ranges Banned. 8354 Inbound / 5048 Outbound Connections Blocked! [39s]

 

[Adamm]

Why is that? So many minus IPs?

Before update:
Mar 3 01:00:04 Skynet: [Complete] 83235 IPs / 1726 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 7960 Inbound / 3837 Outbound Connections Blocked! [4s]
After update:
Mar 3 02:00:01 admin: AB-Solution added entries via ab_dnsmasq_postconf.sh
Mar 3 02:00:01 admin: AB-Solution linked ab_dnsmasq_postconf.sh via /jffs/scripts/dnsmasq.postconf
Mar 3 02:00:03 Skynet: [INFO] Lock File Detected (save) (pid=24477) - Exiting
Mar 3 02:00:06 Skynet: [Complete] 83235 IPs / 1726 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 8223 Inbound / 3837 Outbound Connections Blocked! [6s]
Mar 3 02:00:46 admin: AB-Solution: file 3 download failed or file is not hosts file, using backup file
Mar 3 02:04:32 admin: AB-Solution updated blocking file, 771248 domains are now blocked
Mar 3 02:04:35 admin: AB-Solution counted ads before log files reset (triggered by update-hosts.add)
Mar 3 02:04:35 admin: AB-Solution blocked 57,148 total 0 week 0 new ads
Mar 3 02:04:35 admin: AB-Solution rotated dnsmasq log files
Mar 3 02:25:49 Skynet: [Complete] 202 IPs / 13 Ranges Banned. -83033 New IPs / -1713 New Ranges Banned. 8347 Inbound / 3837 Outbound Connections Blocked! [49s]
Mar 3 03:00:01 Skynet: [Complete] 202 IPs / 13 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 8347 Inbound / 3837 Outbound Connections Blocked! [1s]

Edit:
After updating ab-solution and skynet its back to normal:
Mar 3 13:26:25 Skynet: [Complete] 87837 IPs / 1789 Ranges Banned. 87635 New IPs / 1776 New Ranges Banned. 8354 Inbound / 5048 Outbound Connections Blocked! [39s]

Hard to say exactly, but I assume the website where most of the lists are hosted had some brief downtime so they failed to download during the usual update period.

 

[Adamm]

Hey Adam, little thing but, do you think you might consider changing your firewall-start install/update lines from a 'sed d' followed by echo >> firewall-start, to a single sed s or sed c to just replace the line in situ, in a more tidy fashion? It'd just a little nicer to my other entries in the script, and means any comments I put in there stay in the right places and order vs your entry... (same for firewall-stop for that matter).

(Of course you're already vastly better than the Entware3 install routine which just nukes the services-start script assuming you couldn't possible give a crap about what might be in there now... That was fun to discover... )

Sounds reasonable. What I can probably do is scan for existing Skynet entries in files like firewall-start, if one exists use the replacement command, if nothing exists append it to the end of the file. Also I agree that entware's installation method is excessively aggressive and silly how it nukes every startup script, very frustrating.

Edit; This change is now live in v5.8.5 along with some minor fixes from throughout the week.

 

[vibroverbus]

Groovy. Yer the man.

 

[JacquesR]

Skynet 5.8.5 seems to be working fine (and thank you for this fine tool, Adamm!), but I've got a persistent issue I'm hoping someone can advise on. Just below the count of IP's and Ranges banned, I see

grep: /tmp/syslog.log: No such file or directory
[: bad number

Manually creating the syslog.log in /tmp/ resolves this, but invariably the message reappears, and my syslog.log file (USB installation) has vanished. What is deleting it, and how can I stop this occurring?

 

[vtol]

Trying to import the TALOS bl http://talosintel.com/feeds/ip-filter.blf which does not seems to work though.

Skynet: [ERROR] No Content Detected - Stopping Import

Perhaps because the URL is redirecting?

 

[Adamm]

Trying to import the TALOS bl http://talosintel.com/feeds/ip-filter.blf which does not seems to work though.

Skynet: [ERROR] No Content Detected - Stopping Import

Perhaps because the URL is redirecting?

Yeah thats it, curl wasn't following redirects, I put out a hotfix so curl will always follow them in every function (you will need to force update as there was no version change).

fyi; Banmalware already has this list included as part of firehol_level3 if you use that feature.

Skynet 5.8.5 seems to be working fine (and thank you for this fine tool, Adamm!), but I've got a persistent issue I'm hoping someone can advise on. Just below the count of IP's and Ranges banned, I see

grep: /tmp/syslog.log: No such file or directory
[: bad number

Manually creating the syslog.log in /tmp/ resolves this, but invariably the message reappears, and my syslog.log file (USB installation) has vanished. What is deleting it, and how can I stop this occurring?

Skynet relies on syslog being present in /tmp for almost all logging related functions. Is there any reason you use a non default location & how did you specify it.

 

[JacquesR]

Skynet relies on syslog being present in /tmp for almost all logging related functions. Is there any reason you use a non default location & how did you specify it.

No, I simply ran the install code from post #1 in SSH, and chose the default menu options. I've just checked the folder structure on the router, and Skynet is definitely in /jffs/scripts. This was also a fresh install, as I only got the router (86U) a couple of months back, and did a fresh install rather than try to copy anything from my 68U. The USB drive is the same one as used in the 68U, though - might that be relevant?