Skynet Skynet - Router Firewall & Security Enhancements

[M@rco]

It should have, the backup command preserves the original directory structure in a tar file.

I'm not sure, but I think I've rebooted my router in the meantime, because I installed dnscrypt-proxy afterwards. Maybe the old directory was wiped, because it's not an actual mountpoint anymore?

As for manually copying from the tar archive, that did the trick, thanks!

Feb 19 16:56:48 Skynet: [INFO] Startup Initiated... ( debug banmalware autoupdate usb=/tmp/mnt/KINGSTON )
Feb 19 16:57:10 Skynet: [Complete] 90338 IPs / 1844 Ranges Banned. 90338 New IPs / 1844 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [21s]
Feb 19 17:00:05 Skynet: [Complete] 90338 IPs / 1844 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 1 Inbound / 0 Outbound Connections Blocked! [4s]

 

[M@rco]

I think I stumbled upon a minor bug. I wanted to follow debug output to whitelist an IP, but accidently selected the wrong option. To be able to reproduce it, I've posted the steps below:

Select Menu Option:
[1]  --> Unban
[2]  --> Ban
[3]  --> Banmalware
[4]  --> Whitelist
[5]  --> Import IP List
[6]  --> Deport IP List
[7]  --> Save
[8]  --> Restart Skynet
[9]  --> Temporarily Disable Skynet
[10] --> Update Skynet
[11] --> Debug Options
[12] --> Stats
[13] --> Install Skynet / Change Boot Options
[14] --> Uninstall

[r]  --> Reload Menu
[e]  --> Exit Menu

I selected 11, Debug options:

[1-14]: 11

Select Debug Option:
[1]  --> Temporarily Disable Debug Output
[2]  --> Show Debug Entries As They Appear
[3]  --> Print Debug Info
[4]  --> Cleanup Syslog Entries
[5]  --> SWAP File Management
[6]  --> Backup Skynet Files
[7]  --> Restore Skynet Files

I selected 2, Show Debug Entries:

[1-7]: 2

Select Watch Option:
[1]  --> All
[2]  --> IP
[3]  --> Port

I accidentally selected option 2 instead of 1

[1-3]: 2

and left the IP blank and hit Enter to return to the menu:

[IP]:

 Is Not A Valid IP

Select Watch Option:
[1]  --> All
[2]  --> IP
[3]  --> Port

and then selected the correct option, 1:

[1-3]: 1

Watching Logs For Debug Entries (ctrl +c) To Stop

 Is Not A Valid IP

marco@RT-AC68U:/tmp/home/root#

It looks like it starts printing debug entries, but then the previous error shows up again en immediately after that Skynet exits...

I hope this is helpful to track it down.

 

[Adamm]

I hope this is helpful to track it down.

Thanks, I've fixed this in v5.8.1.

The issue was actually present in about 50 different locations during validity checks as I shortsightedly assumed users would always retry the same command so never unset the vars, only if they used the exit/back commands (which I found a bug with too that was also corrected).

 

[Hello World]

Here is a list of Xbox links that should be white-listed:

attestation.xboxlive.com
cert.mgt.xboxlive.com
ctldl.windowsupdate.com def-vef.xboxlive.com
device.auth.xboxlive.com
eds.xboxlive.com
help.ui.xboxlive.com
licensing.xboxlive.com
notify.xboxlive.com
title.auth.xboxlive.com
title.mgt.xboxlive.com
www.msftncsi.com
www.xboxlive.com
xbox.ipv6.microsoft.com
xboxexperiencesprod.experimentation.xboxlive.com
xflight.xboxlive.com
xkms.xbolive.com
xsts.auth.xboxlive.com

Can we whitelist URL's on Skynet (vice IP's)? Something on Skynet is blocking my X-Box One. I tried using debug to monitor blocked IP's but nothing showed up.

 

[Hello World]

I think this is one of the most useful commands of Skynet about troubleshooting so maybe it would be better to add this line into the "Usage" part of the #1 entry of this topic. Somewhere that everyone can see it easily

Agreed. Or better yet, add it to the options page. I'm not a Linux guy and always have to revert back to this page to find out what the command is. Thanks.

 

[M@rco]

Can we whitelist URL's on Skynet (vice IP's)? Something on Skynet is blocking my X-Box One. I tried using debug to monitor blocked IP's but nothing showed up.

Sure you can. Launch Skynet by typing firewall, select 4 to whitelist and take your pick:

Select Whitelist Option:
[1]  --> IP/Range
[2]  --> Domain
[3]  --> Port
[4]  --> Refresh VPN Whitelist
[5]  --> Remove Entries
[6]  --> Refresh Entries
[7]  --> List Entries

But if you tried following the debug entries and nothing shows up, I doubt whether it's Skynet that's blocking traffic from or to your XBox One.

 

[Mutzli]

Can we whitelist URL's on Skynet (vice IP's)? Something on Skynet is blocking my X-Box One. I tried using debug to monitor blocked IP's but nothing showed up.

Using IP addresses to white-list Microsoft Xbox live servers might not last long. They are known to switch server IP addresses frequently.

 

[vibroverbus]

Apologies in advance if this is (probably will be) because of some dumb mistake on my part...

Tried to install Skynet today for first time, "Vanilla" option, installed to USB, created SWAP.... seemed to complete 100% fine, but then I just get all red 'FAILED' when starting up the interface. Is there some other setting dependency or prep I have to do? Do I have something turned off or on that has to be changed? The doc for stuff like that seems pretty slim and the install seems to go so slickly I assumed it was all set...

Here's debug output:

Router Model; RT-AC3200
Skynet Version; v5.8.1 (20/02/2018)
iptables v1.4.15 - (eth0 @ num.num.num.num)
ipset v6.32, protocol version: 6
FW Version; 384.3_0 (Feb 13 2018) (2.6.36.4brcmarm)
Install Dir; /tmp/mnt/sda1/skynet (103.7G / 110.0G Space Available)
SWAP File; /tmp/mnt/sda1/myswap.swp (512.0M)
Boot Args; /jffs/scripts/firewall start banmalware autoupdate usb=/tmp/mnt/sda1
grep: /tmp/mnt/sda1/skynet/scripts/ipset.txt: No such file or directory
No Lock File Found

Checking Install Directory Write Permissions... [Passed]
Checking Firewall-Start Entry... [Passed]
Checking Services-Stop Entry... [Passed]
Checking CronJobs... [Failed]
Checking IPSet Comment Support... [Passed]
Checking Log Level 1 Settings... [Passed]
Checking Autobanning Status... [Failed]
Checking Debug Mode Status... [Disabled]
Checking For Duplicate Rules In RAW... [Passed]
Checking For Duplicate Rules In Filter... [Passed]
Checking Skynet IPTable... [Failed]
Checking Whitelist IPSet... [Failed]
Checking BlockedRanges IPSet... [Failed]
Checking Blacklist IPSet... [Failed]
Checking Skynet IPSet... [Failed]

Any help much appreesh.

 

[Goobi]

@Adamm you had replied to a query I had in the AB-Solution thread asking about the excessive logging to the syslog. Have you found anything out about potentially reconfiguring the syslog file in such a way that these drops are logged to another file? Right now those drops are logged at such a rapid rate that after a short period of time the log wraps and has nothing but drops making it difficult to view the syslog for other messages of interest. Thanks in advance.

 

[DonnyJohnny]

Apologies in advance if this is (probably will be) because of some dumb mistake on my part...

Tried to install Skynet today for first time, "Vanilla" option, installed to USB, created SWAP.... seemed to complete 100% fine, but then I just get all red 'FAILED' when starting up the interface. Is there some other setting dependency or prep I have to do? Do I have something turned off or on that has to be changed? The doc for stuff like that seems pretty slim and the install seems to go so slickly I assumed it was all set...

Here's debug output:



Any help much appreesh.

After you installed, Wait 2 min before you restart Skynet as they are still processing the Banmalware list....

 

[Adamm]

Can we whitelist URL's on Skynet (vice IP's)? Something on Skynet is blocking my X-Box One. I tried using debug to monitor blocked IP's but nothing showed up.

I manually resolved all the IP's from the list provided and found one conflicting IP which has since been removed from the telemetry filter (134.170.179.87). Upon updating banmalware on your end this should be unbanned, let me know if you are still having further issues.

 

[Adamm]

Apologies in advance if this is (probably will be) because of some dumb mistake on my part...

Tried to install Skynet today for first time, "Vanilla" option, installed to USB, created SWAP.... seemed to complete 100% fine, but then I just get all red 'FAILED' when starting up the interface. Is there some other setting dependency or prep I have to do? Do I have something turned off or on that has to be changed? The doc for stuff like that seems pretty slim and the install seems to go so slickly I assumed it was all set...

Here's debug output:

Strange, the firewall-start entry was created, so upon installing Skynet should have started up accordingly. Try use the following command;

sh /jffs/scripts/firewall restart

If that doesn't work, look for any Skynet related errors in your syslog. The only thing that really springs to mind is it could possibly be a mounting issue with your USB as it doesn't have a label, I suggest adding one.

 

[Adamm]

@Adamm you had replied to a query I had in the AB-Solution thread asking about the excessive logging to the syslog. Have you found anything out about potentially reconfiguring the syslog file in such a way that these drops are logged to another file? Right now those drops are logged at such a rapid rate that after a short period of time the log wraps and has nothing but drops making it difficult to view the syslog for other messages of interest. Thanks in advance.

The router uses the trimmed busybox syslogd, with that being said it does support using a specified syslog.conf but it currently is configured not to by the firmware. Killing this process and restarting it using the desired flag I don't see as a great option, but possibly @RMerlin could look into enabling this option and then adding custom conf.add and .postconf support like he does with other services.

Beyond to purge the syslog so you can look for unrelated entries its as simple as running any Skynet command or opening the main menu.

 

[vibroverbus]

Strange, the firewall-start entry was created, so upon installing Skynet should have started up accordingly. Try use the following command;

sh /jffs/scripts/firewall restart

If that doesn't work, look for any Skynet related errors in your syslog. The only thing that really springs to mind is it could possibly be a mounting issue with your USB as it doesn't have a label, I suggest adding one.

Thanks Adamm. I've tried the restart, both CLI and via menu. All I get is this...

Feb 20 08:37:38 rc_service: service 7830:notify_rc restart_firewall
Feb 20 08:37:39 miniupnpd[6544]: shutting down MiniUPnPd
Feb 20 08:37:39 nat: apply nat rules (/tmp/nat_rules_eth0_eth0)
Feb 20 08:37:39 custom_script: Running /jffs/scripts/nat-start
Feb 20 08:37:39 custom_script: Running /jffs/scripts/firewall-start (args: eth0)
Feb 20 08:37:39 miniupnpd[7859]: HTTP listening on port 41263
Feb 20 08:37:39 miniupnpd[7859]: Listening for NAT-PMP/PCP traffic on port 5351

Tried re-install too, no love. Will try to give my USB a label... I've just used dev names so far...

The other thing that I've just noticed is that no cron jobs get set up for updates or anything. Files all look like they're being created on the USB fine. I'm going to pull the USB drive and check it w/ GPARTED and see if something looks funny there but if the cron is a telltale let me know.

(oh and one bug thing... minor fault-tolerance menuing issue, if I try to run a menu choice that calls for a filename - Import etc. - it will correctly not prompt me for the filename because it recognizes that Skynet is not running, but then when I enter any other command after that, suddenly I get a prompt for a filename... sounds similar to above where the script gets stuck thinking the user is still in the previous command...)

 

[skeal]

Thanks Adamm. I've tried the restart, both CLI and via menu. All I get is this...



Tried re-install too, no love. Will try to give my USB a label... I've just used dev names so far...

The other thing that I've just noticed is that no cron jobs get set up for updates or anything. Files all look like they're being created on the USB fine. I'm going to pull the USB drive and check it w/ GPARTED and see if something looks funny there but if the cron is a telltale let me know.

(oh and one bug thing... minor fault-tolerance menuing issue, if I try to run a menu choice that calls for a filename - Import etc. - it will correctly not prompt me for the filename because it recognizes that Skynet is not running, but then when I enter any other command after that, suddenly I get a prompt for a filename... sounds similar to above where the script gets stuck thinking the user is still in the previous command...)

I can verify this cron job missing after updating two routers from 5.7.9 to 5.8.1 the firewall restarts but when you check "cru l" the update cron is not there as well as the update software cron. If you restart the router the cron jobs are successfully created and the program functions normally. Confirmed on ac68u and ac3100.

 

[Adamm]

(oh and one bug thing... minor fault-tolerance menuing issue, if I try to run a menu choice that calls for a filename - Import etc. - it will correctly not prompt me for the filename because it recognizes that Skynet is not running, but then when I enter any other command after that, suddenly I get a prompt for a filename... sounds similar to above where the script gets stuck thinking the user is still in the previous command...)

Thanks, I fixed this in 5.8.2. I forgot to break off old while loops if Check_Status failed, so after issuing the new command Skynet would go back and continue where it should have ended on the previous failed command. Took me a minute to get my head around what the hell was going on

Tried re-install too, no love. Will try to give my USB a label... I've just used dev names so far...

The other thing that I've just noticed is that no cron jobs get set up for updates or anything. Files all look like they're being created on the USB fine. I'm going to pull the USB drive and check it w/ GPARTED and see if something looks funny there but if the cron is a telltale let me know.

Basically nothing will happen until the Check_Settings function is complete, it seems your setup is failing during startup somewhere. Try start Skynet manually and see if any errors occur;

sh /jffs/scripts/firewall start banmalware autoupdate usb=/tmp/mnt/sda1

 

[Adamm]

I can verify this cron job missing after updating two routers from 5.7.9 to 5.8.1 the firewall restarts but when you check "cru l" the update cron is not there as well as the update software cron. If you restart the router the cron jobs are successfully created and the program functions normally. Confirmed on ac68u and ac3100.

I was not able to replicate this;

Select Menu Option:
[1]  --> Unban
[2]  --> Ban
[3]  --> Banmalware
[4]  --> Whitelist
[5]  --> Import IP List
[6]  --> Deport IP List
[7]  --> Save
[8]  --> Restart Skynet
[9]  --> Temporarily Disable Skynet
[10] --> Update Skynet
[11] --> Debug Options
[12] --> Stats
[13] --> Install Skynet / Change Boot Options
[14] --> Uninstall

[r]  --> Reload Menu
[e]  --> Exit Menu

[1-14]: 10

Select Update Option:
[1]  --> Check For And Install Any New Updates
[2]  --> Check For Updates Only
[3]  --> Force Update Even If No Updates Detected

[1-3]: 1

Skynet: [INFO] New Version Detected - Updating To v5.8.2...
Skynet: [INFO] Skynet Sucessfully Updated - Restarting Firewall

Done.
admin@RT-AC86U-2EE8:/tmp/home/root# cru l
00 2 * * Thu /tmp/mnt/Main/adblocking/addon/update-hosts.add cronjob #AB_UpdateHosts#
20 5 * * * /tmp/mnt/Main/adblocking/addon/rotate-logs.add #AB_RotateLogs#
25 2 * * * sh /jffs/scripts/firewall banmalware #Skynet_banmalware#
25 1 * * Mon sh /jffs/scripts/firewall update check #Skynet_checkupdate#
0 * * * * sh /jffs/scripts/firewall save #Skynet_save#

 

[vibroverbus]

Basically nothing will happen until the Check_Settings function is complete, it seems your setup is failing during startup somewhere. Try start Skynet manually and see if any errors occur;

sh /jffs/scripts/firewall start banmalware autoupdate usb=/tmp/mnt/sda1

So I cleaned up my USB drive a bit (tho in hindsight don't think that was a problem). Found some errors with system script files where they were exiting before getting to the added skynet lines, due to a unique scripting thing i have going on... Thought I caught all those but it still wouldn't auto start. Did a reinstall and reset of everything (required a bit of hand doing since I decided to do this AFTER putting a label on my storage partition... had to go hand find the swap file entries and delete those as well as the file itself... Reinstalled, nope, nothing... then noticed that there was no ipset.txt file and the .log file was empty, then saw your note here.

Running start manually seems to have completed the whole setup , now looks good! All setup. Core issue I think was with my script files but now that I've noticed and sorted that, should be good.

 

[Adamm]

Found some errors with system script files where they were exiting before getting to the added skynet lines, due to a unique scripting thing i have going on...

Would make sense, I suggest not to use files like firewall-start (or any similar files) for manual scripting but rather point them to an external file for situations like this, much like Skynet's entry in the file.

Now we have the cause of failure, I suggest restarting the firewall service to make sure the issues with scripting were indeed corrected and the firewall-start event runs as expected.

 

[Adamm]

But it hasn't restored them to /mnt/DTSE9/skynet/* (which was the old mount point). Can I just shut Skynet down, manually copy them to /mnt/<newdevice>/skynet and /mnt/<newdevice>/skynet/scripts/ and then fire up Skynet again? Would that work?

I improved the backup/restore functions in v5.8.3, it should now be able to handle install directory changing. They also now use gzip compression which should reduce filesize of the backup by about x10 in my testing.