What's new

Skynet Filter Validator v0.7 - Skynet Firewall Filter List IPv4 Integrity Validator

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

Latest iteration (v0.4) is out there, including the regex fixes made by @SomeWhereOverTheRainBow that were identified by @visortgw! Thank you!

Bonus -- summary included at the end of the validation sequence:

1678132364762.png
 
@Viktor Jaep: A suggestion for installation command string for this script (and all of your others), use "chmod 755" instead of "chmod a+rx". I am seeing the "chmod a+rx" resulting in wide open permissions (i.e., 777). Thanks.
 
Maybe I'm missing something... but I'm not seeing any screenshot or other info to come to that conclusion... but I bet you're right. ;)
Probably just need to have the script force system binaries over optional.
 
Last edited:
Another “fix” to this bogus IP problem is to stop trusting threatview.io since most - if not all - of the “problems” have come from their feeds.
I get the sentiment, but how do you stop errors, that breaks functionality, of a script that's meant to help protect you.
A filter script that throws out bad entries doesn't seem like a bad idea.

Is this something that @Adamm can contribute / incorporate into fixing?
 
I get the sentiment, but how do you stop errors, that breaks functionality, of a script that's meant to help protect you.
A filter script that throws out bad entries doesn't seem like a bad idea.

Is this something that @Adamm can contribute / incorporate into fixing?
I think someone should send this check script to the guy that runs threatview.io!

Certainly there can be improvements made to Skynet’s regex for IPs, but it’s also worked just fine with reputable lists.

This script itself doesn’t change how Skynet works. It will still break if a bad list is used, since it downloads the lists for itself.
 
I get the sentiment, but how do you stop errors, that breaks functionality, of a script that's meant to help protect you.
A filter script that throws out bad entries doesn't seem like a bad idea.

Is this something that @Adamm can contribute / incorporate into fixing?
I agree. While it is highly concerning threatview.io has bad ip addresses within its list, it is equally concerning skynet has no checks in place to prevent such from breaking its usage. While we as users can easily stop using a list if a bad ip address is in the list, skynet did not make it easy to figure out/discern which list was causing the problem. Instead, skynet completed operations as if nothing was wrong. It wasn't until users read their statistics that they notice a different story- in fact statistics showed all their entries had not loaded.
 
@Viktor Jaep: A suggestion for installation command string for this script (and all of your others), use "chmod 755" instead of "chmod a+rx". I am seeing the "chmod a+rx" resulting in wide open permissions (i.e., 777). Thanks.
I concur. 777 is too permissive.
Next iteration of the Skynet Filter Validator v0.5 is out there, including more optimizations and fixes by @SomeWhereOverTheRainBow... I've also made the chmod change per @visortgw's suggestion... @Adamm... please feel free to pilfer these methods if you think they can help make Skynet more solid.
 
It’s from /opt/bin/grep versus /bin/grep.

Probably just need to have the script force system binaries over optional.

Trying to understand how to reduce instances of these issues with conflicting binaries... is it best practice to call "grep" in your code with an actual path, as in "/bin/grep"?
 
@Viktor Jaep Have you benchmarked v0.5 against v0.4? I am seeing what took about 0:21 under v0.4 now takes 1:20 for v0.5 for https://raw.githubusercontent.com/ViktorJp/Skynet/main/filter.list.
If anything, it seems the same or faster to me. I have noticed that sometimes it will bog down on a big list, but I can't be sure if it's because it's taking longer to download it sometimes, and faster on other times... I blamed it on internet weather and how hard the list servers are being hit at that time.
 
Last edited:
If anything, it seems the same or faster to me. I have noticed that sometimes it will bog down on a big list, but I can't be sure if it's because it's taking longer to curl download it than other times, where it might be downloading faster...
What might help optimize the speed is downloading the output directly to temporary file then performing operations and delete the file when finished, versus piping the output from curl which could be time consuming on larger lists.
 
@Viktor Jaep

test using v.5

1678220229161.png


Test using v.4

1678220293824.png


here is a retest of v.4

1678220447574.png



retest of v.5 after testing v.4

1678220558581.png


another repeated run of version 5

1678220788028.png


I will do one for v.6 in a second...

Better yet, you can tell the biggest factor is the download operations.
 
Last edited:

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top