What I would like to do with skynet is to add an aggregation feature so when users stumble upon adding a domain that is already covered by a netblock in use the addresses can be aggregated to keep the lists smaller instead of the overly inflated 500,000 single IP list.
If this can also be done with filter lists it might be advantageous because ip addresses already covered by an aggregated country or ASN can be removed from the main single IP list to reduce the complexity of having a giant size single IP list.
Entware has a package called aggregate similarly there is a python package called aggregate6, but both would mean skynet would have an Entware dependency, which is not something I am looking to do. Especially since skynet has gone this long without relying on entware dependencies. There may be away to use AWK to do the aggregation, but I am still researching this.
To clarify by what I mean to aggregate, let us say I am blocking 100.2.3.4/32, but I already block the netblock 100.2.3.0/24. At this point, there is no longer a need for me to block 100.2.3.4/32 since it is covered in the /24 netblock already. This creates an extra not needed block entry. Now fast forward to country blocking, lets assume
@Viktor Jaep blocks RU through country block, but then uses a blocklist that has over 5000 block entries from RU. This is a very wasteful blocklist since it is taking up 5000 of his spaces he could have used for a different 5000 single IP entries.
I already started aggregating my own list
Contribute to jumpsmm7/GeneratedAdblock development by creating an account on GitHub.
github.com
and my allow list.
And I have started a pattern of finding bugs in skynet and submitting pull request to
@Adamm
*Optimize Skynet use of subnet markings via utilization of regexp logic. (e.g. ensure /32 ranges should not wind up in Skynet-BlockedRanges, when single IP's are optimally used in Skynet-Blacklist ...
github.com
Here are my current specs using the patched version I submitted to
@Adamm in a pull request using my aggregated blocklist.
View attachment 49824
