Yet another malware block script using ipset (v4 and v6)

[Martineau]

Make that
"/tmp/mnt/EXT4"
for both instances.

Indeed, whilst aesthetically pleasing from a programming standpoint (and should be encouraged as good-practice), technically this doesn't alter the assignment of the concatenation of the two "strings" to the variable.

 

[Martineau]

Hey I'm not finding the saved info on my usb drive from this script it gets lost in a reboot. Did I screw something up here? EXT4 is my usb drive root.

################################################Customise for local use #############################################
if [ -d  "/tmp/mnt/"EXT4 ];then
    DIR="/tmp/mnt/"EXT4                # <== USB Location of IPSET save/restore configuration
else
    DIR="/tmp"                                #         NOTE: /TMP isn't permanent! ;-) but allows testing of save/restore
fi

HHMMSS="168:00:00"                            # <== Specify retention period to keep Blacklist entries or passed via 'init reset' hh:mm:ss' invocation
                                            #            e.g. 168 hrs = 7 days
#####################################################################################################################

What happens when you manually issue

./IPSET_Block.sh   save

Is the 'IPSET_Block.config' file created on the USB drive?

NOTE: This should be moved to the appropriate thread.

EDIT: Ahh.. I see that a duplicate post is in the more appropriate thread

 

[redhat27]

Not sure if just me, but this seems to break dns-crypt?

I use dnscrypt and this script just fine without any issues

 

[shooter40sw]

Hi @redhat27 you should place this on the wiki also

 

[skeal]

What happens when you manually issue

./IPSET_Block.sh   save

Is the 'IPSET_Block.config' file created on the

NOTE: This should be moved to the appropriate thread.

EDIT: Ahh.. I see that a duplicate post is in the more appropriate thread

What happens when you manually issue

./IPSET_Block.sh   save

Is the 'IPSET_Block.config' file created on the USB drive?

NOTE: This should be moved to the appropriate thread.

EDIT: Ahh.. I see that a duplicate post is in the more appropriate thread

Yes sir the IPSET_Block.config is created and populated now and I have not changed anything since my initial post on this issue. I didn't see these files the first time I checked. IPSET_Block.config is created and empty.
Is it possible I have it right or.....?

 

[Martineau]

Yes sir the IPSET_Block.config is created and populated now and I have not changed anything since my initial post on this issue. I didn't see these files the first time I checked. IPSET_Block.config is created and empty.
Is it possible I have it right or.....?

Well IPSET_Block.config should never be 'empty' if the 'save' command has been issued unless this was an existing bug in v3.04 that was probably rectified when I rewrote some of the logic in v3.05

So even if you issue the following:

./IPSET_Block.sh   init   reset
./IPSET_Block.sh   save

So even if the member count for the Blacklist IPSET is 0, then the file would still contain the 'create' directives for the IPSETs.

P.S. So you do have the two IPSET_Block.sh cron jobs?

Abject apologies....really should be in the other thread...

 

[skeal]

Well IPSET_Block.config should never be 'empty' if the 'save' command has been issued unless this was an existing bug in v3.04 that was probably rectified when I rewrote some of the logic in v3.05

So even if you issue the following:

./IPSET_Block.sh   init   reset
./IPSET_Block.sh   save

So even if the member count for the Blacklist IPSET is 0, then the file would still contain the 'create' directives for the IPSETs.

P.S. So you do have the two IPSET_Block.sh cron jobs?

Abject apologies....really should be in the other thread...

Well IPSET_Block.config should never be 'empty' if the 'save' command has been issued unless this was an existing bug in v3.04 that was probably rectified when I rewrote some of the logic in v3.05

So even if you issue the following:

./IPSET_Block.sh   init   reset
./IPSET_Block.sh   save

So even if the member count for the Blacklist IPSET is 0, then the file would still contain the 'create' directives for the IPSETs.

P.S. So you do have the two IPSET_Block.sh cron jobs?

Abject apologies....really should be in the other thread...

yes i have both cronjobs and i ran those commands and everything looks ok except IPSET_Block.configbak is still empty.

 

[redhat27]

Indeed, whilst aesthetically pleasing from a programming standpoint (and should be encouraged as good-practice), technically this doesn't alter the assignment of the concatenation of the two "strings" to the variable.

Generally true, but double quotes help in preserving some special characters like & or \ or '

 

[redhat27]

Hi @redhat27 you should place this on the wiki also

Will do. Maybe in a few days. I noticed that swetoast removed his scripts. It is sad that he decided to go that route.
I am planning to incorporate additional lists in this script, that will also include the sources that blocks telemetry, shodan and project 25499 scanners, so there will only one (this) script to run

 

[bayern1975]

As if we don't have enough already

Here is yet another malware blocking script that uses ipset (v4 and v6). So why put up another one?

• Simpler
• Quicker (10-20 seconds typical runtime depending on CPU load, processing power and bandwidth)
• Less CPU cycles for your router to load the lists (kinda saying the same thing as above)
• No files created in /tmp (if for any reason script exits abnormally, /tmp files eat up precious RAM)
• Also supports both ipset v4 and ipset v6
• Can also be run in cru

you should add all microsoft spying, telemetry etc. in this script too....

 

[redhat27]

you should add all microsoft spying, telemetry etc. in this script too

That is exactly what I said in the post before yours (#49)

 

[Jack Yaz]

So will this then incorporate https://github.com/RMerl/asuswrt-me...allation-instructions#tor-and-countries-block (which I believe is yours anyway?) I'll have to keep an eye out so I'm not doubling up on the blocking!

 

[redhat27]

I see no reason to combine country and tor blocking to this script. Malware sources change frequently, very unlike addresses assigned to countries and tor exit nodes. These are not harmful in the first place and need not be in the same category of malware.

An example of where country or tor blocking is useful is when you do not want sources that are in some countries, or sources that are anonymous (using tor) to initiate a connection to your router. The blocking is in the INPUT chain

 

[Jack Yaz]

Makes sense, will telemetry etc be removed from that script then, or will the proposed lists be separate from the telemtry included in your other script?

 

[redhat27]

Microsoft telemetry is also a fairly static list and can be argued to be kept in a run-once type script. Also, it is a fairly small list (50 sources or less) compared to published malware sources, that are typically in several tens of thousands to hundreds of thousands.

So including the small set of telemetry, shodan and a few other scanners adds at most a hundred of so more entries. Also since these sources fall in a category that can be considered as an invasion of privacy, adding a hundred or so these IPs to a much larger list of malware sources seems acceptable to me.

 

[redhat27]

I have a new script that is ready to deploy. Major changes are:

This blocks around 112k single IPs and around 5k CIDR ranges via an enhanced list is primarily sourced from firehol that @Adamm was very kind enough to curate and share with me. I've also mixed in the small set of (relatively static) list of M$ telemetry, Shodan and Project 25499 scanners in there. It is ready to be uploaded to github, but this change brings a few downsides:

• The script now will write out one file in /tmp as a pre-process step (it's deleted right afterwards)
• My runtime went from ~15 seconds to ~42 seconds (about a threefold increase) But considering that IPs blocked increased from 34k to 112k and CIDRs increased from 0.8k to 5k, it should be acceptable.
• The script now splits the single IP sets at 64k sets, as ipset-v4 still cannot have a set more than that so there are YAMalwareBlock1IP, YAMalwareBlock2IP and YAMalwareBlockCIDR sets now

The changes are ready to deploy, but I haven't uploaded them to github yet. Please let me know if you see any issues with the proposed changes. If there are no objections, I'll need to update the OP when I deploy the changes so OP still holds true.

 

[shooter40sw]

Hi! IMHO I will and want to stick with malware blocking, maybe TOR and malware, although Im using the country and TOR script, but I striped telemetry from it, so Im using that modified script and this one, will the users be able to select what can be blocked or not?
Great work and thanks !

I have a new script that is ready to deploy. Major changes are:

This blocks around 112k single IPs and around 5k CIDR ranges via an enhanced list is primarily sourced from firehol that @Adamm was very kind enough to curate and share with me. I've also mixed in the small set of (relatively static) list of M$ telemetry, Shodan and Project 25499 scanners in there. It is ready to be uploaded to github, but this change brings a few downsides:

• The script now will write out one file in /tmp as a pre-process step (it's deleted right afterwards)
• My runtime went from ~15 seconds to ~42 seconds (about a threefold increase) But considering that IPs blocked increased from 34k to 112k and CIDRs increased from 0.8k to 5k, it should be acceptable.
• The script now splits the single IP sets at 64k sets, as ipset-v4 still cannot have a set more than that so there are YAMalwareBlock1IP, YAMalwareBlock2IP and YAMalwareBlockCIDR sets now

The changes are ready to deploy, but I haven't uploaded them to github yet. Please let me know if you see any issues with the proposed changes. If there are no objections, I'll need to update the OP when I deploy the changes so OP still holds true.

 

[RMerlin]

Since new flavours of those block scripts seem to be trendy lately...

Any of you considered implementing it at the RPDB level? Might possibly be more efficient than iptables...

/ # ip route add prohibit 4.2.2.2
/ # ping 4.2.2.2
PING 4.2.2.2 (4.2.2.2): 56 data bytes
ping: sendto: Network is unreachable

(runs away to hide)

 

[redhat27]

Interesting idea. It would be nice to see an example of ip route used in conjunction of hundred thousands of discrete ip's or ip ranges. Firewalling with ipsets seems to be the commonsense/obvious approach.

 

[redhat27]

will the users be able to select what can be blocked or not?

Sure, once the url list is downloaded, its not redownloaded till its manually deleted, so all you need to do is remove the url for telemetry and scanners if you do not want to block those. I'll tell what to remove once I upload the new script and the new lists