Malware Filter / bad host IPSET

[swetoast]

add sources again either by adding it manually or downloading the file again.

wget --no-check-certificate https://gitlab.com/swe_toast/malware-filter/raw/master/malware-filter.list -O /jffs/malware-filter.list

do this both @skeal and @eclp

 

[eclp]

ASUSWRT-Merlin RT-AC87U 380.66-beta1-g7b22cbf Fri Apr 21 18:45:56 UTC 2017
...@RT-AC87U:/tmp/home/root# cat /jffs/malware-filter.list
...@RT-AC87U:/tmp/home/root# ipset -L malware-filter_ipv4 | wc -l
7
...@RT-AC87U:/tmp/home/root# ipset -L malware-filter_ipv4_range | wc -l
7
...@RT-AC87U:/tmp/home/root# wget --no-check-certificate https://gitlab.com/swe_toast/malware-filter/raw/master/malware-filter.list -O
/jffs/malware-filter.list
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/root/.wget-hsts'. HSTS will be disabled.
--2017-04-28 09:01:55--  https://gitlab.com/swe_toast/malware-filter/raw/master/malware-filter.list
Resolving gitlab.com... 52.167.219.168
Connecting to gitlab.com|52.167.219.168|:443... connected.
WARNING: cannot verify gitlab.com's certificate, issued by 'CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 842 [text/plain]
Saving to: '/jffs/malware-filter.list'

/jffs/malware-filter.list         100%[=============================================================>]     842  --.-KB/s    in 0s

2017-04-28 09:01:56 (38.2 MB/s) - '/jffs/malware-filter.list' saved [842/842]

...@RT-AC87U:/tmp/home/root# sh /jffs/scripts/malware-filter
system: Malware Filter (ipv4) loaded 15179 unique ip addresses that will be rejected from contacting your router.
system: Malware Filter (ipv4) loaded 0 unique ip ranges that will be rejected from contacting your router.
...@RT-AC87U:/tmp/home/root#

 

[swetoast]

so download the list with the command above

wget --no-check-certificate https://gitlab.com/swe_toast/malware-filter/raw/master/malware-filter.list -O /jffs/malware-filter.list

its a fairly simple problem the list wont populate cause it has no sources, sources are required to add stuff to block either upload a new list to the router or download using the command then verify that malware-filter.list is populated with sources.

 

[swetoast]

@eclp cat /jffs/malware-filter.list, output again

 

[eclp]

ASUSWRT-Merlin RT-AC87U 380.66-beta1-g7b22cbf Fri Apr 21 18:45:56 UTC 2017
...@RT-AC87U:/tmp/home/root# cat /jffs/malware-filter.list
http://cinsscore.com/list/ci-badguys.txt
http://malc0de.com/bl/IP_Blacklist.txt
http://sanyalnet-cloud-vps.freeddns.org/mirai-ips.txt
http://www.abuseat.org/iotcc.txt
http://www.malwaredomainlist.com/hostslist/ip.txt
https://feodotracker.abuse.ch/blocklist/?download=ipblocklist
https://lists.blocklist.de/lists/bots.txt
https://lists.blocklist.de/lists/ssh.txt
https://ransomwaretracker.abuse.ch/downloads/CW_PS_IPBL.txt
https://ransomwaretracker.abuse.ch/downloads/LY_PS_IPBL.txt
https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt
https://ransomwaretracker.abuse.ch/downloads/TC_PS_IPBL.txt
https://ransomwaretracker.abuse.ch/downloads/TL_C2_IPBL.txt
https://ransomwaretracker.abuse.ch/downloads/TL_PS_IPBL.txt
https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
https://zeustracker.abuse.ch/blocklist.php?download=badips...@RT-AC87U:/tmp/home/root#

 

[swetoast]

think ive narrowed down an possible issue, will update later on a bit busy today but it has to do with https sources and wget not usuing the ca bundle it has at its disposal

 

[skeal]

add sources again either by adding it manually or downloading the file again.

wget --no-check-certificate https://gitlab.com/swe_toast/malware-filter/raw/master/malware-filter.list -O /jffs/malware-filter.list

do this both @skeal and @eclp

I get this when running the above command:

ASUSWRT-Merlin RT-AC68U 380.65-4 Wed Mar 29 04:40:14 UTC 2017
:/tmp/home/root# wget --no-check-certificate https:
//gitlab.com/swe_toast/malware-filter/raw/master/malware-filter.list -O /jffs/ma
lware-filter.list
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/root/.wget-hsts'. HSTS will be disabled.
--2017-04-28 09:20:58--  https://gitlab.com/swe_toast/malware-filter/raw/master/malware-filter.list
Resolving gitlab.com... 52.167.219.168
Connecting to gitlab.com|52.167.219.168|:443... connected.
WARNING: cannot verify gitlab.com's certificate, issued by 'CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 842 [text/plain]
Saving to: '/jffs/malware-filter.list'

/jffs/malware-filte 100%[===================>]     842  --.-KB/s    in 0s

2017-04-28 09:20:59 (38.8 MB/s) - '/jffs/malware-filter.list' saved [842/842]

:/tmp/home/root#

So I manually added the entries.

Now I run malware-filter and this is the result:

:/tmp/home/root# /jffs/scripts/malware-filter
system: Malware Filter (ipv4) loaded 15179 unique ip addresses that will be rejected from contacting your router.
system: Malware Filter (ipv4) loaded 0 unique ip ranges that will be rejected from contacting your router.
:/tmp/home/root#

After a reboot the sleep command seems to have helped a lot I now have pretty good results all around. Even helped my dynamic ip blocker load right....lol.

 

[swetoast]

yeah the router is not a super computer too much and nothing will work right even too much filtering so filter only critical things not just for the sake of it

 

[swetoast]

test version again for @skeal and @eclp

https://gitlab.com/swe_toast/malware-filter/raw/master/WIP/malware-filter

quick and dirty fixes verifying that the list are downloaded before commiting to firewall and no https check since your wget is acting like a muppet

off for the rest of the day to enjoy my day off

 

[skeal]

test version again for @skeal and @eclp

https://gitlab.com/swe_toast/malware-filter/raw/master/WIP/malware-filter

quick and dirty fixes verifying that the list are downloaded before commiting to firewall and no https check since your wget is acting like a muppet

off for the rest of the day to enjoy my day off

Enjoy your day!! Rebooted with ver28 and everything looks just fine so far!

Things are still rocking right along!! Great fix bud!

 

[eclp]

@swetoast
Great! Now it works perfectly ... with IPv4, why IPv6 not?

Apr 28 12:00:39 system: Privacy Filter (ipv4) loaded 118 unique ip addresses that will be rejected from contacting your router.
Apr 28 12:00:39 system: Privacy Filter (ipv6) loaded 7 unique ip addresses that will be rejected from contacting your router.
Apr 28 18:47:27 system: Malware Filter (ipv4) loaded 46026 unique ip addresses that will be rejected from contacting your router.
Apr 28 18:47:27 system: Malware Filter (ipv4) loaded 851 unique ip ranges that will be rejected from contacting your router.

Many thanks

 

[swetoast]

very simple there are no ipv6 lists yet

 

[swetoast]

https://gitlab.com/swe_toast/malware-filter/blob/master/CONTRIBUTING.md

So i made a little contributing guide to the project if anyone wants to pitch in, the reason i want it this way is so its easy to find and debug issues if they are present and with that said here is an new test version of the script

https://gitlab.com/swe_toast/malware-filter/raw/master/WIP/malware-filter

rev 28

• ignore https certs when downloading sources
• restructuring script for easier manangment
• added download message if malware-list is missing along with number of sources in the list

 

[Csection]

https://gitlab.com/swe_toast/malware-filter/blob/master/CONTRIBUTING.md

So i made a little contributing guide to the project if anyone wants to pitch in, the reason i want it this way is so its easy to find and debug issues if they are present and with that said here is an new test version of the script

https://gitlab.com/swe_toast/malware-filter/raw/master/WIP/malware-filter

rev 28

• ignore https certs when downloading sources
• restructuring script for easier manangment
• added download message if malware-list is missing along with number of sources in the list

I tried the "Test ver. 28" and it ran just fine here!
I really LIKE this script!
I also LIKE Martineau's IPSET_Block script and have both of them running in cron.
They are a good compliment to RMerlin's firmware!
Thank you for your hard work!

 

[Xentrk]

test version again for @skeal and @eclp

https://gitlab.com/swe_toast/malware-filter/raw/master/WIP/malware-filter

quick and dirty fixes verifying that the list are downloaded before commiting to firewall and no https check since your wget is acting like a muppet

off for the rest of the day to enjoy my day off

Congrats and thank you for another updated version of malware-filter. 4 routers updated and working. Slightly different results. But close enough.

***AC88U Router with All TRAFFIC
system: Malware Filter (ipv4) loaded 36361 unique ip addresses that will be rejected from contacting your router.
system: Malware Filter (ipv4) loaded 852 unique ip ranges that will be rejected from contacting your router.

***AC88U Router with Policy Rules
system: Malware Filter (ipv4) loaded 35726 unique ip addresses that will be rejected from contacting your router.
system: Malware Filter (ipv4) loaded 852 unique ip ranges that will be rejected from contacting your router.

***AC88U Router with Native WAN
system: Malware Filter (ipv4) loaded 36361 unique ip addresses that will be rejected from contacting your router.
system: Malware Filter (ipv4) loaded 852 unique ip ranges that will be rejected from contacting your router.

***D-Link 880L with DD-WRT
system: Malware Filter (ipv4) loaded 31375 unique ip addresses that will be rejected from contacting your router.
system: Malware Filter (ipv4) loaded 851 unique ip ranges that will be rejected from contacting your router.

Edit: Oh yes, no usage messages on the router installed at the school with this version! Woot!

 

[swetoast]

yeah they are never gonna match so >30k sources is good enough

 

[swetoast]

rev 28 is live in repo since no issues was found

• ignore https certs when downloading sources
• restructuring script for easier manangment
• added download message if malware-list is missing along with number of sources in the list
• additional restructuring according to https://gitlab.com/swe_toast/malware-filter/blob/master/CONTRIBUTING.md

https://gitlab.com/swe_toast/malware-filter/raw/master/malware-filter

 

[swetoast]

big tnx to @Adamm for making malware-filter and How to Dynamically Ban Malicious IP's using IPSet (adamm version) work perfectly together

 

[swetoast]

closing up shop. this isnt worth my time anymore with the constant lack of imagination from the other scripters make it not worth it it was fun now its not.

 

[skeal]

closing up shop. this isnt worth my time anymore with the constant lack of imagination from the other scripters make it not worth it it was fun now its not.

Sorry to hear this you are a good script writer bud!