Malware Filter / bad host IPSET

Nutz2U2

Occasional Visitor
Hi to Everyone,

Wish to thank ALL the boys and girls who made this script workable.

Jan 12 15:02:55 system: Malware Filter loaded 35632 unique ip addresses.

Trying the "Privacy Filter" next ;).
 

radionerd

New Around Here
I am trying to install the latest v7 malware script. I am not great with unix commands, but am very persistent. My head hurts, Little help?
I'm Just trying to get into the cd /opt
Please don't tear my head off, I know there has to be a simple command to allow

[email protected]:/# cd /opt
-sh: cd: can't cd to /opt

I'm blocked by forum filters again
 
Last edited:

Cedarhillguy

New Around Here
I am trying to install the latest v7 malware script. I am not great with unix commands, but am very persistent. My head hurts, Little help?
I'm Just trying to get into the cd /opt
Please don't tear my head off, I know there has to be a simple command to allow

[email protected]:/# cd /opt
-sh: cd: can't cd to /opt

The /opt directory probably doesn't exist. Try cd / to get to the root directory and then ls -trl which will list the directories and files. You can place the script in another directory such as /jffs/scripts, but if so.. would need to modify the path in the script. If you want to make a directory the command is mkdir directoryname
 

radionerd

New Around Here
The /opt directory probably doesn't exist. Try cd / to get to the root directory and then ls -trl which will list the directories and files. You can place the script in another directory such as /jffs/scripts, but if so.. would need to modify the path in the script. If you want to make a directory the command is mkdir directoryname
Thanks for the response, I tried to attach the code but got blocked by the forum's filters. opt shows up. Trying to post a jpg of it since forum filters are blocking me.
ScreenHunter_280 Jan. 16 19.46.jpg
 

radionerd

New Around Here
My problem was /opt was simlinked to a deleted ->tmp/opt.

I'm rocking the script, Thanks guys!

Code:
Jan 17 21:58:13 kernel: net/ipv4/netfilter/ip_set_iphash.c: iphash_retry: rehashing of set malware-update triggered: hashsize grows from 88573 to 132859
Jan 17 21:59:24 system: Malware Filter loaded 33069 unique ip addresses.
 

radionerd

New Around Here
Thanks for working on the script!

I have been looking at blocked IP's from my logs, then in ipset -Test to see if they are in the malware-filter block list, nice!
Code:
ipset v4.5: -T requires setname and IP
Try `ipset -H' or 'ipset --help' for more information.
[email protected]:/jffs/scripts# ipset -T malware-filter 74.82.47.14
74.82.47.14 is in set malware-filter.
[email protected]:/jffs/scripts# ipset -T malware-filter 2.178.244.64
2.178.244.64 is NOT in set malware-filter.
[email protected]:/jffs/scripts#

Code:
/usr/sbin/iptables -L FORWARD -v -n
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    1    40 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           set malware-filter src,dst reject-with icmp-port-unreachable
1527K 2017M ipttolan   all  --  *      br0     0.0.0.0/0            0.0.0.0/0   
 777K   70M iptfromlan  all  --  br0    *       0.0.0.0/0            0.0.0.0/0 
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            224.0.0.0/4
2287K 2085M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED
    0     0 logdrop    all  --  !br0   eth0    0.0.0.0/0            0.0.0.0/0   
    0     0 logdrop    all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           state INVALID
    0     0 ACCEPT     all  --  br0    br0     0.0.0.0/0            0.0.0.0/0   
  630 51065 SECURITY   all  --  eth0   *       0.0.0.0/0            0.0.0.0/0   
16782 1512K NSFW       all  --  *      *       0.0.0.0/0            0.0.0.0/0   
  614 50105 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           ctstate DNAT
16168 1462K ACCEPT     all  --  br0    *       0.0.0.0/0            0.0.0.0/0   
[email protected]:/jffs/scripts#
 

Goobi

Regular Contributor
Hello everyone,

When I try and run the latest script off the wiki (v7), it appears to just hang after the wgets. This is what I see:

[email protected]:/jffs/scripts# /opt/bin/malware-block
/opt/var/cache/malware-filter/malware-list.pre 100%[============================================================================================================>] 160.20K --.-KB/s in 0.1s
/opt/var/cache/malware-filter/malware-list.pre [ <=> ] 2.12K --.-KB/s in 0s
/opt/var/cache/malware-filter/malware-list.pre [ <=> ] 10.55K --.-KB/s in 0.1s
/opt/var/cache/malware-filter/malware-list.pre 100%[============================================================================================================>] 16.99K --.-KB/s in 0.05s
/opt/var/cache/malware-filter/malware-list.pre 100%[============================================================================================================>] 253.30K --.-KB/s in 0.1s

My /opt/var/cache/malware-filter directory has the following files:

[email protected]:/tmp/home/root# ls /opt/var/cache/malware-filter
malware-filter.list malware-list.pre

Looking at the script, this is what it appears to be "hung" at:

cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt

I am running 380.64 on a 68U. I got two additional IPSETs running (BlockedCountries & MicrosoftSpyServers) with no problems so I think ipset itself is fine, but for some reason, the malware script seems to simply hang after wget. I did not see any errors in the log. Did I maybe typo something in the script? Here is the unaltered lines from the script:

get_list () {
mkdir -p $path
wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
}

As of now, it has been sitting there for about 20 minutes and top does not show any relevant activity. Any suggestions to point me in the right direction would be greatly appreciated.
 

Nutz2U2

Occasional Visitor
Hi,
I believe it appears to be hung. I run similar scripts in my services-start like:-
sh /opt/bin/privacy-block
sh /opt/bin/malware-block
sh /jffs/scripts/p2p

followed by:

# Prepare IPset Lists and stop w10 snooping
cru a privacy-filter "5 0 * * * /opt/bin/privacy-block"
# Prepare IPset Lists and Malware Blacklists
cru a malware-filter "10 0 * * * /opt/bin/malware-block"
# Prepare IPset List and P2Pguardian
cru a P2P-filter "15 0 * * * /jffs/scripts/p2p"
# Prepare IPset List and Country Block
cru a country-filter "45 0 * * * /jffs/scripts/country"

Via shell i do iptables -L and none of the filters show - neither do any of the cronjobs in cru -l.
Wait a few minutes, privacy-block has completed, then malware-block and finally p2p script.
They then show up 1 by 1 in iptables -L.
After the scripts are completed, services-start completes and the cronjobs show in cru l.

It just takes some patience. At night-time they run without any interaction from me either.

But it did take a while to get it right o_O.
 

swetoast

Guest
Hello everyone,

When I try and run the latest script off the wiki (v7), it appears to just hang after the wgets. This is what I see:

As of now, it has been sitting there for about 20 minutes and top does not show any relevant activity. Any suggestions to point me in the right direction would be greatly appreciated.

sometimes the host of these files have timeouts it happned to me too cant control it the only thing you can do is to try again but im gonna investigate if there is a possability to have timeout via wget if that happens so it skips lists.
 

Goobi

Regular Contributor
sometimes the host of these files have timeouts it happned to me too cant control it the only thing you can do is to try again but im gonna investigate if there is a possability to have timeout via wget if that happens so it skips lists.

Thanks for pointing me in the right direction. It appears that the following hosts on the list are unavailable at the moment based on ping:

http://cinsscore.com/list/ci-badguys.txt
http://sanyalnet-cloud-vps.freeddns.org/mirai-ips.txt

I commented those entries out and the script worked.
 

swetoast

Guest
So timeout value added so if the lists arent available then it skips.

Code:
#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile
# Testers: shooter40sw
# Revision 8

path=/opt/var/cache/malware-filter                      # Set your path here
timeout=60                                              # Set your timeout value here (seconds)
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL=''  

    ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --timeout=$timeout --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

echo "adding ipset rule to firewall this will take time."

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset -N malware-filter $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice -n 2 ipset $SYNTAX malware-filter $i ; done
fi
else
    nice -n 2 ipset -N malware-update $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice ipset $SYNTAX malware-update $i ; done
    nice -n 2 ipset $SWAPPED malware-update malware-filter
    nice -n 2 ipset $DESTROYED malware-update
fi

iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?
 
Last edited:

jimf

Occasional Visitor
@swetoast, I tried adding the timeout independently. I also added --tries=3. Maybe I'm impatient, but the timeout by itself didn't seem to be working.
 

swetoast

Guest
Code:
#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile, jimf
# Testers: shooter40sw
# Revision 9

path=/opt/var/cache/malware-filter                      # Set your path here
retries=3                                               # Set number of tries here
timeout=60                                              # Set timeot value here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL=''

    ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --tries=$retries  --timeout=$timeout --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

echo "adding ipset rule to firewall this will take time."

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset -N malware-filter $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice -n 2 ipset $SYNTAX malware-filter $i ; done
fi
else
    nice -n 2 ipset -N malware-update $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice ipset $SYNTAX malware-update $i ; done
    nice -n 2 ipset $SWAPPED malware-update malware-filter
    nice -n 2 ipset $DESTROYED malware-update
fi

iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?
 
Last edited:

jimf

Occasional Visitor
I would do both. I'm not sure what the default timeout is, but remember it's setting 3 values (DNS, connect, and read). This is what I did:
wget -q --tries=3 --timeout=15 ...

Not sure what the default timeouts are, but this way I know what to expect. wget leaves it up to the system.
 

icornish

Occasional Visitor
Hi,

I have implemented this script, and seems to work OK. Is it possible to include the Country & TorNode blocking that is covered in the first script on the wiki page by adding the appropriate URL's to the list file?
 

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top