What's new

Malware Filter / bad host IPSET

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

do you see the the entry created by the script in your forward chain?
Code:
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination       
    0     0 REJECT     all  --  any    any     anywhere             anywhere             match-set malware-filter src,dst reject-with icmp-port-unreachable

Also... where are you pinging from?.... note the post here
http://www.snbforums.com/threads/malware-filter-bad-host-ipset.35423/page-2#post-298028
 
Last edited:
hi @swetoast Im using the malware script and thought it was working, but it looks like its not, I can ping a site thats in the malware filter

Code:
 ipset --list
Name: malware-filter
Type: iphash
References: 1
Header: hashsize: 1024 probes: 8 resize: 50
Members:


One IP failed to respond

Code:
99.68.146.9
:/tmp/mnt/sda1/blockip# ping 98.215.45.16
PING 98.215.45.16 (98.215.45.16): 56 data bytes

But other did respond

Code:
 cat malware-filter.txt |grep 221.125.113.178
221.125.113.178

:/tmp/mnt/sda1/blockip# ping 221.125.113.178
PING 221.125.113.178 (221.125.113.178): 56 data bytes
64 bytes from 221.125.113.178: seq=0 ttl=47 time=316.413 ms
64 bytes from 221.125.113.178: seq=1 ttl=47 time=315.809 ms


And in the iptables -L -n -v comand there is no rejected packets

Code:
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           set malware-filter src,dst reject-with icmp-port-unreachab

Any ideas?
Thanks
 
What do you get if you run this:
It should output all ip:s in that list.
Code:
ipset -L malware-filter
 
Code:
#!/bin/sh
# Original script by swetoast. Updates by Neurophile & Octopus.
path=/opt/var/cache/malware-filter                      # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

ipset -v | grep -i "v4" > /dev/null 2>&1

if [ $? -eq 0 ]; then
     ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
else
     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
fi

case $(uname -m) in
armv7l)
    MATCH_SET='--match-set'                             # Value for ARM Routers
;;
mips)
    MATCH_SET='--set'                                   # Value for Mips Routers
;;
*)
    MATCH_SET='--match-set'                             # Value for Wildcard Routers
;;
esac

get_list () {
        mkdir -p $path
        wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

 run_ipset () {

get_list
ipset --destroy malware-filter > /dev/null 2>&1         # Delete the filter so it doesnt clash with the update

if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
ipset -N malware-filter iphash
while read line; do ipset add malware-filter $line; done < $path/malware-filter.txt

iptables-save | grep malware-filter > /dev/null 2>&1 || \
iptables -I INPUT -m set $MATCH_SET malware-filter -j src,dst REJECT
fi }

run_ipset
exit $?

lets try this one last time and see if this works for all routers need to start versioning soon to keep up with demand for this script so i know what version everyone is running..
 
Last edited:
Looks like something is wrong because this is the result:

Code:
 ipset -L malware-filter
Name: malware-filter
Type: iphash
References: 1
Header: hashsize: 1024 probes: 8 resize: 50
Members:

:/tmp/home/root#




hi @swetoast Im using the malware script and thought it was working, but it looks like its not, I can ping a site thats in the malware filter

Code:
 ipset --list
Name: malware-filter
Type: iphash
References: 1
Header: hashsize: 1024 probes: 8 resize: 50
Members:


One IP failed to respond

Code:
99.68.146.9
:/tmp/mnt/sda1/blockip# ping 98.215.45.16
PING 98.215.45.16 (98.215.45.16): 56 data bytes

But other did respond

Code:
 cat malware-filter.txt |grep 221.125.113.178
221.125.113.178

:/tmp/mnt/sda1/blockip# ping 221.125.113.178
PING 221.125.113.178 (221.125.113.178): 56 data bytes
64 bytes from 221.125.113.178: seq=0 ttl=47 time=316.413 ms
64 bytes from 221.125.113.178: seq=1 ttl=47 time=315.809 ms


And in the iptables -L -n -v comand there is no rejected packets

Code:
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           set malware-filter src,dst reject-with icmp-port-unreachab

Any ideas?
Thanks
 
The ipset is still empty, thanks for your help!

Code:
/tmp/home/root# ipset -L malware-filter
Name: malware-filter
Type: iphash
References: 1
Header: hashsize: 1024 probes: 8 resize: 50
Members:



Code:
#!/bin/sh
# Original script by swetoast. Updates by Neurophile & Octopus.
path=/opt/var/cache/malware-filter                      # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

ipset -v | grep -i "v4" > /dev/null 2>&1

if [ $? -eq 0 ]; then
     ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
else
     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
fi

case $(uname -m) in
armv7l)
    MATCH_SET='--match-set'                             # Value for ARM Routers
;;
mips)
    MATCH_SET='--set'                                   # Value for Mips Routers
;;
*)
    MATCH_SET='--match-set'                             # Value for Wildcard Routers
;;
esac

get_list () {
        mkdir -p $path
        wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

 run_ipset () {

get_list
ipset --destroy malware-filter > /dev/null 2>&1         # Delete the filter so it doesnt clash with the update

if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
ipset -N malware-filter iphash
while read line; do ipset add malware-filter $line; done < $path/malware-filter.txt

iptables-save | grep malware-filter > /dev/null 2>&1 || \
iptables -I INPUT -m set $MATCH_SET malware-filter -j src,dst REJECT
fi }

run_ipset
exit $?

lets try this one last time and see if this works for all routers need to start versioning soon to keep up with demand for this script so i know what version everyone is running..
 
Think i found the issue just gonna run some test to ensure that this time its right

Same issue
Code:
ipset --save > /mnt/sda1/blockip/malware-filter.txt
:/tmp/home/root# ipset -L malware-filter
Name: malware-filter
Type: iphash
References: 1
Header: hashsize: 1024 probes: 8 resize: 50
Members:


Nothing in the ipset. And my malware-filter.txt had entries before the change in script but now it has this

Code:
 cat /mnt/sda1/blockip/malware-filter.txt
# Generated by ipset 4.5 on Fri Jan  6 07:32:29 2017
-N malware-filter iphash --hashsize 1024 --probes 8 --resize 50
COMMIT
# Completed on Fri Jan  6 07:32:29 2017
 
In INPUT chain you should always use: DROP
iptables -I INPUT -m set $MATCH_SET malware-filter -j src DROP
iptables -I FORWARD -m set $MATCH_SET malware-filter -j src,dst REJECT
 
im gonna need a day or two cause when i run each line separate it works flawlessly when i run it as a script it breaks so give me time

its specificly this line

for i in `cat $path/malware-filter.txt`; do ipset add malware-filter $i ; done that breaks it if you run it as a single line in separate script it loads the ips into the filter but inside the script it breaks

Code:
Name: malware-filter
Type: hash:ip
Revision: 0
Header: family inet hashsize 16384 maxelem 65536
Size in memory: 395704
References: 1
Members:
24.182.214.6
185.35.62.86
193.106.81.206
211.38.71.81
37.53.95.16
187.161.7.88
85.97.186.11
88.248.204.116
180.178.132.26
31.42.188.10
61.91.37.18
83.104.230.199
93.116.147.15
185.46.248.129
31.129.111.16
37.29.47.183
91.244.25.238
201.166.237.149
46.233.15.123
111.23.15.130
213.231.18.241
50.186.230.154
13.54.22.118
178.174.209.174
107.150.44.58
 
Open to suggestions...

Code:
#!/bin/sh
# Original script by swetoast. Updates by Neurophile & Octopus.
path=/opt/var/cache/malware-filter              # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

ipset -v | grep -i "v4" > /dev/null 2>&1

if [ $? -eq 0 ]; then
     ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
else
     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
fi

case $(uname -m) in
armv7l)
    MATCH_SET='--match-set'                 # Value for ARM Routers
;;
mips)
    MATCH_SET='--set'                       # Value for Mips Routers
;;
*)
    MATCH_SET='--match-set'                 # Value for Wildcard Routers
;;
esac

get_list () {
        mkdir -p $path
        wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list
ipset --destroy malware-filter > /dev/null 2>&1         # Delete the filter so it doesnt clash with the update

if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    path=/opt/var/cache/malware-filter
    ipset -N malware-filter hash:ip family inet hashsize 2048 maxelem 65536
    for i in `cat $path/malware-filter.txt`; do ipset add malware-filter $i ; done   #THIS IS THE BROKEN LINE THAT MAKES THE SCRIPT NOT FUNCTION AS INTENDED
fi

iptables-save | grep malware-filter > /dev/null 2>&1 || \
iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
}

run_ipset
exit $?
 
Hi All,

I’m back. I had to step away for the afternoon. It appears there has been a lot of conversation in that time.

I can get ping replies from many sites in the malware-filter.txt file. I pinged those ip address from windows cmd prompt and got replies. I then pinged from the router SSH session with the same result. Here is more info that was requested:

Code:
admin@RT-AC88U-5248:/jffs/scripts# uname -m
armv7l

Code:
admin@RT-AC88U-5248:/jffs/scripts# ipset -L malware-filter
Name: malware-filter
Type: hash:ip
Revision: 0
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 8248
References: 3
Members:

Code:
Ouch, I have 3 entries in Chain FORWARD for malware filter. Probably from running the script 3 times?
Chain FORWARD (policy DROP)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             match-set malware  -filter src,dst reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             match-set malware -filter src,dst reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             match-set malware -filter src,dst reject-with icmp-port-unreachable
TCPMSS     tcp  --  anywhere             anywhere             tcpflags: SYN,RST /SYN TCPMSS clamp to PMTU
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             state RELATED,EST                                                ABLISHED
DROP       all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere            state INVALID
ACCEPT     all  --  anywhere             anywhere
SECURITY  all  --  anywhere            anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate DNAT
ACCEPT     all  --  anywhere             anywhere

Code:
admin@RT-AC88U-5248:/jffs/scripts# /usr/sbin/iptables -L FORWARD -v -n
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination                                               
    0    0 REJECT     all  --  *      *      0.0.0.0/0            0.0.0.0/0 match-set malware-filter src,dst reject-with icmp-port-unreachable
    0    0 REJECT     all  --  *      *      0.0.0.0/0            0.0.0.0/0 match-set malware-filter src,dst reject-with icmp-port-unreachable
    0    0 REJECT     all  --  *      *      0.0.0.0/0            0.0.0.0/0 match-set malware-filter src,dst reject-with icmp-port-unreachable
 9963  587K TCPMSS     tcp  --  *      *      0.0.0.0/0            0.0.0.0/0                                                            tcpflags: 0x06/0x02 TCPMSS clamp to PMTU
 749K  986M ACCEPT     all  --  tun11  *       0.0.0.0/0            0.0.0.0/0                                                 
 434K  41M ACCEPT     all  --  *      *      0.0.0.0/0            0.0.0.0/0 state RELATED,ESTABLISHED
    0    0 DROP       all  --  !br0   ppp0    0.0.0.0/0            0.0.0.0/0                                                 
    0    0 DROP       all  --  !br0   eth0    0.0.0.0/0            0.0.0.0/0                                                 
    0    0 DROP       all  --  eth0   *       0.0.0.0/0            0.0.0.0/0    state INVALID
    0    0 ACCEPT     all  --  br0    br0    0.0.0.0/0            0.0.0.0/0                                                 
    0    0 SECURITY   all  --  ppp0  *       0.0.0.0/0            0.0.0.0/0                                               
    0    0 ACCEPT     all  --  *      *      0.0.0.0/0            0.0.0.0/0   ctstate DNAT
 6692  426K ACCEPT     all  --  br0    *      0.0.0.0/0            0.0.0.0/0
 
Open to suggestions...

Code:
#!/bin/sh
# Original script by swetoast. Updates by Neurophile & Octopus.
path=/opt/var/cache/malware-filter              # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

ipset -v | grep -i "v4" > /dev/null 2>&1

if [ $? -eq 0 ]; then
     ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
else
     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
fi

case $(uname -m) in
armv7l)
    MATCH_SET='--match-set'                 # Value for ARM Routers
;;
mips)
    MATCH_SET='--set'                       # Value for Mips Routers
;;
*)
    MATCH_SET='--match-set'                 # Value for Wildcard Routers
;;
esac

get_list () {
        mkdir -p $path
        wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list
ipset --destroy malware-filter > /dev/null 2>&1         # Delete the filter so it doesnt clash with the update

if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    path=/opt/var/cache/malware-filter
    ipset -N malware-filter hash:ip family inet hashsize 2048 maxelem 65536
    for i in `cat $path/malware-filter.txt`; do ipset add malware-filter $i ; done   #THIS IS THE BROKEN LINE THAT MAKES THE SCRIPT NOT FUNCTION AS INTENDED
fi

iptables-save | grep malware-filter > /dev/null 2>&1 || \
iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
}

run_ipset
exit $?
If the malware-filter is referenced by an iptable it can't be destroyed right? so you would have to remove the rule first.......

Code:
x, destroy [ SETNAME ]
Destroy the specified set or all the sets if none is given.
If the set has got reference(s), nothing is done and no set destroyed.
 
yeah but its used by the kernel sooo we need to swap it, so it need a bigger re-write and today im busy hence my comment before about a few days..

so if anyone is up to the task and feels like rewriting so that the list gets loaded proper and then the rules get swaped on a update else all of this has to wait until i have more time on my hands.
 
Thanks for working on this @swetoast. No hurry here. I will check back periodically to see if there is an update. I greatly appreciate everyone's help.
 
Untested, remember adding an ipset might take time so dont rush it when testing.

Code:
#!/bin/sh
# Original script by swetoast. Updates by Neurophile & Octopus.
# Revision 1

path=/opt/var/cache/malware-filter              # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

ipset -v | grep -i "v4" > /dev/null 2>&1

if [ $? -eq 0 ]; then
     ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
else
     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
fi

case $(uname -m) in
armv7l)
    MATCH_SET='--match-set'                 # Value for ARM Routers
;;
mips)
    MATCH_SET='--set'                       # Value for Mips Routers
;;
*)
    MATCH_SET='--match-set'                 # Value for ARM Routers
;;
esac

get_list () {
        mkdir -p $path
        wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    path=/opt/var/cache/malware-filter
    ipset -N malware-filter hash:ip family inet hashsize 2048 maxelem 65536
    for i in `cat $path/malware-filter.txt`; do ipset add malware-filter $i ; done
fi
else
    path=/opt/var/cache/malware-filter
    ipset -N malware-update hash:ip family inet hashsize 2048 maxelem 65536
    for i in `cat $path/malware-filter.txt`; do ipset add malware-update $i ; done
    ipset swap malware-update malware-filter
    ipset destroy malware-update
fi


iptables-save | grep malware-filter > /dev/null 2>&1 || \
iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
}

run_ipset
exit $?
 
Last edited:
Untested, remember adding an ipset might take time so dont rush it when testing.

Code:
#!/bin/sh
# Original script by swetoast. Updates by Neurophile & Octopus.
# Revision 1

path=/opt/var/cache/malware-filter              # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

ipset -v | grep -i "v4" > /dev/null 2>&1

if [ $? -eq 0 ]; then
     ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
else
     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
fi

case $(uname -m) in
armv7l)
    MATCH_SET='--match-set'                 # Value for ARM Routers
;;
mips)
    MATCH_SET='--set'                       # Value for Mips Routers
;;
*)
    MATCH_SET='--match-set'                 # Value for ARM Routers
;;
esac

get_list () {
        mkdir -p $path
        wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    path=/opt/var/cache/malware-filter
    ipset -N malware-filter hash:ip family inet hashsize 2048 maxelem 65536
    for i in `cat $path/malware-filter.txt`; do ipset add malware-filter $i ; done
fi
else
    path=/opt/var/cache/malware-filter
    ipset -N malware-update hash:ip family inet hashsize 2048 maxelem 65536
    for i in `cat $path/malware-filter.txt`; do ipset add malware-update $i ; done
    ipset swap malware-update malware-filter
    ipset destroy malware-update
fi


iptables-save | grep malware-filter > /dev/null 2>&1 || \
iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
}

run_ipset
exit $?
Works ok for me...takes a while to build the set though.... maybe not a good idea to leave it in firewall start
 

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top