What's new

Malware Filter / bad host IPSET

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

iptables: No chain/target/match by that name
Do MIPS routers use an older version of iptables because of the kernel version?
perhaps its the iptables-save command thats setting off this error when the rule doesn't exist.
When i try on here its silently ignored
Code:
# Generated by iptables-save v1.4.14 on Mon Jan  9 21:41:39 2017


tOmsK@RT-AC68U-4690:/tmp/home/root# iptables-save | grep great-table
tOmsK@RT-AC68U-4690:/tmp/home/root#
 
Code:
iptables --version
iptables v1.4.14

same version on my router although the no chain might be cause it deletes the rule that doesnt exist

Code:
iptables-save | grep malware-filter > /dev/null 2>&1 || \
if [ $? -ne 0 ]; then
    iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi

is my proposal for the problem.
 
Code:
iptables --version
iptables v1.4.14

same version on my router although the no chain might be cause it deletes the rule that doesnt exist

Code:
iptables-save | grep malware-filter > /dev/null 2>&1 || \
if [ $? -ne 0 ]; then
    iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi

is my proposal for the problem.
yes this is possible.... if i directly delete the rule twice in the command line i get the same error
Code:
tOmsK@RT-AC68U-4690:/tmp/home/root# iptables -D FORWARD -m set --match-set malware-filter src,dst -j REJECT
tOmsK@RT-AC68U-4690:/tmp/home/root# iptables -D FORWARD -m set --match-set malware-filter src,dst -j REJECT
iptables: No chain/target/match by that name.

Doesn't quite explain how we never saw the error on ARM routers, where @shooter40sw saw it on his MIPS router. have to wait for him to give it a burst i guess...
 
Well here it is so anyone with a Mips can test see if it works.

Code:
#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile
# Testers: shooter40sw
# Revision 7

path=/opt/var/cache/malware-filter                      # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL=''   

     ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    ipset -N malware-filter $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice ipset $SYNTAX malware-filter $i ; done
fi
else
    ipset -N malware-update $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice ipset $SYNTAX malware-update $i ; done
    ipset $SWAPPED malware-update malware-filter
    ipset $DESTROYED malware-update
fi

iptables-save | grep malware-filter > /dev/null 2>&1 || \
if [ $? -ne 0 ]; then
    iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?
 
Hi guys, testing the rev 7 script

Code:
t/sda1/entware/var/cache/malware-filter# ./blockip.sh
/opt/var/cache/malware-filter/ma 100%[===========================================================>] 160.12K   279KB/s   in 0.6s
/opt/var/cache/malware-filter/ma     [ <=>                                                        ]   2.34K  --.-KB/s   in 0.006s
/opt/var/cache/malware-filter/ma     [ <=>                                                        ]  10.51K  57.7KB/s   in 0.2s
/opt/var/cache/malware-filter/ma 100%[===========================================================>]  17.00K  --.-KB/s   in 0.1s
/opt/var/cache/malware-filter/ma 100%[===========================================================>] 300.53K   261KB/s   in 1.1s
/opt/var/cache/malware-filter/ma 100%[===========================================================>]  10.34K  --.-KB/s   in 0.1s
system: Malware Filter loaded 35807 unique ip addresses.


Ipset -L

Code:
/tmp/mnt/sda1/entware/var/cache/malware-filter# ipset -L
Name: malware-filter
Type: iphash
References: 4
Header: hashsize: 199288 probes: 8 resize: 50
Members:
220.134.89.70
192.249.59.79
194.44.53.190
85.105.130.116
220.134.251.52
78.10.162.174

Now I had 4 entries in the iptables -L view with the ipset, I run this one 2 times and did not add the line again, the 4 entries because of the cru running everyday. Today I´ll be late home so maybe tomorrow I can boot the router and test it fresh.
This test was made with already 4 entries and no router reboot, Im connecting remotely

Is there a way to put this on the firewall script and run like 10 minutes after the router boot up?, I dont mind updating the ipset once a week or when I reboot the router
 
bahh atleast you dont get the error message you can delete the duplicates manually

Code:
iptables -D FORWARD -m set --set malware-filter src,dst -j REJECT

have to revise my code so it deletes the old entries i dont want to have it in firewall-start cause then it wont be updated rather have it as a cronjob
 
wonder if its the || \ thats not passing the string or if its the whole command

thing is under arm i have no issues it deletes and updates without issues but with the mips firmware it seems like swing and miss upon updates..

same script 2 different results every freakin time
 
Last edited:
wonder if its the || \ thats not passing the string or if its the whole command

thing is under arm i have no issues it deletes and updates without issues but with the mips firmware it seems like swing and miss upon updates..

same script 2 different results every freakin time

Im testing it again, after deleting manually the entries, its running for the second time, have to wait until it runs, but for now the first error did not appear , but nothing is certain until a reboot and check

EDIT:
With this new script it does not add the line again, only one line in the iptables, so I think you did it!, just need to reboot to test, thing that at this moment I cant do
 
Last edited:
Good im rewriting the iptables part so it secures that the rules arent in duplicate this script should check for the filter in the list if its present it deletes the chain and then adds it again so it cant be duplicated.


Code:
#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile
# Testers: shooter40sw
# Revision 7

path=/opt/var/cache/malware-filter                      # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL=''   

    ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset -N malware-filter $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice -n 2 ipset $SYNTAX malware-filter $i ; done
fi
else
    nice -n 2 ipset -N malware-update $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice ipset $SYNTAX malware-update $i ; done
    nice -n 2 ipset $SWAPPED malware-update malware-filter
    nice -n 2 ipset $DESTROYED malware-update
fi

iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?
 
Last edited:
Good im rewriting the iptables part so it secures that the rules arent in duplicate this script should check for the filter in the list if its present it deletes the chain and then adds it again so it cant be duplicated.


Code:
#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile
# Testers: shooter40sw
# Revision 7

path=/opt/var/cache/malware-filter                      # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL=''   

    ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice ipset -N malware-filter $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice ipset $SYNTAX malware-filter $i ; done
fi
else
    nice ipset -N malware-update $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice ipset $SYNTAX malware-update $i ; done
    nice ipset $SWAPPED malware-update malware-filter
    nice ipset $DESTROYED malware-update
fi

iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?
the nice commands will need a value.....
 
Good im rewriting the iptables part so it secures that the rules arent in duplicate this script should check for the filter in the list if its present it deletes the chain and then adds it again so it cant be duplicated.


Code:
#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile
# Testers: shooter40sw
# Revision 7

path=/opt/var/cache/malware-filter                      # Set your path here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL='' 

    ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset -N malware-filter $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice -n 2 ipset $SYNTAX malware-filter $i ; done
fi
else
    nice -n 2 ipset -N malware-update $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice ipset $SYNTAX malware-update $i ; done
    nice -n 2 ipset $SWAPPED malware-update malware-filter
    nice -n 2 ipset $DESTROYED malware-update
fi

iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?


Just tested runned the script two times after deleting manually the old one, and dit not duplicate, looking all good!
 
Good hope it ran a little bit lighter since nice was invoked in the script not using all CPU for the block filter
 
-n 2 good enough for older models ? or should i strangle the process more ?
Afraid i have a "dumbness" of -n19 in that knowledge area.... perhaps the experts can weigh in there.
 
Thanks for the link .... i actually meant that i don't have an idea of what value would be good to assign to an older MIPS router in this case, rather than being dumb about the nice command itself. Theres a nice (pun intended) piece about nice and renice here
http://www.thegeekstuff.com/2013/08/nice-renice-command-examples/
unless one of the forum experts is willing to suggest a ballpark figure that might provide a good tradeoff , then i guess the best thing to do is try couple of values and see what happens. No idea if a niceness of 2 would make a noticeable difference.
 
I had a chance to reboot the router at home for some minutes, but same error guys

Code:
# /opt/var/cache/malware-filter/blockip.sh
insmod: can't insert '/lib/modules/2.6.22.19/kernel/net/ipv4/netfilter/ip_set.ko                                                                                   ': File exists
/opt/var/cache/malw 100%[=====================>] 160.12K   277KB/s   in 0.6s
/opt/var/cache/malw     [ <=>                  ]   2.34K  --.-KB/s   in 0.005s
/opt/var/cache/malware-filter/m     [ <=>                                                    ]  10.51K  55.6KB/s   in 0.2s
/opt/var/cache/malware-filter/m 100%[=======================================================>]  17.00K  51.6KB/s   in 0.3s
/opt/var/cache/malware-filter/m 100%[=================================

The script slams the CPU for 5 minutes 100%, nothing new there

Everything else works ok

second run of the script, maintains only one entry of the script, so thats ok!
Thanks
 
I had a chance to reboot the router at home for some minutes, but same error guys

Code:
# /opt/var/cache/malware-filter/blockip.sh
insmod: can't insert '/lib/modules/2.6.22.19/kernel/net/ipv4/netfilter/ip_set.ko                                                                                   ': File exists
/opt/var/cache/malw 100%[=====================>] 160.12K   277KB/s   in 0.6s
/opt/var/cache/malw     [ <=>                  ]   2.34K  --.-KB/s   in 0.005s
/opt/var/cache/malware-filter/m     [ <=>                                                    ]  10.51K  55.6KB/s   in 0.2s
/opt/var/cache/malware-filter/m 100%[=======================================================>]  17.00K  51.6KB/s   in 0.3s
/opt/var/cache/malware-filter/m 100%[=================================

The script slams the CPU for 5 minutes 100%, nothing new there

Everything else works ok

second run of the script, maintains only one entry of the script, so thats ok!
Thanks
what do you get if you
Code:
lsmod | grep "ipt_set"
 
Result
Code:
mp/home/root# lsmod | grep "ipt_set"
ipt_set                 1504  1
ip_set                 16576  5 ipt_set,ip_set_iphash,ip_set_nethash
That means the modules are loading ok.... i had a small concern because i noticed the insmod error in your last post.
bear in mind that "nice" changes the processes priority , however if there isn't another process running with a more negative priority, it will still max out the cpu. Some light reading here if that kind of thing floats your boat
https://www.cyberciti.biz/tips/how-do-i-find-out-linux-cpu-utilization.html
 

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top