Menu
What's new
By:
Filters Better Search

Malware Filter / bad host IPSET

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

Results as follows: RT-AC87R Entware installed.

system: Malware Filter loaded 23919 unique ip addresses.
real 2m 13.30s
user 2m 0.46s
sys 1m 56.97s
 
swetoast said:
So ive been running test since the new revision and 30000+ takes around 5~ min on a RT-AC56U kinda curious to hear results from others. I am interested in both routers with and without entware installed.

just do this command
Code: 
time malware-block
and when the script ends it prints out 3 stats i used the "real" as measurement for my tests so please post avg time and router model and if entware was present.
Click to expand...

Hi, I have not installed this version 10, I have 9 installed, that commando does not work for me, but the script is working and the ipset full.
Code: 
/cache/malware-filter# time malware-block
time: can't execute 'malware-block': No such file or directory
Command exited with non-zero status 127
real    0m 0.00s
user    0m 0.00s
sys     0m 0.00s

And that is the name of the script, yesterday I factory reset the router and configured everything from scratch
regards
 
The "time" command did not work. I then installed the entware package time:

Code: 
opkg install time

My results are as follows:

Code: 
system: Malware Filter loaded 35002 unique ip addresses.
159.67user 107.07system 2:48.78elapsed 158%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+0minor)pagefaults 0swaps

I reran as follows to make the output easier to read:
Code: 
time -f "\t%U user,\t%S system,\t%E elasped, \t%e real" malware-block

And the output is:
Code: 
system: Malware Filter loaded 34997 unique ip addresses.
        161.19 user,    105.15 system,  2:45.65 elasped,        165.65 real



 
Last edited:
I have an RT-AC68U (800mhz CPU), no entware installed. Script version 9.
Running the time /jffs/scripts/malware-block manually reports:
Real: 6m 24.48s Loaded 34,977 IP addresses.

I've been running a slightly modified script that captures the begin/end times and records the total execution seconds in the system log. Here are some states from previous log entries.
40245 loaded 7min 33sec (453sec)
38990 loaded 7min 16sec (436sec)
32267 loaded 6min 15 sec (375sec)
32094 loaded 5min 52 sec (352sec)
31188 loaded 5min 45 sec (345sec)
35016 loaded 6min 21 sec (381sec)
 
swetoast said:
So ive been running test since the new revision and 30000+ takes around 5~ min on a RT-AC56U kinda curious to hear results from others. I am interested in both routers with and without entware installed.

just do this command
Code: 
time malware-block
and when the script ends it prints out 3 stats i used the "real" as measurement for my tests so please post avg time and router model and if entware was present.
Click to expand...
Around 05:10 for both RT-N66R (single-core MIPS) and RT-AC68U (dual-core ARM).
 
Tnx everyone for reporting your stats gonna put up a list on load times on the wiki based on all your information so that new users gets a sense on how long the load time takes for the script.
 
This is a wonderful script. Have a suggestion for consideration. I noticed when the script updates, it resets the iptables statistics, so you don't know if any devices attempted to reach those blocked IPs.

Code: 
iptables -vnL FORWARD

Chain FORWARD (Policy DROP 0 packets, 0 bytes)
pkts    bytes  target  Prot opt in out source     destination
    0       0  DROP    ALL  --  *  *   0.0.0.0/0  0.0.0.0/0      match-set  malware-filter src

I'm still a beginner with Linux, but learning by searching the internet. I've modified a script merlin posted for firmware update, so it will send a email if devices do attempt to reach the malware blocked ports between malware updates. It checks the iptables packet count and if it does not = 0, then sends the email.

malware-email
Code: 
#!/bin/sh

if [ "$(iptables -vnL FORWARD | grep -w 'malware-filter' | cut -c1-5)" != "    0" ]; then

   # SMTP parameters
   SMTP="smtp server address ie smtp.outlook.com"
   PORT="465"
   USERNAME="email account"
   PASSWORD="email password"

   # Mail Enveloppe
   FROM_NAME="Router"
   FROM_ADDRESS="from email address"
   TO_NAME="name"
   TO_ADDRESS="to email address"


   ### Do not change below
   echo "From: \"$FROM_NAME\" <$FROM_ADDRESS>" > /tmp/mail.txt
   echo "To: \"$TO_NAME\" <$TO_ADDRESS>" >> /tmp/mail.txt
   echo "Subject: Malware filter block results" >> /tmp/mail.txt
   echo "" >> /tmp/mail.txt
   iptables -vnL FORWARD 1 >> /tmp/mail.txt

   curl --url smtps://$SMTP:$PORT \
     --mail-from "$FROM_ADDRESS" --mail-rcpt "$TO_ADDRESS" \
     --upload-file /tmp/mail.txt \
     --ssl-reqd \
     --user "$USERNAME:$PASSWORD" --insecure

   rm /tmp/mail.txt

fi

It can be scheduled to run separately from the malware-block, however I've added the below line in malware-block; above the first case statement, so it runs just before it updates the ipset.

Code: 
$path/malware-email
 
Does this work for non entware xargs?.... shame for those folks to miss out.
Code: 
#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile, jimf, spalife
# Testers: shooter40sw
# Revision 10

path=/opt/var/cache/malware-filter                      # Set your path here
retries=3                                               # Set number of tries here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL=''

    ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --tries=$retries --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

echo "adding ipset rule to firewall this will take time."

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset -N malware-filter $HASH $OPTIONAL
    cat $path/malware-filter.txt | xargs -P10 -I {} ipset $SYNTAX malware-filter {}
fi
else
    nice -n 2 ipset -N malware-update $HASH $OPTIONAL
    cat $path/malware-filter.txt | xargs -P10 -I {} ipset $SYNTAX malware-update {}
    nice -n 2 ipset $SWAPPED malware-update malware-filter
    nice -n 2 ipset $DESTROYED malware-update
fi

iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?
 
no it doesnt and thats nothing i can do anything about :( so either install entware or do it the old slow way.
 
swetoast said:
no it doesnt and thats nothing i can do anything about :( so either install entware or do it the old slow way.
Click to expand...
Oh well... the busybox xargs must be pretty stripped down..... thats really basic usage :(

Maybe the busybox version just doesn't like the -P max-procs option. If thats the case we could just omit the -P10 from the xargs line and optionally use that instead of iterating through the for i loop for the s-l-o-o-o-w version ( maybe a "little" faster if it works)

Code: 
cat $path/malware-filter.txt | xargs -I {} ipset $SYNTAX malware-filter {}
Replaces
for i in `cat $path/malware-filter.txt`; do ipset $SYNTAX malware-filter $i ; done
And
cat $path/malware-filter.txt | xargs -I {} ipset $SYNTAX malware-update {}
Replaces
for i in `cat $path/malware-filter.txt`; do ipset $SYNTAX malware-update $i ; done
 
Last edited:
swetoast said:
cool ill review and test and if its good ill merge :) creds as usual :)
Click to expand...
You may want to compare the speed for piping cat through ipset in entware with the -P10 option against the existing version too, it seemed pretty spiffy for me when i tried.
 
Revision 11 is up

huge tnx to tomsk for finding a way to speed up installations without entware

Code: 
#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile, jimf, spalife
# Testers: shooter40sw
# Revision 11

path=/opt/var/cache/malware-filter                      # Set your path here
retries=3                                               # Set number of tries here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL=''

    ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --tries=$retries --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

echo "adding ipset rule to firewall this will take time."

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset -N malware-filter $HASH $OPTIONAL
    if [ -f /opt/bin/xargs ]; then
    /opt/bin/xargs -P10 -I "PARAM" -n1 -a $path/malware-filter.txt nice -n 2 ipset $SYNTAX malware-filter PARAM
    else cat $path/malware-filter.txt | xargs -I {} ipset $SYNTAX malware-filter {}; fi
fi
else
    nice -n 2 ipset -N malware-update $HASH $OPTIONAL
    if [ -f /opt/bin/xargs ]; then
    /opt/bin/xargs -P10 -I "PARAM" -n1 -a $path/malware-filter.txt nice -n 2 ipset $SYNTAX malware-update PARAM
    else cat $path/malware-filter.txt | xargs -I {} ipset $SYNTAX malware-update {}; fi
    nice -n 2 ipset $SWAPPED malware-update malware-filter
    nice -n 2 ipset $DESTROYED malware-update
fi

iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?
 
Rev. 12 is up

Changelog: added local lan ip cleaner, too ensure that local lan ips don't get banned

Code: 
#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile, jimf, spalife
# Testers: shooter40sw
# Revision 12

path=/opt/var/cache/malware-filter                      # Set your path here
retries=3                                               # Set number of tries here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL=''

    ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --tries=$retries --show-progress -i $path/malware-filter.list -O $path/malware-list.tmp
        awk '!/(^127\.)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)/'  $path/malware-list.tmp > $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
        if [ -f $path/malware-list.tmp ]; then rm $path/malware-list.tmp; fi
        if [ -f $path/malware-list.pre ]; then rm $path/malware-list.pre; fi
 }

run_ipset () {

get_list

echo "adding ipset rule to firewall this will take time."

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset -N malware-filter $HASH $OPTIONAL
    if [ -f /opt/bin/xargs ]; then
    /opt/bin/xargs -P10 -I "PARAM" -n1 -a $path/malware-filter.txt nice -n 2 ipset $SYNTAX malware-filter PARAM
    else cat $path/malware-filter.txt | xargs -I {} ipset $SYNTAX malware-filter {}; fi
fi
else
    nice -n 2 ipset -N malware-update $HASH $OPTIONAL
    if [ -f /opt/bin/xargs ]; then
    /opt/bin/xargs -P10 -I "PARAM" -n1 -a $path/malware-filter.txt nice -n 2 ipset $SYNTAX malware-update PARAM
    else cat $path/malware-filter.txt | xargs -I {} ipset $SYNTAX malware-update {}; fi
    nice -n 2 ipset $SWAPPED malware-update malware-filter
    nice -n 2 ipset $DESTROYED malware-update
fi

iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?
 
Hello to everybody.
First of all I apologize for my poor english, well a have an RT-AC68U with arm processor at 800Mhz, I have installed merlin 380.65, and after I have installed entware; then I tried to run the script object of this topic.
I cut and paste the version that is in the wiki, but I'm not able to run the script corrctly.
This is the error i get from ssh interface:
Code: 
./malware-block.sh
ipset v4.5: Kernel ip_set module is of protocol version 6.I'm of protocol version 4.
Please upgrade your kernel and/or ipset(8) utillity.
insmod: can't insert '/lib/modules/2.6.36.4brcmarm/kernel/net/netfilter/ipset/ip_set.ko': File exists
insmod: can't insert 'ip_set_nethash.ko': No such file or directory
insmod: can't insert 'ip_set_iphash.ko': No such file or directory
insmod: can't insert 'ipt_set.ko': No such file or directory
/opt/var/cache/malware-filter/malware-list.tmp       100%[=====================================================================================================================>] 160.31K   363KB/s   in 0.4s   
/opt/var/cache/malware-filter/malware-list.tmp           [ <=>                                                                                                                  ]   2.12K  --.-KB/s   in 0s     
/opt/var/cache/malware-filter/malware-list.tmp           [ <=>                                                                                                                  ]  10.88K  --.-KB/s   in 0.05s 
/opt/var/cache/malware-filter/malware-list.tmp       100%[=====================================================================================================================>]  15.20K  94.8KB/s   in 0.2s   
/opt/var/cache/malware-filter/malware-list.tmp       100%[=====================================================================================================================>]  27.24K  --.-KB/s   in 0.1s   
/opt/var/cache/malware-filter/malware-list.tmp       100%[=====================================================================================================================>]  58.11K  --.-KB/s   in 0.04s 
/opt/var/cache/malware-filter/malware-list.tmp       100%[=====================================================================================================================>]  19.27K  --.-KB/s   in 0.001s
adding ipset rule to firewall this will take time.
--set option deprecated, please use --match-set
iptables v1.4.14: Set malware-filter doesn't exist.

Try `iptables -h' or 'iptables --help' for more information.
system: Malware Filter loaded 19831
unique ip addresses.
I understood that the problem is th version of ipset, but I'm not able to correct the problem. Does anybody help me?
 
myname said:
Hello to everybody.
First of all I apologize for my poor english, well a have an RT-AC68U with arm processor at 800Mhz, I have installed merlin 380.65, and after I have installed entware; then I tried to run the script object of this topic.
I cut and paste the version that is in the wiki, but I'm not able to run the script corrctly.
This is the error i get from ssh interface:
Code: 
./malware-block.sh
ipset v4.5: Kernel ip_set module is of protocol version 6.I'm of protocol version 4.
Please upgrade your kernel and/or ipset(8) utillity.
insmod: can't insert '/lib/modules/2.6.36.4brcmarm/kernel/net/netfilter/ipset/ip_set.ko': File exists
insmod: can't insert 'ip_set_nethash.ko': No such file or directory
insmod: can't insert 'ip_set_iphash.ko': No such file or directory
insmod: can't insert 'ipt_set.ko': No such file or directory
/opt/var/cache/malware-filter/malware-list.tmp       100%[=====================================================================================================================>] 160.31K   363KB/s   in 0.4s 
/opt/var/cache/malware-filter/malware-list.tmp           [ <=>                                                                                                                  ]   2.12K  --.-KB/s   in 0s   
/opt/var/cache/malware-filter/malware-list.tmp           [ <=>                                                                                                                  ]  10.88K  --.-KB/s   in 0.05s
/opt/var/cache/malware-filter/malware-list.tmp       100%[=====================================================================================================================>]  15.20K  94.8KB/s   in 0.2s 
/opt/var/cache/malware-filter/malware-list.tmp       100%[=====================================================================================================================>]  27.24K  --.-KB/s   in 0.1s 
/opt/var/cache/malware-filter/malware-list.tmp       100%[=====================================================================================================================>]  58.11K  --.-KB/s   in 0.04s
/opt/var/cache/malware-filter/malware-list.tmp       100%[=====================================================================================================================>]  19.27K  --.-KB/s   in 0.001s
adding ipset rule to firewall this will take time.
--set option deprecated, please use --match-set
iptables v1.4.14: Set malware-filter doesn't exist.

Try `iptables -h' or 'iptables --help' for more information.
system: Malware Filter loaded 19831
unique ip addresses.
I understood that the problem is th version of ipset, but I'm not able to correct the problem. Does anybody help me?
Click to expand...
AC68U on 380.65 should be using ipset V6.... if you just type ipset at the command line what does it say?

And just to be sure of the script.... what is the version number from the wiki...you may have an old version from somewhere.
 
Last edited:
Thaks for your answer, the vesionof ipset is
Code: 
admin@RT-AC68U:/tmp/home/root# ipset
ipset v4.5: no command specified
Try `ipset -H' or 'ipset --help' for more information.
And the versione of the script from the wiki is revision 13
 
An other information, if give the command
Code: 
admin@RT-AC68U:/tmp/mnt/sda1/entware-ng.arm/bin# ipset -v
ipset v4.5, protocol version 4.
ipset v4.5: Kernel ip_set module is of protocol version 6.I'm of protocol version 4.
Please upgrade your kernel and/or ipset(8) utillity.
this is the result
 
Something a bit strange going on here.... the malware filter script is trying to load the ipset modules for version 4 because it is testing for it in the same way. However i believe Merlin has built 380.65 with V6 . You may have to do a factory reset to clear all your old nvram values if you have come from a much older build.

Im on the same hardware/firmware as you but....
Code: 
ipset v6.29, protocol version: 6
 
You must log in or register to reply here.

Similar threads

A
Policy-based port routing to bypass NordVPN client
Replies
10
Views
1K
Don->
D
K
Can't get SNAT/MASQ to work.
2
Replies
24
Views
2K
eibgrad
eibgrad
R
Script to detect backup wan and disable certain devices from internet access
Replies
0
Views
333
RandomUser777
R
M
Firewall doesen't block custom rules, neighter Skynet
Replies
0
Views
1K
MamaLing
M
M
Entware Guide to Adding 2FA to Incoming Ports on AX86U
Replies
6
Views
863
ColinTaylor
C
Similar threads
Thread starter Title Forum Replies Date
Swistheater Malware /jffs/updater script. Asuswrt-Merlin 167
R Wireless MAC (Band filter filtering out Guest network) Asuswrt-Merlin 2
G Feature request: firewall url filter log Asuswrt-Merlin 0
TanyaC cannot add domain to router URL filter Asuswrt-Merlin 15
C How to filter out foreign countries Asuswrt-Merlin 3
D Network Services Filter - rules don't persist? Asuswrt-Merlin 13
Nas.CloudBusinessPortal Network Services Filter Asuswrt-Merlin 0
P Inbound OpenVPN Server IP address range filter Asuswrt-Merlin 2
HOgi Manage WLAN-MAC-Filter list via command line (ssh) in a file? Asuswrt-Merlin 3
J Allowing just one device to subnet (iptables or Network Services Filter?) Asuswrt-Merlin 0

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!

Members online

Total: 700 (members: 19, guests: 681)
Top