Menu
What's new
By:
Filters Better Search

Monitoring Web Usage with Asuswrt-Merlin

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

g3m1n1 said:
Thanks a lot for posting this! I do have one issue, I can't seem to run the showwebmon.sh script as a script. If I enter the commands from showwebmon.sh one line at a time, including the while loop, I do get the same output you show in your example. However, I am not able to execute the script itself.

Here is what I get when I try to run the script:
Code: 
admin01@RT-AC68U-2D18:/jffs/scripts# ls -l
-rwxr-xr-x    1 admin01  root           304 Mar 15 10:33 firewall-start
-rwxr-xr-x    1 admin01  root           448 Mar 15 11:38 showwebmon.sh
admin01@RT-AC68U-2D18:/jffs/scripts# showwebmon.sh
-sh: showwebmon.sh: not found
admin01@RT-AC68U-2D18:/jffs/scripts# /jffs/scripts/showwebmon.sh
-sh: /jffs/scripts/showwebmon.sh: not found
admin01@RT-AC68U-2D18:/jffs/scripts# sh showwebmon.sh
': No such file or directoryn_recent_domains
showwebmon.sh: line 12: syntax error: unexpected "done" (expecting "do")
admin01@RT-AC68U-2D18:/jffs/scripts#

I'm not sure what the problem is because the commands do work when typed in manually. In the router's config I have jffs enabled as well as "enable jffs custom scripts and configs". I must be overlooking something obvious. Any ideas?
Click to expand...

The execution path ($PATH) doesn't include the /jffs/scripts/ folder.

If you have /jffs/scripts/ as the current folder, try typing ./showwebmon.sh as that should work.

The error you got when using "sh showwebmon.sh" looks like a syntax error in the script itself; perhaps an extra line feed or carriage control. What editor did you use to create the script? What does the script look like when you "cat showwebmon.sh"?
 
Last edited:
ScottW said:
The execution path ($PATH) doesn't include the /jffs/scripts/ folder.

If you have /jffs/scripts/ as the current folder, try typing ./showwebmon.sh as that should work.

The error you got when using "sh showwebmon.sh" looks like a syntax error in the script itself; perhaps an extra line feed or carriage control. What editor did you use to create the script? What does the script look like when you "cat showwebmon.sh"?
Click to expand...

Hey ScottW, thanks for replying.

I used notepad++ to create the script file, then uploaded it with winscp, then used chmod to make it executable.
Code: 
admin01@RT-AC68U-2D18:/jffs/scripts# ./showwebmon.sh
-sh: ./showwebmon.sh: not found
admin01@RT-AC68U-2D18:/jffs/scripts# cat showwebmon.sh
#!/bin/sh
# Reformat the webmon output to convert epoch date to general date string
domains="/proc/webmon_recent_domains"
dsort="/tmp/dsort.tmp"
cat $domains | sort > $dsort
while read webdate webip hostname
do
  webdate=$(date -d @$webdate)
  #webdate=${webdate//GMT $(date +"%Y")/} # exclude year in output
  webdate=${webdate//GMT /}  # include year in output
  echo -e "$webdate \t$webip \t$hostname"
done < $dsort
rm $dsort
exit 0admin01@RT-AC68U-2D18:/jffs/scripts#
 
g3m1n1 said:
Hey ScottW, thanks for replying.

I used notepad++ to create the script file, then uploaded it with winscp, then used chmod to make it executable.
Code: 
admin01@RT-AC68U-2D18:/jffs/scripts# ./showwebmon.sh
-sh: ./showwebmon.sh: not found
admin01@RT-AC68U-2D18:/jffs/scripts# cat showwebmon.sh
#!/bin/sh
# Reformat the webmon output to convert epoch date to general date string
domains="/proc/webmon_recent_domains"
dsort="/tmp/dsort.tmp"
cat $domains | sort > $dsort
while read webdate webip hostname
do
  webdate=$(date -d @$webdate)
  #webdate=${webdate//GMT $(date +"%Y")/} # exclude year in output
  webdate=${webdate//GMT /}  # include year in output
  echo -e "$webdate \t$webip \t$hostname"
done < $dsort
rm $dsort
exit 0admin01@RT-AC68U-2D18:/jffs/scripts#
Click to expand...

If somehow it got into the wrong format cat won't show it....what happens if you open the script in vi? Any ^M characters?
 
john9527 said:
If somehow it got into the wrong format cat won't show it....what happens if you open the script in vi? Any ^M characters?
Click to expand...

Yep, that was the problem. Before I even checked the original scripts I saved in notepad++ I just re-wrote them in-shell using Vi and they worked perfectly. Checking the other scripts shows ^M at the end of every line. Kind of annoying, I figured notepad++ would account for that if the language was set to Shell. I even tried changing the encoding in notepad++ but that did not help either. Thanks for the excellent script example, btw.
 
This has been working great for me so far. Thanks a ton to everyone who contributed.

I was wondering if there is a way to use ipt_webmon and iptables to capture the full URL of websites or links instead of just the domain itself.
 
g3m1n1 said:
Yep, that was the problem. Before I even checked the original scripts I saved in notepad++ I just re-wrote them in-shell using Vi and they worked perfectly. Checking the other scripts shows ^M at the end of every line. Kind of annoying, I figured notepad++ would account for that if the language was set to Shell. I even tried changing the encoding in notepad++ but that did not help either. Thanks for the excellent script example, btw.
Click to expand...

you should look for settings in winscp to use ascii transfer mode, since it upload the script to router, not notepad++.
 
Ok, I need some help with a script for my Asus router.

Using the information in this thread, I have successfully set up the web activity monitor. I wrote a daily maintenance script which converts everything in /proc/webmon_recent_domains to a readable format, copies it to my USB drive, flushes the iptables and then reboots the router. My problem is, it's only doing the first two tasks. I can run the "shutdown" "init 6" or "halt" command from the shell and it works just fine. I can also run "iptables --flush" without issue. Running them from the script is not working. Can someone please steer me in the right direction?
 
SullyTheUnusual said:
Ok, I need some help with a script for my Asus router.

Using the information in this thread, I have successfully set up the web activity monitor. I wrote a daily maintenance script which converts everything in /proc/webmon_recent_domains to a readable format, copies it to my USB drive, flushes the iptables and then reboots the router. My problem is, it's only doing the first two tasks. I can run the "shutdown" "init 6" or "halt" command from the shell and it works just fine. I can also run "iptables --flush" without issue. Running them from the script is not working. Can someone please steer me in the right direction?
Click to expand...
Is there a particular reason you are doing a flush/reboot (doing a reboot will reset the tables)? If you post your script, we may be able to help out.
 
john9527 said:
Is there a particular reason you are doing a flush/reboot (doing a reboot will reset the tables)? If you post your script, we may be able to help out.
Click to expand...
After converting/logging everything in /proc/webmon_recent_domains, I want to wipe the contents so I don't get duplicates the next time I append to my logs. Since this isn't a real file, I was trying to use the "iptables --flush" and then reboot to accomplish it.

The two lines in my "firewall-start" script that aren't working are:

iptables --flush
reboot

I've tried using the full path and sudo to no avail.

If you know of an easier way to wipe the contents of "webmon_recent_domains", that would be extremely useful to me. I'm not well versed with the iptables command.
 
I'm working on something else right now, so can't test it....but try this....

iptables -t filter -I web_monitor -p tcp -m webmon --clear_search --clear_domain
iptables -t filter -D web_monitor -p tcp -m webmon --clear_search --clear_domain

You may need to add a sleep command between the lines to give it time to process.
 
Here are the commands I used to enable web monitoring:
Code: 
iptables -t filter -N web_monitor
iptables -t filter -I FORWARD -o eth0 -j web_monitor
iptables -A web_monitor -p tcp -m webmon --max_domains 2000 --max_searches 2000

I ran the first command you suggested, and it did clear the contents of /proc/webmon_recent_domains. I ran the second command and got "iptables: Bad rule (does a matching rule exist in that chain?)".

Here's what "iptables --list" returns after running your commands:
Code: 
Chain INPUT (policy ACCEPT)
target  prot opt source  destination 
DROP  all  --  anywhere  anywhere  state INVALID
ACCEPT  all  --  anywhere  anywhere  state RELATED,ESTABLISHED
ACCEPT  all  --  anywhere  anywhere  state NEW
ACCEPT  all  --  anywhere  anywhere  state NEW
ACCEPT  udp  --  anywhere  anywhere  udp spt:bootps dpt:bootpc
DROP  all  --  anywhere  anywhere 

Chain FORWARD (policy DROP)
target  prot opt source  destination 
web_monitor  all  --  anywhere  anywhere 
ipttolan  all  --  anywhere  anywhere 
iptfromlan  all  --  anywhere  anywhere 
ACCEPT  all  --  anywhere  anywhere  state RELATED,ESTABLISHED
DROP  all  --  anywhere  anywhere 
DROP  all  --  anywhere  anywhere  state INVALID
ACCEPT  all  --  anywhere  anywhere 
DROP  icmp --  anywhere  anywhere 
ACCEPT  all  --  anywhere  anywhere  ctstate DNAT
ACCEPT  all  --  anywhere  anywhere 

Chain OUTPUT (policy ACCEPT)
target  prot opt source  destination 

Chain FUPNP (0 references)
target  prot opt source  destination 
ACCEPT  udp  --  anywhere  PQ  udp dpt:55523
ACCEPT  udp  --  anywhere  Aimee-Laptop  udp dpt:56216
ACCEPT  udp  --  anywhere  PQ  udp dpt:58499
ACCEPT  udp  --  anywhere  PQ  udp dpt:62999
ACCEPT  udp  --  anywhere  PQ  udp dpt:56717

Chain PControls (0 references)
target  prot opt source  destination 
ACCEPT  all  --  anywhere  anywhere 

Chain iptfromlan (1 references)
target  prot opt source  destination 
RETURN  all  --  anywhere  anywhere  account: network/netmask: 192.168.1.0/255.255.255.0 name: lan

Chain ipttolan (1 references)
target  prot opt source  destination 
RETURN  all  --  anywhere  anywhere  account: network/netmask: 192.168.1.0/255.255.255.0 name: lan

Chain logaccept (0 references)
target  prot opt source  destination 
LOG  all  --  anywhere  anywhere  state NEW LOG level warning tcp-sequence tcp-options ip-options prefix `ACCEPT '
ACCEPT  all  --  anywhere  anywhere 

Chain logdrop (0 references)
target  prot opt source  destination 
LOG  all  --  anywhere  anywhere  state NEW LOG level warning tcp-sequence tcp-options ip-options prefix `DROP '
DROP  all  --  anywhere  anywhere 

Chain web_monitor (1 references)
target  prot opt source  destination 
  tcp  --  anywhere  anywhere  WEBMON --max_domains 300 --max_searches 300
  tcp  --  anywhere  anywhere  WEBMON --max_domains 2000 --max_searches 2000

If you look at the last chain, it looks like it added another entry with 300 as the max domains/searches. That must be the default setting after it clears. I tried deleting the rule with "iptables -t filter -D web_monitor -p tcp -m webmon --clear_search --clear_domain" but it threw "bad rule".

I tried the same command, replacing "--clear_search --clear_domain" with "--max_domains 300 --max_searches 300", but it still wouldn't remove it and thew "bad rule".

I wonder if "iptables --flush web_monitor" would work? I'm at work right now, maybe I'll try that when I get home.

Hope the extra info helps, thanks for you time!
 
Last edited:
SullyTheUnusual said:
I ran the first command you suggested, and it did clear the contents of /proc/webmon_recent_domains. I ran the second command and got "iptables: Bad rule (does a matching rule exist in that chain?)".
Click to expand...
SullyTheUnusual said:
I wonder if "iptables --flush web_monitor" would work? I'm at work right now, maybe I'll try that when I get home.
Click to expand...
Half way there....it did clear it!:)

Try this sequence (assuming you made the firewall-start script with the first set of startup commands)

iptables -t filter -I web_monitor -p tcp -m webmon --clear_search --clear_domain
service restart_firewall
 
john9527 said:
Half way there....it did clear it!:)

Try this sequence (assuming you made the firewall-start script with the first set of startup commands)

iptables -t filter -I web_monitor -p tcp -m webmon --clear_search --clear_domain
service restart_firewall
Click to expand...

Running those two commands at the shell prompt works brilliantly! However, they will not execute via script. I have even tried supplying the full path to "iptables" and "service".
 
SullyTheUnusual said:
Running those two commands at the shell prompt works brilliantly! However, they will not execute via script. I have even tried supplying the full path to "iptables" and "service".
Click to expand...
90% there :)

How are you editing your script? Double check by opening the script on the router in vi and check that the lines aren't corrupted (contain a ^M character).
 
Had to ask the easy question first.....

How are you calling your script? Are you using Merlin's scripting capability? with a cron job? I know both iptables and service commands work in scripts as I've used them. We may be at the point where you need to post your script to get some additional eyes on it.
 
john9527 said:
Had to ask the easy question first.....

How are you calling your script? Are you using Merlin's scripting capability? with a cron job? I know both iptables and service commands work in scripts as I've used them. We may be at the point where you need to post your script to get some additional eyes on it.
Click to expand...

Yes, I'm using a cron to execute my daily.sh script. Everything in the script runs fine except the "iptables" and "service" commands.

Here's my daily.sh script:
Code: 
#!/bin/sh
# Reformat the webmon output to convert epoch date to general date string
domains="/proc/webmon_recent_domains"
dsort="/tmp/dsort.tmp"
cat $domains | sort > $dsort

while read webdate webip hostname
do
  webdate=$(date -d @$webdate)
  #webdate=${webdate//GMT $(date +"%Y")/} # exclude year in output
  webdate=${webdate//GMT /}  # include year in output
  echo -e "$webdate \t$webip \t$hostname" >> /tmp/mnt/sda1/logs/domain.log
  done < $dsort
  rm $dsort
  exit 0

# Clear webmon filter

iptables -t filter -I web_monitor -p tcp -m webmon --clear_search --clear_domain
sleep 3
service restart_firewall

Here's my firewall-start script:
Code: 
#!/bin/sh

# Enable webmon monitoring
  if [ -z "$(iptables -n -L FORWARD | grep 'web_monitor')" ]; then
  modprobe ipt_webmon
  iptables -t filter -N web_monitor
  iptables -t filter -I FORWARD -o eth0 -j web_monitor
  iptables -A web_monitor -p tcp -m webmon --max_domains 2000 --max_searches 2000
  fi

# Add crons

cru a dailymaint "0 0 * * *" /jffs/scripts/daily.sh
 
SullyTheUnusual said:
Yes, I'm using a cron to execute my daily.sh script. Everything in the script runs fine except the "iptables" and "service" commands.

Here's my daily.sh script:
Code: 
#!/bin/sh
# Reformat the webmon output to convert epoch date to general date string
domains="/proc/webmon_recent_domains"
dsort="/tmp/dsort.tmp"
cat $domains | sort > $dsort
 
while read webdate webip hostname
do
  webdate=$(date -d @$webdate)
  #webdate=${webdate//GMT $(date +"%Y")/} # exclude year in output
  webdate=${webdate//GMT /}  # include year in output
  echo -e "$webdate \t$webip \t$hostname" >> /tmp/mnt/sda1/logs/domain.log
  done < $dsort
  rm $dsort
  exit 0
 
# Clear webmon filter
 
iptables -t filter -I web_monitor -p tcp -m webmon --clear_search --clear_domain
sleep 3
service restart_firewall
Click to expand...

You need to take the 'exit 0' out of the script segment I posted :)
 
Hello All
As I wrote in the main fork thread i tried everything to get webmon workin on my RT-N66U
Still unable to find a reason boecause it could not work
It runs smotly without any problem either starting in firewall- start either entering commands manually
I disabled very "optional" service on my router, disabled every possible startup of packages.
NAT i disabled. VPN client is configured but not active
But whatever I try, i always get no data..

cat /proc/webmon_recent_domains | wc -l
always returns 0

Here it is my last IPTABLES list, i cannot see anything wrong there but if someone could take a look please let me know if you see a possible reason for webmon not working

Code: 
Chain INPUT (policy ACCEPT)
target     prot opt source               destination      
ACCEPT     all  --  anywhere             anywhere         
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:https
ACCEPT     all  --  anywhere             anywhere         
ACCEPT     udp  --  anywhere             anywhere            udp dpt:https
logdrop    all  --  anywhere             anywhere            state INVALID
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere            state NEW
ACCEPT     all  --  anywhere             anywhere            state NEW
ACCEPT     udp  --  anywhere             anywhere            udp spt:bootps dpt:bootpc
logdrop    all  --  anywhere             anywhere         

Chain FORWARD (policy DROP)
target     prot opt source               destination      
monitor    all  --  anywhere             anywhere         
ipttolan   all  --  anywhere             anywhere         
iptfromlan  all  --  anywhere             anywhere         
TCPMSS     tcp  --  anywhere             anywhere            tcp flags:SYN,RST/SYN TCPMSS clamp to PMTU
ACCEPT     all  --  anywhere             anywhere         
ACCEPT     all  --  anywhere             anywhere         
logdrop    all  --  anywhere             anywhere            state INVALID
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
logdrop    all  --  anywhere             anywhere         
ACCEPT     all  --  anywhere             anywhere         
DROP       icmp --  anywhere             anywhere         
ACCEPT     tcp  --  anywhere             anywhere            tcp flags:FIN,SYN,RST,ACK/SYN limit: avg 1/sec burst 5
ACCEPT     tcp  --  anywhere             anywhere            tcp flags:FIN,SYN,RST,ACK/RST limit: avg 1/sec burst 5
ACCEPT     icmp --  anywhere             anywhere            icmp echo-request limit: avg 1/sec burst 5
ACCEPT     all  --  anywhere             anywhere            ctstate DNAT
ACCEPT     all  --  anywhere             anywhere         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination      

Chain FUPNP (0 references)
target     prot opt source               destination      

Chain PControls (0 references)
target     prot opt source               destination      
ACCEPT     all  --  anywhere             anywhere         

Chain iptfromlan (1 references)
target     prot opt source               destination      
RETURN     all  --  anywhere             anywhere            account: network/netmask: 192.168.50.0/255.255.255.0 name: lan
RETURN     all  --  anywhere             anywhere            account: network/netmask: 192.168.50.0/255.255.255.0 name: lan
RETURN     all  --  anywhere             anywhere            account: network/netmask: 192.168.50.0/255.255.255.0 name: lan

Chain ipttolan (1 references)
target     prot opt source               destination      
RETURN     all  --  anywhere             anywhere            account: network/netmask: 192.168.50.0/255.255.255.0 name: lan
RETURN     all  --  anywhere             anywhere            account: network/netmask: 192.168.50.0/255.255.255.0 name: lan
RETURN     all  --  anywhere             anywhere            account: network/netmask: 192.168.50.0/255.255.255.0 name: lan

Chain logaccept (0 references)
target     prot opt source               destination      
LOG        all  --  anywhere             anywhere            state NEW LOG level warning tcp-sequence tcp-options ip-options prefix `ACCEPT '
ACCEPT     all  --  anywhere             anywhere         

Chain logdrop (4 references)
target     prot opt source               destination      
LOG        all  --  anywhere             anywhere            state NEW LOG level warning tcp-sequence tcp-options ip-options prefix `DROP '
DROP       all  --  anywhere             anywhere         

Chain monitor (1 references)
target     prot opt source               destination      
LOG        all  --  anywhere             anywhere            state NEW LOG level info prefix `webmon'
           tcp  --  anywhere             anywhere            WEBMON --max_domains 2000 --max_searches 2000

Thanks for your attention and for any help

BR
ocraM
 
Last edited:
You must log in or register to reply here.

Similar threads

Meetoul
Beta [Fork] Asuswrt-Merlin for RT-AX58U_V2
2
Replies
33
Views
10K
Tech9
Tech9
JGrana
uKasa uKasa - A utility script that manages Kasa smart outlets and switches with Asuswrt-Merlin routers
2
Replies
29
Views
2K
L&LD
L&LD
orcan
AiMesh Configuration With Asuswrt-Merlin 3.2
Replies
2
Views
2K
Adooni
A
garycnew
[RESOLVED] Asuswrt-Merlin 384.19 AiMesh Ethernet Backhaul Connectivity Issues
Replies
10
Views
3K
garycnew
garycnew
RMerlin
  • Locked
Release Asuswrt-Merlin 386.1 is now available
68 69 70
Replies
1K
Views
321K
RMerlin
RMerlin
Similar threads
Thread starter Title Forum Replies Date
v.y.k Traffic Monitoring - Save frequency - issue Asuswrt-Merlin 7
U Solved Cannot upload self-signed cert for web UI Asuswrt-Merlin 6
M Allow admin web GUI access on another interface (tailscale0) Asuswrt-Merlin 0
B Solved Client list in Network Map is empty and changing the System Log verbosity in the web GUI does nothing Asuswrt-Merlin 7
L When I access the System Log --> IPv6 tab the web server crashes Asuswrt-Merlin 0
E Web proxy running locally on ASUS Router Asuswrt-Merlin 2
A Enable Web Access from WAN can't stay enabled Asuswrt-Merlin 2
G VPN Client with Public IP, split tunnel web server over VPN and other traffic not on the VPN Asuswrt-Merlin 4
K The dreaded "HTTPS is a relatively more secured web access protocol. We strongly suggest to enter the ASUS router setting page via HTTPS protocol." Asuswrt-Merlin 16
D Access offline devices list from Web UI Asuswrt-Merlin 5

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!

Members online

Total: 964 (members: 21, guests: 943)
Top