What's new

uiDivStats uiDivStats - WebUI for Diversion statistics

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

I'm not sure exactly why but uiDivStats increases the CPU usage dramatically during high usage of dnsmasq, which means during heavy web surfing between multiple web pages but I guess this is expected but I'm not sure this is caused by uiDivstats directly.

This is with uiDivStats;

View attachment 24096

This is without;
View attachment 24097

maybe this is because of USB Stick write processes of uiDivStats I'm not sure but this is what I see.

On the other side; I can't feel any problems like web pages stuck or any other slowness.
Please provide screenshots/outputs of:
Code:
atop #install from entware opkg install atop
htop #install from entware opkg install htop
df -Th | grep mnt | grep dev
 
I'm not sure exactly why but uiDivStats increases the CPU usage dramatically during high usage of dnsmasq, which means during heavy web surfing between multiple web pages but I guess this is expected but I'm not sure this is caused by uiDivstats directly.

This is with uiDivStats;

View attachment 24096

This is without;
View attachment 24097

maybe this is because of USB Stick write processes of uiDivStats I'm not sure but this is what I see.

On the other side; I can't feel any problems like web pages stuck or any other slowness.
I have security systems constantly chatting and i do not experience cpu issues. how do you have your diversion setup?
 
Please provide screenshots/outputs of:
Code:
atop #install from entware opkg install atop
htop #install from entware opkg install htop
df -Th | grep mnt | grep dev

Code:
admin@FaTiii:/tmp/home/root# df -Th | grep mnt | grep dev
/dev/sda1            ext2           14.7G      1.3G     12.6G   9% /tmp/mnt/entware

https://imgur.com/a/Iy8jmHd

atop is too complex for me :D but as I check htop I saw taildns and pixelserv-tls at the top most of the time, and this is when I try to access many pages in a couple of seconds.
 
how do you have your diversion setup?

Code:
Diversion 4.1.12                 by thelonelycoder

 RT-AC88U (armv7l) FW-384.17 @ 172.24.5.1

 965,296  blocked domains by  2  hosts file(s)
 356,841 t  14,528 w  10 n ads since Jun 17 08:26

 /mnt/entware | Size 14.7G | Used 1.3G (9%)         

 d   Diversion Standard   enabled
 c   communication        DivUn stats backup FWun

 a   ad-blocking          to IP 172.24.5.254
 l   logging              dnsmasq.log 25.5M

 ep  pixelserv-tls        172.24.5.254 v2.3.1

 b   blocking list        Large Mon @ 1:00
 el  edit lists            210 w  22 b  0 wb  936 yt

 f   follow dnsmasq.log

 ac  update ads counter   Auto-update  au  off
 ds  dnsmasq settings     on
 sw  manage swap file     /mnt/entware 1.0G
 ct  change theme colors   green

 sh  shell history        on
 cj  show cron job(s)     
 sf  show file content   

 e   exit  u  update                less options  o

Code:
Dnsmasq settings:

 See the Dnsmasq man page what these options do:
 http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html

 1. Disable Dnsmasq settings
 2. log-async         5
 3. cache-size        1500
 4. bogus-priv        on
 5. domain-needed     on
 6. log-queries=extra

 Enter selection [1-6 e=Exit]

Disabling Dnsmasq settings does not change anything. I'm using Sandisk Extreme 16GB USB3.0 in USB 2.0 port.
 
Code:
Diversion 4.1.12                 by thelonelycoder

 RT-AC88U (armv7l) FW-384.17 @ 172.24.5.1

 965,296  blocked domains by  2  hosts file(s)
 356,841 t  14,528 w  10 n ads since Jun 17 08:26

 /mnt/entware | Size 14.7G | Used 1.3G (9%)   

 d   Diversion Standard   enabled
 c   communication        DivUn stats backup FWun

 a   ad-blocking          to IP 172.24.5.254
 l   logging              dnsmasq.log 25.5M

 ep  pixelserv-tls        172.24.5.254 v2.3.1

 b   blocking list        Large Mon @ 1:00
 el  edit lists            210 w  22 b  0 wb  936 yt

 f   follow dnsmasq.log

 ac  update ads counter   Auto-update  au  off
 ds  dnsmasq settings     on
 sw  manage swap file     /mnt/entware 1.0G
 ct  change theme colors   green

 sh  shell history        on
 cj  show cron job(s)
 sf  show file content

 e   exit  u  update                less options  o
Code:
Dnsmasq settings:

 See the Dnsmasq man page what these options do:
 http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html

 1. Disable Dnsmasq settings
 2. log-async         5
 3. cache-size        1500
 4. bogus-priv        on
 5. domain-needed     on
 6. log-queries=extra

 Enter selection [1-6 e=Exit]

Disabling Dnsmasq settings does not change anything. I'm using Sandisk Extreme 16GB USB3.0 in USB 2.0 port.


turn off the custom DNSMasq options inside diversion, your issues should go away. those options are default in dnsmasq.

ds dnsmasq settings on

turn the ds feature off.

Also: You could dump your dnsmasq logfile. see if that helps.

the size of your log file...
dnsmasq.log 25.5M
 
Last edited:
Code:
admin@FaTiii:/tmp/home/root# df -Th | grep mnt | grep dev
/dev/sda1            ext2           14.7G      1.3G     12.6G   9% /tmp/mnt/entware

https://imgur.com/a/Iy8jmHd

atop is too complex for me :D but as I check htop I saw taildns and pixelserv-tls at the top most of the time, and this is when I try to access many pages in a couple of seconds.
atop screenshot is fine. htop shows that network map is using all of your CPU. try navigating to a static WebUI page (or logging out) and repeat the tests.
 
atop screenshot is fine. htop shows that network map is using all of your CPU. try navigating to a static WebUI page (or logging out) and repeat the tests.

Logged out and opened many web pages in about 15-20 seconds, all loaded fine with no error and I got this;
1.jpg


Then I've uninstalled uiDivStats; flushed DNS of my PC and deleted databases of firefox and restarted.

Screenshot_6.jpg


this time (after uninstalling uiDivStats) I've tried to open 40 tabs as soon as possible and for your reference here my 9900K CPU usage during this;

Screenshot_7.jpg


I mean second test should be more intense for router but it didn't even sweat.
 
Logged out and opened many web pages in about 15-20 seconds, all loaded fine with no error and I got this;
View attachment 24100

Then I've uninstalled uiDivStats; flushed DNS of my PC and deleted databases of firefox and restarted.

View attachment 24101

this time (after uninstalling uiDivStats) I've tried to open 40 tabs as soon as possible and for your reference here my 9900K CPU usage during this;

View attachment 24102

I mean second test should be more intense for router but it didn't even sweat.
I experienced this same issue with the DS setting in diversion enabled along time ago before UiDivstats was around. I disabled it and rebooted the router and the problem went away.
 
I experienced this same issue with the DS setting in diversion enabled along time ago before UiDivstats was around. I disabled it and rebooted the router and the problem went away.

Could you please send your current DS settings? Even I disable settings I think I can change bogus-priv and domain-needed.

Let me change the settings as same of yours reboot and check
 
Could you please send your current DS settings? Even I disable settings I think I can change bogus-priv and domain-needed.

Let me change the settings as same of yours reboot and check
Mine are complety disabled meaning I do not have that diversion "experimental" feature enabled.
 
Mine are complety disabled meaning I do not have that diversion "experimental" feature enabled.

OK; disabled and rebooted the router, installed iuDivStats and waited for a while for a cool down.

Screenshot_1.jpg


And did the same test with 40 tabs and here :(

Screenshot_2.jpg


additional notes I would like to mention.

- When I did the test with 40 tabs when iuDivStats is not installed; the CPU Load average reached its max just after all tabs are loaded
- When I did the test with 40 tabs with iuDivStats is installed when all tabs are loaded CPU load average for 1 min was 0.78 after about 10-15 seconds it reached 1.45.

I guess this shows this is not because of iuDivStats it maybe because of database or something else being written into USB drive.

I've never seen good write speeds with my router, I've tested again. it changes between 2 MB/S and 45 MB/s but this is USB 2.0 and I'm not sure how it's 45 MB/S :D

Screenshot_3.jpg


Maybe it's time for me to reset router to factory defaults and start over :D I haven't done this for more than one year I guess :D
 
That's a point. What filesystem and number of partitions do people have on their USB drive?
Code:
admin@RT-AC86U:/tmp/home/root# df -Th | grep mnt | grep dev
/dev/sda3            ext4            6.3G      2.7G      3.2G  46% /tmp/mnt/entware
/dev/sda1            tfat            7.9G      1.3M      7.9G   0% /tmp/mnt/data

I've had no issue on this version aswell.

on a 512GB SSD attached via USB 3.0 port.

Code:
ASUSWRT-Merlin RT-AX88U 384.17_0 Sun Apr 26 02:29:04 UTC 2020
Makaveli@RT-AX88U-0B10:/tmp/home/root# df -Th | grep mnt | grep dev
/dev/sda1            ext4          436.6G      1.3G    412.0G   0% /tmp/mnt/Scripts

Cpu usage normal.

 
Here we go.
Uninstalled 2.1.0
Deleted dnsqueries.db
Installed 2.1.0

Diversion and scMerlin are the the only scripts I currently have installed on 384.17. It has been at least a year since a factory reset and looking at an old screen shot I took of Diversion after first installing the version is 4.0.6.

Now seeing the cpu spiking to 100% and alternating between cores. It's been going at it for about 20 minutes now.

/dev/sda1 ext2 7.2G 2.1G 4.8G 30% /tmp/mnt/Diversion

upload_2020-6-17_11-21-56.png


upload_2020-6-17_11-22-9.png



upload_2020-6-17_11-22-19.png
 
Here we go.
Uninstalled 2.1.0
Deleted dnsqueries.db
Installed 2.1.0

Diversion and scMerlin are the the only scripts I currently have installed on 384.17. It has been at least a year since a factory reset and looking at an old screen shot I took of Diversion after first installing the version is 4.0.6.

Now seeing the cpu spiking to 100% and alternating between cores. It's been going at it for about 20 minutes now.

/dev/sda1 ext2 7.2G 2.1G 4.8G 30% /tmp/mnt/Diversion

View attachment 24109

View attachment 24110


View attachment 24111
htop is showing hardly any cpu usage - do you see the 100% there or on network map? honestly, im starting to think network map is a steaming pile of s**t that causes the router to work overtime to show anything
 
htop is showing hardly any cpu usage - do you see the 100% there or on network map? honestly, im starting to think network map is a steaming pile of s**t that causes the router to work overtime to show anything

Good catch. htop is not showing the spikes when I monitor it and it is showing around 1-4% usage. But...the last log entry from uiDivStats is:
Jun 17 11:05:23 uiDivStats: Starting first run of stat generation...
Should there be a log entry stating that the generation has completed? It's been running for over three hours now.

I am seeing values in the uiDivStats tab. On the DNS Queries graph the x-axis jives with the time that I installed 2.1.0. It's showing roughly 300 total DNS queries and ~50 blocked.

Just in case it might make a difference my stats config in Diversion is:
upload_2020-6-17_14-23-42.png


My USB stick does have an activity light and it seems way more active that it used to be. Pretty much going nonstop with a pause here and there.
 
Good catch. htop is not showing the spikes when I monitor it and it is showing around 1-4% usage. But...the last log entry from uiDivStats is:
Jun 17 11:05:23 uiDivStats: Starting first run of stat generation...
Should there be a log entry stating that the generation has completed? It's been running for over three hours now.

I am seeing values in the uiDivStats tab. On the DNS Queries graph the x-axis jives with the time that I installed 2.1.0. It's showing roughly 300 total DNS queries and ~50 blocked.

Just in case it might make a difference my stats config in Diversion is:
View attachment 24114

My USB stick does have an activity light and it seems way more active that it used to be. Pretty much going nonstop with a pause here and there.
Looks like the "complete" message only prints to the console at the moment, and wont appear in the log file. I assume that's where you're looking? I'll fix that for the next release
 
Yes, I am looking in the system log for the "complete" message.

My initial atop didn't show this but I'm seeing it now.
upload_2020-6-17_16-6-42.png

It's hard to see but it's showing the dsk busy 100%.

After uninstalling uiDivStats the GUI cpu graph settles down and my USB stick light stops flashing non stop. And now dsk only shows up intermittently in atop.
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top