uiDivStats uiDivStats - WebUI for Diversion statistics

[Jack Yaz]

Hi Jack,
Updated to V2 and rebooted to make the tab show in LAN but to no avail. I went back into amtm and selected the j5 option. i got some Entware install and config activity but something is hanging and i'm not being returned to the menu

Enter option  j5
Downloading http://bin.entware.net/armv7sf-k2.6/Packages.gz
Updated list of available packages in /opt/var/opkg-lists/entware
Package grep (3.4-1) installed in root is up to date.
Package sqlite3-cli (3310100-1) installed in root is up to date.
Installing procps-ng-pkill (3.3.15-4) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/procps-ng-pkill_3.3.15-4_armv7-2.6.ipk
Installing procps-ng (3.3.15-4) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/procps-ng_3.3.15-4_armv7-2.6.ipk
Configuring procps-ng.
Configuring procps-ng-pkill.
uiDivStats: New version of taildns.tar.gz downloaded

were you running v1 previously or did you hop on the beta train?
what do you see for

ls -la /jffs/addons/uiDivStats.d

 

[Jack Yaz]

The update via amtm seemed to go well, but the web GUI shows no stats.
When updating in the uiDivStats> 1, the stats are regenerated with:

awk: cmd. line:1: Division by zero
uiDivStats: Stats updated successfully

Any suggestions?

do you still see this now? i'd expect this to happen if there are no queries logged whatsoever (which i will patch, but you shouldn't see it after things have been running for a few seconds, assuming you have active network clients)

 

[StefanoN]

were you running v1 previously or did you hop on the beta train?
what do you see for

ls -la /jffs/addons/uiDivStats.d

Hello
Some problem of tomsk ..
With your command I see:

ASUSWRT-Merlin RT-AC86U 384.18_alpha1-g5f6b535e1a Fri May 29 04:00:52 UTC 2020
admin@RT-AC86U-81D8:/tmp/home/root# ls -la /jffs/addons/uiDivStats.d
drwxrwxrwx 3 admin root 0 Jun 6 11:00 .
drwxr-xr-x 11 admin root 0 Jun 2 10:13 ..
-rw-rw-rw- 1 admin root 0 Jun 6 10:39 .upgraded
-rw-rw-rw- 1 admin root 0 Jun 6 10:39 .upgraded2
-rw-rw-rw- 1 admin root 728 Jun 6 11:00 SQLData.js
-rw-rw-rw- 1 admin root 24 Jun 6 10:00 psstats.htm
drwxrwxrwx 2 admin root 0 Jun 6 10:39 taildns.d
-rw-rw-rw- 1 admin root 33 Jun 6 10:39 taildns.tar.gz.md5
-rw-rw-rw- 1 admin root 8334 Jun 6 10:00 uidivstats.js
-rw-rw-rw- 1 admin root 11598 Jun 6 10:00 uidivstats.txt
-rw-rw-rw- 1 admin root 28934 Jun 6 10:30 uidivstats_www.asp
-rw-rw-rw- 1 admin root 535 Jun 6 10:00 uidivstatsclients.js
-rw-rw-rw- 1 admin root 814 Jun 6 10:00 uidivstatstext.js
admin@RT-AC86U-81D8:/tmp/home/root#

 

[tomsk]

were you running v1 previously or did you hop on the beta train?
what do you see for

ls -la /jffs/addons/uiDivStats.d

I was previously running V1.... not brave enough to dabble in the beta for that one

tOmsK@RT-AC68U-4690:/tmp/home/root# ls -la /jffs/addons/uiDivStats.d
drwxrwxrwx    3 tOmsK root             0 Jun  6 13:30 .
drwxrwxrwx   10 tOmsK root             0 Jun  6 13:27 ..
-rw-rw-rw-    1 tOmsK root            28 Jun  6 13:30 SQLData.js
-rw-rw-rw-    1 tOmsK root          5060 May 18 17:00 psstats.htm
drwxrwxrwx    2 tOmsK root             0 Jun  6 13:30 taildns.d
-rw-rw-rw-    1 tOmsK root            33 Jun  6 13:30 taildns.tar.gz.md5
-rw-rw-rw-    1 tOmsK root         14169 Jun 23  2019 uiDivStats_cache.tar.gz
-rw-rw-rw-    1 tOmsK root          6249 May 18 17:00 uidivstats.js
-rw-rw-rw-    1 tOmsK root          8715 May 18 17:00 uidivstats.txt
-rw-rw-rw-    1 tOmsK root         28934 Jun  6 13:20 uidivstats_www.asp
-rw-rw-rw-    1 tOmsK root           591 May 18 17:00 uidivstatsclients.js
-rw-rw-rw-    1 tOmsK root           814 May 18 17:00 uidivstatstext.js

if i try the j5 option now i just get

 Enter option  j5
Downloading http://bin.entware.net/armv7sf-k2.6/Packages.gz
Updated list of available packages in /opt/var/opkg-lists/entware
Package grep (3.4-1) installed in root is up to date.
Package sqlite3-cli (3310100-1) installed in root is up to date.
Package procps-ng-pkill (3.3.15-4) installed in root is up to date.

 

[Jack Yaz]

Hello
Some problem of tomsk ..
With your command I see:

ASUSWRT-Merlin RT-AC86U 384.18_alpha1-g5f6b535e1a Fri May 29 04:00:52 UTC 2020
admin@RT-AC86U-81D8:/tmp/home/root# ls -la /jffs/addons/uiDivStats.d
drwxrwxrwx 3 admin root 0 Jun 6 11:00 .
drwxr-xr-x 11 admin root 0 Jun 2 10:13 ..
-rw-rw-rw- 1 admin root 0 Jun 6 10:39 .upgraded
-rw-rw-rw- 1 admin root 0 Jun 6 10:39 .upgraded2
-rw-rw-rw- 1 admin root 728 Jun 6 11:00 SQLData.js
-rw-rw-rw- 1 admin root 24 Jun 6 10:00 psstats.htm
drwxrwxrwx 2 admin root 0 Jun 6 10:39 taildns.d
-rw-rw-rw- 1 admin root 33 Jun 6 10:39 taildns.tar.gz.md5
-rw-rw-rw- 1 admin root 8334 Jun 6 10:00 uidivstats.js
-rw-rw-rw- 1 admin root 11598 Jun 6 10:00 uidivstats.txt
-rw-rw-rw- 1 admin root 28934 Jun 6 10:30 uidivstats_www.asp
-rw-rw-rw- 1 admin root 535 Jun 6 10:00 uidivstatsclients.js
-rw-rw-rw- 1 admin root 814 Jun 6 10:00 uidivstatstext.js
admin@RT-AC86U-81D8:/tmp/home/root#

OK so yours has completed the upgrade, so should load after finishing the stat generation (which with an empty db should take no time at all). If you Ctrl-C and re-launch, what happens?

 

[Jack Yaz]

I was previously running V1.... not brave enough to dabble in the beta for that one

tOmsK@RT-AC68U-4690:/tmp/home/root# ls -la /jffs/addons/uiDivStats.d
drwxrwxrwx    3 tOmsK root             0 Jun  6 13:30 .
drwxrwxrwx   10 tOmsK root             0 Jun  6 13:27 ..
-rw-rw-rw-    1 tOmsK root            28 Jun  6 13:30 SQLData.js
-rw-rw-rw-    1 tOmsK root          5060 May 18 17:00 psstats.htm
drwxrwxrwx    2 tOmsK root             0 Jun  6 13:30 taildns.d
-rw-rw-rw-    1 tOmsK root            33 Jun  6 13:30 taildns.tar.gz.md5
-rw-rw-rw-    1 tOmsK root         14169 Jun 23  2019 uiDivStats_cache.tar.gz
-rw-rw-rw-    1 tOmsK root          6249 May 18 17:00 uidivstats.js
-rw-rw-rw-    1 tOmsK root          8715 May 18 17:00 uidivstats.txt
-rw-rw-rw-    1 tOmsK root         28934 Jun  6 13:20 uidivstats_www.asp
-rw-rw-rw-    1 tOmsK root           591 May 18 17:00 uidivstatsclients.js
-rw-rw-rw-    1 tOmsK root           814 May 18 17:00 uidivstatstext.js

if i try the j5 option now i just get

 Enter option  j5
Downloading http://bin.entware.net/armv7sf-k2.6/Packages.gz
Updated list of available packages in /opt/var/opkg-lists/entware
Package grep (3.4-1) installed in root is up to date.
Package sqlite3-cli (3310100-1) installed in root is up to date.
Package procps-ng-pkill (3.3.15-4) installed in root is up to date.

It seems like your database hasn't been through the upgrade/creation yet. Can you run the below?

sqlite3 /opt/share/uiDivStats.d/dnsqueries.db
.tables
.index
.exit

 

[StefanoN]

OK so yours has completed the upgrade, so should load after finishing the stat generation (which with an empty db should take no time at all). If you Ctrl-C and re-launch, what happens?

With ctrl-C I cannot stop the process. I had to get out of the shell.
After the start generation (point 2 of the menu) seem all ok.

Thanks

 

[tomsk]

It seems like your database hasn't been through the upgrade/creation yet. Can you run the below?

sqlite3 /opt/share/uiDivStats.d/dnsqueries.db
.tables
.index
.exit

Ok Done

tOmsK@RT-AC68U-4690:/tmp/home/root# ls -la /jffs/addons/uiDivStats.d
drwxrwxrwx    3 tOmsK root             0 Jun  6 13:30 .
drwxrwxrwx   10 tOmsK root             0 Jun  6 13:27 ..
-rw-rw-rw-    1 tOmsK root            28 Jun  6 13:30 SQLData.js
-rw-rw-rw-    1 tOmsK root          5060 May 18 17:00 psstats.htm
drwxrwxrwx    2 tOmsK root             0 Jun  6 13:30 taildns.d
-rw-rw-rw-    1 tOmsK root            33 Jun  6 13:30 taildns.tar.gz.md5
-rw-rw-rw-    1 tOmsK root         14169 Jun 23  2019 uiDivStats_cache.tar.gz
-rw-rw-rw-    1 tOmsK root          6249 May 18 17:00 uidivstats.js
-rw-rw-rw-    1 tOmsK root          8715 May 18 17:00 uidivstats.txt
-rw-rw-rw-    1 tOmsK root         28934 Jun  6 13:20 uidivstats_www.asp
-rw-rw-rw-    1 tOmsK root           591 May 18 17:00 uidivstatsclients.js
-rw-rw-rw-    1 tOmsK root           814 May 18 17:00 uidivstatstext.js

 

[Jack Yaz]

Ok Done

tOmsK@RT-AC68U-4690:/tmp/home/root# ls -la /jffs/addons/uiDivStats.d
drwxrwxrwx    3 tOmsK root             0 Jun  6 13:30 .
drwxrwxrwx   10 tOmsK root             0 Jun  6 13:27 ..
-rw-rw-rw-    1 tOmsK root            28 Jun  6 13:30 SQLData.js
-rw-rw-rw-    1 tOmsK root          5060 May 18 17:00 psstats.htm
drwxrwxrwx    2 tOmsK root             0 Jun  6 13:30 taildns.d
-rw-rw-rw-    1 tOmsK root            33 Jun  6 13:30 taildns.tar.gz.md5
-rw-rw-rw-    1 tOmsK root         14169 Jun 23  2019 uiDivStats_cache.tar.gz
-rw-rw-rw-    1 tOmsK root          6249 May 18 17:00 uidivstats.js
-rw-rw-rw-    1 tOmsK root          8715 May 18 17:00 uidivstats.txt
-rw-rw-rw-    1 tOmsK root         28934 Jun  6 13:20 uidivstats_www.asp
-rw-rw-rw-    1 tOmsK root           591 May 18 17:00 uidivstatsclients.js
-rw-rw-rw-    1 tOmsK root           814 May 18 17:00 uidivstatstext.js

What did the . commands show please?

 

[Jack Yaz]

For a sanity check, I rolled back to my install to v1.3.2 and the upgrade went through without issue :/

I wonder why it isn't working for you @tomsk !

 

[tomsk]

What did the . commands show please?

tOmsK@RT-AC68U-4690:/tmp/home/root# sqlite3 /opt/share/uiDivStats.d/dnsqueries.db
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
sqlite> .tables
dnsqueries         dnsqueriesdaily    dnsqueriesmonthly  dnsqueriesweekly
sqlite> .index
idx_dns_clients  idx_dns_domains  idx_dns_time   
sqlite> .exit
tOmsK@RT-AC68U-4690:/tmp/home/root#

 

[Jack Yaz]

tOmsK@RT-AC68U-4690:/tmp/home/root# sqlite3 /opt/share/uiDivStats.d/dnsqueries.db
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
sqlite> .tables
dnsqueries         dnsqueriesdaily    dnsqueriesmonthly  dnsqueriesweekly
sqlite> .index
idx_dns_clients  idx_dns_domains  idx_dns_time  
sqlite> .exit
tOmsK@RT-AC68U-4690:/tmp/home/root#

Weird...it looks like a stat update started since the views exist as does the indexes.
Try running

touch /jffs/addons/uiDivStats.d/.upgraded
touch /jffs/addons/uiDivStats.d/.upgraded2

Then option 2 again. If no dice, try a reboot and then option 2 please

 

[tomsk]

Weird...it looks like a stat update started since the views exist as does the indexes.
Try running

touch /jffs/addons/uiDivStats.d/.upgraded
touch /jffs/addons/uiDivStats.d/.upgraded2

Then option 2 again. If no dice, try a reboot and then option 2 please

After creating the files i was able to enter the menu again instead of going straight to the entware process.

On entering option 2 i got

Choose an option:    2

uiDivStats: Stale lock file found (>600 seconds old) - purging lock

awk: cmd. line:1: Division by zero
awk: cmd. line:1: Division by zero
awk: cmd. line:1: Division by zero
uiDivStats: Stats updated successfully

Still no Tab so i rebooted...... selected option 2 now and some churning going on somewhere i guess... will give it a few minutes and see if it recovers

 

[Jack Yaz]

After creating the files i was able to enter the menu again instead of going straight to the entware process.

On entering option 2 i got

Choose an option:    2

uiDivStats: Stale lock file found (>600 seconds old) - purging lock

awk: cmd. line:1: Division by zero
awk: cmd. line:1: Division by zero
awk: cmd. line:1: Division by zero
uiDivStats: Stats updated successfully

Still no Tab so i rebooted...... selected option 2 now and some churning going on somewhere i guess... will give it a few minutes and see if it recovers

I assume you do have dnsmasq enabled? I recall seeing you active in the unbound thread a lot

 

[tomsk]

I assume you do have dnsmasq enabled? I recall seeing you active in the unbound thread a lot

Yes i made sure dnsmasq is primary DNS before i started the upgrade .. i can't see any unbound manager code that would potentially interfere.. but its hard to tell when you're not entirely sure what you are looking for

 

[Jack Yaz]

Yes i made sure dnsmasq is primary DNS before i started the upgrade .. i can't see any unbound manager code that would potentially interfere.. but its hard to tell when you're not entirely sure what you are looking for

Can you check taildns is running?

/opt/etc/init.d/S90taildns check
ps | grep taildns

 

[tomsk]

Can you check taildns is running?

/opt/etc/init.d/S90taildns check
ps | grep taildns

tOmsK@RT-AC68U-4690:/tmp/home/root# /opt/etc/init.d/S90taildns check
 Checking taildns...              alive.
tOmsK@RT-AC68U-4690:/tmp/home/root# ps | grep taildns
 3349 tOmsK  1420 S    {taildns} /bin/sh /opt/bin/taildns
 3357 tOmsK  1420 S    {taildnstotal} /bin/sh /jffs/addons/uiDivStats.d/taildns.d/taildnstotal
 3370 tOmsK  1456 S    {dnsmasqtotal.aw} /usr/bin/awk -f /jffs/addons/uiDivStats.d/taildns.d/dnsmasqtotal.awk
 3371 tOmsK  1424 S    {taildnstotal} /bin/sh /jffs/addons/uiDivStats.d/taildns.d/taildnstotal
20234 tOmsK  4656 D    grep taildns

The uiDivStats tab has appeared! (as has the Diversion tab which went missing during the t/shoot)
Everything seems to be hackling now.
What do you think went wrong during the update?

 

[Jack Yaz]

tOmsK@RT-AC68U-4690:/tmp/home/root# /opt/etc/init.d/S90taildns check
 Checking taildns...              alive.
tOmsK@RT-AC68U-4690:/tmp/home/root# ps | grep taildns
 3349 tOmsK  1420 S    {taildns} /bin/sh /opt/bin/taildns
 3357 tOmsK  1420 S    {taildnstotal} /bin/sh /jffs/addons/uiDivStats.d/taildns.d/taildnstotal
 3370 tOmsK  1456 S    {dnsmasqtotal.aw} /usr/bin/awk -f /jffs/addons/uiDivStats.d/taildns.d/dnsmasqtotal.awk
 3371 tOmsK  1424 S    {taildnstotal} /bin/sh /jffs/addons/uiDivStats.d/taildns.d/taildnstotal
20234 tOmsK  4656 D    grep taildns

The uiDivStats tab has appeared! (as has the Diversion tab which went missing during the t/shoot)
Everything seems to be hackling now.
What do you think went wrong during the update?

No idea - maybe rebooting too soon after updating?

 

[tomsk]

No idea - maybe rebooting too soon after updating?

It shall remain a mystery then... anyway thanks for your help ... and great work....now to update the YazFi

 

[pattiri]

I have the same problem that tomsk had. I've seen the same things and just rebooted the node. Currently; it stucks here.

admin@FaTiii:/tmp/home/root# uiDivStats
Downloading http://bin.entware.net/armv7sf-k2.6/Packages.gz
Updated list of available packages in /opt/var/opkg-lists/entware
Package grep (3.4-1) installed in root is up to date.
Package sqlite3-cli (3310100-1) installed in root is up to date.
Package procps-ng-pkill (3.3.15-4) installed in root is up to date.

There is uiDivStats tab but it has no info.

Additionally I've removed procps-ng-pkill manually and tried again. it seems it's stucked when configuring procps-ng-pkill.

admin@FaTiii:/tmp/home/root# uiDivStats
Downloading http://bin.entware.net/armv7sf-k2.6/Packages.gz
Updated list of available packages in /opt/var/opkg-lists/entware
Package grep (3.4-1) installed in root is up to date.
Package sqlite3-cli (3310100-1) installed in root is up to date.
Installing procps-ng-pkill (3.3.15-4) to root...
Downloading http://bin.entware.net/armv7sf-k2.6/procps-ng-pkill_3.3.15-4_armv7-2.6.ipk
Configuring procps-ng-pkill.