uiDivStats uiDivStats 3.x

  • ATTENTION! You'll notice a Prefix dropdown when you create a thread. If your post applies to one of the topics listed, please use that Prefix for your post. When browsing the thread list you can use the Prefix to filter the view.
  • ATTENTION! As of November 1, 2020, you are not able to reply to threads 6 months after the thread is opened if there are more than 500 posts in the thread.
    Threads will not be locked, so posts may still be edited by their authors.
    Just start a new thread on the topic to post if you get an error message when trying to reply to a thread.

Jack Yaz

Part of the Furniture
v3.0.1
Updated 2021-08-05

A graphical representation of domain blocking performed by Diversion.

uiDivStats is free to use under the GNU General Public License version 3 (GPL 3.0).

This project is hosted on GitHub

Love the script and want to support future development? Any and all donations gratefully received!
PayPal donation
Buy me a coffee

Supported firmware versions
You must be running firmware Merlin 384.15/384.13_4 or Fork 43E5 (or later) Asuswrt-Merlin

Installation
Using your preferred SSH client/terminal, copy and paste the following command, then press Enter:
Code:
/usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/jackyaz/uiDivStats/master/uiDivStats.sh" -o "/jffs/scripts/uiDivStats" && chmod 0755 /jffs/scripts/uiDivStats && /jffs/scripts/uiDivStats install

Usage
To launch the uiDivStats menu after installation, use:
Code:
uiDivStats

If this does not work, you will need to use the full path:
Code:
/jffs/scripts/uiDivStats
Updating
Launch uiDivStats and select option u
 
Last edited:

Jack Yaz

Part of the Furniture
Screenshots:

 
Last edited:

Jack Yaz

Part of the Furniture
Please note that at midnight (1 past) uiDivStats will perform maintenance on the database. The first time this runs after upgrading from uiDivStats 2.x involves creating new database indexes to increase performance, but can take a long time. My database is roughly 1GB in size and my RT-AC86u took an hour to complete index creation. During this time uiDivStats will stop logging DNS queries so you will have a gap in your data, but this ensures the maintenance can complete as quickly as possible.

Future maintenance jobs will run at the same time every night but are much quicker (<5 minutes, as conditions as above).
 
Last edited:

SomeWhereOverTheRainBow

Part of the Furniture
uiDivStats 3.0.0 will be dropping tomorrow (Monday 7 June)

Please note that at midnight (1 past) uiDivStats will perform maintenance on the database. The first time this runs after upgrading from uiDivStats 2.x involves creating new database indexes to increase performance, but can take a long time. My database is roughly 1GB in size and my RT-AC86u took an hour to complete index creation. During this time uiDivStats will stop logging DNS queries so you will have a gap in your data, but this ensures the maintenance can complete as quickly as possible.

Future maintenance jobs will run at the same time every night but are much quicker (<5 minutes, as conditions as above).
Great job, Jack!. keep on chuggin' along!
 

QuikSilver

Very Senior Member
awesome! can't wait!
 

Jack Yaz

Part of the Furniture
v3.0.0 is now available

PLEASE NOTE THE INFORMATION HERE: http://www.snbforums.com/threads/uidivstats-3-x.72966/post-693164

Changelog:

  • NEW: Overnight database maintenance to ensure query performance is fast
  • NEW: Configure how long data is kept in the database
  • NEW: Configure how many recent results are displayed in the WebUI
  • NEW: Option to refresh query log in WebUI on demand
  • NEW: CLI menu shows URL for WebUI page
  • NEW: CLI commands for "about" and "help"
  • NEW: Add option to reset database
  • IMPROVED: Statistic generation is now much faster due to improved use of database indexes and optimised queries. This will take effect after first maintenance run at midnight
  • IMPROVED: CPU intensive tasks are now run with a lower priority to minimise hogging the CPU
  • IMPROVED: Overnight database maintenance will remove invalid entries such as those with a timestamp in the future
  • IMPROVED: Hostname matching for client IP addresses
  • IMPROVED: Update function now includes a prompt rather than applying update
  • IMPROVED: Use colours in CLI menu to highlight settings
  • IMPROVED: Use of keyboard keys r,f for chart functions (reset zoom, toggle fill)
  • IMPROVED: Use ajax to load dependent files in WebUI to avoid complete page load failures if a file was unavailable
  • IMPROVED: Stale processes will be cleared as necessary
  • CHANGED: Use of cache/tmpfs for query logging is now the default in place of direct write to disk
  • CHANGED: taildns will wait for NTP to sync before starting, with a maximum wait of 10 minutes
  • CHANGED: Increase NTP timeout from 5 minutes to 10 minutes for core script
 

Jack Yaz

Part of the Furniture
Has anyone run into problems following this upgrade?
 

QuikSilver

Very Senior Member
Has anyone run into problems following this upgrade?
Unless I'm overlooking something mine seems to be perfectly fine. If it is then it must be a minor issue.
 

L&LD

Part of the Furniture
None that I've seen.
 

Kingp1n

Very Senior Member
I updated recently and I haven't seen any issues.
 

Jack Yaz

Part of the Furniture
My main concerns were if anything hung during the first nightly maintenance as that will take ages to re-do the indexes, but then after that you should see 10-20s for normal generation and sub 2 minutes for full refreshes
 

jsbeddow

Senior Member
My main concerns were if anything hung during the first nightly maintenance as that will take ages to re-do the indexes, but then after that you should see 10-20s for normal generation and sub 2 minutes for full refreshes
True, the initial evening when this version was installed (through the GUI this time too!), I lost all the graphs/stats, but everything was back in place by the next morning, so I didn't bother mentioning it before and figured it was expected, as you had told us to expect one long maintenance session initially.
 

Jack Yaz

Part of the Furniture
True, the initial evening when this version was installed (through the GUI this time too!), I lost all the graphs/stats, but everything was back in place by the next morning, so I didn't bother mentioning it before and figured it was expected, as you had told us to expect one long maintenance session initially.
it should all be worth it. i spent a significant chunk of time learning about SQL performance improvements and analysing the queries uiDivStats uses to ensure they were all covered by an index to make them quick, as well as remove multiple commands when 1 would do, just needed some advanced syntax
 

SomeWhereOverTheRainBow

Part of the Furniture
v3.0.0 is now available

PLEASE NOTE THE INFORMATION HERE: http://www.snbforums.com/threads/uidivstats-3-x.72966/post-693164

Changelog:

  • NEW: Overnight database maintenance to ensure query performance is fast
  • NEW: Configure how long data is kept in the database
  • NEW: Configure how many recent results are displayed in the WebUI
  • NEW: Option to refresh query log in WebUI on demand
  • NEW: CLI menu shows URL for WebUI page
  • NEW: CLI commands for "about" and "help"
  • NEW: Add option to reset database
  • IMPROVED: Statistic generation is now much faster due to improved use of database indexes and optimised queries. This will take effect after first maintenance run at midnight
  • IMPROVED: CPU intensive tasks are now run with a lower priority to minimise hogging the CPU
  • IMPROVED: Overnight database maintenance will remove invalid entries such as those with a timestamp in the future
  • IMPROVED: Hostname matching for client IP addresses
  • IMPROVED: Update function now includes a prompt rather than applying update
  • IMPROVED: Use colours in CLI menu to highlight settings
  • IMPROVED: Use of keyboard keys r,f for chart functions (reset zoom, toggle fill)
  • IMPROVED: Use ajax to load dependent files in WebUI to avoid complete page load failures if a file was unavailable
  • IMPROVED: Stale processes will be cleared as necessary
  • CHANGED: Use of cache/tmpfs for query logging is now the default in place of direct write to disk
  • CHANGED: taildns will wait for NTP to sync before starting, with a maximum wait of 10 minutes
  • CHANGED: Increase NTP timeout from 5 minutes to 10 minutes for core script
HE IS BACK!<3 who is back? JACK IS BACK!
 

umrico

Occasional Visitor
My main concerns were if anything hung during the first nightly maintenance as that will take ages to re-do the indexes, but then after that you should see 10-20s for normal generation and sub 2 minutes for full refreshes
Good morning! Thanks so much for this! Just wanted to ask/provide feedback. Was the tmpfs supposed to be enabled automatically upon update/first maintenance mode? Mine was still off a couple days later.

I just also saw there was a hotfix this morning. I updated to that but tmpfs was still off (however it hasn't got to tonight's maintenance mode yet).

In any case i just enabled it. Is that for the better? I have an AX-86U.

Thank you again!
 

Jack Yaz

Part of the Furniture
Good morning! Thanks so much for this! Just wanted to ask/provide feedback. Was the tmpfs supposed to be enabled automatically upon update/first maintenance mode? Mine was still off a couple days later.

I just also saw there was a hotfix this morning. I updated to that but tmpfs was still off (however it hasn't got to tonight's maintenance mode yet).

In any case i just enabled it. Is that for the better? I have an AX-86U.

Thank you again!
tmpfs is a new default but won't be imposed on existing installs. if you have a super fast USB/SSD connected then by all means use no cache, but from testing I've found tmpfs provides less hangups on the router
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top