vnStat [Release] vnStat-on-Merlin - UI, CLI and email - data use and data limit monitoring - R1 and R2

dev_null

Senior Member

vnStat-on-Merlin - Release versions - R1 and R2​

Update 18 July 2021: Upgrade! For ARM and AARCH-based routers, we've got some great news: vnStat-on-Merlin Release 2 (R2) is now available for non-MIPS devices. This version is based on vnStat 2.7, recently included as an Entware package. More details below and here.
  • For supported routers, in addition to updating the underlying engine of vnStat, Jack Yaz has worked some of his charting magic, building on the existing charts which are generated by vnStati.
  • More notes here (and see the changelog in the next post).

Update: 15 August 2021: minor fixes to R2 and introduction of trend lines in the interactive charts within R2. No changes in R1.
Update: 27-Dec-2021: vnStat-on-Merlin (dn-vnstat) version 1.0.3 (R1) and 2.0.3 (R2) and later check for amtm messaging existence. If available, vnStat-on-Merlin will use amtm messaging preferentially. If amtm messaging is not configured, will continue to use Diversion (for the time being).
  • See changelog in next post.


Original announcement:

I'm pleased to announce the release of vnStat-on-Merlin version 1.0.2 (aka R1) - for all Merlin-supported routers and version 2.000 (R2) for ARM and AARCH architecture Merlin-supported routers.

I'd like to acknowledge @Jack Yaz who is responsible for the programming behind the concept - Jack is a true partner in every sense of the word!

The main vnStat-on-Merlin github page is located here: https://github.com/de-vnull/vnstat-on-merlin; the changelog is located here.

Background

  • This is an implementation of vnStat for use on AsusWRT-Merlin routers. This effort was started to enable accurate measurement of data use in a local database (for privacy reasons), and as an alternative the internal monitoring tool, Traffic Analyzer > Traffic Monitor, which will periodically record false “17GB” usage bursts. This "false spike", as some have called it, only occurs on some routers on some firmware (I personally experienced it with RT-AC68U firmware on RT-AC66U_B1).
  • Accurate tracking of data usage became a particular concern when Comcast/Xfinity stated their intention to implement 1.2TB caps nationwide.
    • The totals are consistent with the firmware "Traffic Monitor/Traffic Analyzer".
    • The totals reported by vnStat are slightly higher than those currently being reported by Comcast (about 10-12%).

Notes about data use, units and monitoring

  • vnStat-on-Merlin R1 (in the 'Legacy' branch) uses the Entware version of vnStat, currently version 1.18. This is an older version of the application. This version has certain limitations, described here: https://github.com/de-vnull/vnstat-on-merlin/blob/main/more-info.md
  • vnStat-on-Merlin R2 uses the Entware version of vnStat2, currently version 2.7. This is a recent release of the software. It is supported by ARM and AARCH architectures only.
  • vnStat-on-Merlin data-use and data-limit reporting should be considered a guide, an approximation of actual use. vnStat-reported totals may or may not match that reported by your provider, your cycle may start and stop on a different day of the month, a partial month (especially the first month) or the data use reported could be affected by variables such as hardware acceleration, settings that bypass the TCP/IP stack or as mundane as scheduled reboots. You must conduct proper due diligence to determine if the usage reported by vnStat aligns with your provider.

Initial installation and updates

  • Initial installation and update from R1 - choose one of the following procedures:
    • Running amtm and installing from the menu will identify the architecture of the router and install the proper version
    • Choosing u update from within the vnStat-on-Merlin menu will identify the architecture of the router and install the proper version
    • Clicking the Update button from within the vnStat-on-Merlin UI will identify the architecture of the router and install the proper version
      • Upgrading from R1 to R2 will erase any custom vnstat configurations. You may need to redo any custom settings (e.g., date formats).
  • Additional information may be found on the Github page.

Minimum requirements​

  • AsusWRT Merlin version 384.19 or later for R1.
    • R1 has been tested on 384.19 through 386.2_6 release version.
      • Earlier versions (384-series only) may likewise function; kindly report any further experiences (include router model and firmware version).
      • Initial testing on John's fork appears to demonstrate expected functionality. Kindly report any further experiences (include router model and firmware version).
    • AsusWRT Merlin version 386.2 or later for R2.
      • R2 has been tested on 386.2_6 release version.
      • No version of 384.x nor John's fork have been tested (no supported hardware available) - please report success/failure.
  • AMTM, Diversion* and Entware.
  • Update 27/28-Dec-2021:
    • * vnStat-on-Merlin (dn-vnstat) version 1.0.3 (R1) and 2.0.3 (R2) and later checks for amtm messaging existence. If available, vnStat-on-Merlin will use amtm messaging preferentially. If amtm messaging is not configured, will continue to use Diversion.
      • The Diversion requirement is now deprecated in vnStat-on-Merlin. In a future update (TBD), only amtm messaging will be supported.
      • You should start amtm, then follow the prompts to initialize and test the emailing component (amtm > em).

Views​

The AddOns tab showing the UI (with the "rate" column removed via vnstat.conf setting), and the "daily report" collapsed; Asus-theme:

Screenshot_2021-02-28 dn-vnstat-gr-xp.png
New (R2) JY charts:
New-JY-charts-R2.png


The dn-vnstat menu (CLI via SSH):

Menu

An example of the email message output - html and plain text options:

Email_sample

Txt-daily.PNG


Trend lines (R2 2.0.1 and beyond):
Trendlines_1.PNG



Updating from manual/alpha/beta 1:​

CLI version of vnStat​

  • For non-UI users, the CLI (non-UI) install and usage instructions can be found on the vnStat-on-Merlin Github landing page.
  • Instructions on how to install email notifications using the non-UI version are in the readme.md on the landing page.

Feedback welcomed! Please list your router model and firmware version in your posts.

Please keep all vnStat-on-Merlin issues contained to this thread.​

If it expires, another will be opened to replace it.
 
Last edited:

dev_null

Senior Member
Notes, versions and errata.

Version R1 (based on vnStat 1.18):
  • 11-Apr-2021: version 1.0.1. Minor changes, including:
    • New format for monthly total in output listing: i.e., yyyy-mm (dd) where dd is the start of the cycle in a given period
    • Add "Router Friendly Name" to email subject line (can be modified in Diversion email set-up)
    • Add cycle start day to usage string: i.e., "You have used XX % (YY GB) of your ZZ GB cycle allowance, the cycle starts on day 1 of the month"
  • 11-Jul-2021: version 1.0.2. Minor fixes to correct an error which manifested itself as an "arithmetic error"; additional hot-fix to mitigate a related error.
  • 28-Dec-2021: version 1.0.3. Updating to amtm to 3.2.1 or later, R1 1.0.3 amtm messaging is now used, thus removing the requirement for Diversion, even in R1. The existing setup (using Diversion) will work for a while still, but you should set up and test amtm messaging, then update to 1.0.3. Note: amtm messaging ('em') may not be visible on the amtm menu, but can be opened by typing em.

Version R2 (based on vnStat 2.x):
  • 18-Jul-2021: version 2.0.0.Major update, moves supported router architectures (ARM and AARCH) to vnStat 2.x.
    • Upgrade to vnStat version 2.7
    • Data usage is reported in megabytes (base-1000), so no correction is necessary to monitor against ISP bandwidth usage (see this)
    • Minor changes in the built-in charts, including a "shadow" to the daily and monthly usage with projected usage
    • Interactive charts
      - Data usage, by minutes, hours and days
      - Data use comparisons (current week vs previous week, by day of week)
      - Data use trends (current week, previous week, two weeks prior) -
    • Minor changes to the text-based daily reports. These now focus on the most recent periods (last day, previous week, recent 6 months) and include yearly use.
    • Choice to save data to USB rather than JFFS
  • 16-Aug-2021: version 2.0.1.Minor fixes and addition of trend lines.
  • 27-Sep-2021: version 2.0.2. Minor fix to the email symlinks to address an email piggybacking change.
  • 27-Dec-2021: version 2.0.3. Posted here: https://www.snbforums.com/goto/post?id=731816
    • The primary change is to check for amtm messaging existence. If available, vnStat-on-Merlin will use amtm messaging preferentially.
    • If amtm messaging is not configured, will continue to use Diversion (for now; see below).
    • Notes:
      • The Diversion requirement is now deprecated in vnStat-on-Merlin R2. In a future update (TBD), only amtm messaging will be supported. This change is only for R2.
      • vnStat-on-Merlin R1(primarily MIPS-based routers and older firmware versions) still uses Diversion messaging credentials.
        • Also on R1, there is a "false update" message that may show for vnStat-on-Merlin if you update to amtm 3.2.1. You may be shown that v 2.0.3 is available, but when you attempt to update you are informed - correctly - that the 1.0.2 version is the latest. You should ignore this update prompt in 3.2.1. The R1 update to 1.0.3 appears to fix this notification.
    • Other minor configuration, performance and back-end tweaks to charts.
  • 31-Dec.2021: v. 2.0.4. Post here: https://www.snbforums.com/goto/post?id=732843
    • End-of-year-housekeeping: this update includes minor updates to charts, interface id, and uninstall functions.
 
Last edited:

dev_null

Senior Member
FAQ, wishlist, and mea-culpas.

Errata:
  • If, after updating Entware, you see errors such as:
    Code:
    vnstati: error while loading shared libraries: libwebpdecoder.so.4.1.1: cannot open shared object file: No such file or directory vnstati: error while loading shared libraries: libwebpdecoder.so.4.1.1: cannot open shared object file: No such file or directory
    vnstati: error while loading shared libraries: libwebpdecoder.so.4.1.1: cannot open shared object file: No such file or directory
    cp: can't stat '/jffs/addons/dn-vnstat.d/images/vnstat_m.png': No such file or directory
  • You should run this command: opkg remove vnstati2 libgd libwebp && opkg install vnstati2 to reinstall certain necessary packages.
    • This is an Entware update issue.

Wishlist:
  • Potential "trend line" for R2 interactive charts - added to v 2.0.1
 
Last edited:

elorimer

Very Senior Member
Smooth update to 1.0.

1. In the UI, the cli display looks like it is trying to be centered, which is hard to read.
2. I wonder, in the default configuration file, for the colors, if it would be helpful to comment what the different lines cover, and what to use for common colors.
 

elorimer

Very Senior Member
I must be doing something wrong. I get the centered part for the Top 10, but my CLI looks like it was barfed up:
Screenshot 2021-03-28 120157.png
 

dev_null

Senior Member
I must be doing something wrong. I get the centered part for the Top 10, but my CLI looks like it was barfed up:
This is what it looks like on all my routers... hopefully Jack sees something wrong with yours (tested on Chrome, FF, Edge)

Screenshot_2021-03-28 dn-vnstat.png


EDIT: If I (mentally) delete those semi-colons and % signs at the end of some of your lines, it all gets back into alignment. Jack, are those padding?
 

elorimer

Very Senior Member
which browser/OS? can you share your vnstat.conf so I can compare if you have changed any output styles?
Chrome on Windows 10Pro

vnstat.conf:
Code:
 vnStat 1.18 config file
##

# default interface
Interface "eth0"

# location of the database directory
DatabaseDir "/opt/var/lib/vnstat"

# locale (LC_ALL) ("-" = use system locale)
Locale "-"

# on which day should months change
MonthRotate 1

# date output formats for -d, -m, -t and -w
# see 'man date' for control codes
DayFormat    "%x"
MonthFormat  "%b '%y"
TopFormat    "%x"

# characters used for visuals
RXCharacter       "%"
TXCharacter       ":"
RXHourCharacter   "r"
TXHourCharacter   "t"

# how units are prefixed when traffic is shown
# 0 = IEC standard prefixes (KiB/MiB/GiB/TiB)
# 1 = old style binary prefixes (KB/MB/GB/TB)
UnitMode 0

# how units are prefixed when traffic rate is shown
# 0 = IEC binary prefixes (Kibit/s...)
# 1 = SI decimal prefixes (kbit/s...)
RateUnitMode 0

# output style
# 0 = minimal & narrow, 1 = bar column visible
# 2 = same as 1 except rate in summary and weekly
# 3 = rate column visible
OutputStyle 2

# used rate unit (0 = bytes, 1 = bits)
RateUnit 1

# number of decimals to use in outputs
DefaultDecimals 2
HourlyDecimals 1

# spacer for separating hourly sections (0 = none, 1 = '|', 2 = '][', 3 = '[ ]')
HourlySectionStyle 2

# try to detect interface maximum bandwidth, 0 = disable feature
# MaxBandwidth will be used as fallback value when enabled
BandwidthDetection 0

# maximum bandwidth (Mbit) for all interfaces, 0 = disable feature
# (unless interface specific limit is given)
MaxBandwidth 1000

# interface specific limits
#  example 8Mbit limit for 'ethnone':
MaxBWethnone 8

# how many seconds should sampling for -tr take by default
Sampletime 5

# default query mode
# 0 = normal, 1 = days, 2 = months, 3 = top10
# 4 = exportdb, 5 = short, 6 = weeks, 7 = hours
QueryMode 0

# filesystem disk space check (1 = enabled, 0 = disabled)
CheckDiskSpace 1

# database file locking (1 = enabled, 0 = disabled)
UseFileLocking 1

# how much the boot time can variate between updates (seconds)
BootVariation 15

# log days without traffic to daily list (1 = enabled, 0 = disabled)
TrafficlessDays 1


# vnstatd
##

# switch to given user when started as root (leave empty to disable)
DaemonUser ""

# switch to given user when started as root (leave empty to disable)
DaemonGroup ""

# how many minutes to wait during daemon startup for system clock to
# sync time if most recent database update appears to be in the future
TimeSyncWait 10

# how often (in seconds) interface data is updated
UpdateInterval 30

# how often (in seconds) interface status changes are checked
PollInterval 30

# how often (in minutes) data is saved to file
SaveInterval 1

# how often (in minutes) data is saved when all interface are offline
OfflineSaveInterval 5

# how often (in minutes) bandwidth detection is redone when
# BandwidthDetection is enabled (0 = disabled)
BandwidthDetectionInterval 0

# force data save when interface status changes (1 = enabled, 0 = disabled)
SaveOnStatusChange 1

# enable / disable logging (0 = disabled, 1 = logfile, 2 = syslog)
UseLogging 2

# create dirs if needed (1 = enabled, 0 = disabled)
CreateDirs 1

# update ownership of files if needed (1 = enabled, 0 = disabled)
UpdateFileOwner 1

# file used for logging if UseLogging is set to 1
LogFile "/opt/var/log/vnstat.log"

# file used as daemon pid / lock file
PidFile "/opt/var/run/vnstat.pid"


# vnstati
##

# title timestamp format
HeaderFormat "%x %H:%M"

# show hours with rate (1 = enabled, 0 = disabled)
HourlyRate 1

# show rate in summary (1 = enabled, 0 = disabled)
SummaryRate 0

# layout of summary (1 = with monthly, 0 = without monthly)
SummaryLayout 1

# transparent background (1 = enabled, 0 = disabled)
TransparentBg 0

# image colors

CBackground     "475A5F"
CEdge           "475A5F"
CHeader         "475A5F"
CHeaderTitle    "FFFFFF"
CHeaderDate     "EAE303"
CText           "FFFFFF"
CLine           "B0B0B0"
CLineL          "EAE303"
CRx             "C5C5CE"
CTx             "0EC009"
CRxD            "-"
CTxD            "-"
 

elorimer

Very Senior Member
This is what it looks like on all my routers... hopefully Jack sees something wrong with yours (tested on Chrome, FF, Edge)

View attachment 32533

EDIT: If I (mentally) delete those semi-colons and % signs at the end of some of your lines, it all gets back into alignment. Jack, are those padding?
Yes, those things at the end are the rate bars in the text. From the email version:
1616952320978.png
 

dev_null

Senior Member
Yes, those things at the end are the rate bars in the text. From the email version:
Ah. Those were disabled by default in beta 2 and later. Really find them quite useless.
I think the best thing to do is turn off text-center as it breaks the layout if rate columns etc. are turned on @dev_null
Merged. Please update.
 

QuikSilver

Very Senior Member
Typo? "stats file present on your router." (remove from?)
1616965442504.png
 

elorimer

Very Senior Member
Those were disabled by default in beta 2 and later.
Holdover from being an early adopter!

Maybe, though, you are right that that element should be deprecated and go back to centering in the UI.
 

dev_null

Senior Member
Holdover from being an early adopter!

Maybe, though, you are right that that element should be deprecated and go back to centering in the UI.
There are probably more than a few early adopters. But I agree that the left-alignment is not attractive.

You can drop that column by editing vnstat.conf setting OutputStyle 0.

@Jack Yaz : could we sed change that setting and go back to centering?

PS - we love early adopters! You got us where we are today! (And that is in no way, shape, or form, the queen's english).
 

Jack Yaz

Part of the Furniture
There are probably more than a few early adopters. But I agree that the left-alignment is not attractive.

You can drop that column by editing vnstat.conf setting OutputStyle 0.

@Jack Yaz : could we sed change that setting and go back to centering?

PS - we love early adopters! You got us where we are today! (And that is in no way, shape, or form, the queen's english).
do we want to enforce the 0 setting, or reset it as a one-time thing?
 

dev_null

Senior Member
Hotfix with OutputStyle 0 and centering has been merged.

@elorimer - can you give it a few minutes to propagate and then do an update (u), then report back?
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top