Restore Traffic Stats

[Jeffrey Young]

One last thing. In the spreadsheet, I believe the header rows are frozen. Make sure there is not data hidden by the frozen rows (check the column numbers to make sure they are sequential).

 

[sbsnb]

Did you reboot by any chance on March 28?

No. August 21, 2021 and March 31.

I am not familiar with the AX models, but is the var/lib directory a ram drive. In other words, it is over written on a reboot. Possible that the data is being replaced from that history file at boot?

As far as I can tell it's a ram drive. I am presuming that's where rstats directly stores its data and where the UI reads from. If that's not a correct assumption, then things may be different than I am interpreting them.

Make sure there is not data hidden by the frozen rows

There are no hidden rows

 

[sbsnb]

It's just strange that the UI shows data that is not in that file and simultaneously doesn't show other data that is in that file.

 

[Jeffrey Young]

It's just strange that the UI shows data that is not in that file and simultaneously doesn't show other data that is in that file.

Agreed. I have never seen that. The RSTAT data file is a straight forward fixed length data file. I can share the structure over the weekend if you like.

My guess right now is some kind of corruption in the data file? I have a cron job that takes a daily snap shot of the RSTAT file at midnight each day as I have had the file become corrupted before. At least with the daily snap shots, I can go back to when the monkey business started and get most my data back. Last time the data file became corrupted coincided with the two year anniversary that I started collecting stats. Since the file only keeps 24 months, I am thinking the roll over was not handled by the rstat process properly.

Other than resetting your stat file I am out of ideas.

 

[sbsnb]

Taking snapshots with rsync or something is a good idea, but that's what started this whole thread. I don't know how you're supposed to restore the data once the need arises.

Back in the Tomato days I used a script to back up my stats to my FTP server nightly. It worked really well and I had years and years of data available on my WRTG54GS. I don't know why with much more powerful devices with way more memory we're so limited on the traffic data.

 

[Jeffrey Young]

I have aways just turned off rstat from the gui, replace that stat file from the backup, then restart the process. Just don't click on reset the file.

 

[sbsnb]

That's what I tried, but the data looked the same. I am going to kill rstat and wipe out all the stats files in /var and the USB and start again and see if that helps.

 

[Jeffrey Young]

It is possible that the AX is doing something weird. I can't find the history file in my AC86U.

Good luck. If I think of something different, I'll let you know.

 

[Keenan]

My daily data goes all the way back to January 30th, so that would fit with the 62 days mentioned earlier.

I absolutely believe that as I've retained traffic data going back months as well, but with this last 386.5_2 I lost it all; in previous updates, the data was retained. Not a big deal, just curious about it.

I just reset it and now it's even showing me monthly traffic(traffic monitor graph) which it wasn't doing before which is something that made me think something wasn't quite right.

 

[sbsnb]

There's clearly some other data repository. I did killall -SIGTERM rstats and wiped out all the data files in both locations. I restarted rstats and the data is still there in the UI.

 

[sbsnb]

Found it in /jffs/.sys/TrafficAnalyzer/TrafficAnalyzer.db

 

[sbsnb]

A quick pragma integrity_check on the TrafficAnalyzer.db file says it's fine. I guess I'll see what happens next time.

 

[sbsnb]

So I pulled the TrafficAnalyzer.db file from a backup of JFFS I had performed before flashing and it is also truncated at precisely 3/1/2022 00:00. It appears this is unrelated to flashing or rebooting.

 

[sbsnb]

Further digging:

I started pulling the TrafficAnalyzer.db files from multiple backups. It's clear from analyzing these files that data is being purged on a rolling basis. For example, a backup I performed on 3/19 only has data from 3/1 on. Another backup I performed on 3/2 only has data from 2/1 forward. From all appearances it looks like within a few days of the beginning of the month, the previous month's data is purged. I'm assuming that the plan is for TrafficAnalyzer.db to only keep the current month's data and the aggregated data is offloaded to the /var/lib/misc/rstats-history.gz file.

If my speculation is correct, my first guess is that when data is being written to /var/lib/misc/rstats-history.gz it's overwriting previous data instead of appending. That would make sense. What doesn't make sense is why it's not happening to everybody.

 

[Jeffrey Young]

Do you have bandwidth monitor enabled by chance?

 

[sbsnb]

Are speaking of this page? I don't see a way for it to be disabled.

 

[Jeffrey Young]

It's off. It was a long shot. Was thinking that bandwidth monitor might be fiddling with rstat somehow.

Maybe time to think about vnstat

 

[sbsnb]

It looks interesting, but it seems it hasn't been updated in almost a year. Is it a dead project?

 

[ColinTaylor]

The traffic analyser database and the rstats file are two completely different and unrelated things.

 

[Jeffrey Young]

No, not dead. You just have it turned off. If you hot the app analysis slider and turn it on, you start to get data. I was just wondering if a bug is there and old data from this (also known as traffic analyzer) is getting into your rstat data.

Like I say, just a shot in the dark.