[Release] NVRAM Save/Restore Utility

Xentrk

Part of the Furniture
Presentation of my backup concept:

Conditions:
1. daily backup of the jffs
2. daily backup of the usb-device (sda1-Entware Repository)
3. daily backup of the cfg using nsru, rotation 7 days

Solutions:
1. daily backup of the jffs with rsync on usb-device (sda1)
################################################################
script:
################################################################
#!/bin/sh
#
logger -t $(basename $0) "jffs_bak start [[email protected]]"
#
# copy the data
rsync -av --delete /jffs/ /tmp/mnt/sda1/jffs_bak/
#
logger -t $(basename $0) "jffs_bak finish [[email protected]]"
################################################################

2. daily backup of the usb device (sda1) as a tar file on a CIFS-mounted NAS or ssd device
################################################################
script:
################################################################ #!/bin/sh
#
# SOURCEDIR is the path to Entware installation place
SOURCEDIR=/tmp/mnt/sda1
# OUTPUTDIR is the path where backups should be created
OUTPUTDIR=/cifs1/sda1_hell04/
# NUMBERBCKS means how much old copies of backup should be left
# NUMBERBCKS=0 means backup will be overwritten every time and no old copy will be left
NUMBERBCKS=0
EXCLUDE=/tmp/mnt/sda1/exclude.txt
if [ -f ${OUTPUTDIR}/sda1.tar.gz.bck${NUMBERBCKS} ];
then
rm -f ${OUTPUTDIR}/sda1.tar.gz.bck${NUMBERBCKS}
fi
count=$(($NUMBERBCKS-1))
while [ $count -gt 0 ]
do
if [ -f ${OUTPUTDIR}/sda1.tar.gz.bck${count} ];
then
mv ${OUTPUTDIR}/sda1.tar.gz.bck${count} ${OUTPUTDIR}/sda1.tar.gz.bck$(($count+1))
fi
count=$(($count-1))
done
if [ -f ${OUTPUTDIR}/sda1.tar.gz ] && [ $NUMBERBCKS -ne 0 ];
then
mv ${OUTPUTDIR}/sda1.tar.gz ${OUTPUTDIR}/sda1.tar.gz.bck1
fi
tar -cf ${OUTPUTDIR}/sda1.tar.gz -X /tmp/mnt/sda1/exclude.txt -C ${SOURCEDIR} .
################################################################
script with swp-file:
################################################################
#!/bin/sh
#
# SOURCEDIR is the path to Entware installation place
SOURCEDIR=/tmp/mnt/sda1
# OUTPUTDIR is the path where backups should be created
OUTPUTDIR=/cifs1/sda1_hell04/
# NUMBERBCKS means how much old copies of backup should be left
# NUMBERBCKS=0 means backup will be overwritten every time and no old copy will be left
NUMBERBCKS=0
if [ -f ${OUTPUTDIR}/sda1swp.tar.gz.bck${NUMBERBCKS} ];
then
rm -f ${OUTPUTDIR}/sda1swp.tar.gz.bck${NUMBERBCKS}
fi
count=$(($NUMBERBCKS-1))
while [ $count -gt 0 ]
do
if [ -f ${OUTPUTDIR}/sda1swp.tar.gz.bck${count} ];
then
mv ${OUTPUTDIR}/sda1swp.tar.gz.bck${count} ${OUTPUTDIR}/sda1swp.tar.gz.bck$(($count+1))
fi
count=$(($count-1))
done
if [ -f ${OUTPUTDIR}/sda1swp.tar.gz ] && [ $NUMBERBCKS -ne 0 ];
then
mv ${OUTPUTDIR}/sda1swp.tar.gz ${OUTPUTDIR}/sda1.tar.gz.bck1
fi
tar -cf ${OUTPUTDIR}/sda1swp.tar.gz -C ${SOURCEDIR} .
################################################################

3. not available, my problem: rotation max. 7 days in the nsru / backup-directory
better: direct backup as cfg and tar file on CIFS-mounted NAS or ssd device
@ Xentrk: can you help me?
possible problems:
1. USB device could age faster / become defective
Solution: daily jffs backup directly to CIFS-mounted NAS or ssd device
################################################################
sript:
################################################################
#!/bin/sh
#
# SOURCEDIR is the path to Entware installation place
SOURCEDIR=/jffs
# OUTPUTDIR is the path where backups should be created
OUTPUTDIR=/cifs1/jffs_hell04/
# NUMBERBCKS means how much old copies of backup should be left
# NUMBERBCKS=0 means backup will be overwritten every time and no old copy will be left
NUMBERBCKS=7
#EXCLUDE=/tmp/mnt/sda1/exclude.txt
if [ -f ${OUTPUTDIR}/jffs.tar.gz.bck${NUMBERBCKS} ];
then
rm -f ${OUTPUTDIR}/jffs.tar.gz.bck${NUMBERBCKS}
fi
count=$(($NUMBERBCKS-1))
while [ $count -gt 0 ]
do
if [ -f ${OUTPUTDIR}/jffs.tar.gz.bck${count} ];
then
mv ${OUTPUTDIR}/jffs.tar.gz.bck${count} ${OUTPUTDIR}/jffs.tar.gz.bck$(($count+1))
fi
count=$(($count-1))
done
if [ -f ${OUTPUTDIR}/jffs.tar.gz ] && [ $NUMBERBCKS -ne 0 ];
then
mv ${OUTPUTDIR}/jffs.tar.gz ${OUTPUTDIR}/jffs.tar.gz.bck1
fi
tar -cf ${OUTPUTDIR}/jffs.tar.gz -C ${SOURCEDIR} .
################################################################
The post would be easier to read if you put the code examples in CODE blocks.

upload_2020-5-31_9-44-14.png


For item 3, here is an example of using the find command to check if a file is older than seven days. If so, take some action.

Code:
if [ "$(find "$DIR" -name "$IPSET_NAME" -mtime +7 -print)" = "$DIR/$IPSET_NAME" ]; then
      Download_AMAZON "$IPSET_NAME" "$REGION"
fi
 

TonyK132

Senior Member
Navigate to the backup directory. For the files that failed, locate the nvram restore script for the files that failed. It will look like this: nvram-restore-202005021835_RT-AC88U-8248.sh

Scroll down to the bottom of the file and look for the section was last saved. It will look like this:

echo "Restoring [User Adds]"

See if there is a common theme among the failed backups on the last section saved.

Also, closely monitor and check for any messages in the system log at the time the nvram-save.sh ran for clues.
I have some data for you about this problem. I have a few days where the backup did not complete. See the attached files. For the Restore scripts, I included the end of the file when the backup did not complete completely and one where it did.
 

Attachments

Xentrk

Part of the Furniture
I have some data for you about this problem. I have a few days where the backup did not complete. See the attached files. For the Restore scripts, I included the end of the file when the backup did not complete completely and one where it did.
Thank you. Unfortunately, the system log didn't provide any useful information to help pinpoint the root cause. Have you run a health check scan of the storage device lately? The health check utility is on the router home page. Or use the fsck utility to scan for bad sectors?

I came across this utility to check for error. Not sure if it supports your device though and I have not used it.
 
Last edited:

TonyK132

Senior Member
I tested the thumb drive. All indications are that it is OK. I have it connected to USB2.

One thing I noticed is that spdMerlin was running 2 minutes after the start of the backup script. I changed the time when spdMerlin runs and based on 1 day of data, the backup script completed with no problems. I'll keep watching it, but is it possible that another script like spdMerlin could interrupt the completion of the backup script?
 

Xentrk

Part of the Furniture
I tested the thumb drive. All indications are that it is OK. I have it connected to USB2.

One thing I noticed is that spdMerlin was running 2 minutes after the start of the backup script. I changed the time when spdMerlin runs and based on 1 day of data, the backup script completed with no problems. I'll keep watching it, but is it possible that another script like spdMerlin could interrupt the completion of the backup script?
Thanks for the update. A possible conflict with spdMerlin is the only clue we have right now. I'll set up a batch job on my router to see if I can spot any issues.
 

Xentrk

Part of the Furniture
I just pushed an update to nvram-save.sh to fix a typo in the "Saving[User Adds]" message. "variables" was misspelled as "WARiables".

Code:
 Saving [Captive Portal settings]          2 secs (22  variables, Bytes=0       Total=8600 )
 Saving [Traffic Control settings]         0 secs (11  variables, Bytes=0       Total=8600 )
 Saving [VLAN Settings]                    0 secs (6   variables, Bytes=0       Total=8600 )
        #[Astrill VPN settings]            1 secs (?   variables)
 Saving [User Adds]                        1 secs (0   WARiables, Bytes=0       Total=8600 )
 

ttgapers

Senior Member
Here is the systembackup.sh script I use. I amended from this forum when this script was manual. @Xentrk I haven't used your installer - but used your updates due to this. So here it is for the community's use and perhaps amended to your release!!

Code:
#!/bin/sh

UTIL=nvram
BLOC=/mnt/<usb-label>/$UTIL
BARC=$BLOC/archive
BKUP=$BLOC/backup
RETENTION=7

CRUNAME=$UTIL
cru l | grep $CRUNAME >/dev/null || cru a $CRUNAME "0 5 * * * $0 $*"

#exit if directory doesn't exist or not mounted.
[[ -d "$BLOC" ]] || exit 1
[[ -d "$BKUP" ]] && rm -rf $BKUP
[[ -d "$BKUP" ]] || mkdir $BKUP
cd "$BLOC"

/mnt/<usb-label>/nvram/nvram-save.sh -clk -b
#mv "$BKUP" "$BKUP.$(date +"%F_%T")"
sleep 30
mv "$BKUP" "$BKUP.$(hostname).$(date +%Y%m%d%H%M)"
mv "$BKUP.$(hostname).$(date +%Y%m%d%H%M)" "$BARC"


#careful here.  You do not want this running if ntp hasn't synced.
find $BARC -name "backup.$(hostname).20*" -mtime +$RETENTION -exec rm -rf {} \;


exit 0
It needs definition of the variables for
UTIL - name of your scripts folder
BLOC - backup location
BARC=Archive folder
BKUP= Backup folder (for current runs)
RETENTION= Retention period

I tie it together by having this in my service-start as the script above creates the scheduled task:

Code:
# Add NVRAM backup job & run retention
#/usr/sbin/cru a Backup "0 5 * * * /tmp/mnt/sauron-usb/nvram/nvram-save.sh"
/jffs/scripts/systembackup.sh
Hope you find it useful....I think it was in the original nvram thread...
 

ozzed

Regular Contributor
@Xentrk
Hi!

I'm currently running Version 380.70 on an 87U. Is it still true that I would need to run the script in Migration-mode in order to restore the values between the 380.70 and the newest version? And if yes, what settings would then be lost? All I've found is transmit power but in the documentation it seems suggested that there may be other things as well. I'm very thankful for the time spent making it work between updates. I've been putting off upgrading because the old script didn't support the newer versions and it's too much work to re-enter settings. :)


 

Xentrk

Part of the Furniture
@Xentrk
Hi!

I'm currently running Version 380.70 on an 87U. Is it still true that I would need to run the script in Migration-mode in order to restore the values between the 380.70 and the newest version? And if yes, what settings would then be lost? All I've found is transmit power but in the documentation it seems suggested that there may be other things as well. I'm very thankful for the time spent making it work between updates. I've been putting off upgrading because the old script didn't support the newer versions and it's too much work to re-enter settings. :)


Migration Mode is for moving the settings from one router to another. In migration mode, model specific information is not included.

There have been enough changes to several key nvram vars and their formats since 380.70 and 384.17 that warrant caution against using the utility. Especially the format of dhcp_staticlist and dhcp_hostnames which are used for DHCP static reservations. The conservative and safe approach is to
  1. take screen shots of your settings
  2. update to 384.17
  3. factory reset
  4. manually enter settings
After that, you can use the NVRAM Save/Restore scripts between firmware upgrades.
 

ozzed

Regular Contributor
Migration Mode is for moving the settings from one router to another. In migration mode, model specific information is not included.

There have been enough changes to several key nvram vars and their formats since 380.70 and 384.17 that warrant caution against using the utility. Especially the format of dhcp_staticlist and dhcp_hostnames which are used for DHCP static reservations. The conservative and safe approach is to
  1. take screen shots of your settings
  2. update to 384.17
  3. factory reset
  4. manually enter settings
After that, you can use the NVRAM Save/Restore scripts between firmware upgrades.
I see. Thank you for the clarification :) I guess it's time to start shooting some screens.
 

Darktheleon

New Around Here
I'm sorry if this is a repeat question/problem, I couldn't find anywhere else that addressed it. When using the latest version of this and Merlin I can not get past Step 3 Downloading / Installing NSRU. Am I doing something wrong? I have two degrees in PC/Networking and 25 plus years in experience. Mostly working with Windows, I'm a linux Idiot and like to delegate those jobs to my minions. But not being able to get past step 3 on a home network, that's just embarrassing. Below is the command line and response. Please SNB Forums please help, you're my only hope!!

ASUSWRT-Merlin RT-AC86U 384.17_0 Sun Apr 26 02:28:53 UTC 2020
[[email protected][host]:/tmp/home/root# /usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/Xentrk/nvram-save-restore-
utility/master/nsrum" -o "/jffs/scripts/nsrum" && sleep 5 && chmod 755 /jffs/scripts/nsrum && sh /jffs/scripts/nsrum
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Failed to create the file /jffs/scripts/nsrum: No such file or
Warning: directory
7 19113 7 1371 0 0 5960 0 0:00:03 --:--:-- 0:00:03 7879
curl: (23) Failed writing body (0 != 1371)
 

Xentrk

Part of the Furniture
I'm sorry if this is a repeat question/problem, I couldn't find anywhere else that addressed it. When using the latest version of this and Merlin I can not get past Step 3 Downloading / Installing NSRU. Am I doing something wrong? I have two degrees in PC/Networking and 25 plus years in experience. Mostly working with Windows, I'm a linux Idiot and like to delegate those jobs to my minions. But not being able to get past step 3 on a home network, that's just embarrassing. Below is the command line and response. Please SNB Forums please help, you're my only hope!!

ASUSWRT-Merlin RT-AC86U 384.17_0 Sun Apr 26 02:28:53 UTC 2020
[[email protected][host]:/tmp/home/root# /usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/Xentrk/nvram-save-restore-
utility/master/nsrum" -o "/jffs/scripts/nsrum" && sleep 5 && chmod 755 /jffs/scripts/nsrum && sh /jffs/scripts/nsrum
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Failed to create the file /jffs/scripts/nsrum: No such file or
Warning: directory
7 19113 7 1371 0 0 5960 0 0:00:03 --:--:-- 0:00:03 7879
curl: (23) Failed writing body (0 != 1371)
Is the jffs partition enabled? Administration -> System. If it's not enabled, select Yes for the two options below, Apply the settings, then reboot.

upload_2020-6-30_10-15-53.png
 

Darktheleon

New Around Here
Is the jffs partition enabled? Administration -> System. If it's not enabled, select Yes for the two options below, Apply the settings, then reboot.

View attachment 24388
Quoting a well known phrase "D'oh"
Thanks Xentrk! Sometimes it's the easiest things that get overlooked, other times its the obvious ones. Oh well, working fine so far.
Thanks again,
 

QuikSilver

Very Senior Member
@Xentrk Thanks for this awesome tool! I was able to test it out for the first time yesterday (trying to track down a config or router issue). I noticed when I restored the nvram settings that the custom/manual DHCP clients under LAN DHCP server weren't set back. Is that something I overlooked or that list doesn't get backed up?
 

ech

Regular Contributor
Maybe this is because I saved and restored on the latest (384.18) Merlin, but when I did so, everything seemed to be OK, with a few exceptions:
- the custom traffic stats location was lost and went to the default ram location
- custom scripts were disabled even though I had them enabled prior to save/factory-reset/restore.
(RT-AC68U)
 

ttgapers

Senior Member
@Xentrk - qq, if the script backs up a symlink file, for some reason I can't copy the resulting backup off the usb over SMB - I get permission issues.

Is there something in the script that we can use to not backup the symlink?
 

gjf

Senior Member
First of all many thanks to the author who created and maintain this script. It's really useful.

As for me - I missed a possibility to create a number of backups and to move them from USB to HDD. So that's why I have created a script that adds this functionality and it can be added to cron for a periodical backup.

Code:
#!/bin/sh

# Author is Gray Jack the Fixxxer

# SOURCEDIR is the path to NSRU folder
SOURCEDIR=/mnt/swapusb/nsru

# OUTPUTDIR is the path where backups should be created
OUTPUTDIR=/mnt/HSH_HDD/Download2/Backup/NVRAM

# NUMBERBCKS means how much old copies of backup should be left
# NUMBERBCKS=0 means backup will be overwritten every time and no old copy will be left
NUMBERBCKS=2

if [ -f ${OUTPUTDIR}/nsru.tar.gz.bck${NUMBERBCKS} ]
    then
        rm -f ${OUTPUTDIR}/nsru.tar.gz.bck${NUMBERBCKS}
    fi

count=$(($NUMBERBCKS-1))
while [ $count -gt 0 ]
    do
        if [ -f ${OUTPUTDIR}/nsru.tar.gz.bck${count} ]
            then
                mv ${OUTPUTDIR}/nsru.tar.gz.bck${count} ${OUTPUTDIR}/nsru.tar.gz.bck$(($count+1))
            fi
        count=$(($count-1))
    done

if [ -f ${OUTPUTDIR}/nsru.tar.gz.bck ] && [ $NUMBERBCKS -ne 0 ]
    then
        mv ${OUTPUTDIR}/nsru.tar.gz.bck ${OUTPUTDIR}/nsru.tar.gz.bck1
    fi
  
${SOURCEDIR}/nvram-save.sh

tar -czf ${OUTPUTDIR}/nsru.tar.gz.bck -C ${SOURCEDIR}/backup .
rm -rf ${SOURCEDIR}/backup/
mkdir ${SOURCEDIR}/backup/
 
Last edited:

TonyK132

Senior Member
I'll set up a batch job on my router to see if I can spot any issues.
The failure happened again last night after it had been working fine for more than a week. Here is a snippet of the logs when it failed.

Jul 6 00:13:00 RT-AC86U-1BD0 backupRouterConfig: Started
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: NVRAM User Save Utility - Version 30.3 POSIX Code Updates
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: NVRAM User Save Utility - Version 30.1 Martineau Hacked for v384.xx+!!
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Saving RT-AC86U settings from firmware 384.16_0
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Runtime options: -b
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Using standard NVRAM variable file: nvram-merlin.ini Version=30.3.0
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Running in Backup Mode

Here is a snippet of a run that worked with no problem.

Jul 4 00:13:00 RT-AC86U-1BD0 backupRouterConfig: Started
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: NVRAM User Save Utility - Version 30.3 POSIX Code Updates
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: NVRAM User Save Utility - Version 30.1 Martineau Hacked for v384.xx+!!
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Saving RT-AC86U settings from firmware 384.16_0
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Runtime options: -b
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Using standard NVRAM variable file: nvram-merlin.ini Version=30.3.0
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Running in Backup Mode
Jul 4 00:13:41 RT-AC86U-1BD0 nvram-save.sh: Complete: User NVRAM saved to /tmp/mnt/Data/nsru/backup/nvram-restore-202007040013_RT-AC86U-1BD0.sh
Jul 4 00:13:43 RT-AC86U-1BD0 nvram-save.sh: Complete: JFFS directory saved to /tmp/mnt/Data/nsru/backup/jffs-202007040013_RT-AC86U-1BD0
Jul 4 00:13:59 RT-AC86U-1BD0 backupRouterConfig: Finished 202007040013

Let me know if there is anything I can do to instrument this for you to help in finding this bug.
 

Xentrk

Part of the Furniture
@Xentrk Thanks for this awesome tool! I was able to test it out for the first time yesterday (trying to track down a config or router issue). I noticed when I restored the nvram settings that the custom/manual DHCP clients under LAN DHCP server weren't set back. Is that something I overlooked or that list doesn't get backed up?
On the HND routers, some nvram vars are stored in /jffs/nvram. For those nvram parms, you need to run the jffs-restore.sh script to restore the nvram parms.
 

Xentrk

Part of the Furniture
The failure happened again last night after it had been working fine for more than a week. Here is a snippet of the logs when it failed.

Jul 6 00:13:00 RT-AC86U-1BD0 backupRouterConfig: Started
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: NVRAM User Save Utility - Version 30.3 POSIX Code Updates
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: NVRAM User Save Utility - Version 30.1 Martineau Hacked for v384.xx+!!
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Saving RT-AC86U settings from firmware 384.16_0
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Runtime options: -b
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Using standard NVRAM variable file: nvram-merlin.ini Version=30.3.0
Jul 6 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Running in Backup Mode

Here is a snippet of a run that worked with no problem.

Jul 4 00:13:00 RT-AC86U-1BD0 backupRouterConfig: Started
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: NVRAM User Save Utility - Version 30.3 POSIX Code Updates
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: NVRAM User Save Utility - Version 30.1 Martineau Hacked for v384.xx+!!
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Saving RT-AC86U settings from firmware 384.16_0
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Runtime options: -b
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Using standard NVRAM variable file: nvram-merlin.ini Version=30.3.0
Jul 4 00:13:00 RT-AC86U-1BD0 nvram-save.sh: Running in Backup Mode
Jul 4 00:13:41 RT-AC86U-1BD0 nvram-save.sh: Complete: User NVRAM saved to /tmp/mnt/Data/nsru/backup/nvram-restore-202007040013_RT-AC86U-1BD0.sh
Jul 4 00:13:43 RT-AC86U-1BD0 nvram-save.sh: Complete: JFFS directory saved to /tmp/mnt/Data/nsru/backup/jffs-202007040013_RT-AC86U-1BD0
Jul 4 00:13:59 RT-AC86U-1BD0 backupRouterConfig: Finished 202007040013

Let me know if there is anything I can do to instrument this for you to help in finding this bug.
This is a difficult one to debug and find the root cause since there are no error messages. It just appears to stop writing to disk.

Have you run diagnostics on the attached storage? You can unmount the USB on the main page and run the Health Scanner to see if there are bad sectors. Or, manually run the 'e2fsck -p' command after unmounting the disk.
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top