BACKUPMON BACKUPMON v1.5.10 -Mar 1, 2024- Backup/Restore your Router: JFFS + NVRAM + External USB Drive! (**Thread closed due to age**)

[visortgw]

The code is literally this...

    {
    printf "Date/Time: $(date +'%b %d %Y %X')\n"
    printf "Asus Router Model: ${ROUTERMODEL}\n"
    printf "Firmware/Build Number: ${FWBUILD}\n"
    printf "EXT USB Drive Label Name: ${EXTLABEL}\n"
    printf "\n"
    printf "SUCCESS: BACKUPMON completed a successful primary backup to destination: ${BACKUPMEDIA}."
    printf "\n"
    } > "$tmpEMailBodyFile"

For some reason it's inserting this block of garbage in the middle of all this. Perhaps it doesn't like blank printf "\n" lines? Let me try a different variation, and see if that sticks...

That makes zero sense! All of that "junk" is header info inserted by the email server.

 

[Viktor Jaep]

Added a new item under the setup menu that allows you to test your AMTM email functionality. @CaptainSTX... please give this a shot, and see if it gets rid of that block of text garbage in your regular backup emails?

What's new?
- PATCH: Added AMTM Email testing as a menu item off the main setup menu. This allows you to give the AMTM email capabilities a quick test, providing verbose on-screen feedback during the process.

Download link:

curl --retry 3 "https://raw.githubusercontent.com/ViktorJp/BACKUPMON/master/backupmon-1.5.3b2.sh" -o "/jffs/scripts/backupmon.sh" && chmod 755 "/jffs/scripts/backupmon.sh"

Significant screenshots:

The new item "te" to allow for AMTM email testing

 

[Ripshod]

@CaptainSTX What OS? Which email client?

 

[CaptainSTX]

@CaptainSTX What OS? Which email client?

I am running I am running Merlin 386.6 on the router, sending using a free email account from GMX to my Gmail account. My PC is running Windows 10 and is where I check the email.

 

[Viktor Jaep]

I am running I am running Merlin 386.6 on the router, sending using a free email account from GMX to my Gmail account. My PC is running Windows 10 and is where I check the email.

Did b2 make any difference for you?

 

[Ripshod]

I am running I am running Merlin 386.6 on the router, sending using a free email account from GMX to my Gmail account. My PC is running Windows 10 and is where I check the email.

And are you using a browser or email client? which one?

*EDIT* Read below

 

[Ripshod]

And are you using a browser or email client? which one?

Nevermind. Issue confirmed.
I have all the info sent in an email from a gmx email account to a gmail account, but no issue with other sending accounts. This appears to be something with the gmx servers and I'm having a play now.

This is a TEST to check & verify if sending email notifications is working well from BACKUPMON.
Message-ID: <[email protected]>
X-Provags-ID: V03:K1:0CY+Zndpn5mwlFIdl0s5hryjpmz/aAo7RsmmOfnf1EyXZ9S4q56
 N98ZVaHDSAni64eZ04yUbGm9nQ2MORwsqHQoiPWfgXh1hHeTrKgdB4Sx9MIoeljccWY1B35
 0HkPtEfGFiUWQ3eyxDdMmfV3pQmYvr9re2VAvO6BVWz4tJTzEypcTGaxRYISRO9/Hz+2SbS
 0C4Xk5QOUJp+8s0119WZQ==
X-Spam-Flag: NO
UI-OutboundReport: notjunk:1;M01:P0:/NqoAarcKVU=;VTXmOtj9ioFLG1AKOY1FhR4yphN
 q5hbkO+X5zwZSAEGxVD4OxECuEE9JwEpCd8A+boQ2v7Ek4lbbpvip0ofD+g9AuETEGjicvEwf
 1td2o7iG97iNFln0skFVQnQr/8GVlCk0gk8ULXssyxLxlr/nqFRv3Z58zzbTYhKoYJVT62Taz
 EI9q5JlUHNuwjHVADa77SzttAvpx/Vmbm7emP7McEmdP0kWHtRNuV4uiHY1yaVkvVIHKYmQz/
 fBsunoBMC5uH8Ajp+oGp/QtbZEnd+xNvvSVwpXLN9VK6zWZJG9ZF9RJwL+WtJmZ16HafjjQtw
 wfOsRwG+N4qDkQS9jt1gULvVpZp8FI1Uu8dxALKyV7W3Kt3moYit1ngbDJlAp+eO0kv+10osw
 F/7Go5w58awoNAPHyYJ9xbAxYlPUzXqyYHbomW9sdgSgEFcMZUo2eAYihAIfVjKvecKHx5NjG
 AavP/xwPtJxb279He6lzxhbV1PSGSPNK2HpYU5OkvLm3Ab1ZSG4YBfTcAc1PSo0lRnAardqJx
 2p6T2JUFFSwi4S1hOZeWYohFr7Wc4F3grutii7mXe5Nwi5GL8BBBFcVz8VE6pdaR29nYVniDp
 JkXKhT4P3HWZeHhEEtzK1y+4PRUqhGbuwdiebP11IyvLtbvkDvu364RC/p8AMVuSlBi2nXlpL
 T9iLgzh6nn17bKj4cWMbRxymmIDznS1jKXoTnhp/yy+mgYgHbI/hhCNAof9RD0fX/Sg4bjgEy
 ab5NQqyxbz1E9g7mJ1JLncH8GECcutVC30XdsUgRXPe1rPOtVlDx1aD5KBkbQlJ52J/bFARuo
 1Vdt3lUgeVFSNkGvcBQt5hfpYxVzif/iKSWfLp483Tdfg=

Sent by the "backupmon.sh" script.
From the "AX88U" router.

2024-Feb-13, 07:30:19 PM GMT (Tue)

I can find no setting that can change this, so all I can suggest is ignore it or use another email provider.

 

[CaptainSTX]

Nevermind. Issue confirmed.
I have all the info sent in an email from a gmx email account to a gmail account, but no issue with other sending accounts. This appears to be something with the gmx servers and I'm having a play now.

This is a TEST to check & verify if sending email notifications is working well from BACKUPMON.
Message-ID: <[email protected]>
X-Provags-ID: V03:K1:0CY+Zndpn5mwlFIdl0s5hryjpmz/aAo7RsmmOfnf1EyXZ9S4q56
 N98ZVaHDSAni64eZ04yUbGm9nQ2MORwsqHQoiPWfgXh1hHeTrKgdB4Sx9MIoeljccWY1B35
 0HkPtEfGFiUWQ3eyxDdMmfV3pQmYvr9re2VAvO6BVWz4tJTzEypcTGaxRYISRO9/Hz+2SbS
 0C4Xk5QOUJp+8s0119WZQ==
X-Spam-Flag: NO
UI-OutboundReport: notjunk:1;M01:P0:/NqoAarcKVU=;VTXmOtj9ioFLG1AKOY1FhR4yphN
 q5hbkO+X5zwZSAEGxVD4OxECuEE9JwEpCd8A+boQ2v7Ek4lbbpvip0ofD+g9AuETEGjicvEwf
 1td2o7iG97iNFln0skFVQnQr/8GVlCk0gk8ULXssyxLxlr/nqFRv3Z58zzbTYhKoYJVT62Taz
 EI9q5JlUHNuwjHVADa77SzttAvpx/Vmbm7emP7McEmdP0kWHtRNuV4uiHY1yaVkvVIHKYmQz/
 fBsunoBMC5uH8Ajp+oGp/QtbZEnd+xNvvSVwpXLN9VK6zWZJG9ZF9RJwL+WtJmZ16HafjjQtw
 wfOsRwG+N4qDkQS9jt1gULvVpZp8FI1Uu8dxALKyV7W3Kt3moYit1ngbDJlAp+eO0kv+10osw
 F/7Go5w58awoNAPHyYJ9xbAxYlPUzXqyYHbomW9sdgSgEFcMZUo2eAYihAIfVjKvecKHx5NjG
 AavP/xwPtJxb279He6lzxhbV1PSGSPNK2HpYU5OkvLm3Ab1ZSG4YBfTcAc1PSo0lRnAardqJx
 2p6T2JUFFSwi4S1hOZeWYohFr7Wc4F3grutii7mXe5Nwi5GL8BBBFcVz8VE6pdaR29nYVniDp
 JkXKhT4P3HWZeHhEEtzK1y+4PRUqhGbuwdiebP11IyvLtbvkDvu364RC/p8AMVuSlBi2nXlpL
 T9iLgzh6nn17bKj4cWMbRxymmIDznS1jKXoTnhp/yy+mgYgHbI/hhCNAof9RD0fX/Sg4bjgEy
 ab5NQqyxbz1E9g7mJ1JLncH8GECcutVC30XdsUgRXPe1rPOtVlDx1aD5KBkbQlJ52J/bFARuo
 1Vdt3lUgeVFSNkGvcBQt5hfpYxVzif/iKSWfLp483Tdfg=

Sent by the "backupmon.sh" script.
From the "AX88U" router.

2024-Feb-13, 07:30:19 PM GMT (Tue)

I can find no setting that can change this, so all I can suggest is ignore it or use another email provider.

Interesting. GMX account doesn't generate this header when run from other amtm applications, or when I have other IoT devices that send emails when some event triggers a notification email. I will just ignore as I only reported it to help debugging beta version.

 

[Viktor Jaep]

Interesting. GMX account doesn't generate this header when run from other amtm applications, or when I have other IoT devices that send emails when some event triggers a notification email. I will just ignore as I only reported it to help debugging beta version.

I'm a bit at a loss on this one... because the mechanism to create the message body is extremely straightforward, and can't understand how something is inserting itself in there when it gets sent. I don't see this behavior at all when using outlook.com or google.com. But I will continue plugging away as possible causes, and asking @Martinski for his ideas and opinions as well...

 

[Viktor Jaep]

Nevermind. Issue confirmed.
I have all the info sent in an email from a gmx email account to a gmail account, but no issue with other sending accounts. This appears to be something with the gmx servers and I'm having a play now.

I can find no setting that can change this, so all I can suggest is ignore it or use another email provider.

Thanks much for giving this a shot, @Ripshod!

 

[CaptainSTX]

I'm a bit at a loss on this one... because the mechanism to create the message body is extremely straightforward, and can't understand how something is inserting itself in there when it gets sent. I don't see this behavior at all when using outlook.com or google.com. But I will continue plugging away as possible causes, and asking @Martinski for his ideas and opinions as well...

The only thing I can see is different is the time it takes for backupmon to run. Even with an exclusion file the process takes several hours to complete. I don't know when the email program gets called and if it is sitting there for several hours waiting for the results to report and in the interim some garbage creeps in.

 

[visortgw]

The only thing I can see is different is the time it takes for backupmon to run. Even with an exclusion file the process takes several hours to complete. I don't know when the email program gets called and if it is sitting there for several hours waiting for the results to report and in the interim some garbage creeps in.

Something is drastically wrong if it takes several hours to complete -- mine takes a minute or two, even backing up to a remote NAS across a VPN, including email messages. You may wish to reassess your exclusions file list.

UPDATE: I just timed mine to be ~2 minutes (including 20 seconds "settling" — 10 seconds each) for primary backup to local NAS (2.5 Gbps backbone and NAS NIC) and secondary backup to remote NAS (35 Mbps ISP upload). Entire backup folder size on NAS is 27 MB. Exclusions file as follows:

myswap.swp
entware/var/log/*
skynet/skynet.log
entware/var/lib/unbound/unbound.log
entware/share/uiDivStats.d/*.db
*.tar
*.gz

 

[Ripshod]

The only thing I can see is different is the time it takes for backupmon to run. Even with an exclusion file the process takes several hours to complete. I don't know when the email program gets called and if it is sitting there for several hours waiting for the results to report and in the interim some garbage creeps in.

What is the final zipped size of your usb drive stored on the backup server?

 

[CaptainSTX]

Something is drastically wrong if it takes several hours to complete -- mine takes a minute or two, even backing up to a remote NAS across a VPN. You may wish to reassess your exclusions file list.

I have a 256 GB SSD attached on the router's USB 3 port. Got the drive for $15 which is only reason I have a drive that big. I have all the recommended exclusions in the exclusion file. Only things on SSD are scripts and swap file (excluded). I have played around with exclusion file and have not been able shorten backup time which is why I run only weekly late at night since it really makes a demand on processor. Open to suggestions on how to shorten time.

EDIT: Do not back up to my NAS only to the USB drive.

 

[Tech9]

Something is drastically wrong

This RT-AX86S router has very little free RAM available after boot.

Open to suggestions on how to shorten time.

Your router is loaded with scripts and perhaps swapping due to low RAM.

 

[visortgw]

This RT-AX86S router has very little free RAM available after boot.

My GT-AX6000 also has very little unused RAM — currently, 92% used (78 MB free). This is typical for UNIX-based systems. My swap usage is minimal as well — 48.75 MB of 2 GB SWAP currently used — due to memory management by UNIX.

 

[Ripshod]

EDIT: Do not back up to my NAS only to the USB drive.

So are you backing up the ssd to itself? You would need to add a wildcard exclusion for the backups folder.

 

[visortgw]

I have a 256 GB SSD attached on the router's USB 3 port. Got the drive for $15 which is only reason I have a drive that big. I have all the recommended exclusions in the exclusion file. Only things on SSD are scripts and swap file (excluded). I have played around with exclusion file and have not been able shorten backup time which is why I run only weekly late at night since it really makes a demand on processor. Open to suggestions on how to shorten time.

EDIT: Do not back up to my NAS only to the USB drive.

Backing up my same router from USB SSD to USB SSD using the same exclusions file takes ~45 seconds, including 10 second "settling" added by @Viktor Jaep.

 

[visortgw]

So are you backing up the ssd to itself? You would need to add a wildcard exclusion for the backups folder.

If @CaptainSTX is indeed backing up to the same SSD (source and destination the same), backups are of limited utility — backups disappear with a failed drive.

 

[Viktor Jaep]

The only thing I can see is different is the time it takes for backupmon to run. Even with an exclusion file the process takes several hours to complete. I don't know when the email program gets called and if it is sitting there for several hours waiting for the results to report and in the interim some garbage creeps in.

HOURS!?

Something is drastically wrong if it takes several hours to complete -- mine takes a minute or two, even backing up to a remote NAS across a VPN, including email messages. You may wish to reassess your exclusions file list.

I concur with that... there's something very wrong! Backing up my SSD's main backup takes about 20 seconds... and is backed up over the network. All together, it takes about 57 seconds from start to finish. The final backup size of the main ssd tar.gz ends up being about 22mb. It's just entware stuff, skynet, etc. Not a heck of a lot. Swap is excluded.... What does your drive size look like?

ViktorJp@GT-AX6000-3C88:/tmp/mnt/ASUS-SSD# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                50.6M     50.6M         0 100% /
devtmpfs                409.2M      4.0K    409.2M   0% /dev
tmpfs                   497.5M    760.0K    496.7M   0% /var
tmpfs                   497.5M      2.8M    494.7M   1% /tmp/mnt
ubi:data                 16.8M    128.0K     15.8M   1% /data
ubi:defaults              5.7M    380.0K      5.0M   7% /tmp/mnt/defaults
ubi:jffs2                44.5M      5.0M     37.1M  12% /jffs
tmpfs                   497.5M      2.8M    494.7M   1% /tmp/mnt
ubi:defaults              5.7M    380.0K      5.0M   7% /tmp/mnt/defaults
tmpfs                   497.5M      2.8M    494.7M   1% /tmp
ubi:defaults              5.7M    380.0K      5.0M   7% /tmp/mnt/defaults
/dev/sda1               228.2G      5.6G    211.0G   3% /tmp/mnt/ASUS-SSD
tmpfs                   497.5M      2.8M    494.7M   1% /www/require/modules/menuTree.js

So are you backing up the ssd to itself? You would need to add a wildcard exclusion for the backups folder.

You might be backing up your backups to your backups, and they just keep multiplying in size! Do you ever see tar errors saying "backup so-and-so already included in tar" or something of the sort?