What's new

Tutorial A Simple Guide to Using RSync on Asus Merlin to Backup Network Drives

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

BreakingDad

Very Senior Member
Ok based on my own recent experience of installing RSync this is a basic guide, as requested, on how to get Rsync working on Merlin Firmware. Please READ the whole guide first before attempting any code in this guide.

This guide assumes you know the basics of logging into your router using putty and WiNSCP, have JFFS enabled and are running Merlin Firmware.

First of all I added two USB drives to my router, you can also use Network Drives or NAS Drives. For me I have two 2tb usb drives, both formatted NTFS in my case, however the second drive has a small EXT4 partition for my usual swap file for Skynet. If you wish to see your drives over your network you need to have Media Server and NetworkWork place enabled under USB Application. This is not really relevant to RSync, but this is how my setup works.

Then to start off we log in using Putty as usual. Type in AMTM and then go to ep to manage Entware Packages, from here you can see that you probably do not have RSync Installed.

To Install RSync enter
Enter opkg install rsync

Like magic this will install rsync, and this should now show when you manage Entware Packages through AMTM.

For me I wanted exact backups of the main drive onto the backup drive, to achieve this manually I entered the following commands (I have four folders to backup, you may have one, you may have ten, adapt to your own needs)

rsync -av --delete /mnt/My_Passport_Ultra/Films /mnt/NETWORK-BACKUP/Backup/
rsync -av --delete /mnt/My_Passport_Ultra/TV /mnt/NETWORK-BACKUP/Backup/
rsync -av --delete /mnt/My_Passport_Ultra/Software /mnt/NETWORK-BACKUP/Backup/
rsync -av --delete /mnt/My_Passport_Ultra/Music /mnt/NETWORK-BACKUP/Backup/

Note that the -delete switch will delete the original file off the backup drive and replace it with the new one from the main drive effectively creating a Mirrored Drive, the -v switch will show a human readable log of what you are doing, so you can see it has worked.
Also note that on the second drive in the code you do not need to enter the folder it is going into, if you do (like I did) you will end up with a duplicate folder within a folder.
You can modify the routes to suit your own network drive locations if you are using NAS drives etc.

Right, we do not wish to do this manually each time though right?, we want this to run everynight when we are sleeping like babies.

To do this the easy way we need to create it as a Cron Job, Now you can manually add these, or if you are a bit thick like me you can use a lovely command called CRU that Merlin has kindly given us.

You will notice that below we have replaced the -v (verbose) switch with the -q(quiet) switch, this is because we do not need a readout as this will be happening overnight without us reading what is going on, it will just work. There are advanced ways of logging everything however this is a basic guide, I will explain more on this later.

So we enter the following to do this:

cru a filmsbu "0 01 * * * rsync -aq --delete /mnt/My_Passport_Ultra/Films /mnt/NETWORK-BACKUP/Backup/"
cru a tvbu "0 03 * * * rsync -aq --delete /mnt/My_Passport_Ultra/TV /mnt/NETWORK-BACKUP/Backup/"
cru a softwarebu "0 05 * * * rsync -aq --delete /mnt/My_Passport_Ultra/Software /mnt/NETWORK-BACKUP/Backup/"
cru a musicbu "0 06 * * * rsync -aq --delete /mnt/My_Passport_Ultra/Music /mnt/NETWORK-BACKUP/Backup/"

Note that the switch 0 01 the first 0 simply means zero minutes past the hour, and the 01 means 1AM, therefore my jobs run at 1AM 3AM 5AM and 6AM respectively. You can name them anything you like, I chose filmsbu etc just to mean films backup.

Right now this should all work correctly overnight, all being well.

We can check that the jobs have been entered in our Crontab file by enter crontab -E in our putty console.

However what happens if we reboot? well we would lose all of our Cron jobs, so we now need to enter the CRU Commands onto our service-startup file to make sure they reload after a reboot.

The easiest way I found of doing this is to log in to the router using WinSCP, then searching the router for "services-start"

Simply double click the file when WinSCP finds it to edit.

and then adding all the lines we just did above.

cru a filmsbu "0 01 * * * rsync -aq --delete /mnt/My_Passport_Ultra/Films /mnt/NETWORK-BACKUP/Backup/"
cru a tvbu "0 03 * * * rsync -aq --delete /mnt/My_Passport_Ultra/TV /mnt/NETWORK-BACKUP/Backup/"
cru a softwarebu "0 05 * * * rsync -aq --delete /mnt/My_Passport_Ultra/Software /mnt/NETWORK-BACKUP/Backup/"
cru a musicbu "0 06 * * * rsync -aq --delete /mnt/My_Passport_Ultra/Music /mnt/NETWORK-BACKUP/Backup/"

Your services-start file should look something like this:

#!/bin/sh

cru a flexqos "30 3 * * * /jffs/addons/flexqos/flexqos.sh -check" # FlexQoS Addition
cru a filmsbu "0 01 * * * rsync -aq --delete /mnt/My_Passport_Ultra/Films /mnt/NETWORK-BACKUP/Backup/"
cru a tvbu "0 03 * * * rsync -aq --delete /mnt/My_Passport_Ultra/TV /mnt/NETWORK-BACKUP/Backup/"
cru a softwarebu "0 05 * * * rsync -aq --delete /mnt/My_Passport_Ultra/Software /mnt/NETWORK-BACKUP/Backup/"
cru a musicbu "0 06 * * * rsync -aq --delete /mnt/My_Passport_Ultra/Music /mnt/NETWORK-BACKUP/Backup/"

Then click the save button to save the services-start file.


Note I have FlexQOS so I have an extra line already installed for that, this is not relevant to RSync but any existing code should be left intact.

That is basically it, your backup drive should now automatically mirror your main drive every night.

Note that you have to change the paths to suit your own drive locations, you can mirror as many folders as you like.

Note the first time it runs it can take forever if you have tb's of data, for me this was not an issue as I had already manually backed up the drives, so when it runs over night the only thing it does is the changes, ie new files added, changed or removed.

There are many ways you can adapt this to your own needs, adding logging and redirecting the log to a saved file, however for my needs I am keeping it simple, there is further information and help on the forums if you wish to get more fancy!

I hope this guide assists anyone who wishes to set up a simple backup drive. Feel free to continue the discussion below, or ask for advanced help, there is plenty out there!

Lastly as with any guide, exercise extreme caution with valuable data and ensure you have other backups of any data you consider important. This guide covers the very basics and is to be used at your own risk, as stated below by @dosborne potential errors in copying this code could result in a loss of data and additional logging (switches) may be required for debugging purposes for some users in the event of this occurrence.



Thanks to @ColinTaylor @dosborne @Jeffrey Young @L&LD @Martineau @JGrana for all their assistance and guidance when I initially set this up.
 
Last edited:
Alot of people dont know about rclone either that is available through opkg, it will work about the same as rsync, but also I have a aimesh node that uses rclone every night to backup my work files to google drive.
Amazing what can be done via cli on the routers.
 
Good job on the guide, but a couple of words of caution.

It is important to note that automating any file activity can be "dangerous" and that steps should be taken to mitigate the potential for disaster (been there, done that).

First, although more or less covered in the guide, you should always test and verify the exact behaviour of a command before making part if your routine. This is critical when a command runs as a privileged user and is told to delete files. An extra space or a misplaced slash can ruin your day (particularly important on other devices or distros where it may be a lot harder to recover from deleting your root directory). Putting quotes are the source and destination paths can help protect you as well as it protects against paths with spaces that can be problematic for novice users. Most rsync package include a "test" parameter so the output will show what would be performed in non-test mode without actually performing the actions. This is a great way to learn and see what will happen.

Along those lines, I strongly suggest NEVER using "silent" mode on any command like this, and doubly so if you automated the task (cron) as there is no way of knowing what went wrong. Instead, select a reasonable verbosity level and use redirection commands to save the output to a log file. You never know when it will come in handy, but you can be sure it won't help you if you don't have one :)

A link to a complex example can be found here https://www.snbforums.com/threads/can-someone-do-a-simple-guide-to-rsync-please.70525
 
Good job on the guide, but a couple of words of caution.

It is important to note that automating any file activity can be "dangerous" and that steps should be taken to mitigate the potential for disaster (been there, done that).

First, although more or less covered in the guide, you should always test and verify the exact behaviour of a command before making part if your routine. This is critical when a command runs as a privileged user and is told to delete files. An extra space or a misplaced slash can ruin your day (particularly important on other devices or distros where it may be a lot harder to recover from deleting your root directory). Putting quotes are the source and destination paths can help protect you as well as it protects against paths with spaces that can be problematic for novice users. Most rsync package include a "test" parameter so the output will show what would be performed in non-test mode without actually performing the actions. This is a great way to learn and see what will happen.

Along those lines, I strongly suggest NEVER using "silent" mode on any command like this, and doubly so if you automated the task (cron) as there is no way of knowing what went wrong. Instead, select a reasonable verbosity level and use redirection commands to save the output to a log file. You never know when it will come in handy, but you can be sure it won't help you if you don't have one :)

A link to a complex example can be found here https://www.snbforums.com/threads/can-someone-do-a-simple-guide-to-rsync-please.70525
Agree 100%, and I do mention logging, however I consider this guide just the very basics for a simple solution for what I wanted to achieve. That said I have added some cautions to the guide based on your comment.
 

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top