Solved Can someone do a simple guide to Rsync please

[BreakingDad]

So I followed this guide https://rewster.uk/2016/03/11/running-rsync-on-asus-router/

and I got the sh /jffs/scripts/post-mount line and it returned

swapon: /tmp/mnt/sdb2/myswap.swp: Device or resource busy
sh: can't open '/opt/S57rsyncd.1'

Now I am lost, I couldn't work out how to install rsync from entware. Will have a look tomorrow when my brain is refreshed. Surely it can't be that hard

I just want to daily back up a music folder to another music folder, a films folder to a films folder over 2 usb drives.

help !

 

[Jeffrey Young]

Are both USB drives connected to the same router?

The link you provided seems to be a lot of trouble compared to a simple user script and a cron job to run your backup nightly.

For instance, the following in a user script called nightly by a cron job should do what you want. You just need to supply the source and destination paths

#!/bin/sh

    rsync -a --delete --exclude 'myswap.swp' srcpath destpath

 

[ColinTaylor]

You don't need to do any of that. You can just run the rsync command from cron.

EDIT: @Jeffrey Young beat me to it.

EDIT 2: But don't use --delete unless you really want extraneous files in the destination*** to be deleted.

*** Check the documentation.

 

[BreakingDad]

Would appreciate a step by step guide if anyone has time.

From start to finish.

 

[Jeffrey Young]

To start with, what is source and destination paths. In short, the steps will be to;

1. Build the script that will do the work (basically the above).
2. Determine when you want the script to run, then add the cron job (cru a .........)

Give me the source and destination folders and the time you want the script to run. I will look at tomorrow sometime (nice to have something to do in covid lockup)

Edit 1: As @ColinTaylor pointed out, also let me know if you want the destination folder to exactly match the source for each backup. Also, does your backup drive have enough space to handle multiple backups?

Edit 2: @ColinTaylor, I just noticed @BreakingDad is from your neck of the woods. I am just getting ready for bed, but you are likely just getting started. I don't mind you helping out if you get board

 

[BreakingDad]

I'm a bit of a noob at this.

I have entware installed

I try to install rsync with okpg and it just comes back with a load of commands.

People say "just use entware to install rsync" but I don't know how to do this even.

I have to admit I don't even know how to get rsync installed without doing it using the method above which didn't work.

Basically all I want is to copy files from (automatically)

/mnt/My_Passport_Ultra/Music (main drive) or should that be tmp/mnt/My_Passport_Ultra/Music (The folder appears twice in winscp)

to

/mnt/NETWORK-BACKUP/Backup/Music (backup drive)

Both drives are 2tb and no where near full. I have already manually copied two of the folders, but they take about 3hrs each so I have two more to do.

and the same for other folders - Films / TV / Software

So if for example I delete a song off the main drive, it deletes off the backup drive

If I upload a new film to the main drive, it automatically backs it up onto the backup drive and replaces any previous version.

Basically a cloned version of the main drive onto the backup drive. It can syncronise at say 2AM daily.


Right now I have to copy stuff and delete it off both to achieve my backup aims.

What I do know how to do is use putty, amtm because it's easy, I have entware installed and updated. I have a jffs partition and 10gb space with 2gb swap. I also have winSCP and can edit text files.

Basically I need a guide that says something like

Go to putty, login into router, type "install rsync", then edit this file, that file, bingo job done.

I tried looking at the wiki and and all kinds of websites, but there is no definitive how to for getting this working for noobs.

Thanks for help Jeffery and Colin



PS if anyone would like to write a full amtm addon to do this with a GUI I will personally post you a large box of cookies with chocolate chips in.

 

[ColinTaylor]

I have entware installed

I try to install rsync with okpg and it just comes back with a load of commands.

Log into the router with PuTTY.
Install rsync with opkg install rsync

Is it now installed?


P.S. Remove any files/changes you may have previously made with that "bad" guide.

 

[BreakingDad]

Log into the router with PuTTY.
Install rsync with opkg install rsync

Is it now installed?


P.S. Remove any files/changes you may have previously made with that "bad" guide.

oh my, that worked first time, simple when you know the command.

Package rsync (3.2.3-3) installed in root is up to date.

Also another question, What is rsync Daemon all about? some guides mention it, others say it is not required.

Also another question, although rsync seems to now be installed, I don't seem to have any config files for it in opt/etc - I am guessing this is because I havent run it yet as such?


Files already removed from previous

I will have to continue tomorrow, I have worked 6 days a week for last month and am exhausted. Its 23:40 here.

Thank you for help thus far, and will take a look tomorrow, I'm glad the basic rsync is now installed.

 

[ColinTaylor]

Next manually test a copy from the command line to make sure it works as you want.

Just like the normal cp (copy) command make sure you're not copying older files over newer ones by mistake.

So to copy the contents of /mnt/My_Passport_Ultra/Music into /mnt/NETWORK-BACKUP/Backup/Music while also preserving the attributes and file structure:

rsync -av /mnt/My_Passport_Ultra/Music/ /mnt/NETWORK-BACKUP/Backup/Music

Note that there is a trailing / on the source path and there is not a trailing / on the destination.


N.B. If you're going to use rsync you really should read the documentation. Most of it can be skipped over as it won't interest you, but it does include lots of useful examples. It's also helpful to at least be aware of some of the more advanced things it can do (like the --delete option in @Jeffrey Young's script).

Also another question, What is rsync Daemon all about? some guides mention it, others say it is not required.

It is for running rsync as server. This is useful if other machines on the network want to backup to this one. In your case this is an unnecessary complication because you're only copying files locally.

Also another question, although rsync seems to now be installed, I don't seem to have any config files for it in opt/etc - I am guessing this is because I havent run it yet as such?

You don't need any config files unless you are setting up an rsync server.

 

[Martineau]

oh my, that worked first time, simple when you know the command.

Package rsync (3.2.3-3) installed in root is up to date.

FYI
Entware install requests for a package don't necessarily issue a message explicitly indicating a successful install

opkg install rsync

Installing rsync (3.2.3-3) to root...
Downloading http://bin.entware.net/aarch64-k3.10/rsync_3.2.3-3_aarch64-3.10.ipk
Installing libacl (2.2.53-1) to root...
Downloading http://bin.entware.net/aarch64-k3.10/libacl_2.2.53-1_aarch64-3.10.ipk
Installing libzstd (1.4.5-2) to root...
Downloading http://bin.entware.net/aarch64-k3.10/libzstd_1.4.5-2_aarch64-3.10.ipk
Configuring libacl.
Configuring libzstd.
Configuring rsync.

opkg install rsync

Package rsync (3.2.3-3) installed in root is up to date.

i.e. The above message ONLY appeared because rsync was already installed, and the opkg install rsync request found no applicable update(s) in the Entware repository.

 

[BreakingDad]

FYI
Entware install requests for a package don't necessarily issue a message explicitly indicating a successful install

opkg install rsync

Installing rsync (3.2.3-3) to root...
Downloading http://bin.entware.net/aarch64-k3.10/rsync_3.2.3-3_aarch64-3.10.ipk
Installing libacl (2.2.53-1) to root...
Downloading http://bin.entware.net/aarch64-k3.10/libacl_2.2.53-1_aarch64-3.10.ipk
Installing libzstd (1.4.5-2) to root...
Downloading http://bin.entware.net/aarch64-k3.10/libzstd_1.4.5-2_aarch64-3.10.ipk
Configuring libacl.
Configuring libzstd.
Configuring rsync.

opkg install rsync

Package rsync (3.2.3-3) installed in root is up to date.

i.e. The above message ONLY appeared because rsync was already installed, and the opkg install rsync request found no applicable update(s) in the Entware repository.

Thanks yes, I am aware of this and entered the command twice, I posted that return to let @ColinTaylor know it was properly installed.

Now my brain is switched back on I am going to delve further, and am awaiting update from Jeffery or Colin. Jefferey won't be about yet, being it's about 4am in Canada however

 

[BreakingDad]

So I have entered command rsync -av --delete /mnt/My_Passport_Ultra/Films /mnt/NETWORK-BACKUP/Backup/Films which seems to be backing up the films directory, however it has created a films folder with films for some reason.

I have corrected this by logging out of putty and re entering rsync -av --delete /mnt/My_Passport_Ultra/Films /mnt/NETWORK-BACKUP/Backup/ which is now copying nicely.

This is a "test run", I need to add this to cron I believe next. Cron actually seems relatively easy.

Every days a learning day, I;ve found a good guide here https://www.howtogeek.com/135533/how-to-use-rsync-to-backup-your-data-on-linux/ at my friends howtogeek.

I may not need to call on the services of Jeff and Colin yet !

I am amazed how slick this process is, you can even backup to a zip !

 

[Martineau]

This is a "test run", I need to add this to cron I believe next. Cron actually seems relatively easy.

Every days a learning day, I;ve found a good guide here https://www.howtogeek.com/135533/how-to-use-rsync-to-backup-your-data-on-linux/ at my friends howtogeek.

For convenience, RMerlin firmware has the cru utility to be able to manage the cron schedules from the command line without needing to manually edit the 'cron configuration' file (reduces the impact of potentially damaging the configuration of ALL the existing schedules)

e.g.

cru -?

Cron Utility

add:    cru a <unique id> <"min hour day month week command">
delete: cru d <unique id>
list:   cru l

and I found the following useful to code/test the schedule definition

e.g. <"min hour day month week command">
Crontab Guru

 

[Jeffrey Young]

Good morning @BreakingDad. Just getting the day started and see you have made great progress. The use of the final / still trips me up even after 30 years. That is why we do dry runs

If you want a crack at this yourself as a learning experience, I will back off. Otherwise, when I get back from church this morning I'll help write a simple two line script to do the job and how to add the cron job to the init-start script so that your "cru a" command will survive a reboot.

Cherrs

 

[JGrana]

IMHO, rsync is one of the most useful and versitle commands in Linux. It backups up, mirrors (with --delete), copies large amounts of files, clones disks etc. And pretty much whatever Linux runs on you will find it. From servers to tiny IoT devices.

If you haven't found this link, it's a good read on rsync:

Rsync Command in Linux with Examples

rsync is a fast and versatile command-line utility for synchronizing files and directories between two locations over a remote shell, or from/to a remote Rsync daemon.

linuxize.com

 

[BreakingDad]

Good morning @BreakingDad. Just getting the day started and see you have made great progress. The use of the final / still trips me up even after 30 years. That is why we do dry runs

If you want a crack at this yourself as a learning experience, I will back off. Otherwise, when I get back from church this morning I'll help write a simple two line script to do the job and how to add the cron job to the init-start script so that your "cru a" command will survive a reboot.

Cherrs

Good Morning,

Ok right now thanks Jeffrey, as a Catholic I am glad to hear you still get to go to Church, ours are all closed right now, and zoom with a solo priest does not convey the same atmosphere or community for me.

Thank you for all the help guys, really appreciate it. If I get stuck I will shout.

I have now copied all the movie,music and software folders, got rid of a rogue folder that somehow got onto my jffs partition yesterday. Working on the tv shows now, then I will try adding to cron

rsync -av --delete /mnt/My_Passport_Ultra/Films /mnt/NETWORK-BACKUP/Backup/
rsync -av --delete /mnt/My_Passport_Ultra/TV /mnt/NETWORK-BACKUP/Backup/
rsync -av --delete /mnt/My_Passport_Ultra/Software /mnt/NETWORK-BACKUP/Backup/
rsync -av --delete /mnt/My_Passport_Ultra/Music /mnt/NETWORK-BACKUP/Backup/

 

[ColinTaylor]

So I have entered command rsync -av --delete /mnt/My_Passport_Ultra/Films /mnt/NETWORK-BACKUP/Backup/Films which seems to be backing up the films directory, however it has created a films folder with films for some reason.

What did I say ?

rsync -av /mnt/My_Passport_Ultra/Music/ /mnt/NETWORK-BACKUP/Backup/Music

Note that there is a trailing / on the source path and there is not a trailing / on the destination.

N.B. If you're going to use rsync you really should read the documentation. Most of it can be skipped over as it won't interest you, but it does include lots of useful examples. It's also helpful to at least be aware of some of the more advanced things it can do (like the --delete option in @Jeffrey Young's script).

 

[BreakingDad]

What did I say ?

Sorry sir missed that last night, as I said was exhausted

 

[BreakingDad]

So I think I've cracked it.

Added using the cron util with the following

cru a filmsbu "0 01 * * * rsync -av --delete /mnt/My_Passport_Ultra/Films /mnt/NETWORK-BACKUP/Backup/"
cru a tvbu "0 03 * * * rsync -av --delete /mnt/My_Passport_Ultra/TV /mnt/NETWORK-BACKUP/Backup/"
cru a softwarebu "0 05 * * * rsync -av --delete /mnt/My_Passport_Ultra/Software /mnt/NETWORK-BACKUP/Backup/"
cru a musicbu "0 06 * * * rsync -av --delete /mnt/My_Passport_Ultra/Music /mnt/NETWORK-BACKUP/Backup/"

crontab-e returns

30 3 * * * /jffs/addons/flexqos/flexqos.sh -check #flexqos#
25 11 * * * sh /jffs/scripts/firewall banmalware #Skynet_banmalware#
9 1 * * Mon sh /jffs/scripts/firewall update #Skynet_autoupdate#
0 * * * * sh /jffs/scripts/firewall save #Skynet_save#
51 */12 * * * sh /jffs/scripts/firewall debug genstats #Skynet_genstats#
0 01 * * * rsync -av --delete /mnt/My_Passport_Ultra/Films /mnt/NETWORK-BACKUP/Backup/ #filmsbu#
0 03 * * * rsync -av --delete /mnt/My_Passport_Ultra/TV /mnt/NETWORK-BACKUP/Backup/ #tvbu#
0 05 * * * rsync -av --delete /mnt/My_Passport_Ultra/Software /mnt/NETWORK-BACKUP/Backup/ #softwarebu#
0 06 * * * rsync -av --delete /mnt/My_Passport_Ultra/Music /mnt/NETWORK-BACKUP/Backup/ #musicbu#

I (think) this is it. Will delete a file later off main, and see it its gone from backup tomorrow


Thanks all.

 

[Jeffrey Young]

You can combine your cron syntex as "cru a 0 1,3,5,6 * * * command"

EDIT: Sorry @BreakingDad,I did not see that you have multiple command parameters. Ignore the above comment.

Also, you may need to enclose your actual rsync command in quotes to ensure all parameters get passed to rsync. That is way I usually perfer to build an independent script, then call the script via cron.

To Colin's point, you do understand that --delete will ensure a mirror copy of the source. Any files that you inadvertently place in the destination will get removed when rsync runs. It sounds like this is what you want, but I am making certain.