Operating rsync


We are backuping target server through SSH/rsync from the backup server. Alternatives are:


rsync’s default behaviour:

Note --ignore-times does not only ignore timestamps, but also filesize…

        -I, --ignore-times          don't skip files that match size and time

So if you need to ignore the timestamps only, instead you should use,

        --size-only             skip files that match in size

Typical usage for backups:

rsync -avz --delete source dest

Note. using compression (-z) as its over the network. For a local rsync or a dedicated link eventually disable this option. Also remove this options if you’re dealing with already compressed archives, music, films or anything not really worth an additional compression.

Note. if you add a trailing slash to source/, this means its content will be straighly replicated w/o the ‘source’ foldername into the destination.

Note. /path/to/backup/folder/ doesn’t have to exist yet, rsync eventually creates it.

Note. you may add –rsh=rsh to speed up the sync on a secure network.

Note. add -e “ssh -palt_port” to use an alternate SSH port

From backup server to hosts by ssh

In this example, the host to be backed up (let’s say the target) has its local backups as archive files in the /var/backup/ folder. The backup server will sync its mirror each night into /path/to/backup/folder/. It is the backup server that connects to the target, so if the target server is compromised you would still have its backups and backup logs available.

On the backup server

Make sure rsync package is installed. Not enabling the daemon.

Configure SSH without a password and check,

ssh target

disconnect from the session and try,

rsync -avz --delete root@target:/var/backup/ /backup/target/

Note the trailing slash on /var/backup/, this means its content will be straighly replicated w/o the foldername

You can now automate the thing,

crontab -e

add e.g.,

0 3 * * * /usr/bin/rsync -avz --delete root@target:/var/backup/ /path/to/backup/folder/

note. the verbose output will send to you daily by email (eventually configure mail aliases).

Check the latest backups are correctly synced, ls -ltr /path/to/backup/folder/ Note. rsync preserves timestamps so for this setup where we are just replicating tarballs, you can see the date of the original-local-backup on the target.

Additional notes

You might need to tweak the options a little bit. Instead of -a (which stands for -rlptgoD), you may need to avoid p o g D in case you are re-generating some shit every time. Hence this is size-only.

rsync -rltvz --size-only --delete source/ SERVER:/destination/folder/

Something usefull you might want to add to your scripts, not to cumulate rsync processes in parallel,

tmp=`ps aux | grep rsync | grep -v grep`
[[ -n `ps ax | grep rsync | grep -v grep` ]] && cat <<-EOF && exit 1
unset tmp

BTW, don’t be afraid of multiple rsync processes, as one job may use three processes, that’s totally normal:


Why does rsync spawn multiple processes for me? http://serverfault.com/questions/547165/why-does-rsync-spawn-multiple-processes-for-me

Why does rsync forks itself? And why one such forked process is almost kinda idle (as seen in iotop)? http://serverfault.com/questions/460423/why-does-rsync-forks-itself-and-why-one-such-forked-process-is-almost-kinda-idl

Nethence | Pub | Lab | Pbraun | SNE Russia | xhtml