Operating `rsync`

Size & timestamp (default)Size & timestamp (default)

Typical usage

rsync -avz --delete src dst

Beware of the trailing-slash on src/. If there is, its content will be replicated onto destination, not the folder.

Default behaviour

Check if files are identical in size
Check if timestamps have changed

Size-onlySize-only

This does not only ignore timestamps, but also filesize

-I, --ignore-times    don't skip files that match size and time

Therefore we need to ignore the timestamps only with

--size-only      skip files that match in size

Size-only & no permsSize-only & no perms

Instead of -a (which stands for -rlptgoD), you may also need to avoid p o g D in case you are re-generating some shit every time. Hence this is size-only.

rsync -rltvz --size-only --delete src dst

Checksum & no permsChecksum & no perms

in case you need to maintain different file permissions on source and destination

rsync -rltvz --checksum --delete src dst

Additional notesAdditional notes

Only use compression (-z) if that is over the network and not dealing with already compressed archives like music, films or anything not worth the cpu overhead.

dst/ doesn’t have to exist yet, rsync eventually creates it

Eventually switch to --rsh=rsh to speed up the process on a trusted network

Eventually add e.g. -e "ssh -palt_port" to tune SSH client options

Something usefull you might want to add to your scripts, not to cumulate rsync processes in parallel,

tmp=`ps aux | grep rsync | grep -v grep`
[[ -n `ps ax | grep rsync | grep -v grep` ]] && cat <<-EOF && exit 1
RSYNC IS ALREADY RUNNING, ABORDING
$̂tmp
EOF
unset tmp

One rsync job may have two child processes, in the end you have three

the generator
the sender
the receiver

Remotely sync from one server folder to anotherRemotely sync from one server folder to another

make a backup first

ssh $dest cp -a $target/ $target.bkp/

proceed with the dual sync

ssh $source tar czpf - ABSOLUTE-PATH/| ssh $dest tar xzpf - -C / -P

see what changed

ssh $dest diff -rbu --brief $target.bkp/ $target/

and do not forget to reload the daemon when needed on destination server

TroubleshootingTroubleshooting

   bash: line 1: rsync: command not found

==> you need rsync installed on both ends

ResourcesResources

Parallelized rsync-like clone https://github.com/pscedu/psync

Why does rsync spawn multiple processes for me? http://serverfault.com/questions/547165/why-does-rsync-spawn-multiple-processes-for-me

Why does rsync forks itself? And why one such forked process is almost kinda idle (as seen in iotop)? http://serverfault.com/questions/460423/why-does-rsync-forks-itself-and-why-one-such-forked-process-is-almost-kinda-idl

bi-directional sync

https://stackoverflow.com/questions/1602324/how-do-i-synchronize-in-both-directions –> –update

two remote hosts: alternatives

https://unix.stackexchange.com/questions/58037/recursive-scp-without-following-links-or-creating-a-giant-tar-file

https://stackoverflow.com/questions/11030394/is-it-possible-to-make-scp-ignore-symbolic-links-during-copy

https://unix.stackexchange.com/questions/183504/how-to-rsync-files-between-two-remotes

https://serverfault.com/questions/449705/why-is-it-not-possible-to-use-two-remotes-for-rsync

https://stackoverflow.com/questions/29941966/rsync-cwrsync-in-gitbash-the-source-and-destination-cannot-both-be-remote

Operating rsync