Operating rsync

Size & timestamp (default)

Typical usage

rsync -avz --delete src dst

Beware of the trailing-slash on src/. If there is, its content will be replicated onto destination, not the folder.

Default behaviour

Size-only

This does not only ignore timestamps, but also filesize

-I, --ignore-times          don't skip files that match size and time

Therefore we need to ignore the timestamps only with

--size-only             skip files that match in size

Size-only & no perms

Instead of -a (which stands for -rlptgoD), you may also need to avoid p o g D in case you are re-generating some shit every time. Hence this is size-only.

rsync -rltvz --size-only --delete src dst

Checksum & no perms

in case you need to maintain different file permissions on source and destination

rsync -rltvz --checksum --delete src dst

Additional notes

Only use compression (-z) if that is over the network and not dealing with already compressed archives like music, films or anything not worth the cpu overhead.

dst/ doesn’t have to exist yet, rsync eventually creates it

Eventually switch to --rsh=rsh to speed up the process on a trusted network

Eventually add e.g. -e "ssh -palt_port" to tune SSH client options

Something usefull you might want to add to your scripts, not to cumulate rsync processes in parallel,

tmp=`ps aux | grep rsync | grep -v grep`
[[ -n `ps ax | grep rsync | grep -v grep` ]] && cat <<-EOF && exit 1
RSYNC IS ALREADY RUNNING, ABORDING
$̂tmp
EOF
unset tmp

One rsync job may have two child processes, in the end you have three

Remotely sync from one server folder to another

make a backup first

ssh $dest cp -a $target/ $target.bkp/

proceed with the dual sync

ssh $source tar czpf - ABSOLUTE-PATH/| ssh $dest tar xzpf - -C / -P

see what changed

ssh $dest diff -rbu --brief $target.bkp/ $target/

and do not forget to reload the daemon when needed on destination server

Troubleshooting

   bash: line 1: rsync: command not found

==> you need rsync installed on both ends

Resources

Parallelized rsync-like clone https://github.com/pscedu/psync

Why does rsync spawn multiple processes for me? http://serverfault.com/questions/547165/why-does-rsync-spawn-multiple-processes-for-me

Why does rsync forks itself? And why one such forked process is almost kinda idle (as seen in iotop)? http://serverfault.com/questions/460423/why-does-rsync-forks-itself-and-why-one-such-forked-process-is-almost-kinda-idl

bi-directional sync

https://stackoverflow.com/questions/1602324/how-do-i-synchronize-in-both-directions –> –update

two remote hosts: alternatives

https://unix.stackexchange.com/questions/58037/recursive-scp-without-following-links-or-creating-a-giant-tar-file

https://stackoverflow.com/questions/11030394/is-it-possible-to-make-scp-ignore-symbolic-links-during-copy

https://unix.stackexchange.com/questions/183504/how-to-rsync-files-between-two-remotes

https://serverfault.com/questions/449705/why-is-it-not-possible-to-use-two-remotes-for-rsync

https://stackoverflow.com/questions/29941966/rsync-cwrsync-in-gitbash-the-source-and-destination-cannot-both-be-remote


HOME | GUIDES | LECTURES | LAB | SMTP HEALTH | HTML5 | CONTACT
Copyright © 2024 Pierre-Philipp Braun