Does rsync --inplace write to the entire file, or just to the parts that need to be updated? (for btrfs+rsync backups)
I was reading several guides how combine btrfs snapshots with rsync to make an efficient backup solution with history. However it all depends on if rsync --inplace
modifies only those portions of files that actually changed, or if it overwrites the whole file sequentially. If it writes the whole file then it seems that btrfs will always create a new copy of the file, which would make the idea much less efficient.
If you pass rsync two local paths, it will default to using "--whole-file", and not delta-transfer. So, what you're looking for is "--no-whole-file". You also get delta-transfer if you requested '-c'.
Here's how you can verify:
$ mkdir a b
$ dd if=/dev/zero of=a/1 bs=1k count=64
$ dd if=/dev/zero of=a/2 bs=1k count=64
$ dd if=/dev/zero of=a/3 bs=1k count=64
$ rsync -av a/ b/
sending incremental file list
./
1
2
3
sent 196831 bytes received 72 bytes 393806.00 bytes/sec
total size is 196608 speedup is 1.00
Then touch a file and re-sync
$ touch a/1
$ rsync -av --inplace a/ b/
sending incremental file list
1
sent 65662 bytes received 31 bytes 131386.00 bytes/sec
total size is 196608 speedup is 2.99
You can verify it re-used the inode with "ls -li", but notice it sent a whole 64K bytes. Try again with --no-whole-file
$ touch a/1
$ rsync -av --inplace --no-whole-file a/ b/
sending incremental file list
1
sent 494 bytes received 595 bytes 2178.00 bytes/sec
total size is 196608 speedup is 180.54
Now you've only sent 494 bytes. You could use strace to further verify if any of the file was written, but this shows it at least used delta-transfer.
Note (see comments) that for local filesystems, --whole-file
is assumed (see the man page for rsync). On the other hand, across a network --no-whole-file
is assumed, so --inplace
on its own will behave as --inplace --no-whole-file
.
Here the definite answer I guess, citing the correct part of the manual:
--inplace
[...]
This option is useful for transferring large files
with block-based changes or appended data, and
also on systems that are disk bound, not network
bound. It can also help keep a copy-on-write
*************
filesystem snapshot from diverging the entire con‐
*******************
tents of a file that only has minor changes.
rsync's delta transfer algorithm deals with whether the entire file is transmitted or just the parts that differ. This is the default behavior when rsyncing a file between two machines to save on bandwidth. This can be overriden with the --whole-file
(or -W
) to force rsync
to transmit the entire file.
--inplace
deals with whether rsync
, during the transfer, will create a temporary file or not. The default behavior is to create a temporary file. This gives a measure of safety in that if the transfer is interrupted, the existing file in the destination machine remains intact/untouched. --inplace
overrides this behavior and tells rsync
to update the existing file directly. With this, you run the risk of having an inconsistent file in the destination machine if the transfer is interrupted.
--inplace
overwrites only regions that have changed. Always use it when writing to Btrfs.