rsync to compressed archive of file
I take backups of a server with rsync, but those backups are quite large, and I'd like to compress them. Is there a way to have some sort of wrapper around rsync to gunzip the file in the backup, rsync the changes from the live file, then re-gzip the file as soon as it is copied?
I.e. if the source files on the live are foo, bar and baz, the backup has foo.gz, bar.gz and baz.gz.
To restate: I want compressed files at one end and uncompressed files at the other end. I don't want to compress before rsyncing, because even with --rsyncable, it makes rsync less efficient. I know about the -z option to rsync. I don't have space on the backup machine to store all the files uncompressed.
I'm not sure I understand what you're trying to do, but you could create gzipped backups on the main server making sure that they're readily rsyncable and not bother decompressing anything in transit.
gzip(1)
--rsyncable
While compressing, synchronize the output occasionally based on
the input. This increases size by less than 1 percent most
cases, but means that the rsync(1) program can much more effi‐
ciently synchronize files compressed with this flag. gunzip
cannot tell the difference between a compressed file created
with this option, and one created without it.
I think that rsyncing to a fuseCompress mounted filesystem might be the best option. The project's history specifically mentions them making improvements to rsync's performance with it.
http://code.google.com/p/fusecompress/
https://github.com/tex/fusecompress/