Resume rsync over SSH after broken connection?

I have to transfer large amounts of data (>80 GB) over ssh using rsync. Everything is working fine, but the DSL connection where the backup data is sent from will drop once every 24h for up to 3 minutes (switching providers is not an option).

How do I:

1. Automatically restart the transfer when the connection is back up?

2. Make sure there are not by accident two rsync commands running at the same time?

Solution 1:

The following should be helpful:

#!/bin/bash

while [ 1 ]
do
    rsync -avz --timeout=60 --partial source dest
    if [ "$?" = "0" ] ; then
        echo "rsync completed normally"
        exit
    else
        echo "Rsync failure. Backing off and retrying..."
        sleep 180
    fi
done

When the connection dies, rsync will quit with a non-zero exit code. This script simply keeps re-running rsync, letting it continue until the synchronisation completes normally.

Solution 2:

This does much the same as Peter's answer, but gives the user the option of which remote file he wants, and where he wants to save it (as well as conducting the rsync over ssh). Replace the USER and HOST with your username and host respectively.

#!/bin/bash
echo -e "Please enter the full (escaped) file path:"
read -r path
echo "Path: $path"
echo -e "Enter the destination:"
read -r dst
echo "Destination: $dst"
while [ 1 ]
do
    rsync --progress --partial --append -vz -e ssh "USER@HOST:$path" $dst
    if [ "$?" = "0" ] ; then
        echo "rsync completed normally"
        exit
    else
        echo "rsync failure. Retrying in a minute..."
        sleep 60
    fi
done

The rsync options used here enable the progress stats during transfer, the saving of partial files upon unexpected failure, and the ability to append to partially completed files upon resume. The -v option increases verbosity, the -z option enables compression (good for a slow connection, but requires more cpu power on both ends), and the -e option enables us to conduct this transfer over ssh (encryption is always good).

Note: Use this only if you have public key login enabled with your ssh, otherwise it will ask you for a password when it restarts (killing all functionality of the script).

Solution 3:

supervisor daemon (a process control manager) could work very well after creating rsa certificates of both sides, with a similar configuration as follows: (/etc/supervisor/ supervisord.conf is the configuration file path on debian based systems )

[program:rsync-remoteserver]
command=rsync -avz --progress [email protected]:/destination /backup-path
stdout_logfile=/out-log-path  
stderr_logfile=/errlogpath

Solution 4:

A quick and dirty way to do this when you don't feel like opening up an editor:

while ( ! rsync -avzP <source> <dest> ); do sleep 1; done

-a  archive mode
-v  increase verbosity
-z  compress file data during the transfer
-P  same as --partial --progress