make a cronjob wait for previous rsync job to finish

I'm using rsync to backup some data from one server to another. All works fine, but it might take longer to finish depending on how much data there is to transfer.

Is there any gauranteed way to ensure that an rsync command doesn't start before the previous one finished using a cronjob?

For example, every hour I run the rsync command, but its possible the transfer takes more than 1 hour to complete, so the next one would start before the previous one finishes.

Solution 1:

You can implement some kind of locking. This will print the number of rsync processes still running:

pgrep -cx rsync

And this will run the rsync only if no other rsync process exists:

pgrep -cx rsync || rsync ...

Using -x will prevent from accidentally matching unwanted names (for example "foobarsynchronizator" or "not_an_rsync_totally" - it works just like pgrep -c ^rsync$)

Solution 2:

You can use the flock command to help you do this e.g. In this case flock -n is probably what you want as it will cause an immediate failure of the command if it can't obtain the lock e.g.

30 * * * *  /usr/bin/flock -n /tmp/myRsyncJob.lck /path/to/your/rsyncScript 

Solution 3:

If you're willing to consider other tools, you could also have a look at rdiff-backup. It uses librsync to do backups, and saves a configurable number of deltas/increments. It also locks so that only one rdiff-backup process can run at any given time.