Linux: Continuously synchronize files, one way

Solution 1:

You can also use inotifywait from the inotify-tools package.

inotifywait -r -m -e close_write --format '%w%f' /tmp | while read MODFILE
do
    echo need to rsync $MODFILE ...
done

Solution 2:

Lsyncd would be a good solution for this.

Lsyncd watches a local directory trees event monitor interface (inotify or fsevents). It aggregates and combines events for a few seconds and then spawns one (or more) process(es) to synchronize the changes. By default this is rsync. Lsyncd is thus a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or blockdevices and does not hamper local filesystem performance.

Bottom-line, it uses the same kind of tools to do the job (inotify and rsync) as suggested by other answers, but it's easier to set up for someone not familiar with shell scripting.

Solution 3:

I need this a lot since my code needs to run on remote boxes and I write code on local machine. I found a nice tool which you can use to continuously monitor your local folders and sync them to remote or local folder: https://github.com/axkibe/lsyncd

A simple command to continuously sync a local dir with remote machine over ssh will be:

lsyncd -log all -nodaemon -rsyncssh <local_path> <user>@<ip> <remote_path>

Just like with any other rsync command, make sure you give the folder path right and check before you run the command. I almost had killed one of my remote machine because I missed to give a correct destination directory. Make sure yo don't miss out the remote path and don't use '/' unless you know what you are doing.

Solution 4:

If you need to observe filesystem, then inotify is the way to do it. I would write a simple python script using pyinotify to perform sync when filesystem get changed. See documentation. You might also checkout the autosync.py for some inspiration. Have fun.

Solution 5:

What I did once is have a bash script running ls -l in a loop (with some sleep) and comparing to the previous output. If it changed, do your synchronization.

#!/bin/bash

listcommand="ls -l $*"

newfilelist=$( $listcommand )
while true
do
   if [[ $oldfilelist != $newfilelist ]]
   then
      oldfilelist=$newfilelist
      # run your synchronization tool
   fi
   sleep 10 || exit 2 
   newfilelist=$( $listcommand )
done

Start this script in a new terminal with the file names as arguments (after putting in your synchronization tool).

(I used this to start a compilation, not to synchronize, but this would work a similar way.)