How can I sync my home folder over network and internet?

I use my laptop with Ubuntu-desktop to do all my work, but I also have a low-end desktop over at my office just sitting there. I've decided I'm going to install Ubuntu-server on it and use it to mirror my entire laptop home folder, to make things easier when I decide to format my laptop's hard-disk.

Whenever I'm at work, both machines are connected to a network and communicate easily (and high-speed) via ssh. When I'm not at work, the desktop is still accessible via ssh. Ideally, the syncing would take place automatically in the background, whenever I change something. It only needs to be one way: the changes I make on the laptop have to be synced over to the server, but the inverse is not necessary.

I know there's software for this out there, my question is: What software can I use to achieve the above objectives and also take full advantage of local-network speeds when I'm at work? Since I'll sometimes deal with large files, the syncing process needs to realise that the two computers are sharing a local network, and then take advantage of that (instead of always syncing through the internet).

Just to be clear, over-the-network syncing is actually more important to me here than over-the-internet syncing. I ideally the software would check if the former is available and, if not, try the latter; but if that's not possible, the first case is my priority.

Hope this isn't too long. Thanks in advance.

If you can only connect to your office server via SSH, then your best choice is to use rsync: it can use SSH as a transport protocol and uses a smart algorithm to speedup the transfer of large files, by sending only the changed blocks.

Since you only need one-way synchronization, just set up passwordless SSH authentication from your laptop to the office server, and then you can start with a command as simple as:

rsync -e ssh -a $HOME/ myuser@officeserv:

adding --include and --exclude options to refine the list of files/directory that you want to synchronize. For instance, transferring program settings ("dot files") can be risky if the two computers do not run the same OS (same version). My suggestion is to start by excluding all "dot files" (so, use --exclude="$HOME/.[a-z]*") and then selectively add the configuration directories of programs that can safely be shared (this has to be seen on a program by program basis). In addition, web browser cache and $HOME/.cache can always be excluded. See the "FILTER RULES" section in the rsync man page for a detailed discussion of the include/exclude rules syntax.

However, rsync does not have a "continuous-operations" mode, so you will have to run it periodically from your crontab.

I would suggest using rdiff-backup over rsync. It's pretty much just rsync++.

unlike rsync which just a 1-1 mirror that transfers diffs, rsync has a history mechanism. So if you screwed up and deleted something important, you can revert back to a week ago and get it back.

On the ubuntu "server" you just need to have ssh running to apt-get install rdiff-backup.

on the client. I would either run this manually.. or via a crontab.

#!/bin/bash

  #backup.sh

  rdiff-backup -v5 --exclude path-to-annoyingfile  /home/user/ hostname::/media/data/snapshots/laptop 
  ## this will cleanup any backups older then 2weeks.  Adjust this as needed.
  rdiff-backup --remove-older-than  2W --force  hostname::/media/data/snapshots/laptop


  #!/bin/bash
  ##Restore.
  rdiff-backup -v5 --restore-as-of now --force user@hostname::/media/data/snapshots/laptop  /home/user

If you open up your port 22 to the internet.. you should be able to run the same script either ways.. just have it your hostname resolve locally to your local IP.

ie. backupSrv.penguins.org would resolve to the external IP of. 2.3.4.5 but inside your lan: to 192.168.1.253

There is no complete, pre-packaged solution to your problem that I know of. You probably have to write some small script yourself for that.

The syncing itself could be done by rsync, as already explained by Riccardo Murri. Rsync only transfers the changed parts of files, so it is perfect for this task.

You can use the NetworkManagerDispatcher to execute scripts on connection/disconnection of a network interface. So you could write a script that checks if you are on the correct network and then calls rsync. This way your data would be automatically synced when you connect to your company network.

To periodically sync your data you can use cron, as already mentioned.

You should also consider security when using rsync via ssh. To synchronize without user intervention you'll need a passwordless keyfile. Anyone who has this keyfile can gain access to the server it belongs to. I would strongly recommend to encrypt your home folder to protect the data on the notebook and on the server.