Dropbox-Like File Sync without a central server [closed]

I'm looking for a way to sync the secondary drives on my two Macs across the internet. I have consistent VPN connectivity between the two macs, and I can access them as if they are sitting right next to each other.

I'm looking for a way to implement a Dropbox-like experience where if I modify a file on one Mac, the file would get pushed to the other Mac (and vice-versa). I also need the ability to throttle the bandwidth (so our little 1 megabit upload pipe doesn't get overloaded). Any suggestions?

Oh, and using dropbox is kind of out of the question at the moment since my drive is 500GB. Thanks!

Solution 1:

Check out Bittorrent Sync. It has the sync functionality like Dropbox, but without the cloud.

Solution 2:

If you want to go hardcore, you can use rsync command natively in OsX, like *nix staff.

Before everything, you must have the permission to access to the remote host by ssh. OpenSSH server is preinstalled on Mac OsX and you don't need to install any additional package. Just need to do some system settings. To enable the OpenSSH server on Mac OS X, open System Preferences and click to Sharing. And than, check the Remote Login box to enable SSH, then select the radio button labeled All Users from the Allow Access For section.

You can now access to the remote host by ssh.

Now, you must create a public access key for passwordless access from local guest machine to remote host over ssh. For that you must get your hand dirty a little bit. :)

First, determine if you already have authentication keys. In the Terminal, run:

sudo ls -la /var/root/.ssh

If you see "id_dsa" and id_dsa.pub, then you can skip the rest of this section.

On the client machine, run the following in the Terminal:

sudo ssh-keygen -t dsa -f /private/var/root/.ssh/id_dsa -C "comment about this key"

AFter created access key in local guest machine, you need to copy the guest's public key to the host's authorized_keys file. You can do this with a simple Terminal command that appends the public key to the list of authorized keys:

sudo cat /private/var/root/.ssh/id_dsa.pub | ssh root@remote_host_address 'cat - >> ~/.ssh/authorized_keys'

The command below that will perform an incremental backup of your root filesystem in local guest machine on to your remote host:

/usr/local/bin/rsync -aNHAXx --protect-args --fileflags --force-change --rsync-path="/usr/local/bin/rsync" / root@remote_host_address:/Volumes/Backup/GuestMachine

You can change parameters of rsync for change synchronization behaviors. If you add this command into crontab, synchronization can be run in any time cycles. For example:

sudo crontab -e

append line below to crontab:

*/30 * * * * /usr/local/bin/rsync -aNHAXx --protect-args --fileflags --force-change --rsync-path="/usr/local/bin/rsync" / root@remote_host_address:/Volumes/Backup/GuestMachine

This will be run synchronization in every half hour.

Here is the sample Python script:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""
Cron automated synchronization script.

Desc: Synchronize remote folder into local machine. Duplicate processes 
      are disallowed until running process finished.

Usage:
    :: Change required variables as _user, _password, _domain etc.
    :: Edit crontab 
    # crontab -e

    :: Append line below.
    */30 * * * * python synchronizator.py 2>&1 &

Author: Sencer HAMARAT (RecNes)
E-Mail: [email protected]
"""

import shlex
from subprocess import Popen, PIPE
import logging as log
import sys

__author__ = "Sencer HAMARAT"

_user = 'username'
_password = 'password'
_domain = 'example.com'

_expectation = "Enter passphrase for key \'/home/%s/.ssh/id_rsa\':" % _user
_rsync = '/usr/bin/rsync --partial --progress -avvz -e'
_pub_key = '/home/%s/.ssh/id_rsa.pub' % _user
_ssh = '/usr/bin/ssh -i %s' % _pub_key
_remoteDir = '/home/%s/backup/' % _user
_localDir = '/home/%s/backup/' % _user
_command = '%s %s %s@%s:%s %s' % (_rsync, _ssh, _user, _domain, _remoteDir, _localDir)
run_command = shlex.split(_command)

_logFile = "logfile.log"
_logFormat = "%(asctime)s %(levelname)s %(name)s %(process)d %(threadName)s %(module)s:%(lineno)d %(funcName)s() " \
             "%(message)s\n"
log.basicConfig(filename=_logFile, level=log.DEBUG, format=_logFormat)

log.debug(u'Command will run: %s' % _command)

try:
    running_command = Popen(run_command, stdin=PIPE, stdout=PIPE, stderr=PIPE)
    if running_command.poll():
        log.debug(repr(running_command.poll()))
        sys.exit()
    if _expectation in running_command.communicate():
        running_command.communicate(_password)
    print running_command.communicate()
except Exception as e:
    log.debug(repr(e))
finally:
    sys.exit()

Solution 3:

Owncloud is a possible solution if you want web based access.

You will need your own server to host Owncloud on, however it's more of a server based solution where your 2 clients can access what they need at will, but there will only be 1 copy of the file in question, which will be situated on the Owncloud server.

Solution 4:

I recently stumbled across Syncthing. Lot of cool feautures.

http://syncthing.net/