Skip to content

Instantly share code, notes, and snippets.

@Tho85
Last active August 7, 2024 12:55
Show Gist options
  • Save Tho85/6045429 to your computer and use it in GitHub Desktop.
Save Tho85/6045429 to your computer and use it in GitHub Desktop.

Build your own private, encrypted, open-source Dropbox-esque sync folder

Prerequisites:

  • One or more clients running a UNIX-like OS. Examples are given for Ubuntu 12.04 LTS, although all software components are available for other platforms as well (e.g. OS X). YMMV
  • A cheap Ubuntu 12.04 VPS with storage. I recommend Backupsy, they offer 250GB storage for $5/month. Ask Google for coupon codes.

Software components used:

  • Unison for file synchronization
  • EncFS for folder encryption

Set up file synchronization with unison

On your server:

  • Install Unison:

    $ sudo apt-get install unison
  • Add a user for our project and give him a decent password:

    $ sudo adduser encbox

On your client(s):

  • Install Unison:

    $ sudo apt-get install unison
  • Enable key-based authentication for SSH (replace your.vps.com with your VPS' hostname):

    $ ssh-keygen
    $ cat ~/.ssh/id_rsa.pub | ssh encbox@your.vps.com "mkdir ~/.ssh; cat >> ~/.ssh/authorized_keys"
  • Create the folder holding our encrypted files:

    $ mkdir ~/.encbox
  • Create a unison profile in ~/.unison/encbox.prf (replace your.vps.com with your VPS' hostname):

    $ mkdir ~/.unison; tee ~/.unison/encbox.prf <<__EOF
    root = /home/$USER/.encbox
    root = ssh://encbox@your.vps.com/encbox
    prefer = ssh://encbox@your.vps.com/encbox
    
    backups = true
    times = true
    terse = true
    
    repeat = 2
    __EOF
  • Create an upstart file to start synchronization automatically (in /etc/init/encbox.conf):

    $ sudo tee /etc/init/encbox.conf <<__EOF
    description "encbox"
    
    start on desktop-session-start
    stop on desktop-shutdown
    
    setuid $USER
    setgid $USER
    env HOME=$HOME
    
    respawn
    
    script
        bash -l -c "unison encbox; sleep 2"
    end script
    __EOF
  • Start Unison:

    # either run in foreground:
    $ unison encbox
    
    # or run it as a system service:
    $ sudo service encbox start # starts your Encbox
    $ sudo service encbox stop  # stops your Encbox

    If unison complains about archive files on your client, run it once with the -ignorearchives flag:

    $ unison -ignorearchives encbox

Create an encrypted folder using EncFS

On your client(s):

  • Install EncFS:

    $ sudo apt-get install encfs
  • Mount encrypted directory:

    $ encfs ~/.encbox ~/Encbox

    Agree to have your target directory created, then choose p to use EncFS' preconfigured paranoia mode. Give your folder a decent password.

  • That's it! All files in ~/Encbox will be encrypted and synced securely between your VPS and all clients.

Tips

  • Polling:

    Unfortunately, Unison doesn't support file system notifications (i.e. inotify, libnotify), so it polls your file system for changes every 2 seconds (if you set repeat = 2 in your profile as we did above). This isn't nice, but I did not observe any negative effects on my notebook's battery life. Feel free to set the repeat parameter to some higher value if you observe negative effects. Bandwith usage for polling is negligible as well (~0.9 kB/poll, ~3.5 kbit/s).

  • Unison version

    Rumor has it you should use the same Unison version on your server as well as on all your clients, or you will run into problems. I didn't test it, though.

  • Android / iOS app:

    This setup is compatible with Boxcryptor Classic's smartphone apps. You just have to create an EncFS folder in expert mode (x) and follow the instructions at Boxcryptor's Support Desk. Then install a WebDAV server of your choice and expose /home/encbox/encbox via WebDAV (not in this document's scope).

@priyadarshan
Copy link

It seems unison is "is no longer under active development". http://en.wikipedia.org/wiki/Unison_(file_synchronizer)

Copy link

ghost commented Jul 21, 2013

I'm using Seafile in production as a replacement for Dropbox. Supports repository-level encryption of data at rest, open source and a user-friendly version management GUI. Clients for Windows / Mac / Linux and Android / iOS.

@reyman
Copy link

reyman commented Jul 21, 2013

There is also git annex :)
http://git-annex.branchable.com/

@TobiX
Copy link

TobiX commented Jul 21, 2013

Isn't this what git-annex was designed to do? With the "new" git-annex assistant, it's pretty user-friendly, too.

@gltiich
Copy link

gltiich commented Jul 22, 2013

I created a DropBox clone not long ago, used to have the instructions online but not anymore. I mostly relied on lsyncd which supports inotify and is open source;

https://code.google.com/p/lsyncd/

My goal was to make it cross-platform without writing any Windows services, so I created a 'mini vm' which mounted a windows directory via SMB, then that VM ran lsyncd and vola, inotify against a Windows directory. It actually worked but was way too crude, lol.

Unison is a bad way to go, wasteful and inelegant.

There just will not be a good DropBox clone until someone decides to write cross-platform services. Until that day you probably won't do better than a solution built on lsyncd.

There's also incrond, probably not a great option because it'd require a lot of scripting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment