Gitdocs: Open-source Dropbox using Ruby + Git | Hacker News Reader

Gitdocs: Open-source Dropbox using Ruby + Git

rubyflow.com

85 pointsjoshbuddy29 comments

This is a good effort and it's always good to see people using a SCM repository as a collaborative document store. Even better that it has such a nice simple command-line.

However, all four things that make Dropbox Dropbox for me are missing:

* A UI so straightforward that my technophobic mother-in-law (or even a project manager) can use it

* Block-level sync (in particular, the rsync algorithm) with server-side dedupe on binary files (git is suboptimal in my experience on large binaries, and it's unclear whether this project resolves the issue). Incidentally, integrating that with simple versioning behaviour is what lifts dropbox into the "Ooh, clever, I see what you did there" category for me.

* Local-area peer-to-peer sync.

* Click-to-share instant access granting.

Building an open-source tool that has all four is on my to-do list. Unfortunately it's quite far down that list. Anyone else having a crack at it?

I'm having a go, as a side-side project, at defining a strictly RESTful protocol for implementing DropBox-like functionality - I'm building client and server components which will be FOSS. I started this mainly as a learning experience for designing a strictly RESTful interface, but I plan to use it personally. Some random notes:

- Strictly RESTful interface (HATEOS, single base URL, server controlled namespaces)

- Simplicity over performance - easy to do everything on the command line using curl

- Supports block level transfers

So far this has stricly been for my own amusement/education - happy to write something up and put the code on github if anyone is interested.

I think it needs more strict RESTfulness

Quite... the dangers of posting in a hurry :-)

While I was repeating myself a bit there I meant to get across that when I started I thought I knew what a RESTful interface was (nice URLs, HTTP GET/POST/PUT/DELETE) then I discovered that the term is used pretty loosely and a lot of what are described RESTful interfaces aren't really anything like what Roy Fielding meant.

I originally linked this without context. But, now I see they've given up on the UI aspect. :-(

http://www.syncany.org/

From my perspective, the main Dropbox feature that all the "thin layer on top of VCS" clones miss is the conflict-resolution behavior of Dropbox. If both you and someone else edit a file called "stuff.txt" at the same time, you'll end up with two files: "stuff.txt" and "stuff.txt (other guy's version)". Neither person's changes are blown away, there's never any merge conflicts, and the history remains linear.

I'll have to see if Gitdocs does anything like that.

Oo. Great idea. I'll add that.

It's not as simple as it sounds. What if there's already a file called "stuff.txt (other guy's version)"? There's lots and lots of edge cases. You basically need a function that can always return an appropriate but not-already-existing filename.

I think graceful file conflict resolution will be an essential part of gitdocs and we will work to handle these edge cases as best as we can. Thanks for bringing it up.

You might want to look at this bit: https://github.com/hbons/SparkleShare/blob/master/SparkleLib...

It tries to copy the Dropbox conflict behaviour.

Interesting. I'll have to try it out again.

Although, if I'm reading this correctly, the logic isn't quite right. It uses the timestamp of the time when the conflict is detected, when it should really use the conflicted file's last-modified time, or maybe the timestamp of the conflicting commit.

"stuff.txt (sha1)" works. User-friendliness of it depends of user base (and might actually be a feature as it traces back to the exact history point)

This requires another "merge strategy" in git terminology.

http://www.cs.potsdam.edu/cgi-bin/man/man2html?1+git-merge#l...

Git is terrible at dealing with big binary files and by terrible I mean very slow, it's optimized for text files only. Linus wrote about that a while ago, you can google it.

While I'm a big fan of open source in general and of using standard components yadda yadda yadda

For me the biggest draw of Dropbox is that it's not self hosted but that it's run by an entity that's 1) large enough to not go away any time soon, 2) fairly unscary (unlike, say, facebook), and 3) 100% dedicated to keeping the service running so I don't have to worry about my files anymore.

Self hosting only gets you 2).

But in this case, we are storing your files in a simple git repo defined by you. A git repo you could store on say github or bitbucket of which both are git hosts that are not going away anytime soon. In a way, this is just replacing a hosted dropbox with a hosted git repo should you choose to use it that way.

Do github and bitbucket look kindly upon storing gigabytes worth of data on their services? The overviews of their plans don't mention anything about size, but it seems to me you're stretching their terms and conditions quite a bit when you start using their services for hosting gigabytes worth of binary data (which is what I use Dropbox for, ymmv)

For me the biggest misssing feature of all approaches to online folder synchronisation (including Dropbox, in a sad way) is secure backend encryption. For me, this includes an open source client and a free choice, where to store my files. Ideally I could have several decrypting clients to work with my files, while an arbitrary amount of encrypted clients would store my files on various backends, one of which could be a company offering this as service.

It seems like Gitdocs fits the requirement for encryption. Just use git+ssh for the remote of the repo, and transfers will be encrypted. Then, I guess, you need to set up ecryptfs or something on the server side so that things are encrypted on-disk, and then you can back up the encrypted files.

Not simple, but possible.

I would use this for my company if I were able to install something as an OS X app. I'm not comfortable on the command line, and I want contractors and new employees to come on board just as easy (I don't want to have to admin their machines.)

This is very interesting indeed. I wouldn't mind paying for an OS X app that would take care of this problem (the setting up on each machine).

Not a bad idea at all. Gui wrappers for osx, and later linux and windows would be an awesome way to allow this to be used by less tech saavy people in the future. I suspect an osx cocoa wrapper would be pretty easy to write. We also plan to build out a web front-end that allows users to browse,edit,and upload files from right within their browser.

Does this address any of the issues people have with SparkleShare? (Another dropbox clone based on git - http://www.sparkleshare.org/)

Well, it's probably much simpler. And it relies on fsevents to do the pushes. But, aside for that, what issues are people having with sparkleshare?

Lack of windows support and the difficulty to install the server side components are the two ones that I've heard a lot of complaints about.

SparkleShare doesn't require any special server side parts: just git-core, which may be easy or hard to set up depending on your sysadmin skills.

We plan to address those issues, namely cross-platform support and easy install, and of course striving to keep things as simple as possible.

Hm, if I use this to store large files 50+ MB that don't compress well, won't the local repo grow out of proportion soon? Git will keep around the history. Or is this disabled?

How does this compare to lipsync in terms of binary files?

https://github.com/philcryer/lipsync

This is cool. A simple hack, but opens up many possibilites like versioning and remote backup of documents.

Reply on news.ycombinator.com