SparkleShare: Open Source Dropbox clone coming soon | Hacker News Reader

SparkleShare: Open Source Dropbox clone coming soon

sparkleshare.org

113 pointsbarnaby50 comments

This isn't a threat to dropbox becaue (as I read it) you have to set up your own server, or maybe hook up on a friends server. While this is certainly a great feature that a lot of people will value, the beauty of dropbox is that it just works. Without you having to install a server.

I see the following obstacles with installing a server for sparkleshare:

1) You need a server. This might be obvious, but normal people don't have one.

2) You need to monitor your server. If the architecture is anything like dropbox it doesn't need to have 100% uptime since synchronizing will just be delayed a bit, but you still need to check in every now and then to make sure everything is OK.

3) Installing a server has never been easy. There is always some sort of problem with port forwarding, IP adresses, proxies, etc. unless you have a server somewhere in the cloud which most people don't. Normal people are barely aware of what a server is. See point 1.

4) Your backups are still your worry. What makes dropbox great is that I know that if they lose my data they have a serious business problem - they'll be held accountable. I'll write rants on my blog and everyone on HN will read it. They'll go through great lengths to prevent this. Besides their knowledge of backup procedures, RAID disks, redundancy and what have you is far far greater than mine. (At least I hope so :-)) If I host my files on my own server I'm the one that's accountable.

I see this as a great service for companies that have some IT staff and aren't comfortable having all their company files hosted by another company, and hackers that prefer their own server. But it's not a threat to dropbox, which is getting popular among non-techies. My mom even has dropbox installed.

If Sparkleshare develops some momentum, you could easily see third-parties compete to offer Sparkleshare storage as a service, in a similar manner to how Dropbox works today.

You could.

But you'd only introduce more variables for the enduser. All of a sudden you'd have to decide which third-party service to sign up for, and many of them might just be a guy in a basement running the service to pay for college tuition. You'd have to check up on this before you know for sure.

The complete setup procedure for dropbox is as follows:

1) Navigate to dropbox.com

2) click download

3) doubleclick the file when you're done downloading

This is very hard to compete with in terms of ease of use. No strange choices to make, no weird preferences to fill out, no problems. It just works.

Sure, but dropbox themselves aren't exactly a blue chip corporation with a long-established reputation. There's no need for additional third-party configuration, either: $X.com could easily host a custom build of SparkleShare configured for their site. Hence, the straightforward installation process you describe could work equally well with s/dropbox/$foo/g.

It is a Y Combinator backed company, and has some reputation.

> All of a sudden you'd have to decide which third-party service to sign up for, and many of them might just be a guy in a basement running the service to pay for college tuition. You'd have to check up on this before you know for sure.

Isn't this the same for everything? What you seem to be advocating is that we centralize everything so that no one has to think or research when making purchases.

> This is very hard to compete with in terms of ease of use. No strange choices to make, no weird preferences to fill out, no problems. It just works.

But you have to decide to use DropBox in the first place. The crux of your argument seems to be that DropBox is easy because they are the only choice.

Using that logic, one could make the case that the entire Internet should run off of a single email provider with branded client software.

Basically my point is that, however you cut it, this service introduces complexity because you have to set up a server.

What makes dropbox so successful is the utter simplicity of something that used to be hard. The internet is littered with companies that have tried to solve this problem but didn't gain traction because it didn't just work. Dropbox is (to my knowledge) the first service of its kind that you install and forget. The thing that most people forget on HN is that 95% of the population don't even know what a server is - and they don't care.

The thing is, sadly, that open source has a pretty dismal track record of creating software that is easy and intuitive to use. There are many reasons for this. One is that open source software tends to be targeted towards developers instead of end users. Another is that it's often design by committee and mailing lists.

I'm not saying that this project doesn't have merit, or that it wont work. I'm just saying that they'll never be able to target my mom.

Exactly, right now dropbox is 9.95/month for 2Gig and frankly overpriced. I would love to have that right now at 1.99/month. Maybe unreasonable... but I like competition.

Well, $9.95/month for 50GB, but I agree that some competition would be welcome.

$9.95/month for 50GB seems a bit steep to me.

Uninformed question...I assume dropbox syncs machines via their server (ie: all data is synched up to their server first, then back down?) Or does it do peer to peer? (and if so, $9.95/50GB seems really expensive.) While less reliable, this would save them tons of bandwidth, and while it wouldn't work for some people, for me it would be perfect as I always have at least one machine that is on the net that would suffice for my master.

They do both.

They sync directly to other machines over LAN and also send the files to their servers. Bandwidth is saved through other means. The files are split up into chunks for which they compare hashes. If a chunk has already been uploaded to their servers by ANY Dropbox user it won't get reuploaded again. This is especially effective for media sharing, because lots of people will download the same torrents and put them in their Dropbox folders to watch somewhere else.

Downvoted because dropbox is a y-combinator spawn, or we don't like competition?

How would you like 2G for free? Because that's actually the deal with Dropbox. No charge unless you want >2G.

I meant 50 gig - the paid account.

It depends what they define as a "server". If you have to set up a Linux box somewhere with decent upstream, then yeah, I'm not spending any time on that. And I'm the sort of person who could do it.

However for what I use Dropbox for, I'd be perfectly fine if the "server" was just an old Windows box running out of my basement. If there's nothing more onerous than a port-forward it might be useful.

But either way I doubt the guys at Dropbox have anything to worry about. To hurt them this doesn't have to appeal to the average Dropbox customer. It only has to appeal to the average paying Dropbox customer. There's a big difference there, but probably not big enough to matter.

Being open source, I suspect it's only a matter of time before someone writes a plugin/fork that works with Amazon S3. The user would then only need to sign up for an S3 account and that largely removes all four of the roadblocks you mention.

Some thoughts:

IMO, the biggest challenge of making this server-less is NAT traversal. Discovery is also a problem if this becomes an open standard.

Setting up an open SparkleShare server that can bridge traffic between two NAT'ed clients might be able to solve the problem. It'd be a plus if there was a way to take the bridging server out of the loop once data transfer starts (I don't know if that's technically possible though). The same SparkleShare server could be a default discovery server for new users, so they get a "it just works" experience (e.g., run the setup on two machines, with the same login credentials they find each other through the discovery server).

Designed correctly, the server-less and the hosted scenario should function the same - in a hosted scenario you just happen to always have one client online.

I'll wait until JumpBox offers this and toss it up on EC2.

From the FAQ:

Q: Why is it written in Mono/C#? A: Because I hate freedom.

Why C-sharp / mono ?? This could almost be done (on Linux at least) with a copy of rsync, git (for the revision control) and a shell script.

C-sharp and mono will allow it to run on all 3 platforms (OSX, Linux, and Windows).

If you look at the source, you'll see it is already based on git. C# is just used to glue it all together and provide a GUI.

There isn't much source code there though. It looks more like a proof of concept. I doubt anything is "coming soon" here.

I was exactly wondering about mono when I viewed the README. The toolsets needed are already there.

Perhaps this was done to make it easier to port to other OS later?

This could almost be done (on Linux at least) with a copy of rsync, git (for the revision control) and a shell script.

Er, well, yeah, but you'd be polling almost constantly if you're trying to emulate what Dropbox does. I don't know exactly how Dropbox knows when I add or change a file -- I assume it ties in to the kernel. Also, rsync wouldn't work over HTTP, which is a big disadvantage in some of the environments I've been working in lately.

My big question, though: why isn't someone doing this for headless servers? Why do they always assume people just want this for desktops? At least you can hack a headless version of Dropbox on Linux.

I think it is sad that the "Open Source" Community (in quotes, because there is really no such thing and the term is inprecise) didn't come up with Dropbox on its own earlier.

There was rsync, there was unison, but none of them worked as transparently and flawlessly as Dropbox does. And now — we copy the successful entrepreneurs.

I somehow don't feel it is right — I would have loved for it to be the other way around.

Will we wait until somebody solves the other problems (E-mail, spam and E-mail attachments come to mind) and then copy their solution? Is that the best we can do?

librsync, libbzip, and a host of other LGPLd libraries are what make Dropbox go. There would be no Dropbox without the ""Open Source"" community. Dropbox--along with a lot of other companies (namely, Apple)--derive a lot of their success from going the last mile, that is wiring all these powerful OSS technologies together in a package that is intuitive and easy to use. Engineers typically are either uninterested in doing this, and that is why an open source Dropbox never came along.

My criticism of the Open Source community has been downvoted into oblivion. I think this is rather sad. I'm not against open source, in fact I regularly contribute to many projects. But I also think one should not be blind to the outside world.

I'm fairly excited about this, I hope it's "just works" appeal come's close to the DropBox experience.

I run the IT shop for a non-profit, which has just started to expand globally. I've been brought in to help ease the pain. We've got nearly everything web-based, except for the damn "shared drive". We just can't afford to set up a permanent VPN connection to all of our new offices (and all of the road-warriors)... nor do I want to.

I'm testing Alfresco now (http://www.alfresco.com/), which is basically an open-source SharePoint. It's web-based, with file versioning (and other bells and whistles) but best of all it emulates a SMB/CIFS share, so the transition is easy for the main office. Hopefully it works well and I can implement it.

But as an avid dropbox user, even this huge leap feels so old-school. It'd be great if remote users could "subscribe" to some of the folders, so they could work with files more quickly and offline. It's the future.

An open-source DropBox allows for corporate implementations ("my data stays on my servers" types), and also permits tie-ins to other open-source projects (like Alfresco). The service won't be as bullet-proof and smooth, but the mashups will be interesting.

Edit: DropBox should really offer corporate solutions. We're poor, and I think we'd spend money if I could sync our shared folders out to their servers, and let all my users access it from their beautiful web interface. That alone (without the client features) would be worth it.

Synctus (my startup: http://www.synctus.com) does something similar. It is an always-on hardware appliance for each office that Just Works - including NAT traversal. It presents an SMB/CIFS share that is the same regardless of which office you are at. Drop me a line if you think it could be useful for you.

Synctus is different from Dropbox and presumably SparkleShare because it focuses on emulating a single office SMB/CIFS share exactly, including locking so that synchronisation conflicts never happen - and this means that it needs each appliance to be usually-on (so road warriors still need to VPN in to access a particular share or use Dropbox or similar).

Neat product. This won't work for us since I'm trying to get away from CIFS (it's the transition that's hard), but I can imagine many situations where this would be very useful.

Sure beats fighting with windows replication, and has a lot of benefits over rsyncing.

Try http://www.tonido.com. It offers a remote drive feature. Will satisfy all your requirements listed here.

I saw Tonido in This Week in Startups episode 55. I immediately downloaded it. Very impressive. I am in love with it already. I rate high up there next to DB.

Hmm... interesting. Can you give user-specific logins to keep track of who's changed what?

You can create shares and assign guest accounts to them. The users will be able to load as a drive in their computers from LAN and over internet.

I would suggest you to try the workspace application that comes with Tonido. Some NGOs are using it successfully to sync tasks, calendar, notes and files. The closest application that you can compare is microsoft groove.

I'm curious what is in the Dropbox TOS that they dislike so much.

I don't know about them, but, as I mentioned at http://news.ycombinator.com/item?id=1340242, I find the following passage very bothersome:

> You agree not to do any of the following while using the Site, Content, Files or Services:

> o Post, publish or transmit any text, graphics, or material that: (i) is false or misleading; (ii) is defamatory; (iii) invades another's privacy; (iv) is obscene, pornographic, or offensive; (v) promotes bigotry, racism, hatred or harm against any individual or group; (vi) infringes another's rights, including any intellectual property rights; or (vii) violates, or encourages any conduct that would violate, any applicable law or regulation or would give rise to civil liability;

It seems to me to require you to (a) ignore it or (b) think very, very carefully about what files it is OK to backup (is it OK to back up a work of fiction? Then what does it mean that you will not back up material that is ‘false or misleading’?). I wrote them about this, but never received a reply.

I'd like to use this for corporate use - but if that means data will be stored offsite somewhere where I don't have control over it - that's a problem. If I could deploy a server at work, and use similar features but be sure about what's happening - that would be great.

That's the whole idea of this project, isn't it?

This is cool. I think simple solutions will arise as well- companies can build platforms on top of the service and sell support packages- similar to what EY, heroku, and all the other EC2/S3 platforms already do or like red hat has been doing for years. There's a lot of cool potential for this...

I was thinking about something similar, but using SVN as the server and making a Windows client first.

I would love to see something like this using App Engine and Google Storage.

Can package up and sell to enterprises.

Using "clone" loosely here.

it's cool! i want to set this up in my local server

OMFG! I've been hoping for THIS exact technology. I wanted to set up dropbox like environments on all my machines so I can share unlimited data across my LAN this way every single machine acts as a back-up for every other machine on my entire lan set up when it comes to data I very much care for (My Documents, All my photos, etc.) Its not much, 20GB total? If that much. I can even have some machines mirror that to external drives. The idea though is to have them all usable from anywhere and backed up via LAN. Yea so this sounds great.

If we don't need a "server" or do but all machines are servers just one is considered the gatekeeper, this is perfect. Just install everywhere and go.

I use CrashPlan to achieve a similar result. I highly recommend you try it.

CrashPlan will backup to your network (free), your friends (free), your external drives (free), and optionally to their servers (paid) all in parallel. Anywhere you install their software you can share a backup volume/folder and it is automatically visible to your other crashplan installs as a backup destination. On top of that you can use your own encryption key, so the data is protected even in case of a subpoena to the backup servers.

I agree - this would be a fantastic tool. And, frankly, although it might be a Dropbox clone, I'd probably use both. It's not a threat to Dropbox; it's a complement.

It is already there. Try Tonido distributed backup.

In order to compete with dropbox youll have to be as easy to use. People dont use dropbox for the storage they use it because it makes collaboration easier - even collaborating with those not so computer savvy.

hopefully the upload speed is faster than 40kb/s :-D

Reply on news.ycombinator.com