Hacker News Reader

Comment by DeepDuh | Hacker News Reader

sigh Dropbox should not be used as a backup system. A system that synchronizes live should never get that role, except if they can guarantee that old data is never overwritten, new is always appended. This is not the case with dropbox - I've experienced multiple scary occurrences of old versions going to the nirvana with certain user actions. In certain cases the old data just appears to be gone, in other cases the web interface shows them but a restore results in an error message. Especially data move appears to be buggy. Dropbox appears to be simple, but the backend processes are really not and there is too much going on for it to be a reliable backup system, especially if you also share some of your data to team mates.

edit: emphasis formatting, niceifying.

View on HN

No sigh was necessary, I understand. I even mentioned that I have a NAS and off-site backup... so either you didn't read it all or you stopped the moment you encountered the word "Dropbox" and started typing.

That said, people DO use DropBox as backup.

If you take a walk around the British Library and asked every PhD student working there how they "Backup" their research and work in progress, I bet every single person who believes that they have a backup will say "Dropbox", and the only exceptions will be a few who don't really have a backup.

I know that because I ensured my girlfriend does have a real backup solution in place that is tested. Not one of her peers seems to.

DropBox is used for backup because they've made file sync so damn easy that most people can be convinced that if a file exists in many places, it is backed-up.

My whole point is that now storage for long term backup is priced in a way that is affordable to most, that consumer services may emerge that offer true backup to consumers and can successfully migrate people from lesser solutions (DropBox, stacks of CD-ROMs, etc).

One of the things about backup is that it needs to be easy. Currently the size and cost of backups make it expensive, and the only way to reduce the cost makes it difficult (HDD local copies stored at a friends' house for example).

By reducing the cost, perhaps we can finally increase the ease... and then a day may come in which most people have a real backup solution.

"I even mentioned that I have a NAS and off-site backup... so either you didn't read it all or you stopped the moment you encountered the word "Dropbox" and started typing."

No offense, but this post doesn't quite match what you wrote originally. My sigh was in response to the phrase "Dropbox should work here, ...". You didn't state security as your concern as to why not use Dropbox, rather it was cost. This might lead someone who only needs <2GB backed up to believe that Dropbox is perfectly fine for that task.

"DropBox is used for backup because they've made file sync so damn easy that most people can be convinced that if a file exists in many places, it is backed-up."

And that's exactly what I'm scared about and why I'm saying it again and again that you shouldn't do it - if only one person listens and avoids potential data loss because of it I've already reached my goal.

"One of the things about backup is that it needs to be easy. Currently the size and cost of backups make it expensive, and the only way to reduce the cost makes it difficult (HDD local copies stored at a friends' house for example). By reducing the cost, perhaps we can finally increase the ease... and then a day may come in which most people have a real backup solution."

Agreed. IMO no consumer backup system is quite there yet. TimeMachine is very close, if only it would do better logging and have some more intelligence about warning messages.

"No offense, but this post doesn't quite match what you wrote originally. My sigh was in response to the phrase "Dropbox should work here, ...". You didn't state security as your concern as to why not use Dropbox, rather it was cost. This might lead someone who only needs <2GB backed up to believe that Dropbox is perfectly fine for that task."

A fair point.

In my case, Dropbox use is in addition to local RAID (scratch) NAS (network scratch, access of larger files) + off-site (backup).

I only use Dropbox for syncing and sharing.

The sync vs backup is an interesting one, simply because most consumers couldn't tell you the difference.

For example: Q: "Are your contacts backed up?". A: "Yes, they're sync'd to Google".

I did conflate my scenario with thinking about my girlfriend's peers in my post. And then reacted from my perspective again... my bad.

"Agreed. IMO no consumer backup system is quite there yet. TimeMachine is very close, if only it would do better logging and have some more intelligence about warning messages"

Vigorous agreement here too, except for the TimeMachine bit as that is Mac only and doesn't work for <insert any other system or device that isn't Apple Mac OSX).

"Vigorous agreement here too, except for the TimeMachine bit as that is Mac only and doesn't work for <insert any other system or device that isn't Apple Mac OSX)."

Well yes, obviously it only works in a mac-only household/office. I still think it's a good solution for non technically experienced users who only have macs since it's so simple that you could literally explain your grandma how to setup. I don't think mobile devices are that important on the other hand. The important data on them should usually be synched to your computer and as long as that is backuped, you should be fine. On windows I think that the built-in backup since Windows 7 is finally decent, albeit not yet grandma-proof ;).

To be fair though, Dropbox is way better than any other backup solution consumers usually use. Personally, I've never seen data loss occur on Dropbox, but I'm sure it can happen - it's just way less likely than the average user messing up their own backup.

The ideal solution is the Quadfecta (is that a word?) - Dropbox for (in my experience) excellent versioning/synchronizing (Never failed me) + Backblaze (or its ilk) for continuous Offline backups + Super Duper (weekly/whenever) - for Image Backups - + Something (Arq?) on top of Glacier for long time off-site-archival.

For $50 in software (Arq+SuperDuper), $100 for an external HD, and less than $25/month ($4-backblaze, $10 Dropbox, $10 Glacier) you have a backup system that is next to air tight for a Terabyte of Data and a working set (on dropbox) of 100 Gigabytes.

I use a combination of Dropbox and S3.

I have my MacBook and a Linux server linked to my Dropbox account. So changes in my documents are synced to my Linux.

My Linux run three cron-jobs. One daily, one weekly and another monthly. The command is.

s3cmd sync --delete-removed ~/Dropbox/documents/ s3://backup-daily/

There are buckets for weekly and monthly too.

Note: the command is not exactly like that, check the man page.

That way I have backed up all my documents very cheap.

One thing to be a little careful of - you are sending all of your data to S3, which is, of course, the backing store for Dropbox.

Dropbox stores their stuff in S3 a little different. It's not a 1:1 correspondence between user files and objects under Dropbox's S3 account. The fact that they use S3 as their backing store means very little. It certainly sounds good to have S3 in back when you talk about scalability and durability, but 1) they could just as easily use something else, and 2) depending on their sharding strategy, a single lost object could impact multiple files at the user level.

Right - the point I was trying to make, is he was putting all his eggs in one basked. If anything catastrophic happened to S3, he might lose both his S3 as well as his Dropbox backups.

If you are going to the effort of having dual-backup systems, may as well try and find something that can't be impacted by a single disaster.

Ah, right - definitely.

Wait, do I understand this correctly: You sync your dropbox to S3? Do you overwrite your previous daily when you do so?

Yes I do it, I have the weekly and monthly.

And yes, I might have daily-monday / daily/tuesday.... and so on, and it would work better, but this way works for me. There is a lot of room for improvement, and it is not hard to implement with cron-jobs and more buckets.

You could try adding objects with the date in their names to a 'backup' bucket rather than create separate buckets for each backup. S3 also supports object expiration, so you can set your daily backups to expire after 7 days and weekly backups to expire after 4 weeks, for example.

Do you know about worst case analysis?

Let's say your data loss occurs on the Sunday, 31th of a month at 23:55 and then gets synched across all your S3 backups (or it could occur at some point before that but you don't notice it). And poof goes your data.

THIS. I'd say at least half of the time I went to retrieve a good revision of a file from backup, it was already too late and the backup was trash as well.

He could just turn on object versioning.

Though you make a good point, your comment could have avoided sounding pretentious if you shortened it by removing the first few words.

Also, please don't use uppercase for emphasis, as mentioned in the guidelines [1]. If you want to emphasize a word or phrase, put asterisks around it and it will get italicized.

[1]: http://ycombinator.com/newsguidelines.html

Alright, I've edited it, thanks for the heads up. Dropbox and backup in one phrase tends to awaken my temper as I've had some horrible almost data losses (as in I've had to restore data from the dropbox cache folder that could basically be purged at any point).

The dropbox client itself keeps the old versions of files around for a few days. It doesn't matter if the backend gets confused.

Yes, I'm aware of that. This helps, up to the point where you don't notice that something's gone for 'a few days'. Let's say you have your student project's folder shared with two other colleagues. You take a few days off, use your PC for casual browsing, meanwhile your colleagues are working on the project. At the end, one of them (who doesn't quite understand how dropbox works) deletes the files while the other is still working on it and dropbox gets confused. Your PC gets synched at a moment where you don't notice. Come back from your holidays and you will have a nice surprise waiting.

The colleague that was actively using the files doesn't notice them disappear?

Can you explain how dropbox has lost history for you in the past?

It doesn't matter whether he notices it or not, you can't count on social factors when it comes to backup systems. He might notice it but then he thinks, oh no problem, it's all in Dropbox anyway, I'll talk to Dylan16807 when he gets back.

I've never had any complete data losses until now, but I've had this situation where the Dropbox cache was the only place to retrieve files twice and it showed me that it can't be trusted this way. I'm certainly not gonna wait until the real deal happens just to have a personal story on how Dropbox can go horribly wrong. I've been shown the possibility and that's enough for me.

As for specifics, the case I mentioned is the one I saw where it can go wrong. IMO the only really save way to use Dropbox without additional backups is if you never share folders and only ever use one system with write-access at the same time - which is not the usual use-case of that product.

I just want to understand better why you had to resort to the cache. The only time I've ever done that was when I accidentally deleted a bunch of data and didn't want to restore each file by hand or put in a support ticket.

I can't quite put my foot down on what the scenario is exactly. One time might have been a case where a folder with lots of small files got moved away and Dropbox only allows single file restore, which would have taken hours, possibly longer than the lost work. Once I've done something stupid where I wanted to exchange a whole Dropbox folder with a different version. I switched off the client on one PC (by mistake, obviously) and exchanged the folder on the other, then switched it on again. Then I became aware the the old folder had important stuff in it. The versioning was corrupted at that point, many old files would either not show up online or would produce an error message.

So usually there is some user behaviour involved, yes. But the whole point of a backup system is that you can rely on it, even when the user behaves stupidly up to a certain degree. Sync is for day to day collaboration and data management, it's not for backup.

Dropbox Pro with the Packrat addon claims to store all version indefinitely.

That said, we still keep multiple backups.

If dropbox works well for you for source control, that is great. But frankly, if you get to the point where you have to start writing hacks to keep it working, it is probably time to move onto something designed for the task.

>A system that synchronizes live should never get that role, except if they can guarantee that old data is never overwritten, new is always appended. //

Why don't Dropbox enable this use-case. It would surely be very easy to implement. I guess that pack-rat does do this in a way but seems like overkill.

I'd like something along the lines of duplicating all files but requiring confirmation of deletions and overwrites.

Reply on news.ycombinator.com