Ask HN: What's your favorite online-backup tool? | Hacker News Reader

Ask HN: What's your favorite online-backup tool?

53 pointssabj87 comments

Hey HN -

It's almost 2011, and my backup solution still calls for semi-regular huddles between my laptop and a few external drives, which I randomly scatter between home and school for redundancy. This is pretty low-tech, all things considered, and I'd like to enter the future. Right now at best I have a hodgepodge of things that I will try to upload to Google Docs or suchlike. Not so pretty!

Where to start? I could set up from cron scripts and buy Amazon S3 space, but that really doesn't seem to be the most elegant solution (perhaps for some software projects, but not for a personal basis).

Options I'm familiar with are, most prominently, Mozy (mozy.com) and Carbonite (carbonite.com). I guess using something like Dropbox could fulfill this need to some extent as well.

Would love to hear feedback about these services for automated backups and any other alternatives I should consider.

Thanks!

View on HN

Tarsnap. Simple, cheap, efficient, honestly secure.

I can't imagine storing personal files and photos somewhere "out there", managed by someone else, in readable form.

Seriously, how do you Dropbox (etc) fans sleep at night?

Tarsnap is great

Tarsnap - for home and work. Fantastic service.

I love tarsnap. I have some custom scripts to manage backups easily. I've been thinking of cleaning them up and turning them into a rsnapshot style system.

Another + for tarsnap, although I only use it for server backups (git repos and databases). I should probably start using it to backup my laptop.

My dropbox partition is completely enveloped in a TrueCrypt partition.

I don't particularly have many concerns with the security of Dropbox, but I do keep some personally sensitive information on there, and TrueCrypt is how I sleep at night.

Doesn't this make your syncs terribly slow? It will basically have to send up your entire partition every time you make a change in any file there.

I think Dropbox uploads diffs instead of the entire file based on the speed of synchronization of my yojimbo database.

AFAIR : Truecrypt is designed so that changed plain blocks map to changed secure blocks : i.e. the whole secure image doesn't change if one plain file does. (Makes sense from a speed perspective too).

Isn't this insecure? IIRC, the best way to backup data on a secure partition is to create another secure partition (with a different key/password) and sync files between them.

love tarsnap, but seriously... someone please make a web-gui (you can paste your ssh keys in a webpage a la github) or atleast a java gui.

It does get to be too painful sometimes.

Wouldn't you have to upload at least a read-only subkey of your tarsnap key not your ssh key. Github works because sshd only requires a copy of the public key to be stored to log in with the private key.

This would destroy the secure model of tarsnap. If you truly want this then don't use tarsnap use dropbox.

I was not trying to propose a "better" way - my understanding of tarsnap's security is deeply limited. Yet, I love how it is exposed as a scriptable command.

In my (very) limited knowledge - I am able to do retrieve backups on my machine using my key, which I supply to the command line. I was thinking of how it would work in a web based environment. Perhaps I could copy paste my key into a textbox, and I could retrieve my backups from a webpage. This would only last for the session (tarsnap would not store the key).

I know that you can think of a million ways that this can be exploited, but that is MY cross to bear. If I am prepared to do this, because I need immediate access to my backups from another machine - then please let me do this (basically, give me a way to access and create my backups in less than two steps).

I personally do believe that this would be a killer feature.

Dropbox's website suggests that they store files using some sort of AES-256 encryption and they cannot read them: https://www.dropbox.com/help/27 I wouldn't make the claim that they are more (or as) secure as Tarsnap (especially since there is no page as detailed as http://www.tarsnap.com/crypto.html on Dropbox's website), but this suggests that files on Dropbox are not readable by the company.

"Suggests." However, the fact that there is a web GUI tells us they can read the data.

...when they are provided with your password.

If you forget your password, can Dropbox reset it for you? If they reset your password for you, can you still read your data? If the answer to both of these is "yes", then dropbox can read your data without you providing your password.

(Honest questions -- I don't use Dropbox, so don't know.)

This comment is cryptographically correct.

+1, https://www.dropbox.com/forgot is a password reset link.

Besides, there is much more to cryptography and building a secured system than just using that crypto suit or another (e.g. AES256-HMAC-SHA256 etc.)

I love this kind of services that just work, no need for flashy menus. Plus the pricing model is convenient.

Anything I was uploading to the interwebs for serious storage would already be lovingly encrypted in a TrueCrypt container... usually.

Wow. I go AFK for a few hours (orchestra concert, fundraiser for something) and when I come back online I see this thread. I click on it, all ready to type out a comment saying "I kinda like this tool I wrote for myself, called Tarsnap"... and I find that the top comment, with 3x as many points as any others, is recommending Tarsnap.

I love my users.

Seriously, consider writing a client for Windows and your business should increase 10 folds.

Tarsnap will support windows eventually. But seriously, if Tarsnap usage increases tenfold overnight, the service will collapse under the load the following morning. Making sure that the service keeps running smoothly takes priority over attracting new users.

I like you, and I like your service. I especially like that your service is clearly designed to be a successful operating business rather than a 'sell it to google' startup. I wish you much success.

It's pure selfishness, really. I use Tarsnap, and lots of my friends do too. If I sold Tarsnap and ran, I wouldn't have good backups any more, and I would have a lot of ex-friends upset with me for not taking better care of their backups. :-)

Tarsnap is great. Some things that would have been nice: * Automatic archive naming. I would love to just type tarsnap-backup stuff/ or something, and it would pick a name based on the folder name and a timestamp. * It seems a little too easy to delete archives. Perhaps a way to lock an archive so it cannot be deleted or modified without unlocking first. * If you backup a symlink to a folder, tarsnap will only backup the symlink itself. Perhaps this is the best way to do it, but it means you may think you have backed up something, but actually haven't. A warning message perhaps?

tarsnap is great. The problem is you are relying on a single brilliant person (cperciva). If he gets hit by a bus, etc you're screwed.

It depends on the platform and what type of backup you are looking for.

It used to be that a backup was a full or image backup. It is a bit funny how I used to say I have a backup and it meant a backup of my entire computer. If I didn't, I'd say I had a backup of the database, or a backup of home. Now if someone says I have a backup I assume it is just of some of their files.

There are really now three types of backups.

Image or full backups - These are far more useful. You can actually fix your computer without reinstalling and reconfiguring everything.

Partial backups - Files and Folders, unfortunately letting the software try to select the "Important Files" is fatally flawed on windows. They generally select based on file name and location. If you use any software slightly off mainstream you will want to make sure those data files are backed up.

Sync - Sync software maintains a copy online which is good for most files and is essentially an extra copy or backup. But some files or directory structures are troublesome like build directories, databases (especially email databases like outlook pst files). Make sure they support multiple versions...

Begin shameless plug...

Hybir Backup recently won "Software Product of the year" from the DaVinci Institute.

I wrote Hybir Backup https://www.hybir.com It is windows only for now. Hybir Backup performs full online backups and full local backups (simultaneously). Bare metal restores can be done from the windows PE 3.0 recovery environment.

The interesting part to the technology is the data identification and global data de-duplication technology. Essentially only data unique to your PC needs to be uploaded. First backups on relatively clean machines are pretty quick. If you have a bunch of unique user data, you will have to upload it just like the other services. The advantage is all of those other files are backed up too.

During a full restore only files that are truly different need to be restored to fix the computer greatly speeding the restore process when a computer won't boot, but the disk is still functional hardware wise.

Another cool feature is that if you backup online only, and need to do a bare metal restore, you don't have to download the full image. You can simply backup another PC to a USB drive. You probably need another PC to burn the recovery CD you didn't burn before the problem anyway. Any data that is common and identical on the USB drive will be copied from the USB drive. The unique bits will be downloaded at the same time. You have a fairly good chance of not having to download the OS, MS Office, etc.

The software is free to use for local image backup and supports network drives. It works great for a small office environment and includes the data identification technology. This makes it really efficient storage wise. It is probably the only free local backup solution that has global data de-duplication.

I've been pretty happy with Crashplan ( http://crashplan.com/ )so far. The free version will allow you to backup to one or more remote machines which you can seed via a hard drive if you want to avoid uploading hundreds of Gb for the initial backup. They have options to backup to their data center too for a competitive rate (unlimited storage).

Transfers are encrypted, de-duplicated and compressed, supports file versioning and it all works very well.

The only downside I've had is that their Java client uses a lot of memory both on my OS X and Linux boxes (haven't tried Windows) - It's using nearly 600Mb of resident memory right now.

I'd like to second and stress that CrashPlan supports client side encryption using a private key. This means that _nobody_ can ever get to your files without the key (not even you), no matter how much access is granted to the CrashPlan servers. From what I remember from my research, they were one of the few online backup providers that offers this option.

I also love the fact that you get email alerts about the backup status for every destination (in one email) local or to remote.

I've been extremely happy with the service. It keeps multiple versions of files and backs them up continuously if you have CrashPlan+. This has already saved my bacon more than once.

The icing on the cake is that you can designate some storage as a backup well of sorts and have your family (or other computers) back up to this destination. This is very easy to setup and has worked flawlessly for me, no networking or "what is my ip" voodoo.

I like this, but... how is it free?

I didn't say it was free... Neither did the OP ask for a free solution. So...

Ah, I understand: my grammar was unclear. I just saw the first page, where it said "free," and I was confused about what the economics of the plan were. I don't care if it is free, and in fact that free factor was a concern, since I was wondering how it stayed in business. Clicking through and seeing the pricing, I understand now. The question was more "how does it stay in business," not, "in what way is it free." : )

I didn't realize that you were the original poster, sorry.

I admit their product offering is a bit confusing. It does make sense though:

You can install CrashPlan (the regular, free version) on any number of computers and set them up to back to each other/external drives/ftp/etc. This works wonders for family computers, for example. At this point its all free.

If thats not enough for you you can buy the "Plus" version of the software that offers continuous backup, stronger encryption, etc. This is where it stops being free; you're down something like $60 for the software.

Regardless of which version of the software you use, however, you can pay for CrashPlan Central, which is effectively another backup destination that happens to be on their servers, this is the "online" portion of their offering. The cost is more or less on part with the Mozys out there.

So technically if you are OK with backing up just between your existing machines and you don't need up to the minute backups, their free offering will work just fine for you and it works well.

crashplan looks great. but is anyone else skeptical about the unconditional "unlimited" offerings?

I have a Mac Pro with 5+TB of storage. Lots of HD home videos and photos.

I've used Crashplan for over a year now. Works great on Windows and OS X.

I use JungleDisk (https://www.jungledisk.com/) - it's good because it can be used for backups, syncing, and previous versions of synced folders. It also does multiple users and multiple operating systems (Mac/Windows - probably does Linux too but I haven't checked).

It's also quite cheap.

I previously used Mozy, which was excellent, but only does backup. Mozy was much more efficient over the wire, better at notifications and better about resuming very large backup sets. But unfortunately doesn't do synced folders and the rest.

Jep, jungledisk is really good.

Do you use the old just-pay-the-S3 client or the new $2/month service?

Workgroup edition - which is $4 per month per account. It includes the first 10GB of storage on rackspace's cloud for each account, which seems to pool.

I'm surprised Duplicity hasn't had a mention. It supports full and incremental, zipped encrypted backups (using GPG) to S3/SCP/FTP/local/etc. Under the hood it uses librsync for fast binary diffs.

Duplicity + S3 is somewhat similar to tarsnap. It's not quite as friendly - you have to put a bit more effort in - it doesn't provide easy key generation, and it doesn't support snapshots, but you have no reliance on any third party service other than S3.

If the Duplicity software disappeared off the planet you could still recover your files - it's just split gzipped encrypted files.

Oh, and you can install it on OSX using homebrew: `brew install duplicity`

http://duplicity.nongnu.org/ (unfortunately the site seems to be down for me right now - here's the cache: http://webcache.googleusercontent.com/search?q=cache:2s6jQju... )

+1 for duplicity. With the command line tool, you can also:

1- Easily go back in history (if you erased a file and want it again, it's here, in a previous incremental backup).

2- Ask to automatically make a full backup every X months.

3- Ask to automatically remove incremental and full backups older than X months

4- Select the granularity of the backup buckets (very useful to avoid having hundreds of files or if your connection is unreliable).

5- Upload a backup bucket while preparing the next one in parallel.

6- Use S3 reduced redundancy storage if you want (cheaper...)

Duplicity is written in Python and the API is easy to use if you want to rely on a Python script instead of a shell script.

The fact that you can use S3 was a deciding factor: it is a really cheap solution with a big ecosystem of tools and it should still be there in 10 years...

Assuming you want Windows, try the fast & free application called Duplicati:

http://code.google.com/p/duplicati/

Their 1.0 release is out and works excellently with S3. It can't be set up as a Windows service yet, but that functionality is on its way.

For Linux, my favorite is another free utility called Amanda Network Backup:

http://amanda.org/

Supports S3 and MySQL backups. Really easy to set up and has commercial support if you need it.

I like it! I used to use some various commercial products to this end, but found them rather unpleasant. I like the aesthetic here and might put this to use even for some side projects. Thanks!

My personal favorite (Mac only) is Arq: http://www.haystacksoftware.com/arq

It uses Amazon S3 for storage, has versioned backups, very lightweight and fast, and leaves all Mac metadata/permissions intact. I've had 0 crashes or problems in a year of constant use.

They also supply an opensource tool you can use to restore your backups, in the event that you don't have access to your licensed copy of Arq (like if your laptop gets stolen)

I've been happy with Backblaze. Good cpu usage, and I restored files easily with a nice web interface.

I switched from Mozy because the client was much more efficient. If you're looking at Mozy and Carbonite, definitely check out Backblaze.

(Also, I met the CEO a few years back--seemed like a smart guy).

I love Backblaze too. They also have a great blog and have posted about their custom hardware a while back http://blog.backblaze.com/

I won a Spideroak ( http://www.spideroak.com ) plan and have been using it for about a month now. It works, uploads are fast, I can't comment on how easy a restore would be. The only thing that's really annoying is that a search for updated files uses 100% CPU.

Other than that I can warmly recommend JungleDisk ( https://www.jungledisk.com/ ), been using it do backup my source code and documents for about a year now without any major problems. I don't even notice it runs anymore. I'm using it with the S3 backend.

Do a trial run of a restore. I've seen too many backups fail/be incomplete to trust that backup works without validation; test that you can get back what you need to.

The most comically tragic "failure to effectively test restore" story I'm aware of is from one of our customers, who previously had their own home grown backup system with encrypted archive shipping offsite, monthly test restores, and so on.

It was only after a total failure that they realized the gpg keys to decrypt the backup were only contained _in_ the backup. Their test restores before were happily using keys from the user's home directory.

This is one of the many reasons it makes sense to have a company that specializes in backups help you with backups. :)

(I cofounded SpiderOak)

I use ADrive. ( https://www.adrive.com/login ) They give you 50GB free. No SSL support for free-users, so you might want to encrypt sensitive files. (Use the link above to log in with SSL for free.)

But to be really honest with you, I'm considering leaving ADrive for my own external HD only. That's the safest, not always easiest, solution.

Don't be too quick to overlook Dropbox. I use it for almost everything.

I use Dropbox in combination with Crashplan. Dropbox for "live" files, and Crashplan for "archive" files.

If I wanted to store everything, 100 GB would be too small.

I love AltDrive.com. But I'm partial since I created it...and took 5 years doing so. I think tarsnap is good but it seems more for hackers and IT types. Reading their blogs, they do seem to well understand security and encryption.

AltDrive has a free two month trial for either home or business users. You can control your own encryption key too. It is highly secure and uses AES-256 EAX mode encryption. It works in Windows, OSX, Linux. Plus other OSs for business users. It is easy to use and is full featured. There is also a white label offering for the business service.

Check it out. http://altdrive.com I'm always looking for feedback of any kind that would help improve it.

rsync.net

Not cheap, ($.80gb/month) but bulletproof, and about as hacker-friendly as it gets. As you may be entirely unsurprised to hear, you can use rsync with it, as well as sftp, webdav, etc, etc, etc.

I had heard about this before, and I think it is really great - for small amounts. For backing up my laptop hard drive (~250 GB) and everything I normally would keep backed up (another... 1 TB?) or even some fraction thereof, this would get SUPER expensive. :(

What I really need backed up: - Documents (~5GB) - Photos (~20GB)

What I would _like_ backed up includes everything to do a full drive recovery, but beyond that, music [40 GB] and other harder-to-replace things.

If I can't backup all my backups, that's fine, but I think under even the bare-bones situation, rsync would be outside my budget :)

I use rsync to back up compressed text, .rc files, etc. The critically important stuff, that in meatspace would go in a safe. Photos, music, and movies (the kind of stuff that would be on bookshelves) get backed up to terabyte hard drives in USB external boxes.

The difference here is that the important stuff is also on those drives. "Importance" here means how many times it's replicated, and how far apart the backups are.

One of my favourite things about rsync.net is the support: you seem to get access to an engineer (or at least someone that knows what they are talking about) immediately.

Dropbox and external HDs for now. Trying out AeroFS as a possible migration path; I have outgrown the reasonable price point of cloud storage.

Dropbox for me. The seamless multi-machine syncing just adds another dimension thay makes it a must have for me.

Tried Carbonite and CrashPlan, but have settled on Mozy to cloud backup my 2TB RAID network drive. $5/mo or $55/yr, unlimited storage for one 'machine'. Good client tool for the Mac with bandwidth throttling and intuitive file picker. And fast indexing.

Has no-one on here hit problems with backblaze's number of file limitations and xml data structures?

see: http://harijan.wordpress.com/2009/04/27/the-real-backblaze-b...

what the author does not make clear is the statement: "the bzfileids.dat is causing my computer to stop."

this is how i found the article originally, the bzfileids.dat on my computer was too large and at that point the only solution is to reinstall the software. which invalidates your old backups and they are purged after 30 days.

Backup-manager - http://www.backup-manager.org.

I posted a blog post about using it on server - http://www.yepcorp.com/blogs/pavel-karoukin/simple-backup-so...

I am using it to backup both my dedicated server and laptop.

Basically, this is bash script with config file and some dependencies to Perl libraries to upload files to S3. Since it's bash script - it's very customizable.

We used to recommend Backblaze to our clients (and set a few up with it), but when they announced a while back that they had entertained a purchase offer, I pulled the plug on that. I can't in good conscience recommend an online backup service that's not in it for the long haul.

We now try to get people set up with Carbonite, mostly because of its pricing, ease-of-use, and compatibility with most platforms.

I'd love to find something with a good white-label reseller program, but no luck so far.

You may like SpiderOak's white label program: https://spideroak.com/partners/

(I cofounded SpiderOak in 2006.)

IIRC, SpiderOak was a close contender but it didn't meet one of our requirements.

We were looking for an offer where we would handle support and invoicing of our clients, with compatibility for Windows and Mac, while being able to keep costs under $10/month/client. We're happy to tell our clients about the service we're using.

Pretty much everything seemed to break on one of those points; either the service wanted to do the invoicing, or it was prohibitively expensive.

No one has mentioned my current favorite, iDrive. I didn't want to spend more than $5-10/month on online backups, and found that Tarsnap, Dropbox, JungleDisk, etc all got expensive too quickly to back up 100GB+.

I wanted to like Mozy but the mac client crashed on me and the uploads were slow.

iDrive gives you 150gb for $5/month. Plus the mac client is reliable, the uploads are quick, and they let multiple computers share space if you want.

I used Mozy for a while. It was OK, but the desktop/server distinction doesn't always make sense. Never did have to try it in a substantial restore, though.

Nowadays, I keep most of my stuff backed up locally on external hard-drives, but also started using dropbox recently for smallish set of files (sharing, mostly, but serves as a backup of sorts as well).

For quick file uploads (to share or when I'm hopping around campus), I'm surely going to plug my new project, Crate [1] (though I have it better than most, with access to the sexy Crate for Mac app).

[1] http://letscrate.com/register?inv=hn

I sought around for one for awhile to little avail; Dropbox works for me. I have relatively modest needs.

One interesting open-source solution is Areca: http://www.areca-backup.org/

Multi-platform, encrypts, compresses, incremental/differential backups to FTP, network drives, local drives, usb sticks and it has a GUI.

What's wrong with s3? It's cheap and simple. If you use linux, try http://live.gnome.org/DejaDup -- it provides a dead-easy gui to automate backups (to s3, or to other kinds of storage).

It seemed like it would be relatively more complex to set up / run, and more expensive to store large bunches of files. Good middle-range option. I like to dual-boot sometimes, which might mean hairier fixes, but more than that I don't want to have to do any tech support for my backup solution if anything goes wrong. Experiments, software, I'm fine dealing with the consequences, but with my data, I don't want too many DIY fixes since the results can be really unpleasant!

Dropbox and Amazon S3. Carbonite and Evernote in lesser use.

Backblaze, a ReadyNAS as a Time Machine server and JungleDisk. I know I could drop Backblaze or Jungledisk, but I never got around to deciding between them.

I'd place another vote for SpiderOak. The fact that you can pool storage between computers on a consumer plan is great, and their DIY API looks promising.

I've been pretty happy with SpiderOak but I haven't needed to use it to recover files yet -- I feel that the interface may be a little too cumbersome than it should be.

http://macsoftwarespotlight.com/ is offering 35% off SpiderOak pricing (less than an hour left). their DIY API seems simple enough and hacker-friendly, although i hope they standardize their API so it's easier to migrate from, say S3.

Ubuntu One is free for a certain amount (relatively large) and larger plans are really reasonable. But, not sure if you're running any Linux boxes.

Consider - http://www.phpmybackuppro.net/ nice tool and its free

I'm satisfied with BackBlaze (http://backblaze.com)- $5/mo

Sugar Sync

I've never compared; I just use the first one someone invited me to use. 5GB free.

Desktop: CrashPlan Server: tarsnap

dropbox

no one mentioned sugarsync?

can some people chime in with a multimachine option that also allowsaccess by smartphone?

Reply on news.ycombinator.com