Hacker News Reader

Comment by StavrosK | Hacker News Reader

If someone open-source, encrypted backups at that price, I would buy it in an instant. What I want is, basically, rdiff-backup on EncFS, but I haven't found a way to hack the two to work together.

View on HN

* Arc http://www.haystacksoftware.com/arq/

* JungleDisk https://www.jungledisk.com/

Arc supports encrypted backup to Amazon S3 and/or Glacier, works with occasionally connected external hard disks perfectly. I use this to backup RAW photos from Aperture stored on an external USB hard disk (at least 50GB worth).

Very nice, thank you! It looks like neither supports headless Linux, though...

Can you describe your workflow for aperture?

It's nothing too special. I use a separate Aperture library per year, each stored on an external hard disk. It's a Western Digital "My Book Edition II" with 2 x disks in RAID1 configuration for redundancy. This is configured to backup to Amazon Glacier using Arq, which is clever enough to only backup when the drive is connected and only uploads the new or changed files. Works quite effectively and once configured I don't really need to think about it.

I have a friend that backs up his photos to my computer using a combination of ecryptfs, rdiff-backup, and rsync. It goes something like this:

    mount -t ecryptfs ~/.photos-backup-crypted ~/photos-backup
    rdiff-backup ~/photos ~/photos-backup
    rsync -a --del ~/.photos-backup-crypted remote-server:photos-backup

That's pretty much what I want, but it requires you to have as much free space as the thing you want to back up.

What I wanted to do was mount EncFS in reverse mode (so it gets plain files and creates an encrypted virtual volume) and rdiff-backup that to a remote host, which works very well. However, the problem is that, when your directories are disparate, you can't easily make them appear as one directory, so, to back up 5 locations you'd have to do this 5 times.

I can fake it using AUFS or similar, but it's too large a dependency. Someone wrote an EncFS patch to let it follow symlinks in reverse mode (https://code.google.com/p/encfs/issues/detail?id=147), which would solve the problem, but I don't think EncFS is maintained any more.

Have you tried Duplicity (http://duplicity.nongnu.org/)? I use it for backups and think it's great. You can even backup directly to Amazon S3.

I have. Unfortunately, due to the way it encrypts files (it basically turns everything into one big, encrypted tarball) it needs to redo the whole thing from time to time, as diffs stack and it gets unwieldy.

Combined with a slow home connection, it basically never managed to upload all my files, and it could never resume. That's why I want to go the EncFS route, I don't care if file sizes and numbers are visible if it makes uploading resumable, sane and easier.

It's not just about metadata being visible. Duplicity is designed to work when the remote machine doesn't have the private key to decrypt your files. If you use EncFS instead, anybody who compromises the server with your backups can also get the key to decrypt them.

EDIT: Never mind, I just saw your other comment and learned about reverse mode. That's a simple but clever feature.

Nah, they can't get your private key if you transfer just the encrypted files. That's the whole point, really, but obviously they can get it if you mount it on the server itself (and thus disclose your passphrase).

SpiderOak provides encrypted backup for any types of files for around $0.09/GiB. When i signed up for a free account, i got a 10% off email a few days later.

I currently use SpiderOak, but I'm nothing if not paranoid, and they aren't open-source. Other than that, it's pretty good for backups, and quite cheap, too.

Yeah, you wonder why they don't open-source their front-end client code so that it can be easily audited to make sure the encryption is done right. It would go a long way to increasing some users' peace-of-mind. Then again, since it's all javascript, you could probably look at the html source, but it's probably minified and obfuscated.

It's not JS, it's Python, AFAIK. I talked to their CTO (I think it was the CTO) about it, who said that it was a big priority with them, but they couldn't open-source it yet. It would do a lot for my peace of mind, though, and I wouldn't look for alternate solutions if everything was okay with it.

Oh, so it's not a web-based client. Sorry, I confused it with Mega. To be honest, why not roll your own alternate solution? This one here is just a frontend to S3 and glacier. It shouldn't be too difficult to write a script that takes all the files in a directory, encrypts it with GPG or OpenSSL, and then uploads it to glacier.

Actually wait, yeah, that's pretty feasible. It seems that glacier pricing is free for low storage volumes. https://aws.amazon.com/glacier/pricing/

The problem is all the diffing and stuff. You can get half a terabyte of storage for $3/mo, so glacier support isn't necessary, but getting accurate diffing is hard (there's a reason why rdiff-backup is as large as it is, and duplicating the entire thing doesn't make much sense).

I would much prefer to write a small layer to glue EncFS and rdiff-backup together (and I started doing just that: https://github.com/skorokithakis/encrups), but I haven't managed to find an elegant solution for the directory aggregation yet.

Tarsnap too expensive?

Around 8 times more expensive than this, yeah.

Note that the $0.04 price is likely "cold storage" via glacier, which is cheap for write-only storage ($0.01/GB/month) but it's much more expensive than S3 for retrieval ($50/million requests to S3's $1) and there's ~4h round-trip to get data back according to Colin's old analysis of Glacier for Tarsnap: http://www.daemonology.net/blog/2012-09-04-why-tarsnap-doesn.... I'm guessing the data in cold-storage will not be interactible-with or visible in the interface.

Note that the second-to-last paragraph of Colin notes the possibility of tarnsap "glacializing" a complete machine dropping storage costs to 3~5c/GB/month, but it not be readable or deletable save by "deglazializing" the machine (15~20c/GB) to move the data to S3 for retrieval or deletion.

Yeah, the data I have is pretty much just old projects, photos, stuff like that that I archive for sentimental purposes, and that I never really need, but don't really want to delete either.

I'd be fine with something that's basically just "keep this data for me just in case all my other backup locations go down".

Reply on news.ycombinator.com