Hacker News Reader

Comment by segf4ult | Hacker News Reader

Because tarsnap is cheap, incredibly well documented, open source, and run by an awesome guy. It's an all around win-win.

View on HN

Rsync.net is even cheaper, has no requirement for a custom client, and arguably are more dependable because they're not just reselling S3

Edit: not to mention they offer actual support not just "contact the author" email link as a last resort.

I contacted the author today. He responded to me in 30 seconds.

Try in 18 hours. Can you call him when something fails?

I'm not saying he isn't responsive I'm saying depending on a one-man-band who is responsible for the client software, server software and the underlying storage system (ie he is the owner of the s3 account) seems like a huge risk.

I assume he still has to sleep, at least on some days. :)

I personally just trust Colin's crypto skills more than anyone.

So you're saying you trust a single developer to both write an encryption tool and run the servers it talks to more than the combined possibilities using existing open source tools to create backups by encrypting data locally and storing it remotely via ssh/sftp?

Yes, when it comes to crypto I'd put my in trust in highly talented people over trusting my own ability to glue together a collection of OSS tools anyday.

You seem to have misunderstood me.

I didn't suggest you should write your own encryption tool. There are numerous open source tools for creating encrypted backups, some do deduplication first too.

If the tool doesn't happen to support remote storage, a simple rsync or scp fills that part.

Literally the only thing unique about this service is the use of the term picodollars and the single individual it's all reliant on.

It's the dropbox discussion all over again. We know how that turned out, don't we?

Would you care to elaborate?

When Drew first do a "Show HN" [0] (before it was a thing, actually), there were a lot of response about how it doesn't do anything new that couldn't be already done by a technical inclined person (see the first two top comments in the posts).

To make a comparison with tarsnap, while it's probably possible to do encrypted backup manually with a combination of shellscript and such, there are just too many moving pieces that can go wrong. Where do you store the backup? Someone mentions S3, but even managing backup on S3 with deduplication is not something trivial, and managing the encryption process is definitely not something most of us can say with confident we won't mess up. I can imagine a thousand way that I encrypt something, then unable to decrypt it back.

And then maintenance is also an issue, if I'm using a set of OSS tools, I would have to make sure the tool is being maintained, and to follow any potential disclosure on bug/ updates etc. With Tarsnap, I know I will get an email from cperciva if something comes up.

[0]: https://news.ycombinator.com/item?id=8863

As I already stated I never said you or I or most people should write our own encryption tools.

There are many open source backup tools. They offer a wide range of features such as data deduplication, references/hard links to simulate total backups without copying unchanged files, data compression, data encryption, logging, reporting, remote storage and/or remote sync.

Not all tools offer all features. Not all features work the same way, but there are many options.

Those that don't offer remote storage/sync can be setup very simply to backup locally and then sync/copy to your remote file store of choice - another server, s3, rsync.net, etc

The majority of these tools are shipped as part of Linux distribution repos, so there are almost certainly many more people using them, and multiple people with a vested interest in maintaining them.

And for reference, I agree with the comment(s) about Dropbox. The only difference is that they offer a more intuitive GUI which so far is lacking in open solutions.

I didn't mention writing encryption tools, I was simply saying that plugging all the available tools to use together is non-trivial.

Tarsnap to encrypted backup is what dropbox to file syncing (to a certain extent, obviously). I can understand why for someone knowledgeable like you, the benefit isn't obvious, just like we don't see the benefit of dropbox over other tools. But certain demographics will see tarsnap/ dropbox as value added, and is willing to pay for them (with good reason, too).

I know a lot of developers who have never spin up an EC2 instance, can't get their way around setting up a server, and certainly is not interested in maintaining an offsite server for backup. To them, tarsnap with its command line provide enough simplicity to be used (of course, it can be much better, as patio11 and alot of people pointed out).

Plugging together?

I'm talking about working tools. They either do everything when invoked, or write to a file/dir on disk that can then have rsync invoked to copy offsite.

Im talking about maybe a 4 line shell script, if that.

If someone can't handle that amount of setup, maybe they shouldn't be the person setting up mission critical backups?

Check out the key roles; you can split up writing and deleting archives, so - for example - a hacked machine can't delete the archives. This is nice.

tarsnap is not open source:

"While the Tarsnap code itself has not been released under an open source license, some of the "reusable components" have been published separately under a BSD license"

http://www.tarsnap.com/oss.html

The source code for tarsnap is available to view, so you could audit/inspect it yourself, but it is not under an open source license.

But its not cheap, which was my point. 100GB of storage costs:

$300/year at tarsnap

$36/year at S3

Finally, numbers other than picodollars and gigabyte months and unpredictable deduplication. This convinces me I don't want to store 4TB there at a huge cost($12,000 if it's really $300 a year for 100GB) compared to buying two 4TB drives (~€250 per 3-4 years) and placing them at a friend's with free bandwidth.

Don't get me wrong: managed, off-site encrypted backups are very attractive, and I might be willing to pay a premium, especially for software from a trusted person, but not the cost price hundredfold.

Add another $1k to that for data transfer (assuming you only upload that 4tb once)

Tarsnap isn't intended to be used as one-time backup like that, and it's super expensive if used that way. It's very cheap when used to backup (almost) the same 4GB for 1000 days in a row, which is what a lot of people/businesses need for their backup solutions.

It's not one time, I'd be incrementally writing updates to the disks. With a raspberry pi or something, the power costs are near negligible.

Rough estimate here:

If you upload 4tb in a year, that's 333.33gb/month

So for tarsnap that equals

- $1k/year in data transfer charges (4000gb * $0.25 transfer charge * 12 months)

- $83/month per month of data (333gb * $0.25 storage cost/month)

- $6.4k/year for the first year ($83 * 78 cumulative months in a year)

So $7.4k for 12 months resulting in 4tb

If usage stays the same each year will add $12k to the incremental yearly cost

> that's 333.33gb/month

I have 4TB of data, which changes an unknown amount (probably around 20-50GB per month) and grows slightly (probably 5-15GB per month).

In any case, thanks for the calculation. Tarsnap is apparently not for the common person who wants to back up everything including their media.

That actually works out worse - $14K for the first year, $13K for the second year.

That sort of backup is what AWS Glacier is for, is it not?

I guess, I haven't really looked at it yet. And I'd have to find my own software to encrypt it before uploading. Tarsnap's software is one of the major selling points, at least to me.

I hate to think the cost if you had to restore that data from glacier though.

How much do you save after dedublication? Tar-snap could be a lot cheaper if you do frequent backups or you often change little in huge files.

Backup tools like attic (which I use) include automatic deduplication. There are surely minor differences in implementation, but tarsnap isnt the only implementation of deduplicating backup.

Reply on news.ycombinator.com