Tarsnap – cleaning up old backups | Hacker News Reader

Tarsnap – cleaning up old backups

dan.langille.org

74 pointstosh27 comments

Can someone shed some light on why one might use tarsnap over another service? The tarsnap pricing model seems wild ($0.25/gb/month) considering you can do the exact same thing for 10% the price with something like Borg for deduping/encryption [0] + Rclone for syncing to the cloud [1] + S3 (which is what tarsnap uses on the backend). And this is only one of many different toolsets that can do the same dedupe/encryption/sync. And you can do it for even cheaper using something like Backblaze B2 instead of S3. Despite this, I see tarsnap recommended a lot amongst the tech crowd, but don't understand why. What am I missing?

0: https://borgbackup.readthedocs.io/en/stable/

1: https://rclone.org/

tarsnap is simple. The rest of the stuff you mention requires me to set things up and write scripts to wrap it all and sign up for AWS and monitor it and ....

Also your method might be much cheaper, but tarsnap is still inexpensive for most use cases. So how much actual money is saved vs. the effort level put in?

If there is one thing that TarSnap isn't, is inexpensive.

I previously used Tarsnap, but I moved to using Borg on rsync.net. Borg can do all the stuff Tarsnap can do and comes with 'borg prune' included. In fact for Tarsnap I would have needed to write a custom script to perform regular pruning.

By relying on rsync.net's automated ZFS snapshots I get the same or similar protection compared to Tarsnaps's write-only keys.

For my personal use case cost wasn't an issue, though. It's less than 10 GB of data stored.

After many years, I moved from tarsnap to Borg as well (hosted with borgbase.com).

It wasn't about price for me either. On the few occasions I actually had to restore some files, Tarsnap was glacially slow. That made me nervous if I ever hit a situation where I had to do a full restore.

Like you say, there's the quality-of-life features that I now don't have to script. Pruning, missed backup alerts, monthly reports are all things I've realized were sorely missing from my Tarsnap experience.

BorgBase does not do scheduled pruning? Not even via script?

I believe it inherits the same pruning behavior of Borg itself.

I don't know the architecture of Borg storage too closely, but I assume that since it is all end-to-end encrypted, Borgbase cannot do the pruning for you.

If you use append-only keys, they can't (destructively) prune, you have to use regular keys to do that.

Setting up borg + rclone + B2 took me about an hour to set up and zero time since then. Something like Duplicati or Duplicacy makes it even simpler.

I'll agree that tarsnap is still 'simpler' since I guess maybe it takes 15 mins to set up versus an hour, but I'm still surprised that someone would consider the 45 minutes saved to be worth possibly hundreds/thousands of extra dollars in backup costs.

>Also your method might be much cheaper, but tarsnap is still very inexpensive for most use cases. So how much actual money is saved vs. the effort level put in?

What are the use cases you use tarsnap for? Is tarsnap only used for small amounts of data or something? I guess I just cannot get over the hurdle that tarsnap seems obscenely priced given the cost of storage these days. You can get 2TB of storage for ~$120/yr on something like Dropbox/B2/Google Drive/etc. That same 2TB would cost six thousand dollars a year on tarsnap!!

I switched from tarsnap to Wasabi plus restic. It’s much cheaper ($.0059/gig) and I have no complaints so far.

Yeah I don't put anywhere close to that kind of storage in there, so I get it. I just have critical small files, mostly documents and code.

Hard to agree with you.

Setting up restic/duplicacy/kopia/etc is easy and you get to choose your storage from wide array of choices that's pretty much 1/10 the price. Does tarsnap even have retention rules?

Just because someone behind may be famous, charging 10 times for something that's not even better is absurd and should be seen as an obsolete service when there are a dozen others with competitive pricing.

I see you're happy to pay $100 for something I pay $10 because making a cron job is tough for you.

> I see you're happy to pay $100 for something I pay $10 because making a cron job is tough for you.

This is against HN rules FYI. No reason for this kind of trolling/goading. My use case doesn't even come close to matching that cost anyway.

"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."

https://news.ycombinator.com/newsguidelines.html

Maybe not a good enough reason, but the author of tarsnap is very active on HN (cperciva). He's also very smart... ("Dr. Colin Percival studied mathematics at Simon Fraser University, entering at age 13, before going on to a doctorate in Computing at Oxford University, where his thesis covered problems in string matching, data compression, and file synchronization.")

Pricing definitely doesn't look smart when it's the cost from 15 years ago maybe at 10 times the competitor.

Tarsnap's easy for both me and accounting, works, and has very good support. I don't feel like setting up a solution that is not understandable to the person who will look at this while I take vacation.

I've learned that the ultimate test of documentation, training, scripts, and good services is "can I go on an undisturbed vacation" when the less experienced people have to run the ship.

I think it's basically mostly because the Tarsnap tooling has a very principled design, is reasonably easy to use for power users with basic use cases, and perhaps most of all because the operator of the service (Colin Percival) is a tremendously well respected figure in the cryptography world.

I describe the tarsnap tool as being "principled" because it draws a pretty tight boundary of what functionality is in-scope. As a result, I think most real "backup" scenarios will require some kind of tooling to be used as a wrapper (e.g. GFS or even simpler retention scheduling).

I was a Tarsnap user for some time, but in the end I've found that using Duplicati with B2 is actually easier than Tarsnapper and significantly less expensive, as my usage totals a bit over 7TB - not affordable for me with Tarsnap. As a result I haven't been recommending Tarsnap for some time.

I do think it's important to emphasize that "B2" or "S3" does not necessarily mean doing scripting or whatever yourself - probably the biggest advantage of choosing one of these commodity storage services is that they are supported by a wide variety of open-source and commercial tools, generally with much a much more sophisticated featureset than the tarnsap tool. Duplicati as mentioned is a more GUI-centric (web interface) tool with block dedup and retention management for .NET/Mono. "rclone" is a command-line tool for Linux that's fairly similar in usage to the tarsnap tool but also has additional features. Both can be set to encrypt backups before uploading. Both support either S3 or B2 (and I assume also Google Cloud Storage, Azure Storage, etc).

tarsnap is dead simple to use if you have classic Unix sysadmin knowledge, it takes care of the encryption/deduplication work behind the scenes, I trust it to work properly and be secure because the creator is someone who has integrity (you can email him any time and get no-nonsense and refreshingly honest replies) and the price only becomes an issue if you’re storing massive amounts of data, at which point you have probably outgrown it.

I use a great tool called tarsnapper to cull old backups. It lets you configure schedules like "keep daily backups for the last 30 days, then keep a backup every 3 months for the last year, then keep yearly backups".

prunef is also good at this

tarsnap --list-archives | prunef | xargs tarsnap -d

https://git.sr.ht/~apreiml/prunef/

I strongly recommend "--no-run-if-empty" on xargs pipelines like these.

It depends on which OS you're on.

GNU xargs: Normally runs the command at least once, but you can turn that off with "-r" or "--no-run-if-empty".

OpenBSD xargs: Like GNU, but only has the short "-r" version of the flag.

FreeBSD xargs: Never runs the command if the input is empty, has "-r" (but not "--no-run-if-empty") as a no-op compatibility flag for GNU.

macOS xargs: Never runs the command if the input is empty (has no "-r" or "--no-run-if-empty" flags).

I see no reason not to also provide the compatible long option on FreeBSD: https://svnweb.freebsd.org/changeset/base/365954

I have two custom python scripts that manage my tarsnap process, one that performs my daily backups (dumps various dbs the performs backups) the other scrubs backups older than 45 days based on the name of the backup skipping over. All automated backups are named with an ISO timestamp. Has worked rather well so far, and when needed I can disable the cron that deletes temporarily if needed.

Using tarsnap for my dotfiles, which are ~17mb in total and costs ~$0.0001 daily (lol). Will worth every penny, once you reach it!

Made by HN user cperciva

Funny I just found out through yesterday's thread about top HN comments: https://news.ycombinator.com/item?id=24531972

There's tsar for doing grandfather-father-son backup rotation:

https://support.powertools-tech.com/safonso/tsar

Reply on news.ycombinator.com