Hacker News Reader

Comment by tlamponi | Hacker News Reader

Interesting read, as with most of ben's blog.. And yeah, buffering is definitively required to get acceptable speed out of tape tech.

If you want a LTO Tape solution with more bells and whistles you could check out Proxmox Backup Server's tape support:

https://pbs.proxmox.com/docs/tape-backup.html

We also rewrote mt and mtx (for robots/changers) in rust, well the relevant parts:

https://pbs.proxmox.com/docs/command-syntax.html#pmt

https://pbs.proxmox.com/docs/command-syntax.html#pmtx

The introduction/main feature section of the docs contain more info, if you're interested: https://pbs.proxmox.com/docs/introduction.html If you have your non-Linux workload contained in VMs and maybe even already use Proxmox VE for that it's really covering safe and painless self-hosted backup needs.

Disclaimer: I work there, but our projects are 100% open source, available under the AGPLv3: https://git.proxmox.com/

View on HN

Do you run a service where I can give you data and reasonable money, and you store it on tapes for me? Low cost cloud storage prices seem very distant from this, because presumably it's usually spinning rust and not tapes that are doing the storage. I'd be into a cheaper, larger storage service where this was offered.

What is reasonable money? AWS Glacier Deep Archive is around $1/TB/month. Since it includes Multi-AZ replication for "free", you'd have to store multiple tapes in multiple facilities to get the same durability with tapes.

Retrieval costs are additional of course, and depend on how quickly you need access to the data, but if you just want to store data long term in case of disaster, $1/TB for multi-AZ replicated data seems like pretty reasonable pricing.

LTO-6 tapes hold 2.5TB of data (uncompressed), assuming you store 2 for redundancy, you'd need to find a place that will store them for $1.25/tape/month to break even, plus you're paying $25 for the tape itself, so over 3 years, that's almost another $1/month/tape. Plus the tape drive itself is around $1500.

You can use newer tape technology for better economies of scale, but your buy-in cost is higher due to the higher price of the tape drive, so you'd need a pretty high volume of data to break even.

Glacier cost in the cheapest region is 3.6$/TB/month. Plus at least 50$ to download that terabyte once it's needed(if hardware that you're backing up is not in AWS), and I don't even factor in retrieval costs. You can get HDD storage cheaper than this(twice as cheap with some providers) if you are willing to use dedicated servers. And they come with unlimited traffic. And you can use hardware there for something. Glacier is expensive AF.

S3 Glacier Deep Archive is the closest equivalent to off site tape storage.

From their pricing page:

S3 Glacier Deep Archive - For long-term data archiving that is accessed once or twice in a year and can be restored within 12 hours - us-east-2 (Ohio)

All Storage / Month $0.00099 per GB

https://aws.amazon.com/s3/pricing/

Sorry, I confused it with Archive access tier. Still, you need to spend at least 50$ to download it from AWS.

This is deep archive offsite tape storage, not something you'd need to restore often.

When I last managed offsite tape backups, I never planned on really needing to retrieve the data -- I had the data on disk and on the most recent tapes. (I did do periodic restore tests)

If I had to restore the data, I wouldn't care how much it costs (within reason).

And its awesome until you notice that just the transit fees to restore it work out to roughly $100 a TB. Which even for someone with a small auto loader (10-20 tapes), quickly buys a new tape backup system should they ever have to restore 100TB+.

So, if your 100% confident you won't ever have to restore it, its a great deal. I was considering it for my personal backups but the ~3-5k a full restore would cost me is unpleasant on top of the $30-50 a month the storage would cost me. Over a 3 year timeframe that is $1000-1800 just in storage fees. So, maybe I will just buy another USB JBOD...

I suspect that over 100T, tape is still much cheaper, especially if the offsite storage is a fire safe at the CTO's house. Once your in the few PB range its a no brainier and actual offsite storage services start to look inexpensive. https://spectralogic.com/wp-content/uploads/white-paper-iron... says that its ~$1 a month per tape to store it at iron mountain, but I'm pretty sure there is a nice volume discount, one of the previous places I worked used them for tape storage and the actual storage bill was a joke in comparison to the pickup fee.

Glacier is for storing records that by law you have to keep for a certain period of time (e.g. 6 years for expenses, tax & accounting, etc.) but that you have no intention of ever retrieving yourself. It's not a cheap backup solution.

Who is accessing their off site backups regularly? I spent 7 years of managing IT for a 500 person company and we never needed to pull in the offsite backups, those were for disaster recovery. After decades of managing my home backups, I've never had to pull in the off-site backups (which started as CD's (the DVD's) mailed to my parents, then a hard drive, and now cloud backup (not AWS)).

I always have local backups which I'd use to recover a deleted file or crashed hard drive, it would take weeks, maybe months to pull down my cloud backup unless I pay for a hard drive to be shipped to me (or in AWS's case, a Snowball, I could have 100TB restored in under a week using a Snowball)

No, we don't provide hosting services - only the software, i.e., Proxmox VE for Hypervisor (VM and Linux container), clustering, hyper-converged storage (Ceph, ZFS integrated directly and most Linux stuff somewhat too, then Proxmox Backup Server with PVE integration, can do duplicated and incremental sending of backups and save that to any Linux FS or, well, LTO Tapes, at last (at least currently, we got more up the pipeline) there's Proxmox Mail Gateway, the oldest project and a bit of a niche, but there's not much else like it available today anymore.

> and you store it on tapes for me?

I mean, we can do client-side encryption and efficient remote syncs, so such a service would be possible to pull of with PBS, but no, we don't got the bunker or dungeon to shelve all those LTO tapes at the moment :-)

The docs seem to indicate that if you have a directory structure (say over NFS share) you want to backup to tape, you must first back it up to a datastore (read: local storage on the PBS), that you cannot backup the files straight from a directory structure you must first store them locally in the datastore?

Yes, normally tape is a second level backup for the worst case and long term archival.

Tapes normally are not kept on site, to ensure that even a destruction of the office/datacenter/... like, for example, a fire does not make one lose all data.

With PBS this also allows to leverage the whole thing in a more flexible, efficient and powerful manner, as one can do tape-backups async, the deduplication layer can be used so that subsequent backups only need to add the delta of new backup chunks from the content addressable storage (note, the backups themself are still full backups, and one can configure a schedule for when a new media set needs to be created), besides that backing up file-level to tape is normally pretty slow compared to writing out big chunks that are ordered by physical disk location.

That is really impressive! Are the Proxmox tape utilities separate from Proxmox itself? I have a Synology NAS that I'd like to back up to tape. I actually have a tape library, but I haven't seen anything that looks like a simple solution for this until now.

Well, the CLI tools are not really couple to Proxmox Backup Server and could be built for most somewhat modern Linux distros, quite possibly also other *nix like systems.

The whole tape management is in the common PBS API, so that'd be a bit harder to port but not impossible. For example, I made some effort to get all compile on AARCH64 (arm) and while we do not officially support that currently there are some community members that run it just fine.

So, maybe, but could require a bit more hands-on approach. If you run into trouble you could post in the community forum (<https://forum.proxmox.com>).

Reply on news.ycombinator.com