Hacker News Reader

Comment by robotmay | Hacker News Reader

I debated doing this recently but ultimately spent a lot of money on an LTO-6 tape drive instead, just due to the cost efficiency over time for larger amounts of data. I justified it to myself as I needed to either expand my NAS or clear stuff off it, and the drive was the same price as buying two new HDDs to expand the NAS thanks to the obscene price inflation at the moment. As I’d likely need to further expand my NAS next year (I’m shooting a lot of photo and video) I think this works out a much more cost effective way of storing this data long term when I likely don’t need to access it much.

View on HN

After reading reading the "Environmental and shipping specifications for LTO tape cartridges" [1] I came to the conclusion that tapes are not that great.

    Environmental   Allowable       Recommended 
    Factor          Storage         Storage
    Temperature     16 to 32°C      16 to 25°C
                    (61 to 90°F)    (61 to 77°F)

In the summer it's not unusual to reach temperatures over 30°C in the house.

There's also this story [2]:

> I had a room full of servers, many with 16-36 disks per server and had an AC failure. Things turned off as planned, but it hit 115°F [46°C] for a bit. None of the disks died, majority of the tapes did.

[1]: https://www.ibm.com/docs/en/storage-deep-archive/1.3.1?topic...

[2]: https://news.ycombinator.com/item?id=21066981

Anecdotal, but...

I doubt that the recommended storage conditions for qic tapes were much different from the above; and I've seen a system boot from a qic-120 tape that had been kept in a non-climate controlled ministorage locker in Santa Fe NM for 17 years.

Holy cow, thank you for this. On that same IBM[1] link, it reads:

Maximum wet bulb temperature (Only LTO-8 and earlier cartridges) / Maximum Dew Point temperature (LTO-9 and later cartridges), all across the board, allowable / recommended / shipping: 26°C (79°F).

In Florida, we like to refer to 79°F dew points as "Tuesday."

More to the point, the time I want those LTO cartridges to earn their keep is during a disaster scenario. The most likely disaster scenario--other than fire--is hurricane. High temperature and humidity, and no power to run the HVAC to mitigate those. (Unless you have generators with fuel to run for days, and your HVAC, generators, and fuel tanks survived.)

Oh very interesting! This isn't (currently) too much of a concern for me as I live in Wales, and though it's ridiculously humid here there's not much risk of it hitting those temperatures inside my house the vast majority of the year. I shoot a lot of photographic film as well so I'm used to storing these sorts of media, though I probably won't be shoving the tapes in the fridge beside the film. Currently they sit in my windowless office.

Here in the north of Scotland where it never really gets below 5°C or above 25°C making it t-shirt weather all year round, I think the tapes will be okay in a dry reasonably-insulated storage room.

They’re engineered like this because 99.9% of tape storage media is sold to corporations who store their backups in climate controlled buildings with backup generators.

As a person instead of a company, can't you just buy a stack of hard drives instead of a stack of tapes?

Even with the inflated prices nowadays 3tb 3.5" sata drives are cheaper than 2.5tb LTO-6 tapes.

Personally I don't trust optical. I had some DVD-RW drives and many of them became unreadable after a very short time.

(I don't know what the absolute vs practicate reliability factors are for any of these)

EDIT: Hmmm... https://www.howtogeek.com/the-thousand-year-disc-that-failed...

FWIW both CD- and DVD-RW are less reliable than the read-only counterparts. M-Disc is supposedly very good.

I have used tape (of various flavours, QIC, DAT, LTO) and DVD+R and DVD-R for decades. Bluray has been the most reliable of all of them for me overall.

Tape is amazing, and resilient, but the drives are fragile and expensive.

The sad part is that Panasonic/Sony/Pioneer have pulled out of making drives (I believe Panasonic still make some for their own vertically integrated products), which just leaves Hitachi-LG.

I see several used LTO-6 tape sales on Ebay, for about $13 per tape. The cheapest used SAS drive I see is about $30. I'd trust a used tape over a used hard drive every time. Tapes have a 30 year life span, I can't imagine a hard drive would come close.

I'm not sure I'd trust a used tape. They're generally only rated for a couple hundred or so writes. They'll totally sit on a shelf for 30 years, but writes tear them up.

If I write and verify the write, then that's good enough for me. I don't only buy used tapes, I have about 100 that are new/unused that my important data go to, and really important stuff goes on at least 2 tapes.

Ignoring the cost of a tape drive, tapes are cheaper per TB, yes. I wouldn’t write off used hard drives though - buying only the cheapest per TB used enterprise drives, I’ve only had a ~5% failure rate across roughly 200 drives with 2-5 years of runtime (while I’ve owned them - they usually have 4-6 years when I get them). I usually get rid of them once they hit 8-10 years of uptime (or if any get any uncorrectable sectors).

If you only need ~50TB it’s probably both cheaper and easier to just buy hard drives, even today, factoring in the cost of the tape drive. Maybe even 100TB. I’m personally backing up ~225TB off-site these days and I still find AWS S3 glacier deep archive to be cheap enough - at current prices I’ve decided the hassle of doing tape again isn’t worth my time unless I’m over 300TB.

Blu-rays are way more durable than CDs and DVDs for some reason

The Blu-ray spec demands a better coating.

It's a design feature [0], since people got annoyed with CDs and DVDs scratching so easily.

[0]: https://en.wikipedia.org/wiki/Blu-ray#Hard-coating_technolog...

the problem I've have with CD-Rs and DVD-Rs has nothing to do with scratching but due to degredation of the organic dye. I think this is less of an issue with BD-Rs, but I think BD-Rs can still cheap out and use the organic dyes.

The problem with optical disks is that nowadays they have a too small capacity in comparison with magnetic tapes or magnetic disks, so they are useful only for archiving small amounts of data, of a few TB at most.

If you want to archive up to 100 TB or even 200 TB, the cheapest way to do that is with HDDs, but you must keep in mind that you must use at least duplicate HDDs for everything, preferably stored in different locations, to avoid losing data due to files that become corrupted after a long term storage or due to some accident. Moreover, you must migrate the data to new HDDs periodically, e.g. every 5 years.

If you want to archive several hundred TB of data or more, magnetic tapes become cheaper and they are more reliable than HDDs. You still need to migrate the data, e.g. every 10 years, but much less because of the risk of corruption or mechanical failures, but because the tape format may become obsolete and appropriate tape drives that can read it might no longer be available.

The biggest advantage of LTO is that it's designed to be offline. Hard drives start to get weird when they've been sitting still for too long.

> The biggest advantage of LTO is that it's designed to be offline.

It's been a while since I did enterprise backup, but when I did, most backup software had a 'bootstrap procedure' that you could use even if you were completely destroyed.

First, most enterprise software does not need a license for restorations, only for backups. In a disaster recovery scenario you're unlikely to have the keys.

At end of every backup run/day, an e-mail is sent out with the tapes and offsets that have the indexes and configuration backups. You can send this message to an e-mail address that is separate from corporate e-mail (in case that gets wiped as well); back in the day you could send it to /dev/lp0 (and stick a copy in the box that the offsite tapes are transported in).

So even if your entire infrastructure is wipe (e.g., see NotPetya and Maersk), there's still a starting point for restoring your restore-backup server, and rebuilding up your infrastructure.

Remember: your backups have to be inaccessible to attackers.

The new type type of laser heat assisted drives might change that. Laser and heat to write data. Kind of similar to old mini discs

I've been thinking about different techniques for making my NAS backup-able to cold storage and... the thing that has given me pause with tape is how closed-down the ecosystem for the LTO drives themselves seems to be. What's your strategy for knowing you'll be able to access your tapes to transfer them forward to the next generation in 10-15 years when the tapes start reaching the end of their dependable life?

Perhaps for some use-cases, the value of the data degrades at a rate proportional to its persistence layer, and doing a re-migration is not useful.

Looks like I'm a bit out of date, but older generations of LTO drives were able to read tapes from up to two generations ago, and write tapes from one generation back. Looks like this isn't the case since LTO-8. It looks for 8 and 9 theycan read and write to tapes back one generation. 10 breaks the pattern and can only read LTO-10 tapes.

https://www.lto.org/lto-generation-compatibility/

If you have money for LTO-10 drives and tapes, then backwards compatibility shouldn't really be a problem. The drive alone is $20,000 to $30,000. If you can afford an LTO-10 setup, then an array of lower LTO drives seems affordable.

> is how closed-down the ecosystem for the LTO drives themselves seems to be.

Linear Tape Open is remarkably Open for an enterprise product with Open in the name.

Personally, I feel like archival storage of personal digital data is not really solvable. I feel pretty confident about keeping my online storage going while I'm able. After that, there's nothing effective to preserve it when I'm gone. That's not a huge deal for me, I don't expect my heirs to want or need these things... Photobooks with annotations are more likely to last and have some chance of being apprechiated.

EBay has a bunch of LTO-3 tape drives for sale for <$300. LTO-3 is 22 years old and can read LTO-1, originally released 26 years ago. I expect them to still be for sale 20 years from now, albeit fewer in number. I see no reason why LTO-7/8/9 drives won’t be readily available for the foreseeable future.

I guess the practical strategy is to treat migration as part of the backup plan rather than something you deal with in 15 years

I am also considering an LTO solution of some kind, but I also found out that LTO drives seemingly need some TLC and maintenance if you don’t want your tapes to jam up. I feel like at least with optical media, it’s nearly impossible to screw it up.

I really wish a multi-terabyte optical media solution took off, because that would be the path I’d ultimately take, instead of going with expensive LTO devices and media.

I bought a used LTO5 tape drive from ebay, and I've been getting used LTO5 tapes. I've never had a problem. The tape drive lives out in my garage inside my backup server. I have a 10Gbit link so I can write 140MB/s over ethernet from the server inside the house. It works perfectly, has for years, nothing "jams up" (so far). I've had to use a tape drive cleaner cartridge once since I got this thing years ago for routine maintenance. I verify writes for anything important.

Optical just didn't make sense to me. Overall it was way more expensive, and a lot slower to burn BluRay discs.

The max BluRay disc size is 128GB (which is smaller than a lot of SD cards these days), and the fastest you can write to a BluRay disc is 18MB/s. That's a lot of of swapping discs to burn my many Linux distros.

And those 128GB BluRay discs cost between $30 and $40 each. Used tapes (with a lot of life left in them) on ebay are far less expensive per TB - the lowest I've paid is about $1.40/TB for LTO5 tapes without a case. I typically pay about $4/TB for unused LTO5 tapes with a case on ebay. It's getting a bit more difficult to find those deals, but they come up every now and then.

In bulk the 128gb blu ray discs aren't nearly as expensive -- I picked up a 25 pack for about $230 18 months ago

The bigger problem is finding a drive that can read and write them -- the 128gb format support is a bit harder to find. I thought I would be good with my nice Pioneer BDRW drive but it didnt have the support so I had to buy an LG unit. So now my desktop has two BDRWs -- how very 2006!

I recently got 90 LTO5 tapes for $200 total. That's 135TB. A 25 pack of Blu ray will get you to 3.2TB.

The LTO5 drive cost me about $150 on ebay, and an HBA card was $30. Yay for datacenter hand-me-downs.

At least where I am, Bluray disks are currently cheaper (in bulk) per megabyte than hard drives are.

Tape reigns supreme in cost per megabyte, but the infrastructure costs are astronomical.

SD card is an atrocious archival/backup format as data retention is rated in low single digit years - though I am continuously amazed I can store 100GB or more on something so tiny. I would love a true archival SD card format.

>Tape reigns supreme in cost per megabyte, but the infrastructure costs are astronomical.

A used LTO5 tape drive cost me $150, an HBA card cost me $30. That's literally the only "infrastructure" I needed to acquire. The tape drive has been running fine for several years. It lives in a tower PC, so I'm not sure what astronomical cost you are talking about. Sure, a new LTO-10 drive costs $20,000+ but I don't need the latest and greatest. I just need cheap, fast storage.

I've had very mixed success procuring used tape drives in the past, with a very high rate of failures. Looking at LTO-5 drives available to buy secondhand in Australia, they range from $300-$1000, predominantly shipped from the US or China. Maybe it is just because I've been burned before with used tape drives, but that is probably more than I am willing to spend.

Unfortunately I don't have either a tower PC or a server in which to install it, being predominantly an iMac/Laptop user - so would need to factor in an external enclosure which are certainly not cheap.

It feels to me the broader issue is that consumer archival media is dead, with the (dying) exception of optical disk. I would love consumer tape drives to become a thing again - lower capacity, smaller, and USB-C connected.

I can see this making a lot more sense for photo/video. The Holocron approach is neat when your truly irreplaceable data fits on a handful of discs, but I would not want to burn and manage dozens of them every year

Yeah I'm tending to shoot around 1TB of data per wedding I cover now that video is in fashion for it. Thankfully it's exactly the sorta thing tape is suitable for - once I'm done I can just archive a couple of weddings off per tape. It's generally very unlikely I get requests for that data later (once they've ordered prints) so doesn't make much sense keeping it on spinning disks in my NAS.

> ultimately spent a lot of money on an LTO-6 tape drive instead

I'm semiseriously considering getting the old LTO3 library from work and putting it to use. Sure, it's "only" 400GB per tape but I have a literal vanload - the racking takes up the same volume as a mid-size Ford Transit - of new and used LTO3 tapes to dispose of too.

A petabyte would only be 800kg or so!

Reply on news.ycombinator.com