Managing tape drives and libraries with the Unix/Linux CLI | Hacker News Reader

Managing tape drives and libraries with the Unix/Linux CLI

blogs.intellique.com

68 pointswazoox43 comments

It's always weird to remember that what we use tar for every day (packing and unpacking archives on disk) is by design a strange corner case and not its actual purpose.

I'm surprised still that so many people sleep on tape. It is a very mature and stable technology with the best dollars/gigabyte ratio of any storage technology out there. Start up costs are a bit unpleasant but purchasing secondhand is reasonable and economical, and unlike the cloud the costs are mostly one time instead of forever. The tapes are durable and last for decades, so that initial outlay will last you a very long time.

The tapes might last for decades, but support for them won't. Newer LTO standard drives can only read tapes a couple generations back at most. So you're pretty much forced to copy your stuff over if you want to stay current.

That's just it, there is no need whatsoever to "stay current" unless you are unhappy with the capacity of the tapes. For multiple generations now, that has been the only difference. The last major feature was slightly better compression followed by LTFS, and that came out on LTO 5 and 6… In 2010 and 2012.

If your drive breaks down while you’re trying to do a restore, you’re going to want to get a spare one quickly. That may become a problem if you choose not to stay current.

In which case I'm going to eBay where I probably got the original drive in the first place over 10 years ago, and a quick look finds thousands of listings for both standalone and internal drives, at good prices, from reputable sellers with good history, going all the way back to LTO2 (which is almost 2 decades old!).

This is what I mean, LTO sits in this niche where it is/was popular in the enterprise and so the secondhand market is big. I would be willing to wager that I will be able to find LTO 6 drives in another 10 years, with no problem.

There's the very interesting new feature of LTO7M and 8M, which adds 50% capacity to your LTO7/8 tapes when formatting them with a - respectively - LTO8/9 drive. A 30$, 9TB LTO7 tape is the cheapest storage you can buy.

It's a one-way conversion, and then you're stuck with that one LTO generation; a LTO7M tape won't work anymore in a LTO7 drive.

It's a moot point.

If you are buying the whole system new then you buy M-capable drives.

If you are stuck with G7 drives - then you don't format your tapes to M, even if you have 1 G7M drive.

And if you value your data, ie you want to able to recover it someday then you need N+1 drives of needed generation and format anyway.

It's not just the expense. Backwards compatibility is only guaranteed up to the last two generations of tape media. If you hold onto these things for decades, you might find the drives required to read them no longer exist.

> If you hold onto these things for decades

It's always amusing when people bash tapes for the lack of backwards compatibility, but...

Hey, try to find a working system with PATA drives support? Try to connect SATA150 drive to SATA-III port? How about SCSI-320?

People like to talk, but I always said what if your data has a value then it's your responsibility to have means to backup and to restore it. If your data "is extremely valuable" but all your C-level are approving is a single drive decade old library - you can write your backups straight to /dev/null, because your data is worthless for your C-level.

I'm not "bashing" tapes, I'm pointing out the fact they have limited backwards compatibility guarantees. Given the massive expense of tape drives and the fact you need to keep replacing them and the tapes just to keep up with new LTO standards puts this almost perfect solution out of reach of mere mortals like me. I'm not an enterprise, there is no "C-level" approving anything.

> I'm not "bashing" tapes

Oh, this wasn't pointed at you at all, sorry for the misunderstanding.

> Given the massive expense of tape drives and the fact you need to keep replacing them and the tapes just to keep up with new LTO standards

Well, you don't need to keep replacing them. The rolling upgrade (gradually replacing the tapes and drives for a new generation) makes sense only if you have tens to hundreds of drives and thousands and thousands of tapes - ie for those insane enterprise libraries. For everything else, be it a small enterprise or a "mere mortals" it would be always cheaper to just buy a whole new library/autoloader.

And if the cost of initial buyout is too much then of course tapes aren't for you... along with the compatibility woes.

SCSI 320 HBAs are pretty easy to find in the usual places nowadays. And they work on modern systems.

Until they don't. And modern systems doesn't have PCI-X slots. Sure, there are still Adaptec cards out there... which doesn't exists as a company since 2010/2016. And I have a first hand experience with HP P800 not working in PCI-E 2.0 slots. Are sure your old SCSI320 card would work in something like PCI-E 4/5 or even newer?

I didn’t know about sg_logs: that could be useful. One issue I have encountered and one that isn’t really addressed is that there are different paths (sg, st and nst) for a tape drive, but it isn’t always easy to know which of those device paths is associated with which drive in the changer. It is also possible that the /dev/sg* path number will be different from the st and or nst number. IBM has a lin_tape daemon that reassigns /dev/ns* to /dev/IBMtape* so that’s extra fun if you need to access this information in a repeatable fashion. Beyond the tools mentioned in the article udevadm -a can provide helpful information about a drive.

One thing I forgot to mention is the M format. Starting with LTO7 media used with LTO8 drives, if you label the tapes with labels ending with M7 instead of L7 the drive will format the tape when first writing it in the 8M format, adding 3TB of capacity to it.

Of course these media can only be read using drives of the next generation.

Notice that first formatting a M9 tape can take up to 45 minutes. That can be a surprise when you first try it! Take this into account if you're time-constrained.

Confusingly, LTO7 tapes with "M7"-ending barcodes are called "8M", and LTO8 with "M8"-ending barcodes are called "9M". Don't ask...

LTO does seem to be the last bastion of closed source vendor lock-in. That being said, even if prices dropped by 75%, I'm not sure our shop would look to do LTO instead of a series of hard drives. 20 TB hard drives are now $400.

If you fall in the right place on the curve $75 / LTO-8 cartridge still makes a lot of sense. It doesn’t take many $400 hard drives to make up the price for a tape element at that per-cartridge price. (Assuming you can deal with sequential access, etc.)

Hdds are soo fragile. They constantly break. A proper 3-2-1 backup solution is incomplete without tape IMHO.

At least HDDs are fast, cheap, don't have massive up-front investments. You can take a couple of HDDs, RAID them together for reliability and you can still write and read data at humane speeds.

A RAID array isn't a backup. Your data is still live on it, an "rm -rf" away from destruction. HDDs fare poorly when shelved for a long time, RAID are even worse in that regard. OTOH a tape on a shelf is perfectly safe for 5 to 10 years or more (I recently restored 1998/99 sensitive data from LTO2 tapes written in 2003).

>A RAID array isn't a backup.

Who said it was?

>Your data is still live on it, an "rm -rf" away from destruction.

How is accidental deletion not a problem with tapes?

Unless you store your tapes right next to your cache of neodymium magnets, that should be pretty easy. Writing to a tape is very deliberate generally. Much more than writing to disks.

With current software and hardware RAID solutions, we also have massive up-front cost. We can't buy drives as needed and add them to the system one at a time.

I wish it was easier to incrementally expand capacity of RAID setups. ZFS promises RAID-Z expansion but it still hasn't been released. Btrfs has a flexible block allocator that supports incremental expansion as well as heterogeneous drive pools but unfortunately the parity support cannot be relied upon to this day.

RAID is not reliable.

Source: Former HPE Storage dev.

Nothing prevents you from using open source software with LTO, which is a multi vendor format. You can read tapes written with an IBM drive with a Quantum drive...

LTO 8 tapes are 80$ or so, formatted as LTO8M in a LTO9 drive they hold 15TB uncompressed. LTO7 are like 30$ and hold 9TB uncompressed in LTO7M format.

Also this post exists in French here https://blogs.intellique.com/tech/2021/08/20#BandesCLI

I was able to purchase a LTO-7 library full of tapes a few years ago for basically nothing, and still haven't gotten around to using it. There seems to be very little in terms of free, automated software for managing the system and performing tasks such as backups. tar doesn't exactly cut it for hands-off usage.

The best I've found is using Veeam Backup & Replication, but it's Windows only and not great still.

> free, automated software for managing the system and performing tasks such as backups

Did you come across Amanda[1] or bacula + bacularis[2]?

1: http://www.amanda.org/

2: https://bacularis.app/doc/index.html

I hadn't seen Amanda before. Bacularis looks like it might help with the complexity of setting up Backula. I might try that out.

It's hard to get the motivation to mess with these tools that have relatively little documentation/examples and are purely config file based.

That's fair. I used to use Ark, which was mostly a proprietary front-end for bacula, and that exposed me to the kind of futility of a "user-friendly" tape interface. Every tape operation from picking to winding to r/w takes so much time, and has so many subtle ways that it can fail, that it defies attempts at friendly user interfaces.

Bacula is great software, but yes, fiddly to set up and SCSI tape drives / changers aren't really straightforward either. It will serve you well once all is set up though.

I worked for a company using Bacula quite successfully for many years with a 24 tape system. It was much better than the proprietary software the system came with.

Wow, amanda. That brings back memories of using that newly released software (~2000) for swapping tapes daily on a drive hooked up to a sparc 5.

For backups I'm using BareOS, which is a fork of Bacula. For archival... We wrote our own software, which is mostly free (AGPL3). I should give our github some love though :) https://github.com/Intellique

you may check out SEP Sesam, its cross platform and has an community license.

Can anyone comment on the brands of tape you can purchase? Given it is an open format what sets the brands apart? Are there records of failure rates like backblaze has done with HDDs?

Currently for LTO8 tapes you can buy the following brands:

* Fujifilm

* Quantum

* IBM

* HPE

* Oracle

* Dell EMC

Actually only Sony and Fuji make tape. All the brands use one or the other. However they use two different technologies (metal oxide and metal evaporate) so the general recommendation is to use both for long-term archive. I generally use HPE or Overland (Sony-based) and Fujifilm.

Hi guys, reposting because it may interest you. In particular using LTFS on the command line get little love on the internets...

I feel is missing a clear way to identify self rewind tapes,otherwise you may be reading the first file multiple times.

I currently use removable hard drives for backups, but was considering the idea of adding a refurbished LTO tape drive to my XigmaNAS based NAS, however I couldn't find any information about feasibility, compatibility etc. Does anyone have some pointers, assuming it is doable?

A tape drive actually led to my adoption of Linux.

Around 1999, I decided that a QIC tape was the best backup solution for my home network. At the time, I had a 486 running Windows, and two Apollo 425t running OpenBSD. The 486 may have also dual-booted into OpenBSD.

Anyway, I had a bare-bones 386 sitting around and I had upgraded the 486 so much that the spare parts were available to reconstitute the 386. I found a nice floppy tape drive, consumer grade, and wouldn't you know that OpenBSD didn't have floppy tape drivers. So I installed Linux on the 386 and it was a dedicated backup server.

At the same time I had a SyQuest EZ 135 that I was very fond of, and those 135MB cartridges could hold good stuff, like a complete complement of Windows updates. But its capacity was too low to be a backup solution, of course; it was storage on-the-go.

Reply on news.ycombinator.com