50TB IBM tape drive more than doubles LTO-9 capacity
blocksandfiles.com
blocksandfiles.com
IBM's had several generations of tape tech in the can, and has managed the rollouts with calm competence for at least back to '97 when I got started in the backup business.
Their research is a prominent part of the LTO platform, too. My observation of that landscape was that they gave toys to LTO at a rate calculated to extinguish the utility of tape research in other places. This means there's a plausible competition, which IBM wins all the time.
This may sound like I'm down on them for it; I'm not. Their hardware was rock solid, performed as advertised.
[0] https://en.wikipedia.org/wiki/Linear_Tape-Open#Generations
I would guess so.
Similarly, HAMR and MAMR will be milked for years in the world of magnetic hard drives.
Compression really is a game changer when you're over a certain amount of data you need to protect, and backup technology does some really clever space savings at basically all stages, making it hard to find a good and easy metric to help sell to management. It's a lot easier to tell management:
"These tapes hold 50 TB of production data, and we have about 45 TB in production. Backups and restores will be faster, we'll have fewer tapes to manage for backups"
than it is to explain the different types of data you need to move to tape, the expected compression ratio, etc.
That being said, I do agree they market it too much and it gets confusing, as it should only be a reference for implementation and quick calculations.
I suppose compression ratios vary a lot between users of these things, right ? And if you are backing up user-generated data, you don't know in advance how well it will compress.
So looks like 50TB raw if I am reading this right.
For these sorts of solutions you are usually not using just 1 tape. The tape is the cheap part. The drives cost a decent amount though. So in your case if the data does not fit. You buy another tape and pop it into the existing drive and finish the backup. Also you would have some sort of rotation schedule and at least a few dozen tapes. If you are doing it right offsite and onsite rotation too. An extra tape in the mix of that is not something they would worry about. When I did this for a small org I had the weekends to do a full backup of all of our computers. Then during the week I would do incrementals. At the end of the week I would ship off the prev week to an offsite and start the next full backup. Not perfect but gave us a onsite 1 day return offsite (building gone) 1 week out. The offsite would keep about 6 months of tapes and return them about once a month.
Except when it isn’t, but those customers know who they are.
Can't you just do that estimate in your head? If you know your data mostly text and has a 3:1 compression ratio, when you buy your tapes you can multiply the raw capacity by 3, if it's mostly compressed images (i.e. gif, jpg, etc), you're probably not going see much compression at all.
Does going with some tape drive manufacturer's estimate really help you?
Sure, if "production" means database. Which is sometimes sensible and sometimes extremely far off.
> than it is to explain the different types of data you need to move to tape, the expected compression ratio, etc.
To some extent, but you mostly just need two numbers. Database capacity and photo/video/audio capacity.
It seems they went all in on the "compressed" value after missing the doubling of the real space in LTO6.
Standard written text typically runs about 50%.
Compressed audio, image, and video data of course compress very little, if any.
tank compressratio 1.14x -
tank/backup compressratio 1.43x -
tank/movies compressratio 1.01x -
tank/music compressratio 1.01x -
As you can see, my actual file backups (servers and clients) compressed somewhat, but movies and music I got nothing (this is from ZFS but tape compression will be similar).If all your competitors report 30 TB*, and you report 15 TB, you may have a better product, but what the people with purchasing power want is bigger numbers, and you advertise a lower number
*with 3:1 compression
But in fact this did happen to some extent. LTO tapes used to advertise with 2:1 compression ratio. Then that became 2.5:1. And now IBM is doing 3:1
And that's not because we just discovered how to improve on Lempel-Ziv. The switch to 2.5:1 was around 2012. LZMA is 20th century tech, and people still use gzip today. Compression technology doesn't account for the advertising number inflation there.
If anyone else is wondering if this is radioactive, it likely isn’t.
> While natural strontium (which is mostly the isotope strontium-88) is stable, the synthetic strontium-90 is radioactive and is one of the most dangerous components of nuclear fallout, as strontium is absorbed by the body in a similar manner to calcium. Natural stable strontium, on the other hand, is not hazardous to health.
Because of this, strontium is one of the most abundant elements in the universe, among those that are heavier than the elements of the iron peak, i.e. among the elements that cannot be produced by fusion, in stars.
There are various living beings which use either salts of strontium or salts of barium for their skeletons, instead of the more common salts of calcium (because the salts of Sr and Ba, especially the sulfates, are less soluble than the corresponding salts of Ca).
In magnetic tapes, either barium ferrite or strontium ferrite are preferred to ferromagnetic metals that have been used in the past, because they already are oxides, so they cannot be oxidized by the air, which enables long lifetimes for the magnetic tapes, of at least thirty years.
This was very interesting. I quit paying attention to tape formulations decades ago, but even when paying attention, I only knew of the names vs actually knowing this level of detail.
https://www.webmd.com/osteoporosis/strontium-treatment-osteo...
In any case, here's a lovely FDA safety data sheet of strontium ferrite. It... uh.. apparently doesn't cause pain when rubbed into the eyes of rabbits and guinea pigs. No LD50 tests were taken for the sheet though, so that's bittersweet I guess.
https://www.advancedenergy.com/globalassets/non-resource-lib...
Zero mismatch.
> While natural strontium (which is mostly the isotope strontium-88) is stable, the synthetic strontium-90 is
Later at my first job I was responsible for performing tape backups and rotating them. The most recent would go into the fire safe, and the one before that would go with the owner to take home.
e.g. LTO 3TB costs 30 EUR RDX 2TB costs 265 EUR
https://buy.hpe.com/us/en/storage/disk-storage-systems/remov...
Ideally I would be able to rent a tape drive, even if it costs 100 dollars per day it wouldn't really matter.
Did you have any specific discs in mind?
I recently bought "Sony Blu-ray BD-R XL 128GB", but I couldn't find any info whether its organic or inorganic.
The company that actually made M-Discs also went bust and there's a lot of conflicting information on the Internet whether or not the current M-Discs are real M-Discs or not.
* https://www.ebay.com/b/LTO-6-Tape-Data-Cartridge-Drives/3997...
Here's an eight-slot SAS library:
It's reasonably possible to hit a target of 2x hard drive cost with loose tapes but you need a lot of data.
And trying to match the price of hard drives with $10/TB tapes will require piles of data and piles of tapes.
It would be faster, easier, and cheaper to backup to a second set of disks and store them at a relative's or friend's place.
Some people keep a copy of their data at their place of work. Personally I'm opposed to that.
- Having the data be offline, ie, not available to an attacker which somehow managed to take control over your network
- Having the data be in a physical format which can be easily put in a safe or in the back of a car when there is a risk of flood or fire
- Having data on a media that won't regrade over a period of say, 10-30 years. Hard-drives tend to fail
Both of these issues can be largely mitigated by simply plugging the HDDs in every year or so and re-writing the data (maybe run badblocks while you're at it to ensure the drive is still in good health), but, as throw0101a mentioned, this takes time and effort.
[0]: https://serverfault.com/a/51893
[1]: https://www.partitionwizard.com/clone-disk/do-hard-drives-go...
How long a given SSD will last before losing data depends on the chips it is built with. I remember reading that Sandisk rates their flash storage as being able to survive a year without power (though I read that years ago and cannot find a source for it now), though [0] says 2-5 years, so technology has likely advanced since I last read about this. Either way, you should probably try and power up your SSDs at least once a year to keep the cells charged. This is less work than to keep an HDD healthy for cold storage, but it certainly isn't "set and forget" like tapes or optical disks.
Edit: I remember pulling my first SSD, a 60 GB Intel 330, from an old laptop and, despite having not been powered on for 3 years, it still booted. But that was MLC/DLC [1], which only has 4 voltage states, and I only saw that it booted, not that all my old data was intact.
[0]: https://www.easeus.com/resource/does-ssd-need-power.html
[1]: https://www.intel.com/content/dam/www/public/us/en/documents...
Manual labour.
There are tape libraries with robots that can take a tape out of a slot, put it in a drive, and then put it back automatically without humans needing to do anything. If you want to move the tapes offsite, there is some manual work to move the tapes from the import/export slot to a container.
A large library like the StorageTek SL8500 can have up to 640 drives, 100'000 tape storage slots, and 2'880 import/export slots.
Big cloud providers have disk replacement robots, so it's not unfeasible for robots to perform the "unplug and store the HDD somewhere".
But another advantage tape has is durability.
[citation needed]
I have no idea if anything like this is actually in use, I have my doubts. Also, a system like this probably isn't terribly useful, a Cartesian gantry robot would be much faster and way more robust than an articulated arm strapped to a mobile robot platform.
Wow the scale is just immense, that's pretty cool!
So, yeah, spinning rust might rust.
And if you replace dead disks every month or two, a non-networked server can probably manage more reliability over 10-30 years.
(The three major tiers are A: I accidentally deleted a file or directory yesterday, so I need to get a copy from snapshots; B: the machine I was working on exploded, so I need to rebuild it and repopulate it with my data; C: archive for some need in the far future)
Suppose you have large quantities of data that you need to preserve for legal reasons -- it's always legal reasons, unless you are in academia or similar -- but you will not be accessing much. That's when you use tapes.
For most other purposes, you want a snapshotting filesystem or an offsite filesystem made from those snapshots.
If you just want to store lots of data in a single place I think MAID still beats the tape at any scale (especially if you want automated retrieval)
It's discouraging that it's so complicated which made me back out of the purchase a few times, but it was well worth it.
The tape itself is quite cheap (LTO-9 is 18TB for $100), but the drives are expensive ($5000+ for an LTO-9 drive), so you need quite a bit of volume to make it worth it.
You can get an earlier breakeven point by buying used equipment for older generations. But no matter what you do, at small scale hard drives will be cheaper (and can be put on a shelf or in a safe too)
Essential when data is created faster than the locally available internet bandwidth which in media industries is quite often.
For larger amounts of data, the money savings vs. HDDs increase quickly.
The break-even point vs. HDDs is lower for long-term archiving than for short-term backups, because the lifetime of tapes is much longer (it is limited not by the degradation of the tapes but only by the risk that compatible tape drives might become hard to obtain), so you need multiple HDDs to store data for the lifetime of a single tape.
Right now you can get 18TB drives for $290 so you can get 306TB for $4,930
Have any data to back your claim?
So perhaps not 100% immune, but partial immunity to bit-rot is a pretty important part of the LTO spec.
I suppose that's anecdote. :) But it's what I've got.
Search for "year" in the specification of HPE's Ultrium LTO media:
* https://www.hpe.com/psnow/doc/c04154430
Or IBM's white paper:
S3 is a fantastic piece of architecture and service and the fact that the common dev has such tools and service available instantly from their personal laptop or even phone is mind boggling.
We are very fortunate that AWS led more services being made available to common folk than what had been done in the past.
Its not all about cost or hardware ownership--the value these near infinitely scalable services is easy to ignore but astounding at the grass roots level.
Of course, I never have, don't now, and never would run a business on someone else's computer though... all the hyperscalers will happily charge you more for 3 months worth of compute than the cost of the hardware the compute is being done on. Would I rather build a server for $5k or pay $1600/mo to rent that exact same server? Such a difficult decision! /s
The benefit is that you don't have to buy the entire server in order to use a portion of the resources on it. If you're going to use the entire server, buy the entire server — if you're only going to use a portion of a server, why buy an entire server?
Unless your compute needs fall below those of an Rpi Zero, it is almost always cheaper to buy physical hardware once that doesn't have a monthly bill attached.
Sure, scalability is a real value proposition from the hyperscalers that isn't the same with physical hardware, but is scalability worth paying 10x+ in OpEx for what could've been 1x in CapEx? For most businesses, I have a hard time making financial sense of that one.
There are no free lunches. I do not disagree buying hardware is cheaper but operational overhead and opportunity cost are real.
only if your data usage decreases over time.
There's a break even point, and that break even point is closer than you think. But by then, you're locked in. It is also the same reason why they are able to charge you more money than a self-setup storage system, since once you're locked in, the cost of switching is almost always just slightly higher than staying.
US case law is abundantly clear that web scraping is 100% legal. Further, I have never signed any agreement with AWS (I don't even have an AWS account) regarding AUP or anything like that, so I have no contractual restrictions on this either.
When it comes to doing basic napkin math and publish the results, that I find highly ethical, actually.
ala, i was gonna murder you, but at least you were warned!
So I think the practical difference is mostly between praising S3 (as synonym for "the cloud" when Amazon wasn't even mentioned, no less) and warning at least to use it diligently, which when it comes to costs might often be not using it at all.
The comment I replied to said:
> someone else's server in Virgina
While everyone may operate there now it was exceedingly common in mu experience to be referring to AWS when referencing Dulles / Ashburn / NoVA or just Virginia since AWS started there first.
So I figure S3 is the correct comparison to not having a 50 TB tape drive under your desk when "the cloud" is being discussed.
Most of us have used tape drives in the past for backups only to find that hard drives work better, are more reliable overall, can be accessed randomly, and are cheaper.
Imagine using a 2Gig DAT tape for backups these days, as I did do way back in the past. Or a 60MB QIC cartridge, as I did do a few years earlier than that DAT tape drive.
Am I reading this right? At that throughput it would take about a year to fill a compressed tape (assuming 3:1 ratio is achievable).
Or is there some parallelism involved? (i.e. many tracks written in parallel to the tape, I don't know a lot about tape drives)
(50e12 bytes / (400e6 (bytes/second))) = 125000 seconds ~= 35 hours ~= 1.45 days
(50 terabytes) / (400 megabytes) = 125000
Sony made digital cameras that recorded to floppy disk (the Digital Magnetic Video Camera, or MaViCa), and Agfa made one that recorded to PocketZip, the 40MB shrink of Zip disks. Both random-access media, but that isn't crucial for a digital camera. Could easily record to linear media, and in fact a later Digital Mavica did just that, onto CD-R discs. So recording to tape wouldn't be a stretch at all.
Instead of deleting photos on the camera, you could perhaps mark them for deletion but then have that take effect at offload time. Then blank the tape and use it again. At a time when 64MB flash cards were barely affordable, a 500MB tape would've been a gamechanger.
I use cameras as an example because they're probably the use-case that benefits most from a capacious writable medium in a portable format.
Of course in reality it depends on the data you feed it so the ratio has always seemed odd to me but it is what it is.
dd if=/dev/zero of=tmp.img bs=1G count=1 oflag=dsync
And got -
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.49586 s, 113 MB/sThe drive appears to be - WDC WD160EDGZ-11B2DA0, it's a 16TB disk, which I thought would be SMR, due to its size, but maybe I'm wrong.
Just found https://www.reddit.com/r/DataHoarder/comments/qf5ar3/is_this... which seems to indicate it's CMR.
This will reduce performance
only marginally slower than your average SSD
Cold tier cloud storage absolutely has tape behind it. You can even send tapes to some cloud vendors.
Broadcasting is another giant customer. Companies like WBD have Exabytes or Zettabytes of digitized programming (movies, series, documentaries) which would be impractical to store on disk. The typical workflow is for the program scheduler to direct the robot to fetch the tape containing a program 24 hours in advance of the broadcast time and copy it to disk, then erasing the disk copy after broadcast.
However, even those rarely accessed records often have more accesses per year than the hard drives can support - so you are forced to buy 400 drives instead of 200 drives and keep each half full.
End result: There is a lot of 'almost free' space on the hard drives, as long as you are happy to only read/write the data very rarely.
I don't think hard drive based storage and tape bases storage have quite the same use case.
An 18TB tape is basically the same size as an 18TB 3.5" hard drive.
I am using older LTO-7 cartridges, with a raw capacity of 6 TB. The newer LTO cartridges must be compatible mechanically with the old cartridges, because the same drive must be able to read and write them.
The LTO cartridges are smaller than 3.5" drives, they have maybe about 2/3 of the volume of a 3.5" HDD.
Moreover, a tape cartridge is many times lighter than a HDD. I have suitcases with 20 tape cartridges that are as convenient to carry as any typical suitcase.
A luggage carrying 20 HDDs would have a very noticeable weight, and its volume would be great not only due to the bigger HDDs, but it would require serious padding to avoid HDD destruction if the case were dropped accidentally.
* https://tapeandmedia.com/lto-ultrium-9-tapes/
If your data is compressible then you can of course store more. The drives themselves use Streaming Lossless Data Compression, but of course you can send them things like gzip, JPEG, MPEG files which are already compressed; SLDC:
* https://www.ecma-international.org/wp-content/uploads/ECMA-3...
LTO-10 is supposed to have 36TB raw and is estimated for perhaps 2024.