Coughlin: SSDs will not kill disk drives

Hard drive technology pretty much always got more complex as time goes on. Do you think bits shrink themselves?

20+ years ago, we had "Get Perpendicular" explaining why perpendicular bits were a good idea. https://archive.org/details/get-perpendicular

All future technologies are more complex than our past techs. I mean, should we say that future SSDs are doomed because they're going for 4-bits per cell and 300+ layers of silicon rather than the single-layer single-bit simple designs from 10 years ago?

If it is not plugged in, i.e. unpowered, don’t count on anything more than three months. Thread with sources: <https://news.ycombinator.com/item?id=27573332>

Modern hard drives have warrantee limits on how many bits you can read/write during their service lifetime, since they lower the head from about 10nm to 1-2nm during reads and writes, and head lifetime is correlated with the number of hours it spends at that 1-2nm height.

With workload specifications in the range of 500TB/year (IIRC - I took a quick look and couldn't find any recent specs) that works out to less than QLC levels of endurance. It's not the same, though - if you read/write every byte of a hard drive 300(?) times the failure rate rises and the vendor gets nervous; if you overwrite QLC 3000 times it's on its last legs, and 1 or 2K more writes will almost certainly kill it.

SSDs have quite predictable durability, mostly because the unit that fails is less than 1GB, so the law of large numbers kicks in.

Note also that "durability" is a soft target - the normal failure mode is that blocks retain their data for shorter and shorter times before hitting the ECC error limit, so if your storage system moves data around every few months you can push the flash harder than in e.g. a laptop, where you don't want to risk losing all your data if it sits powered down on a shelf for half a year.

There are at least these two reasons: proper testing of endurance of a device is not possible, you can only test that in a pretend kind of way. We are talking about years of service that have to be somehow emulated in at least weeks... and, of course, you cannot really account for physical behavior of materials from which the disk is made by simply running multiple I/O workloads. Now, add to this that the larger the storage capacity, the harder it is to check durability because throughput becomes the bottleneck. I.e. if you emulate device wear by running more I/O workloads, then, proportionately to the size of the device, you will be able to run fewer of them per unit of storage, because you are bounded by the throughput.

Any real (good) tests so far rely on previous generation of devices, and don't necessarily reflect the current situation. I.e. if your SSDs survived for only five years, who's to say that the next generation you will buy will last more or less? They are very likely not the same kind at all...

Also, it's really silly to measure disk durability in units of time... I mean, most tests that intend to measure durability model it by running I/O workloads, so, they'd typically measure durability in something like "how many times can a unit of storage be written over". The usage patterns vary dramatically across different kinds of workloads. So, if, eg, you are running a build server, you will wear your storage a lot faster than if you are running a (well-configured) database server, and still much faster than if you were running a video streaming service, and still much faster if that's a (well-configured) Web server... and the difference could be an order of magnitude between these.

That’s a neat feature, but I’d expect a lot of software to behave badly if local file system operations suddenly started exhibiting tape-level latency.

Add a deep sleep mode and bam, you got a stew goin’.

Or what HDD companies would themselves. I recently was forced to buy WD JUCT, which are SATA-2 (!), 5400rpm, 16MB. There were literally no other non-SMR 2.5 drives at tgat moment, even WD REDs.

This is true when you compare them to media designed for long-term storage. But HDDs aren't all that terrible at it. I recently needed to dig out a 30 year old hard drive to recover some data from it, and it worked flawlessly.

Flash replaced mini-drives in iPods when the price of "enough" flash (2GB or so) dropped to that of a mini-drive.

SSDs replaced HDDs in laptops when the price of a "big enough" SSD (256 MB?) became competitive with an HDD.

Every reduction in flash cost after each transition was a win for the vendor, since they could keep storage constant and reduce the price. (unlike HDDs, where the price stays constant and the amount of storage goes up)

In each case some users (over-represented here) wanted more storage, and vendors didn't care, or were happy to sell more flash for a price premium.

Enterprise systems aren't single drives, but I believe they still have a concept of "enough" - if the savings from HDD are marginal, or the performance loss from concentrating data on fewer and fewer HDDs becomes too much, they'll switch to pure flash. Anecdotally that's already happening. (also, to be honest, another driver is probably because people like to spend their employer's money on high-tech shiny things, which often pays off better career-wise than saving money)

For the Googles and Amazons of the world there will probably never be a value of "enough", and HDD won't fade until the IOPS/TB ratio becomes ridiculously bad. Maybe not even then, as HDD may still be the best way to store data for a few decades.