Backblaze Drive Stats for Q1 2022
backblaze.com
backblaze.com
Backblaze shouldn't complain though ;)
Some interesting answers to this and other questions.
"Due to the quantities we need and the visibility of the posts, drive manufacturers are motivated to give us their best possible product." ....
And the best thing? These bad boys are only 250 euro's for 14TB or 300 euro's for 18TB of raw storage capacity.
The "Age vs AFR" plot is getting there but it should be flipped and is hard to read; error bars would also help.
Also percentiles would be useful: people want to minimize the probability of failure.
> Is there a failure list for write heavy drives?
To be clear about what these drive failure stats are and what they are not: Backblaze runs a data storage service with about 214,000 hard drives in it right now. We don't run any specific tests or induce issues on purpose FOR the drive failure stats, we just report what occurred in our datacenter.
Sometimes readers think we're carefully running a "study", but it's more just what we have experienced as honestly as we can offer it up. If the reads and writes and seeks our drives experience matches your particular application, great! Or maybe it is just interesting to read.
Now, we do save (and publish) all the raw data, and some other awesome people out there have done various analysis on it, which always makes us happy also. You can find the raw data here: https://www.backblaze.com/b2/hard-drive-test-data.html At this point it goes back almost a full decade.
Someone with some time could correlate that to failure rate. My hypothesis is of course it's correlated--but by how much?
We looked into it a little, some notes written up here: https://www.backblaze.com/blog/what-smart-stats-indicate-har...
Short summary is there are a few SMART stats that seem to predict failure way more than others, which is probably obvious. But we aren't PhDs in statistics and it isn't our area of focus, so....
This guy wrote a paper based on the Backblaze SMART data: https://etd.ohiolink.edu/apexprod/rws_etd/send_file/send?acc...
These 5 guys wrote another paper based on the Backblaze SMART data to train up a Bayesian network to predict failures: https://ieeexplore.ieee.org/document/8489097
This is another article of predicting hard drive failures using the Backblaze SMART data: https://karthikna.github.io/Prediction-of-Hard-Drive-Failure...
I can't comment on their findings, but it's DEFINITELY an interesting thing to study now that we have almost 10 years of these drive stats across a pretty big drive farm.
I'm building https://github.com/stonecharioteer/renfield for this purpose. Before I get around to it, I'm trying out git-annex, but I must say I don't like the git approach to files.
I wonder if they've done any proper Cox proportional hazards lifetime analysis across the different models? The stats for this sort of problem are well developed. Of course, the hazard is not going to be proportional, I suspect, bit I'd love to see that come out of the regression...
Generally I only buy enterprise drives
Continually reading the block seems doomed but it works a surprisingly large number of times, and when it's successful the block is often put somewhere else on the drive transparently (and the old block is marked as "bad"). This drastically improves the lifetime of the drives since they're slowly wearing down vs "going bad" because of a single block being unrecoverable.
This is not the only difference, but it's one thing to consider.
It's similar to how the linux kernel can be compiled with low latency in mind; which causes lower throughput but makes the desktop experience much nicer (ime). It's just trade-offs. Nothing is just "objectively better" it's a different trade off.
According to the link I shared, they aren't lower quality, they're higher quality than enterprise drives.
But that's an interesting note about the drive firmware!