Dispelling Backblaze's HDD Reliability Myth

I read the article with great anticipation, but the author fails to poke any more holes in Backblaze's test that they themselves haven't already caveated.

Backblaze already acknowledged that their usage environment doesn't reflect that of normal consumer, yet the author harps on the point.

The author tries to debunk Backblaze's claim that it found no correlation between drive temperature and failure rate. Yet he presents no evidence that Backblaze's methodolgy was wrong.

Backblaze had large enough sample to have even distribution of drives in various temperature, rack, and load environment. I doubt they put all Seagate drives in the middle shelf of Ver. 1 racks where it was the hottest, but put all Hitachi drives in the bottom of the Ver. 3 racks where it was cooler and more stable.

Unless there is another study that shows failure rate of these drives in normal consumer environment, Seagate's 3-5x failure rate should give people a pause.

>"Backblaze claims that drive temperature doesn't affect drive life. That is counter to the observations of many others, including drive manufacturers."

Google has a whitepaper[0] on a large deployed sample size of drives which disagree with the author's claims here. I agree that more data from backblaze would be nice, but their claims do seem to have more plausibility than the author suggests. (If he has methodology issues with the Google paper, that would be more interesting)

[0]- http://static.googleusercontent.com/media/research.google.co... (pdf warning)

"With varying numbers of drives for each model, it is possible that some bad batches may have made their way into the sample pool, thus further skewing the numbers."

So, to measure drive reliability, the author recommends that you don't count unreliable drives, since it may "skew the numbers"?

This Tweaktown article offers nothing in the way of factual evidence to discount any of the hard work Backblaze put into that blog post.

Shame on Tweaktown for poo poo'ing on Backblaze's effort.

I'm curious why the author believes that beating the ever loving crap out of drives in harsh conditions and seeing which ones come out, overall, on top creates a myth. Perhaps the Backblaze article is better headlined "which hard drives seem to hold up well under pressure." Consumer environments usually have crappy, unreliable power (power strip chained to power strip plugged into an outlet from the 1960s), random vibrations (especially with kids), and poor air flow conditions (who here has done Family IT and had to carve the dust out of the case stuck behind a non-moving fan?). The only thing a consumer environment lacks from Backblaze's comparison is 24/7 read/write operations.

This article is astonishing. I try hard to give the benefit of the doubt to authors (as opposed to salespeople, speechwriters, and advertisers), and assume that they have good intentions and believe what they are saying. But I can't find any explanation for this article other than paid hit-piece or personal vendetta.

Here's the article's conclusion:

The data from Backblaze should not influence a purchasing decision by any consumer, regardless of what type of drive they are purchasing. The innumerable variables, and lack of documentation, ensures the results are unreliable. Even for the winners, the results aren't good; the failure rates are exponentially higher than those observed in the real-world. One should question whether these companies could survive financially with the massive warranty return rates in real-world scenarios.

And here is some of the logic:

It is the release of the data, in handy charts and graphs that encourage misrepresentation, which brings out the data-storage stickler in me. HDD manufacturers spend billions of dollars in R&D, and their labs are designed to characterize and measure the reliability and endurance of their storage solutions.

This last line converted from to "undecided" to "paid propaganda", as I doubt that vendettas come with such clearly presented "suggested talking points". It's worth reading if just to see such a stunning example of the art form.

The author of this article (Paul) appears to ignore the law of large numbers - while Paul's arguments against using the figures from Backblaze as measures of drive reliability would hold if Backblaze's analysis was based on a small sample size, because their sample size is relatively large, Paul's criticisms don't hold. Paul's argument is that any analysis of hard-drive reliability should control for factors like vibration and temperature as these vary in all environments - but in fact, because Backblaze has taken large (enough) sample of drives, these variations are already controlled for (ie cancelled out). Sure the consumer is probably never going to experience a failure irrespective of whether the buy a seagate or WD drive - but that's not the point - the point is that the long tail of the distribution, WD and Fujitsu beat Seagate for reliability. That is still valuable information for people, even if they aren't going to be using their drives in the same type of environment as Backblaze.

Warning: Browsing this site on ios mobilesafari will randomly taskswitch into the app store app, onto the buy page for some ios game.

Of course it isn't a large scale scientific experiment with controls in place for all the variables they are running a business and the data is a by-product (albeit one which they can use to optimise their own business).

But it is the largest scale testing I'm aware of that publishes the manufacturer names. For that alone it is valuable. I also like the fact it is harsher than normal environment, I don't want the most reliable disk if it never gets touched or used or to have to baby it.

That article was just like reading the Backblaze post, if the Backblaze post had been split into three pages and covered in ads.