Backblaze Drive Stats for Q1 2021

It's marketing (and effective, too). I became a Backblaze customer after reading one of these reports (many years ago!).

If it only would also backup attached USB sticks.

Gives valuable insight on drive stats at a scale that individuals can't ever reach as well.

In addition to the other mentioned reasons, I guess it's valuable feedback for their providers, and if that feedback is acted upon, it's beneficial for them.

> why do they share this info? Is it to show they’re reliable or just for curiosity? Or some other reason?

It has many facets. It's attractive for us, nerds. It also helps their providers to spot problematic models. They also show their technical prowess, and lastly I always take a look to the most failing models and try to avoid them in the data center, if I can.

"Transparency breeds trust. We’re in the business of asking customers to trust us with their data. It seems reasonable to demonstrate why we’re worthy of your trust." - Blackblaze

Is it, have you had good experiences loading the backups? I was on the lookout recently for some offsite backup solution and came across this blog post that didn’t inspire confidence in backblaze.

https://messengergeek.wordpress.com/2018/03/09/backblaze-rev...

Yev from Backblaze here! We do it because it's interesting! Initially, since we're pretty transparent, we were hoping that others would join in the fun and share their stats so we'd know how our environment and hard drive stacked up against others - but no one at our scale has really done that yet. We do have a lot of off-the-record confirmation that our experience is somewhat similar to others, which is neat - but we were just trying to be transparent and share something interesting from our infrastructure. A lot of folks jumped on it and found it interesting so we keep it going!

Plus along the way some folks find out about us and sign up for the services we offer (B2 Cloud Storage and Computer Backup) and that's nice too! Plus we also like these conversations and at the end of the day, it's fun!

A well written SSD firmware simply slows down with age. It will never get to failure because the slowdown gets so extreme it becomes unusable. The drive also gets slightly smaller (by passing write failures to the OS to mark sectors bad).

Thats because a "worn out" flash sector is never fully worn out - it can still store some data, just less than the error correction can correct. It is possible to combine two sectors to have extra error correction data to still recover a sector. Now you have less than half the performance.

Worn out flash also doesn't hold data long - perhaps only a few hours before too many bits have flipped and it is unreadable. To fix that, you need to rewrite the data, which slows everything down more.

And now that you have a bunch of unreliable sectors, you also need "super-sectors" which can do sector-based hierarchical erasure coding to recover data from sectors where even the methods above have caused data to be lost. This slows down writes even more.

In the worst case, reading a single sector requires reading every sector on the drive to reconstruct. Clearly thats going to be slow enough the drive will have stopped being used long before that.

Sadly some drive firmware doesn't implement some or all of the above, so they appear to have "failed" and become unreadable, which IMO is inexcusable when it's very easy to design so that worn out drives become slow instead.

I can only speak about the Exos drives, but the Ironwolf (Pro) drives are basically just the Exos drives relabeled for the consumer market. I'm actually about to buy 6 14TB Exos x16 drives to replace 7 6TB Exos 7E8 drives that are 5 years old. Frequently you can get the Exos drives cheaper than Ironwolf anyway.

I would definitely recommend skipping any current large WD drives for home use. < 6TB are SMR and problematic in a NAS, and 6TB+ have a very irritating noise that no-one seems to be able to diagnose (sounds like a scanner when idle). I bought some 8TB WD Golds, and I was expecting them to be louder, but they do something very weird when idling and it's an extremely penetrating sound. It seems to be present on most large models: https://community.wd.com/t/strange-noise-coming-from-10tb-dr...

Backblaze has said in the past that they don't buy enterprise drives because they didn't notice any difference from consumer ones. I don't know if that's still their policy.

Disclaimer: This is my personal experience from being an HPC sysadmin and old school computer enthusiast.

If there's one trend I've seen from using generations of HDDs, excluding some problematic generations like first SATA Seagate Barracudas and early WD Caviars which died for no reason at all, newer generation HDDs are always more reliable from previous generation, regardless of their class (datacenter / consumer).

For the last 10 years or so (starting with the introduction of first WD Green / Blue / Black series), the HDDs are exceptionally reliable unless you abuse them on purpose (like continuous random read/write benchmarking).

I've replaced two 11 year old WD Blacks w/o any problems this year to upgrade to two IronWolf Pro NAS drives, because I wanted something dense and PMR. At office, I changed an old Seagate Constellation ES.2 (aka Barracuda enterprise) drive since it started to develop bad sectors (which I removed from an old disk storage unit anyway). IIRC, it was around ~10 years old too with a much heavier workload history.

Looks like the most differentiating factors between enterprise and consumer drives are the command sets they support and features they bundle. NAS and other enterprise drives have features to make them more reliable in harsher conditions (heat, vibration, operational knocks induced by hot swapping, etc.).

If you're getting an enterprise disk with a storage unit, you're probably also getting disks with special firmware developed for this brand anyway, so they're not off the shelf enterprise drives.

At the end of the day, for normal operating conditions, device class doesn't matter for the home user, but for density and speed, you might need to get an enterprise drive anyway.

Andy for Backblaze here: I looked at that 3-4 years ago. It looks like power cycling increased failure rates, but we don't power cycle our systems very often, maybe 1-2 a year, so not the best use case. This is on my list for a relook one of these days, if we find anything interesting we'll let folks know.

On the bright side it didn’t lose you your data!

black swan spotted

That very well could be on the delivery person / packing of the hard drive.

Hard drives are quite fragile: if they're dropped hard by the delivery person (ex: dropped onto your concrete steps), they could break.

Andy for Backblaze here: A while back we did an analysis of drive failure over time, i.e. the bathtub curve. It is probably a good idea to update that, as I believe we are seeing lower failure rates upfront these days.

Depends on the SSDs and the cause of failure. There have been high profile cases where a firmware bug meant an absolute cap on hours running.

I've been involved in server farms with thousands of (mostly Intel) SSDs and (mostly WD) spinning drives; the spinning drives tended to have pre-failure indicators, but we couldn't figure out any indicators before SSD failure and generally they would just completely disappear from the bus when they did fail. The failure rate was signficantly less though. Our write rate wasn't very high and tended to be small writes; for busy disks, more than what we could do with a spinning disk, but usually not near the capability of the drives.

Andy at Backblaze here. We do look at drive model failure over time. We did a post on this topic several years ago, 2015? At the time, most drive followed the bathtub curve of failure, but I'm not sure that is still the case. I think its time to update that report.