Hard Drive Stats for Q3 2017
backblaze.com
backblaze.com
- (mentioned this before) invoices need improvement. No accountant will accept this as of now. Please take a look at e.g. Digitalocean for how to do this right. I mentioned this at least 5 times to support...but nobody seems to care.
- upload speed is really bad from Europe (might be purely due to distance). Tried uploading one TB (@1.25MB/s), did around 25GB per day. Switched to a Hetzner Storage box, doing consistently 75GB per day right now.
- deleting your bucket (with data in it) requires to install a command-line tool which is really inconvenient. Took me 30 minutes+ and still threw some weird java.exception afterwards when clicking on „delete bucket“. Why not put a simple „delete“ button in the web ui?
Btw, what is missing in the invoices?
For us to be able to move from S3 we would also need to be able to create multiple API keys where we can set what buckets they have access to.
That's really useful to know. I am looking to switch from Amazon Cloud Drive for my cloud backups (now their pricing scheme has changed) and one criteria was much faster uploads. Sounds like B2 won't be it then, shame.
It's so bad.
Side note: One thing that terrifies me about B2 is the ability to make a bucket public (when it's private and intended to be). Would be nice to have some kind of lock on it to say "never private" or at least make it harder to change that opening the same settings page as other stuff.
Per-bucket keys would be nice too.
Love to see some performance stat to see if read /write IOP, MB/second, etc change over time on per model base.
Other than complete failure, do you collect check the any other early warning failure data such as ECC change over the life time of the drive? --- Would be a fun big data exercise for an intern to work on.
Also what's the usage patterns for your HDDs? read/write/spin down/sleep time in term of life time of the HDD.
I assume mostly write, some read. But also very low percentage is active?
Yes, we collect temperature data. If you download the dataset, you'll find a daily sample of the SMART data (including temperature) for every drive.
One statistic I'd really like to see included is the disk usage of each drive as a percentage of their total capacity. Would there be any correlation between that and the failure rate?
(Source: https://en.wikipedia.org/wiki/S.M.A.R.T.)
edit: No that can't be it. Maybe this data isn't available, hmm...
When 12TB HDD fails, it's equal to four 3TB HDD's failing at once.
They have to maintain more racks to do that (how much does it cost them to have their drive arrays running st 90%?) but that drops the amortized cost of replacing the drives.
But for me it might be cheaper to have one big drive that fails twice as often, because the odds of a drive being dead on any given Tuesday and the costs of addressing that might be low enough to offset the price differential.
It seems that you could lead this and give the industry a standard way, with vendors producing the fitting hardware, and allowing others to maintain their own vaults. Maybe this is a business advantage you don't want to lose right now. Then, I'm sorry to have asked :). If not then I think you might benefit by lower prices if the hardware is readily available and not a Backblaze custom production run.
There are many places who cannot for technical or won't for political reasons offload their storage to S3 and friends and need it all locally most of the time which would use a vault or two.
As for getting the servers, there's an entire company that popped up from one of our early storage pod producers, Protocase, you can buy servers (very reasonably priced btw) based on our designs here -> http://www.45drives.com/.
For now I'll stick with the plain offer. ^__^;
I even had RMA'd ones die within a week.
e.g.
6TB - BB have 4 times as many Seagate drives (and nearly 4x logged time) as WDC, but more WDC drives have failed. (4.6%WDC versus 1.2%)
8TB - Annualised failure is lower than HGST (1.1% & 1.2% versus 1.7%). Nearly 25,000 Seagate drives and 150 odd failures.
10TB - 0 failures over 13,720 drive days so far
12TB - 0 failures over 500 drive days so far.
Maybe I'm misreading something, but it looks from their data that the major Seagate issues have been put behind them now?
Yes. Per the description, that table is only drives that remain in service as of 2017. There have been "troublesome" Seagate models that failed at such high rates that Backblaze finally just pre-emptively pulled them, and those drives are not in the "cumulative" table they show in the OP.
https://www.backblaze.com/blog/3tb-hard-drive-failure/
> Maybe I'm misreading something, but it looks from their data that the major Seagate issues have been put behind them now?
That's what people said after the 7200.11 series. And after the 1.5 TBs shit the bed. And after the 3 TBs shit the bed. They're good for a while, then Seagate releases yet another model that shits the bed.
You can actually see that in the link above.
> While this particular 3TB model had a painfully high rate of failure, subsequent Seagate models such as their 4TB drive, model: ST4000DM000, are performing well with an annualized 2014 failure rate of just 2.6% as of December 31, 2014. These drives come with 3-year warranties and show no signs of hitting the wall.
2 years later they've released two new 4 TB models that fail at 50-100x the norm again.
While they may have a greater failure rate percentage, the price often offsets that for Backblaze's needs. Your needs may vary.
It's a lot easier to get Seagates in large volumes than some of the other brands - Toshiba specifically is problematic, iirc HGST maybe also.
Also, I didn't say Seagates were always shit. But some of their models are, and they are the only brand with these issues, and it's been a repeated problem for them over the last 10 years.
>Our next drive stats post will be in January, when we’ll review the data for Q4 and all of 2017, and we’ll update our lifetime stats for all of the drives we have ever used. In addition, we’ll get our first real look at the 12 TB drives.