Hacker News Reader

Comment by cm2187 | Hacker News Reader

Right. Though I doubt backblaze is using RAID in the first place.

Backblaze does use RAID (erasure coding): data is striped across 20 drives, with 17 needed to complete a read. See:

https://www.backblaze.com/b2/storage-pod.html

"For Backblaze Vault Storage Pods each is one of 20 pods needed to create a Backblaze Vault. A Backblaze Vault divides up a file into 20 pieces (17 data and 3 parity) and places a piece of the file on each of the 20 Storage Pods in the Vault. We use our own implementation of Reed-Solomon to encode and distribute the files across the 20 pods, achieving 99.99999% data durability. We open-sourced our Reed-Solomon encoding implementation as well."

They do not use RAID. They use userspace erasure coding of data stored in files on plain ext4 IIRC.

Maybe they use RAID for the OS.

It's software RAID. RAID isn't a piece of hardware, it's a system design.

Yes it is a system design.

Backblaze's main reliability system is erasure coding[4] of shards of files, which is not RAID[1][2]. RAID mirrors disks or volumes on block or filesystem layers, not files in userspace.

That being said, I stand corrected in that they did use RAID at some point in time[3] (RAID6 on 13+2 drives), and may still be using some form of RAID. OTOH their reliability calculations don't seem to consider RAID - calculating with individual drive failures and shard rebuilds.

[1] https://en.wikipedia.org/wiki/RAID

[2] https://en.wikipedia.org/wiki/Non-RAID_drive_architectures

[3] https://www.backblaze.com/b2/storage-pod.html

[4] https://www.backblaze.com/blog/cloud-storage-durability/

As a lay person I don't understand why they wouldn't, could you give some more information as to why?

For these large scale infrastructures, they typically use JBOD, i.e. no RAID whatsoever and they achieve redundancy by maintaining multiple copies of the data over multiple disks spread around the datacentre. So it's not hardware RAID that requires a certain response time (I suspect the drives mentioned here drop because they timeout as it take a long time to random write on an SMR disk), more like a distributed software RAID. I think they likely also have their own file system so they are not dependent on a fixed block size, which allows them to save space if they have loads of small files.

I think that's about half right: for large storage infrastructure, you would indeed eschew local RAID, but instead of storing multiple copies, you'd use Reed Solomon encoding to stripe a single copy across many disks/servers/failure domains with a configurable number of added parity stripes. Full copies are really expensive!

I believe azure uses full copies [1]. I assume the other cloud providers must do the same.

[1] https://docs.microsoft.com/en-us/azure/storage/common/storag...

Google seems to use erasure coding for some (iirc. multi-AZ nearline, or so) storage classes, last I checked.

https://www.backblaze.com/blog/reed-solomon/

It's a similar idea but they distribute the copies between machines, not just across disks in the same machine. https://www.backblaze.com/blog/reed-solomon/ So the drives in each box are not in a RAID configuration.

RAID only protects you from disk failures, not machine failures. If you want to protect against machine failures (whole server offline), which might be an availability concern, then you would naturally want to replicate the data at a higher level. Once you’re doing that, it makes sense to only replicate the data at a higher level, because for a given level of safety, it is more efficient to replicate the data once, in one layer, than to replicate the data multiple times, at multiple layers.

RAID is only effective for workloads small enough that you care about a single machine.

Reply on news.ycombinator.com