Hacker News Reader

Comment by bestboy | Hacker News Reader

If I understood it correctly, then they are using simple physical disk mirrors for redundancy. To me that seems like a huge waste of disk space. Parity based redundancy schemes like RAID-Z3 are way more space efficient. I do understand that parity based schemes need more time to heal/rebuild on drive replacements, but that does not seem to outweigh the huge amount of wasted disk space IMHO.

View on HN

The paired disks are in a different physical location so they also provide a degree of geographic redundancy.

Short of splitting a RAID array across two physical locations (a terrible idea), your proposal would require them to run mirrored RAID arrays in both locations.

This would give them greater redundancy, but would be a less efficient use of raw disk space than their current solution. It would also be more complex and difficult to maintain, and have performance impacts.

There are tables for bandwidth and iops vs differing raid levels, mirroring is generally the best.

True, but I do not consider nearline archiving to be a workload that requires lots of IOPS.

Besides the cross-DC issue others have mentioned, erasure coding everything can also exacerbate CPU or memory bottlenecks. Not sure if this is an issue for IA, but on my last project data would be initially replicated and then transparently converted to erasure codes after some time. I believe that some other exabyte-scale storage systems work similarly.

Reply on news.ycombinator.com