Hacker News Reader

Comment by atYevP | Hacker News Reader

Yev from Backblaze -> We don't need the boot drive to be that "large", but it's more of a pricing thing. An SSD, even at 256GB would be more expensive than those 5400RPM drives, so we just go with what's cheap :)

View on HN

Reliability has a cost, too. You can get a 60GB SSD for cheap (Amazon suggests it's about the same price as the slow 500GB HDD), and the SSD will fail at a much lower rate than that HDD.

Yea fair point, might just be easier to order the 500GB ones through our suppliers along w/ our hard drive purchases, but yea, we might go down to SSDs soon. We're constantly iterating on these things.

When you have another 60 HDDs in the same case, I guess you think about reliability differently.

The 60 HDDs aren't single points of failure. {Edit: I mean for the server, not for the whole system.}

And a raid1 pair of HDDs for the system disk is more expensive than a small SSD, more fussy, and the SSD is still less likely to totally fail.

It doesn't sound like the boot drive is a single point of failure since data is stored Reed-Solomon coded in chunks across many pods. If one data drive fails, the whole pod has to go down for maintenance for it to be replaced. The only difference is that you get to choose when to take a pod down for maintenance to replace a data drive.

That's the situation I was talking about, yes. It's much cheaper to go into the datacenter once a month to replace failed data disks than to have to go in to promptly replace any system disk HDD that fails, in order to not have 60 idle disks.

Sure, that would make sense if they only had personnel in the datacenter once a month, but: "we replace about 10 drives every day" [1].

[1] https://www.backblaze.com/blog/vault-cloud-storage-architect...

To clarify, I'm not saying that they wouldn't be better off with an SSD boot drive. I'm arguing that having an HDD boot drive, given their setup, isn't awful.

What about network boot? Or do you find that to be too unreliable?

Not 100% sure if we've given that any thorough testing, but the ops team is looking at most available options before building these out. That said, if they haven't tried it, they'll read this and give it a go :)

Another option -- check out the very low profile USB thumbdrives (the one I've got sticks out about 4mm or so). Or see if you can get motherboards with an internal USB or SD socket. Maybe combine this with PXE boot for initial setup (the pod gets built with a blank internal thumbdrive, and on first boot it installs the Linux kernel and initial ramdisk image on the thumbdrive). Edit: It looks like the motherboard you use does have a USB header on the motherboard, where you can plug in a boot thumbdrive.

Supermicro also sells a sata plug mounted storage device it would have enough for a kernel to be loaded.

though i think a PXE or similar boot option would be a smarter move. just enough to bootstrap something to run in ram. (alternatively a minimal kernel and dedicate a section of the disks in the chassis to holding the running software (500MB x 45 with decent levels of erasure coding is a fairly minimal slice.) the kernel could then pull down the code after first boot (or updates)

Amazon apparently sells a bunch of 64GB and 120GB SSDs for less than the $53 cost of your boot drive.

Reply on news.ycombinator.com