Yes I agree with your points. On our 1U boxes we run 8 drives in RAID6. On our 2U boxes we run 22 drives in RAID6 + 2 hot spares. So we use a similar approach in having hot spares on the bigger boxes.

As pointed out in the article, using distributed block storage means you need a pretty high performance storage network and low latency is just as important as high bandwidth.

We already have a physically separated storage network that runs in parallel to our public network. This is used primarily for drive traffic over iSCSI. We have a standard gigabit redundant network for this. As a physically separated network it means we don't get data integrity issues caused by DOS for example.

Moving to distributed block storage will mean we will upgrade our storage back-end to either 10Gbps Ethernet or Infiniband. The advantage of Infiniband is that as a cloud provider we will have essentially a large grid which is what Infiniband is meant for. We can also use 40Gbps per port so with dual networking can go up to 80Gbps relatively cost effectively. Just as important is the super low latency. Suffice to say we are ahead of the software on this one :-)

I think its important to point out that as storage moves to these sorts of systems it will be come increasing difficult to replicate such a setup in a redundant fashion on dedicated hardware. As a cloud provider we spread the cost over many customers.

Best wishes,

Patrick, CEO, CloudSigma