Hacker News Reader

Hacker News

Comment by oasisbob | Hacker News Reader

Full thread

oasisbob

I think this is a neat development, but doesn't this post overstate the case for reduction in complexity?

I hope a Racker will correct me if I'm wrong, but:

- With the exception of the boot device, you're dependent on non-local storage over the network.

- You're not going to get dedicated networking gear

- OnMetal is still dependent on the virtual networking layer (OVS?)

I can appreciate having more bang for the buck, but there are enough shared components here that I would be very hesitant to assume that that MTBF crosses the imaginary line where you can eschew fault-tolerant architecture patterns.

View on HN

For the network, you're getting 2x 10Gbit connections, configured in an HA-Bond, and plugged into a real switch. There's no current dependency on a virtual networking layer.

How do IPs get assigned? Is that still over nova network?

That's good to know. How and where are ACLs handled?

Engineering manager on OnMetal here.

Our first iteration, you'll get two networks: "ServiceNet" is our intra-DC network, which is unmetered (ie, free bandwidth) and "PublicNet" which is the internet and will be billed just like we bill for other cloud server bandwidth. Collectively you get 10Gb/s across these, which you can break down however you want. To you these just show up as VLANs which will be configured on build.

We're working on hooking these boxes up to our "Cloud Networks", which are per-tenant software defined networks. If this sounds like a fun project, hit me up, contact info is in my profile.

What makes you think it is non-local? Two of the options are RAM only (32GB and 512GB). That's like memory chips plugged into the motherboard (what modern normal user computers often only have 2 to 4GB of). You would run a RAM disk if your software needed something more than the boot drive to run using that. That's why the numbers are so large.

The other option is PCIe Flash, that's a solid state "disk" plugged into the motherboard expansion slot, basically. Again since that flat out state PCIe, they are saying it is local.

Are you saying it's appropriate to run a database on a RAM disk? This product is offered without a disk, so it has to be coupled with a SAN or similar remote storage system.

Our IO flavor has 2x 1.6TB PCIe flash cards, this is how we see most users running databases or otherwise persisting "hot" data. I've gotten to play with these over the last few months, and its really incredible how fast they are - they totally crush even high end SSDs. People have traditionally used these in a tiered setup to accelerate spinning disks, but the costs have come down to where it hardly makes sense to use anything else for low latency persistent storage.

We're working to hook this up to our existing block storage offering as well, which will offer more flexibility to attach SSD or spinning disk backed block devices to any flavor.

Can you recommend a specific brand of PCIe flash card? How reliable are they?

We are running these in OnMetal: http://www.lsi.com/products/flash-accelerators/pages/nytro-w...

Reliability is similar to an SSD. The underlying storage tech is basically the same: some flash chips, with a controller that handles wear leveling and so on. Just that instead of running operations through a SATA controller it uses the PCIe bus, which offers higher bandwidth and lower latency.

Interestingly, Apple is shipping PCIe flash in all (I think) of their latest laptops.

Oh, I see, just plain SSDs with PCIe instead of SATA. Yes, that seems logical.

Reply on news.ycombinator.com