RAMCloud - All Your Storage in DRAM | Hacker News Reader

RAMCloud - All Your Storage in DRAM

ramcloud.stanford.edu

64 pointsgjenks16 comments

From recent coverage of Open Compute on /.:

RAM Sled: Facebook wants to replace the leaves and run it on a RAM sled with between 128 GB and 512 GB of memory, and for $500 to $700 per sled. Only a basic CPU would be needed. Total queries would be 450,000 to 1 million key queries per second. http://slashdot.org/topic/datacenter/how-facebook-will-power...

Not really any way to know what Facebook is up to here: 512 GB is normally done via 4 sockets, 4 channels, 2 dimms per channel, 16GB dimms; whether FB intends on disrupting any of the factors in this equation for getting mass memory on a single system will be interesting to see.

It has been rather shocking to me that FB-DIMMS & other approaches which allow a lot of RAM to be chained to one another- really deep channels- hasn't seen any widescale success. RAM is cheap enough: would that we could plug in a lot of it.

If you just care about capacity and cost you can build a system with a $20 ARM SoC for every two DIMMs. The ultimate is BlueGene-style packaging with an SoC on each DIMM, but Facebook's probably not ready for that.

Other direction you could go would be to put a bunch of the RAM on a peripheral bus (e.g, PCIe).

Similarly, I've been waiting to see an extended modern version of the iRam Box[0], but it doesn't seem to be coming. The 4GB limit is useless now, but imagine the same with hundreds of gigabytes.

[0]: https://en.wikipedia.org/wiki/I-RAM

Somewhat relevant, but on a much smaller scale I suppose:

I had to geocode 100million addresses and used PostGIS to get it done. It was painfully slow on disk, so I rented some big-ram servers, created ramdisks, configured postgres to run from those ramdisks, and it was off to the races.

What would have taken days of processing was taken down to several hours. The speed improvement was amazing. Even SSD couldn't touch it.

How is RAMCloud different from something like Redis?

It is intended to be durable and available despite host crashes and localized hardware failures. They discuss getting 64GB back into DRAM in 0.6seconds from secondary backup store. The trick is to have many hosts share the burden, so that when one host goes down, 1/100 of its memory is quickly sucked into each of 100 machines from another 100 disk/flash backups. Neat.

So RAID across network noes.

The idea is not that weird. Lots of applications run from RAM most of the time, needing long cache warmups before performing well. If you keep your entire storage in RAM this isn't necessary, and you can do away with the caching layer.

This looks like the "RAM SAN" products, where there's a bunch of memory and some spinning disks with enough local battery backup to flush to disk. I think the main difference is that they want the data aggregated across multiple machines, which I don't think that previous solution had.

Can anyone shed light on how persistence is achieved? The wiki page doesn't seem to come to any conclusions:

https://ramcloud.stanford.edu/wiki/display/ramcloud/Data+per...

Hire more server huggers, make extra sure das-blinkenlights stay blinking, buying a second backup generator.

And- obviously enough- off site redundancy; multiple data centers. Which they talk about the difficulty of: 10Gbps pipes between DCs is not a lot compared to write throughput of a single computer's main memory, much less a DC's. For some compressible checkpoint logs there might be a way forward.

Perhaps we need a new *-cast technology: after any-cast we'll all be needing n-cast, which sends our traffic to the n-nearest DC's? :)

If it's a distributed network, data can be distributed across nodes using a RAID algorithm.

Here's the long answer to your question: http://www.stanford.edu/~ouster/cgi-bin/papers/ramcloud-reco...

I often run stuff in ram, the improvements are dramatic. My latest deed was dumping/restoring 20GB of RRDs; doing it in tmpfs was FAST. For running programs stuff can be kept in sync with the likes of anything-sync-daemon. SQL can get trickier, but it can be done.

What's odd is they didn't just use nbd + ram disks + RAID. Building a whole key-value store around it makes it a lot less general purpose, and there are perfectly good key-value stores that run off RAM...

Reply on news.ycombinator.com