RAMCloud
ramcloud.stanford.edu
ramcloud.stanford.edu
As a use-case example, those units have been in use for at least ten years by CCP Games [http://community.eveonline.com/news/news-channels/press-rele... and http://community.eveonline.com/news/dev-blogs/apocrypharrrrr...] as extreme-low-latency database storage for a fairly large cluster. At the time there were jokes of the staff needing to get special clearances just to see demo hardware before making their decision and eventual purchase due to restrictions related to their use by government and military customers.
Maybe I'm oversimplifying it or looking at it wrong, but I wanted to share my 2c while I was thinking about it. I'll take a better look later.
We have in less then a decade, all of a sudden remove most of our I/O bottleneck. From NAND SSD, to Intel Xpoint, and Giant amount of memory all within 10 years.
If I have a write-heavy application, for instance a time-series database, does it mean that eventually I'm going to consume all the RAM, then page cache, and eventually my throughput is going to be disk IO bound?
The backup copies of the data is stored on a configurable amout of _other_ systems in the cluster. They receive it to memory, and then the write is acked to the client. Then the backup is written asyncronous to disk on those systems.
You could configure that backups are not written to disk at all, if you have so much memory in the cluster (and do not care for that additional level of durability). Or the size of the buffer of unflushed data which would cover some longer write spikes.
The point is more that the backup copies become "free" once they are written to disk. Otherwise the cluster could only store 25% of it's memory size if you want 3 backup copies.