Hacker News Reader

Comment by mey | Hacker News Reader

"128GB is now starting to become a limitation. With 16 cores that is only 8GB/ core."

This comment gave me pause, I feel like I'm falling out of pace with the evolution of commodity server systems. We've gone from being network (10/40gbe) and disk io (fixed with SSD, PCIe NVRAM and Fiberchannel) bound in the the near past to swing towards memory capacity bound?

View on HN

I think that's more of a reaction to the "fixes" to disk IO still not being fast enough.

The more you can keep in memory, the better off you're going to be, even with the fastest SSDs in the world.

See this article: https://queue.acm.org/detail.cfm?id=2874238

In a high-end server, with PCIe flash or NVRAM, you're increasingly unlikely to be IO bound.

My takeaway was that the single JBOD approach won't work in next generation systems as a JBOD of SSD/NVME/PCI-SSDs will be greatly limited by the controller, and that infrastructure complexity will increase presumably by having many JBODs with smaller number of disks.

Which is silly, because controllers can simply be designed to handle more data, and likewise they can be made more power efficient. So, far the only real limitation has been the interface, which off-course can be changed.

I'm not sure why that is your main take away from that article. The point is that relatively slow disk controllers designed for rotating magnetic storage are a bottleneck when you connect fast enough Flash storage. Connecting the storage via PCIe alleviates this bottleneck, to the point that you are no longer bound by the traditional constraints of small caches and large, slow permanent storage.

That was a great read, thanks for sharing!

I wonder how soon we will see the inevitable convergence of RAM and "storage". XPoint is still slower than RAM, but is a path towards unification.

XPoint still wears down; it's said to be 1,000 better than NAND flash but that's still highly finite.

Couple of iterations still and wear down suddenly won't matter anymore!

Isn't 8 GB per core still reasonable? Especially if you can share indices and buffer caches across cores, for example. There must still be diminishing returns for caching.

Depends on your dataset. Systems like Databases perform much, much better the less you have to touch a storage controller. One of the things that's going into Intel's 2017/2018 platform is the support for 6 channels of memory, and not all of that needs to be DRAM. There's a new standard, NVDIMM, that lets you use non-volatile memory as RAM, and take advantage of its much higher densities. So, if you have things like a database, you can map a much larger dataset into actual memory, instead of needing to just have the indexes in there.

I do a load of work with high throughput genetics sequencing. One of the programs that processes the data for a sample is single-threaded, but requires about 40GB of RAM. I currently have a server with 384GB RAM and 64 CPU threads, so 6GB/thread. RAM capacity is a definite bottleneck, since I can only run 9 at a time, instead of 64.

It all depends.

"reasonable" is entirely context-dependent. Some workloads are CPU or network bound. Others are memory bound.

That is correct.

NVMe-backed storage, 10 GigE-backed network IO, and 16-core CPU gets you pretty far nowadays for ~$5K USD, and RAM starts to be the limiting factor. Disk IO and network IO can be saturated pretty easily, and that kind of work usually involves a lot of RAM so here we are.

Keep in mind, Phi offers an additional 8 GDDR controllers at 2 lanes a piece. If you really ran into that 128GB wall, you could basically use your Phi as an off-loader and make those pages addressable. Profile your program and find the fetch patterns, then chuck in a cheap Phi as a memmapper for the LRU and/or older generation stuff. (I'd imagine you can borrow a lot of the last 20 years of GC research.) It's a non-trivial (but certainly doable) task.

That quote from the article makes sense if the author is thinking about virtualization. He is imagining running 16 VMs on it.

Honestly, our production systems are 256GB RAM and 16 GB/Core.

So no, I don't think so.

What's going on with this 128GB number? What's the actual limit here? I thought we had 40 bits of physical memory to work with.

The board has 4 ram slots, and 32gb is generally available (64gb is still in 'soon' territory, I think). Xeon e5 v3 has four ram channels instead of two for the processors in the article, and can handle three dimms per channel (or at least the board in the article only haa two slots per channel).

I assume there's some market segmentation happening here, but it's also a way to reduce board size and pin count.

That makes sense. I didn't realize these had so few channels. What I was concerned about was possibly baseless market segmentation engineering, in the same way that they elide ECC from the "Core" line of CPUs.

Samsung has announced they had a 128GB Stacked DRAM ready soon. So 4 DIMM could fit 512GB Memory.

So i guess the problem is with address space as mentioned above.

The Xeon Ds have had a max of 128 gigs, or 37 bits of addressing, since they came out last year. For Skylake, they have bumped up the addressibility to 36 bits from 35 bits, which is where it had been for a number of processor generations. For the E5 Xeons, the amount of physically addressable RAM is at 46 bits, and has been for a couple generations. Intel's used addressible RAM for a while as one of their segmentation methods to push people towards their higher end if their usage is RAM heavy.

it's market segmentation, sure, but also every bit of address line that you don't use translates directly into watts wasted.

Reply on news.ycombinator.com