Hacker News Reader

Comment by nynx | Hacker News Reader

Moving ram over to pcie makes a lot of sense. Glad that’s on the roadmap.

I think there are a lot of misconceptions here in other sub comments.

For starters, the DRAM manufacturers may possibly be the biggest penny pinchers in the semiconductor business. They're too cheap to use differential signaling even on GDDR memory. I'm not saying that PCIe RAM is never going to happen, but as somebody who works with DRAM controllers and DRAM vendors, I can not fathom a world where DRAM manufacturers put a PCIe controller on each chip, especially one capable of Gen 6.0 speeds! The extra die area, complexity, and bringup testing necessary for a PCIe controller is insane

Second of all, the latency would really destroy system performance. The DDR protocol is designed to minimize latency (well kind of, there are still speed targets that need to be met.) The whole point of using a parallel bus with unscrambled data is to eliminate encoding/decoding latency. PCIe uses a Serializer/Deserializer (SerDes) that's scrambled on one end and then de-scrambled on the other end using a LFSR. It also goes through an adaptive tap filter. After that, the ordered sets from each lane are assembled and sent through a FIFO for clock domain crossing/timing recovery. All of those steps add latency to the system, and that's before we get to the Data Link Layer and Transaction layer. While I'm not well versed in CXL, I know that it shares the same Physical layer (a SerDes Phy.)

Now you might imagine a chip with one PCIe end point that fans out to an array of DRAM chips on the back end, and you don't care that much about latency. But then that makes PCIe RC and EP in the system redundant because you could put the DRAM controller where you would otherwise put the PCIe controller. Now it might make packaging easier and allow for a cheaper version of DDR to be implemented, but is there a net savings? and does it justify the latency hit? Maybe. I'm not saying it's never going to happen, but I wouldn't bet on that tech.

Is it possible one manufacturer "takes the leap" and provides ram modules that are faster than competition. Therefore driving the transition purely on competition alone?

No, because they're in an unspoken, totally not illegal oligopoly not to dump prices.

Doubtful. PCIe Gen 5.0 x16 over DDR5 is a huge cost increase, a huge latency increase, and only a minor bandwidth increase (about double after accounting for protocol overhead.) At that point, you might as well widen your memory bus and use 2 DRAMs. The memory controller doesn't get much more complicated because both chips share a CA bus, and the DRAMs are cheaper. I don't work for a DRAM vendor, but I would imagine that the front end logic in the DRAM (the DDR controller inside the DRAM) uses way less than half the die area that a PCIe Gen 5.0 x16 EP would use.

There's a significant momentum behind CXL [1]. Samsung already announced a CXL DRAM module [2]. From where I sit, it's all but inevitable that this technology will get broader adoption, at least with large CSPs. Its benefits are just too good to pass up. The extra latency is estimated to be no worse than a NUMA hop [3]. Many companies are already figuring out how to best deal with the more complex memory topologies that CXL will enable [4, 5].

But I'm biased (I work for Intel), so take what I say with a grain of salt.

[1] - https://www.computeexpresslink.org/members

[2] - https://news.samsung.com/global/samsung-electronics-introduc...

[3] - https://www.lenovoxperience.com/newsDetail/283yi044hzgcdv7sn...

[4] - https://arxiv.org/pdf/2203.00241.pdf

[5] - https://arxiv.org/pdf/2206.02878.pdf

Well I'll be...

Though the renderings indicate that this is a CXL controller connected to DRAMs on the back end. It makes sense, since the front end logic for CXL is die area intensive. This is basically an SSD without all of the complexity of wear leveling.

> The extra latency is estimated to be no worse than a NUMA hop

I was almost going to mention that in my previous comment and point out that this might be a thing in enterprise applications where SRAM is plentiful, but not likely to outright replace DDR in most consumer computing devices as the parent comment seemed to suggest.

> This is basically an SSD without all of the complexity of wear leveling.

With all the benefits of an enterprise SSD (EDSFF in most cases) form factor. And that's what data center folks want.

> not likely to outright replace DDR in most consumer computing devices

Right. I expect that CXL will initially be deployed as an internal transparent optimization for cloud providers and large data centers, with the latency and complexity (to an extent) hidden at hypervisor/orchestration layers. Some sophisticated software might have special support. Eventually we might get S3-like disaggregated memory services for cloud instances enabled by pooling (pure speculation on my part).

The potential benefits of this may vary though depending on how the OS exposes things.

For example, exposing this as a ram disk while possibly useful is not really optimal. Treating it more like NUMA ram, that simply lacks any cores where the memory is "local", could be more viable. The OS could be programmed to shuffle out memory that is not frequently used over to there, as a faster alternative to paging to disk, etc. Some new madvise calls to let apps mark regions as not latency critical giving the OS permission to move the memory over even if it is frequently accessed could also be interesting.

> They're too cheap to use differential signaling even on GDDR memory.

Makes it quite impressive that they're able to push 20+ Gigabit/s/pin even in dual-sided / T-layouts (though that also uses PAM4). Of course, comparing historical GPU layouts it's pretty obvious how the RAM chips are creeping ever more closely to the GPU die - weird how that is.

The DRAM signaling is very strange, I worked briefly on a hardware DDR controller and the timing reference was over pins called dqs. These pins sent a signal to represent the bit transition so they could be safely captured by the receiver. All good, except the pins were bidirectional so when transmitting you sent a reference on the same pins, so lost the reference from the other side. The issue then was when you just switch from trasmitting to receiving, you have to quickly re-lock to the sending reference, which is a pain in the ass, and really complicates things. DDR phys could be simplified if dqs was unique per TX and RX directions, but nope, penny pinching on the signal count.

I don't think that is big of an issue. I have designed a memory controller and since the SDRAM protocol is really unambiguous which party owns the dqs lanes at what point I never had a problem with this. I'm curious what problems you encountered.

The main issue is not who owned it, but the fact a pll (or similar) was needed to create a local clock locked to dqs, but dqs from the sender would dissappear for an unknown number of cycles depending on the rx vs TX schedule. So that local pll could drift and would need to be re locked once dqs from the sender was presented again

There is a long tradition of serial memory interfaces. RDRAM/XDR DRAM, FB-DIMM/AMB, Intel SMB, OMI DIMM/DDIMM, etc.

They seem to have most success in high end server CPUs where memory capacity and throughput are important. They have cost, power, latency disadvantages, so their forays into low end servers, PCs, etc has been more sporadic.

Forgive me if this is an ignorant question, but couldn't you use PCIe RAM like cache in the same way a CPU's cache levels work? It's not as fast, but could feed the regular RAM faster than from another source.

The latency is going to be much too high, when the performance numbers turn up people will decide they don’t want it.

A similar thing happened with Intel Optane (which is now dying).

Memory latency is only bad when you have a cache miss. That’s why mainframes have gigabytes of cache between their processors and memory.

You can mitigate that with some in-package HBM. IIRC, Xeon Phis came with up to 16GB of it - you could in theory have a workstation without any memory sticks.

I would hope and expect that CXL gets latency low enough to make it viable. These days you're mostly waiting for the core of the ram chips to respond as everything else races around it.

For Optane, I'd say it's doing so badly because it's barely cheaper than RAM, which means it's only useful in niche situations.

It'd be interesting to see a desktop system with almost everything running over PCIe. You could drop the chipset entirely at that point.

Amd's A300 and X300 chipsets for AM4 are just a bit of rom. Modern CPUs don't really need a chipset; AM4 chipsets connect to the CPU through PCIe, anyway.

We'll be right back in the S-100 bus days

AMD EPYC is already that, a SoC - there is no EPYC chipset. They all have 8x 16-lane multi-function interfaces, which can be configured to work as PCIe, xGMI (inter-socket communucation), or SATA, as well as built-in USB, eMMC, LPC, UART, I2C, SMBus, GPIO.

Yeah, but somebody might insist that the IO-die means that the chipset has moved onto the CPU but still exists.

There’s IBM’s OMI as well.

https://openmemoryinterface.org/

what would ram over pcie help with?

I'm no expert but I bet some users would enjoy RAM over a switchable PCIe fabric. They'd take a latency hit but would have the potential for massive amounts of memory.

Many things. Capacity, Bandwidth, Serviceability [1]. It also paves the road towards disaggregated memory [2].

edit: The official Compute Express Link (CXL) website has an article about the benefits of this technology [3].

[1] - https://www.nextplatform.com/2021/09/07/the-cxl-roadmap-open...

[2] - https://pmem.io/blog/2022/01/disaggregated-memory-in-pursuit...

[3] - https://www.computeexpresslink.org/post/the-benefits-of-seri...

Reply on news.ycombinator.com