Comment by Lramseyer | Hacker News Reader

Comment by Lramseyer | Hacker News Reader

Parent Full thread

I think there are a lot of misconceptions here in other sub comments.

For starters, the DRAM manufacturers may possibly be the biggest penny pinchers in the semiconductor business. They're too cheap to use differential signaling even on GDDR memory. I'm not saying that PCIe RAM is never going to happen, but as somebody who works with DRAM controllers and DRAM vendors, I can not fathom a world where DRAM manufacturers put a PCIe controller on each chip, especially one capable of Gen 6.0 speeds! The extra die area, complexity, and bringup testing necessary for a PCIe controller is insane

Second of all, the latency would really destroy system performance. The DDR protocol is designed to minimize latency (well kind of, there are still speed targets that need to be met.) The whole point of using a parallel bus with unscrambled data is to eliminate encoding/decoding latency. PCIe uses a Serializer/Deserializer (SerDes) that's scrambled on one end and then de-scrambled on the other end using a LFSR. It also goes through an adaptive tap filter. After that, the ordered sets from each lane are assembled and sent through a FIFO for clock domain crossing/timing recovery. All of those steps add latency to the system, and that's before we get to the Data Link Layer and Transaction layer. While I'm not well versed in CXL, I know that it shares the same Physical layer (a SerDes Phy.)

Now you might imagine a chip with one PCIe end point that fans out to an array of DRAM chips on the back end, and you don't care that much about latency. But then that makes PCIe RC and EP in the system redundant because you could put the DRAM controller where you would otherwise put the PCIe controller. Now it might make packaging easier and allow for a cheaper version of DDR to be implemented, but is there a net savings? and does it justify the latency hit? Maybe. I'm not saying it's never going to happen, but I wouldn't bet on that tech.

There's a significant momentum behind CXL [1]. Samsung already announced a CXL DRAM module [2]. From where I sit, it's all but inevitable that this technology will get broader adoption, at least with large CSPs. Its benefits are just too good to pass up. The extra latency is estimated to be no worse than a NUMA hop [3]. Many companies are already figuring out how to best deal with the more complex memory topologies that CXL will enable [4, 5].

But I'm biased (I work for Intel), so take what I say with a grain of salt.

[1] - https://www.computeexpresslink.org/members

[2] - https://news.samsung.com/global/samsung-electronics-introduc...

[3] - https://www.lenovoxperience.com/newsDetail/283yi044hzgcdv7sn...

[4] - https://arxiv.org/pdf/2203.00241.pdf

[5] - https://arxiv.org/pdf/2206.02878.pdf

Well I'll be...

Though the renderings indicate that this is a CXL controller connected to DRAMs on the back end. It makes sense, since the front end logic for CXL is die area intensive. This is basically an SSD without all of the complexity of wear leveling.

> The extra latency is estimated to be no worse than a NUMA hop

I was almost going to mention that in my previous comment and point out that this might be a thing in enterprise applications where SRAM is plentiful, but not likely to outright replace DDR in most consumer computing devices as the parent comment seemed to suggest.

> This is basically an SSD without all of the complexity of wear leveling.

With all the benefits of an enterprise SSD (EDSFF in most cases) form factor. And that's what data center folks want.

> not likely to outright replace DDR in most consumer computing devices

Right. I expect that CXL will initially be deployed as an internal transparent optimization for cloud providers and large data centers, with the latency and complexity (to an extent) hidden at hypervisor/orchestration layers. Some sophisticated software might have special support. Eventually we might get S3-like disaggregated memory services for cloud instances enabled by pooling (pure speculation on my part).

The potential benefits of this may vary though depending on how the OS exposes things.

For example, exposing this as a ram disk while possibly useful is not really optimal. Treating it more like NUMA ram, that simply lacks any cores where the memory is "local", could be more viable. The OS could be programmed to shuffle out memory that is not frequently used over to there, as a faster alternative to paging to disk, etc. Some new madvise calls to let apps mark regions as not latency critical giving the OS permission to move the memory over even if it is frequently accessed could also be interesting.

The DRAM signaling is very strange, I worked briefly on a hardware DDR controller and the timing reference was over pins called dqs. These pins sent a signal to represent the bit transition so they could be safely captured by the receiver. All good, except the pins were bidirectional so when transmitting you sent a reference on the same pins, so lost the reference from the other side. The issue then was when you just switch from trasmitting to receiving, you have to quickly re-lock to the sending reference, which is a pain in the ass, and really complicates things. DDR phys could be simplified if dqs was unique per TX and RX directions, but nope, penny pinching on the signal count.

I don't think that is big of an issue. I have designed a memory controller and since the SDRAM protocol is really unambiguous which party owns the dqs lanes at what point I never had a problem with this. I'm curious what problems you encountered.

The main issue is not who owned it, but the fact a pll (or similar) was needed to create a local clock locked to dqs, but dqs from the sender would dissappear for an unknown number of cycles depending on the rx vs TX schedule. So that local pll could drift and would need to be re locked once dqs from the sender was presented again

> They're too cheap to use differential signaling even on GDDR memory.

Makes it quite impressive that they're able to push 20+ Gigabit/s/pin even in dual-sided / T-layouts (though that also uses PAM4). Of course, comparing historical GPU layouts it's pretty obvious how the RAM chips are creeping ever more closely to the GPU die - weird how that is.

There is a long tradition of serial memory interfaces. RDRAM/XDR DRAM, FB-DIMM/AMB, Intel SMB, OMI DIMM/DDIMM, etc.

They seem to have most success in high end server CPUs where memory capacity and throughput are important. They have cost, power, latency disadvantages, so their forays into low end servers, PCs, etc has been more sporadic.

Forgive me if this is an ignorant question, but couldn't you use PCIe RAM like cache in the same way a CPU's cache levels work? It's not as fast, but could feed the regular RAM faster than from another source.

Is it possible one manufacturer "takes the leap" and provides ram modules that are faster than competition. Therefore driving the transition purely on competition alone?

Doubtful. PCIe Gen 5.0 x16 over DDR5 is a huge cost increase, a huge latency increase, and only a minor bandwidth increase (about double after accounting for protocol overhead.) At that point, you might as well widen your memory bus and use 2 DRAMs. The memory controller doesn't get much more complicated because both chips share a CA bus, and the DRAMs are cheaper. I don't work for a DRAM vendor, but I would imagine that the front end logic in the DRAM (the DDR controller inside the DRAM) uses way less than half the die area that a PCIe Gen 5.0 x16 EP would use.

No, because they're in an unspoken, totally not illegal oligopoly not to dump prices.

Reply on news.ycombinator.com