Hacker News Reader

Comment by toast0 | Hacker News Reader

I'm not sure I really understand why Optane went in the DIMM slots instead of the PCIe slots. I know PCIe storage is usually NVMe, but you could also expose the storage as a large memory mapped region. You wouldn't need explicit support from the cpu/chipset/motherboard, although you probably want resizable BAR support, cause the region would presumably be pretty large, and maybe you want to do something for older systems that can't handle a giant region. From the software side of things, PCIe mapped memory is more or less the same as physical memory; from the hardware side, you'd be taking a different path to get there, so you'd have different bottlenecks, but PCIe5 is here with 2x the bandwidth and there's a limited number of applications that really max out their PCIe lanes.

View on HN

PCIe Optane was a thing and it achieved 10us latency whereas today's fastest SSDs get 40us. IIRC the DIMM version of Optane was <1us, literally an order of magnitude faster!

I'd expect PCIe to contribute ~3us latency and interrupt/context-switch/kernel to contribute ~5us. You could eliminate the latter, but 3us is still kinda slow.

An order of magnitude faster, but still a few cache hierarchies away from where the actual computing is happening.

But the real dealbreaker was that there are also similar indirections in the other direction. It's just so very, very common in modern architectures to not permanently store data on the first drive it hits. That first level of persistence is usually just some queue log where the write takes a short breath, safe from power failure for the first time, before getting passed on to some more distributed persistence story. Optane wouldn't change this general setup at all, because those layers exist for more reasons than just the power failure scenario. Optane might change implementation details of that first layer, but it wouldn't question the layers. Because optane is only answer to power failure, not to the data center burning down.

Optane is either the fastest SSD or the slowest RAM. If it is doing the job of RAM it might hurt performance, not help it. It’s not clear adding another level to the cache is really going to help.

For consumers notebook, Optane could be fast swap drive. It's pity manufacturers didn't get this done. Swap should be just fast to make application switch fast enough. Optane could do it.

Far too specific, plenty of work loads where that wouldn't make a difference at all. And of those remaining, chances are that a bump to the next main memory price tier would completely change things, more so than optane swap.

Where I'd expect the technology to really shine is in flash controllers: give that thing a generous serving of optane cache to play with in its fancy wear leveling rites, power-loss-persistent and sufficiently closely integrated (same board) to allow software to confidently assume "committed" without the flash even touched if the write happens within a tight working set (you might want to set certain file system parameters up for "it's ok to write the same address many times"). You could even include the optane in the advertised capacity! And the swap space use case would be included without even trying.

They kinda did that with Optane+flash 1TB M.2 consumer drives like the H10 and H20 series. Unfortunately they never merged the controllers so each one only gets 2 PCIe lanes and you need Intel's software (which is only compatible with Intel CPUs because why not) to do software RAID (which has a terrible reputation on Windows for losing data)

It achieved 14us latency (by acking writes before they hit flash) which is extremely good (better than today's fastest SSDs) but the 32G Optane cache doesn't go very far when most enthusiast consumers are gamers that blow the cache every time they download an update. A lotta games are also larger than 32G so if you use more a few large programs, the cache just isn't large enough. So you end up with a product that's kinda expensive, but still not quite good enough to take the performance crown.

Wow, so close, yet still a total failure. So the data would have to hit PCIe when copying/moving between optane and flash?

Yep! With the older CPUs it was released alongside, I believe it was even worse: The data would go through PCIe into the CPU cache, then into DRAM, and then back down PCIe to flash while wasting a few CPU cycles along the way. It looked terrible on benchmarks (literally half the MB/s compared to cheaper flash SSDs that used all 4 lanes) but wasn't too bad in practice since NVME drives can do many requests in parallel.

The real issue was size. I've got 100M CPU cache (5800x3d), 24G VRAM (3090), 64G DRAM, and 2TB flash. I'd need at least 128G of Optane for it to make sense in my cache hierarchy and improve game load times. I could get the $3000 data center Optane SSDs, but that's kinda hard to justify when it's as expensive as my entire PC.

I mean... with modern memory compression and SSD speeds, is this really neccesary? If you've got an NVMe SSD, you've most likely got enough bandwidth to never notice the swap kicking in (especially with a generous swappiness value). Sure there might be a couple workloads that could benefit from that kind of upgrade, but I think most people building memory-optimized rigs would rather just buy more memory, since it goes in the same DIMM anyways.

I mentioned consumers notebooks as main target for such optimization. A lot of "memory" for apps and real hibernation are main profits.

They did do PCIe Optane (e.g. https://www.intel.com/content/www/us/en/products/sku/129833/...)

Sure, but looks like that was NVMe, not just a big bunch of bytes, so whatever magical future was enabled by having a big bunch of non-volatile bytes wasn't enabled by this. (unless there was a way to change it into big bunch of bytes mode?)

My guess is that to get low latency you have to be interacting with a memory controller that expects low latency (like dram) I don’t know how many cycles it would take as a difference, but my impression is that pcie trades throughput for latency compared to dram

IIRC permissible lane skew in PCIe is like 30 ns, which means that the latency due to deskewing alone in PCIe already approaches main memory latency.

Probably because the way they did it allowed it to participate in the caching hierarchy with MESI and all the rest of it.

[Article author here]

FWIW I tried to explain why that was the important aspect of the tech in the article.

I also covered it in some depth in my most recent FOSDEM talk, which is linked in the article. As is the script if you don't have time for the video.

I guess it's because DIMM is physically closer to CPU than PCIe, so you get better bandwidth thanks to that (think that the data travels at the speed of light - the closer the faster).

Reply on news.ycombinator.com