The Changing Relationship Between Memory, I/O, and Network Bandwidth

The other element which appears unsaid is that in a typical datacentre, your bisection bandwidth is typically << (num computers * network bandwidth per computer). Or in other words, even if your computation is bandwidth and not latency starved, you're not obviously going to be able to do a gigantic data shuffle quickly unless it's within a rack. This is to say, I'm not sure _just how much_ this would affect your overall target system architecture at present. Once you're switching at a couple of terabits things are likely quite different in those terms.

The other element which is a bit scary is that it's fairly rare these days for mass market companies to index deeply on tech which isn't available in public clouds, so until AWS supports this sort of thing, it's unlikely that many folks will target it. You can kind of see this with the Optane PDIMMs - they looked absolutely fantastic, but given you couldn't get them on any AWS instance there wasn't much point actually trying to use them outside of very specific applications - as a software engineer this hardware lets me build my software very differently and in a simpler way, but how can I possibly risk architecting based on that if it then cannot support a customer's cloud migration?

Obviously v different in HPC contexts.

And it should always be said - latency is very different between these. Memory latency still measured in nanoseconds, PCIe latency still measured in 10s of microseconds, about 3 orders of magnitude difference.

> Also they used DDR5 4800 in their comparison instead of DDR5 6400.

Are there server processors on the market yet that support DDR5-6400? Intel Sapphire Rapids and AMD Genoa only support DDR5-4800. Faster DIMMs are available (especially in the consumer market segments) but it seems quite reasonable to base this kind of analysis on what's actually officially supported and widely available.

Check that you have installed one module, or two modules, one per each memory channel. If two modules are installed in wrong slots, they can use the same channel and this can result in a slowdown.

One a 13900 core, with default clock, should be able to read at max. rate 89.2/2 = 44.8 GB/s (small gigabytes = 10^9). One DDR5 module running at 6000 MT/s has maximum throughput 48 GB/s. Granting some unknown slowdowns due to error corrections and other things I would be expecting to see at least above 40 GB/s. With two RAM modules and two or more cores working, one should be able to get above 80GB/s.

Perhaps memtest does something that limits its memory performance (maybe it uses old less-efficient instructions for reading memory instead of SSE/AVX) or maybe it makes a factor 2 error in calculation of the speed.