Hacker News Reader

Comment by andersa | Hacker News Reader

> Fetching data from disk is something that everyone strives to avoid since it takes approximately 10x more time than using it from main memory.

Did they mean 10 thousand times? Or is the in-memory version that inefficient?

View on HN

I think they’re playing it safe with their points of comparison. DDR5 supports 64,000 MB/s per channel and 4x NVMe PCIe5 SSDs support 10,000 MB/s. Depending on how many memory channels and what RAID you use, I think 10x improvement over high performance storage is unimpeachable. Memory latency being better than SSD latency will really benefit memory depending on workload, but I don’t think you can just throw one number out there to represent that.

Now if you’re comparing to spinning rust, memory is definitely going to blow it away, but commodity hardware isn’t running tens or hundreds of TBs of memory. Memory to SSD comparisons seem right.

its also may be limited on iops/s. you _can_ really just work with latency, but you need to consider the pipeline depth.

For example the latest AMD Genoa has 12 memory channels per socket, at dual socket and with enough DIMMs that's a 75x speed advantage even if you compare with RAID0 of high perfomance NVMe.

> even if you compare with RAID0 of high perfomance NVMe.

Do you mean a RAID0 of just two or four NVMe SSDs? It's absolutely ridiculous to count aggregate DRAM bandwidth across two CPU sockets and not do the same for PCIe lanes. A fair comparison is that Genoa has about twice the DRAM bandwidth as it has PCIe bandwidth, though in a fully-loaded database server some portion of the PCIe bandwidth will be used for networking rather than storage.

FWIW I think you have the aggregation backwards, dual socket Genoa would have 24 channels of DRAM but sacrifice some of the PCIe lanes for the interconnect. Your numbers work out right though as single socket is actually about 1:1 in RAM vs PCIe bandwidth so dual socket would still come out roughly 2:1.

I think 10x is a fair rough number though, depending on your access pattern.

Dual-socket Genoa would be 24 channels at DDR5-4800 (38.4 GB/s) for a total of ~921.6 GB/s. Typical PCIe configurations are 64 or 80 lanes per socket for dual-socket, so 128 or 160 lanes total, at PCIe 5.0 speed that's ~504 GB/s for 128 lanes total or ~630 GB/s for 160 lane configurations.

Single-socket Genoa would be 12 channels of DRAM (~460.8 GB/s) and 128 lanes of PCIe 5.0 (~504 GB/s), but none of the previous comments were specifically about single-socket Genoa and I wasn't going to silently switch from considering dual-socket in one sentence to single-socket in the next sentence.

Ah yes, I forgot about the 48 lane 2 socket interconnect mode which does allow you to still aggregate some additional lanes.

10,000 is for spinning rust. With SSDs on NVMe it's about 50x, so pick 10x or 100x to be your number.

This should have been a revolution in DB design, IMO, but it kind of hasn't been.

Recent database engine designs tend to be bandwidth bound. The difference in bandwidth between memory and modern storage hardware is much smaller than you might expect. Really taking advantage of that storage bandwidth is more difficult than memory, as it requires some top notch scheduler design.

In a similar vein, this was news to me recently:

> here’s a chart comparing the throughputs of typical memory, I/O and networking technologies used in servers in 2020 against those technologies in 2023

> Everything got faster, but the relative ratios also completely flipped

> memory located remotely across a network link can now be accessed with no penalty in throughput

The graphs demonstrate it very clearly: https://blog.enfabrica.net/the-next-step-in-high-performance...

Yeah, I think this is a brilliant thing in part because it makes disaggregation of hardware infrastructure plausible for high-performance, high-scale data-intensive workloads. It also turns some deeply embedded assumptions about software design and architecture for things like databases completely on their head. This shift in hardware balance is probably the biggest change to fundamental assumptions in database architecture since SSDs started to become a thing, and most people haven't internalized the implications yet.

It is an fun time to be working in high-scale data systems.

10x is reasonable-ish for bandwidth not for latency

Based on this resource https://gist.github.com/hellerbarde/2843375 1MB sequential read from SSD just 4x slower than the same read from main memory. For random reads main memory faster of course. I believe author meant some average value.

I guess when you compare pure performance of hardware, 10x is very optimistic from the perspective of disk. Probably the author based this number on some some specific application/database context / bias in the measurements. But yea, pure hardware difference might be hugely different compare to that number.

Yes, even with SSDs it seems like 10x is very optimistic. It should be several orders of magnitude.

Yes we used this number based on some of our workloads and it is definitely lower bound... As I am thinking now we should have moved this lower bound a bit up yes

Reply on news.ycombinator.com