Whoa there... let's not compare devices with 20+ GB/s and latencies in nanosecond ranges which translate to half a dozen giga-ops per second (aka RAM) with any kind of flash-based storage just yet.
If you want to know the upper bound on your per-core RAM bandwidth:
64 bytes (the size of a cache line) * 10 slots (in a CPU core's LFB or line fill buffer) / 100ns (the typical cost of a cache miss) * 1000000 * 1000 (to convert ns to ms to seconds) = 6400000000 bytes per second = 5.96 GiB per second RAM bandwidth per core
There's no escaping that upper bound per core.
Nanosecond RAM latencies don't help much when you're capped by the line fill buffer and queuing delay kicks in spiking your cache miss latencies. You can only fetch 10 lines at a time per core and when you exceed your 5.96 GiB per second budget your access times increase.
If you compare with NVMe SSD throughput plus Direct I/O plus io_uring, around 32 GIB per second and divide that by 10 according to the difference in access latencies, then I think the author is about right on target. The point they are making is valid: it's the same order of magnitude.
I've since bought all the pieces but the CPU; they are all sold out. So I got a 6 core 3600XT in the interim. I bought fairly high binned RAM and overclocked it to 3600Mhz, and was surprised to cap out at about 36GB/s throughput. Your 6GiB/s per core explanation checks out for me!
could you point me to a little reading material on this? I know what an LFB is, more or less, but what queueing delay, an dhow does that relate to cache misses? Thanks.
It means if a system can only do X of something per second, then if you push the system past that, new arriving stuff has to wait on existing work in the queue, and things take longer than if the queue was empty. You can think of it like a traffic jam and it applies to most systems.
For example, our local radio station here in Cape Town loves to talk about "queuing traffic" when they do the 8am traffic report, and I always think of Little's law.
Bufferbloat is another example of queueing delay, e.g. where you fill the buffer of your network router say with a large Gmail attachment upload and spike the network ping times for everyone else sharing the same WiFi.
Here is where I got the per-core bandwidth calculation from: https://www.eidos.ic.i.u-tokyo.ac.jp/~tau/lecture/parallel_d...
> In the DRAM region we’re actually seeing a large change in behaviour of the new microarchitecture, with vastly improved load bandwidth from a single core, increasing from 14.8GB/S to 21GB/s
Yeah, that's odd. But the article's really about cache, so maybe it's a mistake. Next para says
> More importantly, memory copies between cache lines and memory read-writes within a cache line have respectively improved from 14.8GB/s and 28GB/s to 20GB/s and 34.5GB/s.
so it looks like it's talking about cache not ram but... shrug
Flash is special in that its latency is still considerably higher than that of DRAM, but its throughput can get reasonably close once you have more than a handful of SSDs in your system (or if you're willing to compare against the DRAM bandwidth of a decade-old PC). Extracting the full throughput from a flash SSD despite the higher-than-DRAM latency is what requires more suitable APIs (if you're doing random IO; sequential IO performance is easy).
My understanding of distributed computing history is that the last time network>local storage happened was in the 80’s, and most of the rest of the history of computing, moving the data physically closer to the point of usage has always been faster.
Just as then, we’ve taken a pronounced software architecture detour. This one has lasted much longer, but it can’t really last forever. With this new generation of storage, we’ll probably see a lot of people trotting out 90’s era system designs as if they are new ideas rather than just regression to the mean.
Same as it ever was.
[0] - https://www.intel.com/content/www/us/en/architecture-and-tec...
The thing is, up until this point, for the entire history of computers, storage was so relatively slow compared to memory and the CPU that drivers could be quite simple, chuck requests and completions into queues managed by simple locking, and the fraction of time that requests spent inside the driver would still be negligible compared to the time they spent waiting for the disks. If you could theoretically make your driver infinitely fast, this would only amount to maybe a 1% speedup. So there was no need to spend a lot of time thinking about how to make the driver super efficient. Until suddenly there was.