Marvell Announces First PCIe 5.0 NVMe SSD Controllers: Up to 14 GB/S

ROFL. Thanks. Keep thinking about the good old days.

I am sure if you go back a few years you can find systems that have the same or less memory bandwidth than this has now, but they have both been moving forward enough that SSDs are still not close. That being said, between bandwidth and random access times, swapping out to disk backed virtual memory is very pragmatic and isn't the death of performance it used to be.

These 2U or 4U cases, or tower desktop cases, were designed to efficiently remove heat from storage devices, as well as the 3000W++ that the dual socket CPU and 8way GPUs will pull.

10W is tiny for desktop and server. Barely a factor in overall cooling plans.

It seems to be kinda in between RAM and flash spec wise.

As the platter density increased, the head could glide over more data at the same spindle speed and you would get increasingly higher sequential speed. But you would not get much better latency - physically moving the head to a different track could not be done significantly faster. With each new generation, you would get only marginally better speed for many workloads that users actually care about. The latest HDDs could saturate SATA bus with sequential reads, but would still dip into KB territory (not MB, let alone GB) for sufficiently random-access workloads.

SSDs are similar in a sense that they can be massively parallelized for the increased throughput, but the latency of an individual cell is much harder to improve. Benchmarks will saturate the I/O queue and reach astronomical numbers by reading many cells in parallel, but for most desktop users, the queue depth of 1 (and the individual cell latency) is probably more relevant. That's why a lowly SATA SSD will be only marginally slower than the newest NVMe SSD for booting an OS or loading a game.

I've got a cheap SSD that will change your mind. It has amazingly poor performance though, I agree with the general concept that while quantitative differences can be measured, there's not a qualitative difference between competent SSDs.

That reminds me of non-volatile SRAM modules you can get by Microchip. How it works is that its just normal SRAM, but each cell is connected to a matching flash cell and when the mdoules voltage drops below a threshold, it writes the SRAM contents to flash in parallel. When the voltage is restored, it does the reverse. The user-observable effect is that you have normal SRAM (speed and it doesn't wear out) but its non-volatile. In reality, its a clever trick, but since the flash is only written when the module is powered off (not per-write to the module), you get a lot more use before it wears out.

Of course these modules are tiny (a few KB), but I still thought it was pretty interesting!

Those graphs cut off the essentially vertical latency spike that results from enqueuing requests faster than the sustained rate at which the drive can serve them. For a different view in terms of queue depth rather than throughput, there are some relevant graphs from an older review that predates io_uring: https://www.anandtech.com/show/11930/intel-optane-ssd-dc-p48...

Generally speaking, latency starts increasing long before you've reached a drive's throughput limit. Some of this is inevitable, because you have a relatively small number of channels (eg. 8) and dies to access in parallel. Once you're up to the throughput range where you have dozens of requests in flight at a time, you'll have constant collisions where multiple requests want to read from the same plane/die/channel at once, and some of those requests have to be delayed. But that's mostly about contention and link utilization between the SSD controller and the NAND flash itself. The PCIe link is pretty good about handling transactions with consistently low latency even when on average it's mostly busy.

There was another post I saw recently comparing the increase in disk SIZE over the last 30 years vs the increase in disk SPEED vs LATENCY (so size in GB, speed in GB/s, latency in IOPS) - and how size increases far outstripped speed which outstripped latency, though all had improved.

Found it! The key is IOPS/GB as a metric.

https://brooker.co.za/blog/2021/03/25/latency-bandwidth.html

https://louwrentius.com/understanding-storage-performance-io...

https://louwrentius.com/understanding-iops-latency-and-stora...

Related: Notice how the public cloud marketing material tends to focus on scalability over other metrics? That's because scaling horizontally for them is easy: They just plop down more "stamps" -- a set of clusters and controllers that is their unit of scale. Need 1,000 more servers in US East? Plop down 10 more stamps of 100 servers. Easy!

Except of course this involves an awful lot of networking, with long cable runs and many hops and virtualisation layers. The end result is that you can't get anywhere near the underlying storage latency.

Azure's Premium SSD has "write flush" latencies of about 4 milliseconds according to my measurements, which is easily 100x slower than what my laptop can do with a now very outdated Samsung NVMe SSD.

Notice that if you go to their marketing page, they talk about "low latency" and "lowest latency", but they have no numbers? Meanwhile the MB/s and IOPS is stated with numbers: https://azure.microsoft.com/en-us/pricing/details/managed-di...

My logic always worked like this: if IOPS is input / output per second, surely that means 10,000 IOPS of, say, the usual random 4kb file write at QD1, then that means I can get 10,000 of those in one second. So, a higher number must be better.

I had no idea that IOPS needs to take latency into account. After having done some reading I am still somewhat confused as to how this really works, but for future reference I will use latency as a measurement of performance again, as that is indeed what I thought IOPS was.