Comment by mananaysiempre | Hacker News Reader

Comment by mananaysiempre | Hacker News Reader

Parent Full thread

> I/O is inherently very slow compared to computation.

Not anymore, no. Your SSD, before any caching, does gigabytes per second of sequential reads. For any bytewise processing, except the most trivial of tasks, you’ll struggle to get above a few hundred megabytes per second with scalar (native) code. To actually keep up with a modern SSD, you’ll virtually always have to hand-write SIMD loops, minimize the number of syscalls with tools like io_uring, or possibly be smart about distributing tasks across cores without ruining the access pattern.

For instance, simdjson is famously fast but I don’t believe it can keep up with say a high-end PCIe Gen 4 SSD like a Samsung 990 PRO, let alone the latest-and-greatest (and, literally, hottest) Gen 5 stuff. And I know of no Unicode normalizer that would be able to do a gigabyte per second on general inputs (not ASCII, not Latin-1) simply because the latency for dependent lookup table accesses is absolute murder.

Your analysis is correct if and only if the data is on the same machine as the calculations. If the data comes from another machine, it comes at network speed. If it comes from the internet, it comes at non-local network speed. That's very different from SSD speed.

No, you're comparing apples and oranges. All an SSD sequential read is doing is copying data from one place to another. So you should be comparing SSD bandwidth to memory bandwidth, not SSD bandwidth to (time it takes to execute some arbitrary algorithm). Or you should be comparing SSD bandwidth when performing millions of tiny random non-sequential reads and writes, to the algorithm time.

What your comment demonstrates is that it is possible in some cases for I/O to be fast enough to not be a performance bottleneck for certain kinds of programs. But not that I/O is not slow.

Muratori et al. like comparing speeds to (single-core) memory bandwidth (dozens of GB/s) and that’s a reasonable upper bound, but generally it seems to me that, unless you operate on huge elements and don’t do very much with them, you won’t get within an order of magnitude of it. Even if you think about RAM exclusively, the headline numbers are for sequential reads and things will slow down dramatically if you actually perform random accesses (IIRC, DDR5 is about as slow as DDR4 there in terms of physical time units, so much slower in terms of bus cycles). Meanwhile, in a real situation, you’re going to be bound by compute long before that.

And I think you’re being unfair labelling my couple of examples “some arbitrary algorithm[s]”: my choice was indeed arbitrary, but it’s also immaterial. The general setup would be that you’re processing elements in a loop and that your iterations are serialized (as they usually more or less are before you get around to optimization). A loop body of even three lines of C is likely to have a latency of 5–10 cycles or so, and you’re running on a core clocked somewhere from 5 GHz (desktop) to half that (server). So the best you should expect is ~500 MB/s if your elements are bytes, ~2 GB/s if they’re 32-bit integers, etc. For very simple tasks (that are also somehow not susceptible to vectorization), it is possible to not lose this order of magnitude and get down to almost 1 cycle/element in scalar code, but that requires heroic effort[1].

[1] https://github.com/powturbo/Turbo-Histogram

Classic latency vs throughput problem. 10s of GB/s of disk bandwidth doesn't help when my problem is serialized durable writes.

Reply on news.ycombinator.com