Hacker News Reader

Comment by zbentley | Hacker News Reader

> If you're I/O bound, that either means the problem doesn't require much computation - which is possible but fairly rare

This is backwards. I bet that by count, many more programs are written in domains where they're necessarily IO bound than the inverse. Anything that uses the network for its core functionality, anything reliant on a datasource whose aggregate contents are O(memory)+ size, or anything reliant on slow peripherals (lots of embedded software) are in this class.

Scientific simulations, HFT algorithms, video games, LLMs, etc.--the stuff in the other class--aren't inconsequential, but they're dwarfed in number by the class of software that spends 99+% of its time waiting for IO. Hell, entire programming languages (node.js) have been created in response to that proportion.

View on HN

Bottleneck is a misleading word here. Yes the network is slow. But you can still save 200ms in response time by working on your CPU.

Neither I nor GP used that word.

You're not wrong in that there are some programs that could be doing more work in CPU/memory while waiting for IO, but those, too, are dwarfed by the number of programs that can't really do anything meaningful until IO completes. Anything that RPCs or IPCs data is generally going to be waiting for a complete RPC IO to finish before doing compute (and even the most granular RPC protocols tend to communicate in pretty big, slow chunks to maximize throughput). Lots of software waiting on local hardware (e.g. storage) is similarly doing IOs in pretty big pieces--maybe page-sized, or disk-block-sized, or file-sized--and can't do much meaningful CPU work until that's done. In embedded, it often behooves programs to get as much IO-sourced data read or written as is possible with available resources before switching back to CPU work--doing this increases throughput on slow hardware, and can also improve power efficiency.

Put another way: modelling IO as a stream with something like io_uring won't save the end user much latency if the completions inside the ring wait for slow, batched IO, or if application code needs to see completed transactions before proceeding.

Latency, throughput, power, hardware cost--those often trade off, and there's no free lunch.

Modern networks can transfer data more quickly than CPUs can process it. Starting at 100Gbps.

That’s completely backwards by orders of magnitude. Modern CPUs outrun L1 cache speed by a lot, L1 cache speed is on order of 1000-5000 GB/s, a GB/s is real bits used, is 8x of Gbps, and network speeds include significant overhead bits for frame packing and other data unused for the actual payloads.

And a GPU does orders of magnitude past this.

Networks are not even close.

https://chipsandcheese.com/p/a-new-year-and-new-tests-gpu-l1...

Is that the caliber of hardware I can expect to be working with on a cheap vps?

What in tarnation?

Reply on news.ycombinator.com