Hacker News Reader

Comment by rwmj | Hacker News Reader

Coroutines are fun, but in real code please consider using actual threads. Modern processors have many cores, but coroutines will (often) only use a single core.

Edit to add: This is a real world problem too. Until recently qemu, which extensively uses coroutines, would put a lot of its block device I/O through a single thread. This caused some performance issues. Kevin Wolf and others have spent years of effort fixing this so modern qemu will use many threads for I/O (this work will appear in RHEL 9.4).

View on HN

The only connection between threads and coroutines is that some single-threaded language runtimes only have coroutines, so you might occasionally use them where threads would be a better choice.

Coroutines are a way of structuring single-threaded execution, and a useful one. The example in the Fine Article of a producer-consumer pattern is a good one, attaching a stream to a parser isn't a parallel algorithm so threads are useless for writing it.

Naturally, using a single-threaded paradigm for work which could be performed in parallel is inefficient, but coroutines aren't a poor man's parallelism, they're a control structure which functions on its own terms. They can be combined productively with threads, such as using an event loop in a web server to thread (as in needle) coroutines through various blocking events with a dispatcher, and the runtime can spin up a thread per core to parallelize this, which reduces per-thread coordination to checking the depth of each thread's work queue and farming the request to the least congested one.

Bob Nystrom makes this argument best, I think, in his two-parter on loops and iteration[1,2]. Looping over data structures is of course only one example of how one can apply coroutines, but a very established one. The canonical problem requiring coroutines[3] is also essentially about doing that.

Or for those who want something different there’s the elevator (and elevator-userbase) simulation from TAoCP volume 1, also an essentially concurrent problem with little to no parallelism or I/O to it.

[1] https://journal.stuffwithstuff.com/2013/01/13/iteration-insi...

[2] https://journal.stuffwithstuff.com/2013/02/24/iteration-insi...

[3] https://wiki.c2.com/?SameFringeProblem

> attaching a stream to a parser isn't a parallel algorithm so threads are useless for writing it.

Couldn't it be done in 2 threads? The output of the decompressor thread feeds to the input of the parser thread.

It could be, but given the sometimes astonishing costs of the—effectively—network protocol we know as cache coherency (thousands of cycles if you’re not careful), it’d be a giant waste in many of the cases where stackless coroutines would be perfectly appropriate.

> coroutines will (often) only use a single core

That's generally the desired behavior. If you have decoupled, parallel workloads they're going to naturally be working on disjoint data. The idea behind coroutines is that you have some kind of local workload with synchronous data that, for whatever reason, is easiest to express "inside out" with a function that gets to loop over something and "push" the results to its abstracted consumer whose code lives somewhere else, vs. the natural functional paradigm where the inner loop is a conceptual "pull" controlled by the caller.

Thank you for eloquently expressing an observation I probably should have learned years ago.

There's often a sweet spot to be had in mixing threads and coroutines, where you have a coroutine scheduler instance per thread, and a thread created per core.

Then rarely, if ever, migrate coroutines across schedulers, and rarely, if ever, share data between coroutines on different schedulers.

Coroutines can enable an ergonomic concurrent programming style while avoiding the need for any locking at all via cooperative scheduling. You generally end up with higher scheduling latencies, but potentially quite high throughput by removing any need for atomics/locking overheads, and no timer constantly interrupting execution for preemptive scheduling.

Right, that's what qemu has ended up with.

> please consider using actual threads.

Bad advice in general.

Why would you run separate thread if you only want is to iterate over nodes in a tree (as an example of non flat collection).

It’s never bad advice to consider something.

To the contrary, consideration takes time, and rules of thumb are valuable to mitigate overthinking.

"Consider using threads" is only safe advice if the person doing the considering knows how to dealt with the (usually unwanted) non-determinism threads introduce.

Only a small fraction do, but threads look so simple on the surface the rest don't realise they are walking into a mine field.

No, it's frequently bad advice to consider something. See eg https://www.xkcd.com/1445/.

Real world and toy examples are very different. The example isn't like what people are using coroutines for in the real world. I'd urge you to look at how coroutines are used for inversion of control (quite correctly) in qemu.

> Coroutines are fun, but in real code please consider using actual threads.

Coroutines are lightweight and trivial to synchronize. They are perfect for small bits of incremental computation, like iterators and tokenizers. Maybe you're thinking of green threads?

I agree for typical async IO code which needs to wait for external events (like an IO operation to finish), but sometimes in other situations threads are not an option because there would be too much synchronization required.

For instance in my emulators, the CPU emulation is a switch-case state machine which is very similar to the coroutine approach described in the article, trying to move this idea to threading would require a synchronization between multiple threads on each emulator clock cycle which is somewhere between a few dozen and a few hundred host CPU clock cycles. That's not realistic, at least for emulating typical 8- and 16-bit home computers. For emulating 'modern systems' where the hardware components are not as tightly coupled as in old-school 8- and 16-bit machines, threading makes more sense though.

See here to get an idea how that CPU emulation works (only the first few sections are needed to understand the concept): https://floooh.github.io/2021/12/17/cycle-stepped-z80.html

Coroutines are fun, but in real code please consider using actual threads. Modern processors have many cores, but coroutines will (often) only use a single core.

Threads and coroutines have different purposes. Coroutines are more about logical structure.

That seems like an orthogonal concern to structuring control flow, though it is much more difficult if you intend to use coroutines across multiple threads. There's nothing stopping you from using both threading and coroutines.

Coroutines are good for modelling concurrency which is different from parallelism. Concurrency is useful for abstraction and expressiveness. Parallelism is useful for making your code run faster by running parts of it in parallel on multiple cores. You could make concurrent programs run faster on multiple cores by distributing the coroutines which don't share state on multiple working threads in a thread pool, thus mixing concurrency and parallelism...but they are still two different things with different purposes.

Not just that, but the scaling problems with threads are usually massively overstated. It's true that thread switching has quite a bit more overhead, but it's been optimized a lot since the bad old days of 15+ years ago. (Plus, unless you're using a massive number of threads it's very unlikely that thread switching is going to be your bottleneck.)

Unless you're on QNX 7 of course...

Something something exception that proves the rule?

I'm not familiar with QNX other than knowing that it is a an RTOS... which I imagine imposes some constraints that complicate things quite a lot?

It runs in millions of cars and other embedded systems, not that niche tbh. Your "exception" is the rule for a lot of engineers.

But yes, they took pride that process switching is as efficient as thread switching which is a slick way of selling that they never optimized threads to be better, unlike Linux. Until QNX8 there is also the "big kernel lock"...

> It runs in millions of cars and other embedded systems, not that niche tbh.

Linux (in some mutilated form) runs on billions of devices. Hell, even JavaME does; or at least did, not sure what current status is. That's not really a worthwhile comparison.

My 'niche' mention was in relation to number of devs who'd be familiar with it.

> Your "exception" is the rule for a lot of engineers.

Now, that's a fair comparison... but I wager "a lot" is still not nearly the number of engineers/devs that work with e.g. Linux. That's not to cast any aspersions, of course. After all it's often not really a choice of the engineer's part.

Not sure how we got to this tangent, tbh.

This is out of the question for real time apps. Co-routines are an elegant solution to implement cooperative multitasking in such cases.

Threads with implicitly shared memory are more or less impossible to use safely, particularly in a language like C. Coroutines for concurrency, and multiprocessing with explicitly shared memory for parallelism, is a better approach.

Reply on news.ycombinator.com