The Downsides of C++ Coroutines
reductor.dev
reductor.dev
> Finally at the end of the function the stack space initially reserved get’s reset to where it was initially when the function first call happens then returns to the caller.
This post could use some editing. I'm having to reread each paragraph several times to figure out its intended meaning. Most sentences are separate paragraphs with careless mistakes that make me feel the author was being chased by someone when writing them and couldn't take a breath.
That being said, it is a great article. C++ and coroutines is a story that has been going on for a long time, and the result surprised me. In a bad way.
One bit me right from the start. I copied out an example and it crashed, and it turned out (after hours of searching, reading - the compiler and sanitisers sure weren’t any help) that the problem was that I’d inadvertently made a parameter const& (force of habit) and bound a temporary to it.
My answer to this is simply that I choose not to use coroutines. If I can’t force a compilation failure when I do something dumb, that spooks me.
For a feature released in 2020 it has far too many footguns. Ranges was similar when it came to lifetime footguns. It’s just something that makes it hard to take seriously the claims that it is legacy code that is the reason C++ has a bad rap for safety. Coroutines and ranges are modern features that can shoot your foot off if you don’t know the implementation, which is kind of contrary to the point of making a friendly wrapper over it all.
I agree with your conclusion of not using C++ coroutines, though. It seems like the design falls somewhere in the "worst of both worlds". I would rather either use a library that implements coroutines with the minimal amount of C and inline assembly if performance is critical, or some higher level abstraction that works well with all other language features.
We were using coroutines about ~12 years ago in embedded contexts, this was with Lua which has very good support and being a managed language avoids all the footguns here while still allowing very fast interop with native code(at least in the case of LuaJIT).
I hate to drag Rust into every C/C++ conversation but this is one area where the language really keeps you within the guardrails. Callbacks are hard to use correctly in Rust, less because of the language and more just do to the messy lifecycle aspect of them. You can side-step that with shared_ptr/Arc but then you're stepping into memory leak territory when you have a circular reference(and the atomic ref counting isn't cheap either).
C++ has always been a bit of an outlier to me for that reason.
Same problem with ranges. The footguns are remnants of abstracting something complex with a friendly interface and failing to secure it. It’s great for people who know the implementation, and it’s obvious where memory issues appear - but if you don’t know the implementation then you end up with holes in your feet.
I live eat and breathe a few deeply technical things (including c++) but am hampered by lack of proficiency at clear communication and expression. Becoming increasingly self-aware of this, seeing a fine article like this about a topic I understand, "in the wild" only underscores the importance. :)
I can feel myself automatically rewriting it, just like how typos jump out.
I wonder if that would be actually useful (a rewrite) to anyone.
That said, I think they are still very much worth it. Older asynchronous programming libraries in C++ are so verbose and so much worse than coroutines that it’s an obvious choice to use coroutines.
Also, there is another hazard that the author does not mention in this article: RAII lock wrappers. Holding a lock across suspension points is super dangerous. At best it wastes performance to leave it locked when blocked. At worst it can create deadlocks or corrupt the lock if it is released on a different thread than it was acquired.
I agree, but the other choice is to have traditional threads of execution that block. This simple strategy has delivered more successful projects than any other.
You don't even need coroutines for this to be dangerous. Holding locks over callback invocations is a pet peeve of mine in PR reviews. Callback invocations, like suspension points, can inject arbitrary operations into our code, which can easily break prior invariants, yet look innocuous for the casual reader.
We use seastar.io a thread per core framework and locks are "async" friendly in that they yield for access instead of blocking. Also embracing fully async message passing between threads simplifies the programming model a ton.
Some parts like the one about lazy coroutines seem to only be an issue because coroutines in C++ can theoretically resume on any thread. If they were restricted to the current thread by default (like in Python+Twisted I think?) then you would still be able to use them for many use cases but with less cognitive overhead.
The author seems to prefer stackful coroutines aka green threads, which are essentially just user mode threads. Handoff is implicit deep inside 'blocking' functions. I rarely see the downsides of them discussed, but they have their own problems: They are still threads so you often need locks. You don't know where a function will hand off, and you could accidentially call a really blocking function or run a long computation and ruin responsiveness. And the type of the function no longer reflects if it is blocking or not (the famous colored functions).
https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/b...
Ironically, C++'s design is heavily related to C#, as the initial proposal was done by Microsoft and shares many of the same ideas, including how to create runtime aware awaitable types.
Making code nonblocking without threads and without callbacks = happy case for async. Writing multithreaded servers focussed on throughput is a whole other can of worms, which is basically my point.
Blocking vs non-blocking can be solved with naming conventions, like Sync vs Async suffix (works well in node.js for instance)
(also getting rid of colored functions is a good thing!)
This can trick 2 parts of your program into thinking they have exclusive access to the same thing at the same time. E.g. they could both grab the same thread local or both enter the critical section of a recursive mutex.
I think the colored function thing is often thoroughly misunderstood. There is a real difference between a function that returns `string` vs. `Future<string>`. It's not arbitrary but just a matter of typing. Languages could have more syntactic sugar to bridge both worlds of course. And you can get rid of the distinction as goroutines etc. show.
But actually I wonder if it would be useful to keep some colors. Maybe you could have an effect system and mark functions as "computationally expensive" / "blocking" vs. "computationally trivial". The compiler would prevent you from calling the blocking functions from the GUI thread, but you could `async` or `go` them to another thread and resume when finished.
A coroutine is a thread of execution that can yield to another. The scheduler is thus under userspace control.
The C++ thing is syntax sugar over a control flow transform which looks kind of similar, except you have to annotate all the functions in the call tree and can't do anything that can't be desugared to the same runtime system that was there anyway.
The primary downside of C++ coroutines is then clear. They aren't coroutines, and by squatting on the name, make it borderline impossible that C++ will ever have coroutines. This is annoying as they're one of the things which really needs compiler support to do well.
I am not a C++ person, but I protest this characterization. You obviously know the categories and I suppose you know the history, but I will recount them so that everyone else understands my objection.
O.G. coroutines emerged in a world where subroutines were, by and large, not reëntrant. Function parameters, local variables, and return addresses were static, and there was no call stack in the modern sense. This is why the old timers said that coroutines were a generalization of subroutines; only the jump instruction of the call/return really needed to change.
Once call stacks became common, coroutines fit awkwardly, and people tried to adapt them in various ways.¹ The world has settled on two designs for reconciling coroutines to the call stack: thick and thin coroutines. Thin coroutines allow suspension within the body of the coroutine but not within subroutine calls; this way the size of the coroutine’s state can be known at compile time and its interaction with the stack is relatively clear. Thick coroutines (i.e., green threads) can be suspended within a subroutine call, and thus require their own slices of stack — either separate from the main stack or copied from and to it as the coroutine is suspended and resumed.
Thin coroutines are absolutely coroutines. They are truer to the original definition than thick coroutines are! They are more limited than thick coroutines, true; whether that makes them better or worse is a matter of design trade-offs. But they certainly deserve the name.
[1] Simula 67, to my understanding, treated objects as a kind of coroutine instance where function definitions in the coroutine body became methods that closed over its local variables.
The version that shipped can be done in the compiler front end. On the happy path it compiles to zero cost relative to writing the branches by hand. Machine architecture independent.
The version that didn't ship requires language runtime support. It involves allocating memory for the new stack and storing the live registers to it on yield. It's per-platform machine code, with varying overhead depending on how much control the compiler gives over calling conventions. Yield then looks a lot like a function call (and sometimes upsets branch predictors).
The full/stackful/green/thick/etc version works very like a posix thread without the pre-emptive scheduler, and needs language runtime support for exactly the same reasons that pthread_create does. They're zero cost if not used - they don't change the calling convention of other functions - but the yield usually can't be optimised out at compile time if they are used.
Naming things is indeed difficult and definitions do tend to shift over time. However the "C++ has coroutines now" feature box check doesn't bear up under scrutiny if one expects said coroutine to support the same operations that coroutines support in other languages.
Do it in a scratch refactoring, and wait a week or two before you consider merging it. And make sure you are emotionally as ready to discard as you are to land it.
But in practice I've more often seen the opposite problem, where organizations end up stuck on C++11 for a decade for no technical reason. It's good to explore the new stuff and eventually adopt what you can use.
The post mentions that it might be the caller's or the callee's responsibility to keep the object alive until the end of the coroutine. This is purely based on conventions however, and different libraries might have different conventions. If the caller is responsible for it, extra care needs to be taken whenever the function is called -- the code shown in the post seems fairly complex to me and easy to get wrong. Also, I am not sure how the callee could safely implement keeping the object alive if lazy coroutines are used: Even if the first statement in the coroutine is retaining a strong reference on the object, there might still be a time between the call and the initial resume of the coroutine where the object is destroyed. I think it would have been great to provide explicit capture lists for coroutines, similar to lambdas.
All of this gets especially confusing once you try to use lambdas together with coroutines. AFAIK, C++ lambdas are basically just structs overloading operator(). In a coroutine, only the "this" pointer of the structure is captured and the caller needs to ensure that the object is not only alive, but also at the same memory address until the end of the coroutine. This is very easy to get wrong in my experience.
They're also very useful if you've ever had to create a bare metal multitasking system. Much easier for state management than older style "while (true)" loops with a million state variables so functions can resume via a switch/case as pseudo-coroutines. (Well, easier if you don't have to implement the coroutine mechanism yourself.)
Are not stackless supposed to be more performant? In which cases? Yes I know their virality, potential heap allocations, etc.
First, stackful coroutines use the coroutine stack for everything they do. Stackless coroutines can use the normal thread stack for synchronous calls, and that stack can be shared across any number of coroutines. Per-coroutine allocation is only needed for asynchronous calls.
Second, for stackful coroutines you need to allocate the entire stack up front, and usually you have no way of knowing how much stack might be needed, so you need a conservative upper bound. Normal thread stacks have sizes in megabytes. (That doesn't necessarily correspond to actual memory consumption, since the OS will only reserve physical memory as needed, but the physical reservation for a given stack can only grow, not shrink. And even just allocating the virtual space has a cost.) Most of the time you can get away with stacks that are much smaller, only a few kilobytes, but at the cost of potentially crashing when you've consumed too much stack; it's hard to statically analyze maximum stack usage.
Stackless coroutines will, in general, only allocate memory as needed for each coroutine invocation, so not only are you wasting less memory, you don't have to worry about hitting an arbitrary limit. Allocation elision makes things more complicated since, as the blog post notes, you can end up wasting some memory, but compared to stackful coroutines it's peanuts. But they have the downside that heap allocations and deallocations are expensive; plus, splitting a "stack" of nested calls into separate heap allocations, usually far away from each other in memory, is worse for cache locality.
But, considering the accelerated releases post C++ 11, I guess I'm not surprised.
Who would have thought?