Why the C++ committee didn't go with 'Stackful coroutines AKA Fibers' http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136...
Raymond Chen's thoughts on where they are still useful (hint they really aren't): https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=10...
Though I'm not impressed with the thread-local storage errors argument, as it's just an argument against low quality implementations.
In a good implemention of fibers or green threads, TLS APIs should be replaced by fiber-local storage using the same APIs and fast access method, and FLS doesn't have those errors.
Perhaps there's another argument against FLS, but the TLS errors argument should have considered FLS rather than leaving it implied that there's no solution.
I do not think that FLS (at least by default) is the right solution as it would consume a lot of memory, increase the switch overhead and add complexity for little benefit.
Also, regarding shared libraries, how can compiler cache the TLS address, since the compiler versions that compiled the shared library may be different that the version which is compiling the executable that links to it, and thus may have different caching behaviours?
thread_local x;
void foo() {
x = 10; // x in thread 1
foo(); // moves this coroutine to thread 2
x = 20; // might erroneously write to x of thread 1
edit: also yes, forcing inlining off is not guaranteed to disable cross procedure optimizations. At the very least you have to disable function cloning. In fact I see that GCC now has attribute noipa which is probably a better solution.
LTO is a thing though, putting it in a separate translation unit doesn't help when static linking.
TLS has its own issues, but in every situation where this optimisation is fine with TLS, it's fine with FLS too.¹
If you call a function in a fibers / stackful coroutines environment, and the function has a context switch inside (say it does some blocking I/O), by the time the function returns the original context is valid again.
> I do not think that FLS (at least by default) is the right solution as it would consume a lot of memory,
Depends on what you store in FLS. As with TLS, there are situations when it's beneficial for memory, compared with storing replicated copies of data all over the place in arguments and data structures, and situations when it isn't beneficial.
In some it will consume less memory than stackless coroutines, which store copies of the same data in async structures anyway.
> increase the switch overhead
If we're still discussing the article's case that stackless coroutines are better, FLS switch overhead is usually lower than stackless/async overhead.
In the stackless/async version you have compiler-generated loads and stores to context-specific structs that need to be executed every time something context switches for an await.
FLS overhead is updating a single pointer. Even that may be free, because you're going to update some kind of context structure pointer anyway. Even in the base case, stackless will update an equivalent pointer at absolute minimum, but often more.
This is really the same argument for TLS being sometimes useful in threaded programs specifically for performance, as opposed to explicitly passing around copies of pointers and storing them in data structures all over the place. (Of course there are non-performance reasons why people choose TLS or avoid it too.)
> add complexity for little benefit
The benefit is faster programs, and a simpler programming model. You may reasonably disagree :-)
But consider, the Linux kernel is basically all fibers with FLS, and that's a top performer :-)
With regard to programming model: In Linux, it was found to be very much simpler to use fibers (called "tasks" in Linux) than to build an async I/O state machine for filesystems. This is why Linux didn't get useful async I/O in filesystems for a long time, because the state machine version of filesystem operations, though people attempted, was just far too complex in practice to implement for all corner cases, so its async properties were unreliable.
--
¹ At least no problems that are specific to FLS. If you're pointing out there are issues with TLS (without fibers) due to shared library loading/unloading or lazy memory instantiation, that's an issue but it does not support the article's implication that TLS is ok while FLS is not.
The alternative to FLS is not storing duplicated data but use TLS and making sure that the compiler does not miscompile your code, either because you are extremely careful or because the compiler provides an option for it. Unfortunately compiler writers do not really see a future in stackful coroutines and are not really willing to provide a 'couroutine-safe' TLS mode.
Regarding the switch overhead, the fastest switch routine I have implemented does not store or load anything from any context-specific struct, only swaps a couple of registers, so an extra load might have a cost (trivial for async io, but can be noticeable for generators). Also, the way TLS is implemented on x86-64 for example is via a segment register which would need to be changed at every coroutine switch; updating it is significantly more expensive compared to a normal register [1].
Finally, as far as I know, the kernel does not use thread local/fiber local storage, instead heavily relies on per-cpu data. I advocate the same solution for coroutines, where, if you use a 1:1 mapping from OS threads to cores, the per-cpu data would correspond to the TLS of the underlying OS thread.
[1] Agner reports a reverse throughput of ~13 cycles on Neahalem. I think he stopped testing on newer architectures as it is a rarely used instruction; it needs to be expensive because it is not just a register load but it also has to update shadow limit registers. It probably has second order effects on the TLB as well.
It does use FLS. Everywhere you see "current->" or "current_thread_info()" in the Linux kernel, it's FLS, not per-cpu.
There's a lot of those, some of them wrapped in other macros and inlines. "current" refers to the current task, which is the kernel's name for what the article calls fibers.
Per-cpu data is used in the kernel as well, and I agree with you about using the analogous thing for coroutines which is TLS for certain things. Indeed, a FLS implementation almost certainly uses TLS for its root-of-data pointer and keeping tracking of the current thread's coroutine queue.
> Agner reports a reverse throughput of ~13 cycles on Neahalem. [...] it is not just a register load but it also has to update shadow limit registers
You're thinking of a segment register load, but these days we would use the WRFSBASE or WRGSBASE instructions, which don't update the segment descriptor (shadow limit etc) and don't use the GDT/LDT.
Windows has supported this for userspace scheduling for a long time (but Windows has good fiber support with native FLS anyway), Java coroutines want to use it for performance on Linux, and Linux kernel looks like it will enable this soon; the CPUs have supported it for years.
When those instructions aren't available, since the "current coroutine" is almost certainly a TLS variable itself that must be updated on context switch, use that as the base for FLS too. Then FLS access is one extra level of pointer indirection on access, but still hoisted and cached. For portable coroutines-and-threads code, use a portable TLS variable or pthread_getspecific() for this.