> In Rust, if you're actually doing any compute work, you're stalling out the async system.
Just use threads? Or add explicit yields.
Just use threads? Or add explicit yields.
That's so 1984 Macintosh. "Cooperative multitasking".
The usual effect of that style of programming is stuck window managers.
Go's solution is threads, just with a particularly idiosyncratic M:N implementation. Rust threads are 1:1. But they're the same concept. You can even get M:N if you really want it (nobody does, which is why mioco gets no use).
Computation needs to be scheduled to a CPU or core for parallel progress to occur. That means kernel threads.
A thread is too inefficient for the fast calls, but no thread is disastrous for the rest of the program's concurrency for the slow calls.
So you're reliant on the library, not written by you, to always choose an efficient behaviour for each call. In other words, to decide dynamically when to use a thread, and when not to use a thread. And that library is reliant on all of its transitive dependencies in the same way.
It's a nice ideal, but it's not realistic.
And it still doesn't work all that efficiently anyway. If you call a library function many times, or just many functions, and any of those calls (outside your knowledge) has some long-running part where a thread ought to be used, to do it efficiently a single thread would span your code's multiple calls, and not just last for one library call.
Essentially, to know whether to make a library call in a thread or not, and at what level to use a thread, you need to know what the behaviour of that call is going to be in advance. In practice, even when carefully optimising programs, we almost always assume one behaviour or another. Maybe measure, and assume the measured behaviour will remain similar. This doesn't have good adaptive behaviour when the assumption is wrong or out of date, so it doesn't work well for calls to things whose duration varies a lot, and especially not for large systems of calls between different modules, each of which has unpredictable timing.
I first encounted this issue in a video games engine, long before Go or Rust existed (and before async-await). NPCs usually did a tiny amount of calculation, and there where tens of thousands, called every display frame, so this had to be fast. But some of them, known only to themselves, occasionally decided to make network calls, read cached data (which might need network calls or not), or call the filesystem, or do intensive "AI" planning using a burst of CPU. To maintain the frame rate while efficiently processing all NPCs, it was necessary to detect when some deep library call from an NPC had used the NPC's "short call" time budget, and switch it over to a thread.
In that video games environment, the Go model worked. No matter how each NPC was coded and by whomever did it, an NPC might slow down if it called something slow but the game would stay smooth, and all other NPCs would continue to update at the full frame rate. The Rust model would result in the whole game frame rate stalling, which is much less acceptable. The threads-when-anything-could-be-slow model was too slow, and the use-threads-when-you-will-be-slow model gave NPC code too much power to affect the system outside the NPC.
But let's run with pre-emption, because that does apply on single cores.
No, it doesn't work out to be a global cost, assuming you mean performance (as opposed to developer cognitive cost). I have experience with this - the game engine. We don't half-ass performance in games like that, it's a central factor. We optimise for the highest performance on middle-of-the-road hardware, and measure constantly.
The Rust model is same or worse on each global performance metric that matters in that scenario. The global metrics are things like graphics throughput, tail latency and variance, interactive response, task starvation. So the "cost" of "pre-emption" ranges from zero to negative.
I will grant you that pre-emption adds shared resource locking overheads if you're comparing with an async-await model that uses only a single thread for all tasks globally, so locking is not required. (I.e. JavaScript). But that's not the Rust model.
Pre-emption has a small, positive local cost, in that you need to check when to pre-empt. But that can be made small. You don't need a kernel to do it, and you don't need to be constantly reading clocks either. The cost is a few nanoseconds per microtask, i.e. book-keeping like with any execution loop, and some periodic interrupts or signals, which you might have anyway. Alternatively you can use performance counters on some CPUs. The actual act of pre-empting is also cheap - it's just a coroutine jump with some book-keeping. But it's also comparatively rare compared with async task transitions, so the cost doesn't really matter anyway.
Regarding scheduling and how task-switching is done, async Rust and Go are fundamentally the same model (and BTW, the authors of Tokio never hide that they took a lot of inspiration from the Go runtime). The three big differences are the following:
- Go's tasks are stackful, they have their own stack, which is grow-able (the stack is copied into a bigger stack when it's full)
- Go's yield points are automatically inserted by the compiler based on heuristics which vary depending on the version of the compiler. Whereas Rust yield point are manually inserted by the developer. This resonate strongly with the philosophy of both languages (best-effort automatic vs explicit manual control leading to the best performance) which can also be seen on the topic of memory allocation (automatic boxing based on escape analysis vs manual Box pointer).
- Go only has green thread, which means that if the runtime is failing to keep its promises[1][2], you don't have much alternative. Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload.
[1] https://github.com/golang/go/issues/10958 [2] https://github.com/golang/go/issues/36365
In-userspace stackful context switches are not at all slow in a sane userspace-switching runtime. They are approximately a combination of setjmp() and longjmp(), but without saving all registers. Typically a few tens of nanoseconds or less on a modern CPU. Pre-emptive context switches are nearly always stackful, even if non-pre-emptive (async) context switches are stackless in the same runtime.
A pre-emptive context switch in a userspace co-operative scheduling system is slower than non-pre-emptive context switch if it is caused by a dedicated interrupt of some kind. In the case of userspace pre-emption, generally a signal and return, typically single digit microseconds.
However, the cost there is mainly the signal. If the pre-emptive context switch is caused by a signal that triggered anyway for another purpose, the actual context switch is, again, cheap.
In the model I described (the game environment), pre-emptive context switches are rare. It wouldn't matter if they took longer, because >= 99.99% of context switches are co-operative in that model. The important factor is that no task blows the frame budget or prevents other tasks from running at the full frame rate. In web services a similar target is tail latency, in the presence of diverse tasks you cannot predict in advance.
> meaning you can choose what works best for your workload.
Exactly. And as I've tried to explain, for some varying, complex workloads whatever you choose statically has poor metrics; it must be adaptive to maintain good timing metrics.
> Rust yield point are manually inserted by the developer
Indeed, and in the case of NPCs in a game engine, or a large program composed of many libraries written by hundreds of different authors (e.g. some web servers), that causes complex, interdependent performance characteristics, where timing behaviour in one of them ruins performance for everything, unless they are isolated using threads, in which case it's too slow. There is no "the developer" who can ensure this doesn't happen.
(Aside, if Rust's type system helped with this, that would be great, the same way types help with other "programming in the large" safety issues, but it doesn't address timing characteristics as far as I know.)
> leading to the best performance [..] Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload.
You could summarise my point as: "For some dynamic workloads, especially in timing-sensitive large programs with many components working independently and unpredictably, neither async tasks or OS threads perform best for your workload (or even adequately sometimes). The optimal (or required) combination requires some dynamic responses, and cannot be achieved solely by static placement of yield points and thread initiations."
(This speaks more to the fact that goroutine-per-entity wouldn't be feasible than to the idea that thread-per-entity would be.)
I'm talking about the scheduling model rather than the specific implementations in Go and Rust. The game engine I worked predates them both, and at the time, there is no way 10,000+ OS threads could advance in every rendering frame. Just entering and exiting the kernel for each thread would take longer than the frame budget. It had to be a userspace queue.