EDIT: People talk about swapping executors etc but it seems like 99.999% of the programming applications you don't need anything like this, nor does something like Python supports this anyway and we're all fine with it.
EDIT: People talk about swapping executors etc but it seems like 99.999% of the programming applications you don't need anything like this, nor does something like Python supports this anyway and we're all fine with it.
It depends on what you're comparing it against. If you are comparing Rust async against other languages, then the major difference is Rusts borrow checker. In every implementation of async, you effectively move whatever state is needed off the stack and into a separately allocated memory area. In Rust that separate area is a closure.
This interacts badly with Rusts borrow checker. The borrow checker needs to know the lifetime of any object you deal with. It has two "base truths", by which I mean life times it already knows about that you can derive other life times from: static and the stack. If they don't suffice you have to handle life time management yourself and at run time using Rc or Arc or something. Being forced to do that complicates your types and slows the code down. The root cause is async in Rust removes one of those two base truths: the stack. So now you are forced to write that ugly manual life time management code far more often.
This is unique to Rust. Every other language I know of that implements async has garbage collection, so while it remains true they also move stuff off the stack it doesn't change anything. You still use the same types, and apart from sprinkling async's and await's here and there and indenting your closures, your code remains the same.
This is also why I think green threads are a much better fit for Rust than async. Under the hood green threads and async are very similar: they are both ways of doing event driven I/O. Their performance characteristics are near identical. The main difference is while async forces you to move your state to a different area, green threads you do it as before and store it on the stack, just like normal code. In fact green thread code looks identical to normal code. The only change you have to make to convert some code to green threads is change the name of the I/O calls to use non-blocking versions (which is something you also have to do with async, of course). But since the stack is still available all those fights with the borrow checker async creates go away, as does all the extra syntax async requires.
It doesn't come for free of course, so the run time performance of green threads and async is not absolutely identical. In async every task shares the one stack, whereas in green threads they each get a new one. This creates some extra memory overhead, chews up considerable address space (which normally isn't backed by memory) in order to protect against stack overflow, and it costs a bit more to set up a stack. But once a task is setup green threads are going to be bit faster you aren't moving stuff and and off the stack and you don't get hit with those additional run time life time checks you were forced to introduce for async. Mitigating green threads overheads somewhat, a process that is handling 1000's of concurrently connections is unlikely to be running one a machine that is memory constrained, so the extra memory probably doesn't matter. (In reality a Raspberry Pi with 4GB of memory can handle 1000's of stacks.) And it's likely to be a 64 bit machine where address space is nearly free, so the "considerable extra address" space also doesn't matter.
Still, I can think of once place it does matter. You can have two styles of generators: ones take the async approach and ones that take the green thread approach (ie, allocate an extra stack to each generator). Rust nightly does have generators and they currently take the green thread approach(!). But, generators tend to be short lived. You tend to use them to iterate over an array, rather than serving a web request (the typical use a task is put to). That means the overhead of creating the stack for a green thread isn't a small proportion of the time a long running task takes, but could well dominate the time it takes to iterate over a small array. Thus async is a much better fit for iterators.
Currently Rust has this arse about: it has async for long running tasks, and green threads for generators in nightly. With this announcement it looks like this will be 1/2 fixed. Great!
> Every other language I know of that implements async has garbage collection,
Small note: C++ also has it these days, and does not have GC. You are (as far as I know) left to your own with object lifetimes, as with non-async/await.
The current async approach requires the compiler to statically calculate a fixed size for each invocation context. Except when it can't, in which case you have to dynamically allocate space for each invocation. That same logic could be used to transparently optimize stack creation, opportunistically avoiding both a pessimistically large stack and the costly guard pages. The compiler could even choose to instantiate the generator stack on the caller's stack, just as today.
async Rust optimizes the common case but completely neglects the hard case. In theory Rust users should be able to have their cake and eat it too, especially given that the difficult static analysis work already exists to support the current async model.
(I've made this point before and feel like I may have forgotten some counterpoints. I apologize in advance if that's so.)
The issue generated a lot of hot air at the time, far more than I have time to read, but I think I have the gist of it.
The performance issues were due to an implementation choice. All green thread implementations I'm aware of hide the code colouring event driven I/O introduces, and Rust did the same. The hiding has to be done at run time. That translates to every I/O call has at some point chose the blocking or non-blocking implementation. This is typically done using vtables (dyn in Rust), but whatever mechanism is used, it introduces runtime overhead. Worse, it slows down everything - including code that doesn't use green threads. It gets radically worse if you try to hide C calls blocking.
Async has the same issue, some solves it by making coloured code the programmers problem. If they had of made the same design decision for green threads the performance issues go away. We don't have to speculate about that. There is a green thread crate out there called "may", then has independent benchmarks covering it, async Rust implementations, and other languages. "may" beat everything at one point, but it's a very noisy benchmark so the only conclusion I would draw from it is that "may" looks to run at the same speed as async.
As for the rest of the issues: they revolve around needing a separate stack. I tried to cover the trade-offs above. Summarising: green threads lead to simpler code that's easier to write and theoretically could run faster than async, but have larger setup overhead and use more memory.
Just reasoning aloud here. That looks to be another similarity with green threads and async. Async also requires a big fat runtime that isn't part of the language. Instead you have to pick an async colour - such as tokio. That's pretty much what happens with green threads now. You have to pick a runtime such as the "may" crate, which forces the programmer to choose yet another colour.
So many colours, yet they are all just event loops underneath. Colours cause fragmentation, fragmentation is the mortal enemy of reuse.
It seems like the very least the language could do is provide a set of trait's for event driven I/O that mirror the existing I/O library in std. Then the library writers wouldn't have to colour their code by using a particular event loop implementation. I suspect it's easy enough for green threads or async, but accommodating both styles of event loop would be hard.
I do completely agree with you that it would be great for std to pull in more traits from `futures` and elsewhere to allow it to be easier to write code against different runtimes! I think part of the intent was to get Futures out, see how they get used, and then to go from there. Hopefully we're getting into the "go from there" stage now, which is some of what this article gets into
Sometimes I feel like the only way to get really good async I/O is to rewrite the whole damn thing to use io_uring. Avoid the blocking syscalls as a whole.
("Rewrite" because this changes e.g. read buffer management. The fundamental APIs will have to change, to enable the performance gains.)