In Defense of Async: Function Colors Are Rusty
thecodedmessage.com
thecodedmessage.com
For instance, the postgres driver for rust (which is an excellent driver, by the way), is built in on async primitives. That's a good thing, because it means that it can be used effectively from async code.
Unfortunately, a "hello world" program that uses the client takes 38s to build and downloads 65 dependencies, many of which are at 0.x.y versions and I have no idea what they do (parking_lot_core v0.8.5? slab v0.4.5?).
This is not just an issue of build time, but overall confidence in the quality and security of the code, and knowledge of who the authors even are. Also, it just adds complexity and magic, and sometimes I just want my code to be simple and tight and understandable from beginning to end.
One idea to resolve this is if rust has a standard futures runtime API, and you can just choose one in Cargo.toml or something. By default it would be a built-in single-threaded runtime, but you could choose tokio if you want. If you want to call async code from sync code, there would be an easy way to do it (maybe more syntax?) that would use whatever runtime you chose in Cargo.toml.
https://crates.io/crates/slab/reverse_dependencies
And then looking into the biggest results and understanding community sentiment. Ultimately you need to trust the authors. Do this enough and you start seeing the same popular projects, making it easier to trust things.
> sometimes I just want my code to be simple and tight and understandable from beginning to end
I hear ya! This is one reason I wrote my own async runtime, and the output of “cargo build” fits on my screen. :) But I don’t recommend this unless you have a LOT of time.
However are we going to expect crate authors to repeatedly reinvent wheels? Would we trust them if they did?
If you had a clear metric for all the parts that go into a library you wouldn't worry about how many there were. And if the library author likewise cared about their libraries metric, they wouldn't arbitrarily choose low-quality libraries.
Friction isn't useful here imo. Trust is just as bad with high friction, just maybe less moving pieces. Not having verified metrics seem to me the real problem. Some sort of safety rating that bubbles up from all the dependencies. Changes to software owners would also have to cause issues in this system too.
Put another way, if you go and build Firefox from source, the number of Rust crates and NPM modules it pulls in automatically during its own build is more than the total number of packages in Linux From Scratch and Beyond Linux From Scratch combined. Not every library needs to be the size of glibc, but Rust and Node have gone too far in the opposite direction.
I don't know what you have in your Nix build, but even Linux From Scratch is already bloated because it includes a bunch of build and test tools you don't need to just run an OS. The Arch base system is 27 packages.
Sure, because dependency management for C is a joke^H^H extremely cumbersome and so you get coarse-grained packages that bundle lots of functionality and are a pain to upgrade.
Why is number of dependencies something you care about? Bundling 10 functions together and calling them a library doesn't make them any easier to audit than auditing them individually, all it does is bloat programs that only needed one of them. Getting supply chain attestation to scale is a legitimate problem, but a solvable one - e.g. we could have organisations that would vouch for a large range of authors or packages.
Well to be clear, i'm talking about the dozens of applications primarily that make "my workstation". All the crap and varying duplicated versions, all of it.
Much of my dependencies aren't JavaScript or Rust based, and there are still dozens if not hundreds of dependencies. Is Nix at fault for letting me too easily install things? Maybe each individual app needs to have less dependencies? My workstation is built on leagues of other dependencies, is my workstation less secure because of it? Probably. Would installing less things be more safe? Sure... but throwing away my computer would be too.
The problem isn't the count imo. The problem is how we vet them. To not only judge a library/app, but to automate a way to bubble up this vetting. To let consumers know that the one package/library/etc they chose to use increased their risk by a significant amount. To know when a package updates that it is no longer secure, as it is owned by a different author. etc.
In summary: It's easy for me to install packages on my OS. Most OSs in fact. Making it more difficult to install wouldn't help me make safer choices.
And probably more importantly, if `proxy-auditor-crate` drops a dependency because they no longer trust it, then I definitely want to be notified.
In Rust, this is not the case and leads to fragmentation.
There are a lot of high-quality libraries which cannot be efficiently used in an async context (e.g. Diesel). And since Rust isn't shipped with a default runtime (tokio, async-std, ...), some crates are incompatible with the rest of your program. I understand the reasoning (there is no "one size fits all") but it makes things more complicated.
This then opens up alternative runtimes to tokio (async-std, smol, glommio, monoio)
Then your suggestion of a single threaded executor exists already: https://github.com/enlightware/simple-async-local-executor
Granted, in practice many libraries end up building a hard dependency on tokio
I don't think it's a coincidence that both of the examples here are actually quite reasonable as standalone crates. They're important abstractions, but just a little bit too niche to justify a place in the standard library. That said, I agree that learning what all these crates mean is daunting, and that finding ways to make that easier somehow would be valuable.
parking_lot has actually been considered for inclusion in libstd[0], as a better alternative to the native mutex implementation (parking_lot mutex are smaller, can be const-initialized, can be moved, and tend to perform better - all while not depending on external, non-rust code). The effort is currently on hold, but it's possible that it will resume in the future.
Personally I have much more confidence in the community de facto standard implementation of something than I do in the half-baked version that someone writes themselves to avoid pulling in a dependency.
When async got added, this simplification stood in conflict with the new explicit user-space scheduler (reactor-executor in Rust terms). Today, std is a minefield within an async app — your program may deadlock non-deterministically — without as much as a compiler warning. Every project member needs to worry about intricate details of different types of mutices and which std::fs APIs are kosher, and so on.
I would have preferred a plug-in system where you could use the same application code with different scheduling backends - threaded by default, the M:N model or even the M:1 model if you're feeling web scale. This way, code reuse would have been a breeze.
What we got was a mess of application code where the compatibility matrix (what code is compatible with what runtime) is not clear, enforced or even well documented. Even the proponents of the current model acknowledge that the current ecosystem is fragmented.
This was always true, even prior to async. Rust's thread safety features have never prevented deadlocks. They prevent races, but not deadlocks. RAII makes some patterns that can cause deadlock more avoidable. Nothing in Rust prevents you from spawning a thread that locks a mutex and then never gives it up.
One of my issues is that Rust continues to pile ad hoc solutions, without looking for more fundamental approaches. For example, in the async case, it was rushed in the current form instead of properly solving generators and linear types. Yes, it's probably helped with the language adoption, but it's a short-term win at the expense of the longer-term language health.
this is how we get the mess that is C++
Are you the one withoutboats always chimes in here to correct by pointing out that there was actually tons of discussion and alternative designs evaluated? I seem to remember the last time rust async was discussed here the people who actually worked on the feature being pretty peeved by people claiming this.
However, I totally get why Rust chose to go down that path. It's the way to go if you want to have zero-cost abstractions, even if it's a PITA. And since you're programming in Rust, you've already accepted that PITA's in the name of performance and low-level access are occasionally alright.
Yet there is no production quality storage engine written in rust, whereas there are several of those in go.
Ergonomics matter. Iteration time too. Rust fails hard there.
Sugar around thread yield, unwrap and context return? That seems like a lot of possible gotchas to simply be hidden and lurking under every benign looking synchronous method.
Remember that the point of async/await is cooperative threading and not pre-emption. Hiding these details makes it much harder to reason about. If you don't know when your execution will be yielded, you're essentially being pre-empted, no?
I'm not a rust programmer, but personally I think that stackless coroutines are probably a necessary evil in system languages as full stack switching might have too much overhead. But let's stop pretending that it is actually a good thing.
Still I think that even with stackless coroutines await should be the default (the async keyword would be used if you explicitly want the promise), which, in addition to better ergonomics, would allow the async-ness of a function to be inferred, possibly as a function of its type parameters. This at the very least would avoid duplicating a lot of functionality.
You misunderstand. Its not about whether your thread can be pre-empted, its that your logical execution can hold exclusive access to a thread. There are common design patterns that use exclusive threads as a means of synchronization.
The borrow checker or some other explicit critical section markers are a better soluiton.
That really depends on the platform. Async rust is also being leveraged to build RTOS on embedded platform for instance. If async rust was preemptive by default, that would be impossible. It's not a matter of performance there, but of correctness.
The reason for that is that the point of a RTOS is providing guarantees: A certain task should be guaranteed to run within a certain time. To provide that guarantees, the RTOS needs to support preemption and prioritization. That allows it to schedule a certain task even if another task is running for too long. I'm not aware about any RTOS which makes use of cooperative multitasking.
I think it's worth noting that the big operating system kernels - perhaps the ultimate target of "system languages" - tend to use stackful coroutines rather than stackless.
Stack switching tends to be similar speed overhead than stackless, despite surface appearances in how the code may appear. Stackless tends to do more memory allocation and freeing, though.
Even full OS threads are an acceptable answer for the vast majority of applications. It is good and proper for embedded programmers and OS designers and such to have options as powerful as Rust, but that does not mean the average program written in Rust is embedded, an OS, or needs to be able to scale to hundreds of thousands of threads.
(I actually think reaching straight for async/await is a mistake, personally. It's a tool for more extreme situations, not something you reach for when you're writing a program that's going to intermittently consume a fraction of a percent of CPU, or run a quick job where it may need maybe 10 threads if that and terminate. A lot of people badly overestimate the resources they need and jump straight for the extra-mega power tools for problems where they are literally a dozen orders of various magnitudes (CPU, RAM, etc.) away from needing them for their task.)
(It is extra ironic to be terrified of threads when working in a language like Rust that has so thoroughly tamed them!)
Yes, there are compromises, so you get the flip side of what I said above: Just because something has costs doesn't mean it doesn't have benefits. It is a benefit that you don't have to annotate everything yourself and the compiler and runtime just takes care of it.
"Remember that the point of async/await is cooperative threading and not pre-emption."
I disagree. Cooperative threading is a disaster. It doesn't scale. We tried it, very very hard, with an entire major consumer OS for decades. The point of async/await is precisely to get something like pre-emption into something that isn't using full, real threads (as its primary organization). That it functions like cooperative threading is one of the disadvantages that happens to be an acceptable compromise at most scales, not a goal.
It doesn't need to scale because it solves a different problem. It find extreme success in UI and graphics programming where UI systems are dominated by a single UI thread for synchronization. GPU have been single threaded until recently.
You can argue you do not like it but I can't see how you can argue against it's success.
That said, I'm pretty interested to see a popular multithreaded/pre-emptive UI framework in the wild.
However, this is not possible. When you have concurrent code, whether you make it obvious or hide it, you have contention over any shared mutable state.
async/await is meant to have the pieces with contention be cooperatively multitasked, and the yielding of control is explicit so that you can understand that the state going into the call may be different than the state once you come out - any number of unrelated things may have run in between when you awaited and when control returned.
Not having coloring means that you have turned an explicit cooperative system to an implicit or even preemptive system. There are definite advantages and disadvantages to both, with the difficulty in tracking down and debugging concurrency issues in implicit/premptive systems being the reason that there has been so much research into other approaches such as async/await, actors, and structured concurrency.
Rust has been able to thread since long before async/await was introduced. It was introduced because for some of the use cases of Rust the overhead was unacceptable, not because prior to async you had no options at all to have that sort of functionality.
let promise = async {something+someother();};
// other computations
let result = await promise;
ETA: perhaps it would be a good idea to have a thread-unsafe keyword, though I'm not sure if rust needs that since rust has a really stringent resource pointer/reference aliasing guarantee.
ETA2: I misunderstood async semantics for rust, it's not a fork/join but send/poll. In any case, I still think you should be able to declare async at the call site, not the function site. That doesn't seem to clash.
But that would require starting other threads, which requires everything to be `Send + Sync`, has the overhead of multithreading, and can't be managed with any sort of scheduler (without platform-specific hacks). Unless you use green threads, and that's discussed in the article.
edit to match yours: not sure what a "green" thread is.
In any case, if threads are not an option, an async may always be ignorable, after all, a program may not rely on the order of the execution of its threads/its preemption; that would be a race condition. Textbook definition.
As I see it, the fork/join model is very useful for "shingling" code paths that don't have to happen in sequence, but _could_ happen in parallel.
Final edit: nothing is otherwise stopping the implementer of async to add a pre-started threadpool to the runtime, with whatever configurable parameters one would want.
Post final edit: I think fork/join vs send/sync are just implementation details. But I respect Rust for going with send/sync, just that it doesn't change all that much in terms of the programming interface.
See https://rust-lang.github.io/async-book/01_getting_started/02...
At my work i mix a lot of compute heavy work in an async context, and if i let this compute heavy stuff run on my async threads i'd potentially starve my runtime. Tasks wouldn't get woken up on time, etc.
The preferred pattern is to avoid long running tasks in the default thread pool as it's a shared resource. Long running tasks should be executed somewhere else, either some new thread pool or just a new thread or just don't use async and make it your caller's problem. The point is that you don't want to consume the limited shared resource.
In your case, what would happen? Would a new thread be created? Is it ok to use the default pool? Did the synchronous method writer even consider this? It's hard to say.
Now, does this need to be enforced by the language? No. Developers could simply append "_Async" to make it clear a method can be put on the default thread pool or not. That's still a form a coloring though. Even if you consider it a choice by the caller, it's still a color that needs to be determined and you never want to exhaust your default thread pool.
That said, there are other reasons why calling an async method should have different semantics than calling a normal sync method. Just saying that the "can use shared threads" aspect of async methods can probably live as a convention.
normal synchronous code never gives up control until it's done. so it'll block the cooperative/async scheduler until it finishes, whether it's maxing out the CPU or just waiting idly for a network request to respond. if there's anything else happening on that scheduler, it has to wait, which can lead to rather terrible behavior.
and last but not least: you generally cannot move async code between threads. async tends to imply you want same-thread concurrency guarantees, to avoid the cost of cross-thread mutexes and whatnot. so while you could in theory just interrupt an async scheduler and move it to a new thread, in practice that's not usually safe, so it isn't done.
you could build a system that allows jumping between threads though - that's essentially what green threaded languages do under the hood (like Go). which gives you a middle-ground of "nice thread-like semantics" but "nice async performance", but neither quite as good as a pure system in sometimes-important edge cases.
---
all that said, yes, you can just dedicate a thread to a single scheduler, run a synchronous func on it, and it has practically no additional cost over using a normal thread. but you have to choose to do that, it won't happen by default.
This is the big point. I'd love to see anyone put forth a concrete, valid reason why this couldn't be implemented, because as it stands, it appears that the function coloring problem is exclusively due to poor language design, rather than a technical limitation (where an example of a technical limitation would be escape analysis, which is hard).
Do you actually want this? Currently, you can assume that a function will not change threads mid execution, and that that thread will also not be used to execute other things. Implicit yielding breaks that assumption. Cooperative multithreading becomes impossible because it's much hard to reason about when a method call will yield execution or not.
Of course you need a runtime to drive that future forward, so it's more:
let result = runtime.block_on(future);
If a function does this, it must be async- there's no other way to implement it. If a function does not do this, there is no reason to make it async in the first place- it will never suspend. (Things get uglier when it may or may not need to await something, e.g. if it accepts a closure parameter. Rust doesn't really handle this today, but one approach would be to make the function polymorphic over async-ness- this is similar to but not really the same as what it sounds like you're talking about.)
Async functions are for language-level cooperative multitasking, not concurrency per se. The difference between fork/join (what you imagined async to be) and what it actually is, is more than an implementation detail- it's a semantic change to how the program behaves. If you want to make something happen in the background at the call site, there are already ways to do that, they're just not spelled "async fn."
Other parts of the language are conditioned on the return type. For example the ? operator only works if the return type is Option or Result. It seems like `await` could inspect the return type in the same way.
No difference, but while the feature was being explored early users found it simpler if they didn't need to think about lifetimes on the returned future. Sometimes sugar is just sugar; same reason that `for` loops exist instead of just using `while`.
You need the collaboration with the compiler to make it all work well. And that means keywords and language-level infrastructure. We tried no sugar, and it was rough: http://aturon.github.io/tech/2018/04/24/async-borrowing/
(And a proc macro didn't work out either)
A normal function returning `impl Future` has normal synchronous control flow, cannot ever use `await`, and then eventually returns a value whose type implements the Future trait. You can't just return an int, because ints don't implement `Future`- you have to return something that is `poll`-able.
On the other hand, an `async fn` merely looks like it has normal synchronous control flow, with some `await`s sprinkled in. But when you call it, none of that code runs- instead it immediately constructs and returns a value (of anonymous type, similar to a closure) whose type implements `Future`. The code written in the body has its control flow ripped apart at `await` points, much like a CPS transform, and it goes in the `poll` method of that value's `Future` impl.
It's technically possible that the language could have been designed the way you describe- if an `await` is present, that transformation happens, otherwise it doesn't. But because of the nature of the `await` operator, this is much messier than `?`. For example, when the transformation happens, you need to be able to return an `int`, and when it doesn't you must return an `impl Future`, unlike `?` where you always have to return a `Result` or `Option` regardless of your use of `?`. And you wouldn't be able to tell at a glance whether the function body runs immediately when it's called, or whether it waits for the Future to be `poll`ed. Plus, the language also has `async { .. }` blocks that do the same transformation in an expression context, so it's kind of nice to be consistent with that.
Introducing a new keyword is a breaking change because existing code may have a variable with the same name. So `await` is a conditional keyword in C#, conditioned on whether the function is marked `async`. And other languages have since just copied that choice.
fn bar() -> impl Future<Output = u8> {
async {
let x: u8 = foo().await;
x + 5
}
}I think it's obvious that Rust chose `async fn` for familiarity with other languages. As you say `async` is just sugar on the return type, so logically it should decorate that type, not the fn keyword. "async fn" is even misleading since the function itself returns synchronously (and immediately): it's the returned value where the async-ness lies.
So I think Rust's use and placement of `async` is for familiarity, which traces back to C#'s original choice. Not to say that's a bad rationale.
Admittedly I didn't follow any of the Rust async design discussions so I could be completely wrong about all of it.
You've got the causality backwards- it's the async-ness that causes it to return immediately. Async is primarily about the execution model of the entire function, and the return type is only changed in service of that model.
The course of the design discussions started without async or await, using library-based Future combinators. Async was introduced as a way to desugar control flow and move knowledge of Futures into the compiler for borrow checking reasons, not out of any intention to mimic C#.
I don't agree with this analogy. With Result, it's ?; – with async, it's .await; . That's not overriding or overloading; it's explicit.
But for Rust, I think having colored functions is probably the right call. Asynchronous operations are fundamentally different at the OS/machine level. Since Rust is a language where code deliberately has very high affinity to what the underlying system is doing, it makes sense to make that distinction visible to users even if it means that it's harder to write and reuse asynchronous code.
Is async-style the best possible tradeoff to achieve the best result when programming with blocking calls?
The fact that you should not be applying the same approach to a very similar problem (blocking CPU-bound code) kinda sucks, which means that you need to be employing multiple concurrency approaches in the same program.
I am all for syntactic sugar to make concurrency easier and less error prone, but I'd prefer it if the same approach trivially worked for all sorts of concurrent functions without heavy penalties to execution.
In short, "the best approach" to me is one that:
1. Is suitable for all sorts of concurrency
2. Has a low memory overhead and low switching costs like the async functions today
3. Is the most "ergonomic" (leading to fewest bugs, easiest debugging and easiest maintenance): yeah, this one is not too well defined either, which is why I would measure developer satisfaction and things like defect rates to establish which one is it, not that this makes it any easier to define :)
4. Provides "enough" safety and encapsulation without impacting performance (much)
If 4 has a huge cost, perhaps go with special-cased "async isolated" concurrency primitives when you need added safety in addition to "async", and if the cost is low, allow not paying it anyway with the inverse ("async unsafe"): basically, different defaults depending on what's realistically achievable and nature of the language.
I think that ideal lies somewhere near the "non-cooperative multitasking" async programming approach, but I am not sure, and I haven't played with enough recent developments in programming languages either to know if any one of them gets close.
But having thought through it a bit with this comment, I think my main gripe is how your (developer's) decision of the concurrency tool (threads, async/await, callbacks, green threads...) affects your syntax simultaneously (eg. why should awaiting a compute-intensive task be problematic? that's an implementation detail a developer should not have to worry about if all else is equal).
foo :: Monad m => m ()
in Haskell. If that goes to rust, maybe we can have "async polymorphism". This, and the far more easier "mut polymorphism" (no more as as_mut boilerplate) means we get best of both worlds: color and code reuse!
As far as the complexity goes: it’s a systems language without a true “fat” runtime like C# or Go. You can’t do the sorts of pretty abstractions you can do in those languages. You can’t hide the details very much. Rust async is syntactic sugar around what would be much more verbose nested callback patterns. It hides that nastiness but you still need to have a mental model of what is actually going on.
I think Java’s approach with Loom is going to be a big win over C# there, as someone that just wants to get stuff done and is a fan of both.
Of course Java also has some frameworks like Netty and things built on top of it - which won't benefit. But I feel like even though those are great from a performance point of view, they are actually more niche in the overall java world.
If it goes too many levels deep, we have to ask what we are trying to achieve and perhaps consider a better approach (a specialized state machine? threads?)
You're not making the case here, guys.
Why are interrupts not used? or are they?
source: epoll, select(bsd at least), isn't kqueue also like a multiplexed epoll?