How to think about async/await in Rust
cliffle.com
cliffle.com
It didn't start because it is so awesome, it started because JS can't do parallel any other way. That's the long and short of it. People wanted to use JS in the backend for some reason. The backend requires concurrency. JS cannot do concurrency. Enter the event loop. Then enter some syntactic sugar for the event loop. And since JS is popular, async became popular.
Code written using threads is, at least to me, much more readable and easier to reason about. Each section in itself is just synchronous code. The runtime/kernel take care of concurrency. The overhead is negligible in a day when we have greenlet implementations. It works for both i/o bound concurrency and cpu bound parallel computing. It doesn't require entire libraries rewritten to support it. There is no callback hell. It scales both horizontically and vertically. Modern languages support it out of the box (Hello `go` keyword).
I realise that this is going to get a lot of downvotes. I don't really care. To me, async is just "cooperative multitasking" with a quick paintjob. We left behind that paradigm in Operating Systems decades ago, and with good reason.
JS doesn't have any language-level abstractions for parallelism (async or not) but you do have Web Workers[0] and process forking (depending on runtime) to get actual parallel programming. JS async deals with concurrency, not parallelism.
Threads are the opposite: They are interfaces for parallel programming and their use is orthogonal to how your application handles the resulting concurrency.
You say "the runtime/kernel take care of concurrency" - are you telling me you never write a mutex or implement locking logic? Because that's what "taking care of concurrency" is. I'd choose refactoring callback-/Promise-hell over debugging a complex deadlock any day (unless intra-process parallelism is actually a requirement, which may tip the scale in the other direction).
In the context of doing concurrency and parallelism in Rust, I'd 100% agree that the JS/C#-style async/await approach isn't necessarily always the best approach and it's good to consider alternative idioms if your requirements call for it. For anyone writing "apps" or glue-libraries, though, I'm thankful that they stay away from spawning threads all over my system by default and that they need more tools than "learn Rust async in 15 minutes" gives them to become dangerous.
Messing up your single-threaded event-loop concurrency can hog roughly a single CPU core and cause OoM. Messing up thread-based concurrency can have larger implications on the hosting system.
[0]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers...
We still need them if we have multiple parallel tasks (coroutines spawned non-locally) using a shared resource across multiple yield points.
As long as the accesses to the shared variable are separated in time, sharing is fine.
This is correct code:
let mut foo = 1;
async { foo += 1 }.await;
foo += 1;
println!("{foo}");
int foo = 1;
std::thread ([&] { foo+=1; }).join();
foo+=1;
std::cout <<foo <<'\n';
mut buffer: &[u8] = ...;
loop {
select! {
_ = stream.readable() => stream.read(&mut buffer),
_ = stream.writable() => stream.write(&mut buffer),
}
} select! {
_ = channel.readable(), if queue.has_free_space() => read(&mut queue),
_ = channel.writable(), if queue.has_data() => write(&mut queue),
}
In any case acq/rel is the only thing required here and it is extremely cheap.
edit: in any case we are discussing synchronization and 'has_free_space' 'had_data' are a form of synchronization, we all agree that async and threads have different performance characteristics.
I don't think it is a universal solution. What if I am interested in parallelism as well, only not for the coroutines that operate on the same data? If my app handles 10k connections, I want them to be handled in parallel, as they do not share anything so making them parallel is easy. What is not easy is running stuff concurrently on shared data - that requires some form of synchronization and async/await with event loops is a very elegant solution.
You say that it can be handled with an SPSC queue and it is only one ack/rel. But then add another type of event that can happen concurrently, e.g. a user request to reconfigure the app. Or an inactivity timeout. I can trivially handle those with adding more branches to the `select!`, and my code still stays easy to follow. With threads dedicated to each type of concurrent action and trying to update state of the app directly I imagine this can get hairy pretty quickly.
Threads are both for parallelism and for concurrency. Threads have been used for decades on machines without hardware parallelism.
> are you telling me you never write a mutex or implement locking logic
aside from the fact that mutexes vs futures is completely orthogonal to async vs threads, I definitely prefer dealing with mutexes. 99% of mutex usage is trivial and deadlocks are relatively easy to debug. The issue with locks is their performance implicaitons.
No, it really doesn't. I mention both concurrency and parallelism, and their main difference.
> Threads are the opposite: They are interfaces for parallel programming
No, they are not. Threads can do both. When waiting for an i/o bound operation, a thread can simply sleep. Added bonus: A thread basd implementation supports io bound concurrency and cpu bound parallelism using the exact same principle, and letting the kernel/runtime take care of the details.
> are you telling me you never write a mutex or implement locking logic?
Pretty sure I never said that. As for what I prefer to debug: Most Mutex-based synchronicity tasks that come up in practice are easy. And if "complex deadlock" does occur, it's usually pretty clear what resource was locked. Debugging that is just a question of going over all callers that access that resource.
And as mentioned before, all that code is synchronous. So each one of them is easy to reason about.
So yea, all in all, I prefer debugging problems arising from deadlocks over wading through callback-hell. By a huge margin.
Oh, and all that is before we even talk about using CSP as an approach to synchronizing threads, which makes it both harder to mess up, and again easier to reason about.
If the comment is not conflating concurrency and parallelism as you say, mind expanding on this part?
JS has neither.
Of course it is. Concurrency gives the impression to the user that parallel processing is being done, even when it's not. That's why my parents old 386 could render a moving mouse cursor and a progress bar at the same time (usually).
Concurrency lets you do things "in parallel" even if you can't actually do them in parallel.
Certainly. That part is a sentence written to be short and catchy. It sacrifices precision for reasons of brevity and style. It also doesnt mention either concurrency or parallelism, it just uses the word "parallel".
This is acceptable, because the post goes on to more precise statements later on, quote:
It works for both i/o bound concurrency and cpu bound parallel computing.
I mean if you're fine with blocking I/O then obviously you don't need async, but on the other hand having non-blocking I/O is the whole point of async ^^
Most node code I see in the wild is just a simple `await loadData()` which doesn't block the main node thread but does block that code flow until the data returns. This is roughly the same as what would happen in a normal blocking multithreaded language other than the extra overhead of a thread. If you don't have enough threads (or they are efficent enough in your language of choice) for this overhead to be an issue then you are adding all this complexity for almost no benefit.
Basically it comes down to if you trust your language of choice's threads more or less than your language of choice's event scheduler. Since Node is fully single threaded there isn't really an option but with other languages, a single thread per worker is much simpler.
In python it is even more opaque which to use as the CPython itself is singled threaded so you are comparing its thread implementation to its event scheduler implementation. For this small win you get to rewrite all your code to new, none-standard apis.
> Most node code I see in the wild is just a simple `await loadData()` which doesn't block the main node thread but does block that code flow until the data returns.
Agreed. Higher level languages tend to discourage or outright decide not to expose asynchronous I/O. Instead, they optimize blocking I/O within their own runtime - skipping the higher resource needs for the system schedule and thread representation.
If I am writing a web server in C or C++, I'm likely writing asynchronous I/O directly. I may also decide to use custom memory strategies, such as pooling allocators.
If I write one in classic Java, I'm allocating two threads to represent input and output for each active connection, and hoping the JVM knows how to make that mass of threads efficient. In Go, I'm likely using a lot of goroutines and again hoping the language runtime/standard library figured out how to make that efficient.
Java packages like NIO/Netty and Go packages like gaio are what expose asynchronous programming to the developer.
The footgun is that it is hard to use an asynchronous I/O package when you have a deep dependency tree that may contain blocking code, perhaps in some third party package. This was one of the attractions to server-side javascript; other than a few local filesystem operations, everything sticks to the same concurrency model (even if they may interact with it as callbacks, promises or async/await)
A language + runtime like Go or Erlang doesn't so much have "blocking" or "non-blocking" IO as the terms simply not applying. I see them yielding far more confusion than understanding when people try to come from Node and apply them to such threaded runtimes.
But if you had to force a term on such a system, the better understanding is that everything in a large-number-of-threads language+runtime is non-blocking. Both terms yield incorrect understanding, but that one gets you closer to the truth.
If you get into a time machine back to the 1980s, then yes.
You will not necessarily utilize more CPU cores by spawning additional Web Workers because they are not inherenent parallel. The actual performance of Web Workers depend on how your browser and OS schedules threads.
They are OS threads despite the mountain of misinformation on the Internet about them implying that they are truly parallel. They are not.
> [Web workers] allow long tasks to be executed without yielding to keep the page responsive
You don't get extra calculation performance with web workers. You just create the illusion of a smooth experience because you don't block painting. It does not complete faster.
https://github.com/jschaf/web-worker-experiment/
> You don't get extra calculation performance with web workers
The primary purpose of web workers is extra calculation performance. From MDN:
> Workers are mainly useful for allowing your code to perform processor-intensive calculations without blocking the user interface thread
I remember doing async in C++ with CORBA and ACE's Reactor pattern about 25 years ago, it was not beautiful nor easy.
But if memory serves, most interpreted server side languages used for web programming in the late 2000's didn't have mature multithreading or async capabilities and most of the deployments consisted on exec/forked full application servers. I would also bet that this is exactly what made Node.js popular.
To each its own, async is just another tool in the proverbial belt.
If you have compute-bound components, things get more complicated. If you have threads, locks, and async, all in one program, things get much more complicated. I'm not sure that's a win. At some point, it's easier to use something like Go's green-thread "goroutines", which try not to block, but can block if they have to.
Yes, but this is very unusual. In a web server, you have pool of threads that can respond to incoming connections. When one is blocking, another is ready to go. All the transition are handled transparently by the kernel.
I do agree however that async as a concurrency approach sucks.
In regular threaded programming, cancellation is a bit more painful as you need to have some type of cancellation token used each time the thread waits for something. This a) is more verbose and b) can lead to bugs where you forget to implement the cancellation logic.
No, it isn't. Nothing is stopping your threading library from implementing the same thing. It just turns out it is a bad idea to kill threads at random points in time because they may own things like locks. Or in the case of async await doing something that is thought to be atomic.
A co-routine that yields in the middle of an atomic operation is an oxymoron. Anything can happen before you're scheduled again.
In the end, its about expressing a state machine in a more concise implicit way, which is a suitable level of abstraction for most use cases.
> In regular threaded programming, cancellation is a bit more painful as you need to have some type of cancellation token used each time the thread waits for something. This a) is more verbose and b) can lead to bugs where you forget to implement the cancellation logic.
Yeah of course Rust just makes cancellation so easy by allowing the Futures to be dropped. What about the resources these Futures could have allocated within the context that are not just memory? You are saying it as if async/await somehow solved the whole problem of stack unwinding.
In Python, cancellation causes an exception to be injected at the await site, which allows it to clean up whatever resources it likes (even if that means making other async calls). If you use Trio or the new TaskGroup in asyncio (inspired by Trio) then an exception leaking out of one task causes the others to be cancelled, and then the task group waits for all tasks to complete (successfully, with exception, or cancelled). It's extremely nice and easy to write reliable programs.
In principle, I think many of these ideas could be applied to threaded IO. But I haven't seen it done in practice.
That's how POSIX deferred [1] cancellation works. An uncatchable exception is thrown from blocking calls if a thread is requested to terminate. As POSIX is C centered, you can imagine that handling exceptions was never popular, but it should work fine in C++. For some reasons it wasn't added to std::thread though.
[1] there is also async cancellation, but friends do not let friends use PTHREAD_CANCEL_ASYNCHRONOUS.
But not in JavaScript.
Yeah as I said async does not, in fact, "provide easily cancellable execution patterns".
It is, and not only to you. It is a way to save a call stack until a runloop calls it back.
But what I can’t agree with is parallels with OS. Coop MT is only problematic in OS MT. When it’s your code there’s no unknown bad actor, and having multiple cooperative (mostly waiting) processes without scheduling them on a thread pool is a useful concept regardless of threads availability.
E.g. when you have to wait on multiple sources, the options you have are:
- serialize
- perform few non-blocking calls and wait for any/all of them to complete
- schedule them as tasks on a thread pool and wait for their completion
Async can do all three, it’s orthogonal. I’d say that just awaiting on PMT task completion is much more convenient that setting up locking primitives. Same for NB polling. Promise is just an abstraction and all it does is waiting for an event to fire on a current thread’s runloop while retaining the comfort of a lexical scope, all with a couple of keywords.
> I’d say that just awaiting on PMT task completion is much more convenient that setting up locking primitives.
Don't use the primitives then - write e.g. a parbegin/parallel-map atop thread primitives, and use that.
For example, the blog authors project is an OS running on minimal resources that would not be appropriate for any threading model I'm aware of.
This is a wee operating system written to support the async style of programming in Rust on microcontrollers. It fits in about 2 kiB of Flash and uses about 20 bytes of RAM (before your tasks are added). In that space, you get a full async runtime with multiple tasks, support for complex concurrency via join and select, and a lot of convenient but simple APIs.
If you'd like more info on the story around the adoption of async/await in rust you can see this excellent talk by Steve K. https://www.infoq.com/presentations/rust-2019/.
As for threading vs. explicit yielding (e.g. coroutines), I’d say it’s a matter of taste. I generally prefer to see where code is going to yield. Something like gevent can make control flow confusing, since it’s unclear what will yield, and you need to implement explicit yielding for CPU-bound tasks anyway. Its green threads are based on greenlet, which are cooperative coroutines.
Cooperative multitasking was a big problem in operating systems, where you can’t tell whether other processes are looking for CPU time or not. But within your own code, you can control it however you want!
Wanting to do asynchronous I/O to disk is only useful if you're aiming to bypass the cache. In practice it is very hard to reach higher performance when doing that though.
But I also think the interest in things like io_uring in Linux reflect that people are open to asynchronous file IO, since the kernel is doing asynchronous work internally. To be honest, I don't know much about io_uring though - I haven't used it for anything serious.
There's no perfect choice (as always) -- After all, for extremely high-performance scenarios, people avoid the async nature of IO entirely, and dedicate a thread to busy-looping and polling for readiness. That's what DPDK does for networking. And I think for io_uring and other Linux disk interfaces have options to use polling internally.
Pretending they're the same under some generic I/O concept only goes so far.
Yeah, but this could already be solved without "async/await compiler magic" in native code just with OS primitives, for instance with Windows-style event objects, it might look like this in pseudo-code:
const event1 = read_async(...);
const event2 = read_async(...);
const event3 = read_async(...);
wait_all(event1, event2, event3);
Looks just as convenient as async/await style, but doesn't need special language features or a tricky code-transformation pass in the compiler which turns sequential code into a switch-case state machine (and most importantly, it doesn't have the 'function-color problem').
(this really makes me wonder why Rust has gone down the Javascript-style async/await route with function coloring - the only reason why it remotely makes sense is that it also works in WASM).
If it does the latter, the stack will keep growing with each nested wait call:
main -> runEventLoop -> someCallback -> wait_all -> runEventLoop -> anotherCallback -> wait_all -> ...
The async/await transformation to a state machine avoids this problem.
(the operating system's thread scheduler is basically the equivalent to the JS "event loop").
In no way does it mean that you don’t need an event loop because threads exist, as was the contention here.
2. It absolutely does mean you don't need the main thread event loop for non UI-events.
Async would let you yield at the gather.
Taking into account that an async/await runtime also needs to switch to a different "context", the performance difference really shouldn't be all that big, especially if the task scheduler uses fibers (YMMV of course).
As someone who’s done asynchronous programming in Rust before Futures (I’ll call it C style), then with Futures, then with async/await, it’s because it is far simpler. On top of that it allows for an ecosystem of libraries to be built up around common implementations for problems. Without it, what you end up with is a lot of people solving common state machine problems in ways that have global context or other things going on which make the library unportable and not able to easily be reused in other contexts. With async/await, we actually have multiple runtimes in the wild, and common implementations that work across those different runtimes without any changes needed. So while I’m disappointed that we ended up with function coloring, I have to say that it’s far simpler than anything else I’ve worked with while maintaining zero overhead costs allowing it to be used in nostd contexts like Operating Systems and embedded software.
E.g. I can do:
loop {
select! {
_ = src_channel.readable() => src_channel.read(&mut buffer),
_ = dst_channel.writable() => dst_channel.write(&mut buffer),
}
}
This is not possible to achieve with threads directly. If using blocking I/O + threads model, then you'd need to dedicate one thread for reading and one for writing and then synchronize access to the shared data structure (where using a queue/channel also counts as synchronization). Which obviously would be much harder to get right.
Not obvious to me I'm afraid. Using CSP, this is almost trivially easy. All access to the data goes through a guardian thread. Accessing the resource is just sending a message.
And mutexes are not hard either.
crossbeam_channel FTW
Business logic programmers shouldn't be dealing with machine level parallelism and async unless heavily abstracted, such as in a job queue or evented message queue.
JMP or RET is how the machine transfers control flow at the machine level. So coroutines are a natural solution to switching between code at the machine layer.
If you're working in Javascript, then you shouldn't have to worry about this stuff.
Cooperative multitasking is elegant, within a process for scheduling but not as the main approach for the operating system to switch between processes, it's a subtle difference.
If the operating system depends on cooperative multitasking, some buggy processes can keep control flow to themselves. But using cooperative multitasking inside a process for code elegance, is a good way of scheduling and decoupling concerns.
I have a lightweight thread runtime similar to Go and I find event loops really interesting, I want to make the pain of async and parallelism go away.
Concurrency and parallelism are different concepts. Many many years ago it was easy to confuse them both because there were no parallelism. You had a single big core and CPU pipelines were much simpler.
I won't delve into details, although I found them fascinating, but concurrency and parallelism are different tools. I confess I found the name "concurrency" not useful.
Concurrency allows you to transfer control from one piece of text (I mean executable code) to another while it waits for the return.
Parallel means instructions are being executed, well, in parallel.
The OS scheduler does not inspect the code, nor know that the next instruction will be a noop sleep. Some languages with runtime environments provide basically functionality pra intercepting calls and nudging the OS.
Attempts in the past of requiring each application to be clear about sharing control but it failed. One single bad application could hang and compromise the entire system. As a matter of fact, some RTOS uses this premise of development.
Async has been implemented by providing a runtime library which saves the context and swap tasks. The control is only hidden from the programmer.
I do not know about Golang, but I suppose coroutines are implemented in a different way, as it seems to me, that the compiler handles this. But I don't know.
Huh? It's certainly the case that people who program have dedicated neural circuitry.
Most of the work is not being done by language centres[0]
[0] https://hub.jhu.edu/2020/12/17/brain-activity-while-reading-...
Then there's `#[tokio::main(flavor = "multi_thread", worker_threads = 2)]` which actually lets you use multithreading with async/await which is just a nice feature to have, given that Python can't do this due to the GIL. But asyncio in Python is also so great.
It’s also possible to do async-like behaviour - without the async/await language feature - in C# using LINQ; so you could argue C# has had the capability (like F#) since LINQ was released.
But, I believe C# was the first mainstream language to implement the async/await method-splitting coroutines state-machine (as a language feature)
C#, and other languages that have taken the C# approach [to async/await], don't have first class continuations (well, C# does with LINQ, but that compromises most ways the average OO dev works). They implement async/await with first-class keywords that indicate where to slice a method in two.
In my language-ext [1] project I have added the LINQ operators to `Task<T>` which allows C# tasks to be used in the same way that Async is done in F#.
[1] https://github.com/louthy/language-ext/blob/main/LanguageExt...
Incidentally there are papers that are trying to improve on async in C++ by trying to sneak in generalized do-notation.
https://en.wikipedia.org/wiki/Concurrent_Pascal
Simula,
https://en.wikipedia.org/wiki/Simula
And plenty of other ones,
C# really helped to popularized asyc/await as language feature, even though F# had something similar first.
So, no it is not a hype. These types of techniques have been used for a very long time.
Event Loops are an old technique and not a hype. I am aware of that. One of my earliest C experiments as a kid was writing a snake-implementation for the terminal. I still have that code, and it uses an event loop to process the input.
The hype I am talking about, is an assertion currently "en vogue", that asynchronous should be the default method of doing concurrency.
This hype, imho, started with the ubiquitous use of JS as a backend implementation language. JS couldn't work another way, so this is what all these JS programmers used, and JS is super popular. So, it must be cool and great. And that's how a workaround for a language limitation became a popular paradigm.
https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
https://news.ycombinator.com/item?id=36597229 (fresh repost)
Discussed previously:
https://news.ycombinator.com/item?id=8984648 (8ya)
https://news.ycombinator.com/item?id=16732948 (5ya)
https://news.ycombinator.com/item?id=23218782 (3ya)
https://news.ycombinator.com/item?id=28657358 (2ya)
NodeJS did it right, and it's hard to call a 14-year-old technology a fad.
I would like to know more detail about this claim.
The scenario you are describing is one were 64-128 OS threads are fully blocked waiting for IO. If that's the case, is it likely that you will have additional unused IO resources that could be being utilized?
Also, what overhead do you see as the main limit on spawning a lot of threads? Is that the CPU time of context switching? If so, in this scenario CPU is not the bottleneck, and switching between processes will be nothing, especially with a 32-64 core CPU.
This is a genuine question as I have never worked on an application that got close to maxing out either approach.
One likely scenario is that you've issued 128 RPCs to some other services and are waiting to hear back. Even if each RPC is, say, on a separate TCP connection, your network stack can handle plenty more.
> Also, what overhead do you see as the main limit on spawning a lot of threads? Is that the CPU time of context switching? If so, in this scenario CPU is not the bottleneck, and switching between processes will be nothing, especially with a 32-64 core CPU.
I don't remember what specific feature of OS threads contributes the most to overhead, and maybe someone else can answer this better. But context-switching burdens both the CPU and RAM (due to saved stacks).
> This is a genuine question
I always assume this anyway. Maybe not on Reddit ;)
I think I'm going to do some research and see what benchmarks/measurements I can find.
Also can say at least that paging to/from disk or using memory compression would dominate all other overheads, and it's not something to rely on here.
Hmm, wasn't the whole point of doing things in event loop because it way out perform thread-based architecture? Like when Nginx way out performs Apache? On a single core CPU of course.
Edit: It is not that event-loop is better than thread-based, just in a web server scenario it just perform much better.
For rust specifically, though, green threads/coroutines were discarded because they are not zero-cost.
You can have good ergonomics and performance with stackful cooperatively scheduled tasks instead of a stackless sync/await abstraction.
Async/await makes sense when you have so many tasks that you cant afford to dedicate a full stack to each of them and segmented stacks or heap allocated frames are not an option (for performance or compatibility).
I'm curious, what reason?
I grew up on Python and C#, and only know async/await, never done real threading (C# async is threading and coroutines under the hood, Python is just coroutines, single-threaded). I find that way of writing code very elegant, as one can encode points of blocking/switching explicitly. A bit like encoding logic into the type system (cliffle has an article on the type state pattern, a good read!): the underlying async implementation can change without code adjustments.
However, that same reason doesn't apply within a single app, because a single app by a single author _can_ safely co-operate with itself. So co-operative time sharing can work and make sense within single app.
That's not the case in any non-trivial app. Any real program truly has dozens if not hundreds of authors whose code you're using sight unseen, and which may be difficult to modify.
And your program getting stuck can be just as bad as the OS getting stuck. Suddenly your program doesn't reply to API requests, or doesn't relinquish some expensive resource (like an expensive VM), or some such.
In days gone by, processes who got to run on the single core that contemporary CPU had available, had to actively relinquish control of the core back to the kernel.
If a single process refused to do so, e.g. because the program hang, there was nothing the kernel could do about it, and the entire OS was blocked. The scheduler never ran, no other process would get CPU time, the whole thing was dead in the water, and all you could do was kick the "Reset" button (if present) or pull the power cord and reboot.
Obviously, this is a very bad situation for an OS, which runs many processes from many sources. And because of that, we ditched this system, and went on to preemptive multitasking, where control is relinquished back to the scheduler after a time whether the process is okay with that or not.
Async basically re-invented that system in userspace. We have an event loop, and we have processes that actively yield control to it. What happens if a subroutine refuses to do so? There is nothing the event loop can do about that.
And it's really easy for this to happen. All it needs is a single synchronous call, say, to an external datastore, somewhere deep down in the callstack, and the awesome throughput of async goes bye bye.
val another_path = await readFile(path);
val data = await readFile(another_path);
console.log(data.length);
val another_path = readFileSync(path);
val data = readFileSync(another_path);
console.log(data.length);
In Lua and Zig we have "cooperative multitasking" but we get to use the same library for both kinds of applications :)
Javascript is limited by the single-threaded browser runtime, and the way that WebWorkers were bolted on later didn't help much because WebWorkers with message passing are too inflexible to implement even green-thread-style task switching.
The state-machine code-transformation was indeed the only way out of this dilemma.
Async/await is essentially high-level language syntax sugar and it shouldn't matter whether it is implemented via code-transformation, or with green-threads or real threads, or a combination of all those under the hood (but in reality it matters because there's a difference between 'code slices' being scheduled on the same thread or different threads when dealing with shared resources).
And in C#, you can absolutely combine it with Task.Run if you want it to run be concurrent. This feature made asynchronous operations so much better than all the patterns that came before it - and I think they goes for every language that ended up copying it.
Threads are a resource hog. They take a lot of system resources, and so you usually want to have as few of them as possible. This is a problem for applications that could, in theory, support thousands of concurrent connections, if not more. With a basic thread-based model, you need 1 thread per connection, and if you have long-lived connections with infrequent traffic, those threads mostly do nothing but consume precious system resources. When you're waiting for data, the thread is blocked and does nothing. With async/await, you can have far fewer threads, maybe even just one, and handle blocking through a system call that wakes a thread up whenever any one of the currently blocked tasks is ready to progress. In languages with much lighter thread alternatives, such as Go's goroutines or Erlang's Beam processes, this problem basically doesn't exist, and so those languages don't need async/await at all.
Not really on any decent operating system, but if they are too heavy, there's still fibers aka green-threads aka stack-switching (which at least on Windows are an operating system primitive - but can be implemented in user code on any system that gives you direct access to the CPU stack and registers).
I doubt that the async-await state machine code transformation which 'slices' sequential function bodies into many small parts which are then jumped in and out frequently is any better for performance than stack-switching (in async/await you still need to switch a 'context pointer' on slice-entry/exit instead of the stack pointer).
One obvious advantage of the state-machine code transformation is that it also works in very limited single-threaded runtime environments without access to the callstack (like WASM).
In any case, from the user perspective, async/await should just be language syntax sugar, how it is implemented under the hood ideally shouldn't matter (e.g. it should also be possible to implement it on top of a task scheduler that runs on fibers or threads instead of a state-machine code transformation).
As an aside, there is a lot of confusion in this thread between general async operations and async/await.
[1] but of course C++ screwed it up by requiring hard to remove allocations.
My best guess is that it's because of implementation limitations in CPython and likely other interpreters. StacklessPython is a fork of CPython with real coroutines/fibers/green threads but apparently they didn't want to merge that patch. Very disappointing, because async/await is a nearly useless substitute for my desired usecase (embedded scripting languages with pauseable scripts).
Because queues backed by thread/process pools for serving web requests has sharp edges.
Not possible with the C stdlib IO functions though (that's why it would be nice to have optional async IO functions with completion callback in the C stdlib)
PS: just calling a blocking read in async/await code would have the same effect though, you need an "async/await aware" version of read()
What do you see as the main limitations of spawning 2048 threads in a pool in this scenario?
I can run millions of goroutines on a laptop. These get mapped to a relatively small number (number of available CPU cores with default settings) by the runtime.
I agree. They are if we spawn OS threads everytime we need a thread. The equivalent in async would be spawning the entire overhead of the event loop every time we need concurrency.
Obviously, we don't do that.
WorkerPools don't need respawning. Greenlets don't need respawning. Virtual Threads handled by the runtime don't need respawning.
This is an interesting case. Is it difficult to recover state in the case of an error in such a connection? If not, then you could just use that ability. If so, that seems fragile.
Also, this doesn't sound like an inherent limitation of the design approach. Couldn't the linux kernel just improve the performance of that case?
> those threads mostly do nothing but consume precious system resources.
You mean a small amount of virtual memory?
https://medium.com/young-coder/5-misconceptions-about-asynch...
And also that talk on the event loop : https://www.youtube.com/watch?v=cCOL7MC4Pl0
It's “easier” because it lies to you and makes you assume that everything is sequential, but it's not, and sometime that “everything is sequential” abstraction is leaky, and you can't really see what's going on without diving to the bottom of every functions.
I've been accustomed so much to the transparency of async/await, that I now whish we had the same kind of thing for functions using blocking syscalls (for instance you could annotate the function with the `blocking` keyword and need to use `block` to call it) so you known you need to spawn a new thread if you don't want to wait until the completion of some I/O-bound function.
Can you explain what details leak?
The sequential model was developed for programming because that's a natural way to reason about proccesses. `if then else`. `do this, then do that. The "async/await" designers seem to agree, as they attempt to tame async by emulating this behavior.
Note that to do anything other than sequential is extremely complicated to reason about, not because of computers, but because of logic/math. All sorts of concerns like: race conditions, synchronization, dead lock, etc are inherent.
Any approach that does not directly address these issues is the one that's creating a leaky abstraction.
> so you known you need to spawn a new thread if you don't want to wait until the completion.
All functions take "blocking time" to execute. It's a spectrum of how long you want to wait.
You answer half of it a few lines later:
> All sorts of concerns like: race conditions, synchronization, dead lock, etc are inherent. > Any approach that does not directly address these issues is the one that's creating a leaky abstraction.
By writing `await` you're telling your reviewers, coworkers and even your future self than your program stops executing sequentially at this step, and that other concurrent task can do things in the meantime. When using blocking code, the same thing can happen, but this is hidden from you.
But in my perspective as a back-end engineer, the biggest issue is related to latency: with annotations you know (and tell others: code is written once but read many time) what takes significant time, with threads and hidden yield point you don't. It looks sequential, but the latency is an observable behiavor that show it's not: the definition of a leaky abstraction.
> All functions take "blocking time" to execute. It's a spectrum of how long you want to wait
That's technically correct, but keep in mind that the magnitude difference between your typical REST API call and a CPU instruction is roughly the same as the difference between the size of a football field and the distance to the Sun…
I'm not convinced. I don't see how it is hidden from you. The blocking code works exactly as written. The mistaken assumption would be that your program will never be premempted or have to wait for resources.
> the magnitude difference
Rarely is a function call in C a single instruction. They are typically algorithms of non-constant complexity. So yes... but you're also picking the most extreme comparison. What about `fopen` vs `partial_sort`?
No, it really doesn't. If I make the wrong assumption that "everything is sequential", then it's not the paradigm lying to me.
Pretty much the first thing I learned about threads: I have to assume that my code can, and will, be interrupted at arbitrary points.
As long as I keep that in mind, there is very little that can surprise me. Because the other side of that coin reads: "Unless it branches into more threads of execution, each block itself will run sequentially, no matter what", which makes it very easy to reason about each block.
The rest is a matter of synchronizing these interruptions to a useful outcome, which, as stated elsewhere, CSP and modern languages integrating the primitives for that natively, make really easy.
I find it rather odd that you say "easier to reason about.". I find it much harder to keep a mental model in my head which thread currently does what compared to async/await code which you can write like synchronous code. You generally don't have to be that hyper-aware.
This can be transparently solved by the GUI library. The main thread does a loop and polls events from a queue. Those events are generated by a gui on another thread. It can be designed so gui itself can be manipulated on the mainpulated on the main thread, and the event handling and rendering is double buffered, or synchronized for you.
> easier to reason about
The style of code OP is describing looks like `if then else`. You can reason about the state of the system using traditional programming logic.
For example, you would read data fragment D0 from a network socket, and right before doing any work on it, you ask the OS to fetch data fragment D1, etc. This would is faster than reading and working in distinct time intervals. And despite whatever you seem to believe, it would also be faster than reading and working using thread parallelism. Because even if you eliminate the synchronization overhead or you devise a good lock-free algorithm, thread context switches still have a massive overhead, not to mention issues related to memory bandwidth and cache coherence.
Still, how does one gather the nerve to call a feature present in most modern languages, from C# to Zig, a hype?
Even if the above condition doesn't hold, you would have to be very careful to do better with threads.
Is it just an arbitrary design decision that NodeJS is single-threaded?
I can see this.
More mature model of that is message passing like in Erlang or in a bit more bastardized version, in Golang. And it works there because you can write "normal" code with no colored functions and other baggage that JS-like async brings with it. And it works beautifully on multi-core machines.
async is just strict, shitty subset of that where you're limited either by bad implementation (any single-process scripting language) or language limitations (no GC in Rust would make Erlang/Go-like message passing much harder).
JavaScript is older than all of those. (Libuv in particular was harvested from Node, which was itself built on top of JavaScript.)
Obviously JS didn't invent callback-based async IO, but I think you're forgetting how old JS is and how relatively new that style of IO is.
The article is about async/await, but the comments I'm replying to seem to be about asynchronous programming in general. JS was doing async for many years before async/await was added.
What changed my mind was accidentally making a (shitty/incomplete) async system while implementing a program "The Right Way" using threads and synchronization primitives. The program is for controlling an amateur telescope with a lot of equipment that could change states at any moment with a complex set of responses to those changes depending on what exactly the program is trying to accomplish at the time. Oof, that was a confusing sentence. Let's try again; The telescope has equipment like a camera, mount, guide scope, and focuser that all periodically report back to the computer. The camera might say "here's an image" after an exposure is finished, the mount might say "now we're pointing at this celestial coordinate", the focuser might say "the air temperature is now X", and the guide scope might say "We've had an error in tracking". Those pieces of equipment might say those things in response to a command, or on a fixed period, or just because it feels like it.
Controlling a telescope can be described as a set of operations. Some operations are fairly small and well contained, like taking a single long exposure. Some operations are composed of other operations, like taking a sequence of long exposures. Some operations are more like watchdogs that monitor how things are going and issue corrections or modify current operations. When taking a sequence of long exposures the program would need to issue commands to the telescope depending on which of those messages it receives from the telescope or the user; If the tracking error is too high (or the user hits a "cancel" button) we might want to cancel the current exposure. If the air temperature has changed too much we might want to refocus after the currently running exposure is finished. If the telescope moves to a new celestial coordinate we probably want to cancel the exposure sequence entirely. So, how do we manage all that state?
The way I solved it was to make a set of channels to push state changes from the telescope or user. Each active operation would be split into multiple methods for each stage of that operation, and they would return an object that held the current progress and what it needed to wait on before we could move onto the next stage. That next stage would be triggered by a controlling central method that listened for all possible state changes (including user input) and dispatch to the next appropriate method for any of the operations currently running. To make things a little simpler I made a common interface for that object that let the controlling central method know what to wait on and what to call next. This allowed me the most control over how different concurrent operations were running while staying completely thread-safe. It was great, I could even listen to multiple channels at the same time when multiple operations were happening concurrently.
At this point I realized I'd accidentally made an async system. The central controlling method is the async runtime. The common interface is a Future (in rust, or Promise in js, or Task in C#). Splitting an operation into multiple methods that all return a Future is the "await" keyword. Once I accepted my async/await future, operations that were previously split across multiple methods with custom data structures to record all of the intermediate stages evaporated and became much more clear.
I'm still using multiple threads for the problems that benefit from parallel computation, but making use of the async system in rust has made implementing new operations much easier.
That's just not true. It started because starting a thread for connection isn't scalable at all. Asynchronous programming was in use way before nodejs even in languages that have proper threads.
Sure, there are other ways to handle these things like polling state, but async makes it trivial and readable.
But this mostly applies to JS which has a kickass implementation of async/await. Whatever C++ tried to do, it's an awful mess so there I still use threads with busy loops, polling, etc., whatever makes sense for the task at hand.
The fact that JS simply has no other options for doing anything concurrently, might have something to do with that.
Some believe that async makes the code easier to reason with, especially in cases where num_threads << num_sessions. This is a very subjective engineering conversation. You could certainly make strict threads work in any situation with some DIY, but I would be near the front of the line to make an argument to at least start with async/await if there is a paying customer involved. I think handling I/O becomes a joy when you have these abstractions at your disposal.
The other angle is performance. If you are reaching for async programming out of the gate because you want to go fast, you are making an epic mistake. Unnecessary context switching between threads will chop multiple orders of magnitude off a single thread best case. Unless you are 900% sure that the cost of communicating between threads is worth the squeeze, you should stick with a single-threaded paradigm (or use async/await responsibly).
Now, you may decide that losing performance in order to leverage more "sugary" programming primitives is worthwhile. We certainly make that decision many times over throughout - Interpreted languages, GC, etc. While I could implement our webapp using a socket select server and still easily meet our performance objectives, the complexity of managing this is not worth it. Async/await would still kick my ass in terms of performance because the runtime has been so carefully tuned around it. I can beat it in latency terms, but only for a trivial # of clients.
There are two different subcultures pushing for asynchronous. You are correct, one is based on people learning to program in JS and suffering from either Stockholm syndrome or some other problem that blocks them from thinking in sequential IO.
The other grew mostly out of the 10k problem at the late 00's, and is very correct on their assessment in that asynchronous code allows for some high-performing architectures that are much better than anything you can get synchronously.
Those two groups are trying to solve different problems, with different architectures, in different parts of the software stack. Rust's async supports both of them, so the discourse is really confusing.
It's worth noting that during the 1990s, multi-threaded programming did not completely dominate as the model for network servers. I believe it was mostly due to multi-threading support was uneven across the different UNIX flavours of the day, but regardless of the cause, some popular servers (mostly notably Apache httpd) started out as multi-process based, using a pool forks the same way you'd use a thread pool. Other servers were written using the 1990s incarnation of asynchronous programming, essentially using select() or poll() (or WSAWaitForMultipleEvents on windows). From the programmer's perspective, these act mostly the same epoll, but are just less efficient.
It is during that time that the C10k problem and its asynchronous solution was experiencing its peak hype cycle that high-level languages got interested in the game, and implemented asynchronous I/O with callbacks. I believe it started with Python and Twisted, but node was the poster-child. OS-level threads were either not supported by the language (Node.js) or severely encumbered by having a GIL (Python). Green threads or coroutines would have probably been a better fit for this languages, but if you're just writing a library or a runtime for a language you don't control, that's harder (of course, gevent in Python went and manage to do that anyway).
By the time async/await came to Javascript, this wasn't part of a hype. Javascript has already widely adopted callbacks and then promises as a bottom-up, library oriented solution. Most I/O APIs were promise-based. Even if ES6 added go-like coroutines, all the APIs you had were already accepting a callback or returning a promise. You'd still had to do something like "await(myApi())" every time you're calling that API, not to mention having to introduce synchronization primitives to the language and watching code that never had to care about synchronization before break.
Async/await by itself, is not really asynchronous programming. Behind the scenes, it is implemented asynchronously (just the same as I/O in goroutines is!), but the programmer is writing code that looks linear and synchronous. The real trend nowadays is to eschew synchronous I/O and hide the complexity of asynchronous I/O behind synchronous-looking code. Explicitly asynchronous programming (like callbacks or non-awaitable promises) is just as trendy as Ruby on Rails or flip phones, that is - yeah, sure, it was fairly trendy back in the 2005.
Nowadays you've got two popular M:N thread models for running multiple synchronous tasks which perform asynchronous I/O behind the scenes: The green thread model (Go, Java's Virtual Threads) and the state machine transformation model (a.k.a. async/await). If you think the async/await model is inferior to the green thread model used by Go, that's a different story. I think each has its own pros and cons, but claiming that only async/await receives hype is untrue. The green threading model receives a fair share of its own hype ("Which color is your function"), and its usually the proponents of the green threading model who claim that their model is strictly superior while the other model has no merit at all, and not otherwise.
If you go back to Rust, Rust definitely tried the green threads model, as many people have already said. It had to abandon it. Go is not to be a full-spectrum systems language, and can get along pretty well with being garbage collected and running its own scheduler. Rust has to run on some environments and contexts where you just can't do that. Rust is also very sensitive to overhead introduced by features (that's the entire "zero-cost abstraction" theme), and it does not shy away from adding some complexity in exchange of performance. Otherwise why won't it just do away with lifetimes altogether?
While I concur the callbacks hype in JS was probably misguided (although understandable), I find it hard to believe that the decision to use async/await in Rust was based on hype.
It's partially hiding what's under the surface.It makes programs harder to reason about. And, worst, it "infects" whole programs pushing async further and further up the call-chain, as the "sync to async" boundary is not a good story in Rust.
Finally the fact that async in Rust is effectively "tokio", it means this framework also gets all over the place, leading to dependency bloat and framework-itis. It potentially makes things that could be simple and re-usable components much more complicated and coupled to tokio than they could be.
I would have rather seen the language provide a nice way of doing continuation passing and an explicit coroutine construct.
That's my hot take. Personally I try to avoid async and stick with explicit concurrency & communicating by channels, until I am in a situation where I'm provably blocked primarily on I/O in a way that it would make sense to reach for it.
But I'm also not writing web services.
I don't think this is the right way to design libraries; I think the right thing to do is to expose the state machine etc that drives the I/O and then provide async and sync entry points to the same thing. But 'async' keyword makes it too easy to just sprinkle it all the way down, and now you've imposed your lifestyle choices on your user.
Now that you mention it, this reminds me of ObjC where promises were a cool new thing and got put into tons of libs for no reason. The fad died down after a year or so.
However, it doesn't really address how you'd construct a state machine with an external tick. It's pretty common to have a state machine called at a fixed frequency. I guess to implement that (using the pending!() Macro between state actions) you'd need to implement a custom executor?
The difference between:
async fn bla() {
doWork();
waitForSecs(x);
doMoreWork();
waitForSecs(x);
lastBit();
}
fn somethingElse() {
// state is persistent
match state {
FirstState => doWork(); state = SecondState;
SecondState => doMoreWork(); state = ThirdState;
ThirdState => lastBit(); state = Done;
_ => ()
}
The benefit of the second example is that it can be completely synchronous with other parts of the system. You know that when your global tick happens, all the state transitions also happen. If each function manages their own time delays, that's not a given.
It depends how deep you want the ticks integrated with your async code. At minimum you can do:
async { loop { next_tick_time().await; do_tick(); } }
But both approaches are IMHO pointless. Async is meant to be a sugar on top of events and run code only when the events happen, not run it all the time at a fixed timestep.
select! {
() = wait_for_tick() => println!("tock"),
v = woken_thing() => println!("woke with {v}"),
}That said, I do not know if there would be an easier way to implement the state machine in the article using structured concurrency over async. Maybe that is actually one place where async would be better.
I need to look into that.
However, for those in the comments arguing that async is better for I/O-bound stuff, I heavily disagree.
I implemented a multiplexer system. You can start any number of operations you want and then multiplex them. This blocks until one operation is done, yes, but hey, you have threads, so use another to do something else if you need.
But this multiplexer allows me to decide what function gets called for each type of task that finishes. This means that the caller still controls what to do then the "future" completes.
So it's equivalent to async, but it's still synchronous. Nice.
I can have it multiplex on multiple types of things too. I actually haven't implemented asynchronous I/O with it yet; I mostly use it to multiplex on child processes.
So I agree with one of the top-level comments: async is a hype. There are better ways for most use cases.
Based on your description this is equivalent to async/await implemented with callbacks but not async/await implemented via polling.
I know it doesn't sound like a lot, but it is.
const fn = async (arg) => { ... }; // calls some RPC
const results = await Promise.all(args.map(fn))
fn main() {
tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.unwrap()
.block_on(async {
println!("Hello world");
})
}
So I would still start at https://rust-lang.github.io/async-book/01_getting_started/02... And if you want to go deeper, think about how you'd build a massive IO-bound server without async (the answer will look like async but with extra steps).
If you don't await your variable contains a future - how are you using that like e.g. an int, without a compiler error?
This means the compiler will emit a warning (can be upgraded to an error) if you forget to await a future even if it doesn't return anything.
[1]: https://doc.rust-lang.org/nightly/src/core/future/future.rs.... [2]: https://doc.rust-lang.org/nightly/src/core/result.rs.html#49...
And turning warnings into errors just encourages people to write 'let _ = ...' to get rid of the error.
> And turning warnings into errors just encourages people to write 'let _ = ...' to get rid of the error.
No? writing `let _ = make_future()` will clearly not await the future, why would you do it instead of just adding `.await` ?
Using `let _ = ...` is sometimes fine for Result if you really sure you don't care about the potential error you got but it's a no go with futures.
warning: unused implementer of `Future` that must be used
--> src/main.rs:9:5
|
9 | foo();
| ^^^^^
|
= note: futures do nothing unless you `.await` or poll them
= note: `#[warn(unused_must_use)]` on by defaultWith async however, if "await" is the default, then as soon as an async function calls another async function, it would block, completely defeating the point of async in the first place.
I guess you could flip the rules and say that within an async function async is the default and within a regular function await is the default, but actually in most languages a regular function can't call an async function directly because async needs to propagate all the way to the event loop. So you'd just have async as the default again.
My explanation sucks but if you want to go into this rabbit hole look up "stackful vs stackless coroutines".
Creating a future in Rust does not have any side effects like running the future in background. This is not JS. Creating a future is just creating an object representing future (postponed) computation. There is nothing spawned on the executor. There are no special side effects (unless you code them explicitly). It works exactly as any other function returning a value, hence why should it be syntactically different?
If you called something that returned a future but you forgot to use the returned future - how is that different from e.g. opening a file for write and forgetting to write to it or from creating a User object and discarding it immediately, forgetting to save it to a database? There isn't really a difference, and therefore all those cases are handled by `#[must_use]` warning.
Contrary, an `await` is an effectful operation. It can potentialy do a lot - block execution for arbitrary long time, switch threads, do actual computation or I/O... So I really don't understand why you want to hide this one.
Maybe the naming is confusing - because `await` does not really just `await`. It runs the future till completion. You should think about it more as if it was named `run_until_complete` (although it is still not precise, as some part of that "running" might involve waiting).
> Creating a future in Rust does not have any side effects like running the future in background. This is not JS. Creating a future is just creating an object representing future (postponed) computation. There is nothing spawned on the executor. There are no special side effects (unless you code them explicitly). It works exactly as any other function returning a value, hence why should it be syntactically different?
> Contrary, an `await` is an effectful operation. It can potentialy do a lot - block execution for arbitrary long time, switch threads, do actual computation or I/O... So I really don't understand why you want to hide this one.
> Maybe the naming is confusing - because `await` does not really just `await`. It runs the future till completion. You should think about it more as if it was named `run_until_complete` (although it is still not precise, as some part of that "running" might involve waiting).
A normal function cannot switch threads.
foo(); // executed on thread 1
doSomeIO().await;
bar(); // possibly continued on thread 2
And also being forced to read distant code to understand if given snippet is correct would be a maintainability nightmare.
I've had enough problems dealing with Java exceptions which are allowed to pop up from anywhere and are not visible in the code.
Once you do that, you do not need a call site annotation that a function can be preempted as the compiler will check it for you.
Rust is uniquely equipped to enforce these guarantees.
> In Go you have to explicitly state that a function is to run in the background via "go fn(...)".
In Rust you have to explicitly `spawn` a task to detach it from the current coroutine and make it run in background. Typically this is much more costly than not spawning and executing async function concurrently as part of the same coroutine's state machine (and Go actually doesn't give you that option at all).
> In the async/await world you can't tell by looking at a function call if it will block until its done.
foo().await(); <-- blocks
foo(); <-- doesn't block
warning: unused implementer of `futures::Future` that must be used
For similar reasons you don't want `clone()` to be implicit or rethrowing errors to be implicit (like exceptions in Java).
Awaiting implicitly would hide a potentially long and important operation. Await typically means the control is yielded back to the executor and it can switch to another task. You don't want it in a language that wants to give as much control about performance as possible to the developer. Being able to see that "this fragment of code will never be preempted" is a great thing for predictability. Rust is not Go/Java - nobody is going to celebrate achieving sub 1 ms latency here.
Additionally there are certain things you are not allowed to keep across await points, e.g. mutex guards or other stuff that's not safe to switch between threads. E.g. using a thread-local data structure across await points might break, because you could be on a different thread after await. If await was hidden, you'd likely be much more surprised when the compiler would reject some code due to "invisible" await.
> foo(); <-- doesn't block
> warning: unused implementer of `futures::Future` that must be used
> Additionally there are certain things you are not allowed to keep across await points, e.g. mutex guards or other stuff that's not safe to switch between threads. E.g. using a thread-local data structure across await points might break, because you could be on a different thread after await. If await was hidden, you'd likely be much more surprised when the compiler would reject some code due to "invisible" await.
That's fair point, but traditionally you don't use blocking functions in async contexts at all. It is fairly easy to lint for by prohibiting some inherently blocking calls eg.g std::io, although they might sneak in through some third-party dependency.
This doesn't have an easy solution because Rust is a general purpose language that allows different styles of concurrency adapted best to the situation, instead of one-size-fits-all like Golang.
Rust has means to annotate functions so maybe there will be some automation to deal with that in the future, similar to how `#[must_use]` works now. E.g. `#[blocking]` or whatever.
> I'll have to check the compiler settings.
This is with default compiler settings.
> Why couldn't the compiler clearly state the reason for the error though?
Stating the reason is probably solvable problem, but there is another problem: what if 5 layers down the call chain something suddenly introduces a potentially blocking (awaiting) operation? This would mean that some code that previously compiled now has to stop compiling even though it hasn't changed and even though none of the signatures it uses changed. I guess it would break things like separate compilation.
And again, it would be less readable than it is now. Now it is fairly simple - you don't have to look down the call chain to know that something can do await.
It would. `.await` works only inside `async` context. So if the method wasn't async at the top level, then adding `.await` somewhere down the call chain would force changing all the signatures up to now become `async`.
So you cannot just freely add `.await` at random places that don't expect it. Which is sometimes a blessing and sometimes a curse. Definitely when trying to hack a quick and dirty prototype this is a slowdown. But it is really good when you aim for low latency and predictability.
You can't know that in general. Any regular Go function could spawn a goroutine return immediately too. In JS a "blocking" function could call setImmediate(…) and return too. Even in C, a function could spawn a thread and return immediately too.
You never know at the call site whether a function will block or not, in any language.
So I think polled futures actually are closest to knowing this, since the block-or-not decision can be bubbled up to the caller. In Rust the "doesn't block" example would more likely be `runtime.spawn(foo())`, since the executor is not built into the language, so spawning asynchronously is easier when left up to the caller.
but as you point out else thread, you can still hide blocking and potentially expensive operations in any function, so not seeing await give no guarantee that the operation won't block (it only guarantees that the operation won't return to the event loop, assuming that the rust event loop is not reentrant).
Hence await doesn't really protect any useful invariant.
In Rust an async function is really just a const fn that synchronously only constructs and returns a state machine struct that implements the Future trait.
So
async fn foo(x: i32) { }
essentially desugars to
const fn foo(x: i32) -> FooFuture { FooFuture { x } }
struct FooFuture { x: i32 } // technically it's an enum modelling the state machine
impl Future for FooFuture { ... }
You have to explicitly spawn that onto a runtime or await it (i.e. combine it into the state machine that your code is already in). So that's actually really cool about how Rust handles async; that an async fn really isn't doing any magic, it just constructs a state machine and never interacts (or spawns) with a runtime at all, so it never starts running in the background, you are always in full control. And by throwing the future away, you are essentially cancelling it, there's no need to interact with any runtime either.
... and that didn't click for me until I understood that concurrency (async/await) and parallelism (multiple OS threads and processes) are different executions modes which the Tokio runtime in rust allows you combine those different modes.
Meanwhile when I read greenthreading, I think of lightweight threads like goroutines or Java's virtual threads.
Am I wrong?
Maybe it's inaccurate to say async is a type of greenthreading and better to say it's a form of application-layer context-switching comparable to greenthreading.
I feel the goal of a tool should be to make common patterns easy to represent: so "sprinkling async everywhere" only doesn't work because of some common desirable pattern not being easily representable in modern languages and gotchas of the languages.
I have a lightweight thread scheduler written in C and I communicate between threads with a lockless ringbuffer. IO threads do IO. I like the idea of event loops and libuv but I want parallelism.
I wanted to understand how to compile async/await so I wrote a multithreaded unrolled state machine in Java. Each async keyword sends to another thread. I haven't got it to send to a different thread each time to load balance, I need to spend more time on it.
task1:
handle1 = async task2();
handle2 = async task3();
handle3 = async task4();
// at this point, I want task2, task3, task4 to be going on in parallel ON DIFFERENT threads
await handle1;
await handle2;
await handle3;
thread(s) = state1(yes) | send(message) | receive(message2);
thread(r) = state1(yes) | receive(message) | send(message2);
I am coming from the perspective of states and behaviour. I have read about session types but my syntax is not inspired by them.
Types are important and useful but I am more interested in the rigid parts of code that types data flow through otherwise known as control flow. I feel it's an ignored part of computer science.
My goal is easy parallelism, asynchronocity and reactivity.
The Cilk papers in particular are very very good. They discuss the programming model, the compilation strategy, the scheduling algorithms and more.
edit: this one for example http://supertech.csail.mit.edu/papers/cilk5.pdf
This works in Rust:
let thread_s = spawn(join!(state1(yes), send(message), receive(message2));
let thread_r = spawn(join!(state1(yes), receive(message), send(message2));Honestly, working in embedded... I don't want my state machine hidden like this. I want it up front and explicit, documented, and observable. I want to be able to query components to find out what state they're in. I want to be able to keep metrics on their transitions. I want logging at the transition points. And I want the whole thing specified in code and comments, and I want to be able to see up front how it's using its resources and I want each component to describe its potential state movements. Even better if I can use declarative/formal analysis tools to check them, too (see e.g. stateright)
The risk with the async call-flow pattern is the creeping emergence of new undocumented state transitions, and potential new error states and edge conditions that come from them.
Threads are for working in parallel, async is for waiting in parallel.
async fn my_state_machine() {
set_pin_high();
defer! { tristate_pin(); }
sleep_for(Millis(100)).await;
set_pin_low();
sleep_for(Millis(100)).await;
// Pin gets tristated here
}
Surely the example code _does_ ensure that both the minimum and the maximum wait times are approximately 100ms between changes to the pin? The functions set_pin_high() and set_pin_low() are just normal functions AIUI; they are not async (are not awaited). In the previous snippets the author used a pending!() macro, but that is not used here.