What async promised and what it delivered
causality.blog
causality.blog
Not really. The author provides Go as evidence, but Go's CSP-based approach far predates the popularity of async/await. Meanwhile, Zig's approach still has function coloring, it's just that one color is "I/O function" and the other is "non-I/O function". And this isn't a problem! Function coloring is fine in many contexts, especially in languages that seek to give the user low-level control! I feel like I'm taking crazy pills every time people harp about function coloring as though it were something deplorable. It's just a bad way of talking about effect systems, which are extremely useful. And sure, if you want to have a high-level managed language like Go with an intrusive runtime, then you can build an abstraction that dynamically papers over the difference at some runtime cost (this is probably the uniformly correct choice for high-level languages, like dynamic or scripting languages (although it must be said that Go's approach to concurrency in general leaves much to be desired (I'm begging people to learn about structured concurrency))).
Java had green threads in 1997, removed them in 2000 and brought them back properly now as virtual threads.
I'm kinda glad they've sat out the async mania, with virtual threads/goroutines, the async stuff just feels like lipstick on a pig. Debugging, stacktrackes etc. are just jumbled.
Like their purpose/implementation everything is just so different, they don't share anything at all.
https://docs.rs/tokio/latest/%20tokio/runtime/struct.Handle....
I assume that answers your question.
And for my money I prefer async/await to the structured concurrency stuff..
There are also related discussions on other platforms that are worthy to read.
Certainly it’s true that Go invented neither, both Erlang and Haskell had truly parallel green threads without function coloring before Go or Node existed.
You could imagine a programming language that expressed “comptime” as a function argument of a type that is only constructible at compile-time. And one for runtime as well, and then functions that can do both can take the sum type “comptime | runtime”.
A while back I just started leaning in. I write a lot of Python at work, and anytime I have to use a library that's relies on asyncio, I just write the entire damn app as an asynchronous one. Makes function coloring a non-issue. If I'm in a situation where the two have to coexist, the async runtime gets its own thread and communication back and forth is handled at specific boundaries.
Yes, having to rewrite literally all of your code because you need to use an async function somewhere is an issue.
An even bigger issue is that now you have two (incompatible!) versions of literally every library dependency.
I was talking about when writing from scratch.
There's no "inherently synchronous structure", at least not in Javascript. The nature is synchronous, asynchronous is an illusion built on top of it. Which is why you can easily block an "asynchronous" program:
while (true) {}
JavaScript execution is synchronous on a single call stack. That's why they added Workers which is different to async.
Rust's Tokio and co are also blocking. You need threads to get something that's not an inherently synchronous with merely a facade or cooperative asychronicity.
I have to use Python as as an example since I don't have much experience with JavaScript, but when you're using asyncio, that's single threaded non-blocking IO (asynchronous). Each use of "await" yields execution back to the event loop, and it can can schedule some other task to run. Tasks can run asynchronously, but no in parallel.
When you use the multiprocess library you are actually creating new threads that run in parallel (I'm ignoring the threading library because it just muddies the water). That's also asynchronous execution.
I don't know the semantics as well in JavaScript, but I'm sure the principle is the same. At certain points you can yield to the runtime and it will schedule other pending tasks to execute while the current one is pending.
My point was that mistake I see people make (in Python) is they think of their program in blocking terms by default. So they get this frustrating coloring problem because they are trying to shoehorn in non-blocking calls. If instead you design the application from the start with asyncio in mind, it makes things much simpler.
Sure, but my comment was not about parallelism compared to asynchronous, but about the idea that Javascript is above the "blocking first mindset".
Javascript depends on a blocking (single-threaded, run-to-completion) backend. The asynchronicity on top of this blocking layer is an abstraction based on cooperative yielding.
Whereas e.g. Erlang's asynchronicity is part of the runtime model itself.
My point, coming from Python, was that whenever I can I write the entire application as an asyncio app, which cuts down on the function coloring problem. That doesn't mean you don't have to be careful, it's super easy to accidentally block the event loop.
You can make any async system synchronous. It's much harder to mske a sync sydtem asynchronous. (Misquoting from something Erlang-related).
There are many cases when I don't care if a function call is asynchronous. I'm happy to wait for the result. Yet too many systems tell me I can't, for no good reason.
> Function signatures don’t change based on how they’re scheduled, and async/await become library functions rather than language keywords.
The functions have the same calling conventions regardless of IO implementation. Functions return data and not promises, callbacks, or futures. Dependency injection is not function coloring.
The original function colouring essay was much more about JavaScript's implementation than a general statement.
If JavaScript had exposed a way for a synchronous function to call back into the runtime to wait for an async function to complete, it would still be just as coloured, but no one would be complaining about colour (deadlocks yes, but that's another kettle of fish).
EDIT: with the exception of doing IO on a freshly allocated, in-memory buffer that doesn't escape the function call.
(it's what I'm planning for my own lang, having `trace` be "blessed" to not make a function unpure despite I/O)
Okay, then by that definition, Rust doesn't have colored functions either, because `async fn foo() -> Bar {` in Rust is just syntax sugar for `fn foo() -> impl Future<Output=Bar> {` (that's a normal function that returns a future), and `Future` is just a normal trait that provides a `poll` method; there's no different calling convention anywhere. So which is it? Either Rust and Zig both have colored functions, or neither do.
Basically, any function can handle any effect, they don't need to be marked a special way like async await.
Now function colouring is interesting but not for the reason these articles get excited. Recolouring is easy and has basically no impact on code maintenance. BUT if you need that code path to really fly then marking it as async is a killer, as all those tiny little promises add tiny delays in the form of many tasks. Which add up to performance problems on hot code paths. This is particularly frustrating if functions are sometimes async, like lazy loaders or similar cache things. To get around this you can either use callbacks instead or use selective promise chaining to only use promises when you get a promise. Both strategies can be messy and trip up people who don’t understand these careful design decisions.
One other fun thing is indexeddb plays terribly with promises, as it uses a “transactions close at end of task” mechanism, making certain common patterns impossible with promises due to how they behave with the task system. Although some API designers have come up with ways around this to give you promise interfaces for databases. Normally by using callbacks internally and only doing one operation per transaction.
That depends on the language/framework. In some languages, `await foo()` is equivalent to `Future f = foo(); await f`. In others (e.g. Python), it's a primitive operation and you have to use a different syntax if you want to create a future/task. In Trio (an excellent Python alternative to asyncio), there isn't even the concept of a future at all!
Is this because the functions are async or is that because most of the time async is used for things that are I/O like and therefore susceptible to these kinds of delays?
This is a solved problem in C#. You can use ValueTask<T> instead of Task<T> and no promise will be allocated if it never awaits.
However, in embedded rust async functions are amazing! Combine it with a scheduler like rtic or embassy, and now hardware abstractions are completely taken care of. Serial port? Just two layers of abstraction and you have a DMA system that shoves bytes out UART as fast as you can create them. And your terminal thread will only occupy as much time as it needs to generate the bytes and spit them out, no spin locking or waiting for a status register to report ready.
I wish to God that the rust library devs would admit to this fact - say that async rust should stay for embedded runtimes usecases, but we shouldn’t be forcing async across the majority of general purpose computing libraries. It’s just not a pleasant experience to write nor read. And it really doesn’t give any performance benefits.
More than once, we have wanted to improve the performance of some path and been able to lift the sequential model into a stream, evaluated concurrently with some max buffer size. From there, converting to true parallel execution is just a matter of wrapping the looped futures in Tasks.
Obviously just sprinkling an async on it isn’t going to make anything faster (it just converts your function into a state-machine generator that then needs to be driven to completion). But being able to easily and progressively move code from sequential to concurrent to parallel execution makes for significant performance gains.
If you just want to build a normal backend service, you can't escape async libraries. Wrapping the async functions with `block_on` is not ideal I'd rather just have access to standard sync primitives that don't need me to bring an entire async runtime into the system.
My ultimate point is - I would be happy if async stayed in its own world. But the fact is async has completely polluted the rust library landscape and you can't escape it. I'm working on a project that I hope to show rust users that async isn't needed for performant backend services, and that the code can be written much simpler without it.
You also don’t need to spawn() futures to await them. Spawn enables parallelism on the multithreaded runtime, holding join handles, etc. If all you need is to execute concurrent code, though, the various combinators and functions in the futures crate lets you do so without having hard requirements on Send/Sync. The large majority of the concurrent code I write uses nothing specific from the tokio crate, including spawn.
As is often the case in rust, the compiler is also telling you the correct thing. If you’re using the multithreaded runtime and spawning, your code may execute in another thread, so it has to be Send/Sync, and since the ownership of the future is transferred to the executor, it must also be ‘static.
All of your arguments are just mental workarounds trying to justify how fucked the rust ecosystem is for traditional backend services.
The project I'm working on is specific to making traditional kernel threads faster (150-200 nanosecond context switches compared to 1500-2000 nano seconds for normal kernel threads). It requires a user scheduler but you can swap those out without any changes to how you write rust. In my testing, it's not only faster than async rust but also much easier to write. I hope it convinces people like you, that are hell-bent on defending the current state of async rust, that there are better paradigms and we don't have to be locked in to shitty, verbose concurrent code.
Send/Sync/static is not needed on tokio’s local runtime, which doesn’t require any adjustments to your libraries.
Passing data between threads requires Send/Sync/static, except for certain cases like scoped threads, so making OS threads faster doesn’t seem to solve that issue like using a local runtime would.
Many async libraries (though certainly not all) are runtime-independent. If your library doesn’t have to spawn, it is easy to write runtime-independent code. I would like to see some spawn traits brought into std to make it easier to write libraries that have to spawn, though.
I’ll always try new ways of doing things, but you are making the assumption that the way you feel is the way everyone feels, and totally dismissing the opinions of those who don’t. It puts me off of whatever solution you might be proposing, since you clearly don’t have the empathy to understand the full range of positions of the people whose problems you’re ostensibly trying to solve.
I’m not trying to convince you the way you feel is wrong, but you are wrong that everyone thinks writing async code is miserable. There are times where it’s hard, or where the compiler emits confusing messages about async closures being not generic enough, but on the whole I enjoy writing async rust, so shoot me.
My entire goal is to show that coding the same server with pre-async hyper vs post async hyper is nicer and more performant than async rust. I hope to show you it in just a few days.
The other advantage is a rough classification in the type system. Not marking a function as async means that the author believes it can be run in a reasonable amount of time and is safe to run eg. on a UI main thread. In that sense, the propagation through the call hierarchy is a feature, not a bug.
I can see that maintaining multiple versions of a function is annoying for library authors, but on the other hand, functions like fs.readSync shouldn’t even exist. Other code could be running on this thread, so it's not acceptable to just freeze it arbitrarily.
But in ordinary JS there just can't be a race condition, everything is single threaded.
Two async functions doing so is not a data race either.
Serially, completely synchronously overwriting values is none of these categories though.
Concurrency is needed for race conditions, parallelism is needed for data races. Many single threaded runtimes including JS have concurrency, and hence the potential for race conditions, but don't have parallelism and hence no data races.
What we may argue over (and it becomes more of a what definition to use): IO/external event loop/signal handlers. These can cause race conditions even in a single threaded program, but one may argue (this is sort of where I am) that these are then not single threaded. The kernel IO operation is most definitely not running on the same thread of execution as JS.
I think I have been fairly consistent in the definition of a data race as a type of race condition, where a specific shared memory is written to while other(s) read it with no synchronization mechanism. This can be safe (most notably OpenJDK's implementation is tear-free, no primitive or reference pointer may ever be observed as a value not explicitly set by a writer), or unsafe (c/c++/rust with unsafe, surprisingly go) where you have tearing and e.g. a pointer data race can cause the pointer to appear as a value that was never set by anyone, causing a segfault or worse.
Saying that fs.readSync shouldn't exist is really weird. Not all code written benefits from async nor even requires it. Running single threaded, sync programs is totally valid.
Maybe, but is it useful to have sync options?
You can still write single threaded programs
Sync options are useful. If everything is on the net probably less so. But if you have a couple of 1ms io ops that you want to get done asap, it's better to get them done asap.
and async prevents this how?
In a good API design, you should exposed functions that each do one thing and can easily be composed together. The 'readSync' function doesn't meet that requirement, so it's arguably not necessary - it would be better to expose two separate functions.
This was not a big issue when computers only had a single processor or if the OS relied on cooperative multi-threading to perform I/O. But these days the OS and disk can both run in parallel to your program so the requirement to block when you read is a design wart we shouldn't have to live with.
No, it tells the OS "schedule the current thread to wake up when the data read task is completed".
Having to implement that with other OS primitives is a) complex and error-prone, and b) not atomic.
Even websites had this problem with freezing the browser in the early AJAX days, when people would do a synchronous XMLHttpRequest without understanding it.
i don't see it as very useful or elegant to integrate any form for parallelism or concurrency into every imaginable api. depends on context of course. but generalized, just no. if a kind of io takes a microsecond, why bother.
Because JS is single threaded, fs.readSync will freeze the entire app. The only case where I would find that acceptable is in cli-scripts. But that could also be achieved with nodejs’ support for top-level await. There's perhaps a slight overhead from the Promise being created, but JS-Engines have so many optimizations that I don't even know if that matters. If nothing else is scheduled, awaiting a promise is functionally the same as blocking. Even in rare cases where you do want to block other scheduled events from running, you could achieve that with an explicit locking mechanism instead.
You could argue that filesystem access is fast so blocking everything is fine, but what if the file happens to be on a NAS somewhere?
To give a concrete example: computer music languages, such as SuperCollider, need concurrency to implement musical scheduling. Imagine a musical sequence where you play a note, wait N beats, play another note, etc. Often you want to play many such sequences simultaneously. Stackful coroutines provide a very elegant solution to this problem. Every independent musical sequence can be modelled by a coroutine that yields everytime it needs to wait. The yielded value is interpreted by the scheduler as a delta time after which the function should be resumed. In this sense, SuperCollider users have been doing async programming since the early 2000s, long before it became mainstream.
It's typically less than a hundred kilobytes and (on the systems I've benchmarked using std::thread) it takes 60usec (wall time in userspace) to create and destroy a thread.
Threads have gotten so fast that paying the async function coloring price makes very little sense for most software.
There is an extremely small set of domains where simple threading doesn't suffice, and async/await is too high a price to pay across the entire software ecosystem just to slightly optimize those domains.
I agree with this 100%.
If you're getting paid to bin pack jobs that do lots of concurrent I/O into a server infrastructure, then yes, building these complex async "machines" and forcing everyone to do the extra labor to use them is necessary to avoid a lot of waste.
But for everyone else, it's a huge waste of time. Even low end modern embedded Linux systems are capable of running thousands of threads.
Synchronous code is always simpler to write, easier to reason about, more straightforward to test, and faster to debug. Nobody should ever reach for async by default.
Why is reserving a megabyte of stack space "expensive"?
> and takes roughly a millisecond to create
I'm not sure where this number is from, it seems off by a few orders of magnitude. On Linux, thread creation is closer to 10 microseconds.
Guess it's not a huge issue in these 64-bit days, but back in the 32-bit days it was a real limitation to how many threads you could spin up due to the limited address space.
Of course most applications which hit this would override the 1MB default.
Because if you use one thread for each of your 10,000 idle sockets you will use 10GB to do nothing.
So you'll want to use a better architecture such as a thread pool.
And if you want your better architecture to be generic and ergonomic, you'll end up with async or green threads.
You don't pay for stack space you don't use unless you disable overcommit. And if you disable overcommit on modern linux the machine will very quickly stop functioning.
It’s a programming model that has some really risky drawbacks.
Operating systems can shrink the memory usage of a stack.
madvise(page, size, MADV_DONTNEED);
Linux also supports mremap, which is essentially a kernel version of realloc. Supports growing and shrinking memory mappings.
stack = mremap(stack, old_size, old_size / 2, MREMAP_MAYMOVE, 0);
They sure shouldn't exist to the unused region of the stack though; if they do, that's a bug (because anything could claim that memory now). You should be free and clear to release stack pages past your current stack pointer.
The correct tool for myriad of small tasks is coroutines / green threads / async tasks, so why spend any energy optimizing threads for that purpose instead of what they are already good at?
Coroutines / green threads / async tasks will all do this too, but there's something to be said for using/relying on the system scheduler instead of bringing your own in in addition.
In C it's impossible to even get the stack pointer without dropping to assembly or using compiler builtins. It's hard to know where the stack starts or even how big it is.
1.On a system that is handling 10k concurrent requests, the 10GB of RAM is going to be a fraction of what is installed.
2. It's not 10GB of RAM anyway, it's 10GB of address space. It still only gets faulted into real RAM when it gets used.
My example (and the c10k problem) is 10k concurrent connections, not 10k concurrent requests.
> 2. It's not 10GB of RAM anyway, it's 10GB of address space. It still only gets faulted into real RAM when it gets used.
Yes, and that's both memory and cpu usage that isn't needed when using a better concurrency model. That's why no high-performance server software use a huge amount of threads, and many use the reactor pattern.
No, it literally is not. The "memory" is just entries in a page table in the kernel and MMU. It shouldn't worry you at all.
Nor is the CPU used by the kernel to manage those threads going to be necessarily less efficient than someone's handrolled async runtime. In fact given it gets more eyes... likely more.
The sole argument I can see is just avoiding a handful of syscalls and excessive crossing of the kernel<->userspace brain blood barrier too much.
Only if you never free one of those stacks. TLB flushes can be quite expensive.
I've written massively concurrent systems where each connection only handled maybe a few kilobytes of data.
Async io is a massive win in those situations.
This describes many rest endpoints. Fetch a few rows from a DB, return some JSON.
They also say:
>The system spends time managing threads that could be better spent doing useful work.
What do they think the async runtime in their language is doing? It's literally doing the same thing the kernel would be doing. There's nothing that intrinsically makes scheduling 10k couroutines in userspace more efficient than the kernel scheduling 10k threads. Context switches are really only expensive when the switch is happening between different processes, the overhead of a context switch on a CPU between two threads in the same process is very small (and they're not free when done in userspace anyway).
There are advantages to doing scheduling in the kernel and there are advantages to doing scheduling in userspace, but this article doesn't really touch on any of the actual pros and cons here, it just assumes that userspace scheduling is automatically more efficient.
I feel like we're now, what, 20, 25 years on and people still haven't adjusted themselves to the fact that the machines we have now are multicore, have boatloads of cache, or how that cache is shared (or not) between cores.
Nor is there apparently a real understanding of the difference between VSS and RSS.
Nor of the fact that modern machines are really really fast if you can keep stuff in cache. And so you really should be focused on how you can make that happen.
So much so that they'll sign themselves up for async frameworks that thread steal at will and bounce things all over cores causing cache line bouncing and associated memory stalls, not understanding what this is doing to their performance profile.
And endure complexity, etc. through awkward async call chains and function colouring.
Most people's applications would be totally fine just spawning OS threads and using them without fear and dropping into a futex when waiting on I/O; or using the kernel's own async completion frameworks. The OS scheduler is highly efficient, and it is very good at managing multiple cores and even being aware asymmetrical CPU hierarchies, etc.. Likely more efficient than half the async runtimes out there.
Further, the benches that are showing best there are non-thread-stealing scenarios, not tokio.
I also suspect simply tuning the thread-based workloads more aggressively would have the same effect.
When I profile high throughput tokio applications there's way too much contention on shared atomics, mostly inside tokio's scheduler itself. On lower core count machines and where the workload is I/O heavy, this is probably fine. So, yes, web servers.
But I'm very interested in applications that scale on machines with lots of cores and where CPU is a large part of the equation.
> Likely more efficient than half the async runtimes out there.
The benchmark shows the opposite: 2 (multithreaded async runtime) vs 7 (threads) * 10^8 ns per request for 2k requests/s.
> non-linear scaling problems.
oh, look closely, the relative gap increases with #requests/s
Your level of self-assured snark is way out of proportion to both the article and chart I'm looking at.
Props to the person who wrote the article. Your use of it however, is dubious.
Equally, if a megabyte of stack is a lot for your usecase, can't you just ask pthreads to reserve less? I believe it goes down to like 16k
The program could use one page's worth of stack space, which is optimal. The program could use like 200 bytes of stack space, which wastes the rest of the page. The program could recurse all the way to 9.9 MB of stack usage, stop just before overflow and then unwind back to constant 200 bytes stack space usage, and never touch all those pages ever again.
While virtual memory allocation does not require physical allocation, it immediately runs into the kinds of performance problems that huge pages are designed to solve. On modern systems, you can burn up most of your virtual address space via casual indifference to how it maps to physical memory and the TLB space it consumes. Spinning up thousands of stacks is kind of a pathological case here.
10µs is an eternity for high-performance software architectures. That is also around the same order of magnitude as disk access with modern NVMe. An enormous amount of effort goes into avoiding blocking on NVMe disk access with that latency for good reason. 10µs is not remotely below the noise floor in terms of performance.
Because they wrote “thread per task” which I assume to mean something like “each os thread handles the work submitted by one user”.
This is beside the point but, something like io_uring is still significantly better than doing threadpool nvme io.
Async precisely improves disk/network I/O-bound applications because synchronous code has to waste a whole thread sitting around waiting for an I/O response (each with its own stack memory and scheduler overhead), and in something like an application server there will be many incoming requests doing so in parallel. Cancellation is also easier with async
CPU-bound code would not benefit because the CPU is already busy, and async adds overhead
See e.g. https://learn.microsoft.com/en-us/aspnet/web-forms/overview/... and https://learn.microsoft.com/en-us/aspnet/web-forms/overview/...
Assuming for the sake of argument that they are together, you're still saving stack memory for every thread that isn't created. In fact you could say it allows the CPU to be idle, by spending less time context switching. On top of that, async/await is a perfect fit for OS overlapped I/O mechanisms for similar reasons, namely not requiring a separate blocking thread for every pending I/O (see e.g. https://en.wikipedia.org/wiki/Overlapped_I/O, https://stackoverflow.com/a/5283082)
So avoiding that would mean a server can handle more traffic before running into limits based on thread count.
I'll publish my results shortly. I did these as baselines because I'm testing finishing the User Managed Concurrency Groups proposal to the linux kernel which is an extension to provide faster kernel threads (which beat both of them)
I'm pretty close to being done - I'm hoping to publish the entire GitHub repository with tests for the community to validate by next week.
UMCG is essentially an open source version of Google Fibers, which is their internal extension to the linux core for "light weight" threads. It requires you to build a user space scheduler, but that allows you to create different types of schedulers. I can not remember which scheduler showed ^ results but I have at least 6 different UMCG schedulers I was testing.
So essentially you get the benefits of something like tokio where you can have different types of schedulers optimized for different use cases, but the power of kernel threads which means easy cancellation, easy programming (at least in rust). It's still a linux thread with an entire 8mb(?) stack size, but from my testing it's far faster than what Tokio can provide, without the headache of async/await programming.
Using async for languages like Rust or C++ is cargo cult by people who don't know what the hell they're doing.
[Caveat: there's a use case for async if you're doing embedded development where you don't have threads or call stacks at all.]
The UMCG implementation allows kernel thread context switches to happen in 150-200 microseconds, compared to the 1500-2000 microseconds for normal kernel thread context switches. My goal is to show that if UMCG could be merged into the Linux run time then then it would be competitive with async rust without the headache.
I mean, I suppose we could move the scheduling and tracking out of kernel mode and into user mode...
But then guess what we've just reinvented?
Honestly, it's a mostly obsolete approach. OS threads are fast. We have lots of cores. The cost of bouncing around on the same core and losing L1 cache coherence is higher than the cost of firing up a new OS thread that could land on a new core.
The kernel scheduler gets tuned. Language specific async runtimes are unlikely to see so many eyeballs.
EDIT: Something else to consider is what if your REST calls needs to make 5 queries. Do you serialize them? Now your latency can be worse. Do you launch a thread per query? Now you need to a) synchornize b) take x5 the thread cost. Async patterns or green threads or coroutines enable more efficient overlapping of operations and potentially better concurrency (though a server that handles lots of concurrent requests may already have "enough" concurrency anyways).
There’s a benefit in being able to code the handling of a request in synchronous logic. A case has to be made for the particular application that it would cause performance or resource issues, before opting for asynchronous code that adds more complexity.
I totally agree there are applications for which this is overkill and adds complexity. It's just a tool in the toolbox. Video games famously are just a single thread/main loop kind of application.
Why does async make it harder to reason about when resources are released?
The execution flows of individual async tasks are still linear, much like individual threads are linear.
Scheduling (tasks by the async runtime vs threads by the OS), however results in random execution order either way.
If there is a slow resource, both, async tasks as well as threads will pile potentially increasing response times.
Wether async or threads, you can easily put a concurrency limit on resources using e.g. semaphores [1]:
- limit yourself to x connections (either wait or return an error)
- limit the resource to x concurrent usages (either wait until other users leave, or return an error)
Regarding blocking the main loop: with async and non-blocking operations, how would something block the main loop? And why would the main loop being blocked cause API calls being timing out? Is it single threaded?
[1]: https://docs.rs/tokio/latest/tokio/sync/struct.Semaphore.htm...
Think about what happens:
1. Request one hits an await in foo()
2. Runtime switches to request two in bar() until it awaits
3. Runtime switches to request three in baaz(), which blocks the loop for a while
4. Request one gets a socket timeout or expired API key
That error in #4 does not tell you anything about #2 or #3, and because execution spreads across everything in that process you have to check everything. If it was a thread, you would either not have the problem at all, it would show up clearly in request three, or you’d have a clear informative failure on a synchronization primitive saying that #3 held a lock for too long.
That makes it harder to control when memory is allocated or released in garbage collected languages, too, because you have to be very careful to trigger gc before doing something which can suspend execution for a while or you’ll get odd patterns when a small but non-zero percentage of those async requests take longer than expected (i.e. load image master, create derivative, send response needs care to release the first two steps before the last or you’ll have weird behavior when a slow client takes 5 minutes to finish transferring that response).
Arguably that’s something you want to do anyway but it dramatically undercuts the simplicity benefits of async code. I’m not saying that we should all give up async but there are definitely some pitfalls which many people stumble into.
No such thing. In a preemptive multitasking OS (that's basically all of them today) you will get context switching regardless of what you do. Most modern OS's don't even give you the tools to mess with the scheduler at all; the scheduler knows best.
Linux kernel uses 8k stacks (TBH, it's been a while), but there's also some copy-on-write overhead. Still, this is not the C10k problem...
Libraries like Tokio (mentioned in the article) have support for this built-in. Goroutines sidestep the issue completely. C# Tasks are batteries included in that regard. In fact function colors aren't an issue in most languages that have async/await. JavaScript is the odd one out, mostly due to being single-threaded. Can't really be made to work in a clean way in existing JS engines.
Take Rust: you return `Result<T,E>`, you are coloring your function the same way as you are when using `async`. Same for Option. Errors as return values in Go: again, function coloring.
One of your nested function starts taking a "serverUrl" input parameter instead of reading an environment variable: you've colored your function and you now need to color the entire call stack (taking the url parameter themselves).
All of them are exactly as annoying, as you need to rewrite the entire call stack's function signature to accommodate for the change, but somehow people obsess about async in particular as if it was something special.
It's not special, it's just the reflection that something can either be explicit and require changing many function signatures at once when making a change, or be implicit (with threads, exceptions or global variables) which is less work, but less explicit in the code, and often more brittle.
And result and option usually mean something else. Option is a value or none. None doesn't necessarily means the function failed. Result is the value or an error message. You can have result<option, error>
That's different then async where you can call the other type.
Async functions are colored because they force a change in the rest of the call stack, not just the caller. If you have a function nested ten levels deep and it calls a function that returns a Result, and you change that function to no longer return a result because it lost all its error cases, you only have to change the direct callers. If you are ten layers deep in a stack of synchronous functions and suddenly need to make an asynchronous call, the type signature of every individual function in the stack has to change.
You might say "well, if I'm ten layers deep in stack of functions that don't return errors and have to make a call that returns the error, well now I have to change the entire stack of functions to return the error", but that's not true. The type change from sync to async is forced. The error is not. You could just discard it. You could handle it somehow in one of the intervening calls and terminate the propagation of the type signature changes half way up. The caller might log the error and then fail to propogate it upwards for any number of reasons. You aren't being forced to this change by the type system. You may be forced to change by the rest of the software engineering situation, but that's not a "color".
For similar reasons, the article is incorrect about Go's "context.Context" being a coloration. It's just a function parameter like anything else. If you're ten layers deep into non-Context-using code and you need to call a function that takes a context, you can just pass it one with context.Background() that does nothing context-relevant. You may, for other software engineering reasons, choose to poke that use of a context up the stack to the rest of the functions. It's probably a good idea. But you're not being forced to by the type system.
"Coloration" is when you have a change to a function that doesn't just change the way it interacts with the functions that directly call it. It's when the changes forcibly propagate up the entire call stack. Not just when it may be a good idea for other reasons but when the language forces the changes.
It is not, in the maximally general sense, limited to async. It's just that sync/async is the only such color that most languages in common use expose.
Threading methodology is unrelated though. How exactly the call stack is scheduled is orthogonal to the question of whether or not making a call to a particular function results in type changes being forced on all function in the entire stack.
There may also be cases where you can take "async" code and run it entirely out of the context of any sort of sceduler, where it can simply be turned into the obvious sync code. While that does decolor the resulting call (or, if you prefer, recolor it back into the "sync" color) it doesn't mean that async is not generally a color in code where that is not an option. Solving concurrency by simply turning it off certainly has a time and place (e.g., a shell script may be perfectly happen to run "async" code completely synchronously because it may be able to guarantee nothing will ever happen concurrently), but that doesn't make the coloration problem go away when that is not an option.
Here's the list of requirements: 1. Every function has a color. 2. The way you call a function depends on its color. 3. You can only call a red function from within another red function. 4. Red functions are more painful to call. 5. Some core library functions are red.
You are complaining about point 3. You are saying if there's any way to call a red function from a blue function then it's not real. The type change from sync to async is not forced any more than changing T to Result<T,E>. You just get a Promise from the async function. So you logically think that async is not a color. You think even a Haskell IO-value can be used in a pure function if you don't actually do the IO or if you use unsafePerformIO. This is nonsense. Anything that makes the function hard to use can be color.
I have inside information about how that is not the case since I typed, but deleted before posting, some points about how Haskell is one of the few languages that can create arbitrary numbers of colors in libraries, but it started spiraling when I started trying to characterize when a new color was created. It is, for instance, not as simple as "it's monadic", because types that implement "monad" that allow extracting of the value like List or Maybe don't create colors even though IO does, so I just deleted it, especially when my thoughts turned to trying to explain how Haskell is also one of the few languages that can abstract over color. (Although I gather zig is giving it the college try with sync/async.) Using monad as an example is just an invitation to trouble since way more people think they understand it than actually do.
A function simply taking an annoying amount of parameters is not a color problem, because the entire essence of "color" is the transitivity. If you don't have the transitivity flowing up the call stack, you don't have color. Just being "hard to use" is not sufficient... the proof of which being that we've had "hard to call" functions for many more decades than we've had widespread "color" in our functions and nobody was talking about this transitivity problem until we started having async functions. That is a new problem... I mean, new on the relevant time frames... the whole color thing has been ongoing for years now, but it's still relatively new.
well, this isn't really true - at least for Rust:
runtime.block_on(async{});
https://docs.rs/tokio/latest/tokio/runtime/struct.Handle.htm...
This is a language specific problem, not a language pattern one.
I am not doing that. The caller (which is the only one being blocked here) is sync anyways and just wants to call an async function, so no async scheduler is blocked.
So I don't buy that async colors are fundamentally different.
Like in Haskell there is the IO monad used to denote the IO effect. And there are unsafe ways to actually execute it - does that make everything in Haskell impure?
As soon as you start using function arguments instead of using a global variable, you are coloring your function in the exact same way. Yet I don't think anyone would make the case that we should stop using function arguments and use global variables instead…
Effects are another way of making functions incompatible, for better or worse. It can be done badly. Java fell into that trap with checked exceptions. They meant well, but it resulted in fragmentation.
Sometimes it’s worth making an effort to make functions more compatible by standardizing types. By convention, all functions in Go that return an error use the same type. It gives you less information about what errors can actually happen, but that means the implementation of a function can be modified to return a new error without breaking callers.
Another example is standardizing on a string type. There are multiple ways strings can be implemented, but standardization is more important.
Or if you mean that returning a new error breaks API compatibility, then yes that's the point. If now you can error in a different way, your users now need to handle that. But if it's all generic and inferred, it can still just bubble up to wherever they want to do that with no changes to middle layers.
In this way, declaring a type guides people to write calling code that doesn't break, provided you set it up that way. It makes things easier for the implementation to change.
Sometimes you do need handlers for specific errors, but in Go you always need to write generic error handling, too.
(A type variable can do something similar. It forces the implementation to be generic because the type isn't known, or is only partially known.)
Go also basically requires you to write actually generic error code (if err != nil return err), so it feels like errors are more work than they are. In Scala (especially ZIO) propagation is all automatic and you just handle it wherever you'd like. The only code that cares about errors is the part generating them and the part handling them. But it's all tracked in the type system so you can always see what exact errors are possible from what methods. And it's all simple return values so no trickiness with exceptions (e.g. across async boundaries).
Throwing the baby out with the water like what go sort of does with its error handling is no solution. The proper solution is a better type system (e.g. a result type with a generic handles what go can't).
For effects though, we need a type systems that support these - but it's only available in research languages so far. You can actually just be generic in effects (e.g. an fmap function applying a lambda to a list could just "copy" the effect of the lambda to the whole function - this can be properly written down and enforced by the compiler)
That isn't function colouring, but rather plain incompatible APIs/runtime. You could have the equivalent with non-async ecosystems.
The issue arose because Javascript didn't have threads, and processing events from the DOM is naturally event driven. To be fair, it's a rare person who can deal with the concurrency issues threads introduce, but the separate stacks threads provide a huge boon. They allow you to turn event driven code into sequential code.
window.on_keydown(foo);
// Somewhere far away
function foo(char_event) { process_the_character(char_event.key_pressed) };
while (char = read())
process_the_character(char);
The history of Javascript described in the article is just a series of attempts to replace the horror of event driven code with something that looks like the sequential code found in a normal program. At any step in that sequence, the language could have introduced green threads and the job would have been done. And it would have been done without new syntax and without function colouring. But if you keep refining the original hacks they were using in the early days and don't the somewhat drastic stop of introducing a new concept to solve the problem (separate stacks), you end up where they did - at async and await. Mind you, async and await to create a separate stack of sorts - but it's implemented as a chain objects malloc'ed on the heap instead the much more efficient stack structure.
I can see how the javascript community fell into that trap - it's the boiling frog scenario. But Python? Python already had threads - and had the examples of Go and Erlang to show how well then worked compared to async / await. And as for Rust - that's beyond inexplicable. Rust has green threads in the early days and abandoned them in favour of async / await. Granted the original green thread implementation needed a bit of refinement - making every low level choose between event driven and blocking on every invocation was a mistake. Rust now has a green thread implementation that fixes that mistake, which demonstrates it wasn't that hard to do. Yet they didn't do it at the time.
It sounds like Zig with its pluggable I/O interface finally got it right - they injected I/O as a dependency injected at compile time. No "coloured" async keywords and compiler monomorphises the right code. Every library using I/O only has to be written once - what a novel concept! It's a pity it didn't happen in Rust.
while (char = read())
process_the_character(char);
element.onkeydown = window.greenThread(processKeyEvents, ...);
...
function processKeyEvents(green_thread, ...) {
while (event = green_thread.yield(...)) {
/* process the key event */
var foo = call_some_function_that_blocks(...);
/* more processing */
}
}
Code is effectively still coloured, because a call that can block can't be called from a context that doesn't allow block (like DOM events). async makes this colouring obvious but it comes at the expense of new keywords, new concepts like promises and futures, and manually creating the stack using linked object on the heap. A real stack is far faster than allocating stuff on the heap. But to repeat the key observation above: async and green threads are semantically near identical, it is only the implementation is different.
One selling point used for async is it makes blocking calls obvious, and that would somehow avoid bugs. The claim (willfully?) ignored that green threads (aka, cooperative multiasking) were the earliest form of multitasking - they have been around for over 50 years at this point. They were never a source of the sort of bugs they claimed would happen.
There are a bunch of use cases for it outside of implementing concurrency in a single threaded runtime.
Pretty much every GUI toolkit I've ever used was single threaded event loop/GUI updates.
Green threads are a very controversial design choice that even JVM backed out of.
Did they? Project Loom has stabilized around Java 21, no?
Not sure if inspired by it, but async/await is just like Haskells do-notation, except specialized for one type: Promise/Future. A bit of a shame. Do-notation works for so many more types.
- for lists, it behaves like list-comprehensions.
- for Maybes it behaves like optional chaining.
- and much more...
All other languages pile on extra syntax sugar for that. It's really beautiful that such seemingly unrelated concepts have a common core.
It's interesting to wonder about the C# world where those things were more unified.
It's also interesting to explore in C# all the existing ways that Linq syntax can be used to work with arbitrary monads and also Task<T> can be abused to use async/await syntax for arbitrary monads. (In JS, it is even easier to bend async/await to arbitrary monads given the rules of a "thenable" are real simple.)
I tried once to hack list comprehensions into JS by abusing async/await. You can monkey patch `then` onto Array and define it as flatMap and IIRC you can indeed await arrays that way, but the outer async function always returns a regular Promise. You can't force it to return an instance of the patched Array type.
Having been instrumental in accelerating bringing async/await to JS, it definitely was the case that it came from C# and we eagerly were awaiting its arrival in JS and worked with Igalia to focus on it and make it happen across browsers more quickly so people could actually depend on it.
That's a mischaraterization. They were abandoned because having green threads introduces non-trivial runtime. It means Rust can't run on egzotic architectures.
> It sounds like Zig with its pluggable I/O interface finally got it right
That remains to be seen. It looks good, with emphasis on looks. Who knows what interesting design constraints and limitation that entails.
Looking at comptime, which is touted as Zig's mega feature, it does come at expense of a more strictly typed system.
There was a fair bit of time between the two, to the point I'm not sure the latter can be called much of a strong motivation for the former. Green threads were removed pre-1.0 by the end of 2014 [0], while work on async/await proper started around 2017/2018 [1].
In addition, I think the decision to remove green threads might be less inexplicable than you might otherwise expect if you consider how Rust's chosen niche changed pre-1.0. Off the top of my head no obligatory runtime and no FFI/embeddability penalties are the big ones.
> Rust now has a green thread implementation that fixes that mistake
As part of the runtime/stdlib or as a third-party library?
No, you appear to have no idea what you're talking about here. Rust abandoned green threads for good reason, and no, the problems were not minor but fundamental, and had to do with C interoperability, which Go sacrifices upon the altar (which is a fine choice to make in the context of Go, but not in the context of Rust). And no, Rust does not today have a green thread implementation. Furthermore, Rust's async design is dramatically different from Javascript, while it certainly supports typical back-end networking uses it's designed to be suitable for embedded contexts/freestanding contexts to enable concurrency even on systems where threads do not exist, of which the Embassy executor is a realization: https://embassy.dev/
The interoperability problems came from a design choice they made: they wanted to invisibly swap between OS threads if a blocking call was made. That was the wrong design choice for Rust. It is not something green threads require - it's an additional burden they imposed on themselves.
> a fine choice to make in the context of Go,
Correct. It was.
> enable concurrency even on systems where threads do not exist,
And that was their mistake. Green threads are just user space stacks, which any CPU that has a stack pointer can support. They don't need OS threads. But in the initial Rust implementation they added "invisibly supports blocking calls" implementation. That does indeed need full OS thread support, and breaks the C interface.
They could have just abandoned that mistake, implemented pure green threads and they would have been done. But no, they instead they implemented async. So instead we got new keywords, had to wait years for various new language features to stabilise (like pin) and lifetime 'static. I'm sure the language designers spent many happy man months solving the problem "How do we represent a suspended function as a type-safe State Machine?".
What we got for all that effort is something so famously difficult to use, even experienced Rust programmer shy away from async. Whereas with green threads a programmer could just reuse their knowledge and mechanisms Rust has for sharing data across multiple stacks, for async they have now to battle the borrow checker on a new front - the language to absorb their function state into 'static area that has fewer lifetime guarantees. The alternate green thread implementation could have had just two colours - blocking and non-blocking, now we have a colour for every async library. Whereas with green threads they could have stabilised the interface to underlying the event loop, we are now stuck with every library writer having their I/O in their own mini API, which they swap for each async implementation.
And while the type-safe State Machine they did come up with is an engineering marvel, it's very complex and requires data copies under the hood. Green threads just reuse an existing mechanism for saving a function's state - the CPU's IP Address, the registers and the stack. It's so simple anyone can understand it, it isn't something additional to learn because the program's main already uses it, it is very, very fast because it's undergone decades of refinement, and it's also type safe!
But instead in what we got the syntax is a mess, the error messages are incomprehensible, and we re-implemented the CPU's stack management in software (invariably slower) because they didn't want to write the logic to grow a hardware stack.
The language will be better off recognising it was all a huge mistake, and moving towards standardising on a green thread implementation like mioco. Everyone who has to manage 100,000 lightweight processes would be immensely grateful for the simpler, faster API. Some of the truly impressive stuff done for async, like knowing how much stack some function calls can take, would be really useful for green threads. You know the size of the stack you need - to the byte! So it's not a complete loss.
From what I understand the interop problems are more fundamental than that. The Rust devs wanted zero-cost FFI, but FFI for green threads would necessarily involve moving data between the green thread stack and the C stack, which is not zero-cost. Furthermore, the Rust devs wanted "zero-knowlege" FFI, in that calling code wouldn't need to know it's calling into or being called from Rust. That means that you can't assume whatever is on the other side of the FFI boundary knows how to switch/grow/otherwise handle green thread stacks, and inserting shims to do so would also not be zero-cost.
> and we re-implemented the CPU's stack management in software (invariably slower) because they didn't want to write the logic to grow a hardware stack.
If I'm understanding you correctly, part of the problem is that Rust didn't have a good mechanism to grow stacks. Segmented stacks were abandoned because of their allocations and performance cliffs, and stack copying a la Go is not feasible due to the inability to update pointers to the new larger stack. Rust ended up just using larger stacks for its green threads which kind of defeats a major reason to use green threads in the first place.
But for a long time (I think even till today despite that there is as an optional free-threaded build) CPython used Global Interpreter Lock (GIL) which paradoxically makes the programs run slower when more threads are used. It's a bad idea to allow to share all the data structure across threads in high level safe programming languages.
JS's solution is much better, it has worker threads with message passing mechanisms (copying data with structuredClone) and shared array buffers (plain integer arrays) with atomic operation support. This is one of the reasons why JavaScript hasn't suffered the performance penalty as much as Python has.
The job wouldn’t have been done. They would have needed threads. And mutexes. And spin locks. And atomics. And semaphores. And message queues. And - in my opinion - the result would have been a much worse language.
Multithreaded code is often much harder to reason about than async code, because threads can interleave executions and threads can be preempted anywhere. Async - on the other hand - makes context switching explicit. Because JS is fundamentally single threaded, straight code (without any awaits) is guaranteed to run uninterrupted by other concurrent tasks. So you don’t need mutexes, semaphores or atomics. And no need to worry about almost all the threading bugs you get if you aren’t really careful with that stuff. (Or all the performance pitfalls, of which there are many.)
Just thinking about mutexes and semaphores gives me cold sweats. I’m glad JS went with async await. It works extremely well. Once you get it, it’s very easy to reason about. Much easier than threads.
You can't always choose to write straight code. What you're trying to do may require IO, and then that introduces concurrency, and the need for mutual exclusion or notification.
Examples: If there's a read-through cache, the cache needs some sort of lock inside of it. An async webserver might have a message queue.
The converse is also true. I've been writing some multithreaded code recently, and I don't want to or need to deal with mutexes, so, I use other patterns instead, like thread locals.
Now, for sure the async equivalents look and behave a lot better than the threaded ones. The Promise static methods (any, all, race, etc) are particularly useful. But, you could implement that for threads. I believe that this convenience difference is more due to modernity, of the threading model being, what 40, 50, 60 years old, and given a clean-ish slate to build a new model, modern language designers did better.
But it raises the idea: if we rethought OS-level preemptible concurrency today (don't call it threads!), could we modernize it and do better even than async?
I've been programming for 30 years, including over a decade in JS. You need sync primitives in JS sometimes, but they're trivial to write in javascript because the code is run single threaded and there's no preemption.
> What you're trying to do may require IO
Its usually possible to factor your code in a way that separates business logic and IO. Then you can make your business logic all completely synchronous.
Interleaving IO and logic is a code smell.
> The Promise static methods (any, all, race, etc) are particularly useful. But, you could implement that for threads. I believe that this convenience difference is more due to modernity, of the threading model being, what 40, 50, 60 years old, and given a clean-ish slate to build a new model, modern language designers did better.
Then why don't see any better designs amongst modern languages?
New languages have an opportunity to add newer, better threading primitives. Yet, its almost always the same stuff: Atomics, mutexes and semaphores. Even Rust uses the same primitives, just with a borrow checker this time. Arguably message passing (erlang, go) is better. But Go still has shared mutable memory and mutexes in its sync library.
> But it raises the idea: if we rethought OS-level preemptible concurrency today (don't call it threads!), could we modernize it and do better even than async?
I'd love to see some thought put into this. Threading doesn't seem like a winner to me.
> Multithreaded code is often much harder to reason about than async code, because threads can interleave executions and threads can be preempted anywhere.
No, green threads / fibres or whatever you want to call them explicitly don't interleave executions. They are a form of cooperative multitasking. Async/await is another form of co-operative multitasking. One former just builds on what we already have. The latter re-invents the universe.
By the by, the blocker for Javascript green threads wasn't preemption, mostly because there isn't any. It's that Javascript has a "run to completion" model. If the DOM calls a javascript event (which is effectively how all javascript is invoked in a browser), it doesn't block, so it always runs to completion. Green threads break that model. It's not a insurmountable break - the DOM events could always still return immediately, but they could start a green thread that returns to them as soon as they block. Thinking about it, the change is possibly smaller than language changes required by async/await.
If you can reason about where an await is, you can reason about where a green thread yields. The only difference is that one of them clutters your syntax and the other doesn't.
If you use them with a multithreaded executor (eg in Go), of course they interleave executions. I suppose all your green threads / fibers could run on a single CPU core. But what's the point? How would that be an improvement over what we have now?
I suppose you could make something similar to async/await but with a yield() operation whenever a call wants to block. This would allow blocking read() and so on. But its basically async/await but without declaring functions as async. And without needing to explicitly await. Await points would be implicit and invisible. But if you do that, any function call you make could yield before returning. As a result, you could no longer easily reason about interleaving. I call foo(). Does it yield to other threads before returning? I have no idea. I could read the code of foo(), but maybe foo will change between minor versions of the library.
This would lead to an avalanche of bugs. Lots of javascript code quietly depends on the lack of interleaving for correctness. Javascript guarantees that while my (non async) function runs, no other code gets executed. Adding threads, even if its via cooperative multitasking, would break that invariant. It would break all sorts of programs which are working correctly today.
> The latter re-invents the universe. [...] Thinking about it, the change is possibly smaller than language changes required by async/await.
Did you write much javascript before async/await and before promises? Javascript at the time was already async. We just implemented async execution through callbacks. ("Callback hell"). Over time, functions tended to go down and to the right. Promises were added as 3rd party libraries. Then promises were standardised. And later, async/await was added as syntax to help you work with promises. Async / await in javascript was an incremental change to give us new syntax to do what we were already doing. JS already had an event loop and promises. Async/await just added syntax.
Threads (cooperative or preemptive) would be a massive change to JS. It would cause an endless parade of bugs, and frozen websites. To say nothing of your notion we could casually reinvent DOM events. That ship sailed a long time ago.
> The only difference is that one of them clutters your syntax and the other doesn't.
One of them is explicit about where and when a thread blocks. Whether or not something is "blocking" (async) is part of the API. Threading (incl cooperative threading) hides this information. Personally, I much prefer this information to be explicit. I need to know as a programmer whether or not execution will be interleaved.
yes.
> But what's the point? How would that be an improvement over what we have now?
> But its basically async/await but without declaring functions as async.
You answered your own question: yes, you get what you have now, without all the overhead of async, await, promises and futures.
> But if you do that, any function call you make could yield before returning.
A green thread could be an instance of a particular type, so `input = self.yield()` would fail if you aren't a green thread. So no, not "any function" - just ones that instances of a green thread, or are passed a reference to one.
> Does it yield to other threads before returning?
It could if you pass it an instance to a green thread, otherwise it can't.
> This would lead to an avalanche of bugs.
It doesn't. Cooperative multitasking is at least 1/2 a century old at this point. The bugs you're imagining will happen mostly aren't an issue. To the extent they do happen, it's because someone hasn't thought about two control flows modifying the same data structure. Yes, that happens, but it happens in all single threaded code - async included. It's why we hate side effects. It's what Rust famously prevents with its borrow checker even in the face of side effects. It's not avoided by async. The explicit colouring does not help to prevent it - it's just overhead.
FWIW the one issue cooperative multitasking does often introduce is that they can take a long time to execute, so other cooperative tasks don't run in a timely fashion. Exactly the same thing can happen with async of course. It's not usually a problem in browsers, but in embedded solutions where cooperative multitasking is commonly used, it's a real issue because they are often real time. Ask me how I know.
> Javascript guarantees that while my (non async) function runs, no other code gets executed.
This remains true. You are getting confused by your mental model of threads as a form of concurrency. There is no concurrency going on there. Semantically it is near identical to async / await. The principle difference is in async / await, the program is explicitly creating each stack frame on the heap using manually allocated objects. In addition to the mental overhead that creates, it slower than using a real stack like green threads do. But now for the truly bizarre twist. Can you guess how modern javascript engines get around that speed issue? Wait for it .... they create an explicit stack ... that looks like what green threads would use anyway! And as a wonderful side effect - you get real stack back traces again. The irony is almost palpable. https://v8.dev/blog/fast-async
> Threads (cooperative or preemptive) would be a massive change to JS. It would cause an endless parade of bugs, and frozen websites. To say nothing of your notion we could casually reinvent DOM events. That ship sailed a long time ago.
I agree the ship has sailed at this point. The rest of the assertions you make there are wrong.
This assertion stands out: frozen websites. Can you tell me how they are going to block? There are no blocking calls in javascript now. The things you would await on now would be passed a green thread handle. But the javascript scripts events called from the DOM have no green-thread handle, so they can't block.
> Personally, I much prefer this information to be explicit. I need to know as a programmer whether or not execution will be interleaved.
You don't. You've just been conditioned to think that because you've never done it any other way. But the reality is people have been using cooperative multitasking for a long, long time. It pre-dates threads and async. The issues and bugs you are proclaiming would happen don't arise.
Good on the V8 team. Sounds like they’ve figured out a way to get the performance of green threads with the better ergonomics of effects systems (async await). Great!
You sound like an expert in cooperative multithreading. If async await can use real stacks, what actual benefits are there to cooperative multithreading? Why prefer them over what JS has now? Pitch them to me.
Oh, right. As you said, the ship has sailed. I think you could bolt green threads onto javascript now without ill effects - apart from bloating the language. I can't see anything that could go badly (certainly no avalanche of bugs). But in javascript green threads are only mildly more ergonomic than async. I wouldn't be bloating the language for such a small return.
Rust is a different position. The current async implementation has two big black hairs. Firstly, they had to come up with a type-safe way saving the functions current state. By state, I mean what a function normally stores on its stack. What they came up with is a work of art in some ways, but it doesn't work well with the borrow checker. The borrow checker insists you prove that you have exclusive use of a variable while it exists. Things on the stack have a limited lifetime (the function call), so the compiler knows they don't exist for very long. Even with that small lifetime it's a battle, but it's workable. Async persists that state, usually to the heap, which can effectively live forever. That wreaks havoc with the borrow checker, causing comments like this: https://news.ycombinator.com/item?id=37436274, quote: "Yes, async is effectively a much harder version of Rust ...".
The second issue is colouring. In the current Rust async implementation of large chunks of it is left to libraries, like tokio. Each of these libraries has to provide their own I/O. They aren't compatible. So if you want to use a cute new HTTP server, you are out of luck unless they provided a version that talks to the async library you are using.
The library writers do their best to accommodate by providing interfaces to the popular async libraries. That forces them to do a extra work. Whereas before they could just call `std::file::File::read()`, now they have to abstract all the I/O they do to a different module, and provide an implementation of that module for each async library they want to support.
The outcome can only be described as a mess, and that's putting it politely. It's harming uptake of the language. It wasn't like they didn't know it was coming either - there were comments pleading for a better implementation. And it wasn't as if weren't better solutions weren't already apparent - they had green threads before, they made some wrong turns with its implementation that needed to be fixed. And it's not like these solutions were harder to do than the async implementation they came up with. Async needed new standard library features to stabilise (like `Pin<>`) and introduced new keywords - none of which was needed for green threads. (Although some would be useful for an efficient green thread implementation - like knowing the maximum amount of stack a function could use.)
In the face of all that, they persisted with async. You'd need a sociologist to explain how that happened - to my engineering brain it's inexplicable. Unlike Javascript it isn't just mildly ergonomic implementation of the same thing, it's a serious mistake - well worth the effort of throwing out and replacing.
- You can't name the type of a impl Future.
- They play terribly with the borrow checker because the borrow checker can't handle self referential types.
- There's no future executor in the standard library. You need 3rd party libraries. And the most common library is tokio, which is a whale.
- Despite all the work, there's still no async streams in the language.
- Pin. !Unpin. pin_project. Unsafe pin_project. What are we even doing.
But async works really well in javascript. Maybe where we disagree is that I don't think any of these issues are because async itself is a bad idea. But, async has become the place dreams go to die in rust. Look at the issues above. They're all problems with rust's type system, borrow checker and standard library.
What I think rust needs is:
- A way to have self-borrows in a struct. Types with self borrows would be implicitly pinned.
- A way to name the return value of a function. Eg let x: ReturnType<some_func>. People have been saying this is right around the corner since 2019.
- Generators. Futures are built on top of generators inside the compiler. But generators have - for some reason - never been exposed in stable rust. I think generators should have been stabilised first - since all the problems you need to solve to make generators work well (self referential types, return values you can name, etc) are things futures need too.
Unfortunately I think that ship has sailed too. I try to avoid async rust whenever I can. Its such a pity. I'm hoping someone makes a rust 2.0 language at some point which fixes this situation.
Generators are an interesting case. For example, if you implemented a Vec iterator as a generator, it becomes:
fn vec_iter(&self) {
for index in 0..self.len() {
yield &self[i];
}
}
fn next(&self) {
if (self.index >= self.vec.len()) {
None
} else {
self.index += 1;
&self.vec[self.index - 1]
}
}
However ... the async implemented all the mechanics needed to get rid of that green thread stack allocation when the size of the stack is known, as it is in this case. The state saving stuff they created for async could be used to translate that stack to a type. It would, surprise, surprise, contain just `index` - analogous the iterator type we have to manually create for event drive code. So compiler could translate the green thread to the same implementation as the event driven code, but you get to use the compact (and very familiar) syntax of a stack machine.
I found it interesting to see what happens for a more complex generator - like something that returns every node in a tree. You can do it recursively, which is simple clear code, but you don't know the size of the stack so the trick used for the vec iterator (translating it to a type) can't be used. Or you can manually store the state you stored in the stack with a recursive implementation in a Vec<> instead. Both require a memory allocation, but they are different. One is just normal malloc that must be reallocated and moved as the allocation grows. The other can use the OS's stack implementation, that doesn't move as it grows. If you re-used stacks, the OS's stack implementation would be faster in a long running program.
Notice that the transformation from a generator to async implementation is arguably more complex than the same transformation for green threads, especially for the tree traversal.
That observation is one of the reasons I'm such a strong proponent of green threads. The other is a simpler mental model. Unlike async, you don't have to expose the inner mechanisms it depends on, like futures.
As I understand it, the implementation of async in the rust compiler grew out of the implementation for generators in nightly. Its the same continuation-passing transformation that lets you implement both await and yield in your fictional example.
> Notice that the transformation from a generator to async implementation is arguably more complex than the same transformation for green threads, especially for the tree traversal.
Yeah for sure. Another nice thing about green threads is that the compiler doesn't need to invert the call stack. I suspect you'd get smaller binaries in many cases. A lot of the complexity of async in rust comes from moving stack variables into a hidden struct as part of this transformation. For example, this function:
async fn foo() -> impl Future {
let x = 5;
let y = &x;
await someexpr();
// ...
}
enum FooFuture {
AwaitPoint1 { x: usize, y: &'a usize }
}
> That observation is one of the reasons I'm such a strong proponent of green threads. The other is a simpler mental model. Unlike async, you don't have to expose the inner mechanisms it depends on, like futures.
Fair. But as I said earlier in this thread, I like the mechanism (futures) are exposed. I like that "async" is part of a function signature. I like that you need to be explicit about which functions yield, when, and where. I want programming languages to have more effect systems - for example, it would be great to have a nopanic effect. I just ... find it much easier to enjoy async in javascript.
If you use a thread per connection (or green threads like Go), you don't also need async. If you have async (eg nodejs), you can get great performance without threads. You're right that they can also be combined - either within a single process (like tokio in rust). Or via multi-process configurations (eg one nodejs instance per core, all behind nginx). But they don't need to be. Go (green threads) and Nodejs (async, single threads) both work well.
Of course we're comparing them. We all want to know who wore it better
> The job wouldn’t have been done. They would have needed threads. And mutexes. And spin locks. And atomics. And semaphores. And message queues. And - in my opinion - the result would have been a much worse language.
My point is that you do need mutexes, spin locks, etc with async as well, given that you have a multi threaded platform. So no, we have basically 2x2 stuff we are talking about with very different properties (async/sync x single/multi threaded).
No, Javascript isn't a shared-memory multi threaded platform. It would only need mutexes and so on if we added threads to javascript, as the comment I was replying to suggested should have happened:
> At any step in that sequence, the language could have introduced green threads and the job would have been done
You can call async methods without immediately calling await. You can naively await as late as possible. They'll run in parallel, or at least how ever the call was configured.
In Javascript, promises are eager and start executing immediately. They return control back to the caller when they need to wait. So in practice, all of your promises are running concurrently as soon as you create them.
In Rust, futures are lazy don't start executing until they are awaited. You have to use various features of your chosen runtime to run multiple futures concurrently (functions like `spawn` or `select`). But that interface isn't standardized and leads to the the ecosystem fragmentation issue discussed in the article. There was an attempt to standardize the interface in the `futures` crate, but none of the major runtimes actually implement the interface.
https://docs.oracle.com/en/java/javase/21/core/virtual-threa...
Almost. JavaScript adopted async because it was a programming language designed to slot into someone else's event loop. Other programming languages, at least on the server, that needed lightweight threading didn't bother with any of this, they just shipped their own managed stacks. But UI code practically demands to own its own event loop and requires everything else live as callbacks inside of it. And JavaScript, because it was designed to live in a browser, inherited these same semantics.
I understand that some devs don’t want to learn async programming. It’s unintuitive and hard to learn.
On the other hand I feel like saying “go bloody learn async, it’s awesome and massively rewarding”.
Funny, because it was supposed to be more intuitive than handling concurrency manually.
I find it interesting how in software, I repeatedly hear people saying "I should not have to learn, it should all be intuitive". In every other field, it is a given that experts are experts because they learned first.
I never thought "programming languages are a failure, because they are not intuitive to people who don't know how to program".
My point being that I don't judge a tool by how intuitive it is to use when I don't know how to use it. I judge a tool by how useful it is when I know how to use it.
Obviously factoring in the time it took to learn it (if it takes 10 years to master a hammer, probably it's not a good hammer), but if you're fine with programming, state machines and message passing, I doubt that it will take you weeks to understand how async works. Took me less than a few hours to start using them productively.
Other fields don't have the same ability to produce unlimited incidental complexity, and therefore not the same need to rein it in. But I don't think there's any field which (as a whole) doesn't value simplicity.
Now if you take the chainsaw without spending a second thinking about learning to use it, and start using it like a manual saw... no doubt you will find it worse, but that's the wrong way to approach a chainsaw.
And I am not saying that async is "strictly better" than all the alternatives (in many situations the chainsaw is inferior to alternatives). I am saying that it is a tool. In some situations, I find it easier to express what I want with async. In others, I find alternatives better. At the end of the day, I am the professional choosing which tool I use for the job.
But concurrency is hard and there's so much you syntax can do about it.
After you’ve learned the paradigm and bedded it down with practice.
If you come from callbacks it is (almost) purely an upgrade, from threads is it more mixed.
At least in JS. I don’t find async in rust anywhere near as nice to use. But that’s a separate conversation.
It forces programmers to learn completely different ways of doing things, makes the code harder to understand and reason about, purely in order to get better performance.
Which is exactly the wrong thing for language designers to do. Their goal should be to find better ways to get those performance gains.
And the designers of Go and Java did just that.
Technically, promises/futures already did that in all of the mentioned languages. Async/await helped make it more user friendly, but the complexity was already there long before async/await arrived
If I want sequential execution, I just call functions like in the synchronous case and append .await. If I want parallel and/or concurrent execution, I spawn futures instead of threads and .await them. If I want to use locks across await points, I use async locks, anything else?
If you don't need the performance and scalability then it is not unreasonable to argue that async isn't worth the engineering effort.
I didn't coin that term, the Oxide folks did: https://rfd.shared.oxide.computer/rfd/0609. I want to emphasize that I don't think futurelocks represent a "fundamental mistake" or anything like that in Rust's async model. Instead, I believe they can be fixed reliably with a combination of some new lint rules and some replacement helper functions and macros that play nicely with the lints. The one part of async Rust that I think will need somewhat painful changes is Stream/AsyncIterator (https://github.com/rust-lang/rust/issues/79024#issuecomment-...), but those aren't yet stable, so hopefully some transition pain is tolerable there.
> The pattern scales poorly beyond small examples. In a real application with dozens of async calls, determining which operations are independent and can be parallelized requires the programmer to manually analyze dependencies and restructure the code accordingly.
I think Rust is in an interesting position here. On the one hand, running things concurrently absolutely does take deliberate effort on the programmer's part. (As it does with threads or goroutines.) But on the other hand, we have the borrow checker and its strict aliasing rules watching our back when we do choose to put in that effort. Writing any sort of Rust program comes with cognitive overhead to keep the aliasing and mutation details straight. But since we pay that overhead either way (for better or worse), the additional complexity of making things parallel or concurrent is actually a lot less.
> At the function level, adding a single i/o call to a previously synchronous function changes its signature, its return type, and its calling convention. Every caller must be updated, and their callers must be updated.
This is part of the original function coloring story in JS ("you can only call a red function from within another red function") that I think gets over-applied to other languages. You absolutely can call an async function from a regular function in Rust, by spinning up a runtime and using `block_on` or similar. You can also call a regular function from an async function by using `spawn_blocking` or similar. It's not wonderful style to cross back and forth across that boundary all the time, and it's not free either. (Tokio can also get mad at you if you nest runtimes within one another on the same thread.) But in general you don't need to refactor your whole codebase the first time you run into a mismatch here.
Don't get me started on how you need to move "blocking" work to separate thread pools, including any work that has the potential to take some CPU time, not even necessarily IO. I get it, but it's another significant papercut, and your tail latency can be destroyed if you missed even one CPU-bound algorithm.
These may have been the right choices for Rust specifically, but they impair quality of life way too much in the course of normal work. A few years ago, I had hope this would all trend down, but instead it seems to have asymptoted to a miserable plateau.
what bother me the most, is that aside from async, I _really_ do like the language and appreciate what its trying to do. otherwise I would just turn away from the whole mess. this just landed really badly.
This isn't true at all. The Send+Sync+'static is basically the limitation of tokio::spawn
https://emschwartz.me/async-rust-can-be-a-pleasure-to-work-w...
Change the executor, and the bound changes.
> Async ruined Rust for me, even though I write exactly the kind of highly concurrent servers to which it's supposed to be perfectly suited. It degrades API
It refers to async in Rust, and everyone else is responding as if it is talking about async in Rust. That's a mischaracterization. You don't have to use the Tokio executor.
It's a bit like saying, "Graphics in Rust are ruined. I need to use Vulkan to make my game."
No, you don't. If your use case is unique enough, use something else. There are bindings for OpenGL, DirectX, etc.
- Hardware interrupts
- Threads
- Thread pools / Rayon
- GPU (Cudarc, FFI, WGPU etc)
- SIMD
- Write firmware for multiple cores
But yes, once you go dining on other people's crates you definitely get the impression that you have to, because tokio gets its fingerprints all over everything.
But also there are non-thread stealing runtimes that don't require Send/Sync on the Future. Just nobody uses them.
Because Rust is the language that tokio ate.
Which ones ?
It uses N:M threading model - where N virtual threads are mapped to M system threads and its all hidden away from you.
All the other languages just leak their abstractions to you, java quietly doesn't.
Sure, java is kinda ugly language, you can use a different JVM language, all good.
Don't get me wrong, love python, rust, dart etc, but JVM is nice for this.
This whole thing is basically snake oil. The best thing backend can do instead is have dedicated thread pool where each real thread has its own queue of limited size. Each element in queue would contain input and output state of request and code to deal with those. Once queue grows over certain size the backend should simply immediately return error code (too busy). Much more sound strategy in my opinion.
There are more complex cases of course (like computationally expensive requests with no io that take long time). Handling those would require some extra logic. Async stuff however will not help here either
Java has this too.
My primary inspiration for the concept is theorem proving languages like Lean in which typeclasses ("interfaces" in the OOP terminology) are implemented using structures passed down as arguments ("vtables" in the OOP terminology) separately from any receivers (values of the type implementing the interface, which doesn't actually need to exist for Lean). Typeclasses (and interfaces) are an effect, albeit a simple and limited one. Lean can't express effects in their generality due to totality requirements, but the same mechanism would work perfectly well for effects too. As for the "vtable" aspect: the primary distinction in implementing typeclasses using exposed vtable passing is that the language does not in any way limit the programmer to zero or one implementations of a typeclass per receiver type(s) (cf. orphan rules in Rust, cf. compiling effect systems to witness-passing, etc.).
1. It makes asynchronous programming look synchronous. I do not like things being other than they appear. The point was touched on with the:
getOrders(user.id),
getRecommendations(user.id)
2. On a related issue CPU bound code can block the thread of execution and stop any concurrency in its tracks
In Rust there is the added problem of shoehorning it into the memory model which has lead to a lot of hairy code and tortured paradigms (e.g. pin)
you skip the function coloring.
Virtual threads in JVM are similar.
I play around with real time audio, and use state machine/event loop. A very powerful, if verbose, method to do real-time programming, I cannot see how asyc/await could achieve the same ends
My trite slop bashing was days ago:
Sounds like the second-chance pool (dang's description at https://news.ycombinator.com/item?id=26998308)
The next decade will be a proliferation of hackers having fun with io_uring coming up with all sorts of patterns.
Solved at the OS layer, sure. This article is about solving it at the language layer.
It's been solved for a long time at the OS layer. I mean, didn't VAX have something like overlapped IO in 1980? Unix had asynchronous IO as well with `select`.
It's 2026 and I'm starting to hate the internet.
Sure, it comes with its own issues like large stacks (10k copy of nearly the same stack?) and I predict a memory coloring in the future (stack variables or even whole frames that can opt out from being copied across virtual threads).