Let futures be futures
without.boats
without.boats
Everything else about “why can’t I mix these functions” is a direct result of that broken consistency exposed by the OS. As such I’m not sure there is a zero-cost abstraction (stackful coroutines is close) that any programming language can build to improve the situation.
Meanwhile, the trend is clear - every OS has adopted (and is adopting more) non-blocking syscalls because they are direly needed. The only benefit blocking syscalls offer is sugar to improve syscall ergonomics.
I think if more people talked about it this way it would become clear that adding a “blocking” tag to the syscalls and bubble that tag up the call stack is the right next step to depreciating those legacy OS APIs. I don’t mean to say we should accept poor ergonomics but adding the blocking tag is a great first step to 1. Reminding people of the problem. 2. Identifying areas where research is needed to improve ergonomics and replace the existing “blocking” tag with minimal downsides.
Edit to add: I haven't personally done much work with async Rust yet, but the thing that appeals to me about async Rust is intra-task concurrency. So I'm not just trying to counterbalance the complainers and make you feel better; I really believe that Rust's futures and async/await syntax were not a mistake.
I also like the point that maybe(async) isn't terribly compelling. Recently, I've been looking at building an async parser, and the sync version of the interface is pretty simple: fn parse(r: &mut dyn Read) -> Result<ParsedObj> [1]. In theory, I could slap an async on it, change Read to AsyncRead, and now I have an async version. Except it's not usefully async: the consumer of the data can't do anything until everything has been fully read in. So what you want in the asynchronous version of the API is something that looks like a stream of parse events. But that API is way too much faff for anyone trying to use the API synchronously. Or is it? If you have a high-level method that distilled the stream of output events into a simple, easy-to-use answer, then you end up with something that looks maybe(async)-able [2].
Where I think the author gets it wrong, though, is in asserting that it's statefulness that causes maybe(async) to break down. State per se isn't the problem: our underlying parser above is statefully turning a stream of input data into a stream of parse events, and this could sometimes work well with maybe(async). The problem arises when you're multiplexing many streams at once. In synchronous code, there's just no way to do that kind of multiplexing (as pointed out earlier in the blog post), while it's a pretty key design feature of async code. The trouble really comes in when you have a system whose outer interface is a 1:1 stream interface, but which internally needs to go through a 1:N piece and an N:1 piece. It looks like it could be maybe(async)-able from the outside, but its implementation is hopelessly not maybe(async)-able.
[1] Honestly, a lot of parsers go a step further and just read in the input buffer as a &[u8] and rely on zero-copy, but that interface turns out to be even less useful in an async context.
[2] But the faff remains if you can't resort to one of the pre-canned methods. How much of an issue that ends up being is left to the reader to decide.
This is not uncommon in big-world games. I hit it in my metaverse client. It's a standard Windows feature. Windows file systems have "FindFirstChangeNotificationA", which lets a process monitor a collection of files for modifications. That has scaling issues for large numbers of files, but it's the same problem. Similar ideas exist in the database world. You'd like to be able to ask "tell me when the results of this SELECT change". That's been played with as a concept, but didn't go mainstream.
In this area you can turn a manageable special case into a really hard generic problem. This is useful if you need a thesis topic, less useful if you need a working system.
To be honest, the system that we designed wasn't that dissimilar from promises. It was a dynamic collection of callbacks (using built-in C# events), not unlike then/catch delegates. I've worked on it about 10-12 years ago, exactly when JS world had discussions on what flavour of promises was the best, and it was one of the main sources of inspiration.
* Object wants some asset.
* Asset might be in memory in a cache. If so, satisfy request from cache.
* If asset is not in cache, queue up request, possibly on a priority queue that gets re-prioritized. Merge requests for same asset.
* When request reaches front of queue, see if the things that wanted it still want it. Remove dead requests. Discard entire request if all requestors no longer need it.
* Lock against two asset fetches for same asset being performed simultaneously.
* Fetch and preprocess asset
* Check again that asset is still wanted.
* Deliver asset
* Unlock against two asset fetches for same asset.
This is almost generic, except that once level of detail becomes involved, it gets less generic.
Does anyone have some thoughts on this? I just lack the knowledge to really evaluate that sort of take. There's a lot of discussions on whether async Rust is "good" or "bad", or the right abstraction or not, but it's pretty clear that (at this point) we're more less stuck with it (for better or worse), but that's not what I am asking about.
What I am asking is whether a proper effect system would have genuinely, and legitimately improved or significantly simplified the whole state of Rust async or not, and how. A lot of the async Rust problems sound like "hard" problems, and I'm a bit skeptical that a feature like that would really reduce the "heavy lifting" you have to do.
Discussed most recently @ https://www.abubalay.com/blog/2024/01/14/rust-effect-lowerin... / https://news.ycombinator.com/item?id=39005780 It's a viable approach if you can take the time to get it right. Async rust started out with more of a limited, MVP approach, where they first did the simplest thing that would surely work, and then went on from there.
It does sound very appealing, but I'm not sure there's enough prior art to steal from, even now, but especially not when Rust was hatching. It would probably be too big a gamble. I'm checking out Koka in that space, but it's early days.
This seems... a bit specific? Was it a reference to a specific new language or was that more of a poignant wish?
Not very new, seeing as Erlang was open sourced by Ericsson in 1998. (The language itself is from 1986 but was proprietary up to that point.)
> You might go do that, in a less than optimal world.
Eh bien! mon cher Pangloss, lui dit Candide, quand vous avez été pendu, disséqué, roué de coups, et que vous avez ramé aux galères, avez-vous toujours pensé que tout allait le mieux du monde? Je suis toujours de mon premier sentiment, répondit Pangloss; car enfin je suis philosophe; il ne me convient pas de me dédire, Leibnitz ne pouvant pas avoir tort, et l’harmonie préétablie étant d’ailleurs la plus belle chose du monde, aussi bien que le plein et la matière subtile.
"I have always abided by my first opinion," answered Pangloss; "for, after all, I am a philosopher, and it would not become me to retract my sentiments; especially as Leibnitz could not be in the wrong: and that preestablished harmony is the finest thing in the world, as well as a plenum and the materia subtilis."
If you're a language/platform like Swift or Rust (see also Erlang or OCaml) you're gonna end up with some kind of userspace task scheduler because OSes don't give you enough control over threads. If you're a systems language you're gonna use stackless coroutines because speed/memory and C interop. In Rust, you use their task scheduler via async/await, but sadly the ergonomics aren't wonderful and there's now weirdo problems like what reactor/engine/whatever they standardize on etc. The interesting and good stuff will be built on top of async/await, probably shouldn't have touted it as the interface.
My hypothesis here is that something like concurrency/parallelism exists on two levels. You've gotta get the underlying implementation right, and you have to expose a mental model and affordances for using it. The problem is that languages that are good at the first thing are usually real bad at the second thing. More broadly I think this is a trilemma--pick any 2 for your language:
- small
- can implement performant concurrency/parallelism
- can express a high-level mental model of concurrency/parallelism
No one should be surprised that Rust will throw "small" overboard here. I think the only question is how many models will they support? Will they do CSP AND actors (they're different!)? Will there be some kind of OpenMP type thing? Will there be new keywords or--shudder--new SYNTAX?
---
I feel like I have to say Go's solution to this is bold and elegant. Giving up straightforward C interop was pretty gutsy, and the affordances for working with its task scheduling system are familiar and intuitive. I want to stress this (because TFA goes way out of its way to be a little shitty w/ Go): this worked absolutely great. Tons of good software is written in Go. It is super effective. It spent its innovation tokens on experience and tooling and that was really successful (i.e. type systems aren't the only way to fight data races). There were exactly zero debates about stuff like await syntax or what engine to use/standardize. In the time it took Rust to ship async/await, probably 7 billion actually useful Go programs were shipped. Just because someone else's values aren't yours doesn't mean they're the wrong values. I think it's cool that people like Boats are being really thoughtful about how to do this and sharing their thoughts and processes out in the open. I just think there's room in this crazy world for at least two points of view, and that we could do with a little less sneering.
Rust builds support for all of these models based on it's basic underlying "shared ^ mutable" and "Send/Sync" safety guarantees. It should be quite clear based on this whether you're sharing access to some read-only data, staying within a single-threaded context with mutability, or relying on some sort of synchronization primitive (be it atomic data, locks, r/w mutexes etc). The fact that we understand some uses of these facilities as involving patterns such as "CSP" or "Actors" is quite beside the point.
> We should aspire not to simplify the system by hiding the differences between futures and threads, but instead to find the right set of APIs and language features that build on the affordances of futures to make more kinds of engineering achievable than before.
The stuff you listed, along with async/await, makes this possible. Isn't that what TFA is saying?
So every time you call foo(); you anticipate that it might resolve immediately, or it might take some time, but it won't block any other function (unless it actually does some cpu bound operation)
And any time foo(); does something asynchronous, all of its callers, and its callers callers, become implicitly asynchronous too.
Of course you would have to have some primitive for when you actually want to do several things concurrently within the scope of a single function rather than blocking that function, but that doesn't sound too bad and in JavaScript you effectively need to use Promise.all most of the time anyway.
I'm sure there's some major downside I'm missing -- but what is it?
Even is your language run time is fully async, it's really hard to do anything useful without touching C/C++/the kernel/system libraries, and as soon as you do that all those guarantees go out the window, because you cannot rewrite the world in $newlang.
Where this breaks is with FFI: if you cannot intercept blocking calls in foreign functions, you block useful threads or even deadlock. This is what the quote in the article is about:
> The cost for the native compatibility with the C/system runtime is the “function coloring” problem.
func yield[G AnyGenericEvenInterface](promise G) G {...}
I'd have preferred the language expand yield() to take an argument which yields until the ((promise)d) variable is resolved to a value. This makes it more clear that the work is being passed off to runtime and will wait for that value.
I'm still somewhat confused that promise constructions don't have any well defined start time https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... They could start instantly or defer entirely until waited on by any of the resolution methods. It seems more complex than light threads / goroutines synchronized by communications queues and other first class language objects.
Indeed, async v. sync should be a matter of the application and the application only.
I don't really understand this point.
In a hypothetical world where
(a) threads are cheap, and
(b) threads support some form of cancellation (with an ecosystem that expects this and is built around it),
then you don't need "intra-task concurrency" as a fundamental primitive. You can achieve the exact same semantics by just having a helper function that, given N closures representing work to perform, spawns a new thread to run each one, and waits for any/all of them to finish, while making sure to cancel all those threads if it itself is cancelled. (Or if you want to manually manage future objects for more control, you can do that too. But a future just represents work being done on a thread.)
Compare to Go. Go has cheap threads. Boats criticizes Go's "threads, locks and channels", but the fact that you have to interact with those primitives directly, rather than using a helper function as described, is really a limitation of Go's type system, not an inherent result of basing concurrency on cheap threads.
With one exception: Go threads do not support cancellation. Cancellation is extremely important! Without it, you have to manually pass down timeouts and other forms of cancellation into each function that performs I/O, and it becomes impossible to write that kind of generic helper function.
And of course, Rust has to fit into an existing thread ecosystem that does not meaningfully support cancellation, and does not make threads cheap.
So the tradeoff for Rust remains. Still, I don't see this anywhere near the way boats sees it. In my ideal world, if I didn't care about performance or compatibility, I wouldn't want "a language in which all functions are coroutines, meaning that all functions can yield." I don't think yielding needs to be visible to the user in any meaningful way; why does it need to be? It can be a hidden implementation detail, just like it is with green threads or OS threads. But I'd add on a concept of cancellation.
…In short, I think my ideal model is something like Trio, but I haven't had the opportunity to actually use Trio. I hope that changes.
(edit: and I'm looking forward to when boats does respond to the Trio blog post.)
As an aside, when folks mention algebraic effects in statically typed languages, delimited continuations and effects are equally powerful (possibly identical?). IMO effects seem to be about type safety and static typing, which does allow some neat optimizations.
> I don't think yielding needs to be visible to the user in any meaningful way; why does it need to be?
It's an interesting question. Users definitely want coroutines they can yield themselves - this is was iteration is all about, for example, and there are other similar patterns (the entry API is ultimately a kind of coroutine).
At the same time, the highest performance low level interfaces for asynchronous IO are ultimately just asynchronous iterators & asynchronous sinks over shared memory ring buffers, io-uring is the most famous example but there are others like AF_XDP.
I think you're right that a kind of cancellable virtual thread ultimately would have analogous affordances to a future. But a future is just a specific class of coroutine (that yields a unit type), so if you will have the coroutine mechanism for the rest of the system, why would you not extend it to include IO instead of adding a separate mechanism to erase the fact that IO is happening? What you also get from this is that if a function yields the bottom type you know it doesn't perform IO.
Can anyone (withoutboats?) elaborate on this? If “stackful” coroutines means functions with an ordinary-ish stack that can yield, then those stacks are just memory, and you can take references to them. Heck, you can do this, slowly and painfully, in C, with setcontext. And you can mostly do FFI with a degree of care about stack sizes. You can even yield across FFI boundaries. In some sense, the coroutine objects implicitly created by Rust and C++ are really just stacks minus the hardware stack pointer part.
Also, I found this amusing as a kernel programmer:
> the implementations of each of these things are completely different. The code that runs when you spawn an async task is nothing like spawning a thread, and the definition and implementation (for example) of an async lock is very different from a blocking lock: usually they will use an atomics-based lock under the hood with the addition of a queue of tasks that are waiting for the lock. Instead of blocking the thread, they put this task into that queue and yield; when the lock is freed, they wake the first task in the queue to allow it to take the lock again.
When blocking a thread, Linux (and most conventional operating systems) put the thread into a queue, mark the thread as non-runnable, and yield, and when the thing they’re waiting for is ready, they mark one or more queued threads runnable so the executor, I mean scheduler, will run them so they can try again :)
Most OSes do this a lot more slowly than can be done in userspace.
See http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136... for an in-depth analysis. The short version is that stackful fibers turn out to be a terrible choice. The author (Gor Nishanov, for the C++ ISO committee) explicitly says that they should not be used - this is even in spite of the fact that Windows API explicitly supports fibers as part of the programming interface, at a far deeper level than just POSIX setcontext.
Call stacks are just a data structure like any other. That they happen to be managed by the compiler to store things like local variables doesn't change this. Consider that there are algorithmic ways to convert from function calls to this data structure (defunctionalization) and back (refunctionaliztion), and the correspondence between recursive programs and iterative programs using an explicit stack.
The question here is, for any given stack, what requirements does it place on the memory layout and allocation for that stack? Gor's analysis covers a few of these, but not everyone has the same requirements and not every implementation style makes the same tradeoffs.
And you can mix and match these styles within a single programming language! Rust async functions and C++ coroutines both still store much of their local state on the native C stack- they only use their state machines to store local state (including the state machines of their callees) that lives across suspension points. This is what enables Rust to support APIs like spawn_blocking or block_on- you can run different parts of a program using different styles of stacks.
Call stacks are a part of the system API/ABI; your fancy ersatz stacks not so much. This is kind of a key consideration in the above analysis. The whole point of having Windows API support for fibers was to try and obviate that basic pain point, but ultimately this has been shown not to be very successful.
> Rust async functions and C++ coroutines ... only use their state machines to store local state ... that lives across suspension points.
Yes and this is briefly mentioned in that analysis as a key advantage of stackless coroutines compared to fibers. If you might call a function that requires 500k of stack, a fiber has to provide for that amount of stack space to be available; a stackless coroutine doesn't need to store the 500k of data, provided that it doesn't live across await points.
The reason Windows fibers failed is not because they were stackful, but because they tried to use only one stack at a time.
As long as you have that, it can be segmented, or it can be contiguous and movable for growth if you have a GC and/or forbid pointers to locals, or it can be perfectly sized if you do a whole-program analysis on stack usage, or it can be large and non-moving like the C stack.
And you’re right that the mutex in the kernel is implemented with a similar algorithm - I just meant that the code in the library you’re using is very different, because it is a relatively thin wrapper around the relevant syscalls whereas the async one is implemented in userspace.
It definitely makes me wonder if the original sin of Rust and many of the other colored-function-based concurrency systems was limiting the enforcement of color-can-only-call-color to only async functions. Would a language that disallows calls back from async functions into blocking functions make sense? Probably not. But what if we split "blocking" into two categories: functions that provably blocked for constant time (or even constant-less-than-X time; simple systems like BPF can verify that already, so I'd assume it's possible to get more nuanced here with effort) versus functions that did not (including I/O, while-true-with-non-statically-verifiable-condition)? If we disallowed calls from async functions into functions of the latter category but not the former, would that be ergonomic/accessible for programmers?
(A minor quibble/criticism of an otherwise great post: I think the discussion of "blocking functions" should get a crisp definition up front, qualifying that "blocking" in this case means undesirably long or potentially indeterminate blocking of executor threads, so readers don't get confused and wonder whether the implication is that any synchronous function call is inherently--rather than just potentially--toxic to good async behavior).
Continuing to think about the idea of "async code can't call 'bad blocking' code": I suspect that one of the reasons this isn't widespread is that it really does require compiler/runtime validation that code down a call stack is good/bad. Erlang almost got there, but (just like Boats points out happened with Rust) the C FFI chimera stopped them in the home stretch and they settled for asking users to tag dirty vs. clean NIFs. But maybe compiler awareness of call path blocking risk is advanced enough today that such a system could be implemented? It definitely wouldn't be simple (...he said, as if he were anything but a rank amateur when it came to understanding compilers). It definitely would solve the mutex-held-across-await-point class of problems.
Two other points that stood out to me that I thought were worth mentioning:
1. The rejoinder to the blocking-reqwest-vs-ureq situation is well put. Honestly, Boats could have gone further here: I think that a large proportion of the complaints/reluctance around using synchronous APIs that wrap block_on or whatever are fundamentally aesthetic/emotional objections arising from people's bad vibes around including or separately instantiating an async runtime inside a library just to provide a blocking API. In practice, the compile overhead is rarely severe and nearly always paid extremely infrequently, and the runtime overhead in time/memory is negligible in release builds. Supporting the emotional resistance here is, I think, a lot of user ignorance around the practical differences between, say, how a current_thread Tokio entry point runs Rust futures and how a libuv loop or a boost::asio context or whatnot run futures in JS/C++/etc. While Rust and Tokio aren't magic (and while I would love more crates to be async-to-syncable with pollster alone), I think that folks often project assumptions about heaviness/performance from other systems onto parts of the Rust ecosystem where they don't hold as true.
2. The Golang callout is fair (and was hopefully cathartic to write), but in addition to being punchy I think there's a lesson to be learned there, albeit a bitter one: Go is widely considered to have wildly succeeded at/"solved" the ergonomics of concurrent colored-function programming for a bunch of technical reasons that Boats calls out ("this is for network servers, C interop's not priority 1", growable stacks, GC, nonzero cost scheduler, etc.), but I think it's also important to note that the problem was addressed to completion in the initial version of the language. Rust's async functionality was not; it was added later. And while it was added carefully, I think that the fact that it was not part of the first coat of paint inevitably increases the severity of some of its pain points, technical constraints aside. I think the lesson that features in the first draft are often the most beloved/powerful holds true in other ways here: consider the trait system and borrow checker, two of the most important features of Rust that were present in v0 (or at least very early on? I don't honestly know) compared to Go's slow and acrimonious adoption of generics in 1.18. I guess another way of putting the lesson is "get it right the first time", alternatively phrased as "hindsight is 20/20". Or maybe I'm extrapolating patterns from unrelated technical phenomena, who knows?
Rust async support is explicitly not complete. They've got an entire working group that's focused on improving it as per https://rust-lang.github.io/wg-async/vision/roadmap.html .
> consider the trait system and borrow checker, two of the most important features of Rust that were present in v0
This is quite wrong in fact, early versions of Rust were very Go-like, with a focus on green threads and GC. Even Rust 1.0, released in 2015, had a very restrictive borrow checker compared to what's available today. Rust's editions system has allowed for a whole lot of pretty seamless evolution.
Re async not being there from day one, this is indeed a big problem. it resulted in certain painful technical compromises (eg Pin) & probably at least ans importantly resulted in cultural / educational issues. The narrative around concurrency in rust initially centered how ownership & borrowing solve data races and it’s hard to see how stackless coroutines fit into that. To many users it clearly feels like an alien invasion.
The questions with adding Move were strictly around backward compatibility & useability, not with determining the semantics.
[1] https://vorpus.org/blog/notes-on-structured-concurrency-or-g...
I am deeply aware of njs’s notes on structured concurrency & have been working on a post engaging with it. This post was not about structured concurrency & so it wasn’t relevant here.
> The popular concurrency primitives – go statements, thread spawning functions, callbacks, futures, promises, ... they're all variants on goto, in theory and in practice. [...] Therefore, like goto, they have no place in a modern high-level language.
That's a really long and abrasive article ("Everyone else's stuff is GOTO, but mine isn't") to read just to get to "Wait for your return values before moving on".
No mention of any kind of synchronisation primitives, liveness guarantees, determinism, channels, transactions...
> His claim is that structured concurrency is the only way to make Ctrl+C handling work in a way how users naively expect it to work.
> And it’s easy to see why: If the exceptions are propagated up the call tree, freely crossing the thread boundaries, the KeyboardInterrupt exception is eventually going to reach the top of the call three (main function) and exit cleanly.
> However, there’s a hidden race condition involved: Imagine two threads in a scope, one gets KeyboardInterrupt and the other one raises an unrelated exception. If the latter exception bubbles up to the parent thread faster it will cancel the thread that’s processing KeyboardInterrupt and Ctrl+C would get lost.
I wonder if progress has been made since 2019.
I don't think there's a race condition in what you've described. Trio and that article is about single threaded async applications, but it's true that it could apply to threads in principle. But, in that case, I would expect cancellation to only be propagated at specific points, much like Trio's checkpoints [1]. You can see exactly that concept applied to threads in C#'s CancellationToken object [2]. You can only cancel a C# Task (which can be running in another thread or asynchronously in this thread) by using one of those, and the cancellation only takes effect when the Task checks it (e.g. by passing the token to a suitable async routine, or by explicitly calling CancellationToken.ThrowIfCancellationRequested()).
So, in the example you gave, there's two possibilities: (1) A race happens because an exception is thrown right after a checkpoint (i.e., cancellation check) and the cancel exception gets thrown at that checkpoint - but that's not really a race (except in your application logic) because that's what you asked for. Or, (2), you get to the point of throwing an exception in one thread after cancellation has been requested from another thread (but before you got a chance to check it) - but in that case the exception will propagate to the top of the thread's call stack (if not otherwise caught) and out of the thread into your nursery-like concept. So you'll end up with a multi-exception containing the KeyboardInterrupt and the other thread's exception.
In the second case, there's a "race" in that you requested cancellation of a task but it finished before it had a chance to be cancelled. But that's always a risk, even with single-threaded async applications. I think it's just fundamental to the concept of cooperative cancellation.
[1] https://trio.readthedocs.io/en/stable/reference-core.html#ch...
[2] https://learn.microsoft.com/en-us/dotnet/api/system.threadin...
(Aside: Personally, when using Trio in any application, I run it in a secondary thread, and I save the main thread for just waiting for KeyboardInterrupt (or for the secondary thread to finish naturally). That way, I can propagate the cancellation to the right bit of my application, which allows shutdown in an orderly fashion. I view Trio's default KeyboardInterrupt handling as just a decent attempt for those that aren't writing a serious enough application to need really reliable handling.)
this is relevant because miller explains why he designed promises the way he did, although since he wrote the dissertation in 02006, he couldn't yet explain how he adapted them for incorporation into the javascript standard. following his dissertation, he was on the ecmascript committee for a number of years, which is how we got e-style promises in javascript, although obviously that also involved the committee as a whole being convinced that they were a good idea. (much of the dissertation is actually about object-capability security, but chapters 13–19 are about promises.)
the great benefit of promises is that you eliminate data races, where one procedure is correct only if a particular piece of mutable data remains unchanged, while another concurrently runnable procedure changes it, so the interleaving of their accesses can result in a failure. consequently, you don't need locks or queues to communicate between different concurrent procedures; you can just use a shared data structure with no locking. this is because promises are an alternative to threads. if you're familiar with async rust, you will note three glaring problems with this:
1. rust eliminates data races through the type system, so the great benefit of promises doesn't exist in this context
2. tokio runs multiple tasks concurrently, so if you do have data races, promises won't save you
3. so you do in fact use channels and locks in async rust
so i am puzzled about why you would ever want to use promises in rust, and i guess this post is an attempt to answer that question. it explains why people have this bizarre misconception that promises are just an efficiency hack, because in rust that's all they are, and those people have never used promises in javascript or e or the other children of e such as spritely goblins.
this bears repeating. before reading this post i didn't realize this, but rust async/await is just an efficiency hack. even the section of this post entitled 'i don't want fast threads, i want futures' never brings up, as far as i can tell, any correctness or flexibility advantages for async rust, only efficiency advantages ('to multiplex arbitrarily many perfectly-sized tasks on a single thread', 'much better performance characteristics' etc.) there's brief lip service to concision ('concurrency-related boilerplate about spawning threads', which i don't think actually exists) before returning to the main theme of performance. then it proposes some truly astoundingly terrible ideas about automatically introducing race conditions if you await with a lock held, in the name of being 'even more optimal', at which point we're well into 'mongodb is web scale. if /dev/null is web scale i will use it' territory
a terminological quibble: i wish people wouldn't refer to promises as 'futures'; a future is, or anyway originally was, a different approach to the concurrency problem, one where reading from a perfectly normal variable would block your thread until that variable's value had been computed (allowing other threads to run). this approach does not have the advantages of promises. promises are what async rust implemented, just under the name of 'futures'. really, though, confusing terminology seems to be the least of the problems with rust's implementation of the paradigm
I don't know about this one. Programmers who use async Rust often talk about how they appreciate that `await` is explicit because it lets them coordinate state using cooperative scheduling. This is maybe a bit higher-level than just data races, but it is about avoiding certain kinds of race conditions.
> it explains why people have this bizarre misconception that promises are just an efficiency hack, because in rust that's all they are
I'm not sure about this either. The post argues that they are more than an efficiency hack in Rust, too- that intra-task concurrency is a useful control flow structure and promises are what enable it. That intra-task concurrency happens to share a thread is a performance benefit but that's incidental to its flexibility benefits.
> promises are what async rust implemented, just under the name of 'futures'.
Well, it's not like Rust's promises are exactly like JavaScript's promises either. JavaScript promises are continuation/callback based; Rust's are poll-based. In a sense they're like your earlier definition of "futures," except that the future API lets its caller decide how to block when the result is not yet available.
i don't agree with your implication that data races are a low-level thing. they can be, but the canonical example of a data race is a lost write to a bank account, which is not just layer 7 (the application layer) or layer 8 (the user) but layer 9 (the economic production system that employs the user)
> JavaScript promises are continuation/callback based; Rust's are poll-based
this is obviously important for the implementation of a so-called async runtime such as tokio, but as i see it, from the point of view of most of the user code, this is an implementation detail. i was talking about the semantics promises provide to things like an http client, not how those semantics are implemented
> the future API lets its caller decide how to block
it's not clear whether in this phrase you are referring to the real future api or to what rust confusingly calls 'futures'. in case you mean the former, i want to clarify that, in the futures interface defined in 'multilisp: a language for concurrent symbolic computation', (halstead, 01985, https://dl.acm.org/doi/pdf/10.1145/4472.4478), you cannot distinguish a future from the value that it eventually produces, and therefore the caller cannot decide how to block; it doesn't even know it's blocking. as halstead explains:
> An operation (such as addition) that needs to know the value of an undetermined future will be suspended until the future becomes determined, but many operations, such as assignment and parameter passing, do not need to know anything about the values of their operands and may be performed quite comfortably on undetermined futures. The use of futures often exposes surprisingly large amounts of parallelism [now conventionally called 'concurrency'] in a program, as illustrated by a Quicksort program given in Figure 1. (...)
> These “futures” greatly resemble the “eventual values” of Hibbard’s Algol 68 [39]. The principal difference is that eventual values are declared as a separate data type, distinct from the type of the value they take on, whereas futures cannot be distinguished from the type of the value they take on. This difference reflects the contrast between Algol’s philosophy of type checking at compile time and the Lisp philosophy of run-time tagging of data.
i would claim that, since halstead's paper has been cited 1658 times in google scholar, including 25 times in the last year, his definition of 'future' is very much in current use, and so it is sort of an act of intellectual vandalism to promote a conflicting but confusingly similar definition
I don't mean low level in that sense, just that they're a narrower concept than general race conditions.
> it's not clear whether in this phrase you are referring to the real future api or to what rust confusingly calls 'futures'.
I'm referring to Rust's API.
i'd still be delighted to see examples of how you can coordinate state changes with 'await'. git repositories containing purported examples would be fine, i'm not suggesting you write an entire async rust program in an hn comment
from my point of view, at this point, rust async seems to be a terrible design blunder. rust's design for massive concurrency is statically proving race-freeness, and that's what makes rust such a promising development. async in rust seems like the kind of development that could destroy the community and render the language worthless, while simultaneously torpedoing public understanding of halstead's futures model (though i don't think it's very promising) and javascript's async approach to concurrency (which is), ruining their chances to succeed as well. so i'm highly motivated to investigate any evidence that i'm wrong
What an extreme take!
Async Rust as-it-is makes perfect sense for Rust's primary audience: folks who desire control/performance enough that they would otherwise be using C/C++. To call async an "efficiency hack" is to underestimate how important that efficiency is to Rust's existence. If async didn't work the way it does, it wouldn't have gained traction within the Rust community (folks would have just ignored it and kept writing state machines by hand) and it wouldn't have been able to act as a draw for C/C++ developers. It's a killer feature, and has enabled the language to thrive.
Not only is it a killer feature, the way it is designed is the only way it could have been. See: https://without.boats/blog/why-async-rust/
async in rust is the opposite, and to me it looks like such an expensive way to get that efficiency that it strongly undermines the reasons for adopting rust
I don't think that's charitable. While there's not, like, a concise bulleted list of async's advantages other than performance, many of those advantages are discussed throughout the piece:
- Cancellation (and the corollary, the ability to externally control, suspend, or time out arbitrary sets of computations at yield points).
- Ergonomic (rather than just performant) heterogenous multiplex-waits. Doing that in non-future-oriented systems isn't just potentially slow, it's fiddly and a breeding ground for bugs in many languages, including Rust.
- The ability to outsource the geometry of concurrency and parallelism to an async runtime rather than spending brain and editor cycles deciding things like e.g. how many handler vs. acceptor threads you're going to manage, and whether executors will launch/stop lazily or eagerly.
Those aren't necessarily unique to Rust, but they are definitely discussed and are definitely unique benefits of an async/promise-based concurrency system.
Also, minor nit:
> tokio runs multiple tasks concurrently
I think you mean "in parallel" here; the whole meta-system is concurrent, but data races re-emerge in Tokio (or rather, impose sizedness/pinnedness restrictions on data in async contexts in Tokio) where they aren't in JS because Tokio can run in true threaded parallel.
> - Cancellation (and the corollary, the ability to externally control, suspend, or time out arbitrary sets of computations at yield points).
i'm not convinced by this one; i'd need to see more details, but my intuition is that it's just an efficiency hack, and i'll explain why
in a shared-nothing system like erlang (or pre-threading unix) you really can externally control, suspend, or time out arbitrary sets of computations. if you have to wait for a yield point, you only sort of can; a task stuck in an infinite loop that therefore never yields will not get a chance to be canceled. moreover, if a task is holding a lock when it yields, that could be either because its own correctness depends on some shared mutable data structure staying consistent, or because it currently has such a structure in a temporarily inconsistent state that it needs to restore to consistency
in the second case, how do you cancel it? you need to somehow restore the consistency, and nobody outside the task has the information required, and if it just releases the lock and exits instead of continuing, the next task that depends on the inconsistent structure will probably crash. so you can't cancel it at the next yield; you have to wait until the task yields without any locks held. and similar remarks apply for suspension, except that the potential problem is not crashing but a deadly embrace, a particularly pernicious kind of deadly embrace because it's hard to tell who was responsible for unsuspending the task
but let's suppose that being able to sometimes cancel, suspend, or time out a computation is a useful feature. we must ask, then, when is it useful? if a task is computing a result that is no longer needed (perhaps the user has already been shown a 'timed out' error, for example) what is the harm of allowing it to continue as long as it wants? it depends on the task, of course, but it had better not be crucial to correctness, for two reasons. one is that, as explained above, the cancelation might have to be delayed for an arbitrarily long time so that the task can suspend while not holding any locks. the other is slightly more subtle: if the task's correctness depends on it getting canceled early enough, then its correctness is dependent on the other task that kills it getting scheduled early enough, which is generally not something that can be guaranteed
consequently, i believe the only case where task cancelation can be useful in a shared-memory system like this is when it's just an efficiency hack and doesn't actually affect the system's semantics
but maybe there's something i don't understand about this (my ignorance of async rust is truly vast) so please let me know. git repository urls containing purported counterexamples would be especially welcome
> - Ergonomic (rather than just performant) heterogenous multiplex-waits. Doing that in non-future-oriented systems isn't just potentially slow, it's fiddly and a breeding ground for bugs in many languages, including Rust.
it's possible i'm not understanding what you mean; can you explain with code? it sounds like the kind of thing you could put into a library function in a system using multithreading; you can solve it without reconsidering your entire programming paradigm
> - The ability to outsource the geometry of concurrency and parallelism to an async runtime rather than spending brain and editor cycles deciding things like e.g. how many handler vs. acceptor threads you're going to manage, and whether executors will launch/stop lazily or eagerly.
this one is purely an efficiency hack; in a threaded system you can spawn a new handler thread for each request which terminates when the request is done. maintaining a thread pool is itself purely an efficiency hack
> I think you mean "in parallel" here; the whole meta-system is concurrent, but data races re-emerge in Tokio (or rather, impose sizedness/pinnedness restrictions on data in async contexts in Tokio) where they aren't in JS because Tokio can run in true threaded parallel.
i don't think so. what looks like 'concurrency' at one level of abstraction can look like 'parallelism' at another, but if by 'parallelism' we mean running multiple streams of instructions at physically the same time, no, you can totally get data races in a multithreaded system that is running on an in-order single-processor system and therefore has no parallelism at all, and this was common until 20 years ago. even if the processor is bit-serial, it makes no difference
From a certain perspective, the knowledge that suspension/cancellation can only occur at yield points shrinks (or at least makes somewhat more vislble) the risk surface of areas where arbitrary cancellation can leave the program in a corrupt state.
I do agree that more work is needed in this area to ensure automatic state reconciliation after cancellation (the mutex-held-during-await issue). Python makes a decent stab at this: its async/await concurrency implements cancellation as an exception raised at await points, allowing cleanup to occur before proceeding with cancel. Unfortunately, that comes with the consequence of allowing routines to choose to defer cancellations, or ignore them entirely. Perhaps that could be mitigated with an async/await system that makes cancellation a bidirectional exchange, changing observable behavior at the canceler when the cancellee does or doesn’t actually terminate? I think the Rust folks are moving in this direction with async-drop, though I personally do not like the idea of implicit custom destructors in the first place, and worry that allowing them to effect async cancellation control flow will hurt rather than help. That’s a separate topic, though.
All that said, I think that struggle is a somewhat inevitable outcome of a platform that supports shared memory between routines and C interop. If you’re willing to give up on those, then sure: you can preempt or cancel with much more flexible semantics. But I don’t think that’s innately better: first, C interop is critical for some platforms, as Boats mentions. Second, while you might be right in pooh-poohing some of my other points as “just efficiency”, adopting an Erlang memory sharing model is really expensive—impactfully so for a much wider variety of programs than the ones for which, say, stackful vs. stackless coroutines make a difference.
On to your next point: how important is yield-point cancellation, really? I think that
> the only case where task cancelation can be useful in a shared-memory system like this is when it's just an efficiency hack
isn’t correct.
Firstly, “efficiency” is kind of a moving target. Is it merely efficiency if we can, say, take advantage of the fact that memory used by cancelled coroutines is freed once they’re cancelled? Consider this code:
https://gist.github.com/zbentley/f45ea39843f533f5f54b23f8a49...
If cancellation of the first call to memory_hungry() did not trigger the drop of x (90% of system RAM), then we’d be unable to successfully call memory_hungry() a second time after the select!, since that would put us into OOM.
Like, sure, there are all sorts of ways to accidentally leak resources in the event of cancellation (we could mem::leak it, could have memory_hungry() spin and block the task, and so on). But this is one fairly happy path where a very important resource (memory) is correctly managed by the cancel-at-yield model.
Secondly, what about the cancellation of routines that keep something alive? For one example, consider routines that send heartbeats or liveness probes: cancelling those can be very meaningful to the program’s semantics. If your rejoinder is that that’s a bad idea because of the chance of a while-true loop blocking routine causing a spurious event loop hang, I’d respond that we’re either now discussing a whole different set of constraints (RTOS behavior), or that we’re back at the original tradeoff between a costly-and-C-incompatible thick runtime like Erlang’s, and some inescapable risks that come with the territory of using both coroutines and ordinary pthreads.
On to the ergonomics of heterogenous multiplex-waits. Consider this code that waits for the first of a timer, a background thread doing some work, and an IO event:
https://gist.github.com/zbentley/d05696b1f6268372cf858b157c6...
Regardless of what cancellation semantics are or are not present, how would you implement that code using threads? It’s certainly possible, but it’s tricky: once you have the background thread created, how do you wait for it with a timeout (so you can detect the other events occurring before that)? You probably put a condition variable/mutex in it and wait on that with a timeout, since the underlying pthread_join doesn’t take one. That’s extra work. Do you farm the timer out to another background thread? The cost is negligible unless you need a huge number of them, but it’s some extra work to code up if you do. Now how do you wait on either of two background threads’ conditions with a timeout? Perhaps you switch to a semaphore--make sure you use it right to avoid deadlock (extra work)! But the socket read is I/O, so perhaps it makes more sense to model the timer as a timerfd and poll/epoll/select over both FDs instead: tricky decisions, extra work. Or should the timer just be tracked in memory and implemented via the timeout to select/epoll (decisions, work)? Then, how to combine waiting on that IO multiplexer with waiting on the background thread? Perhaps we just pretend the thread is a process and give it one end of a pipe to write to when it finishes, and select/epoll on all of our file descriptors to detect what happens first: a nice consistent outcome, but lots of extra work (congratulations, you’ve written a rudimentary event loop! Hope you didn’t write any bugs!). Or maybe this is all in a request handler, and while we might be OK with thread-per-request, we aren’t OK with multiple threads per request, so we move the background job to a thread pool via a queue, or the timer to a pooled timer thread, or the IO to an epoll thread, so we need to bring in some kind of system that models channels … you get the idea.
Backing up: sure, all of those are solvable problems using non-async/await models. But that’s not “just efficiency”: that is a lot of decisions to make and more than a little code that you need to get right just to wait on three heterogenous events. We haven’t gotten into how to bookkeep contingent state machines (e.g. background-thread completion triggers a different continuation than timer completion) or how to add more types of event (signals, kqueue, inotify) into the multiplexer.
In other words, you are 100% correct, in that this is
> the kind of thing you could put into a library function in a system using multithreading
That’s what an async/await system is: a good utility library interface (that may or may not use threads) to code that wraps all of those concerns while providing a sane continuation and cancellation system.
That’s not nothing! Especially when you consider that it’s one of the few means of modelling that kind of concurrency without trading away runtime efficiency (no reduction counts, GC pauses, or added golang-style preempt points), resource conservation (it’s generally sensible and not as hard to understand as bare threads re: memory, and also does reasonable things re: file descriptor and background thread counts so those don’t get exhausted), and FFI models.
If those aren’t priorities for you, then yeah, it might make more sense to use an Erlang-ish or Golang-ish system, or just good ‘ol pthreads. But I don’t think it’s fair to say “efficiency is the only benefit”.
> you can totally get data races in a multithreaded system that is running on an in-order single-processor system
Fair enough. s/parallelism/preemptive concurrency models/.
1. I updated the gists a bit for clarity, hopefully without accidentally moving any goalposts.
2. In addition to the tradeoffs mentioned in the sentence "If those aren't priorities for you...", I didn't really respond directly to your points about loop-blocking causing event loop hangs.
The possibility of loop-blocking causing the entire concurrency system (rather than just, say, one request's routines) to malfunction is a real drawback of async/await systems. Whether or not it's a deal breaker for your using such a system boils down to trust, I think. If your program is running a lot of background routines whose behavior you don't trust to have a low likelihood of loop-blocking, then async/await might be problematic for you. This is analogous to deciding whether to include a piece of functionality as a library linked into your main program, or to include it as a subprocess/IPC interface instead. If you don't trust that functionality not to e.g. corrupt memory, you may choose the latter.
I think Tokio specifically is better at mitigating those drawbacks of async/await concurrency models than many other systems, due to its abilities to a) provide fine-grained and accurate detection of loop-blockers, and b) to move task execution between threads to get around potentially stuck routines. Those are just mitigations, not solutions, but they're substantially better than what's offered by e.g. JavaScript.
Promises don’t prevent data races. They only don’t exist in JavaScript because the runtime is implicitly single threaded.
if you want to select on multiple promises, it's relatively easy to do in javascript; you make a new promise for the disjunction of the multiple promises (e.g., using .withResolvers()), and attach a callback to each of the original promises that resolves the disjunction promise when it fires
this of course does not prevent the other promises from getting resolved, because that would happen in some other piece of code, perhaps in response to a network i/o event or a filesystem i/o event; knowing whether a particular turing-complete event handler is going to resolve a particular promise (so you could avoid running it) would require solving the halting problem
It does not if you control the implementation of the language primitives ;) I’m not smart enough to answer exactly how, but Andy Winford explains how CML does it using the notion of optimistic and pessimistic stages. https://wingolog.org/archives/2017/06/29/a-new-concurrent-ml
fundamentally one of the major things promises are useful for is waiting on network packets which may never arrive. (in the small, sometimes the network in question is sata or usb, but that makes surprisingly little difference.) there's really nothing you can do to keep some other host from sending you a response packet to a request you've already sent it, and when your network interrupt fires, unless you've brought the network interface down entirely, your interrupt handler needs to do work to buffer the packet (if necessary) and inspect it to see where to route it to. if that turns out to be a promise, all of this is 'making progress on' a promise that may be canceled or whose fulfillment may have no effect. (you could, however, limit the resources you dedicate to such useless work: an important optimization in many contexts, but not one that needs language-level support)
so even controlling the implementation of the language primitives doesn't help
I think selection is about what is visible to your program. I.e. that even if that packet arrives, if you choose another path in the select, then the recv() is guaranteed not to happen. You are right that a program doing this has to handle this case correctly, which I think is what the whole Rust future cancellation conversation is about.
1. The words are completely awful. There's nothing "not synchronous" about an async function. Maybe at best "not synchronized".
2. The idea of pretending async functions are still "functions" is kinda terrible.
This is really a huge problem in say python where the architecture (iirc) is via iterator/yield, so your function "actually" has multiple outputs.
Maybe a radical idea in PL design would be to actually make "async functions" be some sort of other thing in the same sense that, for example, pascal had "procs" and "functions"
You could easily solve the coloring problem by creating some sort of builtin that does an immediate, blocking execution of the function by compiling out all the suspend sites, or keeps them "in place" in the case where you want to keep things as if they are futures.
This makes no sense to me. An async function is just syntactic sugar for a (normal in every way) function that returns a promise.
My novice understanding of the Rust model is that an async function is compiled down to a state machine and a set of functions which execute parts of the async function's code and then manipulate the state machine. Those manipulations have two broad categories of side effects: they can wake up/be observed by pollers, and can change the state of later parts of the task corresponding to an invocation of the original async fn (e.g. by setting variables for subsequent functions in the set--code on the other side of an internal await point that uses the result of the await).
In other words, async functions in some languages are more than just sugar for "delegate to event loop and return promise resolved by delegate completion".
My short summary would be in languages that result in function-coloring due to async/await, using Future/Promise directly avoids the coloring and the ecosystem is better off.
Places where async/await are ok are JS (because both the browser and node/deno) are for the most part in an async context, and Go since everything's a goroutine hence all one color.
Languages such as Java have worked well with Future/CompletionStage and doesn't really need the convenience of writing async/await which translates the rest of the program after await into a hidden 'rest of program' callback. That async/await syntactic convenience is not worth the complexities added to libraries. Rust including async/await was a poor 'cargo cult' choice. I would welcome a Rust fork/ecosystem without it.