Hacker News Reader

The way you have quoted this suggests it is being said by the author, whom you are disagreeing with. It's not: it's being said by someone else (in an issue filed against the repo), and the author also mostly disagrees.

Sorry, yes this is not from the mouth of the author, however he seemed to agree with the premise that async/await is unergonomical and that performance is its only reason to exist, which I am trying to dispute (at least in the context of how Rust does it, which is much better than the JS version for instance)

Indeed. I read it as something written by the author. Double-checking revealed it was written by spacejam that did post the same argumentation over and over here on HN.

I haven't mentioned async Rust on HN since 2015.

It seems to me that these are two orthogonal topics. One thing is how you represent tasks, either using OS threads or async tasks. And the other is how you structure concurrency. Maybe I'm missing something, but I think there is nothing preventing the use of those structured concurrency patterns using OS threads as the base for tasks. Then you get some nice benefits of doing this such as proper stack-traces and easier debugging.

The killer use case for async tasks is when you need hyper-concurrency, e.g. hundreds of thousands of concurrent tasks. In that case, as the article mentions, you can't use OS threads anymore. Of course there are some use cases requiring this level of concurrency, messaging servers come to my mind, but there are also many, many use cases were you need a lower level of concurrency, like a few hundred concurrent tasks max. In those cases I think using OS threads can work pretty well with less complexity.

The advantage of async tasks for structured concurrency lies in task cancellation, which is intrinsically linked to the notion of "task ownership". If you are using an OS thread to offload some task, and then realize that you don't need that task's result anymore, your safest bet is to let the thread run until the end and then discard the results it produces. Other options include adding custom cancellation logic to the thread and remembering to call it at the appropriate time. Nobody checks that you are doing this correctly, which means you may leak resources such as the thread's memory or a TCP connection. On the other hand when using async/await in Rust, the fact of owning a future (i.e. owning the promise that will return you the value when it's done) implies ownership of the task's resources, such as memory, file descriptors, or TCP connections. Dropping the future before it completes means that the task will stop and all resources will be freed/closed immediately, and this is checked statically by the compiler.

POSIX thread cancellation has existed with defined (though complex) semantics for ages. It's a ginormous ugly mess, but it is an alternative to run-to-completion or custom logic.

In practice this is the source of a tremendous number of bugs when async tasks don't actually clean up shared state in their Drop impls.

Anywhere you have an .await in async code you could have a checkpoint in a thread that allows for cancellation. That's the main cancellation advantage - that the author is forced to write those to consume other async functions.

All haskell threads are cancellable. This does mean you have to take extra care when using certain constructs.

So one of the things I realized writing Erlang is that when concurrency is 'free' (or so close as to be indistinguishable in most use cases), more things end up being easy to write concurrently than we traditionally think.

An instance I ran into personally was, effectively, task scheduling. Sure, I could have done the 'normal' thing, of a priority queue being populated from the database on some interval, having some thread reading from that queue, sleeping until the first item needs work, pulling it off, throwing it onto a threadpool. Have to take care to ensure the threadpool is large enough for the maximum amount of concurrency I need, have to make sure that I'm careful in what data structure I use for the priority queue (I need to make sure I'm not adding the same task multiple times to it, and that when adding items to it I'm not locking it), make sure the polling thread can't throw (or at least, when it does, it restarts or kills the program and that then restarts), a few other niggles here and there too. And a whole 'nother level of complexity if tasks lead to follow up tasks (i.e., a task represents a state machine through a series of transitions, which themselves take a sizable amount of time, to where just leaving them on the thread is a bad idea, since it uses up the threadpool).

In a 'free concurrency' world, I just spin up a new concurrent process per task for some window (same as how many items I added to the priority queue). And that's basically it. Each process can step through its state machine, sleeping in between tasks for however long, without issue.

I think the more important aspect of that quote is just about performance vs. code. I see many cases where people are hyperoptimizing on whether or not their framework consumes 2 or 15 microseconds per request when the work they are going to do takes 100 milliseconds.

If you like the async style better, then fine, use it. Sometimes you win like that, where the thing you like better is also faster. But don't worry so much about the performance.

Web frameworks is another place I see this a lot. Crossing the streams, if you've got an incoming web request, unless your framework somehow consumes and discards the web headers, a real web request is already many kilobytes just to represent the incoming headers by the time it gets to your handling code. Using async because it has ~200 bytes per task vs a thread allocating 10K out of the box at that point doesn't make much difference because the HTTP request itself is blowing out the difference.

The spread in orders of magnitude in what is expensive and what is not has gotten so significant on modern systems that you can easily get developers sitting there optimizing nanoseconds while throwing away seconds. The old school assembly-style premature optimization where we're trying to save every bit and cycle has mostly passed away, but its replacement seems to be this; frantically benchmarking how many millions of requests per second some framework or feature can handle as if it matters when your code is going to take 500ms.

A lot of web requests are tens of millliseconds due to the latency of speaking to the database. The abolity to fire several requests to the database instead of serialising them is one of this optimisations that you really can’t do in threaded model without introducing other asynchronous systems components.

you have a good point about http request size. But any high performance framework would not buffer the whole request to later parse it. it will do incremental parsing.

Meaning you don’t need to read all the bytes from the tcp socket before deciding which route to take. And also the handler for that route is given a stream object and will just read as many byte as it need.

Speaking of futures_unordered and similar patterns, I think a part of the "async promise" that has failed is the lack of concurrency for a single user request by default in most languages.

That is, the 'easy' path is to write code such as the following (in vaguely C# pseudocode):

    var p = await GetUserPermission( username );
    var c = await GetServerConfig();
    var m = await GetMessageOfTheDay();

Assume each await call is potentially an expensive SQL query or REST API call.

The problem with that is that this is strictly sequential, synchronous code that is merely "dehydrated" and "rehydrated" to reduce overheads during the waiting periods. It is strictly slower when executed on a server that is not very busy! It must be, because it does the exact same work in the exact same order as the ordinary synchronous version, except now with extra state machinery and complex error handling woven throughout by the compiler.

Scalability is not everyone's concern. Scalability is for the FAANG sized companies. I care about the individual user experience, and async does nothing for that by default.

I mean, sure, you can write much more verbose code along the lines of:

        var p_t = GetUserPermission(username);
        var c_t = GetServerConfig();
        var m_t = GetMessageOfTheDay();
        await Task.WhenAll(new Task[] { p_t, c_t, m_t });
        var p = p_t.Result;
        var c = c_t.Result;
        var m = m_t.Result;

But noone does this, for some values of noone. I've never seen code like this in the field.

In fact, let's test this. I'm reviewing an asynchronous ASP.NET application developed in 2020 right now. It's a large app, with literally thousands of uses of the "await" keyword, at least 3500 files use it.

The only use of "Task" static methods are seven uses of FromResult(). That's it. Zero uses of WaitAll(), WaitAny(), or ContinueWith()!

This is typical.

It's not that asynchronous programming is hard, it's that it is unergonomic to gain a latency benefit out of it. Most applications need lower latency, not higher throughput. Hence, for most programmers, most of the time, asynchronous programming is next to useless. It's just extra noise and more failure modes.

That .NET syntax using Task.WhenAll seems quite bad, which might be part of the reason why not many people bother (disclaimer: I don't do C# or ASP.NET). In Rust it would be:

    let p, c, m = join!(
        GetUserPermission(username),
        GetServerConfig(),
        GetMessageOftheDay()
    );

(you don't even have to write await when using the join macro)

With such simple syntax available it seems obvious to me that one would want to use it as often as possible, and it's also much simpler (and probably cheaper) than dispatching those three tasks to a thread pool.

var r = await Task.Whenall(f1,f2,f3); Console.WriteLine($"{r[0]}, {r[1]}, {r[2]}");

f1,f2,f3 are all async fn's, that is all you have to do

yeah, the original example is showing the unwieldy version of the syntax.

The original example by me did not assume that asynchronous functions all return the same result type.

Mist opportunities for concurrency are between unrelated tasks (because related tasks often dependencies between them). Unrelated tasks tend to have unrelated return types.

When tasks are unrelated, you also likely don't need them all at the same time for the next stage of pipeline. You can simply await it when you need it.

  var p_t = GetUserPermission(username);
  var c_t = GetServerConfig();
  var m_t = GetMessageOfTheDay();
  function_to_call1(await p_t, await c_t);
  function_to_call2(await m_t);

This does not look any more complicated than a non-async function. Not sure how this example justifies your claims.

Besides, even if your example is valid, the usage of Task.WhenAll has nothing to do with your claims either. The use of async/await is majorly for scalability. Being able to make several network calls concurrently is not the major concern. Even if you await at each async call, you still achieve better scalability because threads won't be blocked for async calls and can work on something else.

I guess I am that no one. I come at all this from writing queues from scratch and using threads or processes for concurrency. I also had a lot of fun writing my own networking hot loops with select/poll/spill/kqueue when my work needed it, so I guess I am extra sensitive to making concurrent things actually concurrent. But I would not dream of making three independent requests like that sequentially. There are other patterns you can use besides waiting for all tasks to finish, especially if you can do some processing after the first are done, but all in all why wouldn’t you make them concurrent aside from liking seeing await/async all over the place?

In Javascript, this is a typical rookie mistake. Every newcomer would do it once, get lectured about `Promise.all` in code review, and move on.

Honestly, I'd be really surprised if this was a common practice in C#.

Rust doesn’t allow you to do this.

What makes you think so?

From a quick ddg search, it looks it does: https://docs.rs/futures/0.3.8/futures/macro.join.html

All you have to do is wrap multiple futures into a single one and then await on the combined one. There is no programming language on earth that can prevent this.

My teams/company uses it all over, so maybe depends on the context you work in?

And FWIW, this explicit form is often unnecessary - if you kick off each task they will run in parallel and then just await each task only when the result is needed, it can look a lot cleaner:

        var p_t = GetUserPermission(username);
        var c_t = GetServerConfig();
        var m_t = GetMessageOfTheDay();

        var foo = isAuthorized(await p_t);
        // more code here
        var msg = ( (await c_t).ServerName + await m_t) );

True, this doesn't work in Rust though, because nothing at all happens before the first time you poll a future, so you need an explicit task (but as other pointed out, it's pretty straightforward thanks to the `join!` macro).

Same as F# It uses 'cold' tasks, unlike C# that uses 'hot' tasks that start running immediately. But the F# way can compose in more advanced ways.

Call me noone then.

I write this kind of stuff all the time because parallelizing long-running tasks without dependencies is one of the easiest wins when it comes to wall-time.

But this kind of optimization is somewhat orthogonal to async/await. You don't need fine-grained async to optimize long-running tasks, you could just throw a bunch of closures into a threadpool for that purpose. Async only makes sense when you're interleaving thousands of tasks with readiness/completion based IO.

It's rare that I personally write these kind of optmisations in web apps, but quite often do for backend processing services (and then, only for "embarrassingly async" operations such as hitting a database or HTTP API).

You don't need to create a Task[] because WhenAll is set up for varargs. This is fine:

    await Task.WhenAll(p_t, c_t, m_t);

Or you can just await the threads before you need them. They're already started and running at this point.

You also probably want to avoid using Result and just await the completed task for the nicer unwrap syntax. Plus, you don't want to get into the habit of using Result as its a blocking call. Same with WaitAll and WaitAny. Ideally you would never use those. ContinueWith is also not very needed if your style is to use the more plain await syntax. Those methods are more to bridge blocking and async code so an async from the start app might use async extensively and never those methods.

Perhaps search for WhenAny and WhenAll?

I have written a few programs like this - but not in languages which have async/await! In languages with manual async, getting here by refactoring is fairly easy.

I've been using C# for around 20 years, basically since it was first released.

I never personally had any issue with working with threads and locks, finding it simple enough to reason about them, though I understand lots of people felt differently. When async/await first came to C# around 10 years ago, I grumbled because I didn't see the point; I found it much harder to reason about the flow of code, and initially at least, stack traces were a shitshow (things are much improved, but there is still a lot of cruft in async stack traces).

But async/await was heavily pushed, and "real" threading is almost relegated to the sidelines for most developers. Although having said that, I find that junior devs in particular really struggle to really grok async/await.

Anyway, several more years on, and I have mixed feelings about async. Because Microsoft has gone all-in on async/await, I think it's really easy to work with when building web apps and APIs with ASP.NET Core/MVC - there is barely any "developer overhead" at all, really. Web apps very often hit things like HTTP APIs and databases, and with how easy it now is, there is little reason not to use async/await. Yes, for small loads there is a tiny performance loss due to the runtime setting up async state machines, but it really is almost always completely insignificant - even moreso with the advent of ValueTask, and again more recently with pooled ValueTasks. Yet the gains can be tremendous.

But for non-web apps/APIs, I feel differently. I spend a lot of time writing server-side processing services, and things like Windows services for desktops (in the infosec space), and I've gone all-in on async/await because Microsoft has gone async-first. Hell, a lot of stuff is async only now, so unless you want `.GetAwaiter().GetResult()` everywhere, you have little choice. Anyway, these systems are more complex than web apps, because with web apps, most of the real complexity is hidden away in the framework. But here you have to deal with work queues, caching, pooling, serialisation etc all by yourself. And with async/await, it can be hard to reason about the flow of code, and it's really easy to break things in ways that are really painful to diagnose. And it means that every.single.stacktrace contains async cruft that you need to sift through. Which is not fun.

Anyway, this is much longer than I meant, but my conclusion is that I'll continue to use async/await for web apps and REST APIs (because, why not), but for services, I'm going back to the threadpool, green threads and synchronization primitives, and only using async/await in a limited way where it provides clear value - not async all the way down from the entrypoint.

Welcome back, my beautiful, green threads! (⌐■_■)

> Anyway, this is much longer than I meant, but my conclusion is that I'll continue to use async/await for web apps and REST APIs (because, why not), but for services, I'm going back to the threadpool, green threads and synchronization primitives, and only using async/await in a limited way where it provides clear value - not async all the way down from the entrypoint.

AFAIK, .Net doesn't support "green threads" and they repeatedly confirmed that there are no future plans to do so. Additionally, M:N threading model has serious interop issues as evident in Go, which is a no-go for system languages. Personally, I don't see a need for green threads since kernel threads are fast enough and don't use that much RAM as people tend to believe. And when they are not, sure, go async/await.

TIL: I've been using the term "green threads" incorrectly for years! [0]

I had actually meant "normal", OS-level threads.

[0] https://stackoverflow.com/a/42454139/25758

I find your comment about stack traces a bit weird: of course, when all your work is sequential and you can use only threads, you will have a nice stack trace for free, when async stack traces need a lot of support from the tooling.

But most of the time you not only use thread, but also several synchronization primitives (locks, channel, etc.) and when doing so, regarding stack trace you are in an even worst situation than what async stack traces gives you (“some thread changed this shared-memory value and now it's not what you expected, but you have no easy way to know which one did and when, good luck”).

Maybe if you spray threads around at random :), but in real-world use I find it much easier to pinpoint where the problem occurred, and the path taken to get there. Also, at least with threads you can get the thread ID and/or name.

Regarding shared, mutable state - if multiple async "threads" can access that state, then you still need to guard it, but usually with an async-capable means.

> Regarding shared, mutable state - if multiple async "threads" can access that state, then you still need to guard it, but usually with an async-capable means.

Sometimes, but not as often, because the scope of your async function is often the only “shared state” you need.

async/await is for the concurrent stuff and threads are for the parallel stuff. Two different things. If your code is I/O-bound, use async/await. If your code is processor-bound thing, use threads.

Async/await paradigms exist in several languages, but with C#, async/await is generally considered the "modern" and unified way to handle both IO bound and CPU bound tasks.

The runtime will generally schedule IO bound tasks to run on the threadpool.

> The runtime will generally schedule IO bound tasks to run on the threadpool.

Well, that's not correct. Unless you explicitly call Task.Run or Task.Start (or other similar methods) no new thread is created. The compiler generated state machines don't require the threading mechanism to work. In fact the overhead for async/await is mostly the extra code generated for the state machine and error handling. At runtime, there's no thread switching overhead.

Yes, I meant using Task.Run; I was simplifying, as I'd assumed (wrongly) you were familiar with async/await from another language.

Otherwise, from memory, the runtime spec doesn't actually guarantee that await won't run on a threadpool thread - it will under certain circumstances.

And then there are further nuances if there is a synchronisation context and ConfigureAwait(false) is used, as the continuation will be scheduled on a threadpool thread.

> Async programming also allows for structured programming, where each task is an owned resource of a parent tasks, which means that lifetimes of tasks can be controlled and runaway threads can't exist

You're confusing mechanism with semantics. Here's an article on how Java's Project Loom achieves structured concurrency using its new virtual threads capability: https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

And the article cited above that proposes the idea of nurseries for structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

Arguably, structured concurrency as described above is easier to obtain when using threads as your underlying mechanism, because the vast majority of code is serial[1]. That there are a handful of critical regions where you want to express concurrency relationships doesn't mean we have to discard threads. That's throwing the baby out with the bath water.

Self-promotion: I had stumbled on the idea of "nurseries", independently and many years before the above were published. See https://github.com/wahern/cqueues It's nominally a non-blocking "threading" API for Lua. (In Lua coroutines are also called threads.) But note the plural, continuation queues. It's trivial to instantiate a queue, which is similar to a nursery. This was by design. Many cqueues projects naturally end up with a tree of thread controllers/schedulers. It doesn't work on Windows (yet) because it relies on the fact that kqueue, epoll, and Solaris Ports descriptors can be recursively polled.

[1] Serial != synchronous/blocking.

It has been a pleasure using the Erlang runtime to scratch my concurrency itch while avoiding the async / await bandaid.

Seemingly synchronous on the inside. Async on the outside. With (nearly/practically) unlimited processes/greenthreads.

Best combination of things I have come across.

> Async programming also allows for structured programming, where each task is an owned resource of a parent tasks, which means that lifetimes of tasks can be controlled and runaway threads can't exist (if one is avoiding tokio::spawn).

That's unfortunately far less reliable in practice than it seems on the first glance: You might never know whether any async function you call spawns something else, or makes use of `spawn_blocking`, `block_in_place` or any other function which isn't a pure state machine.

If you try to cancel any of those, you will get either excessive blocking or end up with runaway tasks.

A better solution for this is real support for structured concurrency, as available in Kotlin, Python Trio and now coming to Swift async functions. This doesn't really require immediate cancellation - as favored by Rust futures. It works better with cooperative cancellation, where cancellation is requested asynchronously and ongoing tasks are supposed (but not forced) to listen and follow the cancellation recommendation.

That's interesting to hear - but how much of an investment is it to climb the that mountain (or hill) that makes you comfortable with working with the async model?

I'm not the best qualified to answer this question as I do spend a lot of time reading about programming languages in general, and even though I was able to grasp Rust's async/await very fast, I probably owe it to previous knowledge of a relatively large variety of programming paradigms. I'll thus rely on other commenters that seem to agree that it's really not as hard as you would expect. In particular Rust helps a lot in making sure you don't make too many mistakes so I'd wager that learning to do correct async/await in Rust is probably easier than in, say, Javascript.

Rust's async model takes very little time to grasp. It's very explicit. Nothing runs in the background (contrast that with NodeJS). You have strong static types to help you to know when you got a Future, you can decide where/when to await it.

It's programming with threads, where you have a thread pool and pipes to put tasks onto that and a helper function/macro. Await does this under the hood of course.

That is until you get a weird error or try to do something more complicated.

This is perfectly valid for NodeJS, Python (async and/or threads, hello GIL[0]), and a host of other languages/runtimes.

Also I agree that multi-threaded Rust is probably the best alternative to async Rust.

[0] I have no problems with the GIL, but it's yet another factor to consider when a python multi-threaded program stops working as intended

Not sure how you can compare then. You can’t do with the threads what you can do with async functions.

?

You can do exactly the same thing, after all async is just a big auto-generated state-machine that puts jobs onto a thread pool and waits for them.

Nginx is hand crafted (artisanal!) event-driven C, which is exactly how async runtimes also work. A big loop (the event loop, usually an infinite `while` blocked on epoll()). For example NodeJS uses libuv for this.

The big advantage of first class async support is that we don't have to do this by hand. Plus it makes some optimizations easier (eg. putting things on the stack instead of assigning each event handler a slice of some global [heap allocated] structure).

Or maybe I'm simply misunderstanding what you meant. In that case cloud you clarify please?

Not OP but: I found it surprisingly manageable. I think the disconnect for many people is they think they'll understand it simply by using it [0]. For me, investing a short time reading some of the design articles/documents really helped it click.

[0]: Which is fair, I wouldn't be surprised if this was the best way for some.

using threads (or, my preference, stackful coroutines) does not prevent you from using futures for pipelining and composing computations. But it avoids having to explicitly [1] chain continuations to wait on them.

[1] I count await as explicit as it forces the awkward top level only suspend model.

more expressive and powerful can also mean harder to wrap tour head around.