Hacker News Reader

Comment by Animats | Hacker News Reader

That's a huge help. I only need about 20 threads in Rust, some of which are compute-bound. So involving "async" is totally the wrong tool for the job. Goodbye, Tokio.

View on HN

> So involving "async" is totally the wrong tool for the job.

Sadly with so many things having gone async-first (or only) it’s become difficult not to end up with an async runtime anyway, or not be forced to use an async system. I wanted to build a small web-based tool for local, didn’t really find anything which was not async.

You might like Zig's attitude towards this question. Async/sync decision is a single compile-time decision there. The jury's still out whether that's a good idea though.

I recently wrote an IO abstraction over io_uring using Zig's async/await.

Here's how you would do a write()/fsync()/read() with this (https://github.com/coilhq/tigerbeetle/blob/beta/src/io.zig#L...):

  const offset: u64 = 0;
  const bytes_written = try io.write(fd, buffer_write[0..], offset);
  try io.fsync(fd);
  const bytes_read = try io.read(fd, buffer_read[0..], offset);

Other sync functions can use this asynchronous IO completion code in a synchronous style (as this snippet shows) and still get all the zero-syscall and asynchronous performance of io_uring. What this is actually doing under the hood is filling SQEs into io_uring's submission queue ring buffer and then later reading completion events off io_uring's completion queue ring buffer, so it's fully asynchronous in the I/O sense but this hasn't spilled out and leaked over into the control flow. The control flow is as it should be, nice and simple and synchronous.

Beyond this, Zig still allows you to explicitly indicate concurrency with the `async` keyword, for example if you wanted to run multiple async code paths concurrently.

But the crucial part is that Zig's async/await does not force function coloring on you to do all of this: https://youtu.be/zeLToGnjIUM

Pretty incredible on Zig's part to be able to pull this off. Huge kudos to Andrew Kelley. Also, thanks to Jens Axboe and io_uring, what you saw above was first-class single-threaded or thread-per-core, there's no threadpool doing that for you, it's pure ring buffer communication to the kernel and back, no context switches, no expensive coordination. Pure performance. There's never been a better time for Zig's colorless async/await. The combination with io_uring in the kernel is going to be explosive. It's a perfect storm.

I’d happily take an async-by-default world over a world where some APIs only exist through blocking calls. A classically threaded program can easily block on a future, but wrapping a blocking call in an otherwise asynchronous program is complicated, expensive and error prone work.

Btw, have you read this: https://async.rs/blog/stop-worrying-about-blocking-the-new-a... async-std allows to run blocking calls without hoops rather efficiently:

   async fn read_to_string(path: impl AsRef<Path>) -> io::Result<String> {
       std::fs::read_to_string(path)
   }

It doesn't have await inside! My mind was blown as I saw that.

> should a task execute for too long, the runtime will automatically react by spawning a new executor thread taking over the current thread’s work.

That is a super interesting strategy, though obviously only works when you can « afford » a multithreaded scheduler.

Anyway I wonder how they manage this, signals?

> though obviously only works when you can « afford » a multithreaded scheduler.

Yeah, for example in comparison actix-web only uses single threaded workers - one per core. Future in actix-web doesn’t have to be Send or Sync, and I think it’s incompatible with what async-std is doing here. That design is almost certainly one of the reasons actix-web tops phoronix

Nothing so fancy.

Each worker thread runs in a loop executing a queue of jobs. On every iteration it sets an atomic progress flag to true.

The runtime in which it's contained polls its workers every 1-10ms, atomically swapping in false and checking to see if the previous value was also false - if so, it steals its task queue and spins up another worker to execute it.

https://github.com/async-rs/async-std/blob/ceba324bef9641d61...

It also really doesn't scale. It'll do fine on your average <10 core laptop, but once you get on a multi-package system you're going to find you're constantly thrashing memory because it is making disruptive scheduling decisions and your pooled tasks have poor context locality.

Read the first sentence.

> This blog post describes a proposed scheduler for async-std that did not end up being merged for several reasons.

I don't think it's a particularly good idea in the first place - it's basically an automatic watchdog-driven block_in_place(). It doesn't remove the problem of blocking in futures, it just limits the damage to the local task rather than blocking the entire executor.

That's fine in the simple case of future-per-task, but it's pretty common to be polling multiple futures concurrently within one, so it's not a general solution.

> A classically threaded program can easily block on a future, but wrapping a blocking call in an otherwise asynchronous program is complicated, expensive and error prone work.

It’s really not though, at least as long as the parameters and results are Send. For instance Tokio has a spawn_blocking which runs the function on one of the blocking threads it spawns on-demand specifically for that use.

Meanwhile « blocking on a future » requires adding and managing an entire async runtime and its interactions with the rest of the program, and locking up the runtime is a very real possibility.

I understand the desire to stave off dependencies but managing an async runtime should only be a simple function call or two. How do you end up locking up the runtime with something like that?

These days it is possible to eliminate almost all blocking calls in Linux apps. File opening was a long persisting one but io_uring fixes that. Async sockets and file i/o to already-open fd's have been around forever.

mio and mio_httpc are options if you're the kind of person like me who finds 'async' worse than event loops.

The only reason you wanted to use “async” was because of micro-optimization on thread context switches?

That's the most commonly given reason for using async/await so a lot of people assume threads are way more heavyweight than they actually are.

What other reason were you thinking of?

Ease of understanding multithreaded code and wait on results or perform standard control flow constructs in a multithreaded environment?

This is a great example in Node on useful combinators that with async await make it easy to express parallel programming concepts with familiar tools. No manual IPC, no fork/join child PID/thread ID handling, etc.

https://github.com/sindresorhus/promise-fun

The same abstractions (or many of them) exist in Rust, but I think the above is illustrative of the ways we can combine async object returning functions and then use await to hide the complexity of the state machines needed to drive them.

That this abstraction that makes code easy to read and write also performs better is the icing on the cake. The former prevents bugs and keeps code quality high, and that is worth much more.

> Ease of understanding multithreaded code

Rust is not Javascript. Using threads is actually a lot simpler in Rust than async/await.

I don’t know about Rust but in every other language I’ve used threads were easy to use and understand, except when it came to some bits like signals, which at least on Linux are no longer a big problem. Main thread runs a hot loop to look for data to process, then hands it off on a queue to a worker thread out of a pool. That thread is then solely responsible for processing the event and passing the result either back to the main thread or to the next thread in the pipeline via the same queue mechanism. Last thread to handle the result or the exception frees the resources. It might not be ergonomic for all types of code but it certainly isn’t hard to understand what everything is doing and easy enough to debug since each thread can be tested individually to check its functionality.

Sure, threads are easy to understand. The difficult is when you get a concurrency bug but that can happen with single threaded async/await code anyway.

Also threads are definitely not easy to use in all languages. E.g. C++ gives you very little help (no channels for example), and JavaScript makes starting threads difficult and moving/sharing memory is limited to primitive arrays.

A queue implementation in C is easy to create and understand if you don’t have a library for it handy. Combined with a mutex and/or a spin lock and once you’ve grokked pthreads’ mental model you should have the primitives. But those are all guns that shoot both ways if you aren’t careful.

You just described having the main thread have to wait on multiple threads to complete processing of data, worker threads handling signals and IPC and moving data between threads, and then some sort of shared signaling to ensure resources are freed.

So, code potentially laden with use after frees, double frees, shared and mutable data, and so on.

No offense to you, but I would be leery of trusting that code in any languages except a handful. Certainly not C/C++, and if it were written in Rust, I would hope it would use a thread combinator library and channels.

It certainly is easy to make a mess of it with C/C++. It can be done well and safely but there are no guard rails. I have written this code in C and trusted it to run as intended and it did. I wouldn’t stake human lives on it but that wasn’t my requirement at the time. Val grind and other code analysis tools certainly didn’t complain and I had no memory leaks. Rust didn’t exist at the time. One specific project wrangled about 1000 worker threads, a logging thread, a network server thread, a signal processing thread, and a main control thread to the tune of a very large number of requests per second on commodity hardware. In running it for I think 4 years I had one memory leak initially that Valgrind quickly found. Could probably write that service with a lot fewer LOCs today with a language like Rust of course and with all kinds of memory safety. But at the time it worked well. Oh and it had to do all kinds of fun low level networking stuff with elevated privileges so double danger :)

Threads easy to understand? Hmm, even the termination part? With all these old school network APIs?

I find async/await to be easier to use than alternatives. I see two categories:

- A. Async/await - compiler saves and resumes functions.

- B. Message based - Golang, Erlang, threads with messaging.

With category A, I can use my IDE to jump to every function that is called and easily follow the computation.

With category B, all of these connections happen at runtime with messages.

When you have a tree of tasks all which may save/resume many times, async/await it easier to understand than launching a thread per IO event.

This a confused notion. A useful way to think of Go and Erlang is that they automatically and transparently insert async/await each time you call a function that performs I/O. Messaging between different application tasks is completely orthogonal and can have use cases in languages with async/await as well.

I probably should have put the categories as:

A. Implicit messaging using the languages function syntax (async/await).

B. Direct messaging using a message passing feature of the runtime (Erlang, Golang)

Note: I mean "messaging" in the context of a single OS process, that possibly has many threads (so within a single language runtime).

Async/await is still implicit messaging, but it appears like a regular function call - which in my opinion is easier to understand. Using function args/return for input/output is something every developer already knows.

In contrast, Erlang and Golang require you to use some type of messaging feature in addition to functions.

> A useful way to think of Go and Erlang is that they automatically and transparently insert async/await each time you call a function that performs I/O

The part they are missing from async/await is the ability to easily get return values without messaging, and do this recursively for a large tree of functions.

E.g. getting a return value from `go x()` requires messaging, but with async/await you could do `const p = x(); const ret = (await p); // return value received at a later time with no messaging.`

Both of them will require you to create some type of messaging topology to return the values (which makes your program a mixture of (regular functions + messaging features) vs async/awaits "everything looks like a function").

> The part they are missing from async/await is the ability to easily get return values without messaging, and do this recursively for a large tree of functions.

No, they do not. In Elixir for example if I call:

      bytes = File.read!("filename.txt")

`bytes` will have the data returned from the function call immediately, with no need for message passing or awaiting the result. Under the hood, it is still asynchronous evented I/O. If I want to explicitly await for flow control reasons (await all of or one of multiple events) that is available in the stdlib in the `Task` module. E.g.

      t1 = Task.async(fn -> do_this_thing() end)
      t2 = Task.async(fn -> do_this_other_thing() end)

      Task.await_many([t1, t2])

You can accomplish most things without ever calling send/receive or writing your own gen_server etc.

I see, I did not know that.

Last time I used Erlang (pre-Elixir), the `bytes` example would require you to set up a request/response with a blocking `receive`.

Although this emulates async/await (AA), under the the "async" emulated functions is message passing that must keep track of the connection between requests and responses at runtime (E.g. with state mapping request ids to response ids).

I think the key issue is that the inputs and outputs are disconnected in the static program text (and only connected dynamically at runtime).

Two contexts that matter for understanding how a system transitions between states are:

1. Program editing/reading.

2. Runtime.

I think AA is superior for understanding the system as a whole in both these contexts, because at edit time the IDE jump to def/show all usages allows you to understand every function that will be called, and at runtime you can get a stack trace to understand where the current function came from, and where it is going.

With message passing runtimes, both 1 and 2 require extra mental models on the part of the programmer, because they also need to understand the network topology (which either is not possible statically, or requires extra tooling on top of functions).

Message passing breaks down your system into CSP's, which makes it easy to understand each sync process, but hard to understand the whole system, as the same program-writing-process that allowed you to break down your components is working against you when you need to put them together again to understand the whole system.

I could be wrong as I have not used modern IDE's or debugging tools with message passing runtimes lately.

Sorry, what does tokio have to do with "async"? The default implementation uses posix threads, no?

It's a runtime for running lightweight tasks (`Future`s, async functions) on top of it. What is not async about it? And of course it still needs posix threads. The executor needs to run somewhere, and the only somewhere that an OS offers is a thread.

Sure, but I didn't think anything about async functions implied running tasks. Isn't it just syntactic sugar over futures? You certainly don't need to use the tokio runtime in order to use async functions.

So, it's not clear why you'd abandon the async syntax just because you're compute bound.

I'll admit it's a bit disconcerting not getting a reply but I'll get over it.

Reply on news.ycombinator.com