I thought Rust had other, better ways to create non-blocking code so I don't understand why to use async instead.
[0] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
In fact, Rust does have a great solution for nonblocking code: just use threads! Threads work great, they are very fast on Linux, and solutions such as goroutines are just implementations of threads in userland anyway. (The "what color is your function?" post fails to acknowledge that goroutines are just threads, which is one of my major issues with it.) People tell me that Rust services scale up to thousands of requests per second on Linux by just using 1:1 threads.
Async is there for those who want better performance than what threads/goroutines/etc. can provide. If you don't want to deal with two "colors" of functions, you don't have to! Just use threads.
"""Three more languages that don’t have this problem: Go, Lua, and Ruby.
Any guess what they have in common?
Threads. Or, more precisely: multiple independent callstacks that can be switched between. It isn’t strictly necessary for them to be operating system threads. Goroutines in Go, coroutines in Lua, and fibers in Ruby are perfectly adequate."""
What more do you need?
Say you have 1000 threads. To handle a request each one needs to make 50ms of external or DB calls. In one second, each thread can handle 20 calls. So you can handle 20k requests/second with 1000 threads. But Rust is so fast it can serve 500k requests a second. So with regular threads, you need ~25,000 threads. The OS isn't going to like that.
With async you can run a single thread per core, with no concurrency limits. So you get your 500k requests without overhead. With fibers you just run 20k fibers which is a little bit of overhead but easy to do.
This is the core reason everyone is pushing async and fibers in fast languages. When you can push a ton of requests/second but each one has latency you can't control, regular threads will kneecap performance.
In "slow" languages like Python, Ruby, etc, async/fibers don't really matter because you can't handle enough requests to saturate a huge thread pool anyways.
But yes, eventually, for very heavy cases (more than what I would call "high") you will want async/await.
With 8k stack for each, you can easy have 10k-100k threads in a low-end system
And IMHO the added code complexity is not worth the trouble.
The thing is, this is just that - your opinion, generalized as The Truth. But engineering is about making the right trade-offs. Often threading will be fine, you'll win simplicity, and all is good. But sometimes you really need the performance, or your field is crowded and its a competitive advantage. Think large-scale infrastructure at AWS, central load-balancers, or high-freq-trading.
Heh? Where?
With a simulated load of ~20 users we were running over 1000 threads.
Several posts in the chain say that 20k+ threads is "fine". Not unless you have a ton of cores. The memory and context switching overhead is gigantic. Eventually your server is doing little besides switching between threads.
We had to rewrite our s3 code to use async, now we can do many thousands of concurrent uploads no problem.
Other places we've had to use async is a proxy that intercepts certain HTTP calls and user stats uploader that calls third party analytics service.
Just sayin it's not that unusual to need async code because threading overhead is too high
[0]: https://ziglang.org
The idea is that you can set the global "io_mode" mode to blocking, mixed or evented and I/O functions will switch their implementation accordingly. The type of the function will then, if I got that right, propagate up the call stack and turn functions that touch it transparently into either normal or async/awaitable funtions.
Nice way to avoid a bifurcation of the ecosystem into red/green functions. Its a bit magical maybe, any other trade-offs?
I mean, don’t you know at compile-time whether you want something to be async or not? If so, it should be handled by the type system, not by mutating a variable at runtime.
https://news.ycombinator.com/item?id=20676641
It's trivial to turn async into sync in Rust. You can use ".poll", "executor::block_on", et cetera.
Turning sync into async is harder in any language. Even Go with it's easy threading. That's a good argument to make async the default in libraries in Rust, but since async isn't stable yet, that would have been hard to do 5 years ago.
well in most languages you can wrap sync into async. so it's not "hard". it's just harder to have NON blocking code. i.e. in c# there is a difference between:
`await Task.Run(() => Thread.Sleep(5000));`
and
`await Task.Delay(5000);`
both will wait for 5 seconds but one will waste cpu cycles while the other won't.
It is not easy to do in a cirrect and performant way. "async" doesn't mean "code that runs in another thread". You can have a single threaded runtime running async code (that's usually the case for javascript).
The "async-ness" is in those cases provided by the use of non-blocking primitives for IO, network etc. If a function is making a blocking call to the file system even if you make it async it will not help since the main thread will still be blocked on that system call.
The performance will also be quite different: waiting for data on 10000 sockets in a non-blocking way is quite different from having 10000 threads doing the same.
Elixir's Task module (in the stdlib):
future = Task.async(fn ->
do_something_here
end)
...do_other_things...
result = Task.await(future, timeout)
some_list_of_values
|> Enum.map(fn value ->
Task(fn -> do_something_with(value) end)
end)
|> Enum.map(&Task.await(&1, timeout))This is largely why Python async took so long to mature, because so much inbuilt functionality was making IO operations transparently using core sync impls that locked up any async executor.
Console IO operations are actually message call to a "global group leader" which performs the operation, so they are async (and atomic). This can sometimes be confusing if an operation (such as logging) has a bunch of middlemen with an IO operation as a side effect. It's worth the atomicity, though, so none of your IO calls are interrupted by another IO call. Also, if you run a command on a remote node which dispatches IO as part of its own process, the IO will be forwarded back to its group leader (which is on your local node), which is useful for introspecting into another VM.
Disk IO is also different; each open file descriptor effectively gets its own "thread" that you send IO messages to. There are ways to bind a file descriptor directly to your current "thread", but you "have to be more careful when you do that" - you do that if performance is more important (and I have done this, it's not terrible if you are careful).
Network IO is also different; the erlang VM kind of has its own network stack, if you will, but you can set up a socket to be its own "thread" or you can bind a socket into a thread so that network packets get turned into erlang messages.
Handling blocking is all done for you by the VM, which is preemptive and tries to give threads fair share of the VM time.
When people say that programming the erlang VM is like doing everything in its own os, they aren't kidding. Except unlike linux, where your communications are basically limited, you get to interact with your processes via structured data types with coherent language (and also IPC calls are way cheaper than OS processes).
> Does Elixer overload IO operations to be async in async contexts?
Maybe the right way to answer this is: When in Elixir, presume everything is async.
This makes it a hell of a lot easier to reason about.
Is it 0-cost abstraction? I mean, is `sync_read` will compile to the same code like `async_read.poll`? Because turning sync into async is kind of trivial as well: just spawn new thread for that sync block.
- Create an epoll descriptor.
- Add my socket to that descriptor.
- Poll the descriptor for a readiness notification.
- Read the descriptor.
Those first three system calls weren't required in the synchronous version, and unless the read is large enough to overshadow them, they represent some additional cost. But that cost is required by the OS itself, not by Rust's abstractions.
Someone with more experience writing Mio code might want to jump in and correct me here though.
No, they don't. Goroutines have stacks, while Rust async code does not. Go has to start stacks small and copy and grow them dynamically because it doesn't statically know how deep your call stack is going to get, while async/await compiles to a state machine, which allows for up-front allocation. Furthermore, Go's M:N scheduling imposes significant costs in other places, such as FFI.
Besides, for the vast majority of apps, OS threads are not significantly different from goroutines in terms of efficiency. Rust doesn't have a GIL and per-thread startup time and memory usage are very low. It only starts to matter once you have a lot of threads—as in, tens of thousands of clients per second—and in that case it's mostly stack size that is the limiting factor.
This is not true for a use case with a lot of connections, additionally context switch cost a lot more now with all side channel attack mitigations on.
M:N threading was slower than 1:1 in Rust.
Sync is a rudiment of our close past. We use it when we need to shave off development costs.
> What you don’t use, you don’t pay for. And further: What you do use, you couldn’t hand code any better.
In that mind-set, it is completely okay that `sync_read` and `async_read.await` can totally compile to something different, as they abstract different things.
Boats has some more thoughts on this here: https://boats.gitlab.io/blog/post/zero-cost-abstractions/
Rust had legitimate reasons for taking the approach that they did. One can agree that they made the correct decision without excusing and obscuring the consequential costs.
Arguing that the problem doesn't exist if you only stick to functions of a single color isn't a rebuttal, it's an admission! But the fact of the matter is async functions have real limitations and costs, which is why they're not the default in Rust, which in turn is why any Rust program will always have some mix of differently colored functions. But, yeah, the fewer of one color and the more of the other color, the better. That's the point.
Maybe new code will be exclusively async and existing code will switch over.
Some (but not all) of which might even benifit from async... although graphics code has it's own solutions to many of these problems, and it certainly wouldn't be the bread and butter of your core render loop.
1) For performance reasons, your GPU consumes command buffers after a decent delay from when your CPU requests it. This means async logic crops up for screenshot/recording readbacks, visibility queries, etc. assuming you don't want to simply stall everything and tank your framerate.
2) New lower level graphics APIs expose the asyncronous logic of command submission more than ever before, limiting safe CPU access to memory based on what the GPU is still accessing. This sometimes spills into higher level APIs - e.g. bgfx buffer uploads can either take a reference to memory (fast) - which you must keep valid and unmodified for a frame or two (asyncronous, and currently difficult to expose a sound+safe API for to Rust) - or it can make an extra deep copy (perf hit) to pretend it's behaving in a more syncronous fashion.
3) Resource loading is heavily asyncronous. You don't want to stall a game out on blocking disk I/O for a missing minimap icon if you can just fade it in a few seconds later. I might not have 10,000 GPUs to drive, but I've certainly had 10,000 assets to load, semi-independently, often with minimal warning.
So yes, Rust still has colors, but it doesn’t matter because a red function can call a blue one without a problem and vice versa. You’re right in saying that async functions have a cost and shouldn’t be used indiscriminately - so just use them when it makes sense. As opposed to JavaScript, Rust doesn’t make you commit to one or the other early and either face major refactors in the future or pay the price of async when it’s not required.
P.S. I think there are some caveats for library authors and also to blocking the thread on a single future, but maybe more qualified people can comment on those.
Held, in 2015 but doesn't any longer since js had async/await.
This blog post isn't really interesting anyways, and its popularity mainly comes from the zealotry of gophers.
There are hacks like “deasync”, but I personally wouldn’t use it.
https://github.com/abbr/deasync
Rust can block on an individual future so, say, a sync callback can still take advantage of async functions.
What cannot be done is to perform a blocking call on a Promise from a sync function. And that is by design because JavaScript has a single threaded runtime.
Five years ago Rust still had green threads. Literally every standard library I/O function was async, and the awaits were always written for you with no effort.
Its literally taken five years to get back to an alpha thats not as good, and we'll still have to wait for a new ecosystem to built on top of it. I know not everyone writes socket servers and so forcing the old model on everyone probably doesn't make sense long-term, but I still have to shake my head at comments like this.
D made a similar mistake by requiring GC/runtime from start and now even though they added ways to avoid it the ecosystem and the language design are "poisoned" by it an itmakeas it a very hard sell in some places where it could be sold as a C++ successor.
Because rust made the right choice in time it's now a contender in that space, if it chose to go down the runtime required/custom threading model route it would have much less practical appeal. If you can swallow runtime/threading abstraction overhead why not just bolt on a GC and use Go
C++11 introduced a GC API in the standard library, and one of the biggest C++ game engine does use GC in their engine objects, Unreal.
C++ on Windows makes heavy use of reference counting (which is a GC algorithm from CS point of view), via COM/UWP.
The biggest problem to overcome is religious, not technical.
Not sure if Ref counting is a good example here, as there is no runtime monitoring the object graph hierarchy and of course Rust it’s self uses ref counting in many situations.
Reference Counting is a garbage collection implementation algorithm from CS point of view.
RC has plenty of runtime costs as well, cache invalidation, lock contention on reference counters, stop the world in complex data structures, possible stack overflows if destructors are incorrectly written, memory fragmentation.
The new I/O system is better in several ways. First, as you acknowledged, not everyone writes servers that need high scalability. M:N has no benefit for those users, and it severely complicates FFI. Second, async is faster than M:N because it compiles to a state machine: you don't have a bunch of big stacks around.
I'm not saying it was the wrong decision five years ago, but it definitely was a choice and there could have been a different one. I was responding to someone who said async wasn't an option five years ago.
M:N is the parallelization level. I'm actually not sure if Rust is M:1 or M:N or both based on configuration.
M is the number of concurrent process in the language, basically the number of user thread. These user threads can be implemented to be stackful or stackless, up to the language. The N is the number of OS threads.
At least that's always been my understanding.
It also was constantly crashing and had weird semantic issues. I very much prefer the current state, even if I'm a bit sad that async/await has taken us so long.
proc readLine(s: Socket | AsyncSocket): Future[string] {.multisync.} =
while true:
let c = await s.recv(1)
case c
of '\n':
return
else:
result.add(c)
The main challenges I see are around usability within the language design on how best to propagate and compose them.
[0] https://www.reddit.com/r/rust/comments/cjcwmu/is_there_inter...
No thanks.
Edit: I just remembered that in cooperative multitasking, it's probably possible for the OS to safely save the program stack pointer, meaning the program doesn't have to unwind its stack when yielding, unlike async programs. Never mind, that makes the two models quite different. However, in practice, programs written for cooperative multitasking really should be structured just like async programs in order to be responsive (so users can, for example, interact with the GUI while downloading files in the background.)
Which one? It’s “cooperative” ie not unpredictable. The points where one can block are predictable and documented explicitly, otherwise how would the programmer know they won’t block forever. The same should hopefully be the case for async/awaitable apis.
In fact where async/await will actually give up control are harder to tease out.
The differences are really not as big as they would seem.
I don’t see how this increases overhead to deal with either.
Basically, coop multitasking and async/await operate on the exact same execution framework, the latter just gives convenient syntactic support.
Perhaps you should see how typescript turns async await into js.
[1] If implemented with care, not doing syscalls in the middle of async primitives and using fast nearly-O(1) algorithms for timers, etc. it can be incredibly fast. And of course Rust also gives enough room to mess up all that nice determinism.
So the event handler gets called immediately? No that’s not right. What would be the point of that? The event handler or continuation obviously needs to be scheduled on something that is awaitable. Meanwhile, other concurrent tasks may be able to run.
> This is the essence of asynchronous programming. There are no tasks, no yielding, practically no overhead and everything is deterministic
This is just totally wrong. Especially re tasks: https://docs.python.org/3/library/asyncio-task.html#creating...
There is nothing inherent about async and await that prevents “yielding”... the issue of yielding and semaphores is a concurrency issue and since async and await are used in concurrent programming environments, the same issues apply.
While it is true async and await don’t require any kind of cooperative concurrent framework to work, that is kind of their whole point for existing. A single task async/await system isn’t terribly interesting.
It's kind of like this: async/await is syntactic sugar for higher-order abstractions around event loops. At the level of event loops and event hadnlers there is no awaiting anymore. And the whole point of event loops is to not run event handlers concurrently, that's why they are even called loops, they invoke handlers one by one in a loop deterministically without concurrent tasks and once there is nothing more to run they just block and wait for new events. Obviously you can run multiple event loops in parallel, but you shouldn't share memory between them, as it defeats the purpose, is always slower and is never really necessary, you can just use asynchronous message passing to communicate between event loops when you have to.
> A single task async/await system isn’t terribly interesting.
And yet this is the whole point of async/await, promises, futures and event loops. All of them exist to avoid mistakes and performance problems of shared memory concurrency. I mean, really, if you have semaphores or mutexes in event handlers, futures, promises or async functions - you are in a broken concurrency model zone.
As far as I understand, cooperative is far more efficient than preemptive, but unsuitable for poorly written or untrusted code.
I wish to learn and would really appreciate your assistance if you are willing to help.
Well..
await new Promise((res, rej) => { setImmediate(res); })
Isn't the alternative WCiYF is proposing to allow the caller to treat any function asynchronously, while having no way to discern whether doing so might be counterproductive?
It's sometimes called "cold futures".
'async' exists because Python has that GIL bullshit and so Python programmers had to invent that fifth wheel of 'async programming'.
Programmers in other languages then got jealous because they, too, wanted a complex, unnecessary framework that pollutes the whole runtime and serves to differentiate regular programmers from 'rockstar' programmers.
And so async got fashionable and barely-literate coders now think async is magic performance dust that will automatically make your program run 1000% faster.
TL;DR - it's just fashion, give it five years and we'll be reading posts about how async sucks and that it's stupid legacy tech invented by bonehead dinosaurs.
C# introduced it in 5.0, which came out in August 2012. The Python proposal (PEP 3156) for an async library was posted in 2012, the proposal (PEP 492) for async/await syntax in 2015, and implemented in Python 3.4 and 3.5 respectively, I believe. So C# predates Python by about 3 years.
From what I can gather, Python was influenced by C#. But C# doesn't have a global lock, and that's not why it has async/await.
Edit: Added PEP reference.