Monoio – A thread-per-core Rust async runtime with io_uring
github.com
github.com
[1]: https://windows-internals.com/i-o-rings-when-one-i-o-operati...
or for never :P
why not prove like... "this is the best. all other platforms/circumstances/situations are subpar"
if the performance differences are enough, i can picture people making excuses to avoid all of those other platforms (or just never using this... more likely)
Supporting them is smart because people expect it, and not supporting it would be bad optics, but if a language legitimately only targeted Linux, and was better for it, I'd be fine with that - they target the most popular OS by far.
Apparently we haven't published platform statistics since 2019, but according to that years' survey: https://blog.rust-lang.org/images/2020-03-RustSurvey/32-what...
* 55% of Rust users develop on Linux
* 24% develop on Windows
* 23% develop on macOS
So yeah Linux was the most popular, but dropping effectively half of your users isn't always a great idea. Of course, some people will build OS-specific software in Rust, and that's great! But it is a tradeoff you're making.
But if a language said "we're not going to support those" I wouldn't care at all, and a massive number of use cases - the majority, I think - would be solved with that language.
In terms of what you develop on, that's a whole other story. The majority, of course, are on Linux. But I'd be interested to know what platform they target.
I'm not sure that's as obvious as you're making it sound. The number of people who use Linux on the desktop is absolutely minuscule compared to the combined user base of Windows and MacOS. It's probably not as lopsided for developers, but I've never seen anything to imply that most developers in general are on Linux, and I'd honestly be surprised if that's the case given how much smaller the portion is in terms of people I know or have worked with, and that's as someone who does not personally own any laptop or desktop that runs anything other than Linux.
Perhaps MS could pay for crater runs on Windows? Are crater runs done on Windows now?
Looks like yes, https://github.com/rust-lang/crater/blob/master/docs/agent-m...
Some of rust's most well-known and widely distributed success stories for a start - Firefox & Dropbox
That's also the most exciting thing about io_uring for me: how it enables a simple, single-threaded and yet highly performant thread-per-core control plane, outsourcing to the kernel thread pool for the async I/O data plane, instead of outsourcing to a user space thread pool as in the past. It's much more efficient and at the same time, much easier to reason about. There's no longer the need for multithreading to leak into the control plane.
My experience with io_uring has been mostly working on TigerBeetleDB [1], a new distributed database that can process a million financial transactions a second, and I find it's a whole new way of thinking... that you can now just submit I/O directly from the control plane without blocking and without the cost of a context switch. It really changes the kinds of designs you can achieve, especially in the storage space (e.g. things like LSM-tree compactions can become much more parallel and incremental, while also becoming much simpler, i.e. no longer any need to think of memory barriers). Fantastic also to now have a unified API for networking/storage.
So much good stuff in io_uring. Exciting times.
As an aside, Google translation integrated in Chrome breaks the formatting of the code blocks, which is surprising since they're in a <pre> block.
1. Be able to timeout a read of a socket
2. Be able to select over multiple sockets, and read whichever is ready first
Usually a combination of both.
epoll/io_uring(I guess? I only ever did research on epoll) seem like the solution being handed to me on a silver platter, however my understanding is if you want to use either of those you're meant to use async in rust, and that while there are some libraries which provide interfaces for this kind of behavior outside of async, they're usually very ad-hoc and that the community is just very laser focused on async as a language construct. What I don't understand is why does Rust consider it necessary to introduce async, futures, runtimes, an executor, async versions of TcpListeners, Files, etc for this?
Why can't I just have a function in the standard library that takes a slice of std::net::TcpListeners, a timeout, blocks, and then gives me whichever is ready to read, when its ready to read, or nothing if the timeout is reached? Its not like I was going to do anything else on that thread while I wait for a packet to be received, it can happily be parked.
Instead I have to select a runtime, and libraries compatible with the runtime, replace all the TcpListeners from the standard library I'm using with tokio TcpListeners or whatever, deal with API change, and now I have to deal with the "turtles all the way down" problem of async as well.
That's not even to get into the whole nightmare that a lot of really sick libraries which I want to use in a blocking nature, are now only providing async APIs, which means its "async or the highway". I am very much not happy about this situation and I don't know what I can do about it. It seems the only response I ever get is "just use it, its easy" but that is not at all convincing me. I don't want to use it!
Async lets one process handle millions of sockets.
If you want to handle millions of sockets with threads, you can use `mio` [0]. Mio's API has footguns that can cause UB. If your wire protocol is complicated, you may find yourself implementing something like async, but worse.
Edit: For those wondering who Bytedance are, I can save you a Google search. These are the people making TikTok.
From what I read (in Chinese), this "Monoio" is meant to be used for the proxy/whatever part of their next-generation service mesh. So maybe a corp-wide thing.
I'm pleasantly surprised. Have many other Chinese tech companies embraced FOSS?
Plus, Tokio is unique in that it's one of the very few runtimes in existence that's work-stealing (meaning the task can move off its original thread). Most other runtimes in other languages do not have that requirement, meaning you can use traditional Cell/RefCell/Rc instead of their slower, atomic variants.
Right now, the best you can do is write a thread pool that spawns a Tokio LocalSet to run !Send futures. In fact, this is what Actin-web does to achieve its crazy performance. Web requests finish so quickly you rarely need work-stealing to achieve good performance, and often the cost of atomics/stealing is greater than the performance gain.
https://docs.rs/tokio/1.14.0/tokio/task/fn.spawn_local.html
> Plus, Tokio is unique in that it's one of the very few runtimes in existence that's work-stealing (meaning the task can move off its original thread). Most other runtimes in other languages do not have that requirement, meaning you can use traditional Cell/RefCell/Rc instead of their slower, atomic variants.
Most other languages don't have any concept of Send/Sync, or non-atomic Cell/RefCell/Rc. Rather than the compiler stopping you you only find out about the issue when you hit it, and if you're lucky you can skate by a long long while (or alternatively the language has no way to sync and everything's send).
Work-stealing runtimes may be more common than you think because of that: Erlang's BEAM, Go's scheduler(s), Java's fork/join, .net's TPL, many most if not all OpenMP implementations, Apple's GCD, ... implement work-stealing in various measures.
Also... thread-per-core makes work-stealing even more necessary because the OS can't perform the balancing? The only ways to avoid work-stealing eventually becoming necessary (for a general-purpose scheduler) is either to have a completely single-threaded scheduler, or to not have an application-level scheduler at all and use OS threads.
> Are people doing this? Does it work? :)
> Kind of every single successful async framework since the inception of eventloops :-)
> libevent, libev, libuv, boost asio, GTK, QT, seastar, nginx, javascript + node.js, dpdk, netty, grizzly, dart, etc.
> Besides Rust the main frameworks which tried to do move tasks between executors are Go and C#'s Threadpool executor (although I think the ASP.NET default executor might have fixed threads).
> Therefore the state of the world is actually more that Rust would need to prove that its approach of defaulting to thread-safe is a viable alternative than questioning the effectiveness of single-threaded eventloops. Their effectiveness in terms of reducing context switches and guaranteeeing good cache hit rates was more or less what triggered people to move towards the model, despite the ergonomic challenges of using callbacks.
Are you arguing rust’s async runtimes should be single threaded, right after having somehow expressed interest in a non-single-threaded runtime?
If we didn't have the Send + Sync requirements, and multiple async functions are running concurrently on the same thread, and multiple of them locked the same RefCell, might that cause a panic?
Hopefully.
Can you go into this a bit more or provide a code reference? I'm new to writing async Rust code and am interested in this idea.
I brought it up to tide's maintainers and got a bunch relevant links. Feel free to click through the tide issue for more context.
- https://github.com/http-rs/tide/issues/837
- https://github.com/rust-lang/wg-async-foundations/issues/87
- https://github.com/rust-lang/wg-async-foundations/issues/128
Best of the best modern systems programmers gotta get good sometime. Not sure if it's happening yet. Ok here's one point of call: https://github.com/tokio-rs/tokio-uring
And I integrated Hyper as an example: https://github.com/DataDog/glommio/blob/master/examples/hype...
And the performance was blisteringly quick (6x better latency streaming from a file compared to Nginx).
yes, check out `actix-rt`
Where they screwed up was not providing the machinery to make libraries agnostic of the runtime an end user wants to use in their program, so libraries either depend on a specific runtime explicitly or use features to allow users to switch runtimes at compile time. This causes a lot of headaches for library maintainers and end users both.
There’s a lot of interest in adding said machinery (through collections of traits in std) to enable libraries to be generic over different runtimes, but a solution is still some ways off.
Because it was considered inimical to the core values and purpose of the language, which is to be a systems language.
An async runtime being a language feature means a runtime is a language feature, and that is very much undesirable (in fact it used to be part of the language and was removed as it "settled" into its niche from its original design, which was much higher level and more applicative).