Io_uring is not an event system

The main benefit to me has been in cases that previously required a thread pool on top of epoll, it should be safe to get rid of the thread pool now and only use io_uring. Socket I/O of course doesn't really need a thread pool in a lot of cases, but disk I/O does.

I dug up the thread with the benchmarks: https://github.com/axboe/liburing/issues/189

io_uring does win on more recent versions, but it's not like it blows epoll out of the water - it's an incremental improvement.

Specifically for websockets where you have a lot of idle connections, you don't want a bunch of buffers waiting for reads for long periods of time - this is why Go doesn't perform well for websocket servers as calling Read() on a socket requires tying up both a buffer and a goroutine stack.

I haven't looked into how registering buffers works with io_uring, if it's 1-to-1 mapping or if you can pass a pool of N buffers that can be used for reads on M sockets where M > N. The details matter for that specific case.

Again where io_uring really shines is file IO because there are no good solutions there currently.

Io uring can in theory be built to subscribe to any syscall (though it hasn't yet). I don't believe epoll can do things like stat, opening files, closing files, and syncing though.

epoll: tell me when any of these descriptors are ready, then I'll issue another syscall to actually read from that descriptor into a buffer.

io_uring: when any of these descriptors are ready, read into any one of these buffers I've preallocated for you, then let me know when it is done.

Instead of waking up a process just so it can do the work of calling back into the kernel to have the kernel fill a buffer, io_uring skips that extra syscall altogether.

Taking things to the next level, io_uring allows you to chain operations together. You can tell it to read from one socket and write the results into a different socket or directly to a file, and it can do that without waking your process pointlessly at any intermediate stage.

A nearby comment also mentioned opening files, and that's cool too. You could issue an entire command sequence to io_uring, then your program can work on other stuff and check on it later, or just go to sleep until everything is done. You could tell the kernel that you want it to open a connection, write a particular buffer that you prepared for it into that connection, then open a specific file on disk, read the response into that file, close the file, then send a prepared buffer as a response to the connection, close the connection, then let you know that it is all done. You just have to prepare two buffers on the frontend, issue the commands (which could require either 1 or 0 syscalls, depending on how you're using io_uring), then do whatever you want.

You can even have numerous command sequences under kernel control in parallel, you don't have to issue them one at a time and wait on them to finish before you can issue the next one.

With epoll, you have to do every individual step along the way yourself, which involves syscalls, context switches, and potentially more code complexity. Then you realize that epoll doesn't even support file I/O, so you have to mix multiple approaches together to even approximate what io_uring is doing.

(Note: I've been looking for an excuse to use io_uring, so I've read a ton about it, but I don't have any practical experience with it yet. But everything I wrote above should be accurate.)

select, poll, epoll, are all the same model of blocking and signalling for readiness.

The problem with the former occurs with large lists of file descriptors. Calling from user to kernel, the kernel needs to copy and examine N file descriptors. When user mode comes back, it needs to scan its list of file descriptors to see what changed. That's 2 O(n) scans at every syscall, one kernel side, one user side, even if only zero or one file descriptors has an event.

epoll and kqueue make it so that the kernel persists the list of interesting file descriptors between calls, and only returns back what has actually changed, without either side needing to scan an entire list.

By contrast, the high level programming model of io_uring seems pretty similar to POSIX AIO or Windows async I/O [away from readiness and more towards "actually do the thing"], but with the innovation being a new data structure that allows reduction in syscall overhead.

epoll is based on a "readiness" model (i.e. it tells when when you can start I/O). io_uring is based on a "completion" model (i.e. it tells you when I/O is finished). The latter is like Windows IOCP, where the C stands for Completion. Readiness models are rather useless for a local disk because, unlike with a socket, the disk is more or less always ready to receive a command.

Tie difference between select and epoll is just that select sets a bitmap of file descriptors, so if you are listening on 5000 sockets, you then have to scan the resulting bitmap (80 or so 64-bit ints) to find the active ones, while epoll gives you an event-like structure telling you something has happened on file descriptor N, which you can then go and check, without having to scan a bitmap. So that is more efficient if you have a lot (thousands) of active fd's. I don't remember for sure but select may actually have a limit of 1024 fd's or something like that. So epoll was preferred over select for high levels of concurrency, before io_uring arrived. But, select worked ok for the purpose and epoll was simply a refinement, not a game changer. Scanning a few hundred words of memory is not that slow, as it will all be cached anyway.

The io specificity is expected to be a temporary situation, and that part of the name may end up being an anachronism in a few years once a usefully large subset of another category of syscalls has been added to io_uring. The shared ring buffers aspect definitely is an implementation detail, but one that does explain why performance is better than other async IO methods (and also avoids misleading people into thinking that it has something to do with the async/await paradigm).

If the BSDs hadn't already claimed the name, it would probably have been fine to call this kqueue or something like that.

Thanks for your detailed thoughts on this, I am definitely not involved at all and just onlooking from the sidelines. The name does initially seem quite esoteric but I can understand why it was picked. Thinking about it more 'io' rather than 'syscall' does make sense, and Windows also does use IO in IOCP.

I am hoping io_uring goes this way (maybe just call it uring). It would make people design APIs with a thought to how they could be async'd.

Fwiw posix specifies readv and writev already; you may want to take a look at those.

It's not really the same, though; readv and writev are still synchronous APIs, they just do more at once.

Well, they have space for 255 ops :)

The entire point of the article is that it is not just for IO.

There's nothing stopping io_uring from gaining more non-IO operations, and indeed that appears to be the plan.

kqueue is not really a replacement for io_uring. That said, FreeBSD can make a POSIX AIO request signal completion to a kqueue, which is a similar model of async I/O.

But as has been pointed out elsewhere, io_uring is also about making other syscalls asynchronous.

People keep saying this but IME that's not how the Solaris Event Ports API works at all. The semantics of Solaris Event Ports is nearly identical to both epoll+family and kqueue. And like with both those others (ignoring io_uring), I/O completion is done using the POSIX AIO interface, which signals completion through the Event Port descriptor.

I've written at least two wrapper libraries for I/O readiness, POSIX signal, file event, and user-triggered event polling that encompass epoll, kqueue, and Solaris Event Ports. Supporting all three is relatively trivial from an API perspective because they work so similarly. In fact, notably all three let you poll on the epoll, kqueue, or Event Port descriptor itself. So you can have event queue trees, which is very handy when writing composable libraries.

There's already plans for a new futex-based swap_to primitive, for improving userland thread scheduling capabilities. There was some work done on it last year, but it was rejected on LKML. At this rate, it looks like it will not move forward until the new futex2 syscall is in place, since the original API is showing its age.

So, it will probably happen Soon™, but you're probably still ~2 years out before you can reliably depend on it, I'd say.

It might! I'm not sure if it's an exact fit or not but the User Managed Concurrency Groups work[1] Google is trying to upstream with their Fibers userland-scheduling library sounds like it could be a match, and perhaps it could get the upstreaming itcs seeking.

[1] https://www.phoronix.com/scan.php?page=news_item&px=Google-F...

I think some of the *BSDs have (or had) it. Linux almost got it at the turn of the millennium, with the Next Generation Posix Threading project, but then the much simpler and faster NPTL won.

We talked about adding that to Linux in the 90s too. A simple, small scheduler-hook system call that would allow userspace to cover different asynchronous I/O scheduling cases efficiently.

The sort of thing Go and Rust runtimes try to approximate in a hackish way nowadays. They would both by improved by an appropriate scheduler-activation hook.

Back then the idea didn't gain support. It needed a champion, and nobody cared enough. It seemed unnecessary, complicated. What was done instead seemed to be driven by interests that focused on one kind of task or another, e.g. networking or databases.

It doesn't help that the understandings many people have of performance around asynchronous I/O, stackless and stackful coroutines, userspace-kernel interactions, CPU-hardware interactions and so on are not particularly deep. For example I've met a few people who argued that "async-await" is the modern and faster alternative to threads in every scenario, except for needing N threads to use N CPU cores. But that is far from correct. Stackful coroutines doing blocking I/O with complex logic (such as filesystems) are lighter than async-await coroutines doing the same thing, and "heavy" fair scheduling can improve throughput and latency statistics over naive queueing.

It's exciting to see efficient userspace-kernel I/O scheduling getting attention, and getting better over the years. Kudos to the implementors.

But it's also kind of depressing that things that were on the table 20-25 years ago take this long to be evaluated. It's almost as if economics and personal situations governs progress much more than knowledge and ideas...

Which is every program that wants to be fast, on modern computers. So...

Say more about this? You're probably right but I haven't thought about it before and I'd be happy to have you do that thinking for me. :)

> Supporting exec sounds like it could be a security issue.

Why would it be any more of an issue than calling a blocking exec?

> why on earth you would call exec that way

The same reason as why you would want to do anything else with io_uring? In an async runtime you have to delegate blocking calls to a thread pool. Much nicer if the runtime can use the same execution system as for other IO.

I've been intending to try process-to-process message passing using a lock-free data structure very much like io_uring where the processes share pages (one page read-write, the other page readonly, and inverse for the other process) on hardware with L2 tightly-integrated memory (SiFive FU740). The result being zero-copy message passing at L2 cache speeds. [for an embedded realtime custom OS, not for linux]

huh, interesting. As far as I can tell the main advantage this has over IOCP is that you can get one completion for multiple read requests.

Looks like they took a lot of the concepts from Winsock RIO and applied them to file I/O. Which is fascinating because with network traffic you can't predict packet boundaries and thus your I/O rate can be unpredictable. RIO helps you get the notification rate under control, which can help if your packet rate is very high.

With files, I would think you can control the rate at which you request data, as well as the memory you allocate for it.

The other thing it saves just like RIO is the overhead of locking/unlocking buffers by preregistering them. Is that the main reason for this API then ?

I would be very interested to hear from people who have actually run into limits with overlapped file reads and are therefore excited about IoRings

That's what IOSQE_BUFFER_SELECT is for.

But some process (kernel or user space) still needs to spend the computers finite resources to read it?

If the work happens in the kernel process or user process it still costs the same?

It also depends heavily on your kernel version. There are really wide swings.

lock-free ring buffers still have locks for the case when they are full, so it would be interesting to see how the kernel behaves when you never read from the completion ring.