Can anyone comment on how this interface compares to state-of-the-art async i/o in other *unix o/s? Is Linux playing catch up here or innovating?
For non-blocking sockets and the like you have what grew out of the readiness model from select and poll. Today on Linux that's epoll.
This model never worked for files on disk. There is the POSIX aio API for that. On Linux, glibc implements that and it's always been based on synchronous I/O in thread pools because the kernel support isn't good. And I guess there's been Linux specific things like io_submit(2), which as the article states only worked with O_DIRECT and sometimes fell back to synchronous I/O.
It says the new mechanism works with both files and sockets, but I imagine it's still reasonable to use epoll for sockets, and only use this new stuff for files on disk. One good thing about the readiness model for sockets is you don't need to allocate memory upfront every potential read the remote host might trigger.
The amount of work that goes into supporting, debugging, developing, improving, and designing the sheer number of ways that we do things, at literally hundreds of layers of abstractions all which are slightly wrong and each of which does things their own way with their own tradeoffs.
I'm damn glad there are people a lot smarter and more dedicated than I am that keep this stuff running so that I can store a value in localstorage in a browser and have it punch through all of those layers in a fraction of a moment down to writing bits on one of like a dozen storage mediums on any number of OSs via multiple interfaces in multiple browsers.
The model for the various FD polling mechanisms is "wake me up when read(2) won't return EAGAIN when I call it later". That is very natural for sockets, because what read(2) is doing is draining a kernel side receive buffer, so the question is "wake me up when the buffer isn't empty". If the FD were a file on disk, knowing ahead of time that read(2) isn't going to return quickly is kind of a clumsy task.
But isn't this exactly what happens? A process running in kernel mode calls schedule() and sleeps waiting for I/O. The reason for calling schedule() is that it already does know read isn't going to return quickly. But maybe I'm completely misunderstanding your explanation about why current AIO on Linux is clunky?
For all of these cases I suppose you could make the I/O fail with EAGAIN, but this seems like a lot of potential for spurious wakeups.
[1] - https://news.ycombinator.com/item?id=19818899
The ring buffer is definitely innovation.
kqueue is neither epoll nor aio but a common queueing interface for events, which includes io readiness and aio completion as separate types. Linux prefers to split these as different types of fds. FWIW, this wasn't due to obliviousness of the existing design - https://yarchive.net/comp/linux/event_queues.html.
Honestly your comment sounds very like a regurgitated old and somewhat outdated Bryan Cantrill rant.
Luckily, in the end epoll ended up supporting multiple queues at least.
I do believe that ET is good though.