Hacker News Reader

Comment by bluetomcat | Hacker News Reader

It's a fundamental Linux API for writing event loops of any kind. In addition to watching sockets, you also have eventfd (alternative to semaphores and condition variables), timerfd (timers managed by the kernel), signalfd (handle signals safely and synchronously), pidfd (handle child process events).

They allow you to handle heterogenous events inside that same loop, reducing the usage of threads.

For async disk IO, io_uring is the way to go.

View on HN

Of course, io_uring has had many security vulnerabilities, as it essentially forms a new ad-hoc programming language with its sequencing operations and doesn't seem to have a cohesive story around security. It will likely suffer many more vulnerabilities. This is presumably why many corporations, including Google, limit its use in production.

As a lot of complication and issues seem to be around sequencing and the adhoc language, hopefully the community will eventually settle on using eBPF for all programming within io_uring [1].

[1] https://lwn.net/Articles/847951/

Do you know of a good resource for learning how to write event loops using these primitives? I feel like there are some extremely common and robust patterns there, but my day job mostly involves high-level abstractions like async/await so I never get to dig into how it really works.

I am not expert in these but I thought Tornado's ioloop was readable enough for me to learn more event loops. Mostly, it was being implemented in pure Python.

https://github.com/tornadoweb/tornado/blob/branch4.5/tornado...

(Had to be in 4.5 because the newer versions 5.x and 6.x, it's switched to Python's stdlib asyncio)

You might be interested in a pure Zig implementation of these primitives by Mitchell in his "libxev" event loop library: https://github.com/mitchellh/libxev

David Beazley’s Python Concurrency from the Ground Up[1] is a great introduction.

[1] https://www.youtube.com/watch?v=MCs5OvhV9S4

Have a look at how memcached does it. It's a bit more interesting than most examples I've seen as insofar as I can remember it combines epoll with multithreading. Be aware that many of the tutorials and criticisms of epoll are a bit outdated as the API has evolved a little over time.

io_uring and epoll can be combined, in both directions. Either add poll SQEs to the ring for an epollfd or have the uring submit readiness events to an eventfd that can be added to epoll.

Reply on news.ycombinator.com