Hacker News Reader

Comment by jchw | Hacker News Reader

I have two complaints about sockets:

1, bidirectional sockets probably should've been ≥2 fd's, not 1.

2, non-blocking semantics and poll/select suck.

Can't blame anyone in 1983 for not getting "async I/O" right since we're still struggling; at least now we have some decent answers (like io_uring on Linux.)

View on HN

> non-blocking semantics and poll/select suck.

I'd argue that it's not that the semantics suck, it's that there are too many of them. We have O_NONBLOCK, multiple multiplexers (for sane reasons--I don't begrudge 1980s folks for not thinking about fd set size and copy overhead for select(2) either), and others. What's worse, they don't all work with all FDs--not only are regular files not nonblock-able in the same way that sockets are, but all sorts of other FD-exposed capabilities (signalfds, timerfds, memfds, pidfds) do or don't support nonblocking semantics and multiplexing in all sorts of weird ways.

If those old system designers had stuck with keeping the async IO syscall space small (e.g. "you only get select/poll" or "you only get read(fds) and read_noblock(fds, timeout)") and consistent (by drawing a hard line at "if you expose something as an FD, it must support all APIs that generically handle FDs"), we would have ended up in a better place. Sure, that would have slowed down some implementations (e.g. vfs drivers), but would also have massively sped up development against a lot of these APIs.

Ah, well, hindsight is 20/20 I guess.

Allready in 1983, 4.2BSD had 3 different attempts at doing multiplexed/asynchronous I/O:

Non-blocking I/O: "O_NONBLOCK" with "EWOULDBLOCK" (or "EAGAIN")

Signal-driven I/O: "O_ASYNC" with "SIGIO"

Synchronous I/O multiplexing: "select()"

All 3 had various defects, especially with a large number of concurrent I/O actions.

In UNIX-derived operating systems, after 1983 there have been many other attempts to implement something better than these 3 (starting with System V "poll" and with POSIX AIO, and then with various incompatible approaches in Solaris, FreeBSD and Linux), but none were good enough and most were seriously inferior to methods of doing asynchronous I/O that existed in some IBM and DEC operating systems decades earlier.

In my opinion, only io_uring has finally solved the problem of multiplex asynchronous I/O in Linux, and in a manner much better than in all older operating systems. I consider all the many older alternatives that exist for liburing as obsolete.

This. I'd break-out in a cold sweat if I had to work on this again.

Windows has a better async I/O implementation than Linux and has for decades

No, although this was true until around 2019 when Linux introduced io_uring and finally had a genuine innovation. io_uring wound up giving us a unified asynchronous I/O layer that handles just tons of things. (And it is genuinely a step up from the overlapped I/O APIs of Windows, as it avoids eating syscall transition overhead in many cases.)

Of course, Microsoft then went on to introduce IORing in 2022, continuing their legacy of never being afraid to adopt good ideas. But still - it would be wrong to suggest Linux is currently behind on async I/O. It hasn't been for years.

What would all the other FDs be instead? Like an eventfd or epoll handle, it obviously couldn't be an FD so what would it actually be?

Sorry, what I meant was that FDs that aren’t sockets should be uniformly handled by the async IO syscalls rather than having to parse man pages or trip over EINVALs to figure out the answers to questions like “can I select(2) on a pidfd? Can I O_NOBLOCK a regular on-disk file?”.

For a concrete example, I should be able to select/poll/epoll any file descriptor. The multiplexers can short-circuit return for things where readiness is inherent (e.g. block device files/vfs, I’m not asking for the moon a la waiting for NFS shares to report ready or something).

I should be able to issue non blocking reads on special file descriptors (timerfds, signalfds, eventfds, and so on).

This is analogous to my other unachievable fantasy: everything that exposes a filesystem interface must work with inotify/kqueue/whatever. No exceptions for procfs/NFS/etc.

At some point this fantasizing just becomes me griping about what made it to market/the worse-is-better philosophy generally, though. And yet it moves, I guess!

Or you throw it all out and do Windows-style overlapped I/O instead.

Windows NT had overlapped IO decades before Linux. The Microsoft kernel team were alright.

Windows NT had many things decades before Linux, I've gotten used to it. There are things Windows has that I still sort of wish Linux had, I mentioned one of them just last night (RDP fast user switching/session roaming.)

If we're just strictly talking about low level things, another good one would be synchronization primitives, which I guess we now have some of in Linux verbatim at this point, if only for the sake of emulation. (And of course futex2.)

I don't think the kernel prohibits fast user switching but you'd have to implement it at the desktop layer too.

Think of ctrl-alt-f7, f8 etc. This still works today, you can run X servers as different users on different VT numbers and switch between them. All that's missing is better UI.

The kernel definitely doesn't, although the VT layer kind of sucks and probably should've never been used to multiplex graphical sessions. (I mean, I get why it was, but oh well.)

I was talking about this:

https://news.ycombinator.com/item?id=49093002

And it winds up being mostly a thing you have to implement in the compositor, but none of them do so far to my knowledge.

And of course the same issue applies if you wanted multiple physical seats juggling sessions between them.

That's systemd limitations. Who even uses systemd?

> That's systemd limitations.

It's kind of the opposite, systemd-logind provides better seat management than without. I attempted to implement the same concept using libseat and seatd unsuccessfully though I do not remember exactly what I got caught on. (not in kwin but a toy compositor; I have been at this problem for a little while now.)

That said, most desktop systems have only implemented systemd-logind in a limited way so far that basically just does what the VT system + display manager was already doing, that's what would need to be worked on in order to make this a reality. I was able to accomplish a prototype with only patches to Kwin, kfreerdp and plasma-login-manager, no need to patch systemd-logind or anything like that.

> Who even uses systemd?

Well for one thing, almost all of the major distributions; Ubuntu, Debian, Fedora/RHEL, Arch Linux, NixOS, openSuSE? and of course SteamOS now.

The only major non-systemd Linux distributions I can think of are Alpine Linux and Gentoo. And Android if you wanted to count that, but I think it is special enough to be considered its own OS that just is Linux-based.

Then it's libseat and seatd limitations. Why rely on a system that doesn't do what you want? Bypass it and DIY what you actually want.

I personally use a systemd-based distro (NixOS) and logind is able to handle this use case. The point of trying to support libseat was for the sake of non-systemd distros. (Though elogind would probably work too.)

I certainly wouldn't DIY this; nothing would support my DIY version, which would defeat the purpose.

The Linux platform gives you full control. If the box you seek doesn't exist, you have to create your own box instead of thinking inside someone else's box that doesn't even do what you need.

I'm going to be frank; there is literally no point in replying to me only to not read what I said again. I am running into KDE/GNOME/etc. limitations, not logind. And I both can and have submitted patches to both projects, so I do not need my own box.

Windows strengths: IRPs and fantastic async I/O, ETW events, tooling- windbg xperf driver verifier time travel debugging, RDP, doesn’t claim everything acts like a file

Linux strengths: file system, open source ecosystem, ssh, lockless algorithm support

Mechanisms equivalent with the overlapped I/O of Windows NT (1993) already existed 30 years earlier, e.g. in IBM OS/360 and PL/I (1964/1965).

The main features that were better in Windows NT than in the UNIX-derived operating systems were inherited from the DEC VAX/VMS operating system (1978) (e.g. WaitForMultipleObjects) and a part of them had been inherited from the even earlier operating system DEC RSX-11M (1974-11) (Dave Cutler also had a major role in those operating systems, so there is nothing surprising about this; Microsoft had to pay a big compensation to DEC, on the order of $ 100M, for the features that were obviously taken from the DEC operating systems).

UNIX was very simplified in comparison with the operating systems that were used on bigger computers, and for some things its derivatives never caught up with those older systems, except after 2000. Only "futex" (2002) and "io_uring" (2019) have advanced the state of the art clearly beyond what already existed for AIO on the IBM mainframes around 1964/1966.

To be fair, I think that was the entire challenge in the first place: the contemporary microcomputers were relatively limited on what they could do, so you couldn't necessarily afford to just do everything mainframes were doing. More importantly, it's not clear you would've wanted to. Having the taste and good sense to know what to "steal" and how to "steal" it was already hard enough work. A lot of good ideas took a while to come over, but a lot of probably bad ideas also never did.

It's an evergreen interesting fact that Dave Cutler took perhaps a bit too much inspiration from prior VMS work to NT, but I also think there's no need to put an asterisk on it anymore than there is UNIX or anything else; it does ultimately stand on its own. Like great artists, great software architects steal.

Including the use of systems programming languages with bounds checking, PL/I, PL.8, PL/S, NEWP, ...

I’m not sure you can call anything in Linux/Unix as equivalent to overlapped I/O in Windows given that there aren’t any IRPs

I just can’t understand why the Linux kernel has avoided tracking completions for so long and has such hard time with async I/O. Windows has had IRPs and overlapped I/O for decades, asyc is the default and there just isn’t much reason to ever use synchronous I/O.

3, fds should have been UUIDs

fds should have been IPv6 addresses. I will not elaborate on why or how.

It's such a common problem to write to a stale file descriptor. It may be closed which is fine, at least you will have an obvious error. But what's much worse is when the stale descriptor actually works, because something else in the program opened a new fd and your stale descriptor is not stale anymore, it just points at something unexpected.

Opaque handles are often a blessing.

Maybe UUID would be overkill (it certainly would have been back in the day of 16-bit unix machines) but something fairly large would have helped a lot.

Even just incrementing and rolling over a 16-bit value would have helped, instead of situations such as closing stdout and stderr, opening a file, and now random error logs sprinkle into your PDF or whatever.

Yes, "you are holding it wrong", but it should be hard to "hold it wrong", not easy.

Joking aside I pretty much agree with this. Windows NT Object handles work pretty well. I assume they are pointers of some kind under the hood, but either way a machine word is probably large enough as it seems unlikely you would ever need to handle more open handles than you have bytes of addressable memory.

Reply on news.ycombinator.com