Operating system threads are always going to be (more) expensive
utcc.utoronto.ca
utcc.utoronto.ca
If you look a the old KSE work, the general gist was that if you were about to block in a syscall then you'd effectively get a signal-style longjmp back to your userland thread scheduler. You'd pick another thread and continue running all in the same process/task context.
There were many problems with what we did and how we did it, but the unavoidable fundamental problem at the time was that it inverted assumptions about costs of low level primitives. Important(TM) software was optimized for the world where threads and blocking were expensive and things like pthread mutex operations were cheap. Our changes made threads and blocking trivially cheap but added non-trivial overhead to pthread mutex etc operations. Applications that made extensive use of pthread mutexes to coordinate work dispatching on a precious small pool of expensive threads were hit with devastating performance losses. Most critically, MySQL. We'd optimized for hundreds of thousands of threads rather than the case of multiplexing work over a few threads.
It became apparent that this was going to be an eternal uphill battle and we eventually pulled the plug to do it the same way as Linux. We made a lot of mistakes with all of this.
Today I believe all of these operating systems (except MacOS, LOL) have either futex or an equivalent technology (the Windows thing works on bytes rather than aligned 32-bit values, so that's cool) and so it seems like only contention could be more expensive as the whole point of a futex is that uncontended acquisition is a single CPU store and that's userspace, the kernel is nowhere near it.
Apple are even less talkative than Microsoft, but it is possible that the reason libc++ uses this is that it's actually guaranteed and if properly asked Apple would say so. It would be nice for (newer) macOS to get the same promises as the mainstream platforms with respect to these primitives.
Yes, switching from user to kernel mode and back is expensive. But let's count the syscalls.
Model 1: Threads. 1 blocking read(), 1 blocking write(). Plus cost for the OS scheduler.
Model 2: 1 "I want to read", 1 "can I read now?", 1 actual read, 1 "I want to write now", 1 "can I write now?", 1 actual write. Multiply as necessary if the read or write are only partial. Add IPC cost as necessary if you have more than one thread handling async events.
Measuring by the syscall overhead, blocking I/O is much more efficient.
There are other costs to it, though, so it usually consumes more resources per handled socket and turns out to be more expensive than async I/O. But it's not the syscall cost causing it.
epoll batching only works in respect to the event notifications from kernel to user space. The only other thing that could be considered batched is that you can switch from "I want to read" to "I want to write" without telling the kernel first that you don't want to read anymore. But that's hardly worth mentioning.
Nonblock read(), if EAGAIN: append fd into event waiting queue, wait event and read() again
Nonblock write(), if EAGAIN: append fd into event waiting queue, wait event and write() again
For loaded systems EAGAIN case is quite rare and syscall count is same.
I, for example, implement an async web server and your case basically never happens. You get a connection, you call read, there is not going to be data yet. In fact calling read immediately is basically wasted effort, so I'm not doing it. I save one syscall over your proposal right away. Optimizations like TCP fast open might change this over time, but definitely won't when you do SSL. You need to wait for a request and then for replies. SSL involves several additional back and fros over plain TCP.
Writing is worse unless you tell the OS to provide ungodly buffer space per connection, which you shouldn't, because then your server won't scale.
> The less that doing things with threads crosses the CPU's security boundary into (and out of) the kernel, the faster the threads go but the less we can really describe them as 'OS threads' and the harder it is to get things like forced thread preemption. And this applies not just for the 'OS threads' themselves but also to their activities. If you want 'OS threads' that perform 'synchronous IO through simple system calls', those IO operations are also transitioning into and out of the kernel. If you work to get around this purely through software, I suspect that what you wind up with is something that looks a lot like 'green' (user-space) threads with asynchronous IO once you peer behind the scenes of the abstractions that programs are seeing.
The issue has more to do with thread lifecycle management, ie creating, maintaining a stack and destroying the thread. When you have very lightweight tasks, those add up and have no direct mitigations. That’s a big reason we moved runtimes into userspace.
I wish I could see the perf chart of these. If anyone finds them please link. It’s important to have objective facts to base these discussions in.
For thread creation, the author of the original post said "somewhere between 10 and 20 microseconds". This matches my own (very simple) benchmarks on my Ubuntu desktop, which was able to create and destroy just over 100k threads/second.
Depending on your use case, using a thread pool could mitigate this cost entirely.
Asynchronous IO: the next billion-dollar mistake?
Your phrasing here suggests the NT kernel can/does have API’s that reach further down, is that correct, or am I reading too deeply? Haha