Boosting Nginx Performance with Thread Pools
nginx.com
nginx.com
Most people when hearing about "thread pools" in the context of a web-server think about using multiple threads for handling separate requests, which is NOT what this is about. It is using threads to enable some blocking syscalls (read and sendfile) to run asynchronously from the main event loop.
There's already a library for that! http://software.schmorp.de/pkg/libeio.html
A bit ironic since this is an article about reducing blocking for improving performance.
https://www.usenix.org/legacy/event/usenix99/full_papers/pai...
Great to hear that.
So the answer is it probably would have been implmented years ago if that was the case.
In practice, I've seen plenty of errant bugs because of race conditions in sites that start to come under heavy load. I wish more people would take the time to understand how their platforms work. That said, I've really come to appreciate the node.js approach.
Thread is a wrong idea in the first place - by broking isolation of processes (share nothing principle) they brought in the whole new class of problems with locking and synchronization. Only threads that share nothing is a reasonable choice, but without sharing the whole concept makes no sense anymore. So there are kernel lightweight processes which seems to be good choice for offloading the blocking operations from the main loop.
BTW, Erlang does it right from the very beginning.)
It's a fairly common practice to spawn 2n processes/threads (n processors) to allow half to block on I/O and system calls though.
Note that kqueue(2) in BSD-land supports a unified interface for async IO for both sockets and files, so you can have a proper event loop without having to resort to reading files in a thread pool. If Linux had something similar, a nginx wouldn't need to integrate a threadpool for this (though it might for other things, such as CPU-intensive plugins).
The detail I apparently skipped is that uncached file reads aren't handled uniformly through epoll (which I'm surprised about). I don't see why files should be handled any differently than sockets. etc... in regard to non-blocking I/O using epoll.
Although, my issue is that everyone tends to look to methods starting with aio_ to do asynchronous I/O. Those are fairly bad interfaces (POSIX AIO) and inefficient (effectively threadpools). Using the nginx model with an epoll/kqueue event loop is a better architecture.
That only works for read, write and fsync.
There are plenty of other blocking syscalls that people need to use.
e.g. An important missing one is getdents.
Glibc's posix aio (aio_*(3)), on the other hand, does not use Linux's kernel aio AFAIK. It probably uses thread pools. It also uses signals to signal completion. It is not generally considered performant.
Well at least it isn't a 503