Hacker News Reader

Comment by rdtsc | Hacker News Reader

A few places I could see using the reactor pattern over the "concurrency unit (i.e. green process, thread, co-routine) per connection context" is in very high performance scenarios and where callback chains are very short.

Think proxies (haproxy), routers, forwarders, web servers (nginx) etc. Where memory context per connection should be minimal, and everything should be as close as possible to the select/poll/epoll loop.

Funny enough, this also includes demos, quick example scripts, and benchmarks. I wonder if that what hooked most people to the reactor pattern -- small examples in Twisted, Node, etc, look pretty easy and simple. But when you start adding business logic and callback chains evolve into callback/errback trees 10 levels deeps when things get very scary.

View on HN

You make a very important point here. When you're creating a very high-throughput network server, you want to use as little Reactor and proactor I/O makes it very easy to reason about execution order and more importantly memory usage. This kind of code usually gets written in C (or at the very best C++), and minimum memory usage per-socket is carefully planned.

Co-routine context and and green thread stack size can be tweaked and optimized sometimes, but if you want to have precise control over memory allocated per-connection, reactor/proactor is hard to beat.

Besides, to be 100% honest, C and C++ just doesn't have native support for any other concurrency model. That's probably the main reason why Node.js was designed to use callbacks. I'm pretty sure it would use generators or async functions if it was designed today.

Funny thing about "high-performance scenarios" is that the reactor pattern is often slower than other patterns. If you want to use the reactor pattern, you have to use asynchronous IO. But, asynchronous IO involves making more syscalls. Blocking IO is actually rather fast. The performance is differences are going to depend on a lot of particulars.

If someone has told you synchronous I/O is slow, they've obviously misunderstood the entire C10k argument going on for a while.

The problem with synchronous I/O is not speed, but blocking. Blocking means the only possible concurrency model is based on processes or threads. Consequentially, most performance problems with synchronous I/O stem from the thread model: context switching, synchronization, memory overhead, thread-pool starvation and so on.

> Blocking IO is actually rather fast.

Right. Yeah there was a presentation about Java concurrency patterns about that. Basically dispelling the myth that is everything non-blocking and asynchronous will be faster than the old school blocking thread / socket.

Interestingly, I believe, haproxy and nginx use a hybrid model. They have a worker thread per CPU, all listening to the same socket using epoll from multiple threads! When data arrives they all get woken up and then use a shared mutex to decide which one will handle the request. (Now later kernels fixed that one can have exclusive wake-up across the same socket).

Nginx implementation may have changed since the last time I delved into it (and I delved pretty deep), but at least 2 years ago that was not the case. Nginx did have some vestiges of an abandoned multi-threaded implementation attempt from the 0.x days behind compiler flags, but beyond that it was thoroughly single-threaded, but there is a master process which forks itself into multiple processes.

There is a so-called accept_mutex, but I'm pretty sure you can't avoid that if you want to have multiple cores handle connections from the same port. Even Erlang would have to do that somewhere behind the scenes. Newer Linux kernel versions support the SO_REUSEPORT which is meant to address this situation - I guess this is what you're referring as exclusive wake-up accross the same socket?

Interestingly, Java 7’s "native non-blocking IO" actually uses exactly that – you have a ServerSocket, and, if a user connects to the server, it spawns a Thread and gives it a normal SocketChannel.

This completely depends on the kernel.

> proxies (haproxy), routers, forwarders, web servers (nginx)

...all stuff that people should not implement in javascript in the first place.

Someone at a place I used to work got the Node.js fever and stuck node's http proxy in front of our public facing servers. That was an epic disaster from multiple points of view: stability, performance, debuggability etc.

Or you use promises/observables (with arrow functions) or generators or async functions, and they don't. 2014 called, they want their node problems back.

Basically almost none of the criticism in this whole thread is true today. The only remaining true bit is that node still relies on cooperative multitasking and one of the tasks can hog the CPU of a single worker. Which isn't very good, but still, way better than say, the good old Rails 1 request per process model.

Someone should probably do a proper benchmark to show how different numbers of workers at different CPU workloads affect a node service's response time / latency. Especially with multiple processes (cluster), I would bet the effect would be much better than what people expect it to be.

Reply on news.ycombinator.com