Socket Sharding in Nginx Release 1.9.1 | Hacker News Reader

Socket Sharding in Nginx Release 1.9.1

nginx.com

47 pointsUncle_Sam13 comments

I was under the impression the `SO_REUSEPORT` implementation on Linux was rather broken, as evidenced by HAProxy's graceful reload still dropping packets[1]. I'm curious as to why Nginx has implemented this when you can design your Nginx config around the problem while not encountering the broken implementation.

[1] http://engineeringblog.yelp.com/2015/04/true-zero-downtime-h...

One of the contributors of REUSEPORT support is a long time Dragonfly BSD developer. No broken Linux implementation there.

How does that help?

Obviously they are telling you that *BSD is better than Linux and that you should switch.

It makes the implementation interesting for Dragonfly, at least. So why not do it?

And if it's offloading some work onto the kernel, having a popular tool support it may push Linux developers to fix the support.

The issue that HAProxy is encountering with SO_REUSEPORT is not an issue for Nginx's use case.

In the HAProxy case one of the sockets belongs to the old server which is going away, and a new connection can be assigned to that socket right before it does so. For Nginx, all of the sockets are accepting new connections.

What happens during the reload case for nginx? At reload time, the previous-generation worker that bound to the socket with SO_REUSEPORT will need to close its listener, just like haproxy.

In nginx worker processes don't open and don't close sockets. They inherit them from the master process.

But isn't that what reuseport changes? Each worker opening it's own socket?

No, they don't (and they can't, since they usually don't have such privileges). The master process opens all sockets for each worker process.

When reloading, nginx starts up a new set of workers and duplicates the sockets across to them with SCM_RIGHTS. None of the sockets is closed - only duplicate file descriptors.

I secretly hope Nginx could implement socket fd sharing via sendmsg, so my websocket worker have direct access to the client TCP connection. No more relaying.

Why don't you submit a feature request for that?

Reply on news.ycombinator.com