Nginx 0mq transport layer for communicating with upstream servers | Hacker News Reader

Nginx 0mq transport layer for communicating with upstream servers

github.com

52 pointswlll23 comments

How is this different than Mongrel2?

Wasn't Mongrel2 created solely because NGINX didn't support ZeroMQ.

@ZedShaw, thoughts - comments?

From what I understand, core Nginx decided that supporting ZeroMQ was a non-starter because of the threading model that ZeroMQ uses. I'm not sure if that's just not true anymore, or if the restrictions that this new module places on it (namely, exclusively REQ/REP pairs) make that not apply.

> if the restrictions that this new module places on it (namely, exclusively REQ/REP pairs) make that not apply.

ZeroMQ I/O threads are still being used with REQ/REP, so this doesn't change anything in that matter.

Also, exclusive use of REQ/REP pair isn't restriction, but current limitation of the module ;)

I'm not sure how well adopted mongrel2 has been. There ain't many commits by Zed these days and I don't see blog posts about using it. I don't know why though.

Commits to Mongrel2 are happening near daily. It's in the development branch.

https://github.com/zedshaw/mongrel2/commits/develop

I've been using mongrel2 pretty much since the beginning. I host my own websites with it. Although they are very low traffic I ran some performance tests using blitz.io and the results were impressive. Less than 1% conn errors with a 512MB linode and a simple Django app (no database).

I'm also blogging about mongrel2 lately: http://daltonmatos.wordpress.com

I was one of the first committers on Mongrel2 (actually, second behind Zed) and I still am (though mostly as a historical accident, I haven't contributed anything of substance since SSL).

In my opinion, this is great. I think Mongrel2 is an excellent proof-of-concept, and if it provided the impetus for a much more widely used project like nginx to go down a similar path (though it's just a module right now), then everyone wins.

That said, this certainly isn't in nginx mainline, and it's very feature poor as compared to Mongrel2. One could not simply swap in nginx with this module for an existing Mongrel2 installation.

Alex, do you know if Mongrel2 have any interesting production deployments? In terms of visibility, throughput, or any other metrics. Outside from the architecture discussions, I haven't see any mentions of it.. (and I'm a fan of the architecture).

> it's very feature poor as compared to Mongrel2. One could not simply swap in nginx with this module for an existing Mongrel2 installation.

That's because this is _transport_module_ and what Zed did with Mongrel2 is build a level 7 protocol on top of ZeroMQ messages.

Even if this would support PUSH/PULL + PUB/SUB sockets right now, you would still need an _upstream_module_ that understands Mongrel2's level 7 protocol.

Head's up, I've started working on both:

- PUSH/PULL + PUB/SUB support for ngx_zeromq (transport module),

- ngx_mongrel2 (upstream module).

Should be ready sometime next week.

Mongrel2's ZMQ protocol was designed for lightweight messages against async handlers. Instead of a synchronous ZMQ REQ/REP pair that this nginx module uses, Mongrel2 used a combination PUSH/PULL sockets to send the request, and PUB/SUB to get a response.

Say you want to map /chat to a handler. Mongrel2 opens a PUSH socket, and a SUB socket connecting to one or more chat handlers. The handlers open a PULL and PUB socket. Mongrel2 will spread requests to /chat across the available handlers, and receive all published responses. It's a pretty smart way to handle multiple handlers receiving requests at the same time. It's been a while since I looked at Mongrel2, so it's very possible I have some of the details wrong...

> How is this different than Mongrel2?

It has nothing in common with Mongrel2 other than the fact that both use ZeroMQ transport instead of TCP.

Cool stuff, but I'm wondering why they went with REP/REQ instead of ROUTER/DEALER. That would dramatically reduce the number of file descriptors needed (indeed, a single socket could handle any number of backends / requests).

Also, the fact that ZMQ is message, not stream, oriented makes it non-ideal here. You get the whole response in one big chunk. Across an internal network, this may not be a huge deal, but it's not a desirable property.

It's conceivable that a variant that broke responses up into discrete messages might be better, but this would have more complication in implementation, and I wonder how much it'd have to offer over straight HTTP.

Also, I wonder if the true ideal protocol for this is SPDY without SSL. It just seems like a no-brainer.

This is just an initial spike. The conversation started here: https://github.com/chaoslawful/lua-nginx-module/issues/35.

> Also, I wonder if the true ideal protocol for this is SPDY without SSL.

SPDY does not exist without SSL, nor will it.

Also, SSL does not add anywhere near enough overhead to matter for this kind of application, especially when you just leave the connection established. SPDY would work quite well as a protocol for this purpose.

Speed aside, maintaining certs for all your backend servers would suck.

> SPDY does not exist without SSL, nor will it.

Can you elaborate on this, or provide a link with more context? Almost all TLS/NPN references are missing from the current SPDY draft spec. (I've found webpages that repeat the SSL requirement assertion, but none that source it or explain why.)

The SPDY draft spec doesn't seem to include any of the encryption bits; I don't know why. I've read documents explaining the details before (as well as expressing very strong opinions about the (lack of) merit of a modern protocol supporting a non-encrypted mode), but I can't seem to find them at the moment.

As far as I know, SPDY's protocol always builds on an SSL session, not a raw socket connection. Opportunistic SPDY connections to HTTP servers rely entirely on SSL "next protocol negotiation" (NPN); a SPDY server will indicate that it supports SPDY to an HTTPS client via NPN. SPDY also uses SSL to avoid interference from proxies that would otherwise break attempts to use SPDY on what started out looking like an HTTP connection.

> Cool stuff, but I'm wondering why they went with REP/REQ instead of ROUTER/DEALER.

Because this is just proof-of-concept hacked over 2 days (+ README file). It probably wouldn't be even released at this point, if I didn't promise it to @igrigorik.

I'll look into adding other socket types sooner or later.

> Also, the fact that ZMQ is message, not stream, oriented makes it non-ideal here. You get the whole response in one big chunk. Across an internal network, this may not be a huge deal, but it's not a desirable property.

ZMQ and ngx_zeromq support multipart messages, so this is only true if your backend sends it as one big chunk.

I've used frickle's ngx_cache_purge in production for over a year: it is rock solid. I'm looking forward to finding a use for this plugin!

thanks!

Why? Is it faster, more robust, something, anything????

Proof-of-concept, nothing else at this point.

Reply on news.ycombinator.com