Mongrel2 v1.6 Released | Hacker News Reader

Mongrel2 v1.6 Released

sheddingbikes.com

47 pointscwb19 comments

I'm still not quite sure where Mongrel2 fits in the stack. Is the idea to just take care of the nitty gritty details of writing an asynchronous HTTP server and expose it to app servers via ZeroMQ? Like an asynchronous version of FastCGI?

If I already have an app server that has a good asynchronous HTTP server built-in (Tornado, Node.js, etc) is there any reason I'd want to use Mongrel2?

From the documentation:

"Other web servers will let you talk to any language as a backend, but they insist on using HTTP proxying or FastCGI, which is not friendly to asynchronous operations."

Maybe I don't understand why HTTP proxying isn't async friendly? Node.js does it quite well.

Not trying to knock Mongrel2, I'm just trying to understand better.

Zed if you're listening, some example deployment scenarios would be helpful.

Good questions. It fits at about the sample place you'd put nginx or apache, with it in front of your other language application stacks. Mongrel2's protocol also tends to remove the need for any "middleware" like WSGI or Rack since it's protocol is already similar to what those do (minus the insane chaining and wrapping they do).

The reason I say "HTTP isn't async" is that, while it can be async over a single socket, you can't really do N:M responses with it. For example, you can't have nginx send a request to one Python backend, and then have 6 other backends respond directly to the client before closing the connection in a 7th. Mongrel2 does this by sending 0MQ messages to one target, but allowing any handler to remotely "write to the browser socket" with any number of 0MQ response.

The closest you can get with HTTP is having specially written proxies that know how to do individual HTTP requests to each backend, and then take those responses and use chunked-encoding to build the final response. I believe Amazon does this fairly effectively for their applications (or used to).

So, I think the confusion on the term "async" is that you're thinking "async socket IO via callbacks in Node" vs. "10 processes asynchronously responding to one browser".

That answer it?

That helps, but I guess it begs the question of why you would want "10 processes asynchronously responding to one browser"? And what does that really mean?

It doesn't sound particularly useful for traditional HTML responses (unless you had some kind of simple template system to aggregate them into a coherent response), but I could see it being interesting for WebSocket applications, especially multiplexing multiple backend services over a single connection. For example, on Facebook's chat and news feed services multiplexed over one connection. Is that a typical use case?

It would be really awesome if Mongrel2 could support something like Socket.IO or Orbited, which provide a WebSocket-like API to the browser, but automatically negotiate another transport if WebSocket isn't available.

Maybe it already does, I don't know. I'd just like to be able to use something like WebSockets without worrying about the capabilities of the browser or backends (Socket.IO is for Node.js and a few other platforms. Orbited seems like a dead project, but I believe it used the STOMP protocol).

> That helps, but I guess it begs the question of why you would want "10 processes asynchronously responding to one browser"? And what does that really mean?

Two use cases: A background task that involves a few systems, but you don't want to block the main app stack. Simplest case is say image crunching on an upload. More complex case is what you'd do with something like Gearman.

Second: You can carve up your pages into each piece and plug the pieces together. This would require a bit of coordination, but you could have a backend for headers, one for content, side bars, footer, etc. and they all respond when they get results directly to mongrel2's socket with a chunked-encoding.

> It would be really awesome if Mongrel2 could support something like Socket.IO.

It already supports Flash sockets and long polling, so Socket.IO should just work. Haven't tried it though. Check out the docs and look at the chat example in examples/chat/ for a quick (slightly old) demo that's just using jssocket.

This is pretty cool. One thing tho:

The @routename construct is still not explained in the documentation, but used in that documentation.

I hate to be this guy, but I pointed that out to the mailing list before the last release, along with a request for advice on how to proceed with fixing the docs, and got no reply. I figured since Zed bragged about fixing the routing docs, they would have included some sort of mention of the mysterious symbol, but alas, no.

Oh yeah, forgot about that. Here's the ticket:

http://mongrel2.org/tktview/8506d37df4c3bdcebc43233003ad119e...

This is a project to watch. Language agnostic web server with a 0MQ back-end to automagically take advantage of multi-threading/multi-core on modern hardware. Nice.

Plus he is running unit tests against code samples in the manual. Brilliant!

Here's the coverage report:

http://mongrel2.org/static/coverage/

This release had a drop of about 17% from the last run, but I'll bring it back up in the next one.

I'm wondering why TNetstring types come after the data. This means you have to buffer the entire data before you can parse it. Is this intentional?

It's for backwards compatibility with regular old netstrings. It works very well in practice. Implementations are dead-simple and super fast in just about every language.

Yep, you have to anyway. It's to prevent buffer overflows so that you can't "cheat" and try to stream it. You have to read it, allocate, then save it. It also means you can abort after just reading the size and ':' if it's too big. Rather than designing your protocol to allow insane buffer sizes to "stream", just have things send reasonable easily buffered chunks. That's more reliable and easier to manage.

The other reason was so it was backward compatible with netstrings. The : was already used by netstrings, but the ',' wasn't so overloading that was easiest.

I'm constantly amazed at how much stuff Zed cranks out.

Welll, to be honest, life sort of pushed Mongrel2 to the side temporarily. The book, teaching classes, and the new job took over until they stabilized. But, it's back and people were using it in production already so this means more progress to come.

But dude, to put it in perspective, I wish I were half as focused as you. Like I said, continually impressed. Mad props.

TNetStrings seems quite similar to Bencode less the size prefix.

http://tnetstrings.org

http://enwp.org/Bencode

(sample implementation) https://github.com/bittorrent/bencode/blob/master/bencode/__...

Yep, except, not crazy. :-)

Care to elaborate? :)

DevOps folks are going to love the upcoming "config server" protocol that Zed is proposing to do for v1.8.

That's where I'm getting that feature idea from. I think the control port, config anywhere, fast dynamic restarts, and filters will make it the bee's-knees for hosting.

Reply on news.ycombinator.com