Hacker News Reader

Comment by benjamincburns | Hacker News Reader

This kind of validates an idea I've been flirting with: a Heroku-like service which routes requests via AMQP or similar message broker and actually exposes the routing dynamics to the client apps.

From a naive, inexperienced view the idea of having web nodes "pull" requests from a central queue rather than the queue taking uneducated guesses seems to be a no-brainer. I can see this making long-running requests (keep-alive, streaming, etc) a bit more difficult, but not impossible.

What am I missing? This seems so glaringly obvious that it must have been done before...

View on HN

That is exactly how an app I develop at $WORK works.

Requests come in on a front-end, it gets passed off to a router that has multiple different workers connected. The workers send a request to the router letting the router know that they are ready to start responding to requests. The router hands the worker a request, marks the request as being worked on, and moves on to the next request. It is a basic Least Recently Used queue at that point, and if all workers are busy but worker number 3 which received work last finished first, he gets handed new work instantly.

The worker then sends the request back to the router, which sends it back to the appropriate front-end that originally responded to the user.

We are using ZeroMQ for our communication.

For our use case we can handle around 200 requests a second from a TCP/IP connected client to our router, to a worker and back to a client. That is with 3 backend workers, which are hitting the disk/database.

It has worked very well for us.

The pull model is very hard to implement because the router behaves like a proxy for a much larger set of dynos (think tens of thousands). When you have 10K clients yielding "i'm available" 10 times a second, you have a nightmare, it's not sustainable.

A possible solution for the proxy and the dynos to agree on a protocol where the proxy passes a request to the dyno and the latter can give up with a status code that says "retry with another dyno". This could go on to up to the 30s timeout limit that Heroku has now.

In Mongrel2, app servers subscribe to a named ZeroMQ queue and, when they're done, they send the response on a different queue.

You can actually configure an arbitrary number of different queues if you like, switching on request path and some other stuff I don't just now recall.

I believe Mongrel2 (http://mongrel2.org) is close to what you're talking about. It uses zeromq (http://www.zeromq.org) to talk to a backend application and the backend application talks back to the web server using the same protocol.

Not exactly nodes pulling messages off a queue, but closer to something like that?

It has: https://github.com/paulj/trapeze

You might also be interested in what's been done in the literature; look under "Staged Event Driven Architecture" aka SEDA.

Mongrel2 and OK Web Server both support the SEDA approach.

The problem with this model is you'd need to re-engineer all existing web applications to work with the new service. If I just want to boot up an open-source Rails/Django/node.js/etc app on your service, how would I do it?

Reply on news.ycombinator.com