Hacker News Reader

Comment by floatboth | Hacker News Reader

So, it's like mongrel2, but a bit worse: AMQP is centralized, unlike ZeroMQ.

That was my thought as well, though I never really understood the point of Mongrel2.

It's basically a reverse proxy that speaks to upstream application servers using a custom protocol over ZeroMQ instead of HTTP over TCP? Why is this better than just using HTTP?

Does anyone here use Mongrel2? Do you like it?

I haven't used Mongrel2, though I have used ZeroMQ, but I can try to answer this question:

> Why is this better than just using HTTP?

In SOA most of your services aren't going to be exposed publicly. HTTP is a great protocol for public facing servers, but HTTP is a very clunky protocol. For private services, it's a pretty big benefit (performance, scalability and ease of parsing) to skip HTTP and use something else. ZeroMQ gives you several messaging patterns that you would never get from HTTP.

That makes sense if your services do not follow the request-response pattern (e.g. background workers). But what if they do? In Polyglot, the services very clearly follow a request/response pattern because they're handling web traffic. What sense, then, does it make to use a message queue?

Persistence? Makes no sense for web traffic. Even if the message is persisted to disk, it's useful for a few minutes at most before the user gives up and closes the tab.

Language-independence? You don't need a message queue for that. You can do that with regular HTTP.

Scaling and load balancing? Ditto.

I'm not sure what your comment is getting at. I think HTTP is fine for the Web, and any service exposed and designed for use by the Web is going to have to use HTTP. But if you're going to use a framework like Polyglot, you're going to have several services and any of those services not directly communicating with the Web doesn't need to speak HTTP.

What I'm getting at is why those services shouldn't speak HTTP. I understand that they don't need to speak HTTP per se, but I get the feeling that your comment is implying that such services should speak a non-HTTP protocol, while I think that it's fine even if those services speak HTTP.

Persistence can be very important depending on your needs. Save-and-forget comes to mind.

That makes sense. There are also good reasons to just HTTP: one less technology to understand/troubleshoot, reuse existing tools and middleware.

Yup, I agree with that and in a team environment that may be very valuable. HTTP is definitely the "JavaScript of protocols." However, I don't think learning other protocols is a ton of overhead, and learning is always a good thing ;)

We've had a similar argument internally. In load testing, we're not bounded by the speed of our JSON parsing, and if we were we'd probably just swap that endpoint to use a different message body provider before moving to a totally binary RPC model.

Especially in the Java world, where Jackson + Afterburner is fast enough for most cases. Protobufs/Thrift will smoke it in most performance tests, true enough, but when you're waiting on a database or algorithm to run, what's JSON serialization?

There's certainly benefits at extreme scale, but not enough to justify the loss of tooling that comes with it until necessary.

This might be a weird edge case, but in Rust, 0MQ bindings came really quickly, as its protocol was easy. This let Rust do web stuff _far_ earlier than the still-ongoing work to actually write a server in Rust itself.

At least in Ruby world, we steal Mongrel's parser over and over and over and over. Seems like it might be easier to not have to do that, and just use Mongrel2 with a 0MQ library. I haven't actually done this, though...

We use Mongrel2 at Fanout.io. I love it. It's part of our general ZeroMQ architecture of having lots of components that each do one thing well.

ZeroMQ is an improvement over plain HTTP primarily because you can use fewer internal pipes by interleaving requests and responses over the same sockets.

We even created a HTTP over ZeroMQ spec to help standardize this approach: http://rfc.zeromq.org/spec:33

What do you mean by internal pipes? Do you mean file descriptors? Why is reducing the number of file descriptors a good thing? After all, the total amount of traffic stays the same.

If you have a lot of fds, then you have fd polling issues to deal with for high performance.

To be honest I've not actually benchmarked anything comparing the throughput of an efficient fd poller to multiplexed pipe. It just feels nice to avoid the problem, considering what kind of effort can go into those pollers.

Some discussion from Zed about fd polling: https://web.archive.org/web/20120225022154/http://sheddingbi...

The advantage of Mongrel2 is this: http://zef.me/4502/message-queue-based-load-balancing

Message queue based load balancing.

Also dynamic load balancing without restarting any daemons.

AMQP/Rabbit can use TLS (even if not leveraged in this project, something not possible for ZeroMQ IIRC). Also, define "centralized". An intelligent AMQP client properly falls over to another node in a cluster, and Rabbit offers good durability guarantees while still allowing quorum and tolerating node failure.

But we're talking about HTTP requests here. The RabbitMQ durability is useless for this use case. Suppose the RabbitMQ node fails, and the admin notices after 2 minutes and solves the problem after 2 more minutes. The user who initiated the HTTP request has long pressed Stop in his browser.

True, durability wouldn't help here, however, individual Rabbit queues can fail over to alternate nodes if the master node for that queue stops responding.

Exactly what I was wondering. How different from mongrel2 it is.

Reply on news.ycombinator.com