Hacker News Reader

Comment by curiously | Hacker News Reader

any real reason you should use zeromq over rabbitmq?

The two approaches are very different. RabbitMQ centralizes and uses a fairly heavy asymmetric protocol (that I originally designed, so I know this pretty well), which makes the server a major hub, and clients rather stupid and slow.

There are things you just cannot do with AMQP and thus RabbitMQ. The main one, and why we originally started ZeroMQ, was multicast from publishers to subscribers. How else can you get message rates of millions per second?

ZeroMQ (and Nano and the many similar efforts that have come to life since we started) gives every process the same capability. Your laptop can talk directly to ten thousand peers, and do it rapidly.

We do have a broker, Malamute, which works nicely for various patterns. I use this in lots of projects. However not in the conventional sense of starting a process on a box somewhere. I use Malamute as an actor thread, to coordinate events and workload between other actor threads, in a single process.

ZeroMQ makes this kind of scale magic quite easy. Take a look at the CZMQ library and you'll see how wise use of ZeroMQ transforms even an old language like C. Thread-to-thread messages work the same way as process-to-process messages. You can't even think of such things using AMQP.

It does change the way you build distributed systems.

I want to use ZeroMQ but stuck with RabbitMQ because I am using Celery.

I hate Celery and I hate RabbitMQ because it was so difficult to get stuff working the way I wanted which makes me wonder if it would've been better if I just wrote my own simple job queue.

They're fundamentally different things. For your needs, how about rq?

http://python-rq.org

ZeroMQ (in certain configurations) is broker-less if I recall correctly. The clients connect to each other, rather than going through a broker.

Last I checked, ZeroMQ is more of a "low-level" library/framework that provides easy-paths to more higher-level functionality that would be comparable with what you'd expect from a standard message queuing system.

Anyone got a different take from it? I only did the tutorials quite a while back.

Yep, it's best to think of ZeroMQ as networking library than an "MQ". It makes communication between processes easier than writing BSD sockets code by hand.

For example, the Mongrel2 web server uses ZeroMQ to drive backend handlers. You probably wouldn't dream of transporting HTTP requests and responses over a brokered message queue, but ZeroMQ is different since it's "just sockets".

Yes, ZeroMQ is brokeless. However, you can implement a broker with ZeroMQ. In addition, it should be noted that ZeroMQ buffers messages when the connection is disconnected.

If micro (possibly nano) seconds matter, you can't use a broker system like rabbit. 0mq is a set of wrappers around tcp and udp and allow you to do network programming at extreme performance levels generally not available without a whole lot of mucking about.

While this doesn't answer the question, one thing I've found (see my other comment for why I don't care for 0mq) is that Rabbit really doesn't like to be full as a queue.

It really requires that you can eat messages off as quick as you can put them on the queue, or else it slows down -- which means you can't pull them off as quickly, and then things basically get to "cascade failure" type problems.

Maybe others have other experiences - Rabbit is fast, but I'd probably evaluate other options if you want your queue to actually, well, queue.

zmq is not a message broker like rabbitmq. its more like a concurrency library. it kinda feels like working with sockets, except its not just sockets.

hn-thread: https://news.ycombinator.com/item?id=6739231

This is not entirely accurate. The difference is that ZeroMQ doesn't have a centralized broker, not that it doesn't allow you to work with sockets. Lots of ZeroMQ projects utilize sockets for communication. In fact, because there isn't a broker, ZeroMQ gives you more freedom as to the underlying communication medium.

In the Postface section one of the contributors writes:

"We investigated different solutions to find something suitable for our needs. We tried different message brokers (RabbitMQ, ActiveMQ Apollo, Kafka), but failed to reach a low and predictable latency with any of them."

Reply on news.ycombinator.com