Hacker News Reader

Comment by whalesalad | Hacker News Reader

One is a tomato, the other is an orange. From a distance they might look alike but they really are two completely different tools. This is a pretty solid explanation of the differences with good illustrations.

Rabbit can do everything Kafka does - and much more - in a more configurable manner. Kafka is highly optimized for essentially one use case and does that well. Nothing in life is free, there are trade-offs everywhere. I am not privy to which one is theoretically faster - but once you reach that question methinks the particular workload is the deciding factor.

View on HN

Rabbit is an arse to scale past one broker. It was possible, but a pain, that might have changed.

Kafka is just a pain full stop.

The official rabbitmq controller for kubernetes is a breeze. Scales wonderfully without almost any effort.

At a previous company, about 10 years ago, we had roughly 10 RabbitMQ instances (brokers), all isolated. The system was essentially partitioned by queue server. We had a directory-ish service that would associate clients with their assigned node. It worked well, except if a client got too large we might have to move them to another queue.

>> Rabbit can do everything Kafka does - and much more - in a more configurable manner. Sure, if you're doing like 10's of MB/s. RMQ is fast compared to AK if you're not adding durability, persistence, etc. Try to run gigabytes per second through it though, or stretch across regions, or meet RTO when the broker gets overloaded and crashes.. Get your shovel ready! ;)

Kafka itself is dumb but scalable and resilient, it's the client ecosystem that's massive compared to RabbitMQ. Count 10 stream processing, connectivity, ingestion or log harvesting platforms that use RMQ as it's backend, then name 10 languages that have supported libraries for RMQ.. then compare that to Kafka.

I am not an expert in either and have only worked with Kafka. At a past job I had to write a connector job to parse and sanitize some extremely dirty, unstructured data and pass it along somewhere else. RabbitMQ supports this? What is the one use case of kafka? I think you have it backwards.

> parse and sanitize some extremely dirty, unstructured data and pass it along somewhere else

can you be more specific? that to me sounds like hello world for either of these tools. "santize data" is an application level concern that neither rabbit or kafka would handle. as far as "pass along somewhere else" again both tools can do.

It was a Sink Connector. I don’t know what it was or wasn’t supposed to do but I was asked to do it, as is often the case in tech. I could have done any number of transformations in that process though, which I’m not sure rabbitmq supports

It sounds to me like you aren't really even sure what you built. I have operated both rabbit and kafka at scale I definitely do not have it backwards :)

No, I’m not, because it was years ago, and I’m asking for clarification because what was said immediately sounded wrong to me (I’ve managed a lot of rabbitmq deployments) and you’ve not really given one other than an appeal to authority. guess I have my answer. Can’t find anything that suggests rabbitmq natively supports anything like sink connectors. thanks.

> I am not an expert in either and have only worked with Kafka.

> I’ve managed a lot of rabbitmq deployments

... ?

You do not need to be an expert in something's internal workings to manage/monitor a deployment. Surely this does not need to be explained further.

> I think you have it backwards

You do need to be an expert when you start telling other people they’re wrong, though.

A classic exchange on the internet.

So let me get this straight. You've used Kafka once, RabbitMQ never. You don't really know what you did with Kafka. But you somehow know that RabbitMQ cannot do the thing which you don't really remember anymore. Doesn't make much sense to be honest.

Nobody can really have any sources for RabbitMQ being able to do it if you don't know what it supposedly cannot do. The way you descibed it, is that you simply read data and then did something with the data and passed it to somewhere else. RabbitMQ obviously can do it.

> So let me get this straight. You've used Kafka once, RabbitMQ never

Not true, and some also for the rest of your snotty comment, I'd have a response but it's best not to engage trolls. Another commenter answered the question I had. Good luck.

Also, did you register solely to make this comment? Pretty sad display, really.

> Can’t find anything that suggests rabbitmq natively supports anything like sink connectors

Kafka doesnt natively support them either. That would be Kafka Connect. I guess you could use it as an MQ, but it wouldnt be a very good one. Its more used as a data integration platform. If you want more MQ-like functionality OOTB on top of Kafka you would want to use something like Kafka Streams instead.

Thanks for this clarification, this is what I was after.

Reply on news.ycombinator.com