Hacker News Reader

Comment by hnrodey | Hacker News Reader

I'm very well versed on RabbitMQ. We use it internally in a .NET codebase.

Anyone considering RabbitMQ needs to read up on "network partitions", how to build your cluster to avoid them (odd number of nodes and pause_minority), your recovery strategy for when a network partition occurs (it will occur), your personal/organizational tolerance for message loss and a plan for how you will upgrade your cluster at some later date (ensure you architect your application to handle whatever type of upgrade strategy you will pursue).

There are definitely ways to operate to minimize these failures but you SHOULD KNOW ABOUT THEM before your add this service to your environments.

View on HN

If you're using RabbitMQ on .Net, I highly recommend using NserviceBus. It's made working with queues so easy. It handles maintaining a connection and retrying/acknowledging messages for you.

I worked on NServiceBus years ago (not just used, but actually was an active developer on the project). It's an excellent piece of software and Udi Dahan really knows what he's talking about.

Hindsight is the best site. That's definitely what I would do if I was starting a new project using RabbitMQ. Although I'll defend myself on this front; I inherited our RabbitMQ project from the developer who left the company 7/8 of the way through the implementation. I had the "make it work" directive and not the decision making luxury he had from the beginning.

This. We used Rabbit in our platform for a few years and it was an absolute disaster. Network partitions mainly. In retrospect I'm sure we were doing it wrong but that really wasn't obvious at the time.

Can you talk a little bit about how you've managed your RabbitMQ infrastructure? Also if you've done any comparison to Azure Message Queues and what were the pros and cons against Rabbit?

I'm looking to pitch adding a message queue to our infrastructure (at a .Net shop on Azure), and I'm sure there will be some questions about the comparisons between the two. Unfortunately, that's been tough to really track down.

>Can you talk a little bit about how you've managed your RabbitMQ infrastructure?

From a ten thousand foot view, two or three node clusters running in non-prod environments on virtual machines running Windows. In Prod, three node clusters on Windows virtual machines.

All work to install and configure RabbitMQ is done manually. Sadly enough.

I'm on the application/architecture side of this equation but I know enough about our infrastructure to perhaps answer follow-ups or more specific questions.

Our application is single tenant (so each customer is deployed in their own isolated area) so we use virtual hosts to isolate each customer within the cluster.

>Also if you've done any comparison to Azure Message Queues and what were the pros and cons against Rabbit?

Definitely looked in to the Azure native queueing options but it's been awhile. Azure Message Queues is an AMQP compliant messaging system that seems fairly robust. To be transparent, I have no production experience with this product. If your company/department is in to managing virtual machines then they might want/prefer to go with RabbitMQ. However, if they're in to PaaS systems then I'd probably roll with Azure Message Queues and never look back.

Thanks for the response. From the other responses in this thread, it seems like the admin of the nodes/cluster is not overly onerous. Would you agree with that statement? Also, being a .Net shop, the Windows VMs make sense, but is there any tradeoffs to running Rabbit on Windows, as opposed to Linux?

I think part of the sell is how we would manage the admin component of a Message Queue, which tilts things towards Azure Message Queues as it's PaaS. We're mostly IaaS at the moment, and starting to see some of the admin overhead that comes with managing that infrastructure ourselves. We're not ready to jump onto a PaaS solution for the things we've grown accustomed to managing, but for something brand new, I think my company would be open to it.

Architecturally, we'd lean on it initially for background job processing, which is currently at a scale where our homegrown, db-backed solution is starting to show it's weaknesses. Once it's in place though, I think it could leveraged as a key component to decouple subsections our application and give us more flexibility with scaling and deployment.

RabbitMQ runs perfectly fine on Windows too. As others mentioned in the comments, RabbitMQ supports a great variety of use cases. If you want to reach out for help, you can find my contact in the article.

Download and set aside a copy of the erlang and rabbitmq installers if you're running on windows... I've had issues on many occassions with the erlang installer being unavailable or very slow to download.

>From the other responses in this thread, it seems like the admin of the nodes/cluster is not overly onerous. Would you agree with that statement?

Yes.

>is there any tradeoffs to running Rabbit on Windows, as opposed to Linux?

Should be fine to run on Linux assuming you (or you have) people are who are comfortable admin'ing Linux servers. I think that a Windows admin would get frustrated to setup/configure RabbitMQ on a Linux server. There's also a container advantage as RabbitMQ is published to Docker only with officially maintained Linux images.

>We're not ready to jump onto a PaaS solution for the things we've grown accustomed to managing, but for something brand new, I think my company would be open to it.

I'd push you to figure out why Azure Messages Queues would not work for you. If there's no compelling "no" argument then you'll thank yourself later.

>Architecturally, we'd lean on it initially for background job processing, which is currently at a scale where our homegrown, db-backed solution is starting to show it's weaknesses.

We pursued RabbitMQ for very similiar reasons (queueing mechanisms via SQL Server tables and stored procedures). Keep in mind that you still need something to submit the job (initiate the background task). RabbitMQ is not going to automagically schedule anything for you. We have a couple applications that use the tool Hangfire for job scheduling and in one case, the Hangfire job simply sends a message to RabbitMQ.

Also, if Redis 5 is already part of your stack then you should look at their Streams feature before adding anything like RabbitMQ or Azure MQ.

Unfortunately streams were released after we introduced RabbitMQ to our application and I really wish we could just focus on Redis.

I'll add one comment, if you aren't doing a really large number of queued items (under 50k messages every few minutes), Azure Storage Queues are pretty nice and easiest to use imo.

Reply on news.ycombinator.com