RabbitMQ Highly Available Queues and Clustering using Amazon EC2

As I often have to repeat to people: don't use RabbitMQ unless you need durable messaging. That is to say, if losing a message is catastrophic, and in the event your broker goes down, you are guaranteed to still retain your messages, then I'd recommend RabbitMQ. For example, you do not want to lose a queued financial trade.

Most people really don't need durable messaging. They are typically offloading asynchronous work jobs via messaging. This is where a broker-less technology like ZeroMQ makes sense. You can achieve the same high-availability architecture without having to worry about the availability of your RabbitMQ choke point separating you from your worker instances.

As a side note, some of these features (multi-node synchronization, distributed configuration management, HA) are made possible because of Erlang, which RabbitMQ is written in. It is a classic case of 'the right tool for the right job' example in my book.

Do not forget about the netsplit scenarios. In the case you have two datacenters, and one has network problems, you will end up in a split-brain scenario. Both datacenters may believe they are available or down. A way to solve this, could be to use more boxes and majority voting, with something like apache zookeeper. But none of this comes out of the box.

Maybe I'm missing something but how does the client know what ip address/hostname to connect to? Does it just use some sort of multicast to find nodes?

Does AWS count as highly available?