Hacker News Reader

HNHacker News

Top New Best Ask Show Jobs

Comment by yayitswei | Hacker News Reader

Parent Full thread

yayitswei·

What are you guys using for durable queues? I'm currently evaluating Redis and RabbitMQ.

View on HN

Pairs of RabbitMQ brokers, used in a Publish-One-Subscribe-Many fashion for availability.

Scaling RabbitMQ at SoundCloud: http://vimeo.com/69960105

Edit: usually with durable queues on SSDs

Edit: text version of the talk: http://blog.pivotal.io/pivotal/case-studies-2/scaling-with-r...

I wouldn't use either of those if you need durability.

What would you use? And why not RabbitMQ durable queues?

Probably referring to:

http://aphyr.com/posts/315-call-me-maybe-rabbitmq

But all systems like this have weak points, just look at the other systems aphyr has analyzed.

Kafka may be worth a look. It's the best message delivery system I've seen in the wild.

Anyone using Kafka with AWS? I'm concerned that it won't be partition tolerant enough to use it there.

http://aphyr.com/posts/293-call-me-maybe-kafka

We chose RabbitMQ, but we're achieving durability by storing messages in a backup database (CouchBase) that appears to handle partitions better. Still not ideal, and more complicated than we'd like.

This isn't helpful. It's a perfectly valid question that even provides insight into some possible candidates. The candidates might not be the best options, but they commonly serve this purpose.

What is it about RabbitMQ or Redis durability that you find lacking? What alternative do you recommend?

I've looked and have not found any alternatives that are better.

Did you rule out Kafka? If so, on what basis?

and here I am, staring down the barrel of Erlang. again

What type of issues are you having with Erlang?

Reply on news.ycombinator.com