I know this isn't a stylish opinion, but I've been using beanstalkd (https://beanstalkd.github.io/) in production for years now and never had a problem. Once in a while the server it's hosted on will have troubles, but the process itself is bulletproof and handles thousands of jobs/s without breaking a sweat. It has compacting binlogs, so for mission critical jobs that cannot be recreated you can save them to disk. For jobs that can be re-created, you can run an in-memory instance that can easily handle more jobs than most people here will ever need.

If you want redundancy or write-scaling, you can run 2+ of them side by side, round-robin your jobs to the available instances, and have your workers read from them (RR as well).

Beanstalkd is a lifesaver for job queues. It's the first thing I reach for when I need a queue. I'm excited to see how Redis' Disque turns out, but until then I'm happy with bean.

I've evaluated replacing bean with something like Kafka, but it just doesn't make any sense at all. A distributed log is NOT a queue, and even using something like RabbitMQ gets tricky if you want to pull jobs instead of push them. For dedicated worker queues at small-to-medium scale, I cannot recommend beanstalkd enough. No, it doesn't have native sharding or multi-region replication, but like I said, I run millions of jobs/hour through it and never needed any of that crap.