Hacker News Reader

Comment by gilad | Hacker News Reader

RQ [0] (Python-based with Redis backend) is another option; works great. [0] https://python-rq.org/

I love RQ. Nothing but good experiences using it several times over the past ~6 years.

However if you are doin mission critical work in your queue that cannot be replayed Celery is still the best python flavored option IMO

Others above have been suggesting Huey as an alternative. Do you know of any reason that it falls short of Celery in a mission critical situation?

Anything using redis will be riskier than AMQP by default.

Redis can be tuned to be safer (such as persisting on every key change) but this all comes with tradeoffs.

Redis such a great piece of tech I use it first as often as I can.

Sorry why is Redis riskier?

Because it's default configuration and (most) naive setups offer no redundancy? It is almost entirely in-memory with no easy eviction controls? If you are going to snapshot on every change because the thing you are doing needs to happen another purpose built queue like rabbitmq is going to perform better in this situation, especially during failures.

Clustering is another topic entirely.

I've had mixed experiences with RQ. It works great most of the time, but I've observed that you can get into weird states if you OOM your system or otherwise throw curveballs at it. Overall I think Python makes it easy enough to handroll a queue management system that I would generally recommend that over RQ.

Reply on news.ycombinator.com