Hacker News Reader

Comment by sylvinus | Hacker News Reader

I've worked 4+ years with Celery on 3 different projects and found it incredibly difficult to manage, both from the sysadmin and the coder point of view.

With that experience, we wrote a task queue using Redis & gevent that puts visibility & tooling first: http://github.com/pricingassistant/mrq

Would love to have some feedback on that!

View on HN

I'll check this out. I recently started looking around for an alternative to celery. I literally just got over a celery-related bug that took way too long to diagnose and one that took even longer on my previous project.

I'm not very happy with the community either. What with the dispersed, incomplete documentation, multiple discussion forums, and snide responses, I'm really getting ready to wash my hands of it.

This. In my experience Celery's capabilities are greatly oversold, both by itself and others. Most problems Celery purports to solve it tends to just overcomplicate and often not really solve at all. And most the time I've dug into the code I'd rather I hadn't (discovering that the thing you'd assumed was implemented fairly bulletproofly, really wasn't).

And don't get me started on RabbitMQ.

That sounds like exactly what happened to me.

I asked about a progress bar, for a long running web request on Stack Overflow, and Celery seemed to be the accepted way to do that.

I manged to get it set up eventually. Realized a month or so back that it hasn't been running, and it has taken me about 3 times as long to get it up again as it did on the first try. I am sure there must be an easier way.

My major bone of contention is the frequency and stability of releases. There are too many releases and not enough testing before each release. I have frequently found myself trying out a new release because it included a patch I wanted, to only find it has broken something else.

Looks interesting. Can't find any links to the docs?

I can't find any either - it looks like mrq is a front-end dashboard for python-rq: http://python-rq.org/

MRQ is heavily inspired by RQ to which we switched from Celery (http://www.slideshare.net/sylvinus/why-and-how-pricing-assis...)

However it is a complete rewrite because we felt we couldn't add gevent support and other features to provide extreme visibility without major changes. If you don't need those 2 things, you may want to check out RQ instead for now, it's still a very good piece of software.

Awesome!

I see now that mrq supports concurrency why python-rq does not (at least in a stable fashion).

I'll try mrq for the gevent integration. It's great that you guys are actively working on improving it. Python-rq is great too, but it hasn't been updated in a while and I don't think concurrency is on the radar.

We are writing more docs now and putting a small website up, currently they are in the README. We provide support by email, there are already a few third-party users using it in production.

Reply on news.ycombinator.com