Hacker News Reader

Comment by reubano | Hacker News Reader

Why should I use this over the simplier RQ?

RQ is certainly useful (with finer control over messages), but we've been having a lot of problems with it lately in production due to it being memory bound (Redis). Improper code deploys or insufficient/stuck workers would quickly explode the queues and make the broker go oom in matter of hours. Memory is also very expensive. With KQ/Kafka I was hoping it would provide us with a lot more to buffer for human errors and scale better.

Ok, that makes sense. I've never dealt with the scale you are talking about so have yet to run into these issues with redis. It may be best to put this info in the readme so people can decide for themselves when to use RQ or KQ.

Or the proven Celery

This Celery? https://news.ycombinator.com/item?id=12844206

I mean, Celery is a pretty amazing library. It has flaws, but I doubt this new project has no flaws. They are just unknown still.

For that specific bug in question, does KQ have logic around DST switch? I was not able to find any. So it looks like you would have the same bug in KQ, right?

Dunno why people would blame the Celery devs for that bug - if you're setting the clock on production servers to a non-monotonic TZ you are going to have a bad time.

So you basically blame people for living in countries that use daylight saving time? O_O

Also, I would understand if it were tasks between 2AM and 3AM that run twice in transition from summer time to winter time (there are two 2:30 that day, after all), but the bug affected tasks scheduled for the middle of the day.

Reply on news.ycombinator.com