Hacker News Reader

Comment by drpancake | Hacker News Reader

Hey everyone. The idea here is take advantage of the new async/await ORM features in Django 4.1* to build a lightweight background task queue for Django that runs in a single worker process without any external dependencies (i.e. no Redis/RabbitMQ).

It's pretty basic and experimental. Feedback and PRs welcome!

*The Django ORM is still using sync-to-async compatibility layer behind the scenes but they plan to phase this out in future versions.

View on HN

I love the simplicity of this idea because for lots of sites the database works just fine as a queue backend and it reduces the amount of infrastructure needed. I currently use https://github.com/dabapps/django-db-queue for https://devmarks.io which also uses the database to store tasks instead of a dedicated queue infrastructure. `Django Q` also has an option to use the database, but I haven't tested it at all: https://django-q.readthedocs.io/en/latest/configure.html#orm. And if you are already running `redis` for your site, https://github.com/rq/django-rq is another option.

The one benefit of this package is that it is async-first which will be beneficial as Django continually adds in more async capabilities. Nice work! I'm looking forward to trying this out and seeing how it works!

Thanks! I wasn't aware of django-db-queue. As far as I can tell it runs in a single process so I guess running tasks will block any others from running. That's probably acceptable for some use cases.

Interesting that django-q has an ORM broker mode. Looks like it forks workers using multiprocessing under-the-hood.

I've been very excited to see Django embrace asyncio, and your package looks like a great way to do async task queues.

I've built a lot of very low traffic Django sites that are all behind user login for only a few users (think company internal-only CRUD tooling) and being able to use a task queue without having to set up a Redis thing is a big bonus for me.

Excited to see how this evolves, thank you for sharing it!

Exactly. For many projects it's overkill to set up an external broker like Redis or RabbitMQ. That's where I think Chard fits in.

This looks awesome! I'll definitely check it out. Always on the lookout for minimalist django libraries. I'll give it a go in a couple weeks. Thanks for making it and sharing!

Do let me know how it goes :)

> without any external dependencies (i.e. no Redis/RabbitMQ)

You still depend on a database with the `Task` model. This would be a no-go for that reason, since there's no reasonable way to have an impact on its behaviour, outside of creating a custom database router to avoid having every third-party library hitting the same database as core logic.

If you absolutely must use a model, take a look at enumeration types[1] for a slightly "neater" way to declare choices.

[1]: https://docs.djangoproject.com/en/4.1/ref/models/fields/#enu...

Sure but for 99.9% of Django projects you safely assume there is a database available. I'm not entirely sure why that's a no-go.

Reply on news.ycombinator.com