Show HN: Chard – simple async/await background tasks for Django

This looks awesome! I'll definitely check it out. Always on the lookout for minimalist django libraries. I'll give it a go in a couple weeks. Thanks for making it and sharing!

> without any external dependencies (i.e. no Redis/RabbitMQ)

You still depend on a database with the `Task` model. This would be a no-go for that reason, since there's no reasonable way to have an impact on its behaviour, outside of creating a custom database router to avoid having every third-party library hitting the same database as core logic.

If you absolutely must use a model, take a look at enumeration types[1] for a slightly "neater" way to declare choices.

[1]: https://docs.djangoproject.com/en/4.1/ref/models/fields/#enu...

I love the simplicity of this idea because for lots of sites the database works just fine as a queue backend and it reduces the amount of infrastructure needed. I currently use https://github.com/dabapps/django-db-queue for https://devmarks.io which also uses the database to store tasks instead of a dedicated queue infrastructure. `Django Q` also has an option to use the database, but I haven't tested it at all: https://django-q.readthedocs.io/en/latest/configure.html#orm. And if you are already running `redis` for your site, https://github.com/rq/django-rq is another option.

The one benefit of this package is that it is async-first which will be beneficial as Django continually adds in more async capabilities. Nice work! I'm looking forward to trying this out and seeing how it works!

I've been very excited to see Django embrace asyncio, and your package looks like a great way to do async task queues.

I've built a lot of very low traffic Django sites that are all behind user login for only a few users (think company internal-only CRUD tooling) and being able to use a task queue without having to set up a Redis thing is a big bonus for me.

Excited to see how this evolves, thank you for sharing it!

Good idea! Added a section:

https://github.com/drpancake/chard/blob/master/README.md#roa...

It depends a lot on how the application is set up.

> So, my ideal async task thingie would be to start an async worker on the fly when there are tasks; the worker will be killed when the task has finished (or stay alive some time to pick any new tasks, whatever). So no plumbing of running a worker indenepdently.

If you are running a single process in a single thread, then you have one async event loop scheduling all async tasks. In that context, background jobs can run and stay running as long as the event loop is running, and you won't have any problems unless you accidentally call some blocking routine in the background task (then you'll block up the whole event loop because it's all still single-threaded).

You could even have multiple subthreads with one event loop in each, or multiple subprocesses, each with one or more subthreads, each with its own event loop. There is nothing about the design of the Python language that prevents you from doing this. The GIL is annoying but not a dealbreaker.

Things get trickier when you are dealing with a pool of workers being managed by some external process. I can't speak for other languages, but this is the industry standard setup in Python web development. You could of course disregard the fact that anything is different about your setup and go about the DIY process of spawning threads or async background tasks, building machinery to manage their liftetimes, etc. But then you're circumventing not only your web framework but also the master server process that manages the worker pool, and things could get really weird and messy.

Exactly how weird it gets, and how much DIY framework hacking you need to do, depends on the implementation details of your particular framework and the "master server" system that you are using.

Thus we ended up with standalone task queueing/running tools like Celery and Dramatiq (and now I guess Chard), which let you avoid fighting your framework and writing your own background task background task runner, at the expense of running a separate daemon and pool of workers. From the perspective of a developer, this is a great tradeoff because it lets me write web framework code that's focused on serving web stuff, and it lets me write background task code that doesn't have web framework details cooked into it. You could even go a step further and use something like Google Cloud Tasks or Pub/Sub if you want to avoid managing and monitoring another thing.

But I think it's important to be clear that these systems exist not because of anything particularly weird or bad about Python itself, but because of how Python web frameworks generally are expected to work.

That's right although you could take a synchronous function and wrap it with sync_to_async() and it would work. But I'm not sure about the performance implications of that.

https://docs.djangoproject.com/en/4.1/topics/async/#asgiref....