Hatchet looks pretty awesome. I was thinking about using it to replace my Celery worker. However, the problem is that I can only use the gRPC client to create a task (correct me if I am wrong). What I want is to be able to commit a bunch of database rows altogether with the background task itself directly. The benefit of doing so with a PostgreSQL database is that all the rows will be in the same transaction. With traditional background worker solutions, you will run into two problems:

1. Commit changes in the db first: if you fail to enqueue the task, there will be data rows hanging in the db but no task to process them

2. Push the task first: the task may kick start too early, and the DB transaction is not committed yet, it cannot find the rows still in transaction. You will need to retry failure

We also looked at Celery and hope it can provide a similar offer, but the issue seems open for years:

https://github.com/celery/celery/issues/5149

With the needs, I build a simple Python library on top of SQLAlchemy:

https://github.com/LaunchPlatform/bq

It would be super cool if Hatchet also supports native SQL inserts with ORM frameworks. Without the ability to commit tasks with all other data rows, I think it's missing out a bit of the benefit of using a database as the worker queue backend.