Does this do what you are looking for?
https://docs.sqlalchemy.org/en/20/orm/session_transaction.ht...
It's the backing DBMS that will enforce atomicity. Postgres and sqlite allow the semantics of nested transactions with SAVEPOINTs (I don't know about other RDBMSes).