Not OP, but I do this with SQLite in memory DB. If each unit only adds the tables the test needs, this is super fast, probably around 2ms per table.
This is the same risk with mocking/stubbing, but for integration tests it's important that you're testing with the actual system used in production.
In our case, we decided it was a worthy trade off in terms of developer feedback times on our python microservices, because most of the SQL there is INSERT only. In our golang services which do 90% of the interesting SQL, we spin up PG in a docker container.