You have to be careful by exactly telling the LLM what to test for and manually check the whole suite of tests. But overall it makese feel way more confident over increasing amounts of generated code. This of course decreases the productivity gains but is necessary in my opinion.
And linters help.
I worked on a team where we had someone come in and help us improve our tests a lot.
The default LLM generated tests are bit like the ones I wrote before that experience.
I agree with this.
I've found I need a whole separate cycle of test writing to get proper (in both scope and accuracy) coverage.
It does help tremendously with all the boilerplate of tests, and it seems to be quite good at setting up numerous tests for all combinations of variables. It does have to be done explicitly, though.
And you do need to mind when a test fails whether it fixes the test or the code.