But some of the buggiest stuff I've dealt with were in codebases that had full coverage. Because none of the tests were designed to test the original intent of the designed code.
In another view, this might just be a fancy way of doing snapshot testing, use AI to generate all the inputs to produce a robust snapshot, but realize the output isn't unit tests, it's snapshots that report changes in outputs that devs will just rubber stamp.
From the PR: unit tests: what are they good for?
Answer: Personal opinion - writing unit testing is not fun. It becomes even less appealing as your codebase grows and maintaining tests becomes a time-consuming chore.
However, the benefits of comprehensive unit tests are real:
Reliability: They create a more reliable codebase where developers can make changes confidently
Speed: Teams can move quickly without fear of breaking existing functionality
Safe Refactoring: Code improvements and restructuring become significantly safer when backed by thorough tests
Living Documentation: Tests serve as clear documentation of your code's behavior:
They show exactly what happens for each input They present changes in a human-readable format: "for this input → expect this output" They run quickly and are easy to execute This immediate feedback loop is beneficial during development
The problem is that this assumes that the tests or the method was written correctly in the first place. If the behavior in the method is wrong and the tests are validating that the behavior is wrong, then you pay an extra tax. First to fix the behavior of the method, then to fix the behavior of the tests.
That's why automatically generating unit tests is in my opinion adding a bomb to your codebase. The only exception is stuff like basic parameter testing but even that can be questionable at times (is null a valid input at any point for example) unless you know the intent of the code, and AI can't really grasp at the intent.