Hacker News Reader

Test cases being nonsensical and not testing any real world cases was extremely common even before AI, in my experience.

When teams rely on metrics like code and branch coverage, developers will write tests that only serve to satisfy those metrics. I've seen tests that end with "assert true == true"; they don't test anything and only exist to "cover" a branch.

The situation has improved vastly because of AI. AI agents frequently write good tests that have a real chance of actually catching bugs and improving the reliability of the application.

It's human nature to use a probabilistic strategy of cheating until one cheat works. We will never eradicate that, and the careful person that checks the output of AI will always be a minority.

This happens all the time. I looked at a PR, many tests were broken, told the guy he needed to at least fix the previously working tests, got tons of slop like changing assert 200 to 500.