Hacker News Reader

Comment by lateforwork | Hacker News Reader

The first thing you should have AI write is a comprehensive test suite. Then have it implement the main functionality. If the tests pass that is one level of verification.

In addition you can have one AI check another AI's code. I routinely copy/paste code from Claude to ChatGPT and Gemini have them check each other's code. This works very well. During the process I have my own eyes verify the code as well.

View on HN

The advice that everyone seemed to agree on at least just a few months ago was to make sure _you_ write the comprehensive tests/specs and this is what I still would recommend doing to anyone asking. I guess even this may be falling out of fashion though…

Generate with carefully steered AI, sanity check carefully. For a big enough project writing actually comprehensive test coverage completely by hand could be months of work.

Even state of the art AI models seem to have no taste, or sense of 'hang on, what's even the point of this test' so I've seen them diligently write hundreds of completely pointless tests and sometimes the reason they're pointless is some subtle thing that's hard to notice amongst all the legit looking expect code.

There is no need to write tests manually. Just review the tests, and make sure there is good coverage, if there isn't ask AI for additional tests and give it guidance.

I was anticipating that having AI write code to pass tests (human and/or AI written tests) would be worthwhile, but in practice, I've found that even models such as Opus 4.6 Thinking, High Effort simply "cheats", or rather, fails to generalize much too often. It's occurred to me that perhaps I need some amount of randomness in the tests to keep the models honest, but it feels wrong. We'll see.

Reply on news.ycombinator.com