Hacker News Reader

Have it run fuzz and test suites. Get with it man. Most of my LLM projects have massive test suites that do a far better job then I ever would have.

View on HN

So you're fine with your code having unintended behaviour, as long as that unintended behaviour passes a test that the agent wrote?

My preferred approach is to read and understand everything the LLM produces AND have it create test suites (which I also read and understand). The LLM can help you with that too - just have it breakdown and explain the code at each iteration.

If it's code for a pacemaker, it should be reviewed by multiple humans. If it's code for an unimportant side project, I will never read it. Between those extremes are shades of grey.

Part of our job in this new era is to understand the worst-case consequences of a bug given how the code interacts with the world, then allocate our effort based on that understanding. This can only be done on a case by case basis.

If you don't, at a bare minimum, read the code once though, you don't know how it interacts with the world.

Did you look at those test suites? Sometimes they test nothing.

Yes, but not all - it has been a long time in my experience that frontier LLMs have written nonsense tests.

Reply on news.ycombinator.com