Hacker News Reader

Comment by noosphr | Hacker News Reader

It really isn't.

It's s about subpar models trained on subpar data doing subpar work.

The only reason why anyone takes it seriously is that we've had a glut of subpar developers for 30 years so it all balances out in the end.

View on HN

No it really is about the test suite, and provably so. As another poster pointed out, speed is a superoptimization problem and the test suite provides the constraints. If the constraints are appropriately set, even a naive genetic algorithm will eventually improve the outcome over time, provided suitable mixing of the proposal scheme. LLMs provide measurably better proposals than naive approaches, so the entire chain is sound. The issue really is an inability to set appropriate constraints on what the user is looking for, and poor quantification of the multiple objectives one should try to balance in practice. What's great is that's a human problem. Diverting that to the models is obviously a disaster.

I agree that there has been a glut of subpar developers for years, and that has lowered the bar significantly. This is mostly because core values shifted. So let's keep our eyes on what really matters rather than acting elitist.

Tests do not prove the absence of bugs.

I hope you understand the context in which that was said. The point of that statement is that the only way to rigorously verify correctness of a program is by using formal methods. Those are often too difficult to achieve by humans, which is why there is an entire program of developing autoformalization using LLMs. You are supporting what I have said.

In practice, no one rigorously "proves" that their program works at present, and well-written tests do suffice. The definition of "well-written" here is circular, granted, but there isn't really an alternative. Even strong programmers should be writing high-quality testing suites.

Llms are great at converting natural language to a spec because that's a translation from one language to another.

Only a moron would then use them to code up that spec.

Reply on news.ycombinator.com