Hacker News Reader

Comment by NickNaraghi | Hacker News Reader

It's a funny thing to write, like an article in an old newspaper that aged quickly. I suspect that this will be wildly out of date within 2-3 years.

View on HN

There is this belief that in 2-3 years AI will be much better and all the gripes people have with AI use today will be solved. Honestly, personally, I think that optimism will age poorly. But to say it out loud at work or post publicly probably hurts my career prospects.

I think it's already out of date with verifiable reward based RL, e.g. on maths domain. When "correctness" arguments fall, the argument will probably just shift to whether it's just "intelligent brute force".

The set of tasks for which "correctness" is formally verifiable (in a way that doesn't put Goodharts Law in hyperdrive) is vanishingly small.

"stochastic genius"

It's already out of date because it makes no sense. If it's true that the superficial signals of quality were once somehow good enough to keep the entire economy on the rails (it's not true), surely you can have an LLM look at given piece of work and extract comparably useful signals of quality or effort.

> If it's true that the superficial signals of quality were once somehow good enough to keep the entire economy on the rails (it's not true)

It was true. The negative signals (we called them "code smells") weren't the be-all-end-all of reviews, they indicated to the reviewer where to spend more effort. It got us 90% of the benefit of an in-depth review with 10% of the effort. But with LLMs eliminating this, we now have to spend all our effort on everything, taking a lot more time and energy overall.

I think it’s true that we were able to establish trust and produce good work without verifying every detail — what I’m suggesting is that signals of that kind were not a very important factor. And code smells still work!

Reply on news.ycombinator.com