Hacker News Reader

Comment by serjester | Hacker News Reader

This misses that if the agent is occasionally going haywire, the user is leaving and never coming back. AI deployments are about managing expectations - you’re much better off with an agent that’s 80 +/- 10% successful than 90 +/- 40%. The more you lean into full automation, the more guardrails you give up and the more variance your system has. This is a real problem.

View on HN

Sutton might have said you just need a loss function which penalises variance and the model will learn to reduce variance itself. He thinks this will be more effective than hand coded guardrails. He's probably right.

I don't know how you write that loss function mind you. Sounds tricky. But I doubt Sutton was saying it's easy, just that if you can do it then it's effective.

Penalises on training? Not runtime? The risk is that.

You don't have to tolerate agent/AI going haywire. In a simple example, say of multiple parallel generations. It's compute intensive and it reduces the probability of your agent going haywire. You need mechanisms and evals to detect the best output in this scenario of course, that is still important. With more compute, you are preventing your final output to be haywire despite the variance.

Do you have a real world example of this? Claude Code for example doesn’t fit the pattern of “higher success but more variance.” If anything the variance is lower as the model (and tightly coupled agent) gets better.

The only AI I've ever dealt with is unwillingly, when companies use AI chat bots to replace human support. They certainly make me want to leave and not come back.

Reply on news.ycombinator.com