Hacker News Reader

Comment by xg15 | Hacker News Reader

It's not wrong, but I find the underlying corrolay pretty creepy that actually trying to understand those problems and fix errors at edge cases is also a fool's errand, because why try to understand a specific behavior if you can just (try to) finetune it away?

So we'll have to get used for good to a future where AI is unpredictable, usually does what you want, but has a 0.1% chance of randomly going haywire and no one will know how to fix it?

Also, the focus on hardware seems to imply that it's strictly a game of capital - who has access to the most compute resources wins, the others can stop trying. Wouldn't this lead to massive centralization?

View on HN

>So we'll have to get used for good to a future where AI is unpredictable, usually does what you want, but has a 0.1% chance of randomly going haywire and no one will know how to fix it?

Just like humans. I don't think is a solvable problem either.

It’s not “just like”, because humans can be held accountable. Also, I suspect that the distribution of failure modes is actually substantially different between LLMs and humans.

Late to reply, but my point was that even with the ability to be held accountable on pain of unemployment, imprisonment or even death, sometimes even these disincentives are not enough to stop a certain small percentage of humans from going haywire. It's all just something we have to live with.

Add more humans and LLMs to correct for errors. If humans sometimes go crazy and try to randomly end the world at a rate of 0.1%, requiring two humans to turn two keys synchronously to end the world reduces the error rate to 0.01%.

So, to avoid depressed AIs ending the world randomly, have a stable of multiple AIs with different provenance (one from Anthropic, one from OpenAI, one from Google...) require a majority agreement to reduce the error rate. Adjust thresholds depending on criticality of the task at hand.

Let's hope they're not correlated.

Reply on news.ycombinator.com