Hacker News Reader

Comment by josteink | Hacker News Reader

> Because they were trained on internet arguments, and nobody on the internet has ever admitted they don't know something

Or people’s questions were left unanswered (because nobody knew), and there was nothing for the LLM to learn there.

So I guess LLMs can’t learn the absence of answers?

View on HN

Yes, they can. Meta described a clever way in their paper on training Llama3 [1] (in section about factuality). The idea is to sample several answers to a question that you know the answer to. Let an LLM decide if the given answers are different from your known truth. If so, you found a question, that you can train your LLM in the next post-training round to answer with "I don't know". Do that a couple hundred of times and your LLM will identify neurons that indicate doubt and from here on have the ability to answer with "I don't know".

[Edit] The article here also mentions a paper [2] that comes up with the idea of an uncertainty token. So here the incorporation of uncertainty is already baked in at pre-training.[/Edit]

[1] https://arxiv.org/pdf/2407.21783 [2] https://openreview.net/pdf?id=Wc0vlQuoLb

Or, the ai is not thinking at all, just predicting. RTFA

That’s my point. In a unanswered question, there is nothing to predict (ie «learn» or «think» which I never said), so the AI/LLM will predict using a different source which has responded something, correctness or relevancy be damned.

What’s with your tone btw? RTFA? To me that feels quite unwarranted and in violation of the site guidelines.

Chill out, eh?

Reply on news.ycombinator.com