Hacker News Reader

Comment by hnuser123456 | Hacker News Reader

There should probably be a little more effort towards making small models that don't just make things up when asked a factual question. All of us who have played with small models know there's just not as much room for factual info, they are middle schoolers who just write anything. Completely fabricated references are clearly an ongoing weakness, and easy to validate.

View on HN

LLMs by definition do not make facts. You will never be able to eliminate hallucinations. It's practically impossible.

Big tech created a problem for themselves by allowing people to believe the things their products generate using LLMs are facts.

We are only reaching the obvious conclusion of where this leads.

To be sure, a lot of this can be blamed on using AI studio to ask a small model a factual question. It's the raw LLM output of a highly compressed model, it's not meant to be everyday user facing like the default Gemini models, and doesn't have the same web search and fact checking behind the scenes.

On the other hand, training a small model to hallucinate less would be a significant development. Perhaps with post-training fine-tuning, after getting a sense of what depth of factual knowledge the model has actually absorbed, adding a chunk of training samples with a question that goes beyond the model's fact knowledge limitations, and the model responding "Sorry, I'm a small language model and that question is out of my depth." I know we all hate refusals but surely there's room to improve them.

All of these techniques just push the problems around so far. And anything short of 100% accurate is a 100% failure in any single problematic instance.

A talk I went to made the point that LLMs don't sometimes hallucinate. They always hallucinate -- its what they're made to do. Usually those hallucinations align with reality in some way, but sometimes they don't.

I always thought that was a correct and useful observation.

> effort towards making small models that don't just make things up

But all of their output it literally "made up". If they didn't make things up, they wouldn't have a chat interface. Making things up is quite literally the core of this technology. If you want a query engine that doesn't make things up, use some sort of SQL.

If you disable making things up, LLMs will not work. Making stuff up is literally how they work.

I am aware, but think about right after a smaller is done training. The researchers can then quiz it to get a sense of the depth of knowledge it can reliably cite, then fine-tune with examples of questions beyond the known depth of the model being refused with "Sorry, I'm a small model and don't have enough info to answer that confidently."

Obviously it's asking for a lot to try to cram more "self awareness" into small models, but I doubt the current state of the art is a hard ceiling.

> then fine-tune with examples of questions beyond the known depth of the model being refused with "Sorry, I'm a small model and don't have enough info to answer that confidently."

This has already been tried, llama pioneered it (as far as I can infer from public knowledge, maybe openai did it years ago I don't know).

They looped through a bunch of wikipedia pages, made questions out of the info given there, posed them to the LLM and then whenever the answer did not match what was in wikipedia, they went ahead and finetuned on "that question: Sorry I don't know ...".

Then, we went one step ahead, and finetuned it to use search in these cases instead of saying I don't know. Finetune it on the answer toolCall("search", "that question", ...) or whatever.

Something close to the above is how all models with search tool capability are fine tuned.

All these hallucinations are despite those efforts, it was much worse before.

This whole method depends on the assumption that there is actually a path in the internal representation that fires when it's gonna hallucinate. The results so far tell us that it is partially true. No way to quantify it of course.

Do you have any links on tracing a NN for a hallucination/unconfidence neuron? I do worry that it's possible there isn't an obvious neuron that always goes to 1 when bullshitting, but maybe with the right finetuning, we could induce one?

Nope, I haven't come across a work that does that. But I also haven't looked in a couple months.

I don't think there is any math showing that it's the models size that limits "fact" storage, to the extent these models store facts. And model size definitely does not change the fact that all LLMs will write things based on "how" they are trained, not on how much training data they have. Big models will produce nonsense just as readily as small models.

To fix that properly we likely need training objective functions that incorporate some notion of correctness of information. But that's easier said than done.

Given that the current hypewave is already going on for a couple years, I think it's plausible to assume that there really are fundamental limitations with LLMs on these problems. More compute didn't solve it as promised, so my bets are on "LLMs will never not do hallucinations"

How do you know how much effort they're putting in? If they're making stuff up then they're not useful, I think the labs want their models to be useful.

Reply on news.ycombinator.com