What does it mean when an LLM “hallucinates” & why do LLMs hallucinate? | Hacker News Reader

What does it mean when an LLM “hallucinates” & why do LLMs hallucinate?

labelbox.com

9 pointsandsoitis22 comments

Open articleView on HN

> LLM hallucination occurs because the model's primary objective

Stopped reading here. A model has no objective. It has only output.

The only objective is on the part of "AI" marketeers and it is there that the euphemism "hallucination" originates.

"A model has no objective. It has only output."

Is creating output not an objective?

Objective signifies intent, no?

Maybe, somewhere, in the minds of the creator of the model.

But an objective function is a core element of ML models, no? And LLMs have objective functions, so far as I'm aware.

Sure, but it doesn't need to be organization the LLMs objective. A soldier could have an objective that they don't agree with or didn't choose themselves.

In star Trek there's the prime directive, the enterprise crew follows it because it's a core requirement of the federation.

Just because the LLM has an objective to answer queries for example does not mean it chose it. or has the intent itself as sapient being would.

No, it has only input. Divergence from reality (insofar as it can be centered) is a hallmark of human thought as well, so perhaps the models are simply performing.

It's reasonable to call minimising the training loss function the "objective" of the LLM, I think (said loss functions are often called objective functions, after all), though we must careful that the overloading such term in an anthropomorphic fashion. Whether "producing clear and coherent text" is a good characterisation of a "predict the next token" loss function is another question though. I would say it probably isn't, but it's a resonable hypotheses to say that the nature of the training is why the output tends to bias towards plausible but incorrect text as opposed to some variation of "I don't know".

Agreed there is a semantic argument here. I don’t think anyone is anthropomorphizing LLMs. I’ve had to adopt “objective” with one customer instead of “task” (my preferred default term) because their industry overloads “task” with something else. Now that said, as you point out, the real objective is next token prediction not “help the user with a truth bomb.” and it just happens to have emergent accidental usefulness (I think a natural bias from the training data).

They (chatgpt et al) do move closer to “clear and coherent” by adding extra layers such as a beam search on LLM outputs. Good to remember that ChatGPT et al are products, not bare metal LLMs.

> I don’t think anyone is anthropomorphizing LLMs

The "AI" marketeers' use of "hallucinate", "objective" and "predict" does precisely that.

> It's reasonable to call minimising the training loss function the "objective" of the LLM

This article isn't doing that, though. What it is calling the model objective is "producing clear and coherent text".

> Whether "producing clear and coherent text" is a good characterisation of a "predict the next token" loss function is another question though.

I agree. But lets us be clear that prediction too is a misnomer. Simply generating a word coherent with prior words is not prediction.

Objective as in objective function.

So "LLM hallucination occurs because the model's primary objective function is to generate text that is coherent and contextually appropriate"?

I'm not buying :)

Hallucination is all LLMs are capable of doing. Whether those hallucinations align to truth, falseness, or neither is secondary. They're trained on text but they don't recall text, they just hallucinate "beliefs" (heavy scare quotes) that aren't rooted in any notion of logic, just empirical statistical association (assuming the training was "successful" (again heavy scare quotes)).

So LLMs hallucinate because that's what their architecture demands of them for the purpose of generating outputs. It's really that simple.

Very true. Arguably the only hallucination here is on the part of the output's credulous reader.

This article reads like SEO blog spam. Gross to see on HN.

The titled and numbered paragraphs also give me chatgpt vibes

Ilya is wrong, hallucinations is not a bug, it’s a feature and it can’t be fixed with RLHF.

Indeed, and even his primary example shows that. He says "hallucination in large language models can be attributed to a lack of ground truth from external sources" and quotes the ground truth that disproves the fabricated poem he cites.

Hallucinates is a misnomer when used with LLMs. The more precise word, if we use neurology terms for LLM processes, is confabulate.

I consider the idea of naming a whole field “artificial intelligence” unfortunate. Many people think, because of the name, that artificial neurons are not just inspired by real ones, but are a perfect digital model of them. We don’t need to add more confusion by misappropriating terms from other sciences (to be honest, “hallucinate” is also not great by this rule).

I like calling it "bullshitting", because the art of bullshitting isn't about lying, it's about not caring whether or not things are true if they can make a good story.

(But seriously, "confabulate" is much better. The thing I don't like about "hallucinate" is that it gets the agency wrong. Hallucinations are something that happens TO people, that they recover from.)

Hallucinations are also something very personal and internal, not something you can really share with another other than a description.

chatGPT is like a subordinate afraid of looking bad or being chastised for not knowing something it tells you what it thinks you want to hear. bullshitting is what I call this too.

If you use retrospection, and tell it to only give answers that are vetted or true or that it should retrospect on the validity of an answer because they could be true but only in so much as the LLMs cut-off date, for instance using outdated docs.

i personally think everyone working on ai should band together and create an open source framework or tool that's essentially an LLM verification API to verify truthyness.

Reply on news.ycombinator.com