There isn't really such thing as a "hallucination" and honestly I think people should be using the word less. Whether an LLM tells you the sky is blue or the sky is purple, it's not doing anything different. It's just spitting out a sequence of characters it was trained be hopefully what a user wants. There is no definable failure state you can call a "hallucination," it's operating as correctly as any other output. But sometimes we can tell either immediately or through fact checking it spat out a string of text that claims something incorrect.
If you start asking an LLM for political takes, you'll get very different answers from humans about which ones are "hallucinations"
There's definitely room for a better label, though. "Empirical mismatch" doesn't quite have the same ring as "hallucination," but it's probably a more accurate place to start from.
Sure, but that would require semantic mechanisms rather than statistical ones.
If someone wants info to make their model to be more reliable for a specific domain, it's in the existing papers on model training.
Is is possible for a chess engine to compute the next move and be absolutely sure it is the best one? It's not, it is a statistical approximation, but still very useful.
People say it's "anthropomorphizing" but honestly I can't see it. The I in AI stands for intelligence, is this anthropomorphizing? L in ML? Reading and writing are clearly human activities, so is using read/write instead of input/output anthropomorphizing? How about "computer", a word once meant a human who does computing? Is there a word we can use safely without anthropomorphizing?
[1]: And please don't argue what's "wrong".
You will be told that linear algebra is just a model and the fact that epistemology has never turned up a decent result for what knowledge is will be ignored.
We are meant to believe that we are somehow special magical creatures and that the behaviour of our minds cannot be modelled by linear algebra.
If a company does a thing that's bad, it doesn't matter much if the work itself was performed by a blacksmith or by a robot arm in a lights-off factory.
> We are meant to believe that we are somehow special magical creatures and that the behaviour of our minds cannot be modelled by linear algebra
I only hear this from people who say AI will never reach human level; of AI developers that get press time, only LeCun seems so dismissive (though I've not actually noticed him making this specific statement, I can believe he might have).
No, it’s more specific than just wrong.
Hallucination is when a model creates a bit of fictitious knowledge, and uses that knowledge to answer a question.
You can argue if it matters how a wrong answer came about ofc but there is a difference
You can still get that with zero bad labels in a supervised training set.
Multiple causes for the same behaviour makes progress easier, but knowing if it's fully solved harder.
Context is "don't call it hallicination" picked up meme energy since https://link.springer.com/article/10.1007/s10676-024-09775-5 on the thesis that "Calling their mistakes ‘hallucinations’ isn’t harmless: it lends itself to the confusion that the machines are in some way misperceiving but are nonetheless trying to convey something that they believe or have perceived."
Which is meta-bullshit because it doesn't matter. We want LLMs to behave more factually, whatever the non-factuality is called. And calling that non-factuality something else isn't going to really change how we approach making them behave more factually.
If they could predict facts, then these would be gods, not machines. It would be saying that in all the written content we have, there exists a pattern that allows us to predict all answers to questions we may have.
It logics its way to it.
By predicting the next word in a sequence of words.
Sure? It kinda sounds plausible? But man, if it’s that straight forward, what have we been doing as a species for so many years ?
TLDR: Sure. A rose by any other name would be just as sweet. It’s when I use the name of the rose and imply aspects that are not present, that we create confusion and busy work.
Hey, calling it a narrative is to move it to PR speak. I know people have argued this term was incorrect since the first times it was ever shared on HN.
It was unpopular to say this when ChatGPT launched, because chatGPT was just that. freaking. cool.
It is still cool.
But it is not AGI. It does not “think”.
Hell - I understand that we will be doing multiple columns of turtles all the way down. I have a different name for this approach - statistical committees.
Because we couched its work in terms of “thinking”, “logic”, “creativity”, we have dumped countless man hours and money into avenues which are not fruitful. And this isnt just me saying it - even Ilya commented during some event that many people can create PoCs, but there are very few production grade tools.
Regarding the L in ML, and the I in AI ->
1) ML and AI were never quite as believable as ChatGPT. Calling it learning and intelligence doesnt result in the same level of ambiguity.
2) A little bit of anthropomorphizing was going on.
Terms matter, especially at the start. New things get understood over time, as we progress we do move to better terms. Let’s use hallucinations for when a digital system really starts hallucinating.
To me the real danger comes from when the models get things wrong but also correct at the same time. Not so much in software engineering, I doubt your average programmer without LLM tools will write “better” code without getting some bad answers. What consents me is more how non-technical departments implement LLMs into their decision making or analysis systems.
Done right, it’ll enhance your capabilities. We had a major AI project in cancer detection, and while it actually works it also doesn’t really work on its own. Obviously it was meant to enhance the regular human detection and anyone involved with the project screamed this loudly at any chance they got. Naturally it was seen as an automation process by the upper management and all the humans parts of the process were basically replaced… until a few years later when we had a huge scandal about how the AI worked as it was meant to do, which wasn’t to be on its own. Today it works along side the human detection systems and their quality is up. It took people literally dying to get that point through.
Maybe it would’ve happened this way anyway if the mistakes weren’t sort of written into this technical issue we call hallucinations. Maybe it wouldn’t. From personal experience with getting projects to be approved, I think abstractions are always a great way to hide the things you don’t want your decision makers to know.
We won't, and we'll see this constant distraction.
Well, parent is lamenting the lack of lowerbound/upperbound for "hallucinations", something that cannot realistically exist as "hallucinations" don't exist. LLMs aren't fact-outputting machines, so when it outputs something a human would consider "wrong" like "the sky is purple", it isn't true/false/correct/incorrect/hallucination/fact, it's just the most probable character after the next.
That's why it isn't useful to ask "but how much it hallucinates?" when in reality what you're out after is something more like "does it only output facts?". Which, if it did, LLMs would be a lot less useful.
LLM don't need to be perfect fact machines at all to be honest, and non-hallucinating. They simply need to ground statements in other grounded statements and identify the parts which are speculative or non-grounded.
Otherwise, how do you prove the grounding isn't "hallucinated"?
That's simply not true. You're confusing how they're trained and what they do. They don't have some store of exactly how likely each word is (and it's worth stopping to think about what that would even mean) for every possible sentence.
It's a simplification. Temperature also influences it to not always be the most probable character, as an example.
It is somewhat humorous when humans have ontological objections to the neologisms used to describe a system whose entire function is to relate the meanings of words. It is almost as if the complaint is itself a repressed philosophical rejection of the underlying LLM process, only being wrapped in the apparent misalignment of the term hallucination.
The complaint may as well be a defensive clinging "nuh uh, you can't decide what words mean, only I can"
Perhaps the term "gas lighting" is also an appropriate replacement of "hallucination," one which is not predicated on some form of truthiness standard, but rather THIS neologism focuses on the manipulative expression of the lie.
Hallucination might not be the best word, but I don't think it's a bad word. If a weather model predicted a storm when there isn't a cloud in the sky, I wouldn't have a problem with saying "the weather model had a hallucination." 50 years ago, weather models made incorrect predictions quite frequently. That's not because they weren't modeling correct weather, it's because we simply didn't yet have good models and clean data.
Fundamentally, we could fix most LLM hallucinations with better model implementations and cleaner data. In the future we will probably be able to model factuality outside of the context of human language, and that will probably be the ultimate solution for correctness in AI, but I don't think that's a fundamental requirement.
People still want it to be used for thinking.
This isnt going to happen with better data. Better data means it will be better at predicting the next token.
For questions or interactions where you need to process, consider, decompose a problem into multiple steps, solve those steps etc - you need to have a goal, tools, and the ability to split your thinking and govern the outcome.
That isnt predicting the next token. I think it’s easier to think of LLMs as doing decompression.
They take an initial set of tokens and decompress them into the most likely final set of tokens.
What we want is processing.
We would have to set up the reaction to somehow perfectly result in the next set of tokens to then set up the next set of tokens etc - till the system has an answer.
Or in other words, we have to figure out how to phrase an initial set of tokens so that each subsequent set looks similar enough to “logic” in the training data, that the LLM expands correctly.
Humans also confabulate but not as a result of "hallucinations". They usually do it because that's actually what brains like to do, whether it's making up stories about how the world was created or, more infamously, in the case of neural disorders where the machinery's penchant for it becomes totally unmoderated and a person just spits out false information that they themselves can't realize is false. https://en.m.wikipedia.org/wiki/Confabulation
This is a very "closed world" view of the phenomenon which looks at an LLM as a software component on its own.
But "hallucination" is a user experience problem, and it describes the experience very well. If you are using a code assistant and it suggests using APIs that don't exist then the word "hallucination" is entirely appropriate.
A vaguely similar analogy is the addition of the `let` and `const` keywords in JS ES6. While the behavior of `var` was "correct" as-per spec the user experience was horrible: bug prone and confusing.
What am I suppose to call that?
That being said, people new to the field tend to believe that these models are fact machines. In fact, they are the complete opposite.
No real way to mathematically prove this, considering there is also no way to know if the training data also had this “hallucination” inside of it.
Investigate it with the tools of psychologically, as suited for use on a new non-human creature we've never encountered before.
https://www.nytimes.com/2023/11/06/technology/chatbots-hallu...
For us, we treat hallucinations as the ability to accurately respond in an "open book" format for retrieval augmented generation (RAG) applications specifically. That is, given a set of information retrieved (X), does the LLM-produced summary:
1. Include any "real" information not contained in X? If "yes," it's a hallucination, even if that information is general knowledge. We see this as an important way to classify hallucinations in a RAG+summary context because enterprises have told us they don't want the LLMs "reading between the lines" to infer things. To pick an absurd/extreme case to show a point, the case of a genetic research firm, say, using CRISPR and finding they can create a purple zebra, if the retrieval system in the RAG bits says "zebras can be purple" due to their latest research, we don't want the LLM to override that knowledge with its knowledge that zebras are only ever black/white/brown. We'd treat that as a hallucination.
2. On the extreme opposite end, an easy way to avoid hallucinating would be for the LLM to say "I don't know" for everything thereby avoiding hallucinating by avoiding answering all questions. That has other obvious negative effects, so we also evaluate LLMs for their ability to answer.
We look at the factual consistency, answer rate, summary length, and some other metrics internally to focus prompt engineering, model selection, and model training: https://github.com/vectara/hallucination-leaderboard
Some people are surprised by smaller models having the ability to outperform bigger models, but it's something we've been able to exploit: if you fine tune a small model for a specific task (e.g. reduced hallucinations on a summarization task) as Intel has done, you can achieve great performance economically.