I’m not sure we really suffer from it. In our internal analytics AI tends to slow employees down and make them less productive. This is in general. The exception is experts using it, where LLMs increase their value output by an ok margin. Especially within our own programming team LLMs have proven a real challenge. On one hand they are fancy auto-complete which will speed an experienced developer up by so much it’s hard to ignore. On the other hand it takes an experienced developer to know when they get things wrong. Not the things that literally won’t work, but the things that will work poorly. I haven’t been too involved outside of our team, but I imagine it’s the same in any field.
Which is where the “hallucination” term sort of back-fired. It was a good way to make people buy into the value of LLMs by making the mistakes oddities almost negligible. The issue is that those mistakes can have such massive impacts that the entire trust in the AI industry falters. I mean, we had one of the CEOs ask if we could switch to Linux now that Windows includes AI… obviously we can’t do that without going bankrupt, but it tells you something about the worry in the non-tech enterprise top.
We might say the same thing about people.
Ultimately, just how problematic is it to label something as a hallucination? Are investors about to be massively duped? If I create a mechanism to reduce hallucinations and I call it therapy, is that really problematic?
No, it would be fair to say they have a "grasp" of predicting the next word in a given sequence of words based on a set of words in their training set. Hallucination then is what people call their inherent tendency to run into a situation where the probability of the next word being predicted "wrong" is high. And once one "wrong" word has been predicted the probability that the next word is also "wrong" rises exponentially.
LLMs do not have any grasp of reality. They just predict text based on trained patterns. Too many people have been fooled into believing that LLMs can understand anything about reality, but a word-based description of reality is not the same as reality.
> We might say the same thing about people.
If you want to reduce a human being down to being a word predictor, then I guess you could say that?
LLMs know a whole lot more about the uncertainty of their predictions than they say.
GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r
Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824
The Internal State of an LLM Knows When It's Lying - https://arxiv.org/abs/2304.13734
LLMs Know More Than What They Say - https://arjunbansal.substack.com/p/llms-know-more-than-what-...
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975
Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334