You must have a very peculiar definition of a hallucination.
Why would an LLM reciting a fact correctly be hallucinating?
Why would an LLM reciting a fact correctly be hallucinating?
So if it's "hallucinating" a probable continuation which asserts something which is [incidentally] understood to be completely wrong or not in the source material by humans, it's going through exactly the same process to arrive at a continuation which [incidentally] is understood to contain only accurate statements or valid summarizations
Am I now capable of predicting the future?
Suppose I wrote the book to be as banal (i.e. highly probable) as possible.
Am I predicting the future now? And, how impressive is it?
If you write a book of random predictions without any insight the vast majority of them will be false, so even if few of them are right it is not impressive nor anyone would say you're capable of predicting the future.
In comparison, the OP states that GPT-4 predictions are 97% correct. And yes, I would say that is pretty impressive. If 97% of anything I say about the future was correct I would be considered a wizard and probably be a billionaire.
Isn't this step for step exactly what Nostradamus did?
And you just hallucinated that! It's just, if there was a system that only talks in an oddly definitive hypotheticals, they can be correct about a lot, and GPT-* are exactly that.
Fact of the matter is that sota LLMs are highly accurate predictors for many topics, certainly above any living human in terms of total AUC of correct predictions on fact based questions. Some humans are better on certain topics, but noone can match total AUC since LLMs has such breadth.
LLMs are fine - people are attributing superpowers to them when they discuss hallucinations.
LLMs do not "think". They created the correct text as they were modeled to do.
The observer feels that the facts are wrong.
That's an issue with the observer, not the model. The model was never trained for facts it was trained for text.