It might also lead to recursive garbage generation. If the AI has flaws and those flaws are from the initial training, then I see no way, how the flawed data can ever generate clean data.
"The datasets are massive, and curating them into buckets of fact/fiction by hand would be next to impossible. "
And it is not impossible. It is just a lot of work. And maybe work we just have to do, if we want to create reliable AIs, that do not fail at random times.
Now this alone might not completely remove hallucinations, but solid training data is just the base of it all.
(and since we are living in the area of fake news, I welcome all efforts towards established facts and data out of general principle)
This is purely hypothetical, but I imagine that the internal mechanisms the model uses to invent stories out of thin air are different from the mechanisms the model uses to recall precise facts. Providing additional labels could help sharpen the division between creative and factual writing tasks.
> I imagine that the internal mechanisms the model uses to invent stories out of thin air are different from the mechanisms the model uses to recall precise facts.
Nope! If you input "What is the capital of the United States of America?", it's probably going to output the correct answer, because the question (and answer) probably appeared many times in the training data. If you input "What is the capital of the Gronk Republic?", whatever output you get is generated via the exact same mechanism.
No one knows for sure exactly what happens inside a 500B parameter model. I’ll just leave you with GPT4’s response to your question “What is the capital of the Gronk Republic”.
“As an AI language model, I am not aware of any existing country or political entity called the "Gronk Republic." It could be a fictional or hypothetical place, in which case the capital would be determined by the creator or context in which it is mentioned. If you are referring to a real location, please provide more information or clarify the name of the place you are asking about.”
Have you tried “arguing” with it and try to gaslight it by insisting that it’s non-fictional?
For that reason, I'd say the 'intelligent' answer would be a clever play on the notion that this celebrity was the ruler of a 'republic' of some sort, and associating some kind of location or thing to serve as the 'capital' of either a physical or conceptual 'republic', playing along with the gag.
So, you'd get 'Boston'.
Or you'd get 'Gillette Stadium… the END ZONE'.
(since I had to google all that, first answer is the biggest city representing the New England Patriots, and the second abstracts the 'republic' to be 'the area Gronk rules', depicting him as returning to his property by scoring points in football)
The thing is, this sort of wild associativeness IS 'intelligence' and it's simultaneously complete hallucination. It's valuable. Hallucination is potentially the most valuable thing an AI can do. Without it, you're doing nothing but regurgitating the work of others.
I work day in and day out at a job that demands I exploit associativeness, and that makes it not easy for AI to simply step in and replace me, but by the same token I can SPOT when it's moving in that direction, and I can exploit the abilities of AI (such as Stable Diffusion) to hallucinate and make wild associations, because I know the contexts in which that is useful. To handicap this is a bad mistake. You're asking AI to do the wrong things when you're asking it to be free of error. If it's perfectly free of error it's not intelligence anymore, it's a dataset.
It took me a minute and a little googling to track down who Gronk was, what he did, the rules of football, and why 'Gillette Stadium, the END ZONE' is actually a fantastic and intelligent answer to the question. It's a creative 'slipping' of the grounds of the question, in the absence of a literal answer, to provide a satisfying figurative answer that reframes the question in an unexpected way. When AI is able to do this, that will itself be a useful kind of intelligence… and we are already slightly there, without realizing it.
Training the model on "accurate" data isn't going to change the fact that you're operating at a completely different level of abstraction from the model -- one that it can't operate at, because that's simply not how it works.
This is the same problem that image models have. A human artist works with shapes, values, textures, colors, etc. An image model works with pixels, with zero higher-level abstractions or reasoning informing its output.
Though I agree there is something ontological missing. All of these are flat basis. There's nothing spanning the recursive dimension. It can't draw a picture within a picture within a picture...to some specified depth n because it does not work with abstractions of "objects". The joke of Rene Margrit's "treachery of images" is lost on the AI.
No, only difficult.