GenAI Predictions
tbray.org
tbray.org
> If there were a way to eliminate the hallucinations, somebody already would have. An army of smart, experienced people people, backed by effectively infinite funds, have been hunting this white whale for years now without much success.
Research has been going on for what, like 10 years in earnest, and the author thinks they might as well throw in the towel? I feel like the interest in solving this problem will only grow! And there's a strong incentive to solve it for the important use cases where a non-zero hallucination rate isn't good enough.
Plus, scholars have worked on problems for _far far_ longer and eventually solved them, e.g. Fermat's Last Theorem took hundreds of years to solve.
"Write me a story about the first kangaroo on the moon" - that's a direct request for a hallucination, something that's never actually happened.
"Write me a story about the first man on the moon" - that could be interpreted as "a made-up children's story about Neil Armstrong".
"Tell me about the first man on the moon" - that's a request for factual information.
All of the above are reasonable requests of an LLM. Are we asking for a variant of an LLM that can flat refuse the first prompt because it's asking for non-real information?
Even summarizing an article could be considered a hallucination: there's a truth in the world, which is the exact text of that article. Then there's the made-up shortened version which omits certain details to act as a summary. What would a "hallucination free" LLM do with that?
I would argue that what we actually want here is for LLMs to get better over time at not presenting made-up information as fact in answer to clear requests for factual information. And that's what we've been getting - GPT-5 is far less likely to invent things in response to a factual question than GPT-4 was.
To me, there’s a qualitative question of what details to include. Ideally the most important ones. And there’s the binary question of whether it included details not in the original.
A related issue is that preference tuning loves wordy responses, even if they’re factually equivalent.
The strongest argument in my mind for why statistical models cannot avoid hallucinations is the fact that reality is inherently long-tail. There simply isn’t enough data or FLOPs to consume that data. If we focus on the limited domain of chess, LLMs cannot avoid hallucinating moves that do not exist, let alone give you the best move. And scaling up training data to all positions is simply computationally impossible.
And even if it were possible (but still expensive) it wouldn’t be practical at all. Your phone can run a better chess algorithm than the best LLM.
All of this is to say, going back to your Fermat’s last theorem point, that we may eventually figure out a faster and cheaper way, and decide we don’t care about tall stacks of transformers anymore.
If for "solved", you want AI to be as accurate and reliable as simply retrieving the relevant data from an SQL database? Then hallucinations might never truly "get solved".
If for "solved", you want AI to be as accurate and reliable as a human? Doable at least in theory. The bar isn't high enough to remain out of reach forever.
To me, this looks like an issue of self-awareness - and I mean "self-awareness" in a very mechanical, no-nonsense way: "having usable information about itself and its own capabilities".
Humans don't have perfect awareness of their own knowledge, capabilities or competences. But LLMs have even less of each. They can recognize their own inability or uncertainty or lack of knowledge sometimes, but not always. Which seems like it would be very hard but not entirely impossible to rectify.
That's just wishful thinking.
First, that kind of code is one of the things that's most valuable for AI to get good on, so a lot of effort will be directed to that. More efficient low level systems help with recursive self-improvement. Writing JS frontends doesn't.
Second, systems programming is a domain with objective quality metrics that makes it a good RL target.
Third, the models are already very good in that domain. They obviously don't have perfect taste yet, so having an experienced systems programmer with a vision guiding the model will be much better than fire and forget.
When implementing a new optimisation is 10% the cost that it used to be when done manually, the tradeoff for which ones are worth implementing changes.
But in addition it's way easier to justify exploration and speculative ideas when they're so cheap to test out.
* Translators & transcription
* Waymo doing real world shuttling of people (not strictly genAI I guess)
* AI art being used in places where it's "good enough" - blogs, powerpoint etc
* Call centers
* Boilerplate coding
* Beginnings of robots "understanding" their surroundings for trivial warehouse tasks
...a job apocalypse that does not make, but to me that is sufficient evidence that replacement is feasible in principle for some things and combined with trajectory we'll see *some* displacement and societal impact. Whether that constitutes "mass" layoffs remains to be seen, but I feel a surprisingly low % number will already rattle societies foundations. Just because my job might be safe doesn't mean a sudden +5% in unemployment doesn't affect my community.
It would be interesting if the author were to try to express these with a timeline in a way that is falsifiable, optionally with some kind of measure of certainty.
A lot of futurist predictions are too vague to be falsifiable. Or they’re already happening, in which case they’re not really predictions.
I have many friends and coworkers who think the AI race will come crashing down. But they are not selling their stocks. I’d love for some AI skeptic to help me make sense of this mess.
"The market can stay irrational for longer than you can stay solvent" (or in this case, for longer than you're willing to lose on gains).
Some individuals are simply less risk averse (or more influenced by FOMO) and more willing to play the musical chairs game for longer.
Yet, let’s look at intelligence for a sec… it’s been evolving for quite some time and isn’t stopping at humans. Humans are building intelligence at a rate far faster than biological evolution. It is almost inevitable (pending humans wiping ourselves out by any number of catastrophic failures of governance) that we will build intelligence that supersedes our own. Yeah?
I’m a co-founder of Aloe (https://aloe.inc) - a generalist AI that recently became state of the art on the GAIA benchmark. As I was hand-checking the output of our test to ensure Aloe had done the task, not just found answers in some leak online, I had a real come-to-Jesus moment when it hit that this agent is already a better problem-solver than most of the adults I’ve worked with in my career. And this is the floor of its capability.
It is a humbling moment to be human. The few humans at the helms of companies developing these technologies will inevitably reshape the trajectory of humanity.
There is no central goal. We made a cool toy and we’re trying to figure out what to do with it.
Starting with that incorrect premise means all your conclusions are wrong.