Generative AI doesn't have a coherent understanding of the world
news.mit.edu
news.mit.edu
The very headline is based on a completely faulty assumption, that AI has any capacity for understanding at all, which it doesn't. That would require self-directed reasoning and self-awareness, both of which it lacks based on any available evidence. (though there's no shortage of irrational defenders here who somehow leap to say that there's no difference between consciousness in humans and the pattern matching of AI technology today, because they happened to have a "conversation" with ChatGPT and etc).
And this right here is why its so frustrating to me that the term "AI" is used for LLMs. They are impressive for certain tasks, but they are nothing close to artificial intelligence and were never designed to be.
I mean, I get it. These things are mainstream and close enough in some ways to what we see in science fiction that the term evokes the sci-fi meaning of the term rather than the computer science meaning.
I've always considered AI as a much more complex concept than simply appearing to be human in speech or text. Extremely complex prediction algorithms that appear on the surface to be creative wouldn't have met my bar either.
An LLM can predict how a human may write a white paper, for example. It will sound impressive, but the model and related algorithms seem to have absolutely no way of logically working through a problem or modelling the world to know if the ideas proposed in the paper might work. Its a really impressive sounding impersonator, but it is nothing like intelligence.
It's actually long been noted that the bar keeps rising, that when we figure something out it stops being called AI by a lot of people. I don't think that's individuals raising their bar necessarily though, but stuff becomes more mainstream or younger people being introduced to it.
The same thing could be said about Virtual Reality where the bar raises each time the tech gets closer to making VR indistinguishable from reality.
> It's actually long been noted that the bar keeps rising, that when we figure something out it stops being called AI by a lot of people.
That's interesting, I've seen it the other way around. In the 90s I don't remember anything related to AI depicting something as simple as fancy text prediction or human-like speech. Going back further and Hal from Space Odyssey didn't seem like an AI that was just predictive text.
Its always possible my expectations were born from science fiction and not academic research, but that's a pretty big gap if academics considered AI to only require natural language processing and text prediction.
How often have you talked to someone and showed them evidence through logic or other means and they just don't get it? Or just don't want to accept it?
Are people who do not properly self reflect, non-intelligent?
AI only needs to be as good as a human and cheaper
A LLM has no such concept. It is not truly capable of accepting your rational arguments; it may nod emphatically in its response after a few sentences, but once your input leaves its context window, all bets are off. We can retrain it with your particular conversation, and we will increase likelihood it will more readily agree with you, but that is likely a drop in the bucket of the larger corpus.
But this is well past technology and into philosophy, of which I can safely say my mental model for the topic would likely be rather frustrating to others.
Setting aside the current limitations of LLMs for a moment, this is just a really unfortunate view on a new tech in my opinion. Its nothing unique here so I don't mean to single you out. I've seen it plenty of other places. The idea, though, that we effectively imply that most humans are idiots and we just need to invent something that is as good at being a functional idiot but for much less money leaves humanity in a terrible place. What does the world look like if we succeed at that?
To you main question, though, I think differences in human opinions and ideas can be easily misconstrued as a lack of intelligence. Our would also consider environmental factors of the average person.
I'll stick with the US as its what I am most familiar with. The average american has what I'd consider a high level of stress, chronic health conditions, one or more pharmaceutical prescriptions, and our food system is sorely lacking and borderline poisonous. With all those factors in play I can't put too much weight behind my view of the average person's intelligence.
Affluent societies that abandon an ideal of personal development bound to forces that bring to a catastrophic configuration of society, with devastating effects: the absurdity, the risks and impact, the diversion from propriety merit all focus.
I shall add: the Great Scarcity today is... Societies.
Nonetheless, i do wish for a Star Trek like future and we can't reach it as long as we humans work normal jobs we can automate away.
Secondly, the very fact that you can't convince some people to "get it" (whatever the hell that thing may be, regardless of whether you're the one who's actually failing to get something at the same time) is evidence of their self directed reasoning. For better or worse, it's there, along with a sense of self that even the ostensibly dumbest person has, excepting maybe those with truly profound mental disabilities.
None of these things apply to LLMs, so why not drop the semantics about similarity or much less equivalence? Sel awareness is a sensation that you yourself feel every day as a conscious human being in your own right, and no evidence whatsoever indicates that any LLM has even a smidge of it.
AI is a set of engineered tools and they need to be adequate for the job.
When we speak about general Intelligence, it will have to be acute and reliable. Surely the metric will have to be well above low bars for humans, already because for that we already have humans available, and because, as noted, that low bar level is easily not «adequate for the job».
We've had the debate if animals were self-aware in the first place, some seem to be based on studies, some don't. But all animals have intelligence, and the ability to problem solve to different degrees, and they most certainly learn based on experience and evolution.
Artificial Intelligence means intelligence that was created artificially, how intelligent and whether or not it is self-aware (it's not), is a different debate from "it's not intelligence".
ChatGPT put it best itself:
> Yes, I possess artificial intelligence, which enables me to process information, answer questions, solve problems, and engage in conversations. However, my intelligence is based on patterns and data, not personal experience or consciousness. Let me know how I can assist you!
AI isn't Skynet, Wintermute or HAL 9000, those are fictional and are what we dubbed ASI (Super), not AGI (General) or ANI (Narrow), ASI is self-aware, the others are simple intelligence that can either do one thing, or multiple things to different degrees of quality comparatively to humans (not always better, not always worse).
So yes, LLMs have intelligence, and you can claim the same for other software. Whether or not they're equal to or superior to humans is irrelevant since animals are inferior to us in intelligence to different degrees.
At the end of the day Intelligence can mean different things to different people, so I can't just outright tell you that you're right or wrong, I can only express my viewpoint and why I think LLMs as well as some other software are AIs.
EDIT: Should also note that words take on different meanings, and AI is an easy buzzword to use for the general public, so the word took on extra meaning, and it did so decades ago. We've been calling NPCs in video games AI for decades. There's likely other examples that don't come to mind where AI has been used far longer than LLMs.
We started in 1956. https://en.wikipedia.org/wiki/Dartmouth_workshop
That kind of understanding has no relation with «self-directed reasoning and self-awareness».
Why? For sure not out of logic.
The assumption is that if you just read a lot of information, more than a normal human ever can read that you get something which can be similar to what a human builds in their brain through living.
LLM as the end result of a adult human state.
It's not that far fetched.
Especially when you see how hard it is to discuss things purely logical and not through feelings, opinions and following behavior patterns of others.
AI is currently missing pieces it needs to be a "complete being", but saying there's nothing there is falling into the same reductive, "scientific" nitpicking that brought us "fish can't feel pain, they just look like it and act like it".
I don’t see why something would need to be entirely coherent to be useful. If you have a program which, when given a statement, assigns a value between 0 and 1, to be interpreted as if it were a probability of the statement being true, it needn’t define an entirely coherent probability distribution.
Human perception is coherent because it tends to accurately model the physical world from which it's constructed, by means of sensory input and memory, and if this were not the case, we would have gone extinct as a species ages ago. While it is true that the model of human perception isn't perfect, it would be incorrect to call it incoherent to the same or similar degree as the incoherence demostrated by LLMs, as jmole's earlier comment implies.
Even if one must consider the model of human perception incoherent, one must also consider it vastly more coherent than whatever LLMs model reality on, if anything.
LLM's do not have that concept, and you'll notice very quickly if you ask chemistry questions. Atoms appear twice, and the LLM just won't notice. The approach has to be changed for AI to be useful in the physical sciences.
https://www.science.org/doi/10.1126/science.adg9774
https://www.technologyreview.com/2024/10/18/1105880/the-race...
The parent poster posted a paper where an AI guesses the Hamiltonian (nice, but with iterative methods you at least get an idea of the error associated with your numbers, not sure how far anyone should trust AI's there), maybe methods that do guesswork on topological networks would help. But I haven't seen those yet.
You're inability to recognize high dimension relations between word tokens is no less evidence you lack coherent understanding.
You can't say that given <conditions>, 1-phenylpropane rearranges to 1,2-diphenylpropane, you cannot pull whole atoms out of thin air for an indefinite amount of time. No human would make such a mistake, but AI does, and what's worse, it is impossible to talk the AI out of its strange reasoning.
AI gets trained on a lot of different worlds (books, stories, science, etc. )
Irrelevant, because for the intended purpose (reasoning over a reliable representation of the world) there is no need for an interactive stream - the salient examples of interaction, cause and effect, are provided by the dataset.
A large perception model "LPM" will not be superior to a large language model "LLM" if its underlying system is not superior.
Imagine trying to understand the world if you were simply given books and books in a language you had never read… and you didn’t even know how to read or even talk!
So it’s pretty incredible it’s got this far!
But what you describe is basically done by any human on Earth : you are born not knowing how to read or how to talk and after years of learning, reading all the books may give you some understanding of the world.
Contrary to LLM though, human brains don’t have a virtually infinite energy supply, cannot be parallelized and have to dedicate their already scarce energy to do a lot of other things than reading books including moving in a 3D world, living in a society, feeding themselves, doing their own hardware maintenance (sleep …), pay attention not to die every single day etc etc.
So, for sure, LLMs _algorithms_ are really incredible, but they are also useful only if you throw a lot of hardware and energy into them. I’d be curious to see how long you’d need to train (not use) a useful LLM with only 20W of power (which is more or less the power we estimate the brain is using to function).
We can still be impressed by the results, but not really by the speed. And when you have to learn the entire written corpus in some weeks/months, speed is pretty useful.
You don't simply learn on training data, you are taught by other humans who were in turn taught by more humans still. When you do learn genuinely new information, you know exactly how it makes sense to you, and you can share it with others in a way optimized for them to incorporate it as well.
LLMs aren't just learning to speak and read, they are developing from scratch a framework for comprehending the training data thrown at them. An LLM has no concept of phonetics because it has no ears with which to hear nor a mouth with which to speak. It derived english from pulses of electricity.
Leave a child in a box with no human contact and a library of books and perhaps it will ascribe meaning to the symbols written on the pages, but it will never know what they actually mean. Even if they lived a million lifetimes, and were able to recognize that the symbols appear with certain regularity and could infer some rules of the language, they would never know what concept the string "comprehension" refers to.
Interesting -- I'm surprised this joke didn't land harder.
Thought experiment-- let's say you have the PERFECT transformer architecture and perfectly tuned weights with infinite free compute and instantaneous compute speed. What does that give? Something that allows you to perfectly generalize over your dataset. You can perfectly extract the signal from the noise.
Having a "world model" implies a model of the world and thus implies a model of the self distinct from the world. Either that or the world model incorporates the self, right? Because the thing doing the modeling must be part of the world in order for it to have a model of the world.
So, given a perfect transformer and perfect language model, you will at best get a perfect generalization of the dataset.
You cannot take a dataset that has no "world model" and run it through a perfect transformer and somehow produce a world model, aka, cannot turn lead into gold-- you have created information from thin air and now we are in the realm of magic and not science. It does not matter how much compute you throw at the problem.
Looking at the model is a red herring. The answers are in the dataset.
[1] https://en.wikipedia.org/wiki/Tabula_rasa
[2] https://press.princeton.edu/books/hardcover/9780691181226/th...
[3] https://www.amazon.com/Archaeology-Mind-Neuroevolutionary-In...
[*] One of the things that frustrates me the most in the discourse on LLMs is that people who should know better deliberately mislead others into believing that there is something similar to "intelligence" going on with LLMs -- because they are heavily financially incentivized to do so. Comparisons with humans are categorical errors in everything but metaphor. They call them "neural networks" instead of "systems of nonlinear equations", because "neural network" sounds way sexier than vectorized y=f(mx+b).
You seem to be conflating a «self» and «the thing doing the modeling». But there is no need to confuse the "self" as a perceptual object part of the world model, and the processor. The "world model" is the set of the ideas (productive to statements) describing reality. The perceptual self may not be greatly important in the world model (it will not be for a large amount of problems we will want to propose to the reasoner), and the processor will be just a "topic" in the world model, in the descriptive form (the world model contains descriptions of processors).
The expression about a «dataset that [would have] no "world model"» makes no sense. The dataset is the input of protocols ("that fact was witnessed"), the world model is what is reconstructed from the dataset. You have a set of facts (the dataset): you reconstruct an ontology and logos (entities and general rules) that make sense in view of the facts.
Respectfully, this is false. The easiest non-technical intuition for why this is false is because the transformer can be trained on any dataset-- if you give it a random noise dataset, it does not produce "reason" but random noise. If you give it a dataset of language full of nonsense, it will produce nonsense. Using the term "reasoning" is an inappropriate anthropomorphism of the vectorized equation y=f(mx+b), which is effectively what a transformer is.
Respectfully again, it's not productive for me to argue the rest of your points because they proceed from this false premise.
The thing I would encourage you to think about is the relationship between the dataset and the transformer.
When you write about «tak[ing] a dataset that has no "world model"», that is what we normally do: we gather experience and make sense of it, and that happens through functions that can be called "reasoning" (through "intelligence" etc). We already "«create[] information from thin air»" - it's what we do.
If the «Perfect transformer architecture» cannot reach the proper «world model» through its own devices, it must then be augmented with the proper modules - "Reasoning", "Intelligence" etc.
The point was, "Reasoning", "Intelligence" etc. do not seem to need «self-aware sentien[ce]».
Perhaps at this point we are merely speculating on an existence proof-- whether or not there exists such a dataset that contains a sufficient "world model" (for some definition of "world model"). I assert that such a dataset would necessarily include a concept of the self, otherwise the "world model" would be flawed. You seem to disagree.
That's fine.
My argument does not hinge strongly on whether or not sentience is required for such a world model, it is sufficient to say that no such dataset exists that will provide a transformer with sufficient information to condition weights which will have an accurate world model, sentience or not.
The "self-aware sentience" bit was hyperbolic prose for the sake of the joke, but the joke is that anyone pedaling "if only I had more money for compute I could make AGI" is shilling snake oil. No amount of infinite compute can turn an incomplete dataset into anything more than an incomplete dataset, information is conserved.
Hopefully that point is not too controversial.