Why Mastering Language Is So Difficult for AI
undark.org
undark.org
People who don't believe the LLMs have a world model are wrong because they are mistaking a physically weak world model for no world model. GPT-3 doesn't understand physics well enough to embed models of the referents of language into a unified model that has accurate gravity and motion dynamics, so it maintains a much more dreamlike model where objects exist in scenes and have relationships to each other but those relationships are governed by literary relationships instead of physical ones and so contradictions and superpositions and causality violations are allowed in the model. As multimodal transformers like Gato get trained on more combined language and sensory input their world models will become much more physically and causally accurate which will be reflected in their accuracy on NLP tasks.
The word "model" is another of those words that Minsky called "suitcase words"--- they can be used to mean many things. I don't think Marcus is saying that that LLMs have "no model", just that they don't have a model of the type that a symbolic system could have. He gives many examples of deductions that a symbolic AI system can easily do, which GPT-3 is simply incapable of.
It seems clear to me that if the CLIP-like part of Imagen or Stable Diffusion can take an image made of pixels and yield "cat" and similarly take "dog" and produce a 3D neural radiance field that we can light just like any other 3D model and recognize as a dog then there must be an accurate and useful model of both how vision works and what dogs look like and the English relationship between those two things inside the machine.
I also wish Gary Marcus was replying to Google's Minerva paper instead of GPT-3. The ability to answer multi-step symbolic problems is basically here. https://ai.googleblog.com/2022/06/minerva-solving-quantitati...
I'd also note that decades of attempts at automated theorem proving with symbolic systems haven't yielded similarly impressive results. Now we have deep learning models helping with proof search.
It's feels a bit similar here. There are clear limitations to AI by just NNs, namely they like Stable Diffusion must be trained on so much data, far more than any single artist will ever see, and they still fall short sometimes. With NLP it seems a little more obvious because the limitations seem to lag actual speech understood by humans still, even with the massive dataset again beyond what any single human will ever digest. There are clear issues here, merely reciting to critics what your model of intelligence is doesn't address the shortcomings these systems have which any literate person can perceive. And gesturing at the data horizon (you need even more data??) is not convincing either.
Apparently this cat (the interviewed author) has some baggage, but the things he's talking about (symbols) have already been studied and used by philosophers and linguists long before AI or computers were a thing. It helps to build upon what other researchers have already done, I don't know why NN researchers don't seem to map what they are doing to existing research on intelligence in other areas, namely cognitive science. Do NN researchers have any contact with linguists or cognitive scientists generally?
Again, being a little ignorant of his baggage, I do think a hybrid approach sounds like it would be promising. Symbolic AI failed in the 70s, NN is getting a bit of the way there now and is pretty useful but misses a few things, why go all in on just one type of approach?
[0] s is basically the center of mass energy, what particle physicists mean when they say the "energy" of an interaction (collision)
Our vision alone uncompressed is about 13TB per hour (!) (assuming 8k per eye resolution, "full color", and about 20hz sample rate). By the time a kid knows to play with a color book, they saw an equivalent of 4 PB of imagery.
For comparison Dall-E 2 only seen 175TB. And GPT-3 only a few TB.
People are working on your symbolic logic hybrid approach, but not much interesting has happened yet. This is a bit expected, since the short amount of time, and the fast scaling these language models have done in those last two years. It's been hard to outscale them with any other approach, let alone by the particular one of symbolic logic.
I think the main discripancy with particle physics, is that every iteration of bigger language models are also significantly better, no diminishing returns in sight.
Better for concrete applications as well. There is therefore no need to claim these models are scaled for scientific reasons in the search of AI, the accountants are happy as it is.
You specifically mention Stable Diffusion, but SD has an analogue to the learning of an individual, textual inversion, which can adapt the NN to express new concepts with as few as a handful of examples.
The article correctly recognizes this, but then fails to actually make any meaningful contributions. Say the model was in fact conditioned on the basis of a present reality, would that change the interviewee's mind? The fact that it's difficult to say simply hints that the interviewer didn't really do that great of a job drilling down to the interesting questions.
Sure we can, there are diseases which greatly hampers your ability to understand the world but doesn't hamper your verbal abilities at all, Williams syndrome for example. They are great at generating nonsense stories that doesn't make sense. GPT-3 is like that but much worse.
Say we apply the diagnostic criteria of William's syndrome to language models and deduce that they meet the criteria for diagnosis, then the diagnostic itself ie: the test results are the grounding for the conclusion not the computational substrate. In that way the discussion surrounding the computation substrate is a red herring or at the very least a unclear and roundabout way of grouping of the characteristic we're actually interesting in.
Byron can understand what's happening in the world around him and can communicate it through body language (just see the warmth in his eyes and his smile) but when he speaks everything comes out in a stream of nonsense.
If we tried to apply a language-model-centric view of intelligence we would rate Byron is unintelligent. That would clearly be a mistake, as anyone can plainly see that he is intelligent.
The contrapositive would be to show a clearly intelligent human with no communication ability that can still model the human world accurately. Past such examples don't tend to show this, though the co morbidity with no early human contact and severe developmental nutritional deficiency makes it hard to be sure.
> conditioned on the basis of a present reality
If we grant this hypothetical, we might find what it "says" fundamentally alien to us --- we might not even know it is saying anything at all.
Spiking neural networks are possibly closer to being biologically analogies, but they kind of suck to train and don't have good performance.
It takes a lot of computer "neurons" to simulate a single human neuron, and we can't simulate it perfectly in any case.
The way airplanes fly doesn't even vaguely resemble how birds fly. Same with neural networks and brains.
Wouldn't the fact that people's facts do not match up prove that they do not?
There's some romanticism in common between magicians, con artists, and fraudsters about how unclearly 'normal people' perceive the world so perhaps there's some truth to what you say. And the perpetual problem with artists and visionaries is that they see the world before everyone else does. For the one group they see things as they are now while everyone else seems to see things as they were 2, 5 years ago. For the other they see what the world could be, by exercising the tools near at hand.
One wonders then how an AI that actually understand things would be seen. As a trickster? A misunderstood intellectual? A madman?
Intelligence has been concisely defined as "the ability to profit from one's experiences." GPT-3 seems unable to profit from its ample experience being told who the president is.
People get so incredibly gullible when an exciting fantasy like sentient AI is in front of them. ELIZA was a laughably simple program and still fooled people.
Anyway. There are three things that NLP is notoriously bad at:
1. Using world knowledge to interpret utterances in a given context. The article itself provides a neat example of that with United-Statian presidents.
2. Contextualisation that goes beyond the text that the utterance is found in.
3. Assessing the pragmatic purpose of an utterance.
But of course, "linguistics is useless for NLP!", the codemonkeys say.
Or you can embrace the bitter lesson, which is that symbolic techniques are considered harmful, like the future winners of the AI race have: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
>No, it unironically IS useless.
There's an example of why Linguistics isn't useless in _the very comment that you're replying to_: pragmatics. That is, modelling the _purpose_ of an utterance is essential to process it.
Without that pragmatic modelling, I believe that NLP will keep failing to deliver what it should, and it'll continue being just a cute toy to keep codemonkeys busy with their blackboxes. Granted, I may be wrong (I'm not stupid to claim certainty over the future), but I do not think that I am.
>You're welcome to bark up the symbolic tree along with Chomsky, Gary Marcus, and others who are about to fade into obscurity.
Damn, could you lend me that crystal ball that allows you to claim certainty over the future?*
It's perfectly possible that they fade into obscurity. It's also perfectly possible that they get hailed as heroes of language processing. Or anything in between. We, in 2022, are in no position to know it. The same applies to any other linguist that is interested on natural language processing.
Specifically on Chomsky. He often spouts some stupid reductionist shit and, paradoxically, unnecessary complexity to keep that reductionism intact. Even then, his approach looks saner than "NEEDZ MOAR NODES X-D".
>Or you can embrace the bitter lesson
"If I say so then you should accept it" is not a valid argument, sorry.
>which is that symbolic techniques are considered harmful, like the future winners of the AI race have
Another instance of "I KNOW THE FUTURE! CHRUST MEEEE" through your comment...
Checking the link,
>The ultimate reason for this is Moore's law
It's already common knowledge that Moore's Law is dead, and brute force only goes so far.
*That specific sentence is a great example highlighting the role of pragmatics. Humans can easily get what I did there; NLP should too, but it doesn't.
Not sure why you criticize a man for having different research interests than you in a completely different field.
But that's just me hoping, that there's anything worthwhile in the comment that you're replying to.
(It reminds me a little of the early years of aviation. In 1901 the US Navy's chief engineer called manned flight a "vain fantasy", & in 1903 the New York Times suggested flying machines would be possible in "one million to 10 million years". But it was accomplished later in 1903, with rapid improvement after that. Things that seem impossible now can become humdrum in a few decades.)
What are the current best results from Marcus, or others using his preferred approaches?
Constantly asking for clarification is exhausting and gets you labeled as pedantic and "difficult". So most people don't do it. I'd guess that a fairly high percentage of human communication results in a misunderstanding. This is largely masked by human inaction or ability to disguise their lack of understanding. The AIs have no ego and no incentive to hide their confusion.
It really frustrates me when people, particularly people on here specifically respond to criticism of NNs and the like by saying nonsense like "but YOUR brain is a neural net!" as if all your brain is but mimickry after all and there's nothing else. There is this need to reduce all the mind's complexity down to a model that you seem to understand, just because you understand it, which seems like such a lazy, reductionist way to think. Reductionism in general has value but it tends to narrow the mind.
I feel like Gary Marcus is right, you don't just need the mimickry even if it plays a role, you do need "symbolic" understanding somewhere, and that makes it easier to determine whether Trump is president vs. Biden at the moment, as they talk about in the article. You do need concepts (symbols) deep down somewhere, it seems so bizarre to think you don't because no one even in their own lives lives that way or thinks that way. I mean literally right now, me reading and writing comments on hacker news is all about ideas and concepts, symbols in the philosophical sense. I feel like even if it started as mimickry when I was an infant, eventually the mimickry takes a life of its own, I figure out concepts themselves, and I no longer even work in mere mimickry, I work with symbols.
It reminds me of Baudrillard's concept (unintended pun, I think) of simulacra, it's not that simulacra are just representations of the real and are thus fake, like money represents some amount of food or gold or something and so money is fake. Money, as we all know, is in fact very real! Our daily lives are shaped around it, they shape our reality, things like loans, credit, bank accounts, stock markets eventually become things we care about. Money is therefore hardly fake. Eventually the simulacra develop a reality of their own, in fact, they become hyperreal.
Ideas are the same, and any real AI will need them.
Still, it’s impossible for me to shake the feeling that he just likes to hear himself talk and doesn’t want to see progress in AI.
I think I trust him more than Sam Altman though, not that anyone’s being asked to make that judgement.
He says he wants to see progress in AI. He wants people to get their heads out of the neural network cul-de-sac and use other tools, too. His quote:
"Imagine a world in which iron makers shouted “iron,” and carbon lovers shouted “carbon,” and nobody ever thought to combine the two [to make steel]; that’s much of what the history of modern artificial intelligence is like." [1]
An adjustment will happen when the limitations of these purely statistical methods play out and burn a few self-driving car investors. Unfortunately, it will also probably taint the entire field of AI and start another AI winter for a decade or so.
[1] https://nautil.us/deep-learning-is-hitting-a-wall-238440/
I have no accent or such, plain standard German,same for my wife and Kids - Alexa drives us crazy with wrong understood episodes and sometimes titles.
BTW a parrot knows a big deal about the world. Not much about human language but still more about language than GPT-3. Of course a 737 doesn't know how to fly but it still flies people around the world. Landing on a branch, not much.
GPT-3 is closer to a talking toaster than a Parrot.
> I think that people are led to believe that this system actually understands human language, which it certainly does not. What it really is, is an autocomplete system that predicts next words and sentences. Just like with your phone, where you type in something and it continues. It doesn’t really understand the world around it. And a lot of people are confused by that.
> They’re confused by that because what these systems are ultimately doing is mimicry. They’re mimicking vast databases of text. And I think the average person doesn’t understand the difference between mimicking 100 words, 1,000 words, a billion words, a trillion words — when you start approaching a trillion words, almost anything you can think of is already talked about there. And so when you’re mimicking something, you can do that to a high degree, but it’s still kind of like being a parrot, or a plagiarist, or something like that. A parrot’s not a bad metaphor, because we don’t think parrots actually understand what they’re talking about. And GPT-3 certainly does not understand what it’s talking about.
I think this is an easy accusation to disprove, and we can do so with a battery of simple tests!
To test whether a language-model is only constrained by mimicry, we can just give it a task that it has never seen before. One of my favorite tasks is the construction of novel analogies.
Here's a kind of "template" that I use often for prompting novel analogies...
PROMPT: You might not realize at first, but “________” and “________” actually have a lot in common. For example,
You can fill-in-the-blanks with various different phrases and see how well the LLM is able to construct a coherent analogy. I have an LLM running in a beta environment (trained on the literary corpus at prosecraft.io), so I'll ask it to run a few of these tests for me.
For example, here's what we get when we ask for an analogy between “writing poetry” and “driving a race car”...
PROMPT + COMPLETION: You might not realize at first, but “writing poetry” and “driving a race car” actually have a lot in common. For example, your first time in a race car, you’re scared. But after a few races, you’re confident. Or in poetry, the first time you write a poem, you’re scared and unsure. But after you’ve read and improved the poem many times, you’re confident. And you are a better writer because of it.
I'm pretty sure that the training corpus doesn't include any metaphors about writing poetry and driving race cars, so the model is relying on a hierarchy of connected abstractions in order to navigate a novel pathway between two disparate concepts. In this case, the central connective-tissue of the analogy is the fact that writing poetry and driving race cars can be scary for beginners, but with more practice, the practitioner increases their skill and confidence.
It's truly astonishing that a system comprised primarily of matrix multiplications on word vectors is capable of achieving this level of abstraction on such a complex cognitive task, but there ya go.
That's what the "deep" in "deep learning" refers to... The LLM training process decomposes language understanding into a hierarchy of layered composite concepts, and the inferencing process can use those different layers of meaning to compose and manipulate novel concepts.
With the "transformer" architecture, and the "self-attention" mechanisms, an LLM is capable of learning a vocabulary of symbols, a hierarchy of relations between those symbols, and a set of operators for transforming those symbols and relations.
We can try it again with “playing chess” and “rock climbing”:
PROMPT + COMPLETION: You might not realize at first, but “playing chess” and “rock climbing” actually have a lot in common. For example, at first glance, chess is a fairly abstract and rigid game. But as you play and gain skill, you realize that certain situations require a certain decision from you. For example, you realize that certain sequences of moves are either bad or good depending on your opponent’s next move. And as your skill increases, you find the moves you’re used to thinking are bad now can be good, and vice versa. The same is true with rock climbing. And with each successive attempt, you realize how much you’re actually improvising in terms of where you place your feet on the wall, how you orient your body, what holds you, where you lean.
In this example, the AI clearly shows that it has an underlying real-world model for the meaning of both chess-playing and rock-climbing, and it was able to find a semantic parallel between those concepts: that they both involve improvising a sequence of strategic moves toward a goal.
Pretty cool, huh?!
It's true that there are still a lot of missing capabilities in these models. For example, a temporal model would certainly help LLMs answer questions about history and current events more accurately (Who is the current president of the US? Who was the previous president?)
Likewise, since most LLMs operate at the "token" level, they actually don't understand how most words are spelled or pronounced, and they're not good at answering questions about rhythm, rhyme, alliteration, etc. For example...
PROMPT: The word “whale” rhymes with the word
COMPLETION: “blue”
Huh? Obviously, the model doesn't have enough information about the pronunciation of words to solve this problem. But if we trained a model jointly on language-as-text and language-as-speech, then there's no reason the model couldn't integrate those sensory data into a unified model. And again, it would be a multi-layered hierarchical model. Not just a lookup-table to be used for mimicry.
If you've been following along with CLIP (VQGAN, CLIP-guided diffusion, Stable Diffusion, etc), which is a joint-sensory model trained on text + imagery, it's easy to imagine future projects that might train joint-sensory models on other modalities (voice, music, locomotion, time, physics, mathematics, etc).
There's no reason those models can't be trained too!
But it makes sense to start with text and imagery, and then bootstrap the other models from there.