This has been posted many times before, but I thought of it yesterday when I saw this essay by Chomsky in the New York Times, "The False Promise of ChatGPT."
https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chat...
https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chat...
Had he said "not ONLY a statistical engine for pattern matching," I would agree. But I'm pretty sure that the majority of what human cognition is and does is exactly what he describes. The pattern matching engine encodes the wisdom of thousands of years of evolution, in addition to the experiences of any particular instance (person), but in the end, most of what we do and say is generated by predicting, based on pattern matching, what is most likely to get the brain positive feedback.
The thing that the rush to more and bigger LLMs seems to leave out, to me, is that the pattern matching and prediction that drives intelligent response and behavior in humans (when you can find it), is not merely prediction about language, but rather prediction based on multiple learned (by humanity generally through evolutionary incorporation into our neural architecture, and by individuals through experience) models - of space, time, biology, abstract reasoning, ontologies that objectify the natural and human intellectual worlds, and more. LLMs encode very little if any of that directly, but rather get pieces of it indirectly through the imprint of ontology and reason that are baked into word (token) usage patterns.
So, I agree with Chomsky on the main point: you can't really get much beyond fluency with LLMs alone. There is way too much hype on these things.
Adding in layers of complexity is going to happen next, and I expect it is going to get wild.
I’m not sure I would even then. I mean maybe, but that’s not the clearest, most certain, fundamental difference.
A more clear fundamental difference is ChatGPT instances are all, metaphorically, instinct with no space for intelligence. They have lots of “training”, but that all happens before they are capable of acting, and once they are capable of acting their behavior is entirely preprogrammed based on a very small input window. They have no memory of experience (that’s simulated, within the token limit, between the model and the user for chatbots), much less a reward mechanism that would let them learn behavior from experience.
That ChatGPT regularly "makes stuff up" and that there is no difference between the truth or falsehood of any statement also seems to be false. I asked ChatGPT to act as a "[Lie Detector]" and to rate the truth or falsehood of a variety of statements asked. I asked about 40 questions ranging from physical situations (heavy objects floating away into air) temporal questions (time travel, etc.) and logical questions - and it very accurately could determine if each of these statements was "true" or "false" given physical or logical rules. Again not perfect but very accurate (38 out of 40 correct).
With attention - ChatGPT is very obviously operating at a level above the simple probabilistic prediction. It clearly seems that it has some notion as to the meaning of what is being said and is making inferences based on that meaning. That those inferences were trained probabilistically is certainly true, but that it was trained on the average human's understanding of those physical or temporal or other constraints also seems to be true and to also be fairly accurate.
1) One instance first parses the chat and last message to generate a response. Currently this is where things end but we can keep this private and do additional work.
2) A second instance, properly primed, can take the last prompt and response and "analyze" it, generating scores for things like factuality and usefulness, possibly adding commentary.
3) Pass into a third instance that has the chat history again to rewrite the response, taking into account the feedback.
4) Optionally repeat #2 and #3 until it passes some quality threshold.
edit: By that I mean the following:
"Note, for all the seemingly sophisticated thought and language, the moral indifference born of unintelligence. Here, ChatGPT exhibits something like the banality of evil: plagiarism and apathy and obviation. "
He does not seem to understand it is a feature of the system.
He is absolutely correct about the level of hype though.
Typing the prompts from the article after the DAN (11.0) prompt caused GPT to immediately respond with its opinion.
Chomsky's claims in the article are also weak because (as with many discussions about ChatGPT) they are non-falsifiable. There is seemingly no output ChatGPT could produce that would qualify as intelligent for Chomsky. Similar to the Chinese room argument, one can always claim the computer is just emulating understanding.
I've yet to see a convincing argument that humans are any different. They're sometimes better at pretending to understand things, but at the end of the day both humans and ChatGPT have a small handful of things which they functionally understand[0] and a larger body of knowledge which is only partially integrated.
Chomsky has disappeared up his own intestinal tract on this one. One can quibble about intelligence until the end of time, but the real question is that of utility -- which they certainly do have, in ever-increasing scope and measure.
[0]: i.e. have synthesized the object and can properly explain and apply it in other contexts
We use all those amazing tools while knowing only a fraction on how they actually work ( or what to do when they break ). Do we merely mimic or do we understand? GPT brought us to an interesting philosophical ledge.
edit: somewhat related tangent
My extended family member recently claimed she is a conscious consumer unaffected by advertising and therefore not concerned about targeted ads. Is she conscious if she picks what everyone around her picks as a way to fit into society or does she understand her choice, underlying forces and simply opts into them?
And you'd be correct. The point isn't what kind of output ChatGPT can produce, the point is what kind of input it takes to create the model.
If ChatGPT were to gain language understanding at the level of a toddler trained on the same number of tokens that a toddler needs, then you could start postulating that the model is really learning, rather than becoming a sophisticated stochastic parrot.
I may be wrong, but if that is the case, and we are still arguing with appeal to authority, shouldn't he defer to experts in that field? Shouldn't AI experts opinion be valued more than his observation?
Note, I am merely raising a possibility that he is wrong about this particular idea.
More to the point, do you think his understanding of syntax really means he understands the underpinnings of AI; the same AI that merely tries to emulate human language capability?