Caricaturing Noam Chomsky
garymarcus.substack.com
garymarcus.substack.com
Statistical models turn out to be enough to get past the Turing test. They're probably going to be better at interpreting vague requirements and making decisions than most humans too. They're better at art than the average, and better at explaining themselves than most humans.
[0] https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chat...
Using an LLM and being amazed by its response is not doing the Turing test. The Turing test is mostly a thought experiment anyway.
In some sense ChatGPT is actually engineered to fail the test.
Also, although I understand the concerns over super intelligent AGI, I suspect that in the short- to medium-term, that won't matter either. What will matter is highly optimized LLM's putting high swaths of the white collar workforce out of work, and the economic upheaval that will ensue.
Just to add some science fiction to the discussion: what if we are in an AI/human flippening moment? Where humans used tools for million years and orchestrated the goals while now we are moving to being more like a tools, in the sense that we are necessary some times but completely unnecesary in others.
Shiny tech will provide copious benefits, but cannot be what it is not: organic genius.
Humans are indeed 'trained' on a much larger set of data than 'just' the whole text of the internet. Indeed orders of magnitude more: think of all of the video/audio/olfactory/social cues/physics we absorb every day of our lives through all of our senses. How is that not infinitely more data than the public available text on the Internet? What is to say that a way larger LLM-like model would not be an AGI? why should human intelligence NOT be just a statistical engine?
After all, haven't we proven already that physics is more-or-less statistics at heart (quantum mechanics)? that doesn't make physics non-deterministic at human scale. So why would the result of an LLM-like model trained on reality be not-deterministic or wrong? How is a high-confidence statistical result different than reality?
In other words, nothing in Chomsky's article proves that humans are NOT statistical pattern matching engines. I am not saying that we are BTW, but it's an intriguing possibility that should be researched. After all, statistical methods and ML have made a lot more progress in the last 10-15 years than the MIT-driven, deterministic school of thought did in the previous 30. Cambridge lost, plain and simple.
Advances on probabilistic search will have equal facility with hallucination as with the real.
Regardless of whether Chomsky's article "proves" your negative, there is plenty of evidence that LLMs are not even as good as dogs and cats at forming a theory of the world around them. That we might draw a conclusion that dogs, cats, and people work differently from LLMs seems more like common sense.
What we lose in the drowning is the ability to distinguish quality. We forget which way is up, where the air is, and swim down further until it is too late.
- LLMs are sometimes said to be “just” shallow pattern matchers, “just” massive look-up tables or “just” autocomplete engines. These comparisons amount to a form of (methodological) reductionism. While there’s some truth to them, I think they smuggle in corollaries that are either false or at least not obviously true.
- For example, they seem to imply that what LLMs are doing amounts merely to rote memorisation and/or clever parlour tricks, and that they cannot generalise to out-of-distribution data. In fact, there’s empirical evidence that suggests that LLMs can learn general algorithms and can contain and use representations of the world similar to those we use.
- They also seem to suggest that LLMs merely optimise for success on next-token prediction. It’s true that LLMs are (mostly) trained on next-token prediction, and it’s true that this profoundly shapes their output, but we don’t know whether this is how they actually function. We also don’t know what sorts of advanced capabilities can or cannot arise when you train on next-token prediction.
- So there’s reason to be cautious when thinking about LLMs. In particular, I think, caution should be exercised (1) when making predictions about what LLMs will or will not in future be capable of and (2) when assuming that such-and-such a thing must or cannot possibly happen inside an LLM.
https://www.erichgrunewald.com/posts/against-llm-reductionis...
Imagine an "AI" that "knows" about rhyme and meter and can generate a seemingly infinite collection of catchy songs that follow the II V I chord progression, only the sequence of words doesn't necessarily make any sense.
LLMs are a more powerful version of that kind of thing that does put the words in a sequence that feels like natural human language. Since they are trained on information, the sequences chosen often seem informational, even though they are just patterns. It's really a lot like autocomplete.
In other words, a lot of useful information may accidentally be encoded into the model, but so is a lot of stuff that looks/feels similar but is nonsense.
It’s completely true that LLMs are trained on next-token prediction (although some, like ChatGPT, are then additionally trained using reinforcement learning with human feedback). It’s also completely true that this fact profoundly influences the texts they generate. So I don’t think it’s unreasonable to call LLMs autocomplete engines or to emphasise next-token prediction. But I think it’s subtly misleading:
- Though LLMs were trained to optimise success on next-token prediction, that is not necessarily what they do. We don’t know what it is they do. The training process reinforces behaviours/heuristics in the model that tend to cause it to make better next-token predictions on in-distribution data. This does not mean that those behaviours/heuristics are fundamentally “about” optimising next-token prediction, especially when the model encounters out-of-distribution data.
- The usual example here is human evolution. Humans were shaped by a process that optimised for reproductive fitness. This gave us a bundle of drives such as family kinship, prestige and sexual pleasure – drives that aren’t fundamentally about optimising for reproduction, which becomes evident as we enter a new environment – one with contraceptives, say.
- Optimising for a task for which intelligence is useful encourages the optimised thing to become more intelligent. Sam Altman gave expression to this last week when he wrote, “Language models just being programmed to try to predict the next word is true, but it’s not the dunk some people think it is. Animals, including us, are just programmed to try to survive and reproduce, and yet amazingly complex and beautiful stuff comes from it.”
- The forms of intelligence that are useful in doing next-token prediction are different from those that are useful in human reproduction, but I think there’s a considerable overlap, as (1) some fundamental abilities, for example using and applying concepts, just seem very broadly useful and (2) the data LLMs are trained on are written by humans, for humans and often about humans and things that matter to us.
I think the "they're just autocomplete" take also hides other properties of LLMs, like them seeming to (as mentioned in another comment, and in the post) contain and use world models, and being able to learn general algorithms.
The point is is that dismissing it because it's "just patterns" ignores the fact that nobody has proven that human cognition isn't also "just patterns". If it is, then that undermines the whole justification for the dismissal. So the dismissal assumes the conclusion until that evidence is presented.
Chomsky's point that we have no basis for judging LLMs as cognition machines because we don't know what cognition is, is correct. Building these machines may produce useful tools, but it also doesn't advance studies of cognition, either.
We do know one characteristic of cognition as it occurs in nature. There seem to be natural affinities to distinguish the real and true from the hallucination. Perhaps that comes from persistence and physicality. How do we give that to ChatGPT?
That's not Chomsky's point at all. He's saying they are emphatically not understanding or intelligent in any way, and that's not justified.
> Building these machines may produce useful tools, but it also doesn't advance studies of cognition, either.
I couldn't disagree more. We have here apparently intelligent systems completely unlike naturally occurring ones. IMO, you'd have to be crazy to think that comparing and contrasting the properties of these systems won't provide any valuable data about the nature of intelligence, perception and cognition as a whole.
In fact, we're already seeing papers discussing how LLMs basically refute Chomsky's whole model of language [1].
> So Noam's position is basically that doctors will never discover Vitamin C unless they develop chemistry.
I disagree with this analogy. LLMs and machine learning is playing with chemistry, and Noam's position is more like, "You'll never understand any chemistry unless you first figure out subatomic physics". That's just obviously false, we can understand and do quite a bit of chemistry without a full understanding of quantum field theory. At best, such knowledge impacts our understanding at the fringes of chemistry.
Literally the author used the analogy so I have no idea what your point is if you disagree with the author's analogy. It tells me you are not thinking through the piece analytically.
It's not an uninteresting paper, but by the author's own admission, he doesn't give an astute reader good reason to proceed to the next section, since that was the analogy that he tried to draw.
Those arguments and the empirical data that follows are exactly why I think my analogy is better, but it seems silly to dismiss the remainder of a paper based on that flawed analogy.
I'm also seeing this paper circulated on social media so I automatically start squinting when random people start promoting it that way. A quick search on Google shows that the author has had a 10 year long back and forth disagreement with cognitive linguists such as Chomsky's colleagues. That's a bit of omitted context that this social media amplification does not help with when portraying ongoing research debates.
Which is quite useful to do some tasks: writing fiction (it now wrote at least 6 characters in multiple rpg campaigns I participate in) and probably others (I used it to write tests data, but in fine I had to revalidate it by hand. Could do better.)
I do not see LLMs be more than that in the near future though. More informed, more accurate bullshit generators? Yes probably.
And I'm not saying it isn't impressive and very close to human behavior. I still think we're missing a part in our models though, and until we find it, breakthrough will be limited to improvements on accuracy and information.
Beyond that, many intellectual capabilities are also useful for successful bullshitting, so even if we can confidently say that that's all they do (in some sense), that doesn't mean they don't also do something-like-human-reasoning etc.
I don't really know how to explain it, but to me we have at least two kind of reasoning, swallow and deep. LLMs could well be capable of the first.
I think the whole stable diffusion and ChatGPT thing will come and go, almost as self-driving cars have. If you remember, just eight or so years ago, everybody was absolutely convinced that they were right around the corner. There was an insane amount of hype. Now, that hype has largely come and gone, with several companies abandoning their self-driving approaches or just flat out failing at them. There is still some hype, but I feel it has faded as people have come to grips with the premise, much less the difficulty of the implementations. The focus seems to be shifting back to driver assistance features.
Another thing is VR. So these stable diffusion and GPT methods are having their time in the sun. Yet again, just like VR and self-driving, proponents are trying to will these things into existence. Yes, they are impressive in some very specific cases, but they are not general. Just like stable diffusion, these GPT things will just flood the Internet with junk, either filling people with incorrect knowledge or driving people away, leaving the only remaining application to be advertising and trying to sell stuff.
Humans are seemingly absolutely obsessed with technology trying to replace humans and reality. That is the common thread between all of VR, self-driving, stable diffusion, and now ChatGPT. But this is both hard and destined for failure. People keep discovering that the role of technology is to augment humans and not to replace them. Engelbart knew this 60 years ago.
But Stable Diffusion and ChatGPT are here, and the last one has more than 100 million users. Unlike self-driving cars, the excitement behind this technology was not created by the promise of building "Stable Diffusion" or "ChatGPT", the excitement was created by the people who tried these models for free and saw their expectations subverted. I think the difference is abysmal.
Chomsky and his critics agree here. Chomsky doesn't like it, his critics are more open.
Our brain processes information much more efficiently so we humans still have an advantage for certain tasks.
Self driving cars excited people but they're still not on the market so of course it's easy to forget.
But why would people forget this and go back to doing laborious writing work themselves?
i think the idea is that by trying to replace humans we learn more about what it means to be human.
I have my own reservations about his opinion but would never state them in the combative and sneering tone often employed by his critics.
Can't help but feel like they personally have a problem with him, and they're using this piece as a cudgel to "take a swing".
If we are evaluating LLM's purely on outcome alone, it seems only fair to compare them to humans that don't have fully functioning cognitive capabilities. We wouldn't discredit a schizophrenic person that hallucinates, we know that this person can also see reality for what it is, most of the time. I just don't think that finding absurd examples, usually after careful prompting, is a slam dunk against LLM's at all.
They can, it's just that Chomsky's dismissal of "weight adaptations" as insufficient is not actually justified by any evidence. The Transformer model augmented with memory is Turing complete, so it can in principle learn to compute anything using only weight adaptations.
* On the Computational Power of Transformers and its Implications in Sequence Modeling, https://arxiv.org/abs/2006.09286
Our brains are much more complex than an artificial NN, though. So the mere thought that the two somehow share functionality after a wildly different learning process is somewhat far-fetched.
Well sure, because CFGs can have unbounded depth, and all real brains are finite. But they can learn bounded CFGs:
https://arxiv.org/abs/2112.09174
> Our brains are much more complex than an artificial NN, though.
Yes, brains have a few orders of magnitude more parameters, and lots of evolutionary baggage. It's not clear how many parameters would be needed without that evolutionary baggage. Time will tell.
> So the mere thought that the two somehow share functionality after a wildly different learning process is somewhat far-fetched.
I don't know what precisely you mean by "share functionality", but it doesn't seem like a stretch to suggest they are in some sense isomorphic given Turing equivalence. Rule 110 doesn't look like it should share much with Turing machines or the lambda calculus or SKI combinators, but these are all equivalent in a formal sense.
> in some sense isomorphic given Turing equivalence
Then you're brain is just like any CPU, which is very far from what such a statement tries to express in the context of NNs.
Yes, the brain is like a CPU + a specific program running on that CPU. The properties of that program are where the isomorphism comes in. So to come full circle back to your original point, we don't yet know that "weight adaptations" are not a sufficient operational model for that program.
In contrast, the world is too complicated to learn through exposure. Something has to structure our learning process. Language is (in principle) not learnable, only when you make enough assumptions. Those assumptions must be coded somewhere in your brain and probably genes.
Tabula rasa sounds nice, but it is untenable and utterly lacks evidence.
People who rely on LLMs to do their work in domains they understand find them useful in the sense of hefting another useful tool as humans have done throughout their existence. That's it. Among serious users of LLMs as a productive tool there's no wistful wonder that they may be speaking to a truly understanding intelligence. So Chomsky and the rest of the critics don't really have a foot to stand on when they say things like
> It is at once comic and tragic, as Borges might have noted, that so much money and attention should be concentrated on so little a thing — something so trivial when contrasted with the human mind, which by dint of language, in the words of Wilhelm von Humboldt, can make “infinite use of finite means”, creating ideas and theories with universal reach.
Why is it comic and tragic? LLMs are genuinely useful tools and one of my biggest productivity boosters at work since I learned how to make a computer do my bidding. Of course huge amounts of money are being poured in when it's so helpful across a number of domains for getting work done.
Really, folks, there's nothing to see here. Make use of these marvelous tools and continue doing what humans have done best, a truly innate human inheritance, to use tools to support our goals.
> However useful these programs may be in some narrow domains (they can be helpful in computer programming, for example, or in suggesting rhymes for light verse), we know from the science of linguistics and the philosophy of knowledge that they differ profoundly from how humans reason and use language.
I think you and he agree on your main point, but his main point is about something else.
Not that this is really relevant to this topic, but that's simply not true. He doesn't support Russia's interests, he's simply acknowledging that when dealing with powerful nations, we have to consider their interests and reach compromises where their interests conflict with ours in order to avoid the horrors of war. You know, diplomacy 101.
Has been doing that for decades.
He is intellectually dishonest and morally bankrupt. This makes his thoughts on other topics suspect as well.
Moreover how do you do diplomacy with Hitler2.0?
Thinking that giving up yet another piece of (for you) unknown land to one of those will bring peace makes you sound really naive.
Speaking of intellectual dishonesty making other thoughts suspect...
Chomsky used some examples in his article of what LLM's would get wrong. It is clear that he hasn't even tried, because chatGPT in fact gets those examples right. Why does he get a free pass for such errors?
Sounds like Aristotle when he claimed that women do not have wisdom teeth for reasons (he made up), instead of bothering to check.
You might want to check your wife's teeth :)
What did Scott Aaronson contribute of comparable impact?
The tragedy seems to be that during the hype of any technological bubble, someone of the scale of Chomsky is needed to make at least someone hear the obvious. But maybe it's also a comedy.
However, it doesn't matter. All flying cars will be on RISK V and Rust and on the blockchain soon. That's what's main.
No, the comedy is that he's saying things that seem completely obvious but are actually completely wrong.
This, I fail to understand. Clearly, Chomsky started with an exposition of the concept of negativity. How can this be missed by anyone with cursory familiarity with western philosophy? The rest of the article (including the passage regarding the apple) is pretty much a series of excursions on this theme, how LLMs systematically miss negativity.
Anyways, what I tried to suggest was that there is some base for reasoning that what we're encountering here is more like a cargo cult of conceptuality than conceptuality itself at work. And that it is still on us to decide, if want to be persuaded by this to land our planes of reason next to it, or not.
One thing grammar does not do is constrain grammatically-correct language to produce only the truth. It is as easy for humans as it is for LLMs to say false things, to construct false reasons for believing those falsehoods, and to be inconsistent about it. The argument Chomsky uses to claim LLMs will never lead to and understanding of the mind can equally be deployed against the prospect of understanding the mind through studying grammar.
If anything, the success of LLMs, in producing grammatically-correct language about as well as humans do, is evidence for this position. They also present a significant challenge to the Poverty of Stimulus claim[1], which is often used in support of the universal grammar hypothesis.
In the early 20th. century, western metaphysics took what has been called the 'linguistic turn'[2]. Personally, I am deeply skeptical of the proposition that studying human language will - or could, in principle - lead to knowledge about how the world must be, and the ability of people to hold inconsistent beliefs and to argue for falsehoods is one basis for that skepticism.
In all fairness, this belongs only to context-free grammars. Also, I'm somewhat concerned that we may see a resurgence of behaviorism promoted along the lines. (It may be worth remembering what kind of regime this had been.)
As for behaviorism, it can stand or fall on its own merits (or lack of them), just as should the alternatives.
If a clock was correct 95% of the time, would you say it has no understanding of time? To me it seems that both Marcus and Chomsky are resorting to a sort of dualistic theory here. They concede that these LLM's are capable of impressive displays of cognition, but wait! It's not real cognition of course, it's fake, unlike the one in humans. Well, we don't know enough about cognition in humans to reject the hypothesis that we are also communicating or thinking based on similar principles. ChatGPT is by far the most successful automaton for understanding and producing language we have ever seen. The language Chomsky uses in the op-ed and in his reply here - "(...)Therefore they are telling us nothing about language, cognition, acquisition, anything" - is pure hubris and completely unjustified.
Similarly, we could sprinkle formic acid on the floor in a pattern that fooled ants into following it in search of food, but there is nothing "correct" about it, merely "realistic".
Asking GPT3 to create a persuasive essay is impressive in the same way. It "knows" how to string words together in a persuasive way, but just as some salespeople may not care about how factual their statements are, ChatGPT generates the veneer of realism but will often generate something that in spite of its realism is utter nonsense, or is even mostly "filler words" with little meat.
The veneer, filler words and style that humans (and LLMs) use in language are the parts that more similarly mirror the chemical signals used by insects. They are what we use to manipulate and to persuade rather than to convey understanding. This is why LLMs are scary when we consider their use in political misinformation campaigns, but less scary when we consider them writing dissertations about deep scientific topics.
My point is that we should not make these types of distinctions about how "we" "reason" versus how an LLM does. For example, I have no idea if your reply came from ChatGPT, but it sounds pretty intelligent to me. In that sense, your output is "correct" because it led me to believe I am arguing with an intelligent being. The analogy to a broken clock which is correct twice a day is ridiculous, because ChatGPT, on average, is quite capable of producing meaningful replies to a very complex problem (natural language queries).
I don't quite get your analogy to insects. If we figure out an analog pheromone to the ones ants use to communicate, then we found out something very meaningful about ants. Likewise here, I am quite certain ChatGPT is not producing language in the same way a human does, but is it using the same principles? Chomsky and others are prescriptive in saying it is not, yet their arguments are not convincing.
Chomsky has around half a million citations to his scientific work, mostly on language. The guy is amazing. At age 94, he is productive, sharp and very well spoken. I tend to take him more seriously on this.
One problem is that, experts in one domain opine about domains they have not worked on. Another problem is, some people apparently hold a view that, if you make a product that works and sells, nothing else matters. Everyone will rush to earn a living around it. They feel empowered to take any position. So Science doesn’t matter if it’s not aligned, products do. Aaronson didn’t publish a proper article addressing Chomsky’s arguments.
"[Note for people who might be visiting this blog for the first time: I’m a CS professor at UT Austin, on leave for one year to work at OpenAI on the theoretical foundations of AI safety. I accepted OpenAI’s offer in part because I already held the views here, or something close to them; and given that I could see how large language models were poised to change the world for good and ill, I wanted to be part of the effort to help prevent their misuse. No one at OpenAI asked me to write this or saw it beforehand, and I don’t even know to what extent they agree with it.]"
This has been my biggest issue around tech recently, and the whole conversation on ChatGPT and the latest generation of AI stuff just makes it a lot more obvious. They rush to make it and sell it, and that's all that matters. Societal externalities be damned. There's a lot of people way too invested in making sure stuff doesn't get discussed, all around.
There's already a myriad of extractive NLP solutions available that organizations could be utilizing (like any simpler BERT-based model.) Unfortunately, they don't come with the veneer of misleading ethos that GPT/LLMs pretend to have.
Personally, I see some potential for customer-facing uses, but that's because customers like novelties.
A rudimentary intelligence capable of basic object-recognition and permanence* would be an actual step towards AGI over LLMs, in my professional (but still fallible) opinion.
Something as basic as an insect, or hell, even protozoa, is all we would need to say that we're on the way. Once we have that, tying it with an LLM to give it denotative capabilities would be the boom we're all hoping for/terrified of.
[*] this is key, but we don't need to codify this so much as find a way for the object-recognition to build up to it. Even children take a while to develop object permanence (hence peek-a-boo being fun for them - you really are disappearing.)
The generality of the intelligence is unquestionable, at least in a language domain. Discounting it as AGI because a fly is better at being a fly seems besides the point.
1. It has agency. 2. It can move about and manipulate the world in a general as opposed to predefined sense.
Edit: that final point is why I propose an in-to-out approach over an out-to-in approach (which is what LLMs are). Let the intelligence make itself, instead of us trying to make intelligence.
Well, that’s ridiculous out of hand. What makes you say something like that? If they feel intelligent, pass intelligence benchmarks and are designed to be intelligent— why not ascribe to them intelligence? Is it because they aren’t wet?
I don't call any of the ML models that I create "intelligent" - I call them probabilistic. Feigning intelligence with probabilistic methods is nothing new (eg ELIZA). If you can scientifically demonstrate that feigning intelligence is equivocal to de facto intelligence, with something more than "feelings", I'll be happy to read your write-up.
No problem with probabilistic methods. Human intelligence is also probabilistic (ie, predictive coding). Maybe the biggest question: do you think these will be exponentially more powerful in 5 years, learning from failures, self-improving and generally economically “a very big deal” due to human replacement? Because that’s what AGI is supposed to do, rapidly—and that’s why it is worth acknowledging the possibility.
From my perspective, if it barks like a dog, walks like a dog… fulfills all the “dog” benchmarks… even if it came about through unusual means, you’d better consider the possibility.
All these experts Pooh-poohing the possibility of AGI because “I’ve worked with LLMs” is really unconvincing. The debate isn’t balanced because most professionals wouldn’t want to claim AGI for professional reasons.
Most professionals in my field would probably consider this a career-defining moment that would land one in the halls of the greats. I don't really follow this thinking.
And I get the "looks like a duck, quacks like a duck" argument, but as I've repeatedly stated, that's not good enough.
Frankly, I don't buy the "predictive coding" theory - at least, not to the extent that I believe the brain is only probabilistically modeling the phenomena it experiences. This ignores the glaring fact that our senses in-of-themselves exhibit some astounding capabilities right out-of-the-box, and predictive coding has no ground to stand on without the cross-referential analysis of the senses. Human babies are developing their intelligence long before any heavy stimuli. Hearing is the most active before birth, eyesight - our most rich sense - doesn't kick in until after birth. Nonetheless, babies come out of the womb with intelligent reflexes, such as rooting. That bundle of rudimentary intelligence is the basis upon which predictive coding begins - a functional default. An LLM is a set of statically-coded weights, incapable of context switching or object permanence. A child may take a while to understand the difference between "you" and "me", but they still can intuit "thing A" and "thing B". LLMs cannot - at least not without being paired with another model of some sort. And that model will be what tips the scale, not scaling up the features of what's merely a beautiful implementation of the logit function.
You also see that some basic production rules (for basic reflexes) could mean the difference between AGI chatGPT and chatGPT today. Because of how it would help chatGPT orient towards the world, I suppose?
I buy that critique because I do view a cybernetic loop as essential to intelligence — there must be some goalstate (however vague) to be achieved successfully.
But, have you seen many claim that chatGPT is AGI? My issue is not with your argument (it’s good!) but the lack of acknowledgment for the reasonability of the position that chatGPT can be called AGI. Most people DO buy predictive coding. And many would view the missing “reflexes” in chatGPT just a matter of good UI/UX (for instance). Yet, I haven’t seen anyone stand up and systematically argue that chatGPT is AGI. (But I’m not on Twitter, so…)
As for the assertions being made - at first, no, but once the Bing Search fiasco kicked off, I've seen more and more (here on HN) making the claim that ChatGPT and its kin are the beginning. Ultimately, it doesn't irk me - once we hit the same plateau as usual, the hype will boil down - but I do feel a bit concern at what may come of poorly-wielded LLMs before the industry wises up to their risks. A lot of people take ideas they see on HN back to their jobs, so we have a bit of responsibility to manage such hype here.
With hype I think of blockchain (wherein I never understood why removing people from decision-making through automated contracts was worth so much) or the metaverse (where I never understood how they envisioned a world with tech strapped to their face).
The public communication about AGI isn’t exactly a hype cycle. People should, instead, be expecting a fairly dramatic change in many areas of life over the next 3-5 years.
This is pretty unfair IMO. Aaronson wrote (emphasis mine):
> In this piece Chomsky, THE INTELLECTUAL GODFATHER GOD OF an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded.
He never wrote that Chomsky himself was involved in this efforts, only that he was influential to it (which may or may not be true, but Marcus never argues that point).
Vs.
Therefore they are telling us nothing about language, cognition, acquisition, anything.
As anything with Chomsky, the hyperbole misses the point. A useful piece of technology still has uses. An airplane still flies even if it tells us nothing about how birds learn to fly or think about flight or even navigate the globe during an annual migration.
https://dnyuz.com/2023/03/08/noam-chomsky-the-false-promise-...
In the late 1990s, some undergrads would see Chomsky campus-wide talks as an opportunity to mock him, but it seemed to be about disagreement with his "politics". "Dept. of Leftist Linguistics", and worse. (The student body seemed overall libertarian, and of overall narrow exposure to the world outside STEM, compared to my previous university.)
Not my main point but to be more nuanced I think ChatGPT could produce *some* amount of original science in places where people written about some phenomena but weren't able to connect the dots (come up with analogies shedding new light etc.). But ChatGPT won't go out looking for the dots.
I think it won't go looking for new dots because of 2 reasons.
1. It can't. It operates on language alone
2. It doesn't ask questions by itself. It only gives answers. It doesn't have the drive to seek knowledge. I'll try to illustrate my intuition: There was a gorilla Koko. She was taught to communicate with humans and could answer questions. But Koko didn't ask questions. She was observing the world but didn't have the concept of unknowns. The sphere of things she didn't know simply wasn't existing for her. I feel like ChatGPT is the same in that regard.
1. I can't find the article. I can be mistaken about Koko, but my illustration still stands.
2. (So many tangents....) Maybe 'curious' model could be built, but ChatGPT would only be a language part of the bigger model. Whole other paradigm would be responsible for curiosity part
Question. Mathematics originates in axioms. Maybe that's why it fits the world. Because people observing the world chose the axioms that made sense to them. So when it's advacing to it's logical conclusion it still happen to fit the reality. If mathematics wasn't formalized already would ChatGPT ever invent mathematics? Would it come with any kind of axioms?
https://www.reddit.com/r/chomsky/comments/11gfo8m/why_are_ch...