Comment by twarge | Hacker News Reader

Comment by twarge | Hacker News Reader

Chomsky's article really seems to show the results of decades of living in an echo chamber.

>On the contrary, the human mind is a surprisingly efficient and even elegant system that operates with small amounts of information;

True, but then

> it seeks not to infer brute correlations among data points but to create explanations.

This is such an odd dichotomy, and feels really wrong to me. Explanations are to me just further correlations.

> a young child acquiring a language is developing — unconsciously, automatically and speedily from minuscule data — a grammar, a stupendously sophisticated system of logical principles and parameters.

Chomsky apparently has children, but clearly may have forgotten what it's like for them to learn language! It's years of trying to communicate, constantly, failing, and learning.

Now, go ask ChatGPT to Write an ode to <obscure topic> in iambic pentameter and I bet that it will be as astonishing, creative and fluent as a native speaker.

> Chomsky apparently has children, but clearly may have forgotten what it's like for them to learn language! It's years of trying to communicate, constantly failing, and learning.

You are correct that children learning language are constantly failing in some manner. There has been a lot of study of the language acquisition of children. However, the way in which children fail and do not fail is specific. Some things children will fail at a number of times until they learn. On the other hand some rules of grammar, logical principles and parameters children seem to pick up immediately, and in Chomsky's view are even innate.

So you can divide theoretically possible failures into two categories - ones theoretically possible that are not made, and ones that are theoretically possible and are made. The first category is what he is talking about.

Also - there are different languages with different rules, and people learning a language have to learn the rules of English which are different from Spanish which are different than Japanese. Although in Chomsky's view, these languages are all so similar (compared to say the recursively enumerable language a Turing machine uses), they are virtually identical from a theoretical point of view.

Also - children make grammar errors such as first saying "I go to the store" (correct) and then "I goed to the store" (incorrect). These errors were not made by data they acquired! The errors children make are showing they are beginning to understand grammar rules, it's a sign they are going off a series of rules and not just regurgitating data they have acquired.

I'm not here to argue about Chomsky's basic claims (which I do not find useful), but I am here specifically to argue against the idea that children easily learn language with somehow way less data than ChatGPT. Those who claim otherwise simply have not spent a couple years with a baby.

ChatGPT is trained on some healthy fraction of the internet. You're arguing that the amount of information a baby consumes to learn language is comparable to the size of the internet?

A baby almost certainly consumes far more data before it speaks than has been fed into GPT-3, yes.

GPT3's input is something like 45TB of data. It's a significant fraction of all text humans ever produced, to the point researches are concerned we may run out of text within the next few years.

Normal healthy language exposure for children is around 100KB/day, we can round up to 40MB/year.

No, I have not spent ~1000000 years around a baby.

Chomsky is a fool and this article demonstrates it clearly.

First we don't really know how the mind works and therefore we really have no idea if the language models are path toward AGI or not.

He doesn't have an idea how a child obtains a language. You really need a lot of data to acquire the grammar, no, minuscule data is not enough.

I have observed it personally with my multilingual children, their mother tongue, to what they have the least exposure and what is the most complex, is not still not fluent for them and yes, they make obvious grammar mistakes that demonstrates that they have not fully internalized it.

His claims are simply false, easily refuted by empirical data.

I would call it fallacy of English. English is stupid simple language. Most languages are not.

I have to say I don't find "I have kids" a compelling dismissal of a lifetime of work, nor do I agree that English (or any other language people speak natively) could be described as "stupid simple" (and really, do you think Chomsky did not consider any other language before positing a universal grammar?).

It is not my fault that he made his claims too easily refutable by claiming that "young child acquiring a language is developing speedily from minuscule data". I have a comparison between two languages that are both not simple as English and there has been clear correlation between the exposure time (the amount of data) and fluency and the difference between the exposure has not been minuscule. I have noticed the same by other similar children.

I honestly don't know what he did or did not consider considering that he has made very bad judgement about two aggressor states: Serbia and Russia based on his anti-US stance. It clearly demonstrates that he is able to not think clearly.

Nobody says that exposure to the target language is irrelevant and Chomsky's political views are completely beside the point.

He didn't claim that it is irrelevant, he said that it can be very small. This is clearly wrong based on empirical evidence that can be quantified based on hours of exposure to one or another language. It is enough to show a single counter example to his claim.

Explaining away genocide is not a political view.

As far as I know: Chomsky did not support the Khmer Rouge - he questioned information about the Cambodian genocide.

I read Henry Kissinger's China book. He seemed to be not purely happy about US Cambodia politics. (but reasons)

"Allegations of United States support for the Khmer Rouge"

https://en.wikipedia.org/wiki/Allegations_of_United_States_s...

Reply on news.ycombinator.com