Toddlers’ grammar skills not inherent, but learned, new Stanford research says | Hacker News Reader

Toddlers’ grammar skills not inherent, but learned, new Stanford research says

news.stanford.edu

48 pointsstared44 comments

See also: On Chomsky and the Two Cultures of Statistical Learning by Peter Norvig: http://norvig.com/chomsky.html

This basically postulates that language learned is a process similar to a supervised machine learning algorithms, and this, perhaps, is a more realistic view that Chomsky has. The machinery has been evolved, but "the training data" is required to make it work.

Be careful of the trap: for quite a while now, each generation has simply treated it as "realistic" to match the brain's workings to that generation's popular new computational technique.

What's interesting is that Google's Translate, via neural network training, seems to have found an efficient structure underlying many languages:

https://www.newscientist.com/article/2114748-google-translat...

It is learned because the brain has specialized circuitry to be able to learn it.

The machinery obviously has been evolved and is inherited, while a grammar of a particular language has been learned.

There is also a strong hypothesis that the structure of the specialized areas of the brain "reflects" the general notions of the shared environment, such as things, attributes, processes which corresponds to nouns, adjectives and verbs.

Like visual cortex has its own hard-wired heuristics and presuppositions (that use environmental cues such as shadows, distances, illumination) which are used in the process of scene decomposition and interpretation, the language area should have its own hard-wired heuristics, which, presumably, are used in grammar learning.

The knowledge (heuristics) are encoded in the structure, not in some kind of information.

Why "obviously" ?

It is obvious after taking a decent psychology course, like the one of Paul Bloom at Yale.

> The knowledge (heuristics) are encoded in the structure, not in some kind of information.

When you're talking about neural computation, distinguishing between an evolved network structure/architecture and "information" (or at least biases toward certain kind of information) is not very useful. The nature of these systems as computing substrates is that they encode information in network structure and rules for updating that structure.

> they encode information in network structure and rules for updating that structure

I think use of the term 'information' is not accurate. There are no bits and the whole information theory, it seems, is synthetic and has nothing to do with how the brain actually works. It is not a digital machine but electro-mechanical, if you wish.

Respectfully, this is begging the question :)

You're correct that neural networks don't encode information in the way current digital machines do (i.e. with explicit records/files/etc encoded in bytes). I would disagree that this is the only useful view of information. And certainly, from a Shannon information perspective, the network structure contains information (in the Shannon sense) about the environment or problem for which it was optimized.

> the network structure contains information (in the Shannon sense) about the environment

... the way a map represents a territory instead of how a guide book does ;)

Or the way a 3D structure made out of the same 20 joined amino-acids could represent either an enzyme (the code) or a protein (the data) and that there is no distinction between code and data (which is the great insight realized in the original Lisp. It is not a coincidence that Lisp was the language or the classic AI. Sorry for the reminiscence - just love this part).

It is said that babys who are spoken to in multiple languages speak later than those who are spoken to in just one language. Could this extra 'learning' cause the delay in speech?

How is this not obvious? Who could claim that grammatical understanding is innate given that grammar is tied to the language and the language itself is learned? Studying how grammar is learned certainly seems like a valuable endeavor but I don't understand the "debate" over whether it's innate or learned.

To simplify, the Chomskyan position is that there are certain shared characteristics of all human languages' grammars, and that these characteristics reflect structures in the human brain.

Fair enough. The article explained that poorly and made it sound as if there is a real contingent of psychologists who think that babies magically know grammar as opposed to learning it.

Given this reframing of the research, I fail to see how this study demonstrates anything, though. If the brain has special structures that allow humans to learn grammar, you would still expect to see the same pattern of "terrible, broken grammar -> mediocre grammar -> good grammar" that we see. We see the same learning pattern for things that the brain does have special structures for, such as... well... everything from sight to motor control.

Like most ticklish philosophical problems, I think it becomes less obvious the deeper in to the problem you look.

There are a number of considerations at play here, and a long history of violently disagreeing about them.

Is language tied to grammar? Or is grammar tied to language[1]? There has been a lot of work seeking to examine the common underpinnings of languages that seem otherwise disparate, and a somewhat flexible core grammatical structure has been proposed as one of these unifying components. The speculative step some take next (including people like Chomsky) is that the reason that all of these common patterns appear is that the brain has evolved some specific hardware that serves as the basis for a set of communication protocols that we call natural languages, and all natural languages instantiate this basic hardware in different ways[2].

If you think this kind of hardwiring is possible, then another obvious-but-not-obvious question follows. Can language really be learned[3]? Can you take a clean slate of neurons (some arbitrary configuration of connections) and, given enough input data, end up with a system that understands language (to frame it as a machine learning problem, which it may or may not be)? Or is it closer to a game of connect-the-dots; the bare framework is there, built into the way the brain arranges itself over the course of millions of years. Input data is just required to figure out exactly what the final arrangement of the picture is, whether it looks like English or Russian or Mandarin.

The current mood in cognitive neuroscience academic circles seems to be the former case, that you can 'simply' throw enough data at a general-purpose learning model and get a language-speaker out. Partly due to the success to deep learning approaches in what were previously very difficult problems in NLP. But I don't think it's fair to say there's nothing worthy of debate here.

---

[1] Or is this a false dichotomy and there's some even weirder relationship between the two.

[2] For a primitive example, languages with SVO ordering vs. SOV ordering. Different instantiations of the same triple with subjects, objects, and verbs. Ray Jackendoff took this another step with formulations like XBar grammar, which was trying to demonstrate how a universal, parameterized grammar might look.

[3] Grice had a fun theory that the observable signal of language is _so_ noisy that it's impossible to learn outright, regardless of the number of examples you're exposed to. And thus, everyone speaks a slightly different language, and some sets of these idiolects are, roughly, mutually compatible. I don't think he ever did any math to demonstrate this though.

Thank you. As a parent of a toddler my first thought was the same as the op's but after reading your post I get why it might require more examination.

Maybe stated in a different way, how good do your priors have to be to learn and understand a language? With an infinite amount of training data, any language can be learned. The question is what prior assumptions are required to learn a language in a reasonable amount of time?

> Grice had a fun theory that the observable signal of language is _so_ noisy that it's impossible to learn outright, regardless of the number of examples you're exposed to. And thus, everyone speaks a slightly different language, and some sets of these idiolects are, roughly, mutually compatible. I don't think he ever did any math to demonstrate this though.

I'm not sure how you'd reduce this to something mathematically demonstrably, but it seems to me that it's the only thing that could be true. The alternatives is that languages are real, distinct, concrete things that exist independent of people, rather than arbitrary lines drawn by people around the continuous and evolving variation in ways people communicate.

So, the only hunch I have is that there might be a kind of cryptographic/information approach to it. If you treat natural language as a noisy encoding of some underlying set of intents (whatever sentience is), then the question becomes "is there enough Information in the signal (natural language) to accurately model the underlying intents statistically?". If the answer is 'no', or 'yes, but very badly', then it seems likely to me that there's some shared hardware/firmware (in this framing, the functional equivalent of a decryption key) between people. If the answer is 'yes', or 'yes, mostly at least', then there's no need for any kind of universal grammar or such, and you can learn language from a blank state. Basically, is natural language something like a one-time pad, or is it more like a breakable form of encryption?

Of course, there are plenty of assumptions in that model (for one, can you even separate 'language' from 'sentience' like an information model would suggest. I'm not so sure you can, but then I've always been somewhat sympathetic to Sapir-Whorf). I'm also not a competent enough mathematician in these fields to see where all the holes in this line of thinking are.

Because you're assuming the grammar is tied to the language, rather than the structure of the human brain. There may be some grammatical primitives that the brain structure supports which enables the grammar of language to function on top of them. This is one of the common linguistic explanations for why you can teach a human to understand grammar, but not a dog, for instance.

Dogs can learn. They just can't learn grammar. Why? Because the human brain is primed with grammatical primitives that a dog brain hasn't got.

Personally, I think that's a bunk explanation, but this conclusion is not obvious either way.

Original researcher here. For context, people are not arguing that nothing is learned, but instead that children come innately prepared with abstract categories of "noun" and "determiner" even if they don't know which words are nouns or determiners or even exactly how these categories are combined.

Here's a paper that makes this argument: http://www.ling.upenn.edu/~ycharles/PNAS-2013-final.pdf

Hey, Michael. Thanks for posting. This is a more understandable claim than the one I read from Stanford News. That was I guess a misunderstanding on my part.

I'll take a look at the paper you linked, but I'm also curious, how does your technique make a distinction between rule-based grammar being fully learned vs mostly or partially learned? That is, how can you say that improved grammar over time is evidence against innate ability to understand/use grammar? It seems we have an innate ability to walk in that we seem to be "wired for it", but we are still terrible at it for a bit over a year on average. The fact that we get noticeably better over time doesn't seem to rule out innate ability. This of course assumes "innate ability" doesn't mean "completely unlearned".

Sorry for the slow response. Our model is - we think - an advance because it actually allows us to infer the amount of shared structure between different words. This is an index of how much generalization the child is doing. If they do no generalization, that's evidence for what's called "item specific learning" (no abstract grammatical rules). If they do complete generalization, that's the innateness hypothesis in the paper I linked. What we found was - of course - a mixture, but critically with something close to item-based learning in the very earliest data from the very youngest kids.

Nothing is obvious. If this is obvious to you then well done - give yourself a little pat on the back and smile to yourself. But in general nothing is obvious and everything must be questioned.

How long is the one side of this thing going to talk past the other?

I am surprised this required research. How is this not obvious ? What's next ? Algebra skills and essay writing? You could pretty much do this research for every thing we ever learn from school to college and publish endless amount of papers.

No this is not obvious. Throughout human history and even today algebra, writing and reading are hard skills for humans which require years of training to master.

Ability to speak however comes very naturally without special training. Human societies independently developed isomorphic grammar. For example Navajo Indians helped American soldiers translate their messages into Navajo and back so Germans wont understand it.

This sort of evidence lead scholars like Chomsky to believe that human brain might have some in-built ability aided by evolution to parse some basic grammatical rules.

> Ability to speak however comes very naturally without special training.

It literally takes years of practice for children to speak well. Maybe it's natural but we tend to dismiss the difficulty because it happens early.

I would think if we had dedicated hardware for grammar subsequent languages wouldn't be so difficult to learn either. It's easier to learn algebra than a second language.

It is believed that second language acquisition is "harder" than first language for a number of reasons, such as the placidity of the brain crystallising as we age and, potentially, that any special hardware we may have has a finite capacity (because physics). People research this (my wife is doing her PhD on it currently) and the evidence is neither obvious nor conclusive.

Children learn language without instruction. That is all. Clearly, there is innate linguistic capability in humans.

Without instruction? They are watching people speak for years. They even have people "teach" it to them. Can you say, "Daddy"? While slowing down and enunciating.

We are unfortunately very aware that when isolated, kids don't learn to speak. So it isn't innate.

Children learn to build with magnatiles without instruction. That is all. Clearly, there is innate magnatile building capability in humans.

My granddaughter (1y 8m) uses flossers without instruction. Clearly, there is innate tooth hygiene in humans.

No, what's happening is she's mimicking the adults in the house. She's observing and repeating. She's done the same thing with dolls: put them on her shoulder, pat them, make shushing noises ... because we do that with her; because she see's others do that with other babies.

"Instruction"? No. But the only "innate" behaviors this child has involve eating and curiosity. Everything else is spurred by that innate curiosity.

Then why teach grammar at all if we know kids pick it up naturally? There seems to be a threshold beyond grammer needs to be taught? Or was the hypothesis that there is no such thing.

There's a difference between formal and conversational grammar

you dont need to learn "grammar" in school/lessons to know how to speak a language.

Language planning/standardization - ensuring prevalence of standard varieties of a language, suppressing non-standard varieties or at least letting people understand how and why they're non-standard

(David Foster Wallace wrote a long piece about this in 2001 called "Tense Present" http://harpers.org/wp-content/uploads/HarpersMagazine-2001-0... and there's also a long story about nationalist movements trying to design, and teach, a standard prestige version of their national languages, which does require grammar instruction to create and maintain because it's somewhat different from all or most students' home language varieties)

Written vs. spoken language - may have different grammatical norms

Non-native speakers - want explicit instruction because they lack native-speaker intuition about some aspects of grammar

Appreciating language intellectually - curiosity and appreciation about how your language works, independent of your ability to produce sentences other speakers accept as correct

In a way, it's the deepest and most perennial controversy in linguistics.

https://en.wikipedia.org/wiki/Universal_grammar

https://en.wikipedia.org/wiki/Poverty_of_the_stimulus

But it's hard to present the context of the argument when describing an individual research paper, so maybe the research summary ends up sounding at first like "when they're little, children learn how to speak their languages!", which definitely sounds like "wait, why does someone even bother publishing a paper about that?".

Learning has to start from some prior or inductive bias. The question is how much knowledge of how language works is built into childrens' innate priors. Chomskyans have typically argued that the innate prior ("universal grammar") gets you almost all the way to understanding language, such that you only have to set a few parameters to get to fluent understanding of a real language.

Thanks, this makes the most sense to me. The problem more generally stated then for more fields is that how much abstraction do we carry innately and how much must be taught. Applies to math as well. It appears to me that ability to count is innate. But beyond that I don't know what else. Would be great to find out.

I'd disagree that "counting" is innate. The mental recognition of scale and quantity may be innate, but "counting" seems to be a more elaborate matter of embedding scale and quantity recognition into an abstract mental or linguistic framework.

That is, to count you need to understand number, but to make decisions about scale and quantity, you don't.

Just today I heard a little girl crying to her father (in romanian): "You are giving!" when she actually meant (by the situation and raised hands) "Give me the toy!".

God save us from most decisions made because something is obvious (to someone).

Reply on news.ycombinator.com