Neuroscientists find new support for Chomsky’s “internal grammar” thesis

I think it's a quite nice analogy. There are structures present, but they have to be trained as well. Suppose you have a hierarchical Dirichlet process as prior, you still need data to be able to define what clusters with what. And of course I won't postulate such a simple process as representing our newborn brains.

From Wikipedia: "At birth, visual structures are fully present yet immature in their potentials. From the first moment of life, there are a few innate components of an infant's visual system. Newborns can detect changes in brightness, distinguish between stationary and kinetic objects, as well as follow kinetic objects in their visual fields. However, many of these areas are very poorly developed."

Your extreme case would be that there is a priori no reason to have language processed in certain parts of the brain, like Broca's or Wernicke's regions.

It is of course possible that these are just strategically located. However, newborns with damage to these regions by oxygen deprevation at birth can already suffer from linguistic impairments.

The idea that some complex systems can simply grow without inputs is not unfounded in the biological world -- witness, for example, grazing animals such as cows and giraffes that plop out of the birth canal, shakily stand up, and being trotting around. They couldn't have trained that system in the womb.

Similarly, human babies who are growing complex neural systems over years might look like they are learning, but instead they are just growing. That's not to say that we don't learn things later on-- certainly we do-- but that's not evidence that babies are necessarily learning all the skills they become capable of in toddlerhood.

I think what the comment you're responding to meant is that this study shows no evidence for an innate internal language or grammar, which I thought was Chomsky's most controversial claim.

Grammar in most human languages is structured to be information-theoretically efficient, which often winds up being vague, ambiguous, and context-dependent. It's basically a time-space trade-off for statistical inference.

That was kinda my impression as well, but I don't want to say much more as I'm so far from an expert on this and I'll probably just make an idiot out of myself. Still, as you say, it is interesting work in its own right.

Because in English word order is important (in fact, in most languages)

It would be fun to learn a completely independent word order language (I suspect Latin is like that) but I suspect this is one of the reasons this aspect died in romance languages, it's harder to understand and relying on word order makes it easier

I actually had to read the sentence about 3 times, and force myself to read it slowly word by word, before I spotted it.

We often don't read one word at a time.

I agree with you that "massively parallel" is probably drastically overstating it, though.

For fun, I once had two friends speak to me simultaneously, one in each ear. We swapped places and did that to each other a few times. Result: nope, our language understanding unit is not parallel. You can either take in a wall of sound, OR focus to a string of words and understand its meaning. It might feel like you can almost do both at the same time and pick up two different threads of conversation at the same time, but it's always just beyond your reach. At best you can swap between threads quickly, if the conversations permit it (so forget it both/all conversations are rapid-fire and heavy with nuance etc).

I also note that our language generation units are not parallel. We can utter one syllable at a time. So maybe there's not much point in understanding multiple syllables at a time?

I'd say because the probabilistic structure of language is such that it must be processed sequentially to be decoded correctly. The interpretation of what comes next depends on what came before. Information-theoretically, this happens to be an efficient and compact way of encoding information in the auditory medium (where input unfolds over time, rather than being presented all at once like in images).

Whether processing is "massively parallel" depends on what level of analysis you're assuming. At some level, processing in the brain is obviously massively, since that's simply how the brain's laid out.

I don't see how sequential processing bears any implication to whether or not language is or can be learned through statistical pattern matching, but I'd love to hear your angle on this.

The opportunity for massive-parallelness is in disambiguating a huge number of possible interpretations. Before you can construct the possible candidates, you first need to have processed enough of the sentence. But since you can process new sensory input at the same time as disambiguating previous input, you can get the garden path effect. What I find interesting about the garden path effect though is that it's so artificial, so the way language is used normally seems to purposely avoid ambiguities that will trip up listeners/readers.

Consider this text: "Well the whole point of it was... I mean, when I was in school, we often-- sometimes we would go hunting after school, and I mean, you know -- look, nothing like this ever-- we could never have imagined it."

That's 5 or 6 sentence fragments, one after another. If you processed it in a naively serial way, you just end up with one big ungrammatical sentence. I'm only able to clarify it with punctuation (mdashes and commas) because I could parse it correctly.

A parallel system would benefit not from parsing such a string, but creating a set of candidate interpretations, (all the words together, this is a series of three phrases, this is a series of five phrases, etc) and determining the most meaningful match.

I don't read one word at a time, but scan larger chunks. For instance, I processed "if it's so massively parallel" simultaneously as one "picture".

Do you really read one word at a time? Try making a sheet of paper with a cutout that is approximately word-sized, and move that peephole over the text. Then you're reading one word at a time.

If that makes no difference to your reading efficiency, you could have some cognitive disability affecting reading.

I wonder if the serial limitation is purely historical, or if our intelligence system is somehow fundamentally unable to process massively parallel transmission, even if such has been available during our evolution?

Certainly, we recall associations in a seemingly massively parallel fashion, but our conscious thought is apparently serial. We think this, then that... wait, isn't this really that? There may be non-deterministic threads going in the background, but they are more about association or "fit" than working something out.

We also model things narratively, a serial representation which is also a nice match for events over time - also serial.

BTW: didn't spot the error at first. But I've seen typos in books on second or later readings, which (evidently) also eluded the author and copy editor.

However, that is very different from the quoted passage. It sounds like these "neuroscientists and psychologists" don't think our brain has any notion of "grammar" at all, and everything can be explained better with statistical relations. I find that very hard to accept, and I wonder if a more reasonable position (= there is no special part of brain reserved for learning grammar) was summarized poorly into a much stronger hypothesis.

Well, of course, if we reduce everything as much as we like, then we could say everything is about statistical relation of a gazillion variables (or quantum mechanics, if you go far enough). But that's not useful, isn't it? Even if we cannot pinpoint a neuron and say "This neuron will fire if it sees a past perfect!", if the brain as a whole acts like it has an underlying notion of grammar, then I think it's fair to say that it "knows" grammar.

And, forgive me if I'm wrong, but I'm pretty sure that my brain acts like it has some notion of English grammar. :P

Edit: Hmm, I think I caused confusion by using the word "innate". Apparently it can mean (1) existing from the time a person or animal is born, and (2) existing as part of the basic nature of something.

Chomsky proposes (1), and many disagrees. (I don't find Chomsky's arguments particularly convincing, either.) But the sentence I quoted (which didn't use the word "innate") sounds like it was against (2).

I always think that "human is born with some miracle thing" is quite suspicious. How can orderliness be born out of no where (no work spent, no energy consumed)? The probability of orderliness to be spontaneously assembled itself must be infinitesimally small.

Grammar might evolve along the way (due to statistical learning) but it seems too good to be true if it's born out of no where.

If it's not about innateness but about the claims of "internal grammar" from the press release then it because less clear that there would be many people who would reject such a position. I think this is a misleading on the part of the press release. Whether something was innate or learned, I think most people would agree that knowledge of language is internalized. So then the question remains whether you call it grammar or something else, and whether statistics plays a part should have no bearing on that if you ask me.

Google "Stanford parser". This is a state of the art natural language parser and like all parsers it relies on a grammar. The grammar it uses is a dependency grammar, a different grammar than the phrase structure grammars proposed by Chomsky and also one that is built in a probabilistic manner, but a grammar nonetheless.

Also, it's really not the case that you get better performance out of neural network models of language, let alone "human like performance". We're very far from that still.

No not really. Barring concrete physiological evidence, innateness is simply an unfalsifiable position. I do think you can say that consensus among scientists might have shifted against innateness, but that is definitely not because of some computational model such as RNN with some passing resemblance to observed behavior of human neurons. The way these networks are configured and trained has nothing to do with real brains, nor does the abstraction in neural nets leave room for any electrochemical effects.

> The interesting question is why RNNs are able to produce things that resemble formal languages at first pass.

Is it? A neural network can achieve that by simulating a Turing machine or simpler automaton. I don't see what that brings to the table? That's just doing something we could already do, but less efficiently. I think the interesting thing is the opposite: if the RNN can do something which couldn't be done before with other models.