Wait, is this true? There are people who seriously suggest that we don't have an innate notion of grammar?
The article doesn't actually provide any support for the "innate" or "universal grammar" hypothesis as it is typically understood, though. This is really about whether the de-facto grammar that emerges from whatever the "language faculty" has a particular hierarchical and recursive structure. And yes, there are people who seriously argue against that, too. The most common alternative model that I have been exposed to is analogical template matching, which actually does a pretty good job at a lot of NLP tasks.
However, that is very different from the quoted passage. It sounds like these "neuroscientists and psychologists" don't think our brain has any notion of "grammar" at all, and everything can be explained better with statistical relations. I find that very hard to accept, and I wonder if a more reasonable position (= there is no special part of brain reserved for learning grammar) was summarized poorly into a much stronger hypothesis.
Well, of course, if we reduce everything as much as we like, then we could say everything is about statistical relation of a gazillion variables (or quantum mechanics, if you go far enough). But that's not useful, isn't it? Even if we cannot pinpoint a neuron and say "This neuron will fire if it sees a past perfect!", if the brain as a whole acts like it has an underlying notion of grammar, then I think it's fair to say that it "knows" grammar.
And, forgive me if I'm wrong, but I'm pretty sure that my brain acts like it has some notion of English grammar. :P
Edit: Hmm, I think I caused confusion by using the word "innate". Apparently it can mean (1) existing from the time a person or animal is born, and (2) existing as part of the basic nature of something.
Chomsky proposes (1), and many disagrees. (I don't find Chomsky's arguments particularly convincing, either.) But the sentence I quoted (which didn't use the word "innate") sounds like it was against (2).
Does your brain "know" the formulas that describe projectile motion? Is it running a physics model in your brain?
As for throwing the basketball, most people know (#) what the arc should be, and can immediately after the throw tell if a throw will hit or miss with reasonable accuracy, assuming a viewpoint that allows them to extract the movement accurately. The difficulty there is learning to execute that throw.
(#) edit, know as in "have learned", not as an innate ability.
Even if you consider spins, throwing a ball is only about 9 degrees of freedom. It is too small to make good prediction on how the brain works, so Occam's razor dictates that it probably doesn't solve a differential equation. (Also I could try throwing a baseball, and I'll find that I have to re-learn everything, so it's reasonable to assume that my brain was "optimized" to just throw one particular kind of balls to a particular height.)
Human language is so mind-bogglingly complex that even if you just restrict yourself to a dozen words you can easily utter sentences that were never spoken in entire history of English. Yet my brain has no problem dealing with that.
Sure, we could say that it's all statistical relation, but those relations have to stack (almost) recursively and connect phrases that are a dozen words apart. At what point do we stop calling them a heap of relations and instead call it what it is, i.e., grammar?
My point was that Newton's formulas are simplified models of how the world works. As a result, just because you know how to throw a ball doesn't mean you "know" Newton's formulas. Similarly, just because you know how to speak doesn't mean you "know" grammar.
The question to ask to help us answer this question would be: is grammar a simplified model of language?[1]
[1] This depends on whether you're a linguistic prescriptivist or descriptivist.
Using language, on the other hand, requires us to really think.
(Any errors in this comment due to the fact that I wrote it without thinking.)
Being in a flow state seems very similar to being in the zone.
Not consciously, no. I mean, do you really think about what you say when you speak in your native language, or do you just say it? It's actually a milestone in learning a foreign language - switching from thinking in native and translating into foreign to just thinking in foreign. If you find yourself running your internal monologue in a foreign language then you know your brain groks it.
And generally, mastering a task means switching it to unconscious mode. A good driver doesn't consciously think about turning the steering wheels or pressing pedals; they think about car (or rather, car+themselves system) moving faster or slower, going here or there. A soldier is trained until most of his tasks are muscle memory. A programmer doing what they really knows how to do will find themselves in the state of flow, etc. Mastery of a task is mentally abstracting it into just another basic capacity, just like breathing or moving your hands.
I'm not sure I agree, not entirely at least. "Thinking while speaking" is something I associate to speaking a second language that you don't really master. Then you really think in one language, think about the translation, and speak. If I speak in my native language or in English, I don't think about the words I'm saying or about how I'm using the language, there's a much bigger component of instinct and thoughts can be much more impulsive.
I always think that "human is born with some miracle thing" is quite suspicious. How can orderliness be born out of no where (no work spent, no energy consumed)? The probability of orderliness to be spontaneously assembled itself must be infinitesimally small.
Grammar might evolve along the way (due to statistical learning) but it seems too good to be true if it's born out of no where.
The interesting question is why RNNs are able to produce things that resemble formal languages at first pass.
Google "Stanford parser". This is a state of the art natural language parser and like all parsers it relies on a grammar. The grammar it uses is a dependency grammar, a different grammar than the phrase structure grammars proposed by Chomsky and also one that is built in a probabilistic manner, but a grammar nonetheless.
Also, it's really not the case that you get better performance out of neural network models of language, let alone "human like performance". We're very far from that still.
The base of the parser relies on a grammar to describe possible sentence structures, but actually most of the work in disambiguation is done using statistics, or with neural networks. The question what is and what is not grammar is rather arbitrary, there can be a continuum between simple rule-like and statistical regularities.
Ultimately the models in NLP are rarely making any kind of cognitive/neuroscientific claim about being plausible models, just effective ones. There's a specific field of computational psycholinguistics which does investigate those things.
> Also, it's really not the case that you get better performance out of neural network models of language, let alone "human like performance". We're very far from that still.
I wouldn't make such claims these days because deep learning methods are gaining ground very fast. There are many tasks at which the deep learning model is better than traditional models. Furthermore, there are already deep learning models which are better than humans at image labeling.
We've been here before. Back in the Olden Days of GOFAI, expert systems used to routinely outperform human experts at all sorts of cognitive tasks (medical diagnosis being a typical example). There was a huge amount of excitement and people promising wild things were just around the corner. A few years down the line, there's the same excitement around a completely different technology and expert systems are nowhere to be seen. So I'll keep my expectations at about mid-range and wait for another ten years before I say I know exactly what's going on.
No not really. Barring concrete physiological evidence, innateness is simply an unfalsifiable position. I do think you can say that consensus among scientists might have shifted against innateness, but that is definitely not because of some computational model such as RNN with some passing resemblance to observed behavior of human neurons. The way these networks are configured and trained has nothing to do with real brains, nor does the abstraction in neural nets leave room for any electrochemical effects.
> The interesting question is why RNNs are able to produce things that resemble formal languages at first pass.
Is it? A neural network can achieve that by simulating a Turing machine or simpler automaton. I don't see what that brings to the table? That's just doing something we could already do, but less efficiently. I think the interesting thing is the opposite: if the RNN can do something which couldn't be done before with other models.
That, what you say- that's the business. An RNN (any ANN) is not a human brain. It is one model of the human brain, based on our very limited understanding of human brains. It doesn't matter what ANNs can and cannot do, how well or bad they perform, it tells us nothing much about how we do language; or vision, motion, decision-making, anything.
No it isn't. It's loosely inspired, but the goal is to make something that works, not to simulate what we know of the human brain. There's a different field, computational neuroscience, which does try to make faithful models of the human brain.
Hell, you can communicate pretty well by just dropping nouns in sequence. That we have no problems understanding information presented in such way, or maths, or music notation, or programming languages, suggests that we don't have "internal grammars".
Edit: by "innate", people mean that it's something you're born with, rather than learned through experience.
If it's not about innateness but about the claims of "internal grammar" from the press release then it because less clear that there would be many people who would reject such a position. I think this is a misleading on the part of the press release. Whether something was innate or learned, I think most people would agree that knowledge of language is internalized. So then the question remains whether you call it grammar or something else, and whether statistics plays a part should have no bearing on that if you ask me.