The interesting question is why RNNs are able to produce things that resemble formal languages at first pass.
Google "Stanford parser". This is a state of the art natural language parser and like all parsers it relies on a grammar. The grammar it uses is a dependency grammar, a different grammar than the phrase structure grammars proposed by Chomsky and also one that is built in a probabilistic manner, but a grammar nonetheless.
Also, it's really not the case that you get better performance out of neural network models of language, let alone "human like performance". We're very far from that still.
The base of the parser relies on a grammar to describe possible sentence structures, but actually most of the work in disambiguation is done using statistics, or with neural networks. The question what is and what is not grammar is rather arbitrary, there can be a continuum between simple rule-like and statistical regularities.
Ultimately the models in NLP are rarely making any kind of cognitive/neuroscientific claim about being plausible models, just effective ones. There's a specific field of computational psycholinguistics which does investigate those things.
> Also, it's really not the case that you get better performance out of neural network models of language, let alone "human like performance". We're very far from that still.
I wouldn't make such claims these days because deep learning methods are gaining ground very fast. There are many tasks at which the deep learning model is better than traditional models. Furthermore, there are already deep learning models which are better than humans at image labeling.
We've been here before. Back in the Olden Days of GOFAI, expert systems used to routinely outperform human experts at all sorts of cognitive tasks (medical diagnosis being a typical example). There was a huge amount of excitement and people promising wild things were just around the corner. A few years down the line, there's the same excitement around a completely different technology and expert systems are nowhere to be seen. So I'll keep my expectations at about mid-range and wait for another ten years before I say I know exactly what's going on.
No not really. Barring concrete physiological evidence, innateness is simply an unfalsifiable position. I do think you can say that consensus among scientists might have shifted against innateness, but that is definitely not because of some computational model such as RNN with some passing resemblance to observed behavior of human neurons. The way these networks are configured and trained has nothing to do with real brains, nor does the abstraction in neural nets leave room for any electrochemical effects.
> The interesting question is why RNNs are able to produce things that resemble formal languages at first pass.
Is it? A neural network can achieve that by simulating a Turing machine or simpler automaton. I don't see what that brings to the table? That's just doing something we could already do, but less efficiently. I think the interesting thing is the opposite: if the RNN can do something which couldn't be done before with other models.
That, what you say- that's the business. An RNN (any ANN) is not a human brain. It is one model of the human brain, based on our very limited understanding of human brains. It doesn't matter what ANNs can and cannot do, how well or bad they perform, it tells us nothing much about how we do language; or vision, motion, decision-making, anything.
No it isn't. It's loosely inspired, but the goal is to make something that works, not to simulate what we know of the human brain. There's a different field, computational neuroscience, which does try to make faithful models of the human brain.