Hacker News Reader

Comment by gizmo686 | Hacker News Reader

No. Innate grammar has always been about how humans aquire language, not how any possible system which understands human language must posses that innate grammar.

View on HN

Trying to put his in an uncontroversial way: the human brain (or a brain plus paper and a pencil) can be turning complete/equivalent. Therefor a human sitting down with a pen and pencil could, in a painstakingly long time, compute the backwards and forward passes of a transformer network.

Therefor a human with no understanding of grammar/language, and using no innate biological circuits, could process grammar and respond with language.

The flaw in this argument would be how to teach a human to do this without grammar ...

But that has never been proven that this is how indeed human acquire language; it is essentially a hypothesis. We may as well do it the way LLMs do - some undifferentiated networks acquires the grammar by unknown means.

LLMs are universal approximators and can pick up patterns in sequences that are very different from Human languages. Sure, they don't have many inductive biases and can understand language, but as a consequence require a tremendous amount of data to work. Humans don't, which implies a certain bias towards Human language built into our heads. A bias is also implied by the similarities across Human languages, though what structure(s) in the brain are responsible is not exactly clear.

It still does not proof anything, as claiming that "there is certain bias for Human Language built into our heads" is quite different thing that saying there is some universal grammar in the brain structures, as much we do not have innate abilities to comprehend calculus or play chess, yet we still able to learn it, with a lot less training information than LLMs. In fact 2 books will suffice for the both.

My comment was more of a response to

> We may as well do it the way LLMs do

We almost certainly don't learn the way LLMs do, it's just too data inefficient.

And I don't see what current LLMs can say about a universal grammar in the Human brain, unless there is proof that a LLM-style attention mechanism exists in the brain, and that it is somehow related to language understanding.

We don’t learn language from textbooks though.

Chomsky has explicitly answered this: Moro has shown in experiments that humans do not appear to be able to learn arbitrary grammatical structures in the same way as human-like (hierarchical) languages. Non-human like languages take longer to interpret and use non-language parts of the brain.

LLMs on the other hand can easily learn these non-human grammatical structures which means that they are not the way humans do it.

Reply on news.ycombinator.com