Large language models are poor theories of human linguistic cognition

> One central factor behind this failure is the learning method: LLMs are trained using backpropagation, which pushes the network in very un-scientist-like directions and prevents it from generalizing and reasoning about inputs in anything even remotely similar to how humans generalize.

It's hard to take this backpropogation dis as fact given the performance and progression of AlphaGo, AlphaGoZero, AlphaZero, MuZero.

> When instead of backpropagation we trained networks using Minimum Description Length (MDL) — a learning criterion that does correspond to rational scientific reasoning — the networks were able to find perfect solutions to patterns that remained outside the reach of networks trained with backpropagation. Could there eventually be future LLMs (MDL-based or otherwise) that would strike us as scientist-like? Perhaps. But if such models do arrive they will be very different from current LLMs. In the meantime, any suggestion that LLMs are automated scientists should be treated with suspicion.

Seems like the critique is specifically about how current LLMs operate and not about them in principle.

Sounds a bit like Chomsky’s critique of Skinners “Verbal Behavior”, which kind of put him on the map back in the 60s. Though Skinner gave us teaching machines, ping pong playing pigeons, clicker training and, indirectly, ChatGPT.

The author assumes that asking GPT to “take longer and reconsider” is something that an LLM actually does.

As in, temporally.

The author claims that “even after taking longer to reconsider” an answer, ChatGPT was still wrong.

The author appears to misunderstand how LLMs work.

"The distinction between likely and grammatical, which all humans have, is entirely foreign to ChatGPT and its fellow LLMs."

Because grammar is not determined by frequency.