A lot of people dismiss LLMs based on how ridiculously simple next token prediction is.
Human nerve cells look ridiculously simple relative to what a clump of them can do. So does a transistor that CPUs are built out of.
Human nerve cells look ridiculously simple relative to what a clump of them can do. So does a transistor that CPUs are built out of.
If you were to look at human brain cells without ever having seen a functioning human brain built from one, you might well question if they'd be enough too.
What do "truly" and "understand" mean here, when LLMs are known for making errors and generating semantically correct but contextually meaningless results? Do they truly understand everything in the universe, or is the appearance of understanding the result of that universe being composed of data created by humans, so that relevant and meaningful results are statistically more likely than not?
A hand looks simple to a child. to a doctor? orthopedic surgeon?
worse, to an untrained eye, your response looks like it is from an llm. it would explain missing the nuance. not the only explanation by far, but when combined with talk of genetic effects in the footnote…
Out of all that complex biology in the brain maybe just a small part of it is contributing to intelligent behavior. Most of it is for support or redundancy. So I don't care you can equal a neuron with a 3 layer MLP with 1000 neurons or more. A model that fits on a DVD can speak fluent language, write code, and solve tasks - that is the useful capacity we are looking for.
That's not what the OP or the literal quote you pulled from them said. They said a single neuron is ridiculously simple _relative_ to what a clump of them can do.