> if we transcribe your brain into a simulation and give it a tickrate, you will be just autocomplete too.
The underlying mechanisms for producing language are very different, in the same way birds and airplanes have different mechanisms for flying.
The underlying mechanisms for producing language are very different, in the same way birds and airplanes have different mechanisms for flying.
LLMs are more like human minds than we are willing to admit. Such reluctance is perhaps the least surprising aspect of any of this.
I disagree, language is the product of, not the mechanism for thought. People who lose their faculty of speech (or haven't gained them) have complete thoughts and executive function.
CoT is a hack to use language (i.e. autoregression) to simulate reasoning, and it's very effective at it. Human minds can acquire, hold and use axioms as building blocks for actual reasoning, LLMs use statistical likelihood.
This is not even slightly true. Even when trying, humans commit logical mistakes at notable rates, because the behavior of our minds is inherently nondeterministic. Human minds are not built for logic and need to twist themselves into knots and rely on symbolic representations to do it. There is such a thing as valid reasoning - predicate calculus, decision theory - and humans only emulate it with some accuracy, in hacky ways, and only because our brains were imperfectly taught to do that over millenia of evolutionary pressure. LLMs are much the same way.
> humans commit logical mistakes at notable rates
But then again, maybe to support your point people frequently say "start from first principles" when those are usually the thing that needs to be found, not the place you start. But LLMs, like humans, love to be confident about things they aren't sufficiently trained on
Equally, even if your axioms are right your logical execution can still fail.
Neither individual humans nor LLMs get either of those steps right as a default. Just as LLMs must use hacky CoT and other systems of error checking and correcting to even begin to emulate reasoning, humans must do the same thing.
We are apes. We are driven by emotion and solve problems either when we are forced to or as a sport. We did create the ultimate automata of logical execution: the deterministic von Neumann Machine, but a human brain in no way reflects that operation. Nor, as you have observed, does an LLM despite using one as its substrate.
langauge is only produced by the final layer every tick. and every layer can only interact with other layers on the same level.
the mechanics of LLMs and the restriction in how we can train them makes it appear as though all we are doing is forcing language onto them but once RL gets involved all bets are off regarding what's happening inside them (it's quite possible that a static corpus alone is sufficient for all the bets being off).
Which is actually an important part of the Navier-Stokes conversation. Solving hard problems expands the vocabulary. The problems are hard, illustrating a region we know where the language is insufficient. So the point isn't so much to solve the specific problem, but to figure out how to discuss problems like it.
But there's a big difference between talking about something in an extremely convoluted manner then people struggle to understand and inventing a new word that simplifies our discussions.
Though this is grossly oversimplified. It's a HN comment, not a lecture on metamathematics or metaml
Human egos are probably the reason we also resisted the concept of heliocentrism.