We can possibly say math is not learned, but a mental models of abstractions are developed. How? We dunno, but what we do know is we don’t learn by figuring the common features between all previously seen equations only to guess them later…
Mind operates on higher and higher levels of abstractions building on each other in a much fascinating way, very often not with words, but with structure and images.
Of course there are people with aphantasia, but i really fail to see how any reasoning happens in purely language level. Someone on this forum also noted - in order to reason one needs an ontology to facilitate the reasoning process. LLMs don’t do ontologies…
And finally, not least though, LLM and ML people in general seem to equate intuition to some sort biased.random(). Well intuition is not random, and is hard to describe in words. So are awe and inspiration. And these ARE part of (precondition to, fuel for) humanity’s thought process more that we like to admit.
Chain of thought is like trying to improve JPG quality by re-compressing it several times. If it's not there it's not there.
There is nothing in the LLM that would have the capability to create new information by reasoning, when the existing information does not already include what we need.
There is logic and useful thought in the comment, but you choose not to see it because you disagree with the conclusion. That is not useful.
Those people sure showed us, didn't they? Ah, but "it's different this time!".
>It's not thinking
>it compressed the internet into a clever, lossy format with nice interface and it retrieves stuff from there.
>Chain of thought is like trying to improve JPG quality by re-compressing it several times. If it's not there it's not there.
There is more data to add confidently, it just has to re-think about it with a renewed perspective, and an abstracted-away higher-level context/attention mechanism.
Empirically speaking, I have a set of evals with an objective pass/fail result and a prompt. I'm doing codegen, so I'm using syntax linting, tests passing, etc. to determine success. With chain-of-thought included in the prompting, the evals pass at a significantly higher rate. A lot of research has been done demonstrating the same in various domains.
If chain-of-thought can't improve quality, how do you explain the empirical results which appear to contradict you?
It's literally in the second line of the abstract: "While CoT has been shown to improve performance across many tasks..."
A lot of CoT to me is just slowing the LLM down and keeping it from making that premature conclusion... but it can backfire when it then accidentally makes a conclusion early on, often in a worse context than it would use without the CoT.
Natively, I understand these to influence the probability space enough to weaken the emergence patterns we frequently overestimate.
Every model has the model before it, and it's academic papers, in it's training data.
Changing the qualifiers pulls the inference far away from quoting over-trained data, and back to generalization.
I am sure it has picked up on this mesa-optimization along the way, especially if I can summarize it.
Wonder why it hasn't been more generally intelligent, yet.
I'll rank those three fruits from largest to smallest:
1. Grapefruit 2. Orange 3. Blueberry
The grapefruit is definitely the largest of these three fruits - they're typically around 4-6 inches in diameter. Oranges are usually 2-3 inches in diameter, and blueberries are the smallest at roughly 0.5 inches in diameter.