Hacker News Reader

Comment by samrus | Hacker News Reader

I like the idea of more reccurance in the transformer level. Chain of thought always seemed so clunky. Its just not the way the human brain processes information. Its an extrmeely crude approximation at best

View on HN

it is what I do to solve hard problems through.

easy stuff happens by itself, but with a system large enough you need a scratchpad and a rubber duck.

One thing about the reasoning is that models are trained to generate a chain of thoughts, but it doesn’t have to be correct, accurate, or reflect the underlying logic of the LLM. It’s the same problem we have with the output, it is something plausible, but not that reliable

I do the same thing in my head. There is no underlying logic to an llm. Logic is an external construct alien to human like forms of reasoning.

>but it doesn’t have to be correct, accurate, or ...

why we don't do GAN here, ie. second model verifying correctness/accuracy/etc. ?

Yes, both the output should be "milestones" of sorts, like lemmas and theorems in math. Important plateaus that serve as a launching pad to the next phase. Regurgitating every thought potentially degrades signal:noise ratio.

The hidden states of the tokens likely contain more semantic information than can be extracted by the final projection into token space.

Actually, how does chain of thought work? Is the LLM actually creating the tokens and then re-reading them, or is the there still a full hidden state under the hood and then the UI just prints that projection?

Can't tell you what's happening in a closed model, but in case of open ones it's just a text stream, same "take all previous tokens, compute next token" mechanism applies. Thinking vs Response is just a state change like between a system message and a user message.

Closed models probably do the same thing internally. What is shown externally is different though: you get a summary of the chain of thought, not the thoughts itself. This is done to prevent distillation.

The latest look we had at a frontier chain of thought is probably in the Huggingface incident report - I haven't actually read it yet but I saw the BlackHat talk, and it included some snippets. The thoughts look like they are approaching neuralese. The words are still understandable but the grammar is weird, simplified. In comparison, Qwen 3.8 27b thinks in valid English.

Right but it's meant to be smarter than you

Chain of thought seems very specifically like an instantiation of the System 2 thinking of the System 1 and 2 thinking Daniel Kahneman popularized.

Is it clunky in that it's a verbalized/languified version of system 2 thinking, and clearly humans do some non-verbal version too?

Some humans apparently only or almost only do non-verbal thinking. It’s called anendophasia.

Sounds like philosophical zombies.

https://en.wikipedia.org/wiki/Philosophical_zombie

Maybe, I wouldn’t know. I just find it fascinating that other people have such a radically different way of experiencing thought. No idea what it’s like.

“For example, if a philosophical zombie were poked with a sharp object, it would not feel any pain, but it would react exactly the way any conscious human would”

Doing non-verbal thinking has nothing to do with not feeling pain, I don’t think it sounds like philosophical zombies at all.

I am one of those "humans", allegedly. ;)

How do you deal with things like mental math or "thinking something through"?

Do you find this study to be accurate or inaccurate in any ways?

https://pubmed.ncbi.nlm.nih.gov/38728320/

Pdf seems available here: https://www.semanticscholar.org/paper/Not-Everybody-Has-an-I...

What I'd like to see is "brain"-machine interface for LLMs that plugs things like calculator and other modules, directly into the neural network instead of accessing them through in-context tool calls.

Why? I talk in my head and then enunciate only that which is relevant. My speech rate inside is incredibly fast.

The purpose of the scratch pad is very often to refer back to it later.

Reply on news.ycombinator.com