Hacker News Reader

There are new research where chain of thoughts is happening in latent spaces and not in English. They demonstrated better results since language is not as expressive as those concepts that can be represented in the layers before decoder. I wonder if o3 is doing that?

View on HN

"You can tell the RL is done properly when the models cease to speak English in their chain of thought" -- Karpathy

I think you mean this: https://arxiv.org/abs/2412.06769

From what I can see, presuming o3 is a progression of o1 and has good level of accountabiltiy bubbling up during 'inference' (i.e. "Thinking about ___") then I'd say it's just using up millions of old-school tokens (the 44 million tokens that are referenced). So not latent thinking per se.

Interesting!

Reply on news.ycombinator.com