Hacker News Reader

Comment by code51 | Hacker News Reader

Why on earth is nobody here talking about the sudden jump to use von Mangoldt function?

The reasoning trace never types Λ, never types "von Mangoldt", and never invokes ∑_{q|n} Λ(q) = log n.

There is a clear discontinuity at play. I remember an article on this, maybe a comment by Terence Tao himself, seen here, but cannot find it.

View on HN

During training they gate with a lot of guardrails the format of the reasoning tokens output. They don't just use a reward for getting the correct answer during training but also reward human readable output. That said, if they didn't, the reasoning tokens that are the most efficient to get to the final correct answer during training would most likely look like a lot of gibberish.

There is a relationship between the tokens in the output in the model's vector space, that is the most important, and something hidden we will never see.

I think that the thought trace is definitely incomplete - you can see cases where it is like and "let's calculate the integral:[no integral calculated]". The train of thought it's on towards the end of the trace looks like an entirely different approach than what it ends up returning, so I think we are just not seeing the part where it hits on the right approach (sadly).

Does DeepSeek's solution look more traceable?

https://chat.deepseek.com/share/nyuz0vvy2unfbb97fv

Thought traces are indeed not an accurate representation of what models actually do. If you ask an AI model to add two values it will do so, then in the next prompt ask it to explain the algorithm it used, it will regurgitate that it used some standard textbook method, whilst in reality it used a completely different algorithm. Thinking LLMs don't record the neural pathways they used.

Reply on news.ycombinator.com