that sounds testable - if you skip the reasoning tokens, do you get the same result?
if not, then there's certainly some bearing, but not necessarily in how we read the tokens as text
if not, then there's certainly some bearing, but not necessarily in how we read the tokens as text
This points to a hypothesis that the content of the tokens is only slightly related to the mechanism by which it improves performance, and that primarily the extra tokens allow the original prompt to be processed more deeply by the model, because earlier tokens will essentially pass through the model many more times than later ones.