I'm guessing this works better because it can always go back and re-analyze the saved context.
Our docs show a diagram here:
https://developers.openai.com/api/docs/guides/reasoning
> Input and output tokens from each step are carried over, while reasoning tokens are discarded.
Keeping reasoning tokens around is better for caching and for remembering past insights, so you might reasonably wonder why we designed it this way. The main benefit of dropping reasoning tokens is that you can fit a lot more work inside the model's context window before you're forced into a slow and lossy compaction step. This was a larger consideration with our earlier reasoning models that had shorter context windows (~200k), longer thinking times (up to ~100k per message), and poor compaction. However, now that we've shipped longer context windows, we've trained our models think much more efficiently, and we've made compaction way better than it used to be, the balance of factors is changing. Tune in Thursday!
This is something I never understood. Why the reasoning is not included until the context is full, then the reasoning stripped optionally to allow the conversation to continue. and only then when its truly full offer a compaction. Was it to optimize caching? Well I guess it doesn't matter now that you hinted that this choice was made because of prior limitations and may change very soon
Models are typically trained (at longer conversations/more turns) either with or without the reasoning still in the conversation. If you train a model with those, then using it without them, the model will perform a lot worse, same vice-versa if you train without but then end up using the model with them.
That's why you'll see some models have it and others don't, and trying to use them another way, will make them worse, they weren't trained like that.
So why aren't the models trained with both? I'm guessing that sort of permutation in the training would lead to double the amount of training time being needed, as you know effectively will have two variants of every session you train on, with and without the reasoning.
Why do they store an encrypted reasoning payload in the session file and pass it to the API? Just a ruse? Reasoning isn’t even that many tokens, you think they’d degrade their model quality like that?
Reasoning messages would be lost immediately after a single tool call, unless you mean they sometimes go back and strip the reasoning channel retroactively, but that would increase costs via cache invalidation. I just don’t see any way it would make sense for them to do.
And wouldn’t this be noticeable by reasoning tokens not being accounted for in the context window usage?