Fooling around with encrypted reasoning blobs

Not necessarily for the "without sharing" part, but to increase the reliability of the jailbreak. The same prompt isn't guaranteed to return the same result, but combining the internal thinking with the prompt might be a more effective way

N.B. This is exactly how seaside, vba, and even arc[1] do server-side state generally: by encrypting the blob-representing-state and sending to the client to be sent back on future requests (where it will be decrypted and rehydrated).

It's an old trick that everyone designing protocols should know, since there are lots of applications beyond AI companies.

[1]: As in, pg's lisp: https://arclanguage.github.io/ref/srv.html#:~:text=The%20pre...

the exchange rate between text and its representation in memory is brutal. here's a bit from a recent article:

>An 82 GB footprint in DDR3 on a 2016 Xeon. About 25 GB of weights and 56 GB of KV cache at the full 262K context. The KV cache is larger than the model.

262k tokens is not much at all. with ~5 characters per token, that's only 1.3 MB of plaintext.

Except the providers also cache the parsing of the prompt (the KV cache), and that has substantial cost savings (easily an 80% saving on typical coding use cases).

That caching is done server side and not passed to the client. Which in turn means they still need state management on the server side, although it perhaps doesn't need the same level of global replication and availability.

they still have to cache the tokens. its not completely stateless.

in theory, every conversation is replayed from the beginning. in practice, its only going to be economical to heavily cache the stable portions of the text as tokens inside the GPU

one of the reasons the Cloud providers have such heavy prompts is because that can be cached for all users, but its essentially poisonong the state before you even start. alot of the variability appears related to changing the context rather than the model.

models are expensive and the bean counters know fine tuning and context changes are cheaper. id guess the IPOs are essentially the SOTA EOL.

While it seems like a good idea, resending a growing context window is very inefficient and costly. Instance pinning would make a huge efficiency gains but also collapse LLM provider revenue. This is something open models could better solve.

It's to prevent you training another model to emulate the reasoning. They want it to be their moat.

Sure they can. I was able to figure out Gemini 2.5 Pro's "Memory" feature's hidden system prompt because the reasoning tokens references the markdown headers by name as "blah blah says I can't refer to this", while the output would never mention them.

Yeah, I get that you can jailbreak and get that info anyway. Also that this is specific to front ends like web chat and less about API usage. But as a sibling points out it's also a good way to make post training other models harder. Mostly a "win/win" for the provider.

They are encrypted to prevent others from training on the reasoning trace. But before anthropic started encrypting the reasoning traces they were signed. The signature was to prevent the user from being able to manipulate the reasoning part of the context because this reasoning part is considered "more trustworthy" by the model. So it would be bad if the user could manipulate the reasoning to convince the model to do something dangerous/against policy. Encryption keeps this property while also preventing the reasoning from being used for training.

Also commenting to say I really enjoyed it.