Hacker News Reader

It wouldn't matter if it was stored only on their servers. As long as they offer the feature to downgrade a chat to a dumber model that can be jailbroken (and the downgrade keeps the reasoning), this trick works.

If CoT wasn't stateless and you instead just got a reference which pointed to the CoT stored on the lab servers, the same vulnerability would still exist. Since you just need a weaker jailbroken model to read a smarter model's CoT. This being stateless or not doesn't really matter.

The stateless part is also important for enterprise customers that require zero data retention.

(they could scope CoT access per model, but then users couldn't switch models mid-session)

this is a lethal trifecta, but where a chunk isn't even needed

you have a secret to keep that is read by the llm, and untrusted input that wants to exfiltrate it.

by hell or high water, the agent is gonna output that text

strange because, their subscriptions are not stateless. they log everything and send it to 3rd parties for moderation.