Hacker News Reader

Comment by x312 | Hacker News Reader

Super cool that this works. I'm surprised these companies re-use the same encryption key across models!

I wonder if you can use these for attacks, like this previous paper showing that if you know how a model reasons, you can "fake its thinking" to control it? https://news.ycombinator.com/item?id=48631888

View on HN

> I'm surprised these companies re-use the same encryption key across models

I assume switching the model in the middle of a conversation is intended behavior (very useful in coding agents, for example)

Seriously, what does it take to encrypt per session? There are many ways to make it scalable and efficient so I am wondering if this is left like this to allow interested 3rd parties ahem unobtrusively peek what people are doing with the AI.

(Thanks for the link. That’s an interesting idea!)

The provider has the hidden text anyway; this isn’t customer managed encryption.

Sure, but if each session has a unique key then these need to be managed and stored and unauthorized access to these leaves tracks. So all that had to be 'compromised' is a single universally applicable key. Again, the question stands: session based encryption can be scalable and efficient. Why aren't they using it?

The exploit here isn’t a leaked encryption key. It’s pretty likely that they are already using a unique key per conversation. The raw CoT eventually reaches the model, and you can convince the model to share it with you.

Yeah encryption isn't the issue. The only way I see to fix this is if you stop the user from switching models mid-session, or strip out the thoughts when switching models. Either way you're degrading the user experience.

If a different model is using encyrpted blocks of another model, then by definition it is no longer a session scoped bit of information. Since you can give it to any other session and another model, clearly it doesn't even have to be the same user. Therefore, there is only one (set) of universally available key(s) used by all models across all sessions.

Reply on news.ycombinator.com