13 karma · joined April 14, 2025
To me it feels like the difference between "RL performance" and the pretraining / base "knowledge".
Yes you can RL terminal bench to the moon but does the model hold up on out of distribution tasks?
Kind of like trying to navigate a dark room with a laser light, vs a flashlight. Laser is going to go a lot farther much more efficiently but only if you are already pointing it at the right place.
This is only required if you want users to be able to share things with everyone and you are going for the simplest implementation.
If not you could try to keep a record of keys associated with a user, then when a new request comes in look through to see if the user has a valid key to decrypt the COT.
For explicit shares, just add the key used in that one conversation to the users valid keys. For global shares use the global keys. But that's adding more complexity to the system.
If you can play it right, you don't even need to send suspicious prompts to the frontier models. Just use them for regular tasks, extract the encrypted COT blocks and replay it to a cheaper model to get the plain text COT.
But the real question is: Is it okay to steal from a thief's hoard?
But there is no real way to know how much of this "waiting" any lab is doing, if we can get better estimates this way maybe we can gauge how far the open weights models really are.