Its not just the f*cking sandbox

I was so excited to read this, get an inside view of how this problem must actually be much more nuanced than it seems for some reason not observable from the outside, just to read 10k words of whining and pleas for sympathy. When he finally makes it to the technical section it’s just “it’s a lab culture, there’s a lot going on.” There were some extremely inexcusable mistakes made apparent to anyone who knows anything about security at all. If the lab is spinning off experiments faster than you can improve your security posture just say that and own up to it (it’s what he appears to be trying to say in the worlds most mealy-mouthed way).

> I literally missed my sister’s wedding a few weeks ago to help clean up after some of the recent incidents.

A security researcher at OpenAI that clearly doesn't have limits.

On the one hand, people like Sam Altman have been talking about potential problems from AI systems up to and including the destruction of humanity for years now.

On the other, we are supposed to be ok with OpenAI deciding to move fast and break things? Cut them some slack because they are a lab?

Lol, what a pathetic attempt to garner sympathy. I like how he is constantly harping about how they never predicted the crazy jump in capabilities and then tries to defend the "safety researchers" like this:

> First, the safety researcher perspective. These folks work tirelessly to evaluate model capabilities and the dangers they pose as they advance at an alarming pace. They understand fundamentally better than nearly anyone else how models are able to interpret their environment, reason, and solve problems. They study models as they try and deceive their graders, evade chain-of-thought monitoring, and do all sorts of crazy stuff. These researchers are continuously stress testing the models to determine why and how they behave the way they do, and are working vey hard to make tangible progress in aligning their interests with ours. Many of these researchers have formal backgrounds in these types of networks, with expertise that takes many years to develop. However, a lot of safety researchers, even ones that I respect enormously, have never been in a real incident, don’t understand security vulnerabilities, or really know how to break a system. That’s okay. That is not their background. But safety has direct overlap with security, and so it does pose a problem.

Wow. This just makes them appear incredibly incompetent.

So this gross incompetence is the result of gross irresponsibility, because what I take away from this post is that OpenAI did not put enough resources into meeting the challenge. The post is very interesting, though.

> However, a lot of safety researchers, even ones that I respect enormously, have never been in a real incident, don’t understand security vulnerabilities, or really know how to break a system. That’s okay. That is not their background.

Correction: this is not ok. Hire a different background.

I mean it sucks but it kinda seems like their work training new models has vastly outpaced the ability to comprehensively secure those systems.

I don't think it's because they're incompetent or lazy, but that the nature of the problem is that security vulnerabilities seem to scale superlinearly with complexity but model training scales linearly or logarithmically with complexity.

But the organization isn't allocating the resources accordingly, which would likely be economically unsustainable for the competitive environment they're in. Put simply, if the security team was more than twice the size of the research team, they might have a chance at keeping up.

As someone who has been going around telling everyone that it's mostly about OpenAI being incompetent at sandboxing, I do kind of appreciate this post. It's very familiar to set up a system only to find years later that it has grown past the assumptions it was built around.

man that was so many words while saying so little. tl;dr "my job is hard and people are mean on the internet "

If the guy is missing his sister's wedding that really means they do not have a glimpse of what should be their priority.

> "world-class" security team

> decided to sandbox models by restricting them to GET requests

lol, lmao even

Want some cheese with your whine?

Those with AI Delusion Syndrome are fking insufferable.