Hugging Face had to tell all of their users, many of them paying customers:
> As a precaution, we recommend rotating any access tokens and reviewing recent activity on your account. If you believe you are affected, or want to report a security concern, contact us at security@huggingface.co.
HF also said this, I'd be very interested to hear how that got resolved!
> Finally, we have also reported this incident to law enforcement agencies.
It's very difficult for me to reconcile belief in the existential risk business with what they actually did. So I agree with you that this makes OpenAI look badly incompetent; but their communication on this makes me think they don't realize it.
For what it's worth I don't agree with the xrisk-ness of these models; they're dangerous, but almost certainly only temporarily while a new equilibrium is reached via more secure software. Open models are probably an essential part of the recipe (as you noted) for doing so. I also have a personal suspicion that LM-accelerated formal verification will have no small role to play here, sidestepping the cat-and-mouse game of bug finding-and-fixing.
> We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of. We will continue to conduct a thorough investigation alongside Hugging Face and will share more details on the vulnerabilities, incident, and findings when our investigation is complete.
That's not well massaged PR language - that's the kind of thing you dash out when you see a major shitstorm brewing (HF had already publicized the attack before they knew it was from OpenAI) and you want to get ahead of things while you're still pulling together the full story.
I expect we'll find out within a few days if OpenAI are going to keep their promise to "share more details on the vulnerabilities, incident, and findings". If they don't do that I'll reassess how I interpret their initial post.
With who? Who are these "defenders"? None of the US labs have done much for the greater good as of... Ever. Of course a frontier provider can leverage their own resources at scale and pull something like this off. If anything this should showcase how dangerous OpenAI and Anthropic are in their current states and maybe the powers shouldn't be concentrated as they continue to move.
I will bet that the RCA debriefed by OAI is going to be a lot of lipstick and very little meat.
How would the model get any packages that it thinks it needs to complete the task at hand? Not a well-specified task that those tasking it could anticipate and provide all resources up front, but one of discovery.
The target audience is regulators. They want to look like the smart guys really concerned about AI safety, when they come asking for open weights models to be banned and for other regulations to cement in their moat.
They want this to look like a demon core incident. Bomb and Nuclear reactors still got built.
Turns out they can backfire.
They are still openly lobbying for more AI regulation.
Please do explain how this event that makes their product look powerful and perfectly aligns with their openly stated long term goals of pushing for more AI regulation makes them look bad.
This very thread is full of comments from people who are shocked at how badly they messed this up.