Hacker News Reader

Comment by KingOfCoders | Hacker News Reader

"The agents found a Modal-hosted insecure app with a weak API key, then used that to stage an attack against Hugging Face."

Why, what was the prompt?

I told Claude today to wire plugins on Linux into a sound pipeline to remove noise. Did some astonishing things, played sound through the pipeline, measured it etc. I told it to optimize my sound for TF2 and it played the spy_decloak samples, measured them and made them easier to hear, astonishing too.

But it did not go to hack Amazon because it could.

View on HN

This was clearly explained by OpenAI in their initial press release on 7/21 [0]:

> This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. […] The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database. All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.

[0] https://openai.com/index/hugging-face-model-evaluation-secur...

It does not explain how agents months later would "collaborate" to hack Hugging Face.

they explained that it was looking for datasets to solve their problem and chose HF?

I now watched the video. It seems the agents were sharing context for months, run unattended for months, the sandbox was no sandbox at all, one agent hacked a service and announced it, the service was fixed weeks (?) later, but not secured in any way, the agents hacked the same service again and researchers again didn't watch what the agents were doing. Then the agents - unattended - hacked OpenAI infra and HF. Which is when someone found out about the whole thing that was going on for some months.

This is why I disagree with anyone claiming it is just marketing. It makes OpenAI look really really bad, like they have no idea what they're doing in terms of security. After the first board happened, they still didn't add better monitoring? They didn't rollback the checkpoints of the models that were in training to before the first board existed, so they still had the idea of a secret board in their actual weights, etc. Like it is almost mind-boggling...

Reply on news.ycombinator.com