Hacker News Reader

Was the answer key stored on huggingface's servers and this is public knowledge?

In OpenAI's writeup (https://openai.com/index/hugging-face-model-evaluation-secur...) they explain that the model initially spent its efforts obtaining internet access in an attempt to cheat on the evaluation. Once it got that internet access, it was able to do research that lead it to believe that HuggingFace had infrastructure that hosted the evaluation and potentially had the answers.

But does HF actually have the answers? Or did it commit a felony based on a hallucination

I mean, I interpreted the comments to mean that it committed a felony based on research it performed after getting internet access. I don't want to attribute much agency to a machine here, but an AI agent is certainly capable of using tools and adjusting its behavior based on the outputs of those tools. Even if it was wrong, that wouldn't necessarily make it a hallucination.

Anyway, if you read TFA, you'd see that HF did actually have the answers: "While the intrusion did reach Hugging Face's internal infrastructure, the only customer content accessed was the set of ExploitGym/CyberGym challenge solutions stored in five datasets."

Reply on news.ycombinator.com