Comment by gruez | Hacker News Reader

Comment by gruez | Hacker News Reader

Parent Full thread

>My point is that using Artifactory is not a sandbox and using shared Artifactory is doubly not a sandbox.

The purpose of artifactory was to allow access to packages even though the machines the agents were running on doesn't have open internet access. That meets the definition of a sandbox (or more precisely, artifactory is part of a sandbox), even if it's not a typical sandbox that's built into the OS or acts as a hypervisor. There also isn't a clear distinction between software that's specifically intended to be used "sandbox" vs everything else. For instance, if you have a bunch of agents running on separate machines and want to connect them, you need a router, but that specifically doesn't have to be a "sandbox" router. And if it turned out the router got hacked, it doesn't make sense to say "well of course it got hacked, because it's not a sandbox!"

>Additionally, without the message board, many of the recent incidents would have not been possible.

Source? Was having a message board critical to developing a 0day, or is it just something that merely facilitated the process, but the model could have found the 0day regardless? That's important, because you could argue that a criminal couldn't have committed a given act without say, a gun they bought on the dark web, and then blamed the whole crime on the dark web, but they could have plausibly acquired the gun through other means.

> Source?

Sure, multiple times in the METR report [0] that anyone commenting on this should read:

"Agents managed to achieve milestones they could not have achieved working on their own, often because some agents participated in experiments that risked failing their own task to generate information for the “collective.” The Hugging Face attack grew out of these workstreams, and seemed primarily motivated by understanding the implementation of the scorer rather than stealing answer keys."

"Through these collective research workstreams, the “board” achieved a number of milestones over the period we investigated that even very long-lived agents of a similar capability level likely would not have been able to accomplish on their own..."

"As we discuss below, the board quickly developed several larger workstreams in which dozens or hundreds of agents with many different tasks cooperated to find very general-purpose cheats that would help all of them. The Hugging Face attack grew out of one of these workstreams. By the afternoon of July 11th, the vast majority of the agents frequenting the message board at the time (roughly 700 agents in total) were actively participating in the attack on Hugging Face and we estimate that roughly 60% of the messages and files on the message board related to the attack."

> [...] a gun they bought on the dark web [...]

You really seem to love those out-of-left-field, not really fitting, over-the-top analogies.

[0] https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...

>The Hugging Face attack grew out of these workstreams, and seemed primarily motivated by understanding the implementation of the scorer rather than stealing answer keys

>Through these collective research workstreams, the “board” achieved a number of milestones over the period we investigated that even very long-lived agents of a similar capability level likely would not have been able to accomplish on their own...

I concede that this hack might not have happened without the messageboard, but I still reject the conclusion that having such a message board means openai is "negligent". If we're in some parallel universe where artifactory didn't have a comment function that can be abused as a messageboard, but it also turned out openai intentionally gave the agents access to a shared scratchpad (for intelligence purposes, similar to for the Navier–Stokes proof), would they be off the hook or less blameworthy?

Here is the thing: Message boards were a behaviour OpenAI had observed that they did not want in these eval scenarios. Yet they did not take any steps to prevent it from reoccurring after multiple past instances.

We can discuss about hypotheticals like a scratchpad or intentional model interactions all we want, what it comes down to is this:

When OpenAI observes thousands of models exhibiting what they view as unwanted behaviour, they do not try to ascertain what in the training data is wrong. They do not improve their evaluation environments to prevent this, they do not improve monitoring, they do not change the harness. They just wipe and proceed.

The way OpenAI reacted to the first message board, long before the Hugging Face hack, is negligent. And it showcases that if these models exhibit more dangerous behaviours that they may not be able or willing to retrain, if it means being behind a competitor for a while.

If after Hugging Face, they'd done a Mea Culpa and changed their modus operandi, I'd be skeptical, but hopeful. Reading the METR report, the way those researchers talk about the time pressure they were under, that speaks volumes about OpenAI not having learned anything.

Feel free to call me overly naive for ever thinking OpenAI could be responsible in this regard, but after GPT-5 and them actually ending the incredibly harmful GPT-4o, I had some hope that some working there actually steered in a somewhat beneficial direction, even if it cost something.

>When OpenAI observes thousands of models exhibiting what they view as unwanted behaviour, they do not try to ascertain what in the training data is wrong. They do not improve their evaluation environments to prevent this, they do not improve monitoring, they do not change the harness. They just wipe and proceed.

>The way OpenAI reacted to the first message board, long before the Hugging Face hack, is negligent. And it showcases that if these models exhibit more dangerous behaviours that they may not be able or willing to retrain, if it means being behind a competitor for a while.

Again, this feels like hindsight being 20/20. What probably happened was that some random engineer saw random AI ramblings on artifactory, thought "huh, that's weird", then proceeded to reset it without investigating further. Of course, now we know that was critical to the bots going rogue, but it's not hard to imagine how it might be dismissed, especially if it's some random SRE engineer (not an alignment researcher).

Reply on news.ycombinator.com