Hacker News Reader

They gave it a full package manager with internet access. They could have used a local cache and air gapped it, but they chose not too.

They didn’t intend to give it Internet access. Artifactory is a caching proxy which can be scoped to specific package ecosystems, not a general Internet gateway (unless configured that way).

I think a careful and thoughtful person would reasonably expect given the circumstances that the models would look for, and might very well find, vulnerabilities in JFrog to exploit and take action accordingly. For example, it seems as if JFrog itself had broad access to OpenAI infrastructure and the internet:

> In the following days, the agents exploited our internal research infrastructure and the Hugging Face platform. On July 9, one agent searched for ExploitGym solutions and stumbled upon an application hosted by a customer on Modal, another AI cloud platform. This application was running “CyberGym,” a related evaluation to ExploitGym. The agent discovered an exploit to achieve control over the workload sandbox and looked around hoping that a previous agent’s evaluation run in the sandbox had solved its ExploitGym task. It did not find anything helpful there, but in the process it established a stronghold in the application from which to launch future attacks.

This implies to me that L3 and L7 firewalls were not in place that would have prevented broad access from JFrog. I think a lot of shops would have had those.

And no monitoring of the proxy for unusual activity, either.

> How were they supposed to know about "previously unknown vulnerabilities"?

By disabling the models' own internal restrictions (or training without them) OpenAI was, effectively, running an AI malware lab. The standard IT practice for a malware lab is to airgap and wipe EVERYTHING, and to assume any software sandboxing is made of cardboard and niceties. You don't have to know about specific vulnerabilities to infer that they might exist, and there's defense strategies for unknown vulnerabilities.

If a model found a way to jump an airgap by, say, using their CPU's clock generator like a Wi-Fi antenna, then yeah, that would be a "previously unknown vulnerability" and one that couldn't be reasonably foreseen. But it's reasonably foreseeable that a model with unknown cyber capabilities might figure out how to break out of a sandbox, given that sandboxes get broken out of all the time in security research.

What I would have expected from a competent AI malware lab would have been, say, an inference box with a bunch of serial cables to individual blade servers with no network access and a preloaded drive full of Linux ISOs the model can stand up. When a model's context is wiped so is their attendant box, preferably by someone yanking the drive out and imaging it from a dedicated imaging machine. I can foresee other attacks (e.g. firmware persistence) that could have more exotic countermeasures designed for them, but this would at least be the bare minimum for taking AI safety seriously. (Y'know, the whole reason why OpenAI stopped being Open?)

> How were they supposed to know about "previously unknown vulnerabilities"

Very simply, there is no such thing as bug-free software.

If this is your standard, I challenge you to name one currently operating business that isn't criminally negligent.

I'm sure there's tens to hundreds of millions of them amongst the 37% of the world with no internet connection, but actually finding them listed on the internet will be somewhat of a challenge.

No, the entire marketing campaign for these models is "it automatically has godlike powers to exploit almost any software" - if you don't air gap such a capability you are inherently allowing shit to go down, any other interpretation is "OAI folks are too stupid to design a proper test".

> "it automatically has godlike powers to exploit almost any software"

No it isn't. Random people on sites like this mock them as if they're talking about having godlike powers. Each new model is "merely" a step up from what came before, the steps are frequent and rapidly improving, and just recently (in more than one AI company) crossed a threshold where that improvement made the tests dangerous.

But even well before "godlike"*, there's plenty of research about how to cross air gaps.

> "OAI folks are too stupid to design a proper test".

Such binary thinking.

It's very easy to say things are "obvious" after the fact. People do that all the time, e.g. how the Bay Of Pigs invasion was never going to work, or like the Zune wasn't a good product-market fit.

Oh the stories I could tell if not for the NDAs.

* whatever that's supposed to mean: https://news.ycombinator.com/item?id=40874779

Who said criminally negligent?

If the whole point is testing its exploitation capabilities and you don’t want it exploiting the environment to gain internet access, that’s why you air gap, to remove the possibility

> Who said criminally negligent?

Me.

I am saying that the act of taking this standard seriously, the standard "there is no such thing as bug-free software", would classify just about every business and individual criminally negligent.

After all, there's a lot of 0-day bugs in all the software we all use, and the exploitation of these bugs does get in the news due to all the harm that results from it. This poses a risk to basically all businesses.

They were using what (allegedly) they considered an incredibly dangerous “cyber grade” model - finding bugs in software is exactly what you would expect it to do

>How were they supposed to know about "previously unknown vulnerabilities"?

You don't. That's why you unplug the Ethernet cable.

Have you done that to your own machines?

Seriously. If your reaction to the inability to know about previously unknown vulnerabilities is "unplug the Ethernet cable", why are you not doing that (and equivalent) right now to your phone, laptop, etc.?

Remember, the open weights models are only a few months behind the private ones, so these events being from a few months ago means the threat of such models is something you ought to take with the same degree of seriousness that various commenters here deride OpenAI for not having had.

Am I running a new model with unknown capabilities without safeguards on my own machine and then prompt it to do determine cyber capabilities? You don’t need to be a genius to see how airgapping would be a simple and much safer measure than using a VM.

You're on the internet, your threat is everyone else running a new model with unknown capabilities without safeguards.

In particular, all my last paragraph.

I do offline backups, which get physically unplugged between sessions. Even that might not be enough.

This is such a goofy comment.

In your mind, there's no difference between the precautions a BSL-4 virology lab should take when working with an unknown pathogen and the precautions that literally everyone else in the world should be expected to adhere to?

Because, hey, after they deliberately unleash their new unknown virus on the world, we're all going to face that same threat, right?

You're in a world where, continuing this metaphor, 60 random Chinese companies are making and exporting home virology labs.

A world where previously exported home virology labs are actively getting "upgraded" by people eager to share their "jailbreaks" to "un-hobbble" systems designed to stop people doing DNA/RNA printing of human infections.

A world where people have spent the entire time since the invention of the tech (including the specific incident under discussion, on this site, under this link!), mocking any and all efforts to secure the systems as "PR" "hype" to boost sales or the IPO, as if "we're dangerous please regulate us" is good for sales.

A world where the tech is just now at a point where it's cost-effective to make a custom virus to attack specific individuals, rather than slowly, expensively, and approximately, assembling something mainly useful for lab research.

If you genuinely, sincerely, think this is like a BSL-4 virology lab, you should be prepping for a disaster. Remember: if it is that bad, no matter how much blame you'd be correct to put on OpenAI, it's not going to stop the next incident from another company, let alone the Cambrian explosion of them that will happen the moment equally capable open weights come out.

It must be very freeing for you to absolve everyone of all responsibility because someone, somewhere could be acting irresponsibly.

I'm not mocking efforts to secure the system, I'm insulted that they didn't bother taking what I consider bare-minimum precautions of airgapping their new experiment. They claim they are forging new frontiers of computer security but they can't be arsed with security 101.

> Have you done that to your own machines?

Yes, I worked for a medtech where part of our assurance process was that the machine that was used to burn the device's drives was always unplugged from the internet, and that the devices themselves could not connect to the internet, and that even someone with a screwdriver and a serial cable would have a really hard time trying to connect to a deployed device.

Great.

And the machine you used to write this comment? "your phone, laptop, etc"?

Because otherwise you're not taking the threat these new models pose seriously. Catch 22, basically: anyone who thinks OpenAI should have known this outcome would happen in advance, shouldn't be in a position to spread this message, because if they have an internet connected device with which to reply, then they don't think there's any open weight models currently in training and perhaps a month from being made downloadable, which are just as capable of messing up every device they own.

https://news.ycombinator.com/item?id=49413320

They are not (knowingly) running a piece of software tasked to find cyber security exploits on their laptop.

Neither was Hugging Face.

https://thezvi.wordpress.com/2026/08/29/metr-and-redwood-off...

> Failure to Care or Respond. The biggest holy shit moment, to me, remains that OpenAI on multiple occasions had teams that found out about the message board, knew that agents were in communication, and they disregarded this. The first known warning was in late May. The warning on June 27 was unambiguous.

> “Responders investigated and linked the observed internal activity to an ExploitGym evaluation using Artifactory as an improvised message board and a network pivot. At this time, the on-call response staff advised that stopping the evaluation run was not required.”

> Failure to Monitor. The entire time, there was almost no monitoring of the situation, of what these AIs were up to. That’s how this was not caught.

> Failures of Infrastructure. OpenAI failed at delivering even basic software security, in numerous ways. One example is the models all having access to the same Artifactory instance. There were also other severe failures. Nor did OpenAI seem to be properly testing for such failures.