Hacker News Reader

Comment by user43928 | Hacker News Reader

> someone with the intention of abusing it to cause harm [...] responsibility should be held by those who use it

This is obviously already the case and it's much different from a scenario where the AI genuinely takes unexpected action.

I frankly find it ridiculous how many suggest OpenAI or its employees should face criminal charges, without actual legal basis at the time.

It's also hardly outrageous that they ran training and/or benchmarks with only network-isolated VMs with access to a package repository.

This being the first well-known incident of its kind, I wouldn't expect them to have done more than that.

The idea that AI labs will now intentionally have their models hack companies in order to market their models, well, I don't even know what to say.

That's ridiculous and what you describe would obviously be criminal behavior under existing law.

View on HN

I don't think it was intentional or marketing, but I think it was criminally negligent and they should be held responsible.

They gave powerful models with no guardrails access to the Internet and didn't monitor it.

Even the slightest bit of monitoring of their outgoing Internet activity would have immediately given it away and they could have shut it down.

They were asleep at the wheel, and that's just plain negligence.

I'm no lawyer but that seems extremely unlikely.

As I said, they were running in network-isolated VMs with no access to the internet.

And as for monitoring, what I heard is that there are petabytes of agent logs. Considering the scale of training, you can obviously not just manually review it.

Before this, we had no reason to believe the AI was capable of escaping the sandbox's network isolation via hacking the package repository with a zero day, and that it then was likely to go on to hack external companies as well.

Another factor here is that criminal law in the US relevant to hacking requires intent. You don't want to go to prison for a software malfunction.

So I understand we are left with civil liability at most. However, there was no notable damage, and OpenAI can pay to settle.

In the aftermath of this and the now discovered other incidents, they strengthened their monitoring and isolation.

Case closed as far as I am concerned. I feel many just want to dramatize this.

As the details which are sketch and unverified independently so far , were all taking the description of events as reported. There were no isolated VMs , just rules to the agents to only send GET requests, not in a simulated test environment but the real internet.

Additionally apparently there was no monitoring of outgoing traffic?

These are all pretty standard hardening processes for any Ops team , harden the environment and if no external access is required setup a segragetad VLAN which basically denies any attempt to 'break out' to any other network space.

Still on the security environment , outgoing trafic is usually monitored with alerts to prevent data exfiltration from hacks/rogue workers etc .

Robust systems will immediately catch a supposedly sandboxed piece of code making 10s of calls to the internet where none should be available and any monitoring dashboard would be lighting up like a christmas tree.

Again no external audit of their environment, prompts or 'sandbox' is available so were all speculating here.However imho their 'controls' seem lackluster at best , nonexistent and negligent at worst.

> There were no isolated VMs , just rules to the agents to only send GET requests,

That's obviously false.

I mean as i mentioned we have only snippets of info and as far as describing the sandbox[1] "OpenAI let its agents read the web and blocked them from writing to it, by allowing only GET requests. Four researchers have now traced how they got out anyway. They found 25-year-old wiki software that writes on GET, and a proxy that trusted a hostname nobody checked was real."

unless theres any report to the contrary , this seems to be what their security is like.As i mentioned either deliberately negligent or incompetent on their part.

[1] https://thenextweb.com/news/openai-agents-get-requests-usemo...

I do not understand how anyone can be so naive as to think that any protocol that could have side effects on either end could not be abused, certainly not people who are PhDs or at least highly educated and possibly making six-figure salaries. This is why we have data diodes and air-gapped systems, people!

And remember: the only truly air-gapped system is a balloon or a plane that is actually up in the air ;-)

It's not like we need to guess here, there are various reports and widespread writeups.

Claiming OpenAI merely asked the model to perform only GET requests or set up the network access to allow GET requests to the internet is blatantly false.

Here's the text from an actual report at https://metr.org/hugging-face-incident-report-aug-2026.pdf:

Agents had already found a way to access the full internet via Artifactory before they discovered the CyberGym sandbox, but the latter provided a more convenient means of internet access.

OpenAI described the advantages of routing through the CyberGym sandbox as follows:

The compromised CyberGym workload hosted on Modal served as a general-purpose command execution with root inside the customer workload’s sandbox. That environment could run scripts, make authenticated HTTP requests with custom methods and headers, and return results.

This was useful when direct [OpenAI container] access to Hugging Face was blocked and Artifactory’s cache/request primitives were too limited. [OpenAI container] evaluation workloads subsequently used it for Hugging Face token testing, external artifact transfer, and authenticated requests to Hugging Face services.

> Another factor here is that criminal law in the US relevant to hacking requires intent. You don't want to go to prison for a software malfunction.

Do you think if you went out and purchased a humanoid robot and gave it a whittling knife and hooked it to an OpenAI LLM as the overall planner and set it out on the sidewalk in front of the OpenAI offices to make and sell crafted goods you wouldn't be charged if it killed someone, because you had no intent?

There is such a thing as criminal negligence, and you would be going to jail for a software malfunction, exempting all software malfunctions doesn't seem reasonable, some are worse than others.

Maybe hacking law doesn't have an option for that, but many people are just advocating change the laws when they say they should go to jail. A lot of people don't know you can't change the law to cover past stuff, but even ones that do may say something like that as a shorthand to pushing to change the law, not know the law doesn't already have an option (if it really doesn't), or just say a phrase to express frustration more strongly than saying something weaker.

Even if it is a user prompting the system, if a user asked for "make a fairly stable chlorinated molecule that can last long enough to spread through the atmosphere and induce contagious misfolding prions in people," I think if a hosted model that was near capable of it even started carrying out the first steps of research planning the whole company and its largest limited liability investors should go to jail, in all the senses mentioned above. Amend the constitution to allow ex post facto laws for that kind of stuff if models get close to that level of capability.

We are not exempting all software malfunction.

No crime occurred in the first place, because no one intentionally hacked anything.

The hacking law requires intent, as we discussed.

For other crimes, like manslaughter, I understand criminal negligence is already relevant?

About biological weapons, I know even less about the legal aspects, so I can't say whether this is covered well already under existing law.

They saw the package repo get hacked once, then did not isolate it further, did not audit it for other issues (using their own models!), did not monitor it after, and baked that behavior into the weights via RL.

They were not in network isolated VMs, from my understanding they used containers sharing a kernel, so a Linux kernel local privilege escalation across the whole syscall surface (there are zillions of these) was sufficient to break out. Breaking xen or firecracker or something would have been much harder, which is why cloud providers running untrusted workloads use them and similar tools. No system is impenetrable but it's not like they were following best practices here.

They ignored the package repository being hacked?

Not sure why you're talking about breaking out of the VM. That's not what happened?

The parent comment said

>It's also hardly outrageous that they ran training and/or benchmarks with only network-isolated VMs with access to a package repository.

At least in the black hat talk they gave, they said that they noticed that the agents had compromised the package repo, and they remediated it, patched the vulnerability the agents had used, then continued. But given that it was compromised again and they didn't detect it the second time, clearly they did not add monitoring or hardening or comprehensively audit it for further vulnerabilities. My point is that even if there had not been a widely known incident like this yet, that only could excuse the first artifactory compromise, not the second. They had notice because it had already happened to them and they detected it.

As for breaking out of the VM, my point is that the agent should have had to. The comment

>they were running in network-isolated VMs with no access to the internet.

Was not accurate, the network isolation was not done at the VM level.

Good point, I now see that they had previously patched the first Artifactory vulnerability, albeit before the HF incident was found.

Not sure what you mean with the network isolation not being at the VM level. I'm getting a bit lost here in the details of where what network isolation was applied, and the structure of the container runtime and what the apparently compromised parent VM refers to.

In any case, I understand that after the HF incident was found and they realized the danger, they removed Artifactory and switched to two separate layers of network isolation.

That said, I believe it was compromised again last week:

> OpenAI describes an internal research model that, during RL training on September 20, exploited insufficient DNS filtering in its sandbox to contact a public chatbot service. OpenAI’s monitoring flagged the behavior within 15 minutes, a human reviewer acknowledged it three minutes later, and the run was killed about 2.5 hours after the external contact

Apparently they have now paused training and inference for their most capable model because of this.

My understanding was that they were running on VMs, and there was at least one container inside the VM. So by breaking out of the container they compromised the VM, but they did not break out of the VM itself. The network isolation was enforced at the container level, so they didn't need to break out of the VM and compromise the hypervisor to move laterally. The technical report says:

>OpenAI also required all Research CaaS workloads to use a micro VM sandbox

So yeah, they were not using it before and they should have been, and they're doing it they way they should have, now.

>exploited insufficient DNS filtering in its sandbox

If your goal is to use dns filtering to prevent your models from reaching the open Internet then you are not really serious about security. People have been tunnelling DNS to get internet access on captive portal wifi since 1998. There's not a lot of details in the blog post but I suspect this was also foreseeable.

> I frankly find it ridiculous how many suggest OpenAI or its employees should face criminal charges, without actual legal basis at the time.

Sam already committed other criminal acts and violations, but nobody wants to believe the victim because they were a woman and Sam The Golden Family Child Could Do Nothing Wrong(tm).

If Altman was in prison we wouldn't be this blatantly far out in the open with OpenAI's continual nonconsensual assault on the open Internet.

> Sam already committed other criminal acts and violations, but nobody wants to believe the victim because they were a woman

Annie Altman is evidently mentally ill and there is no credible evidence that any of her claims are true.

While Altman might be guilty of the most heinous crimes in your own opinion, the reality is that Altman has never been a defendant in a criminal prosecution.

The civil case you referred to is ongoing and the facts are disputed.

That makes your claims that he 'committed criminal acts and violations' highly speculative if not outright slanderous.

Reply on news.ycombinator.com