I brought this up to someone else and was told that airgapping is apparently much more expensive than I'd naively think.
I still think this is a sign that they are not taking their own rhetoric seriously.
I still think this is a sign that they are not taking their own rhetoric seriously.
It is also a choice to not do any or all of the above.
Not surprised this is always what they have and hack.
Who would use an Agent that spends $10,000 re-implementing some OAuth lib or reverse-engineering a proprietary lib when it's free on the internet?
Whilst it might not be JFrog's threat model, I wouldn't assume it can be used as a full internet proxy.
I don't really mean to defend OpenAI here, but they did make some attempts at sandboxing. Although it does seem that they didn't really know what they were doing.
And we don't just magically know all the consequences of that.
Which is exactly why we do need full, physical air gapping. (Which, yes, would also include self-hosting a mirror of the package repo, if the point of the simulation is to see what's possible with the real package repo.)
Yes, yes they do, but read through artifact proxies are dodgy as fuck, which is why and facebook (and I assume a fuckload others) don't have them.
Also semi-airgapped labs are a lot less expensive than you think at that scale. Once you have to do multi-region VLANs with machine certs before you get access to juicy VLANs, the difference between "no internet for you" and "mostly airgapped" falls to almost zero.
Also I would want an artifact mirror because a) that give a good signal about how the model reacts, and what training material its latched onto, b) it hides what the models are doing from the outside.
if your internal models are so damn good, they should be able to "one shot" airgapping... right?
You know what's better? They already do this per (paid) user - your ChatGPT subscription comes with a Linux VM that you can even legitimately SSH into, just ask your agent to configure it to accept your public key.
They absolutely know how to spin up VMs and configure them. They just made the conscious decision not to for the task where they specifically instructed the agents to hack stuff.
But you can only connect to Wifi if you have Wifi hardware, and RF signals are contained by Faraday cages. Ethernet is still a thing for local connections.
If that was the premise, why run LLMs in a lesser sandbox than a VM? Clearly it's not.
Well, my mental model concludes: because they are incompetent WRT security, or at least they inappropriately trusted a third party that turned out to be incompetent.
These labs are one of the most valuable and heavily funded enterprises in the whole world, that they can't properly air-gap their systems to me reads as if their "agents" and LLMs are not as good as they say they are, because if they were, why would it be hard/expensive to air gap a system? They already scraped most if not all of the internet, where did that data go?
But I'm calling for something stronger than a VM here because we shouldn't rely on the VM being bulletproof just like we shouldn't rely on Artifactory being bulletproof. The access should be controlled at the hardware level. Like, networking on internal LANs only, and the entire thing inside a nice big Faraday cage just in case.
Furthermore, VMs are isolation at the hardware level, particularly through hypervisors. I'd say given current LLM capabilities, it's a reasonable containment.