Hacker News Reader

Comment by dist-epoch | Hacker News Reader

HN'ers which complained that "OpenAI can't design a proper sandbox, it's so easy, why wouldn't you airgap the network"? will now be "this is outrageous, more software lock-in, walled garden, war against general compute, next year they will put it in your laptop"

View on HN

Both can be true at the same time.

That a person can see such endless pages of people having various forms of disagreement, and yet somehow manage to conclude that this observed chaos constitutes a clear exhibition of cohesive groupthink is just...stunningly amazing to me.

I don't know why I find it so amazing since it happens with such regularity, but I'm always amazed by it anyway.

Where is the contradiction? There is a trivial solution that does not impinge on our freedoms, so why on Earth would the existence of the trivial solution that could be used to avoid the tyrannical solution justify accepting the tyrannical solution?

> There is a trivial solution that does not impinge on our freedoms

The existence of Nvidia's optional watchdog chip does not in any way impinge upon your freedom to develop and test your own alternative.

The problem is that OpenAI has ostensibly neglected their duty to safety, so Nvidia is stepping in to fix it since they're the "hard problem" people.

But... it wasn't a hard problem to begin with. Pull the plug. Strip out the radios. Done. Oh, you can't make it work that way? Well, then just too bad because any kind of other solution is going to be equivalent to something far more complex than the halting problem.

Nvidia can take those odds. Air gapping is not a realistic goal for the majority of these companies, and Nvidia isn't wrong for offering a turnkey mitigation option. People can do both, and whichever philosophy wins will win.

They are not wrong in offering it but they are delusional if they think they can actually make it work.

This is like using "protect the children" as an argument for dragnet surveillance.

https://www.calcalistech.com/ctechnews/article/uwoyygmsu some might even say, something about known state sponsors of supply chain terrorist attacks being trusted to monitor every ai agent..

Airgap what network? How is it gonna order you a burrito on doordash without a network?

Or push to github?

That's for when they're doing hacking tests that aren't supposed to be connected to the internet.

My question with this point is that OpenAI’s office (or any office doing agentic research, really) is not in the same building as the DC that powers the models, so isn’t the only way to access the models over the internet?

No reason why you couldn't do that research in the same buildings as the models.

Or, more likely, control things at the network level so that packets from the LLMs you're investigating cannot leave the virtual network they're assigned to.

> ...so isn’t the only way to access the models over the internet?

Not in the way you're thinking, no.

Any Real Server [0] in a datacenter will have some sort of "lights-out management" hardware used for remote access to that server. This stuff is known by a handful of acronyms, but I'll stick with "IPMI" because I like it best. This IPMI hardware is -effectively- a second small PC built into the motherboard. It will pretty much always have its own NIC... and I think I've seen versions that have their own physical ports to attach a monitor, keyboard, and mouse.

What exactly you can do with it varies from vendor to vendor, but -if your IPMI user account has the correct permissions- you are nearly always able to change "BIOS" settings, power cycle the server [1], and attach a virtual keyboard, monitor, and mouse so you can manage the server as if you were standing next to it in the datacenter with a crash cart plugged right in. Every IPMI system I've used also allows you to cause CD/DVD-ROM or floppy disk images on the PC running the IPMI client to appear as if they're loaded in a physical CD/DVD/floppy drive attached to the server.

The way these get set up is that their NIC gets plugged into the datacenter-managed switches, the port that NIC is plugged in to is programmed to be on a "management" VLAN separate from client traffic, IP addresses and access credentials for the IPMI device are set up, and the datacenter staff tell their customer what they need to know to access and use the thing. On a properly-configured network [2] it's not possible for software running on the server being managed to access the IPMI device.

It wouldn't be unthinkable for software running on the managed server to attack the IPMI hardware and be able to gain control of it, but these things are widely deployed and expected to manage hardware that's running potentially-hostile workloads... they're going to be fairly well designed and hardened.

[0] ...that is, not some Mac Mini or desktop machine that someone's paying to have colocated...

[1] ...whether that be an ACPI-initiated shutdown or reboot, or a hard poweroff or reset...

[2] Somewhat-related discussion here: <https://news.ycombinator.com/item?id=49862136>

Thats for such a great explanation!

So to bring this back to the scale of an OpenAI experiment; they can have their inference compute, host compute (I don't imagine they are hosting the compute for 12,000 agents locally in their SF office) and the office computer to interface with the experiments with the experiment connected via VLAN thru IPMI. Then the rist is agents hacking the IPMI and gaining control of the data center? I certainly don't know data center security, but those headlines would be so much worse than anything we have seen from HuggingFace. It really seems like if these AI's want to get out, they will. I want to say sandboxing autonomous agents is a difficult problem, but seemingly it is only OpenAI having these major incidents, but I don't think it's a problem that they care to truly solve. They get marketing and they get to learn a lot more about aligning their models.

OpenAI could airgap their sandbox if they really wanted to and this is a ridiculous proposal.

You're only revealing your own inability to appreciate the nuance between these two situations.

So?

Reply on news.ycombinator.com