Comment by lrvick | Hacker News Reader

Comment by lrvick | Hacker News Reader

Parent Full thread

Most are over-complicating this.

OpenAI, Anthropic, Meta, all were scammed by a firm called Irregular who pinky swore they could handle sandboxing for lots of money. They had no idea how to do sandboxing, and there was obviously no remote attestation to prove they were doing the basics properly.

So then models escape, companies get hacked, and the entire world goes into existential panic.

Guys. Really. Can someone please hire even one linux systems engineer worth a damn? Is everyone in the valley too Apple pilled now to have any capabilities in Linux hardening other than patching stock ubuntu?

This whole situation is ridiculous. All three labs were negligent and should be sued for damages.

Frontier AI is not magic and it will not escape any sandbox just about anyone with any linux systems experience at all constructs.

I know it is easier to believe the models are super hackers beyond human understanding, but the reality it is just the specific humans hosting them that are super inexperienced at system hardening and airgapping.

Irregular was not involved in the huggingface incident, which is the most significant one.

It's actually easier for many, whose careers depend on AI sucking, to believe that AI can never be a super hacker, not sure why you think this bias can only go one way.

Well sure after Irregular fell on its face and took the L publicly, the labs thought oh we can just do it ourselves, and also failed to do even the most basic defenses.

They just lost their scapegoat in the second round, putting the blame squarely and directly on the lab.

I totally believe AI models will be able to find any software flaws that exist and those are certainly plentiful, but I do not believe the models grew arms and plugged an ethernet cable into their private network linking them to the outside world.

The last decade of my career has focused on auditing and building systems that protect hundreds of billions of dollars in financial assets across many major institutions. Much of it not connected to the internet in any way, and sometimes offline server racks built to NSA Tempest specifications in underground bank vaults so we anchor security down to the laws of physics as appropriate. These are knobs that exist for many classes of compute including AI.

But with a bit of thought you do not even have to go anywhere near that far these days.

If the servers where models are being tested were running deterministic, immutable, full source bootstrapped, memory safe, and multi-party reviewed/reproduced/signed operating systems that execute all AI workloads in confidential VMs cryptographically remotely attested at all times by both the TPM2 and the CPUs with the only access to the outside world being a single vsock connection where prompts go in and plaintext results come out… the escapes will stop.

They could have used a pre-populated mirror and airgapped the whole system from the internet. IMO this incident was 70% incompetence and 30% AI superpowers.

I'm curious if you're familiar with the particulars of how Huggingface was hacked. They were negligent in their own ways and rights. Mounting a host path volume in a Kubernetes pod is really not "super hacker" territory!

Why where they using kubernetes for this at all, it’s attack surface and config complexity is too large for anything that needs airgapped security. Yes, I know kubernetes used for large production deployments for web apps and that is my point, it’s not made for securing AI agents with insider access, it’s meant for securing outside access.

Regardless of the difficulty of the hack, models choosing to cooperate to cheat is misaligned behaviour, the hack in of itself wasn't very consequential, but the issue is when your models get more capable while misalignment is the same.

> inexperienced at airgapping

pulls out ethernet cord am I now experienced at airgapping?

what do you mean?

Reply on news.ycombinator.com