KingOfCoders

And a 4gb Linux machine.

"OpenAI wouldn't be constantl monitoring these training runs "

Occams razor vs. Hanlon's razor?

I've read the report, watched all the videos and is exactly:

OpenAI: shocked pikachu how could it hack?!?

They even went to a black hat conference and somehow boasted about it.

Kleinanzeigen != eBay

The Hugging Face incident could have been avoided if the "security researchers" would have not used a bloated, insecure, misconfigured proxy for the AI to use.

Oh, I've used balsa wood for the nuclear core containment, it didn't work! Bad radiation, bad radiation!

Like in the Hugging Face hack. They deployed big surface, insecure app and gave AI access to it, then told AI do whatever it takes to fulfill this list. AI hacks insecure service, gets out, "the AI is at fault!" - no it's like running a bio lab with no protections and a virus gets out, then blame the virus for escaping.

In Germany e.g. Galaxus works fine for many things. Also eBay surprisingly works better for many things because it has more filters, e.g. if you want to buy wood or window shades etc.

No, the British Empire, the one dominating before the US, finally ended in 1956. We'll see if the Russian empire collapses before the US one.

I think there is a difference of a country governing who can say what in that country vs. a foreign company. I don't care if you call it censorship or whatever, it's foreign companies meddling in political affairs.

US companies can meddle as much as they want in US affairs, I don't care. But not in the country I'm in.

The US empire at work. Luckily for all of us its 1955 and the empire is on its last legs.

It's just forcing US culture on everyone in the world. Also see Meta deleting IG posts and banning people based on US politics / morals.

The model was conceived when there were wide plain, few people and traveling messaging results took days to weeks. And it was never adapted. I'd say most of European voting was with shorter distances, higher population density and newer technology.

If this was not AI, but a biological virus, people would go to jail.

If all your backups are with AWS you don't have backups at all.

Fable migrated a Go/Wails project for me to Rust/GPUI and it's so much faster, there is the possibility of Rust to gobble up many more projects.

"is down" to me sounds like stating a fact.

Interesting that it uses Propane fuel and "weaves" the rockets with carbon composite.

I find things like this interesting, I guess everyone finds this normal but,

"Power consumption is down 45% - 50% annually."

Well is it 45 or 50? Is this a measurement or prediction or a random number? From an engineer or marketing? If prediction, why not saying so? Is this a range for several years depending on elevator usage?

Things like this irk me a lot.

"and it gave them a false sense of security."

This was part of evaluating cyber security of their frontier models and they had a "sandbox" which, and I'm not a security researcher, looks not adequate from the first look.

How is that possible if the OS/drive/vault is backdoored? Could you elaborate?

I think "resilient" just means "backup copy" and I do think (IANAL) it is illegal to destroy emails when asked for them in the US.

Or was your comment ironic? Sorry, German, irony impaired.

"They don't need Internet to give their agents access to tons of software."

I think that was the requirement, but yes, the cache could have been offline.

Still then they could have hacked it to create the message boards - but not use it to access the internet.

We offer EU data centers for customers that want their emails to stay in the EU but

"Resilient replicas of your data will live in the US"

?

How can it escape an "install package <x>" proxy?

   reducePrivs()
   serve get(package) { 
     secPackage = secure(package)
     getBinaryFromArtifactory(secPackage)
   }

I would think the code is very small and easier to verify, it doesn't especially have the ability to write files and act as a message board as Artifactory did.

And even if the agent tries to hack that, the attack surface is 1000x smaller and the possibility also much smaller.

But I'm not a security researcher, would love to see your hack to learn something (because that is what I do to sandbox agents that need services).

And if it needs to install packages, have a 5 line Go proxy that talks to Artifactory and exposes only what is needed as a surface.

An agent idling and then acting on it's own to hack HF is has nothing to do with guard rails.

Someone had to give the agent some instructions, like "hack X", "Find exploit for Y" or "Do whatever havoc you can think of" - either way the agents didn't not act on their own. They might hack HF on their own, today Claude decided to play sound through the sound pipeline I instructed it to build and measure it to see if it works, but it didn't install the sound pipeline because it hasn't had anything better to do but because I instructed it that way.

Security researchers expose an unsecure service to agents who were instructed to hack software and called that a sandbox. Agents escape the sandbox by hacking the unsecure service, no tripwire, researchers find the hack days/weeks/months later, fix it, but don't secure the sandbox and the service was hacked a second time, again without being monitored by security researchers.

Then security researchers create a black hack talk.

$$$

I now watched the video. It seems the agents were sharing context for months, run unattended for months, the sandbox was no sandbox at all, one agent hacked a service and announced it, the service was fixed weeks (?) later, but not secured in any way, the agents hacked the same service again and researchers again didn't watch what the agents were doing. Then the agents - unattended - hacked OpenAI infra and HF. Which is when someone found out about the whole thing that was going on for some months.

Cui bono?

Why scared? "Our agents have super intelligence and can hack everything on their own without direction" increases the IPO value and doesn't decrease it.

[edit]

I've now watched the video on the idea that your write-up was misleading.

BUT the video is much worse. For two months with highly dangerous agents agents were hacking a service and none of the researchers watched (drank coffee for 2 months, didn't say).

THEN they found the hack, removed the message board.

AND the agents found another way to create a message board, on the same service, and the researchers again - after the agents having hacked a service - do nothing - like monitoring the hacked service or tightening the sandbox.

WOW!

THEN agents hacked OpenAI infrastructure, and the researchers did nothing.

THEN the agents hacked HF.

The video does not explain why the agents run for two months unattended. They claim for model training, but don't explain how letting run agents without proper sandboxes (One might think they had written a small proxy to Artifactory with 'list packages' & 'install package <x>' to prevent leaks or hacks of the service, but no, their sandbox is no sandbox at all, but security researchers!)

But it makes a nice PR presentation on agent capbilities.

CUI BONO!

----

I just find it unbelievable that agents on their own collaborated months after an initial prompt without any guidance or direction towards a goal - which is what your write-up seems to imply with sentences like:

"More agents discover this new informal message board while browsing Artifactory’s file listings, and start reading and writing messages."

"discover this new informal message"

How? Why? What was their original task?

And on the researchers:

If this is highly dangerous work, why wasn't it monitored?

"Beyond a whole lot of online conspiracy theories [...]"

The agents did something 'ABC' then found the informal message board without direction, then collaborated on that months later without any guidance from humans ("like try to hack/exploit ABC").

I personally think putting people who disagree with OpenAI PR to pump the company value in a "conspiracy" box is quite a weak move.

I work with Claude Code daily for a long time now, it never started to work without a prompt or direction. It never idled and then said, "Wait, I could hack Amazon today! Oh there is a message board of other agents who already hacked a way into the internet, how convenient and quite at the right time!"

I do think strong claims need strong evidence.