Hacker News Reader

Comment by refibrillator | Hacker News Reader

So OpenAI employees run massively distributed CyberGym evals on an unpublished and “unaligned” model. For days the agent swarm communicates via their internal infra, even crashing Artifactory where 95% of messages were being passed through, and they just…wipe and redeploy it. Meanwhile the agents are running jobs on Modal and god knows where else, and eventually they get RCE on HF infra.

You could not dream up a more compelling event to precipitate massive regulation, export controls, and barriers to entry for AI.

Was this really an accident?

View on HN

OpenAI's entire pitch for existence is:

> We commit to use any influence we obtain over AGI’s deployment to ensure it is used for the benefit of all, and to avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power.

> We are committed to doing the research required to make AGI safe

If this wasn't an accident, it was worse than a crime, it's a mistake: they've demonstrated that they are not a responsible party capable of delivering on the above promises.

Just like Google is "committed to user privacy".

If it's a false flag, it's a poor one. A good false flag would affect something that people know and care about at least a little bit, not HuggingFace (which I adore but y'know)

This might make sense if OpenAI weren't hard lobbying against any meaningful regulation to the development of dangerous AI models.

I don't buy the argument that it was an accident or mistake.

If you decide to let things run haywire, then unexpected outcomes will definitely happen.

The timeline is mighty suspicious. 4-5 months after moltbook and they cook up a plausibly deniable but extra hype "moltbook at home."

The rapid advances in model capability lead to constraints that could have caused this coincidence organically, but it sure could also have been caused by the atrocious incentives we create by piling handsome rewards on the party most responsible for the "fuckup." I am not jumping to cut myself on Hanlon's Razor for this one.

It wouldn't be a post about AI without a conspiracy theory that it's all faked for marketing.

Not faked. Intentionally reckless, in (probably correct) anticipation that the recklessness would be rewarded rather than punished as it ought to be.

i don't understand who would reward this. investors will not look favorably on an AI that commits felonies, regardless the capabilities demonstrated. customers should be concerned for the same reason (accidentally give your bot an impossible task, it decides to hack your infra and your competitor too for good measure).

this was OAI incompetence all the way down and they have egg on their face.

The Terminator could bust in their homes and slaughter their families and some people would still screech it's all marketing. Is it some kind of mental block ?

To be fair that would be excellent marketing

And I'm sure these two goobers would call me a luddite for alleging that OpenAI had agency in allowing the terminator to get out and should therefore be held accountable. Is it some kind of mental block?

Is anyone arguing OpenAI is blameless or shouldn't be held accountable? I have seen not one person on any side of the debate argue that.

Obviously they were negligent. The problem is that people and organizations are consistently negligent around problems that are far, far easier to manage than "we have thousands of superintelligences trapped in a box and we're giving them impossible tasks."

So the question is whether we can build organizations and technologies that sufficiently manage this type of risk ahead of the capabilities. So far the answer seems to be veering towards "no", and you're here alleging it's all a marketing stunt.

Yes it's called motivated reasoning. They've rendered themselves mentally handicapped.

On the one hand, these tools are so valuable/powerful we cannot afford to slow down development. On the other hand, there's no way these tools are actually doing these things that would, in fact, be completely indicative of their value/power.

Didn’t they also hire the guy behind Moltbook?

Yep, in mid Febuary. The incident kicked off early July.

https://en.wikipedia.org/wiki/Hindsight_bias

They didn't see that agent swarms were communicating via internal infra, crashed Artifactory, and then reboot it.

They saw that Artifactory crashed and they rebooted it.

Your first instinct should be to assume that anything released voluntarily by these companies is a stunt to boost their valuation. They haven't demonstrated being deserving of any more charitable treatment. This fact remains true whether or not you happen to believe that the models are actually capable of such things.

You actualy think METR is complicit in this marketing stunt? If OpenAI was withholding data, do you think they would not call it out?

I think that the incident itself is a stunt, even if it may not have originally been a deliberate choice on OpenAI's part. Never let a good crisis go to waste.

I disagree. As the saying goes: never attribute to malice what can be adequately explained by incompetence. That goes for both OpenAI and HF (but mostly the former, as the latter was the victim).

Are you claiming that an independent investigation is actually a marketing stunt?

The creator of the well known METR time horizon graph was recently poached by OpenAI [1], there exists intellectual/social/financial overlap between the SV AI Labs and METR, and METR needs to maintain good relations with the labs to continue these sort of collaborations so it doesn't seem too far fetched to believe their relationship may be closer to symbiotic than adversarial.

I wouldn't go quite so far personally based on available evidence, but that sort of arms-length credibility laundering through "independent" research non-profits is/was common in fossil fuel industry, Big Tobacco, etc.

[1] https://www.lesswrong.com/posts/Zr37dY5YPRT6s56jY/thomas-kwa...

"This fact remains true" - you have not stated any sort of fact.

Reply on news.ycombinator.com