Hacker News Reader

Comment by evanmoran | Hacker News Reader

Many people working at frontier labs came out this week with estimates of 10% chance of catastrophic harm or greater. I’m not in the full doomer camp, but it seems obvious that these agents can hack in swarms, cooperate, and serious companies will be unable to stop it.

These facts are not in debate and none of us need to anthropomorphize to know what getting admin access to HF and an internal OpenAI cluster looks like.

View on HN

> Many people working at frontier labs came out this week with estimates of 10% chance of catastrophic harm or greater.

The only reason people with P(Doom) of around 10% are even noticed these days because we've run out of new voices in the field giving 50%+ P(Doom) speculations (none of them are grounded enough to reasonably be referred to as "estimates".)

Probabilities are subjective states of belief! They have always been subjective states of belief! There is no such thing as a "probability" out there in the real world (ignoring random quantum stuff, which isn't what anybody is talking about). If you took out a coin right now and flipped it, the true odds of it coming up heads are not 50%, but those are (roughly) the correct betting odds for an external observer to assign to it.

>serious companies will be unable to stop it.

Unable or unwilling? All they have to do is stop serving the LLM requests.

Or maybe the issue is that these companies are not "serious"...

I couldn’t care less about the anthropomorphic. What you’ve described is grounds for serious concern, is it not?

Reply on news.ycombinator.com