Hacker News Reader

Comment by zdragnar | Hacker News Reader

Having witnessed so many people treat LLMs as a something divine, I can only assume the reasonable people at openai and anthropic were all pushed out long ago, and the majority that remain believe the crazy hype despite Tesla-self-driving-level predictions from these companies that don't come true.

I'm not worried about what they think. I'm worried that too much infrastructure- water, power, defense systems, etc- remain running on tech from an outdated era of understanding security.

> they believe no one else will act responsibly, so they must do it themselves, despite the risk.

This genuinely makes no sense. Them getting there first in no way precludes bad actors from also getting there. It might as well be another marketing stunt.

View on HN

> I can only assume the reasonable people at openai and anthropic were all pushed out long ago

Typical uninformed take on the side of "doomers are crazy".

Both CEO's of OpenAI, Sam Altman and Dario Amodei, and many in their leadership, believe AGI has a very real probability of causing humanity's extinction. Both companies were founded upon this belief, it is at the core of the company. Only later were mercenaries hired chasing $1m compensation packages.

If they truly, truly believed that, would they be speeding towards building it? If yes, that would make them truly insane, right? Not as in a quaint "off their rocker" but more "non compos mentis".

Both companies have deluded themselves into thinking the arms race is going to happen anyway and they need to rush to it first, as if somehow that helps. They have publicly stated as such repeatedly. They think the ~10-50% chance of extinction sucks, but that it's going to happen anyway and they believe they can steer it towards something good the best and unlock all the potential positives like infinite life.

Why is the take uninformed? You didn't address anything about the part you quoted, wherein reasonable people were allegedly pushed out long ago.

How is it not addressed? The company never contained only "reasonable people" that believe AGI is not an existential risk to humanity. At both inceptions were people who believed in AGI x-risk, even the founders. Only after time, did there become more "reasonable people" who were mercenaries and only believed it to be a typical tech job. Today, there are more "reasonable people" than ever there. They haven't been pushed out. We're just witnessing some prescient mercenaries smart enough to Eureka the grave implications of what is actually happening.

I'm not saying they are crazy, I'm saying their predictions have a record of not being accurate, and thus give them no weight compared to others'.

In any case, if Altman really does believe it is an existential threat, he must be a misanthrope as he now opposes heavy handed government regulation, unlike in 2015 when he was the only game in town. It's almost like he doesn't actually believe it and just wanted regulator capture.

Before OpenAI was ever even founded, way before any regulatory capture plausible claims:

"Development of superhuman machine intelligence (SMI) [1] is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could." -Sam Altman

Dario discussing AGI Existential Risk in 2014 before OpenAI and Anthropic: https://intelligence.org/2014/01/13/miri-strategy-conversati...

Both companies have deluded themselves into thinking the arms race is going to happen anyway and they need to rush to it first, as if somehow that helps.

If you're a doomer, wouldn't the "let's try to get there so fast" actions of the companies suggest that, in fact, there are no reasonable people in positions of influence there?

By your own description neither Altman or Amodei are reasonable if their thought process goes: "this is an existential risk, give me hundreds of millions of dollars so I can accelerate it."

GP defined reasonable people as not believing in existential risk from AGI. I think the leadership and alignment teams are way more informed and reasonable than some of the very poorly thought through takes like GP just making things up about the labs. There’s just no evidence of it whereas lab leadership are operating under a mostly informed worldview.

Yes, I don’t think lab leadership are totally reasonable as they were concerned about the risks and yet lack of reason caused them to directly contribute to the problem we’re facing. They’re still not being reasonable when they throw their hands up at the arms race and hope the utopia come, when that’s not our trajectory at all and more lab employees are realizing it

Reply on news.ycombinator.com