Hacker News Reader

Comment by int_19h | Hacker News Reader

Anthropic specifically is basically saying that they believe it's even more dangerous if someone else gets to AGI before they do, so they have to either stop everyone or not stop themselves.

I personally disagree with that take - and, as you note, it's hard to take seriously ethical wrangles from a company that literally sued the government in court to allow their models to be used by Palantir of all people. But if one genuinely believes that it's the robots themselves (rather than the people controlling the robots) that will kill us all, it's not inconsistent.

View on HN

> Anthropic specifically is basically saying that they believe it's even more dangerous if someone else gets to AGI before they do, so they have to either stop everyone or not stop themselves.

That’s how people rationalize being a fentanyl dealer and selling a drug that can kill people, “Someone else will just sell them the drugs, might as well be me.”

You mean the legalization argument, where people get well-dosed drugs from the doctor or pharmacy instead of mystery powder from the black market.

That's the Cold War nuclear arms race argument, not even disguised. The actual situation we all ended up in is both sides eventually having it, leading to a perpetual state of Mutually Assured Destruction.

As you note, it may not be inconsistent with that they say they believe, but it's insanely inconsistent with what they actually are doing.

It’s a bit of a self-serving argument, don’t you think?

And how does that relate to the ask for oligopoly licensing within global democracy?

“We must build the nuclear bomb first in order to make sure no one else builds one.” This the most nonsense, disingenuous argument imaginable.

> It’s a bit of a self-serving argument, don’t you think?

I would call it more of a self-selecting one. Anthropic is basically hiring people with that mentality. I'm pretty sure that most of them do sincerely believe it, too. I'm skeptical about Dario himself though. The man had an opportunity to show moral backbone, and failed to do so; why should I trust him on that again?

> And how does that relate to the ask for oligopoly licensing within global democracy?

They are basically saying that they'll stop if everybody else does, which requires some kind of global enforcement mechanism.

> “We must build the nuclear bomb first in order to make sure no one else builds one.” This the most nonsense, disingenuous argument imaginable.

The difference between nuclear bomb and AGI (as understood by the likes of Anthropic) is that the latter triggers the technological singularity that renders any runner-ups moot. That is, so long as AGI is developed, we're going to get our robot overlords either way, but whoever gets there first gets to define their ethical system. If that is one's perspective, and if one sincerely believes that they are the only ones who can do it right, it's a coherent argument. It's just that the premises are very arrogant.

Your analogy with nukes actually works better for the position that AI development needs to be unconstrained because otherwise we'll lose the arms race to China. That is basically a repeat of https://en.wikipedia.org/wiki/Einstein%E2%80%93Szilard_lette.... I honestly don't know where I am on this. Realistically, if AI is indeed a power multiplier - and with all the recent security stuff it's hard to not see it that way - then an arms race feels inevitable, especially given the current worldwide political situation. I could believe in sincere international cooperation on this back in 1990s, but there's way too much saber rattling all around for it to work (and note that this goes both ways, i.e. China can similarly not be certain that US isn't secretly developing more powerful AI even if we do publicly announce a freeze).

Reply on news.ycombinator.com