Hacker News Reader

Comment by cowthulhu | Hacker News Reader

Everyone writing off AI-related risk in this thread should think about what would need to happen for them to consider AI a serious existential risk. It can be outlandish and unlikely (close call with a bioweapon? unsupervised persistence in a data center?) but people should honestly call their shots and then stick with them.

View on HN

No. The person making the improbable argument is required to provide an affirmative case for that argument.

It's not my job to make the argument for them.

For what it's worth, I have done your suggested exercise, and I find every causal link (including the ones brought up by luminaries like Amodei) to be outrageous and poorly argued. But it's not my job expend effort to make their outrageous arguments better.

I'm not making an affirmative case - I'm looking for others to introspect about this, and anchor their expectations.

If we have a bioweapon close call, would you consider AI an existential risk at that point? If not, how close would we need to get?

You don't need to answer here, or disclose anything publicly - just think and remember.

I don’t really understand this exercise of predicting what I’ll be thinking in the future.

To avoid shifting goalposts

There are no goalposts. No one is trying to score points except you.

I think I'm coming across as confrontational, which is genuinely not my intention - sorry about that. Not trying to score points or anything, just encourage people to introspect a bit :)

I think your viewpoint is totally valid, and I'm not trying to argue against it.

Not confrontational, but proselytizing.

Is there truly nothing an AI system could do (no matter how outlandish it seems today) that would give you pause?

Yes, just like that. What would it take for you to accept Our Lord into your heart?

The bar is high, but there's probably some combination of events (combined with a high confidence in my own lucidity over an extended period of time) that would convince me.

Same same.

But it doesn't do that. "Imagine how you'll be wrong in the future" is just as shiftable as anything else, and has the distinct rhetorical advantage of being unfalsifiable.

The substantive difference is that you're asking me to dream up ridiculously improbable scenarios, which is probably the actual point. Just like "The End is Near Accept Jesus" guy on the street corner gets what he wants as soon as I engage.

To quote a famous movie about doomy AI scenarios: "the only winning move is not to play."

> If we have a bioweapon close call, would you consider AI an existential risk at that point? If not, how close would we need to get?

The thing is, this doesn't really mean anything.

What is a bioweapon close call? What is the process in which a bioweapon close call happens? An AI that hacks all cell phones to emitt anthrax?

It can be subjective, because it's internal to you and not something that gets litigated or whatever. A close call is anything that you personally consider to be a close call.

I'm not asking you to post it here, I'm asking you to think about it and then remember it in the future, should the event ever occur.

No. Why? If you want to convince people something is true you need to actually convince them it’s true. They don’t need to justify not believing you.

I'm not asking for justification, just for people to introspect about what, if anything, it would take for them to consider AI a serious threat. No justification needed, and really no need to post it here, just think about it and remember it in the future.

At this point, the most likely scenario for AI producing a serious existential risk is making us all want to bash our heads in because of "the honest options" and other claude-isms.

Reply on news.ycombinator.com