AI existential risk probabilities are too unreliable to inform policy

aisnakeoil.com

I’ve totally given up trying to have this debate, and everyone else senior in AI in practice seems to feel the same

People want scary powerful terminator/skynet like AI cause, like dictators, they think democratized advanced technology will give them the power they always wanted to live the dream - despite what we know of history

Ok fine.

Since we can’t agree to limit ourselves, lets go as fast as we can and see what happens. That’s what I’m doing

Probably the way to go. There seems limited risk while we are still able to turn the power off. Also I think people have seen enough Terminator movies to not connect present LLMs to nuclear launch systems.

I wouldn’t make that last assumption

I'm not an expert on the subject, but this seems like a very insightful post. I especially liked the part about forecast skill analysis being impossible for tail risks.

Reply on news.ycombinator.com