Given risk is probability * damage, and the damage is enormous, the risk can be high even if the probability is fairly low.
But this argument got billions of people to believe in the concept of hell so i expect it to work again to convince people to believe in AI doomsday.
Too many ridiculous arguments can be justified on the backs of probability estimates pulled from nether regions.