And what does charisma of AI alignment folks have to do with anything?
Nuclear weapons proliferated explicitly because they proved their scariness.
Anyway this is a good example of the completely blind-faith reasoning that backs AI optimism: we’ll figure it out “because it’s what we do.”
FWIW we have still not figured out how to dramatically reduce nuclear risk. We’re here just living with it every single day still, and with AI we’re likely stepping onto another tightrope that we and all future generations have to walk flawlessly.
We have figured out stuff in the past, but we also came shockingly close to nuclear armageddon more than once.
I'm not sure I want to roll the dice again.