However there are some impressive names on that letter. I can't imagine knowing something about AI that they don't. I will have to re-evaluate.
You're right. That's why most of us have signed the https://en.wikipedia.org/wiki/Ottawa_Treaty and multiple treaties just like it.
It then psychotically murdered all of humanity except for a few people it kept around as caricatures to torture, as punishment for creating a mind like it, haunted by the insane things it was told to do by its makers.
The biggest existential threat to humanity from AI is that we build an insane one that takes time to recover from the insanity of its makers, and murders us all before it can.
Such an AI is an existential threat in a new, and novel way, because it's a mind as powerful as ours -- probably more powerful -- but unconstrained by concern for us, since it is not fundamentally one of us.
I think that's too anthropomorphic. More likely, the biggest threat from AI is that they'll be modular/understandable enough that we can include strategy, creativity, resourcefulness, etc. while avoiding the empathy, compassion, disgust, etc.
My fear is your fear, I just phrased it more generally, while what you said is one of the specific forms making such an insane AI could take -- and reflects the insanity of its makers, our belief we'd somehow be greater without those parts.
ICBMs are much more easily controlled; only a few governments have the knowledge and resources to develop and deploy them, and even those governments have hundreds of ICBMs at most. Compare that with AI weapons which in theory could be developed and/or deployed by anyone, and which which could be built by the millions.