Does not anyone worry about this whole human level AI Pandora’s box trap? Why not work on nukes themselves, sounds safer to me
Unlike software, which can be mathematically airtight, there's no reason to assume that physical devices built by humans are unexploitable in the face of the massive intelligence of the AGI. So the AGI will just ask itself which is easier: delicately performing surgery on its own sensors, or building tons of nukes.
Honestly, that people are even worried about AGI doomsday shows to me the power of narrative. Everybody has heard of SkyNet and the sorcerer's apprentice, nobody has heard of the AGI who outsmarted itself by creating digital porn for its objective function. Therefore, through the miracle of the availability heuristic, AGI doomsday it is.