I have a profound understanding of my puppy's psychology, motivations, and capabilities; I even exercise complete control over her physical environment, and yet she ate my fitbit strap (again, goddamit!) as I was typing this very comment.
It's like arguing that humans with all their intelligence are still useless in controlling the situation when confronted with a Tiger in the wild. So clearly the tigers should relax and stop thinking about human-alignment. Tigers can always just switch off the smart humans if they were to try something. The part where this falls apart is "always."
It is terrifying to think that we might one day be puppies to AI as puppies are to us. It is not reassuring at all that we might be able to make things harder for the AI in question every now and again by nibbling at them.
Same goes for the Emu war. I'd like to not be the Emu one day to AI. Even if the AI struggles completely and hilariously to control me, there's a huge power differential here. I'd much rather be the human failing to kill Emus and face my own embarrassment than the Emu going guerilla-style fighting for their life.
I don't get why so few have pointed out why these specific arguments for "don't sweat AGI risk" are so weak.
The delta between a puppy and a human is also a lot smaller than a human and a super intelligent AGI and humans can effectively train dogs.