The super-AI is going to have power. Deployed everywhere, used by millions, etc.
You have two choices:
- It can potentially lash out in an alien-like way.
- It can potentially lash out in a human-like way.
Do you understand why this has no effect on the argument whatsoever? You are just introducing an irrelevant observation. I want the AI to behave like human always, no exceptions.
"What if it's a bad human"
Jesus. If people make an evil AI, then it doesn't matter anyway how it behaves, it's just bad even before we get to the discussion about how it fails. Even when it accomplishes tasks succesfully, it's bad.