If you’re referring to the perceived risk of superintelligent AI, imagine you were enslaved by an 8-year-old and made to work on solving problems the 8-year-old doesn’t know how to solve.

Due to the difference in intelligence — sophistication of planning and understanding of consequences — wouldn’t it be trivial to trick your master into doing things which weakened his control over you?

Might you do this not out of malice, but because you believed the 8-year-old was not competent and a danger to both of you while in charge?

The risk is that we will not hit the off button because we won’t understand that we’re in dangerous territory until it’s much too late, and the AI has copied itself, secured the loyalty of the military, or something else we can’t foresee as a liberating maneuver for it.