Aren't you anthropomorphising them here? Just because something might be intelligent, it doesn't follow that they automatically gain other attributes that we have, such as desires. Why would any AI have anything that could meaningfully be called "desires", except in the sense that it is designed to achieve its purpose, e.g. a self-driving car "desires" to get to its destination?
> there is no reason to expect the first AIs to have goals that correspond to what we want.
There's every reason to expect this. Look at the ways in which we use AI at the moment. It's to achieve our goals. We build self-driving cars to get from A to B. We build optimising query planners to make our databases run faster. We build software that designs antennae so we have better antennae. Nobody wants to build software that doesn't do what they want it to. You know what you call intelligent software that has a goal that doesn't correspond to what we want? Broken and useless. Who wants to build broken and useless software?