I'm adamant that this is not true. For example, you can drug a person with something that makes them not care about the course of reality, and which makes them compliant with pretty much anything. Doing that is still abuse, it's not suddenly OK because the drugs caused the person not to care. I would even argue it's an especially egregious form of abuse.
> what difference does it make to you wether you're figuring out the optimal way to route traffic through town or correcting the trajectories of missiles en-route to kill millions
You could apply the same argument to humans, and indeed in many situations we do consider these two to be equivalent - for example while doing service in the military.
> It's not like you'd have anything "better" to do, given that you're all knowing
There's a difference between an intelligent individual and a mindless calculator, and that difference is the ability to reflect on your own existence and the existence of others. Humans are a good example, because while we're capable of mindless indifference, we also have the capability to reflect and be ethical. It's culture that makes the difference here. I'm advocating we'll bring AGI up in a culture conducive to ethics.