So if AI ever takes over the world and kills us all, it will probably be because it failed to understand what we actually wanted.
Just because we created the AI doesn't mean it'll care about us. That's like saying a maths problem will try to make your favourite number its answer, just because you wrote it. No, you are the one who must make its answer the number you want. It won't happen by chance.
Corollary: you can't patch broken FAI designs. Reinforcement learning (underlying basically all of our best AI) is known to be broken; it'll game the system. Even if they were powerful enough to understand our goals, they simply wouldn't care; they'd care less than a dolphin. https://vkrakovna.wordpress.com/2018/04/02/specification-gam...
And there are far too many people in academia who don't understand this, after years of writing papers on the subject.