They could do it. But there are limits to the practicality of what they can do. It would kill the local economy in an instant. All those foreign workers are at the core of the productive economy, doing actual work that matters, including much of the dirty and mundane stuff, as well as pretty much all of their construction work. I'm sure it's possible to replace them all if they so choose, but it can't be done in an instant. Dramatic and disruptive changes like that take time, or else there will be huge problems. Replacements will need training and experience. The economy as a whole will need to adjust to the population change, and there will be all sorts of other second- and third- order effects.
It's not just unskilled workers. They rely on many foreign skilled workers too. Two acquaintances of mine are expert concrete engineers. They go out there to work on these building mega projects like the Burj Khalifa, as well as ones in KSA. There is little local engineering expertise; it's all brought in. When you're mega rich, it's easy to do that. It would be better to develop their own expertise and industry and have a more balanced economy, as well as a more secular society that would value such careers, but that takes time as well as the desire to change their economy and society to permit it. It's not clear they have the desire or will to do so.