I know you're not attacking Islam, just prefacing that I am not.

Mecca was never important. It was symbolic but that's where it ends. Ignore the Ottomans, even the first few Caliphs, the minute they left Arabia, no one cared about Mecca beyond symbolism.

That you see espoused as "religious" is almost always BS. Its all politics and economics. You cannot develop anything in a desert. Why would you focus on a desert when you can focus on much richer areas by the Mediterranean with already a lot of culture and resources and wealth?