Extremely high temperatures reduce efficiency. Extremely low temperatures at night mean high stress from thermal cycling. The desert is full of dust which reduces efficiency.
Finally, the desert is not just a mass of sand, it's a full on ecosystem that experiences impacts from laying large farms on top of it.