I mean... Not that many.

I've watched a few documentaries and there are barely a handful of houses let's say in Death Valley. And Death Valley is a lot more hospitable than Mars.

To each their own, though, sure!

Well, frozen deserts like Northern Canada, Alaska or Siberia are not getting all hyped up and people often times are trying to get out from there even that it is almost not possible - see i.e. Norilsk

Mars with its -80 degC is much harsher desert than dirty industrial towns of Siberia.