If by "modern society" you mean colonizing the American South-West and California, you are probably right. Those places are deserts and semi-deserts and should have been left sparsely populated, but a combination of corruption, American exceptionalism gave us this.
Thus said, there are plenty of places around the Mediterranean which have always been pretty warm in the summer, and where there is a convention of closing down between noon and 4 PM, but spreading the work-day longer during the evening and the early morning (think 8-12 4-8 instead of 9-5). So I strongly suspect that there is much about this that is a matter of culture and habit.
(Not to mention that your use of modern is pretty wrong historically, since AC is a contemporary invention, definitely not modern, but I'm digressing...)