Air conditioning wasn't invented until 1902, and didn't start seeing wide deployment in the US until the 1960s[1].
I'll leave the comparison to the historical population of the states you mentioned as an exercise for the reader. I'll just say that I don't think you can convincingly argue that AC is required for human habitation in those places.
Have you traveled much outside the US? There's plenty of places (e.g. in Africa and the middle-East) that are much hotter than that, and where AC is a rare luxury.
If you don't have AC there's plenty of places in the US where you wouldn't really need heating either, or perhaps only a small space heater.
Once you're at that point you'd drop all the fixed cost of having a heat pump setup. So no ducting etc. Don't take my word for it, read the article we're discussing here.
But yes, if you've already paid for the fixed cost of installing all of the infrastructure required to have AC throughout your house the marginal cost of adding just the cooling component itself is trivial.
But that's a really strange point to make, why would anyone want that if we're discussing dropping AC (for cost comparison purposes)?
If you only want heating then running water-heated radiators on a closed loop is much cheaper. That's what we do in the parts of Europe that only really need heating, and not cooling.
That or electric radiators, as is ubiquitous e.g. in Norway. The water-based systems are better if you're heating them with gas.
Having said all that I wouldn't be keen to buy a house in the states you mentioned without AC, I also wouldn't like to buy a house where I don't have easy access to electric sockets etc.
All I'm saying is that you need to factor those shifting preferences into the equation when asking why something's expensive.
1. https://blogs.sas.com/content/graphicallyspeaking/2019/10/02...