Land is there it's just being used wrong.
It’s way cheaper to maintain.
I'm actually surprised by how many people here seem to be in favour of packing people into ever denser cities simply for the economy benefits of doing it. I suppose it's just the demographic of people who browse HN.
When you look at pictures of old housing units, it's striking how small they are in comparison to newer construction. As people become wealthier, they demand larger living spaces, which means they want to spread out. Now you can offset some of that spreading out by going vertical, but that's extremely costly (space needs to be set aside for stairs, elevators, you need stronger walls, more complex construction, HVAC, etc).
Although people like to focus on things like skyscrapers, but those are really rare even in places like NYC. Going vertical is mostly for luxury construction, at least in land rich places (not e.g. Hong Kong).
In most places, building lower to the ground is more cost effective so as wealth increases and transportation technology improves, cities become less dense.
There is a study about the secular decline in urban density, showing that cities get about 2% less dense every year, and this trend goes back to at least 1910. So no, it's not just zoning, but in general people prefer lower density.
The study is by the Lincoln Institute, my favorite think tank dealing with land use issues:
https://www.lincolninst.edu/publications/working-papers/pers...
"Using satellite imagery, census data and historical maps, we report on density variation among cities the world over. We find significant differences in the average population density in the built-up areas of a global sample of 120 cities: In 2000, average density was 28±5 persons per hectare in cities in land-rich developed countries, 70±8 in cities in other developed countries, and 135±11 in cities in developing countries. We also find that built-up area densities in this sample declined significantly, at an average annual rate of 2.0±0.4 percent, between 1990 and 2000. We report on the five-fold decline in average tract density in 20 U.S. cities between 1910 and 2000, at an average long-term rate of 1.9 percent per annum,"
Some people see housing as more a means-to-an-end and like the city life. Other people hate the city and want green space and bonus rooms.
The problem is that America is getting more urban (just look at the 2020 census) but our housing stock hasn't build the townhomes and condos needed to support it.