This is highly dependent on the region. I have lived in both the Bay Area in California and in Arizona. They are like studies in fairly extreme opposites. What you talk about is very much how the situation is in California, even in areas where more houses could be built, its always a huge political battle of attrition.

In Arizona, anything goes, they build shit literally anywhere, its flat (for the most part in Phoenix at least) and land is almost literally endless. Property is dirt cheap, houses are too and they have a huge overabundance of houses, I would say probably at least 1/5 of houses in the metro area there are currently unoccupied.

Areas that are desirable and have geographical characteristics that make usable land scarcer (which includes being near a body of water as that limits the land in one direction) are probably always going to have the characteristics of what you describe, IMO. One thing it does lead to is more valuing of older buildings, etc as these areas don't have the "tear it down and build something new" mentality of a place like Arizona.