What do you think would happen to all that “luxury housing” if there wasn’t all the red tape and we allowed as much as the market could bear?
In earlier eras the answer was it didn’t stay luxury housing, but moved down market as shinier and newer buildings kept coming online, and as overeager developers built more fancy housing than there turned out to be a market for.
Then when you have all those landlords and more housing units than are needed, there’s price pressure all the way down, and a lot of upward mobility as people slide up into slightly nicer homes, leaving a bunch of reasonable quality homes open all the way to the bottom of the market.
We haven’t had anything like free market supply of housing since the FDR era, so this example is so far removed from living memory that most people can’t even imagine it.