There's also a lot of open area in the yards and air 10-30ft above single-family homes. There are lot of property owners who'd be willing to demolish the SFH on their lot and build a lowrise multi-unit building with 2-6 units, and the only thing stopping them is that it is illegal to do this.
I have a question that is tangentially related to the OP, but not based on the irrationality of the housing market. What I want to know is: given the way things are today, how do you identify and locate neighbourhoods, towns, and cities where housing will appreciate in value the most? IOW, what is the data you need to make a good "buy low, sell high" decision?
Demand:
Interest rates(good luck predicting that)
Median income(predictable)
School districts (good schools attract people who care about that... as you can imagine that's a spiral one way or the other)
Supply:
Current inventory
Potential inventory
One more factor in the demand side: population. IMO it is hard to predict.
For a given area, population isn't really a demand side thing - it's the intersection of demand and supply.