I would point the finger at NIMBYism and the fact that new developments in places like San Francisco, London, and New York have to hop through many many hoops to get built (and pass muster with a surprising number of "stakeholders").

Here is just one example...

http://www.bizjournals.com/sanfrancisco/blog/real-estate/201...

The TL;DR of this article is that a large new apartment block in SF is being held up by a construction union because the developer is refusing to hire union labor for every contractor role in the project.

Maybe we should reduce the red tape around building new structures and see how far that gets us before plowing more money into public housing and other government interventions.