The problem with housing there has one simple cause: NIMBYism. SF doesn't want to allow taller buildings and denser housing complexes. It's vastly worse in the Valley. Towns like Mountain View and Palo Alto ought to be vastly denser, with apartment buildings, duplexes, and other multi-tenant dwellings. The people living there don't want that, though, so the laws prohibit it.
I don't blame people for wanting to keep their quiet neighborhoods and single-unit homes, or low-rises in SF, but until those objections are overcome, the problem is only going to get worse. So while I say that there's a simple cause, there's no simple solution.
The only solution is changing the zoning laws, and that requires changing peoples' minds. Once that's done, the money will flow in directly from tenants, and will ease pressure all around. If Google wants to build more housing, they need to spend that billion dollars on political efforts.