The supply didn't decrease. The demand increased. That's the proximal cause.
Are you kidding? Remember the big white buses that everyone is mad about? They play a role here.
The valley tech companies have mitigated the natural barrier that would ordinarily limit the number of tech employees living in the city -- the pain of commuting. So rather than staying close to work and building communities where they live, they use the city as a glorified suburb, but with cushy private transportation. There are a variety of things wrong with this, not the least of which is the fact that they reduce housing supply for the natives.
Econ 101 theories about supply and demand don't apply when large external forces are subsidizing the demand side of the equation. One can quite reasonably argue that instead of "building up" (which is expensive and disruptive) SF should just ban the private buses -- it would be cheaper, more natural, and would solve the same problem.
A huge number of people employed on the peninsula would absolutely live close to work and build communities there, if those peninsula cities weren't so hell-bent on preventing dense development. There just isn't the supply there.
And what the f do you mean by "natives", anyway? Native Americans? Everyone else is an immigrant. If you moved here a year before I did, do you have some more legitimate claim to live here? I don't think so.
Of course supply and demand applies to the situation. Your suggestion of banning private buses, while ridiculous, is specifically an attempt to influence demand. Your own suggested solution assumes that the "Econ 101" theory applies.