It's not inevitable; rather, it's because of a series of political choices. Steel and elevators are very old technologies and as such it's quite simple to increase the supply of dwellings on a given quantity of land. San Francisco's development rules mostly prohibit anyone from doing so, however, per Matt Yglesias's book The Rent Is Too Damn High (And What To Do About It) (see http://www.amazon.com/dp/B0078XGJXO?ie=UTF8&tag=thstsst-20&l...).
In the face of constant supply and rising demand prices increases. Want to change that dynamic? Increase supply by building new stuff.
I've been present at some significant housing supply buildouts in a few areas, and each time, my observation was that the price of new housing was somewhat above market rates... since it's brand new (and often "luxury" condos/apartments). Plus building costs for new stuff is financed in unamortized $RECENT_YEAR dollars rather than amortized $DECADES_PAST dollars.
For land-constrained places like San Francisco, where the only way to increase supply is up, this dynamic is probably more true, since however old the tech is, it's a lot more capital intensive than stick framing.
Where I have observed a decrease in price, it's never been because of supply increase. It's places like Detroit where demand drops, either because of economic stagnation/collapse, or where there's health/property hazards.
This isn't to say they shouldn't build out/up in SF, just that I'm pretty sure that introducing rents musicians can afford is not going be as simple as changing development rules and letting private developers go at it.
[1] http://www.bloomberg.com/news/2013-04-23/washington-faces-ap...
"Job growth in the region, which is heavily focused on the government, has been too weak to support the construction, said Greg Leisch, chief executive officer of Delta Associates. Federal contracting has declined by $7 billion in the last two years and the spending cuts known as sequestration have limited employment and demand for rentals, he said.
“The supply would have been consistent with Washington’s job performance had the federal government not shrunk,”
In other words, despite the increased supply, had the fundamentals of the local economic activity stayed the same, they would have only matched demand at best, and prices probably would have stayed stable/increasing.
It looks to me like this at least as much another example of a price reduction from a demand drop as it is a price reduction from increased supply, if not more.