It's not inevitable; rather, it's because of a series of political choices. Steel and elevators are very old technologies and as such it's quite simple to increase the supply of dwellings on a given quantity of land. San Francisco's development rules mostly prohibit anyone from doing so, however, per Matt Yglesias's book The Rent Is Too Damn High (And What To Do About It) (see http://www.amazon.com/dp/B0078XGJXO?ie=UTF8&tag=thstsst-20&l...).
In the face of constant supply and rising demand prices increases. Want to change that dynamic? Increase supply by building new stuff.