One question I'm not seeing asked is, other than the wealth of interesting jobs for those in tech, is SF even worth the price of admission as a city? My experience here so far says "no fucking way".
As someone who has never lived there but has probably spent close to a year of his life collectively visiting there, I'd say it's a mixed bag--even though it's clearly on my list of favorite cities. Pretty good climate (though not as much so as the South Bay), great recreational opportunities in the area, great food, pretty walkable although public transportation is a mixed bag, and certainly some areas are nicer than others (I'll leave the last point at that). As an urban experience (as opposed to the Bay area more broadly, there are probably other cities I prefer. But I definitely see the attraction even if I've never succumbed to it, or indeed living in a city, myself.
Seems people tend to conflate the price of admission with desirability. There are plenty of reasons why rents can rise much faster than inflation, other than desirability increasing.
Yes, things like economic opportunity
Current biggest reason is likely that mortgage interest rates are being minimized by gov't so that banks can unload the houses they're holding at higher prices. When house prices rise, rents tend to as well.