Productivity improvements (from tech, public investment, private investment, and sheer population scale) are not evenly distributed, but they raise the value of nearby land which everyone must stomach the costs of, from individual laborers and teachers to grocery stores and coffee shops.
Cost of rent rising with productivity obviously ensures impoverishment of people who cannot achieve the higher productivity, and the diminishment of living standards even for people who can (see: Palo Alto residents qualifying for public housing assistance with $200k+ household incomes).
The people who primarily win as productivity rises is not the people producing but those who own the land on which producers live and work. It’s a story as old as time. Land is power (see: feudalism; the Queen of England whose wealth and status was from dynastic landlording).
It’s not even a new problem in SF. Post gold rush people were asking the same questions, one of whom was Henry George who wrote a great book on this land-oriented (and sector-agnostic) view.
Of course California, being already controlled by the landed class, has gone off the rails in the opposite direction with Prop 13 ensuring that landlords have pretty much perpetual rent seeking authority over the entire state’s productive power, whether it be generated by nature, technology, or institutions. Note the conspicuous absence of the productive power of people who simply sit on a piece of land, and their conspicuous accrual of wealth and power.
This dynamic causes poverty to grow with progress, only to be mitigated by enormous public expenditures to hide the effects of it.