Commercial and residential buildings have different building codes. It would take years/decades to get approval and actually retrofit buildings. Cities like San Francisco chose to suppress residential housing and focused on commercial buildings because it was cheaper and required less infrastructure and made the city more money through payroll taxes. I'm glad they are being punished for their poor long term planning. Hopefully other cities learn from their mistakes and start to allow mixed use zoning, etc.

Zoning is one issue but the big problem is converting commercial buildings to residential is often just as expensive as building new residential from scratch.

I don’t understand the push to convert to residential. Absent proximity to jobs, who’s clamoring to live downtown in just about any city?

Maybe downtown SF is still “cool” (for whatever value and age group you want to assign that to) and I just don’t know the Bay Area as well as my own city.