This is the problem: "High rents reflect the success of California’s businesses—but also decades of low investment and over-regulation. The California Environmental Quality Act, passed in 1970, aimed to ensure that environmental concerns got a proper hearing in planning and development. In practice, the act has become a NIMBYs’ charter. Four-fifths of all suits filed under it have sought to stop infill development in cities (ie, on land already zoned for building) even though this usually has a smaller environmental impact than building on green fields. California’s development and impact fees are about three times higher than the national average. Zoning laws and parking requirements are onerous, too."
SF isn't even the worst offender compared to cities like Belmont and Palo Alto. While there are real safety concerns that limit development since 1. we're in earthquake country, and also because of 2. some Superfund sites in parts of the Bay Area; most of the problems just stem from current homeowners of single family homes (NIMBYs), who do not want high-density housing near their house. Until this changes, no reasonable amount of income assistance will solve this problem for the poor or the middle class. There's just too little usable land here to be exclusively used for traditional single-family homes.
This has more detail about the problem in SF: https://www.sfchronicle.com/politics/article/How-powerful-is...