Whatever the big picture national issues are, there is no reason the Bay Area should be lagging even compared to the rest of the US. Rich suburbs of tightly packed single family homes are prime fiber territory. Those are the very first places Verizon wired up with FiOS 15 years ago.
I live in NY, and each town or city gets to negotiate franchise agreements that always favor the telcos. You have to wield a big stick with these companies, who divide and conquer.
In my cities case, in exchange for some bullshit (public access tv and filming equipment for the high school) gave Spectrum an exclusive contract for TV.
So Verizon brought FIOS to every town around us, and we’re stuck in the internet ghetto.
The bay area needs to be torn down and rebuilt, but the NIMBYs won't even let adequate housing be built so the chances of that happening are about zero.
If cable internet was still stuck at 25 Mbps or similar, the pressure for fiber internet would be a lot stronger.
But cable internet got good.
The backwards and scandalous aspect of modern American internet is that the ISPs managed to roll out 1 TB data caps without being blocked by lawmakers.
Most consumers aren't willing to pay the monthly cost that ISPs would be need to charge to make overbuilding fiber economically feasible. DSL providers overbuild with fiber because they can't compete with cable and the rising cost of maintenance of a copper plant. Some cable providers will do new builds with FTTH or overbuild when upgrading the existing plant is cost prohibitive. But overbuilding a working coax plant almost never makes since. Spending millions of dollars to offer higher speeds at the same price just doesn't make business sense.