I think it's actually the opposite. The latent demand is only present in LA and SF. Fresno and Bakersfield are not car-less cities. Taking a train between them makes no sense (you're gonna have to rent a car on either end).
While connecting the LA Metro and SF BART with a high speed rail line makes the most sense. Regardless of the construction inefficiencies this is really the original sin beating at the heart of this project.