Its a little bit of hyperbole to say it still looks like the 70s when there used to still be a lot of orchards back then. They never could have anticipated this growth
That's silly. San Francisco isn't the first city to boom. Detroit added more than a million people between 1910 and 1930. Brooklyn added a half a million people per decade in the early 1900s.
And neither of those cities look like Midtown Manhattan or Tokyo or whatever else SF residents seem to imagine is required to support fast growth.
They did it by building vast areas of multi-story mid-density mixed-use developments. There is zero mystery here, both about what was clearly starting to happen in the SFBA 30 years ago and what should have been done about it.
My grandfather told me that his commute from Santa Clara to Lockheed in Sunnyvale involved mostly driving past orchards all through the 1960s and into the 1970s. He left the area in the 1980s, saying that traffic and livability had become too much of a problem. It's been decades of intentionally not addressing the problems - nobody can reasonably claim to not have known or anticipated.
Suggesting that people could have predicted the dot-com boom of the 90s or the tech/startup driven boom of the past 15 years (and that it would be so heavily focused in the Bay Area) seems a little far fetched. It might seem obvious now that it has already happened, but in the 60s and 70s I don't see how you could have known (the industries driving development then were Defence and Manufacturing, both nearly nonexistent now). Plenty of areas in the US were increasing in density and economic development during that time, only one of them became Silicon Valley.
But there certainly have been rapid changes, especially in some currently popular cities. For example, I believe Boton was still losing population in the 1990s and, at one point, when Teradyne moved out that was pretty much the last tech company in the city (including Cambridge) leaving. There may have been some early biotech but pretty much none of the Route 128 computer companies were in the city.
New York City was also in pretty dire straights in the latter part of the eighties.