As for why you don't see more high rises going up, that's due to regulations which increase cost. I saw one tech crunch article that said super computers are now required to comply with all the regulations: https://techcrunch.com/2018/12/08/why-you-need-a-supercomput...
Ironically, affordability regulations actually drive the cost of building up to the point where no one can afford it anymore and nothing gets built. Why is this allowed? A big chunk of the SF population is deeply conservative ( politically liberal), but actually conservative in that they dont want anything to change. Meanwhile more and more jobs are added every day exacerbating a huge imbalance between housing and jobs, and employers continue to wonder why they can't find "talent".
On the plus side, there's no shortage of fun spots in SF: the fisherman's wharf, the embarcadero, the presidio, etc. Lots of fun restaurants and cafes, etc.