Anecdote, I spent some time wandering around east palo alto (at the time, was the murder capital). It didn't feel particularly bad, but the data said otherwise.
You can't trust random people's perception of safety. You can't trust the data either.
In San Francisco, that bargain is unraveling.
The Bay Area has always been particularly badly segregated. Palo Alto is wealthy and posh, East Palo Alto is a dump. San Jose boasts huge tech companies while East San Jose is full of gang fighting. I grew up in one of these violent areas of the Bay (starts with East but that's all I'll mention publicly) and the gap in resources is huge. Bad schools, poor libraries and community spaces, poorly maintained parks, unsafe neighborhoods, no pedestrian affordances, high speed badly maintained roads, the list is endless.
The downtown part of SF isn't really considered a posh area by most of the old money interests in SF, who live in the Northwestern parts of the City. The old political families of SF all live there in huge, beautiful houses and the neighborhoods are very different than the parts of FiDi and SOMA that new money tends to hang around.
The Wire is a great show that goes into some of the politics and challenges of America's containment style of policing.