I rode the bus every day for 10 years in SF, and I've had less than one bad experience per year. Yes, you may see a drug user on a bus (I once had someone smoke meth next to me, for the length of one stop), but assault isn't very common. Likely the most common issue is theft, when you aren't paying attention to your belongings.
The issue is very much racist/classist, because problems and complaints like this are much less common in places where everyone uses public transportation. The public transportation is better funded, more frequent, and better maintained (and cleaned). Rather than being filled with people who are forced to use public transport, it's filled with everyone, and that helps with the general culture/vibe of the transport too.
Live somewhere where public transport is the norm, rather than exception, and get back to us.