I’ve spent a cumulative 6 months there (3 separate trips) including 3 months with a 1 and 3yo, and never was there a point where I felt unsafe. I am a tall white guy and stand out. Met lots of curious, friendly, amused locals that wanted to take pictures with me, especially if they were from the less globalized regions. There is a strong police/security presence everywhere. Even my apartment block had its own 8 person security team.
Watch out for Shanghai coffee scam. Two Chinese female invite male foreigner for coffee where foreigner is forced by large male enforcer to pay large bill perhaps $100's. Guy in my hostel got stuck for $200usd.
Brutish/opportunistic way out: Be physically large. I don't mean to sound like some kind of Chuck Norris badass — because I am absolutely not — but when I stood up after being similarly challenged the game changed. They didn't even try to show their martial arts skills (of which mine are poor at best tbh) or any other proof they would hurt me. They just up and left.
Quite safe. My wife (who is Chinese, I am white) actually becomes annoyed because everyone treats me so well. I never once felt unsafe in China, though I suppose there are parts that you may feel unsafe.