Yes, it's a known problem. Lax law enforcement, most social services are also "downtown."
While the commenter said they wont name the cities, it can be Chicago, Seattle, Los Angeles, San Francisco, New York, etc.
It makes it unpleasant, and if you do have to live in the middle of downtown or such, not only are you paying more - you have to for safety. Safety of a living place, safety of controlled entry, safety of a garage for your car, etc.
In General it's all depressing, you'd imagine downtown cores would be the "best" of a city, but in the USA they generally are not.
Now whether or not it's true insecurity, fear or such can be up to person/gender but criminal statistics do not lie. You can pull up a map and see where the majority of crime is reported of happening - and there you go.
Then you can look at the threshold and severity. But nowadays we live in an era of political correctness or wokeness, so saying or making direct observations of that data can become very faulty quickly, especially once you bring up age, gender and race of the perpetrators.
So eh.