You're not a racist for saying it's bad. You're a racist for saying that the mere presence of nonwhite people has somehow harmed your "community".

If this is what you've taken away from your ancestors' struggles for freedom and self-determination, then I'm sorry to say that they died for nothing.