It's actually a very accurate description of how a group that historically is facing systematic injustices and racism get to have fewer illusions than the comfortable majority.
If anything, it's ANTI-racist.
Not being racist is not the same as believing that racial minorities have absolutely the same experiences and outlooks as the majority.
A group of who had the cops enforce Jim Crow and segregation on them, or arrest, harass and shot them far more often than another group, has a different outlook on this "justice" thing.