Could be because they're used to it. Every country has a caste system of some sort.
Comparing the lack of a legal caste system because “of some sort” to a legal framework is quite the leap.
https://en.wikipedia.org/wiki/List_of_countries_and_dependen...
Coloreds are still hugely overrepresented in the criminal "justice" system, they're still much poorer on average, are still disenfranchised en-masse through gerrymandering and voter ID requirements that accept shooters' club membership cards but not ID cards poorer blacks might possess, etc.
For all intents and purposes, the USA has a caste system that has evolved from an earlier system that was apartheid in all but name.