That is 100% an anglo-saxon world problem since most European languages either differentiate between words for skin color and color name, or using the color name to describe skin color isn't considered racist at all.
I would stretch it even to mark it as an US-only problem.