I have always thought the analogizing of white to “good” and black to “bad” comes directly from absence of light (dark; black) or the presence of light (bright; white).

Because we are not nocturnal creatures, darkness == scary and daytime == security so therefore black == bad and white == good.

I can see how one who is hyper-focused on race might see that as problematic, but to me, “sometimes a cigar is just a cigar.”