Maybe being ginger is connected to IQ. Or maybe a particular ear shape or toe length or something is correlated. It's possible I suppose. But no one is looking in to that because everyone understands it doesn't really matter as not everyone with a high-IQ ear shape will be smart. You still need to judge them as individuals.

Even if there is a connection between skin colour and IQ (which there is not, as far as I know) you'd still need to judge people on their individual merit. It's all about "on average, black people have a lower IQ". Even if true, you can't do anything meaningful with that in any liberal merit-based democracy. White people from rural Alabama might also score lower on IQ tests than white people from NYC. When pressed, even the racist assholes posting in this thread will admit that James Mickens is way smarter than the average white programming/computer scientist (never mind funnier). He certainly is smarter than me and I'm white enough to get sunburns in Ireland.

I would say the entire focus on connection between race and IQ is almost entirely prejudice because it just doesn't matter. Barring than the occasional well-intentioned misguided soul, if you dig in to all the people focused on it then it rarely takes long to find some genuinely racist things well beyond their so-called "just looking at the objective facts".