This seems to me a simple instance of the general fact that, for telling things apart, you only observe as deeply as you need to, and it takes a while to get used to observing deeper.

For example, suppose I learn the phone numbers of my neighbors. Then I meet a friend who lives across town, and his phone number starts with 322-, while all my neighbors' numbers start with 494-, so when I see a caller ID beginning with 322-, then I immediately know it's him. But then I make several more across-town friends, and holy crap now there are four numbers beginning with 322-, so now I learn to pay attention to all seven digits.

Then I meet someone at a summer camp, and his phone number is in a whole freakin' new area code, so if I see "(408)" then I know it's him. But then if I meet more people in 408, I'll have to learn to look at the rest of the number when it begins with 408.

Or, think of backpacks. I have the only black backpack in my class; that makes it easy for me to tell mine apart. But now someone else gets a black backpack, and I say to myself, hmm, well, mine's got thick straps. Then maybe someone gets the exact same model of backpack, and I'm like, dammit, fine, I'll write my name on mine.

Cars. Mine is a big white car. That worked fine for a few years. Then I saw other big white cars. Mine is a Toyota minivan. Worked fine for some more years. Then I saw another white Toyota minivan, but it had different hubcaps. Fine. Now I've seen a few that are the same model and color of car with same hubcaps; gotta look at the license plate to be sure.

With people, there are many ways to distinguish people. Male/female. Long hair, frizzy hair, buzzcut. Really tall, really short. Color of skin. Color of hair. Facial features common to certain races. Fat. Always wears a particular hat or jacket.

At any time, people's methods of telling people apart are likely not much more sophisticated than they need to be, and if they're thrust into a situation in which those methods don't work well, then obviously they'll have trouble telling people apart. And, given time, they will learn and use new methods and get better at it.

So: There's nothing special about race, there's nothing special about faces, and there's nothing special about finding it hard to distinguish things in ways that you're not used to.