Lots of white people also look the same, I grew up in a mostly Mexican environment, living in the Bay Area now, I find it difficult to recognize lots of white people apart from each other.

I'd say it's an ignorant thing to say in the way you say it. Like you said in this post, you just don't know the features to look out for when you're not familiar with them