I'd argue that the cultural perception of the belligerent humans towards their targets is far more important than objective differences.
And you and parent are certainly being overly-reductive compared to the context of the times.
What would a 1930s German have said about being compared to a Polish or Dutch person? Would they have seen themselves as "pretty similar"?
And I'm not sure how many Indians and Pakistanis you've talked to, but they're pretty close to space aliens to each other.
Same with Shia Iraqis and Sunni Iraqis, and both with Kurds. Etc etc.
I don't see how you can say "these peoples are the same" across any modern ethnic or national boundaries without a lot more support without it coming across as "those people all look alike."
Hell, I can localize most folks in the US within five sentences or so.
And yes, there were states that straddled the Mason-Dixon line (and states like Virginia that were shattered because of it), but outside of those limited cases I'm sure the majority of the people involved could have identified "the other" pretty easily.
Point being that if we're finely tuned enough to physical and cultural differences amongst our own species, yet still war on those we can perceive as different... imagine how much stronger that impulse would be against something that didn't even look like us.