In my twenties, my husband spent months going on endlessly about his new best friend "John." John this. John that. It aggravated the hell out of me. He gushed about the guy.
Then one day hubby and John arranged for the two families to meet and it turned out John and his family were black. The surprise showed on my face, which made for a very awkward meeting.
I didn't care that they were black, but my husband had never once mentioned that John was black. So I assumed he was white.
In that moment, I realized this was a racist assumption. This assumption that if he isn't white, my husband will inform me of this fact, was an artifact of a deeply racist culture. And I was tremendously upset to realize I had drunk the koolaid when I thought of myself as an idealist and not racist and so forth.
So I am very uncomfortable with the possibility that I may yet still be obliviously doing things that help keep racism alive, help keep people of color oppressed, help keep doors shut in their face without being aware of it. I don't know the solution to that and I think if you really truly believe in justice and equality, that should bother you. If it doesn't, then you are probably part of the problem and just not really wanting to take responsibility for that fact.
But, yes, nitpicking about a single word (or a couple of words, in this case) is often a cheap way to lord it over people and act morally superior without really doing anything of significance. That much, I agree with.