Clearly you have never dealt with a white man who is better off than certain women, or people from ethnic minorities, and thinks it's because his being white/male makes him inherently better and invokes "meritocracy" all the freaking time.
I have. And I'm a half-Asian male - I can't imagine what it's like to deal with that jerk if I were a woman or an ethnic minority he didn't consider inherently intelligent (yes, really - he was actually surprised I was upset at the things he said because he wasn't talking about me, as if that was the reason for me to disagree with him).
When the word "meritocracy" is used to structurally shut down debates of sexism and racism, it is no longer about meritocracy.