Your excellence in no way diminishes the value of my attempts. It does put my success (or lack there of) into perspective and gives me a meter for what excellence looks like. That's a good thing, though. Having something higher to strive for, and knowing that it's possible to do better, is a significant factor in personal growth.
Height is a spectrum and yes, IQ is a spectrum. "Seven feet tall" is an easily identifiable point along the height spectrum. "Genius" is not an easily identifiable point along the IQ spectrum.
It's a wishy-washy, overly emotive term that lumps certain people into an elite group which, by the popular conception of the word, you're either born into or you're not. It says more about the people who use it than the people they're bestowing the title upon. But hey, if you like the term and get your kicks seeing it bandied about, good for you. I find it trite and counterproductive.
By drawing the analogy to height, I thought the point would be obvious. Telling somebody who is 5'2" that they "didn't work hard enough" after they failed to get drafted into the NBA is just cruel. I think the same argument applies for somebody with an IQ of 85 hoping to win a gold medal at the IMO or a Fields Medal.
People are in elite groups along many different metrics: wealth, intelligence, height, strength, athleticism, etc. Hard work is what it takes for somebody who is already in one or more of those elite groups to succeed on the world stage. Millions more work just as hard but aren't in those groups and typically don't win the big contests.