Your argument sounds so temptingly plausible, doesn't it? Yet, women inarguably have biological limitations compared to men in some respects, but in those areas where they do, I've found that they still really hate it when you say, "you did a great job...for a girl".
Besides, your claim that specially calling out black excellence is different because of their fundamental equivalence is specious. We know that blacks are underrepresented in many high status areas. Thus, we know that there are barriers of some sort or another blocking them from achievement. It's an incredibly weak argument to say that these barriers aren't worth acknowledging, but the barriers to equal average achievement for older people are.
It's not really relevant whether you think old people are feeble on average. I, as everyone, wants to be judged on my own, as having the full potential of anyone else. When you say, man, you did an amazing job for an old person, you are suggesting that a younger person could do it better, or, at the least, I could have done even better when I was younger. But if some younger person could do it better, why aren't they? It's a little obnoxious.
Celebrating someone for having to "overcome" being black or being old or being a woman lowers their status in each case. This is just a fact.