Maybe, but I don't think that most people who say this are trying to belittle their idol, rather complement them on the fact that they have a greater skill than others. That no matter how much time someone else puts into it, they will never be as good.

And there is some truth to this. You are born a good jocky, most people couldn't train hard to be that small.