I think that the way we as humans decide to structure society determines whether life is fair, not natural circumstance.
Just because you are born smart does not mean you are born more deserving.
No amount of exercise will give me the garden hoses for veins that someone I know has, but I can still work to reach my own potential. Should I still be given a job that requires holding a jackhammer over my head for 8 hours a day because it's not fair?
Why is it socially acceptable to be taller or stronger or faster, but not to be smarter or more studious?
Exactly - "to deserve" is a nonsense notion in these contexts! Tall people in the NBA might lead to a more beneficial outcome (more interesting games?) but it does not arise because tall people are more "deserving" of being in the NBA.
Since we get to choose how to structure the human sphere of the "game" of life, it seems better to optimize for an outcome that is fair, ie. equally attentive of people's moral worth.
If that requires giving more $ to some who are more talented in order to better off everyone, then life does seem fair because our society is structured w.r.t. everyone's benefit.