But we first have to accept the real world and get off this childish notion we are all born equal because as you pointed out, we sadly are not.
A 10 year old will be judged and rewarded in a favorable way compared to a 30 year old programmer. My example was to highlight the bias that people already acknowledge as fair. Poverty is a handicap in similar fashion. When we acknowledge the former in age should be judged more leniently and with grater praise, we refer to the statistical odds of a 10 year old being as good as a 30 year old and other 10 year olds.
That's the reality. You are refusing to accept it by removing the background of people in your hypothetical world. Fairness is not based on outcome.
Someone who never had access to computers, internet, mentors, etc would be judged differently compared to someone who always did. At least I would.