Consider children of immigrants. Look at the ones who are tall, taller than their parents. Are they tall because they got good nutrition, etc.? Many people supposed that they have natural tallness genes and their parents were only shorter because of tough times in the old country.
Now look at their children, the grandchildren of the immigrants. They tend back toward average.
The tall kids were tall because, in part, there is a natural variation and they happened to get on one side of the bell curve. Luck, if you will.