Imagine you start a society where nobles have the same "gene quality" as everyone else. Some nobles are pretty smart, others not so much. As nobles would have more children surviving into adulthood (due to better nutrition and living standards) compared to the amount of land/titles they could distribute, some of the children of the nobility would be cast aside. The assumption, and I agree it's an assumption (although a rational one in my opinion), is that the smarter children would tend to inherit, and the less capable children would be dispossessed.
Over time, this constant churn would lead to a concentration of "good genes" among the aristocracy, which would trickle down into the lower classes when every generation some of the nobility lost their position.
I believe that this "trickling down" phenomenon has been proven to have occurred in England during the second millennia by Gregory Clark using census data, although I'll admit I've only heard of his work, not read it.