if luck does not explain 100% of the variance on average more successful people will have better skills than those who are not. It is a pretty big bias to think in terms of 1 or 10 or units
I assume people who believe wealth and success of individuals can be explained by predictable qualities or behaviors of that person have simply never met a rich person in their life.
And you don't believe a "winning group" would hold its own biases?