Observing that money is a proxy for many desirable traits, and helps, and has helped, in bettering survival chances is not social Darwinism. To assume that the unintelligent wealthy are somehow more lucky, and that morons must be filtered out is.
Inherited wealth is much a tradition in the same way gestating your young (outside of the womb) is. Happy equilibrium or not, that is the way culture and nature have co-transformed, and determining the causative reasons is not necessarily blind Darwinism.
Here's a statement: On average, the wealthy are smarter, prettier, and taller, than the not-wealthy. Do you disagree with that? Or do you agree, but attribute the correlation to happenstance?