I don't know many people in Silicon Valley but I know a lot of rich people. They definitely have a lack of compunction and some use confidence games to scam and exploit people.
I am not suggesting they are all bad but 90% act like my friends who live in trailers in the Midwest on welfare.
Human nature doesn't care about privilege. I believe education fails to teach critical thinking and tools to face adversity. And this creates situations where rich people lie, or honest investment bankers launder drug money. Or doctors overprescribe drugs, surgeons do unnecessary surgies and rich men exploit young girls. But they all self-rationalize their decisions and never think they are part of the problem. Arrogance, incompetence, greed, stupidity, etc.
There are a ton of successful incompetent people.
In general, I agree with you that it is the individual's job to transcend these challenges but it is ignorant politicians and lazy wealthy and influential people who sustain a system that has so much friction, so much chaos and so many challenges to overcome to get a fraction of what the alleged "elite" get.
My point is, for the most part there is nothing elite about the 1%. There is no secret group controlling the world. If we want to change things, we can.