"What keeps this going is that each VC thinks they're better than average"

Another thing keeps them going is that the VC Partners get a nice salary for 10 years. Many of them will make more money than many C-level executives. I don't see a downside in being a VC.