Exactly. That's an application of the Pareto Principle where 80%+ of the returns are realized by 20% or fewer of the participants. Without those high-performance outliers, this whole category of investment asset class would simply no longer exist. Once those high-performance outliers regress to the mean (if they do end up doing so), the VC asset class is sunk. If the outliers remain as outliers, then either the asset class will shrink (on that note, read this interesting article on Zombie VCs: [1]) as the outliers will become the mean (the marginal and underperformers will be culled out of the market), or whatever edge they have over the mean will no longer exist. I have a feeling VC will never disappear as an asset class, which implies the former more than the latter.
[1] http://www.daniellemorrill.com/2013/04/zombie-vcs/
and... more recently "New data suggests the decline has been more severe than previously thought, finding fewer than 100 active U.S. VC firms in the technology sector.":
[2] http://www.daniellemorrill.com/2013/04/zombie-vc-shakeout-co...