As the proportion increases, the market distortion you describe would increase, making it easier to pick stocks successfully. So the answer is: there will be an equilibrium and we have just not reached it.
I’ll keep paying $1/3k p.a. to VTI in management fees.
That's how equilibrium is reached. We don't get to know if you are a genius or an idiot, not even after the fact.