Unless you're very bad at investing you take out more than you put in to the system...
However there are risks with too much wealth tied up in too small a demographic group. It reduces the diversity among those investing, thus reducing the diversity of investments. This could distort economic development and leave productive opportunities unexplored. It could also lead to political capture by this monied elite.
In the limit that means they end up controlling all the things.
You would somehow need "little people" wealth growth (in aggregate) to outstrip "rich people" wealth growth for this not to happen, but it can't because the little people have to spend non-negligible fractions (or all of) their income to stay afloat. Basically, the rich win by not actually spending (proportionally) much money.
Thinking of it in exponentials, assuming you invest everything you don't consume, normal people have a drag on their exponential coefficient which is their cost of living. They also tend to invest less efficiently in aggregate due to not having wealth managers and specialised tax consultants and so on.
In an investment race, highest exponent wins. There is a confounding factor of population growth though.
What the economy needs is for massive wealth accrual to be met with equally massive backpressure. Allowing for much higher taxes at the very high end of income/investment returns acts as a kind of non-Newtonian fluid in the flow of wealth, keeping it all from going away from those who haven't even had the opportunity to get some yet.
What a catholic idea, that people should commiserate and suffer for the better of others.
Thankfully human kind is resourceful and since time immemorial will always weasel out of situations they dont agree with.
To the very least it would reduce wages and increase stock granting, which making them more obscure it has other negative effects.
Where difference investment options shine is, as you mention, being able to skirt taxes, or have access to information, or state regulation.
In the tech sector, the requirement of being an accredited investor to be able to invest in startups has the following effects: increases investment returns for investors. Reduce amount of founders and their payoff and lowers wages because they cant trade their stock freely, showing captivity.
On taxes: it is dreamed that better tax policy might nake the rich pay what they owe. But that mindset is what got us here. Id say lower the taxes on the common, and that might not solve it, but would vastly reduce the exponentiality.
On information: in this I part with milton and say that all income of any kind should be public. Society will take a hit on privacy, but this information is too valuable to hide. Knowing that everyone woyld not need to speculate on how the rich make money, and given freedom might even compete with them in the same assets.
Game is rigged, unfortunately, by the people that think they are fixing it.