The problem isn't just that the ultrarich have too much money. It's that they get this money by doing things that make the lives of everyone else measurably worse. Selling us poor-quality products, paying low wages to work in stores selling those products, treating real estate like an investment instead of a human need, overcharging for healthcare, invading our privacy to find ways to more efficiently get us to buy their garbage, etc.
Without the incentive for the ultrarich to rip us off coming and going, we have a chance to improve our quality of life and change the things that are wrong in society, without leaving it to the rich to "fix".