The problem IMHO is rent and profiteering off taxpayer subsidies, both of which more or less force me to pay tribute to the rich.
I don't care one bit if someone else is infinitely richer than me as long as I don't have to pay tribute to them or otherwise be lorded over by them. I would not care if (hypothetically) there were people alive today who were biologically immortal cognitively enhanced cyberpunk demigods with control of wealth flows in excess of present-day US GDP. In an abstract philosophical sense it seems likely that such sentient entities do in fact exist somewhere in our vast universe and I don't see how this likelihood impacts me at all. There are also (statistically speaking) probably a minimum of several hundred million to a billion people smarter, healthier, and better looking than me.
What I do care about is being forced or at least heavily pressured/coerced into paying tribute to those who are above me economically.
A prime example: housing costs are insane and this is partly a result of rich people from around the world parking their wealth in real estate and extracting rent in exchange for little to no investment in anything productive. In general, idle rent extraction is a social ill and should be taxed to hell and back. I would support cuts to capital gains, employment, and income taxes and large increases in taxes on property speculation (non-primary residences) and other non-productive rentier activity. (We might distinguish between productive capitalism and rentier capitalism, though it's hard to draw an absolute black-and-white distinction.)
Other examples of tribute include the many cases where public insurance is used to back-stop private losses while private gains remain private, such as the 2008 bank bailouts or the government paying in part for the cleanup of BP's gulf oil spill, and cases where taxpayer subsidized research and development is privatized for exclusively private gain (and patented!). How many huge companies have been built on the back of government-funded research? (The fact that so much wealth is built on the back of public research and public infrastructure is a common argument for progressive taxation.)
A huge reason I've lost my faith in libertarian politics is that the 2008 bailouts convinced me it's a scam. The rich advocate laissez faire until their wealth is on the line, at which point they become socialists. It's clear to me that for the poor and middle class to forego the use of government coercion is for them to engage in unilateral disarmament. I've heard today's economy described as a casino in which public funds are used to backstop the losses of the highest rollers, and that seems fairly accurate to me.
It's not transformed me into a conventional liberal though. Personally I don't think anyone on the present-day right or left has a clear path out of this. Both sides advocate things that would make the problem worse.