> Given that most stocks by far are held by wealthy people

I also won't pretend to be an expert here, but I think this is where your thinking may be going astray. There are lots of retirement-age people who are counting on their pension funds to fund their retirement. Pension fund managers assume steady growth in stocks, something like 7-10% per year. If stocks don't meet that threshold, major parts of the pension system run into big trouble. Pension plans have trillions of USD in assets. This article [1] estimates ~$9T. Retirement-age people in general are a powerful constituent.

[1] https://www.investopedia.com/articles/credit-loans-mortgages...