How will it harm most people in the short term, exactly?
And the only reason no longer working is a bad thing is that nobody trusts our government to create a universal welfare system.
It does seem like something to worry about for a small percentage of people in the short term (just as the China shock hurt a number of manufacturing workers in the 90s/00s) and maybe a majority of people when AGI exists, but who knows when that will be.
The people at the top are guaranteed to benefit. The people at the bottom probably figure they have a very low chance of benefiting.
If they succeed in their stated goals, most will be permanently unemployed, with no political power and the government will be a.) fascist b.) feudal.
AI CEOs aren't different. At some point we'll have to admit that we're sacrificing others for our own benefit.