On seconds thoughts, interestingly, that difference between "wealth" and "power" also becomes relevant when quantifying your "most" in They can do without MOST of the "we".

Like, how many people do the wealthy/powerful actually need? What level of automation will allow maintaining their lifestyle, including:

1. Biological needs: Food & a sufficiently diverse pool of mates for themselves and their children.

2. A social hierarchy large enough to flex their power beyond mere biological survival (very important to primates). I'd expect this number to be strictly smaller than for 1), so likely not a concern: as long as there are enough people to biologically sustain a population, there are enough people to subjugate.

3. Technological scaffolding to produce, maintain and bootstrap (in case of calamities) said automation. Currently humans are a necessary physical substrate for the continued inflow of energy that automation needs. I don't have the numbers but the footprint to run even a single power plant must be tremendous: a large and complex society, once you include 2nd order effects. Without energy, the machines don't go "VRRROOM" and neither does AI.

And if the human population drops too low to interfere with any of these points, the automation was self-defeating. It doesn't matter how rich or powerful you were.

I mean, I'm sure someone somewhere ran the numbers and has a plan ready, while the rest of us wax lyrical about "just find a new job LOL" and UBI. Although to be fair, there are also preppers, who've taken the above analysis to its logical conclusion.