Tools could be better at supporting rolling updates.

With 20.000 servers, for example, an automated process could roll out updates to 1,000 every hour during 20 hours, check that response times, system load, etc. are within limits for the updated set during each hour, and send out alerts and pause updating when they do not.

So, the user still would press one button to do the update, but the change would slowly take effect, allowing both the system and humans to take action if needed.

Main problem there is to keep things flexible enough to allow somewhat out of the box updates. And of course, that requires that you can run with half your servers on a different version of your software.

you probably also will have to forget doing the entire update in a single transaction.