Even if the rollout was atomic to the servers, you will still have old clients with cached old front ends talking to updated front ends. Depending on the importance of the changes in question, you can sometimes accept breakage or force a full UI refresh. But that should be a conscious decision. It’s better to support old clients as the same time as new clients and deprecate the old behavior and remove it over time. Likewise, if there’s a critical change where you can’t risk new front ends breaking when talking to old front ends (what if you had to rollback), you can often deploy support for new changes, and activate the UI changes in a subsequent release or with a feature flag.

I think it’s better to always ask your devs to be concerned about backwards compatibility, and sometimes forwards compatibility, and to add test suites if possible to monitor for unexpected incompatible changes.

This is a systems problem that can and should be fixed in the system IMO, not by relying on devs executing processes in some correct order.

This is where unit testing / integration testing should be implemented as guard rails in my eyes.

Rollout should be within a minute. Let's say you ship one thing a day and 1/3 things involve a backwards-incompatible api change. That's 1 minute of breakage per 3 days. Aka it's broken 0.02% of the time. Life is too short to worry about such things