depends who the customer is, I'm a customer of AWS and I expect 100% availability, mostly because my customers are everywhere in the world and there's no available window for downtime
Yes, it will require more work and time than just taking the database down and performing the update while it is offline. But as long as the database remains available it doesn't really matter if the update takes 5 minutes or 2 days, just that you can do changes faster than they appear. Since DBMS updates happen at most every few months, that should hopefully not be a problem.
At one of my previous workplaces we had a multi-TB table that could take several days to migrate with the online tooling, and would take 12+ hours to migrate even offline. Nobody wanted to take 12+ hours of downtime (for a busy customer-oriented website) but as long as the db stayed up nobody much cared how long it took.
AWS neither provides nor promises 100% availability. AWS will have SLAs on various services with the penalty only being a discount on your bill.
It's _your_ job to make your service resilient to a point where you are comfortable with your mitigations.
Well, you aren't gonna get it, it's a myth, like "5 nines" and such are, based on that businesses can foresee the unforeseen and plan ahead.
Whether a service is distributed or not, at some point some issue will come up and availability is going to stop for a while.