It's basically impossible to do a zero-downtime upgrade with an RDS. Systems that have zero-downtime upgrade "solve" this problem by not being relational (because it pushes the fault-tolerance to the query developer when they have to build their own relations across systems that are defined by API contract to not always be available).