Amplify's core architectural presumption is to only use serverless options to try to keep costs low. Serverless SQL databases are currently a big hole in AWS's offering. Yes, there's Aurora Serverless, but you need to run it in a VPC, you can't scale to zero (so, for some definitions, not really serverless), v2 threw out their Data API, and v1's Data API had severe scaling issues anyway. v2 is so expensive that you have to very seriously think through the numbers on the small RDS databases and ask what value you're actually going to get out of v2. So what OP proposes can't really be done by the Amplify team.
That said, the fact that Planetscale and Neon (which also have branching!) continue without serious competition from AWS is a serious headscratcher.