You're right, it doesn't happen when developing locally, only in MWAA. This was the answer given by the Airflow team as well and I figured they would punt before I asked.

I realize Amazon is taking an open source project and making a ton of money on it (the instance prices are ridiculous for what you get) and the incentives are misaligned for the Airflow team to help AWS make it better unless AWS paid them to help fix it.

It's crap all around, and Airflow gets a bad rap from AWS's terrible MWAA product based on it.