It depends on your tests and environments. MySQL container should be used for integration and CI/CD testing if you are cost sensitive. Dev or Staging should be end to end testing against AWS RDS.

Whole point is this stuff is for those who think running Dev RDS is too expensive.