Different accents isn't bad data. Your vision of the world of "english is only spoken with an american accent" is what leads to horrendous speech recognition APIs, like Google's.
If your ML model can't handle multiple accents, it is worthless.
If your ML model can't handle multiple accents, it is worthless.
Fortunately, splitting models into separate accent-specialized variants and helping them out with language model training will often help in case the model doesn't cope well enough with the cognitive dissonance.
Which american accent?