> The general point of ML ethics is that a machine learning system should not reinforce bias caused by historically biased training data.

When I think of "biased training data", I think of data that systematically misrepresents reality. Like say taking the prevalence of news reports as a proxy for the prevalence of what's being reported on.

In contrast, the AI ethics people seem to use that term to mean data that doesn't reflect just (as opposed to unjust) causality.

I think there's a deeper issue, which is summed up in the saying "when a measure becomes a target, it ceases to be a good measure": simple correlation-finders - which is what current ML/AI models are - are not fit for the purpose of building control loops. And the "ethics" people are trying to paper this over by manually fixing up all the correlations that people notice aren't (or shouldn't be) casual.