This problem is tightly coupled with issues of correlation vs causation. In real data, correlation is mostly transitive (there are toy exceptions - https://terrytao.wordpress.com/2014/06/05/when-is-correlatio..., but they are just that). This means that if you want to predict something, and that something is correlated with some some uncomfortable association, trying to predict the something without the uncomfortable association can leave little residual behind.

For example, if hair length is societally taboo for gender prediction and I make an algorithm that uses a "politically correct" determination using XY chromosomes, I have also made an algorithm that correlates with hair length. Moreover, if I try to statistically correct my algorithm so that it does not correlate with hair length, I end up with an algorithm that works on the tiny leftover residual created by people who buck the trend, i.e. one that's much more likely to be wrong.

Algorithms find both correlational and causal factors. If 9/10 men are from Mars, and you tell me you're from Mars, it is often via correlation that the algorithm labels you a man. You are not allowed jump to the assumption that, say, the drinking water on Mars is turning people into Men.