Let's say we train the NN using equal number of human faces of all races, and animals faces. Let's say the trained neural net made some mistakes, including a few cases where black faces are recognized as gorilla faces. And this doesn't happen at all or as likely with white faces. The results are horrible, right? And people immediately start to point fingers to the training data and the algorithms, stating the training data is racially biased and/or the training algorithms or even the use of neural nets are racially biased.
But is it really so? It's known that in order to take a black face photo with the same degree of details, the lighting condition and camera settings need to be adjusted. This is an effect purely due to physics. In other words, it could well be by nature, that recognizing black faces is harder than pale faces under the normal camera and scene settings. This is why you have night mode on your phones. It is just harder to take clear photos when less light gets into the camera. And this requires the camera and photography settings to be adjusted.
So, the unwanted results here are still due to the input data. But neither the input data, nor the algorithms contain any racial bias towards the black people. The results might be merely due to the difference between dark faces and pale faces under the natural law of physics.
These are unwanted results due to our social norms, but they are NOT racially biased or racist, because there is no such bias introduced or inherent during any part of the process.
We could and should correct such unwanted results by introducing adjustments to the input data, like improving dark face photography and camera sensitivity. But we can't just label the input data, the algorithms and the people who designed these algorithms as "racist" or "racially-biased". There is zero racial bias that is man-made here. The race just coincide with the side-effect of photography.
Likewise, there will be cases where the reverse is true, like white faces get unwanted results instead of dark faces.
So, while we work towards improving the data quality and the algorithms, we must stop this trend of labeling or calling people and algorithms racists.