It is well supported. History is full of examples of social change resulting from abandoning biases (sometimes freely, though usually due to "encouragement" by other forces, often economic or demographic). From the "Struggle of the Orders" in the Roman republic to the rise of the middle class in Europe, if you are of European descent (and probably even if you are not), then your own education and social standing are likely a result of such change.
> This is anti-inductive. Knowledge can only be contingent based on past experience. This is John Stuart Mill stuff, empiricism.
Quite the contrary. Suppose the dynamics of a system is that it behaves so: -1, 1, -1, 1, -1, etc. If your previous observation was 1, your prediction should empirically be -1. If you are presupposing that the state of the system is unchanging -- and therefore predicting 1 -- that is anti-inductive.
But consider a more interesting system. It can obtain the values 1 and -1, and its initial state is 1, but its future state is identical to your previous prediction. What should your prediction be, then? You cannot make an inductive argument for favoring either prediction, and the correct answer is "whatever you want the result to be".
Finally, consider a more interesting system yet. The first three states are 1, and thereafter the state is the majority of your last three predictions (another interesting next-state formula is: -1, if the previous three predictions are all -1, or 1 otherwise). What should your predictions be, then? The correct answer is still "whatever you want the result to be" (although you may miss a few), but the difference is that this time some people will (incorrectly) argue that predicting 1 is just the "mathematically" right thing to do. It is not. It is simply a system that makes it easy to mistake a conservative policy for a neutral one.
> This does not mean we should just make shit up for some of our sample data/training set.
No, but it does mean we should study society in depth rather than arbitrarily decide that what was is what will continue to be, especially if by making that decision we are forcing that outcome. I propose that we stop making shit up when modeling society by treating past behaviors as future predictions, and instead try to study social dynamics. Once we understand those -- more or less -- we should decide what future outcome we want. Once we know that, we can decide how to factor past data into the system's dynamics so that we can get the outcome we want.
> If you want to enact social change go for it but stop pretending that this has anything to do with statistics, modeling, prediction or math, just go straight to the politics.
I am. I'm just pointing out that no view is neutral. Interpreting past behavior as future behavior is a conservative political action.