Ethics in Machine Learning: Interview with Dr. Hanie Sedghi, Google Brain

The paper was beyond my level of stats-comprehension, so I couldn't figure out the answer to this question while looking it over. How does their methodology get around the 'redundant encodings' problem?

I don't understand what you are proposing here. The way things are done right now is we define some specific features which can include things like age, height, eye color, skin color, default history, average time a person wakes up, whether they prefer pancakes or waffles, income, and pretty much anything else you come up with. Then, we obtain a training set containing instances of the specific values of the features paired with the output variable we want to predict. Then, we pick a model and train it using this data. After training, we can test the model on some new data and measure its accuracy, that is, how often it is wrong, what is the direction of the error (i.e. bias), if it's wrong, by how much (variance) etc.

Some of the variables might turn out to be more predictive, and some variables probably quite a bit less. Some variables might turn out to be predictive even though one would guess they shouldn't. For example, people with brown eyes are more likely to be victims of crime than people with blue eyes. It might seem like a mystery, until you realize that black people overwhelmingly have brown eyes, and they are more likely to be a victim of crime than white people. Here, removing race as a variable will only make you and your model more confused.

It is important to realize that removing predictive variable will not necessarily make your model less "biased" -- in fact, it is likely to make your model more often wrong than before, and therefore less useful. The difficulty here stems from the fact that often laws require you to make "non-discriminatory" decisions, and if you attempt to do that, for example by introducing counteracting bias to the bias you get from reality, your model will be less predictive and so less useful.

Fairness in general is very complicated issue. Suppose our model tells us that blue agents are much more likely to default on loan, and this is in fact the underlying reality. The creditor, to maximize his profit, should therefore charge blue agents higher interest than red agents. If you are blue agents, this is obviously unfair for you -- it's not your fault some other agents who share similar characteristic default more often. However, if we have laws forbidding discrimination based on agent's color, the creditor must then charge both colors of agents equally, and since one is more likely to default, this interest rate would be higher than what red agents would pay if the anti-discriminatory laws weren't put in place. Isn't it unfair towards red agents then? After all, it's not your fault that some other agents who share some characteristic default more often...

It may not be “fair” in some cosmic sense, but traits that you don’t have control over are important for lots of things you want to assess. For example, should modeling agencies not take into account physical appearance?

Additionally, this extends to the usage of latent or inferred variables that behave as a proxy to a protected class - so neural networks and other discriminative approaches outside of a distinctly diagnosable decision tree, could be in a rather grey area legally if used for credit scores, etc.

It's possible that the upshot of a legal challenge could be a crippling cap on the accuracy of models as they explicitly avoid any information derived from any protected factors.

But in reality not all feature dimensions are orthogonal. You can't make the same 'adjustment' for any potential trait bias. Correcting for the purple/orange thing might skew the data on square and round heads, and correcting for that might skew it on pointy and rounded ears and so on and on. The intention is laudable, but many a road to hell is paved with good intentions.

How do you quantify "skew due to differential enforcement"? The proposal is to posit some value for "skew due to differential enforcement", then massage the results to match the baseless bias.

The data[0] shows that the race distribution of the assailant in crime reports is about the same as the race distribution of arrestees. There's no support for the differential enforcement hypothesis.

[0] https://www.ncjrs.gov/App/Publications/abstract.aspx?ID=2560...

If the classifier is being used for medical diagnosis, race, age and sex may be important factors. Ignoring them would amount to systematically ruining innocent hopeful life without any wrongdoing.

The problem usually isn't injustice caused by model errors, because humans make unjust errors all the time; using a model that's better than humans is still a win even if the model isn't perfect. Likewise for medical diagnosis, if a model is better than the average doctor, it should replace them.

The actual problem is that humans mistake the model prediction for something it is not. E.g. if you train a model on the data of people accused of a crime to predict whether they will be convicted, you're modeling P(convicted | accused). Even if you train this model to perfection (zero test errors), you would at best reproduce the existing legal system; which is a win if the model is cheaper. But people tend to mistake the model output for P(should be convicted | accused), which assumes that the legal system is perfect. Or worse, they think of it as P(should be convicted), without accusation.

The solution to your "go in the field and run your classifier on everyone" thought experiment is not to hide race, sex and age from the model. If the model had no access to any features at all, it would just predict the baseline conviction rate, much higher than the baseline crime rate. Using the model output as evidence of a crime would be just as silly, because that's not what the model was trained to do.

The solution is to not even try to predict something for which you don't have any data. Modeling can never make a process more accurate, it can only make an accurate process faster and cheaper (and slightly less accurate). Sometimes the accurate process is to wait for a few years and see what happens, e.g. for cancer mortality or recidivism rates. Sometimes the accurate process is to ask a lot of experts.

Once you have a model for the accurate process that is better than the cheaper process you'd otherwise use, then you can replace it by the model. Instead of a single human judge or a jury, have a model trained on the decisions of multiple judges and juries deliberating much longer than usual. Instead of a single doctor, have a model that has seen the medical histories of thousands of cancer patients and knows exactly which treatments worked.

False positives in classification or prediction are in practice unavoidable with any probabilistic technique for any non-trivial task. Including certain features where they are forbidden by regulation is obviously not done. The problem arises that your techniques can potentially derive those features anyway from correlated proxies.

Meta-physically you are using a probablisitc inference technique to derive from unknown-knowns in your data a given known-unknown predictor. You can try to normalize for known-knowns, blurring the biases you don't want to account, but you can't control for unknowns.

In some cases it is easy to detect bias in the model when there was discrete data for the feature you try to control. E.g. if your data would have an explicit 'gender' field, then you can test how much the outcome of the prediction is not correlated to the specific value of that field. If the data does not have such a feature, but derives it implicitly though e.g. food purchase preferences, that is much harder to detect. You can try to control for that if for whatever reason you deem the model not in line with bias preferences, but due to the implicitness it will be very hard to isolate the impacts.

I feel you will never get a model satisfactory to all, as in the end what is and what isn't accepted as 'neutral-data' is not a scientific but a political statement. There is nothing wrong with that. All our ethical choices are political. Just don't expect universal acceptance.

There is an argument to be made though that we should be looking for causal connections rather than correlative when we think about what fairness looks like. Unfortunately, even with the many recent advances in causal network detection, we're not quite at a point where I would trust causal modeling for this.

That's not entirely true. Causal inference is an enormous field and is just beginning to become more of a part of mainstream ML. ML is largely "minimize predictive error," but it's not limited to that.

I don't like this example-they could be a graduate student