Hacker News Reader

Step 1? Maybe if you don't want your model predicting crime based on race, sex, or other sensitive features, you shouldn't train your model on race, sex, or other sensitive features.

View on HN

This approach doesn’t work. If sensitive features are actually predictive (and they very often are), the model will just learn to predict these features based on the non-sensitive ones. For example, if you know that someone is 30 years old, lives in Bay Area, and earns less then 30k/year, if you bet that this person is neither white nor Asian, you are more likely to win this bet.

I don't like this example-they could be a graduate student

Of course. They could be many other things as well. The question is, what would you bet on, if you didn't want to lose money?

The key issue is that it's not sufficient. A model that explicitly uses race gives quite similar results to a model that "just" uses e.g. zip-codes and all kinds of other information; many of these things are highly correlated.

I'm sure that facebook could give (if they wanted) a very, very good guess about the shade of your skin simply based on your history of likes, pages seen, etc.

Reply on news.ycombinator.com