Hacker News Reader

Comment by Asdfbla | Hacker News Reader

In any case, you shouldn't neglect the subtle but important sources of bias those pre-crime models can have. Here's an interesting talk about it:

https://www.youtube.com/watch?v=MfThopD7L1Y

Basically, one instance of bias is the fact that many crime-prediction models are trained on police data, which means they will predict crime in places more often targeted by the police anyway. Then the model predictions even amplify that effect, since more training data may be generated from the places now more often policed, etc.

There's lots of resources out there on AI fairness these days. I think everyone who tries stuff like crime prediction should read up on that topic.

View on HN

Cathy O'Neil wrote a book called "Weapons of Math Destruction" -- interesting read.

You can listen to an interview she does on econtalk -- interesting to learn more about the hidden biases.

http://www.econtalk.org/archives/2016/10/cathy_oneil_on_1.ht...

>Cathy O'Neil wrote a book called "Weapons of Math Destruction" -- interesting read

+1000. That book should be required reading for anyone working in machine learning. Written by a former Wall Street quant who has the math down cold.

What she knows about rampant bias in allegedly politically agnostic machine learning circles is that the formulation and production of answers is trivial when compared to the formulation and production of questions.

Super-relevant to this thread is her work on recidivism risk scoring algos run on prisoners and defendants. The feedback loops that these algos spur are seriously damaging the lives of huge numbers of persons in the criminal justice system far beyond proportionality for the offenses that brought them there.

So if the AI identifies insider trading at trading firms, banks, etc., we should beware that this would create a feedback loop to look more into the investment and banking sectors and will ignore the mom and pop insider traders? That they go where crime is rampant over where it isn't and that could be a bad thing?

I think it's not inherently bad to have a bias, but it's bad if you don't recognize it. In your example:

If you train an AI on data that solely consists of trading firms & banks, you should recognize that it's an AI biased towards detecting activity at trading firms & banks, and that it might be lacking in other areas.

It becomes dangerous when such an AI is marketed and assumed as unbiased and the source of truth for detecting all insider trading activity.

If we don't have infinite resources (prosecutorial, for example), it makes sense to concentrate on the most salient loci of crime. If after enough resources are devoted to the most egregious locus, a different locus becomes the focus of crime and in turn gets the attention, that's not a bad thing...

In other words if done decently well, they'd obtain data from all places but focus on the problematic areas till they reach equilibrium and then you redirect to the next hotspot, no?

Is it a loci because that is where the crime happens, or is it a loci because that is where all of the crime prevention/detection/prosecution is directed due to external circumstances? You have to be able to correct for this sort of bias in your training data or else you are just baking the external circumstances into the model and pretending that they reflect reality. Start by figuring out how you would obtain data from all places first.

Mom and pop insider trading? How would that work, exactly? Or was it intended as a joke?

Small fry. People like Martha Stewart. She's wealthy of course, but her dipping in is of little consequence. Or your buddy at Google lets you in on something theyre not supposed to. It's the big sharks you care about.

Reply on news.ycombinator.com