Be Careful What You Code For
points.datasociety.net
points.datasociety.net
This is extremely important. The weight of existing prejudice drives and reinforces future prejudice. Garbage in, garbage out. If you start with biased, flawed data, and you look for patterns like the biased, flawed data, you'll just add to the biases and flaws. You need to account for the poor data quality explicitly.
Thus, unless the goal is to perpetuate the effects of discrimination, algorithms need to be carefully set up and trained to understand that "this isn't how things are supposed to be".
Now, take that model and apply a really crude analysis. You would conclude that there is five times more crime in the more heavily patrolled neighborhood. Your algorithm, applied to policy, suggests that crime could be controlled better by patrolling the "bad" neighborhood ten times as much, rather than five times as much.
Now, consider the social result. The "high crime neighborhood" then sees its property values drop, and more homes sold into the rental market. This attracts poorer, rougher people who can't afford the "good" neighborhood. Now crime actually goes up, reinforcing the idea that the bad neighborhood is high-crime.
Now, imagine we've been doing this to those two neighborhoods for generations. What are the likely results of your algorithm? There's your uncomfortable truth.
In the days when Sussman was a novice, Minsky once
came to him as he sat hacking at the PDP-6.
"What are you doing?", asked Minsky.
"I am training a randomly wired neural net to play Tic-Tac-Toe" Sussman replied.
"Why is the net wired randomly?", asked Minsky.
"I do not want it to have any preconceptions of how to play", Sussman said.
Minsky then shut his eyes.
"Why do you close your eyes?", Sussman asked his teacher.
"So that the room will be empty."
At that moment, Sussman was enlightened.
There is always bias (and other types of error) in data. There is even bias in the choice of which data to use and the type of analysis to perform.
If you think that this isn't a problem, you should really read about practices like "redlining"[1]. For many decades segregation was (and still is) enforced by opaque "loan approval" methods that just happened to always deny loans to blacks.
[1] http://www.theatlantic.com/magazine/archive/2014/06/the-case...
See [Researchers have long known that whites are more likely to use and sell drugs. And yet, who is arrested for drugs? Blacks. 13% of the US population is black, but over 60% of those in prison are black.]
So how would this happen? One possibility: if a crime happens more police frequent there. So you find more crime. Because we've over-sampled black neighborhoods (including DNA testing making it more likely to find relatives of someone already in prison) the mistaken belief becomes confirmed, and reinforced by the continued arrests.
I'm unaware of how to call out bad sample sizes in data. However doing so may help highlight invalid assumptions.
Blacks have a six times greater homicide rate than the rest of the US population, according to FBI extended homicide data. Almost half of homicides in the US in the last year I checked were by African Americans.
With the long prison sentences given for violent crimes, violent crimes are disproportionately represented in prison populations. As a thought experiment, imagine that each year one person with a thirty year sentance was committed, and one person with a one year sentence. One the system stabled out, 97% of the prison population would be serving 30 year terms.
You point out a bias in the statistic. Of the imprisoned population. What are the percentage breakdowns for types of crime?
Drugs and violent crime are the majority. So now we can get a breakdown of race. You can see a minor (and slightly misleading, I'll explain) breakdown by race here: https://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/2011/...
The reason the above table is "a bit misleading" is because FBI tracks "Hispanic" as "White". Most people tend to think of "Hispanic" being separate. You'll need to investigate a bit for accurate data/studies regarding this (because I'm not going to while at work), but if I recall "Hispanic" commit roughly equivalent amount of crime as "Black".
Note that this is arrests, not convictions, and it is also possible there is racial bias in courts where Hispanic/Blacks are more likely to be convicted for violent/drug-related crimes and thus over-represent themselves in prisons. Honestly, the data isn't useful because there are too many unknown variables to draw any meaningful conclusions but many people with racially motivated beliefs love to point at this data.
The gap in violent crime is much smaller than 6 times according to this DOJ report:
http://www.bjs.gov/content/pub/pdf/cv14.pdf
summed up in: http://www.amren.com/news/2015/07/new-doj-statistics-on-race...
Comparing the violent crime rates to the % population in the US (63.7% white pop; 42.9% offenders | 12.6% black pop; 22.4% offenders), it's roughly 2.8 times as high for black as whites per person. Bad, but not quite 6 times as high by any measure.
Going back to the original number of total offenders, it would make sense (assuming violent crime is represented disproportionately), that roughly 42% of those imprisoned are white and 22.4% are black; instead blacks account for a whopping 37% of the prison population and whites only 22%.
It would seem something more is going on here.
Honestly I'm not surprised blacks dislike officers either; unarmed blacks are 3.49 time more likely to be shot by an officer according to a bayesian analysis[1]. They also found no relationship between this statistic and crime rate in the area.
[1] http://journals.plos.org/plosone/article?id=10.1371/journal....
To get estimates for prision population percentages you'll need to adjust the convictions for each category by the average number of years served for that crime category. As you said, homicides make up a tiny fraction of crimes and yet a double digit percentage of state prisoners are serving time for homicide.
You'd also need to compare inside a state, since states can have quite different sentencing laws. For example, in California the sexual assault of an unconscious person has either a one year or two year max sentence. Whereas in South Carolina it has ten year max sentence.
the parent referred to bias. the example you're giving is one where the sample is biased because of a poor choice of sampling method. so, i think you're actually agreeing with the parent (as well as danah).
Why do you seem to think that you have to just throw your hands up in the air and accept the bias? Wouldn't the 'best possible analysis given the data available' include compensating for known biases in the data? If I was designing a system for analyzing temperature data, for example, and I knew that every measurement was 10 degrees warmer than the actual temperature, shouldn't my analysis system compensate for that?
Given the contrived arrest rate example above, do I multiply every black arrest by .7 to compensate for bias?
How would a system built this way be useful in practice?
Why are these parallels always drawn against racial divides as well? How are we compensating for the poor white kid missing out or the rich black kid getting benefits he might not need?
It's such a messy problem but I don't think building bias into software is the solution.
I prefer factually correct, biased systems to incorrect, still-biased-just-in-another-way systems.
Yes. And that is exactly the question you should ask yourself in work you do.
It's on all of us to unroot social ills. Even though it's thankless and difficult work.
The truth is that men living in certain areas are much more likely to commit violent crime and therefore get arrested than the average person in the United States. This is the sad reality of the world we live in. It isn't wrong to deploy police resources where they are most needed. If the data shows that these areas are no longer hotspots of crime, then we can reroute resources elsewhere.
The problem is that using arrest/conviction rates to predict crime rates will be inaccurate if you don't correct for the biases that exist in the ways in which crimes lead to arrests and convictions.
In the case of policing, the arrest record is racially biased (the US releases data that shows this bias and it's well researched). If you train on arrest records, it won't predict actual crime because the results will be racially biased.
Being unbiased is actually a very difficult problem. Especially when we are predicting things rooted in social / cultural trends.
Our job is to be correct. If we avoid offense in the process, great.
Programmers are not the personal army of any social cause.
I work hard to accurately model engineering problems, and apply consistent ethics in doing so. If that aligns with your definition of "combating prejudice", great for you, but I need the latitude to critically analyze technical problems without the threat of bullying and shaming for wrong-think.
If I need to understand how surnames work across cultures, understand how time works across calendar systems, or understand what "marriage" means to the people getting married, then that's what I'll do.
If I need to support non-binary 'gender' selections because that's what users want, that's what I'll do. On the other hand, if I need to support binary gender for the purposes of simplified data analysis under the assumption of two biological genders with a low probability of intersex individuals, that's what I'll do, too -- politely, and without rancor, and with all the respect for others I can muster.
> I don’t care what your politics are. If you’re building a data-driven system and you’re not actively seeking to combat prejudice, you’re building a discriminatory system.
Fuck that. Build _correct_ systems first. If you’re making a system that intentionally distorts data for the sake of “combating prejudice”, you’re _lying_. That doesn’t help anyone.
http://edition.cnn.com/2009/TECH/12/22/hp.webcams/index.html
http://www.businessinsider.com/google-tags-black-people-as-g...
Unrelated Example: Every morning, I weigh myself. Today, I found that my scale is inaccurate, and understates my weight by 5%. Therefore, if I am to predict my weight in the future, I must account for the inaccuracy of the scale.
Related Example: There exist police statistics for arrests. Studies have found that arrests are racially biased, and therefore those police statistics do not accurately represent the crime rate, only the biased arrest rate. Therefore, if I am to predict the crime rate in the future, I must account for the inaccuracy of the data set.
Also, citation needed for “studies have found that arrests are racially biased”. That is entirely too hard to measure. Well, it's not that I doubt such studies exist, I simply doubt they're valid in any way, and there are probably other studies with the opposite conclusion.
I certainly agree that racially motivated arrests are a large issue, and should be dealt with. Avoiding misinterpreting data is a widespread issue that in these cases happens to have prejudicial consequences.
I mean, is it discriminatory to say that Compton has a lot of crime and police should patrol it?
Or is this just loosely using the tech community as a demographic to drum up clicks while scraping the bottom of the barrel as far as looking for topics to write about?
There are way too many variables to say anything about whether “blacks commit more crime” OR “police arrest blacks too much”.
http://www.businessinsider.com/google-tags-black-people-as-g...
http://edition.cnn.com/2009/TECH/12/22/hp.webcams/index.html
The prejudice of the programmer (unintentional, as all prejudice really is) can show up in the code just because they forgot to include enough data in their training set.
---
[1] In light of which Ralph Ellison's novel is particularly prescient:
If you are writing a system that is designed to predict arrests for any reason other than telling police officers where to go, then arrest records are precisely the data you want to use without these political correctness coefficients. So advertising legal help for criminal arrests to someone that matches arrest demographics is the correct thing to do. Anything else is stupidity.