Courts are using risk-assessment software to sentence criminals
wired.com
wired.com
After all, the algorithm has been trained on information about recidivism that was collected in a world where racism skews arrest rates, conviction rates, and sentencing.[1] That means the algorithm is almost certainly baking in a racial bias. Now, I'm sure they aren't foolish enough to put "race" in as one of the input factors, but other correlated factors will allow the algorithm to continue to enforce this racism, but now with legal immunity.
[1] Do I really need to footnote this? http://www.huffingtonpost.com/kim-farbota/black-crime-rates-... is one source that addresses all of these, but there are many, many other sources.
Which correlated factors are you thinking of?
I'm not saying these factors can't legitimately be used in a risk assessment, but they could be used to make a good bet on race.
If you were using these proxies to identify the race of a person jailed for a crime, they might be good predictors, but only marginally more useful than just blindly guessing black, since they constitute the majority of incarcerated people.
I think, by attempting to remove the smoke screens a "real racist" would use, you give them a new one - "look, our opponents want to ignore actual data".
But now let's say minorities gets a "resiting arrest" charge on top of their original charge more often, because of many factors such as the police bias, the bias of the minorities towards police, etc. (By bias here I mean all spectrum : racism, but also fear of the police etc).
As long as minorities are treated differently upfront, then the model would treat them differently.
"It found that widely used software that assessed the risk of recidivism in criminals was twice as likely to mistakenly flag black defendants as being at a higher risk of committing future crimes. It was also twice as likely to incorrectly flag white defendants as low risk."
Source: https://www.nytimes.com/2016/06/26/opinion/sunday/artificial...
I'm sympathetic, but how do we keep from causing more harm than good here? It seems like blindly ignoring all data that correlates with race is also bad. We do this in some circumstances already (i.e., IQ tests when doing hiring or admissions) and I think it's a defensible point that we're worse off for it.
This is a very theoretical, futuristic red herring. Race factors into decision-making far, far too much. It's like worrying that if someone developed a perfect, bug-free crypto library, then everyone would use it and we'd have a monoculture - not really a present concern, and in fact a good problem to have (nor is it an eloquent analogy, but I have to run).
I think there are fair existing examples both for and against it, so I'm not sure calling either position futuristic or theoretical is fair.
[1]: https://weaponsofmathdestructionbook.com/ [2]: http://www.econtalk.org/archives/2016/10/cathy_oneil_on_1.ht...
It may be easier to objectively measure and correct for undesired (racial, or otherwise) bias in an automated system than it is to objectively measure and correct for bias in human judgement. And even if it's more difficult to correct for undesired bias in an automated system, at least that correction is more likely to be permanent and applied uniformly, compared to things like giving training courses for humans, which will vary in effectiveness across the recipients of the course, and will need to be refreshed frequently.
There are a lot of qualifications and "maybe"s in that argument though.
In my mind, the ownership of the software is what's troubling. First, it's not public, so nobody can look at the guts. And second, the Loomis decision constructs even more walls around the software. It's hard for people to critically examine and change something they can't see.
[1] https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/1... [2] https://arxiv.org/pdf/1609.05807.pdf
https://www.chrisstucchio.com/blog/2016/alien_intelligences_...
Bias correction is possible, but, tautologically, a model whose structure cannot capture a bias cannot correct for that bias. This means that a modeller must either understand the bias, and accommodate for it in their model, or use a model that might be able to capture unknown structures and run the (serious) risk of over-fitting that model.
It's really not clear to me that much is gained from having very precise decisions made about bail and sentencing. Trying to predict the future is a fool's errand, whether a judge does it or a computer. It'd be better to just set fair, uniform standards (particularly for bail where bail should be granted presumptively unless unique circumstances are present).
Unfortunately, using machine learning for sentencing is just the tip of the iceberg. "Scientism" is rife in the criminal justice system. The U.S. Sentencing Guidelines, for example, are utter gibberish. Sentences are calculated to the month using complex formulas: http://www.ussc.gov/guidelines/2016-guidelines-manual/2016-c....
> The total points from subsections (a) through (e) determine the criminal history category in the Sentencing Table in Chapter Five, Part A.
> (a) Add 3 points for each prior sentence of imprisonment exceeding one year and one month.
> (b) Add 2 points for each prior sentence of imprisonment of at least sixty days not counted in (a).
> (c) Add 1 point for each prior sentence not counted in (a) or (b), up to a total of 4 points for this subsection.
> (d) Add 2 points if the defendant committed the instant offense while under any criminal justice sentence, including probation, parole, supervised release, imprisonment, work release, or escape status.
> (e) Add 1 point for each prior sentence resulting from a conviction of a crime of violence that did not receive any points under (a), (b), or (c) above because such sentence was treated as a single sentence, up to a total of 3 points for this subsection.
But it's not like this is based on an empirical statistical model correlating sentences with recidivism or deterrence effects. It's classic scientism, believing that an algorithmic sentence based on completely arbitrary rules is somehow better than an arbitrary sentence handed out by human judgment.
it may not be better, but, isn't it more predictable?
also, doesn't it protect a judge and the justice system against charges of favoritism, discrimination, and other kinds of bias?
i mean, i'm not a lawyer and i truly don't know ... but weren't these the sorts of reasons for coming up with these guidelines in the first place?
(Provably not, actually.)
The best thing we can do is to admit it's not going to be perfect, and use our own reasoning and feelings, and make sure the process is transparent and the people are held accountable.
so, a criminal defendant and their lawyer have no better chance of predicting the sentence handed down given guidelines than they would if there were no guidelines?
Guidelines are not a black box that spits out a number of month in prison.
The Sentencing Guidelines are exactly a white box (a black box has concealed internal operations) that spits out sentences.
OTOH, since they aren't mandatory (by Supreme Court ruling; they were originally statutorily mandatory), they don't produce exactly the same rule-based system as they superficially represent.
I am still trying to clear myself of a wrongful conviction, a misdemeanor for which I was given the maximum allowed sentence of six months in county jail, that was given to me, in spite of blatant inconsistency across half a dozen witness reports and 2 years of court battles, because the judge said, "I think you are lying" and was angry that he had to see me so many times and I didn't just take a plea deal like a good little citizen.
And I wish I was an exception. But this happens everywhere and I doubt the presiding judge could even recount half of these guidelines. It's all arbitrary.
Perhaps the real objection is the lack of transparency behind the calculations?
It seems to me that there is a big range of acts that are classified as sexual offences, starting with a grey area bordering to 'being a dickhead' to some seriously messed up sexuality.
I suppose it's possible to learn how not to be a dickhead, but I also think it's hard to change your sexuality.
Aren't uniform standards as likely as a universal algorithm? There always will be messy edge cases where human judgment is needed. Mandatory sentencing rules worked out badly (though the current U.S. Attorney General Jeff Sessions is still in favor of them!)
Or am I misunderstanding you?
"Numerology" might be a better word.
I know this is somewhat off topic but the lack of advanced options for GPS routing is such a PITA. It would be trivial to add check-boxes for things like:
"I'm towing a trailer, don't make me take dumb lefts across multiple lanes, avoid clustterfawks and don't make me take unnecessary turns"
"Yes I'm wealthy enough to afford an iphone, that doesn't mean I want you to send me through a $5 bridge toll"
"I'm taking a road-trip, send me on a route that uses ten fewer roads even if it takes twenty more minutes, I don't want to have to look for a turn every 10min"
I know tons of options aren't good for the UI but just hiding all that stuff behind an "advanced preferences" menu or something would be nice.
Just a simple tie in to a weather API that increases the cost of route features that are a PITA in snow would be nice (no I don't want to stop on a downhill to take a >90deg left across 40mph traffic in snow thank you very much).
https://www.google.com/search?q=apple+maps+avoid+tolls
https://www.google.com/search?q=google+maps+avoid+tolls
But I agree with the other ones. Quite a few of them might need better recognition of weight restrictions and other such rules on roadways, though. It in theory shouldn't be hard to develop (self-driving tech already either reads signs or sources data from previous mappings of roads where said signs have been read), but I wouldn't be surprised if this info hasn't yet been catalogued yet.
As far as weight restrictions and stuff, I'm talking even simpler than that. The geospatial data and a lot of the traffic flow is already there. It would be an "summer intern size" project to figure out an algorithm that does an ok job identifying route features and traffic flow situations that are not trailer friendly or delivery vehicle friendly (left turns suck). Even for something other than commercial vehicles this is important. Anyone driving a mini-bus with a student/church/summer camp group will run into a route with a really bad intersection or turn.
If truckers are choosing to save a few hundred bucks by using their smartphones, which don't have the trucker-specific features, surely the fault lies with the trucker for using a consumer-grade product, not with the consumer-grade product for not being professional-grade?
There is a whole host of features that are trucker specific. Trying to reconcile the hard copy guide for the possible route and a phone based route planner is something I don't want to imagine.
Know your clearance and pay attention to road signs. Your regular car insurance may cover rental cars, but it probably doesn't cover rental trucks. So if you peel your top on a low underpass, you may find yourself in a world of pain and regret for a long time. Ideally, if you are in a rental truck, you should already know how to get where you are going without GPS assistance.
I see beer trucks and U-Hauls with the top six to twelve inches sheared off by the railroad overpass at least once a month.
In which case it is possible it would eventually discover that in the USA incarceration is very strongly linked to recidivism. It follows that the algorithm might refuse to incarcerate many convicts.
Which is arguably exactly what the algorithm should do, namely what politicians will/can not: employ evidence to advance the methods and improve the outcomes of the criminal justice system.
More crime in an area -> more cops -> more convictions for little stuff -> more big sentences handed out -> more lives ruined -> more crime -> goto start.
Seems like an easier way to break that loop is to decriminalize the "little stuff" instead of capriciously enforcing it. No algorithm is going to be able to do that.
And is that fair? If the last ten guys to come through who looked like me should have had harsher sentences, is it fair that I get that harsher sentence?
It's dangerous to imagine that the algorithms won't be used just as politically as anything else. The difference is, instead of a mostly transparent process in a legislature, the decisions will be made between powerful people and software developers.
With the introduction of explicit algorithms we also hopefully will have the advantage of better data collection, which arguably can make it harder to justify changes that operate in contrast to stated principles. (Contrast much legislation that serves purposes entirely orthogonal to their stated intent - and which survive because of the lack of data and accountability that would reveal their abject failures at all but their ulterior motives)
Any time we think some group of people is going to be naturally above human frailties, we are wrong; it never works. We find software programmers to be more reasonable because they are more like us, and also we are imagining ideal hypothetical developers - plenty of real ones should be nowhere near making laws for others.
Everything is ultimately politics. If there is power in algorithms, then those who desire to influence things will influence the algorithms. It's just a less transparent mechanism.
You will note my choice of words "on whole", which essentially means a statistical aggregation.
But you are absolutely right that everything is politics. I'm not sure I agree about the transparency. Point of interest: You probably know more about the technical exploits the CIA and NSA had than the legal and political mechanisms that permitted them to come about - and almost certainly never will.
The article complains that AI is a black box for the defendant. How is this any different from judge's brain? You can't peek into his mind to figure out what is behind the decision. Judge can give some justifications, but you won't know if those are the real reasons or if the decision is just mostly based on defendants skin color, socioeconomic background or clothing.
Humans can acknowledge and correct for biases in training data much better than computers can.
EDIT: this person put it much better than I have: https://news.ycombinator.com/item?id=14139772
For example just having your sentencing be before lunch or at the end of the day results in harsher punishment simply due to the judge being hungry or tired.
Same goes for other basic things like women getting lighter sentences, minorities harsher sentences, etc.
So AI probably shouldn't be used for sentencing, but that's more to do with all the stakeholders being clueless about the technology. The people asking for it, paying for it, using it, and building it (hey I can just throw some packages together in R right?) don't know what they're doing and are using it to significantly impact the lives of others.
On the other hand, pretending an arbitrary Judge / Jury / sentencing guidelines also don't form a dynamic system with not well understood effects that's equivalent to random AI in terms of output doesn't exactly help anyone either.
At the end of the day you need people who are honorable who you can trust to do the right thing (as much as it sounds like a saturday morning cartoon).
The problem is, "AI" today is glorified pattern matching. So, it learns what "should be" based on the current state. In other words because the pattern exists it will learn precisely that "women should get lighter sentences and minorities should get harsher ones"
Not only that, they are so good at pattern matching that you don't even need to provide the gender or race for them to identify those as a parameter.
It would be great if we could train an AI to eliminate bias, but as it is they are training to reinforce bias that already exists.
Until we get natural language understanding and context aware AI, using AI for sentencing is a terrible idea.
(There's an analogy to vehicle automation, where we accept a level of error in humans that we wouldn't in automation.)
Why should it stop when it has the potential to be much more efficient? Just give it sufficient oversight, and I'm fine with this.
Here a simple metrics would be the average bail and the percentage of people who appeared on court.
On a job like this, I can't really see how a human could beat a machine. And you don't need any kind of "artificial intelligence". This is just standard statistics stuff that every insurance company and bank does.
See http://blogs.discovermagazine.com/notrocketscience/2011/04/1... for the example of the problem we need to solve.
There's nothing in machine learning that bias it into keeping the status quo (differently from people), there's also nothing there biasing it into optimizing the correct thing (also differently from people).
That said, having machines in a consultative position under a judge may be a good thing.
Like - Yes, I'm very social. I play bridge, take my kids to soccer, I'm a member of the PTA. Yes, I exercise. I have a weight set at home, ride my bicycle, play racquetball at the gym. No sir, I don't do drugs, never touched them. No sir, I don't drink either. Yes sir, I do have a degree, two in fact!
Just in case you might need it at a presentencing interview, of course.
I'm not opposed to using other algorithmic methods, but the algorithm needs to be transparent. Though that would be difficult to do outside of some pretty tightly controlled parameters. We can't currently make a system that can take into account arbitrary facts about the case and weigh ethical implications.
Could we demand that they pass a course in machine learning before they use it?