I'd be shocked to learn that the IRS does not use some form of mechanical computation to determine probabilistically whether or not a given taxpayer should be audited for possible tax fraud in a given year.
But however unpleasant it may be, an audit is not a punishment. (Someone is abusing their power if so.) Not anymore than being selected for jury duty is a punishment.
Also, you used the word "probabilistically". We should not be assigning sentences based on the roll of a die, eh?
Insurance companies use machines and statistical models to set rates. But they also have strong feedback to improve accuracy. They lose money if they get it wrong.
Here we are talking about judgement.
I know far too much about computers to ever credit the idea that these machines can replace a human judge. The people who have put these things into practice are deeply incompetent and should be voted out of office and replaced by folks who won't devolve to crude machine-worship.
Probability is not "roll of a die", it is how reasoning actually works. Human judgement is nothing but a heuristic-driven, semi-conscious probability calculation. When you have data that's more quantified than "vague gut feeling", doing things the right, formal, explicit way leads to better results - that's why machines are getting much better at reasoning than humans already.
So if IRS wasn't using probabilistic models to decide whom to audit, I'd say they'd be sucking at their jobs.
Machines don't reason, they compute. It's not the same thing at all. We can reason about things that cannot be represented in any symbolic fashion. Put another way, no machine can tell you: which schools to attend, whether to marry that person, what to have for lunch, the optimal route to work.
Deciding whom to audit is different than deciding how long you should stay in jail.
The latter situation is most definitely an algorithm whether it be administered by a computer or human -- e.g. 1. is there a person in front of me. 2. Is this person male? 3. Is this person dark-skinned? 4. Is this person within X meters and inside a timeframe of Y minutes from a reported crime?
As for making judgments probabilistically; hate to break it to you, but that's already the status quo. Exhibit A: Probable cause.
And probabilistic thinking is most definitely behind sentencing guidelines. Let's take the recent and still controversial case of Brock Turner, the ex-Stanford swimmer convicted of sexual assault [0].
Aren't the following aspects of the Turner case all a manifestation of probabilities?
0. That his bail was set at $150,000, based on the likelihood of an accused (and not yet guilty) rapist of his status would break bond.
0.5 The likelihood that he did indeed sexually assault the victim and was not ignorant as to her state of consciousness or consent.
1. The recommended minimum sentence of 2 years in prison for his convicted crime.
2. The probation officer's recommendation that he be given a "moderate" county jail sentence.
3. The judge's determination that Turner, based on his lack of prior criminal record, be given 6 months in county jail.
4. The county jail's determination that Turner serve only 3 months in county jail because of good behavior.
5. The post-incarceration penalty that comes with Turner to be thought of as a Tier III sex offender, which means he has to register 4 times a year (presumably based on some statistic that that's enough polling frequency to adequately track him), for the rest of his life.
Note: I'm not implying that you agree with any of these aspects of the case. Just saying that if you're concerned about statistics and likelihood playing a part in our criminal system, then that ship has long sailed. In defense of the machines, if a machine had been the sole determiner of Turner's sentence, we'd at least be able to audit it and know how it weighted things. With the human judge, we don't have a clue as to whether the judge's personal relationship with Stanford was a factor, or even if he read the probation officer's report correctly when giving the seemingly light sentence. Hell, we don't even know how the probation officer interpreted the victim's wishes [1]
[0] http://www.daytondailynews.com/news/news/crime-law/greene-co...
[1] http://www.cbsnews.com/news/probation-official-called-brock-...
The point you're making about the Brock Turner case is well taken; the judge was another standout Stanford athlete in a sport that doesn't have a professional level. My guess is that these algorithms won't be devised by poor black people who dropped out of high school, but they will have a lot to say about how much worse poor black people who dropped out of high school are than people from the demographics of the people who do devise them.
What's your angle?
For me, it's really simple: The day that a machine decides how long I spend in a cage we are going to have a serious problem.
Maybe they don't actually use physical computers to perform the computation (though that'd be a bit weird if they didn't), but I think that's an irrelevant implementation detail as the Supreme Court has upheld the use of an unyielding mathematical formula to determine the length of a prisoner's sentence:
This doesn't seem as bad as the OP, but it's still an abdication of responsibility. Automated authority.
Let me point out something: why are these systems being deployed to decide sentencing, as opposed to trying to figure out what will actually affect recidivism?
I haven't read about that in any of the articles on this: is anyone paying attention to e.g. whether a given sentencing policy is actually reducing crime?
I don't doubt that machines can help us understand and manage ourselves and our complex society, but this seems like machine-worship to reinforce our atavistic ideas around crime and punishment.
I'm glad some people put the effort in rather than just chasing positive feels.
Mass democracy is the root of this problem. One proof is observing the way in which democratic decision-making has devolved chiefly into emotional appeals to in-group moralistic bromides. To me, the degree to which one seriously desires rational thinking as a basis for lawmaking is the degree to which one desires a reduction in the number of cooks in the kitchen.
Rationality can lead you astray, you have to listen to your heart.
Emotions can lead you astray, you have to listen to your head.
If every audit was accompanied by a check for $3000 or so, a large portion of the people audited would feel neutral about the process.
Computer algorithms are much easier to improve than low-paid human clerks, so there's that too.
Care to explain?
The second one follows the principle of American Army - "If we don't know what we are doing, the enemy certainly can't anticipate our future actions". A purely random selection cannot be gamed directly because it doesn't change its behavior in response to external input. However, it certainly can be meta-gamed; e.g. some people could take advantage from the very fact such algorithm is used in the first place. This circles back to the same point I made about the "perfect" algorithm - humans are just too damn good at metagaming.
Still - my primary point was that a) humans are a gameable algorithm too, and b) humans are pretty low-hanging fruit as algorithms go. We can, and should, do better.
[0] - http://www.allproudamericans.com/paimages/here-is-why-americ...