Hacker News Reader

Well hold on. COMPAS performs as well as people but costs less than getting a jury together or paying a professional. So I wouldn't say that's uninspiring.

It also performs as well as their simpler model. But they only evaluated the simple model on data from a single county. There's not enough evidence to determine whether that model would perform as well on other data sets. I would presume, perhaps wrongly, that more work has been done to ensure the generalizability of COMPAS, the results of which work militated against the adoption of a simpler algorithm. Possibly because the algorithm has to avoid racial disparities that might creep up in counties with a different demographic mix. Even if the simpler model performs equally well across many jurisdictions, that merely suggests that a simpler non-proprietary algorithm would be better. But we're still choosing between algorithms, not discrediting the idea of algorithmic sentencing altogether.

As long as you are vaguely positivist then we are surely only "choosing between algorithms" - the question is rather how those choices are made. When we consider computerised solutions we are nearly always forced to radically simplify our frame, dispensing with "judgment". This can be an advantage, because it reduces the scope for individual bias, but it can also be a disadvantage, because it reduces the scope for mitigating systemic biases. It is much more difficult to design a perfect system than it is to apply corrections to a merely good one - capturing what those corrections are, however, is very difficult.

As an aside, do you have a source for the cost comparison, because I didn't see anything in the paper?

In the interests of disclosure and not as an assertion of authority: I must admit that I am a lawyer and my perspective is shaped by my dissatisfaction with the profession's lack of scientific rigor both in the academic and practical spheres.

And as an initial matter, I would point out that there is no judicial procedure which relies on algorithmic sentencing or bail decisions without a judge reviewing the decision. COMPAS is merely used to produce a recommendation for a bail amount which a judge then has to review and approve. Defendants are still afforded a hearing where they can object and raise any extenuating circumstances. So to the extent that judicial discretion can address systemic biases, it already does.

And I certainly recognize the value of judicial discretion. Mandatory minimums alone demonstrate how prioritizing the punitive and retributive goals of the judicial system over simple human mercy can amplify the negative effects of racial and other social biases.

However, my experience has been that the legal field has a strong systemic bias against empirical or statistical techniques. Essentially anything that involves math. There's a running, tired joke at every law school that students pursue JDs because they couldn't get a good score on the math GREs (the LSATs in contrast involve no math).

And to clarify since I am at risk of confusing what we mean by the term bias: I am not denying that the legal profession is susceptible to the social biases you mention, but rather I would argue those biases are impossible to address until the deeper, methodological biases in how the legal profession pursues objective truth are addressed.

Algorithmic sentencing and bail setting software can assist there in two ways. First, it ensures the collection of extensive, objective, and standardized data that can be used to evaluate the potential systemic biases you are concerned about. A single, consolidated, structured database with enormous amounts of information about each individual defendant is a gold mine for researchers. Simply put, we cannot even begin to resolve our biases without first identifying them. Software like COMPAS can help (though I'm not sure if it does in this case because I don't think they provide public access to the data).

Second, the introduction of this software gives lawyers a chance to get used to machine learning algorithms and other advanced statistical tools. They're powerful techniques and need to become a larger part of the standard lawyer's tool kit.

I don't think COMPAS is the terminal destination for sentencing software. In fact, I really hope otherwise. But, as a profession, I think we need to start taking steps to modernize our practices, one little step at a time. Each individual step may be imperfect at first, but so long as we maintain judicial discretion and careful oversight, these tools will be a boon for American justice.

And in response to your last question, I don't have a citation on hand. I've read it somewhere before. Basically they replace a professional clerk or low level prosecutor in the DA's office who prepares a bunch of data and biographical information for the bail recommendation to the judge. It ends up being about one FTE a year and the software costs less than that to license.

Thanks for a comprehensive reply, and I apologise for not seeing it earlier. To frame what follows, my perspective is that of someone who spends a lot of time learning about and using these kinds of techniques.

You seem to appreciate the problems with potential biases, and your point about data collection is well taken, though I would note it doesn't require application of algorithmic sentencing. I would, however, urge you to continue to think about the difficulty of automating the sentencing decisions made by a human, from a mathematical perspective - the problem amounts to performing factor analysis by simply throwing away a large number of factors that are difficult to measure. It is not clear to me that that method will be able to capture those difficult parts of the distribution that people intuitively refer to as justice.

I also urge you to continue to think about the difficulty of optimising that automated sentencing procedure, which requires you to select parameters to optimise for. This problem faces similar difficulties - whereas a human might update their judgment on a wide range of observations (potentially including everything they experience), for an automated procedure we typically have to choose a handful of parameters based on some expert intuition about what is important. The choice of those parameters can be critical, and can expose feedbacks that would be naturally corrected by humans.

The previous two points are often dismissed as merely being difficult challenges, rather than fundamental problems with the techniques, and, whilst I am inclined to agree with that perspective, they are difficult challenges, and the decisions that such systems end up making can (and often do, in my experience) have consequences that weren't expected.

This leads to my political reason for being cautious about these systems, which is motivated precisely by the fact that they can be cheaper, at least when one is only making a comparison based on decisions per dollar, or something along those lines. I am happy for these systems to be used by informed practitioners as a mechanical aid to their decision making, but cost savings are such an evergreen issue that I fear pressures to dispense with the human and "make do" with the automated system will become too strong to bear (for the politicians, not the lawyers).

Ultimately one can imagine a system that is constantly improved over time (an expensive prospect) could result in more objective sentencing - my concerns are that uncaring politicians will exploit the availability of the techniques to cut costs, and that once those costs are cut it will be difficult to justify the (relatively) increased expenditures necessary to fund improvements. I don't have your insider's perspective, but I suppose overall I struggle to accept that saving costs in this way is one of the most necessary reforms to the American justice system.

The COMPAS system seems to have the same or better performance than the others methods they compared it to (including humans), so it seems to work pretty well.

I find it weird that an algorithm doing better or same than humans is called 'uninspiring'. Maybe people have too high expectations of what AI can do ?

The discussion in the paper describes COMPAS as "no more accurate or fair than the predictions of people with little to no criminal justice expertise who responded to an online survey", an assessment that doesn't seem unreasonable given the results presented. So, COMPAS is uninspiring because it provides no demonstrable benefits over even a crude analog of existing methods. Furthermore, it is not competitive against a two-feature linear regression model which only considers age and the number of previous convictions.

As it happens, however, I agree with you - people do have unrealistic expectations of what can be achieved with AI. Replacing humans is, in many domains, a more difficult prospect than might be imagined, because so much of what we do is implicit. Suggesting that a particular problem is "solved" because a system achieves non-significant parity on a handful of measures is short-sighted to say the least.

One question that would be interesting to explore is: what is the predictive score of judges/juries/etc? Is it more or less variable than the COMPAS score?

In other words, do judges perform better than COMPAS? And slightly less important: is the variance lower?

If it isn't, then there is no point arguing whether COMPAS is risky: it would be less so than the alternative.

Although I suppose that the 'crowd of non-experts' is essentially what a jury is and they had about the same performance.

At any rate, it sounds like much less of a hassle to input characteristics into COMPAS and get a probability than getting a jury together.

Besides, I think I'd be much more worried that a jury could influence each other and bias its decision than about am algorithm making unfair predictions.