Hacker News Reader

Hacker News

Comment by a3n | Hacker News Reader

Full thread

a3n

Consider:

- Write the algorithm out.

- Extract all the decision points.

- Pay legal interns to collect all the data, apply the data to the decision points.

- Produce a report.

- Produce a recommendation.

- The judge sentences based on that process.

And now tell the defense that they have no right to any knowledge about the above process. Not the decision points, and not what data was fed to what decision, and not even what the data was.

If you can't release that information to the defense because of trade secrets, then, it seems to me, it's not appropriate as a sentencing tool.

Of course, IANAL.

View on HN

This same stuff is used to decide things like probation length, if someone gets a parole, and what level of severity a sex offender is set at. All of this stuff should be freely available online if it is used in any legal decision making process. As it stands even the police who use it don't know how it decides anything.

You could do automatic differentiation. Take each input, race, sex, age, ... and see how the output changes. If one field has a huge swing it might be a good basis for claiming variable X based discrimination.

And if there's not a small number of perturbations, then it's the proprietary combination of a large number of small, individual perturbations. But either way, since it's proprietary, it's not practically refutable.

Not to mention you'd never get access to do the differentiation, as it would rely on access to the proprietary process; the company would have to be compelled to cooperate.

Sex would have a major impact. At least in the US our justice system is extremely sexist, far more than it is racist. I once saw sex offender guidelines that automatically put the default sex offender sentence as medium for males and low for females (the default was used before an official evaluation was completed, and I would bet some money that the official evaluations discriminated based on sex).

While I don't disagree our legal system is also racist, the racism pales in comparison to the current level of sexism.

That's not a bug, it's a feature! User jawns posted some of their marketing copy elsewhere in this thread:

"COMPAS offers separate norms for males, females, community and incarcerated populations"

And they call that "defensible decisions"! In other words, they want to make sure the software doesn't rock the boat. If there's a bias, they want to preserve it so it doesn't look like their algorithms are "weird".

> In other words, they want to make sure the software doesn't rock the boat. If there's a bias, they want to preserve it so it doesn't look like their algorithms are "weird".

Isn't that what judges do when they follow precedent?

Legal precedents tend not to explicitly single out one gender, race or religion.

No, and judges don't always follow precedent (they theoretically have to when it is binding, but no precedent is binding on the highest court, even it's own.)

When they follow precedent, they are not matching outcomes to outcomes to avoid looking wierd, they are applying what was previously established as the formal decision rule.

That's a very different thing that attempting to preserve biases that may not even reflect the formally-annoubced decision rules in the cases that produced them, but may represent silent deviations from the formal rules.

Reply on news.ycombinator.com