Hacker News Reader

Comment by onos | Hacker News Reader

It seems the appropriate method to judge this approach is in comparison to the baseline referenced at the start: humans making “black box” judgement calls that are subject to the individual judge’s biases. If the computer programmed decision making were (already is?) transparent, then we could critique it and work to ensure something resembling fair is encoded.

View on HN

"Computer programmed decision making" is anything but transparent currently. Just look at the example of Northpointe, Inc.'s COMPAS ("Correctional Offender Management Profiling for Alternative Sanctions") tool, which purports to measure recidivism risk. Even under threat of lawsuit, Northpointe refused to reveal the underlying source because it is a "protected trade secret."[1]

Are judges biased? Of course, and that is not acceptable. However, the answer is not to surrender our system of justice to unaccountable commercial actors.

1. https://www.uclalawreview.org/injustice-ex-machina-predictiv...

“Compas” was also the name of the test used by Wisconsin community colleges to assess students.

It was legally required that all community college students take it, even if they were only taking a single course.

It seemed like a really corrupt arrangement. Want to take a language class to complete a high school requirement? You have to pay $100 to a for profit company to assess your math skills.

Seems community colleges no longer use a test branded as “Compas”.

Edit: it was “Compass” and discontinued in 2015 over concerns of its accuracy.

https://www.insidehighered.com/news/2015/06/18/act-drops-pop...

How would you define 'fair'? I've a few examples, gathered from being alive and hearing what other people think for a few decades.

1. It's fair to punish someone else if you let another off the hook

2. It's fair to treat someone more harshly if you don't like what they've done

3. It's fair to turn the other cheek because they're a friend

4. It's fair to treat someone differently because of where they came from or their skin colour

5. It's fair to give someone a pass because they did you a favour

6. It's fair to be unfair because you owe someone a favour

7. It's unfair to be a victim who doesn't get justice

8. It's unfair to be an innocent person who is prosecuted for someone else's crime

9. It's unfair to not get what you wait

10. It's fair to get what you want

You will find a variation of all of these examples across the world right now.

The point is, fairness is loaded with bias and therefore any algorithm that tries to deal with 'fairness' is going to inherit the bias of those who designed it.

You make really good points. Life is bizarre and human culture is extremely contradictory and difficult to quantify.

And what's worse our definition of 'fair' maybe advancing just like culture is constantly advancing.

African Americans may have been treated poorly by the algorithm back in the 1700s when they we're considered less than a full person by the legal system for example. And maybe in the future drug offenses may be considered not a big deal.

They call the Constitution a living document.

This algorithm may have to be a living algorithm.

What if it was open source and constantly updated and reviewed to reach consensus?

There would probably also have to be human appeal processes.

I definitely think this is intriguing as a first-level sentencing determination though.

> African Americans may have been treated poorly by the algorithm back in the 1700s

Fucking hell, man.

I completely fail to see what's the issue with what OP said, care to explain?

It is completely accurate, algorithms are built by people whom live inmersed on their own temporary and geographically atomized cultures, just look at modern supposed ""unbiased"" algorithms used to ""prevent crime"", it is just racial biases baked in

I am super confused as well. Lol.

This website is really strange.

I suspect the "may have" is what set people off.

Thanks for letting me know.

So strange.

The 'algorithm' is the noun object of the preposition.

Not society or history .

Is modern English education really that poor?

Or is there purposeful misinterpretation in order to be offended?

I just don't get it.

When I read it, as a native speaker, I read "treated poorly by the algorithm" as being a term for the behavior of culture and systems of the time, and not a hypothetical algorithm from the future that took as input the culture of the time and made decisions about fairness.

I think it was the use of "the algorithm" and not "an algorithm" that most affected my reading there - "poorly by an algorithm" comes across, to me, as "poorly by some algorithm", rather than "poorly by _the_ algorithm", implying a specific instance of a thing, already known by the reader or referred back to later, doing it.

Consider "treated poorly by the system" versus "treated poorly by such a system", which has an even clearer connotation, to me.

My first reading as a native speaker was that "the algorithm" meant "the Man". But after several readings, I think he meant that this algorithm would categorize someone who was legally considered less than a citizen as a more likely recidivist. Which is an interesting point. Particularly in the US where there's a notion of illegal immigrants having already broken the law. Would this algorithm consider them to be more likely to be repeat offenders if it were weighted to treat noncitizenship as a factor, or overstaying a visa as a crime? And if so, then why wouldn't it weight slaves as more likely to reoffend. And if it did that, what would stop it from weighting the descendants of slaves the aame way?

I thought the US was somewhat unusual in its lax treatment of illegal immigration. However lying to or sneaking past border officials to enter a country is literally a crime yes (certainly for foreign nationals, and probably not kosher for citizens to do either). Though entering on a visitor visa and overstaying is just a civil violation with a civil penalty (deportation). If a citizen had a record of using false identity documents or trying to evade legally-enforced border crossings, a fair algorithm would presumably have to take that into account just as well.

Interesting example of how context can affect the sentencing and seriousness of the crime!

Well if you enter as a legal visitor through a border checkpoint you give the govt the opportunity to do security checks, restrict import of various things, charge customs taxes. Plus they have an idea you are there. It makes sense that sidestepping some or all of that screening and record-keeping will be treated more harshly.

Yes! Exactly. If humans wrote an algorithm, to determine sentencing of crimes, it's possible that biases and predjudices of the time would be included in the algorithm because they don't realize or don't want to admit that their current ways are unfair.

Much like the constitution did!

But the constitution can be amended to be more fair and so the abstract sentancing algorithm, whatever it may be, would have to be amendable as well.

I hope that statements not racist!

I see how the parents wording is confusing. IMO, they weren't saying African Americans "may have been treated poorly" in the past (I suspect this is quite uncontentious), they were saying that even this algorithm, now, applied back in the 1700s may not have eliminated bias. Instead, it may have simply applied the prevailing biases emotionlessly. It's hard to see the inequities of your system when you participate in it.

> African Americans may have been treated poorly by the algorithm back in the 1700s

Tell me about how your algorithm enslaved people.

If you're asking if I took a time machine back to the 1700's to use an imaginary algorithm, to enslave people, I assure you good sir I did not.

This is indeed one of the potential problems with inventing an algorithm to sentance people, as I mentioned before, is that it would be written to match the biases of the time period.

For context, the issue was, as many posters said, this:

> African Americans may have been treated poorly

On my part, I'm british. "Fucking hell, man!" for me is a way of being exasperated; I was surprised with reading that. It's not a condemnation, it's surprise.

And a phrase containing 'may have' is begging for equivocation, especially when referring to black people as 'african americans' while talking about their treatment.

> Black people may have been treated poorly

^ this is the sentence that I read

Your outrage radar is a little too sensitive pal.

You don't see any problem with the "may have"?

I think what OP meant is "if the algorithm were designed (or perhaps implemented) in the 1700s according to standards of fairness held by those in power at that time, then it might have treated African Americans unfairly (despite being an algorithm)". I'm not a mind reader but that seems to make some sense given the context of the discussion.

You don't have to go back to the 1700s to find African Americans being treated poorly by the judicial system.

It's a weird shifting of the sands that isn't necessary to the conversation, when the context is a subject that is causing real harm now.

That supports my point even more.

If certain groups of people are being treated poorly in modern times then an algorithm that sentences people would possibly include that bias as well.

Is that statement offensive to you?

> If certain groups of people are being treated poorly in modern times then an algorithm that sentences people would possibly include that bias as well.

I’d even change the word “would” to “will” as that has already occurred time and again.

I think you two are violently agreeing with one another, but misunderstanding one another as well.

Come on. Plantation era slavery was great! Free housing, free food, a lot of excercise, no Bay Area rent prices.

What's the big deal?

I can actually see a use case in earmarking court decisions for review that are far off from what may have been expected.

Handing over the decision altogether, however, is like handing justice over to a theatrical magician ("The Great Fully Automated Barnum Court"): it's commercially incentivized, you may not know the trick, because it's a trade secret, and it's not the real thing, either. (But, exactly because of this, people will fall for it.) Also, judgement is very much about discerning correlation from causation, so correlation based data may not be the best foundation, to begin with.

This reminded me of something I read, and I can't recall whether it was in Argentina under the junta, or in North Korea, or in the USSR ...but the gist was that people sentenced for minor crimes or for irritating the state were essentially assigned random sentences and punishments, divorced from or even absurdly light or harsh relative to their suppose crimes, as a way of terrorizing the population.

Or maybe I'm thinking of a Kafka story, or Dostoevsky's mock execution. Or 1984. Shit. I've lost it.

[edit] Might have been from the Killing Fields. Wow, this is really dredging up a lot of wonderful historical references...

Don't forget the film Brazil, where a bug-induced misprinted warrant leads to an obviously wrong person being arrested and then interrogated to death, because how could the machine be wrong?

"Something resembling fair", i.e. unfair? We already have that.

Not that the idea of codifying fairness is new; a certain guy named Hammurabi started the trend a while ago.

The result of humanity's collective attempt to create a Transparent Fairness Flowchart still leaves much to be desired.

Replacing codex with code, will, however, streamline the process by entirely removing (the already meager) accountability for bad decisions, as well making it effectively impossible to correct them:

"Computer says no" [1]

[1] https://youtu.be/0n_Ty_72Qds

Reply on news.ycombinator.com