Hacker News Reader

Comment by SketchySeaBeast | Hacker News Reader

>Algorithms would be another, better way to know exactly what makes the determination if a person should be released awaiting trial or not.

Isn't it a source of constant concern that the algorithms that end up being used in law enforcement inevitably end up employing the same stereotypes that their flesh and blood counterparts have?

View on HN

But wouldn't algorithms be easy to examine? At the least you can test it with similar circumstances and modify the "race" variable and see if the results are consistent or different. Humans have many self-image protective layers when questioned (e.g. "I'm not racist, I have black friends!"), Bbut hopefully algorithms are more transparent...

Machine learning models in particular are famously, fiendishly difficult to examine.

While agree with your general sentiment, you ought to point out that this very much depends on the algorithm in question. Things like decision trees/forests are quite observable.

Race variable is often correlated with other data.

Look at red lining in real estate/banking. They marked black neighbourhoods with red boxes, white neighbourhoods with green. The bank would approve business and home loans depending on if you were in a red or green area. Algorithmically the banks did not engage in a racist practice but systemically they are.

But did they mark red areas because they were black neighborhoods or because historically people in those neighborhoods defaulted on loans?

An algorithm as simple as "mark neighborhoods with high default rates as red, low default rates as green" would probably turn out to be "racist" even if it is also flagging poor white trailer parks with red.

There's no easy way to solve that

At least in the case of historic redlining (i.e. where the term came from), we don't have to wonder - there has been a ton of research done on this, and it was pretty clearly a racially motivated discriminatory practice. A fact which resulting in, among other measures, the fair housing act in the late 60s....

And it is very well known how to test and correct for that. In a human, however, it cannot be corrected. As you point out, it can be very hard to see what rules have to do with races.

I would argue that in some cases it's bloody hard to see that you're doing it yourself.

> But wouldn't algorithms be easy to examine?

Examine by who? And would they be trustworthy?

An algorithm can be racially biased, or have biases that significantly correlate with race, without actually having a "race" variable.

I think what he's suggesting is feeding the input data that differs only on that variable, and if the algorithm produces differing output, then it is by definition racist (and then you send the authors back to the drawing board to figure out why).

That's fair.

They would be, but it turns out that humans make their own similar algorithms without realizing it.

Reply on news.ycombinator.com