The Dark Past of Algorithms That Associate Appearance and Criminality | Hacker News Reader

The Dark Past of Algorithms That Associate Appearance and Criminality

americanscientist.org

78 pointsnkurz12 comments

This article was eye rollingly heavy on moralizing ultimately to only eventually hit on the point that what the algorithms are measuring is the odds somebody will be convicted and not the odds somebody does illegal things. Appearance is known to dramatically affect your chances at trial.

I seem to recall reading somewhere that good looks affect the sentence but that they play a lesser role in if you're convicted.

I remember reading about a similar result in software that judged job applicants. The system was trained on resumes of past applicants and the hired/not-hired outcome. Unfortunately, instead of providing a neutral guide, it just learned the biases of the humans who had made past hiring decisions. Of course, the designers of the system had excluded demographic information, but the system found proxies like extra-curricular activities that correlated with race, gender or social class.

> that criminals are under-evolved, subhuman beasts

This has everything to do with common attitudes. Ask any person about what kind of person does the kind of crime or moral aberration they are most passionate about and you'll get a similar response. People can't imagine that criminals might have had some understandable train of thought prior to committing a crime; or worse that they may have done exactly the same thing in their shoes.

> These algorithms reportedly make false accusations against students with disabilities who move their faces and hands in atypical ways, and Black students have indicated that they have been required to shine bright lights in their faces so as to have their features detected at all.

Warning: I will be using the term police quite broadly here, but it's anyone involved in any kind of policing or detective activity.

Any algorithm used in policing that produces false negatives needs to be retired until it can be proven they can be used provably equally on people, which I suspect is never. The problem is that most people believe that policing should be more efficient. More efficient police work means violating your privacy, breaking algorithms, and using lazy surveillance and data collection. If you want your freedoms intact then good old fashioned police and detective work needs to be just that; the police need to be smarter, not have smarter tools.

> Yet equating convictions with criminality seems to register with the authors mainly as an empirical flaw, in that using pictures of convicted criminals, but not of the ones who were cleared, introduces a statistical bias that skews the results. They said they were “deeply baffled” at the public outrage in reaction to a study that was intended “for pure academic discussions,” which also suggests an unawareness that their work’s flaws go beyond sloppy statistics.

Absolutely a fair criticism. I'm surprised anyone was baffled that the criminal justice system is highly flawed, other than some mathematicians who lead quite prestigious lives. I've long thought it's pretty suspect that there's a morality bar forming (or that has formed) in programming. A force beyond laws can make you unhireable, what a precedent! Really, I find the idea that we might deny entry to programming even for convicted felons that have served their time problematic. Their insights might be quite useful here.

Edit: Changed "Americans" to "people"

> Ask any American about what kind of person does the kind of crime or moral aberration they are most passionate about and you'll get a similar response > .... > Especially Americans can't imagine...

Why is this targeting Americans? In my home country these same biases exist. There's a "group of people" that are deemed more likely prone to criminal behavior.

For some reason America seems to serve as a lightning rod for these types of criticisms, as if the rest of the world is better somehow.

The American context is easier to understand for most people. We're surrounded by American culture on English-speaking websites. The context of <other country> isn't as easy to understand for people that don't know <other country> well.

I'm certainly not saying your struggles don't exist. I live in America and it'd be disingenuous and probably counter-intuitive for me to create an exhaustive list of all people's who suffer these circumstances.

That's to say, I can really only speak in earnest about the devil that I know.

> People can't imagine that criminals might have had some understandable train of thought prior to committing a crime; or worse that they may have done exactly the same thing in their shoes.

People can imagine this. They understand it. They just implicitly recognize that humans are rationalization machines, that their reasons don't matter. Every single person in human history, no matter how abhorrent, had their reasons, had a whole litany of reasons that they could recite. Plenty of them were able to convince other people they were right, too. That's how you get everything from mass surveillance to genocides.

Now, it's part of being human that we recognize that in extraordinary circumstances, maybe those exigencies can justify or ameliorate your wrongs. But every single person in human history is going to be able to explain to you why that applies to their case, that they had to do it.

But the fact remains nearly all of them didn't.

I'm talking about the way we look at average criminals. I don't think we should be looking at average people like they're monsters.

Of course you'd say that, you have the brainpan of a stagecoach tilter.

https://www.youtube.com/watch?v=ySXpRf3ggJE

I had previously written a critique of this article, for a different forum. I'm reproducing it here, slightly edited for this venue:

With concerns of systemic racial bias such a hot topic today, the subject of hidden bias in machine learning systems driven by "big data" has been a frequent question. An article "The Dark Past of Algorithms That Associate Appearance and Criminality", by Catherine Stinson, appearing in the January-February 2021 issue of American Scientist, attempts to make the point that "Machine learning that links personality and physical traits warrants critical review", and largely succeeds at this task. However, most of the piece attempts to provide that critical review, and at this task it fails.

The article starts off offering phrenology as a close cousin to contemporary efforts to use facial recognition to identify criminals, or as "gaydar". A series of examples of it (mis)use in Chinese surveillance, or to monitor schoolchildren to ensure they pay attention in class, capping this off with the claim that Black students have had to endure bright lights pointed at their faces to enable such systems to work. Such applications do seem unfair, and potentially so disruptive as to be counterproductive. So in this way the author is successful in showing a need for critical review.

Stinson goes on to discuss how some of the derided foundations of phrenology - the idea that particular cognitive functions can each be seated in a particular locale within the brain - has now become "a standard assumption in mainstream neuroscience". However, she then claims that despite this, the science has suffered from sloppy methods. One example study is cited that looked for criminality based on photographic input had biased training data by using police photos to represent the convicts, versus professional photos scraped from the web for nonconvicts. It's easy to see how this would skew the results.

The author first claims, "using pictures of convicted criminals, but not of the ones who were cleared, introduces a statistical bias that skews the results", and that seems a fair point. She also says, "the researchers also admitted that taking court convictions as synonymous with criminality was a 'serious oversight'", but offers no explanation for why that should be. My best guess is that we should be able to assume that a "guilty" verdict is the best judgment we can get as to whether someone is guilty of a given crime; but that other people who are guilty of a crime but never caught will be left out of the data set, so our training would bias the system to give a pass to people who don't get caught. I suppose that's fair, too.

It's at this point that I think things start to derail. Stinson goes on to say,

their framing suggests that criminality is an innate characteristic, rather than a response to social conditions such as poverty or abuse, or a label applied to exert social control. […] Part of what makes the data set questionable on empirical grounds is that who gets labeled “criminal” is hardly value-neutral.

That's largely a philosophical question that she doesn't attempt to back up. But the fact that she does make this point, about extrinsic forces being an important contributor, actually defeats her later argument.

Along the way, she feels the need to throw in the standard anti-HBD argument that "psychologists who studied the heritability of intelligence, such as Cyril Burt and Philippe Rushton, had to play fast and loose with their data to make it look like they had found genuine connections between skull size, race, and IQ." I don't intend to debate that here, there's ample material out there showing the correlation between race and IQ (although I don't know about skull size). Either way, it doesn't seem to be relevant to her actual argument about revelations of criminality from facial characteristics.

Her main argument starts

Complex personal traits such as a tendency to commit crimes are exceedingly unlikely to be genetically linked to appearance in such a way as to be readable from photographs. First, criminality would have to be determined to a significant extent by genes rather than environment. There may be some very weak genetic influences, but any that exist would be washed out by the much larger influence of environment.

This is just wrong. Genes do not need to be the determinant here. There are any number of external factors that can point to someone's criminality:

+ A person's age is a strong clue to criminality, and we can judge age based on facial appearance quite readily

"more than half of all homicides in 2014 were committed by 15-to-29-year olds, who only made up 20 percent of the population" [1]

"The relationship between age and crime is one of the most robust relationships in all of criminology. This relationship shows that crime increases in early adolescence, around the age of 14, peaks in the early to mid 20s, and then declines thereafter." [2]

+ Probably the best clue as to criminality is past criminal history. Jailhouse tattoos (like the tear thing) are probably excellent hints here. There are even studies that claim to relate tattoo type and location to the type of crime that an inmate is incarcerated for

"The effect is even more pronounced for those with tattoos on the head or face, who are around 30% less likely to be murderers. Similar associations can be found for perpetrators of domestic crimes." [3]

+ Property crimes correlates with poverty, and poverty leaves its marks on a person's visage as well. (this is where her previous point about criminality not being an intrinsic trait actually undermines the argument). See, e.g., [4]

She continues:

Second, the genetic markers relevant to criminality would need to be linked in a regular way to genes that determine appearance. This link could happen if genes relevant to criminality were clustered in one section of the genome that happens to be near genes relevant to face shape. For a complex social trait such as criminality, this clustering is extremely unlikely. A much more likely hypothesis is that any association that exists between appearance and criminality works in the opposite direction: A person’s appearance influences how other people treat them, and these social influences are what drives some people to commit crimes (or to be found guilty of them).

She's doubling down on the social influence thing, but see my rebuttal about poverty just above.

But more importantly, her argument is again false. As improbable as it may seem to her, we know for sure that at least some important factors linked to criminality are, indeed, found relatively localized in the genome. It's established that genes on the Y chromosome lead to greater criminality: in other words, it's well documented that men are more likely to be criminals than are women.

Males were convicted of the vast majority of homicides in the United States, representing 89.5% of the total number of offenders. [5]

Her conclusion compares these technologies to cold fusion, saying that continued work on these questions is worse than useless because it's not just wasted, it also serves in "propping up colonial and class structures" - bringing non-scientific politicization into the argument. The trouble is, while she's shown that - like any tool - this facial recognition technology can be misused, she hasn't even tried to look into how it might be used for good.

I've been a subscriber to American Scientist for many years, ever since Scientific American became so overtly political. Over the past few issues, American Scientist has been debating with itself about whether it should go in this direction. The editor's note in the previous issue made me worry that it would be going in this direction, and this article solidifies that. Anybody know of a good outlet for scientific information, a couple notches more in-depth than Discover or Popular Science, but still approachable to a well-educated generalist?

- - - -

[1] https://onlinelibrary.wiley.com/doi/pdf/10.1002/978111851963...

[2] https://www.sagepub.com/sites/default/files/upm-binaries/602...

[3] https://webcache.googleusercontent.com/search?q=cache:https%...

[4] https://www.researchgate.net/publication/221674142_The_dynam...

[5] https://en.wikipedia.org/wiki/Sex_differences_in_crime#Stati...

Technologists tend to not think through the humanist problem very well. Just because you can doesn't always mean you should.

Associations between appearance (as expressed genetically) and that person's capacity for something (intellectually, morally, or otherwise) is plain old eugenics. Given it's sordid history and it's horrible effects on the 20th century, we should be banning this sort of work.

Reply on news.ycombinator.com