Hacker News Reader

Comment by Permit | Hacker News Reader

This article is talking about two very different things:

1. Using video/audio of a person to make statements about their personality.

2. Using video/audio of a person to identify them.

The former seems as though it’s largely pseudoscience and should be avoided on the basis that it simply does not work.

The latter may be innaccurate, biased and problematic but it does not seem to qualify as pseudoscience. I would imagine such systems will continue to improve in accuracy. Do others really consider facial recognition and voice recognition (as the title suggests) to be like phrenology?

View on HN

> Do others really consider facial recognition and voice recognition (as the title suggests) to be like phrenology?

I won't jump to that conclusion but I'm skeptical. Phrenology seemed like it worked (to some) until real rigor and statistically significant sample sizes put it to rest. I suspect that may happen to facial and voice recognition too - if limited to a small, known group size or heavily controlled circumstances it works well, but the second you apply it to nontrivial real world populations over time it loses most of its predictive power. Obviously even in limited applications the technology is far more useful than phrenology ever was, just going by the number of people using Face ID.

At the end of the day, the predictions facial/voice recognition algorithms make are far more specific and obviously testable than phrenology ever was, but we don't even have conclusive evidence that humans can precisely identify individuals using only sight or sound without a mental context that would approach a general AI. Even parents, for example, can have trouble differentiating identical twins without contextual clues like personality traits, schedule, or preferred fashion.

IMHO it all relates to what do you mean by "precisely identify" - if we're talking about automating some process that's currently done by humans, then it does not to be able to distinguish identical twins, it only needs to be roughly comparable to what the humans can do. And the "benchmark human" is not a close relative using contextual clues, it's some bored overworked clerk who'd otherwise be looking at the same pictures.

I think whether or not these methods are pseudoscience is irrelevant. Why?

Even if phrenology had worked most of the time, it would still be a terrible idea. Even if phrenologists could classify criminals with 99% accuracy from the shape of their skull, they'd still be screwing over a ton innocent people when their methods were applied to large populations. Putting somebody in prison because of the shape of their skull is a terrible proposition, even if you get it right more often than not.

That’s sad, but we already do lots of statistical analysis in good or bad ways. It would only help to shed more light on bad ways if we digitize it further. It’s unclear to me if what you assume about that digital racism software isn’t already there (e.g. detectives who prioritize suspects by skull shape or HRs doing the same). The fact that we cannot see it doesn’t make it less terrible.

Edit: I’m basing on a premise that working with a lot of subjects, a human inevitably creates a structure in their brain similar to what we discuss (professional deformation is a thing). I bet that it’s often even worse than $subj because a human mind tends to simplify its job for energy consumption reasons.

If it worked there could have been a effort around Applied Phrenology where bumps are added to peoples skulls to improve behavior

I feel the same way about automatic driving.

Even if the accuracy and error rate surpass humans, every death due to an AV failing is one death too much. However, most people seem to be happy with a failure rate lower then my grandma...

Human eyewitness testimony is also imperfect. How often do you see someone or hear a voice that you think you recognize only to find out it was someone else?

What you are arguing is that voice and face recognition are to be accepted as absolute fact, which is not how they ought to be treated. They shouldn't be considered any more reliable than a human, and any use otherwise should be discouraged. Don't throw the proverbial baby out with the bathwater.

Human eyewitness relates to an event. "Did you see this person on the night of January 14" etc. Phrenology is based on immutable characteristics of a person's physiology. Predicting criminality based on any immutable feature seems categorically wrong to me. If facial and voice recognition are used in relation to specific events, that's one thing, but using them to predict some sort of innate propensity for doing crime sounds like pure bathwater.

You might want to check out the bit in the article mentioning the study that claimed you could figure out a person's sexual orientation from their photo.

There are also many well-known algorithms that attempt to predict how nervous or angry a person is from a photo.

Edit: as for criticisms of image recognition systems themselves, they have different false positive rates for different skin colors.

What is your basis for saying that (1) is pseudoscience? A mere photo can be used guess political preferences fairly well. Why not personality?

It may not be pseudoscience. But even if the methods are accurate, it is still prejudice.

> (countable) An adverse judgment or opinion formed beforehand or without knowledge of the facts.

> (countable) Any preconceived opinion or feeling, whether positive or negative.

Suppose for argument's sake that it is fairly accurate.

What would then be the argument that we shouldn't use this information to, say, set insurance premiums, when we already happily accept the use of other prejudicial information such as age and gender to do so?

> we already happily accept the use of other prejudicial information

False premise, I do not happily accept this. I begrudgingly accept it, only because I don't think I have the power to change it. But by speaking out against the automation of prejudice in other domains before those systems have become mainstream, I hope I might make a difference.

I think the cost of car insurance should be a function of your past driving history and the price of your car (if the insurance policy covers damage to your car.) Pulling race, gender, sex or age into the equation might make insurance companies more profitable, but I do not like it.

Why?

In the (theoretical, unrealistic) limit of perfect competition, supposing that the profit margin is the same in both cases, wouldn’t the average purchaser of car insurance would be better off if these things were taken into account than if they weren’t?

Ok, maybe you don’t think averages are the thing to care about.

Uh,

Ok so if you draw the demand curves (not straight lines) for car insurance for both types of people (those born with and without street racing symbols on their irises) And we consider the cases where the insurance companies can set a price that depends on the type, uhh, Well,

Hm, ok yeah I guess those with racing-eyes get a worse deal in the case that they can be discriminated against. (Here I am using “discriminated against” in what is meant to be a way that doesn’t make a value judgment)

But, the point of insurance is not to produce equal outcomes between people. The point of insurance is to reduce the variance in outcomes for each person with as small as possible a worsening of the average outcome for that person. If what you are doing is trying to make outcomes equal between groups, what you are doing is no longer just insurance, but a subsidy.

Is it really more efficient to have the prices be required to be the same regardless of racing-eyes, than it would be to just directly tax those with non-racing-eyes to subsidize those with non-racing-eyes?

Maybe. I haven’t done the math.

> profit margin is the same in both cases

Profit margins would actually be bigger in the case of more profiling. Absent profiling, insurers open themselves up to much more negative selection (e.g. old people being more likely to purchase cheap health insurance that's subsidised by the young), which has the effect of increasing premiums for the customer, decreasing margins for the insurer and chasing away the customers that are the least likely to need to make a claim.

Does this hold in the perfect competition limit? Doesn’t the perfect competition limit have profit margins of approximately zero?

Who's "we"? An outcome of EU gender equality law was that car insurance premiums may no longer depend on gender even though it is correlated with risk.

> A mere photo can be used [to] guess political preferences fairly well

Really? Could you provide me with a link? It seems counter-intuitive. I look pretty much like my barber, but he's right-wing tradcon and I'm oldskool left-wing.

Nope, but your example nails it.

It’s fairly straightforward to guess someone’s age, gender, race and maybe some cultural markers from a photograph.

Those demographic factors are correlated with people’s political views: old white men tend to be Republican, folks with rainbow hair tend to skew liberal, etc.

On a group level, this seems to let your predict political views, but there’s no mechanism that lets it work for an individual.

No, check out the study I linked.

They controlled for race, gender, etc.

And human guessing was just 55 percent accurate versus 72 percent for the ML model.

Sure, but look at what cues their system uses:

"The highest predictive power was afforded by head orientation (58%), followed by emotional expression (57%). Liberals tended to face the camera more directly, were more likely to express surprise, and less likely to express disgust. "

The system isn't telling you anything about the person themselves; it's picking up on artifacts about how different subcultures present themselves in a public profile.

It's your speculation that the underlying reason these features work is because individuals are adopting the behavioral norms of a subculture. This may or may not be the case.

Also, some dimensions of personality are correlated with political belief, so even if these features weren't useful beyond predicting political belief (which itself is an unsubstantiated/speculative claim), there should still be some transitive predictive utility there.

https://www.nature.com/articles/s41598-020-79310-1

Thanks, that's very interesting. The other linked studies are, too.

A more precise term of art is "Speaker Recognition" as opposed to "Voice Recognition", so as not to be confused with "Speech Recognition".

https://en.wikipedia.org/wiki/Speaker_recognition

>Speaker recognition is the identification of a person from characteristics of voices. It is used to answer the question "Who is speaking?" The term voice recognition can refer to speaker recognition or speech recognition. Speaker verification (also called speaker authentication) contrasts with identification, and speaker recognition differs from speaker diarisation (recognizing when the same speaker is speaking).

https://en.wikipedia.org/wiki/Speaker_diarisation

>Recognizing the speaker can simplify the task of translating speech in systems that have been trained on specific voices or it can be used to authenticate or verify the identity of a speaker as part of a security process. Speaker recognition has a history dating back some four decades as of 2019 and uses the acoustic features of speech that have been found to differ between individuals. These acoustic patterns reflect both anatomy and learned behavioral patterns.

It's actually been used in criminal cases:

>Speaker recognition may also be used in criminal investigations, such as those of the 2014 executions of, amongst others, James Foley and Steven Sotloff.

https://www.theguardian.com/media/2014/sep/02/steven-sotloff...

>An investigation is under way to establish whether the man dubbed Jihadi John is behind a second murder after a British-accented man was shown in the video depicting the killing of Steven Sotloff.

>Security sources said that although there were similarities between the voice on the film that emerged on Tuesday and that depicting the murder of James Foley a fortnight ago, the figure is largely hidden in black clothing.

>A man with a British accent was seen in an Islamic State video posted last month in which the American journalist James Foley was beheaded. He was dubbed Jihadi John after one former hostage spoke about three Britons, collectively know as the Beatles, who were among their captors, one of whom went by the name of John. In both videos, the speaker has what appears to be a London accent.

some theory of social identity builds a construct like .. the more socially important (for whatever reasons) the person is, the more detail and currency are in the ID or profile, by many measures. It would apply here like - a lot of detail in identifying a television personality going through your security gates, and as a side-effect a lot of pressure on a person that happens to look a lot like that personality; but ordinary people of ordinary means would have both less detail overall to ID them, and more errors in that, overlapping with others.

Thereby, you would get many effects, like people who fit certain demographic and cultural slots at whatever place and time, get a lot of false positives due to no fault of most of them. other examples possible..

Reply on news.ycombinator.com