Courtroom AI system claims almost 90 percent accuracy in detecting lies

>What are currently the best tech buzzwords that might convince people you have magic pixie dust for sale?

blockchain

The linked study does not support or address lie detection. It is about attacks on AI face recognition. (Making it misdetect or not detect the subject.) This is actual fake reporting. Unless author has made a critical mistake when pasting the link to the study.

I suspect based on the image he meant DARE: https://doubaibai.github.io/DARE/

Which is somewhat less than impressive. The quoted "accuracy" is AUC which weighs all errors the same. It is also not a binary detector. For use in courtroom false positives are much more costly.

Remember this also says nothing about generalization power of the system. Nor how well humans could be trained to detect lies. (Untrained results were pretty impressive.)

Very overblown claims as of now.

There's plenty of science behind micro-expressions and detection of deception.

Paul Ekman [0] is "ranked 59th out of the 100 most cited psychologists of the twentieth century" and I thought the TV show based on his research [1] was very entertaining. (I am surprised that the article used the phrase "micro-expression" without any reference to Ekman's work.)

[0] https://en.wikipedia.org/wiki/Paul_Ekman [1] https://en.wikipedia.org/wiki/Lie_to_Me

The problem I think is, lie detection software is kinda like jet plane software. Even at 99% percent effective can have terrible consequences when its 1% wrong.

What is your skepticism based on?

This is interesting! The ground truth for the study couldn’t have been far removed from the “ground truth”. I am afraid this appears to me as a pattern where any dataset whatsoever is thrown at ML algorithms and the results are heralded as the new way of analyzing data. There is no regard to the source of the data, it’s deficiencies and utter disregard to its applicability to the problem at hand. See the question generation related deep learning papers based on SQUAD dataset.

AI used for selecting Courtroom Extras for Law and Order SVU Season 31.

You're extrapolating a lot here: this is a demonstration in a mock environment. There's no indication that this is going to be rolled out as-is and used as the final arbiter of justice. Yes, denying justice 10% of the time is unacceptable. But that's not what's being presented here.

As an aside, the comment you're referring to was dead on arrival. It looks like that account has been banned.

I agree. But what's the error rate now?

Perhaps one could insist for an explanation and reasonable defense of the outcome by these ML algorithms. Would that be a feasible retort?

They meant https://arxiv.org/abs/1712.04415 (based on a full text arxiv search of the text mentioned in article)

But what is the Jury's alternative (unassisted) effectiveness?

Interesting point though I am not sure how the 5th amendment would apply here. Specifically, I thought that 5th amendment allows you to not answer a question or not testify. After you say something, it looks like using it to make a case against you is a fair game.

you can avoid all this by not testifying.

Accuracy of measuring what? Despite the term, they don't actually detect lies. At best, some of them can detect physiological "events", as when a particular question or word induces acute anxiety. Most interpretation of those events is highly subjective.

The closest thing I've heard of to an objective method, the guilty knowledge test, doesn't center on evaluating the truth of the suspect's statements at all. Basically, the authorities know certain confidential facts about the case, such as the location of a wound, characteristics of a weapon, characteristics of an item stolen by the perpetrator, non-public "calling card" of a serial killer, method used to break into a building, etc.. The interviewer (who should be unaware of the true fact, to prevent something like the Clever Hans effect) is given a list including the true fact and plausible alternatives to incorporate into the questions. The idea is that a reaction specifically to the true fact indicates that the suspect knows something that only the actual perpetrator would know. As far as I know, interpretation of these results is still ultimately subjective because there is simply too much variability in human psychology and physiology to define a useful objective standard of when the test is "failed".

Terrible (50%?). They're so bad their results haven't been admissible in court for years.

I don't think they've been shown to be better than placebo.