Defense of Amazon's Face Recognition Tool Undermined by Only Known Police Client
gizmodo.com
gizmodo.com
That's worrisome indeed. Understanding confidence thresholds is essential to using software like this. Thinking they don't "utilize" one means they don't understand the first thing about it.
Thresholding is not the only way to make use of a classifier that ouputs a confidence score. In particular, the slide shown in the article indicates that they use the confidence score to sort search results in descending order. That means that there is no fixed threshold and the PIO's statement is not worrisome at all.
The system described is much less dangerous (in terms of overconfident decisions) than one that only returns results matching a threshold, because it means that the user also occasionally sees low-confidence results and learns that they can't simply defer decisions to the machine.
It is another tool to justify police action with a low true positive rate.
"We were following the AI leads" is the perfect excuse for parallel construction[1] (evidence laundering).
If Rekognition supports limiting the search to the people in a specific town (or other small group), it might be possible to force someone to be somewhere on the list of matches. That would give bad actors the perfect tool for manufacturing "probable cause".
You're assuming that there is a wide distribution of confidence levels in every search. Its very possible with grainy footage that no result is above a reasonable threshold, but the results will still show an ordered list.
Without _some_ threshold or training in what these kind of scores mean, its easy for a user to think "well they were the best match" and take a negative action like bringing them in for questioning or introducing the match as evidence, even if its only 10% +/- 5%.
(I could be missing something here)
They probably have at least one person (or several) who knows what they're doing. After discussing the situation with the department They probably just told the department to set it to whatever the minimum is because the department probably expressed a desire for the maximum possible number of matches because they want to manually pick through the list rather than trust the (new and untested) machine to spit out a short list.
As other commenters have mentioned, setting the software wide open has the side benefit of exposing the users to a lot of obvious false positives which (hopefully) prevents them from forming a habit of assuming the machine is usually or always right. Unless there's a drastic increase in transparency I'm still not a fan of the police deploying facial recognition tech.
This is essential just a tool to quickly filter a very large data set down to possible matches. Interestingly enough, the police and many of the letter agencies have already had software that does this for over 25 years. Rekogition will likely perform better, but it still seems weird to me that Amazon would make this move at all. What is the impetus? They obviously aren't going to make money off of this, compared to their other income sources.
https://www.theverge.com/2019/1/25/18197137/amazon-rekogniti...
It does provide leads to police officers, which is a great tool. Many criminals are recidivists and are already existing in the database.
Many police officers know their 'clientele' very well by face and name, so they can run checks on the computer anytime they want as long as it is justified. I don't see anything in using a automated way of doing it by picture.
From now on, whenever a computer crime is suspected I'll make sure to forward your name as a lead on the off-chance that you had anything to do with the crime.
Policing should start from available evidence leading to suspects, not start from suspects to turn to see which match the evidence.
How exactly do you believe investigations work? You don't think that rounding up the usual suspects and asking questions is a legitimate investigative tool? There isn't always a direct link from the available evidence to a person. Should they just give up and close the case?
Let me put it another way; someone is stealing packages from your doorstep. You happen to be the neighbor of a person you know has been arrested for burglary. That fact doesn't even enter your thought process?
Past behavior is a good indicator of future behavior and the real world isn't CSI. Detectives need to follow any leads they can get their hands on.
Policing start from any leads available, not evidences. Evidences are used in a court of law.
It's quite simple: a face image is submitted, and then a sorted by "match score" result is available. This is the same gallery, now sorted in respect to the submitted face image. That "confidence threshold" (also called verification threshold, and a number of other similar terms) is the threshold within the sorted results after which one can generally discard possible matches between the submitted face and a face in the gallery because the similarities between the submitted face and the candidate face have little resemblance. Imagine a series of faces on a horizontal row: the left-most is the highest match score, and moving right is the next highest in the sort, and so on moving to the right. The left-most image looks the most like the submitted face image, and as one moves right each image is a little bit less similar.
This confidence threshold loses accuracy when a) the submitted image is a less than a perfect image (often the case), b) the gallery is composed of images that are less than perfect (often the case), and even c) the types of cameras used to generate the images are radically different, with different dynamic ranges and potentially poor video encoding parameters. As a result, a few things are done: 1) multiple images (even if less than ideal quality) (and if available) are added to the gallery for each person, and 2) the confidence threshold is treated as a soft number, with sliders even to dynamically grow and shrink the net cast by the analysis. Also notice this is interactive - that implies a human operator. The entire FR process is a selection guidance tool, not an authoritative selection tool. The FR operator is keenly aware of the fact that they are sifting through a large number of low quality data points. FR is a tool to help sift sand.
Disclaimer: I am a lead developer of a leading FR product. Not Amazon's.
Also, to speak about racial bias in FR: it is a stretch to call it "racial bias", perhaps more of a "low facial dynamic range invisibility". Both people with very dark skin and people with very pale skin have the same situation: there is very low variation from the brightest non-highlight part of their face to the darkest part of their face. This means there is very little distinguishable information for FR to use, if it can even identify the portion of an image that contains a face. Such individuals are simply hard to see with FR, and when identified and tracked as a face, there is very little information to use for differentiation. I don't know if I'd call that a "racial bias". These people are almost FR invisible, which benefits them in this situation, making them very hard to distinguished with FR analysis.
Wish you'd spend your time on something that will move the needle in a more positive direction. The long term applications of the tech that you are making are downright scary.
There is a joke that technology is like four wheel drive (itself a form of technology), you will still get stuck, but in harder places. In other words, the tech that gets you into trouble is not necessarily the tech that will get you out of that trouble, it may need a more advanced technology to solve a problem created by a previous level.
The people at Google and Facebook are slowly figuring out that their data driven perfect future may not be one that they themselves (or their descendants) would like to live in. Face recognition for computers enables a whole pile of use cases that are downright scary and I would love for such tech to be stalled as long as possible.
Or do you even reconcile it at all?