- Show us the side-by-side images of the false-positives. Are the matches plausible?
- What is the demographic distribution of the mugshot database? If the data is disproportionately biased, then that bias would be reflected in the false-positives. A casual skimming of some mughot websites shows a potentially significant racial bias.
At the same time, they do still have the correct final conclusion. Dragnet facial recognition is a bad idea that will produce too many false positives to be of use. Or at the very least, dragnet facial recognition used by people who put too much faith in it is a bad idea. It's at most useful to produce flags for slightly more attention; on its own, I would not consider it anywhere close to enough to arrest someone on.
And it can serve as a demonstration of how a clumsily-put-together system can be used to produce bad results, so, if we correctly assume that there may be people bodging together a system in much the same way the ACLU did, well, hey, it's a valid result then!
Same thing re data setup. Not the point again. Even if law enforcement hires the world's best computer vision experts (fat chance, it'll go to cheapest contractor) this is a note that there can be mistakes that get people killed.
Regarding sketches, they're actually interesting because they force a human to exercise judgement, being ambiguous by nature. This is fine - any law enforcement officer stopping someone who looks like a sketch is going to give them the benefit of the doubt. On the other hand, if a computer tells an officer a person ahead is an 89% match with a known killer, the conversation will start with guns drawn, and that will greatly increase the likelyhood of things going south.
You really don't want to be scanned by a trigger happy cop having a bad day. God knows who you happen to look like from that angle.
That is where the two sides of this argument diverge. First: That things "will start with guns drawn". And Second: That things are more likely to "go south" for an innocent individual incorrectly blamed by this.
If police engage a target with a weapon already drawn, there is a long list of things that person can do that will get them shot, and a very short (and often unclear) list of actions that will keep them safe.
Source: 10 years PMO in the Marine Corps, 1 month SFPD (then quit because their training was so shitty)
For the most part, this sort of article makes things seem like there will be some sort of racial-based, dystopian police future where automated algorithms will target people and have them put into jail. I don't see that happening in the near future at all. The police departments' personnel currently using this probably LOOK at the matches and only then make a further call. Just like I would assume AFIS fingerprint hit results are double-checked by skilled operators after matching to reduce false-positives.
Perhaps lawmakers should consider implementing rules to ban facial recognition software until the developers can prove some like 99.999% accuracy. Otherwise we end up harassing a large number of people, and the credibility of the technology will stuff. As it stand any lawyer will be able to argue that facial recognition isn't admisible as evidence, do to the high margin of error.
You're right that systems have previously learned to infer black -> criminal. It's absolutley possible that could have happened here! Given that not every one flagged is a person of color, there's clearly more at work.
Systems like Rekognition let a user specify what level of certainty the system has that it has found a match. In this instance, the system was told to use 80%. Rekognition can be told to use any number from 0 to 100. Your number of 99.999% could be used without any difficulty whatsoever! Since the ACLU doesn't seem to be sharing enough to reproduce the test, we can't tell what would happen.
Regulation based on accuracy is a wonderful idea! One well worth pursuing. With that said, "accuracy" is perhaps difficult to define here. How would you determine that? How accuracy is measured is highly sensitive to what you use as a dataset, and different test datasets can easily produce very different impressions of how common false positives and false negatives are.
Yet, might there not be other factors to consider? Bear in mind that this isn't pushbutton easy off-the-shelf. Some actual work is required here. There's probably a batch of Python scripts sitting around that did the work. The most charitable assumption is that whoever did this at the ACLU didn't look at how the tool under examination should be used at all and missed everything about confidence levels in whatever few docs they did read. Under the ACM code of ethics, I am comfortable calling that an unethical degree of incompetence and carelessness.
A less charitable interpretation is that they knew and didn't care because it produced the result they wanted. This is similarly unethical, in my mind on a level with the sort of p-hacking prevalent in social sciences.
You're completely right! They just used the default. It's just perhaps possible that there might be other factors worth considering.