Hacker News Reader

Comment by fluxsauce | Hacker News Reader

> but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences.

That's not in https://aws.amazon.com/rekognition/faqs/ - while that threshold may minimize false positives, I'm curious if it was in any documentation that the ACLU saw. Or, to the bigger point, is it in any public documentation?

View on HN

Threshold is a configuration property no different than a dynamo read capacity. Misconfigured properties don’t work the way you expect. I don’t see how documentation can be expected to give black and white guidance to what this threshold should be set to without having more understanding of your use case.

> I don’t see how documentation can be expected to give black and white guidance to what this threshold should be set to without having more understanding of your use case.

It can say exactly what the Amazon response to the Verge dismissing the ACLU test should, and, in fact, it should say that if Amazon really does have such critical, specific domain-specific recommended settings, rather than it being something made up off the cuff to mitigate bad press.

You're unquestionably correct. Amazon should spell out in clear, simple, easily accessible language what level of certainty is appropriate for what type of application, giving concrete examples at every point.

With that said, it's possible that no amount of clear guidance would have mattered here. The ACLU isn't exactly asking for better docs.

If you're someone with minimal understanding of the use and deployment of technologies, who knows only of this technology as something used to detect faces, then it's a glaring oversight.

If you're a researcher making a deliberate choice to use the defaults to simulate a random police officer with no real understanding of their tools, then say as much. And compare with more cautious results.

I would be shocked if LE started using this tech, imprisoned a bunch of people, and used the excuse "Well, we followed the FAQS." I would imagine they would work very closely with Amazon and run a ton of calibration. For the ACLU to say that we used the default settings and it doesn't work correctly is disingenuous.

Assuming that law enforcement is going to spend any amount of time being careful about enforcing correct use of forensic tools is hard to reconcile with the history of the American judicial system. For example, there is essentially no scientific evidence that a polygraph test measures anything at all, but polygraph evidence is considered admissible in many courts and used to be admitted in many more. Moreover, rigorous studies of fingerprint evidence have found them to have false positive rates as high as 1 in 18, but they are still often treated as close to infallible in court. I am not sure why you think facial recognition is going to be different.

I would not be shocked at all if some LE somewhere used the default settings. It seems absurd to assume otherwise. It's not as though LE has a great track record.

https://en.wikipedia.org/wiki/No_Fly_List#False_positives

You are going to be very very shocked my friend!

LE doesn't have the best track record following best practices. For example, they still rely heavily on notoriously unreliable eye-witness accounts and polygraph tests, two things science has largely debunked.

They were slow to let go of phrenology as well.

Reply on news.ycombinator.com