10% of all OK images may be falsely flagged, but what about specific categories of images?
What would the error rate be for an album of legal pornography?
What would the error rate be for an album of a family spending the day at a beach?
What would the error rate be for an album of legal pornography?
What would the error rate be for an album of a family spending the day at a beach?
The article doesn't say what % of OK images are falsely flagged, only what percent of flagged images are OK.
Agree that subsets might have different stats, but there's 0 information in the featured article about any of that!
This isn't even a hypothetical. The case I mentioned above actually happened. Google narc'd on someone who did exactly the thing I mentioned, and they got a police investigation for their trouble. And while the police - armed with exhonorating context - did eventually drop charges, they are still banned from Google for life. Yes, Google was contacted by journalists about this guy and they said they stand by their initial decision to permaban him.
The core danger with any automated scanning system is not the false positive or false negative rate - that is an engineering problem. It will get better. The danger is the amount of legal-but-crime-adjacent activities that will be effectively prosecuted as crimes. I mentioned child telehealth above, but there's other instances of automated prosecution expanding far beyond the letter or spirit of the law. YouTube Content ID is top-of-mind for me; though that has two problems. It both kills fair use, and correctly prosecutes copyright infringements that people expect to fly under the radar.
[0] Child anti-explotation agencies still caution against this because they don't want your kid to get used to getting photographed down there. However, they wouldn't consider this sexual abuse.