If the system was matching against known cryptographic hashes the collision / false positive rate would be small, but the fuzzy matching involved with perceptual hashing necessarily has a greater false positive rate.
And that doesn’t even begin to address the detection of sent and received “explicit images” which are detected on device and don’t have a set of known hashes.
The iMessage bit is different - it's only on device, only on child accounts, and only alerts parents. It's more akin to a parental control feature than anything else.
They have nothing to do with each other, and I've seen a dozen people on HN confuse them. If the message is getting muddled here, it will be hopelessly conflated in less technical circles.
I'm concerned and upset about the CSAM filter for all the reasons that keep hitting the front page, but don't care about the opt-in parental controls at all, and if I had kids, I might want them.
But if I thought the CSAM filter worked like the nudie-detector filter, I'd be wigging out.
Bugs in the application of the code, combined with human complacency and mistakes can certainly lead to errors, even if the cryptographic algorithm itself was perfect.
We really need to bring back comp.risks
So now instead of sending just one nice innocent very high resolution images of "Tokyo City" or something with something horrific hidden somewhere you have to send a few such images.
That is reassuring. I can never believe anyone except me will think about that.
(If the system is too dumb to detect this it is worthless, and if it is smart enough this opens the floodgates for anyone wanting to make trouble for just about anyone.)