How do you see a false positive progressing into child separation?
People who are actually convicted of possessing CSAM don't necessarily lose access to their children at the moment.
I have no way of verifying what these people are telling me, and I can’t help them even if I did. And even more than this: my first reaction is to wonder if they’re all actually pedophiles who are just lying to me, and/or if they’re going to text CSAM at me. (Because that’s the power of this accusation — nobody will trust you once they hear it.)
Some of these folks have had their lives ruined and some have told me they were hospitalized and considered suicide, even though they weren’t actually prosecuted. I can certainly imagine being in their situation due to the mistake of having an asshole friend-of-a-friend in a group chat and some bad settings in my phone. (I have neither, thankfully.)
If none of this stuff convinces you, here is a much more carefully-documented version of a similar story in which a dad took a photo of their toddler’s rash and was subsequently investigated for CSAM. This one has the “happy” ending that the accused is entirely cleared by the police, but never gets his Google account back and has to live with whatever other trauma and shame comes from the investigation: https://www.nytimes.com/2022/08/21/technology/google-surveil...
https://www.bnnvara.nl/artikelen/hoe-konden-1115-kinderen-va...
And then there's of course the (less-common, but happens often enough to be troubling) situation where parents lose their children and never get them back, even though they're innocent of causing any harm.
Note that I'm talking about the broader effect of Child Protective Services when they investigate any kind of reported abuse, not just CSAM possession/distribution.
I hope the “prankster” ends up caught and in prison
This always makes me think how much worse middle school will get: kids are guaranteed to push boundaries, they all have some anger justified or not towards each other or the adults they know, and some fraction will learn how nastily they can weaponize the mandatory reporter system. Even if the truth comes out quickly, there’s so much room for strife first.
There's many intervening steps[1] of consequences which can result. Not to mention chilling effects: it's a standard bit of knowledge now that all my family members worry a bit if they take a photo of my son doing something and he's not wearing a shirt or something. Because no one's entirely sure the extent or impact of whatever scanning systems already exist (hey folks: encrypt everything on the cloud, and thank god for Signal).
[1] https://au.pcmag.com/security/95768/nyt-parents-lose-google-...
At the same time, there are unfortunately many counterexamples of abusive parents not being separated from their children via the same mechanisms. Sometimes this is from the abusers being "good criminals" and not getting caught; but sometimes it's from LEOs being bad LEOs and not doing the catching.
My point was that in a system which surveils ~500M people, even an FP rate of ~0.01% is 50k people. Supposing the LEOs tasked with filtering this into credible (and triable) cases further manage to, through amazing police work, confirm the innocence of 99% of those false positives. That's still 500 falsely accused, but now confirmed suspicious, suspects.