You could argue that a minecraft server is technically in possession of CSAM if that's the case, but you could spend an infinite amount of money looking at various possible sequences and are bound to find many more false positives than true positives.
Services should have a duty to report CSAM when they notice it, but the lengths they should go to search for CSAM should be limited by cost/benefit and privacy concerns.
This type of scenario is what happened with the messaging service Kik, which was reportedly used to distribute CSAM in private chats. Law enforcement agencies said the company wasn't providing timely responses and that children were being actively abused as a result. This is about as damaging of an accusation you can leverage against a company.
Laws against CSAM worldwide are not going away for good reasons, so there is always going to be a justifiable argument that storing certain classes of data is illegal. Hence, anyone wanting to run a cloud service that stores user data will have to obey by those laws, regardless of how proactive they are in scanning for the material. Absolute privacy in the cloud is impossible to achieve with those rules in place.
This has been mentioned on here before, but it's known CSAM possession that's illegal. Apple keeps your files encrypted until its algorithm thinks your encrypted file is too similar to CSAM, and then it decrypts it and sends it to Apple for review. There's a few things here.
- The algorithm is a black box, so nobody knows how many false positives it hits.
- Apple's willingness to decrypt files without the consent of the owner makes the encryption seem like a bit of a sham.
- I imagine many are skeptical of Apple's ability to judge CSAM accurately. If I take a photo of my kids in a bathtub, is that CSAM? What about teenagers in a relationship sharing nudes. The law is a blunt and cruel instrument, and we've gotten away without hurting too many innocent people so far because the process is run by humans, but computers are not known for being gracious.
So we know for sure they're not just using PhotoDNA?
> If I take a photo of my kids in a bathtub, .....
Kinda the same question. If they're using PhotoDNA, then that's not really a risk, right? Isn't this technology well understood at this point?
- There's a system to catch CSAM that is either PhotoDNA or something that works similarly.
- There's a system to detect novel nudes, and notify parents if their children view them.
I think I got these two mixed together.
That's fair. Apple did a shit job of explaining themselves, and it has been compounded by a lot of misinformation (deliberate or not) in response. I'm trying really, really hard to moderate my reaction to this whole mess until I feel like I actually understand what Apple intends to do. I don't make platform jumps lightly.
Which is exactly why these policies are so dim witted.
Dragnet violation of everyone’s privacy while anyone even remotely sophisticated can easily evade it by just encrypting the data upfront.
> (f)Protection of Privacy.—Nothing in this section shall be construed to require a provider to—
(1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b).