Hacker News Reader

Comment by bo1024 | Hacker News Reader

Not complete answers but background: apple’s system works by having your device create a hash of each image you have. The hash (a short hexadecimal string) is compared to a list of known CP image hashes, and if it matches, then your image is uploaded to Apple for further investigation.

A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash, allowing them to trigger false positives any time. That appears to be what we are witnessing.

View on HN

> A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash, allowing them to trigger false positives any time.

This is my understanding too. But is this not also true for other (cloud-based) CSAM scanning systems? Why is Apple's special in this regard?

They aren't.

Apple could have saved themselves so much backlash and not have caused the outrage to be focused exclusively on them if they hadn't tried to be novel with their method of hashing, and had just announced that they were about to do exactly what all the other tech companies had already been doing for years - server side scanning.

Apple would still be accused of walking back on its claims of protecting users' privacy, but for a different reason - by trying to conform. Instead of wasting all the debate on how Apple and only Apple is violating everyone's privacy with its on-device scanning mechanism, which was without precedent, this could have been an educational experience for many people about how little privacy is valued in the cloud in general, no matter who you choose to give your data to, because there is precedent for such privacy violations that take place on the server.

Apple could have been just one of the companies in a long line of others whose data management policies would have received significant renewed attention as a result of this. Instead, everyone is focused on criticizing Apple.

There is a significant problem with people's perception of "privacy" in tech if merely moving the scan on-device causes this much backlash while those same people stayed silent during the times that Google and Facebook and the rest adopted the very same technique on the server in the past decade. Maybe if Apple had done the same, they would have been able to get away with it.

Perception aside, Apple’s system is somewhat better for privacy, since Apple needs to access much less data server side.

There’s a lot of overlap between Apple product buyers and “fight the thoughtcrime slippery slope” hackers. The CSAM abusers are presumably (if they’re not stupid) also fanning the flames on that slippery slope perception, because it’s to their benefit if the hackers defeat Apple.

I don't know. I don't know what other systems use, I just know what I've read recently about Apple's.

This pretty much sums this entire drama up, I think.

But how does this exact attack and scenario not also apply to Google, Facebook, Microsoft etc... who are also doing the same thing on their clouds servers?

I don't know what those companies do, hopefully someone who does know will chime in and answer.

They essentially do the same, just in the cloud meaning they access all images directly.

That’s an oversimplified and misleading description of how the system works, but ok. I recommend reading the technical description, or even the paper linked from that: https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

No, I don't think that's what we're witnessing. No one has yet demonstrated that they can take a hash alone and produce an image that matches the hash. It's true that that would be bad, but right now you still need the original colliding image.

But how would the attacker get the generated image on a person's phone?

I didn't mean to suggest a targeted attack. If the goal is to just overwhelm Apple's system, the attacker doesn't need to target a particular phone, they just need to distribute lots of colliding images to some phones. Even their own phones, since the colliding images wouldn't be illegal.

E.g. send them a whatsapp message that looks innocent

They can see the image - why would they import a random image into their library from someone they don’t know?

WhatsApp has a really weird default behavior: it imports all images you're sent into your photo library.

This is a smart thing to disable, even outside this recent discussion of CSAM.

https://faq.whatsapp.com/android/how-to-stop-saving-whatsapp...

In that scenario this attack is unnecessary. Someone could just send you legitimate child pornography and then immediately tell the authorities that you possess said things.

Yup! It's a legitimately crazy default.

Edit: Though, to be fair, the specific hash-collision scenario would be that someone could send you something that doesn't look like CSAM and so you wouldn't reflexively delete it.

If it doesn’t look like it wouldn’t a human reviewer disregard it once it gets to that point?

Personally I don’t really see the issue.

We don't know how the human review is going to work. Countries with fewer resources are going to find it easier to just arrest/detain any suspects, instead of spending time and money on figuring out which reports are true.

All you have to be is accused of CP for your life to be destroyed. It doesn't matter if you did it or not.

What you’re describing is possible even if you didn’t receive anything.

If the government wants to get you they don’t need this Apple scanning tech, or anything at all really

Yeah, you'd need a very specific set of processes for it to really be a problem.

Whatsapp imports received images into camera roll by default, and icloud sync is on by default. So you just need to get into a WA group the victim is a member of, and have the victim use default settings.

Auto import could be turned on?

Reply on news.ycombinator.com