Comment by pinum | Hacker News Reader

Comment by pinum | Hacker News Reader

Parent Full thread

About that "manual review"... Refer to pages 10 and 11 of the technical paper.

>The server then uses the decryption key to decrypt the inner encryption layer and extract the NeuralHash and visual derivatives for the CSAM matches.

This "visual derivative" term shows up repeatedly. To me, the implication seems to be that Apple doesn't look at the actual suspected image before deciding whether to proceed with a report. Instead, I infer that they only verify whether (as the device reports) the image's neuralhash is indeed present in the NCMEC database. If my understanding is correct, their "manual review" process actually provides no protection at all against collisions or erroneous database entries.

Further supporting this, on page 4:

>Apple reviews each report to confirm there is a match

It only refers to a match, not about whether the image appears to be illegal.

This makes perfect sense from Apple's perspective- who would want to be in the business of reviewing reports of probably-illegal images?- but it means that the references to a manual review safeguard would seem to be false reassurance. Maybe I'm misunderstanding the paper.

Visual derivative is a preprocessed picture itself, and it's part of the "safety voucher".

It may be grayscale and resized/normalized in other ways. Only apple knows exactly what it is.

It's only a matter of time until internet trolls find a way to abuse this. The database is stored on user's devices, so someone downloading it and permuting innocuous images enough until they match the database, and then spreading them for funzies via some underhanded method (wallpaper download sites?)... is not too far fetched.

If they can do a manual review at all, then there isn’t end to end encryption anymore, so I’m missing what the point of client side scanning is.

Apple doesn't currently have end-to-end encryption for iCloud Photos either: https://support.apple.com/en-us/HT202303

Not today. I think moving CSAM from the server where it's done today to device is in preparation for announcing e2e for iCloud photos.

That the insanity of all of this hand waving. Literally every public photo service does this today.

Apple has gotten a ton of heat over this and they haven't once mentioned that e2e on iCloud is something they're working on or that this technology would make possible, so can people stop spreading this narrative that this is their goal? It's completely baseless.

End to end encryption of messages is by comparison easy as the devices can handle all of that internally. However, losing your iPhone is one of the main reasons to have an iCloud backup. Require a user to come up with a private key and any user who lost it also loses all their data.

Most people don’t really want end to end on consumer backup services, because of the associated risks. If however you don’t want unsecured backups you can handle this manually.

Of course nobody wants the company to actually look at your data, but that’s a separate issue.

The main selling point of Apple is how well integrated the ecosystem is, they could make it super simple to backup the private key on your different devices like watch, tablet and laptop.

Most of the time, Apple seems to think through what bad actors will do with their stuff. Their scale and (perceived) reputation basically requires this.

I don't see any reasonable way here. Either somebody looks at the images - i.e. they have some underpaid poorly trained grunt to look at horrible abuse images which is probably illegal to look at whole day, having about 3 seconds to make a decision on each - or what they're doing is just a sham, "Did the computer say 'it matches'? Yes it did! Review complete, match confirmed!" I don't see any non-horrible way of doing non-sham reviews here.

People keep trading away privacy for convenience. We won't have any privacy left if we continue to go down that road. People need to start looking at these tech giants with extreme scepticism. Like we do if companies are from China. We need to think about our American companies the exact same way, and stop trusting them to care about our privacy.

> We won't have any privacy left if we continue to go down that road.

If people look at today's state (phone movement tracking, voice capture, cameras and face ID, etc.) from the mental model of 30 years ago, they would see neither privacy nor freedom. At all.

Sadly, it is not the case of preserving what we still have, if we want freedom and privacy we will have to reacquire them. And a lot of the price will have to be paid in blood.

Before that I thought about dumping Pixel phones for a iPhone. Now I gonna try out CalyxOS on my Pixel 2. Once I finally decided on an upgrade to a Pixel 5 I'll probably give GrapheneOS a shot. Hoping that makes a difference, otherwise it will a dumb phone again.

> who would want to be in the business of reviewing reports of probably-illegal images?

I could think of a couple of companies this could be outsourced to, with strict business and privacy agreements in place, of course, and also conveniently on the lowest end of European minimum wage for the tier 1 reviewers.

Reply on news.ycombinator.com