Hacker News Reader

Comment by Retric | Hacker News Reader

Perceptual hashes are only used to reduce the search space for human review. Apple doesn’t have images in the CSAM database to do a comparison, but if it’s just a picture of a door their going to reject it. Also, because human review is an expense Apple’s incentives are to minimize the number of times it happens, thus the requirement for multiple collisions.

View on HN

I don't really want my family photos reviewed by strangers. "Reducing the search space" of photos on my phone isn't an outcome I want to live with. At the time someone is looking at photos of my, my wife/husband/girlfriend/boyfriend, and my kids, they'd better have a darned good reason (e.g. a search warrant).

I'd also appreciate if Apple let me know if my false positives were reviewed and found to not be CASM.

Don’t upload an image anywhere, else it can be reviewed.

The whole point of Apple's system is that I don't need to upload an image anywhere.

Images from my phone can be stolen and reviewed with no due process, based on proprietary Apple technology.

The system as described only submits its safety vouchers when photos are uploaded to iCloud.

Not saying it will stay that way, but there are three distinct realms of objection to this system, and it's probably useful to separate them:

1. Objections that in the future, something different will happen with the technology, system, or companies; so that even if the system is unobjectionable now, we should object because of what it might be used for in the future; or how it might change. 2. Objections that Apple can't be trusted to do what they say they are doing, so that even if they say they will only refer cases after careful manual review, or that they will submit images for review that were not uploaded to iCloud, we can't believe them, so we should object. 3. Objections that hold for the system as designed and promised; in other words, even if all the actors do what they say they are doing in good faith and this monitoring never expands, it's still bad.

People who have the third kind of objection need to deal with the fact that Apple is basically putting in a system with more careful safeguards than are already in place in many Internet services, even for their "private" media storage or exchange. You likely don't know how the services you use are scanning for CSAM but if the service is at all sizeable (chat, mail, cloud storage) it's likely using PhotoDNA or something similar.

I think there are valid objections on all three bases. But there's a difference in saying "this is bad because of something that might happen" and "this is bad because of what is actually happening".

I think the issue is that the content review is happening on phone, and would be a small change to go from scanning uploaded photos to all photos

Oh yes, I agree. We will see a change in privacy policy before that happens. And Apple will lose a lot of us if that comes to pass.

For many years, it happened in the cloud. Soon it will happen on device and send a message about which item in the cloud is an issue.

I think it’s all about apple moving ML jobs (like Siri) to device to lighten the load on their datacenters.

I saw a story on here yesterday about iphones resetting to default settings after restarting. So people were turning off backups to the cloud, and then finding that their device turned the feature on after sometime.

> Apple’s incentives are to minimize the number of times it happens, thus the requirement for multiple collisions.

How can we be sure they won’t cut costs by increasing worker load? I could see them giving each reviewer less time to review individual pictures before passing it on to law enforcement.

We can't and they probably will, everyone else seems to already be doing so. There's that Swiss federal police report that only about 10% of NCMEC reports are actually relevant (https://fedpol.report/en/fedpol-in-figures/fight-against-pae...)

If they pass false positives to authorities that will open them up to legal action.

Apple's human review is largely useless.

Trolls will be able to easily use tools slightly modify ambiguous adult porn to collide with a "known CP hash".

A human reviewer will see a blurry grayscale derivative of adult pornographic content and hit "report" every time.

This is the threat model I am looking at. It is number one with a bullet. We have already had a court case where an adult actress had to show up in court and prove that she was adult when experts testified that the images were of a non-adult woman.

Baby in the sink? No. But a bunch of the aforementioned? Yeah.

> Perceptual hashes are only used to reduce the search space for human review.

False. The Apple proposed system leaks the cryptographic keys needed to decode the images conditional on the match (threshold of matches) of the faulty neuralhash perceptual hash.

Matching these hashes results in otherwise encrypted highly confidential data being decodable by apple, accessable on their servers to the relevant staff along with anyone who compromises them or coerces them.

Apple can decode the data either way. Their the ones doing the encryption on their servers.

There are two basic reasons for this first it’s a backup service which makes end to end encryption risky, but second they also let users share access to their baked up photos. iCloud > photos > shared album.

Edit: I incorrectly claimed there wasn’t manual review - see below

What is the basis for your understanding?

Apologies, I was mistaken:

“ Only when the threshold is exceeded does the cryptographic technology allow Apple to interpret the contents of the safety vouchers associated with the matching CSAM images. Apple then manually reviews each report to confirm there is a match”

The design goal was no human review for individual matches.

Reply on news.ycombinator.com