Hacker News Reader

You've done nothing to address OP's concerns. The linked twitter thread assumes each actor (NCMEC, FBI, Apple) act in a certain way. There's no "provable" guarantee against an actor acting in bad faith or in a manner inconsistent with certain interpretations of the law (which we've seen routinely with the NSA).

The FBI/NSA can absolutely inject something into the hash list. You're assuming that NCMEC needs to be involved. Or that it would be broadly known to Apple. The reality is that the hash list needs to be updated on a different cadence than iOS itself. So it's likely downloaded rather than baked into the OS build permanently. That means that you can't necessarily rely on an iOS build being signed to know if you have a different hash list from everyone else. Ultimately, a small team at Apple cooperating with a secret court order could release a different hashlist to a select set of devices. There's nothing really stopping that.

Even if Apple didn't comply, we've seen recently how sophisticated cybersecurity companies armed with zero days can manipulate devices easily. If the mechanism for hash lists scanning the device is already built in all it takes is an exploit changing the hashlist and where it reports to which might be much simpler than gaining full access to the device.

It's worse than that. They don't even need to release a different hash list for you, all they need to do is add a few images they know you'll probably have (say from your Facebook or Instagram posts) to the regular DB to meet the match threshold. The people running the database aren't going to be continually going back through old images to double-check they're actually CSAM/theoretically CSAM-related.

How do you know there aren’t bad actors working at the NCMEC? If I know that adding a hash to a list will get it flagged, and I could conveniently arrest or discredit anyone I wanted, I would certainly send people to work there.

How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t.

And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights?

The point is that it is completely an easy way to get tagged by a government or bad actors as a pedophile. It’s sickening that Apple would let this “technology” into their products.

If you don’t trust what Apple says about this, why even argue?

Apple could be doing all of this and more without telling.

I agree with you on the point that the concern here is what various governments may mandate, but if we’re going to argue about Apple’s specific implementation you should probably understand it.

There's nothing to argue. I'm incredibly disappointed in Apple and feel betrayed. I went all-in with the Apple ecosystem because I stupidly and naively believed their commitment to privacy.

You don’t seem to have read what Apple has said on the issue so that feels a bit extreme.

And for the record I’m not for this, but my concerns are more about what various governments may start mandating as this capability becomes an option.

If you want on-device and cloud backup of data that isn’t checked for illegal content, I think that change needs to happen at the legislature not the phone store.

I’m telling you what is going to happen in the future. What they write today is meaningless. In 2016 they fought the fbi in terms of unencrypting data. Then they decided to not encrypt iCloud backups. They didn’t mention in 2016 they wouldn’t encrypt iCloud backups just like they won’t say today that they won’t bend to the Chinese government tweaking their algorithm or their hash list in a few yearsnn

The only logical reason for Apple to implement such a complex system is to give them a defensive political tool to allow them to do things like encrypt iCloud backups and photos. “But CSAM” is a common political tool used against such encryption.

I don’t see this move from Apple making any measurable difference in how well a government can scan your device for arbitrary data.

If it happens it was going to happen anyway, your prediction of the future comes true with or without CSAM scanning, if you are allowing for new government orders and legislation.

The difference is that Apple has built the infrastructure for _global_, continuous image scanning. No need to hack individual phones or know who your targets are, all you have to do is get some images added to list. If you think the Chinese (or most likely American) government wouldn't love to be able to find out everyone who had copies of certain images, say from protests or inside camps, you're far too optimistic.

This is Apple creating a mechanism for worldwide surveillance and declaring they'd never compromise because they're good people who we can trust. Note that they're not making any actual promises that might land them in court if the system is abuses, they're just repeating that they're good people who'd never cooperate with authoritarian regimes.

All you have to do is:

- Get your image added to the list

- Order Apple to scan all images not just ones being uploaded to iCloud

- Order Apple to change the reporting threshold or remove the safety voucher system unless you only care about users with many matching images

- Compromise or order Apple to skip the human review and pass the reports directly to your government agency

Really you have to compromise or order Apple to change every step in the pipeline except “compare images to hashes” which is the part an intern could do in a weekend.

I’m sure governments would love to know which users have “undesirable” images on their device. I’m just saying instead of doing the above steps to take advantage of the CSAM system, they could just order Apple to scan for the images they want scanned for. Apple’s choice is the same, comply or risk participation in that market.

By the same logic, how do you know there aren't bad actors working at Apple's software teams? Or your insurance company or bank?

If you think that FAANG and other tech companies aren't filled with spies from around the world, you're incredibly naive.

> How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t

Spoiler alert: they are building the hashing algorithm, so there is at least someone to confirm everything in the beginning. They are not hashing hashes.

Stakes are so high in this game that they must be very careful in review. This is not about Youtube copyright claims.

> And what about in China?

I don’t think that China even cares a lot. It is mandatory to install some apps by law for some minorities. They have the surveillance already.

> How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t.

As I said, the flagged content is reviewed by an Apple employee before it actually triggers an external report. If the flagged material is not in fact CSAM, it will not be reported.

> And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights?

First of all, the volume of flagged CSAM content is much, much smaller than the volume of YouTube copyright claims. It's entirely plausible to ensure that a human reviews all flags. Second, Apple is actually constitutionally-barred from automating this step entirely. You can thank Neil Gorsuch's decision in United States v. Ackerman for this. [1] The crux is that since NCMEC is a qausi-governmental entity, automatically sending CSAM-matched content to NCMEC without an Apple employee first inspecting the content would constitute an unreasonable search and seizure and would violate the 4th amendment.

[1] https://library.law.virginia.edu/gorsuchproject/united-state...

> Apple states point-blank that they will refuse any demands to add non-CSAM content to the lists.

How would Apple know if non-CSAM was added to the list? Apple does not and cannot curate the list. Apple only receives hashes from NCMEC (and other unnamed government agencies). The government does not allow Apple to verify that this list only contains CSAM. This pledge from Apple is at best misguided at worst intentionally dishonest. Of course nobody can make Apple add non-CSAM to the list: Apple doesn't maintain the list.

Do you work for Apple or the government? You just sounded like a lawyer talking about similar but unrelated things to win the case.