Hacker News Reader

There are numerous incorrect statements in your comment.

First: Apple has disclosed who gets to curate the hash list. The answer is NCMEC and other child safety organizations. https://twitter.com/AlexMartin/status/1424703642913935374/ph...

Apple states point-blank that they will refuse any demands to add non-CSAM content to the lists.

Second: Why can't the FBI / CCCP inject a hash into the list. Here's a tweet thread gamifying that scenario: https://twitter.com/pwnallthethings/status/14248736290037022...

The short answer is that at some point an Apple employee must visually review the flagged photo, and confirm that it does represent CSAM content. If it does not, then Apple is under no legal obligation to report it.

Third: You claim that abusers will simply opt not to use iPhones to distribute their CSAM content rendering the feature useless. This is in fact not how things have played out on other platforms like Google and Facebook that do already scan for CSAM. These organizations report on the order of millions of flagged images per year. [1] Clearly the abusers have simply not moved on to a different platform.

[1] https://www.businessinsider.com/facebook-instagram-report-20...

How do you know there aren’t bad actors working at the NCMEC? If I know that adding a hash to a list will get it flagged, and I could conveniently arrest or discredit anyone I wanted, I would certainly send people to work there.

How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t.

And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights?

The point is that it is completely an easy way to get tagged by a government or bad actors as a pedophile. It’s sickening that Apple would let this “technology” into their products.

If you don’t trust what Apple says about this, why even argue?

Apple could be doing all of this and more without telling.

I agree with you on the point that the concern here is what various governments may mandate, but if we’re going to argue about Apple’s specific implementation you should probably understand it.

There's nothing to argue. I'm incredibly disappointed in Apple and feel betrayed. I went all-in with the Apple ecosystem because I stupidly and naively believed their commitment to privacy.

You don’t seem to have read what Apple has said on the issue so that feels a bit extreme.

And for the record I’m not for this, but my concerns are more about what various governments may start mandating as this capability becomes an option.

If you want on-device and cloud backup of data that isn’t checked for illegal content, I think that change needs to happen at the legislature not the phone store.

I’m telling you what is going to happen in the future. What they write today is meaningless. In 2016 they fought the fbi in terms of unencrypting data. Then they decided to not encrypt iCloud backups. They didn’t mention in 2016 they wouldn’t encrypt iCloud backups just like they won’t say today that they won’t bend to the Chinese government tweaking their algorithm or their hash list in a few yearsnn

The only logical reason for Apple to implement such a complex system is to give them a defensive political tool to allow them to do things like encrypt iCloud backups and photos. “But CSAM” is a common political tool used against such encryption.

I don’t see this move from Apple making any measurable difference in how well a government can scan your device for arbitrary data.

If it happens it was going to happen anyway, your prediction of the future comes true with or without CSAM scanning, if you are allowing for new government orders and legislation.

The difference is that Apple has built the infrastructure for _global_, continuous image scanning. No need to hack individual phones or know who your targets are, all you have to do is get some images added to list. If you think the Chinese (or most likely American) government wouldn't love to be able to find out everyone who had copies of certain images, say from protests or inside camps, you're far too optimistic.

This is Apple creating a mechanism for worldwide surveillance and declaring they'd never compromise because they're good people who we can trust. Note that they're not making any actual promises that might land them in court if the system is abuses, they're just repeating that they're good people who'd never cooperate with authoritarian regimes.

All you have to do is:

- Get your image added to the list

- Order Apple to scan all images not just ones being uploaded to iCloud

- Order Apple to change the reporting threshold or remove the safety voucher system unless you only care about users with many matching images

- Compromise or order Apple to skip the human review and pass the reports directly to your government agency

Really you have to compromise or order Apple to change every step in the pipeline except “compare images to hashes” which is the part an intern could do in a weekend.

I’m sure governments would love to know which users have “undesirable” images on their device. I’m just saying instead of doing the above steps to take advantage of the CSAM system, they could just order Apple to scan for the images they want scanned for. Apple’s choice is the same, comply or risk participation in that market.

By the same logic, how do you know there aren't bad actors working at Apple's software teams? Or your insurance company or bank?

If you think that FAANG and other tech companies aren't filled with spies from around the world, you're incredibly naive.

> How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t

Spoiler alert: they are building the hashing algorithm, so there is at least someone to confirm everything in the beginning. They are not hashing hashes.

Stakes are so high in this game that they must be very careful in review. This is not about Youtube copyright claims.

> And what about in China?

I don’t think that China even cares a lot. It is mandatory to install some apps by law for some minorities. They have the surveillance already.

> How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t.

As I said, the flagged content is reviewed by an Apple employee before it actually triggers an external report. If the flagged material is not in fact CSAM, it will not be reported.

> And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights?

First of all, the volume of flagged CSAM content is much, much smaller than the volume of YouTube copyright claims. It's entirely plausible to ensure that a human reviews all flags. Second, Apple is actually constitutionally-barred from automating this step entirely. You can thank Neil Gorsuch's decision in United States v. Ackerman for this. [1] The crux is that since NCMEC is a qausi-governmental entity, automatically sending CSAM-matched content to NCMEC without an Apple employee first inspecting the content would constitute an unreasonable search and seizure and would violate the 4th amendment.

[1] https://library.law.virginia.edu/gorsuchproject/united-state...

You've done nothing to address OP's concerns. The linked twitter thread assumes each actor (NCMEC, FBI, Apple) act in a certain way. There's no "provable" guarantee against an actor acting in bad faith or in a manner inconsistent with certain interpretations of the law (which we've seen routinely with the NSA).

The FBI/NSA can absolutely inject something into the hash list. You're assuming that NCMEC needs to be involved. Or that it would be broadly known to Apple. The reality is that the hash list needs to be updated on a different cadence than iOS itself. So it's likely downloaded rather than baked into the OS build permanently. That means that you can't necessarily rely on an iOS build being signed to know if you have a different hash list from everyone else. Ultimately, a small team at Apple cooperating with a secret court order could release a different hashlist to a select set of devices. There's nothing really stopping that.

Even if Apple didn't comply, we've seen recently how sophisticated cybersecurity companies armed with zero days can manipulate devices easily. If the mechanism for hash lists scanning the device is already built in all it takes is an exploit changing the hashlist and where it reports to which might be much simpler than gaining full access to the device.

It's worse than that. They don't even need to release a different hash list for you, all they need to do is add a few images they know you'll probably have (say from your Facebook or Instagram posts) to the regular DB to meet the match threshold. The people running the database aren't going to be continually going back through old images to double-check they're actually CSAM/theoretically CSAM-related.

> Apple states point-blank that they will refuse any demands to add non-CSAM content to the lists.

How would Apple know if non-CSAM was added to the list? Apple does not and cannot curate the list. Apple only receives hashes from NCMEC (and other unnamed government agencies). The government does not allow Apple to verify that this list only contains CSAM. This pledge from Apple is at best misguided at worst intentionally dishonest. Of course nobody can make Apple add non-CSAM to the list: Apple doesn't maintain the list.

Do you work for Apple or the government? You just sounded like a lawyer talking about similar but unrelated things to win the case.

What on earth? To clear up some of your false statements:

- You need several hash matches to trigger a review

- The reviewer can of course see what triggered the review (the visual derivative)

- The reviewer would see that the matches are not CSAM, and instead of the report being sent on to the NCMEC it would instead start an investigation of why these innocuous images were matched in the first place

- If the CIA or FBI or CCP wanted to arrest you, there are much easier ways than this

Apple is putting a backdoor in the phones to legitimately frame anyone they want. The commenter's claims are valid and in my opinion not limited those.

Couple of gripes there.

Apple basically controls you phone anyway and have done for years as they can issue patches and os updates.

Also you can turn iphotos off - I've never used the thing in spite of owning various apple devices. I do use Google photos and doubt they are much different in terms of checking for CSAM.

I imagine this very well being extended not just to match photos, but also metadata within the photos. And then the list of things it matches against is extended from CP to other undesirables by society.

After a 12 hour flight - that was of course delayed - Liam was pretty exhausted, but was looking forward to getting to his hotel in the center of Munich. He got to the front of the queue, and handed his passport over to the customs officer. The officer scanned Liam's passport, took it off the reader, and after 20 seconds asked "You flew from Los Angeles today?". Liam replied "Yes...". The customs officer, with his firm German accent, said "I need to check something with my colleague, wait here.". Not that there was anywhere Liam would go.

The customs officer came back with someone else who was slightly older and clearly more senior. The senior officer said "Come with me please", and led Liam to a room at the side of the customs hall. The officer said "Sit down please", indicating to the chair in front of the desk. The room looked like any other office, with a computer on a desk and chairs either side. The officer sat behind the desk and started typing something on the computer. After a few minutes he said "You are wanted by Interpol".

The customs officer explained to Liam that he had been flagged as a photo he had taken 6 months before included a known terrorist, and so by association Liam had been flagged. Liam asked how they accessed his photos - he is tech savy and only takes encrypted backups onto his own devices. The customs officer explained that they didn't need to, as this flagging had been done entirely by his device. The customs officer gave the date of the photo, and Liam found it on his device. He had been on holiday with his girlfriend in Paris, and they had taken a selfie. There was someone clearly visible behind them, and the customs officer explained that the facial recognition had identified this person. Due to privacy laws he wasn't able to say (or even see themselves) who this person was, only that they were on the highest German terrorist watch list.

From the photo it looked quite obvious that this person was just a passer by who glanced at the couple just at the moment they were taking a photo. The customs officer took Liam's fingerprints and asked Liam questions about his trip to Paris - typing the answers into the computer - and then the computer decided that Liam could be released. However the customs officer told Liam that he would be closely monitored while in Germany, and may receive 'check in' calls from police. He told Liam he must answer them otherwise a team will be dispatched to intercept him. Liam was then allowed to go on his way. He was only delayed by 45 minutes, but it wasn't a great start to his holiday in Germany. 3 years later when he visited Germany again the same thing happened, at least he knew what to expect this time...

(This is partially based on something that actually happened to me. Nearly a decade ago my passport was stolen, and every time I go to Germany I need to have a fun conversation with customs officers. Every other country I've visited - including the US - let's me through without even mentioning it)