Hacker News Reader

Comment by simondotau | Hacker News Reader

Intentional false negatives are mostly irrelevant. If you want to store galleries of illegal material on your phone, there are much better ways to do it which don't involve them being visually mangled, even if only slightly.

Intentional false positives require knowledge of a hash. And even that isn't enough to get a human to scan your image. You would need to acquire a library of A1 CSAM images/hashes. You would need to generate images which match both NeuralHash and the unspecified secondary hash algorithm which runs on Apple's cloud servers. And then you'd need to repeat this trick a sufficient number of times to trigger human review, at which point a few people (probably some minions in a developing country) get to look at the images for 1 second before pressing the "not CSAM" button and having the images sent to a technical team in Cupertino for analysis.

View on HN

If both false positives and false negatives are irrelevant, then the whole system and argument is irrelevant.

Huh? False negatives and false positives are irrelevant with respect to the utility of the overall system. It means they don't render the system irrelevant.

Intentional false negatives are irrelevant because catching 100% of CSAM was never stated as a goal of the system. And because it's just not realistic to expect to see any real-world use by CSAM aficionados. Anyone savvy enough to understand the principles behind hashing (and trust some command line tool downloaded from Github) is probably smart enough to have better places to store their child porn than an iCloud gallery.

Intentional false positives are irrelevant because there are, depending how you count them, between two and six other layers which would need to be thwarted or exploited in order for intentional false positives in NeuralHash to result in any adverse outcome.

>Intentional false negatives are irrelevant because catching 100% of CSAM was never stated as a goal of the system. And because it's just not realistic to expect to see any real-world use by CSAM aficionados. Anyone savvy enough to understand the principles behind hashing (and trust some command line tool downloaded from Github) is probably smart enough to have better places to store their child porn than an iCloud gallery.

So if the purpose of the system is not to catch persons dealing with CP, what is the purpose? Breaking user's privacy?

We have a history of evil intentions masked by fighting against CP, drugs, money laundry and crime in general.

There is pretty much no enforcement mechanism against any crime which we can expect to catch everyone. False positives get bounced out by the safeguards, false negatives wouldn't have been caught in the first place... that still presumably leaves a number of true positives who wouldn't have otherwise been caught.

The purpose is to avoid Apple getting into trouble

Facebook is catching millions of CSAM images being directly uploaded by people to Facebook/Messenger of all places. There are still plenty of complete idiots trading CSAM that one could see value in apprehending.

That's what boggles my mind. I really don't understand how someone could be trading in CSAM without knowing how dangerous it is to trade in CSAM.

I believe there is an incredibly wide spectrum of most human qualities and traits.

I see no reason to think that general intelligence, technical awareness, utility maximization, impulse control, or risk-reward calculation wouldn’t have similar distributions such that plenty of people would end up in a set prone to share risky material on such channels.

> That's what boggles my mind. I really don't understand how someone could be trading in CSAM without knowing how dangerous it is to trade in CSAM.

One possibility is that these databases are stuff full of material which isn't actually CSAM. There is no accountability for the content of the databases.

Considering the infinitesimal number of prosecutions compared to the claimed number of hits, we should hope that this explanation is actually the truth...

To the contrary.

A system that detects Covid-19 in infected people and doesn't detect it in healthy people, where the errors of both types (false positives and false negatives) are irrelevant, is not only relevant, it's a very well designed system.

> Intentional false positives require knowledge of a hash.

The code/network detecting this will be run on the client. So yes, a 'hash' or an equivalent will be known.

You would have to hope that the secondary system is robust enough to not fall victim to the same manipulated image. And it really is on Apple to first prove it isn't.

No, it is not. The hash is computed on the client, but hash comparison happens on the server, using some cryptographic magic that only reveals a “visual derivative” if enough images match. It’s fairly complex and I don’t recall the details, but the library of “bad” hashes is not accessible.

No, the entire premise of NeuralHash is that no information about your pictures leaves your system unless it has been flagged as 'potentially CSAM'. NeuralHash itself is basically a DB of CSAM hashes encoded as an artificial neural network.

In your example, each picture on your phone (that you are trying to upload to iCloud) would be hashed, and the hash would be sent to Apple. This is exactly the opposite of what they are promising with NeuralHash.

Edit: I am almost entirely wrong, see the comment reply from krisoft. Apologies!

> In your example, each picture on your phone (that you are trying to upload to iCloud) would be hashed, and the hash would be sent to Apple. This is exactly the opposite of what they are promising with NeuralHash.

No. You are wrong. They do hash each image and they do upload the hash derived safety voucher always. Here is the Apple technical paper: https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

There is a fairly complicated cryptographic algorithm which, according to apple, lets them open the safety voucher only if there is a match with the database.

The user devices only have a cryptographicaly blinded version of the hash database which they can’t turn back to real hashes without apple’s private key. This means that the plain neuralhashes are not available to the end users.

Thank you for the actual citation, I edited my comment to avoid misleading others!

Thank you!

> The code/network detecting this will be run on the client. So yes, a 'hash' or an equivalent will be known.

They use cryptography to conceal the hash database from the users to guard it against accountability.

Because of this, if they stuff the database with political or ethnically targeted images no one will be able to tell, you just have to trust that they haven't done that.

Knowledge of a matching hash isn't a big barrier for another reason: Matching images are widely available. NCMEC reports that they had over 20 million matching images reported to them last year. It's a very reasonable assumption that some 4channer will helpfully troll the internet for likely illegal images and put up a large database of likely matches.

False positives can also go the other way: Modify illegal images to match existing lawful images which are already in circulation. One could likely take legal hardcore pornography, crop it to make it impossible to identify the ages of the talent, add captions to falsely claim that they are under-age, tweak it to match targeted images, and submit it to the databases.

That's all just baseless conspiracy theorising.

The other reason why false negatives are mostly irrelevant is because the hash DB can be updated to include them, so the risk of storing them in iCloud remains the same.

That doesn't fly.

The encrypted list is stored on the devices for security and performance reasons: otherwise it would be too easy to ship people customized lists with arbitrary images. You can't just rapidly revise the list without undermining the privacy protection here.

Also the space of modifications is gigantic, for any given image there should be millions of differently hash variations which can be reached without major visual damage. (at least for neuralhash the figure is billions)... so you can't just stick all of them in the database.

You could stick widely circulating variations in the database, however... sure.

I'm honestly not sure what you're refuting here.

Are you saying the hash DB can't be updated or that already uploaded photos won't be compared to hash updates?

The hashdb is part of the operating system and only gets updated with OS updates.

It can be updated-- at some cost. But if it is very frequently updated then the protection against receiving a targeted list is lost.

To state it another way: Users ask "Why won't the government order apple just gives me a list of the hash of every political image on the internet in order to expose my files?" and the answer is that the database is part of the OS, common to everyone, updated infrequently, so it can't be used in targeted attacks. That protection is lost if the database is frequently updated.

It's also true that already uploaded photos won't be rescanned, but that wasn't the point I was trying to make.

So, you're accepting that Apple's approach is useless for anyone who's "determined" but you still seem to be arguing there's some merit in this complex "architecture" of surveillance.

I disagree that it's useless. In fact I wouldn't be surprised if it's already been effective despite not being implemented yet. The announcement of an impending threat of scanning probably managed to get most CSAM afficinados to swear off iCloud already. And if so, it would be no small irony that the noisy criticism would have ensured that Apple was heard loud and clear by the CSAM community.

So the idea is that mass surveillance is great even if it's effect is scaring criminals while it actually only catches people at random?

We've arrived at the opposite of the common reprise of law. No longer "It is Better that Ten Guilty Persons Go Free Than that One Innocent Person be Convicted" but "It is better that Ten Innocent Persons Be Dragooned Than that One Guilty Person Go Free"

All I'm saying is that I understand why Apple doesn't want CSAM on their servers, while also reserving the ability to implement end to end encryption on photo galleries in future.

It's a tough line for them to walk.

Personally I don't see any substantial moral difference to what Apple was planning versus what Google and Facebook already do, aside for the somewhat icky aspect that some initial CPU cycles are occurring on hardware I own.

I'm equally concerned that a purported need to scan for CSAM could be used by legislators as a way to convince the population that the Government needs more encryption backdoors. Personally I'd rather have my photos inconsequentially hashed than have encryption legislatively broken.

All I'm saying is...

Actually, you've made quite a few claims, including some that even ineffective surveillance is good 'cause it makes criminals worried. But anyway...

that I understand why Apple doesn't want CSAM on their servers, while also reserving the ability to implement end to end encryption on photo galleries in future.

Given you've admitted Apple's filtering is ineffective, you have admitted that criminals could put CSAM on Apple servers.

But weird thing is "end-to-end encryption" is entirely useless if everyone is being surveilled before the start and after the end. Sure, Apple wants to be buzzword compliant with privacy and entirely surveil it's users in reality. I suppose I too can see how that would be appealing.

> Actually, you've made quite a few claims,

The phrase "all I'm saying is" is an idiom, not a literal assertion.

> Given you've admitted Apple's filtering is ineffective, you have admitted that criminals could put CSAM on Apple servers.

Yes, I did. And I responded to that already. I'm not interested in engaging with "if it's not perfect then it's useless" type of arguments.

But is getting them off iCloud the goal? As a society, are we better off?

Potentially yes.

Having illegal activity on a major cloud service makes it harder for those services to offer robust encryption without law enforcement pressuring legislators to mandate backdoors.

This system is a backdoor. We should not be surprised if adding a backdoor reduces pressure to add a backdoor. :)

But in the US it isn't just a backdoor, it's an indirect violation of the user's 4th amendment rights. If the government ordered apple to perform this scanning, the scanning would be unlawful and the result would be inadmissible in court. It is only lawful because Apple performs it voluntarily (or at least that is the pretext maintained in court-- repeatedly Apple staff have maintained to me that the government is requiring them to do this). The government should not get the ability to execute unlawful searches simply because it's able to use soft power to coerce user trusted third parties to perform the searching on their behalf.

Reply on news.ycombinator.com