Hacker News Reader

Comment by quitit | Hacker News Reader

The "scanning" feature that underpins this has been deployed for years, and I dare say wouldn't even be considered unique for photo library managers. If you can search for "cat" and get photos of cats, then your photo library already has the tools necessary for CSAM detection.

I feel the fears about this implementation are largely overblown - equivalent features are not being misused on Google and Meta/Facebook's properties, despite having a much larger audience and less procedural oversight. Rather Apple is the outlier here for being not just late, but also by having minimum thresholds that must be passed and an unskippable step of human review.

I'm yet to hear a coherent argument about the potential dangers of the system that understood the implementation, nor the state of image scanning amongst cloud providers - even this thread seems to believe that Apple is the first here. Apple's approach was the most conservative, Google's goes much farther: https://support.google.com/transparencyreport/answer/1033093...

View on HN

The difference is that there is only 1 IF statement or local/remote config flag that will enable this scanning on your device even if you don't use the iCloud photo sharing feature.

It is super well documented that Apple ALWAYS follows the laws of the land, so the scenario is that a government adds some new hashes that might not be CSAM as you and I define it but might be political stuff(but the citizens can't check). Also now that the code is there Apple can't refuse to add this hashes or enable the feature forcefully when a legal warrant for that territory is presented.

some more config changes and it can start scanning for text in documents too.

Given that Apple wrote OSX, this scenario has been possible for over 20 years.

And it hasn't eventuated. But yet now it is something to panic over ?

Until recently you had a lot of freedom on your OSX device, you could install and uninstall whatever you wanted , you could have a firewall that blocked applications. On iOS and slowly on OSX this freedoms are lost, already when you open your computer, open an application Apple is notified (for your safety), only a few more steps are needed until you will get laptops with iOS like lockdowns and maybe a PRO expensive version for the ones that want the key for the gates.

You can blame them when that happens but it’s odd to blame OS X for being locked when clearly OS X is not… locked.

>You can blame them when that happens but it’s odd to blame OS X for being locked when clearly OS X is not… locked.

OSX is getting more and more locked, iOS is locked so why shouldn't people complain when things get worse.

I was just explaining that the excuse "OSX is 20 years old and it was not locked yet so it will never be locked q.e.d " , this is wrong, you can see clearly how things get more locked in steps and you don't have to wait untill last step to complain.

Keep in mind that you have better keyboards on Apple laptops today because people complained, the CSAM shit was delayed because people complained, the Apple tax was lowered in some cases because developers and Epic complained, so complaining sometimes works.

This argument falls exactly into the point I make about not understanding the implementation or the status quo.

Here's what would need to change about the system to fulfil your requirements.

1. The local government would need to pass a law allowing such a change 2. The hashes would then need to come from the government. Instead of the two intersection of two independent CSAM databases. 3. The review process would need to redesigned and deployed to provide a lower threshold and permit the reviewers to see high resolution images. Neither of these changes are trivial. 4. The reports would then need to go to the government. 5. What's stopping these same governments from requesting such things from Google and Meta - both of which have comparable systems with lower oversight?

Apple don't "ALWAYS follows the laws of the land", one can read into this with the recent "secret" agreement between Apple and China which detail the various ways that Apple hasn't complied with Chinese requests (e.g. the requests for source code.)

I see that you don't live in Turkey. Or Lebanon. Or Egypt, or the United States, or Brazil, or any of tens of other nations where this exact scenario is not only plausible, but is of real concern.

Then you should already be gravely concerned because, as I've stated: Apple is last to the game here. Google and Meta each with significantly larger audiences than Apple are already doing this. Since Meta in particular is social media and the main source of casual political dis/information for the masses this would be the low hanging fruit that you're worried about.

The vague "this is a serious concern because my government is shit" requires a person to ignore a lot in favour of a little.

Sorry, my meta hardware is scanning client-side for CSAM? And you claim it's the same for google; can you tell me which Pixel is scanning client-side?

Which apple device is now scanning client side for CSAM?

>Which apple device is now scanning client side for CSAM?

I think you missed OP point, the grandparent found a bad excuse for Apple, that Google and Facebook do the same so nothing is wrong, but the OP responded correctly that this is false, those companies don't install file scanners on your device so the excuse is bullshit.

> those companies don't install file scanners on your device so the excuse is bullshit.

They don’t install file scanners on iOS because they cannot. They are happy with scanning my contacts and doing who knows what with my photos if I let them, though.

>They don’t install file scanners on iOS because they cannot. They are happy with scanning my contacts and doing who knows what with my photos if I let them, though.

Nobody says that Google or Facebook are good/saints, but there is a difference, my phone did not had Facebook or WhatApp installed, so journalists or activists can just don't install this crap.

  > Then you should already be gravely concerned

I am concerned. I explicitly wrote "...not only plausible, but is of real concern."

What is the alternative? Companies deciding that the law does not apply, just because?

Reverse this for a moment: would it be acceptable that a Chinese tech behemoth decides US law does not apply to their American operation, because they are convinced that western democratic values are dangerous?

This is also a real concern. If you think that rule of law is a good thing, then companies must comply with local laws. The alternative is not a humanist utopia; it is the rule of unaccountable corporations.

>If you think that rule of law is a good thing, then companies must comply with local laws. The alternative is not a humanist utopia; it is the rule of unaccountable corporations.

In this case there is no law to force Apple to scan on device, they can scan iCloud only if the law forces them to do that, or scan only iCloud images that are public or shared if the law forces to do only that.

Before 1989 listen to a specific radio station was illegal , but the good part was that the radio device could not snitch on you , if someone would have invented snitching radio then I am 100% only those radio devices would be legal to be sold in my country at that time. So if you create a snitching device then you invite the bad governments to add the laws after to profit from that and make their lives easier.

> two intersection of two independent CSAM databases.

I don't think that offers meaningful protection. A malicious government could acquire or produce real CSAM, or at least a convincing fake of it, then modify that image subtly to match the hash of the political content they were targeting (I believe such engineered collisions have already been demonstrated.) They would then be able to get that manipulated CSAM into as many independent CSAM databases as they wanted, simply by uploading it to the web then reporting it to all those independent databases.

This isn't feasible for a variety of reasons:

1. Forced hash collisions are fragile in comparison to the genuine material. CSAM hashing is devised in a way to allow change in the image without significant variation to the hash. Forced collisions however don't possess that property as their true hash is different.

2. It's trivial to detect images with false hashes, and different hashing methods will not produce collisions.

3. The task you're proposing, in itself is not pragmatic nor effective: dissenting political materials are easily created fresh (i.e. no colliding hash) and exists in many forms that go well beyond an image.

4. Human reviewers again would see the pattern and adapt the system accordingly

5. CSAM database holders aren't idiots. They understand the significance of their hash libraries and work against manipulation from CSAM peddlers.

Points 1 and 3 assume any government wouldn't choose to target people who had a specific document, I think this is a poor assumption. Hash fragility doesn't seem relevant if both the hashing algorithms and the target content are known to the government

Point 2 and 5 I don't understand, could you expand on it? If the government produced manipulated CSAM that didn't exist in any other pre-manipulation form, how would the manipulation be detected? And supposing it were detected, would CSAM database holders choose not to add that image to their databases just because it was manipulated? If that were the case, it would give pedophiles a way to exempt their images from the database too.

Point 4 is neutralized by the government demanding local human reviewers for phones located in their country. China already does this with iCloud, they could demand the same for the CSAM reviewers, then coerce those reviewers or simply select them for party loyalty in the first place.

Oh you mean apple didn't bend over for China when it involved not giving up their source code but when it didn't involve them losing their IP they did it easily? Great job Apple, stand up move that I can trust.

There's plenty of other examples - but people hunting strawmen are somehow blind.

>1. The local government would need to pass a law allowing such a change

Sure, it is not hard, but already laws can apply, did China had to pass a law like "any company has to update their maps software to conform with our local maps?" or China just showed Timmy a graph with profits from China dropping and then Apple said "yes sir!!!" .

> 2. The hashes would then need to come from the government. Instead of the two intersection of two independent CSAM databases

Yeap, the government will demand for a local database to be used and since the crimes are local you need to send the reports o the local police and a local Apple employee will have to check, not some cheap random contractor.

Funny that you mention maps, because mapping information in China is heavily censored, and has been since 2002[0]. They even wrote their own coordinate-obfuscation algorithm so that you can't mix Chinese and western map data or GPS coordinates and get accurate positions on both sides.

Apple's response to the "what if governments force you to abandon the hidden set intersection stuff you wrote" thing is, "well we'll just say no". Which, unless Apple plans to pull out of hostile jurisdictions entirely or levy their own private military[1], holds no weight. Countries own their internal markets in the same way Apple owns the App Store market.

[0] https://en.wikipedia.org/wiki/Restrictions_on_geographic_dat...

[1] which should scare the shit out of you even more than the CSAM scanner

Did you miss the reality where Apple says yes to China all the times? Including maps and storing the iCloud data in China?

The CSAM scanner does not scarry me as it is today, but the fact that is a step, next year will scan more files, then Google will copy Apple (willing or not) and then most PCs will do the same thing.

Apple just normalized that is fine you have an proprietary file scanner/"antiviruys" installed on yoru device with the properties:

1 it can't be stopped or removed

2 it get's hashes from some government agencies over online updates

3 matches will be reviewed by someone that it said are trustworthy and might get sent to your local police

4 the stupid scanner has stupid excuses to exist, where would have been extremely simple to implement and non controversial to scan the fucking images after the upload and not before.

How do you know that there isn't already a secret order in place to look for a particular list of hashes? Perhaps something that has been sealed due to national security?

If it's triggered, send the details to a particular group of individuals with certain clearance to view and verify.

If they have the ability to do this, they will definitely do this!

Why would apple have released a whole white paper on the feature if the plan was… to do it in secret? Clearly they could have gone the google route, start scanning and not tell anyone about it. It’s so odd to blame them for something they haven’t done.

Of course they could decide later to implement it! Just like any company could choose to do that! It’s the whole point, your phone is closed source. But clearly apple didn’t pick that route, they announced it BEFORE shipping and asked for feedback! And you’re blaming them for that?!

>Why would apple have released a whole white paper on the feature if the plan was

Isn't obvious ? Some guy using an exploit would install a debugger on his iPhone and find this scanning process, Apple PR would be hit. But Apple could do this and gain PR by pretending is a safety feature, the guy with the debugger will see that files are scanned and hashes are sent to the server but he can't prove that the config file won't be updated in the future for everyone or some and the hashes will be pulled from a different database, the threashdold will change and the is iCloud enabled flag will be ignored.

What makes no sense and Apple failed to explain and fanboys also failled to convince me with any evidence, why the fuck not scan the images on iCloud only since we are pretending that only if iCloud is enabled the images are scanned. (there is no credible evidence that this is part of any plan to offer some kind of super encryption that Apple has no keys for)

I mean that the Gov will come along and say:

"See that hashing thing you have, this secret gov. order that you can't tell anyone about says you need to look for this list of hashes too. If you find them, you have to let these 5 people know and if you tell anyone you'll accidentally shoot yourself three times in the back of the head to commit suicide!"

> Why would apple have released a whole white paper on the feature if the plan was… to do it in secret?

Didn't that only happen after a leak?

> The "scanning" feature that underpins this has been deployed for years

Yes, I agree that your phone has the ability to find stuff like this based on ML etc. However, if their algorithm finds something that matches a hash of child porn, you'll be reported (after notification or something, can't remember).

Can you imagine what that would do to people? The risk of fucking this up and destroying someone's life is far too high. It's a stigma that will never go away even if it's proven false.

And what if the government decides that they want to ruin a political adversary? The technology is there to report them, rightly or wrongly, for child porn on their phone and ruin them. Again, proving that it was false will not be enough if it leaked out in the first place.

In my opinion, it's my phone (yes, yes, I know they all dial home to the mothership, I only lease it etc.) and I should have complete say over it and this is one more step away from that goal.

It's on the dystopian highway imo.

> matches a hash of child porn, you'll be reported

After 30 hits (so 30 false positives), and then the photos are manually reviewed by a human being. How did people never understand this? How on _Earth_ would people's live be randomly destroyed by this? The reviewer would take one look (or 30) and immediately realize it's not illicit. Crisis averted. But the chances of those 30 false positives actually all being in someone's iCloud was like astronomically unlikely.

> And what if the government decides that they want to ruin a political adversary?

Then you're already fucked, and the child porn scanning software won't be what makes it possible to ruin your life.

And if the government makes it illegal to have a certain document, or a book, what happens then?

You can guarantee Apple will report you. They have to. It will be the law.

It will be used for human rights abuses. Guaranteed.

This falls under the "you're already fucked" part of my argument. If it becomes illegal to have a file, or a book, or an image, then the entire tech ecosystem we all use everyday is completely untrustworthy.

Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope.

> If it becomes illegal to have a file, or a book, or an image

This is already true in certain circumstances.

> then the entire tech ecosystem we all use everyday is completely untrustworthy.

This depends on what tech you use. The degree to which you cannot trust it, or need to understand that it is actively working against you, varies by platform. Nearly all platforms undermine you in different ways. Some of those platforms do so in a way in which you're ok with.

> Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope.

Agree to disagree. I don't mind cloud services (someone else's computer) scanning my data and working against me. I do mind my computer scanning my data and working against me.

>After 30 hits, the photos are manually reviewed by a human being. How did people never understand this? How on _Earth_ would people's live be randomly destroyed by this?

Many people here simply assume all governments and corporations are simultaneously that evil and that incompetent. The aggressive cynicism of this forum and its reticence to engage with content before commenting undermines its pretense at informed technical acumen and civil discussion in many ways.

I understand this. But am also aware of how processes work out over longer timeframes.

Not too many people get caught, review team gets downsized since it is not a revenue generator. Maybe we can replace it with ML!

How many stories do we have on HN about overworked employees reviewing horrible material? Or about how impossible it is to contact support?

Just because there is a good system in place now doesn’t mean it will stay there.

>After 30 hits (so 30 false positives), and then the photos are manually reviewed by a human being. How did people never understand this? How on _Earth_ would people's live be randomly destroyed by this?

Don't be naive, the governments will demand that matches on the "special" hashes will be reviewed by local government people.

There is a similar feature in the "child safety" package that scans for stuff in chat messages and probably (maybe soon) in documents that are received ), so the code is in place to have an uninstallable "antivirus/antiCSAM/antipiracy" on your device that you must trust both Apple and your goverment that does what is promised but 100% you can't just delete the executable to have piece of mind.

> Don't be naive, the governments will demand that matches on the "special" hashes will be reviewed by local government people.

Seriously, if a government is after you, they will get you regardless of the content of your phone. In a country with decent laws, just a match is woefully insufficient to demonstrate someone’s guilt, and they will need some proof. In other countries, laws do not matter so they can just get you without issue. They have done it for quite a long time and are doing it today.

All of this also points to the complete absurdity of this mass hysteria: it is insane to ruin someone’s life just because they have a picture on their device. It is just thoughtcrime, at this point.

>All of this also points to the complete absurdity of this mass hysteria: it is insane to ruin someone’s life just because they have a picture on their device. It is just thoughtcrime, at this point.

But it happened, some guy was falsely accursed because someone made a typo in an Ip address, he lost his job and he was forcefully kept away from his wife and children.

>Seriously, if a government is after you, they will get you regardless of the content of your phone

That is false, for example before 1989 listening to a certain radio station was illegal here in Romania but the government could not ask the guys the made the radio to give them a list with all people that listened to this radio station. Your claim is that the govberment knows my name and what I did they can fuck me up, that is true but this imnplies the gov already knows and has the proof, in this case without the file scanner they won't know I own political/religions materials that is illegal but after the file scanner runs and reports me , only then I am on the gov list.

I hope is clear, tech makes the job of the bad gov easy, so in this case you as a random user you gai nothing from your device scanning for CSAM but many lose from this, the question is why if you gain nothing and this can be done with a simple iCloud scan? (because for sure it will be extended to private messages and documents)

> so in this case you as a random user you gai nothing from your device scanning for CSAM

This makes it seem like there is no upside at all to the CSAM scanning. There is: we stop more folks peddling child porn, hopefully making a dent in the problem.

>This makes it seem like there is no upside at all to the CSAM scanning. There is: we stop more folks peddling child porn, hopefully making a dent in the problem.

Apple could have scanned iCloud from the start and prevent this problem if it is so big. If they are already scanning then there is no use for scanning on device only if iCloud is on and if they are not yet scanning iCloud then you should ask Tim why is he ignoring the problem, Google and Facebook already reported a lot of abuses, does Tim love CP?

> After 30 hits (so 30 false positives), and then the photos are manually reviewed by a human being.

Sounds like a great gig for a pedophile.

> After 30 hits (so 30 false positives), and then the photos are manually reviewed by a human being.

As much as I think the panic is way overblown, this bit is simply unacceptable, and they need to get nailed just for having thought this was a good idea. Nobody is to review personal photos; this is an insane breach of privacy.

I really just love these comments.

You do know that Apple is scanning your photos server-side, right ?

Just like Google, AWS, Yahoo etc anyone that stores your photos in the cloud.

> You do know that Apple is scanning your photos server-side, right ?

Yes.

They have every right to. It's their servers.

However, this is MY phone (ok, not mine, I don't use Apple devices but you get my point)

There's a fundamental difference between you sending files to a company server which are then scanned, and your own hardware constantly monitoring you for illegal activity so that it can report you to the authorities.

It seems that many of the arguments against this system rely on believing governments (or similarly powerful groups) will strong-arm Apple and fabricate evidence.

I fail to see how this system (which I'm not a big fan of, to be clear) makes it easier if you assume governments can and will fabricate evidence. Doesn't seem you need an iPhone, smartphone, or to have owned or done anything at all in that case.

> equivalent features are not being misused on Google and Meta/Facebook's properties

The objections come when such systems are to be run not on corporate property, but on end user property. I don't see anybody objecting to corporations scanning content uploaded to the corporation's own computers. Scanning content on the user's computer, which is the user's property, is what unnerves people.

Furthermore, while Apple may have implemented this feature in a relatively safe way (I don't fully understand it so I'm not confident of that, but smart people here think it so I'll assume it for the sake of argument), I believe that the general public does not really understand this nuance and Apple normalizing the practice of scanning personal computers will clear the way for other corporations to implement scanning in shittier way. If Microsoft were to start scanning all documents on a Windows computer by uploading those documents to their own servers, is that a distinction that would necessarily be made clear to the general public? I don't think so. The rhetoric will be "Apple does it and that's fine, so Microsoft can do it too". The technical differences in the two implementations will be understood in forums like this, but not by the general public at large.

I agree that there is a need for such changes to be public and debated - my grind comes from those which makes arguments which are neither factual, nor pay attention to the status quo. (Thus leaping to fantasy assumptions which are without basis.)

For example Google and Meta both scan all material that is on their services, both of these providers actually go further by utilising AI to hunt imagery which may not be identified yet. So the idea of this compelling others to scan is false: Apple is last here.

Similar to Google and Meta, Apple's CSAM processes only come into play when the material is uploaded to Apple's servers. The minutiae of where the scanning occurs is irrelevant because Apple's implementation is merely a way of performing the function without giving up the user's privacy (by scanning locally rather than sending all of your photos to Apple's servers for review.)

The scanning process is merely the addition of verified hashes to a system which already uses this same technique to categorise ones photos into searchable images ("waterfalls" "dogs" "bicycles" etc.), because this is an existing system there is no impact to performance/device usage.

> The minutiae of where the scanning occurs is irrelevant because Apple's implementation is merely a way of performing the function without giving up the user's privacy (by scanning locally rather than sending all of your photos to Apple's servers for review.)

This is where you and I part ways. I don't think that's an irrelevant minutiae, because it normalizes on-device scanning. This implementation or others may change in the future and opposition to those future implementations will be hampered by "Apple was already doing it" arguments, which will sound fairly plausible to the non-technical public.

Maybe I'm making a 'slippery slope' argument here, but I've become comfortable with such lines of thinking. My cynicism has been rewarded too often before.

Yet the on device scanning already exists on every platform. This is how the system makes content rapidly searchable.

The context of this discussion is on-device scanning that reports users to the police. It's tedious to spell this out; you should be able to infer it.

> I don't see anybody objecting to corporations scanning content uploaded to the corporation's own computers.

That depends on how "BYOD" the company is. There's an awful lot of companies that provide work machines that most of their employees reasonably use for some sort of personal work, be it checking their bank statements to see if they got a deposit or logging into a handful of services so their settings sync. In my opinion it is unreasonable to expect people to not do this, especially given that the alternative is quite painful with modern software. Corporate searches have a problem of not really having clear limits like a legal search might, so going through e.g. your synced web history when trying to find evidence of credential abuse (check your own audit logs!) is common.

My understanding is that the EU has stronger privacy protections for employees using their employer's computers. Personally, I keep all my private business off my company laptop, if I want to check my bank statements I use my phone (which is not enrolled in any company system, I refuse those offers.) I don't sync a single thing between my personal computers and work computer. But yes, I see your point. Maybe there is some merit to stronger privacy protection laws in this circumstance, since so many users of corporate computers lack the inclination, awareness, discipline to keep their personal stuff off computers they use but don't own.

Photo classification takes place on device as well. The models are generated on Apple's side, but the photos on your device are classified by your device while it sleeps and charges. Check the last line on this page: https://support.apple.com/en-us/HT207368

Reply on news.ycombinator.com