Apple Regrets Confusion over 'iPhone Scanning'
bbc.com
bbc.com
They are scanning images on iPhones and iPads prior to uploading those images to iCloud. If you're not uploading images to iCloud, your photos won't be scanned -- but if you are using iCloud, Apple will absolutely check images on your device.
From Apple's Child Safety page:
> Apple’s method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child safety organizations. Apple further transforms this database into an unreadable set of hashes that is securely stored on users’ devices.
> Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the known CSAM hashes. This matching process is powered by a cryptographic technology called private set intersection, which determines if there is a match without revealing the result. The device creates a cryptographic safety voucher that encodes the match result along with additional encrypted data about the image. This voucher is uploaded to iCloud Photos along with the image.
Yes, they will check the images you have chosen to upload. No ‘scanning is involved’.
Claiming this is ‘scanning users devices’ is just dishonest - it’s obvious that it creates a false dichotomy impression of what they are actually doing.
Don’t do that.
Frame it the way you want. It is the device.
If you say Apple is scanning the device, you are lying. They are not scanning the device. They are scanning photos chosen for upload.
Suppose we know there are people who smuggle drugs on airplanes on their person for the purpose of something terrible, like addicting children or poisoning people. If I run an airport I could say: to stop this, I'm going to subject everyone who flies out of my airport to a body-cavity search. Tim, and Craig, are you OK with this? If I can say, "Don't worry! We have created this great robots that ensure the body cavity searches are gentle and the minimum needed to check for illegal drugs," does it really change anything to make it more acceptable to you?
I was just pointing out a falsehood you wrote about what is actually being done.
Anyway, someone in here can accept what the other can't, so let's leave at that and let history tells.
If Chrome scanned downloaded files for viruses and you described that to someone as "scans your computer for viruses" do you think the listener would come away with an accurate understanding of what was happening and accurate understanding of what they were and were not being protected from?
If BestBuy GeekSquad offered a service to "check your device for problems" and all they did was open your photo collection, would you walk away arguing that it proves the screen and mouse and CPU and disk must function and nobody could expect them to do any more than that, service provided, full marks, that's "the device" checked? Or would you hope that "checking the device for problems" might involve at least exercising all the major features like speakers, wifi, bluetooth, at least once, and preferably with stress and thermal tests?
When the TSA ask you to switch your device on to demonstrate that it's not a bomb, are you on the side of "if the screen lights up, the device is thoroughly and effectively tested and cannot contain anything else" or on the side of "an attacker could make up many ways to make the screen glow while hollowing out the insides, this does not really demonstrate that 'the device' is safe"?
"Device scans photos" and "Apple scans device" imply two very different things about how much is scanned, and you're using the latter because you know that if you describe it accurately readers won't be as panicked as you want them to be.
>They are scanning photos chosen for upload
That's pretty much scanning on the device.
That is different from scanning the device. Saying they are ‘scanning the device’ is a lie.
Yes, Apple could scan the device in future. It’s still a lie to say they are doing it now.
>Yes, Apple could scan the device in future. It’s still a lie to say they are doing it now.
Puh..i am relieved now...wait i don't even have a apple product.
EDIT: For Question below
https://technokilo.com/apple-child-safety-feature-third-part...
>Apple didn’t announce any timeframe about when will they implement child safety features in third-party apps. Apple said that they still have to complete testing of child features and ensure that the use of this feature in third-party apps will not bring any privacy harm
Do they? Where have they said that?
Were you aware of that when you posted it?
The Q&A mentioned has no date or time. No Apple Spokespeople are named. There are no actual quotes. No well known news outlets have mentioned this very consequential detail.
This has all the indicators of a fake.
I'd call that photo scanning... and they are scanning the photos on the device.
This isn’t some ambiguous case that needs to be addressed philosophically. They aren’t scanning anything other than the photos being uploaded.
When people say that the device is being scanned for pictures, they know what that means. So it is fine for them to say that the device is being scanned for pictures.
This is about the 16th time I have seen language just like this used to explain away this concern. I don't know if you realize, but this wording makes it sound like you can select some photos and leave others local. I can find no indication anywhere, including on my phone, that iCloud Photos is anything other than an All Or Nothing singular toggle in iCloud settings. If you have instructions to the contrary, I will be happy to stand corrected.
Seriously, everybody is wording it like this. "Photos you choose..." and similar.
If you intentionally make someone think that anything other than the photos they are uploading are being scanned, then you are deceiving them.
Deceiving people to make your point doesn’t help anything. It just makes you a liar and the other person misinformed.
If this is a massive privacy violation in itself, then you shouldn’t need to exaggerate it.
Still, I think Apple misjudged the whole cloud vs. device thing in this case. They’ve historically preached a lot about how everything should happen on the users device, not the cloud. I think that got myopic for them, and led them to this decision.
But in this case I think users would be much happier if Apple had just said “under pressure from law enforcement we are now scanning photos when they arrive at the iCloud data centers. If you don’t want scanning don’t use iCloud.” Because it’s not so much the scanning of uploaded photos that has people upset, it’s the fact that the scanning and phoning home is baked into the device itself.
Many people (like myself) are worried about the slippery slope where this is turned on for all photos, since why not? Not all abusers will upload their CSAM content to the cloud, why wouldn't Apple flip a flag in the future to scan everything, including photos and downloaded content? If they are serious about fighting CSAM and have this great privacy preserving platform, I don't see why they wouldn't do this?
The point of the feature is to prevent people from using iCloud to distribute CSAM. If you’re recording it with your phone, it’s no different than using an slr camera. The cloud part is what they’re worried about.
edit: like many comments here already say, reporting doesn’t sound terrible for CSAM, but nothing about the feature guarantees it wont be extended to other kind of content.
Right, which is why it's so utterly baffling that they don't do this scanning server-side instead of client-side.
Also, the matches are supposedly only to actual babyporn pictures. We have 0% way to verify that, as even employees of NEMSEC are not all allowed to view them. Such DBs are often full of unreviewed fluff, and why not unrelated photos entirely, cookware, computer cases, who knows, as long as “some degree of matching” with your photos allows Apple to send a .zip to the police.
In terms of the privacy intrusion, what difference does it make whether the images are scanned on your device or on their servers? They're getting scanned just the same either way.
But if they did it on their servers, it would provide a technical impediment to expanding beyond the use of iCloud, and remove the need to trust Apple as much. That seems like it would be much better for users while still allowing the functionality they claim to be seeking.
That's for now. The first update can change that and you will have no recourse.
What makes you say this? They announced this change after all. Why wouldn’t they announce future changes?
And I don't think that would include photos saved in other apps.
Apple promising not to use the scanner is a weak promise they know they can’t keep (NSLs)
Five years from now it'll be: "you're wrong"
HN will as usual agree and take pride in being wrong.
I suppose it's great if you're looking for entertainment value. For rational, informed discussion of the technology and its political and social ramifications, not so much. It's just the same refrain of "we never bother to actually RTFA but we imagine a boot stomping on a human face forever."
Most of the commentary on this more recent issue is similarly misrepresented and inaccurate.
I think Apple's mistake here was a PR one, they shouldn't have announced this until they had e2ee ready. Then they could have announced that which would have gotten most of the (positive) press attention. Then they could have gone into details about how they were able to do it while still fighting CSAM.
It wasn’t “one of the most widely accepted and ergonomic grips people use”.
The entire thing was a non-scandal, it wasn’t a real issue. In a lot of ways this is similar.
Compared to their Keyboard which took nearly 3 years before they have a programme for free repair.
There is no way Apple released their initial PR piece without thinking it through and deliberately fusing all those new features together as one big unassailable initiative. It was typical my way or the highway.
Which also make it funny now that they attempt to distinguish between them and run into same hole that they dug for other people.
But in this case, of course, if you're an adult, the Messages part of this doesn't apply to you at all, and the photos part can be completely avoided by not using iCloud Photos.
Apple:
- Isn't going to be remote work friendly.
- Shut down internal polls on compensation.
- Bows to the FBI, CIA, FSB, CCP.
- Treats its customers as criminals.
- Treats its employees as criminals.
- (Spies on both!)
- Doesn't let customers repair their devices or use them as they'd like.
- Closes up (not opens up) the world of computing. Great synergy with the spy dragnet.
Take your time and talent elsewhere. This bloated whale is bad for the world. There are a lot of good jobs out there that pay well and help society.
There's a facade that we really work for other reasons, and money is just an inconvenient byproduct. During a job interview, you may be asked "Why do you want to work for us?". And for some reason "So I can afford to buy food" is not a good answer.
But usually not solely for the money. And usually there are lines people aren't willing to cross just for the paycheck.
As opposed to what??? Free apple stickers??
Of course we might not know of cases that got resolved by internal pressure, because they got resolved, however we do know this was not one of them.
I don't know what you and tharne are talking about here. There was definitely confusion. HN is a tech forum and I still saw plenty of people here worried about how they would get in trouble for having innocent photos of their own children on their phone. You are allowed to be against Apple's plan while still recognizing that many people didn't understand what exactly was part of that plan.
It was not universally understood that this would only apply to photos sent to iCloud.
It was not universally understood that this was only looking for previously known CSAM.
It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system.
I understand if you consider the where more important than the others, but it is simply a fact that there was confusion on what exactly was happening here.
This is ignorance in extreme.
To the extent that other parts of this story was explained to us by Apple, I did try to clarify some exaggeration in other thread.
A lot of the contention wasn't about the specifics of their plan, but rather how subtle changes could vastly expand the scope of their plan.
"this would only apply to photos sent to iCloud." for now, until scope creeps.
"this was only looking for previously known CSAM." for now, until scope creeps.
"using some sort of hash matching so photos you took yourself would not trigger the system." well this one is immediately concerning even within claimed scope because there ARE going to be false positives that apple records some database. Millions of iphone users are going to have a non-zero "possible childporn" score.
They are building an engine for iphone users to self-incriminate. If they rigidly hold the scope to only what they announced and never expand, it could be argued that this is a reasonable concession to fight CSAM. However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
The how helps show us how changing this system is not a subtle change. It isn't like they can flip a switch and suddenly they are identifying new suspected CSAM on people's phones. That would require a new system since the current one is only hash matching.
>However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
This is an arbitrary line that is being drawn. These are photos that are marked for sending to iCloud. Whether the scanning happens on the phone before they are sent or in the cloud after they sent is largely immaterial when it comes to the impact of the code. People are acting as if the line Apple drew was motivated by technology. That was never the deciding factor. Technology is the easy part here. That line was only a policy line and that policy has not changed. Only photos that are sent to iCloud are scanned. If you fear Apple changing that policy going forward, you should have always feared Apple changing that policy.
This is a strawman. Identifying *novel* CSAM is a very hard problem to do accurately - the reason they can't flip the switch is because they don't have the technical capability. All of the other things people are concerned about are things that apple does have the capability to do.
EDIT to reply since at max thread depth: *novel* image detection was never on the table, I think you missed that word
> This is an arbitrary line that is being drawn.
It seems the vast majority of people in this thread disagree that this line is arbitrary.
EDIT: whether the threshold is "verging on impossible." depends entirely on the effective false positive rate. If apple's claimed 1 in 1 trillion rate is true, it's probably not a concern. However, I find it unlikely that perceptual hashes on portions of images won't have higher false positive rates when subject matter is similar (non-CSAM legal adult porn, or images of children in swimsuits, etc). If that rises to 1 in 1 million for these types of images, that's hundreds of thousands of people being falsely accused.
You just used this as an argument against this system. Why do you fear this if you don't think Apple can even accomplish this technologically?
>It seems the vast majority of people in this thread disagree that this line is arbitrary.
I would argue that people who believe that this decision crossed that line were being naïve to not have always known this was a possibility. I don't think this move brings us any closer to Apple scanning our devices for anti-government memes or whatever the fear is because what was stopping that was always more policy than technology.
Also to go back to your earlier comment, in the time since you posted it has now been revealed that this system needs to trigger 30 times before any action is taken. The odds of 30+ false positives is likely verging on impossible.
Maybe. I wonder how many people are choosing not to respond because they’ll only be voted down by the people who feel very strongly about this. I’ve curtailed and hedged many my contributions to this topic on HN because of this.
This is a common issue when social media is used to debate matters which are highly emotionally asymmetric.
I'm a huge Apple fan and a Mac devotee of 30+ years. I love my iPhone. If this thing becomes real, I will go shopping for another phone.
This is absolute insanity. I can't believe they even thought about launching this.
I'm completely disgusted.
I don't think this is right. The apple pdf on the features says
> With the initial match threshold chosen as described above, iCloud Photos servers learn nothing about any of the user's photos unless that user's iCloud Photos account exceeded the match threshold.
This implies to me that they are using some sort of encryption to prevent iCloud from learning even how many matches there are until the threshold is met.
- Whose fault is it that those points were not clearly communicated?
- Who wrote the perceptual hash matching code?
- Who is allowed to audit the code, the review system, and the hash database?
- Who updates this code?
- Who decides if your phone OS is updated?
- Who decides the iCloud upload defaults?
- Who decides if you are reported?
- Who asked for this feature?
* It was not universally understood that this would only apply to photos sent to iCloud.
Since the scanning doesn't happen on iCloud, this distinction is irrelevant.
"We are going to intrusively scan the subset of your photos that you care enough to back up to the cloud that we've been pushing to you for years" is pretty clear.
* It was not universally understood that this was only looking for previously known CSAM.
It was only looking for whatever is in an opaque database which, according to a third party we don't have any contract with, contains CSAM.
* It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system.
Yeah right, I feel totally safe knowing that I won't be falsely reported to FBI by a "some sort of" hash matching.
Here's a hash function: f(x) = 0 for all x
It's "some sort of" hash, too.
It’s really not irrelevant. A third party photo library that avoids using the PhotoKit library would not be touched by the CSAM detector. There are many of these on the App Store.
One step further and store the photos encrypted, with a custom renderer that decrypts the content on the heap, and that would take some tremendous performance-hitting detection abilities it’s extremely unlikely to ever happen.
OK.
Misrepresenting each other’s arguments is not how you have a discussion.
I hope that's precise enough, and sure, everyone is welcome to make their own decisions regarding this.
> HN is a tech forum and I still saw plenty of people here worried about how they would get in trouble for having innocent photos of their own children on their phone.
They're confused about this. NeuralHash doesn't look for pictures of naked kids. It looks for pictures that are identical to the ones they've put in their signatures list.
The problem is that Apple claims that the signatures in their list are all pictures of sexually-abused kids, but we have no way of verifying that. Heck, they don't even have any way of verifying that. Everyone just has to take NCMEC's word for it.
The public does not know what the false positive rate is for 'average iphone user pictures'. As engineers we can be certain the false positive rate is not zero. This means that some number of iphone users are going to have non zero "possible child pornographer" scores in the apple database.
The false positive rate is crucial to understanding how concerning this should be. If the average iphone user has 1000 photos, and the false positive rate is the claimed 1 in 1 trillion, there is a 1 in a billion chance that you'll be flagged as a potential child pornographer. (~1 in the world will be falsely accused). This seems reasonable enough with the apple-internal screening step.
If the chunking and perceptual hashing functionally ends up having a much higher false positive rate for images which have similarities to the dataset (parents' pictures of kids playing shirtless, legal adult porn, etc), the false positive rate could actually be more like 1 in 1 million or worse. In which case there are potentially hundreds of thousands of people who will be falsely accused by this system.
How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches.
I yet to understand what happens to people who only have those synthetic positives? Regardless of what counter threshold is, can’t those people be hoovered up by a subpoena of counter >0 ?
That's really really really user-hostile design.
Even that doesn't come without issue. How long before '1' becomes the value, because, say for example the number is ten, there's also a horrendous PR spin of "Apple has a high degree of suspicion that you have CSAM on your device, but since there's only 8 images, they won't do anything about it" - "Apple allows up to X non-reported CSAM images on Apple devices" is hard to represent in any positive fashion.
If allowed to go forward, it is only a matter of time before the capability is expanded.
So it's a big no to the scanning capability, you would think that Apple had gotten the message by now.
And the other initiative is also open for abuse, by allowing the device administrator to spy on the user. Admittedly not as bad as the on-device scanning.
EDIT: It has now come out that you need to trigger the system 30 times before Apple acts on it. I can't imagine the odds for someone to have 30 hash collisions.
But I haven't seen any positive discussions about it, which is odd.
I don't like the feature. Putting this on the client device is dubious and should never have made it past the brainstorming stage.
Having said that, technology companies, big and small, are bound in the US to do this. By law. If anything Apple was by far the laggard of the bunch (with reporting counts magnitudes lower than peers, despite a larger customer base). As I said in another comment, no company can protect you from your government.
Much has been made about it being on device, which while a serious optics issue...the hot takes being given on here are manifestly absurd. Like, literally the company that holds all of your data, all of your passwords, all of your info and you need to invent slippery slopes to imagine up what they "might" do?
If they want to have their way with your data, they could have been doing it for decades.
They should never have announced two very different systems at the same time. Contrary to some of the insincere claims given in this very thread, there is massive disinformation and confusion about them. In the end I feel like 98% of the "the end is nigh!" comments are by long time Apple detractors who just see this glorious opening.
And while I still hope that Apple says "Mea culpa, we're just going to scan on the ingress to iCloud Photos", whatever they do in a month this is going to be completely forgotten.
Other companies scan their servers instead. And what law banned E2E encryption?
This potentially means all of iCloud, not just photos, could start to use E2E encryption as well - which is fantastic.
What law do you think banned real E2E encryption?
And I said information. The hash matching is information.
[1] https://www.apple.com/child-safety/pdf/Security_Threat_Model...
They have to scan things that are not photos. What if the bad guys just zip their photos and upload that?
There also needs to be a solution to CSAM uploaded before NCMEC had a chance to tag it, especially to cases where the bad guys uploaded their CSAM and deleted it from their iPhone. What happens then, the bad guys get E2E and nobody can find them? There has to be a technical solution in mind for this, and everything I can think of has implications (Let the iPhone store hashes of deleted images? Would it be enough to scan also during download?)
IMHO, any serious attempt to find CSAM using Apple's client-side approach requires more scanning, and Apple not being forward on that makes me trust them less. Also, the moment they expand the scanning, we should think carefully if there actually are any privacy benefits.
Putting it server side is categorically worse. Putting it in the client SDK for iCloud (architecturally speaking) rather than on cloud storage or in the OS is clearly the better correct technical choice, tying surveillance’s hands in a way server side or OS would not.
Most every client SDK routinely checks content before upload, it’s a best practice. Careful examination suggests this was engineered better than that practice.
(Note: even tech trade posts such as LWN, Stratechery, or Daring Fireball trying to write well about this need to sit down a minute and have how it actually works walked through for them, as do many in this community.)
FWIW, I agree with much of the rest of your post except the rationale for low reporting counts.
It's basically engineering with the goal of creating a perception of privacy rather than actual privacy. I don't really care that much about Apple policing what you upload to iCloud, but this disingenuous architecture does annoy me a bit, and there's also the added insult of having a device in your pocket that by design works against its owner (reminiscent of "treacherous computing"). These problems would go away if they just did the check on the server side.
The architectural reason is to do things the server couldn’t.
> You're uploading a photo to iCloud, encrypted with a key that Apple controls.
The architectural reason is so the server doesn’t have to be able to read the photo, and it need not be a key Apple controls.
Your first two sentences are the exact reason to do it on the client instead of on the server, such that it’s possible to have e2e encryption opaque to the server.
Why? Putting it server-side means that it's only possible to examine the images that they claim they are targeting -- those going to the cloud. That seems like a much better and more private way to do it, because it's putting the surveillance "in their house", so to speak, instead of mine.
The list of apps is such a treasure trove. Signal? Clubhouse? Telegram?
https://developer.apple.com/documentation/security/complying...
I need to research what Signal does on iOS. My next canary is encryption of messaging apps other than iMessage.
Plenty of people believe that the Facebook and Instagram apps are recording audio 24/7 and target you ads based on the speech the apps hear. That doesn't stop people from using the apps.
A few years ago some of the most famous people in their world had their iClouds accounts hacked and had their naked photos leaked. That is a lot of people's worst fear. People literally commit suicide over this sort of thing. It didn't hurt the iPhone's market share.
People largely don't care.
1. The penalty of social ostracism due to the network effect. This is a severe punishment to most humans - particularly in today's socially disconnected world. Without these apps, a large # of people would not have any contact with much of their social circle - including family.
2. Learned helplessness. I think that many people have just given up. Even if they knew how to fight for their privacy, they see time and time again that money always wins.
Who knows how much it will matter in the end? But it is hard to argue it doesn't matter.
Take all the paranoia and “muh privacy” around contact tracing or the European Green Pass: It’s as anonymous as it can be, yet millions of people argue against them across the political chasms that separate them. So yes: unless it becomes a divisive topic (which would be undesirable), this CSAM scanning will go away with the news cycle.
About your comment on celebrities’ “sex-tapes”. I guess it takes a certain kind of extroversion and (positive) narcissism to be one, and they’re expected to be gossiped about or to show their bodies in movies or photos so I don’t think those victims blinked that much - some like P. Hilton probably manufactured some porn to ride the wave. On the other hand, revenge porn did drive ordinary people to suicide because in this case the victims weren’t prepared or expected to let the public to see that.
This is old thinking. In the 90s and 00s it was miraculous to trade in a little privacy for some awesome free service on the internet. Google, Facebook and others became huge, and everyone has been influenced / manipulated by them and people are getting tired of having a family dinner conversation next to their smart speaker and seeing ads for six weeks for Depends diapers because someone told a bad joke at the dinner table.
Disagree.
Something this ludicrously intrusive, over-reaching, reckless and trust-destroying is a dealbreaker for exactly the demographic of technology enthusiasts who influence others purchasing. It may not be instantaneous, but it would certainly propagate.
It's anecdotal, but I know zero people who are NOT reconsidering even lifelong loyalty to Apple over this.
It's just too crazy big a wrong, it's like the company just had a nuke go off inside it and is trying to pretend nothing happened.
They may as well have announced a partnership with Trump to put MAGA engravings on all future products, and then in the ensuing furore say they "regret the confusion", whilst carrying on with it regardless.
Though I suppose in that scenario they'd at least be targeting a significant market.
The same can't be said for the size of "sign me up for software-automated police raids" market. A market whose naiveity-induced initial "size" would rapidly shrink after the first few innocents went down, as they absolutely would.
I mean depending on the targets chat software preferences, could a malicious actor potentially ruin an Apple iOS users life just by sending them an image?
There are many troubling scenarios one could imagine. In fact, there is nothing but troubling scenarios.
It's not a can of worms so much as a wormhole, blasting an endless stream of worms at the speed of light.
I'm left not just questioning Apple's leadership, but - honestly - their mental faculties. Really.
No innocent person wants to walk around with an automated snitch in their pocket - I mean, hacking? Bugs? Oversights? Just the overall preponderance of fear that every millisecond you walk around with this thing, it - and its parent corporation and all the depersonalised machinations that go along with it - could be busy organising a blithely mistaken police raid on your family?
Say, because someone you've never met, sent you a message on that new chat app you forgot you installed, that autosaves all media to your iCloud? Or because someone stole the spare phone you keep in a drawer at work, used it for God-knows what and you didn't even notice it was gone? Or, maybe your teenage son got sent something from his teenage girlfriend who unbeknownst to any of them had her phones images uploaded and subsequently catalogued and flagged? Or any number of other entirely plausible scenarios that provide the very reason we have law enforcement protocols and procedures for reporting crimes and that are complex, nuanced and have evolved over hundreds of years and mountains of cases into a massive structure that exists primarily to protect the innocent from exactly this kind of freaking crazy shit?
And Apple expects people to pay them to carry the weight of all that around with them?
I keep seeing this asserted, but there is never any legal citation. What exactly compels a software company to make their software product scan for CSAM?
I get that a service provider like cloud storage may need to scan what they themselves are storing to avoid possessing such material themselves. And iCloud could scan uploaded blobs all day to fulfill their legal department's recommendation.
But what exactly compels a software developer to include a content scanning function in code they distribute? And does this requirement also apply to the authors of rclone?
Apple only scans photos being uploaded to iCloud photos. Google scans. Facebook scans. Microsoft scans. Even tiny image hosting sites scan.
Apple decided to implement this functionality on device, but they could as easily (with much less fanfare and dissent) have placed it on the ingress to iCloud.
When iCloud scans stored files like Google Drive, nobody complains. It's understandable that risk adverse legal departments have come to the conclusion that doing such things is necessary, to avoid a company being in possession of trivially-discoverable CSAM. And from an individual security perspective, you should consider everything you upload unencrypted to be the subject of similar analysis.
If iOS were to encrypt all files before uploading, making any iCloud scanning mostly pointless, Apple would still not be running afoul of any law. Apple is making general software for end users, and encrypting files to upload would be doing the basic user diligence I alluded to. If users end up using the software to do bad things, those specific users are liable and not Apple.
As far as I am aware, there is no legal requirement for a software developer to modify their software to perform scanning of content it will process while being run by other people. But this is what is implied when you say that Apple is forced to do this by law.
Furthermore if this development was driven by iCloud worrying about legal liability from the combined system, then iCloud should be spun out into a separate company that cannot affect the development of the iOS software.
Law requires reporting CSAM. They are not required to scan for it.
They are surely not required to scan client devices, and that was a foolishly, ill-considered plan (that I still would wager they will abandon), however they absolutely must scan iCloud Photos unless they were technically incapable of doing this. US law doesn't say "you go to jail if you don't", it says "you face enormous liability if you don't".
Those are not US companies but I've never heard of AWS, DO, etc. scanning people's storage either.
Pre-Snowden, that was insourced by the NSA.
Except with an iPhone, you don't have a choice.
I'm tossing the mac and iphone because my phone is mine.
Yep. They just altered that deal. (Darth Vader quote deleted)
Upcoming contenders like Purism [1] and the Pine Phone [2] will start gaining a great deal more traction from this. Other SV firms will sense business opportunity..... If merely 5% of the TAM around mobile is willing to prioritize non-spying features that would be enough to stand up very healthy businesses.
It isn't like an iPhone is very customizable, repairable, or that usable with all the App restrictions Walled-Garden stuff.
I’ll bet you $500 to the charity of your choice that this won’t come to be. Set the terms on how you want to measure the outcome.
Since you want to have a friendly competition around "put your money where your mouth is" will look into whether or not Purism is accepting investments and what the terms are. AAPL valuations are pretty lofty right now at ~$149 a share if you'd be interested in the reverse :)
AAPL closed 90 cents short of an all time high share price today. Why isn’t the market pricing in the loss of market share?
It's a hit to their brand from technically knowledgable people for sure though. When someone asks their tech friend if they should buy Apple, more people will likely say, "yabut," or "nah." We'll see if that makes a difference in a year or three.
To me, it just feels icky. I'm sick of all the spying. I've been in computers since the Commodore. The current computer world is shit because of spying. It killed any passion I had left. It seems like no last vestige of privacy remains.
As for market share, this might barely register on people's radar outside the tech community beyond "Apple is trying to prevent child porn."
There are legitimate privacy concerns, and Hacker News users are right to be upset. But that's not an excuse to pretend like this issue is broadly understood.
1. I don't want to financially support anyone going through my private things in conjunction with what is basically the police, looking for reasons to imprison me. Reasonable suspicion first, thankyou very much.
2. I don't trust them to use this in a politically neutral way. This is too much power.
3. I don't trust them to manage the false positive rate. Everyone knows how reliable software engineers are when they claim a system does something. At heart they aren't 1 in a trillion people.
This is a great time to be outraged. I can't really do much about what they do on their servers, but I can certainly get antsy about what happens on my phone.
They just gave a talk at the USENIX Security Symposium on how this works and the safeguards put in place. The scanning functionality is part of the icloud photo upload module. Security researchers can determine if they change the scanning algorithm in future updates. I don't know what else you can hope for.
Going back to the slipper slope argument, you could say nothing stops them (or Google on Android) from uploading your passcode and share it with X. It's all just software in the end that they write that powers the lock screen and security checks.
If they start claiming they do that then I'll get angry about that too. You'll notice that only the fringe is accusing Apple of being liars or acting in bad faith here. Apple are pretty up front about what they do.
The problem with this plan is they're claiming they are only going to do selective law enforcement (only 1 US law, only things that are strongly supported by consensus and even then only sometimes). That isn't a position with a reasonable foundation, they are going to change their mind. I want them to change it in the "we don't snoop on people's phones" direction rather than open season.
Again, I don't expect them to do this. But they could add a useful level of trust if they were really motivated to do so.
As for no company giving up rights, I did say I didn't expect Apple to actually do this, only that they could. Though, companies do actually make binding contractual promises all the time. It's just that they don't tend to do it with consumers or small businesses who have no negotiating leverage and/or no desire to insist on those promises.
Regarding your hypothetical, anything that's actually legally mandated by the government generally overrides contract law since the courts won't enforce illegal contracts and illegality is a defense to breach of contract. They can always do what the law actually requires.
But we're not talking about the government mandating things, since both the iOS 14 behavior and the iOS 15 behavior comply with the law.
Which means this action is just a charade, as I pointed out. Because fundamentally it doesn't really matter. The gov is going to do what they do, and Apple is not going to create a situation in which an external group can determine how or if they can be sued in court based on how overreaching they feel that day.
Also go check how many arbitration clauses exist in your ToS, just as an exercise. I'm all for banning arbitration clauses and allowing consumers recourse in actual courts. I don't believe we should have companies sign their death sentence prior to doing business, this raises the barrier to entry and results in even less competitors who have to do the same.
I could hope for being able to trust that an intensely personal device such as a smartphone isn't something that I have to be constantly suspicious of.
Admittedly, that ship sailed a long time ago, but I could still hope.
You being able to trust them is a personal choice. It seems people are happy to trust companies that don’t talk about what they do with their data rather than the ones that do. Which, while understandable seems counter intuitive.
You cannot be serious.
Could they write and deploy something overnight that hoovered up everyone's data? Maybe on iOS, less so on OS X, but now they're going to ship with that capability. I don't understand how some people can't see the difference between "they could always push some nasty update," versus literally shipping hardware and software with a backdoor.
Do you know this for certain about any vendor? If a company the size of Apple were pushing the same update to everybody, then it would likely be known about by the world pretty quickly. But... if a targeted signed update is sent to a handful of selected devices, that's harder for the world to find out about. It's risky, but it's definitely technically possible.
But it would be very hard to spot new weights of the neural network and a new list of target hashes. These are pretty much guaranteed to change regularly as they retrain the embedding network and/or change the list of known target images. So it will be very hard if not impossible to see what they're searching for. That latest update could just add the ability to recognize CSAM pictures that had meme texts added to them. Or it could change the embeddings and target list to spot unlicensed posting of copyrighted still frames from movies. Or it could now retrieve any picture of people in police uniform. No way to know if you don't have a hunch and a targeted picture you want to test with.
It's Sir <firstname> or Sir <firstname> <lastname>.
https://www.geni.com/projects/Naming-Conventions-for-Knights...
Let’s compare:
— Apple’s market cap is more than twice of Facebook’s.
— Apple’s user base is less than half of Facebook’s. Even fewer use iCloud Photos (and much fewer upload more than the free 5 GB, shared across photos and other content).
— Facebook is entirely made of UGC with no storage limit for a given user, and their moderators have to review every bit of content to ensure the gore is away from advertising targets’ eyeballs (and, indeed, report CSAM). Apple only needs to review images that trigger multiple hash matches occurring within the same account.
Considering the above, my intuition is that Apple’s NeuralHash would have to be completely broken for a company this size to not be able to afford enough moderators to manage the false positive rate. I sincerely doubt they are so inept, and I’d be willing to live with the potentiality of an Apple employee peeking at a photo of mine once a year (me using Apple tech already implies I trust them enough, as I’d never be able to personally verify every privacy claim they make).
What is worth screaming about, and what is eroding my trust, is the fact that since 2019 Apple’s ToS (quietly changed, presumably, to accommodate this feature currently in the news) give them carte blanche for pre-screening any content that they deem potentially illegal. Unless they tighten up that phrasing to limit it to CSAM only, they’re allowing it to be used as a political prosecution tool.
Edit: Factual error, iCloud Photos does have a free tier.
Anyone trusting Facebook with a level of access to their life comparable to a mobile phone is foolish. There is no way I would pay money for Facebook to control my phone.
If it costs them too much money to have too many people personally looking over every single positive hit to make sure it’s not false, then they’re not going to do it even if they technically could.
I live in a place where the government arrests you for having a VPN installed on the phone and labels you a terrorist outright. My phone is checking with physical frisking on the roadside and content critical of state gets automatic manhandling and trip to the police station where I am treated as a criminal.
How I see this as a problem not because I am not going to buy an iPhone in future, but because such ideology would be made normal. That is what is scaring me
The technical implementation is trying to hide the fact that the design mojo of this system is an actual Backdoor for governments of the world to oppress, censure and do whatever they like.
The technical implementation is optimized towards minimizing the cost for Apple. That's why they scan on the device. And this decision is made with knowledge that in the near future "scanning" will be not limited only to hashes. Scanning and processing will be required for $Some_Cool_Functionality to work.
The same slippery slope applies there.
Something I bet wouldn't have happened when Katie Cotton was in charge. But yeah. Tim Cook thought he need new PR direction. And that is what we got. The new Apple PR machine since 2014.
I googled that name and gawker article from 2014 showed up… and I’m speechless…
Industry is learning from omnibus bills
We drank the Apple Privacy Kool-aid, and now we are holding them to it.
This is totally a battle worth fighting!
Are you sure? My local Apple store is just as crowded as it was two weeks ago.
I think the pros list stays longer than the cons list.
I needn't be holding child pornography to be concerned about a third party viewing my photos, writing, or other media on a device that is just mine and not published, public content.
The weirdly less discussed aspect of this is that anyone who is storing their images of any kind on someone else’s computer and network thinks that nothing could have been viewed before. If Apple or Google or Amazon want to scan the data you store with them they could be doing it, so if that was a concern for a person from the get go then they wouldn’t have been storing their data with third parties to begin with.
I honestly don't understand why this is a relevant point. It's still surveillance.
> that anyone who is storing their images of any kind on someone else’s computer and network thinks that nothing could have been viewed before
I don't think that's the confusion. I think a huge part of the issue is that the surveillance is not taking place on someone else's computer, it's taking place on your smartphone. Yes, Apple says it only happens if you're uploading to the cloud -- but that's just Apple saying "trust us". If they did the scanning on their computers instead of yours, it wouldn't be necessary to trust them on this point.
My point is we've already been taking the same risks and the only reason it’s something now is because it’s a transparent process. It’s always a “trust us” scenario unless a person routinely scans all software they is and all updates for malicious server calls or some other kind of recording of data and maybe opening of a back door.
Where you put these will depend on your view on a lot of the issues, certainly.
But, in the past decade:
- Every interaction with your primary device is now, by default, an opportunity for aggressive data collection, often in ways even the people who write the software don't know (because they rely on tons of other libraries and toolkits that are doing this quietly under the hood).
- The default is now that you use a smartphone for everything, with the desktop experience limited or turned into a crappy version of the smartphone version (Image! Video! Scroll, scroll, scroll, never stopping, always seeing more ads! Text, who cares about that ancient stuff?)
- The default has gone from "If you're alone in a social space, you talk to other people" to "You stare at your phone." Certainly was a trend before, with the Walkman/iPod/etc, but it accelerated dramatically.
- Everything has been turned into either a subscription service, or a "Free-to-play" world in which the goal is addiction and microtransactions.
There are plenty of benefits of smartphones, but culturally we're exceedingly bad at looking at the opportunity costs of new technology, and they're increasingly becoming harder to ignore.
If you can honestly evaluate the device and decide it's a net positive, great. But I know an increasing number of people, myself included, who are evaluating them and saying, "You know, never mind. They're not worth the downsides."
I’m starting graduate school in the fall. A few weeks ago, I went in to pick up my new college ID card. The security guard would not let me into the building until I downloaded an app called “Everbridge” on my phone and used it to answer a series of health screening questions (ie, have you tested positive for COVID in the past 14 days).
The app was for iOS and Android. There was no web version. There was no option to fill out a paper form. I was not warned in advanced. But I guess it wasn’t a problem for anyone (including me), because who the heck doesn’t have a smartphone? It’s like having a wallet now—an expected requirement for modern life, even in situations when an analog solution could have worked just as well.
Again, I'm at a point where I can be a thorny pain in the ass about stuff like this, but you carrying a smartphone, even though you (presumably?) know it's evil means that people can do things like this - expect you to download some large blob of unknown code that you're going to run.
As long as they don't encounter people who literally can't comply, it's fine. It works for them.
I mean, I would have refused to download an unknown app I'd never heard of, but... if I pull out a clearly-not-a-smartphone, what are they going to make me do? Go down the street to Best Buy, buy a phone, and come back?
What if your phone was too old to run the app (which looks like a steaming pile, based on reviews)?
Unless there was something in the application documentation about "owning a modern smartphone and being willing to install random applications as required by the university," I would have plopped right down, pulled out a laptop, and started making phone calls to figure it out.
But, again, I'm at a point in my life where I can be a thorny pain in the ass about stuff like this without any real consequences.
For the past week (entirely related to this being a kicker of a motivation on top of a bunch of other simmering long term concerns over Apple and the tech industry in general), I've been carrying around a Nokia 8110 4G - also known, for very understandable and valid reasons, as "The Bananaphone." It's quite literally curved and bright yellow.
The world hasn't ended yet...
It's a bit less of a step for me than other people because I'm already pretty cell-phone hostile. My iPhone (I regret buying a 2020 SE to replace my 6S under the assumption that the 6S wouldn't get iOS 15, which it's getting... maybe...) was pretty well nerfed to start with - very few apps, literally the only apps on my homescreen were person to person or group chat apps (Signal, iMessage, Google Chat, and the Element Matrix client, plus phone, browser, and camera in the bottom). Everything else had to live in the app library thing, which increased friction to use it, and I really didn't have much on there.
But that has been shut down except for a few 10-15 minute windows the past week, and I've been trying, very hard, to work out the transition back to a "dumbphone" (or, as we used to call them, a cellphone).
The main pain point so far is that all my messaging apps used to come to a central point on my phone - so if someone wanted to contact me, it didn't matter what they used, it would ping me if I had my phone on me. Now, that's split (my wife is the main party impacted, I'm pretty high lag on other platforms anyway). If I'm out and about, I can get SMS, but not Matrix/Signal/Chat. If I'm in my office, I can get all of them, but would rather not have a long conversation over T9 - except some of them don't do a great job of notifying me, depending on what machines are running and muted at any given time. Etc. I'm still working this out, and some of it is simple enough - add audio notifications to my "Chat Pi" by wiring in a speaker instead of relying on my phone to chirp if I get a message in Chat or element. That my M1 Mac Mini is going out the door at some point gives me added motivation to solve this.
When out and about, I do at least have the option of tethering to the banana - so I could carry some other device that handles more than the phone does (which seriously isn't much). I'm debating between going back to a small tablet (2nd gen Nexus 7 would be a perfect form factor), or something like a YARH (http://yarh.io) of some variety - a little Pi based mobile computer thing that is exceedingly "We didn't invent smartphones"punk.
I'm at a point in my life (professionally, socially, culturally, etc) where I can happily do "You're weird... whatever..." sort of things with regards to technology, and I'm going to pull the thread until I either figure out alternatives, or determine that they simply don't exist and I can't live without them.
I thought you were going to say it's cellular, modular, interactivodular.
And we're on Hacker News. People know about ROMs and how to use them. Get a Pixel and throw Lineage onto it. It'll be at minimum $100 cheaper and the specs are pretty damn close (minor trades in either direction).
Yes, and I've used it. I'll admit, it is nice. Apple is great about things "just working," but there are also costs to that feature. I'm not sure why people think this is the only way to have said convenience. The major difference is just that Apple this convenience is by default. Honestly Google provides most of this by default too.
> I haven't used others and I am telling you how it looks from my point of view.
And I've used both, and telling you how it is from my point of view. Stop having strong opinions about things that you admittedly don't have experience with. It isn't a good look.
> I will have to invest resources to discover the things you talk about.
You had to invest resources to adapt Apple's ecosystem.
> And there are lots of assumptions in the things you suggested, like that I am fine with using Google or Linux,
Well Google, Linux, Windows, and third party ROMs like Lineage are essentially the only other options out there, so I'm not sure I made many assumptions. And of the most popular, for phones your options are most likely to be Google (i.e. Android) or iOS. The only other option is... third party ROMs. I guess for desktop you could in fact develop your own (besides Linux or Windows), but I think this would fall under a third party custom ROM (if phone) or Linux (if computer). Then again, you could be a BSD person or someone that insists it is GNU/Linux. Let's be real, that's just pedantic, you know what I mean. I'm not going to name every alternative (that's impossible), especially since I covered by far the most popular ones, which share >90% of user share. Why do we have to be so pedantic?
I don't know man, the more you're fighting against this the more it seems like you're just saying "I don't want to try something new." That's fine, but just be open about it. I get it, new things are scary, different, and require you to relearn some things. But that's a different argument. Just state that argument if that's what you're trying to say. Don't make me work for it. Just be honest.
I already own a Pinephone but it's not at a point where I'd want to use it as a daily driver. But they're only $150-$200, so worth taking a chance if you don't want an Android alternative. You may end up liking it. I do know people who are using it daily. It's just not for me. Not yet.
If you want to look into the Android alternatives further, this HN discussion about CalyxOS went into some great detail about that OS, and about other alternatives too.
On a side note: I went to Apple site trying to find that page for all those new features and I could not find one (at least by going to obvious places). The way I was able to get it to link in my posts is by googling it… this whole thing is not yet obvious to laypeople.
I guess what I'm saying is, at least for me, this backlash is blurring their entire privacy and security pitch. Apple built a walled garden, told me it was for my own protection, then come to find out the cameras are all pointing to the inside.
(There are many ways this can be a slippery slope, but we don't have to pretend they could just so what ever body else is doing just as easily and they just want to do it on your phone because they are lazy or whatever. This is a solution to a legitimate problem and also it turns out that people are rightfully worried about what's next; those two facts can coexist)
The second issue is that it will alert authorities.
In regards to CSAM content those issues may not sound terrible. But the second it is expanded to texts, things you say, websites you visit or apps you use it's a lot scarier. And what if instead of CSAM content it is extended to alert authorities for _any_ activity deemed undesirable by your government
Just to be clear, "false positive" in this case means an innocent person is accused of trafficking in child sexual abuse material. It's likely they will be raided.
Sure, that's bad if you're Apple, but it's a lot worse if you're the alleged predator.
Which could also be spun as "Apple allows X freebies of known/highly suspected CSAM on your device before they'll tell anybody".
The amount of PR failure that has gone into all this is huge and multi-level.
From now on, when asked to check whether a user's encrypted phone contains arbitrary content the FBI wants to know about, Apple can no longer say "we don't have a way to do that." Sooner or later, you can bet they will start doing it, whether they want to or not.
If this feature leads to anyone losing their job due to incorrect criminal accusations it will not even make the papers because we expect the accused are guilty anyway. Apple won't shed a tear until there is a class action.
This seems worded to get a Yes answer. So, yes.
It's a big deal because it's unprecedented (to my knowledge) outside of the domain of malware*. Other cloud providers run checks of their own property, on their own property. This runs a check of your property, on your property. That's why people care now. The fact that this occurs because of an intention to upload to their server doesn't really change the problem, not unless you're only looking at this like an architectural diagram. Which I fear many people are.
A techie might look at this and see a simple architectural choice. Client-side code instead of server-side. Ok, neat. A more sophisticated techie might see a master plan to pave the way for E2EE. A net-win for privacy. Cool. But the problem doesn't go away. My phone, in my pocket, is now checking itself for evidence of a heinous crime.
*I hope the comparison isn't too extra. I was thinking, the idea of code running on my device, that I don't want to run, that can gather criminal evidence against me, and report it over the internet... yeah I can't get around it, that really reminds me of malware. Not from society's perspective. From society's perspective maybe it's verygoodware. But from the traditional user's perspective, code that runs on your device, that hurts you, is at least vigilante malware, even if you are terrible.
I see your point here - this is a slippery slope for Apple. However I don’t see how anyone could achieve both purposes - no fingerprint reporting and prevention of CSAM storage on Apple servers.
Also, a practical thing to do is to just not store your photos on iCloud but use something else for sync and backup - there might be a startup opportunity here if enough people care.
iTunes Match is an iCloud service which (if you buy it and opt-in) scans your local on-device music library for copyrighted songs, tells Apple you have them, and then they let you listen to high quality versions of those songs on all your devices. And it's not filename or id3 tag matching, it's doing a fuzzy match that can identify the same song in low quality rips and in different file formats. It would be concievable for them to scan for banned audio lectures, or scan your whole device outside the iTunes library, or change it to check for other copyrighted files e.g. movies. It could concievably be reporting you to the MPAA/RIAA if it finds certain songs along with your public IP address so they can check if that IP address has ever been logged as torrenting those songs. It could "in future" be changed to look for and report evidence of torrenting or movie copying. There's nothing technical or regulatory(?) stopping Apple from saying "people with CSAM on their computers can't use iTunes Match" and making it scan the computer as a condition of use, is there? There's nothing technical stopping a government from asking "can your iTunes Match scan engine report video files in the iTunes library which match popular Tiannamen Square video MD5 hashes?", is there?
I do get that these are not the same seriousness, iTunes Match isn't (so far as we know) scanning to report crime but in so far as "Unprecedented on-device scanning for known content using an opaque database and a closed-source fuzzy-matching engine, it would only take a small change to make it look for other things, governments will definitely pressure them to do that and since they willingly built this system they will definitely agree, and all you can do is trust them", are they not samey enough to be relevant?
At least in the US, the historical distinction between verygood searches and mal searches is given in the 4th amendment: "no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized."
Of course that has been twisted and stretched before, and this is more of the same. But if literally everything is being scanned, "probable cause" is completely absent from the process. It is a fishing expedition of the exact type that the 4A was designed to prevent.
Also, no one(well, most people) has any issue with photos being scanned in the iCloud. Photos in Google Photos have been scanned for years and no one cares. The problem is that apple said that photos are encrypted on your device and in the cloud, but now your phone will scan the pictures and if they fail some magical test that you can't inspect, your pictures will be sent unencrypted for verification without telling you. So you think you're sending pictures to secure storage, but nope, actually their algorithm decided that the picture is dodgy in some way so in fact it's sent for viewing by some unknown person. But hey don't worry, you can trust apple, they will definitely only verify it and do nothing else. Because a big American corporation is totally trustworthy.
Because that doesn’t sound correct to me…
Ignoring the nasty aspects of doing that kind of work, I don't see any way to buttress the fact that Apple has taken another step on the universally one way street of ever increasing surveillance. And whataboutism in the form of "oh but other cloud providers are already doing this" is an extremely weak argument because I don't want to use other cloud providers. I liked Apple, because irrespective of their real motives (ie money), they seemed to be privacy focused. The backhanded way they tried to sell this "feature" shows that they are just as bad as the others.
Apple's solution to this problem is that their employee will actually verify the picture before sending it to authorities. Which again, is one of the problems people have with this system.
https://www.hackerfactor.com/blog/index.php?/archives/929-On...
I think what people are getting riled up about is not the technical ability, it’s the lack of restraint, the willingness to search through everyone’s personal stuff on their phones. This is like the cops sending a drug-sniffing dog into everyone’s home once a day, with the excuse that it is privacy-preserving because no human enters the premises, and that only truly bad people will get caught. There is a difference between scanning in the cloud and scanning on device. One is looking through your stuff after you’ve stored it in a storage unit, and the other is looking through your stuff while it is still in your home. Apple’s excuse is that you were going to move it anyway, but somehow that doesn’t actually excuse things.
If I own my data, someone processing this data on my behalf has no right or obligation to scan it for illegal content. The fact that this data sometimes sits on hard drives owned by another party just isn't a relevant factor. Presumably I still own my car when it sits in the garage at the shop. They have no right or obligation to rummage around looking for evidence of a crime. I don't see abstract data as any different.
> (f)Protection of Privacy.—Nothing in this section shall be construed to require a provider to—
(1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b).
Which is exactly why these policies are so dim witted.
Dragnet violation of everyone’s privacy while anyone even remotely sophisticated can easily evade it by just encrypting the data upfront.
This has been mentioned on here before, but it's known CSAM possession that's illegal. Apple keeps your files encrypted until its algorithm thinks your encrypted file is too similar to CSAM, and then it decrypts it and sends it to Apple for review. There's a few things here.
- The algorithm is a black box, so nobody knows how many false positives it hits.
- Apple's willingness to decrypt files without the consent of the owner makes the encryption seem like a bit of a sham.
- I imagine many are skeptical of Apple's ability to judge CSAM accurately. If I take a photo of my kids in a bathtub, is that CSAM? What about teenagers in a relationship sharing nudes. The law is a blunt and cruel instrument, and we've gotten away without hurting too many innocent people so far because the process is run by humans, but computers are not known for being gracious.
So we know for sure they're not just using PhotoDNA?
> If I take a photo of my kids in a bathtub, .....
Kinda the same question. If they're using PhotoDNA, then that's not really a risk, right? Isn't this technology well understood at this point?
- There's a system to catch CSAM that is either PhotoDNA or something that works similarly.
- There's a system to detect novel nudes, and notify parents if their children view them.
I think I got these two mixed together.
That's fair. Apple did a shit job of explaining themselves, and it has been compounded by a lot of misinformation (deliberate or not) in response. I'm trying really, really hard to moderate my reaction to this whole mess until I feel like I actually understand what Apple intends to do. I don't make platform jumps lightly.
You could argue that a minecraft server is technically in possession of CSAM if that's the case, but you could spend an infinite amount of money looking at various possible sequences and are bound to find many more false positives than true positives.
Services should have a duty to report CSAM when they notice it, but the lengths they should go to search for CSAM should be limited by cost/benefit and privacy concerns.
This type of scenario is what happened with the messaging service Kik, which was reportedly used to distribute CSAM in private chats. Law enforcement agencies said the company wasn't providing timely responses and that children were being actively abused as a result. This is about as damaging of an accusation you can leverage against a company.
Laws against CSAM worldwide are not going away for good reasons, so there is always going to be a justifiable argument that storing certain classes of data is illegal. Hence, anyone wanting to run a cloud service that stores user data will have to obey by those laws, regardless of how proactive they are in scanning for the material. Absolute privacy in the cloud is impossible to achieve with those rules in place.
Let’s not beat about the bush, if someone wants to store information in a form that can’t be decrypted by Apple, they can. This is a stupid dragnet policy that won’t catch anyone sophisticated.
Apple focused the last years pitching themselves as the tech giant who actually cares about privacy. They seemed to be consciously building this image.
To now implement scanning of private information and then try and sell this obvious 180degree slippery slope turnaround in the most weasel worded “but think of the children” trope is an insult to the customers’ intelligence.
I was a keen Apple consumer because I felt that even if their motivation was profit, this was a company who focused on privacy. It was a distinct selling point.
I certainly won’t be buying more Apple products.
For me, Apple lost the main reason to buy their stuff. If they are going to do the same thing everyone else is doing, I refuse to pay the premium they charge.
I can't imagine how they thought this would go well.
It's another example of Apple being stuck in an echo chamber and not being able to objectively assess how their actions will be perceived.
How many times have they made product and PR blunders like this?
Me (Last month): "Apple is taking privacy very seriously. I'm going to vote with my dollars and switch from Android."
Me (This month): "..."
Apple has spent billions in engineering and marketing to establish themselves as the privacy leader, all wiped away by this idiotic system so full of holes you could serve it on crackers.
People assumed this opens the door for Apple to alerted of any known file uploaded to its iCloud storage.
IOW, they assumed Apple can check what someone is uploading,1 despite alleged "end-to-end encryption" and a gazilion promises of "privacy".
No one except the people managing the "detection software" know what files the hashes represent.
Theres no way for the owner of an Apple computer to verify what files Apple is actually checking for.
Is this confusion. It sounds more like lack of trust.
1 Mind you, for a majority of computer owners the uploading is likely occuring by default, automatically, outside of the owner's awareness. As opposed to the owner consciously deciding to upload a particular file to a computer in an Apple datacenter. Tech cmpanies know that users rarely change defaults.
Remember how Apple had zero accountability:
https://www.newscientist.com/article/dn26133-jennifer-lawren...
https://arstechnica.com/information-technology/2014/09/what-...
Ricky Gervais' advice made sense. Wonder why he deleted it.
And that's the crux of the problem.
I sold every share the day they announced this.
Limiting the scanner to iCloud is a policy decision one NSL away from changing.
I have no idea why they haven't done a 180 yet, this is a bigger failure than the butterfly keyboard. They are letting themselves become the symbol of technological dystopia in the public consciousness. Even an acquaintance who does construction was venting to me about how bad apple's policy is and why she is getting a pixel.
After entirely removing that feature and making a commitment to fight against that kind of future I feel like they owe two more apologies to get on my good side - one for screwing up this bad in the first place and one for insulting my intelligence with their handling of the outcry. This isn't 1990, you don't handwave a mistake this big.
I overheard a group of women on the marketing team at my company talking about how creepy it is and I’ve started having a lot people ask me about it - it doesn’t seem contained to just techie at this point but it is concentrated there. I do think it will continue to grow though, apple has lost control of the narrative around their brand.
This is really, really bad for their brand.
Except this time we’re mad too, and we tell them it’s way worse than they even know.
Apple fucked up big time.
That doesn’t mean you should believe them or that there isn’t a real possibility of the system being abused (that’s partially addressed in the interview), but it also doesn’t mean there was no nuance or thought put into the feature or that it isn’t being misunderstood.
All of those can be true at the same time. You can have a bad system that people disagree with for incorrect reasons. That matters: the effort Apple needs to spend fending off inaccurate criticism is effort not spent understanding and addressing the real issue.
I speculate their hand is being forced by one or more governments and rather than admit that they tried to sell it as best they could. Just speculation.
Look at the stock. It barely moved (up).
Pressure from governments.
The only reason we were even told this was being introduced in the first place is because it's being run on edge hardware (ie, phones). One talk at DEFCON on weird resource/energy spikes on apple devices and its existance leaks to the public domain which is even worse PR. The only difference is that historically such government level analysis has been conducted behind data center black boxes.
Well, you could refuse to pay taxes, but that would land you in jail (like most civil disobedience).
Expect everyone else to follow suit and not apologize for it because "Apple is doing it".
Confusion is the best-case scenario for Apple because people will tune it out. If they had released just the on-device spying, public outcry and backlash would have been laser targeted on a single issue.
Apple gave its legendary fan base a fair few facts to latch onto; the first being that it’s a measure against child abuse, which can be used to equate detractors to pedophile apologists or simply pedophiles (these days, more likely directly to the latter.) Thankfully this seems cliché enough to have not been a dominant take. Then there’s the fact that right now, it only runs in certain situations where the data would currently be unencrypted anyways. This is extremely interesting because if they start using E2EE for these things in the future, it will basically be uncharted territory, but what they’re doing now is only merely lining up the capability to do that and not actually doing that. Not to mention, these features have a tendency to expand in scope in the longer term. I wouldn’t call it a slippery slope, it’s more like an overton window of how much people are OK with a surveillance state. I’d say Americans on the whole are actually pretty strongly averse to this, despite everything, and it seems like this was too creepy for many people. Then there’s definitely the confusion; because of course, Apple isn’t doing anything wrong; everyone is just confusing what these features do and their long-term implications.
Here’s where I think it backfired: because it runs on the device, psychologically it feels like the phone is not trustworthy of you. And because of that, using anti-CSAM measures as a starting point was a Terrible misfire, because to users, it just feels like your phone is constantly assuming you could be a pedophile and need to be monitored. It feels much more impersonal when a cloud service does it off into the distance for all content.
In practice, the current short-term outcome doesn’t matter so much as the precedent of what can be done with features like this. And it feels like pure hypocrisy coming from a company whose CEO once claimed they couldn’t build surveillance features into their phones because of pressures for it to be abused. It was only around 5 years ago. Did something change?
I feel like to Apple it is really important that their employees and fans believe they are actually a principled company who makes tough decisions with disregard for “haters” and luddites. In reality, though, I think it’s only fair to recognize that this is just too idealistic. Between this, the situation with iCloud in China, and the juxtaposition of their fight with the U.S. government, one can only conclude that Apple is, after all, just another company, though one whose direction and public relations resonated with a lot of consumers.
A PR misfire from Apple of this size is rare, but I think what it means for Apple is big, as it shatters even some of the company’s most faithful. For Google, this kind of misfire would’ve just been another Tuesday. And I gotta say, between this and Safari, I’m definitely not planning on my next phone being from Cupertino.
You mean that country which gives a damn about privacy altogether because all those fancy corps are giving them toys to play? You know, those companies which feed on the worlds populations data as a business model. The country which has a camera on their front door which films their neighbourhood 24/7? The country which has listening devices all over their homes in useless gadgets?
You have to be joking or that scale you impose here is useless.
This whole thing will go by fast and there won't be much damage on the sales side. Apple is the luxus brand. People don't buy it for privacy. Most of the customers won't probably even understand the problem here.
The only thing we might be rid of are those songs of glory in technical spheres.
How did that work out for you?
I never did that.
Americans were the topic here. See quote.
Privacy is the main selling point Apple is pushing in their current PR campaigns. They've been slowly building up a brand around privacy with new privacy features.
They've just sunk that entire brand/campaign. Instead of "iPhone, the phone that keeps all your data private", it's "iPhone, the phone that looks through your pictures and actively rats you out to police to ruin your life".
>A majority (72%) of iPhone & iPad users are aware of new privacy changes in recent software updates. When asked how well they understand Apple’s new privacy policies, these were the responses: Extremely well (13%), Very well (29%), Moderately well (21%), Slightly well (9%), and Not well at all (28%). Two in three (65%) users are “extremely” or “very” concerned about their activities being tracked as they use certain websites and apps, while only 14% said they were not at all concerned.
And from https://www.androidauthority.com/android-app-tracking-transp...
>You told us: You really want an Apple-like anti-app tracking feature on Android... Over 30,000 people voted in favor of an App Tracking Transparency feature on Android.
People are becoming extremely conscious of online (and on-phone) privacy issues. Where have you been?
Your statement "don't tell me you believe privacy was at the usual consumers mind when they bought their devices... this is ridiculous or you don't meet many normal users." is itself, ridiculous.
Small example: I run a small app. The number of GDPR-related requests is zero. The number of emails like "can my deleted account and deleted data still be recovered?" is like 1 per month or so.
WhatsApp is also a good counter example. People want privacy, but what they want more is utility and network effects. Most people I know didn’t abandon WhatsApp despite numerous privacy mishaps and I think the same will happen here with Apple. This will blow over – unfortunately.
The worlds largest online companies increasingly making privacy a priority in their marketing are all wrong. Got it.
>Most people I know didn’t abandon WhatsApp despite numerous privacy mishaps
I never said privacy was the top issue, trumping all else. You're absolutely correct that other issues like convenience and network effects are important.
>don't tell me you believe privacy was at the usual consumers mind when they bought their devices... this is ridiculous or you don't meet many normal users.
Is true or false? Because you seem to be contradicting yourself. Are people aware of privacy or not? Is the marketing working or not?
I suspect you were trying to say that people weren't aware previously. Is that correct? Because I don't think anyone would disagree with that.
It isn't like the ultimate goal of protecting children isn't worth fighting for, and the ICMEC considers half the countries in the world having no laws against CSAM to be "simply unacceptable." But companies that insist that everything they host can remain private to everyone are lying to their users, and will have to align their marketing claims with the reality of the law, or this kind of backlash will result.
But in general, there are a lot of other descriptors besides "private" that are nothing more than baseless Corporate Memphis copy.
This is wildly disingenuous.
Apple is putting code on the device which generates a hash, compares hashes, and creates a token out of that comparison. That is 100% of what happens on the device.
Once the images and tokens are uploaded to iCloud photos, iCloud will alert if 30+ of those security tokens show a match, it will alert Apple's team, and they will get access to only those 30+ photos. They will manually review those photos, and if they then discover that you are indeed hoarding known child pornography then they report you to the authorities.
Thus, it would be more accurate to say that apple is putting on your device code which can detect known child pornographic images.
> And it feels like pure hypocrisy coming from a company whose CEO once claimed they couldn’t build surveillance features into their phones because of pressures for it to be abused.
This isn't a surveillance feature. If you don't like it, disable iCloud Photos. Yes, it could theoretically be abused if Apple went to the dark side, but we'll have to see what this 'auditability' that he was talking about is all about.
Honestly, with all of the hoops that Apple has jumped through to promote privacy, and to call out people who are violating privacy, it feels as though we should give Apple the benefit of the doubt at least until we have all the facts. At the moment, we have very few of the facts.
We have every fact we need to know to know this shouldn’t be done, and I’m glad that privacy orgs like EFF have already spoken much to this effect.
Or we can just short circuit the entire issue by deciding firmly we don't want this and punish Apple's behaviour accordingly. Which is what appears to be happening.
> it feels as though we should give Apple the benefit of the doubt
It really doesn't feel like this to me at all. Users have clearly stated: we don't want this. It's time for Apple to simply pull it all back and apologize.
There is no way to determine whether the hashes are about CP or about HK protests.
> ...we should give Apple the benefit of the doubt...
You have to take off your apple branded rose tinted glasses my friend.
Any company as big as apple needs to be scrutinized as harshly and critically as possible.
Their influence on the world is so big that a botched roll out of this sort of tech could be absolutely devastating for so many people, for so many reasons.
I don't care if it's hashed tokens or carrier pidgins. We should only allow companies to act in ways that improve our lives. Full stop.
> Thus, it would be more accurate to say that apple is putting on your device code which can detect known child pornographic images
> If you don't like it, disable iCloud Photos.
> Yes, it could theoretically be abused if Apple went to the dark side [...]
> [...] it feels as though we should give Apple the benefit of the doubt at least until we have all the facts.
No, nobody gets "the benefit of the doubt". The very use of that phrase admits that you are being put into a situation where you could be screwed in the future.
There is zero transparency or oversight into the code that does the scanning, the in-person review process, or the database of images being scanned for.
Apple: We have 29 matches on your device of "known CSAM"[0]. However, despite our confidence level being very high, we won't report it to the authorities because we value your privacy!
[0] for varying definitions of 'known CSAM'.
Check the section "WHAT IS APPLE DOING WITH MESSAGES?" in this article: https://www.theverge.com/2021/8/10/22613225/apple-csam-scann...
This feature is a catastrophe.
I can’t imagine that’s a big issue…
This disinfo really angers me. That is the exact opposite of what I've read up till now. People talking about "NeuralHash" and being able to detect if the image is cropped/edited/"similar". SO what is the truth?
This DaringFireball[0] article states the goal of the system is to "generate the same fingerprint identifier if the same image is cropped, resized, or even changed from color to grayscale."
So while the fingerprint may be "exact", it's still capable of detecting images which have been altered in some way
[0] https://daringfireball.net/2021/08/apple_child_safety_initia...
Of course, that's just a nasty way to imply that the images match exactly.
This is the confusion, it's only photos being uploaded to iCloud.
That said, the argument that many people in these threads are making is that they say it's reasonable to scan photos that are uploaded once they're on Apple's servers, they just don't want them scanned while they're still on their phones. In either case, the same photos will be scanned -- ones which are in the process of being uploaded to iCloud -- the disagreement is just about exactly when in said process it's okay to do so. Which seems like a pretty fine distinction to me?
> The main purpose of the hash is to ensure that identical and visually similar images result in the same hash, and images that are different from one another result in different hashes. For example, an image that has been slightly cropped or resized should be considered identical to its original and have the same hash. The system generates NeuralHash in two steps. First, an image is passed into a convolutional neural network to generate an N-dimensional, floating-point descriptor. Second, the descriptor is passed through a hashing scheme to convert the N floating-point numbers to M bits. Here, M is much smaller than the number of bits needed to represent the N floating-point numbers. NeuralHash achieves this level of compression and preserves sufficient information about the image so that matches and lookups on image sets are still successful, and the compression meets the storage and transmission requirements
Just like a human fingerprint is a lower-dimensional representation of all the atoms in your body that's invariant to how old you are or the exact stance you're in when you're fingerprinted... technically Federighi is being accurate about the "exact fingerprint" part. The thing that has me and others concerned isn't necessarily the hash algorithm per se, but rather: how can Apple promise to the world that the data source for "specific known child sexual abuse images" will actually be just that over time?
There are two attacks of note:
(1) a sophisticated actor compromising the hash list handoff from NCMEC to Apple to insert hashes of non-CSAM material, which is something Apple cannot independently verify as it does not have access to the raw images, which at minimum could be a denial-of-service attack causing e.g. journalists' or dissidents' accounts to be frozen temporarily by Apple's systems pending appeal
(2) Apple no longer being able to have a "we don't think we can do this technically due to our encryption" leg to stand on when asked by foreign governments "hey we have a list of hashes, just create a CSAM-like system for us"
That Apple must have considered these possibilities and built this system anyways is a tremendously significant breach of trust.
http://www.fmwconcepts.com/misc_tests/perceptual_hash_test_r...
The whole point of the system is that you get a matching hash after mirroring/rotating/distorting/cropping/compressing/transforming/watermarking the source image. The system would be pretty useless if it couldn't match an image after someone, say, added a watermark. And if the algorithm was public, it would be easy to bypass.
The concern, of course, is that all of this many-to-one hashing might also cause another unrelated image to generate the same fingerprint, and thereby throw an innocent person to an unyielding blankface bureaucracy who believes their black-box system without question.
"Find all images and tag them if they look like this fingerprint" doesn't mean that. It means: "Find all images and tag them if they look 80% like this fingerprint".
Which also means that it will allow governments to upload photographs of people's faces and say: "Tag anyone who looks like this".
Worse, this will allow China to track down more Uyghurs, find people based on guides in the form of images that are spread around to stay safe from the Chinese government, and countries like Saudi Arabia can start looking for phones with a significant amount of atheist-related images, tracking down atheists, and killing them. Because that's what that country does.
https://www.reuters.com/article/us-china-apple-icloud-insigh...
Is this list of hashes already public? If not, seems like adding it to every iPhone and iPad will make it public. I get the "privacy" angle of doing the checks client-side, but it's little like verifying your password client-side. I guess they aren't concerned about the bogeymen knowing with certainty which images will escape detection.
But hey, I'm just one of the screeching voices of the minority.
[1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
"Indeed, Neural-Hash knows nothing at all about CSAM images. It is an algorithm designed to answer whether one image is really the same image as another, even if some image-altering transformations have been applied (like transcoding, resizing, and cropping)."[1]
[1] https://www.apple.com/child-safety/pdf/Security_Threat_Model...
“The system could only match "exact fingerprints" of specific known child sexual abuse images, he said.”
Or like whatever they, the US govt, or any govt where they want to make money (such as China) wants. Is anyone auditing the blacklist? Is it publicly reviewable? (Since it contains CP of course not)
China can start finding Uyghurs based on the images they tend to share. If we're unlucky (as a world), they can even start searching for particular individuals.
"Save the children" is just the classic political ploy to get a ruling through that's just a precursor for evil things to come.
I'm absolutely disgusted by Apple.
They already were storing the photos unencrypted (or with keys available, at least) on their servers, so any government that was able to push them to add a hash to this scanning system could have gotten them to scan for something in iCloud.
China, in particular, could definitely already be doing that, since China made Apple host all iCloud data for Chinese users on servers inside China that're operated by a Chinese company. See: https://support.apple.com/en-us/HT208351
If Apple's willing to change the new system to do a full scan of all photos on-device and send notifications to them outside of the upload-to-iCloud-with-security-tickets mechanism, they could just as easily have done that with the old system.
What do you mean by capability here?
I fully understand the difference, but I’m just saying it’s not like Apple had to build a massively different system to support this.
Can a government secure Apple's cooperation in that? I have no idea. But it does make a useful subversion of the hash database a more complicated thing to accomplish.
Speaking for myself, if Apple is correct about those odds, I'm not personally feeling creeped out by it. If they're wrong, my opinion could change. I certainly don't have the math and security background to actually verify their claims from the white paper they posted about the system, though.
Getting more information about the hashing function they're using would be nice. It'd make it much easier to see how actually collision-prone this is. I'd be all for them getting some external review of it published, much like the review of the security-tokens they've published. (I appreciate that it's difficult, because providing the hashing function itself to experiment with lets awful people tune their images to be just-distinct-enough that they won't match.)
It's worth bearing in mind that Apple has a fairly strong motivation for the hashing to be good. They have to pay reviewers to look over these matches, and it's bad PR if it turns out that they're massively backlogged.
Where did you see that? I tried to find more info about it, but I didn't find anything.
1) That is an MD5 hash, not a perceptual hash. Apple is not using md5s.
2) It is a false positive, not bad info in the database. All involved acknowledge the possibility of false positives.
NCMEC generated the hashes Apple will use by running NCMEC's collection of forbidden media through Apple's algorithm. And perceptual hashes have more collisions than cryptographic hashes.
Several people said the database includes non CSAM seized in investigations.[1]
And if memory serves well it doesn’t need to be Apple as a whole cooperating, a single employee with oversee power could be enough.
Their approval/report rate will make a FISA court judge blush.
It’s a perfectly planned PR response but no one except the biggest sheep is buying it.
The PR show is kind of besides the point.
The company should only be concerned with following the law, not earning bownie points for extralegal behavior. Making the government happy shouldn't be a thing in a country ruled by law.
The other feature they're packaging with this (nudity warnings for children/teenagers) should be relatively uncontroversial. It seems well designed and respects the user's privacy: It shows a bypassable warning on the device, only sends a warning to parents for children up to 12 years old, and only if the child chooses to view it, and only after disclosing that the parents will be notified. I don't think there is much criticism they'd catch for that, no protests to quell.
On the other hand, the proposal that they're (rightfully) under fire for now is something that they can't easily back out of (they will immediately be accused of supporting pedophiles), and it's basically a "do or don't" proposal, not something that they can partially back out of. The press is also incredibly damaging to the "iPhones respect your privacy" mantra that's at the core of their current PR campaign.
I don't think they expected this level of pushback.
They could just say “Following the unexpected pushback, we will scan iCloud content on server like all other major cloud providers. Unfortunately this also forces us to shelve plans for end to end encryption for photos”.
To me that's the worst of the two. From what I understand, it uses AI trained to detect nude photos. This is much more likely to produce false positives than a hash compare.
Not only is it less accurate by nature, it's a backdoor that can be later used to scan the entire device for whatever photos it's trained to find and report to whoever it's coded to report to.
I don't think that is the most likely situation at all.
Apple has been, as part of their privacy initiatives, trying to do as much as possible on the device. That's how they have been defining privacy to themselves internally. Then someone said "can we do something about CSAM" and they came up with a pretty good technical solution that operates on device and therefore, to them, seemed like it would not be particularly controversial. They've been talking about doing ML and photo scanning object recognition on device for years, they're moving much of Siri to on-device in iOS 15, all as part of their privacy initiatives.
It seems to have backfired in that people actually seem to prefer scanning in the cloud to on-device scanning for things like this, because it feels less like a violation of your ownership of the device.
I think the security arguments about how this system can be misused are compelling and it's a fine position to be strongly against this, but I don't know that there's good justification that Apple has some ulterior motive and is faking caring about privacy. I think they were operating with a particular set of assumptions about how people view privacy that turned out to be wrong and they are genuinely surprised by the blowback.
That's an oxymoron.
Also worth mentioning that the original corpus of data used to build the hashes isn’t inspectable, so we don’t know how much garbage exists on the input side.
A recent blog post suggests that the corpus of CSAM isn’t even very comprehensive, so the value of this system is questionable.
People defending CSAM should go to hell, fast. But are we already done destroying all low-hanging fruits? Did we stop Johnny Savile? Did we put all clerical actors behind bars? Did we extinguish the child porn network behind Marc Dutroux (https://en.wikipedia.org/wiki/Marc_Dutroux)?
And even if we did, would that be enough of an excuse to implicitly accuse anyone? My spouses' family (well-off, so using iDevices) took photos of their young age kids playing, partially naked at the sea. They are now frightened if their photos could be stolen by someone and marketed as child porn.
So unbelievable.
That sounds bad, someone high up at Apple should do an interview clarifying that that's not what's happening!
Perceptual hashes on chunks of images will yield false positives.
Apple says false positives occur at a rate of one in a trillion or something like that. Someone elsewhere in the thread calculated that this would not be frequent enough for false positives to trigger a manual review.
These chunks contain less entropy than the whole image so necessarily have higher chance of triggering false positives than whole-image comparisons.
This is malicious. I do not want them to touch my photos or anything personal. I paid for this device, now it is doing things against my will.
Simply disable iCloud Photo Library, and nothing gets scanned.
None of the usual anti-regulation apologists have pointed out that Apple shouldn’t be forced to download and host potentially-illegal material in the interest of ensuring whether or not it’s actually illegal.
This whole program is their intelligent solution to protecting as much user privacy as possible while still being compliant with the law. On-device hashing is actually pro privacy compared to in-the-cloud scanning (which all other cloud hosting providers are also required to do).
However, I also think this is the poster child of the proverb that the road to hell is paved with good intentions.
Now Apple needs to cancel this misguided initiative and never speak of it again, if they want to salvage some of their reputation.
Agreed that this was not intended to be malicious. Apple has always been pretty clear they they should decide what happens on their devices. This sort of on-device scanning that doesn't serve the user is just the latest example of it, and one that people who would never be affected by the code signing restrictions can relate to.
> Apple has been, as part of their privacy initiatives, trying to do as much as possible on the device. That's how they have been defining privacy to themselves internally. Then someone said "can we do something about CSAM" [...]
As a separate issue, many people in the company certainly do care about privacy, and that may go all the way to to Tim Cook. Who knows.
What is much more important to Apple the company, though, is making money. Governments have been hounding them for years about letting them spy on users. And they have painted themselves into a bit of a corner, by having the most secure phones.
Now the government comes to them with an offer they can't refuse, cloaked in child porn motivations. I believe many (most?) of the people involved are sincere. It's clear they have tried to make the least invasive system that still does what the government wants.
But that's not good enough in the crazy connected cyber-world we find ourselves in today.
Apple doesn't have a motivation to do this themselves. But they will do what they calculate they need to do.
China would happily pay billions of dollars per year for this capability.
China doesn’t need this feature.
It doesn't just feel like a violation, it is one. This opens doors for a number of future abuses of power, and abuses of rights.
The average user has no way of knowing what's in the database of hashes being compared against, and we have no way of controlling what goes into those databases. If some party in the US government decided that they didn't like the Quran, and put a million pictures of it in the database, and served the Apple reviewers with a gag order or some other nonsense, we'd have a 80's dystopian blockbuster as everyday life in our country.
My device, my property, my rules. Apple doesn't get to do a stop and frisk, even if it's "for the good of the children".
Your phone has been Apple’s device since the moment you bought it and turned it on.
Apple has also benefited from the perception that Google (their smartphone rival) spies on people by collecting location information where as I am sure they also crowd source iphone location to improve their Maps and road traffic data
Unlike Android phone, iphone will know about your uploading potentially problematic image beforehand. Yet, it will not alert you or block you from doing so.
Personally, I don't have a problem with the parental control feature in Messages (it's pretty clear what it does and the user can decide whether to use it or not, or the parent for younger kids -- that's exactly as it should be).
I do have a problem with the feature where they scan images on my phone to match against a database of images. To be clear, here's a list of things that don't make me feel better about it: that it scans only a certain subset of images on my phone; that the technology parts of it are probably good; that NCMEC maintains the database of images (is there any particular reason to believe the database is near perfect and has all appropriate quality controls in place to ensure it remains so?)
There are several issues about this that Apple does not address. A big one for me is the indignity and humiliation of them force scanning my phone for CP.
Here's a hypothetical for Craig Federighi and Tim Cook to consider:
Suppose we know there are people who smuggle drugs on airplanes on their person for the purpose of something terrible, like addicting children or poisoning people. If I run an airport I could say: to stop this, I'm going to subject everyone who flies out of my airport to a body-cavity search. Tim, and Craig, are you OK with this? If I can say, "Don't worry! We have created this great robots that ensure the body cavity searches are gentle and the minimum needed to check for illegal drugs," does it really change anything to make it more acceptable to you?
If Congress has to obey the Constitution, then they cannot create an organization which they control, and then push for that organization to execute functions they cannot perform by getting in cahoots with industry.
Are you okay (conceptually, assuming a perfect database and hashing function) with them scanning pictures uploaded to iCloud for this material if the scanning happens on their servers? Or is this a complete "these pictures should never be scanned, regardless of where it happens" position?
If the former, I personally don't feel a distinction between "a photo is scanned immediately before upload" and "a photo is scanned immediately after upload" is very meaningful. I'd be more concerned if there wasn't a clear way to opt-out. I acknowledge that there's room to disagree on this, and maybe I'm unusual in drawing my boundaries where I do.
If the latter... I think that ship has sailed. Near as I can tell, all the major cloud platforms are scanning for this stuff post-upload, and Apple was a bit of an outlier in how little they were doing before this.
It has to match the fingerprint exactly, but the fingerprints themselves are not exact, otherwise they would be useless.
And this is completely beside the point. People's concerns aren't mostly over false positives, they're over the possibility that this feature will be perverted by authoritarian governments. Way to miss the point.
> Mr Federighi said the "soundbyte" that spread after the announcement was that Apple was scanning iPhones for images.
> "That is not what is happening," he told the Wall Street Journal.
That's... exactly what's happening.
https://en.wikipedia.org/wiki/Democracy_Index
> The index is based on 60 indicators grouped in five different categories, measuring pluralism, civil liberties and political culture.
The 2020 report classifies 57 countries as authoritarian regimes, 35 more as hybrids.
What's wrong with you? Why are you still with Apple? Are you paid by them?
I’m not defending Apple, I wish they wouldn’t do this, but I see section 230 levels of lack of understanding out there.
If people understood what was going on, would they be as upset? I don’t know. Apple doesn’t seem to think so.
[1] https://www.engadget.com/2020-01-07-apple-facebook-ces-priva...
>but I see section 230 levels of lack of understanding out there.
Mirror mirror on the wall....
That would require them actually changing their plans though, and it doesn't seem like they're willing to do that (yet).
The distinction is whether the matching happens on-device before upload or in the cloud after upload, it seems. If Apple already does on-device ML, it makes sense they would add more photo processing client-side to take advantage of encrypted or archival blob storage server-side.
Additionally, there’s still the option of using a third-party camera app, which wouldn’t upload photos by default at all.
...for now.
If you store the magazines in your basement, locked in a vault (password protected computer), then we have an expectation of privacy. To have the vault manufacturer sneak into your home, forcibly open the safe, and inspect your magazines is a very invasive thing. Especially because if this wasn't all over the news, no one would ever think such a thing is happening.
Just as you cannot be partly pregnant, you cannot be partly trustworthy on privacy and security. Apple blew all their credibility in one stupid decision to appease the unappeasable authoritarians.
I was impressed a few years ago when Apple wouldn't allow interference with a device that belonged to an unconvicted suspect (I can't remember the details, apologies). But this concession to unmonitored surveillance is really disappointing.
Call me conspiracy theorist, they must have been forced so badly to make this kind of a change.
This also means that if you get big enough, you lose your 1A rights in the USA because the feds will punish you extralegally if you do things they don't like or that make life harder for them.
Sad state of affairs in the USA.
Nothing confusing about it however. When you use a closed platform, things like these are literally the endgame for those corporations -- namely being able to not only have access to all that goes through the devices but to profile you and, in one bright and an ever-so-close future, censor and police you.
I've a made the conscious choice of using Apple because I value my time and energy more than the 0.01% chance of me being wrongfully flagged. Their products are robust and convenient. But with these news I have partially revisited my stance and I'll start pulling some of my erotic photography collections to a private NAS / home cloud server. I wish them luck breaking through my router and several layers of Linux virtualization and containerization.
I really have nothing illegal to hide but the slippery slope of "for the children!" can be used for anything and everything. I won't be a part of their game.
In a few weeks/months it will be "your move, corpos".
----------------------
SIDE NOTE / OFF-TOPIC:
I wonder at what point we'll get to the trope of "non-approved Internet traffic is a crime"? Hopefully not in my lifetime but I believe we're bound to get there eventually.
I bet there are facebook lobbyists pushing for this today.
Off course they have been victim to general scepticism towards big tech companies which has gained traction recently. One might ask, why did they not broadcast clearer messages and why was the "confusion" clarified for so long.
I wonder, though, why the back tracking on the messaging happened. Reputation damage or fear it might affect the bottom line.
With this act, they kind of make anyone guilty unless proven otherwise by scraping their data. Apple was the last man standing regarding telemetry handling, bit better than others. Now, I am not so sure
Earlier, when they said about allowing people to stay on iOS 14, that was a real head-scratcher which now makes sense.
Regardless of how you feel about it, both issues were being completely mixed up by every single person I saw discussing this - even otherwise very technically competent people on this very site.
I've no doubt that it muddied the waters significantly when it comes to discussing this.
https://www.forbes.com/sites/thomasbrewster/2020/02/11/how-a...
Do they mean they haven't been doing it for iCloud Photos, but were arbitrarily doing it for other parts of iCloud?
If they were already scanning, you’d expect more reports since although there is no legal requirement to scan, there is a legal requirement to report detections.
I read elsewhere they they scanned Mail but not Photos.
Which, again, really hits the need for disclosure — so much of the response to this announcement has been heavily shaped by both that secrecy and just springing it on the world without much prior public recognition of this issue.
At some level, if you're uploading files to their servers, you have to trust them. And to a lesser extent if you're using their proprietary software (although you can monitor network traffic and so on.)
> Which, again, really hits the need for disclosure
Isn't that what they did?
There was no reason for these two features to be bundled, other than try to dilute the nefarious one with the benign one.
> [Federighi] said it would do the image-matching on a user's iPhone or iPad (...)
be reconciled with this:
> Mr Federighi said the "soundbyte" that spread after the announcement was that Apple was scanning iPhones for images.
> "That is not what is happening," he told the Wall Street Journal.
without at least one of them be a blatant lie?
Is it the tense of "was" in "Apple was scanning (...)" as opposed to "will start to scan"?
To be clear, this is a distinction without a meaningful difference. Or, if there is a difference, it's that it's actually worse than the alternative (cf. the Stratechery article that's been making the rounds).
If that's right, then this isn't a lie, but it's incredibly mealy-mouthed, misleading, and disrespectful of their customers' intelligence.
Perhaps the nuance is irrelevant or disingenuous, but there is a valid interpretation of his words where he isn’t “blatantly lying”.
"Confusion" is what they're trying to sow now.
But that's exactly what's happening? Most people using an iPhone sync photos with iCloud (especially after they introduced the more cost-effective 2TB Apple One plan), images are scanned before they are uploaded to iCloud, ergo Apple will be scanning the iPhone for images.
Apple regrets "confusion" over iPhone scanning.
We are not confused.
* they use some kind of homomorphic set intersection algorithm as part of it
It's time to expand (or reaffirm) the 5th Amendment to apply to your digital mind as well as your meat mind.
[1]: https://constitution.congress.gov/constitution/amendment-5/
The irony is thaht it's exactly Apple's insistence on bluring the line between products and services which make it not OK, and thus bites them in ass!
I don't find the following argument compelling; Because this tech will be used to scan known CSAM, it will necessarily one day be used to scan for non CSAM. If Apple can implant this tech on your IPhone now, it always could have, and therefore the threat of the government coercing Apple to scan all images for whatever pernicious reasons they can think of has always existed.
CSAM is a massive problem. The solution to how we deal with it will be nuanced and plagued with tradeoffs, but I refuse to be an extremist for either side. I do want something done about CSAM, which is why I am happy that Facebook reports over 10 million instances of it per year from messenger. I also want devices to be mostly private (to assume that a device manufactured by a large corporation would ever be perfectly private in the internet age is delusional). But anyone who acknowledges that CSAM is a problem must also acknowledge that some sacrifice of privacy would be necessary to mitigate it. Or, perhaps one day we can rely on homomorphic encryption to deal with this.
No, because there will always be false positives, which means someone is going to be manually reviewing your photos.
So does this mean that no measure to prevent spread of CSAM should be accepted so long as it has false positives? If we are to trust Apple's numbers, that would be every 1/1trillion images.
https://www.wsj.com/video/series/joanna-stern-personal-techn...
The “think of the children” might have been a good wedge if their long term agenda involves expanding this kind of scanning, but it’s creating some strong ties between CSAM and their usually very “clean” brand.
If their aim was to comply with their reporting requirements and demonstratively prove that this kind of content can’t hide within Apple’s ecosystem (a huge advantage in many peoples eyes) then surely they should have opted for the least controversial option? From reading comments, I think very few would have pushed back against comparing image hashes on their servers after upload, even if that became a blocker to E2EE - that we don’t have, and might not ever get anyway.
So you see, Apple is only going to scan certain things at certain times under certain conditions. So we can all relax now, OK?
Of course this is not the point. But skimming through the article I'm not impressed by these mostly irrelevant bits either.
Oh that's a relief. Good luck trying to get 2 intelligence agencies to cooperate.
/s
https://www.apple.com/child-safety/pdf/Security_Threat_Model...
Specifically, it looks like they will be requiring that hashes exist in two separate databases in two separate sovereign jurisdictions.
"That is not what is happening," he told the Wall Street Journal."
But it is. That's exactly what's happening.
"Apple decided to implement a similar process, but said it would do the image-matching on a user's iPhone or iPad, before it was uploaded to iCloud."
Craig, Mr. Federighi, do you think we're _all_ mooks or what?
https://www.wsj.com/video/series/joanna-stern-personal-techn...
For starters they should exclude photos made on the phone's own camera. Because it's literally impossible for a just-taken photo to appear in this database since that only contains already known content found in the wild. And most people's photos would be original content. So it would alleviate a lot of concern while not harming Apple's goals.
If those goals are indeed what they say they are, of course.
Also, this is not a viable distribution method anyway. Every photo introduces more noise. Like dubbing tapes back in the day but worse.
That isn’t the same as saying it will match things that look kind of the same.
And it's incorrect. The way perceptual hashing works is that an image is shrunk down to a 8x8 or 26x26 etc image and then transformations are applied to them to exaggerate features.
If two images look kind of the same when shrunken down, they will have the same or similar hashes. If two images kind of look the same when shrunken down, then their parent images will also kind of look the same.
Please read the OPs of the two links I posted. They're both from people who work in this field. The latter link is from someone[2] who invented many perceptual hashing methods himself that are used widely across the industry. Both articles touch on this subject, and the first[1] one includes two photo examples. I have built products using these methods, and what is said by these two experts matches my experiences.
[1] https://rentafounder.com/the-problem-with-perceptual-hashes/
[2] https://www.hackerfactor.com/blog/index.php?/archives/929-On...
Nobody is saying false positives are impossible.
Apple is saying false positives are on the order of one in a trillion per user account per year. That doesn’t sound like something that matches images that are only ‘kind of similar’. Yes - cryptographic hashes have much lower false positives rates even than that, but that is a distinction without a difference since both make the risk negligible.
> The way perceptual hashing works is that an image is shrunk down to a 8x8 or 26x26 image and then
Which is it for Apple’s hashes?
There is no point in reading old articles about perceptual hashes if the conclusions don’t apply to Apple’s neuralhash algorithm. If they don’t then reading about other hashes is just a distraction.
What can you tell us about the likelyhood of Apple’s hashes to create false positives?
I'm not really concerned with what you're afraid most people will think. Two images that kind of look like one another will have the same or similar hashes. There are literal examples of this in the links I posted above. And it's literally the point of perceptual hashing, to find images that look similar to a source image by comparing hash similarity.
> If you are going to imply false positives are common, then you need to back it up.
I just did with two links I posted above. Twice.
> Apple is saying false positives are on the order of one in a trillion per user account per year.
Sounds like a claim that wasn't replicated or independently verified. Of course Apple is going to say their system is nearly perfect, that's what all companies do. The onus is on Apple to prove that their marketing claims reflect reality.
> There is no point in reading old articles about perceptual hashes if the conclusions don’t apply to Apple’s neuralhash algorithm. If they don’t then reading about other hashes is just a distraction.
The onus is on Apple to demonstrate that their methods are remarkably different from the rest of the science and industry.
This is like saying the normal principles of computing don't apply to new Apple products because they might have invented a new brand computing paradigm that isn't anything like any classical or quantum computer mentioned in scientific literature at all. Yeah, maybe they did, but it's unlikely and the onus is on Apple to prove it.
What matters is not what I think, but whether you care about making misleading comments.
> > If you are going to imply false positives are common, then you need to back it up.
> I just did with two links I posted above. Twice.
No, you posted some links that are not about Apple’s system, and you can’t explain how they apply presumably because you don’t understand what Apple is doing.
> Sounds like a claim that wasn't replicated or independently verified. Of course Apple is going to say their system is nearly perfect, that's what all companies do. The onus is on Apple to prove that their marketing claims reflect reality.
So this tells us you don’t know what algorithm Apple is using…
…And are accusing Apple of lying, when it is clear that you haven’t read about how they avoid false positives.
I think the onus is on you to prove your accusation.
> This is like saying the normal principles of computing don't apply to new Apple products because they might have invented a new brand computing paradigm that isn't anything like any classical or quantum computer
That just silly. It’s doesn’t take breaking the laws of quantum or classical computing to build a system with a low false positive rate.
One obvious way would be to leverage multiple images rather than just one. Increasing the sample size of a population sample generally reduces the false positive rate.
Have you considered that someone could build a system this way?
But, setting that aside, can you explain how a first order evasion attack can be used against Apple’s mechanism?
They are a real kind of attack in the lab, but it’s not obvious how they could be used to exploit Apple’s CSAM detection.
If you have reason to think they are a real threat, I’m sure you can explain.
At face value, they indeed don’t. But the premise from Apple’s white paper that their mechanism can’t be tricked is not realistic.
> If you have reason to think they are a real threat, I’m sure you can explain.
Yes, I can.
So this wasn’t relevant.
> But the premise from Apple’s white paper that their mechanism can’t be tricked is not realistic.
They don’t say it can’t be tricked. You are misrepresenting them.
>> If you have reason to think they are a real threat, I’m sure you can explain.
> Yes, I can.
No you can’t, because they are not a real threat.
Well it appears the CSAM scanning algo doesn't have Dost[0] scanning built in so, many people will evade this 'utility' made by Apple
Otherwise, like in this case, it just becomes an article basically stating "company trying to gaslight people after they got caught doing something bad".
for example:
Are a black parent's photos of their own children more likely to be falsely marked as CSAM than a white parent's photos of their own children?
>>
Apple: “we are going to make a tool that can scan your phone”
>>
Apple: “Sorry, the government is forcing is to use this tool to scan your phone”
Kind of a funny twist. But what about the core of the issue that you created a new affordance for spies and malware, legitimate or illegitimate or government-backed. Why not implement the whole thing in your (few) own cloud servers instead of billions of phones all over the planet
Oh wait, Apple software don't have bugs though, right? /s
This must be implemented in software. A bug in this software that causes the "only on files that are uploaded" return the wrong boolean value means it will scan local photos.
I never said random files on the filesystem. That's your strawman. In any case, it could well happen as well. Something is deciding what to scan, and that's software.
> end users have no way to confirm whether this claim is actually true
This is true of all proprietary software.
Once they have the capability rolled out, it's just a one-line config change to enable it.
I do not see how this wouldn’t be easily extended to all mountpoints on the device. And again, one needs to have faith and assume the /iCloud binary information on storage is really physically isolated from everything else. Sorry, it is very unlikely they aren’t really scanning, as I said, everything.
Edit: clarity.
I don't like it. I wish it had never happened. Fuck the government. But you are wrong and blowing this out of proportion.
Every tech company that has achieved global scale has gone down this path, and I really don't see what makes Apple so unique that it would be able to resist that temptation.
Show me a company with global clout, out of its hypergrowth phase that has behaved in an ethical way, and that has always stood by their customers and end-users in a responsible and ethical manner. Just one.
I'll list some of the ones that are on the other side, without going into specifics they're well known:
Microsoft
Apple
Oracle
IBM
Monsanto
Pfizer
Sony
Coca Cola
And many others, and more generally
Any tobacco company
Any large insurance company
Almost all of the major banks
Media empires
I prefer the companies I do business with to play on a human scale, something that I can relate to and where there are alternatives. As soon as that's gone it is a matter of time.
I will drop Apple if they proceed, and spread the word as much as I can.
You have to have a database containing cryptographic hashes of offending material and keep that continually updated. That's fairly easy to do cheaply.
Then you have to take a cryptographic hash f every single image being potentially loaded to iCloud and compare. I'm not sure what hashing they are doing but I wonder what the cost per 100,000 hash compare is.
Then if a match is found the flag must be added to a database of...and I'm not sure about this...imei numbers? Or maybe just indexed by AppleID?
If thirty flags are raised against an index j that database then that necessitates that a human must manually verify that it is, indeed, illegal pornography.
That's going to get expensive.
This whole debacle comes down to a problem of trust, where people aren't believing Apple to keep to their word that this isn't a slippery slope to privacy problems in the future. And well, Apple fans tend to be pretty diehard in favor for their company, yet they're (Obviously not all of them) not trusting them on this end.
Whether Apple is right or the critics are right, it's good to see even the diehard Apple fans question the company, though there's A LOT of disinformation going around about this.
They can stand back and say “we just can’t do anything, the users won’t have it” while Facebook keeps drowning in political pressure while doing a thousand times better than Apple.
Apple shouldn't do anything, because their duty is to report the CSAM that is visible to them and no private data should be visible to them.
Facebook has those images in the clear. They aren't doing "a thousand times better", they have an infinite amount more unencrypted images.
This is a reductive apples to oranges comparison that misleads anyone who reads it.
Would you similarly argue that the postal service needs to open every letter and inspect it to ensure there aren't photos of child porn contained within? Should uber drivers be required to search every passenger and their bags for child porn?
NO! Because that's private, and we respect privacy in this country.
Come on and be honest, do you really thing anyone, no matter if they where a cs PhD or a primary school dropout would share the opinion that an app that knowingly provided communications services to al qaeda during planning and execution of 9/11 could excuse themselves with “well the traffic was encrypted so w didn’t do anything wrong”
You cannot completely ignore the fact that bot ethics and politics are human construct and while we have “free speech” that does not in any way mean that you are free to speak anything free from consequences or free from judgement.
The reason that isn’t done has nothing to do with law or ethics, it’s purely not done because it is impractical. In countries where it’s illegal to mail currency, all letters carrying currency are intercepted, because it’s easy to do.
I for one would not for one second mind if they had a machine that made a sound if a letter carrying cp went through was used to intercept cp. Would you really be against this?
Everyone's heard about SWATting... get ready for CSAMming. I don't even know where to begin with services like Pegasus that rootkit a phone floating around. Got a major business deal a rival is about to close... CSAM their negotiators and win the contract.
I'm sure there are variations that wipe themselves without a trace after delivering their payloads.
They don't. They expect you to just trust them.
Edit: toward the end of the interview, Craig says the database can be audited. Obviously not the actual images, but people can verify that the list is the same across all countries, for example.
>"If you are a customer that is using iCloud photo library, which you don't have to, but if you're using iCloud photos to store your photos in the cloud, then what's happening is a uhm, a multipart, uhm, algorithm where there's a degree of analysis done on your device as it uploads a photo to the cloud so that the cloud can do the other half of the algorithm"
Oh so Apple isn't scanning customer devices they're just going to do "a degree of analysis" on them. Got it. What an asshole.
This still isn't great from the perspective that scanning's happening, but it seems better than all your images being scanned server side (which all the other big cloud storage providers do), or all images being scanned on your device.
* Start scanning all images uploaded to iCloud
* Start scanning on-device, but only photos that are to be uploaded, and only alerting after some threshold is reached
No matter what HN says, for them, not scanning is not an option. If you look at it this way, maybe the latter option looks better than the former?
What happens, DoJ fines Apple? Police walk into Apple HQ and arrest Tim Cook?
Not speaking rhetorically here to be clear, I'm actually curious why it's not an option.
Apple has been criticized for lax enforcement of anti-CSAM policies. Facebook and Google reported more CSAM images than Apple does, because Apple didn't previously scan iCloud images.
Is criticism they didn't have as many CSAM matches per year why?
Because I don't believe it was nearly as strong as the criticism this garnered. In fact at this point it's fairly clear they could roll back the change and get more kudos than criticism
Let's pretend you didn't just do that... you're talking about PR risk, but here we're seeing that risk blow up into a full blown scandal on the other side.
Apple already was the most friendly hoster to CSAM, and the bad PR from it was quantifiable and minimal compared to the current PR they're getting.
This isn't a new thing, encryption helps bad guys too. The same reason Apple was the most "friendly to CSAM" is the same reason any E2EE platform would be.
Fair enough, I could have phrased this more carefully. But my point stands, so I'll rephrase it.
To put it more carefully, people here are saying they would prefer the tradeoff of hosting CSAM compared against the tradeoff of the privacy implications of scanning users photos when they're uploaded to iCloud. I personally would not make that tradeoff, as I do not want to host a website that distributes CSAM.
> The same reason Apple was the most "friendly to CSAM" is the same reason any E2EE platform would be.
iCloud photos are not E2E encrypted [1]. iCloud photos allows you to share photo albums with others and publicly. This is the reason I feel strongly about this, because if you don't scan for CSAM, iCloud will be used to distributed it.
I'm saying that the same arguments you're making against their at rest encryption scheme apply to all E2EE communication
The idea being we already went through the "think of the kids" moment for that and now iMessage for example doesn't come up as being a defender of illegal content (at least not as often)
-
And your refined point isn't much better.
You're painting people who are against on device scanning as being pro-hosting kiddie porn, and that's a terrible base for an argument.
It's like saying people who are against banning matches are pro-forest fires.
It doesn't pass a sniff test.
I do no such thing in my revised statement. I'm saying there's a tradeoff, and I come down differently than most on the tradeoff. Obviously no one wants CSAM on their servers, but one of the options will result in more Apple hosting more CSAM. I'm sorry if you still don't like my phrasing, but it's objectively true. Fundamentally, I feel like a lot of people on HN refuse to justify this tradeoff.
When you say that it means people who would do want to host a website that distributes CSAM.
The reality is people are ok with unknowingly being one of many many places that all types of "horrible things" can be hosted because when we look back at history "horrible things" has a very flexible meaning.
We can all agree CSAM is horrible, but what you're failing to understand is that nothing prevents politically unfavored statements from being "horrible things".
And this is a one-way ticket here, once Apple rolls this out there's no longer a way to confirm they're not signaling to authorities over other types of content.
This is especially problematic in countries where Apple has already been known to cooperate with oppressive regimes...
-
Like I said, even painting allowing matches and allowing forest fires as a "trade-off" is disingenuous.
There are a lot of things that allow more CSAM, again like any form of E2EE messaging.
In the end your argument boils down to "it reduces CSAM", but that logic is just making a paperclip machine. Cameras are used for CSAM, so less cameras is less CSAM. Photos might be physically distributed so now lets have printers scan for specific hashes too. Encryption prevents ISPs from detecting CSAM, let's block https...
By itself it's just not a useful argument.
What way are you confirming that at present?
Maybe because they're announcing they're doing it now?
Or is this supposed to be a gotcha about OSS... because unless everyone builds every binary and every compiler from scratch and starts running xray machines on SoCs "knowing what runs on your device" isn't enough, you need to trust some root authority somewhere...
Facebook was something but what Apple doing right now is disgraceful. They destroyed the trust they built in years. At least tech people will remember this and make the right choice. I don't trust Android, now no more apple. We will all be forced to use Nokia 3310's again or use these niche crowdsourced Linux phones which suck majority of the time.
My only question is what is the timeline? Do I need to get new devices by next week, or is there time before this is put in place? And in the interim what can I do within the Apple ecosystem to minimize my privacy attack surface area from CSAM?
>"The threshold is set to provide an extremely high level of accuracy and ensures less than a one in one trillion chance per year of incorrectly flagging a given account."
Does that mean they expect about one in every 2.5 people per year to have at least a single false positive image match? (2.512^30 = a trillion)
"How fast were you going?"
"30."
"30 what?"
"...Speed."
Let X be the number of matches in your iCloud library. Assuming each photo's probability of a match is independent of other photos in the library (shaky assumption), then X ~ Binomial(n, p), where n is the number of photos in the library, and p is the probability of match.
The free plan, which gives 5GB, will store up to 2500 photos taken on a 5 megapixel camera. Assuming that's the most common library size, n = 2500.
So we need to solve for p given P(X ≥ 30) = 1/trillion and X ~ Binomial(2500, p). Notice P(X ≥ 30) = 1 - P(X ≤ 29), and we can use the CDF formula to get 1 - P(X ≤ 29) = 1 - sum_{k=0}^{29} of (2500 choose k) (1 - p)^k p^(2500 - k).
Set that equal to 1/trillion and solve for p. I don't have an easy way to compute that, unfortunately.
[0]: https://en.wikipedia.org/wiki/Binomial_distribution#Cumulati...
Just scan the images on iCloud ... I mean the CCP can scan the iCloud files why cant Apple?
Manufactured obsolescence, walled gardens in the name of "user security", on and on.
No, no. You are not holding the phone right. It's not me, it's you. Apple is the epitome of gaslighting.
It doesn’t add up.
Customers get angry.
Company: "I'm sorry you misunderstood me!"
They also said that because it's on device, security researchers will be able to check any change to the program. (probably via the Apple Security Research Device Program ?)
[1] https://developer.apple.com/programs/security-research-devic...
With new bits of infos too.
Also now that this is a thing how effective will it be at all? Or these sick people that dumb? After all this news? I do hope they are that dumb but who knows.
See, it's comments like this that clearly illustrate that there is confusion, and many people are still outraged over things they don't understand. A neural net is not scanning your phone for CP. You are conflating two things. Just watch the video that you're commenting on before commenting on it.
As for exactly how they'll do the auditing, I'm confused as well.
In other words, they still know better.
You’re 𝚑̶𝚘̶𝚕̶𝚍̶𝚒̶𝚗̶𝚐̶ understanding it wrong!
https://www.independent.co.uk/life-style/gadgets-and-tech/ne...
So now they altered the agreement. And they can because you bought a device that you agreed you wouldn't own.
Apple is using CSAM as the straw man here but in the broader sense they can trace any photo found on the internet back to the originator if that person captured it on their iPhone and stored it in iCloud. CSAM is just the excuse to develop the technology and put it on all their phones. Next they'll use it to track down protestors and other dissenters.
Privacy has been presented as a top-line feature by Apple for many years now. By announcing this feature they have betrayed any trust they may have built. The CEO remaining silent is the icing on the cake.
What value can Apple offer now? The Privacy story is done. Do they have anything else?
“Misunderstanding”
“The screeching voices of the minority”
rained -> reigned
It’s also a bit of a laugh to suggest Jobs didn’t suffer from hubris. See: arguably the Mac itself, $10,000 NeXT Cube, the Power Mac cube…really anything with cubes.
https://www.wsj.com/video/series/joanna-stern-personal-techn...