Apple’s Mistake
stratechery.com
stratechery.com
From the article:
Apple’s choices in this case, though, go in the opposite direction: instead of adding CSAM-scanning to iCloud Photos in the cloud that they own-and-operate, Apple is compromising the phone that you and I own-and-operate, without any of us having a say in the matter. Yes, you can turn off iCloud Photos to disable Apple’s scanning, but that is a policy decision; the capability to reach into a user’s phone now exists, and there is nothing an iPhone user can do to get rid of it.
A far better solution to the “Flickr problem” I started with is to recognize that the proper point of comparison is not the iPhone and Facebook, but rather Facebook and iCloud. One’s device ought be one’s property, with all of the expectations of ownership and privacy that entails; cloud services, meanwhile, are the property of their owners as well, with all of the expectations of societal responsibility and law-abiding which that entails. It’s truly disappointing that Apple got so hung up on its particular vision of privacy that it ended up betraying the fulcrum of user control: being able to trust that your device is truly yours.
Well said.
It's very easy to understand that content on a physical disk I own will not be scanned by some government-friendly surveillance program, and content I upload to somebody else's servers will. I am comfortable with [subsets of] my files existing on both sides of this distinction, but only because I understand it.
Now Apple is letting the surveillance apparatus reach into my own physical hardware, blurring the lines with some overwrought, proprietary crypto-gobbledygook solution. Given the details they've provided, I get why they might have thought this was in line with their general privacy ethos of keeping everything on-device, but the fact is that it is too complicated to build a reliable understanding of in my mental model of my own data privacy--and that's for somebody (me) who is quite technical and has even dabbled in crypto.
Now that I no longer understand Apple's practices as a steward of my private data, I can no longer trust them with it. It's a real shame.
When it stops being about the children, it'll be about the terrorists. When it stops being about the terrorists, it'll be about whatever the next excuse is. This seems like a compromise between those unfortunate scenarios - upholding privacy (we don't see the content) while still attempting to solve for the problems that privacy inherently allows for (we can still identify CP).
Getting CSAM pictures in the iCloud photo library of someone you don't like does not even require physical access sometimes: For example, WhatsApp has a "Save to camera roll" option that's enabled by default: just send a bunch of bad pictures via WhatsApp, those will get synced to iCloud after a little while, and now that person is in big trouble.
According to Apple, the only content being scanned are the images you are storing in iCloud. So how is this breaking your rule?
In a way, nothing changed. Apple could've done on-device scanning forever. In another way, everything changed.
I'm not sure what I'm going to do about it. Problem is that the devices are really really good.
EDIT: And they have CCTV and undercover security officers everywhere. More than you can possibly imagine.
But computers are another story. So far we still maintain some privacy there thanks to Linux.
Would it be any different if Dropbox was scanning files before it uploaded them rather than afterwards? It already computes hashes of your files on your device so that it doesn't have to re-upload existing content, so I honestly don't understand the objection to doing image scanning on your device before uploading your content.
But this is not the time to throw our hands up and say there was nothing we could do all along. That the situation was never ideal does not preclude there being something worth fighting for.
The devices are good, but still Apple has a weak value proposition.
Also, handing them your money probably isn't going to make things better in terms of alternatives.
their devices are really good, but they come with so much crippling for the sake of the walled garden that their goodness is wasted in some fronts.
There's a large list of things they are (or can be with minimal effort) perfectly capable for, but are forbidden for reasons, like reverse wireless charge of other iphones / airpods.
In my book that's a step in the direction of privacy, compared to old status quo.
They did this because they want YOUR phone to be a liability not THEIR servers.
by "US government position" I'm including negotiations and proposed/threatened legislation, not just the current laws on the books.
they do it with 'privacy' in mind and what they come up with is usually better than the worst case but it can still be pretty iffy
eg. a few months back when their notarization/entitlement verification system was being discussed
it's all just their vision of computing (which has some merits), hyper-controlled, locked down and "safe" it's not going to change if you're not comfortable with it you really shouldn't be using their products
As the article points out, the capability to scan things on devices is already deployed on huge swathes of devices in the from of virus scanners and content search indexing. All the photos on iPhones and many Android devices get scanned for text recognition now anyway. I'm sorry, but 'OMG scanning things" is a genie that got out of the bottle decades ago. So no, scanning content is not a dangerous new capability, it's an extensions of existing capabilities to a new use case.
Now arguably this is a further slip down that slope of on-device scanning, but it is not some new watershed moment where scanning is happening for the first time. Also we can see that the trend is not to repurpose existing scanning systems for new purposes. Virus scanning, search indexing, text recognition and now this are all separate mechanisms implemented and operating independently in task specific ways. The prediction that these mechanisms will be subverted for other purposes has not panned out.
The real issue is what is being scanned and why, and the implementation of this system. That's the topic we need to focus on.
that's a ridiculous counterpoint. 1) no iphone has AV 2) AV and content indexing don't secretly contact the feds.
adding a new "background service that sometimes contacts the US government" is a big privacy regression.
This is already a shifted/normalized perspective.
If I rent an apartment or a house, it comes with an expectation of privacy. I don't know where to draw the line exactly, but a dumb storage-only cloud service should carry the same expectations. Things get more complex if a service is actually doing something with the data.
But the broader point is that with cloud services, we've already given up some established legal expectations of privacy without much awareness or realization, let alone resistance.
Disappointing, sure, but not surprising.
Since the introduction of the iPhone, Apple has made it clear that one of their core beliefs is that they alone know better than their users. They have repeatedly demonstrated that they are happy to remove user control wherever possible. Their decision here fits that pattern perfectly.
I'm also curious and I don't mean this in a judging way, but are all the people who are planning to sell their iDevices also going to divest all their retirement holdings from Apple? Stock price is the thing that actually applies pressure. Everyone's comp is based on it. What would happen if enough people divested from these rent-seeking companies? We can't expect the government (though they should) or companies to behave in a way that is pro-consumer, but if we can align incentives we have a chance.
To me, it is such a weird implementation. The feature is about scanning on-device data, not data in iCloud, but for some reason disabling iCloud is the way to opt out? Did I miss any technical details?
Apple is always pretty OK with handing over iCloud data to authorities. Not only that they have already stored Chinese users iCloud data in state controlled entity, this was their whole argument regarding the San Bernardino case -- on-device data is user's sacred privacy, but if the shooter's phone uploaded data to iCloud, they were willing to send a copy to FBI in a heartbeat.
This makes me wonder, is iCloud the underlying technical boundary for privacy?
But Apple doesn’t want to know what they host for you on iCloud.
So they need to be sure that before something is uploaded to iCloud, it’s not CSAM.
So if you don’t intend to upload to iCloud (i.e. you disable iCloud on your phone), then Apple doesn’t care anymore if you have CSAM or not.
It is not a guarantee, but it leaves that door open, which currently would face very strong headwinds to get rolled out.
Is this true? I've always assumed apple can always do that (if they somehow needed to).
Yeah, this is essentially impossible today. Even personal computers have processors with potentially malicious firmware. In order to get a trustworthy processor to run free software on, we need to limit ourselves to hardware from decades ago.
It's gotten to the point I don't we can trust any computer we haven't fabricated ourselves. Anyone can download a compiler and make their own software but hardware requires billions of dollars worth of equipment and personnel. Software freedom is doomed unless we somehow develop a way to fabricate chips at home.
This way Apple can scan for CSAM without having to actually violate privacy by scanning a user's images themselves.
I am a user uploading a photo to the cloud.
Google's servers scan photos after they are uploaded to the cloud, and have done so for the past decade.
>a man [was] arrested on child pornography charges, after Google tipped off authorities about illegal images found in the Houston suspect's Gmail account
https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le...
Apple wants the device to scan the photo before it is uploaded to the cloud.
Both are equally invasive and both can only be avoided by refusing to use the cloud.
To the user - using your line of reasoning - they're both the same, right?
This isn't supposed to be a technical comparison, I just want to point out that the implementation matters as much (or more) than the customers' experience.
Technically Apple can do a scan regardless of wether or not you posted it to iCloud if they so desired to, or were ordered to.
There is also the risk of algorithm mistakes. Without the trust + safety team or any cloud verification, does this just get immediately forward to the FBI?
What is the process to be made non-guilty if the Apple algorithm thinks you have something illegal and you don't if there is no human in the loop involved before it's forwarded to authorities?
At least if you upload to a service someone on the trust + safety team can verify the algo worked properly. I would be very nervous building this kind of system of the risk of false positives! Maybe even so nervous to introduce many more false negatives - which is in itself terrible in the case of something like CSAM.
It is at still nice to see people are finally asking these sort of questions.
The majority of Apple hardware purchasers are probably not engaged in illegal activity. Yet all Apple purchasers are being forced to pay the price for Apple's mistake. Apple's hands are not clean.
No Apple purchaser ever intentionally requested that Apple copy their files to servers in Apple datacenters, or that Apple perform encryption intsead of the user, store and forward the user's personal messages through servers in Apple datacenters. Those were business decisions that Apple made to benefit Apple.
Technology allows, all computer users are quite capable of storing their files on their own portable media, and many are capable of encrypting and then sending messages directly to others, peer-to-peer, without leaving a copy with a third party. Technology allows, Apple has choices. No purchaser ever demanded that device storage capacity be reduced or ports for removable media be removed. Nor did any purchaser ever demand that her message be stored with Apple. Those are choices made by Apple, not users. Users should have choices too. Instead they are corralled into a user-hostile vision of personal computing that props up the world's wealthiest companies who horde money offshore while their domicile country falls into chaotic decline.
As the parent comment states, "One's device ought to be one's property..." Apple has actively and aggressively sought to overcome this basic tenet of common sense. As people online argue about "misinformation" please stop and consider the "lies" that Apple and the companies they compete with are telling the public about what exactly it is they do. Apple has taken Jobs' idea of the "Reality Distortion Field" too far.
Computer users need quality hardware. Apple can design and assemble it and be the best at what they do. Users do not need to be manipulated by a hardware vendor after purchase. Where did that idea come from. The sad fact is the "new" Apple is not competing with other hardware vendors. No other vendor even comes close. Apple has set its sights on competing with "tech" (middleman) companies that do nefarious things and cover up the truth. Apple wants user data after purchase. Why. To compete with "tech" (middleman) companies. Bad decision. Users pay the price.
The easiest way Apple could have fixed this problem with storing illegal files would be to stop storing user files and messages! Be a hardware company. Let purchasers use portable storage or arrange their own "cloud storage" with a more "competent" provider (let another company deal with the inherent flaws of the "cloud" concept^1). Let users discover how to encrypt their own data and use peer-to-peer transmission. Let them use the Apple computer to write programs without needing "permission" from Apple. Apple has only tried to hide these basic capabilities and competencies from the majority of new computer users, strengthening the "Reality Distortion Field".
A long line of bad decisions from a hardware company we once admired. They just keep getting worse. And all for what. Greed. Immense untaxed cash reserves held offshore. Why not be a hardware company and stop trying to compete with "tech" (middleman) company surveillance. Why keep dumbing down and locking down what a former CEO once called a "bicycle for the mind".
1. "Cloud" is a synonym for "someone else's computer." "Cloud storage" is another way of saying "storing files on someone else's computer". Perhaps "Apple" is also a synonym for "someone elses's computer". If Aple can exercise control over a computer after they sell it, how it is not still theirs. Purchasers are just "users", not owners.
Fixed it for you. I hope you are not implying that Google, whose bread and butter has been selling personal information, has resisted the temptation for all these years?
I use Apple and it's an informed choice. I am pretty realistic about their practices. And realistically we don't have a choice. Either you use a smartphone and it's a Swiss cheese of security vulnerabilities -- worst of which are embedded in the device on purpose! -- or you don't use a smartphone at all. This is the era we're living in.
I hope that custom ROM Android communities can win and give us back some freedom but I am skeptical. Still, I am open to try some of the de-bloated Xiaomi ROMs at one point. Or maybe put LineageOS / PostmarketOS on them.
Speculating it is slippery slope arg. https://wikipedia.org/wiki/Slippery_slope
"The strength of such an argument depends on whether the small step really is likely to lead to the effect."
> It is true that a computer, for example, can be used for good or evil. It is true that a helicopter can be used as a gunship and it can also be used to rescue people from a mountain pass. And if the question arises of how a specific device is going to be used, in what I call an abstract ideal society, then one might very well say one cannot know. But we live in a concrete society, [and] with concrete social and historical circumstances and political realities in this society, it is perfectly obvious that when something like a computer is invented, then it is going to be adopted will be for military purposes. It follows from the concrete realities in which we live, it does not follow from pure logic. But we're not living in an abstract society, we're living in the society in which we in fact live. If you look at the enormous fruits of human genius that mankind has developed in the last 50 years, atomic energy and rocketry and flying to the moon and coherent light, and it goes on and on and on -- and then it turns out that every one of these triumphs is used primarily in military terms. So it is not reasonable for a scientist or technologist to insist that he or she does not know -- or cannot know -- how it is going to be used.
-- Joseph Weizenbaum, http://tech.mit.edu/V105/N16/weisen.16n.html
To pretend it's inherently unknowable right up to the point until it's "too late" is kinda getting old.
> Argument from fallacy is the formal fallacy of analyzing an argument and inferring that, since it contains a fallacy, its conclusion must be false.
What if the Chinese government could search every device for images of yellow umbrellas, as part of the HK revolution?
I just don’t see the intrusion.
Firstly, you are syncing your photos to iCloud. It’s equivalent to developing your photos at an old timey film shop.
iCloud happens to have lots of security guarantees, much like the film shop.
But the moment you develop CP, all bets are off. I think most of us lead pampered, selfish lives. Perhaps this is a “think of the children” cry, and perhaps this has been used for many evils over time.
If Apple were to get their lists of hashes from China (a situation they’ve opened themselves up to in the future), then sure, that’s awful. Or if they were doing these checks and alerting law enforcement without iCloud sync, then that’s awful too.
But this doesn’t seem to be a slippery slope, yet. They are targeting a specific case: you uploading your photos, and your photos being of known child sexual abuse.
Yes, there are false positive photos in the database. Maybe there’s even a flower in there. But if you got flagged as a false positive, the moment law enforcement looks at your photos (which, remember, you’ve surrendered to iCloud), you sexually abusing a flower won’t cause any problems.
Or will it? Please, my mind is open. I’d love to change it.
I've been briefed about it. Apple says explicitly that this is not the case. They mention that cropping, transforming, or even desaturating an image won't result in a hash that's so different from the matching one. The similarity will be high enough to detect the photo anyway. They don't seem eager to explain in detail how that actually works, though.
so, Apple now has the ability to reach into every law abiding citizens personal phone - report them to the FBI when the algorithm makes mistakes (and it will - nothing is 100%, particular in image recognition), AND THEN THE PERPETRATORS CAN STILL GET AWAY BY JUST TURNING IT OFF??!??!?!
My goodness, I couldn't imagine a stupider way to fight CSAM.
> Facebook made 20.3 million reports of Child Sexual Abuse Material (CSAM) in 2020.
Were there 20.3 million arrests of offenders? I don't think so. I doubt they caught 20 perps from that data. While it may be a better algorithm, it should be a good warning of how often your' very legitimate and non-CSAM data will be leaked to a "verifier", and worse, to the authorities.
Take a look at how often YouTube channels are taken down by "auto triggers" that are then "reviewed" and you will get a good idea of how often the government is going to screw law abiding citizens. It's not zero, and imho, it will be well above a range of "acceptable fallout".
I don't know of any data from the US but in Switzerland the Federal Police pre sorts all reports from the NCEMC and around 90% are unusable [1] and cannot be acted upon. The remaining 10% are then forwarded to local authorities. There is no data I know of what the final conviction rate looks like. But from what I have gathered from local news paper these departments are usually short staffed.
I suspect that the situation is similar in the US.
[1]: https://fedpol.report/en/fedpol-in-figures/fight-against-pae...
Edit: Fix typo: persorts to pre sorts.
I know it would be rude to keep messaging them but in the end they represent Google and they are suppose to help you, at least morally.
As for the 20.3 million reports, Facebook probably decided it was better to bury the authorities in a massive pile of paperwork than to see a day in court. Plus they get to claim they are doing something.
Not trying to defend Apple here, but we should keep in mind that the goal for Apple is to prevent such photos from being uploaded to iCloud and detecting those that do. If someone turns off iCloud sync, then Apple would argue that it's not their concern to scan photos saved on devices (and thank god it doesn't otherwise the repercussions would have been much more significant).
Better would be educating people: don't use your real name or upload pictures of yourself to the internet, be very careful talking to strangers. Don't let younger children use social media services.
Now they literally need to check everyone’s phone for illegal content.
Man, have I become my parents or what.
Few people have access to the databases. I would bet quite a bit, this will be used for marking possible offenders of all sorts of crimes and "crimes", e.g. having ISIS propaganda material on your phone, or leaked data. Sometimes you may not even know files to be on your phone. Try exploring Telegram's nearby groups with image autodownload for groups activated and see what happens to your "share recent files" dialoge... It's all porn and nazi memes now!
Overall, I am happy Apple admitted this ultimately, because I was conflicted about buying into the ecosystem for the recent hardware appeal. Not conflicted anymore. Not at all. It's Linux/ASOP or ~~die~~ get stressed out. No M1 benchmark or fancy watch health features can make up for the chronic knot in my stomach using Apple's products; die Schere im Kopf . Hope I find a low latency pen input tablet runnig FOSS Android or Linux, too, as I feel uneasy about journaling or drawing on my iPad now. Thanks Apple, I truly hate you too <3
If this was as big an issue as people think, we’d have seen more of it.
That's the point. It satisfies the letter of the demands from the crazies, while not actually accomplishing anything, and creating discourse in the rest of the population to work towards realizing that we shouldn't be bending to the whims of the crazies at all.
The only winning move is not to play.
How useful is this feature? If the goal is to protect children, there must be good evidence that this does, in fact, protect children.
Who really made the mistake? The one reporting cases that are almost definitely true? Or the one reporting so many false cases that authorities are buried in false reports?
Also you could trivially have 1 bad guy with thousands of photos being accounted as 5,000 reports instead of 1 report listing 5000 images.
Where are the hundreds of prosecutions resulting from apple's reports? I can't find them: https://courtlistener.com/
Or just apply image filters to the data, that will kill any neural network detection (since there is an infinite number of filters, specially the deep fake ones).
I have no doubt they will. Pandora’s box has been opened, and okay, today the scanning is only for images uploaded to iCloud, and only for CSAM. But that is not auditable and subject to change at any time without notice.
And today it is only looking at known hashes but the devices already have a capability to analyse images and even run OCR on them, this is not a matter of of these things will be weaponised but when.
Can’t believe this is the same Apple that publicly defended the right to privacy for known terrorists by using the slippery slope argument themselves. The mind boggles.
[0] https://www.zdnet.com/article/canberra-asks-big-tech-to-intr...
This, to me, would be exponentially less privacy invasive as I’ve come to assume all major cloud hosting providers implement something like this (look at Google Drive), but Apple has said that the scanning is done on-device, meaning whether or not you upload your photo library to iCloud, your local photos will be scanned with an on-device database of hashes.
Essentially iOS photos now implement a direct API call to the feds with some vague “human verification” layer if you go above an unknown threshold
This is an erosion of fundamental human rights under the guise of “think of the children“ so that anyone who stands up against this tyranny can be labelled a “pedophile”. 1984 wasn’t like 1984 but 2021 is surely looking that way.
They have, for the past decade.
>The system that scans cloud drives for illegal images was created by Microsoft and Dartmouth College and donated to NCMEC. The organization creates signatures of the worst known images of child pornography, approximately 16,000 files at present. These file signatures are given to service providers who then try to match them to user files in order to prevent further distribution of the images themselves, a Microsoft spokesperson told NBC News. (Microsoft implemented image-matching technology in its own services, such as Bing and SkyDrive.)
https://www.nbcnews.com/technolog/your-cloud-drive-really-pr...
Yes, but only for images being uploaded to iCloud.
> meaning whether or not you upload your photo library to iCloud, your local photos will be scanned
Not in the currently proposed implementation, if I understand correctly.
Disclaimer: All my information about this thing is from news articles; I might be misunderstanding the details.
There is so much rich information in your photos, from memorable places you’ve visited to handwritten family recipes. iOS 15 uses secure on-device intelligence to help you discover more in your photos, quickly find what you’re looking for, and relive special moments.
1) Scan devices for copyrighted music, to play your music you need to have it purchased on Apple 2) Scan devices for copyrighted videos, to play your video you need to have it purchased on Apple 3) Scan devices for copyrighted photos, purchase an Apple license to use this photo 4) Scan devices for copyrighted text, purchase an Apple license to use this text
Apple - Think Licensing
In all honesty I think everyone should treat the government as an adversarial force when it comes to respecting your privacy or rights. Much of bill of rights was introduced as a response to the concerns of the government's overreach. Built in protections to impede totalitarianism.
In that regard, you can't simply rely on the government to legislate towards your interests and assuming that they will is foolish. Creating tools to secure your privacy is essential, not arbitrary.
Apple made privacy their selling point (Privacy. That’s iPhone), and that set the expectations of privacy people have with their products.
“Tech bro movement” what?
In basically every election, almost half of the voters voted for the loser.
Government should be limited in power and scope, never allowed to expand beyond that power and scope, unfortunately since the 1930's we have allowed the US government to expand to a size that is untenable and is at odds with the concept of liberty itself
And you couldn't uninstall them, even though you own the home.
And you couldn't know when a human was reviewing 'suspicious behavior' telemetry, which included images, video and audio of you and your family.
And you couldn't control anything about when and where this happened, except by moving house.
And all of this was framed as a 'safety' problem, and if you complained about this, you were considered one of the 'screeching voices of the minority'.
specifically this box "And you couldn't know when a human was reviewing 'suspicious behavior' telemetry, which included images, video and audio of you and your family."
and this box: "And you couldn't uninstall them"
I can't help but feel that someone has wielded a 'big stick' to get Apple to do this. If not it's a huge misjudgment.
Maybe most of the public do not understand privacy in this absolute sense? Maybe even the public support scanning photoes to find child sex abusers?
It could be a fact that the outrage is only among a very small number of individuals. I won't be suprised if the number is small even among tech persons.
But then you know when you talk with a friend that is not technical and they just ask: what do you think about the new iPhone/new Apple launch?
Well so far the answer was: yeah, great you will have privacy.
But from this on the answer might be: oh, be carefull they are scanning your phone. If this goes into a kind of self-repeting meme, then the details will be lost, but the key phrase that Apple is looking into personal photos will remain.
This could, potentially, do a lot of brand damage on the long run.
Really?
You know some late night comedian is going to do some jokes about pedo's having to switch to android. You think that is damaging to apples' brand?
My guess is some android folks do a follow eventually (as usual).
I'm a parent. Even for those of us who are into privacy etc (yes, I did the early PGP key signing parties, EFF / ACLU stuff etc) I'm having a hard time seeing how this damages Apple's brand. I don't want this crap being sent to my children - PERIOD. If they are on a child account PLEASE screen it.
Folks - pay attention to the kind of laws that will get passed and do get passed. Most folks will throw away a lot of civil rights for these types of issues.
I found the arguments against this surprisingly uncompelling. I saw an HN article about how apple is committing felonies etc - it just didn't seem well founded. And everything is over hyperbolic over the top its insane.
[0] https://www.hackerfactor.com/blog/index.php?/archives/929-On...
1 is arguably a good feature (though it's very intrusive - it has a benefit).
2 is a monstrous invasion of privacy. It has no benefit to you, the user, only a massive potential threat.
Finally, the NCMEC is founded by someone who admitted, if he was judged by the own law he helped pass, he'd be considered a sex offender when he was dating his then-girlfriend.
To claim that you treat privacy as a fundamental human right is an extraordinary claim that requires significant effort and action to back up. To me, "fundamental human rights" apply to everyone - all humans - including children.
Their commitment to this was already called into question for several reasons, including their partial commitment to E2E as well as their actions in other countries (i.e. China).
When you give your users privacy with numerous conditions attached indicating all the times they don't have it, you aren't giving them privacy. Full stop.
It's like going to someone's home, having them tell you that they are champions for privacy and then going to the bathroom and seeing a damn camera attached to the wall. "Oh, that, don't worry. I only look at the footage if something gets stolen."
Apple's rejoinder is that they will simply refuse to do that. And that's great, until you consider that all the iPhones are made in China and China is more than willing to apply immense pressure including but not limited to shutting down Foxconn if they feel strongly about it.
We cannot protect our children by building a dystopia for them to grow up in, and normalizing this kind of invasive spyware on every device is pretty much guaranteeing that.
You can always not buy your child a device.
Parents will recoil in horror at the suggestion, being told what to do and limiting their child’s freedom ! Indeed, welcome to our world, where we suffer huge affronts on our freedom and privacy in the name of “the children”.
I think it just goes to show that people don't actually care about privacy and civil liberties. You can't argue against "think of the children" without being labeled heartless or a pedophile so no one with true influence will argue against it since nobody wants to die on the child porn hill. This is what happens when your thought leaders are all cynical and value money and power above all else.
While I'm a privacy advocate I could see that the arguments were fruitless. The popular conception of the constitution today is that it is a joke. People mock liberties like freedom of speech so you just know privacy is something people do not care about.
No just the start of a whole stream of attacks from WhatsApp / Facebook:
https://www.theverge.com/2021/8/6/22613365/apple-icloud-csam...
Also just read the full range of comments.
The fact that you're saying elsewhere that everyone is misunderstanding and it's all overblown is sort of making my point that it's affecting the brand.
Plus you're saying other firms capture images so that's fine - no! Those other firms haven't made privacy a central feature of their brand.
My guess is that Apple designed this privacy protecting system so that they could deliver the solution on their own terms - and perhaps leading the way on how this could be done before they are hit with a cookie-banner-popup-level solution from bureaucrats.
Regardless on your opinion on whether they should scan or not, both the EU and the UK now have a reference design that protects the privacy of people, and still manages to either identify the people that own that material, or make it more inconvenient to own that material.
As we like to say "Deplatforming works", and in this case, a good useful tool for people that own those pictures is no longer available, and they have to resort to jumping through hoops and relying on more inconvient solutions. The later might not solve the root problem, but introduces friction that gets in their way.
Is it, really? I think they feel this will blow over.
I don't think that the phrase "giving up some amount of privacy to stop child abusers" will sound unreasonable to enough other people that don't really understand privacy. The damage to Apple's brand seems to be coming mostly from technologists and the privacy-conscious. Even if only a handful of people are justifiably arrested because of this change, making the tradeoffs not worth it in their eyes, there are still many other people who would believe that catching even a few more child abusers was the right thing to do.
Reducing the reasoning to "but think of the children" ignores the fact that there are still reasons that CSAM is declared illegal. It seems that most people on these threads are focusing on the fact that this is a privacy disaster - which is absolutely is - but until there is a viable argument that on-device privacy is more important to the general public than shutting down a market that actively produces evidence of child abuse, I'm pessimistic that this will be walked back.
What is needed are studies correlating the spread of CSAM with actual CSA, but they do not exist. At this point it seems to be taken for a fact. The taboo around the subject appears to have disincentivized the creation of such studies.
What I can't think of is why scanning everyone's private messages just in case they might be those terrorists is a worthwhile trade-off in any kind of free society.
There are lots of ways to catch the bad guys, and we'll never catch them all no matter what we do. In countries with a concept of individual privacy before the state, there need to be limits on how much the state gets to snoop.
There are plenty of countries without any such privacy, so it's not like we have to guess what the far end of the slippery slope looks like.
Apple’s brand according to HN has already been damaged by too many trade offs: walling off the App Store, the MacOs system disk, availability of stand-alone OS updates and more.
Extreme temperatures and wildfires are to climate change as tangible customer data leaks are to Apple’s implementation of security and privacy around its ecosystem.
That is, CSAM scanning does nothing to prompt a 114 degree day in an iPhone user’s mind.
Until Americans are hauled into custody the way Belarusians are right now, this will not enter the customer perception of Apple’s brand.
And if that future lies ahead, it will be way too late to matter.
I'm now certainly not going to and am loudly telling all my friends why.
The Elephant in the room is "this will be on Mac OS". So the fact that the user cannot have root access to an Apple mobile device is well established by now. But to have my desktop/laptop computer do things on behalf of third party, whatever the "logic", is total madness. This is not some SaaS app. When I buy a car I expect that I am in control. When I buy a computer I care about my control over my property.
What is this madness? Is due process and innocent until proven guilty bed time stories now?
Everything stallman has said about proprietary software is slowly coming true, and I must admit that now I’m reevaluating a lot of the technology in my life.
The CSAM scanning is only getting added to iOS and iPadOS. If it does getting added to Mac at some point it will be part of Photos.app, not the general OS.
I think that a crucial missing piece is the FBI's argument: Apple is fully capable of developing and signing a "law enforcement iBoot". IIRC, the FBI was even willing to have someone else develop the software and only ask Apple to sign it–which they definitely have the capability to do, and only policy of not signing other people's software stood in the way.
If we agree that Apple was right in 2016, it stands to reason that Apple cannot be compelled to modify its CSAM filter to capture arbitrary contents, or report it at lower thresholds, or expand it beyond iCloud Photos (like to Messages itself). The amount of work they would have to do for it seems like it would be even higher. There are whole infrastructure pieces that just don't exist. What am I missing?
A change to the policy of what kinds of images are scanned is opaque by law, since none of the Apple employees involved can even have any access to the database of hashes they are using. There is also no realistic way for the consumer to understand the true false positive rate, no ability for a third party organization to distinguish false positives from true positives on non-csam images leaving the device.
Additionally, these are just problems in the US. Other governments can and will mandate the use of this tool for other kinds of media they find objectionable in their borders.
And there will be bugs that expose people to the results of these other governments, like the Taiwan flag emoji crashing case. https://www.wired.com/story/apple-china-censorship-bug-iphon...
The large investment from this system is almost certainly the infrastructure to get it on phones, report the results, and run it in scenarios where it will minimize battery impact. What photos on device it is run on does not strike me as a technical challenge once the tool is built, only one with policy implications. And the easy answer to that will be just to check some flag if the phone is in a country that requires all pictures to be scanned.
To that point - I generally think that engineers at ostensibly-privacy-minded companies like Apple are competent, well-intentioned, and good canaries. If I were to open Twitter and see a lot of people "seeking new opportunities" from Apple's security team and not able to give their reasons? It's very possible that a backdoor was built contrary to public statements, and they could not condone the discrepancy.
But here, not only is the list of hashes editable with merely a configuration change, but it is fundamentally a list of hashes that is designed to be secret and non-auditable and supplied by a non-auditable supply chain. In fact, the proponents of this program would argue "don't give the Apple engineers and product managers access to the hash list, nor access to whether test images are matched by the hash list, because it could be used for nefarious purposes if they themselves are perpetrators."
So at any time a photograph commonly used to criticize a regime or commemorate a specific event could be added to the list, and there would be literally no way a well-intentioned engineer even inside Apple could even know about it. This isn't just a technology that could be applied with technical effort to make a backdoor, it's a deployed backdoor that opens up all our devices to supply chain attacks, plain and simple. A state level actor would simply need to convince someone at NCMEC to insert something into the un-auditable hash list (whose source images are never to be looked at in totality by design), then compromise any person or computer in the law enforcement-side reporting pipeline to exfiltrate the identity of anyone with the images in question. That's absurdly dangerous.
I’m not knowledgeable enough to comment about other countries’ policies, but the fact remains that no one has a law enforcement iBoot, as far as we know; and if one exists, CSAM filtering was never the opportunity that repressive regimes were waiting for.
The government or whoever will regularly add hashes to the list. Those hashes can be for anything and it’s not like Apple has any way to verify what they are for. Apple doesn’t really have oversight on what they’re doing if I understand correctly, all the trust is in whoever creates the list of hashes
Additionally, by compelling the NCMEC, the government cannot increase the scope of searching or lower the match threshold.
ANY connected digital media can become a 1984-style spying device if you don't have full access to it or even if you have that but you are not an experienced electrical engineer.
The only defensible platforms are nondigital media and airgapped computing. I, for one, wouldn't shoot a nude with any digital device nowadays. Only exception would be a camera that gets connected only to airgapped computers.
But we are losing that too. In a short time we won't even be able to pay for that Polaroid or to buy a Librephone without being traced in some digital form.
We are losing all the battles but we need at least to prioritize. The cloud is lost, connected devices are lost. We need at least to keep cash payments, and to pressure the government to break up digital monopolies.
I don't want to sound like desperate luddite but I really think that warrants requirements are unenforceable in the digital world and we really really need to keep important parts of our lives in the analog one.
Sorry, but I'm not going back to pre-internet days to accomplish my life goals. You might as well tell people in 1900 that they could live just as well without electricity.
If you want to be a monk or Amish, sure. If you want to be a normal human in pursuit of self-actualization and helping the world, internet is a de-facto critical piece of your life.
My grandma never had a computer, or a smartphone, she doesn't even know how to say the word internet, and still she lives fine, and she knows more informed than I that use internet, reads newspapers, on paper, without stupid ads, listens to the radio, watches TV, calls her friends with the landline phone, and most importantly, a thing that nowadays is lost, she goes outside and talks to people.
It seems like we no longer can talk to people, we are constantly busy with our devices, back in the days if you were on a train what could you do, read a book, or talk with the people sitting next to you, nowadays you use your phone, listen to music with headphones, and isolate from others.
Internet was a great thing, but the internet of the beginning, of the '90 and first '00, where it was a cool thing, where you could have used it to learn new stuff, the internet of IRC chats, the internet of forums, communities.
The internet was good because it was something alternative to the real life, something you did in the evening when you didn't have other things to do, nowadays internet became a substitute of the real life, on the internet you share things you do, you talk with people that you know in person, and then when you meet what have to say since everyone knows everything others have done thanks to social media?
Also in the modern internet privacy is destroyed, back in the days one rule was never use your real name on the internet, and never share personal information. Now it's the opposite, Facebook requires you to use your real name, YouTube wants your ID to watch age restricted videos, they encourage you to share to them all your personal information, that is bad.
I wouldn't return to the days before internet, but I would return to the '90, where internet was something good, and just a cool tool you could have used, but not something fundamental for your life.
Donate to the EFF.
It's technically true what you write, that we can live life without the Internet. But we can live life in a million other sucky ways too, and that doesn't mean that we should. I find it arbitrary how people draw the line, and they are fun to argue, but no amount of good reasons make the line not arbitrary. Abuse has a long history and the older ways of living had their share too.
The bigger authoritarian countries (China, India) will demand, and receive, the ability to match against illegal images and memes, like Tank Man, Winnie the Pooh, illicit tweet screencaps, etc. No one with power in the West will really care.
The slippery slope is actually iMessage scanning, not iCloud Photos. Real time, ML based, it is fully capable of scanning for keywords, the same way as NSA analysts used X-KEYSCORE (in the ancient historical period, before they switched to ML). (Fortunately, NSA (or others) probably won't let it's ML classifiers be distributed, because they could be reverse engineered to see how much NSA knows.)
Wouldn't be surprised if this was agreed to behind closed doors in exchange for other assurances from the government. The government passing laws forcing Apple to do this would be very bad PR, better for Apple to appear to do it voluntarily
You make it sound like the government is a single person with a plan. I don’t think people and departments in a government are very coherent and aligned, it’s a huge number of people with a zoo of mandates and everyone is fighting to be seen as somebody who makes things happen
Care to cite a source for that claim?
Your content should be your content regardless of where it lives.
-Photos -Texts -Social Posts -Notes, Reminders
These are all what make up a person’s life. At one point in the not so distant future all of our data will live in the cloud, are we then property of the cloud providers?
I think ideally things would truly be E2E encrypted, in which case Apple or any other cloud provider doesn't have to trouble themselves about what's on their servers, since no one but the user could ever look at the contents of it. In this case a big blob of data up to the storage limits of the person's plan is the only thing they'd have to worry about.
That is also the other issue with Apple's proposed implementation. It completely circumvents E2EE and makes it entirely pointless. It doesn't preserve privacy in any respect whatsoever.
And to what end? You might capture a few people that are syncing their child porn on iCloud without encrypting it themselves first, but does this really make children any safer? It certainly endangers anyone using an iPhone given the technology doesn't discriminate against what kind of hashes it provides and increases the scope of surveillance on everyone. These sorts of things are extremely hard to undo.
You can still manage your own data so that it is opaque to the cloud providers - encrypt it and don't share the key with them. They'll have no insight into the blobs of data that you're uploading to them.
For comparison, it's pretty obvious that there are certain things in physical realm that citizens shouldn't have. So it would be wrong to claim "my things are mine and no one should be able to inspect them or take away from me, no matter what those things are". Examples for things citizens should not posses: weapons of mass destruction and slaves.
Apple et al argue that CSAM should be digital content that nobody ought to possess. So while you're obviously technically capable of storing it, it shouldn't be legal.
Their CSAM reporting looked awful. They needed a solution. They came up with something that's definitely more elegant and more privacy-oriented than scanning all iCloud Photos.
They went for the more complicated solution, which incidentally is one that less people will properly understand (or care about), therefore limiting the size of the inevitable dent on their privacy reputation.
I think they are also preventing the consequences of some upcoming legislation in the US, UK and other parts of the world to make CSAM scanning mandatory in the Cloud, which would carry the risk of weakening Apple's privacy infrastructure even more.
So, if you're given the choice to go for a haute cuisine gourmet sandwich with a side of shit salad, and a huge shit clam chowder, what would you rather go for? In Cupertino they picked the sandwich.
But they ll be scanning all icloud photos
Firstly, this allows the general public to get an idea of how much governments or police are interfering in people's lives but secondly, the enforcers have to convince someone (usually a judge) that their case has merit before they start defaming someone or taking away their liberty.
We shouldn't be as scared about lacking privacy as we should be about the fact that our privacy can be invaded with little oversight and sometimes without us even knowing about it!
Also, we need to speak up for those who are much more likely to be affected by privacy invasions (black, Muslim, other minorities etc.) even if we are white and don't think it will be a problem for us personally.
It also leads me to this question about CSAM. If it only recognizes existing photos, then it's of no use for photos that I take. So perhaps CSAM on a dedicated camera will never be a thing, and people will gravitate back to such cameras.
Also leads to this question. If Apple isn't scanning my photos ( for which there is no CSAM fingerprint), what photos are they scanning? Things that showed up in my browser?
Worse, it’s nearly impossible for a competitor to market against without risking the general population believing you tacitly approve CSAM. How does Google market Android without coming across like a safe haven for pedophilia?
Even before that phrase, we were already in era of "Don't talk to the cops without your lawyer, especially if you're innocent." Giving those same cops more information is not in your own best interest.
Same happened with Fosta, same with the war on drugs. The actual perpetrators don't get caught, the abuse continues, but the metrics make it look good.
These devices haven’t been “truly yours” for a decade. Hence, jail-breaking. Maybe this is just the first time you‘ve encountered friction against the walled garden and become widely aware of its existence, but that doesn’t mean it wasn’t there before.
A social networking site/platform is one where previously created media is shared. iMessage pictures are not uploaded to iCloud Photos by default (they are part of the usually E2E content with the standard online backup caveat), only photo albums or photo rolls are - and those are overwhelmingly first or second party just-created content.
This means that if comparing against known CSAM hashes, it’s extremely likely for Apple to find orders of magnitude less content in the first place. The only thing that their system can catch is first-party images that end up being distributed and registered with the various hash databases.
Regardless of whether we are talking CSAM or anything else, it is the norm for the amount of content “created” to be significantly (as in several orders of magnitude) less than the material consumed.
I don’t think scanning on the device vs scanning in the cloud is going to change any of that.
If so, any abusive pictures that are discovered during pedophile ring arrests could be traced back to the phone that took them.
They could have not told us this at all, and we wouldn't have found out until some hacker playing around with network interfaces noticed unusual traffic going to Apple's iCloud servers from the bird daemon.
What Apple is doing is distasteful to me, but I'm afraid the market is so poor in comparison, I don't know what else you could purchase besides Dell XPS systems running Ubuntu.
But there's no phone version of that, to my knowledge.
I'm curious if there any possibility of a small claims case forcing such a return.
a few years from now: "it's been such a success we're going to do it everywhere for everything now"
> both the UK and the EU are moving forward on bills that mandate online service companies proactively look for and report CSAM. Indeed, I wouldn’t be surprised if this were the most important factor behind Apple’s move: the company doesn’t want to give up on end-to-end encryption — and likely wants to expand it — which leaves on-device scanning as the only way to satisfy governments not (just) in China but also the West.
Is that not the simpler explanation for this?
If someone is suspected of a crime a warrant can be issued by a judge and the suspects phone compromised under lawful control of law enforcement.
What Apple is wanting to do is pre-crime where everyone is considered suspect until proven otherwise.
These slippery slope type arguments tend to assume that there is no middle ground between sides of this argument.
The subtle take I got is that devices should be completely private... given that is what you pay for. This drives Apple's phone system into that "unremovable bloatware" which I pretty much want to escape from on Android devices without necessarily setting up all the control myself. I wanted to trust Apple's hardware for privacy (and already understood the limits of their privacy in their cloud), but now I feel betrayed as this product can be eventually used for more sinister practices should a government pass laws necessitating it, as highlighted in many other comments.
We must go full FOSS from hardware to software to preserve humanity. Everyone needs to get on board, not just the savvy minority. That's the goal.
Sticking government spyware onto people's phones is a direct repudiation of this philosophy. I'm not surprised this would happen eventually, but I am surprised it has happened this quickly. It has been what, a mere 5 years since the San Bernardino affair? The authoritarian delusion is a strong siren call, but I had expected it to result in ever more App Store restrictions and perhaps a few breaks where the government compelled signing trojan horse targeted updates - not a full rejection of the very idea of user privacy by placing a government agent in every phone.
With this development, Free Software is once again the only option if you'd like to preserve your own digital rights. Its story for mobile historically kind of sucks (Android with varying amount of bad bits stripped out), but that is changing with things like PinePhone. IMO inexpensive devices are a necessity to get iterations into many people's hands, as opposed to the waterfall-feeling model of OpenMoko (etc).
I feel like we almost need a “Constitution for the Digital Age”, which would eg guarantee privacy rights.
If not, it will be an interesting test of how many users Apple will lose over this.
Those Apple devices i might upgrade will now be put on hold, it is simply unacceptable what Apple is doing.
Lose.. to whom? PinePhone? Not trying to be flippant, but... there is no longer any option for the common user.
No, my elderly mother is not going to download and flash a custom de-googled ROM for her OnePlus 9 Pro. Pure fantasy that Apple will lose users because of this. This is a nonstory to everyone who lives and works east of Tahoe.
The App could have done the scanning before sending and I would be free to not install it on my phone if I don't use iCloud photo sharing.
Instead, we now have an ever-present spyware engine embedded into the OS itself that can be abused by policy, as the author points out.
But I guess having a separate App would potentially cut into Apple's bottom line as it would have lessen the opportunities of pushing for iCloud subscriptions.
https://www.apple.com/child-safety/pdf/Expanded_Protections_...
Once it's embedded in the OS there is basically nothing stopping the extension of "features" to also eventually scan everything else at some point.
As an App-only feature, you basically disable that whole functionality if you don't have the App.
If you're an iCloud user, it's not much different. But if you're not, you're basically safe from that particular feature creep of extending the scan to other parts of the system (like whole photo-roll, documents, data from other apps, etc)
In the current iteration they are not using AI or anything to identify "bad" content. They are matching hashes for files on on your device against known bad hashes/content.
I'm not trying to convince you one way or another about it, just clarifying what is happening.
I see Apple getting a lot of flak but I'm surprised as anyone that they have decided to suddenly buck their trend/perception of being pro privacy quite so dramatically.
In truth, the issue is that photos are put on Facebook to share with others and other people will be looking at them and reporting them. Facebook does look at content as well, but the reporting numbers are guaranteed to be vastly different simply because one is a sharing platform and the other is a means to back up your private data.
That winnie the pooh with text over it sure does look like a problem. Can I haz alert to whoever has that?
k. thx.
— Xi Jinping
iCloud in China is controlled de facto by the CCP.
Apple is taking something from its users in the West which its users in China never had.
I walk by an iPhone billboard fairly frequently, all it says is "Privacy".
I can come to only one conclusion as to why Apple is doing this, and it isn't because they're acting on their own free will in the interest of their business and users.
If car manufacturers installed a breathalyzer ignition interlock device on every car (new or currently owned) that would save some lives, but I'm pretty sure very few people would tolerate it.
https://www.reuters.com/world/us/us-senate-bill-seeks-requir...
The more opaque proprietary services a device runs, the less yours it can be. I'm surprised it took this long to remember this concept.
Hello All Linux Devices
Politicians began hating on tech corp and promising changes. We’ve seen attacks on section 230 in the press and bills popping up.
This is not going away. My guess is it’s a hedge against government who decided to pass some messages along.
“Why won’t Apple help us fight this?” being suggested by political grifters in both parties. Cable news will hassle them.
They’ll do this to avoid that.
Apple respected its Users' privacy and therefore only reported a low number CSAM cases since it didn't monitor and scan anything.
Facebook, because it doesn't respect its Users' privacy, wants to monitor and scan everything so it has to report large numbers of CSAM cases (in the millions).
It seems to me that the "number of CSAM cases" is a nice (perverse?) corollary to how much a particular service respects its Users privacy.
1) Either get photos on their phone with the correct HASH, or get some photos that are already on their phone into the HASH list. 2) FBI will be quietly informed by Apple. 3) FBI shows up one day at the targets home or work. They confiscates all electronic devices and ruins that persons life.
However as things stand now, with Apple trying to move more of their revenue to services and the general stickiness of the Apple's platform, this change would be palatable to most users.
In the end, all I can say for sure is that I have lost my enthusiasm for consumer technology.
I think the problem is that it's too late; they expended the effort to build the engine into iOS 15. As far as we know the FBI couldn't compel them to write code like this (see the case with the San Bernardino shooter), but it's easy to believe they could compel Apple to maintain this backdoor in the future.
I've been getting angry about this for the last week, and am just now getting to the point where I can accept the reality that the snowball is already rolling. The government isn't going to step in and prevent this, unless it's to make the backdoor larger, and nobody who was going to buy an iPhone or use iCloud is going to stop because of this. So, it's done. Apple will put a backdoor on your phone and yes, and obviously that backdoor will eventually be used for things beyond the scope of what they're telling you now.
She's 24, but could pass for much younger. I'm paranoid that I'll get flagged.. Especially since all of her tik-toks with any skin shown, or any drinking get flagged.
Considering PhotoDNA, the source code of Apple's implementation, a 30-year Ph.D. level in cryptographic knowledge, and the processes by which neuralMatch actually run are unavailable to the OP, I think OP is justified in the minor paranoia of not wanting to be falsely accused of owning CSAM.
That's precisely the problem with Apple's announcement here. You can't apply common sense to an algorithm and dragnet-style surveillance.
I doubt even Apple specifically knows precisely how things will turn out with this system.
What if you take a picture of your naked baby in the bathtub or your child on the beach, and the picture is very similar to a known CSAM picture with a hash in the database, enough to pass the distance threshold they're using?
The picture would be sent for screening, the Apple screener would indeed say that's a naked baby/child, and soon enough you've got the FBI (or whatever is the equivalent in your country) knocking on your door and arresting you for pedophilia.
This is not accurate. Apple is using perceptual hashes. If the features of an image are close enough to an image in the NCMEC database it may generate a matching hash.
[0] https://www.apple.com/child-safety/pdf/Expanded_Protections_...
[1] https://www.macrumors.com/2021/08/09/apple-faq-csam-detectio...
This is a compromise to rid them from or at least threaten a potential safe space. All real compromises are bad compromises. This seems to be just that
Preach.
Though with Apple phones this has never been the case, unfortunately. Having your photos scanned on upload is just the little cherry on top of the sundae of user disenfranchisement that is iOS.
After all, in the iPhone case, I'm not allowed to root it, install anything I want on it, etc. Apple already controls it, and already decides what I can do with it.
So, if Apple decides that it can't be used to send child pornography, how is it different from 'standard' usage restrictions ? Or is it only the fact that it might trigger a call to law enforcement which is a problem, and without this call, everything would be ok ? In other words, would an iPhone which detects child porn and refuses to do anything ('error : illegal content') with it be ok? Or would this be a violation of privacy?
Now the only thing left is to install some devices that monitor every spoken word in our apartments and check it against, well, CP ain't going to cut it, religious extremism, maybe? Good thing that no one would be that stupid to voluntarily install such equipment in their home...
The alternative now is: no phone. Bye-bye Facebook and Instagram and Twitter and Snapchat and Candy Crush, etc. The mobile app market, and the advertising market, could fall apart if everyone did the same. The economy would be significantly impacted.
What a huge mistake.
This is the crucial part here. The means are there, apple can't say it's impossible to do so, the implementation is simple, and the consequences for individuals with winne-the-pooh photos are horrible.
- Government says Apple has to do it
- Apple announces they will do it
- Apple brand suffers immensely
- Apple cites suffering as a reason to not do it?
Seems like there would be a word/term/phrase for this kind of thing.
I'm so optimistic I'm hoping there is an ulterior motive here or some sort of outside coercion.
Apple once refused the FBI to hack a terrorist's phone because it would lead to a loss of privacy for iPhone users. Now Apple has reversed that decision.
However, the majority of Apple users are uncaring and/or unaware of these recent changes and it will not affect their sales beyond the tiny tech minority.
https://www.apple.com/child-safety/pdf/Expanded_Protections_...
Meanwhile, it feels like almost everyone is taking photos, videos and live steaming the entire thing from high-end smartphones!
Explain your thoughts on that topic. I am pretty sure they have a this email monitored and can filter for "CSAM". Once big enough it might be reported to Tim.
It will be interesting what the effect on sexting will be once we know that some Apple employee can look at you naked because the tool mismatched.
And the people pushing this behind the scenes know that the majority of CSA happens in the home or with adults that the child trusts, so if we are to be fine with reaching into everyones private life, video surveillance in every kid’s bedroom should be far more effective than attacking the distribution of CSAM alone…?
This is it. Of course I'd like child pornographers to be caught.
But right now my expensive phone will be using up valuable charge looking for photos that aren't there.
And in future I can't be sure it won't be searching for evidence of support for the political opposition. (This is not only China or Gulf states: democracy seems on pretty shaky ground in the US, UK and elsewhere right now).
Out of all the arguments on this site against this feature, the battery bit is by far the most unfounded one and super odd to see repeated on a technical forum.
1. what else to scan for.
2. what other OS vendors should add matching functionality because Apple has proven it is doable.
I've been a paid sub to Stratechery since the Amazon acquiring Wholefood article - I think Ben's thinking is really unique and valuable.
But - I've recently stopped my subscription, partly because of change in my own finance, partly because I am no longer interested in what Ben wants to cover - namely privacy, policy, and anti-competitive. I miss the days where companies like Stitch Fix could earn a email. I think Ben is covering important topics - but it's not something that I'm really interested in.
Plus the fact that each post seems to require a large amount of prior knowledge - manifested by the large amount of quotes in each email.
What do you think? Have you found other blogs worth subscribing to?
TL;DR: I think one of three must be correct:
A) NCMEC is seriously neglecting protecting children by accepting a subpar solution. We're getting the client-side scanning precedent without any benefit to society.
OR
B) Apple is not entirely ditching server-side scanning, so again we'll be gaining nothing from client-side scanning aside from normalization of a dubious practice.
OR
C) The client-side solution will end up way more invasive than what even Apple believes it would be.
---
Lets look at the worst scenarios. One or more pedos is actively molesting in order to record CSAM on a iPhone. The iPhone soon syncs to the iCloud, while the pedos distribute the CSAM. This is the scenario we all want to stop most and ASAP.
1) Apple's is a hash-based approach, not an AI based one. So the original upload must be marked as 'clear' by the iPhone, since NCMEC can't yet know about the CSAM.
2) Hopefully, sometimes later NCMEC does find the CSAM and marks it appropriately.
3) In the 'naive' server-side approach, Apple could scan iCloud and find the original CSAM and then the uploaders. This is not what we're doing. This is a client-side approach, and if certain rumors are right Apple will eventually do end2end encryption and so will not even be able to scan server-side. So do we do?
4) One option is for NCMEC to give up and let Apple do a worse solution than server-side scanning where the original perpetrators not only go free but can use iCloud for storage without being detected. That's our option A above.
This is undesirable and unlikely. Even if NCMEC were so compromised, Apple can't want to get all this fire and then implement an non-working solution. Even if Apple were to do that, eventually pressure will force them to implement a working solution.
5) The other option is for the iPhone client to receive regular hash updates, and then periodically rescan any picture that was at one time uploaded to the iCloud (option C). Since the incriminating files may well not even exist anymore on the phone (the pedos in our scenario have good motivation to eventually delete the CSAM to save space even without the issue of client-side scanning), Apple will have to store the hashes of deleted images on the phone in order to periodically rescan them.
So to implement effective client-side scanning, iPhone owners will not be able to ever truly delete an image if it was ever synced to iCloud, even if the owners deleted it from the cloud and their phone. The hash will be stored, and not only that - a rough facsimile of the image should be restorable from the hash*. I can't see any other way to do it, and Apple and NCMEC must want the client-side solution to work. In a world where Pegasus and things like it exist, this is.. not optimal.
6) Of course, Apple could avoid all this by storing the hashes in their iCloud (option B), but what would be the point of client-side scanning than, aside from a dubious precedent?
Am I missing something?
* AFAIK the ability to make a rough facsimile of the original image from the hash is a property of visual hashes required for them to work even on manipulated images, and also required for the manual review system. Apple can't avoid this on a client-side solution - it's either the original image or something close enough so it can be manually reviewed if later found to be CSAM.
Either way, a workable client-side solution has to end up more encompassing than what Apple has done.
I wouldn't be surprised if a completely innocent false positive gets you put on a list indefinitely, with little recourse.
You have more trust in Apple, something i don't. So we see this change in a different light.