If not, what is going to convince them to stop at this point?
I believe that they should scan user data in some capacity, because this is about data that causes harm to children.
However, I believe that they should not run the scan on the device, because that carries significant drawbacks for personal privacy.
I think this is actually a pretty weak argument. Children may be vulnerable, but so are many other segments of the population, including those with disabilities, the elderly, and so on. I guess my question is if we're trying to protect children, then why not go all the way? The answer is because going all the way is dangerous: where does it stop? Why here but not there?
I think a better argument would be: we should scan phones for X because X is illegal, and we've deemed, as a society, that X is wrong/bad/punishable/etc. But the problem with that argument is that we already have ways of checking phones/user data for X, and, thankfully, most have to go through a legal process of probable cause, obtaining warrants, and so on. Apple's proposal completely circumvents the legal system.
This is a complicated question that I'm sure will be litigated in court in the next few years. SCOTUS has ruled that drug sniffing dogs (e.g. in airports) are constitutional, whereas using infrared imaging (e.g. for finding drug grow houses) is not. Are hashing algorithms more like drug sniffing dogs or like infrared imaging?
But if the lack of a warrant is the problem then the main issue would be the third-party doctrine and having no reasonable expectation of privacy when entrusting your data with any company, not just Apple.
Because, if I were to compare to what OP claims here, constant device scanning and then contacting law enforcement is like having a company constantly infrared scan your house for drug growing, and then contact the authorities about it. This is circumventing the authorities' legal limitations by moving the burden of responsibility to a company.
This is a carefully chosen stated reason for Apple to add this functionality to avoid user pushback. The actual uses will vary from country to country, but on the whole will not result in the protection of anyone other than the government or ruling party.
In countries with less civil freedom this will be exploited and used as a tool of oppression against minority political groups.
The on-device scanning capabilities will be expanded in the future, but it can be stopped if Apple care about their users.
Right now it seems they don't care.
Forgot to mention that I'll not be recommending or purchasing any Apple devices until they halt this scanning idea.
Apple cares about selling devices, maximizing revenue. Users are just something required to authorize transactions.
If Apple has to kowtow to the most oppressive regimes in the world to keep doing the above, you bet they will and probably already are. And since it is in the interest of both Apple + Gov't to do shady stuff in the dark, there is a good chance they can hide it the public indefinitely.
There are things you can use technology for to help combat CSAM, but this is certainly not an acceptable way to go about it. Creating tools to help report abuse and illegal imagery in their software, for example, could be useful. This is not.
What if the police come to Apple with a warrant? Should Apple have the responsibility to reveal the user's content to authorities?
Part of the motivation for this type of thing is that E2EE effectively makes the old system of warrants obsolete and hides this data permanently from authorities. Lots of people see that as a huge win. Lots of people see that as a huge loss.
If they lack the capability to do so then they don't need to worry about it.
> Part of the motivation for this type of thing is that E2EE effectively makes the old system of warrants obsolete and hides this data permanently from authorities. Lots of people see that as a huge win. Lots of people see that as a huge loss.
The speculation that this enables E2EE is just that; speculation. It's also not a strong argument because Apple could implement E2EE without snooping people's photographs and they'd be in the same position of being unable to assist law enforcement.
If you have a system in place that intercepts all files before they're encrypted and uploaded to the cloud, then E2EE doesn't serve its purpose of ensuring privacy of people's data. It completely defeats the point.
"They don't need to worry about it" may be enough for you, but it is not a satisfactory moral answer for a lot of people. It shouldn't be tough to understand that some people don't want to actively enable illegal activity.
>If you have a system in place that intercepts all files before they're encrypted and uploaded to the cloud, then E2EE doesn't serve its purpose of ensuring privacy of people's data. It completely defeats the point.
Which is the reason why Apple proposed running this on the device so that those files don't leave the device if they aren't flagged.
You're not "actively" enabling illegal activity. The consequence of protecting people's rights is that illegal and immoral activity can occur under those same protections. It shouldn't be tough to understand that people don't want their rights trampled on to enable surveillance.
> Which is the reason why Apple proposed running this on the device so that those files don't leave the device if they aren't flagged.
How does this solve the issue, exactly? The fact that it's on-device doesn't change the equation here.
I don't want to get into a semantic debate about the word "actively". Suffice it to say that some people don't want to work on a product or support a company that they know is enabling criminal behavior while also refusing to do anything to curtail that illegal behavior.
>It shouldn't be tough to understand that people don't want their rights trampled on to enable surveillance.
Apple recognizes this. Once again, this is why they are doing this on device and not in the cloud like other providers. They are trying to do the scanning in a way that is invades privacy the least.
>How does this solve the issue, exactly? The fact that it's on-device doesn't change the equation here.
It changes the equation because it allows for everything leaving the device to be E2EE. One example is that this would stop anyone at Apple from snooping on your files. If the scanning was done on the server, Apple would need decrypted versions of the files which could be viewed by Apple employees.
I already gave one example of something they could do to help curtail illegal behavior without requiring an invasion of privacy. Some people also don't want to support a company that they know is enabling surveillance, which is what Apple is doing here.
> They are trying to do the scanning in a way that is invades privacy the least.
A violation of your right to privacy is a violation no matter how small you might think it is. They don't need to invade our privacy at all.
> It changes the equation because it allows for everything leaving the device to be E2EE. One example is that this would stop anyone at Apple from snooping on your files. If the scanning was done on the server, Apple would need decrypted versions of the files which could be viewed by Apple employees.
Everything leaving the device isn't E2EE and this doesn't "allow" for everything leaving the device to be E2EE. Stop pushing this false dichotomy. Things can be E2EE without any invasion of privacy whatsoever and there are other options to help combat CSAM that aren't violations of a user's privacy. Stop pretending like this is the only option when it's clearly not.
You said they should add a reporting mechanism, that won't stop anything because if a person already has knowledge that this content is being shared, they could just report it to the authorities. There is no reason to report it to Apple. Like you said, "Apple's role is not the police".
>Some people also don't want to support a company that they know is enabling surveillance, which is what Apple is doing here.
I acknowledged that in my first comment. Some people feel one way. Some people feel another. Apple is acknowledging both sides of this while you are only acknowledging one.
>A violation of your right to privacy is a violation no matter how small you might think it is. They don't need to invade our privacy at all.
This type of black and white ideology can't function in the real world. All rights have limits and the general rule is that your rights end when they start infringing on my rights. Your right to privacy does not supersede my right to not be a victim of a crime.
>Everything leaving the device isn't E2EE and this doesn't "allow" for everything leaving the device to be E2EE. Stop pushing this false dichotomy.
Everything I have seen from people in the know has suggested that this is a step on the path to Apple enabling E2EE encryption for iCloud.
>Things can be E2EE without any invasion of privacy whatsoever and there are other options to help combat CSAM that aren't violations of a user's privacy. Stop pretending like this is the only option when it's clearly not.
Do you have an actual suggestion that you think will cut down on CSAM beyond providing a redundant reporting system?
Recognition isn't something people would always be able to immediately do with CSAM imagery so Apple, which has already created a tool to recognize it, can assist with that. It can also create tools to reduce the overhead of reporting. If you make it easier to report, more people will do it. You're assuming that anyone that ever encounters CSAM would go out of their way to report it, which simply isn't true.
> This type of black and white ideology can't function in the real world. All rights have limits and the general rule is that your rights end when they start infringing on my rights. Your right to privacy does not supersede my right to not be a victim of a crime.
My right to privacy is not an infringement of your rights so this argument has no bearing in reality. Law enforcement requires probable cause to get permission to surveil the population and there's been countless cases thrown out because of this violation, which, as it turns out, was even tested in the context of technology interfacing with NCMEC.
> Everything I have seen from people in the know has suggested that this is a step on the path to Apple enabling E2EE encryption for iCloud.
Those people are also pushing the same false dichotomy that you are. There is no technical reason that this is required to enable E2EE for iCloud, it's purely speculation as to why Apple would roll this surveillance tool out, as a compromise to having a fully encrypted system.
> Do you have an actual suggestion that you think will cut down on CSAM beyond providing a redundant reporting system?
The reporting system isn't redundant, but beyond that, the impetus isn't on the person whose rights are being violated to offer solutions to replace that violation.
The law isn't going to rely on people that might or might not report CSAM if they see it. Average users have no obligation to report CSAM, unlike the law and the companies that store user data. If the law finds CSAM themselves, they will always report it, and in their eyes that only improves the chances of a successful conviction.
> My right to privacy is not an infringement of your rights so this argument has no bearing in reality.
The law would argue that a right to privacy that includes the ability to store CSAM privately is infringing on the rights of others, because allowing CSAM to be consumed and distributed creates a market that incentivizes the further abuse of children. CSAM cannot be produced without CSA taking place, which makes it a part of that cycle.
> There is no technical reason that this is required to enable E2EE for iCloud, it's purely speculation as to why Apple would roll this surveillance tool out, as a compromise to having a fully encrypted system.
That is Apple's own fault for failing to precisely explain their reasoning for designing the system the way they did, and the confusion was entirely preventable.
The reason this is murky territory is because it's on-device, and we have precedence of there being an expectation of privacy on our cell phones via Carpenter v. United States which found that in a very narrow circumstance third party doctrine did not apply. However, other cases have not been tested and there are many factors regarding Apple's implementation that may or may not protect it in an actual case. We don't really have a clear picture of how and when third party doctrine applies with the case of our cell phones, which have been determined to be nearly indispensable in modern society.
But there's a difference between the legality of something and morality to it and while the former is up in the air, the latter really isn't in this case, at least for a lot of people that are arguing for or against this system. In my opinion, it does not matter that Apple's implementation only ever activates if you there is intent to upload to their servers. The fact that it is performed on device is in itself the most problematic aspect. There are no acceptable justifications to be had here. If Apple wants to scan their own servers that is their prerogative. The same cannot be said about my phone.
You said you didn't want Apple to be the police but you know want them to be the judge of what is and isn't CSAM?
>It can also create tools to reduce the overhead of reporting. If you make it easier to report, more people will do it. You're assuming that anyone that ever encounters CSAM would go out of their way to report it, which simply isn't true.
I don't know why you are assuming that potential reporters knowing about CSAM but not reporting it is anywhere close to as common as CSAM being unknown and therefore obviously unreported.
>My right to privacy is not an infringement of your rights so this argument has no bearing in reality. Law enforcement requires probable cause to get permission to surveil the population and there's been countless cases thrown out because of this violation, which, as it turns out, was even tested in the context of technology interfacing with NCMEC.
As the other reply stated, certain content does infringe on other people's rights. Also Apple isn't law enforcement. They don't need probable cause. One of the primary and unmentioned motivators here is that they don't want CSAM ending up on their servers and opening themselves up to legal action.
>Those people are also pushing the same false dichotomy that you are. There is no technical reason that this is required to enable E2EE for iCloud, it's purely speculation as to why Apple would roll this surveillance tool out, as a compromise to having a fully encrypted system.
It is a technical requirement once you accept the moral requirement that Apple doesn't want to enable the sharing of CSAM. Once again, you are ignoring that some people think the morality of enabling (or at least not curtailing) the sharing of CSAM is a moral failing that can't be accepted.
Your argument of it being a moral requirement is dubious. If you really wanted to stop CSAM why not we all live in glass apartments like in Zamyatin's We? To what extent would invading your privacy be acceptable to you if it means we could stop some or even all CSAM distribution and production? Once you decide that it's a moral imperative then you accept that any erosion of privacy in the interest of saving the children isn't off the table.
There's no practical way to test this because there's no incentive for any of those companies to change how they scan for CSAM. None of those companies are going to take the risk of implementing something that could enable future sexual abuse crimes to take place just to improve privacy. The system in place apparently works, but there appears to be little statistical data validating those techniques.
What I would want to know is what alternative method for reporting CSAM could be developed that would meet the current expectations of the law in the number of cases they successfully uncover while also satisfying the desire for absolute privacy under all circumstances. A system that has users reporting content themselves relies on the users acting to incriminate other people, and the criminals can simply improve their privacy by hiding the content out of public view. It also doesn't address the fact that the government doesn't want people storing CSAM on their phones at all, and if they happen to notice you're storing CSAM for any reason, they're going to arrest you for it.
I'm honestly not sure that it's possible for any company to implement true E2EE given the current state of the law. It sounds like a tradeoff had to be made between privacy and the ability for the law to be upheld.
Isn't this in itself a win for reducing distribution of CSAM? The content is already out of public view. You can't just go to some clear web website and find CSAM unless you're incredibly unlucky and all legal entities have systems in place to report it if it does end up on there. Companies like pornhub have fairly recently created systems to reduce the chance of CSAM ending up on their platform by requiring verification. There's certainly nothing stopping most CSAM from being distributed encrypted through official or illicit channels and the sad reality is there probably is significantly more that is than isn't.
Law enforcement has a big task on their hands here and I can empathize with how difficult it can be but you cannot sacrifice an entire population's rights to catch the relatively few bad actors. This erosion of rights was justified after 9/11 to implement the PATRIOT Act and people are trying to use CSAM as a justification to do so now. It's not good.
> I'm honestly not sure that it's possible for any company to implement true E2EE given the current state of the law. It sounds like a tradeoff had to be made between privacy and the ability for the law to be upheld.
There's already companies that do offer E2EE encryption and there is no law requirements to scan for CSAM in the US. I'll concede the point that Google, Microsoft, and Apple are going to be under much more scrutiny of law enforcement and the government than a small privacy focused cloud storage provider, but the point stands that there is no legal reason they must implement this.
Making CSAM illegal isn't merely about preventing the material from being publicly visible. Having a ring of child abusers privately circulating CSAM outside of clearnet by using Tor or by exchanging thumb drives is just as damaging of an act to law enforcement agencies, because it means the market can still exist in a different environment. There will always be other ways to distribute CSAM, but what the law sees as within its reach in order to prevent the spread of CSAM to the greatest extent possible is to pressure companies that offer user data storage into increasing the amount of scanning for such material.
> you cannot sacrifice an entire population's rights to catch the relatively few bad actors.
Unfortunately this is a matter of public opinion, and there are many people out there that would argue instead that we need to give up some amount of privacy in order to better protect the human population.
> There's already companies that do offer E2EE encryption and there is no law requirements to scan for CSAM in the US.
What are the names of these companies? Are you sure there isn't a clause in their terms of service stating they will give up your data to law enforcement if requested, including the contents of encrypted files?
Even though there is no law compelling companies to scan for CSAM, the law is not going to let any of them store CSAM for an indefinite period of time. That would defeat the entire purpose of outlawing CSAM to begin with. Regardless of what's legally required of them, it's in the companies' best interests to ensure that the data stays off their servers. No matter what moral stance they take in deciding to implement CSAM scanning, they're not going to take the risk of being found criminally liable for the data they're storing.
How do you know what "the law" sees as being within its reach? Law enforcement and rights are always going to be at odds with each other.
> Unfortunately this is a matter of public opinion, and there are many people out there that would argue instead that we need to give up some amount of privacy in order to better protect the human population.
I don't really care so much about this hypothetical public opinion. Refrain from using this unknowable nebula of consensus to argue your points for you. Even if there is massive consensus that would not change my argument that it is wrong.
If a company has no means of which to know what is stored on their servers, they have no reason to presume that it is illicit material, even if it in fact is. Removing those liabilities by restricting what Apple can know has been how Apple's been operating for quite some time. Even this implementation could be argued that this is what Apple is attempting to do. The issue is that ultimately, Apple is installing software on your device to work only as adversary to its user by violating their privacy.
> What are the names of these companies? Are you sure there isn't a clause in their terms of service stating they will give up your data to law enforcement if requested, including the contents of encrypted files?
By definition if the third party can decrypt your files it is not end to end encrypted, the third party in this case being Apple. Many files upload to iCloud already are encrypted but since Apple also has access to the key it can decrypt and examine the contents of the files on its servers. With E2EE the only person with access to the private key would be yourself. They could certainly hand over the encrypted files and if law enforcement gained access to your private key they could then decrypt those files, but (hypothetically) not until after they've obtained the private key.
In my mind, what might have actually transpired for Apple to announce this change was a less severe instance of such governmental or legal pressure on Apple to scan as much data as the other major tech companies. Last year Apple only submitted a couple hundred reports to the NCMEC, while Facebook submitted 20 million. Looking at those numbers makes it sound like Apple was a nail sticking out that needed to be hammered down.
How does that change the warrant's effectiveness? Should you be obligated to unlock your phone for authorities? Otherwise the warrant still does nothing because the device's encryption can't be broken without forcing a person to potentially self-incriminate themselves. This is fundamentally the same problem that encryption breaks how warrants used to behave.
Should GM use onstar to record every conversation you have in your car and use AI to flag conversations that relate to child abuse? After all it involves children.
And if down the road law enforcement decides to expand what gets flagged, you shouldn’t care if you aren’t breaking any laws, right?
This is a horrible idea and it’s a matter of if, not when it is abused.
I dont care if it will save billions of lives, your devices should not betray you.
The legal system already provides a means for invading people's privacy given certain criteria, whether by scanning their phones, physically searching their residences and offices, or whatever. The criterion is "probable cause" and the search is launched by a judge signing a warrant that the probable cause exists. That is how the Framers wrote it into the US Constitution (see Amendment 4), because they had dealt with too many bogus searches done by the Colonial governments and wanted to put a stop to that. Things have not really changed since then.
"Think of the children!!!" is a traditional thought-stopper, but really, if there is substantive reason to search someone's phone, just convince a judge to sign a warrant and everything will be fine. Searching people's stuff when they are not under any concrete suspicion is part of why we had a revolution in this country in the first place. I.e., why does Apple hate America, as the memes like to put it?
Google, Facebook, and others are already scanning the images when uploaded to them. We should not be surprised that the images we upload are being scanned. Sometimes for CSAM, sometimes to train ML, and sometimes for other things.
Apple doing scanning images that were uploaded to their servers would just be another company doing the same thing.
If I understand things correctly, these companies even have some legal obligations to do some of this.
> However, I believe that they should not run the scan on the device, because that carries significant drawbacks for personal privacy.
This is about where Apple is doing the scanning. Doing it in icloud is on them. Doing it on our phones is an invasion of privacy. And, it builds a system that can be used for other things. It opens a door, as many others have pointed out.
I wonder if Apple did all of this to scale the implementation. So it wouldn't have to run on their servers. They wouldn't have to maintain and power them. Poor implementation if that's the case.
So now they have a wedge that they can gradually use to expand what is going to be scanned and when.
This is a classic power move straight from 48 laws of power.
Speaking out enough on this gets you called a pedo.
I will not go as far to claim that the people for this are intentionally using children to shield themselves from criticism push out a universal surveillance mechanism. ... but if you wanted to do exactly that, hiding behind the spectre of child abuse is exactly that.
There are some hard questions we should be asking. Facebook claims to have reported some 20 million insances of child porn last year, but there were only a couple prosecutions. Are the matches fake? Is the goal really to just build a huge database of kompromat? If these images aren't a concern why are only an infinitesimal proportion of cases prosecuted?
The data is not what causes the harm. It is the method used to produce (some of) the data that causes the harm.
And the problem is that looking for the data in order to find the people causing the harm runs afoul of the converse fallacy. It's the same fallacy that causes people to think that because most terrorists (that they know of) are Muslims, that most Muslims must be terrorists. Most child abusers have photos of naked children on their phones, therefore most people who have photos of naked children on their phones must be child abusers. It's false in both cases. A friend of mine, for example, has photos of her naked child on her phone that she sent to the doctor to help diagnose a rash.
I guess it depends on whether the system is 100% safe and cannot be exploited;
Since only iCloud photos are scanned, and other cloud storage providers already do this and have been scanning their users photos for a long time. I haven’t heard anyone complain when Google started scanning their cloud photos, which is kind of… interesting.
Maybe if masses of people go into apple stores and save photos on demo phones that collide with the files they're looking for. Do those phones get scanned? Will apple ban their own app store accounts?