Apple’s new abuse prevention system: an antritust/competition point of view
blog.quintarelli.it
blog.quintarelli.it
With that being said, what Apple is doing here is just a blatant violation of that 'trust' and certainly a compromise to their commitment to privacy. Under no circumstances is it justifiable to essentially enlist people's devices to police their owners, while using the electricity that you pay for, the internet service you pay for, and the device itself that you pay for to perform a function that is to absolutely no benefit to the user and in fact can only ever be harmful to them.
It doesn't matter that the net data exfiltrated by Apple ends up being the same as before (through scanning unencrypted files on their servers). The distinction is so obvious to me that I find it incredible that people are legitimately arguing that it's the same, or that it in some way this is actually helping preserve user privacy.
As mentioned in the article, this does absolutely nothing towards protecting children other than directing all but the biggest idiots towards platforms that can't be linked to them, which I'd imagine, they already are.
I suspect you're more wrong than you think about this. People share large volumes of CSAM through lots of different services - I knew someone who worked on the problem at Linked In(!).
HN likes to downplay the actual reality as if it's always some trojan horse, but the issue is real. It's worth talking to people that work on combatting it if you get the chance. I'm not really commenting on Apple's approach here (which I haven't thought enough about), but I know enough that an immediate dismissal based on it 'not helping' is not really appreciating the real tradeoffs you're making.
You can be against this kind of thing from Apple, but as a result more CSAM will be undetected. Maybe that's the proper tradeoff, but we shouldn't pretend it's not a tradeoff at all.
"Robin Hanson proposed stores where banned products could be sold. There are a number of excellent arguments for such a policy—an inherent right of individual liberty, the career incentive of bureaucrats to prohibit everything, legislators being just as biased as individuals. But even so (I replied), some poor, honest, not overwhelmingly educated mother of five children is going to go into these stores and buy a “Dr. Snakeoil’s Sulfuric Acid Drink” for her arthritis and die, leaving her orphans to weep on national television" [0]
[0]: https://www.lesswrong.com/posts/PeSzc9JTBxhaYRp9b/policy-deb...
Edit: After digging in, HN commentary is missing the most relevant details about this particular implementation. iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate.
For iMessage child account ML scanning it’s on device and just generic sexual material restriction - more similar to standard parental controls than anything else (alerts only go to the parent, and only happen on child accounts)
My initial impression is this is a good approach. The risk mostly comes from future applications (like the CCP adding hashes of other stuff that isn’t CSAM like tankman or something).
The frankly ignorant knee-jerk responses from technical HN readers do a disservice and weaken the ability of technical people to push back when necessary.
The point is: This is too easy to abuse.
Most of the HN responses are dumb, get the details wrong, and don’t take the trade offs seriously. That kind of thing causes me to dismiss them entirely.
There are legitimate arguments and risks which I’d concede, but they’re not what most people in the comments are talking about.
"People just don't understand!!" has never been a convincing argument.
The implementation specifically for detecting CSAM can be okay while using that for other purposes can not be okay.
The goal is to stop child sexual abuse, FB reports millions of cases a year with a similar hash matching model for messenger.
> ""People just don't understand!!" has never been a convincing argument."
That's not my argument - I just think most of the HN comments on this issue both miss the relevant details, and are wrong.
There are people who profit from CSAM and in turn this creates a market for abuse. Unless we create structures which disincentivizes these behaviors — right now there are none - they will continue to grow. What Apple is building will basically make any criminal who sells to someone sloppy enough to store in iCloud risk being traced as the origin of the CSAM. Back to the darkest web they go.
Anti CSAM is inevitable. You will find similar systems for all major providers eventually.
I think this is why getting the details right matter because if people are arguing about unrelated nonsense it's harder to focus on the actual risks represented by the questions you're asking.
There are diminishing returns to using the same old excuse again and again. I'd say most people are just tired of the whole "Just think of the children!" into "Ah we got this system in place why not use it against <a little less evil but still illegal thing> too" followed by "It's the law in China/Turkey/Russia/wherever, <Company> can't just ignore it (and thus not help putting reporters, critics and other people into prison)" combo.
What you are saying is basically another rendition of "look the problem of child abuse exists and this could help so it is worth discussing", which is also a variant of the same.
> iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate.
Actually we recently had news where an Apple tried to help cover up their errors [1], the system was supposed to be safe™, doesn't mean the people having control over it can't make mistakes, or worse.
What details are being missed here, exactly? Ultimately it is trivial to expand the hashes to compare to, isn't it? What does it matter that they use CSAM for now? It doesn't remove the involvement of humans in controlling the system, so false positive rate will never be "effectively zero", and it can easily be expanded.
We are left with the same two arguments as always:
- Think about the children
- Just trust the company, they use technology™, no you aren't allowed to check. Yeah they can easily abuse it, but we don't know for sure!
I wouldn't say HN is ignoring details, many people are just tired of the same old loop, there is no reason to put trust into these endeavors and it is reasonable to doubt even the motives.
Now they're saying that if the match is a false positive, it'll be screened by the human reviewer. But that means there's some stranger there potentially looking at my private photos (and, by the way, I wonder which country they'll be located in?). That's already completely and utterly unacceptable - you don't have to wait for any "future complications".
I don’t know enough about the specific implementation or perceptual hashing details and probably pushed back too hard as a result of the other comments at the time (which were comments out of ignorance).
The level of downvotes I received is disproportionate to what I wrote anyway - this is clearly a political issue on HN. The irony is I’d probably align more with the risks being too high position, but it needs to be considered after actually understanding what the true risks are first.
Occam’s razor is that they’re doing what they say they’re doing and not some more complicated future action.
You said earlier upthread, that HN is ignoring the tradeoffs. I find that distasteful, people have considered the tradeoffs of this and they don't like them, but that doesn't mean they need to preface every single statement about the thing with "well actually i've considered the tradeoffs, and I've found them to be blah, and..." That would quickly get tiresome which is why people don't do it.
I'll make a different, similar generalization. Like most people pushing this, you are pretending to care about the "tradeoffs", but you don't. You don't care about the downsides, you just say you do, and then blithely ignore them when they are brought up. What's the golden rule again?
You cannot claim to be making "the real reasonable analysis" and write this. So much for "you're all geeks stuck on technical details". Quite the contrary: I'm sick of bogus software pretending to solve problems for me, while the quality of tech has exponential degraded over the last 20 years (often due to trying to solve some unsolvable problem in a bad way that backfires).
Now imagine you have a 16 year old girlfriend. She sends you a nude photo. Your phone calls the cops on you (it doesn't matter if the phone doesn't quite do this now, it will in the future. They will use their ML crap to detect the age of subjects in photos and explicitness of the photo). You normally wouldn't go to jail for this since 16 is legal in 99% of the civilized world, but thanks to America with their super duper "non-technical" innovations that only big boy white collars can understand, you can go to jail for having a photo of your legal girlfriend.
> big boy white
but why the totally uncalled racism and sexism here?
It does nothing to strengthen your argument and it is just dumb.
I meant "white collar big boys", but I did not bother to edit as I'm writing.
The guy above is claiming everyone who is against apple's yet-another-bogus-TPM-style-snakeoil is a little geek who does not understand anything outside their little tunnel.
Also now that I re-read his comment:
> Edit: After digging in, HN commentary is missing the most relevant details about this particular implementation. iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate.
False: it's a perceptual hash. Ignoring the fact that if for some reason you choose to let people host stuff in your icloud account (perhaps as a neat hack), which may be out of terms of service, but certainly not worth 20 years of jail: perceptual hashes have false positives, and can confuse images that appear harmless but were crafted to look like $badimg. But you don't have to be technical to understand that having your devices police you is bad, you just have to not be blinded by politics and boogeyman your state has sold you.
This is an obvious misrepresentation. An opt-in system to detect when adults are trying to groom kids by sending them porn seems like the opposite of harm. I imagine a lot of parents want that.
As for scanning what gets sent to iCloud. That’s also an opt-in service, and frankly it seems entirely reasonable for Apple not to want their servers to be used as a repository or hub for child porn.
If Apple wishes to scan what's on their servers, that is their prerogative. They can use their compute resources and energy to do so. You needn't install spyware on a person's device that is of no benefit to the user. I'll reiterate, this can only ever be harmful to the user. Its utility right now is at its absolute best and most altruistic and it is still a violation of people's privacy and stealing computing resources from the device owner.
> That’s also an opt-in service, and frankly it seems entirely reasonable for Apple not to want their servers to be used as a repository or hub for child porn.
This will not stop their servers being used as a repository for illicit materials, if that is what you're suggesting.
They have to scan for CSAM by US law.
The EU and UK are in the process of passing laws requiring scanning.
No, it is opt-in. Nothing is being stolen.
For my phone though... no idea. My iPhone is honestly such a solid piece of tech. I don't _want_ to go Google either... so what else do i have?
I know lots of people run de-googled Androids, which i guess works, but i'd prefer to avoid them entirely. Is there anything that works?
edit: I know of the Purism phone (https://puri.sm/products/librem-5-usa/) but that's the only one i know of. Anyone know of others?
edit: Whoa, 2K for that purism phone. fuckin' a.
Anyone who comes across this comment use one?
I wonder if framework will have a mysterious bump in sales because of this
My MBP's battery failed (swelled up) twice while under warranty, but it was a pain to deal with each time: wipe the SSD (because it's soldered on), hand it over to Apple, wait several days for repair, and then restore from backup. And now it's no longer under AppleCare coverage anyway.
In contrast, Framework designed their motherboard so we can even use it as a standalone PC once we're done using it as part of a laptop. That's such a difference in terms of user reparability.
iOS 15: "WTF is this?" SWAT team crashes through window
The example is somewhat contrived.
If a 'friend' takes your phone and has access to it and then uses it to take images of CSAM similar enough to the original image that it triggers the hash match and does this enough times to go over Apple's threshold to flag the account after these images are uploaded to icloud without the original phone owner noticing then yes it might cause a match.
At that point the match is probably a good thing (and not really a false positive anyway) - since it may lead back to the friend (that has the illegal material).
Or anyone who can just text you since imessage backs up to icloud automatically...
I haven't been particularly impressed with Apple's security record[1] lately, and I don't trust them to not mess this up.
1 - https://bhavukjain.com/blog/2020/05/30/zeroday-signin-with-a...
- A 2016 study by the Center for Court Innovation found that between 8,900 and 10,500 children, ages 13 to 17, are commercially exploited each year in the United States. (Center for Court Innovation, 2016) https://www.courtinnovation.org/sites/default/files/document...
- The annual number of persons prosecuted for commercial sexual exploitation of children (CSEC) cases filed in U.S. district court nearly doubled between 2004 and 2013, increasing from 1,405 to 2,776 cases. https://www.ojp.gov/sites/g/files/xyckuh241/files/archives/p...
This is a niche crime from everything I've seen.
Apple, if it were truly interested in the net good of children, could have picked something that impacts more of them (nutrition? early childhood education?), didn't introduce new vulnerability / abuse surface area, and was less politicized.
Maybe Whole Foods or maybe some popular restaurants are better candidates for working on improving nutrition in public schools? Why don’t we let apple contribute where it thinks it can. Maybe with apple that number goes down from 10,000 to 2,000. Wouldn’t that be a celebrated outcome?
Authority and its keeping is the number two law of the jungle. Any power handed over in the name of security, "to stop all crime", is an affirmation, a concretization of its future abuse. You speak of the calculated cost of preventing child abuse as acceptable. What of the abuse of an entire people?
This is not handwavy theoreticals. We already know what happens, in the US, when you push an agenda in the name of protecting the children: it looks like FOSTA/SESTA, which has driven sex workers of America underground and exposed them to more violence, more danger in a profession already one of the most murderous professions in the world. Those murders, in the name of protecting the young, are at the feet of the people who would protect the children with more authority.
What would be insignificant? 1 child? 100? There are 73,000,000 children (under 18) in the US alone. 10,000 is .0001% of that population.
> Why don’t we let apple contribute where it thinks it can.
Apple is the most profitable company in the world. It's a company that prides itself on its imagination and innovation, I wouldn't discount their ability to come up with something.
> Maybe with apple that number goes down from 10,000 to 2,000. Wouldn’t that be a celebrated outcome?
No, it's not. We make trade-offs all the time. The possible harm to Apple's user base is not worth the possibility that this reduces child abuse. There's a possibility these people move on to another platform and this does nothing.
To get that number down Apple creates an entry point for violating the privacy of half a billion users worldwide. Many of them are in China, where pressure from the government has already moved Apple in directions that are harmful to its customers[1].
1 - https://www.nytimes.com/2021/05/17/technology/apple-china-ce...
https://storage.googleapis.com/pub-tools-public-publication-...
If the prevalence is really around 8-30%, this seems a lot bigger than what Apple could even make a dent in. (Because most of the offenders are relatives/acquaintances of the victims. The phones don't seem to have any influence on the underlying numbers.)
Furthermore, criminalizing content again pushes the actual problem deeper into the shadows.
Why there are no routine questions about abuse for kids? At least that would help to identify victims, remove them from the abusive environment, and even potentially help catch the perpetrator.
Problem is that this scanning is necessarily fuzzy and there is going to be a false positive rate to it. And the way that you'll find out that you've tripped a false positive is that the SWAT team will knock your door down and kill your dog (at a minimum). Then you'll be stuck in a Kafkaesque nightmare trying to prove your innocence where you've been accused by a quasi Governmental agency that hides its methods so the "bad guys" can't work around them.
It isn't just "authoritarian regimes" abusing it, it is the stochastic domestic terrorism that our own government currently carries out against its own citizens every time there's a beaurocratic fuckup in how it manages its monopoly on violence.
This is the "Apple/Google cancelled my account and I don't know why" problem combined with SWATing.
[0] https://www.dictionary.com/e/what-is-stochastic-terrorism/
[1] https://www.merriam-webster.com/words-at-play/history-of-the...
How did we get here? How did everything become so politicized and polarized? How did law enforcement become so militarized? How did we as a society become so terrified and distrustful of our neighbours?
So when we have a billion iPhones in the wild taking 10 images a day...1 in a trillion chances happen every few months. Now, if that triggers some further review, maybe that's an acceptable false positive. If it triggers a SWAT team, I don't think it is.
suppose you have a partner who is a 'petite' woman of 34. She enjoys posting nudies on a website, but without her face in that picture. Someone who collects child porn downloads it, because he enjoys that picture. A year later he gets caught by the police and all his pictures get marked as 'verified child porn'. Suddenly you get marked as owning child porn.
https://www.nytimes.com/interactive/2019/09/28/us/child-sex-...
Apple’s thing has some sort of threshold anyway so one image would not trigger it. I don’t buy your example - the CSAM images are not what you’re describing.
They are instead fuzzy classifiers, and thus have non-zero error rates.
How can you do that, considering md5 can have collisions?
For a hash (whether cryptographic or perceptual), there is a chance of random collisions and also a difficulty factor for adversarially-created intentional collisions. The random collision probability has to be estimated based on some model of the input and output space (with cryptographic hash functions, you would usually model them as pseudorandom functions and assume that the collision probability is the same one created by the birthday paradox calculation).
Intentional collisions depend on insight about the structure of the hash function, and there are also different kinds of difficulty levels depending on the nature of the attack (preimage resistance, second-preimage resistance, and collision resistance). Gaining more insight about the structure of the hash function can act to reduce the work factor required for mounting these attacks. That should be true for perceptual hashes just as much as cryptographic hashes, but presumably all of the intentional attacks should start off easier because the perceptual hashes' threat models are weaker and there's much less mathematical research on how to achieve them.
And in AI systems involving classifiers, it was generally easy for people to create adversarial examples given access to the model. Perceptual hashes for estimating similarity to specific known images aren't the exact same thing because it's less like "how much like a cat is this image?" and more like "how much like NCMEC corpus image 77 is this image?", but maybe some of the same techniques would still work. In the cryptographic hash analogy, I guess that would be like trying to break preimage resistance.
To mitigate adversarial false positives one idea is to use the combination of a cryptographically strong hash along with a randomly selected perturbation of the file. Prior to hashing, perturb the file and submit both the hash and the selected perturbation to apple. Apple selects the DB based on the perturbation and proceeds with matching and thresholding.
If the attacker does not know how the image will be perturbed prior to hashing then he cannot generate an image which matches with known CSAM.
I think the rest of us have been discussing how this can and will be abused, by definition by adversaries.
Many of us have also observed for years how systems are abused so we sadly have a gut feeling for this.
I highly doubt that
Based on the fact that ultimately we can't check the system. And based on the fact that at some point in the chain humans are involved. [1]
What we are left with is "trust in Apple" not "trust in math".
If the system was matching against known cryptographic hashes the collision / false positive rate would be small, but the fuzzy matching involved with perceptual hashing necessarily has a greater false positive rate.
And that doesn’t even begin to address the detection of sent and received “explicit images” which are detected on device and don’t have a set of known hashes.
The iMessage bit is different - it's only on device, only on child accounts, and only alerts parents. It's more akin to a parental control feature than anything else.
They have nothing to do with each other, and I've seen a dozen people on HN confuse them. If the message is getting muddled here, it will be hopelessly conflated in less technical circles.
I'm concerned and upset about the CSAM filter for all the reasons that keep hitting the front page, but don't care about the opt-in parental controls at all, and if I had kids, I might want them.
But if I thought the CSAM filter worked like the nudie-detector filter, I'd be wigging out.
Bugs in the application of the code, combined with human complacency and mistakes can certainly lead to errors, even if the cryptographic algorithm itself was perfect.
We really need to bring back comp.risks
So now instead of sending just one nice innocent very high resolution images of "Tokyo City" or something with something horrific hidden somewhere you have to send a few such images.
That is reassuring. I can never believe anyone except me will think about that.
(If the system is too dumb to detect this it is worthless, and if it is smart enough this opens the floodgates for anyone wanting to make trouble for just about anyone.)
The idea is to take an image and have all of its possible derivatives create the same hash.
For example, if a hash was made of the Mona Lisa, any copy no matter how large, small, black and white, would have the same hash.
Think of all the ways the Mona Lisa could be transformed and still be the Mona Lisa.
The combinations of ones and zeros would be in the billions. If not more.
And all those possible combinations of ones and zeros go back to the same “hash.”
That’s extremely resourceful intensive.
My guess is that they are going to transform the images into a very low resolution, black and white thumbnail. Then compare it against known abuse images that have been similarly transformed.
Or they’re using AI. They might be using AI.
Either way, it’s guesswork. How many images might be transformable to the same black and white thumbnail. I don’t know.
Not true. Hash matches are to be human reviewed. So no, people won't get "swatted" accidentally as you allege.
The other concerns people have been voicing are certainly valid though (IMHO).
Until it proves too expensive, then a different AI system will do it instead. I have zero faith that it'll be a fully competent, well trained, well rested, well paid person will actually be doing these reviews in the long run.
Especially since anything flagged by this system is manually reviewed by Apple. So, there would exist counter-evidence for the govt claim.
Don’t misunderstand, I see how this system is ripe for abuse. I was just commenting on the specific claim that there will be automated SWAT call outs (presumably in the US).
The problem is that they can do whatever if you have CP. Emphasis on "have": you might not even know you have it, because all it takes for you to be guilty is for a forbidden bit of data to be on your disk or cloud account. How did it end there? Doesn't matter much. It's unlikely you'll be convicted if someone else maliciously puts CP on your drive, but it's extremely likely your whole neighborhood will know about it before it's settled. Think about that.
> They don’t need Apples system.
Apple's system makes it sure that whenever someone puts CP on someone else's account, it rings the police and starts the nightmare. Without it, there's a few extra steps that make the nightmare much less likely to happen in the first place.
The only argument I made was against the claim that fully automated police dispatches will occur because of this system (in the west).
Making outrageous, tinfoil hat level claims does not help our collective argument against this system.
It just makes us look like paranoid conspiracy theorists.
Let’s make coherent arguments (like the one you made above) instead of fantasy ones.
I mean, seriously, you're assuming as beyond obvious something that's demonstrably wrong RIGHT NOW.
So no, under the proposed implementation (and current judicial requirements in the west) robots dispatching armed police to kill pets or people without ANY human oversight because of a situation that is not time sensitive is mere fantasy.
Don’t misunderstand, I am against this whole thing for the reasons many others have well articulated in this thread.
But the claim that robots are dispatching armed police without any human interference RIGHT NOW is provably wrong.
Show me one incident where this has happened already and I will donate $100 to the charity of your choice.
And yes, the judicial rules and Apples implementation could change in the future. That’s certainly a risk but is not the case RIGHT NOW. I mean, seriously?
That said, I don’t like my phone being a snitch.
This comment suggests this phrase might be cleverly worded, to make it seem like the images are human reviewed, while that actually not being the case: https://news.ycombinator.com/item?id=28096059
I m just imagining the situation where these companies took the initiative to scan all their users data in a situation like the attach in US capitol this year. Creating new affordances for spying always leads to their abuse in the first chance when an extreme circumstance occurs. So there is no excuse for creating those affordances just "because they can"
The first seems pretty arbitrary -- why is it worse to scan files you're sharing with the cloud locally than in the cloud (except potential performance/battery impact, but that seems moot).
If Apple brings this feature to the desktop, it seems likely they'd be using it the same way: files stored in their cloud.
I think it is more similar to drugs, possesing one doesn't mean you are consuming it, yet it is an illegal substance and production, transportation and distribution are understandably not allowed.
This is a wild and in my opinion a wrongheaded analogy. Possessing CP is a crime by itself. It doesn't matter if the person possessing is actually a molester or not. It is just like the possession of drugs being illegal and it does not matter whether the person has actually taken or plans to take those drugs.
Actually it's legislators and the courts that come up with these laws and they are complicated. The question is if these laws reduce child abuse or simply increase spying, and in the end what is the acceptable balance between these two.
> if you polled the US that any sizable portion
No, but they also would agree to that they should have an option to keep their data privately. Maybe the public should be polled about this tradeoff?
For example the case of virtual CP has an interesting legal history https://www.freedomforuminstitute.org/first-amendment-center...
It is fine if this is your argument, but you don't have to wrap this argument in with the very legality of CP. You can acknowledge something is and should be a crime while also being against these type of automated dragnets to find people guilty of said crime.
But there is a clear tradeoff between the two so saying that would be a useless platitude. If we really see the internet as an extension of our vocal chords, then we should have individual rights to it, especially considering the fact that the internet infrastructure is not provided by the governments themselves, only the spying is.
The real crime is child abuse. Material related to that is also illegal because it presumably creates demand for the abuse. Whether that's actually true I don't know.
The purpose of strict liability in possession is to prevent the defense that someone does not know the legal status of an item in their possession. It does NOT prevent the defense that someone does not _know_ something to be in their possession.
For example, it is not a defense to have drugs and claim "but I didn't know they were illegal". It is a defense to claim "I did not know they were there."
In drug cases with actual possession, it is difficult to support a defense of "I didn't know they were there", which is why charges typically result in criminal liability. They drugs were physically on you, and unless you have evidence that someone planted them, it is unlikely you could establish reasonable doubt.
But in cases of electronic material for networked devices, there is most certainly an affirmative defense to counter actual possession and constructive possession. Computer devices are hacked all the time, and network & device logs exist. For example, if a prosecutor agrees to the fact that a defendant had no knowledge of the material, a judge would toss the case and a jury would not convict you. The law is not meant to pedantically convict you of non-crimes.
Does not matter if someone else put it there without your knowledge. Or maybe it matters, but then it's on you (for all intents and purposes) to prove that it was without your knowledges.
„Microsoft removes content that contains apparent CSEAI. As a US-based company, Microsoft reports all apparent CSEAI to the National Center for Missing and Exploited Children (NCMEC) via the CyberTipline, as required by US law. During the period of July – December 2020, Microsoft submitted 63,813 reports to NCMEC. We suspend the account(s) associated with the content we have reported to NCMEC for CSEAI or child sexual grooming violations.“
https://www.microsoft.com/en-us/corporate-responsibility/dig...
That's specious reasoning. Someone who posesses an action movie likes action movies, while someone who posesses child porn likes child porn. One is ok, the other is pretty vile and illegal for a reason.
I don't agree with Apple on this but let's be clear on what is and what isn't
Unless it's planted. [0] Or sent to you. [1] Or (farther out there) happens to be embedded on a site you visited and ends up in your browser cache.
[0]: https://www.nytimes.com/2016/12/09/world/europe/vladimir-put...
[1]: https://www.nytimes.com/2019/06/17/nyregion/alex-jones-sandy...
You don't have to be paying for iCloud, either. There's a free tier, so I'd imagine almost all iPhones are using some tier of it.
iCloud account break-ins aren't exactly rare. An accusation, even if false, could ruin an innocent person's life.
i was quite surprised to see this was the default or at least was setup unknowingly to me.
I know multiple people (most of them in their 50s or older) who started paying for iCloud because they thought it was their only option.
Or are you saying that malicious activity is only interesting if it was on an Apple device?
https://www.macrumors.com/how-to/see-photos-shared-with-you-...
On top of that, at this stage you are right. How long before they move it to every file in your device's storage "because of the children!"
Until people are actively encouraging people dying for people getting killed by paying for gladiator matches, I agree with you it's not the same thing, but I don't think the person you're answering to is talking about action movies.
yet
Filming an action movie does not require actually murdering people.
I'm not defending Apple. I'm saying it's absurd to act like "merely" distributing explicit photos taken without the subject's consent is a victimless crime.
That is because the consumption of child porn induces people to create it. The consumption of action movies does not induce people to murder anyone.
Some proportion of existing content has been discovered, tagged, and hashed. The legally and technically savvy members of the audience would likely know this. In such an environment, "old standard" content would be seen as riskier to hold or exchange. A completely fresh, new abuse image is the least likely to trip any automated monitoring system.
I could also easily see warez-ratio style "you've got to share something new to get access to our content" patterns, to try to discourage law-enforcement infiltration of their groups and to ensure "if I go down, I take you with me" legal leverage.
Stallman predicted a similar outcome, and although he (and many others) thought the end of computing freedom would be due to copyright/DRM, I wouldn't be surprised if "the children" is what eventually pushes things over the edge.
https://www.gnu.org/philosophy/right-to-read.en.html
in a situation like the attach in US capitol this year
...and as much as I'd like to see at least that amount of "watering the liberty tree" directed at Big Tech, it unfortunately would likely lead to even more authoritarian outcomes. Any future fights for freedom will need to happen online and non-violently, but on platforms that are also under their control.
I’m having a hard time finding a reading of this that isn’t advocating violence against tech workers. Is that what you intended?
But it is especially a stretch in the context of January 6th which involved violence directed at people. And the rest of the paragraph laments that future action must be nonviolent.
I did not find it very hard to assume a not-the-worst-possible reading of the comment. What's with the comment police here lately that's labeling various comments? I hope this ain't becoming the new twitter
It is a Thomas Jefferson quote:
“The tree of liberty must be refreshed from time to time with the blood of patriots and tyrants.”
In the context of January 6th it’s hard to make a leap to metaphorical server blood. Datacenters still have people inside them, no mention of servers was made. Only a quote from a man who waged a real war.
> tech workers are replaceable.
Speaking as a tech worker, no, my life is not replaceable.
> I did not find it very hard to assume a not-the-worst-possible reading of the comment.
The not-the-worst-possible reading being what? Mobs attacking data centers?
> What's with the comment police here lately that's labeling various comments? I hope this ain't becoming the new twitter
I feel the same way. So let's tone down the rhetoric. If this post was metaphorical hyperbole it is the kind that got a president impeached.
You can prepare for it, though. Organize some hardware while you still can, teach your children to not trust any device or service, and, more importantly keep your mouth shut. It might be hard for a typical US millennial to grasp the concept, but people that grew up in eastern Europe will be able to understand the concepts.
"In 2018, Facebook (especially Facebook Messenger) was responsible for 16.8 million of the 18.4 million reports worldwide of CSAM"
Apparently they do, yeah.
> Do we really think criminals don’t know mathematics or programming ?
Yes, by and large yes I do. Amon is a great example of this. Surely in the world this author is imagining no criminal would get caught up in a scheme like Amon since there are criminals that know "mathematics or programming", except... no one noticed/blew the whistle and instead the FBI rounded up the users of that device.
But this is the job of the police, using standard techniques to track down first, the people making and distributing this content, and then consumers of it.
It is not Apple's job to put a policeman in everyone's phone.
With lower power sensors, head-mounted devices with always-on-sensors, and whatnot, why not sample real-time hashes that can tip LEO about potential crimes happening?
Then why sacrifice recall in order to achieve high accuracy?
Err on the side of uploading more hashes. Then feed it all into a ML so it can use other data to filter out potential false positives.
Then if in a distance future, any LEO wants to investigate you for whatever reason, the set of potential hashes associated with your account will provide sufficient evidence for any court to authorize further non-hashed data access (it doesn't matter if they were all false positives)
If this is already scanning photo libraries locally, just add the temp dir for app screen captures and they're effectively monitoring your screen too.
Apple has been a thorn in the side of the IC for a long while. IC probably saw an opportunity to gain a bit of leverage themselves via the whole NSO thing, and likely offered their cyber support in exchange of some support from Apple.
I mean c'mon they've been consistently pressed by IC for tooling like what they just launched; it's the least invasive thing(compared to something like a literal backdoor like that NSA_KEY that MS did for Windows) they can offer in exchange for some cybersecurity support from the gov.
idk if that's what's happened, but it's odd Apple would do this at all, and do it right around the time of the NSO thing.
If that's true, as an iCloud user you are exactly as likely to be charged with a crime based on your photos as you were before, but you now get E2E encryption.
Obviously I'd prefer E2E without any scanning. If I wanted to upload a pirated mp3 to icloud, I wouldn't want the RIAA knocking on my door. However, given that scanning was already in place, is this a step forward?
Is this strictly true? I feel like the evidence that a photo was present on specific device is different from evidence that a photo was uploaded by a specific account (and a specific ip address probably).
It seems like it would be far easier for the government to justify a search warrant if they have evidence the photo they are looking for was on a specific device. Just having evidence that a specific account uploaded a photo seems like far shakier grounds to search a specific device, after all accounts are often stolen to be used for criminal purposes and ip addresses don't map cleanly to people or devices.
From what I’ve read the on-phone scanning only alerts after multiple photos and is designed to have a 1 in a trillion false positive rate. If the iCloud scan is similar they would have a strong case for getting a warrant based on uploads.
It's like living in the glass apartments of Yevgeny Zamyatin's We but still thinking we're preserving privacy because we put our items into an opaque box.
Hence when a Chinese photographs such brochure "in the wild" using an iPhone, someone from "the government" will knock the next day and "strongly enquire" about yesterday's photo. Likewise when a Chinese minor receives an iMessage containing such brochure.
This is just _one_ example case of "extension" of the CSAM database as seen fit by some regulatory body.
Next it's elderly people. We don't want our forgetful elders to get lost, do we? What if grandma wanders off but is in someone's picture, surely you want the police to know right that second where she is?
Next up, terrorists! Four adult brown men in an unmarked van are certainly suspicious, especially near a government building. Your Instagram selfies will help the police in the USA shoot even more innocent people for no good reason.
Animal abuse is next. You don't like puppies being abused, do you? Why do you hate puppies? Do you take part in illegal underground dog fights?
Gosh, that video looks like it might have been pirated.
Nice house, but based on your estimated income it's really strange that you have such a big television. Is that safe full of cash? How much cash is on your table?
Is that a bit of dust, flour, or maybe crack cocaine?
Is that person asleep or recently murdered?
Seemingly every aspect of digital technology, from search engines to DNS providers, has been co-opted into the fight against piracy, so I wouldn't be surprised if the media industries started threatening Apple with "contributory infringement" suits if they don't re-purpose this technology for them.
To be clear, this is continued enforcement for years-old regulation. The feature is only enabled in the US where it is required.
The implementation is changing from cloud-based matching (which requires photos to be readable by their cloud infrastructure) to local based matching with threshold tokens (which would allow them potentially to be in compliance while making the system E2E encrypted w two additional key release mechanisms (key escrow via separate audited HSM systems, the given threshold disclosure of the image encryption key)
such a long sentence to say "backdoor"
https://cyberlaw.stanford.edu/blog/2020/01/earn-it-act-how-b...
> Section 2258A of the law imposes duties on online service providers, such as Facebook and Tumblr and Dropbox and Gmail. The law mandates that providers must report CSAM when they discover it on their services, and then preserve what they’ve reported (because it’s evidence of a crime).
Apple is being its usual cryptic self about this, which is once again breeding uncertainty, but I still have hope in the end this will work out.
However since this is part of the opt-in iCloud photos sharing mechanism, it doesn’t appear they have started to exfiltrate data without consent.
I don't in know why you're assuming they have a backdoor built in already. The whole point is that they don't, but they're going to add one.
So they have always been one software update from non-opt-in data exfiltration and remain so.
Aka not E2E and therefore something that Cook should face a fraud charge for saying it is.
Apple needs to make it true e2e yesterday, and tell the FBI that they can either approve of it, or never use an iPhone again.
Apples system is not E2E encrypted if a local program scans files prior to upload.
Apple caved to pressure and had to implement this.
Whatever the angle, this isn't about protecting kids whatsoever. It's about power.
In other words, the author thinks with Apple’s back to a wall, they only needed to make the announcement of this feature for the government to see there are advantages to apple having tight control as well. Now they’ll be able to make that very same argument in court in a public sense, but there’s always a behind the scenes sense with 3-letter agencies as well.
Granted all of that is speculation and who knows what is really driving any of this. The author does have a point that if this first step causes bad guys to move on from these services then that will be future justification to move the scanning further and further upstream to the point where it’s baked into the API’s or something. At that level, Apple would really need a “monopoly” to accomplish such a feat.
It’s certainly an interesting and creative perspective.
But even in the best case, exists then worst abuse cases. That’s the problem. This WILL be abused.
It's time to start dismantling massive chunks of the intelligence community. It no longer works for the citizenry that's supposedly their bosses. (If it ever did.) It's become a power blackhole unto itself.
Even elected officials, up to POTUS, have found themselves unable to control the unelected and unaccountable fiefdoms that make up the intelligence community.
The problem isn’t with them, it’s with the loopholes that let them pay less than people would like.
The IRS can’t do anything.
1. Will the scanning come to MacOS? 2. Will the scanning start to include additional $BAD_STUFF, such as political censoring and/or even other files (video, document, etc.)
I really like Apple hardware. The iPads and the M1 Macs are awesome, but this news makes me hesitant to stay in the Apple ecosystem and will be looking at alternatives. I already run Linux desktops, and I’ll probably move to Linux on laptop.
Also the whole SWAT scenario is a bit far-fetched, as they will most likely read thru your entire life (don't forget they already have access to it) to make sure they don't look stupid on the news.
I am concerned about the slippery slope of if they start scanning files that were never going to go through their servers in the first place though.
Only, my daughter was included in the montage. Why?
She was wearing a shirt with a cat on it.
We are headed for a China style surveillance state and there is no stopping this train.
Maybe if you take pictures of Trump signs...
Maybe if you take pictures of Red states...
Maybe if you take pictures of cis relationships...
Maybe if you happen to be male and white and take a selfie...
What/when do these get reported to the FBI or other woke agencies in government?
It's a very slippery slope AND WE KNOW there are sociopaths in positions of power who'd gladly do such things without a second thought.
I mean this is someone who would know what being on the wrong side of law means - not only federally, but probably quite intimately since Alabama was not exactly known for its acceptance of homosexual people.
Now just imagine if the US still had anti-homosexuality laws (like the majority of the world still does) and your phone is constantly scanning your photos to check for signs of homosexual behaviour. Forgetfully take a selfie with your boyfriend, it gets flagged, sent to the Ministry of Morality, and next thing you know you're being dragged into a van. Best case scenario you're being jail. Worst you're being thrown off a building, stoned, or brutally beaten to death.
That's the future Apple is signing us up for. There is zero chance this stops at CSAM, especially with the Democrats convinced that half the country are the absolute worst people on the planet and not being shy about completely ignoring the rule of law and Constitution to extend the reach of the state to levels that would make a totalitarian blush. This will end terribly.
Except that’s not what this is doing.
For all intents and purposes, “scanning” is just hashing content before it’s uploaded and comparing that hash to a known database. So, unless your picture had been hashed and flagged before (And if you just took it how could it have been?) then there’s nothing for it to match.
Don’t just conjure FUD from a contrived worst-case scenario.
The exfiltration mechanism is the problem. If this were saying a hash match could be used to obtain a search warrant, that would be bad, but it would be far less egregious a breach of security and privacy than Apple adding a backdoor to just grab anything it likes.
Apple will already have the data. You’re already consenting to that when you enable iCloud Photo Library (the only place this is being implemented).