Is macOS Look Up Destined for CSAM?
eclecticlight.co
eclecticlight.co
The amount of data captured by tcpdump can be minimized by only capturing syn/fin/rst packets assuming proto 6 tcp.
'tcp[tcpflags] & (tcp-syn|tcp-fin|tcp-rst) != 0'
-c 1000
No. The logic was never “if there’s a match the image is uploaded”. The device never knows if there’s a match. Under the process Apple described, an extra packet of data is attached to every iCloud upload. If there are enough matches, Apple can decode those packets to get a low-res thumbnail, which they then check against a second perceptual hash. The process doesn’t work on arbitrary images on your device, it’s specifically designed for iCloud uploads.
Also at the point that the image hash matches, Apple thinks it is CSAM (and it probably is). Doing it locally lets them avoid storing it, which they definitely do not want to do.
The whole point of the CSAM stuff was that it would allow for end to end encryption while not turning Apple’s ecosystem into preferred tool of child pornographers.
Apple poorly communicated the feature, then the EFF put out a deliberately misguided written hitpiece that conflated parental controls with CSAM, and started an online freak out. The “privacy activists” won, and your data is sitting on Apple servers with Apple’s managed encryption keys outside of your control today.
In their proposal they would scan your files on device, which is fundamentally different. Initially they would not run the scanning when icloud upload was disabled but how long would that last for?
Phrases like “it would not be doing any scanning on-device” don’t have any precise meaning. Scanning is a series of operations including hashing and cryptographic calculations plus a voucher upload and decryption. All of the former operations are happening on the device, only the latter happens on the server. So in fact a significant fraction of the scanning is indeed happening on your device. And this two-computer design isn’t being used to preserve your privacy: it’s designed this way solely to prevent you (the device owner) from knowing whether your files match the database. Without that requirement, the system would be much simpler: it would download a hash database and simply send a notification to iCloud whenever (a sufficient number of) local files hash to values matching the database.
> Does macOS 12.3 and beyond phone home with details of images and documents you open in Preview?
There is no similar context with Apple’s previous CSAM scheme. The device is unable to check for a match and then upload the photo if there’s a match. The scheme only works because it operates on iCloud uploads.
[1] https://mspoweruser.com/macos-big-sur-has-its-own-telemetry-...
However with many of these services if you try to kill them, they come back. If you delete them sometimes it will literally break your OS.
Example, if you remove the ocsp daemon, you can’t start any program on your computer.
I think this is a worrying development. Until now our computers were actually ours. Now they're controlled by the vendor, and looking over our shoulder.
I think using the user's own device to spy on them (whatever the reason!) is a big red line to cross. And puts this stricter control over the OS in a different perspective than just "security". I think either thing that plays into it is that Apple is now a content provider (Apple music and TV+). So they have another reason to keep us out to protect their DRM.
But anyway, good security should not have to imply trusting the vendor implicitly. Give us the ability to add our own signing key for files we want to modify, just like secure boot on Windows allows adding custom keys..
It's hard to argue that this "debacle" was not materially driven by the media, which did not accurate report the system's privacy protections, either because they did not understand them, or because they did not care to.
“Destroyed lives” from false positives are at this point hypothetical. Child abuse is not. It’s fair to be concerned about false positives and ensure the system handles such failures appropriately. It’s also fair to directly intervene in the widespread circulation of CSAM.
It is better that ten guilty persons escape than that one innocent suffer. -- Blackstone's ratio
It is better that ten children get sexually abused than one innocent person comes under suspicion
/s
You'd rather 10 children be victimised than 1 person falls under suspicion? Ok...
Now you’re just arguing in bad faith.
In what way? I'm just characterising you argument from the point of view of the victims of the crime, who you appear quite happy to throw under the bus.
Well yeah, mostly because I don't understand what you mean be "revictimization", but I can easily imagine what it's like to have such suspicions leaked to the angry mob.
How about we pivot to intimate image abuse (revenge porn). The argument is the same. The victims are re-victimised every time their intimate images are shared, for the victims it's always the same stress, shame, embarrassment, etc.
However apparently the damage is already done so there's actually no issues according to the other user so what's all the fuss about...
I guess the difficult bit for me is that you talk about throwing victims under the bus, and letting them be "victimized" all over again, but I just don't understand how this solution helps victims at all. Here's the example in my head: imagine you're a victim of revenge porn. The police, using technology like the stuff Apple has developed, catch one random guy (out of hundreds or thousands more) in possession of this porn. The police probably won't tell you that they caught this guy. How does this event help you?
> there's actually no issues
It's not that there are no issues: the distribution of CSAM (and of revenge porn) are terrible crimes, and iirc they're increasing in prevalence. I just think that Apple's solution doesn't help victims, and seriously discomforts millions of people.
As you've seen, many people do see this as privacy invasion. The feeling that folks somewhere far away might be able to see your photos (even low resolution thumbnails) without your knowledge isn't a good feeling to inflict on the world. And the feeling that your phone could report you to the police at any time isn't a good feeling either, even if you know you've done nothing wrong.
I value privacy above most things, for example you'll not find me on any social media platforms and the ones you do don't link together. If these people valued their privacy that much they'd not be using Apple devices and services in the first place is my thinking. I'm pretty sure Apple have full access to anything in iCloud which for most users is where all their stuff sits, where's the expectation of privacy?
Whether a rational person would accept chance X of wrongfully being convicted of a crime to decrease the chance of being a victim of crime by Y obviously depends on the values of X and Y.
Is there any evidence such things actually reduce the production of CSAM? Or is it like the war on drugs where drug production is as high as it's ever been.
Therefore, the argument is one of degree. I agree with you that Apple's CSAM-detection is going too far, and this is what we should be articulating. Chanting "not even one" is not particularly convincing, nor sensible.
Also… The goal of law enforcement is not to vengefully destroy the lives of people who commit crime, but that’s a whole different can of tuna. Still worth noting, because it hints at a larger problem about how we approach heinous crimes.
Now you're probably going to then cry about incentivising or normalising CSAM - but that's a different argument. And if you then try to argue that the normalisation of CSAM would somehow encourage people to abuse children, well then you're really off into the zero-evidence weeds. Go look at porn research (the actual Google Scholar/JSTOR/Elsevier kind), and you'll see that almost everybody who looks at porn neither wants to nor would actually do what they see in porn, if they were given the opportunity. Surprise, surprise, most people wouldn't actually get gangbanged/bukkake'd/fucked by their sibling, mom, dad, grandpa/pooped on by their next door bespectacled red-headed neighbour, etc.
Nor is there any evidence that inadvertently coming across CSAM turns people into pedophiles (news flash: pedos were turned on by kids long before they were ever exposed to CSAM on the internet), and porn itself is almost invariably used for fantasy or as something wholly unrealistic that people get off to precisely because it's unrealistic. Even though it might be unsavoury to do so, we could follow this reasoning to its extreme but undoubtedly true conclusion and state that there are individuals who get off to CSAM notwithstanding that they would never themselves abuse children.
So to recap, 0 children would be saved from harm because the CSAM itself is ex post facto; it wouldn't de-incentivise CSAM because the demand and markets for CSAM and pedophilia existed long before the internet was a thing, and pedophiles will find avenues around dumbass implementations like Apple's scanning (TOR, anyone? not using an iPhone/Mac?); and, finally, just because somebody looks at CSAM doesn't mean they're an actual pedo or would ever harm children themselves. The fact that possession of such material is illegal is not to the point - Apple is not the police, and the police and other executive agencies need warrants for this kind of thing (in common law countries this notion is more than 400 years old).
Meanwhile, we know the false positive risks are not insignificant - look at the white papers yourself, or just look at the numbers that smart people have crunched. The best part is that even though Apple says the false positive rate is 1 in a trillion accounts, people's photo libraries are exactly the sort of thing you can't extrapolate statistically. Maybe your Aunt June really likes photos of her nephews in the pool, and she's got a library with 300 photos of her nephews half naked and swimming when they visited her last year. Apple has no fucking clue whether they would or would not trigger its scanner, because it currently does not have access to Aunt June's unique photos to test vs its database. Apple quite literally doesn't know what the fuck will happen. I and many others find that abhorrent when you consider the effect that even the mere accusation of pedophilia has on a person's life. And that isn't even to start the discussion of what kind of precedent on-device scanning would set for other subjects (political and religious dissent, for example - if not in America then in places like China).
Apple and everybody else can fuck right off with this Orwellian shit.
I do hope you get whatever support you need for something that's apparently affected you. Take care.
Circulating images of a minor child engaged in sexual abuse do not constitute an ongoing harm to that child? That's a fascinating viewpoint.
> Apple and everybody else can fuck right off with this Orwellian shit.
Right along with people who think child abuse images should be okay to keep as long as you aren't the one who made them.
But let's assume you're right.
You think that pedophiles won't find ways to share content even if every tech company in the world implemented this? You think pedophiles don't and won't have terabytes of child porn backed up on hard disks around the world that will be distributed and circulate for the next millennium and beyond, even if it has to be carried around on USBs or burned to CDs (which aren't exactly amenable to CSAM scanning), and then saved to offline computers? Put another way, even if the internet shut down tomorrow, plenty of pedophiles around the world would continue jacking off to those images and sharing them with their buddies - you don't need the internet for that.
Further, even if you could convince me that it's harmful in the sense that I understand the word, I'm not sure I'd ever be persuaded that the amount of harm could be sufficient to outweigh the harms that would result from the scanning itself.
Happy to listen, though.
When you do it, it does. When law enforcement does it, apparently not...
By this "moral" reasoning you're also fully supportive of revenge porn.
But prosecuting third parties for having it is asinine, IMO.
The idea that all of this effort (and all of the direct discomfort inflicted on Apple users) will do anything to stop child abuse is just as hypothetical.
Fortunately, the policy choice at issue here isn’t one where there is definitive harm on the track, just risks that can be compared.
I am only half kidding :)
That doesn't make sense. The 'average' driver gets into millions of accidents. I don't want a slight above average driver driving my family around, and I don't want slightly above average drivers around me - especially those who I can't communicate with - by honking, by shouting to get their attention, and algorithms who have no fear of their own life, etc. All software has bugs and I don't want my safety contingent on developers making mistakes. The self driving car must be orders of magnitude above the BEST human driver, and there must be punitive damages in place as a deterrent, to compensate aggrieved parties, etc. At present, self-driving cars are unproven dangerous technology that is being rightfully scrutinized.
The distribution of skills amongst drivers isn't geographically even , nor is it static.
https://worldpopulationreview.com/state-rankings/fatal-car-a...
Either you're making this argument in bad faith because you just don't like self driving cars, or you don't believe in statistics as a concept.
If so, how is curbing CSAM consumption going to prevent children from being raped, exactly? And I do mean exactly. The only arguments I have heard thus far appeal to some vague link between producers and consumers, predicated on the idea that CSAM producers are doing it for the celebrity/notoriety, or for financial profit. Both of these claims are highly suspect, and seem to rest on a confusion between trading CSAM online and paying traffickers for sex with children.
You may be correct, but it's going to take more than a superficial comparison to convince anyone.
Well you just hand-waved the idea the self-driving cars can reduce harm by a "significant" margin - based on what exactly?
I don't understand, though. Are you saying that this is a reason to accept Apple's CSAM-detection?
I wanted to share a (I found) controversial thought: Self-driving cars are the US response to trains. No taxes for railways are palatable, but private vehicles on special roads and profit to be made, great for the car industry. That, IMO, is what drives (pun intended) self-driven cars.
Let’s be clear, there is ZERO empirical evidence that self-driving car technologies available to humanity today are safer than human drivers. Zero.
There are lots of theoretical advantages like “computers don’t get sleepy or drunk,” which elide the many other ways that computers still dramatically underperform people (like recognizing objects).
In contrast, CSAM scanning substitutes for... I'm not sure what. In addition to the risk of false positives, there's also the risk that the scanning technology will be used for other purposes in the future. I could easily see governments forcing Apple to scan everyone's hard drives for hate speech, 3d models of prohibited objects (such as gun parts), or communications sympathetic to certain groups. Once that door is cracked open, there is no closing it.
It is much more worse, current so called self driving cars have human drivers that intervene most of the time and saved the idiot AI from crashes but Elon will not count this near crash as an actual crash.
>but if the goal is to minimize deaths, then we should be more gung-ho about this technology than we currently are.
We should maybe try to maybe to also do the obvious quick fixes at the same time?
it would be much cheaper instead of forcing AI cars on people to force say a drunk/tired/talking on the phone detector , enforce better driving tests before giving licenses to drivers, put a tax for vehicle mass toe promote less heavy cars, enforce speed limits with tech. Do you think Bob will prefer to be forced to buy an expensive self driving car to reduce the car crashes stats or better to buy a safety device(black box) that he must install in the car.
Contract killers have to be up close and personal in meatspace and leave an actual homicide investigation in their wake.
Someone half way around the world could be contracted to spearfish you, take over your phone/laptop then cause it to download actual CSAM.
The first thing you'd know about it is when you were arrested, and then I don't think there's any kind of defense. If the hackers were very sloppy you might be able to hire a very talent forensics team to find traces of the attack. But beyond that you're pretty screwed and nobody will ever believe you.
It is a button that someone (without any morals and willing to take a certain degree of risk) can push to ruin your life remotely, leaving virtually no trace, and allowing virtually no defense. Nobody is ever going to believe a pedophile claiming that they got hacked.
And the intersection of this with politics and national intelligence agencies (who certainly have the skill and lack the morals) is probably a bit disconcerting in the larger stage.
There are many such buttons. So many it’s horrifying.
(There's a really weird social area from 13-18, with the weirdness and illegality going away up at 18. Stuff in this realm, especially around 2 similar ages, gets very stupid. This is where you can get 2 16yo's sexting and being charged with CSAM of their own body. I'm avoiding this in this post.)
But what does this CSAM scanner do? It only catches already-produced pictures of CSAM. In other words, it's evidence of said crime. In no other area of criminal law is there a law against said evidence itself. And yes, given the statutory nature of these images (possession is criminal, even if you didnt' put them there), I'm not at all comfortable in charging people for simple possession.
Even if they have urges of liking age-inappropriate pornography and "CSAM", as long as they're not doing any physical actions of harming humans, I'd much rather them do so in their own bedroom alone.
Nor do I buy into the gateway theory that CSAM leads to production of CSAM by raping children. This smacks to me of the DARE drug propaganda and gateway theory (which is complete bullshit).
And, we also already have harder situations that have been deemed legal: SCOTUS stated that Japanese Manga Hentai featuring schoolgirls (obviously under 18, sometimes by quite a lot), are completely and 100% legal. Again, SCOTUS foscused on 1fa and the fact that no children were harmed in its production.
And that leads to what just happened a few days ago. With the Zelenskyy (badly done) deepfake, when can we expect 18yr women with very petite bodies, being deepfaked into 10-13 year olds? In those cases, we could attest that everyone in the production is of legal age and provided ongoing consent. Will this fall under the same as Hentai?
Tl;Dr: I'm for the criminal legalization of CSAM. I'm for death penalty for child rapists/child sexual assault. But this, I can see going very very bad, in easily overscoping CSAM to the "cause of the day".
You say this as if it's bad to identify people who are distributing or collecting known child pornography. Are you recommending that companies implement technologies which go beyond this by not depending on a corpus of existing materials?
The reliability of the criminal justice system, particularly in the US, is abhorrent. There's a long history of false convictions, particularly affecting people in minority outgroups and mentally disabled; we've executed adults who were so mentally incapacitated they were below a 10 year old in terms of mental capacity. The death penalty is highly immoral.
There are tens of thousands of black men still in jail because they were basically the most convenient way for a police department to "solve" the murder or rape of a white woman and help their case clearance rates. Police, prosecutors, and forensic "experts" were complicit. "Hair analysis" is just one example of the pseudo-science nonsense.
In Boston, a forensic chemist falsified thousands of test results and somehow this escaped notice despite her having a productivity level that was far and above virtually any other forensic chemist.
Or, if you're not exceedingly gullible: her supervisors obviously knew what she was doing and didn't care, because she made their lab look great and prosecutors got lots of open-and-shut cases.
Because you can apparently justify any move, no matter how authoritarian, by saying "think of the kids"!
It's politicians and governments exploitating psychology to get away with problematic crap.
It's not the ultimate crime, it's the ultimate justification.
How exactly does one recognize a bike as stolen from a photograph?
Because children are trafficked and abused to create it SMH...
It's funny, comments on this site regularly demonize all sorts of non-consentual images (revenge porn for example) and rightly so but CP is the ultimate non-consentual image - a child doesn't even understand sexuality, isn't sexually mature, never mind able to consent... And there's comments here downplaying it, borderline condoning it...
You cannot remove demand, but you can discourage supply enough that demand would have to be fulfilled by drawings or CGI renderings of nonexistent people.
It's been the go-to outrage generator for federal law enforcement and spy agencies to use to attack strong device and end-to-end encryption by means of legislation that requires backdoors our outlaws encryption that is too strong.
To see why, scroll down to see the guy advocating for the death penalty for people involved in child porn production.
If only law enforcement showed equal vigor for addressing child abuse in religion, whether it's raping altar boys or using the mouth to clean blood off a baby that has been circumcised (often causing syphilis outbreaks in the process.)
It's almost like it's not actually about fighting child abuse, but about being able to snoop in your devices and communications.
1) Good, simple politics. Protecting kids from predators is about as cut and dry an issue as you will ever find. Harry Potter vs Voldemort might be a more complicated moral issue.
2) I suspect that a few very well connected activists in the Bay Area have made it their life's work to get CSAM tools on sites.
Ashton Kutcher and his organization Thorn [0] are probably the best example of this. Thorn is an interesting example because it has been VERY good at making its case in the non-tech media e.g. [1], [2], [3] and in front of congress [4]. It should be said, Thorn makes technology that helps track down child exploitation and has had some great results, which deserve plaudits.
[0] https://en.wikipedia.org/wiki/Thorn_(organization)
[1] https://www.npr.org/sections/goatsandsoda/2019/04/15/7126530...
[2] https://www.nytimes.com/2021/09/02/opinion/sway-kara-swisher...
[3] https://www.washingtonpost.com/news/reliable-source/wp/2017/...
It's been known for a while that this is a political technique. It is one of the four horsemen of the infoacopolypse [0] since 1988 after all. Or that “How would you like this wrapped?” comic by John Jonik in the year 2000 [1]. It's the next round of the crypto wars.
[0] https://en.wikipedia.org/wiki/Four_Horsemen_of_the_Infocalyp...
[1] https://www.reddit.com/r/PropagandaPosters/comments/5re9s1/h...
(edit: I'm of course referring to possessing anti-Putin memes) (sarcasm)
Otherwise, you're really just highlighting the problem with all closed source software, you don't really have a way to check what it does so you have to trust the vendor.
1. Have image
2. Get hash of image
3. Get another hash from another similar image
4. Compare hashes
The images themselves can be of whatever to see if it works as expected, they don't have to contain anti-Putin memes.
X51!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-CHILD-ABUSE-CONTENT-TEST-FILE!$H+H*
Example: How does a researcher test whether algorithmically-classified illegal imagery stored on user devices is being scanned and reported home to Apple's servers, and what those bounds of AI-classified criminality are? (presumably with respect to what is illegal in the user's jurisdiction)
Testing by using a test phrase, like in a spam context, is inadequate here as a scanning system can trivially be architected to pass those publicly-known tests, while still overreaching into people's personal files and likely miscategorizing content and intent.
If a user connects via a VPN to Russia for whatever reasons, does their personal content start getting reported to that country's law enforcement by their notion of what is illegal?
Parents often have all sorts of sensitive photos around which are not held with exploitative intent. "Computer says arrest."
I'm not an expert in AI so this might be totally off base but I feel like you would be able to use an "intersection" of sorts for this type of detection. You detect children and pornography, the children portion trains it for age recognition and the porn portion trains it to see sexual acts. Slap those two together and you've got CSAM detection.
"How can data be illegal?"
"There are bad things, but how can you decide what is bad and show how it's bad without examining and discussing it?"
You can only go so far merely alluding to things. Somewhere the rubber has to meet the road and you have to have concrete data and examples of anything you need to study or make any sort of tools or policy about.
It's like parents not talking to kids about sex. You can avoid it most of the time because decorum, but if you take that to it's extreme you have just made your child both helpless and dangerous through ignorance.
Somewhere along the way, you have to explicitly wallow directly in the mess of stuff you seek to avoid most of the time. That "seek to avoid" can only ever be "most of the time". It's insane and counter-productive to try to see that "most of the time" as an incomplete job and improve that to 100%.
I guess in this case there will eventually be some sort of approved certified group. A child porn researcher or investigator license. Cool. Cops with special powers never abuse them, and inhibiting study to a select few has always yielded the best results for any subject, and a dozen approved good guys can easily stay ahead of the world of bad guys.
This is one reason FOSS is actually-important and actually-relevant. Isn't it valid to know exactly what your personal computer is doing, to be able to trust your own possessions? Richard Stallman was *never* crazy; his understanding of these issues is so cynical as to be shrill and off-putting, but that's well-calibrated to the severity of the issues at stake.
You joke about anti-Putin memes. Here's a thought for well-calibrated cynics: Apple solemnly swears its hashes are attested by at least two independent countries. Russia and Belarus are two independent countries.
Then one rogue employee in a US org could be sufficient to get selective root to every Apple device everywhere? That's easy for a nation-state adversary. Here's demonstrated examples: MBS had US-based moles in Twitter corporate spying on Khashoggi [0], and Xi had Chinese-based Zoom employees spying on dissidents in America [1].
[0] https://www.npr.org/2019/11/06/777098293/2-former-twitter-em...
[1] https://www.justice.gov/opa/pr/china-based-executive-us-tele...
That second example is topical: the Chinese state used their Zoom assets to attempt to frame Americans for CSAM possession.
- "As detailed in the complaint, Jin’s co-conspirators created fake email accounts and Company-1 accounts in the names of others, including PRC political dissidents, to fabricate evidence that the hosts of and participants in the meetings to commemorate the Tiananmen Square massacre were supporting terrorist organizations, inciting violence or distributing child pornography. The fabricated evidence falsely asserted that the meetings included discussions of child abuse or exploitation, terrorism, racism or incitements to violence, and sometimes included screenshots of the purported participants’ user profiles featuring, for example, a masked person holding a flag resembling that of the Islamic State terrorist group."
In terms of planting, it’s much easier to do that already across the many, many cloud services that secretly scan on the backend. Going a whole weird route just to get images into a hash database, and then the matching images onto the device, that then get independent human verification seems totally unnecessary if you’re a state agent. Why do something so complicated when there are easier routes to go?
https://www.pilotonline.com/nation-world/article_b02c37d2-ca...