Hacker News Reader

Comment by xt00 | Hacker News Reader

Anything that includes client side scanning is a slippery slope to fully controlling your device. Will it be illegal to somehow disable the client side scanning? If so then how long until you are breaking the law when you turn off the government scanner — or are caught “installing a new hard drive” in your computer.. etc..

Is the problem that people can send encrypted things back and forth to each other? Requiring that companies put snooping software on their device is basically the thought police. Not hyperbole but the actual thought police. Today it’s saving the children, tomorrow it’s basically any problem the governments of many nations want to try to solve.

View on HN

> Will it be illegal to somehow disable the client side scanning? If so then how long until you are breaking the law when you turn off the government scanner

And once they've normalized "your computer will spy and inform on you", is there any reason to think that won't expand to things which aren't colloquially "computers" but in fact are now computers?

What about "smart houses"? All your IoT toys are computers. Once phones, laptops and PCs as mandatory reporters has been normalized, is there any reason to think all the other microphones and cameras already in people's houses won't become mandatory reporters too? If they make it illegal to disable client-side scanning on computers, might they also make it illegal to remove the crime-detecting cameras in your own home?

Modern cars already narc on people, logging and uploading GPS traces that can be fed into police dragnets, just like phones. Cops can ask for a log of who's been inside a 'geofence' and where does that data come from? Phones and cars reporting on their owners, generally without their owners knowing anything about it. The 'slippery slope' isn't actually a fallacy if you have enough datapoints to legitimately draw a trend line. And I think we certainly do.

My friend, these are dangerous thoughts! Once we have our wifi-enabled brain interfaces you will get punished for thinking this. Please train yourself to forget about it now before you get in trouble.

But will we be punished for thinking it? Or will the chip simply cut off such trains of thought in the first place? :) "Revolutionary new brain implant improves mental health by suppressing damaging* trains of thought"

The former, in a couple centuries. Centralized monitoring of common thoughts will be a solved problem, and the means to circumvent it won't be known by many. This ideal oppression machine will last for a few centuries.

There have already been experiments in altering personal biases and beliefs through transcranial magnetic stimulation [0][1]. Direct electrical stimulation will eventually permit a greater degree of control.

> We presented participants with a reminder of death and a critique of their in-group ostensibly written by a member of an out-group, then experimentally decreased both avowed belief in God and out-group derogation by downregulating pMFC activity via transcranial magnetic stimulation. The results provide the first evidence that group prejudice and religious belief are susceptible to targeted neuromodulation, and point to a shared cognitive mechanism underlying concrete and abstract decision processes.

[0] https://pubmed.ncbi.nlm.nih.gov/26341901/

[1] https://www.sciencedaily.com/releases/2015/10/151014084955.h...

Show us E-papers please

Can you provide a source for that last paragraph (the cars uploading GPS traces and cops asking for logs)? I'm interested in knowing more, e.g. which country this happens in, are there any checks and balances, is this constitutional?

In America:

Cars logging their location history and police getting that data: https://www.forbes.com/sites/thomasbrewster/2021/04/01/these...

Geofenced dragnets: https://harvardlawreview.org/2021/05/geofence-warrants-and-t...

Not giving owners (or their lawyers) access to data on what the Autopilot thought: https://eu.detroitnews.com/story/business/autos/2020/02/26/c...

Look into OnStar.

In Orwell's 1984, the TV was watching you.

Worse than that.. the billboards were watching you.

I mean go look at modern digital billboards, a bunch of 'em have cameras built in.

yeah, it's a bit creepy how on the nose 1984 was. maybe Orwell was a time-traveler.

I remember talking to my dad about 1984 -- the book not the year -- and being somewhat surprised he didn't find it to be either prescient or honestly much of an interesting book in any case. While when I read it it definitely filled me with a sense of dread. I suppose "in his day" much of the predictions had not yet come true, and he didn't quite grasp how now they most definitely have.

> The Jews will hide behind the stones and the trees, and the stones and the trees will say: Oh Muslim, oh servant of Allah, there is a Jew hiding behind me, come and kill him.

Eerie, isn't it?

The only client-side scanning proposal we’ve ever seen (Apple and NCMEC’s 2021 photo scanning proposal) didn’t even address encrypted messaging. It worked on private photo libraries on your phone. I think it’s very important to reiterate that the targets here aren’t communications between criminals: it’s your private data.

Not the only one. At the same time, they announced a separate scanner for incoming inappropriate photos over iMessage as a parental control feature. Unlike the photo library scanner, this one actually got released. https://www.apple.com/child-safety/

That feature did not report users to the police. It did notify parents automatically, however, if you were a child in a family account. After some feedback from child protection advocates they removed the mandatory parental notification and made it a button that the child has to press.

The police report was the wackiest part of the other feature, yeah.

It would scan photos you were uploading to iCloud, not private photo libraries on your phone. I'm sure you'll agree it's important to correct such a misunderstanding as one of those is a lot more invasive than the other.

Yeah, important to mention that. Still, I don't want my phone to even be capable of doing that, nor do I see the reason behind it when iCloud could just do the scanning itself. That's one big step closer to the described full-private scanning (and just a flag flip away).

The supposed argument was that they wanted to keep the scanning they do in iCloud now (I believe they do it) and yet make iCloud encrypted so that they can't see the images once they leave your device.

So they move the scanning to the device.

Did Apple actually say they wanted to do e2ee iCloud photos when they announced CSAM scanning, or were people only speculating this? I don't remember / can't find an announcement on that. Also curious if there's some law preventing them from doing e2ee without the scanning.

I believe it was speculation based on them saying they wanted e2ee (and now it's available IIRC).

It honestly seems to me like they thought they could negotiate a middle ground without pissing off the Feds or the customers, but they maneuvered it quite badly.

> now it's available IIRC

Oh cool, didn't know that. It's this new "advanced data protection" feature that makes everything in iCloud e2ee except the classic mail/contacts/calendars combo that wouldn't really work with that. https://support.apple.com/en-us/HT202303 is a nice resource on this, and I wish more companies would publish things like this.

It's a technologically impressive feat, and honestly they've done it well.

I'm a bit too chickenshit to try it, as losing my devices is all too likely, but I'm glad it's available for those who need it.

> "Still, I don't want my phone to even be capable of doing that,"

Not capable of what, running software? Communicating with a HTTPS endpoint? Having library code? Running stuff in the manufacturer's interest rather than your interest? All those things happen already in some form or other, and there isn't a cutoff to make the phone incapable of it without hobbling the phone.

> "That's one big step closer to the described full-private scanning (and just a flag flip away)."

iPhone already does scan offline private photos for face and object recognition purposes. And run big blobs of unknown Apple-provided code. It's only your trust in Apple that makes you think it doesn't report anything back now - and nothing at all stopping them from being arm twisted by the authorities to make that scan for something the government dislikes and report on it, as you say a flag flip away. It already does send your location and your surrounding WiFi signals and your voice when you use Siri unless you toggle the privacy settings, and that all came in quietly on regular updates.

Apple walked a fairly narrow line when they announced it, and when they publicly stated that if the authorities asked them to extend the scope of the scanning that they would refuse.

I don't know why they chose to do it on the endpoints rather than in the cloud, but acting like doing it on the cloud would give you any level of protection from them putting intrusive software on your phone is not reality. (Same with Google, Samsung, et al).

> Not capable of what, running software?

Not loaded with trained models on illegal content and wired up to alert the authorities if it finds a match, with presumably several teams within Apple built around that feature. I'm thinking about more than the technical aspects of this.

> It's only your trust in Apple that makes you think it doesn't report anything back now

Yeah, exactly. I trust them enough right now to run tons of stuff without my knowledge on my phone. I don't have the time or knowledge to audit my phone, even if it were Android. If they announced that new feature is going live like it's a thing customers are meant to be ok with, I'd trust them a lot less.

It would start at only scanning content that was going to be uploaded to iCloud. There's literally nothing stopping the process from scanning all images whether they're going to be uploaded to iCloud or not. Such an expansion would use the exact same justification as the iCloud-bound content scanning.

It's a slippery slope that ends up with your phone/computer snooping on texts, call contents, or anything else and then submitting your "crimes" to the authorities.

They could have designed it to do that in the first place, and not announced it at all just hid it away in a point release. They didn't.

> "Such an expansion would use the exact same justification as the iCloud-bound content scanning."

One of the justifications was that Apple are/could be legally responsible for criminal images hosted on their servers, that exact same justification wouldn't apply to offline content scanning.

> "It's a slippery slope that ends up with your phone/computer snooping on texts, call contents, or anything else and then submitting your "crimes" to the authorities."

Not "crimes" in quotes, crimes without quotes. Generally people think law enforcement is important, especially regarding crimes against children. "A slippery slope" which leads criminals being punished is not the argument winning logical fallacy you think it is. The argument against it is around invasion of privacy, rights not to self-incriminate or to remain silent, ownership of device and software, freedom from unreasonable search without prior evidence, whether you can be found guilty by algorithm, et al.

> Not "crimes" in quotes, crimes without quotes. Generally people think law enforcement is important, especially regarding crimes against children. "A slippery slope" which leads criminals being punished is not the argument winning logical fallacy you think it is.

You're missing my point. Content scanning will start with the Four Horsemen of the Infopocalypse[0]. Then it'll move on to "crimes" like blogging about a public figure[1].

[0] https://en.wikipedia.org/wiki/Four_Horsemen_of_the_Infocalyp... [1] https://www.flsenate.gov/Session/Bill/2023/1316/BillText/Fil...

> It would scan photos you were uploading to iCloud, not private photo

There was no way to separate the private photos from iCloud-uploaded photos. It was all-or-nothing, like Android permissions: “Allow govt to scan all your private pictures, or do you wish to have no backup?”

I was perfectly feasible to design the ability to have private photos, but Apple chose not to. Or Apple, in collaboration with the government, chose not to.

The wording "scan your phone" misleads by implying that the photos are scanned because they are on your phone, when really the photos would be scanned because you sharing them with a third party, by the third party. Yes it was all-or-nothing, what it wasn't is all-, there was a nothing and that nothing would have kept your photos private.

(Yes it was feasible to design the ability to have a local photo store which isn't uploaded to iCloud, separate from other photos which are, and call it "private photos", but that's another matter).

If you have iCloud backup setup, does it upload all photos from your private photo libraries to iCloud?

If you turn on ADP those photos will be encrypted with a key Apple doesn’t have. They will still be private photos, of course. Even turning on unencrypted backup doesn’t make my private photos not-private.

You say that sharing your photos, unencrypted, with a third party you don't trust, doesn't make them "not-private" and you accused me of twisting language.

Yes there can be private conversations between two people, but there can't be private conversations between two people where one of them isn't trusted but can hear the conversation and simultaneously, what, can't hear it to keep it private?

It's already happening, and scans even with iCloud turned off: https://sneak.berlin/20230115/macos-scans-your-local-files-n...

This is a person who is linking to an article that literally says the opposite of what they are claiming it says.

> The media erroneously reported [a statement from September 2021] as Apple reversing course.

Linking to an article from December 22, 2022 in which Apple is quoted as literally saying “We have further decided to not move forward with our previously proposed CSAM detection tool for iCloud Photos.”, ie actually and literally reversing course.

If this person cannot comprehend an article, should I trust that they actually checked that what they were seeing was what they were paranoid about?

Btw, the screenshot shows an interaction with api.smoot.apple.com, which is known to be used for spotlight and related services: https://news.ycombinator.com/item?id=8479958 ; mediaanalysisd is used for visual lookup, where macOS tries to identify landmarks and other items in an image to help you find more information about it (https://appleinsider.com/articles/23/01/21/tests-confirm-mac...).

In short, paranoid people are good to use as indicators for further investigation, but are rarely to be trusted as sole sources—even less so when they attempt to ascribe intent.

"erroneously" is referring to his later claims about Apple still continuing to scan. I guess it's a poor choice of words, but I don't see much problems with that.

Your note about that domain is interesting though. I don't regularly use Apple devices, so I'm not particularly concerned by that anyways.

It would scan the private photo library on your phone. There is no “public” photo library on my phone, and the default photo library contains extremely personal photos that I consider private and have not shared with anyone else, so I see no value in torturing language to pretend that this is not my private photo library. You are correct that it would only scan my private photos conditioned on a switch being turned on that would also cause those photos to be uploaded to my private cloud backup account. However this does not make that data any less private to me, and it is very different than scanning my photos in the cloud.

Since late 2022 Apple has enabled Advanced Data Protection, which encrypts all photos before they’re uploaded to cloud storage. With ADP on, my photo library is “private” not just in the common-language sense (it contains extremely personal data I have not shared with others) but also in an opinionated technical sense that these files are accessible only to me. If Apple’s CSAM scanner was deployed today, it would be scanning those photos in cleartext on your phone before unreadable data was sent up to the cloud. You could argue that Apple was making a trade: “hey, it doesn’t matter whether we can read the private data you’re storing, the price of sending even unreadable encrypted private data to our infrastructure is that you must run local software that scans the private photos on your phone,” and that’s a trade you might accept or reject on the merits. However I think it’s extremely important to say it exactly this way and not play language games. Apple was going to mandate local scanning of private photos as the cost of using their infrastructure even to store opaque private bits.

At the time of the announcement in 2021 there were no encrypted photos in iCloud. There was "private photo library only on your device" and "photos shared with Apple (not private)". The scanner would not have scanned private device-only photos.

> "If Apple’s CSAM scanner was deployed today, it would be scanning those photos in cleartext on your phone before unreadable data was sent up to the cloud."

Or enabling Advanced Data Protection could have disabled the scanner, we don't know. Even if it went the way you said, you could still not use iCloud and have private photos on your device, whereas your phrasing is trying to imply that there would be no option to do that.

Apple has been working on end-to-end encrypted iCloud since at least 2018 [1]. In fact they’ve been gradually implementing it since 2015. They finally deployed ADP in 2022. It is ludicrous to believe that in 2021 they designed a client-side photo scanning system whose only conceivable purpose is to be part of an end-to-end encrypted backup system, and yet also believe that system was not intended to be turned on as part of their ongoing (and ultimately successful) encryption rollout.

I think we will have to agree to disagree about the idea that turning on cloud backup suddenly makes my private photo library “not a private photo library.”

[1] https://www.reuters.com/article/us-apple-fbi-icloud-exclusiv...

To add more detail to that, Apple's proposed CSAM scanning worked by computing a hashed value for each photo on your device then compare that to a list of known CSAM image hashes downloaded from Apple. Entirely on your device, aka the "client," as in "client side scanning" (to clarify, Apple's cloud is not the client, your personal device is). Then if you have photos that hashed to a value on the known CSAM hash list (which this isn't MD5 or similar bullshit hash algo, so that would only happen if you either engineered a hash collision or actually had CSAM content) they'd send them over to have a human look at. That's multiple photos, cause 1 match could well be a false positive.

It did a great job at freaking people out hearing about their photos getting scanned and it could be defeated by making a 1 pixel change to any photos a pedo would hide on their phone (since any changes to the image would totally change the hash).

>could be defeated by making a 1 pixel change to any photos a pedo would hide on their phone (since any changes to the image would totally change the hash).

This isn't the way those hashes work. A 1 pixel change would still hash similar enough to be matched. Maybe there are adversarial 1 pixel changes that could break the hashing, but I doubt it.

Even cropping, watermarks and other manipulations like that would still match. "Perceptual hashing", very different to cryptographic hashing. It's basically checking if an image looks "similar enough".

I believe this is why they needed multiple matches, because otherwise there must have been too many false positives.

This may be too oversimplified, but imagine that in a series of CSAM images, there might be, for example, a wall or furniture or something, that could appear similar enough to a wall in one of your own photos. That's a match, off to the gulag with you!

You forgot about Google. (Dad sends child photo to doctor)

What's especially fucked up in that case is that Google suspended his account and didn't restore it even after the cops closed the case and said that it wasn't a crime.

So the lesson is clear - avoid Google as much as possible and use services from separate companies. Email from one company, Chat/IM from a different one and so on. So that if one of your accounts gets suspended for one reason or another - it would not affect the rest.

I look at it as, split apart cloud things that are totally unrelated to the rest. Like there's no way my YouTube creator account would be the same as my personal Gmail/calendar/whatever. And don't put photos in the cloud at all unless it's e2ee. Chat/IM and email in the same place would make sense cause they're related things, except nobody uses Google's chat products.

It's nice how iPhones do the neat AI photo search and montages all client-side, with my Mac backing things up. They're really catering to the less common use case there.

That was a server-side scan IIRC.

> Today it’s saving the children, tomorrow it’s basically any problem

This is how most rights get taken away, not just encryption. Also, we're talking about countries that already have pretty restricted speech. Encryption has to consistently remain popular to survive there, and there are plenty of ways to undermine that.

And it's completely obvious bullshit. Consider any more salient issue affecting children: the leading cause of death for children under 18 is car crashes. The response to this large and growing problem is to a) blame the victims (they shouldn't have been in the road / they should have walked a mile out of their way to return to the spot 100 feet across a road / they weren't wearing construction vests & waving flags / walking to school isn't allowed in the first place) or to wave it away as the cost of doing business.

Were there a shred of consistency in actually advocating for children's quality of life, I could forgive those who are duped by these underhanded tactics. As it stands, there's no actual concerted "save the children" bandwagon that we're being invited to hop on to.

> As it stands, there's no actual concerted "save the children" bandwagon that we're being invited to hop on to.

Maybe one should be created, with a prioritized list of issues, and loudly inform people that if issues at the bottom are being prioritized the proponents may have other motives than the ones they claim.

Impossible. If you rank avoidable deaths by quantity, men arrive in all the top 10 categories, so I’m pretty sure no-one would want to make this list anyway.

Don't say that out loud.

Clearly nobody is suggesting that on average men flock towards more dangerous higher paying professions which until recently also meant higher wages.

Wouldn't want people to make the connection between risk and reward as a factor in economics.

Yeah, but I don't know what this has to do with children.

Yes, CEO is a high-risk profession (you can really hurt your back golfing all day), but coal miner is not.

Plus what does this have to do with mortality in children again?

no, you hurt your general well being by working very late, too little sleep, being stressed etc.

They complain about CEOs but they are like 0.1% of the population. Men also flock in the bottom jobs. They don’t even see 90% of the men.

But yes, the transition between CEO and homeless is surprisingly sudden. It’s indeed a risky job.

Can't understand if this is satire.

Yep, that's why CEOs work less hours than the average American worker... Come on man, dont buy into the propaganda.

If there were a political bandwagon around reducing child deaths, I'd probably still suspect ulterior motives. Also, if it were based on just death counts, I view murder differently from accidents. It's like when people try to compare lung cancer and 9/11.

Agreed on all counts. Even discussion of the statistics on firearms-related mortality tends to sidestep the proportion of suicides. People seem forever to be more interested in pushing their own positions than the hard work of solving the problems, even just one person at time.

> the leading cause of death for children under 18 is car crashes.

Are you sure about that? In the US, firearm-related injuries became the leading cause of death among people aged 1 to 19.

https://www.nejm.org/doi/full/10.1056/nejmc2201761

The pretext of child abuse prevention is used as a plausibly-deniable dog whistle to automatically discredit and demonize anyone who dares speak out against the proposed law.

It's got nothing to do whether the law would be actually effective at curbing such abuse, whether it will have harmful (seemingly-) unintended consequences, whether other solutions would be more effective or have less potential for unintended consequences, or whether the problem is actually widespread enough to even worry about compared to other risks that affect children.

Very well said, but even if we were talking about just CSA, this is still bullshit: the one thing you can do to help prevent child sexual abuse is to talk to children about sex, taboo-free, and have quality sex ed starting at primary school children. Emphasising consent, explaining what is sex and how it works, how it's not okay for an adult to do it with you, how it's not okay for anyone to touch you without your express consent, inculcating healthy views about relationships and sex from an early age, etc.

Unfortunately this is uncomfortable for some people (understandable, it's pretty awkward to talk with a 7yo about sex), but it's what politicians should be pushing if they were really about "saving the children". Monitoring cell phones is not it.

Also how do they prevent this from being exploited by hackers? If there's a backdoor it's naive to think that only the "good guys" will use it.

Also what if some hackers put something on my phone to intentionally trigger this in order to blackmail me or ruin my reputation?

> If there's a backdoor it's naive to think that only the "good guys" will use it.

This is the thinking behind NSA's 'Nobody But Us' saying. If you hoard 0day, assume someone else will discover it given enough time.

[0] https://en.wikipedia.org/wiki/NOBUS

Except in this case everybody knows that there's a backdoor that can be exploited.

> If there's a backdoor it's naive to think that only the "good guys" will use it.

Yet we have learned nothing from KGB, STASI, CIA etc.

> Also what if some hackers put something on my phone to intentionally trigger this

"We just found 5 gramms of Mary jane in your pocket".

Religion will come back. "Repent or god will punish you" /s

> Is the problem that people can send encrypted things back and forth to each other?

I think this is the case, except it’s a feature not a bug. The predictable characters will shoehorn in the concern angle[1].

[1]: https://en.m.wikipedia.org/wiki/Four_Horsemen_of_the_Infocal...

> Requiring that companies put snooping software on their device is basically the thought police.

There is already snooping software on most company devices: Microsoft software and Google software and the Western internet.

It's already snooping on everybody for the government (through at the very least NSA).

It already makes sure the traffic is (hopefully/maybe) only encrypted for everybody else.

This goes for the EU as well. Most Western countries legalized and extended what Snowden revealed about government surveillance.

It's already being used in a dragnet surveillance thought police type of way for decades. At least no doubt in my mind. Call me paranoid, don't care.

I definitely agree surveillance is happening and I don't mean to do a whataboutism, but why such emphasis on Western internet? Are you implying this does not happen elsewhere? If we take a counterpoint to West, India and China both have significant surveillance of their citizen and they account for the other third of the world population.

Let's just agree it's a problem everywhere?

> Let's just agree it's a problem everywhere?

Let's just not, because the ones carrying the baton of "liberty" are the ones that ought be measured by that same stick

China or other countries do not fall on the hypocrisy of saying that your info won't be part of the dragnet, this is not the case with said western govs which then proceed to decry the evil non-western countries for doing the exact same thing and expecting their populations to somehow do something about it¿? ¿? ¿ It is indeed nauseating

> Let's just agree it's a problem everywhere?

In some places they say you are free and have rights, but you don't.

In other places you know you don't have certain rights and act accordingly, because people are not stupid, if they know there's a potential danger ahead, they become more cautious (or they carefully comply, because there are no alternatives).

I would say the first group of counties is more dangerous, because it gives people a false sense of security, lowering their natural defenses, while corporations profit from knowing everything about them, things they said they should never have had access to in the first place, because you have rights, right?

because king pooh doesnt spent disgusting amounts of efforts talking about how china is a shiny beacon of light for freedom, how "we are NEVER gonna be like THEM", "that doesnt happen here".

the west talks big, but in reality western countries are the biggest slimiest dirtiest hypocrites the world has ever known

Also a potential security issue. Many client side virus scanners have accidentally introduced remotely exploitable security holes.

Reply on news.ycombinator.com