Hacker News Reader

Comment by nbzso | Hacker News Reader

Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs.

Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO and there is no law who requires them to "scan" on device. So the logical explanation is "growth". Way to go Apple.

View on HN

>> there is no law who requires them to "scan" on device.

There is. Apple must comply with warrant requests. If they have a system for scanning files on customer devices they must, if presented with a warrant, allow police access to that system. We can quibble about jurisdictions and constitutional protections, but if the FBI shows up with a federal warrant demanding that Apple remotely scan Sandworm101's phone for a particular image, Sandworm101's phone is going to be scanned.

That's an incomplete statement.

Currently, they must comply with warranty requests by scanning if they have the ability to scan.

If they have no such ability (say, because they designed their phones from a privacy-first perspective), the law makes no requirement that they create such a capability.

And that's what pisses people off about this.

Can a warrant compel them to develop the capability?

A warrant alone, very likely not. A court order from a judge (slightly different than a warrant), possibly not nut it's not exactly a well tread area of legislation/law. It would probably run afoul of the first amendment given some readings, but it would have to be tested in court and would likely be argued that requiring a backdoor via some law doesn't cause compelled speech or that making that backdoor isn't protected speech etc. It's a relatively new area to be explored.

Yes. Lavabit. Those warrants demanded that Lavabit alter its system to capture passwords and/or decrypt stored email. Lavbit instead decided to stop operating and delete everything rather than comply. Such warrants have not been fully tested in courts but they do exist.

IIRC they gave up the information AND they shut down

They handed over a hard copy of their private key, a printout. The FBI went to court to demand a machine-readable copy. Then they shut down.

Thanks for that! Didn't know.

Have they been served a warrant?

Just because someone could force you to do something you don't want to do is not really a major concern if you choose, preemptively, to do the thing. The concerning part here is Apple signalling that they have executives that don't see phone scanning as a problem. That is a major black eye for their branding.

Apple's warrant canary disappeared in 2014.

The law gets debatable here. The warrant can ultimately be served only to the owner/responsible party for the system. If apple decides that they own all iPhones, a claim they've loosely made for a long time re Flash etc. Then they could be served a warrant on the device.

If they just sold the device to someone, then the police would have to issue a warrant to that individual. My understanding is that as an individual your options are either to comply with the warrant or commit a separate crime destroying the evidence or refusing the warrant.

Apple does not claim they own all iPhones.

The legal principle is the same, actually. The law says you can install whatever software you want on an iPhone and Apple cannot stop you (jailbreaking is legal). But the law cannot force Apple to make it easy for you to do so.

I haven't kept us with the latest cases, but I'm not sure where it falls on app-based SaaS.

I.e. If I use an app, on a system in which app updates are possible, which encrypts my data locally and stores it on their cloud, then the owner/responsible party for the app is...?

> "the law makes no requirement that they create such a capability."

and they have not created such a capability. This is built into the image upload libary of iCloud. This has less ability to "scan the device" than the iOS updater which can run arbitrary code supplied by Apple and read/write anywhere on the system. This is less able to be extended to a general purpose scanner than that is.

Warrants are only part of the issue. The bigger issue is civil and criminal liability. If someone uploads child porn to iCloud and then shares it with someone else Apple is possessing and distributing child porn.

Today they aren't liable because there is a law that shields them. But that law comes with strings attached around assisting law enforcement. By doing an end run around those strings Apple is not holding up it's end of the bargain and runs the risk of lawmakers removing the liability shield.

If that happens then it's a whole new ball game.

> If someone uploads child porn to iCloud and then shares it with someone else Apple is possessing and distributing child porn.

If someone sends CSAM using Federal Express, is FedEx legally liable for "possessing and distributing" that material? Does FedEx need to start opening up packages and scanning hard drives, DVDs, USB sticks, etc. to ensure that they don't contain any CSAM or other illegal data?

I really struggle with the lengths that people are going to to justify these moves. If we can justify this, it's pretty simple to justify a lot more surveillance as well. CSAM is not the only scourge in our society.

Reminder: Apple tells us that they consider privacy a "fundamental human right"[1]. That simply does not square with their recent announcement of on-device scanning, and some would argue that it does not square with scanning or content analysis anywhere, especially on behalf of government.

[1] https://apple.com/privacy

>If someone sends CSAM using Federal Express, is FedEx legally liable for "possessing and distributing" that material?

Yes:

https://www.dea.gov/press-releases/2014/07/18/fedex-indicted...

That was for drugs but conceptually the same for CSAM.

This is not at all conceptually the same.

FedEx was not held liable simply because their service was used to mail illegal drugs. They were held liable because not only did they know about the specific instances in which it was mailed, they allegedly conspired with the shippers to facilitate the mailings. They were knowingly mailing packages to parking lots where drug dealers would wait to pick them up:

> According to the indictment, as early as 2004, FedEx knew that it was delivering drugs to dealers and addicts. FedEx’s couriers in Kentucky, Tennessee, and Virginia expressed safety concerns that were circulated to FedEx Senior management, including that FedEx trucks were stopped on the road by online pharmacy customers demanding packages of pills; that the delivery address was a parking lot, school, or vacant home where several car loads of people were waiting for the FedEx driver to arrive with their drugs; that customers were jumping on the FedEx trucks and demanding online pharmacy packages; and that FedEx drivers were threatened if they insisted on delivering packages to the addresses instead of giving the packages to customers who demanded them.

They had drug addicts literally stopping FedEx trucks on the street to intercept packages from online pharmacies. I don't see how this specific situation is analogous to Apple at all.

The fact remains that FedEx is not scanning hard drives, DVD or other storage devices going through their network and they don't appear to be breaking any laws by not doing that scanning.

>They were held liable because not only did they know about the specific instances in which it was mailed, they allegedly conspired with the shippers to facilitate the mailings.

Apple knows that CSAM is being sent and conspires to do so (i.e. transmits the image). Conceptually they are the same.

The rest of your post details the practical differences between sending physical packages and digital images.

I'm not even sure where to start here.

FedEx knows that CSAM is sent using its services in the same way that Apple does. That is to say that both companies know that CSAM has been sent historically and both know that is possible to send that type of material using its services, and one could argue that they should assume that their services are used to transfer that data.

Why does one of these companies have to go to such great depths to search out and report these materials and the other does not? If we suggest that Apple "knows" about CSAM on their network, we must also accept that Apple "knows" about many other crimes in which the devices they are sell are used in the planning and execution of those crimes. Why are they only focused on CSAM if they'd also have legal exposure in these other crimes simply for being a hardware and service provider?

Lastly, to suggest that Apple "conspires" to transmit this content is wholly inaccurate. Conspiracy implies two or more parties are in explicit agreement to commit a crime.

>If we suggest that Apple "knows" about CSAM on their network, we must also accept that Apple "knows" about many other crimes in which the devices they are sell are used in the planning and execution of those crimes. Why are they only focused on CSAM if they'd also have legal exposure in these other crimes simply for being a hardware and service provider?

It's not knowledge of a crime. Apple is committing a crime by possessing and distributing CSAM.

>Lastly, to suggest that Apple "conspires" to transmit this content is wholly inaccurate. Conspiracy implies two or more parties are in explicit agreement to commit a crime.

But that is exactly what they're doing. They are agreeing to possess and deliver images of which they know a portion are CSAM.

> Apple is committing a crime by possessing and distributing CSAM.

Apple's liability for content is limited by Section 230. If they find out about specific instances of CSAM on their servers they are required to remove it.

They are not required to hunt for it or otherwise search it out. They are not required to scan for it. Anything currently being done in that regard is voluntary.

> But that is exactly what they're doing. They are agreeing to possess and deliver images of which they know a portion are CSAM.

When you use Apple's cloud services, you agree to multiple legally binding agreements that include provisions about not using their cloud services to store or transmit illegal materials including CSAM.

Someone using Apple's services to do that in contravention of those agreements does not constitute them "agreeing to possess and deliver" that content.

What WOULD constitute them conspiracy and them agreeing to possess and deliver that content would be if they were informed of the specific instance of content being transmitted and delivered and did nothing about it or otherwise enabled its continued storage and distribution.

If Apple launches a system for comparing iCloud uploads to a third-party hash list, then adding the ability to do targeted scans for arbitrary additional law-enforcement-provided hashes would also be a form of creating capability.

The people getting pissed off about this have not, so far as I’ve seen, demonstrated why the law would require Apple to add the capability for targeted scans of arbitrary hashes.

Police can use a warrant to ask you for a video file they suspect you have. They can’t use a warrant to force you to videotape someone—even if you already own a video camera. The same principle applies to Apple.

>> They can’t use a warrant to force you to videotape someone

Maybe in the narrow context of US domestic child pornography investigations. In the wider world it is very possible for police to force such things. Even in the US, CALEA demands that certain companies develop abilities that they would not normally want (interception). The principal that US companies need not actively participate in police investigations disappeared decades ago. On the international level, all bets are off.

https://en.wikipedia.org/wiki/Communications_Assistance_for_...

"by requiring that telecommunications carriers and manufacturers of telecommunications equipment modify and design their equipment, facilities, and services to ensure that they have built-in capabilities for targeted surveillance [...] USA telecommunications providers must install new hardware or software, as well as modify old equipment, so that it doesn't interfere with the ability of a law enforcement agency (LEA) to perform real-time surveillance of any telephone or Internet traffic."

Can you explain why you think CALEA would apply to what Apple announced?

And why such application would have had to wait until Apple announced this? In other words, if the law can force Apple to do things in general, why does the law need to wait for Apple to announce certain capabilities first?

>> why you think CALEA would apply to what Apple announced?

Read what I stated. I said no such thing. I said that CALEA shows that companies can sometimes be forced to be do things they do not want to do, to take actions at the behest of law enforcement that they would not normally do. CALEA would clearly not be applied to apple in this case. It would be some other law/warrant/NSL that would force apple to do something it normally would not do. CALEA stands as proof that such things have long been acceptable under US law.

I understand why you are worried that a more powerful law could be passed by Congress in the future.

I thought this particular conversation was about what Apple can be forced to do by a warrant issued under current law.

> why the law would require Apple to add the capability for ...

This isn't the sort of thing that is mandated by law, but rather requested behind the scenes by espionage, er, "law enforcement" agencies. They might be more or less friendly deals; nice monopoly you have here, it would be a shame if something happened to it...

At least in the United States, this is absolutely false. A warrant cannot force them to do something they have no capability to do.

(a) They created the capability, that's the problem

(b) The world is not the United States

I don’t think this is correct.

1) Even if Apple does roll out this system, it’s not a system for scanning files on customer devices. It’s a system for comparing photos being uploaded to iCloud to a single standardized (everyone has the same one) list of hashes.

2) Apple likely has the legal power to refuse to alter that list of hashes, by the same argument they used against the FBI’s request to bypass the unlock code on specific iPhones.

3) Any argument about what Apple will and will not need to do with this system needs to explain how Microsoft Defender (or other AV products) interact with law enforcement, since those are software systems that scan client devices (ALL files, typically) for signature matches.

> and there is no law who requires them to "scan" on device

Yet. The demands by "concerned parents" (aka fronts for secret services, puritans/other religious fundamentalists and law-and-order hardliners) to "do something against child porn" have grown ever more strong and insane over the last years. (And you can bet that what is used on CSAM will immediately be used to go after legal pornography, sex work, drug enforcement, ...)

Many current and powerful politicians don't understand a single bit about computers, to the point of bragging of never having used one (e.g. Japan's cybersecurity minister) or having assistants print out emails daily and transcribing handwritten responses (can't say more than that this is the situation for at least two German MPs) - but there are young politicians who do, and will replace them over the next decade hopefully. That means it's last chance for lobbying groups to get devastating laws passed through, and we must all be vigorous in spotting and preventing at least the worst of them.

And what is suspiciously lacking in Apple's response: what are they going to do when they are compelled to extend the CSAM scanner by law in the US, India, China and/or EU? It's feasible for Apple to say they'll just be sticking the middle finger towards markets such as Russia, Saudi-Arabia or similar tiny dictatorships, but the big markets? Apple can't ignore these, and especially India and China are heavyweights.

Why would they extend the CSAM scanner? It would be much simpler to just use all the OCR and image classification functions they have already deployed.

CSAM scanning is only useful for areas where Apple really doesn't want to even look at the actual material until they are extremely certain that it's a match. If they want to detect anti-government propaganda or something, there would be no such concerns, they would just do a regex search of your inbox. Infinitely more convenient.

>> useful for areas where Apple really doesn't want to even look at the actual material

Correct. It has plausible deniability built in. Apple is unable to verify that the images the government are looking for are actually CSAM. They could be political. They could be protest images. They could be Winnie the Pooh. Apple can plead ignorance as it blindly scans for whatever the requesting government asks it to scan for.

Nobody really minds that this system is going to be used for CSAM. What everyone recognizes is how ripe this system is for abuse, how easily it can be leveraged by oppressive governments. And Apple can play the innocent.

Why even keep Apple in the loop? Why not just allow government to submit scanning models directly?

Which Apple will dutifully install and run, because they're required by local laws.

> "Which Apple will dutifully install and run, because they're required by local laws."

Which Apple have stated that they won't do, and have designed the system so they can't do that without it being found out: https://news.ycombinator.com/item?id=28221082

Can't you at least post accurate information about this system and support outrage based on facts instead of fantasy?

> Which Apple have stated that they won't do

For your random off-of-the-mill dictatorship, yes.

For the US? EU? China? India? No way they can refuse such a request from these markets. And if they could and get away with it, it would be a very worrying situation in itself regarding (democratic) control of government over global mega corporations.

If you believe "the government can compel any company to do anything", then this system is no worse than any other.

Although how do you think Linus Torvalds "manged to get away with refusing" adding backdoors? https://www.techdirt.com/articles/20130919/07485524578/linus...

Simple reason: every line of code in the Linux kernel passes many eyeballs in the open. Even if someone were to compel Linus Torvalds to commit a backdoor and he would comply with that order, people across many jurisdictions would notice this immediately, rendering the backdoor worthless.

Except that Apple will review the photos once you've matched 30 of them, so it's still not possible for the government to misuse it.

In China, iCloud is run by the government.

It's actually not, but even if it were, that would be yet another reason CSAM scanning is completely irrelevant for government spying.

Any spying really. It's much easier to just look at the images themselves.

> It's actually not

It's Joe Wong's joke, that it's like peeing in the snow in a dark winter night, while there is a difference but it's really hard to tell.

So some underpaid contractor reviewing "visual derivatives" that may or may not be CSAM completely prevents governments from misusing this in your mind?

Considering how high profile and incredibly sensitive this is, I doubt they will hire an underpaid contractor. It's the opposite of app review, where the volume is very high but the stakes are low.

Facebook hires underpaid and poorly treated contractors to moderate exactly this same kind of content. I see no reason to believe Apple will be any different, particularly a few months from now when the general public's attention has shifted to other matters (assuming they're even paying attention right now. I don't think the interests of HN are necessarily representative of the general public..)

> Nobody really minds that this system is going to be used for CSAM.

I beg to differ. It doesn't matter how evil the content is, no scanning of my computers by outside parties, period. More so by scanning law enforcement can even plant legitimate child pornography on people's computers and get convictions all the easier because the system self-reports.

It's clearly not the scanning you are opposed to, it's the reporting. Apple apps, well a lot of apps, already scan all your content, for classifying images, doing OCR, extracting dates and addresses etc. That is nothing new, it's omnipresent.

What about the human reviewers at Apple? Those need to confirm matches, and they obviously look at the photos to do so. Apple can’t claim ignorance.

Also, it has to be two requesting governments.

The current Apple policy to use human reviewers. That might change at a moment's notice and there is no technical reason why Apple couldn't bypass humans for certain requests. One must always judge a new system three ways: how it was meant to be implemented, how it is actually implemented, and how it might be abused by bad actors in the future.

> "and what is suspiciously lacking in Apple's response: what are they going to do when they are compelled to extend the CSAM scanner by law in the US, India, China and/or EU?"

from https://daringfireball.net/linked/2021/08/09/apple-csam-faq - "we will not accede to any government’s request to expand it."

From https://www.msn.com/en-us/news/technology/craig-federighi-sa... - "“We ship the same software in China with the same database we ship in America, as we ship in Europe. If someone were to come to Apple [with a request to scan for data beyond CSAM], Apple would say no. But let’s say you aren’t confident. You don’t want to just rely on Apple saying no. You want to be sure that Apple couldn’t get away with it if we said yes,” he told the Journal. “There are multiple levels of auditability, and so we’re making sure that you don’t have to trust any one entity, or even any one country, as far as what images are part of this process.”" - Craig Federighi

A “government’s request” is different from a country’s law. “This process” is not the only process by which third parties can breach your privacy.

How likely is it that you will have enough colliding images in your photo library to even trigger a review?

I'm guessing you need at least 5 images, perhaps much more, to trigger it. In any case, 1 image is definitely not enough.

For a normal random person, a grey man. In the real world, a single collision could be enough for the police to acquire further access to people they are already looking at. If the police want access to your phone for other reasons (drugs, taxes, illegal speech) they can use that one collision to get a warrant which will give them greater access.

It is akin to cops wanting to search a car. They don't need a warrant. They only need to follow the car until it breaks any number of traffic laws. Then the resulting traffic stops lets them talk to the driver, who seems evasive, which gives them probable cause, which gets them a warrant to perform a full search. The accidental collision, week evidence of an offending image on the phone, opens the door to whatever other investigation they want to do.

I'm not sure what you mean. If we are talking about Apple's new feature, they will not even be alerted until you have enough matches.

And the police probably won't be using any neural hashes if they have access to your device, so I'm not sure what you are talking about.

That is an Apple current policy, not any sort of law. If the FBI wants to know if there are any collisions on a particular phone, that will be shared. If the FBI wants to know of all single collisions, that too must be shared. A corporation's internal policy decisions are nothing when faced with a government official carrying a warrant. If they have data indicating possible crimes, no matter how small, they can be forced to divulge it to investigators.

Unlike Lavabit, I don't think Apple will ever close up shop to protect customer data.

Well of course if you believe that FBI can ask Apple to share anything and they will agree, and not tell us, then what is even the point of this discussion? Then they already have access to every photo, every email, every text. Because Apple controls those apps on your phone.

The difference is that now they confirmed that they do search your private images.

Why would that make it any likelier that they are also secretly searching your files, with another method? Seems completely orthogonal to me.

They showed that they are willing to do it in one case. Why not in other cases?

I thought the system is designed such that the threshold must be met to check the matches, i.e. it’s not possible to follow the FBIs request without changing the entire system.

int matchThreshold = 30;

if (numMatches > matchThreshold) { snitch(customer) && customer.life.ruin() }

Isn't Apple seeding their database with fake hits so that a court can't order them to turn over everyone who's just a few shy of Apple's threshold? So when this database leaks, won't there be a lot of people accused of horrific crimes simply because Apple seeded their database with random hits?

Why would people pay a company to falsely accuse them of horrific crimes?

Since, as I have read, Microsoft and Google already do this, where are all the illicit cop take-downs? I have not heard of any.

They are scanning images on online accounts, stuff that is stored on their systems and system files on machines (files they own). They are not, from what I have read, scanning customer-owned images stored on customer-owned devices.

Every AV (antivirus) software system scans customer-owned files on customer-owned devices. So the question is the same: if it’s easy for law enforcement to deputize such a system, where is the flood of cases built off of evidence gathered this way?

>> Every AV (antivirus) software system

Mine doesn't. If Ubuntu or ClamAV is scanning all my photos and reporting the results to Canonical against my will then I will soon be having words with Mr. Linus.

If law enforcement can force software companies to adapt their software to serve law enforcement purposes, why aren’t Ubuntu or ClamAV doing so already? What makes them better at resisting requests from law enforcement than Apple?

Canonical and ClamAV aren't trying to curry favor with the US government to stave off antitrust action.

>> why aren’t Ubuntu or ClamAV doing so already?

Because if some local police agency like the FBI were altering code on an open source project as large as ubuntu then the world would know about it pretty quickly. The linux community would burn every bridge with Canonical and ubuntu would disappear overnight. The idea of the FBI adding malware to ubuntu unnoticed, malware that openly scans files and reports to remote servers, is the stuff of comic books.

I feel like you are both chasing red herrings.

This is not about child porn nor about what AV software can or cannot do.

It's about normalising mass surveillance and implementing populace control. They don't want another Snowden to scare the public with reports about "backdoors" and mass privacy violations.

They want the coming generation to perceive it as normal. Because all phones do it, hasn't it always been this way, and think of the children.

Oh, these dissidents in $distant_country that will be muted and killed using Apple's shiny surveillance tech? Well, evil governments do what they do. But we are not like that. Over here it is only about the child predators. Trust us.

Apple has been an opponent of these developments for decades. Now they are spearheading it.

Apples solution only scans photos that will be uploaded, and the results of the scan require a server side to be evaluated. The client does not know whether a match occurred.

Apple has publicly stated 30. Plus, there would be a manual review of the images before forwarding to law enforcement. I don't see this collision attack leading to innocent people being handed over to the police. The worst-case (best-case?) scenario is that Apple gives up on the whole hashing system and then there will be zero chance of there ever being end-to-end encryption of photos in iCloud.

For individual users the risk is very small, but when you have a billion users the chances of innocent people being caught up is pretty much 100%.

Platform owners like Google, Apple, Twitter and Facebook should really keep stuff like that in mind when they deploy algorithmic solutions like this.

Like dhosek said, the manual review step should reduce the risk quite a bit, though.

They did keep it in mind, that's why they require 30 matches, which gets the false positive rate down to 1 in a trillion per year, so on average it will happen about 1 per millennium, given a billion users.

So that's per photo library, not per photo. The rate per photo in their testing was 3 in 100 million, then they added a 30x safety margin and assumed it's actually 1 in a million.

https://www.zdnet.com/article/apple-to-tune-csam-system-to-k...

Not sure why the downvotes. You are absolutely correct that a threshold in the number of positives (false or otherwise) must be met.

To be fair though we do not know what the threshold is. But I would guess even higher than 5 — I would presume 12 or more.

I'm no criminologist (IANAC) but when you read about someone getting busted with child pornography they have hundreds or thousand of images — not one, not five. They're "trading cards" for these creeps.

Apple says they expect to use 30 as the threshold.

I saw 30 mentioned somewhere, and I think that’s a more likely guess than 5. When men have a porn collection, it rarely has less than a hundred images, and so the threshold for CSAM detections can be set quite high in quantity, and enjoy a near-zero false positive rate aside from malicious hackers.

(And yes, 90% of CSAM abusers are men, so y’all will have to find some other way to weaken my argument.)

Reply on news.ycombinator.com