Hacker News Reader

Comment by citizenpaul | Hacker News Reader

It's incredibly disturbing to me how little much of the HN crowd seems to understand how these systems like CSAM actually work. I'm not talking about the tech or programming.

This is a fast track tool to putting anyone in jail for the most despised crime a person can commit. The tool has no oversight, no transparency, no counter, no clear your name, no real usage definitions other than relentless spying on the "protect the children" fallacy.

Most people don't understand how life destroying a false accusation is once the government paperwork is started. At that point like it or not you are basically considered guilty by all the agents of the governement by default. Their jobs are to prosecute not eliminate false positives. Once you have been on their radar your life will never ever be the same. And since it is CP related you will not even get helpful people willing to touch it with a 10ft pole cause of fear they may be implicated. CSAM is a dastardly and effective way to establish a totalitarian rule by fear of life in jail, nothing more.

View on HN

> It's incredibly disturbing to me how little much of the HN crowd seems to understand how these systems like CSAM actually work.

Dunno. I remember most people on HN were absolutely pissed this was even being considered. My takeaway is that most people here understand it to a large enough degree that they don't like it.

Stuff like this gives me pause in the tech industry. My brother is in med-school and always joke with him that the worst thing I can at work is take a website down and the worst thing he can do is kill someone.

But with tech like this, you can literally kill someone with it. Sure you're not actually "killing" anyone, but the decisions these systems make can drive someone to end their life, whether or not the accusations by the machine are true. I wouldn't want to be the guy building that software.

lest we forget

> Because of concurrent programming errors (also known as race conditions), it sometimes gave its patients radiation doses that were hundreds of times greater than normal, resulting in death or serious injury.

https://en.wikipedia.org/wiki/Therac-25

> can drive someone to end their life

Don't underestimate the possibility of somebody else ending their lives for them in a fit of righteous rage.

What's disturbing is that nothing has materially changed. For those of us who are running iOS or Android, we're just an OS update away from CSAM landing on our devices again. It's just a matter of time. The question is what will it be renamed to and under what pretext will it be deployed?

Eugh, yep. Apple pretty much confirms its coming in the future:

“Update: Apple spokesperson Shane Bauer told The Verge that though the CSAM detection feature is no longer mentioned on its website, plans for CSAM detection have not changed since September, which means CSAM detection is still coming in the future.”

Expect a much quieter roll out.

My bet is they will settle for a similar solution to other providers and scan the photos on their servers.

That’s still a much better world to live in than “let’s make it OK for people to think they can deploy scanners to users devices”. My device is mine, their servers are theirs.

> My device is mine

Not anymore. Now its a self subsidized subjugation tool. In 10 years it will not even be possible to buy a networkable phone in the USA that is without spyware embedded. Mark my words.

Do you have any reasons to think that they're not scanning photos on their servers? AFAIK every major storage cloud does that, including Apple.

What probably happened is that Apple planned to introduce E2E encryption for iCloud. But they have to scan photos, so they rolled out client scanning to ensure that they're not storing encrypted child porn on their servers.

> Do you have any reasons to think that they're not scanning photos on their servers? AFAIK every major storage cloud does that, including Apple.

Apple already scans iCloud Mail for CSAM, but not iCloud Photos https://9to5mac.com/2021/08/23/apple-scans-icloud-mail-for-c...

We will not make it quiet.. It will be found out and I'm sure the privacy organisations will make a big deal of it.

I have much less of a problem with server based scanning. But my device is mine.

Yep we will complain and we will let the public know what apple is up to. We are always watching the watchers as well. Don't forget that Apple. We won't let you reframe it as "think of the children". Please stick to the hardware and making $$$ business, we don't want you playing keystone cops.

You act like there is a choice. It will be rolled out period the decision has already been made. The only reason it is not out is Apple whined about their image and asked for a delay. The powers that be will keep dipping their toes in the water until the heat dies down then roll it out no matter what. Like many other such instances eventually the majority of the public will suffer from fatigue on the issue and stop caring. They have lives to lead and the people pushing for this are getting paid to do it as their job and also will gain power from it. They will never stop.

"Medical misinformation" seems to be the trendy pearl clutching excuse of the day.

Dude every single comment in the post when this was announced was against it.

There were a few "why are you defending predators?" type comments.

And "I have nothing to hide!" type comments, from few voyeurists.

> from few voyeurists

In this case, wouldn't that be exhibitionists?

And the bUt yOU dOn'T uNderStAnd tHe tEchNolOgy! Really? a communuity of programmers, engineers, and scientists can't understand technology? It's so preposterous.

There were several threads about it. As they increased there was more and more debate. The fact that there is debate at all is indication of support.

It also mirrors what they are anticipating. An initial backlash then less and less as they continually push for it since they are literally getting paid for it and we have to fight back on our own time.

You should at least be somewhat charitable to the other side of this issue which is that CSAM is the result of real life sexual abuse and the market for it causes more. Unless you want a cop stationed every 10 feet you can't stop the abuse, you can only take away the incentive to do it in the first place which is the theory behind most laws. The reason most people aren't constantly worried about being murdered isn't because we have super advanced defense systems but because the fear of punishment keeps people from doing it.

I'm aware that the majority of HN want the solution to be "do nothing" and some quote about liberty and security but you would probably change your tune if you or your kids were trafficked, starved, raped, and the had the pictures and videos distributed to fetishists online. CSAM detection is the government's best option to kill the market for it and make it radioactive.

At least listen to the experience of human trafficking survivors before you say it's all a big conspiracy. Those scary signs on buses, trains, in women's bathrooms, and doctor's offices aren't a joke.

You are making it sound all philosophical when HN crowd is already philosophically behind finding an appropriate solution to the CP problem. “Problem” is not a problem here, current proposed solution is. See it critically and not emotionally.

See this is always how online discussions go.

* "CSAM is -- nothing more -- than a way for governments to establish a totalitarian rule!"

* "Oh come on, that's really uncharitable to the people implementing CSAM detection, at least present the argument for the other side in the way they view the issue."

[loud screeching of goalposts...]

* "Of course we acknowledge the problem of sexual abuse imagery, it's just that we don't like the CSAM detection as a solution... [aaand have no ideas for an effective privacy preserving alternative, and want nothing done in the interim while we search for the unicorn]."

There becomes a point where if you don't want to implement the best idea we've got to address the problem, and don't have any other ideas, then it gets harder to believe you really care about the problem.

>There becomes a point where if you don't want to implement the best idea we've got to address the problem

The assumption here is "something must be done". The fact is that liberty and safety are intrinsically at odds. If we're going to make progress, the question we have to face is how many rescued children is worth how many units of liberty. It's distasteful to present the issue so bluntly, but it's the implicit calculation we do in other cases, e.g. preventing child deaths in car accidents. We all implicitly agree that some amount of death and disfigurement of children is worth the economic and social benefits of cars. Similarly, how much liberty should we give up here?

Of course but at least call a spade a spade, the units of liberty you're willing to trade to solve the problem is basically the metric of how much you care about it.

It's a fine position to take that the harm due to that much invasion of privacy and government involvement in our private lives isn't worth but means precisely that you care about the problem less than someone who is willing to make that trade.

I don't think that follows. One can maximally prioritize children while also believing that all children are better served by a society that protects liberty over immediate safety. How you weigh these issues turns on how you weigh the N-th order effects. It's probably not too controversial to say that eliminating all cars would harm children more and thus children as a whole benefit more from cars than their elimination. But it would be disingenuous to characterize this calculation as caring more about economic benefit than children.

> "The fact is that liberty and safety are intrinsically at odds."

If you own an Apple iPhone, and you have photos on it, and you enter into a contract with Apple Computer to use their cloud service, and you upload photos to the cloud service for them to host on their servers, do you have the "liberty" for them to not be allowed to stop you abusing the service in egregious ways? This isn't a matter of freedom and liberty while you can still you can refuse to use iCloud and refuse to use iPhones. Where, by comparison to your example you have much less option to opt-out of travelling by road, using products driven to you on roads and working with people who travelled by road to get to you.

> "it's the implicit calculation we do in other cases, e.g. preventing child deaths in car accidents. We all implicitly agree that some amount of death and disfigurement of children is worth the economic and social benefits of cars"

We don't all agree in the same way; the amount of agreement is continually changing. Where roads are dangerous, authorities lower speed limits and add traffic calming measures and citizens agitate for cycle lanes and pedestrian zones. Where vehicle exhaust makes for poor air quality, people demand cars must have catalytic converters and be subject to emissions control regulations. When pedestrians die in crashes, car manufacturers add external crumple zones. Around schools, people are employed to stop traffic and help children cross roads safely. We pay educators to teach children about road safety and pay for campaigns aimed at educating drivers about pedestrian awareness, drunk driving, distracted driving, the risks of speeding, using fairly graphic imagery of children being hit and killed to drive home the point. We mandate that older cars have regular checks to make sure they're still road worthy and still have functioning brakes and tyre tread and working lights for safety reasons.

I'm often arguing against cars and the harm they do to humans, but it's not the "either you ban cars or you hate children" boolean you're suggesting it is where everyone who benefits from cars is happy to sacrifice children for those benefits and thinks the current amount of sacrifice is just fine thanks and wouldn't change it if they could.

You can turn off a iCloud. CSAM cannot be turned off or even viewed. Only evil is done in the dark.

Nice job making everyone who opposes totalitarian surveillance systems look like they don't care about abuse. This is exactly why children are the perfect political weapon: it makes people accept anything, and the few of us who resist are lumped in with the abusers.

> The reason most people aren't constantly worried about being murdered isn't because we have super advanced defense systems but because the fear of punishment keeps people from doing it.

No, the real reason most people aren't constantly worried about being murdered is that most of the people we encounter aren't murderers.

> most of the people we encounter aren't murderers

Gods I wish this fallacy had a name, I guess it's a corollary of the Spiders Georg principal. The number of perpetrators is not proportional to the number of victims.

It's not true that 40% of men will sexually assault someone in their lifetime but it's nonetheless true that 40% of women will be sexually assaulted in their lifetime.

Jesus in what country? Even taking some very liberal definition of rape 25% is insanely high. Do you have some source on that?

EDIT: In the US we have a 2000 study quoting 5% and a 2011 study saying "nearly 20%" but their data include attempted rape which is a pretty important distinction. This is coming from Wikipedia though.

Changed it to sexual assault because you're right, the study I was going off of included attempted rape which made it higher, but also only counted penetration which is also kinda dumb. Switching to sexual assault eliminates some ambiguity.

Fine, turn my statement around if you prefer. The reason people aren't afraid of being murdered is because being a victim of murder is rare.

But the statistic you're citing here conflicts with your earlier statement (which is the one I take issue with):

> isn't because we have super advanced defense systems but because the fear of punishment keeps people from doing it.

The "fear of punishment" deterrent is clearly doing absolutely nothing for the 40% of women who have been / will be sexually assaulted.

Fear of punishment does not work when "correct" execution of justice is so rare and arbitrary. That's my point. Police are corrupt. Prosecutors are corrupt. Therefore, there is no justice. Therefore, "fear of justice" doesn't work as a deterrent.

Even if the police were better, if all it takes is a few murdering, raping, psychopaths to do all of the damage, how could "fear of punishment" possibly work?

There is, however, fear that the bureaucracy will decide to come after you in a Kafkaesque horror show...

Nobody is saying “do nothing” about all the depraved stuff some people do with respect to human trafficking, etc.

What people are asking is to not be treated as if they are human traffickers by Apple. If they deploy CSAM, they are in effect saying, “sorry world, we suspect all of you are trying to traffic humans so we need to scan your images”.

That’s not how law enforcement should work in a free and open society. You should be suspected of illegal activity first, then law enforcement can move in and do their job.

Once CSAM is deployed, then it’s much easier to expand the scope of it to include other forms of content, and Apple will have a much harder time saying “No”.

I was really hoping Apple would take the backlash of CSAM to exert pressure against whoever is asking them to do this. “Sorry <large gov entity>, we can’t do that without destroying the decade of goodwill we built up with our customers. You’ll have to legislate this for us to do it”. That would have the advantage of at least a public debate.

> The reason most people aren't constantly worried about being murdered isn't because we have super advanced defense systems but because the fear of punishment keeps people from doing it.

In this case, CSAM is the "advanced" defense system, not the fear of punishment.

Kinda? Detecting CSAM doesn't stop the victim from getting abused in the first place which is the goal. It's to increase the risk of getting caught and punished, and to make the photos worthless.

So do detective work, track down the pedo’s, kick down their doors, shoot/imprison them, and free the children.

Don’t take away the privacy and liberty of hundreds of millions of innocent citizens.

What happened to all the people who visited Epstein’s pedophile island? Any consequences? Nothing so far! But the rest of the population better watch out, Big Brother is going through their photo libraries. It’s all a massive government overreach.

The whole reason these systems are proposed is because "detective work" is ineffective and the harm is ongoing. You gotta a least meet people where they are. Don't you think the level of policing required to actually find people who possess CSAM wouldn't also be a massive overreach? Or are you hoping that "just do more" will come to nothing in practice?

Prosecuting members of the ruling class in this country is a whole separate issue, and one that I'm sure we are in total agreement on sans the "well if rich people are above the law why can't everyone be too" take.

> Don't you think the level of policing required to actually find people who possess CSAM wouldn't also be a massive overreach?

Obviously. Criminalizing possession of data is dumb. It's essentially a declaration that some numbers are illegal. It's the exact same problem faced by copyright. Any attempt to actually enforce these laws at scale will be a massive overreach to the point of tyranny. They are incompatible with the computing freedom we enjoy today.

I don't buy this argument because you can make any law sound silly by reducing it to something absurd. Saying that that the benefit isn't worth the trade in freedom is totally valid but the quip about illegal numbers isn't super persuasive.

"Criminalizing possession of nuclear weapons is dumb, it's essentially a declaration that some molecules are illegal."

"Criminalizing hate speech is dumb, it's essentially a declaration that certain combinations of sound frequencies are illegal."

"Criminalization of murder is dumb, it's essentially a declaration that I certain locations where I store my bullets are illegal."

It sounds absurd because it is absurd. Think about the absolute control that would be necessary in order to stop our computers from working with arbitrary data. Obviously this logic cannot be applied to the physical world.

Somewhat disturbing content following!

> The whole reason these systems are proposed is because "detective work" is ineffective and the harm is ongoing.

In Germany, there recently was a news story about two journalists infiltrating and effectively shutting down an insanely large darknet forum for child abuse material.

That story exploded, because evidently, law enforcement could have done a lot better, as all the journalists did was crawling for webhoster-links and reporting the content there - the hosters immediately followed through - terabytes of abuse were deleted. The admins of the forum even spoke with the journalists and told them, literally no one cared before and the whole userbase was caught in surprise.

Not-so-much-fun facts: IIRC that site had millions of accounts registered; apparently at least 1% of men are attracted to children; it's been mostly non-commercial, user-made abuse material - strongly motivated by that community's demand.

The journalists concluded: It's not enough to persecute individuals and gather evidence. It is absolutely imperative to disrupt those communities, too, as those sick people not just consume abuse material, they very much actively promote the abuse itself in their interactions. Now, it's not single hit, lasting solution, but requires constant effort. However terabytes of abuse material are not moved in an instant, and they are surely not delivered via Tor to literally millions of people. Apparently it takes about two people, who know Python, to take down a platform like that. (The argument of destroying evidence is moot, as you can easily do both, within law enforcement capabilities.)

German podcast, interview with one of the journalists; articles in shownotes: https://lagedernation.org/podcast/ldn269-bka-laesst-bilder-v...

TL;DR: Before everyone's privacy and democratic distribution of power are getting compromised, ... maybe lets make sure law enforcement is actually doing their job first.

>detective work" is ineffective

Source?

Detective work actually is effective. The people investigating already know where huge amounts of trafficking and abuse happen. It happens in the CPS system. The problem is they have made it such a quagmire of rules regulations and gotchas that only the truly angelic and those looking to abuse children to make a quick buck are willing to get involved. I got news for you one of those numbers is greater.

You're conflating viewing CSAM with the physical act of exploitation, through some dubious reference to "incentives" and "the market". I don't think people who abuse children are doing so because it's economically lucrative, but rather because they themselves are dangerously sick in the head. But maybe you know more about this scene than I do. (see how that works?)

Once abuse has occurred and has been "documented", completely preventing the spread of that CSAM requires totalitarian systems of policing communications, which are what is being argued against here. Invoking the original abuse as if the topic at hand would have had some bearing on it is an emotional appeal, not a logical argument.

And the defense of totalitarianism begins. Sure strangers 1000 miles away are definitely going to look out for you better than yourself.

>Those scary signs on buses... arn't a joke

They actually are. If you paid attention to Epstine the people doing this have access to massive resources and you don't actually see this happening in public for the most part. Only very small independent traffickers and no one reports them because they themselves are poor and don't have the time to deal with it on their hour long bus trip to a minimum wage job. In fact if you read about most of the people doing it are related to the victims and if you report them there is little to be done about it because of that.

I agree with you completely. We know exactly how it works. What we object to is Apple acting as a police agent without a warrant. It's as simple as that. It sets up a precedent to do all sorts of similar policing (without reason for suspicion or a warrant) for everything from DMCA to thought crime. We are not luddites. We are also not suckers for the police state and authoritarianism creep.

Thing you haven't mentioned is that with no oversight, no transparency no counter, no clear your name, the tyranny which wants to ruin your life doesn't need to build a CSAM scanner to do it. A tip-off to the police, a falsified claim from an "eye witness", an "accident" putting your name on the wrong report to get the machinery rolling and it will, as you say, roll by itself. See what actually did happen to Sir Cliff Richard[1] where "a complaint to the police" got his house raided and his name dragged through the media and despite not being arrested, not being charged, case being dropped, him suing the media and winning, suing the police and winning and the media apologising, he says his reputation will never be completely cleared - and it probably never will be, and it didn't require any specific tech.

"The System" didn't need to wait patiently for a big tech company to build a complicated system with dastardly moustache-twirling weaknesses before it could affect Sir Cliff, anymore than Big Brother in 1984 needed to wait for Winston to be seen taking action on the telescreen in his house before the system crushed him. If "The System" wanted to accuse you, it could accuse your car of having been seen on a nearby road at the time, your likeness being seen on CCTV, and if you get rid of licence plate recognition scanners and CCTV then it would be "an anonymous witness" seeing you in the area at the time or "overhearing you discussing" something.

> "Most people don't understand how life destroying a false accusation is"

Citation needed? Because we were forced to read 1984 and To Kill a Mockingbird in school and I'm pretty sure a lot of other people were as well; we've seen cult TV series like The Prisoner and The Fugitive focusing on the lives of the trapped trying to escape "the system". Blockbuster films have been made about the power of "The State" chasing people such as Enemy of the State with Will Smith. Films in the pattern "wrongly accused must fight to clear his name" and "stalwart good guy fights the system gone bad" are practically genres on their own - The Fugitive as a film with Harrison Ford, Who Framed Roger Rabbit, Captain America: The Winter Soldier, The Shawshank Redemption, and specifically about someone falseley accused by a child, The Hunt with Mads Mikkelsen which is in the IMDB Top 250 films.

Are you also, for example, complaining about the system where doctors and social workers are mandatory reporters if they have any evidence or suspicion of children being abused? That's a "their word against yours" system which pervades the nation, has no oversight, has no objective criteria, no explanaiton of the workings of the brain behind it, relies on humans when we know for certain that there are doctors who make mistakes and errors of judgement and at least some who would look the other way for money.

At least with the tech company CSAM scanners we can potentially have a society wide discussion about how they work in detail, whether that's reasonable, who could and should oversee them and how, what kinds of data cloud companies are obliged to know about their customers and what is reasonable for them to not know, etc. But even in tech circles we can't actually have those discussions, most of the attempts to discuss the details of how it works are drowned out by people who haven't a clue what Apple announced, misunderstand massive parts of it, conflate two or three different systems, don't seem to understand that other tech companies already do this with much less careful design, and cry tyranny and blame China.

Hypothetically, iMessage or any chat system could be built end to end encrypted, any cloud file sync or file storage could be. One end of the spectrum is that they should be. If that's your position then "the system" will probably work around you, that's the "opt out of modern life" position of the "return to the woods" lifestyle. You can't unmake billions of cheap ubiquitous cameras, you can't unopen Pandora's box, and you probably can't stop people emotionally reacting strongly to topics of child protection. If your reaction to that is "not listening, no no no" then it doesn't seem likely to sway "the system" very much at all.

> "CSAM is a dastardly and effective way to establish a totalitarian rule by fear of life in jail, nothing more.

America has effectively incarcerated a lot of people into for-profit prisons without needing this. China has effectively added a lot of population control by rule of fear without needing this. You don't need a high-tech murder-detector to make people fear being thrown in jail for murder.

[1] https://en.wikipedia.org/wiki/Cliff_Richard#Property_search,...

Australian police are allowed to secretly add data to your phone.

The Surveillance Legislation Amendment (Identify and Disrupt) Bill 2020 gives the Australian Federal Police (AFP) and the Australian Criminal Intelligence Commission (ACIC) three new powers for dealing with online crime:

Data disruption warrant: gives the police the ability to "disrupt data" by modifying, copying, adding, or deleting it.

https://tutanota.com/blog/posts/australia-surveillance-bill/

You are correct. Having children already puts you in the system of totalitarian control. This is worse but off topic. CSAM is about using that same incredibly effective method and extending it even to those that do not have children. You cannot even "opt out" by simply staying off cloud services now your own phone is a spy against you.

Think of the children is a devastatingly effective way to slip past peoples defense and have them willing allow their own subjugation.

Again you are correct I don't know any place to look for stats other than my own experience of how shocked people seem to be when some sort of legal issue comes their way. Along with internet comments about just hire a lawyer that you can find in any comment thread about law.

Yep I'm aware of all the power abuse but this is another extension of it aimed at finalizing an end to end digital world control.

I hate this thing as much as the rest of HN but I think you are thinking too far.

CSAM is just a tool that matches image hashes. It can't make accusations. Matching hashes is not illegal, even having pictures of abused children is not illegal, no prosecutor will take your case if it is the only thing you have. At most it will turn you into a witness. If there is an investigation, it can turn you into a suspect, but suspicion is not an accusation. I mean, if your car is parked next to the place a crime happened, you can be suspected, but it doesn't mean you are going to jail because of where you parked your car.

If, for some reason, you get wrongly accused because the system recorded a match and nothing else, it is not a problem of the tool, it is the much bigger problem of justice not working, blame your shithole country if it happens, not Apple.

I am not saying that false accusations don't happen, but saying that police has no other job than to prosecute is a bit much. Police asked me questions a few times, once for a car that caught fire not far away from where I was, once for a missing person, and once they didn't tell me the reason. Was I suspected? Maybe, I was at the "wrong place" after all, but yet, I never heard from them again. If prosecution was their only goal, I would have been in jail now.

> CSAM is just a tool that matches image hashes. It can't make accusations

who validates the source content of the hashes? what stops the government from adding one tiny bitty hash that they know is uniquely present in let's say, assange's phone?

Again a "what if" scenario.

It is not trivial to identify a picture that is uniquely present in Assange's phone. Without having access to Assange's phone that is. And if you have access to Assange's phone, you probably have way more information than such a trick will give you.

But yes, it is a problem, and a reason why I dislike CSAM. But from "allowing the government to track highly sensitive individuals" to "you, the average citizen, will go to jail", there is a huge gap.

Also, I think one thing many people missed is that a hash is not enough. The only thing a hash does it that it allows a reviewer at Apple to decrypt a matching file. If the picture turns out to be nothing special, nothing will happen.

In fact, that feature is much weaker than people make it, it just puts Apple on par with what others like Google, Microsoft, etc... can do. I think the reason it got so much negative press is that Apple made privacy a big selling point. I think it serves them right, I don't like how Apple treats privacy as just a way to attack Google without really committing to it, but still, for the facts, it is a bit unfair.

Reply on news.ycombinator.com