Hacker News Reader

Comment by icelancer | Hacker News Reader

> Of course, grey noise will never pass for CSAM and will fail that step.

Never? You sure that one or more human operators will never make this mistake, dooming someone's life / causing them immense pain?

View on HN

I can guarantee nobody will see the inside of a courtroom, on charges of possession and distribution of child porn for possessing multiple images of grey noise (unless there is some steganography going on).

Many people never see the inside of a courtroom when false or unproven rape accusations are made against them, but their lives still get ruined because of the negative publicity.

Are you suggesting that perhaps less people should report rape accusations, because it might be awkward for the accused to get negative publicity? Thats messed up.

Those cases are not comparable, because the whole reason they have that impact is that the accusations are usually made publicly (because the whole point is to harm the reputation of one's rapist and warn others, should a conviction prove to be impossible), while CSAM review goes through a neural hash privately on your phone, then privately and anonymously through an Apple reviewer, then is privately reviewed at NCMEC (who - I think - have access to the full size image), and only then is turned over to law enforcement (which should also have access to the full image).

It only becomes public knowledge if law enforcement then chooses to charge you - and if all that happens on the basis of an obvious adversarial net image, the result is a publicity shitshow for Apple and you become a civil rights hero after your lawyer (even an underpaid overworked public defender should be able to handle this one) demonstrates this.

As others have stated in this thread, I think the real failure case is not someone's life getting ruined by claims of CSAM possession somehow resulting from a bad hash match, but the fact that planted material (or sent via message) can now easily ruin your life because it gets automatically reported; you can't simply delete it and move on any more.

We give way, way too much weight in the legal system to eye witness and victim statements, in absence of any corroborating evidence. That's a problem.

But not really comparable, IMO. You won't even know you got investigated until after the original images have been shipped off to NCMEC for verification.

One does not need to go to court to have their life ruined by accusations. Ironically, there's quite a few examples of this over the years for alleged CSAM.

One example that sticks out in my mind is a pair of grandparents who photographed their grandchildren playing in the back yard. A photo tech flagged their photo, they were arrested, and it took their lawyer going through the hoops to get a review of the photo for the charges to be dropped.

Sure, there are always cases - but was their photo of a grey blob that matched a hash but is clearly a grey blob or of a naked child?

If the photo was a grey blob and they had to go through a judicial review for someone to look at the photo and confirm 'yes that is a grey blob' then color me wrong.

I'd view a "grey blob" to be the MVP of hash collisions. I doubt that this will end with grey blobs - I see it ending with common images (memes would be great for this) being invisibly altered to collide with a CSAM hash.

If you need a judicial review to confim that a slightly altered Bernie in Coat and Gloves meme is not the same image as the picture of a child being raped that they have on file then we have way bigger problems.

Here's the thing with CSAM - it's illegal to view and transmit. So nobody, until the police have confiscated your devices, will actually be able to verify that it is a "child being raped."

They'll view visual hashes, look at descriptions, and so forth, but nobody from Apple will actually be looking at them, because then they are guilty of viewing and transmitting CSAM.

I noted in another comment, even the prosecutors and defense lawyers in the case typically only get a description of the content, they don't see it themselves.

This is just not true. Human review is conducted. Apple will conduct human review, facebook conduct human review, NCMEC will conduct human review, law enforcement will conduct human review, lawyers and judges will conduct human review.

Over the years there have been countless articles etc about how fucked up being a reviewer of content flagged at all the tech companies is. https://www.vice.com/en/article/a35xk5/facebook-moderators-a...

Where did you get this idea, scooby doo?

It is not illegal to be an unwilling recipient of illegal material. If a package shows up at your door with a bomb, you're not gonna be thrown in jail for having a bomb.

In theory, sure.

At the very least, you'd be one of the primary suspects, and if you somehow got a bad lawyer, all bets are off.

https://hongkongfp.com/2021/08/12/judges-criticise-hong-kong...

Okay, and when a cursory look at the bomb actually reveals it to be a fisher-price toy, what then?

What is the scenario where a grey blob gets on your phone that sets off CSAM alerts, an investigator looks at it and sees only a grey blob, and then still decides to alert the authorities even though it's just a grey blob, and the authorities still decide to arrest you even though it's just a grey blob, and the DA still decides to prosecute you even though it's just a grey blob, and a jury still decides to convict you, even though it's still just a grey blob?

You're the one who's off in theory-land imagining that every person in the entire justice system is just as stupid as this algorithm is.

Possession of CSAM is a strict liability crime in most jurisdictions.

That is simply not true. There is no American jurisdiction where child pornography is a strict liability crime.

On this topic, the Supreme Court has ruled in Dickerson v US that, in all cases, to avoid First Amendment conflicts, all child pornography statutes must be interpreted with at least a "reckless disregard" standard.

Here is a typical criminal definition, from Minnesota, where a defendant recently tried to argue that the statute was strict liability and therefore unconstitutional, and that argument was rejected by the courts because it is clearly written to require knowledge and intent:

> Subd. 4. Possession prohibited. (a) A person who possesses a pornographic work or a computer disk or computer or other electronic, magnetic, or optical storage system ․ containing a pornographic work, knowing or with reason to know its content and character, is guilty of a felony․

I tried to look up [Antonio] Dickerson v. US, but I don't see any SCOTUS decision on it, only a certiorari petition. Do you have a reference for the decision?

Yep, sorry, Dickerson was an appellant that cited the relevant case law, which is New York v Ferber.

Now, Dickerson rightfully lost and it's appropriate that SCOTUS rejected his case because he was involved in child porn production, not posession, so he can't rely on the Ferber precedent. He had the opportunity to ask the underage person in question their age, and chose not to, which would meet the reckless disregard standard anyway.

I don't see any mention of the reckless disregard standard in the New York v. Ferber decision, either? So far as I can see, it just says that CSAM is outside of the scope of 1A, so long as it's "adequately defined by the applicable state law". Am I missing something in the opinion?

What if it is legal pornography of 21 year olds but disturbed to collide with CSAM? You are aware even defence lawyers are not allowed to look at alleged CSAM material in court right?

The process would not trigger any action. The NCMEC, who can look at the material, and are the people to whom the matter is reported, would compare the flagged image with the source material and reject it as not matching known CSAM.

What if the legal porn of a 21 year old that triggered the collision match looked really really really close? So close that a human can not distinguish between the image of a 12 year old being raped that they have in their database and your image? Well then you might have a problem, legal and otherwise.

> defence lawyers are not allowed to look at alleged CSAM material in court right

I know this is not true in many countries, but cant speak for your country.

You are aware that a lot of CSAM are close ups of say pussies for example, and human anatomy can look very similar?

I'm not talking about images of rape here. I'm taking about images that you'd see on a regular porn site, of adults and their body parts.

You are also aware that CSAM covers anywhere from 0 to 17.99 years of age, and the legal obligation to report exists equally for the whole spectrum?

So let's say I download a close up pussy collection of 31 images of what I believe to be consenting 20 year olds, and what are consenting 20 year olds.

But they are actually planted by an attacker (let's say an oppressive regime who doesn't like me) and disturbed to match CSAM, that is, pussy close ups of 17 year olds. They are all just pussy pics. They will look the same.

Should I go to jail?

Do I have a non zero chance of going to jail? Yes.

If you have content that has a matching hash value and is identical by all computational and human inspection to content that has been identified as CSAM from which the hash was generate then you have a problem.

Without getting into the metaphysics of what is an image, at that point, you basically have a large collection of child porn.

Your hypothetical oppressive regime has gone to a lot of trouble planting not illegal evidence on your device. It would be much more effective to just put actual child porn on your device, which you would need to have to conduct the attack in the first place.

> You are aware that a lot of CSAM are close ups of say pussies for example, and human anatomy can look very similar?

I doubt images that look quite generic will make it into those hash sets, though.

Nearly impossible to verify, though, by construction.

What is the factual basis for you to doubt that?

> I can guarantee nobody will see the inside of a courtroom

This wasn't the question I asked.

> grey noise will never pass for CSAM > You sure that one or more human operators will never make this mistake.

Yes I can say 100% that no human operator will ever classify a grey image for a child being raped. Happy to put money on it.

It's possible that the operator accidently clicks the wrong button.

When dealing with a monotonous task that the operator is probably getting PTSD from, I think the chance is greater than 0%.

Articles about content moderators and PTSD:

https://www.businessinsider.com/youtube-content-moderators-a...

https://www.bbc.com/news/technology-52642633

https://www.theverge.com/2019/2/25/18229714/cognizant-facebo...

Reply on news.ycombinator.com