Google’s Scans of Private Photos Led to False Accusations of Child Abuse | Hacker News Reader

Google’s Scans of Private Photos Led to False Accusations of Child Abuse

eff.org

59 pointscolinprince22 comments

I read the original NYT article as well.

I disagree with the position that the "AI failed." I suspect that the images were pretty much dead on for suspected child abuse image from how they were described in the original article.

What I think is terrible about the story is that Google seemed supremely uninterested in investing any effort whatsoever in helping their customers recover from the false allegations. The customers ended up losing all of the information that they had stored on Google, and the company seemed to treat exoneration of the hideous crime that they accused the customers of to be a triviality not worth their effort.

I am a big fan of Google's technology, and I am deeply wedged into their ecosystem for my personal life (Android, Google Maps, Google Photos, Gmail, etc...). I even pay for premium services. But, it's clear from how they treat Youtube creators and stories like this that, as a customer, they are wholly indifferent to customers as individuals. You have to play defense against their tech.

I don't trust Google with anything that I need to keep either private or access too. The company doesn't care about their "customers" because their actual customers are the advertising companies they sell their "customers" data to. Google has nothing of importance from me because they are unreliable and untrustworthy.

Don't you think that the actions of governments have a lot more to do with what happened here than Google's own policies?

1) Google is forced to do CSAM scanning and report any offending material to authorities by EU

2) they are forced to close accounts where they find this account

3) they are forced to delete any accounts so closed by GPDR regulations

This isn't Google, Facebook or LinkedIn doing this, the only thing Google has to do with this is that they're big enough to be one of the first targets of police forces of multiple nations to implement this.

https://en.wikipedia.org/wiki/EARN_IT_Act

https://european-pirateparty.eu/chatcontrol-commission-prese...

https://en.wikipedia.org/wiki/General_Data_Protection_Regula...

How the police actually deals with these companies:

https://www.zdnet.com/article/signal-unveils-how-far-us-law-...

"Though the judge approved four consecutive non-disclosure orders, the court never acknowledged receipt of our motion to partially unseal, nor scheduled a hearing, and would not return counsel's phone calls seeking to schedule a hearing," Signal wrote.

Oh ... and of course, now many US states have decided that for the police's sacred duty to prevent abortions, the police do NOT have to respect their jurisdiction (they get to attack people for crimes, proven ... or not, committed elsewhere when related to abuse, like CSAM)... Sorry, but the problem here is not tech companies, except in the sense that you may be more or less exposed to this problem depending on how much private info you store with them.

see possible explanation here: https://news.ycombinator.com/item?id=32562052 (basically, the data had already been expunged?)

This is why I don't have a Dropbox account anymore.

I am extremely fortunate that the account that was deleted without recourse only contained data I had copies of on my hard drive, and to my knowledge law enforcement isn't involved.

The article fails to mention the stress and trauma of being accused of having CSAM. That remains to this day ... I'm posting from an alt because even the false accusation carries a potentially career and family destroying stigma.

Something definitely needs to be done about these big tech companies not having any appeals processes when they flag/suspend somebody. I know somebody whose business basically got shut down by Paypal and he never got an explanation or a way to rectify things.

Makes me wonder more and more whether the tech companies are so profitable because of their tech or whether they are so profitable mainly because they cut corners by not hiring people who can deal with customer support. They are the perfect embodiment of faceless bureaucracy and Kafka-esque processes.

This should read as "Google's Scans of Photos Uploaded by User (...)".

Just off the front page:

Google refuses to reinstate account after man took medical images of son’s groin (theguardian.com)

https://news.ycombinator.com/item?id=32560361

I'd use VeraCrypt along with those CSAM enabled cloud, so they cannot read my files in most cases.

(unless they assume my file is so valuable that an exploit is needed to decrypt them)

Previously: https://news.ycombinator.com/item?id=32538805

When their system flagged it as CP and their people reviewed it, how is that not a crime? In their state of mind they knew it was CP and they are not law enforcement or government lawyers. Civilians at the behest of google watched CP amd there was no prosecution! Isn't that another big story here?

I absolutley think CSAM as an idea is a good thing so long as any fault in the algorithms are taken into account and it is law enforcement that does the investigation begining to end. Google has the right to refuse service, but not with cause. They slandered a person and cancelled service before the persons had time to migrate causing considerable material damage. How much monetary value do you place on sentimental pictures or email messages? If money is an issue let us donate to the legal funds of these people but they should absolutley sue the crap out of google.

I don't know if there is a civil cause for action in any CP related law but if google employees who are not law enfrocement or authorized medical professionals saw the private pictures of my child I would be raising hell in form of lawsuits and in every avenue where I can make this known including election campaigns against the DA that didn't prosecute google employees and execs for intentional viewing of CP.

You don't like CSAM as a concept? Ok, I get the arguments against it but laws are too humane against child predators. Until castrations , public executions and death by torture are on the books (prevention) then CSAM like approach is a must (detection). Refusing to take a strong stance on prevention and on detection is enabling the real predators.

This is not a constructive approach to fight child abuse. Such scanning is just a pretense for the state to be able to access private communications and it needs to be stopped right in its infancy. States accessing too much information has been a large detriment on society as it is and those responsible need to be deprived of office and responsibility.

Child abuse needs the help of people invested in children to detect instances of abuse. Internet surveillance can only do so much. The best approach is to delete any content immediately to not let it spread. This is done pretty much anywhere, you won't even need legislation. Perhaps connection info might help authorities, but I guess most perpetrators will take that into consideration as well.

Then get proper inhumane punishment as a deterrent? You cannot have tech that enables harmful content and refuse to all measures to deter harmful content. What people invested in children? Some guy in the backwoods of nowhere takes abusive pictures who will know other than tech companies? Delete the content so it doesn't spread in the mean time the child gets no help? Is that your answer? I don't want them to spy on our private content, in every scenario other than this one I agree with you 100% i am very much against government and corporate surveillance and support e2e but, mine and your right to privacy does not supersede the rights of the most innocent and defenseless members of society to be safe and protected from harm. You want to keep privacy then join me in supporting inhumane punishment for child predators. The deterrent aspect of unthinkably horrible punishment is enough to say there is no need for drastic measures like spying on everyone's pictures. You can't have it both ways.

Are you suggesting law enforcement personnel should be embedded in Google to do the confirmation of CSAM?

Someone has to confirm CSAM and report to the police. Or would you rather an auto-report directly to the police from the algorithm that flagged it with no human oversight?

Auto report to police when there is algorithmic match then the police do their own vetting and decide to open an investigation,prosecute,etc... no punitive action by google until the police decide to prosecute. Google employess looking at cp is a crime since they are not legally authorized or perimitted by the parents to look at the pictures in any capacity. LE/gov also has no authority to delegate this task to undeputized civilians.

The blog post about the EU proposal is insanely scary. It "boasts" about 88% accuracy in "grooming" and 90% accuracy in new CSAM. And it also has this line, which is just so obviously horrendous:

>the proposed obligations on service providers as regards the detection of CSAM are technologically neutral, meaning they do not prescribe which technology should be used for detection as long as the chosen technology meets the high standard of conditions and safeguards to protect all users. It is an obligation of result, not of means.

Literally saying "The ends justify the means" when it comes to detecting CSAM. Remember, it NEVER has to do with protecting anyone. Governments do what gives them more power or they are already incentivized to do, and nothing else. And with no voter pressure against these types of laws, it's obvious what will happen.

https://ec.europa.eu/commission/commissioners/2019-2024/joha... Here is the link to the blog post.

That seems like an uncharitable read. The phrase "an obligation of result, not of means" simply stating that the regulation doesn't choose a specific methodology, as it only prescribes a standard for its results. It's as if regulations for clinical thermometers require accuracy within a 0.1C margin of error, but manufacturers can choose how the device works, and how to make it.

Of course, the scary part is that they would consider an 10% error rate to be OK.

In what universe should I be charitable to the government?

> Literally saying "The ends justify the means" when it comes to detecting CSAM...

I don't read it like that. My reading of it is that the EU don't care what technology is being used as long as the technology works as prescribed, that is, the result is CSAM detection regardless of the means by which it was detected.

I don't read anything in that about government power or government overreach, regardless that these technologies are clearly very intrusive.

That is not what technologically neutral means. It means that the EU will not prescribe MS whatever or Google such and such, but that states can choose software themselves, assuming it works good enough (which apparently is not a very high bar to clear in this case).

How would you determine efficiency then? This will also only ever apply popular services. No random e2e encrypted messenger will implement this.

> It "boasts" about 88% accuracy in "grooming" and 90% accuracy in new CSAM

Aside from principals, this is actually very nice accuracy technologically.

Reply on news.ycombinator.com