Hacker News Reader

Comment by Jtarii | Hacker News Reader

I think a more realistic system would be hashing images and comparing them to known CSAM in some database.

I think Apple was going to implement something like this a few years ago before scrapping it.

Such systems are unfortunately trivially defeated by, for example, adding rings of colourful blocks around the edges of the image. If the scanning systems are then updated to notice rings around the image, bad actors will start cutting images in half and adding the colours there, etc. It's a never-ending arms race that's bound to leave regular people worse off.

Don't you think modern image analysis tools such as llms will be easily defeated by measures such as adding colored rings?

A couple of years ago you could add some pixels to an image to change it's automatic classification from cat to ostrich. But the tech has improved and I think the race has now firmly been won by the side trying to de-obfuscate images, and only in rare scenarios can images actually be obfuscated efficiently and consistently.

> Don't you think modern image analysis tools such as llms will be easily defeated by measures such as adding colored rings?

The LLMs think one specific mystery plant in my garden hiding behind the hedge is velvetleaf; no, wild cotton; no, linden; no, hibiscus; no, mulberry; no, knotweed; no, paulownia; no, catalpa; no, hydrangea; no, grape vine; no, pokeweed.

Many of these claims were trivial for me to falsify with a quick image search, they don't look much like each other or my mystery plant. The things the AI "identified" were often simply not true of the photo.

Basically, even with current tech, you go straight back to false positives and false negatives: https://arxiv.org/abs/2111.06628

That assumes no new material is being created.

"But the call was coming from inside the house": https://www.robertkinglawfirm.com/mass-torts/grok-lawsuit/

I don't really want governments, Apple or Google to build databases of CSAM, thank you :/

It already exists. Most CSAM tools are checking against a hash database.

Why would you NOT want this to exist? This seems the only method to not raise false positives.

Let's delete it then.

Why? Because I don't want such pictures to be kept or archived by anyone. This isn't hard.

I'm unsure how these databases work, although it could simply be a database of hashes, with the file contents deleted. That sounds like the sane, safe way to handle this.

Reply on news.ycombinator.com