Hacker News Reader

What about trolling. Assume 4chan figures out apples algorithm. What now happens when they start generating memes that happen to match known child pornography? Will anyone who saves those memes (or repost them to reddit/facebook) be flagged? What will apple do once flagged false positive photos go viral?

One way this hair-brained Apple program could end is to constantly generate an abundance of false positives, and try to render it useless.

For those old enough to remember “Jam Echelon Day”, maybe it won’t have any effect. But what other recourse do we have other than to maliciously and intentionally subvert and break it?

What recourse? how about not buying apple devices that have always locked the user into a golden cage?

You would be naive to believe other companies don’t do this or eventually will not do this.

In the same way I was naive to believe Apple was different from others and cared about user privacy.

>> Will anyone who saves those memes (or repost them to reddit/facebook) be flagged?

Shouldn't they be?

Umm, no? If someone happens upon some funny cat meme that 4chan users made with an intentional hash collision then they're not guilty of anything.

A poor analogy could be trolls convincing a flash mob to dress like a suspect's description which they overheard with a police scanner. No one in the mob is guilty of anything more than poor fashion choice.

The point made was that there are always flaws in these sorts of approaches that lead to false positives. If you can discover the flawed pattern(s) that leads to false positives and engineer them into seemingly harmless images, you can quite literally do what OP I'd suggesting. It's a big IFF but it's not theoretically impossible.

The difference between this and hashes that require image data to be almost identical is that someone who accidently sees it can avoid and report it. If I can make cat photos that set off Apple's false positives, then there's a lot of people who will be falsely accused of propagating child abuse photos when they're really just sending cat memes.

> like the dogs which conveniently bark at police's secret hand sign

This isn't necessary; the state of the art is for drug dogs to alert 100% of the time. They're graded on whether they ever miss drugs. It's easy to never miss.

Airport baggage drug dogs must obviously have far fewer false positives than that. So alerting on everything can't be the state of the art.

https://reason.com/2021/05/13/the-police-dog-who-cried-drugs...

> Similar patterns abound nationwide, suggesting that Karma's career was not unusual. Lex, a drug detection dog in Illinois, alerted for narcotics 93 percent of the time during roadside sniffs, but was wrong in more than 40 percent of cases. Sella, a drug detection dog in Florida, gave false alerts 53 percent of the time. Bono, a drug detection dog in Virginia, incorrectly indicated the presence of drugs 74 percent of the time.

I was pulled over in West Baton Rouge in 2009, we were driving east in an empty rental box truck after helping a friend move back to Tx. It was 1am, we were pulled over on a BS pretense (weaving across lanes when we signaled a lane change because they had someone else pulled over, so we obeyed the law of pulling over a lane to give them room). I denied their request to search the truck, they had no reason. They called the drug dog, who after the third walk around, "signaled" drugs at the right front tire (after having thir leash jerked). They then "had cause" to search the truck. After finding two small suitcases with clothes (exactly what we told them they'd find), the main cop got really angry with me for "making a mockery of the south", threw the keys at me and told us to GTFO.

I'm 100% convinced drug dogs are trained to "signal" falsely at certain things like a leash tug. It's all BS.

Yikes. Seems like the biggest group of people making a mockery of the south is the southerners like this guy who insist on acting like cartoonish southern stereotypes.

I should also add that dogs and many other animals really like pleasing people. So one doesn't even have to consciously train for outcomes like this. A famous example is Clever Hans, the horse that supposedly could read, do math, and answer questions like "If the eighth day of the month comes on a Tuesday, what is the date of the following Friday?" https://en.wikipedia.org/wiki/Clever_Hans

I've had my bag sniffed at airports at least 50 times, and they've never stopped me. So there must be something else going on as well

The police only call in the dogs when they wish to search and the driver does not agree. The airport doesn't do things this way so the same strategy wouldn't work.

The handler didn't think you looked untrustworthy and didn't hint for the dog to bark.

Drug detection dogs sit down near the bag. They don't bark.

Airport dogs (at least in the baggage claim area) are not sniffing for drugs. They alert on food products that aren’t allowed.

And explosives.

The dogs which alert on explosives are not usually in the baggage claim area.

Out of how many dogs? We’re these outliers or the regular thing?

What feedback loop is built into the system to discourage this from being the regular thing?

To be more precise, if the dog always barks and only missing positives are counted, it is inevitable to never miss. An obvious number cheat.

Dogs are used to protect police from accusations of racism and profiling.

Which is odd as dogs can be just as racist as their handlers want.

Well, presumably at that point, someone in that position would just reveal their own files with the hash an prove to the public that they weren't illegal. Sure, it would be shitty to be forced to reveal your private information that way, but you would expose a government agency as fabricating evidence and lying about the contents of the picture in question to falsely accuse someone. It seems like that would be a scandal of Snowden-level proportions.

Na they will ruin your life even if you are found innocent and pay no price for it.

That's the problem: the terrible asymetry. The same one you find with TOS, or politicians working for lobbists.

Who would a company hire: the candidate with a trial for CP due to a false positive or the candidate without ?

And this is just to address the original concept of this scanning.

As many others have pointed out there is too much evidence pointing to other uses in the future.

> Who would a company hire: the candidate with a trial for CP due to a false positive or the candidate without ?

First time I've seen it abbreviated like that; took me a while to grasp. Well, more of a plausible "Enemy of society" than what I came up with: https://news.ycombinator.com/item?id=28060995

There are literally hundreds of cases of police fabricating evidence and getting caught in court, or on bodycam.

This happens today. We must not build technology that makes it even more devastating.

”Sorry, but collisions happen with all hashing algorithms, and you can’t prove otherwise. It is just a matter of time. Nothing to see here.”

Well, not all hashing algorithms but all interesting or considered useful hashing algorithms, probably.

When dealing with say countable infinite sets you can certainly create a provable unique hash for each item in that set. The hash won't be interesting or useful. E.g. a hash that indexes all the integers n with a hashing function h(n+1)... so every integer you hash will be that value plus one. But this just being pedantic and wanting to walk down the thought.

In the past the FBI used some cryptographic hash. Collisions with a secure cryptographic hash are functionally unobservant in practice (or else the hash is broken).

The use of the perceptual hash is because some people might evade the cryptographic hash by making small modifications to the image. The fact that they'd discarded the protection of cryptographic hashing just to accommodate these extra matches is unsurprising because their behavior is largely unconstrained and unbalanced by competing factors like the public's right to privacy or your security against being subject to a false accusation.

It wouldn't prove anything, because hash functions are many-to-one. It's entirely possible that it was just a coincidence.

You can reveal your files and people can accuse you you deleted the incriminating ones.

Not if you show the file that matches the perceptual hash that "caught" you.

Would a court be compelled to provide that hash to your defence? Arguable as it could be used by criminals to clean their collection. And by that time your life is ruined anyway.

So Apple-users can no longer delete any pictures since Apple might already have reported that photo you accidentally took of your thumb as CP.

The way I see it, this is the only possible purpose this system could have. With the press after this announcement, almost every single person in posession of those materials knows it's not safe to store them on an iPhone. By it's construction, this system can only be effective against things that the owner is not aware their phones are being searched for.

Parallel construction is another way this is often pursued.

> Demand the original intended file for the hash?

Even if they'd provide it-- the attacker need only perturb an image from an existing child abuse image database until it matches the target images.

Step 1. Find images associated with the race or political ideology that you would like to genocide and compute their perceptual hashes.

Step 2. Obtain a database of old widely circulated child porn. (Easy if you're a state actor, you already have it, otherwise presumably it's obtainable since if it wasn't none of this scanning would be needed).

Step 3. Scan for the nearest perceptual matches for the target images in the CP database. Then perturb the child porn images until they match (e.g. using adversarial noise).

Step 4. Put the modified child porn images into circulation.

Step 5. When these in-circulation images are added to the database the addition is entirely plausibly denyable.

Step 6. After rounding up the targets, even if they're allowed any due process at all you disallow them access to the images. If that dis-allowance fails, you can still cover by the images existing and their addition having been performed by someone totally ignorant of the scheme.

I know this is a tough thing to consider, but:

Isn't this a problem generally with laws against entire classes of media?

Planting a child abuse image (or even simply claiming to have found one) is trivial. Even robust security measures like FDE don't prevent a criminal thumb-drive from appearing.

I think we probably need to envision a future in which there is simply no such concept under law as an illegal number.

The police can arrest you for laws that don't exist but they think exist. They don't need to any of this stuff.

> Suppose the authority want to false-arrest you.

Why would they want that?

Corruption. Lack of evidence on some other cases. Personal revenge. Who knows, but list is big.

Ok but but what ends?

Imagine you’re a journalist uncovering corruption perpetrated by the police force or a politician. Can you see how they would be incentivised to arrest you on false charges to halt the investigation and protect themselves?

Goodluck getting a promotion if you never arrest anyone. I know some cops who plant weed almost every spring, just to "bust" it every fall.

performance metrics and careers exists in law enforcement too.

eg:

https://listverse.com/2016/11/06/10-undercover-cops-who-went...

This is a pretty weird question considering the mountains of documentation of authorities doing just that. This is not some kind of hypothetical that needs extraordinary justification.