Hacker News Reader

Comment by imroot | Hacker News Reader

https://towardsdatascience.com/black-box-attacks-on-perceptu...

I mean, who's to say that my hash is actually what they think it is...

...and if it's not uploaded to iCloud for someone to manually review what the image is, is that going to be enough to get a search warrant for my phone?

"AUSA SOMEONE: But, Your Honor, The Defendant's phone has an image that matched a hash of a CSAM picture!"

"DEFENDANT: I'm not going to give you my phone's password."

"THE COURT: Bonk: Stay in jail for contempt."

I understand why they're doing it. I don't agree with it. At all, and I say that as a survivor of childhood sexual abuse.

View on HN

I have always been concerned that this system could be weaponized as a way gain access to someone's account. For example:

- Add the hash of a non-pornographic image to the database

- Using a burner email address, email the non-pornographic image to the target's Gmail address. The target wouldn't think anything of it.

- The innocent image would trigger a CP alert, giving law enforcement the pretense it needs to access the account

This is how the police gets warrantless searches "for drugs" - having identified the subject, call for a dog and have it "alert" on the subject (or the vehicle belonging to the subject), after which there's a probable cause for a search and everything that is found can be used against the subject. The fact that there was no drugs is irrelevant.

Now let's translate it to digital world and eliminate the non-scalable components - and you get AI "alerting" on the content of your account and it's a probable cause for searching you.

But the best part is of course the Apple doesn't even need a probable cause to search you - they aren't the police, so they are free to search anything at their pleasure and then report to the police. And this report, by itself, would be the probable cause, of course.

EDIT: It appears they are using perceptual hashes, which are likely much more prone to collisions. The following math does NOT apply to the tech Apple is using.

>> I mean, who's to say that my hash is actually what they think it is...

Let's do the math. assume they are using a UUID for the hash. "For example, the number of random version-4 UUIDs which need to be generated in order to have a 50% probability of at least one collision is 2.71 quintillion" [1]

"The odds of This number is equivalent to generating 1 billion UUIDs per second for about 85 years." And that's for just 1 collision between two random photos. Considering that the pool of bad hashes is not going to be 2.7 quintillion but a fraction of that, this is more than adequate.

And of course they could use a longer checksum if that's the concern. (Not to take a stance on the parent issue, just to explain that hash's are probably a higher standard of proof than DNA matches)

1. https://en.wikipedia.org/wiki/Universally_unique_identifier#...

They're doing perceptible hashes, not file hashes. This was covered earlier in a different article posted here.

What if an attacker has intentionally produced an image with the same hash as a well known blacklisted image?

Then they could send you the new image and fuck up your life.

Hell, what if you were browsing a site like Facebook or Imgur or Reddit, saw an abuse image, reported it, but it still lives in your phone's browser cache?

This is exactly why I use firefox and switch it to in-memory caching only.

I would assume those sites are already using the same, or similar, algorithms/hashes to detect blacklisted images?

But compare practice to DMCA Takedown notices. Lots of false positives, because there's no penalty for mistakes.

> "The odds of This number is equivalent to generating 1 billion UUIDs per second for about 85 years."

If we're talking about random files, sure.

What if the files are intentionally created to match the desired hash? The malicious actor doing this could be a private party intending to disrupt your life, but it could also be law enforcement as a means of gaining access to your device when it would otherwise require a warrant.

Couldn't someone that motivated come up with a dozen other ways to frame someone?

This seems a little alarmist to me.

They could come up with a dozen other ways apart from swatting too, but swatting still happens and we generally agree that it's a problem where actions should be taken. Political debates over whether that means the police should act in a less militarised fashion, or harsher sentences for culprits are ongoing, but cutting down on avenues for life ruining abuse is generally supported.

So why would we open up new ones?

SWATing is actually the absolute cheapest option - if you want to ruin someone's week from across the nation (or across the ocean), your best bet is a burner phone calling the person's local 911 with a threat that warrants a SWAT team response (I am of course not advocating for anyone to do this, just explaining the process malicious actors currently use to trigger these). Outside of that, you'd need to pay a lot of money for something like a PI or obfuscated malware to steal PII or credentials from the victim.

I hope you don't ever run afoul of an internet mob from somewhere like 4chan! They might SWAT you while DDOSing your IP and doxing you then emailing your iphone dozens of photos that collide with CSAM perceptual hashes.

LEO have never, ever planted a baggie of weed in someone's car. Likewise, LEO would never, ever ensure that a picture that matched a known phash gets on a target's phone.

Once someone develops the tool for that, it WILL become available in the wild. Then, any script kid will be able to "swat" you in a even more insidious way.

Or what if you're a researcher who has downloaded a collection of these assets? Wouldn't you be a very high signal hit, beyond even actual perpetrators of crime? Would you need the protection of an umbrella organization to do research? Or register with the government perhaps?

I'm as against this as anyone,but to be blunt:

This is digital biowarfare material of the worst kind and should be handled as such.

Any researcher downloading this material to anything but lab machines should probably receive a visit from the relevant authorities to make sure they understand the seriousness of it.

The law doesn't have an exception for independent research.

> What if the files are intentionally created to match the desired hash?

That's called a second-preimage attack and is similarly infeasible to achieve against a cryptographically secure hash function.

There are good arguments to be made for why this is a bad idea, but hash collisions aren't one of them.

But this isn't your average hash function.

This is a function that is created to be robust against modifications to the image.

A perfectly robust function would - in my opinion - be one that could detect even a recreation of the original setup.

I don't know where on this scale the function law enforcement use is, but if it is any good I struggle to see how it cannot detect similar legal imagesm

Fifth amendment holds that the state can't compel someone to give up their password.

But that hasn't stopped judges from demanding it and keeping people in jail for contempt of court when they refuse.

But you can be jailed for 18 months for not giving it up

Not if your lawyer is any good.

https://arstechnica.com/tech-policy/2020/02/man-who-refused-...

Nope, federal law states they can hold you for 18 months. So long as they can claim to already know what's on the device (which they will, because Apple pinged them about it), they can hold you for 18 months.

Alright, after further investigation it looks like the Supreme Court has so far declined to rule on this issue so it depends on the state. It sounds like most states supreme courts have ruled that you can't be forced to unlock a device, but some, like pennsylvania say that if the state can prove you know the password they can force you to unlock the device.

Definitely a pretty thorny situation.

Except 100 miles from the border apparently

https://www.aclu.org/other/constitution-100-mile-border-zone

The 100 mile zone is clearly fucked up, but I haven't seen any cases of it being used to force anyone to give up their password. Even with a warrant US citizens do not have to give up their password. The only exception I've seen is a case where the defendant shared some info on their phone but refused to share everything. The judge ruled the defendant waived his fifth amendment rights in that case. I think it's clear that that was the wrong decision, but the solution is clear and easily done, don't give cops any information.

I understand the danger of the false positive but I suspect someone with actual child pornography on their device would trigger a dozen or more matches. Clever software would ignore a single match.

Reply on news.ycombinator.com