Everyone understands well enough how it works and understands how trivially it could be changed to search for other kinds of content.
At some point, the back story behind this will leak out. That will be interesting. This has to be something Apple was asked/pressured/ordered to do. As a business activity it makes no sense.
Bear in mind that over the last week, people in Afghanistan have been frantically trying to erase any evidence of doing things the Taliban doesn't like, such as women playing sports.
This is a fairly typical corporate failure mode.
Sometimes the best thing is to do nothing as they have found out because now they’re stuck in a position where the two exits are either piss off everyone by writing off their privacy stance or piss off everyone else for canning child porn scanning.
What a complete fuck up.
I firmly believe we'll never discover who applied that pressure, or how. And it's hard for me to imagine the US government doing that without some court battle with serious risk of leaks to journalists. I mean, it makes no financial sense, so Apple would fight that. The only thing I can imagine is pressure from a private interest whose market is big enough for Apple to care. And if it is that market, then I think we're in for a tough ride until we can build our own devices.
If you read the documentation on how it's implemented, there's some fairly advanced crypto - private set intersection, threshold secret sharing - that only makes sense if Apple took the line "we have to do this, but we're willing to do it in a really expensive way so that we ourselves have as little access as possible". They went to the effort of running the NeuralHash on the client device, as far as I understand.
The standard implementation on other cloud providers is that the provider has access to your data on the server if they really need it - much cheaper, and makes it technically possible in future to easily change the T&C to "we may use this for market research and to improve our products". I view Apple's client-side implementation as drawing a very big line in the sand saying we will not do that and we are willing to put our money where our mouth is by writing this crypto protocol.
But that certainly raises the question, why would Apple do anything like that at all, unless there's some (current or anticipated) external threat that forced them to?
As to whether we'll ever discover it - I wouldn't have predicted the Snowden revelations, maybe this will come out the next time something like that happens. If it does, I also predict we'll be shocked to discover just how big the market and distribution network for child abuse online is, and how many people are involved.
I don't think it is big compared to what authoritarian states would like to use this for. You're talking about, at the very least, putting everyone in non-democratic states under even more pressure to obey than they're already under. And in democratic states this invites bad actors both in and outside the government. There is no net-gain in justice here. It's wrong across the board.
Welcome back to HN after taking 5 months off, btw. You're a SSC fan?
Yes, I agree the state market is even bigger - China's potential market for surveillance alone is 1.3 billion and rising - but I've heard from colleagues in tech who seriously think child abuse is only a very small number of bad actors. My understanding is the "industry" is almost as out of control as the illegal drugs trade.
I am indeed a SSC/ACX fan (and subscriber).
And now I am down a rabbit hole. Probably for a while. But thanks for that. Seems like the world needs more thinkers like this person, and everyone, including myself could learn a lot from them.
Snowden paid a very high price not many people can afford, especially when they have a family. So I highly doubt we could know it within a reasonable timeframe.
Anything going through a FISA court (Foreign Intelligence Surveillance Court) could come with a gag order, preventing anyone involved to talk about it.
> "Though these capabilities are intended to protect children and to reduce the spread of child sexual abuse material, we are concerned that they will be used to censor protected speech, threaten the privacy and security of people around the world, and have disastrous consequences for many children,"
Many people and organizations fear that Apple's plans are paving the way for governments to scan for more than just child abuse images. (e.g. https://appleprivacyletter.com/)
In that light, FISA is relevant. My statement was to show that not all dealings of the US government are public, because not all courts and legal proceedings are available to the public. So "serious risk of leaks to journalists" won't likely happen in that case.
(One counterpoint to this is that PhotoDNA has been used for scanning cloud images using NCMEC-provided hashes for a decade now and this slippery slope doesn't seem to have happened yet.)
My understanding is that there is a threshold that needs to be met, and then an Apple employee with a very difficult job will need to confirm the image. So even if NCMEC get pressured, I don't see what the result can be.
Replace “FBI” and “Biden” with their equivalents in various other countries and yes, it’s absolutely plausible.
I think it's not really a problem of what is being looked for, but the mere fact that it will be done without a warrant on a personal device.