Comment by meterplech | Hacker News Reader

Comment by meterplech | Hacker News Reader

Parent Full thread

There are many photos one could take without uploading or saving them in the cloud. The question is can I safely store things locally on my device without a multi-national corporation being able to set the policy of what's allowed there. The article cites examples of banned memes/images by the Chinese government.

There is also the risk of algorithm mistakes. Without the trust + safety team or any cloud verification, does this just get immediately forward to the FBI?

What is the process to be made non-guilty if the Apple algorithm thinks you have something illegal and you don't if there is no human in the loop involved before it's forwarded to authorities?

At least if you upload to a service someone on the trust + safety team can verify the algo worked properly. I would be very nervous building this kind of system of the risk of false positives! Maybe even so nervous to introduce many more false negatives - which is in itself terrible in the case of something like CSAM.

Here's how it works, in short:

1. Images which are going to be uploaded to iCloud are matched against known hashes. A match has to be very close in order to actually be flagged as a match. The results are uploaded to iCloud Photos along with your image. 2. If your account hits a sufficient threshold of matches (i.e. you have to have multiple/many images matched) then Apple is alerted and they are able to decrypt low-resolution versions of the relevant stored images. 3. Apple's team performs a manual review of the images to ensure that they are in fact inappropriate images and not false positives. 4. If any of the matched images are actually illegal, then Apple reports you to the authorities.

In other words, there are several safeguards in this process:

1. Only if you're uploading files are the files matched. 2. Only if the matches are very close are they considered matches. 3. Only if you have multiple very close matches is Apple able to decrypt the low-res versions of the images themselves. 4. Only if a human reviewer discovers any of the decrypted low-res images to be illegal content is any of your information shared with anyone else.

In my opinion, this is a vastly better system than literally any other cloud provider at the moment; it allows Apple to find and report even more CSAM with effectively no risk of reporting false-positives to law enforcement, while also preserving your privacy through encryption in transit and at rest.

> The question is can I safely store things locally on my device without a multi-national corporation being able to set the policy of what's allowed there.

Didn't Microsoft just start removing people's torrent clients from their computer without permission and announce that you wouldn't be allowed to use your own PC running Windows 11 Home without using an online Microsoft account to log in?

Reply on news.ycombinator.com