Apple enabling client-side CSAM scanning on iPhone tomorrow
twitter.com
twitter.com
For context, I deeply hate the abuse of children and I've worked on a contract before that landed 12 human traffickers in custody that were smuggling sex slaves across boarders. I didn't need to know details about the victims in question, but it's understood that they're often teenagers or children.
So my initial reaction when reading this Twitter thread was "let's get these bastards" but on serious reflection I think that impulse is wrong. Unshared data shouldn't be subject to search. Once it's shared, I can make several cases for an automated scan, but a cloud backup of personal media should be kept private. Our control of our own privacy matters. Not for the slippery slope argument or for the false positive argument, but for its own sake. We shouldn't be assuming the worst of people without cause or warrant.
That said, even though I feel this way a not-small-enough part of me will be pleased if it is deployed because I want these people arrested. It's the same way I feel when terrorists get captured even if intelligence services bent or broke the rules. I can be happy at the outcome without being happy at the methods, and I can feel queasy about my own internal, conflicted feelings throughout it all.
This kind of thing isn’t even unusual either. I know my parents have pictures of myself and my siblings playing in the bath (obviously taken on film rather than digital photography) and I know friends have pictures of their kids too.
While the difference between innocent images and something explicit easy for a human to identify, I’m not sure I’d trust AI to understand that nuance.
In this case it’s not AI that’s understanding the nuance, it’s authorities that identify the exact pictures they want to track and then this tool lets them identify what phones/accounts have that photo (or presumably took it). If ‘AI’ is used here it is to detect if one photo contains all/part of another photo, rather than to determine if the photo is abusive or not.
Although there is a legitimate slippery slope argument to be had here.
I have to assume humans are involved at some point before journalists are notified. The false-positive will be cleared up and no reputations sullied (except perhaps the reputation of using AI to scan for digital fingerprints).
Documents could also be engineered to trigger false positives, making it difficult or impossible for a corporate whistleblower to photograph incriminating evidence to deliver to the authorities.
So, if the rumors are true and every iPhone will check every photo against an opaque database of perceptual fingerprints, what safeguards exist (beyond "trust us" from the database keepers) to prevent abuse of the feature to suppress evidence and control the flow of information, and which organizations or governments will have control over the contents of the database? As always, who watches the watchers?
This is dangerously naive. The US justice system alone will hound people on goosed up charges and try to get people to accept a plea deal and write a bogus confession. Parallel construction. Additionally if you can't audit the database (I'd bet very few people can, including your senator) how do you know a hash of something not CP wasn't inserted into the database. This entire system screams ready for govt overreach. It's worse than normal since there'll be no public evidence when it's abused.
That you even have to consider sexual interpretations of your BABY'S GENITALS is an affront to me. I have pictures of my baby completely naked, because it is, and I stress this, A BABY. They play naked all the time, it's completely normal.
As that seems unlikely, I guess CSAM just uses a constantly updated database of known hashes for matching.
I say let children be free, no court is going to indict you because you have baby pictures on your phone.
Maybe, maybe not. Bad luck is possible with anything involving police, prosecutors, judges, and juries. Need justification for that point of view? Just look at the number of people who were convicted and spent time in jail who truly were innocent. That doesn't even touch on the possible repercussions that can happen from just being questioned/arrested and later let go.
I recall a story several years ago where someone was getting the film developed at a local drugstore, and the employee reported them for CP because of bath photos. This was definitely a thing before computers with normal every day humans.
Your intent when producing the image was irrelevant.
Just playing devil's advocate, my gut (and I think even considered) reaction is in alignment with surely just about the whole tech industry: it's over-reach (if they're not public images).
After all the courts he ended up with 13 years in prison, where he is likely going to die.
As mentioned in the twitter thread, how does image hashing & recognition tools such as PhotoDNA handle adversarial attacks?[2][3]
[1] https://news.ycombinator.com/item?id=27959755
[2] https://news.mit.edu/2019/why-did-my-classifier-mistake-turt...
[3] https://towardsdatascience.com/black-box-attacks-on-perceptu...
If Android then implements something similar, they have the option to simply run different software, as Android lets you run whatever you want so long as you sign the wavier.
"You're using Android?! What do you have to hide?" -- Apple ad in 2030, possibly
In my (uninformed) opinion - this looks like more of a bad faith move on Apples part that will maybe catch some bad actors but will be a net harmful result for apple’s users and society, as expressed in the Twitter thread.
Others who responded here though also seem to think it’ll be a viable technique.
This should be reason enough for you to not support the idea.
From day 1, it's matching legal images and phoning home about them. Increasing the scope of scanning is barely a slippery slope, they're already beyond the stated scope of the database.
The thing I can publicly say is that the database is not strictly for illegal or even borderline imagery.
NCMEC try to keep the contents, processes and access to the database under wraps for obvious reasons.
By both a legal and moralistic standard they're not CSAM. Not even nearly.
Of course, this is a minority of the content in the database. But even 1 such image is gross neglect of stated purpose in my book.
Why would I have any content in my phone that would be in that database?
- A kitchen with nobody in frame.
- A couch with nobody in frame.
- Outdoor scenery with nobody in frame.
- A bathroom with nobody in frame.
Is it hard to believe you wouldn't download something like this without knowing where it came from?
I'm not talking about borderline stuff. I'm talking about content that has not even a hint of pornography or illegality.
Remember, these databases are essentially signature DB's, and there is no guarantee that all hashes are just doing a naive match on the entire file, or that all scans performedare fundamentally the same.
This is why I reject outright the legitimacy of any Client-based CSAM scanners. In a closed source environment, it's yet another blob, therefore an arbitrary code execution vector.
I'm sorry, but in my calculus, I'm not willing to buy into that, even for CSAM. It won't stay just filesystems. It won't stay just hash matching. The fact there's so much secrecy around ways and means implies there's likely dynamicity in what they are looking for, and with the permissions and sensors on a phone that many apps already ask for, my not one inch instincts are sadly firmly engaged with no signs of letting up.
I'm totally behind the fight. I'm not an idiot though, and I know what the road to hell is paved with. Law Enforcement and anti-CSAM agencies are cut a lot of slack, and enjoy a lot of unquestioning acceptance by the populace. In my book, this warrants more scrutiny, and caution not less. The rash of inconvenient people being rather frequently called out as having CSAM found on hard drives in media with no additional context indicates the CSAM definition is being wielded in a manner that produces a great degree of political convenience.
Again, more scrutiny, not less.
In principle, the OS environment could be made independently auditable - keeping undeleteable signed logs.
A person who creates CSAM likely doesn't just create CSAM all the time, right? Those innocuous pictures get lumped together with illegal content and make it into the database.
The database is a mess, basically. Of course it is. It's gigantic beyond your wildest estimates.
That’s not entirely true. If a police officer finds you in possession of a quantity of CP, especially of multiple different children, you’ll at least be brought in for questioning if not arrested/tried/convicted, whether the images were sexualized or not.
> nor would it ever end up in a database
That’s a bold blanket statement coming from someone who correctly argued that NCMEC’s database has issues (as in I know your previous claim is true because I’ve seen false positives for completely innocent images, both legally and morally). That said, with the amount of photos accidentally shared online (or hacked), to say that GP’s scenario can not ever end up in a database seems a bit off the mark. It’s very unlikely as sibling commenter said, but still possible.
That's why I said it's not inherently illegal. Of course, if you have a folder called "porn" that is full of naked children it modifies the context and therefore the classification. But, if it's in a folder called "Beach Holiday 2019", it's not illegal nor really morally a problem. I'm dramatically over-simplifying of course. "It depends" all the way down.
>That’s a bold blanket statement
You're right, I shouldn't have been so broad. It's possible but unlikely, especially if it's not shared on social media.
It reinforces my original point however, because I can easily see a case where there's a totally voluntary nudist family who posts to social media getting caught up in a damaging investigation because of this. If their pictures end up in the possession of unsavory people and gets lumped into NCMEC's database then it's entirely possible they get flagged dozens or hundreds of times and get referred to police. Edge case, but a family is still destroyed over it. Some wrongfully accused people have their names tarnished permanently.
This kind of policy will lead to innocent people getting dragged through the mud. For that reason alone, this is a bad idea.
With all due respect, please please stop making broad blanket statements like this. I'm far from a LEO/lawyer, yet I can think of at least a dozen ways a folder named that could be illegal and/or immoral.
> This kind of policy will lead to innocent people getting dragged through the mud. For that reason alone, this is a bad idea.
Completely agree.
It's very concerning if the fuzzy hash is too fuzzy, but I'm curious to know just how fuzzy it is.
So, it's theoretical, it's a different algorithm, and it's a case where someone is specifically trying to find collisions via machine learning. (Perhaps by "reversing" the hash back to something similar to the original content.)
The two above posters claim that they saw cases where there was a false positive match from the actual official CSAM hash algorithm on some benign files that happened to be on a hard drive; not something deliberately crafted to collide with any hashes.
I understand it seems like they don't want to give examples, perhaps due to professional or legal reasons, and I can respect that. But I also think that information is very important if they're trying to argue a side of the debate.
I gave that above in a sibling thread.
> I understand it seems like they don't want to give examples, perhaps due to professional or legal reasons, and I can respect that.
In my case, it’s been 7 years so I’m not confident enough of my memory to give a detail description of each false positive. All I can say is that the photos that were false positive that included people were either very obviously fully clothed and doing something normal, or the photo was of something completely innocuous all together (I seem to remember an example of the latter was the Windows XP green field stock desktop wallpaper, but I’m not positive on that).
Some of the false positives were of people, others weren’t. It’s not that the hashing function itself was problematic, but that the database of hashes had hashes which weren’t of CP content, as the chance of a collision was way lower than the false positive rate (my guess is it was “data entry” type mistakes by NCMEC, but I have no proof to back up that theory). I made it a point to never personally see any content which matched against NCMEC’s database until it was deemed “safe” as I didn’t want anything to do with it (both from a disgusted perspective and also from a legal risk perspective), but I had coworkers who had to investigate every match and I felt so bad for them.
In the case of PhotoDNA, the hash is conceptually similar to an MD5 or a SHA1 hash of the file. The difference between PhotoDNA and your normal hash functions is that it’s not an exact hash of the raw bytes, but rather more like the “visual representation” of the image. When we were doing the initial implementation / rollout (I think late 2013ish), I did a bunch of testing to see how much I could vary a test image and have the hash be the same as I was curious. Resizes or crops (unless drastic) would almost always come back within the fuzziness window we were using. Overlaying some text or a basic shape (like a frame) would also often match. I then used photoshop to tweak color/contrast/white balance/brightness/etc and that’s where it started getting hit or miss.
This has me pretty worried - once someone has been tarred with this particular brush, it sticks.
But image hashes are “perceptual” in the sense that the hash changes proportionally with the image. This is how reverse image searching works, and why it works so well.
It is claimed that the chance of being a false-positive for a positive match is one out of a trillion.
> Apple says this process is more privacy mindful than scanning files in the cloud as NeuralHash only searches for known and not new child abuse imagery. Apple said that there is a one in one trillion chance of a false positive.
https://techcrunch.com/2021/08/05/apple-icloud-photos-scanni...
I don't know what the cutoff is, but it doesn't sound like they believe that possession of a single photo in the database is inherently illegal. That doesn't mean this is overall a good idea. It simply weakens your specific argument about occasional false positives.
My point is that we have no way to verify the database wouldn't be abused or mistaken and a lot of that rests on the fact that CSAM is not something people want to have to encounter, ever.
It's kind of like the PCAOB... private 501.3(c) with congressional oversight and funding.
I think the strategy is that the organization is able to do more for helping children internationally if they're not seen as part of the Justice department and the executive, which after the debacle with CBP and "kids in cages", was probably the right call.
Anyone should be able to scan their own library against the database for false positives. “But predators could do this too and then delete anything that matches!” some might say, but in a society founded on the presumption of innocence, that risk is a conscious trade-off we make.
The law requires companies like Google and Apple to report when they find CSAM and afiact they would generate hashes and add to this database if new material is found.
I don't know if there is any oversight in this. It's all done behind closed doors so you just have to trust that the people creating the hashes aren't doing anything nefarious or mistaken and that's a separate point apart from what others have said on here that you should be able to trust your devices you own to not be informants against you.
I have a much simpler rule: Your device should never willingly* betray you.
*With a warrant, police can attempt to plant a bug, but your device should not help them do so.
The vast majority of iPhone users won't consider it a betrayal that they can't send images of child abuse, any more than they consider it a betrayal that it doesn't come jailbroken.
The victims of child abuse depicted in these images may well have considered it a betrayal by Apple that they allowed their privacy to be so flagrantly violated on their devices up until now.
Probably neither would child abusers, since as soon as they send an image of child abuse, they're much more likely to be caught than if it had stayed on their phone.
> a betrayal by Apple that they allowed their privacy to be so flagrantly violated on their devices up until now.
"Their" devices? Once Apple sells an iPhone, it no longer belongs to Apple. Taking "betrayal" to mean "didn't plant backdoors on other people's computers to catch your abusers" is stretching that word far beyond reason.
The rule is that your (note the emphasis) device won't ever willingly betray you. There's nothing here that implicates the majority in any way. Simply, your own device should never work against you.
This actually sounds like a great rule to prevent this kind of authoritarian scope creep.
- Yea yeah
- Great. Give me access to every part of your life so i know you're not a terrorist.
It's a pandoras box. You wouldn't allow the regular search of your home in real life.
That said, law enforcement has a nasty habit of having a rather "binary" worldview. People are either cops, or uncaught criminals. ..and they wonder why they have so much trouble making non-cop friends (DISCLAIMER: I know a number of cops).
With that worldview, it can be quite easy to "blur the line" between child sex traffickers, and parking ticket violators. I remember reading a The Register article, about how anti-terrorism statutes are being abused by local town councils to do things like find zoning violations (for example, pools with no CO).
Misapplied laws can be much worse than letting some criminals go. This could easily become a nightmare, if we cede too much to AI.
And that isn't even talking about totalitarian regimes, run by people of the same ilk as child sex traffickers (only wearing Gucci, and living in palaces).
”Any proposal must be viewed as follows. Do not pay overly much attention to the benefits that might be delivered were the law in question to be properly enforced, rather one needs to consider the harm done by the improper enforcement of this particular piece of legislation, whatever it might be.”
-Lyndon B. Johnson
I'm not. I am very unambiguously against this and I think if word gets out Apple could have a real problem.
I would like to think I am against child porn as any well-adjusted adult. That does not mean I wish for all my files to be scanned without my consent or even knowledge for compliance, for submission to who knows where matching to who knows what reporting to, well, who knows.
That's crossing a line. You are now reading my private files, interpreting them, and doing something based on that interpretation. That is surveillance.
I am very not OK with this.
"Think of the children" is how they force changes that would otherwise be unconscionable.
They've done it with encryption and anonymity for years. Now they're doing it with the hardware in your pocket.
We would all be better off without these things happening, and anyone would want less of it to happen.
Since they are only searching for _known_ abusive content, by definition they can only detect data that has been shared, which I think is the important point here.
So... all your hashes will be uploaded to the cloud? How do you prevent them from scanning other stuff (memes, leaked documents, trump-fights-cnn-gif,... to profile the users)?
Or will a huge hash database of child porn hashes be downloaded to the phone?
Honestly, i think it's one more abuse of terrorism/child porn to take away privacy of people, and mark all oposing the law as terrorists/pedos.
...also, as in the thread from the original url, making false positives and spreading them around (think 4chan mass e-mailing stuff) might cause a lot of problems too.
That isn't how I interpret "client-side".
The privacy implications are far more subtle.
It always starts with child porn, and in a few years the offline Notes app will be phoning home if you write speech criticising the government in China.
This technology inevitably leads to the sueveillance, suppression and murder of activists and journalists. It always starts with protecting the kids or terrorism.
Perceptual hashes like what Apple is using are already used in WeChat to detect memes that critique the CCP.
What happens on local end user devices must be off limits. It is unacceptable that Apple is actively implementing machine learning systems that surveil and snitch on local content.
I wouldn't hold your breath waiting for those regulations to become law in, say, Chine or Turkey or Saudi Arabia. I'd bet even Israel won't pass them, surely NSO have enough political lobbying swing (and probably also suitable blackmail material on sitting politicians).
Think of GDPR, which is essentially each member of the EU saying in unison, "your product/service must comply with these data protection laws, or you can't legally do business with any of our citizens".
Come to think of it, I wonder if this Apple thing would even fly under GDPR?
Possibly? I’m not a lawyer, but if this is about compliance with a legal obligation, and they’re under that category of pressure? I think GDPR would allow that?
Certainly seems more likely allowed than the stuff Facebook complained Apple was preventing them from doing.
I wholeheartedly disagree. A world wide goverment would be catastrophic for whistleblowing. Atleast as a whistleblower you can live somewhat safely in a country opposing your own. With a one world government you would have nowhere to run.
And I wouldn't expect them to protect citizen's interests any better than current governments do. Contrary, I think this lack of balance in the world would embolden them further.
The OP still addresses the inaccurate statement (presented in the form of a question for plausible deniability).
A totalitarian autocracy like China does not need this technology to search for wrongspeech, sadly. You are of course aware that all Chinese iCloud users get their data stored in a special set of datacenters that Apple actually doesn't control.
https://en.wikipedia.org/wiki/Birthday_attack
http://www.wolframalpha.com/input/?i=2%5E64%20%2F%20%288e9%2...
It's a perceptual hash.
State-of-the-art for this field is deep learning, and a /huge/ problem with the DL approach is that you can generate adversarial examples. So for example, a picture of a teacup that is identified by /most/ networks as a dog. It's particularly damning, because it seems like you don't have to do this for particular deep networks, they get tricked the same way, so to speak.
Seriously, folks, we shouldn't celebrate Apple's death grip over their platform. It's dangerous for all of us. The more of you that use it, the more it creates a sort of "anti-herd immunity" towards totalitarian control.
Apple talks "privacy", but jfc they're nothing of the sort. Apple gives zero shits about your privacy. They're staking more ground against Facebook and Google, trying to take their beachheads. You're just a pawn in the game for long term control.
Apple cares just as much for your privacy as they do your "freedom" to run your own (un-taxed) software or repair your devices (for cheaper).
And after Tim Cook is replaced with a new regime, you'll be powerless to stop the further erosion of your liberties. It'll be too late.
Stop. Using. Apple.
But is there a realistically better alternative? Pinephone with a personally audited Linux distro? A jailbroken Android device with a non-stock firmware that you built yourself? A homebuilt RaspberryPi based device? A paper notepad and a film camera and an out of print street map?
My hope is that those of you that share my viewpoint will call your legislators and demand regulations or a break up. There are forces of good within the DOJ that are putting together an antitrust case against Apple, and the more of us that lend our voices, the louder and more compelling the argument.
The DOJ is really the last lever we have, and that's pretty good measure for the power Apple wields.
no idea what the situation is now, but i wouldn't consider a phone that doesn't get those patches (project mainline) on time to be a serious option.
Gnu/linux phones have nonexistent security, other than being niche (so security by obscurity at most). And also, they are not yet usable as a daily driver for me personally, at least.
https://puri.sm/posts/defending-against-spyware-like-pegasus...
https://source.puri.sm/Librem5/community-wiki/-/wikis/Freque...
Whether or not I am allowed to check that my entrance has no locks whatsoever doesn’t make it harder to open it. And the reverse, even if I don’t know the details of the lock in my door, it will not let others pass through.
You absolutely can not make that assertion without being able to verify the lock.
For a fact we know that ios has strong sandboxing, secure bootchain and apps are revokably verified.
Not sure about that. No source code. Also, Pegasus.
Can you expand on this point a little bit?
Basically Micay is a legitimate security researcher who created the project and it was later hijacked by the company funding some of it. That company since then try to badmouth Micay at any place they find and is doing shady things on top of the still open source code base. Micay was so professional to destroy the verification key at the time of the forking.
Even if you don't personally audit it, you still benefit from other people doing it. Especially if the software is reproducible (and many packages are).
Realistically now we have Tizen and Jolla OS, which had backings from Samsung but nobody gave two damn about it.
I bet even if any of these vanilla mobile OS gets big enough they’ll get bought by the 3 giants and suffocated to death just like how Microsoft sniped Nokia.
Were there any recent examples where they failed in those?
They'll happily do evil shit, and execute it poorly. Samsung are _way_ more likely to leak the unnecessarily and possibly illegally collected personal data they hoover up than Google are.
... without any technical guarantee or auditability that any of the hashes they're alerting on are actually of child porn.
How much would you bet against law enforcement to abuse their ability to use this, and add hashes to find out who's got anti government memes or police committing murder images on their phones?
And that's just in "there land of the free", how much worst will the abuse of this be in countries who, say, bonesaw journalists to pieces while they are alive?
This will end badly for humanity.
Now the first few images might be the model completely clothed and not even be porn, maybe there's a picture of her lounging around a pool, then another picture of the pool itself. Still its part of a set of pictures that is known porn.
Heck most porn starts off with actors being clothed (so I hear lol).
So I want to ask what happens if you have a photo that is falsely identified as one in question and then an automated mechanism flags you and reports you to the FBI without you even knowing. Can they access your phone at that point to investigate? Would they come to your office and ask about it? Would that be enough evidence to request a wiretap or warrant? Would they alert your neighbors? How do you clear your name after that happens?
edits: yes, the hash database is downloaded to the phone and matches are checked on your phone.
Another point is that these photos used to generate the fingerprints are really legal black holes that the public is not allowed to inspect I assume. No one wants to be involved in looking at them, no one wants to be known as someone who looks at them. It could even be legally dangerous requesting to find out what has been put into the image database I assume.
That seems like a realistic attack. Since the hash list is public (has to be for client side scanning), you could likely set your computer to grind out a matching image hash but of some meme which you then distribute.
There have been a number of cases where people have found ways to trick CV programs in to seeing something that no human would ever see. If you were sufficiently malicious I imagine it would be possible to do with this system as well.
A malware will definitely be created, almost immediately, that will download files that are intentionally made to match CP - either for the purposes of extortion or just watching the world burn.
I'm usually sticking my neck out in defence of more government access to private media than most on HN because of the need to stop CP, but this plan is so naive, and so incredibly irresponsible, that I can't see how anyone with any idea of how easy it would be to manipulate would ever stand behind it.
Even if you do a content aware hash where you break the file into chunks and hash each chunk, you still wouldn’t be able to magically recreate the hash of a CP file without also producing part of the CP.
It's the weights from the middle of a neural network that they're calling a "hash" because it encodes and generates an image it has classified as bad. Experts have trouble rationalizing about what weights mean in a neural network. This is going to end badly.
If this was a hash then it would be as the parent describes, this is at best a very fuzzy match on an image to take into account blurring/flipping/colour shifting.
It's vastly more likely that innocent people will be implicated for fuzzy matches on innocuous photos of their own children in shorts/swimming clothes than it is to catch abusers.
The other thing is, when you have nothing to hide you won't take efforts to hide it - meaning you'll upload all of your (completely normal) photos to iCloud without thinking about it again.
The monsters making these images know what they're doing is wrong, so they'll likely take efforts to scramble or further encrypt the data before uploading.
tldr; it's far likelier that this dragnet will only even apply to innocent people, than it is to catch predators.
All this said, I'm still in support of Apple taking steps in this direction, but it needs far more protections put in place to prevent false positives than this solution allows. A single false accusation by this system, even if retracted later and rectified, would destroy an entire family's lives (and could well cause suicides).
Look what happened in the Post Office case in the UK as an example of how these things can go wrong - scores of people went to prison for years for crimes they didn't commit because of a simple software bug.
The ones that make national news from big busts do, because the ones that don't get caught much sooner and only make local news, because Google and other parties are have automatic CSAM identification online already (server side, not client side, AFAIK), and are sending hits to Homeland Security.
No, it'll be done on-device.
> How do you prevent them from scanning other stuff (memes, leaked documents, trump-fights-cnn-gif
Nothing. Given that it's only done on their closed-source messaging platform though, nothing is preventing them from reading your messages already.
But yes, it could potentially be used to detect images that the current political party doesn't like.
Because you know in some countries there are materials that local government find more offensive than mere child abuse. And once surveillance tech is deployed it's certainly gonna be used to oppress people.
And you can be sure that there's no way for the PRC, that already runs its own iCloud, to use this. America's favorite company wouldn't allow that.
In Saudi, Bahrain, and Iran there is no minimum age of consent – just a requirement for marriage. In Yemen, the age of consent for women is 9 (but they must be married first). In Macau, East Timor, and UAE, it's 14. [1]
I would allege that in all of those states they would probably find the perceptual hash of government criticism far more important to include on the "evil material" database than anything else.
I'm cynical enough to wonder whether this isn't their actual commercial reason for developing this, with CSAM being a PR fig leaf. Apple is substantially more dependent on China than its major competitors.
First it's iPhone photos, then it's all iCloud files, that spills into Macs using iCloud, then it's client side reporting of local Mac files, and somewhere along all other Apple hardware I've filled my home with have received equivalent updates and are phoning home to verify that I don't have files or whatever data they can see or hear that some unknown authority has decided should be reported.
What is the utopian perspective of this which counterbalances the risks for this to be a path worth taking?
Apple takes care of everything for you, and they have your best interests at heart. You will be safe, secure, private and seamlessly integrated with your beautiful devices, so you can more efficiently consume.
What's not to like about a world where child crime, terrorism, abuse, radical/harmful content and misinformation can be spotted in inception and at the source and effectively quarantined?
In 2021 and 2020 we saw people being arrested for planning/promoting anti lockdown protests. Not for actually participating but for simply posting about it. The scope of what "harmful content" is is infinite. You might agree that police do need to take action against these people but surely you can see how the scope creeped from literal terrorists and pedophiles to edgy facebook mums and how that could move even further to simple criticisms of the government or religion.
It's difficult to say how we draw the line to make sure horrible crimes go punished while still protecting reasonable privacy and freedom. I'm guessing apples justification here is that they are not sending your photos to police but simply checking them against known bad hashes and if you are not a pedophile, there will be no matches and none of your data will have been exposed.
In Germany police requested contact tracing lists from restaurants in investigations.
And the underlying desire for having this information will no doubt prolong the Corona restrictions longer than necessary, which is certainly not in the interest of German citizens.
Oh dear, one thing we could definitely state that German government is absolutely could be trusted to do the right thing
Australia, WA, border pass data: https://www.abc.net.au/news/2021-06-17/g2g-app-data-accessed...
Australia, VIC, check-in data: https://www.theage.com.au/politics/victoria/police-sought-ac...
Australia, QLD, check-in data: https://www.abc.net.au/news/2021-06-29/queensland-coronaviru...
[1]: https://www.golem.de/news/hamburg-polizei-nutzt-corona-konta...
Also the partly state sponsored luca app (check in in locations, festivals, restaurants, concerts) that is privately developed (and riddled with security holes) is already in discussion to use the data on the people to better target them for concert tickets and the like [2].
[2]: https://www.ccc.de/de/updates/2021/luca-app-ccc-fordert-bund...
So we see this data is already in abuse by the state and also by state sponsored private entities.
I believe, that this data, once collected, will only be (ab)used further in the future. In my experience it will be as with all data caches - somebody wants to create additional value from it.
https://fortune.com/2021/02/01/singapore-covid-data-tracetog...
https://www.straitstimes.com/singapore/politics/police-can-a...
https://www.straitstimes.com/singapore/proposed-restrictions...
https://www.straitstimes.com/singapore/politics/bill-limitin...
(Note that one mostly-united political party controls 89.2% of Singapore's legislature seats, and can pass any laws or amend its constitution to their liking.)
Please stop creating accounts to break HN's rules with.
Basically victims of rape don't want imagery of their rape freely distributed as pornography. They consider that a violation of their rights.
It's interesting how many users in this thread are instinctively siding with the offenders in this, and not the victims. Presumably because they made it through their own childhoods without having imagery of their own abuse shared online.
That is infantile. Painting people advocating privacy as siding with offenders is highly insulting.
If the median shelf-life of abuse evidence is shortened, in that the item in question can no longer be forwarded/viewed/stored/..., what does that imply in a world where the demand remains relatively stable?
I despise the abusers for what they do, and the ecosystem they enable. But I also remember first having this argument more than ten years ago. If you, as a member of law enforcement or a child wellbeing charity, only flag the awful content but do not do anything else about it, you are - in my mind - guilty of criminal neglect. The ability to add an entry to a database is nothing more than going, "at least nobody else will see that in the future". That does NOTHING to prevent the creation of more such material, and thus implicitly endorses the ongoing abuse and crimes against children.
Every one of these images and videos is a piece of evidence. Of a horrifying crime committed against a child or children.
There's a small number of victims, a small number of offenders (but much more than "a handful"), and hundreds of millions of other users. This change is in the direct interest of victims, direct opposition to offenders.
Most normal people probably support the measures in solidarity with group 1, HN generally doesn't.
It's a restriction on their liberty and privacy that they willingly support because of the overall positive effects.
Anyway I'll duck out of this now the driveby downvotes annoy me.
But yes, there's a lot of drive-by engagement in this thread, thank you for at least engaging with it directly.
There are kidnapped children being locked inside homes. If you don't open your doors and accept weekly full home inspections, I think it's safe to say you support offenders and hate victims if you oppose this. I mean, we're all against people kidnapping and abusing children.
There's a small number of victims, a small number of offenders (but much more than "a handful"), and hundreds of millions of other home owners. This change is in the direct interest of victims, direct opposition to offenders.
I strongly believe that nobody wants to further victimize people by publicly showing images of their abuse.
And I believe very strongly that putting hundreds of millions of people under blanket general suspicion is a dangerous first step.
Imagine if every bank had to search all documents in safe deposit boxes to see if people had committed tax evasion (or stored other illegal things like blood diamonds obtained with child labor). That would be an equivalent in the physical world.
Now add to this, as discussed elsewhere here, that the database in question contains not only BIlder of victims, but also perfectly legal images. This can lead to people "winning" a house search because they have perfectly legal data stored in their cloud.
Furthermore, this means that a single country's understanding of the law is applied to a global user community. From a purely legal point of view, this is an interesting problem.
And yes: I would like to see effective measures to make the dissemination of such material more difficult. At the same time, however, I see it as difficult to use a tool for this purpose that is not subject to any control by the rule of law and cannot be checked if the worst comes to the worst.
A great deal of skepticism is being given to the NCMEC database in these comments, which I'm surprised by as from what information I have I think this is being exaggerated. At the same time we have no idea whether Apple would even be using that database or another one that they may have created themselves.
Thi sis transmission of funds and there are laws regulating the monitoring of those.
I used bank vaults were you put things into the vaults without the bank often times knowing what is in there. If they knew, they would need to report to authorities.
So Apple doing this scan would be the bank opening all vaults, scanning the contents and reporting things to the IRS (I think this is the tax thing in the US if I am not mistaken - in Germany it would be the Finanzamt).
It's only boring until we get another Hitler or equivalent.
You will make Apple tons of money.
What would be the result of 'curl'ing back a few random hashes as positives from the database? Do I expect to be handcuffed and searched until it's sorted out? What if my app decides to do this to users? A malicious CSRF request even?
(I'll wager) The majority of these calls will be false positives. Now a load of resources get deployed to keep an eye on the device's owner, wasting staff time and compute, wasting (tax funded) government budget that could have gone towards proper investigation.
Another thought - notice that they say "if too many appear". This may mean that the hashes don't store many bits of information (and would not be reversible) and that false positives are likely - ie, one image is not enough to decide you have a bad actor - you need more.
But at Apple's scale, statistically, some law-abiding users would likely get snagged with totally innocent images.
Just a bad idea all around.
Creating collisions could be though, eg. brute forcing a normal picture by modifying random pixels by a bit into matching an illegal content’s hash is a possibility.
1) You willingly delegated the decision of what code is allowed to run on your devices to the manufacturer (2009). Smart voices warned you of today's present even then.
2) You willingly got yourself irrevocably vendor-locked by participating in their closed social networks, so that it's almost impossible to leave (2006).
3) You willingly switched over essentially all human communication to said social networks, despite the obvious warning signs. (2006-2021)
4) Finally you showed no resistance to these private companies when they started deciding what content should be allowed or banned, even when it got purely political (2020).
Now they're getting more brazen. And why shouldn't they? You'll obey.
(2) and (3) are bad, but a tangential bad to this: it’s no good having an untainted chat layer if it’s running on an imperfect — anywhere from hostile to merely lowest-bidder solution — OS. (And most of the problems we find in software have been closer to the later than the former).
(4) for all their problems, the American ones held off doing that until there was an attempted coup, having previously resisted blocking Trump despite him repeatedly and demonstrably violating their terms.
It’s not clear that governments would give the open social networks an easier ride either. It could be argued that distributed FOSS developers are easier to pressurise into adding back doors, unless we officially make EFF our HR/Legal department.
The other problem is workers have a right to be paid. The alternatives are FOSS and/or distributed social media. Who in good conscience would ask a tech worker to give away their labour for free, in the name of everyone else’s freedom?
In a world of $4k rent, who amongst us will do UX, frontend, backend, DevOps, UO, and Security for 7 billion people, for anything but the top market rate?
The real alternative is to attack the actual problem: state overreach. Don’t attack people for using SnapChat — get them to upend the government’s subservience to intrusive law enforcement.
How could this be argued?
All millions of them at the same time?
You'd only need a few important ones, and all you'd have to do is compromise them in one way or another. This can be done via coercion, via money, or by physically or virtually breaking into their system(s).
For example, if money can be an incentive, you can stimulate a FOSS dev to add a NOBUS vulnerability in code. Also, since all the code is public, organizations like NSA can do in-house fuzzing, keeping the findings to themselves.
Independent audits should help against backdoors. Again, the FLOSS nature of software and huge number of developers are essential here.
Here's the hope: the tech workers doing it for 'free' because they're scratching their own itch. So it would not be an act of onerous charity. The techies make some free open source decentralised clone of Reddit, say, then some folks among knitting communities, origami enthusiasts, parents groups, etc. copy it for free and pay to run it on their own hardware.
imho, we have everything in the foss world working tightly except great UX/UI. in my experience in the open source world – which is not insignificant – great UX is the only thing stopping us from a paradigm shift to actual tech liberation.
even outside of corporate funded work/commits, we see an astounding number of people donating incredible amounts of their time towards great quality code. but we still thoroughly lack great UX/UI.
i’m not talking about “good”, we have some projects with “good” UX, but very very few with great.
there are many reasons and I’d be happy to share what some of them are, but in my mind great UX is unquestionably one of two primary things holding us back from actual truly viable software liberation.
Truth is even if our tools evolve, our chains evolve faster.
https://link.springer.com/chapter/10.1057/9780230298125_11
Signals intelligence intercept and analysis centres have been called Black Chambers for a long time, including the first such group in the US, predecessor to the NSA:
Now any person who owns an Android is a potential predator. Also, if you are trying to jailbreak your iPhone, you are a potential predator.
Of course, they won't be shown photos of victims of brutal dictatorships like Russia, Belarus, China, etc. They love it that phone manufacturer keeps their phone free of malware.
They love it that phone manufacturer tells their phone free of malware.
Fixed that for you. See: Pegasus.
Congratz, investigations got a bit harder, but now all people have to life with a tool that will be used against them when needed. No sane person can believe that this isn't used for other "crimes" (how ever those are defined) tomorrow.
The RSA-155 Challenge (512 bits) was not beaten until 1999.
What would concern me is if we see a big revenue stream from their software. Then i'd question them not wanting Linux on their machines.
But imo you already gave them what they want when you buy an M1. I don't see a reason why they care beyond that.
In fact, I predict that Apple will at one point start selling software as a subscription, like SaaS. Other OSes don't fit in that model.
The day apple wants it's OS anywhere it can get it, because it brings in money independent of just it's hardware is the day i actually believe Apple cares about the software revenue.
Right now Apple's software is hardware locked. Google by comparison cares much more about the software side than Apple, imo.
That’s not a difficult prediction. https://en.wikipedia.org/wiki/Apple_Arcade:
“Apple Arcade is a video game subscription service offered by Apple Inc. It is available through a dedicated tab of the App Store on devices running iOS 13, tvOS 13, iPadOS 13, and macOS Catalina or later. The service launched on September 19, 2019 after being announced in March 2019”
They also have “hardware as a service”. https://www.apple.com/shop/iphone/iphone-upgrade-program:
“The easiest way to upgrade to the latest iPhone. Get a new iPhone every year AppleCare+ coverage included Works with your carrier Starting from $35.33/month”
[1] https://www.applemust.com/apples-50b-services-target-just-is...
- (Potentially) programmable top notch security chip with no overhead encryption
- Fanless (Air), cool running, fast processor with very low power consumption
- All metal body
- Top notch HiDPI screen with color accuracy.
- Top notch sensors
- Excellent, illuminated keyboard
- Big trackpad with pressure sensitivity and taptic engine
- Excellent battery life
- Excellent battery endurance
- Very high sound quality, good speakers, good mics.
- High quality webcam
- Light for its class
- WiFi with plenty of antennas, MIMO support and all modern standards, incl. forward facing ones.
- The ability to use nix-darwin for configuration management. It may be incomplete, but there’s no Windows equivalent.
- Higher single thread performance than any other device.
For example, take the monitor - there are monitors with higher refresh rates or more resolution (Retina does 'only' 6K). Also, Mac OS colour processing has no good equivalent elsewhere, so you won't see as much benefit from Apple's monitor without Mac OS.
On the software side, Apple has the unique position to force every developer who cares about their user base to rebuild their stuff for a completely different CPU architecture.
M1-based products are anything but overpriced. The Air’s price is comparable to modern flagship smartphones.
At this point in history, you can't avoid Apple's ecosystem if you want to use their ARM-based laptops/desktops unless it's a hobby of yours. And it's sad, because this is the best hardware I've ever owned.
They're not EliteBook/Thinkpad tough, but they're more than enough unless you're going to handle it rough.
I can calibrate a screen regardless of the OS I use with the help of a good calibration device, and having a good panel is a good start for that. On desktop, there are possibly better panels for pure color accuracy, I won't argue that.
OTOH, on the portable space, their hardware formula is pretty robust.
Worse, it would take some time for me to even trust Linux on M1 laptops to not fail exactly when I need it - that's the nature of reverse engineering. Since they don't have the specs, the only real test is lots of people running it for some time and reliability is more important to me than specs.
As for monitors, Apple's colour advantage is more than good calibration - the entire ecosystem can handle 10bit HDR. That's something AFAIK you won't get elsewhere.
Retina Display when it first arrived was literally THE reason I started switching to Apple ecosystem as it was exponentially better to look at that screen, and I'd even pay more for a 4K display on a MacBook if they brought it.
Everyone betting on the same horse is usually a bad strategy.
By the way, here's what Linus Torvalds (the creator of Linux) thinks about using the M1:
https://www.zdnet.com/article/linus-torvalds-would-like-to-u...
Yes, the desktop has higher throughput. Of course it does. But that doesn't mean I don't feel a fraction of a second's lag whenever I do basically anything, and on the M1 that just... doesn't exist.
So it's not just a matter of feel.
The problem is, you pay the same price for a Thinkpad/Elitebook and you get ridiculous gotchas like missing wireless antennas (at most you can find 2x2, if you're lucky). Or the configuration you like doesn't come to your country, or the vendor doesn't allow custom configuration for your country unless you buy 10+, etc.
OTOH, I pay the premium, I get what I configured, with top notch small specs (antennas, wireless chips, etc.). You can't find a spare 40mm SSD for your Elitebook after three years, but Apple will service your device happily.
If you get the said SSD from the vendor, its quadruple the price, so it's Apple's price territory again.
At that price range, there's no advantage in hardware prices between Apple, HP, Lenovo and Dell. They're equally cheap/expensive. So, IMHO, you pay less on the long run for an Apple laptop, which you can use for 7-8 years without problems.
Yes. It's slightly deeper, much more resistant to dust and one can write with very low effort, but with crisp feedback. It allows me to write at least 10% faster with no effort. I similarly use an old Microsoft Sculpt Comfort at office and Logitech G710+ at home.
Good keyboard is something hard to find.
Is it worth it, though? Is being "better for mobile computing purposes" worth all of this, and whatever else will come next - which we know is just a matter of time, because no one cares enough to stop buying from them?
The only difference in this is who you trust. Be it Apple, the community or someone you pay, you're still trusting that someone else's interests align with yours and they did things correctly.
In other words, this is not a technical problem. It's a problem that needs to be solved through regulation, because 99% of the people can't verify by themselves that their devices are actually private and secure.
Not really. There is a huge difference between trusting a single for-profit entity (who provides backdoor to iCloud in China) or huge number of independent people (each would like to get famous/rich for finding bugs).
But the kernel is only a tiny fraction of the system. There simply is no Linux system that even attempts to solve the problems Apple solves. There could be, but there isn’t - this is what we mean by the term ‘wishful thinking’.
You are speaking as if you know the opposite. Any links?
Plus the hardware is nice and actually works.
I agree with what you say in principle but here I am using an iPhone to type this while it’s been nearly 2 years since I ordered my Librem 5.
Making decent mobile devices that are more secure than an iPhone is not an easy thing.
Secure against entities they don't like. But intentionally insecure against entities they do like.
Are you sure intel, AMD, Arm or windows TPM aren’t snitching on you? Do we need to make our own silicon from ingot?
There’s no technological solution to this problem, only social and legislative.
>The system that scans cloud drives for illegal images was created by Microsoft and Dartmouth College and donated to NCMEC. The organization creates signatures of the worst known images of child pornography, approximately 16,000 files at present. These file signatures are given to service providers who then try to match them to user files in order to prevent further distribution of the images themselves, a Microsoft spokesperson told NBC News. (Microsoft implemented image-matching technology in its own services, such as Bing and SkyDrive.)
https://www.nbcnews.com/technolog/your-cloud-drive-really-pr...
Didn't the Windows 10 TOS extend that scanning to local storage as well?
I don't step in the lowest parts of the internet hell, but I am also not very picky about it. I have never encountered child pornography in 10 years of almost pathological internet usage.
Additionally, by calculating hashes of media on peoples devices, you can quickly determine networks. A private image you shared with your friends? Unique hash and everyone that has it is probably in your network. That is aside from the issue that they would also just read your contacts.
The TPM FUD has really gone out of hand.
1. there's no such thing as "windows TPMs", whatever that means.
2. TPMs basically has zero access to the rest of the system. It's connected via a LPC bus, so there's no fancy DMA attacks to pull off. Over that bus the system firmware sends various hashes of the system state (eg. hash of your bootloader), but that's about it.
That's the specification. Have you actually monitored the bus using probes? Did you check that the TPM is only connected to LPC?
Isn't this the definition of FUD?
>Did you check that the TPM is only connected to LPC?
Dunno, you tell me. https://en.wikipedia.org/wiki/File:TPM_Asus.jpg
The only thing that's changed here is that there is more encryption around, and so legal guidelines are being written to facilitate this, which has been happening for a long, long time.
(I don't disagree with your overall point, and child porn is definitely the thin edge of the wedge, but this isn't new and presumably shouldn't be too surprising for any current/former megacorp as they all have systems like this).
"It's been happening for a long time already, the only difference now is a 0.1 degree increase", says the frog while being boiled alive.
When I read that this shouldn't be surprising, it has an aftertaste of "Dropbox is not interesting/surprising because ftpfs+CVS have existed for well over a decade"
This has been standard practice for well over a decade amongst all big internet platforms.
Like, one can argue that regardless, people's messages should not be readable for any reason, but that's gonna be a tough one to get through a court of law.
The obvious difference here is that backdooring encryption is an all or nothing affair, which may require new thinking (it definitely does).
But the ship around this particular form of backdooring has most definitely sailed.
Like, the only reason Apple is new to this game is because they haven't been in the storage/media sharing business for as long as their competitors.
> 1) You willingly delegated the decision of what code is allowed to run on your devices to the manufacturer (2009). Smart voices warned you of today's present even then.
99% of the population will delegate the decision of what code is allowed to run to someone, be it the manufacturer, the government, some guy on the Internet or whatever. For that 99% of the population, by the way, it's actually more beneficial to have restrictions on what software can be installed to avoid malware.
> 2) You willingly got yourself irrevocably vendor-locked by participating in their closed social networks, so that it's almost impossible to leave (2006).
"Impossible to leave" is not a matter of closed or open, but it's a matter of social networks in general. You could make Facebook free software and its problems wouldn't disappear.
Not to mention that, again, 99% of people will get vendor-locked because in the end nobody wants to run their own instance of a federated social network.
> You willingly switched over essentially all human communication to said social networks, despite the obvious warning signs. (2006-2021)
Yes, it's been years since I talked someone face to face or on the phone and I cannot send letters anymore.
> 4) Finally you showed no resistance to these private companies when they started deciding what content should be allowed or banned, even when it got purely political (2020).
No resistance? I mean, it's been quite a lot of discussion and pushback on social networks for their decisions on content. Things move slow, but "no resistance" is quite the understatement.
> Now they're getting more brazen. And why shouldn't they? You'll obey.
Is this Mr. Robot talking now?
But now more seriously, in December the European Electronic Communications Code comes into effect, and while it's true that there's a temporary derogation that allows these CSAM scanners, there's quite a big debate around it and things will change.
The main problem with privacy and computer control is a collective one that must be solved through laws. Thinking that individual action and free software will solve it is completely utopic. A majority of the people will delegate control over their computing devices to another entity because most people don't have both knowledge and time to do it, and that entity will always have the option to go rogue. And, unfortunately, regulation takes time.
Anyways, one should wonder why, after all these years of these kinds of smug messages, we're in this situation. Maybe the solutions and the way of communicating the problems is wrong, you know.
Not true. If you have interoperability between different networks, you can leave. This is how ActivityPub (e.g. Mastodon, PeerTube, PixelFed) works.
> Not to mention that, again, 99% of people will get vendor-locked because in the end nobody wants to run their own instance of a federated social network.
You just switch to any other instance, because Mastodon doesn't prevent you from doing that.
> The main problem with privacy and computer control is a collective one that must be solved through laws. Thinking that individual action and free software will solve it is completely utopic.
We need both. You cannot force Facebook to allow interoperability when there is no other social network.
If all your friends are in a Mastodon instance and you think that instance is scanning your messages, you'll find it hard to leave because leaving the instance for another that doesn't share messages with that one means stopping communication with your friends.
> You just switch to any other instance, because Mastodon doesn't prevent you from doing that.
Controlled by another third party. Not to mention that, with enough users, there will be feature divergence so "switching" won't be that easy.
Want a real world example? See email. Open protocol with multiple client-server implementations. However, most people use one of the major providers (Google, Microsoft...), there are incompatibilities between clients and even if you "can switch", it's not that easy nor gets done often. Yes, you can switch to ProtonMail or something more secure if you want, but that won't solve the problems of the 99% of people that will use general providers and won't even know they can't switch.
> We need both. You cannot force Facebook to allow interoperability when there is no other social network.
Right now you could force Facebook to be interoperable and be open source and still 99% of the people would be on the original Facebook instance. Again, it's not a technical issue.
Everything is controlled by a third party except self-hosting. Mastodon allows that too. Closed networks don't.
> Yes, you can switch to ProtonMail or something more secure if you want
So you answered your own question.
> but that won't solve the problems of the 99% of people that will use general providers and won't even know they can't switch.
My point is that they are able to switch due to the openness of the platform.
> Right now you could force Facebook to be interoperable and be open source and still 99% of the people would be on the original Facebook instance. Again, it's not a technical issue.
Yes. It's not just a technical problem. But there is a technical side in it. Millions will immediately switch given a possibility. What happens next, who knows.
And my point is that most won't, and the ones that do will still go to another platform that's controlled by another third party and they'll still need to rust that the platform is not doing things they don't like.
> Millions will immediately switch given a possibility.
Switch to where? To another company that could do weird things out of the eyes of the users? Do you think all of those millions are going to run their self-hosted Facebook?
My point is that privacy and security is not something that will be solved by federation or open source. For open source and federation to be useful in that regard, you need most people to actively research and check that the tools that they use are private and secure. If they don't, they're just trusting someone the same way they trust Facebook now. And most people (that includes most people here on HN) don't have both the time and knowledge to do those checks.
In other words, this is a collective issue. Trying to solve collective issues by individual choices is not the best path.
I disagree. Here's why:
> For open source and federation to be useful in that regard, you need most people to actively research and check that the tools that they use are private and secure.
This is the key point. You do not need most people. You need some people. And you can always find some people who verify everything and self-host for you. This is how Signal and Matrix appeared and became (relatively) famous.
And what happens when another app comes and says that "it's secure" and people start using it instead of Signal or Matrix? What happens if Signal starts requiring some payments (running servers is not free) and people move to other apps? Maybe those other apps are open source and federated, but the federation protocol is found later to have a backdoor, or some instances run data mining on the messages, or something like that. Who will be faster, the users flocking to those apps or the few number of verifiers getting to work and detecting those issues?
If you want most apps to be like Signal or Matrix, the solution is easy: push for legislation and certifications that ensure that, no matter the app, a certain level of security and privacy is enforced. It's not perfect, but it's far better than just relying on trusting that some people invest a lot of time on that research.
First, early adopters come and verify it. They bring their friends. If it's really secure and they find no serious bugs, more people join. Then, a bridge is created between the services.
> What happens if Signal starts requiring some payments (running servers is not free) and people move to other apps?
This is a problem with a non-federated protocol actively fighting against third-party apps and servers. It will definitely happen with Signal in this way, which is why I'm not using it and not recommending.
> Maybe those other apps are open source and federated, but the federation protocol is found later to have a backdoor, or some instances run data mining on the messages, or something like that.
Such backdoor will be quick and easy to fix, and to verify that it's fixed. Unlike with Apple's Pegasus. No system is ever 100% secure.
> Who will be faster, the users flocking to those apps or the few number of verifiers getting to work and detecting those issues?
Users are typically very slow to move. See Whatsapp & Facebook. But what's your point?
> If you want most apps to be like Signal or Matrix, the solution is easy: push for legislation and certifications that ensure that, no matter the app, a certain level of security and privacy is enforced.
This is definitely an important thing to do, but it's not enough. There is such legislation already in Europe: GDPR. Unfortunately it cannot dramatically change the industry quickly, because of the monopolies and network effects.
That's quite the optimistic path. What if the app starts being used by teenagers, for example? Or by people with less technical abilities?
> This is a problem with a non-federated protocol actively fighting against third-party apps and servers.
Federated services still need to pay for their servers.
> Such backdoor will be quick and easy to fix
Again, pretty optimistic on that.
> and to verify that it's fixed. Unlike with Apple's Pegasus. No system is ever 100% secure.
Pegasus was external malware. What makes you think a Pegasus for federated servers or open source phones can't exist?
> Users are typically very slow to move. See Whatsapp & Facebook. But what's your point?
Security research takes time, probably more time than users need to move from apps.
> There is such legislation already in Europe: GDPR.
And GDPR has accomplished way more in way less time than technical solutions. I wonder why.
> Unfortunately it cannot dramatically change the industry quickly, because of the monopolies and network effects.
Don't those monopolies and network effects affect the technical solutions you propose too?
My point is that of course you need good technical solutions, but just those by themselves are useless, because most people don't have the time and knowledge to reliably distinguish which ones are good and which ones are bad (and "good" and "bad" are relative too), and other differential features (price, capabilities, ease of use) that are easier to notice will weigh more on their decisions.
This is not a problem unique to tech and privacy. Food security, climate, building safety... almost everything you buy has had the similar issue of how to have "things done right" where deciding whether it's done right or not is hard for most people. Almost everything has been solved (or almost solved) with regulation, and just "better products" haven't been enough.
>99% of the population will delegate the decision of what code is allowed to run to someone, be it the manufacturer, the government, some guy on the Internet or whatever. For that 99% of the population, by the way, it's actually more beneficial to have restrictions on what software can be installed to avoid malware
I do not agree with this. You are saying people are too stupid to make decisions and that is amoral in my opinion.
>"Impossible to leave" is not a matter of closed or open, but it's a matter of social networks in general. You could make Facebook free software and its problems wouldn't disappear.
Data portability is a thing. This was the original problem with FB and thats how we got 'takeout'.
>Yes, it's been years since I talked someone face to face or on the phone and I cannot send letters anymore.
>Is this Mr. Robot talking now?
Using the extreme in arguments is dishonest. We are talking on HN where it is a selective group of like minded people(bubble). How does your delivery driver communicate with their social circles? Or anyone that services you? You will find different technical solutions are used as you move up and down the social hierarchy.
>The main problem with privacy and computer control is a collective one that must be solved through laws.
Technology moves faster than what any law maker can create. We do not need more laws as technology advances but rather an enforcement of personal rights and protections enabling users to be aware of what is happening. It appears you are stating "people aren't smart enough to control their devices" and "We need laws to govern people" vs my argument that "people should be given the freedom to chose" and "existing laws should be enforced and policy makers should protect citizens with informed consent".
I'm not saying that at all. I'm saying that it's impossible for all people to make informed decisions on all the issues that surround them, because of both knowledge and time. And it doesn't just happen with computer and privacy, see food, for example. Do you make all decisions about what's allowed or not in your food chain? It's impossible! Unless you dedicate quite a lot of time to it, you can't know if certain foods have certain ingredients, and whether those are harmful or not. That's why we have regulation on food. We trust that regulation because we need to do more things than just worrying constantly about our food.
In the same way, most people delegate control on what can run on their device because they don't have the time or knowledge to inspect constantly what is running on their devices.
> Data portability is a thing. This was the original problem with FB and thats how we got 'takeout'.
And did takeout solve any problems? No, because it's not a technical issue.
> Using the extreme in arguments is dishonest. We are talking on HN where it is a selective group of like minded people(bubble). How does your delivery driver communicate with their social circles? Or anyone that services you?
The GP used the extreme by saying that "essentially all human communication" has been moved to social networks.
But yes, we do agree that HN is not the real world. So I'd love to know what were the warning signs to people like a delivery driver, or basically anyone that wasn't active in computer circles. Not to mention that, before, social networks, most communication was done through channels controlled by third parties (phone, letters, television). From a non-technical standpoint, things didn't change that much.
> We do not need more laws as technology advances but rather an enforcement of personal rights and protections enabling users to be aware of what is happening.
"Enforcement" is done through laws and regulations.
> It appears you are stating "people aren't smart enough to control their devices" and "We need laws to govern people"
I'm not saying that at all. I'm saying that people shouldn't need to invest a significant amount of time constantly verifying that their devices and networks are doing what they say they are doing, and that laws and regulations should be applied to corporations instead so that people can reasonably trust that the ones offering those devices and networks are doing things somewhat correctly.
And again, this has been done already with quite a lot of things. There are regulations for cars, food, furniture, clothes... Not because people aren't smart to control what they use, but because it's impossible for any one person to have the time and knowledge to control everything that they use.
Imagine applying your argument when talking about, say, carcinogenic substances on food. You could argue that the best way to fight that is for people to grow their own food and check that their food doesn't contain those substances, or trust that some company that sells them food is doing it for them. Or, you could push for regulation and organisms that ensure that those substances don't make their way into the food chain.
Well, this is the same. Most people have other things to do instead of learning to ensure that their devices are secure and private and then checking that for everything they get their hands on. You need regulations so that there's a consensus on what can you expect, and then enforcement so that the products you get actually comply with those regulations.
The suggestion is not that people need to continually invest large amounts of time to manually verify their freedom is being upheld. It is that they should inform themselves once in order to understand the issue and be able to ensure their governments are not secretly becoming totalitarian states.
I'd bet that most people in HN live in democracies.
> Who will put pressure on governments to formulate just laws if people are so hopelessly misinformed and cannot possibly take the time to understand the issue of being parted with their freedom?
You don't need to know biochemistry and medicine to know that you don't want carcinogenic elements in your food, right? In the same sense, you don't need to know how to check that a channel is private and secure in order to push for regulations that make your communication channels secure.
>It is that they should inform themselves once in order to understand the issue and be able to ensure their governments are not secretly becoming totalitarian states.
Which is exactly the same thing I'm proposing: push for regulations that align with their interests.
Yes, but you need to be able to conceive what cancer is and (roughly) how and why it appears in order to understand and fight the concept of carcinogens. Most people today still haven't realized the cancer and the carcinogens in the story of privacy. This is despite the fact that this was widely known in a prior time (see Stasi).
> Which is exactly the same thing I'm proposing: push for regulations that align with their interests.
Yes, except it's not happening because people are complacent, which is why people are fuming when they see stuff like this here.
I pretty much agree with you. That said, the position that this particular issue is one in which regulations can save us is naive. The basic political reality is that the very people/organizations which are pushing for further encroachments on our rights and destruction of our privacy as people are the same people and organizations responsible for regulating the companies which offer us these services, software, and devices. When you live in a world with blatantly malicious state actors (the US Government) pressuring and demanding these encroachments as an end-run around existing regulations (the Fourth Amendment of the US Constitution), whom exactly is supposed to create and enforce these privacy regulations?
> I do not agree with this. You are saying people are too stupid to make decisions and that is amoral in my opinion.
No, it's just saying that most people have other priorities. If you want to make the world a better place, educate more people so that their priorities change towards caring more about the software that runs on their devices, instead of attacking people with weird non-sequiturs.
> I do not agree with this. You are saying people are too stupid to make decisions and that is amoral in my opinion.
How much of the code running on your data do you personally inspect? (Don’t forget device firmware) When your browser ships an update, do you reverse-engineer the binary? Do you review all of the open source code you use looking for back doors?
Would it be accurate to say that you don’t do that because you’re stupid? I don’t think that’s reasonable, any more than it would be to say you should carry around a test kit for any food you are planning to buy at the supermarket.
> Technology moves faster than what any law maker can create.
This is a common claim but it’s too simplistic. Laws do get passed relatively quickly when there’s a clear need - think about how things like section 230 arrived relatively soon after the rise of the web - but in most cases it’s more a clarification of existing laws. For example, cryptocurrency wasn’t mentioned in previous laws by name but the IRS had no trouble taxing it under existing laws.
Privacy shows why the “just let people choose” approach doesn’t work: you the individual have no negotiating clout with Facebook or Google, and there are many cases like revenge porn where the problem is only visible after the decision has been made.
Laws are how societies agree to function. If you don’t like the laws, you need to get involved because there simply isn’t a way to get good results by demanding that the system accommodate people who don’t show up.
Instead, we as a collective decided to offload these tasks to the government and make broad decisions through voting. This allows us to focus on other things (at work, with our actual job, at home, you can focus with what matters for you, whatever that might be).
For instance, I tried to avoid Facebook for a while and it was working well, I just missed few acquaintances but could keep in touch with the people who matter for me. Then suddenly they acquired Whatsapp. What am I to do? Ask my grandmother and everyone in between to switch to Telegram? Instead, I'm quite happy as a European about the GDPR and how the EU is regulating companies in these regards. It's definitely not yet there, but IMHO we are going in the right direction.
Making holier than thou comments about everyone else being sheep isn't helpful or thought provoking. Offer an alternative if it is a bad one (looking at you Mastodon). So here's mine: we need to change the power of digital advertising. Most of the most rent seeking companies generate revenue primarily selling ads to get more people to buy more crap. I want a VAT on all revenue passing through the digital advertising pipeline. My hope is that if these things are less profitable, it will reduce the over-sized impact these companies (social, infotainment [there is no news anymore], search, etc.) have on our economy and life. People are addicted to to fomo and outrage (faux?), I don't that that will ever change but we can try to make it less profitable.
Also, that spirit existed only in an adversary free environment. You may as well say the solution is for everyone to be nice to each other.
The solution is to build new technologies that are privacy preserving, transparent, don’t place trust in a central authority but are resistant to attack.
This is possible but nobody has built it yet.
Problem is you end up with a vastly inferior hardware device that costs nearly as much as an iPhone.
I’m still waiting for my Librem 5 preorder.
Would love to hear if anyone is actually using a Linux phone and enjoying it.
So far though, they do nothing to solve the problems we are talking about. The software is not anywhere near audited, and even if it were, you are still interacting with people and services who are using unaudited software.
Many of the projects leverage a well known OS as their base (e.g. pmOS uses Alpine Linux, Mobian uses Debian), and actively ensure that anything that can be upstreamed is upstreamed. So it's not like you're downloading some random ROM off of XDA.
It’s not even close.
The Pinephone is $150/$200.
> Would love to hear if anyone is actually using a Linux phone and enjoying it.
I have one, and I throughly enjoy it! One of the neatest things about it, and I still have to wrap my head around it, is that it can do anything you can do on a desktop. SSH? No problem? Dev environment? "apt install build-essentials". Want to install XFCE? Knock yourself out!
It is still missing MMS, which is why I don't use it daily, but that should be changing much sooner than later.
You can do all these things with a $10 raspberry pi-w. It’s not obvious why it helps with having security and privacy.
The person asked: "Would love to hear if anyone is actually using a Linux phone and enjoying it."
...so I responded about what I enjoy about using a "Linux Phone". A "$10 raspberry pi-w" is not a Linux Phone.
The question didn't address "security and privacy", so I didn't add it. Though, since you brought it up, I will answer how the Pinephone "helps with having security and privacy".
- I have hardware (cut physical power) switches to kill the Microphone, cameras (independent for front and back), and Wifi/Bluetooth.
- I am running a full GNU/Linux distro that I have a lot more confidence in their want to protect users security/privacy.
- I can run software of my choosing on it (the Pinephone actually defaults to booting on an SD card! so it encourages you to experiement with OSes).
- There is serious effort to run the mainline Linux kernel on it, so it will not be artifically obsolete in 3-5 years (or be sutck on running some ancient swiss cheese kernel in 5 years like my old Pixel/Nexus devices).
- I don't have to install/link to some opaque binaries to even boot the system.
I'm sure I can think of more, but that's all I have for now.
What I have encountered is the requirements for it being a daily driver are user specific. I have seen a fair amount of users say "I need $FOO app to work", where $FOO is some an app only developed for Android/iOS (a common one is a banking app, there is almost no way that company will support a Linux phone). So I guess I would ask what your requirements are before I can say if it is daily driver ready or not.
My threshold for it being a "daily driver" is if I can fully replace the "phone" features of my Android phone, which for me, is: Calling, SMS, MMS, Voicemails. MMS is not yet UI functional, but will be sooner than later.
The good news is I am seeing the Linux Phone movement be a positive feedback loop: the more features add, the more users, and the more folks helping to add features.
In comparison my Galaxy Tab S6 is just so much more black box and some stuff just does not work with no apparent way to debug! Like, I tried to setup a samba or SSH server on it for easy data transfer, with zero success after trying all the related apps in frdoid and elsewhere. It just does not work at all! Most likely some brain dead "security" option one can't override 2without rooting the device that is impossible to track down in the mess that is Android.
Running root used to be very simple and semi-sanctioned. Now you have to essentially rootkit your phone (Magisk) for it.
Most of the AOSP programs are abandoned (thankfully ROM maintainers update them!), and you have to deal with a lot of things not working with Google Play.
I actually completely agree with this.
> This is possible but nobody has built it yet.
Doesn't mean we should stop trying. Here's my $0.02 - I've been building an email solution based on Self-Sovereign Identity ideas, still in progress, but check it out: https://ubikom.cc or https://github.com/regnull/ubikom
Who is suggesting ‘stopping trying’?
To me the only thing that sounds like ‘stopping trying’ is boycotting Apple, as though that has anything whatsoever to do with the problem.
It doesn’t matter what vendor you use. The only thing you can do to help is to identify parts of the solution, explain them, and build them.
I don’t have time to check out your email solution but I hope it is part of the solution.
It became, "Sorry, AT&T carrier, AT&T rules."
Seriously? Perhaps heed the warnings? Whenever Apple tightened the reigns, thousand of apologists came to their defense. I wouldn't even have minded if they kept their obedience to personal decisions. But they extended their enlightenment to others.
And then take what actions, exactly? “Guys this is trouble” is …fine, but without “and we should therefore do”, it’s just kind of spitting into the wind.
Don’t buy or use their products.
The only progress for things like this comes at the political level, through pressure on politicians or corporations. That’s where you should be spending effort if this sort of thing matters to you.
A few Linux nerds buying some fantasy spook-free glibertarian phone will not make a dent in the problem.
Still good to keep up the pressure on public officials. That any interior minister is very happy about that doesn't have to be excused. Anyone else should be more like French people.
However I do think there’s value in gentle, consistent evangelism of privacy in ways that don’t make people feel bad. Most folks actually don’t want their deeply private stuff to be accessible. I’ve found that there are good analogies and ways to think about it that folks can get on board with and start caring to some extent.
I absolutely agree that political progress is the ultimate solution. I think the only way that will happen is if enough people demand it.
People: Ignore warning, buy iPhone
You: "Don't buy Apple products" isn't an action.
What is the follow-on? Which products should we buy then? Any phone other than Apple's? None?
Google phone? This is not an alternative if you care about privacy.
The allegory presented sounds insane (growing your own food) but honestly if you're not rolling your own flashed ROM on a 'droid then your privacy is dead already;
I understand throwing all your eggs in one basket is a terrible idea but Apples walled garden and heavy sandboxing was at least somewhat protective.
While the Pinephone isn't perfect yet, I would argue it will be able to replace Android/iOS in a few months.
Yeah this is the mobile phone equivalent of growing your own vegetables.
I get it, I'm a techy, but until the day is here where rooted phones aren't treated like a special case: for most people if they buy a droid then it's actually significantly worse for their personal privacy than an Apple device.
I don't want that to be the case, but until the pine phone is here and starts being a reasonable alternative; that's how it is.
Sadly...I know. The unfortunate thing is my recommendation for my family/friends for a phone is an iPhone, because it really is the best for security/privacy today, and they at least support their phones for much longer than an Android.
> I don't want that to be the case, but until the pine phone is here and starts being a reasonable alternative; that's how it is.
Fortunately, I think that time is coming sooner than later.
Someone will have to buy the pine phone when it is not a reasonable alternative to support the effort and keep the ball rolling.
It's no surprise why we got to this point.
Because that's who I'm speaking for.
The issue is that "the masses" just "go with the flow" and act like if everybody else is jumping off a cliff then it's ok to jump off a cliff too, and it is these mobs that move the markets and determine which products and services become the most popular.
However, in relatively free countries it is still possible to live one's life on one's own terms and not be in the same herd with the sheeple. It is definitely possible to live life without an iPhone and to have an Android phone with F-Droid rather than the Google Play Store as one's primary phone.
It doesn't even need to be an "action". You will have to be at least conscious of what is happening.
That is not what is happening, right now ( or before that ) many are defending Apple, and also an attitude of not "my problem". Writing off any warning as either pessimistic or conspiracy. Having some healthy dose of skepticism is somewhat an unpopular view in modern day society. Especially in Tech and America.
An action could and will be taken once enough people are conscious of what is happening. Right now the scale and critical mass just isn't there (yet).
BTW I think one of the roots of our problem is that we seem to end up in these really weird "winner take all" distributions with only 2-3 main winners and rarely any serious alternatives. It happens with operating systems, browsers, it also happens within programming language communities, like Javascript, etc. Everybody piles up into one of the 2-3 most popular framework options (React, etc). Same thing for Linux distributions. etc.
It would be nice if we could figure out some sort of nudge or hack that would reduce this tendency and encourage more distribution of mass.
Issues are also being found way faster simply due to a bigger user base.
Chose open standards, use and contribute to FOSS, avoid social networks, get involved in your local community, etc.
No need to go to extreems or complicated plans, corporations follow the customers.
But nobody did listen. Quite the opposite. I never had a facebook account, and now today people are boasting when they leave FB. But 10 years ago ? Oh we were the paranoid extremists .
Even today my friends regularly pressure me to get whatsapp.
IANAL but we (via our elected representatives) can push a law that prohibit restrictions on execution of users' code on their own devices. Or we can split app stores from vendors and obligate them to provide access to third-party stores, like we do with IE and windows.
Also, it's completely doable to stop NSA/Prism totalitarian nonsense.
What we can do as tech people?
- raise awareness
- help people to switch from big tech vendor locks
- help people harming BT by installing adblockers, pihole etc
- participate in opensource (with donations or work)
- probably something else
The most pressing things that should be supported, to have the world I think we want:
1. Mandate open app stores. Your device, your choice. *
2. Mandate open browsers. Your device, your choice. The internet is fundamentally an extension of the OS at this point, so an free (as in speech) connection choice is a requirement for an open OS. *
3. Mandate open apps. Your device, your choice. Installing unsigned apps can be warned, but not prohibited (outside of enterprise devices).
4. Mandate configurable tracking. Your device, your choice. There must be a clear option to disable all tracking, along with an API / payment ecosystem for apps to detect this and request alternative payment. I.e. "free if advertising on + $5.99 if advertising off".
5. Mandate right to repair. The manufacturer must provide necessary technical specifications (hard or soft) for a base level of modification and repair. If the manufacturer no longer supports the device, everything must be released to the public.
* Selection must be offered at time of device setup. Installing alternatives can be warned, but not prohibited (outside of enterprise devices).
I think people have a hard time seeing the ethics in the technology they choose to use. Maybe the next wave of net-natives will be able to rediscover that common thread of rebellion and resist. It's insidious, I'll give you that. It's not obvious what is being surrendered with every participation on these platforms but it doesn't take a genius to see clearly.
A strong step forward that gets nullified when Facebook buys the alternative app you're using, or when the app you're using does things as Facebook does.
You can propose all individual options you want, this is a collective issue that won't get fixed just by calls to individual action.
Pretty cheap, too.
I'd argue: avoid using proprietary networks, avoid vendor lock-in with software and hardware, and use hardware that one is allowed to use their own software on. Champion using open and federated protocols for social tools.
I think solutions exist, but honestly, it isn't easy.
> Making holier than thou comments about everyone else being sheep isn't helpful
I would offer the GP comment isn't necessarily a holier than thou comment, it's a comment of frustration. Frankly, I feel the same frustration.
It's tiring to hear snide remarks of "ohh yeah, we can include you for something because you don't have an iPhone". Hell, I have openly heard, even on this forum, that people don't include folks on social conversations with EVEN THEIR OWN FAMILY because of the dreaded "green bubble". (FYI, MMS is entirely done over HTTP and SMS! How is Apple's MMS client so bad that it can't handle HTTP and SMS?).
Or there is the "why don't you have WhatsApp/Facebook/Instagram/etc." and people think your some sort of weirdo because you don't want to use those networks.
So to be honest, when I see something like that, I think "Well I'm not surprised, this is what happens when you are locked out of your own hardware".
> What are you doing to fix it?
While GP may not be doing anything, others are helping and actively working for alternatives. For example, I have been working to get the Pinephone to have MMS and Visual Voicemail support so I can use it daily. I an very fortunate to work with a lot of very talented and motivated folks who want to see it succeed.
Buy a feature phone or a phone from a vendor who doesn't have this power.
Switch to Linux.
Stop buying from companies that abuse you.
Elect politicians who care about your rights.
You know what's not helpful? Attacking the messenger, regardless of how sanctimonious you think he is.
Nothing, because my phone is rooted Android.
I'm dependent, just as you say, and have no illusions about that.
Getting into this situation wasn't my decision (it was a collective "decision" of our society), and getting out of this won't be due to anything I'll personally do either.
The only difference between me and the average joe is having understood that we have a problem earlier than most.
Then they came for the Socialists. And I did not speak out Because I was not a Socialist
Then they came for the trade unionists And I did not speak out Because I was not a trade unionist
Then they came for the Jews. And I did not speak out Because I was not a Jew
Then they came for me. And there was no one left To speak out for me
It's perfectly legal elsewhere (if a bit weird) to have some Simpsons/whatever mash-up of sexualised images, but if I flew on a plane to the land down under, would I then be flagged?
edit: If this is scanning stuff on your phone automatically, and you have whatsapp or whatever messenger set to save media automatically, then mass texting an image that is considered 'normal' in the sender country, but 'bad' in the recipients, you could get a lot of people flagged just by sending a message.
[1] https://arstechnica.com/tech-policy/news/2010/01/simpsons-po...
And you know what? Most people deserve to be locked in and subject to automatic surveillance. They will wake up when their phone creates a China-Style social score automatically, but then it will be far too late. It's a shame for those people that fought this development for years, though. But the "I have nothing to hide" crowd deserves to wake up in a world of cyber fascism.
which one?
What exactly are you talking about, the OS?
Apple? How? Your other option is Android, who do you choose when they start to do it?
Or when governments decide to mandate that ALL phones need to legally have "scanning all the files on it and report them back to the police database" mechanisms?
The EU? Particularly how? An organization that has been deliberately structured to supersede the legitimacy of nation states and export it's power to all of it's members at the whim -- sometimes it seems -- of some aging out of touch bureaucrats.
I'm not even a #brexiter, btw.
Should Scotland become an independent nation? There was a public debate and people had opinions -- and there were mechanisms in place to act on and make a change, as an example.
There has been no public debate on this in a national sense (anywhere), and also no mechanisms by which people could decide to change it. I'm not sure people deserve it.
When you mention that set of population as deserving the consequences, it does not seem too far to me from "People who want trains instead of cars deserve trains". Is this relevant? The big problem is, people buy controversial services, hence finance them and endorse them, hence strengthen them, and in some cases these services make the acceptable ones extinct: the big problem is that people do not refuse what is not sensible, and sensible people have to pay.
Already here where I live, I cannot get essential services¹ because that practice made them extinct!!!
¹(exactly: public administration, tick; banking, very big tick; medicine, not yet. And you did not mention ___the cars___, and more...)
Other note: you wrote
> nearly no one owns their phone
and some of us are stuck with more reliable older devices, which soon may need some kind of replacement. If you know the exceptions to the untrustable devices, kindly share brand/model/OS/tweak.
(It makes me dream of a version which also is ruggedized and uses a high resolution OLED display... But this can easily already be the new palm companion.)
I hope that the current difficulties in (global) manufacturing will be soon be over.
Edit: having mentioned "pines" was not meant to be a pun - I just realized the odd potential reference to the PinePhone.
I can’t say I’m surprised they are implementing this (if true), under the radar. I can’t imagine a correct way or platform for Apple to share this rollout publicly. I’m sure nothing will come of this, press will ignore the story, and we all go back to our iPhones
But Apple iCloud for ex doesn't intrude on your privacy any more or less than Google Photos.
Edit: Actually, this won't even require an exploit if they also scan media for people who have enabled "iMessage in iCloud".
Just send someone an image in the DB (or an image that's been engineered to generate a false positive) and wait for them to get raided.
Authoritarian regimes love this.
Presumably the same could apply to your phone. Most messengers save images automatically. I presume the images are immediately scanned against hashes once saved. And the report is immediately made if it passes the reported threshold. There’s no defence against this. Your phone number is basically public information and probably in a database somewhere. You have no protection here from abuse, if you’re a normal citizen. I bet most people don’t even turn the auto save setting off on WhatsApp.
> Regardless of what Apple’s long term plans are, they’ve sent a very clear signal. In their (very influential) opinion, it is safe to build systems that scan users’ phones for prohibited content.
> That’s the message they’re sending to governments, competing services, China, you.
The message could equally well be ‘We won’t become an easy political target by ignoring a problem something most people care about like child porn, but we are going to build a point solution to that problem, so the public doesn’t force us to bow government surveillance requests.’
It’s easy to nod along with an anti-Apple slogan, but we need to consider what would happen if they didn’t do this.
Public opinion is not in favor of giving safe harbor to pedophiles and child pornographers, and I can’t see why anyone would even want Apple to fight that battle.
What we do know is that it is a proprietary solution using a proprietary hash which only applies to Apple products and that Apple has always presented themselves as a family friendly company that doesn’t support criminal use cases.
Everything points to this being something Apple thinks needs to be solved before the public asks why they haven’t.
If it was a government demand, we’d presumably see Google responding to it too.
You still haven’t explained why not working to deter pedophiles and child pornographers, is an important battle for them to fight.
They might think it’s a problem they want to solve on their own terms, and that would seem to be what they have done here.
Not solving a problem people care about just because the government also cares about it seems illogical. I think a lot of people like the idea of corporations taking responsibility for the social problems they cause without needing to be forced to do so by the government.
Engineered collisions seem unlikely too. Not impossible. Unless there is a straight up cryptographic defect in the hash algorithm, it seems hard to see how engineered collisions could be made to happen at any scale.
This isn't some web tech startup where a mistake means someones tshirt got sent to the wrong address. Peoples lives will quite literally be ruined over mistakes here.
Is it a once in a million issue? The collision rate matters. It could easily be much higher and then it wouldn’t matter that it was being used at Apple’s scale.
What is the actual attack you are imagining?
Like gay porn, 'save Khashoggi' meme, or a photo from documentary about missing Uighurs.
It's hard to imagine how this could be misused, right?
E.g. how the hashes are computed, where they come from, and what happens when a positive match is detected.
Until we have a clear understanding of these things, the rest is just speculation.
Money citizens of those countries want to give him.
I switched to the Apple ecosystem 2 years ago and have been extremely happy.
I couldn't see a single reason to switch back.
Today that reason came. What goes on on my phone is my business.
I guess fairphone next.
Again, I think have nothing to hide now so I can sat this loud and clear now. Given what recent elections have shown us we cannot know if I have something to hide in a few years (political, religious? Something else? Not that I plan to change but things have already changed extremely much since I was a kid 30 years ago.)
At the end of the day laws are relative so to say. The thought behind such a system is noble indeed, but as we've seen, anything any government gets their hands on, they will abuse it. Classic example being PRISM et al. In theory it's great to be able to catch the bad guys, but it was clearly abused. This is from countries that are meant to be free, forward thinking etc, not any authoritarian regimes.
People in this thread are asking what Saudi Arabia, China etc will do with such power that Apple is adding, you bet your ass that they'll use it for their own gain.
I want to believe in such systems for the good. I want child abusers caught. But a system that equally can be abused by the wrong people (and I guarantee you that will be western countries too) ain't it.
[1] https://www.businessinsider.com/apple-data-china-censors-app...
And at that point the well meaning privacy advocate who worked hard to get some nice policies to protect users is booted off the project because you can hardly tell the shareholders and investors who own the company that you're going to ignore $billions in revenue or let your rival get ahead because of some irrelevant political movement on the other side of the world.
It's happened plenty of times before and it'll happen again.
Why? Is it simply because it fits their world view?
I’m not sure if it’s a rumor or not but there was a thread on HN the other day about Facebook exploring homomorphic encryption for running ads on WhatsApp and I wonder if wires got crossed?
Scanning on phones is a step toward end to end encryption for photos probably. Or scanning not uploaded photos. Or both.
A lot of people love not having options and having these decisions made for them.
I would never want a device like that or with something that scans my device, but I think the vast majority of their customers if they even hear about it will think "I trust apple, they know what's best, it wont affect me"
Im ok with apple doing it because i think most apple users will be ok with it. I would not be ok with it if all Android devices started doing it though.
It’s less “I trust Apple” and more “if I really cared, I’d have bought from another designer”.
This does looks good on paper - caring for customers and their security, peace of mind but tomorrow it might be a total vendor-lock with no ways of installing any other software than one approved by the corporate entities.
[1] - https://www.ghacks.net/2021/08/03/windows-10-blocks-potentia...
My parents took lots of photos of me as a baby/small child. Say lying naked on a blanket or a naked 2yr old me in a kiddie pool in the summer in our backyard. Those are private photos and because it was the 1970s those were just taken with a normal non-digital camera. They were OBVIOUSLY never shared with others, especially outside immediate family.
Transform that into the 2020s and today these type of pictures would be taken with your iPhone. Would they now be classified as child pornography even though they weren't meant to be shared with anyone nor were they ever shared with anyone? Just your typical proud parent photo of your toddler.
Sounds a bit like a slippery slope, but maybe I am misunderstanding the gravity here. I'm specifically highlighting private "consumption" (parent taking picture of their child who happens to be naked as 1yr olds tend to be sometimes) vs "distribution" (parent or even a nefarious actor taking picture of a child and sharing it with third parties). I 100% want to eliminate child pornography. No discussion. But how do we prevent "false positives" with this?
If that sounds absurd - most laws are like that. For better or worse, there's a human who interprets the law, not a computer. It's unfortunate Apple is choosing to elect a computer as the judge here, for exactly concerns like yours.
Unless someone has been distributing photos of your kids as child porn (which would probably be good to know) it's unlikely any of your photos will match the hashes of the photos in that database.
I'm not sure that's how it works, but that's what I've gathered from the other comments on this post.
Also, this functionality isn't something they should be able to implement without telling their end users.
It is also problematic because it will just make the cyber criminals more technical aware of what counter measures they must take to protect their illegal data.
The consequence is very bad for the regular consumer: the cyber criminal will be able to hide, and the government has the possibility to scan your files. End consumer lose, again.
Contrast this with my desktop where, in the press of a few buttons, I am presented with the source code for the CPU frequency scaling code.
Bring on the Linux phones.
I really don’t want to wait half a minute for taking a picture, and would prefer if the phone would not be untouchably warm after the fact.
Ontop of this, it gives apple far too much power. What happens when someone they don't like owns an iphone? They can pull an FBI and put the content onto the device, and having it then "automatically detected".
Apple: Say no more, here they are. Hope you won't imprison all of them, as that would decrease our services revenue substantially, lol.
Also Apple: Privacy is a human right, buy more iphones.
Treat your phones as an enemy. Use real computers with VPN and software like Little Snitch when online. Use cameras for photography and video.
The benefits of this approach are immense. I have long attention span. I don't have fear of missing out.
If governments wan't the future to be painted by tracing and surveillance mediated towards people trough big tech - lets make it mandatory by law. And since big tech will reap benefits from the big data they must provide phones for free. :)
I'm assuming your "real computer" is a mac (since little snitch is mac only). What makes you think apple won't do the same for macos? Also, while you have greater control with a "real computer", you also have less privacy from the apps themselves, since they're unsandboxed and have full access to your system.
On a serious note: If you want privacy/security as a semi-professional use: No mobile phone. https://www.qubes-os.org/ with https://www.qubes-os.org/doc/certified-hardware/
Sounds like this is more about "checking your phone less", than "improving security/privacy". This is evident elsewhere in your advice. eg. "phones as an enemy", but no advice about killswitches for microphone? Or some sort of mitigation against GPS/mobile networking tracking?
But I feel there's a valid argument to be made that if your adversary is the sort of people who'd be feeding Apple image hashes to find people, you're probably be wise to carry a regular phone on which you do boring norm-core sorts of things.
A phone you use to take pictures of cats and pay your rent using banking apps and call your parents - while not using it to communicate with your dealer or your anarchist collective or your friendly investigative journalist.
So, it's rather hard to avoid banking apps.
Also, the PSD2 directive implements the duty of providing API infrastructure for third-parties. [1]
https://www.ecb.europa.eu/paym/intro/mip-online/2018/html/18...
Restraining myself to write something very strong about phone security and general user expectancy and duly expectancy (low) - let us stress again the legal side: how do you prove to a bank that, in case of theft from the account, your device was safe? People who see their money stolen then have controversies with the bank about responsibility.
BTW: PSD2 has been, in many parts, a huge nightmare. Furthermore, healthy parts of it for some reason have not been implemented.
Dude, you carry it around with you, with its radio enabled. You're just fooling yourself.
PhotoDNA and systems like it are really interesting, but it seems like clientside scanning is a dangerous decision, not just from the privacy perspective. It seems like giving a CSAM detector and hashes to people is a really risky idea, even if it’s perfect and it does what it says it does without violating privacy.
If the algorithm and the blocklists leaked, then not only it would be possible to develop tools that reliably modify CSAM to avoid detection, but also generate new innocent-looking images that are caught by the filter. That could be used to overwhelm law enforcement with false positives and also weaponized for SWAT-ing.
https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
- terrorism - child pornography
Try to protest it and you will be prompted with a nice "do you have anything to hide?" question by the masses.
The advertised intention of these tools could not be farther from the truth and people happily fills their pockets at each new launch.
- (Check) fragile
- (Check) best monopoly app store
- (Check) high price
- (Check) phones the police in case of a sha collision
Personally I somehow doubt that MS/Google weren't scanning private content (aka not shared) for this type of material. But can't have transparency with these behemoths.
[1] https://www.euronews.com/2021/07/07/eu-adopts-temporary-rule...
[2] https://www.theverge.com/2014/8/7/5977827/microsoft-tips-off...
[3] https://www.theverge.com/2014/8/5/5970141/how-google-scans-y...
Another side note is about near future when someone comes up with synthetic CP images, will they also be criminalised?
You can read about this directly from a victim via this NYT article: https://www.nytimes.com/2020/12/04/opinion/sunday/pornhub-ra...
Conversely, when I use drugs, I'm paying someone, so I'm actually directly funding criminals. Depending on the country and the drugs, this is often putting cash in the hands of a very violent cartel.
1) Does this work only on iPhones or will it be iPads, as well?
2) Is this part of a system software update? I wonder if that will show up in the notes and how it would be spun. "In order to better protect our users ..."
3) If it is part of the system software update, will they be trying to make it run on older iDevices?
4) Is it just photos in your photo bin, iCloud, or does it start grabbing at network drives it attaches to? I could see the latter being prone to blowing up in their proverbial faces.
The Four Horsemen of the Infocalypse ride again!
A court order should be mandatory, not this blanket scanning.
Thankfully, there is the option to steer clear of Apple devices.
Has apple published its training data somewhere?
This is something that child porn enthusiasts using apple would invest their time and money into. A regular person would not think to do this (yet).
This might be naive, but I would guess that the best way to fight this kind of thing is to let people know more of the case details. People would protect themselves, find the crimes, and stop unwittingly supporting them. For instance, if it can be shown that cryptocurrency or encrypted messengers are used to a significant extent, the community will either find a technical solution, or stop using it.
Back to an Android phone, once I confirm this story is true.
* A car that automatically pulls over when a police cruiser attempts to intercept you
* A front door that unlocks when a cop knocks
* A camera that uses AI to detect and prevent the photography of minors, police, and critical infrastructure
* A Smart TV that counts the number of people in your living room to ensure you aren't performing an unauthorized public broadcast of copyrighted content
Surely, at least one of those sounds ridiculous to you. As well-intentioned as this scanning may be, it violates a core principle of privacy and human autonomy. Your own device should not betray you. As technologists, just because we can do something doesn't mean we should.
The UK and Israel allowed cops to monitor cell phone locations to crack down on unlawful gatherings in private homes.
https://www.theguardian.com/world/2020/mar/17/israel-to-trac...
The problem with allowing this is that you’re paving the way for future tyrants to use it against us.
It's funny how everybody talk about the future. This is happenning now. Remember how a certain german guy took the power some 90 years ago ? He was elected.
People nowadays voluntarily carry tracking devices. This will not stop getting worse until that behavior is denormalized.
The power to be gained from abusing it is beyond irresistable. Expecting those in power to not abuse it is like expecting a heroin junkie to be a good pharmacist.
on the strict premise that tracking is exceptional fair use from a law enforcement agency. Do not mix up the voluntary accolites of Zuck and people who want tools. (The use of 'tools' in the sentence is an originally unintended pun. I will keep the term 'accolites', though tempted to replace it for the pun.)
I know I know, Disneys inability to collect money for every pair of eyes watching their movies during VHS times literally made it one of the poorest companies on the world. /s
Also, painting is a physical object. It's a one of a kind.
EDIT: your objection and many many other questions like that are nicely argued against here: https://www.youtube.com/watch?v=mhBpI13dxkI (The Surprising History of Copyright talk by Karl Fogel)
EDIT1:
More than that. People don't live of royalties. Publishers do. Can people create without being paid upfront by publishers? Obviously yes. They can be paid upfront in kickstarter-like arrangement. They can be paid via donations later. They can even be paid via "retroactive funding of public goods" [0]. Or they can be never paid - just as they are today.
[0] - https://medium.com/ethereum-optimism/retroactive-public-good...
EDIT2:
Also, imagine how much good could come of long-standing IP properties owned by Disney if not for them sitting on it. Vide disaster that is current management of Star Wars.
Or we can talk about the mess you get with too many groups involved? Do you know who has the rights to Westwoods Dune? Westwood itself had a limited time license for games based on the movie Dune, which also had a license based on the Dune books. So you would have to deal with three license owners to make a Game involving the Ordos faction. Have fun convincing EA, whoever owns the movie rights and the original authors family that you can make a new game worth their signature on a new licensing agreement (you could probably manage if you have a small country to sell).
People WILL be a fan of all of this if the alternative is lots of robbery / car theft etc etc.
We see this globally. If the state is not offering security now - they will accept incredible craziness for security.
One note - in most cases folks trust Apple MORE than they would for example the Trump administration. Food for thought.
Maybe one trusts Apple more than <insert politician>, but they cannot so easily elect away Apple.
There was concern about phone's being grabbed - street robberies. Regardless of whether you believe their were sweeping generalizations - apple ended up creating more power for themselves with their activation lock system. If they don't want to let you sell your phone to someone - they can block use of your phone. But folks trust them to operate the system reasonably, and so far so good there.
They have locked down their app store very tightly for a variety of reasons including supposedly for security. Users have accepted that.
Apple's competitors (google photos etc) generally are directly scannable in the cloud by google et al. Facebook and others routinely scan users photos. Youtube scans their videos etc. My guess is apple will explain why they are doing it and users are going to be happy.
And yes, users are linking things like ring doorbells and home security video cameras together or registering them so that the police state can use them.
The technology is there, now we only need the motivation. If politicians decides that they want it now, they can simply orchestrate a media campaign and have it. The next time an "outrageous" act of crime is conducted, they can make sure that it stays at the media attention and be portrayed as "If we don't act now very bed things will happen", then slide in their solution.
* Cars can automatically pull over by installing a cheap cut fuel cut switch that can be activated by short range radio. In many places people are used to add devices for toll collection anyway. People are also used to pay for regulatory inspections on their vehicles.
* For the old cars, simply connect an NFC reader that unlocks the central lock system of a car by a master key. For the new cars, simply make manufacturers add a police master key.
* Commercial drones are already stopping their users from flying over forbidden areas, simply extend that to smartphones. Smartphones have enough power and sensors to identify forbidden locations and persons. Add NFC kill switch, meaning the police can send a signal to lock down cameras.
* There were reports of Smart TVs that record all the time, simply mandate it to all manufacturers and enforce automated inspection of the recordings.
Apple devices already betray their "owners", and they've been doing it for a long time.
You can't repair them.
You can't run your own software.
You can't use a better, more compliant web browser.
Businesses have to pay a 30% tax.
Businesses are forced to use login with Apple and forfeit a customer relationship.
Businesses have to dance to appease Apple. Their software gets banned, randomly flagged, or unapproved for deployment, sometimes completely on a whim.
Soon, more iDevices and Apple Pay will lead to further entrenchment. Just like in the movie Demolition Man, everything will eventually be Apple. Your car, your movies, your music, your elected officials.
If you're interested I suggest you also have a look at photo DNA[0]
Today its csam. Tomorrow "misleading information". etc.
For tech stuff I do need Id like to pick dedicated "appliance like" tools, but each of those requires some research.
It lets "Apple Scan"? So Apple is going to proactively scanning your photos using a tool then install?
So many questions about this. It doesn't add up.
Instead of specifically targeting suspects, everyone is surveyed by default. Welcome to a world of mass surveillance.
As I understand it: it's a tool (that sends a command of some sort) that compels an iphone to perform the hashing match operation, and output results. Is that correct? Does it notify the user?
If I had to build it within apple's privacy framework, that'd probably be my approach: remote command causes sepos unlock of photos library (even running the job on sepos?) to do photo scanning. sepos returns hashes that match
So which Linux desktop sucks the least at the moment?
Contraband detected
If they are only concerned with iCloud accounts then... why not scan in the cloud? Can anyone explain to me why client-side scanning is actually needed here? As far as I'm aware, Apple only E2E encrypts iMessages, not iCloud photos or backups.
US politicians (to say nothing of countries with less individual freedoms) already openly pressure tech companies to censor specific content. And tech companies will do so even to the point of blocking direct, private, 1-1 messages sharing specific news. In that light, Apple's crossing a line to client-side scanning seems deeply concerning.
I don't see how keeping this as narrowly-targeted as it's being advertised would ever be possible or even intended.
I can see law enforcement showing up at my door one day with a search warrant demanding to have a look around, and I would have no idea why they’re there, but they’ll want to look through all my personal belongings.
Worse yet, I might come home from work one day, see my windows broken, see my place has been ransacked and my computers are missing. I would call the police to report a burglary only to hear than that I’m under investigation and they need me to give them the key to decrypt my hard drives.
While this argument can and have indeed happened in other instances, this is akin to saying that we should not give anyone any powers to do anything because it is a slippery slope that they can use it to do bad things.
What then sets out the difference between what a slippery slope and a non-slippery one is? Checks and balances and the history of USA have shown that this is indeed what can reign in the worst instincts of any single entity. History of course have also shown when these failed and these should serve not as a reason to reject the idea of checks and balances but as acknowledging it's imperfection and think of ways to mitigate it.
Two instances: 1) Post 9/11 Patriot Law 2) McCarthy era: https://www.e-ir.info/2011/11/03/the-extraordinary-injustice...
Checks and balances are never a done deal. If we reject checks and balances and as a result reject new tech because of abuse potential, how then should we as a civilisation advance?
As far as I know, this kind of scanning is not legally mandated. So, either they think that this will truly make the world a better place and are doing it out of some sense of moral responsibility, or they've been pressured into it as part of a sweetheart deal on E2E ("we won't push for crypto backdoors if you'll just scan your users' phones for us"). Either way it doesn't thrill me as a customer that my device is wasting CPU cycles and battery life under the presumption that I might possess data my current jurisdiction deems illegal.
For all the acclaim privacy-forward measures like GDPR get here, I'm surprised there isn't more outright repudiation of this frankly Orwellian situation.
https://fightthenewdrug.org/apple-fights-child-porn-by-scann...
On another note—OSes may only be starting to do this, but that same cynicism still leads me to presume that arbitrary closed-source third-party apps — or even closed-source binary distributions of open-source apps (e.g. the App Store versions of FOSS software) — could have been quietly scanning the files people are passing them for CSAM for years now, without telling users about it. It always seemed to me like the kind of thing it'd make sense to quietly slip into a media player like VLC.
The indian government has recently introduced new laws that give them power to dictate terms over many online platforms and broaden their surveillance powers over online social media and messenger platforms. One of the laws dealing with messenger platforms requires the platform to track shared content, especially "origin of content" (first originator) of any content that is shared through their network. (Facebook / WhatsApp has already gone to the court to challenge this, as it claims that they would need to break end-to-end encryption for this and it thus violates indian privacy laws).
Apple's iMessage platform has more than 25 million users, and thus should come under the ambit of this law. But strangely, the indian government seems to have given them an "exception" .... and now we know why.
I worry about the method itself: will a simplest of firewalls be very effective against this? The one that forbids any communication except with few hosts, like a pair of them?
>Hashes using a new and proprietary neural hashing algorithm Apple has developed, and gotten NCMEC to agree to use.
>We don’t know much about this algorithm. What if someone can make collisions?
https://twitter.com/matthew_d_green/status/14230792585163448...
Oh boy..
I suppose that it's a bit like when you do a reverse image search on your favourite search engine. When you upload an image, the engine will try and find images that the ML thinks look the same, even if the bits and bytes that make up the file are different. From what I can see, the similarity detection will be much more specific so as to not generate false positives. As you theorise though, it might be possible to modify images to evade detection if the hash's match specificity is high enough.
All bearing in mind that the pictorial hash also is supposedly designed to be a one-way function to ensure that those who know file hashes don't know what the original contents of the file are.
Privacy is about the only thing keeping innocent people free in today's world.
iOS downloads CSAM scanning code from Apple and runs it locally—hence, client-side tool.
All?
They have to download the hashes in order to compare them, I wonder if a pihole could help here?
Can Samsung phones that are de-googled work ? I am specifically interested in a new phone that Samsung launches, can it be degoogled ?
CSAM? According to Google that's Confocal Scanning Acoustic Microscopy. Or something.
And what with the tweeters? I think my laptop just gave me thighburns from the CPU bloat that clicking on that link caused. Eight seconds to render the page? Why do folks still use this twerker website?
Hint: the relevant thing wasn't to do with my laptop.
Rather than complain about it, I am interested in what alternative solutions exist, or how concerns regarding privacy and abuse of this system could be mitigated.
Sacrificing the privacy of the many to help catch a relatively small amount of (admittedly some of the worst possible) criminals, while simultaneously enabling yet more effective surveillance and oppression by those inclined governments (of which there are plenty) is a pretty terrible idea.
Eliminating the 4th amendment or mandating clear walls sure would make the cops' job easier. But no one thinks that's even a remotely good idea.
There is literature about how obedience enables totalitarianism. Tons of it. Should be part of any schools curriculum in my opinion.
Yet.
Remember SOPA that the internet killed?
stop trading freedom for security.