Apple Photos phones home on iOS 18 and macOS 15

I highly suggest everyone else does their darnedest not too either. Don’t do it in your own software. Refuse and push back against it at $dayJob.

I realize that my small contribution as a privacy and data-respecting SWE is extremely small, but if we all push back against the MBAs telling us to do these things, the world will be better off.

Nowhere in marketing materials or what passes for documentation on iOS we see an explanation of the risks and what it means for one’s identity to be sold off to data brokers. It’s all “our 950 partners to enhance your experience” bs.

The backlash against Microsoft's Windows Recall should serve as a good indicator of just how deeply people have grown to distrust tech companies. But Microsoft can keep turning the screws, and don't you know it, a couple years from now everyone will be running Windows 11 anyways.

It's the same for Android. If you really want your Android phone to be truly private, you can root it and flash a custom ROM with microG and an application firewall. Sounds good! And now you've lost access to banking apps, NFC payments, games, and a myriad of other things, because your device no longer passes SafetyNet checks. You can play a cat-and-mouse game with breaking said checks, but the clock is ticking, as remote attestation will remove what remains of your agency as soon as possible. And all of that for a notably worse experience with less features and more problems.

(Sidenote: I think banking apps requiring SafetyNet passing is the dumbest thing on planet earth. You guys know I can just sign into the website with my mobile browser anyways, right? You aren't winning anything here.)

But most users are never going to do that. Most users will boot into their stock ROM, where data is siphoned by default and you have to agree to more data siphoning to use basic features. Every year, users will continue to give up every last bit of agency and privacy so as long as tech companies are allowed to continue to take it.

It would have been so much easier for them to just send the whole photo as is to a server and process it remotely like Google does.

Are people giving up their privacy? Looks to me it’s being taken without consent, via enormous legalese and techniques of exhaustion.

forced can also mean the whole no privacy by default and dark patterns everywhere.

Aaaaand no.

This is everyone, it is universal, I've met many people "in tech" who also click the most "visually appealing" button because they are trying to dismiss everything in their way to get to the action they are trying to complete.

The microcosm that is HN users might not just dismiss things at the 95%+ rate, but that is because we are fed, every day, how our data is being misappropriated ate every level. I think outside of these tiny communities, even people in tech, are just clicking the pretty button and making the dialog go away.

If you do not want it (and that is >90% of people, who never asked for it, never requested it, but was forced upon them these 'enriched' lies and exposure to corporate greed).

By adding that extra step forcing users to be aware of (and optionally decline) the vendors collection of personal data, it adds a disincentive for collecting the data in the first place.

In other words, opt-in can be thought of as a way to encourage vendors to change their behaviour. Consumers who don't see an opt-in will eventually know that the vendor isn't collecting their information compared to others and trust the product more.

I agree that a lot of people don't read, or attempt to understand the UI being presented to them in any meaningful manner. It really is frustrating seeing that happen.

But, think about the "colorful" option you briefly mentioned. Dark patterns have promoted this kind of behaviour from popups. The whole interaction pattern has been forever tainted. You need to present it in another way.

It is definitely true that most users don't know what they're opting out of, they just understand that they have basically nothing to gain anyway, so why opt-in?

But actually, that's totally fine and working as intended. To be fair to the end user, Apple has done something extremely complicated here, and it's going to be extremely hard for anyone except for an expert to understand it. A privacy-conscious user could make the best call by just opting out of any of these features. An everyday user might simply choose to not opt-in because they don't really care about the feature in the first place: I suspect that's the real reason why many people opt-out in the first place, you don't need to understand privacy risks to know you don't give a shit about the feature anyway.

How many people use a web based camera or web based photo album app?

I suppose in some sense it is, as it a reverse-geo lookup service, but it's also no where near to the front in the location privacy war.

Cell phone providers basically know your exact position at all times when you have your phone on you, credit card companies know basically everything, cars track driving directly, etc. etc.

I can see why some people would be up in arms but for me this one doesn't feel like missing the forest for the trees, it feels like missing the forest for the leaves.

The interesting thing is that Apple has created a cathedral of seemingly objective sexy technical details that feel like security. But since it’s all trust, feelings matter!

So my answer is, if it feels like a privacy violation, it is. Your technical comparison will be more persuasive if you presented it in Computer Modern in a white paper, or if you are an important Substack author or reply guy, or maybe take a cue from the shawarma guy on Valencia Street and do a hunger strike while comparing two ways to get location info.

Something to consider - location is geo-encoded already into photos and doesn't need this uploaded to Apple servers. Searching can be done locally on device for location.

Apple goes as far as to offer a setting to allow the user to share photos and remove the geocoding from it.

Offering a new feature is opt-in.

Unfortunately, against my better wishes, this only erodes trust and confidence in Apple that if this is happening visibly, what could be happening that is unknown.

It seems trivially possible to do this in a more privacy preserving way: geocode the search query and filter photos locally.

No idea how Apple implements it though.

And if you design software that uses tracking and what not. Go fuck yourself.

The only thing I've found awful is the mindset of the people implementing the banners.

That you feel frustration over that every company has a cookie banner, is exactly the goal. The companies could decide that it isn't worth frustrating the user over something trivial like website analytics, as they could get that without having to show a cookie banner at all.

But no, they want all the data, even though they most likely don't use all of it, and therefore are forced to show the cookie banner.

Then you as a user see that banner, and instead of thinking "What a shitty company that don't even do the minimal work to not having to show me the cookie banner", you end up thinking "What a bad law forcing the company to inform me about what they do with my data". Sounds so backwards, but you're not the first with this sentiment, so the PR departments of the companies seems like they've succeed in re-pointing the blame...

Web sites could legally use cookies for non-tracking purposes without cookie banners but considering people have not stopped visiting sites despite the fugly click-through cookie banners makes them a failure.

All it takes is for 50% of the internet users to stop visiting web sites with them, and web site authors will stop tracking users with external cookies.

On desktop and Firefox app, I only browse through private browsing so cookies are mostly irrelevant as session ends as soon as all windows close.

(The "I don't care about cookies" extension is similarly effective, but since I'm already running u-block origin, it makes more sense to me to enable it's filter.)

This would have the added benefit of being configurable and work on a bunch of apps instead of just one at a time too

Operating system level controls, though, provide a single control plane. One can very easily imagine OS-level toggles per application of:

[No Internet, No Internet outside your own app-sandbox, Ask me every time, Everything is allowed].

No opt in from apps required -- they might break if the network is disabled, but the user is still in control of their data.

Users had a global way to signal “do not track me” in their browser. I don’t know why regulators didn’t mandate respecting that instead of cookie consent popups.

Apple IDs could easily have global settings about what you are comfortable with, and then have their apps respect them.

There's little to no conscious choice in this. But there is a lot of money in this. Like... a LOT of money. If I were to try to influence society to be okay with it, it would be a no brainer.

So, to me, it's obvious that society has been brainwashed and propagandized to accept it. But doing so generates hundreds of billions if not trillions of dollars. How, exactly, such manipulation is done is unknown to me. Probably meticulously, over the course of decades if not centuries. I know that the concept of privacy during the writing of the constitution was much, much more stringent than it was in the 70s, which is much more stringent than it is today.

But, I am very confident it is happening.

Try disabling the motd stuff - it's quite pernicious by design.

And removing the ubuntu-advantage package disables the desktop. lol.

Needless to say, if an Apple employee wanted to stalk someone (say an abusive partner, creep, whatever), the fact that this stuff phones home means that the employee can deduce where they are located. I've heard stories from the early days of Facebook about employees reading partner's Facebook messages, back before they took that kind of stuff seriously.

People work at these places, and not all people are good.

[1] https://www.koffellaw.com/blog/google-ai-technology-flags-da...

I hate to break it to you, but these things happen in Linux, too.

It's not the operating system that's the problem. It's that the tech industry has normalized greed.

Seems to me like a great implementation.

I have an incentive to see if the Linux desktop share has increased, so I usually run the survey for them to get my data point in. I also suppose the "gamer" crowed likes to show off how powerful their "rig" is, so I would imagine they commonly also run the survey for that reason as well.

The GDPR only applies to personal data. You can collect things like performance data without opt-in (or even an opt-out option) as long as you are careful to not collect any data that can be used to identify an individual, so no unique device IDs or anything like that. Of course, you should be transparent about what you collect. You also have to be careful about combinations of data points that may be innocuous on their own but can be used to identify a person when combined with other data points.

they did away with that.

For comparison Hunyuan video encodes a shit-ton of videos and rudimentary real world physics understanding, at very high quality in only 13B parameters. LLAMA 3.3 encodes a good chunk of all the knowledge available to humanity in only 70B parameters. And this is only considering open source models, the closed source one may be even more efficient.

I’m really curious how this feature is considered useful. It’s cool, but can’t you just open the photo to view it?

Here, Apple does that for you.

They literally own the code to iOS. If they wanted to covertly track their customers, they could just have their devices phone home with whatever data they wanted to collect. Realistically there would be no way to know if this was actually happening, because modern devices emit so much encrypted data anyway, it wouldn’t be hard to hide some nefarious in all the noise.

Time Cook isn’t some Bond villain, sitting in a giant chair, stroking a white cat, plotting to take over the world by lulling everyone into a false sense of privacy (I mean Zuckerburg already did that). Apple is just a large giant corporation that wants to make money, and is pretty damn open about that fact. They clearly think that they can make more money by doubling down on more privacy, but that doesn’t work if you don’t actually provide the privacy, because ultimately, people are really crap at keeping secrets, especially when a media group would happily pay for a story, even at Apple.

is it even that?

I don't see the benefit of this whatsoever

Erm, you’re aware of the whole Apple intelligence thing right? An entire product that costs Apple money, provided at “no cost” to the user (if you had an iPhone 15). Also every feature in an OS update has a costs associated with it, and iOS updates have cost money for the best part of a decade now.

Has it occurred to you that reason Apple includes new features in their updates is to provide customers with more reasons to buy more iPhones? Just because feature are provided at “no cost” at point of consumption, doesn’t mean Apple won’t make money in the long run, and selling user data isn’t the only way to monetise these features. Companies have been giving out “freebies” for centuries before the internet existed, and the possibility of large scale data collection and trading was even imaginable.

If they were to add this anti-CSAM feature, it is not unreasonable to think that Apple would be forced to add non-CSAM stuff to the database in these countries, e.g. anything against a local dictatorship/ etc. Adding the feature would only catch the low hanging CSAM fruit, at the cost of great privacy and probably human life. If it was going to stop CSAM once and for all, it could possibly be justified, but that's not the case.

I know someone whose MS account was permabanned because they had photos of their own kid in the bathtub. I mean, I guess the person could have been lying, but I doubt they would even have been talking about it if the truth was less innocuous.

And people working with CSAM and databases of CSAM have said it was a very bad idea.

If it’s for the children, then giving up our civil liberties is a small price to pay. I’d also like to give up liberties in the name of “terrorism”.

When we willingly give up our rights out of fear, these evil people have won.

This is not the revelation you think it is. Critics understood this perfectly.

People simply did not want their devices scanning their content against some opaque uninspectable government-controlled list that might send you to jail in the case of a match.

More generally, people usually want their devices working for their personal interests only, and not some opaque government purpose.

If this was a feature on its own then it would not be popular.

Citing national security, some danger will justify its existence.

Apple alone does not control and dictate what goes in, once you reach their level of size and wealth that exceed even developed countries, you ultimately cannot be the controller of your destiny purely as a profit orientated corporation.

ex) Meta, Microsoft, Google

But this is the fundamental issue. The author has no idea if personal data is being shared, they’ve made an assumption based on their lack of understanding. It’s entirely possible that all this service does (and arguably likely), is provide a private way for your phone to query a large database of landmark fingerprints, then locally try and match those fingerprints to your photos.

It doesn’t require send up private data. The phone could perform large geographic queries (the size of countries) for batches of fingerprints to be cached locally for photo matching. The homographic encryption just provides an added layer of privacy, allowing the phone to make those queries in a manner that makes it impossible for Apple to know what regions were queried for.

iOS photos already uses databases to convert a photo location into an address, so you can do basic location based searching. That will involve doing lookups in Apple global address database, do you consider that a violation of people’s privacy?

https://www.swift.org/blog/announcing-swift-homomorphic-encr...

It's tragic, because computing is in a professedly imperfect place right now. Digital privacy is under fire, many payments see a 30% digital service surcharge that is wholly arbitrary, and revolutionary cross-platform standards are being supplanted with proprietary and non-portable solutions that does not benefit any user.

As an American, I am ashamed that our government's dysfunction extends to consumer protection.

I mostly think the reaction to this article is overblown because it appeals popular ideas here about big tech. I think one should be wary of Apple’s claims about privacy: the reason is competition with Google and so they want users to be distrustful of the kinds of features that Google are better at implementing (I don’t want to say Apple isn’t trying to do the right thing either – if you look at accessibility, the competition was very bad for a lot of things for a long time and Apple was good despite the lack of commercial pressure). But I think one should also be wary of articles that make you angry and tell you what you suspected all along. (eg see the commenter elsewhere who doesn’t care about the details and is just angry). It’s much easier to spot this kind of rage-bait piece when it is targeting ‘normal people’ rather than the in-group.

(And to be clear I’m not even bothered by the signature check)

At a certain point you have to figure that they realize it doesn’t matter short of some government entity forcing them to stop. At the very least the protections they put in place (homomorphic encryption, etc) are more than I think most other companies would ever bother doing.

A lot of Apple's defaults are just plain bizarre. Why the hell is Spotlight seeing source code mimetypes and feeding it to the search index?

They are also not using your personal photos to feed the location database, most likely public sources and/or Apple Maps data. If they are relying on GPS-tagged public photos alone, you could probably mess up a system like this by spoofing GPS location en-masse and posting them online for years, but for what purpose?

After all, you wouldn’t actually be trusting the source code given to you to match what’s running on your device, would you?

Of course it is not the whole technology stack, but it is something at least. If your evaluation leads to potential problems, you can create issues right there on the github project!

The problem with this feature is that we cannot verify that Apple’s implementation of the math is correct and without security flaws. Everyone knows there is security flaws in all software, and this implementation is not open (I.e. we cannot review the code, and even if we could review code we cannot verify that the provided code was the code used in the iOS build). So, we have to trust Apple did not make any mistakes in their implementation.

I'm asking: what does an opt in for that really look like? You're not going to be able to give the user enough info to make an educated decision. There's ton of risk of "privacy washing" ("we use DP" but at very poor epsilon, or "we use E2E encryption" with side channel data gathering).

There's no easy answer. "ask the user", when the question requires a phd level understanding of stats to evaluate the risk isn't a great answer. But I don't have another one.

However, "my data is being sent off my device" is incorrect, as GP explained. Metadata, derived from your data, with noise added to make it irreversible, is being sent off your device. It's the equivalent of sending an MD5 of your password somewhere; you may still object, but it is not factually correct to say your password was transmitted.

Owning a phone is a privacy failure by default in the United States.

Maybe they should lump its default state into something that already exists? E.g. assume that if you already have location access enabled for Photos (it does ask!), you've already indicated that you're okay with something about this identifying being sent to Apple whenever you take a picture.

My understanding is that Location Services will, among other things, send a hash of local WiFi network SSIDs and signal strengths to a database Apple maintains, and use that to triangulate a possible position for you. This seems loosely analogous to what's going on here with the compute-a-vector thing.

This is not how you launch a privacy-preserving product if your intentions are good, this is how you slip something under the radar while everyone is distracted.

Or is it better to just trust that mathematics works and thus encryption is a viable way to preserve privacy and skip the dialog?

This is a much greater level of security than what you would expect from a bank, for example, who needs to fully decrypt the data you send it. When using your banking apps over HTTPS (TLS), you are trusting the CA infrastructure, you are trusting all sorts of things. You have fewer points of failure when a key for homomorphic encryption resides only on your device.

"Opting-in by default" is therefore not unsafe.

What is happening, is that people make tradeoffs, and decide to what degree they trust who and what they interact with. Plenty of people might just 'go with the flow', but putting what Apple did here in the same bucket as what for example Microsoft or Google does is a gross misrepresentation. Present it all as equals just kills the discussion, and doesn't inform anyone to a better degree.

When you want to take part in an interconnected network, you cannot do that on your own, and you will have to trust other parties to some degree. This includes things that might 'feel' like you can judge them (like your browser used to access HN right here), but you actually can't unless you understand the entire codebase of your OS and Browser, all the firmware on the I/O paths, and the silicon it all runs on. So you make a choice, which as you are reading this, is apparently that you trust this entire chain enough to take part in it.

It would be reasonable to make this optional (as in, opt-in), but the problem is that you end up asking a user for a ton of "do you want this" questions, almost every upgrade and install cycle, which is not what they want (we have had this since Mavericks and Vista, people were not happy). So if you can engineer a feature to be as privacy-centric yet automated as possible, it's a win for everyone.

That’s starting to veer into unreasonable levels of conspiracy theory. There’s nothing to “cover up”, the feature has an off switch right in the Settings and a public document explaining how it works. It should not be on by default but that’s not a reason to immediately assume bad faith. Even the author of the article is concerned more about bugs than intentions.

This is a dumb take. They literally own the entire stack Photos runs on.

If they really wanted to do a coverup we would never know about it.

If someone wanted to get a hold of your cloud hosted data at that point, they would use their capacity to simply extract enough key material to impersonate a Secure Enclave. That that point, you "are" the device and as such you "are" the user. No need to make it more complicated than that.

In theory, Apple and other manufacturers would already use PQC to prevent such scenarios. Then again, QC has been "coming soon" for so long, it's doubtful that any information that is currently protected by encryption will still be valuable by the time it can be cracked. Most real-world process implementations don't rely on some "infinite insurance", but assume it will be breached at some point and just try to make it difficult or costly enough to run out the clock on confidentiality, which is all that really matters. Nothing that exists really needs to be confidential forever. Things either get lost/destroyed or become irrelevant.

Not really, it's been around for a bit now. From 2021:

> The other major reason we’re talking about HE and FL now is who is using them. According to a recent repository of PETs, there are 19 publicly announced pilots, products, and proofs of concept for homomorphic encryption and federated analytics (another term for federated learning) combined. That doesn’t seem like a lot … but the companies offering them include Apple,7 Google, Microsoft, Nvidia, IBM, and the National Health Service in the United Kingdom, and users and investors include DARPA, Intel, Oracle, Mastercard, and Scotiabank. Also, the industries involved in these early projects are among the largest. Use cases are led by health and social care and finance, with their use in digital and crime and justice also nontrivial (figure 1).

https://www2.deloitte.com/us/en/insights/industry/technology...

I do wonder why we don't hear about it more often though. "Homomorphic encryption" as a buzzword has a lot of headline potential, so I'm surprised companies don't brag about it more.

Closest I come to presenting an opinion on the right way UX was "I'm not sure what the right call is here.". The thing I disagreed with was a technical statement "the only way to guarantee computing privacy is to not send data off the device.".

Privacy respecting design and tech is a passion of mine. I'm pointing out "user choice" gets hard as the techniques used for privacy exceed the understanding of users. Users can intuitively understand "send my location to Google [once/always]" without understanding GPS satellites. User's can't understand the difference between "send my photo" and "send homomorphicly encrypted locally differentially private vector of e=0.8" and "send differentially private vector of e=50". Your prompt "send data about your photos..." would allow for much less private designs than this. If we want to move beyond "ask the user then do it", we need to get into the nitty gritty details here. I'd love to see more tech like this in consumer products, where it's private when used, even when opted-in.

The Chinese government are debatably inhuman. They've literally censored the word "censorship." (Then they censored what people used euphemistically for censorship--"harmonious.") It's funny from the outside but also a miserable state of affairs in 2024.

Why in the world am I supposed to be an expert on homomorphic encryption? How many people in the world are experts on homomorphic encryption?

> which just reeks of survivorship bias.

What does that even mean in this context?

> 1: Feature value

What is the value of the feature? As the article notes, this new feature is flying so low under the radar that Apple hasn't bothered to advertise it, and the Apple media haven't bothered to mention it either. You have to wonder how many people even wanted it.

> If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled.

That's ridiculous. Apple itself has said, "Lockdown Mode is an optional, extreme protection that’s designed for the very few individuals who, because of who they are or what they do, might be personally targeted by some of the most sophisticated digital threats. Most people are never targeted by attacks of this nature." https://support.apple.com/105120

Lockdown mode is basically for famous people and nobody else.

https://en.m.wikipedia.org/wiki/Boiling_frog

The CSAM filtering was a best of class implementation. I'm pretty sure I'm one of maybe a dozen people who actually read the spec before throwing a hissy-fit about "muh privacy!"

The only actual "flaw" was that maybe a state-level actor could make it scan for bad stuff on your device.

BUT they can do it to your cloud data _today_. And if you disabled cloud uploads, the local scanning was disabled too.

Let's look at the primary alternative, Android. It generally does not provide you this level of granular control on network access either without rooted hacks. Apps and the phone vendor can do whatever they want with far less user control unless you're a deep Android nerd and know how to install root-level restriction software.

Restating the layers above, in reverse:

- They don't see either your data or the results of the query (it's fully encrypted even from them where they compute the query -- this is what homomorphic encryption means)

- Even if they broke the encryption and had your query data / the query result, they don't know who "you" are (the relay part)

- Even if they had your query hash and your identity, they couldn't reverse the hash to identify which specific photos you have in your library (the client-side vectorization + differential privacy part), though by the this point they could know what records in the places database were hits. So they could know that you took a photo of a landmark, but only if the encryption and relay were both broken.

https://en.wikipedia.org/wiki/Homomorphic_encryption

> Homomorphic encryption is a form of encryption that allows computations to be performed on encrypted data without first having to decrypt it. The resulting computations are left in an encrypted form which, when decrypted, result in an output that is identical to that produced had the operations been performed on the unencrypted data. Homomorphic encryption can be used for privacy-preserving outsourced storage and computation. This allows data to be encrypted and outsourced to commercial cloud environments for processing, all while encrypted

And the way the data comes back to you is via the third-party relay which knows your IP but nothing else

> Then somehow magically, the pictures on your phone are updated with tags based on Apple's processing....but Apple doesn't know who you are.....

Yes, this is the whole point.

That's kind of like saying "they can take a picture of your face and transmit it, but what if someone goofs an operation and sends your actual head instead".

Any encrypted data the server has cannot 'accidentally' be decrypted, and as someone else explained in this thread they only send encrypted vectors describing features of the image (building, tree, etc) and not the actual image. It's certainly not a fact that "they are looking at your pictures" [1].

[1] "they" being Apple; the other guys could have backdoored the whole OS and Apple employee's computers for all I know

How many landmarks are there to justify sending your data off? Can't the database be stored on the device?