Apple needs to explain that bug that resurfaced deleted photos
theverge.com
theverge.com
From a company that claim themselves as defender of Privacy is a Fundamental Human Right (when they see fit, i.e let's ignore China).
This is from Victoria Song, but given The Verge still published it I am going to give them some credit as well. Strange as Apple's PR machine doesn't seems to be working. But you can also pick up all the other media who had Apple's PR influence.
And for those who still dont know, you should read The Submarine from Paul Graham
Is there a source for the claim?
A post mortem would be fun for me as software developer, but is ultimately of limited value to me or lay people because the cause is already communicated.
I understand that the subject of the issue, it being photos, might be a bit touchy, but the mechanics and the nature of the bug is nothing special and anyone who moved past their “Hello, world” phase will immediately understand the kind of bug that was in play here.
Photos are stored “in” a photo library.
On Apple systems the photo library is just a package (i.e., essentially a folder) and in it is an SQL database that keeps track of photos and their attributes based on a GUID in some 70 odd tables.
The photos and videos themselves are stored in folders within that package (i.e., the file system). But it is ultimately the SQL database that is deemed authoritative and that decides what you see in the Photos app.
Different daemons and chron jobs use the database to sync photos to and from the cloud and to clean up photos when marked for deletion.
All it takes for this to occur is for a photo to be marked as deleted, without it actually being deleted in the underlying folder, for it to seem deleted.
And all it takes for it to show back up is for 17.5 to index through the folders and based on what found “repair” the database.
The database also gets changed from time to time, so it could also simply be a new way of keeping track of deleted photos and in the process of migrating to the new database version taking a conservative approach and assuming that photos that are still present to be wrongly marked as deleted.
It’s always better to restore and let the user decide than to make destructive assumptions.
After this process resurfaces photos then they get synced with iCloud, just like any other photo.
The implication that this doesn’t explain the resurfacing of old photos from years ago and many devices ago is rather weird.
Most people don’t start fresh when they get a new Apple device and instead transfer data over or restore form a backup, putting in place the corrupted database. To say nothing of the database file being synchronized across devices via iCloud.
In fact, that only makes the corrupted database explanation more likely.
In the earlier days of iOS, Apple was still finding its way on how to effectively manage the library, making some significant overhauls in addition to overhauls to switch from Photo Stream to iCloud Photo Library.
So it’s not unlikely it was during that period this issue snuck into the database.
Like I said, it’s unfortunate that it affects photos, but otherwise not a shocking bug by any means and the solution to include orphaned photos back into the library as opposed to destructively deleting them is good practice.
The only thing that might’ve been better is if the user was provided with a prompt informing them of the find and perhaps asking them to make a choice.
It’s clear however that they didn’t think it would be an issue that would affect many users and using scary technical words like “corrupted” go against the kind of language and UX Apple tries to stick to.
A miscalculation perhaps, but hardly worthy of the drama that it’s being milked for.
With the messaging based hacks exploiting images encoders, pdfs etc it seems plausible to me that they added some protections that could have made file I/O more sandboxed and accidentally over-sandboxed it in some scenarios and the error codes weren't being checked.
https://undelete.pullpush.io/r/ios/comments/1cspwh2/my_old_p...
That wouldn't fit the scenario you described.
Does anyone have a simple, cross-platform solution for storing (yet easily accessing) private photos and videos? Or do most people just resort to living dangerously?
Anything else is "living dangerously".
Most people do NOT need security from state level threat actors, they just need software that respects them, which of course no longer includes mainstream operating systems or applications, today.
Apple Releases iOS 17.5.1 with Fix for Reappearing Photos Bug
https://news.ycombinator.com/item?id=40426557
Troubling iOS 17.5 Bug Reportedly Resurfacing Old Deleted Photos
https://news.ycombinator.com/item?id=40366091
iOS 17.5 is allegedly resurfacing pictures that were deleted years ago
EDIT: I appreciate your attempt to clean up your comment in response to feedback, but your top link currently is the same as your bottom link, and even if it were correct it still only has either 2 or 1 comment depending on which of the two submissions by that title you meant to link to. That is well within the range of non-uptake that deserves a second chance.
it 'fell off' because other things replaced it. Natural. Discussion still ongoing here. More discussion last week on a number of other threads, when it was news.
I won't respond further because I don't want this turning into a flame war.
Some of that is because low-quality discussions fall of the front page quickly, but most of it is because the first comments on a post are always low quality, because they're disproportionately from drive-by posters who read the title but not the article or the existing comments. If a post falls off the front page quickly, then those comments become the only ones that get seen, which means even if someone comes along with a more thoughtful comment (unlikely) it ends up buried with no upvotes and no replies.
If you let a "dupe" that's finally catching on stay on the front page for a while, the conversation invariably becomes better as the thoughtful commenters arrive and begin to leave their mark on the conversation.
In other words: just because 5 threads are ongoing and uninteresting doesn't mean that this one didn't stand a good chance of having a good, useful conversation if left alone.
I agree with @lolinder that you are being overzealous about marking dupes. I understand the utility of preventing dupes but your behaviour is coming across as extremely bad faith here especially considering your top link is same as your last link and is pointing to a 5 day old post.
@dang Please look.
I think the POV chosen, one way or the other, reveals an interesting bias, like that white/gold/blue dress. But I find this even more interesting because probably most of us here have worked at software companies as developers, and know personally how utterly normal unintentional bugs like a failed deletion bug would be, particularly if black-box QA is oblivious to it because everything seems fine. No company is immune, and Apple has had some pretty embarrassing bugs over the years.
The clue is in this sentence. The privacy is marketing.
Seeking an explanation from them reeks of coping desparation, but not unexpected if you have bought heavily into it. I'd say this could be a lesson for many, but looking at similar past incidents with them, I am not very hopeful.
Maybe back in the day. However, for the last few years, Apple is in the business of personalized ads in App Store, Apple News, and Stocks etc. So they do "extract value from your personal information" just like other companies such as Facebook, Google etc.
Apple shows what they do with your data even at a per app level: https://www.apple.com/privacy/labels/ as well as reviewing Apple's Privacy Policy vs Meta/Google and see they are night and day from those companies. Again this doesn't make Apple better/good, and others worse/bad, as that depends on who one is and what they value. For instance Google offers far superior services in many aspects than Apple because they take advantage of all that customer data in ways Apple chooses not to.
> Maybe back in the day Assuming you may not be an Apple customer or familiar with all the constant flow of new features in the current day Apple has been releasing focusing on Privacy. https://www.apple.com/privacy/
One person wiped their device as per Apple's instructions, sold it, and the new buyer had the seller's old photos appear on the device!
https://undelete.pullpush.io/r/ios/comments/1cspwh2/my_old_p...
Wiping it should have deleted the encryption keys off that device and destroyed all data, no?
A single handwavy comment about "database corruption" is not enough for such a gigantic issue.
Also, taking something very seriously does not mean there can't be critical bugs.
Database corruption can't explain how the photos can return to an iPad after wiping, if the encryption key has been changed as their documentation claims.
Unless the photos are actually linked to the device and not the actual account, which would be stupid and not how they described it works... or if it isn't encrypted at all - also not how it was marketed/described to work.
A pathetic one-line non-explanation for such a giant bug, that doesn't even align with the facts of what actually happened, is not congruent with the actions you'd expect from a company that cares about privacy.
They need to make a statement explaining how that could happen, and what they've done to prevent it happening again.
I disagree. External statements are guided by legal and comms departments (along with Engineering of course, but eng never has final say on what goes out).
External statements don't necessarily have any relation to how things are being handled internally. And this fuck-up (assuming it's as described) also doesn't reflect whether the eng team made a genuine best effort to ensure full privacy. Teams make mistakes, even huge ones. From personal experience, the biggest bugs come from the smallest root cause -- that single typo in a single file that somehow snaked a narrow path through every test on the production pipeline.
Now, is Apple working hard to fix this? I have no idea. All I know (and my original assertion) is that when I worked there not long ago, privacy was genuinely a Very Big Deal that was a top (often the top) company-wide north star.
Got it.
Luckily, I am not under an NDA, and I can tell you that the Reddit post is nonsense. A straight-up lie when you assume bad faith or poor recollection if you assume good faith.
The scenario described there, and further expanded upon by OP in comments is pretty much impossible. I hedge only because of an astronomical unlikely probability that everything in the universe aligned perfectly.
As you seem to be aware, encryption keys are involved, and that involvement lies at the root of the impossibility.
Say you’re inclined to believe that the Secure Enclave that stores this key has a massive bug that doesn’t delete the key upon wiping. That alone wouldn’t explain a scenario like that.
In addition to not deleting that key, the OS must’ve been unable to detect and try to use that key until some serious potent code was introduced in 17.5.
Also, during the wipe, the encrypted data partition that goes with the key must’ve not been deleted and gone unnoticed by the OS up until 17.5.
In addition, the OS must’ve kept the key intact, and ignored the existence of the encrypted data partition. Creating a new encrypted data partition with an accompanying key and acting as if it was all business as usual.
Then, suddenly, 17.5 comes around. It would have to have seen two encrypted data partitions with two encryption keys, mounted the most recent encrypted data partition, and decrypted it with the most recent encryption key without any issues and hiccups, only to then do something quite miraculous.
It would, at that point, do something that it was never designed to do, namely decrypt and mount the old data partition, all while the most recent one is already mounted, grab only a bunch of old photos from a corrupted database, nothing to else, and merge it into the database located on the most recent data partition.
All this while ignoring many complexities related to key pairs tied to iCloud accounts that I’ve omitted for simplicity’s sake and without throwing up a single error, much less a respring or, more likely, a kernel panic.
Just the part about mounting two partitions alone would cause huge issues.
It’s nearly impossible to do this on purpose due to hardware limitations on storage and the way the Secure Enclave works. To entertain a string of bugs that would execute this perfectly is just silly.
Who needs jailbreakers and the likes of Pegasus spending hours designing chain exploits when the OS stumbles into perfectly executed bugs that defy the law of physics?
Just seems odd to me that he would make that whole story up.
I know it's the internet but there doesn't seem to be a compelling motivation for someone to do that.
I think it's reasonable to expect a more detailed explanation from Apple, when it's closed source software from a company that claims to value privacy, yet exhibiting a confidence-eroding problem like this.
You have a lot of work to do convincing anyone that's the case: https://arstechnica.com/tech-policy/2023/12/apple-admits-to-...
And even if it is challenging, it shouldn't even need to be said that soft deleting photos of all things would be completely and totally irresponsible. This isn't some metadata in a business app, some of those deleted photos are extremely sensitive.
> Sure, you could push your glasses up your nose and say, “Well actually, no file is ever really deleted until it’s overwritten...” And while that is true, a reasonable customer would expect that when Apple says a deleted file is permanently deleted, this sort of thing shouldn’t even be possible.
You could stereotype it as well-actuallying, sure, but the reality is that "deleted files aren't actually deleted" has been the way computer storage has worked and still works, in the majority of cases, for a very, very long time. I don't disagree that a reasonable customer would expect that when they delete a file on their hard drive, it's gone forever. But if I were a writer for a tech publication, I don't think I could credibly write an article indignantly demanding an explanation for an undelete utility.
I work for a large internet/tech company, and all images uploaded by users must be permanently retained somewhere per our legal policies, even if that means sending it to long-term storage after it’s been “deleted”. And if law enforcement submits a request asking for your files, we can pull it back up.
If the US or China tells Apple to give them the contents of your phone now or ever, they damn well likely can and will. Same as Google or as Microsoft would.