iCloud Drive silently deletes your content
dominikmayer.com
dominikmayer.com
- My wife had cloud sync on her iPhone for Notes
- I thought: "we probably don't need that" so turned it off for her
- She turns on her phone and goes "Where are all of my notes?? What did you do?"
- I break out in a cold sweat
- I remember we had an iPad that also had the sync'ed Notes app
- I open that up and immediately turn off WiFi to "save" the Notes
- We call Apple
- The support person ended up figuring out that we could essentially copy and paste each Note to a new folder so they wouldn't be wiped out.
It was a stressful couple hours though while we worked through it. I would also say that Apple could have given some more warning/pop ups of the full impact of this.
PS I also learned that Apple techs can essentially "VNC" into your phone and view things, change settings etc. They ask for permission and I think I may have had to hit "approve" but this is a feature I didn't even think about it till I saw it in action.
You get a message like "all notes will be removed from this device", but without any kind of reassurance that "all 172 are still present in iCloud".
And if you're not an expert in this, it's a huge risk to take. When does deleting something synced locally leave it in the cloud, and when does deleting something synced locally also delete it from the cloud? This is one of the biggest and most dangerous UX confusions that exists right now, not specifially with Apple but all around.
I'd love to be able to do that too when troubleshooting my mom's iPhone over a voice call (or trying to).
I really want this for my parents who live two states away.
I use this all the time for tech support (for family and work).
I'm sure you got an asswhooping. haha
Phones are individual/personal.
Even if the iCloud account is on a shared plan, and one member of the family is responsible for managing it, they should not be making changes to any INDIVIDUAL device settings without checking with that device owner.
I would be livid if my partner had done that for me because they "thought 'we' probably didn't need it, so I turned it off for 'your' iPhone"
She didn’t say anything.
But I’m not doing that again.
But OP's sounded like an optimization decision and would require some process retrospective.
> there's no point getting mad
>> But I’m not doing that again
> consistently demonstrate
So, for example, by not saying anything instead of getting mad. And by leaving the other person’s devices to their own management.
https://support.apple.com/en-us/HT205644
supposedly however switch control works to control other devices on the same wifi network. perhaps there's a vpn trick in the support side to leverage it over the internet at large.
It's similar to FaceTime screen sharing, the support employee can only see a video feed of your screen but can't directly interact with it.
I feel as if your wife’s Problem was not Apple but the person administering her iOS devices.
I don’t think that’s unreasonable.
If I have paid for a service good off boarding is part of the service I am paying for.
Are you serious? "No off boarding" is Apple's entire business model.
Am I the only one seeing this as a potential security vulnerability? Is there a way to permanently disable this?
Of course, but such a powerful feature could also be exploited by hackers, no?
People here constantly flip their shit because of Intel ME exists which could be exploited for similar access, but seem perfectly OK with this feature form Apple.
My employer pays (a lot) for Google products and even then there's no way they would ever help with that, let alone on the phone.
In the past I did get a rep on the phone to help with things like access control but I can already hear her tell me (politely) to get lost if I needed help with Keep notes disappearing (which they occasionally do!).
Workspace support is good in my experience, we interact every 2-3 weeks and they can usually resolve our issues or know when to escalate. You can call or chat. I know that you can also get phone support as a consumer now via Google One, but I haven't had to use it yet personally.
My girlfriend lost an AirPod a couple of months ago, so we went to an Apple Store for replacement. Turns out the case was a Gen 1 case and the AirPod itself was a Gen 2. They had given her a gen 1 AirPod and it wouldn’t sync with the other one.
We went back and the geniuses explained there was nothing they could do. The day after, I called Apple support and the guy almost started yelling at me, how it was our fault for mixing up the cases etc. Eventually he hung up on me.
I called them back two hours later, explained again what had happened (including the previous phone call). She told me she would call the store and apologized for her colleague.
In the end we ended up getting a refund for the wrong AirPod(but got to keep it) and had to buy new ones.
So Apple support, not always great.
The same thing happens to me repeatedly when I'm highlighting PDF's using Books -- every so often, I go back to a PDF and discover multiple pages of my most recent highlights are missing. It saves highlights as you go along, but sometimes for whatever mystery reason, it decides that the version of your PDF from 11:35 is the newest main one, not the one from 11:45.
These problems, of Apple silently overwriting newer content with an older version, have been going on for at least a decade at this point.
And I just don't understand how such a fatal flaw was ever built into the system in the first place, and how it still hasn't been fixed.
Because the two hardest problems in software are cache invalidation, naming things, off by one errors, and buffer overflows from a lack of array index enforcement.
I'm not going to say its all bad, but it is less and less my style (the older I get, especially).
Confirms there is a known problem and could provide an option to terminate it
Wrong clock on some of the servers?
(And I long ago turned off iCloud for Books hoping it would fix the problem, but it didn't. Which makes me suspect it's some kind of "local mode" for iCloud that's still open to these bugs.)
Anyone tired of Apple's contrast-killing orangey background and missing a true yellow color for their highlights should give it a try. (No affiliation, just glad it exists.)
My problem is, that I need to use iCloud Drive for some important files because the app I need to use doesn't support another way to sync.
Without git I wouldn't have figured out which files have changed. (And even with git, I can't be sure that I caught everything. I guess git content can also be altered by iCloud sync conflicts…)
Or maybe even locally if you don't need to commit on other computers, then you'll be certain that git is correct without any cloud intervention
My problem is, that for the time being, some of my files needs to be on iCloud Drive because I rely on one app to edit/create them. Everything else I moved out of iCloud Drive.
I most definitely used Git with binary data without any issues. It makes no assumptions as far as I can tell. It's just not great with very large binaries.
I think you cannot make a completely generic indifferent system in finite time. Maybe your system is optimized to efficiently work with 5 1TB files, or 1 million 5KB files, or just nicely handles a reasonable amount of text files in a neat way while being not extremely complex or hard to maintain & deploy.
If Git does version control better than other control systems but only does it for text then I might be fine with that, I think it's reasonable to have a scope.
I think the golden rule here is to only work directly with the repo on a single device. Otherwise there is a lot of useless churn and more potential for conflicts.
If you want to work with it on multiple devices, you can still sync it around, just work with a new clone of that repo rather than modifying it directly from multiple places. Then push your changes into the synced repo and act like it's really remote instead of local.
This does use more storage, however.
I do have to say, I've never understood why git doesn't work well with cloud drives. File management seems like a straightforward enough set of operations that ought to be bulletproof. I've never understood what precise operation gets corrupted when using git, and how that's possible at all.
And I don't know if the fault is the cloud or git. Is it that the cloud returns errors 0.5% of the time, and git silently ignores those instead of retrying, and corrupts data? Or is it something about git reading and writing so many files so quickly that operations on the cloud drive somehow get lost or out of order but without generating errors? Both seem equally implausible, and yet...
Don't use non-e2ee services.
(Yes, there is an opt-in e2ee for iCloud now. No, nobody is using it because it is off by default and buried in settings.)
I use syncthing (e2ee, free software) to keep my files on all my machines. It has sane conflict handling and nice versioning backups. It even supports untrusted sync-only devices that never see plaintext so you can run a node on a vps or somewhere offsite safely.
iCloud Drive is like, one of the only providers that has e2ee.
Settings > AppleID > iCloud > Advanced Data Protection
You'll see 'Add Recovery Contact' and underneath it you'll see 'Recovery Key' 'Off>'
Tap into it, agree to the notice, you'll get a 28 alphanumeric key and then re-enter that same key to finish the set-up.
Safe and secure providers will never touch the plaintext of any user ever, regardless of the user's configuration (or lack of it).
An example (although not in the e2ee sense) is that time Dropbox allowed logins to any account with any password provided. It illustrates that having a basic test suite over authentication systems wasn't important to them.
While I'd love to turn on e2ee, I don't trust myself enough to not lose the encryption key.
I think it’s less about protecting yourself against targeted attack, and more as a defence gains snooping and physhing and some gathering dirt.
If you're using a password manager, you can put the key in there. In effect this backs up the key (encrypted) to every device you install the password manager on, plus a cloud backup depending on what manager you're using.
You can barely convince the average user to set up their own backups, and of the users that do, how many do you think test their backup recovery procedures?
Again, it comes down to your threat model. Key stored in more locations is potentially compromising security, but is a trade-off everyone has to consider. It still prevents disinterested drag-net style data leakage.
If you are targeted by the big bad, I doubt my operational security would be sufficient to protect me from a motivated attacker. I suspect that there are vulnerabilities in my TPM/random number generators/encryption software that are exploitable. An evil maid or even just a camera positioned over my keyboard would probably be sufficient to grab my data. Plus those $5 wrenches.
” If you enable Advanced Data Protection and then lose access to your account, Apple will not have the encryption keys to help you recover it — you’ll need to use your device passcode or password, a recovery contact, or a personal recovery key. Because the majority of your iCloud data will be protected by end-to-end encryption, you’ll be guided to set up at least one recovery contact or recovery key before you turn on Advanced Data Protection. ”
I’m using it. And I’m perfectly okay with this being the default behaviour.
Much easier to handle version conflicts if the file isn’t a black box.
>(Yes, there is an opt-in e2ee for iCloud now. No, nobody is using it because it is off by default and buried in settings.)
These are contradictory statements. It IS e2ee. I can easily convince friends and family to click through two menus to enable it. I can't easily convince my friends and family to go through the effort of installing syncthing on multiple devices and then go through with the configuration of them.
It's great you're a syncthing advocate, but telling people that icloud drive is bad just because they don't enable a setting that could cause an average user to lose all of their data if they lose the key is a little ridiculous. I GUARANTEE YOU that your grandmother would MUCH rather be able to call Apple to get her pictures of little johnny back when she forgets her password than have it "more secure by default".
That would be the same regardless of whether you were using icloud or syncthing...
The commenter hasn't actually linked any sort of proof or citation (at the time I'm writing this), and assuming it exists at all, it requires you to accept the inbound connection in the first place. My guess is they're extremely confused about the new switch control function that allows remote access, not some nefarious Apple backdoor like you're insinuating.
It's all too simple to click "OK" in an inattentive moment.
Well, not really…:
” Encryption of certain metadata and usage information: Some metadata and usage information stored in iCloud remains under standard data protection, even when Advanced Data Protection is enabled. For example, dates and times when a file or object was modified are used to sort your information, and checksums of file and photo data are used to help Apple de-duplicate and optimize your iCloud and device storage — all without having access to the files and photos themselves. Representative examples are provided in the table below. ……”
https://support.apple.com/en-us/HT202303
For example, file names in iCloud are not e2e, etc. etc.
This is a huge privacy issue.
Moral of the story, don’t store anything in iCloud Drive you like, especially not your books.
I would like to argue this is at least partly the fault of app developers, for adding iCloud support without the conflict handling mechanism.
However, why does Xcode, Apple's own in-house app, not support conflict handling?! Apple's own software should be setting the standard for third-party developers. If they don't handle version conflicts, of course no one else will either.
This is an iCloud bug, not an Xcode one.
Obviously, deleting data is bad.
However, Dropbox's "(John Doe’s conflicted copy)" solution is also pretty awful UX. I can understand why Apple would want to use its uniquely integrated software ecosystem to come up with a better system. I believe they could do it—but they clearly have not.
A different approach would be to add a way to see files that have issues. Similar to how Dropbox shows files that fail to upload.
For Dropbox, I agree. Apple could do it though; that's the advantage of having such great control over your ecosystem.
Just assume you write something, save, close the editor and never open it again. iCloud might replace the content of that file and there is no way for you to know.
The only way to prevent losing your data is setting up a Git repository or open _every important file_ regularly in an editor that supports conflict handling.
So if you want your files to be synced and to be 100% sure you'll have a local copy - you should keep the same file in iCloud Drive and in the "On My iPhone" folder. This is the reason Apple Books, Numbers and all the apps that use iCloud Drive remove local copies all the time.
I made a simple note-taking app using iCloud Drive - iCloud was greedily removing a 2 kb txt-file when the app was closed (sometimes it wasn't removing - it's quite random). It doesn't matter if you have a lot of storage space - iCloud still can remove the local version of a file you constantly use.
The Apple developer support said nothing can be done. Even for a developer there is no way to mark a file as never to be removed.
P.S. In their official docs Apple actually recommends to keep all the user's files in iCloud Drive and _not_ to keep a copy "On My iPhone"
However on a morning I realized the contact of a friend I was texting with the night before was gone. I realized the problem was iCloud.
I reach out to Apple support, and after exhausting all the dummy options, the customer support representative admitted that this was a known issue, but they could not find the root cause. And there was no solution to it.
The asked me to monitor my contacts so that I could find when this deletion happen but I would need to give a few minutes range. Because “an iPhone generates a lot of logs and engineers cannot go through them unless we can pinpoint where it exactly happened”.
I thought that pinpointed the bug within a 8 hours range was already good, but the customer’s representative insisted on being accurate within minutes (saying the engineers said that).
Anyway, as all the hard case that I give to Apple Support, the representative said he would follow-up and call me back but never did. So nothing happened.
I move my contacts to Google and just forgot about it.
My break down is here.
As an aside about not bothering users, iOS silently changed my call settings on some update they pushed on me so the phone wouldn't ring for some callers. Some sort of calming nonsense but I missed a call from my cancer surgeon who I couldn't call back because NHS which freaked me out.
Closing all apps on mac and iOS, and running "killall bird" (bird is ithe icloud sync service) sometimes fixes it.
I would assume that syncing would get bogged down (re-syncs full blobs rather than incremental bytes) rather than expect any kind of data integrity problem.
File management on iOS is a mess too. It’s a shame as there’s so much good stuff in the Apple ecosystem but their cloud file storage is not something I want to get involved with any more than I need to!
"Hanging" with no progress bars, no warning, no errors is the Apple way of doing things still after 15 years.
I wonder how hard it would be to implement a dev-mode where it actually told you why it hung, or what the hell it's "doing" in magic (black box garbage) mode.
You can have a filesystem with intuitive semantics, really flexible and great access control, resilient and dependable, fast and efficient, but in order to share this filesystem across the network, with other OSes by different vendors, the lowest common denominator is observed. And so all those features are stripped away, because only the basics are supported when the OS vendors are competing with each other and there are different machines with different filesystem drivers, different file managers, and apps with different libraries and frameworks for accessing files. There is no way to maintain consistency, even with standards-based formats and protocols, when so many disparate vendors and developers are involved.
Wait, what's that you say? iCloud is maintained by a single vendor? You are telling me that Apple has control of every iCloud client and server?
Hmm, bad bit of luck there.
I recently switched my Obsidian notes off of iCloud. I’m using their hosted sync service, and expected to be better – but don’t have enough experience with it yet to be able to recommend it.
(It seems that certain apps -- the type that I never use -- support file versioning within the app. That's not useful.)
(Aside: I really gotta backup my iCloud Photo Library…)
Dropbox is so heavy and uses system resources, I’d love a lightweight one that I can trust
If you want, there are tons of applications which can talk with Dropbox, with varying levels of capabilities.
Why is it so memory-intensive anyways?
Yes, it's borderline Electron levels of resource waste, but that thing works, and I can't complain.
While Apple can manage that 8GB of RAM relatively well, 16GB RAM is a must on any Mac which is used for serious work.
I think developers all using higher end hardware than the common base models is a big part of the reason bloat like this exists is a problem in the first place. You've got programmers with 128 GB of ram and 32 CPU cores writing application software for people with 8GB of ram. A few hundred megabytes is nothing to the developer, but substantial to the common user who is also trying to use the same machine for several other things simultaneously.
Most developers and companies want to minimize development time and sweat. Instead, they want to push their MVP out of the door and iterate quickly. This results in tons of imported libraries for single functions, unoptimized code, cruft and bloat, which I really don't like. However, even if all the things are optimized reasonably well, my point stands, because running things add up.
Currently, besides two browsers (Firefox and Safari), I have Microsoft Excel, Obsidian, Spotify, Terminal, Apple Mail, Dropbox, Nextcloud, TextExpander, Viscosity and Amphetamine open. Currently 13,43GB of RAM is used alongside ~700MB of swap. Most of this is consumed by my Firefox tabs and WindowServer on macOS.
I’m sure there are downsides but I like the benefit of passive image backup and periodic sync.
For code, use git locally. If I want to share, then publish to a remote VCS (gh, gl, personal gitea instance, ..)
Now, I’ll live with this hassle and Drive supports conflicted copies.
Dropbox has always worked the best. But I’m through with them thanks to their heavyhanded shakedown of me by reducing free storage from 5GB to 2 in an attempt to get me to pay. Alter cockers don’t pay: we can’t just go on LinkedIn to get a raise.
- at some point i saw the sheet (which has a date/time column with the current time, for each row i added) reverting back to a previous version (multiple added rows being deleted / disappearing.) I ended up just removing the file from iCloud and using it locally only.
- You also see the same issue in the Shortcuts app, it then at least shows you the 2 conflicting versions with timestamps and asks which one you would like to keep.
TextEdit.app works great on Mac for this, but there is no equivalent in iOS or in icloud.com.
It's an issue with Xcode/Swift Playground not supporting concurrent access to a shared project.
Any app on any platform that doesn't support this use case will exhibit similar behaviour.
iCloud most definitely adds to the problem.
Changing a shared resource in two apps at the same time will usually cause conflicts unless the app has been specifically designed to support this collaborative style use case.
I had backups, so the files are not actually gone, but my trust in iCloud Drive is.
But dominikmayer has a very good point there in the sibling comment. I think this would only affect files that change though.
The old saying is - it’s not backed up if it’s in less than two places.
Google Photos had better hooks previously, and I can't tell if new iOS limits that shaft NextCloud also affect Google Photos, since I don't use it for a while.
If you don't have enough local storage you need to be more mindful.
www.ibeni.net
‘If you don’t own it, you have no rights’ leads to a dark place.
Sure, I have 126GB 'in the cloud' apparently, but come on. The only fix I could see was to start fresh: so I set up a fresh Messages app on macOS, left it to sync overnight with the screen on until the "Downloading messages from iCloud banner" went away and made sure the important conversations showed "0 images in iCloud".
I then exported what I could via unofficial tools but I still feel like I've lost data, which is painful for me. Such as, searching my earliest known message via another backup showed it in the search results on macOS, but clicking on it showed nothing but a blank conversation. It wasn't even in Messages' SQLite DB after multiple "Sync Now"s and restarts. How the heck did it get that bad? Are the PMs at Apple just that misaligned?
I finally tried deleting the most problematic conversation, which went "ok" on the Mac at first, but caused the apps on my iPhone and iPad to do nothing but crash on open. When the Mac synced next, the conversation reappeared with our past messages, but with random gaps in the history (e.g. messages from yesterday, then a week ago, maybe a month before that, etc -- so a incomplete/weirdly ordered deletion). What a bloody mess. I ended up doing a "Disable & Delete" on iCloud Messages. Now I have to wait the the 30 day waiting period for Apple's systems to wipe hopefully wipe the data in that bucket if I feel like trying again. All this certainly does not give me confidence in using iMessage (even with Messages in iCloud turned off) in the future.
------
I'm just.... very very frustrated. How can they take the most precious of their users' data and give no fucks about it? The same thing with "shared photo albums" where it's not (or wasn't) immediately obvious that they re-compress all images and videos to much lower quality because I guess they didn't feel like deciding whose iCloud storage it would take up. So now my friend and I have photos and videos that are irrevocably, unless we go digging and find the originals we hopefully have elsewhere, compressed.
And now iCloud Drive as in the article? A team needs to be formed at Apple to have a long hard look at how they're handling their users' data and ensure it's done with the utmost of care. This should be a wakeup call to other companies as well. Users are entrusting you with their live's precious moments, or even their worst of times that may need to or want to be recalled upon later and you should be a good steward of that data. In the case of Apple, at least provide a reliable way to backup or export data such as Messages in iCloud, then perhaps I'd have a bit more than my many wasted hours fighting with your software and hoping that I have everything.
"Move fast and break things" really was a terrible mantra.
Now consider that courts will use this evidence in case of a crime being suspected. And they will presume Apple’s backups to be more reliable source of information that anything you can produce.
In Britain this happened, we sent over 500 people to prison for stealing money, but it turned out the IT system was bug ridden and didn’t sync transactions properly.
Are you saying that you saw other people's messages ?
Because this would be one of the worst security issues in history if it were the case.
I'm sad I had to delete the conversation, but this wasn't the first time I was sent an image or something and it.. just wouldn't be there on a device and there was no way to make it download. So it seemed something was corrupt, or misbehaving.
(As an aside while trying to sort everything out over the past day or so, a message from someone else was only on my iPad afterwards (it disappeared from my Mac and iPhone), and toggling sync did not cause it to propagate anywhere..)
People put way too much trust in these computers. It feels like we're learning the same lessons over and over again. If you had to put your trust in 1) your local disk, or 2) a cloud service, then local disk always wins. Common sense.
Granted, I’m using SyncThing between 3 different OS platforms.
It's besides the point: it's certainly not indicative of whether or not it's common sense that iCloud intends to sync documents.
I do pay for cloud storage to sync my reMarkable 2, mostly because they'll void my warranty if I don't.
Those people may learn the hard way that deleting documents on one device syncs that deletion to other devices in the same way that making changes to a document on one device syncs the changes to other devices.
Syncthing is also a good option but people have mixed experiences with that.
You don't actively collaborate on anything with anyone?
You've never worked at an org where the network shares are snapshotted and backed up, but desktops aren't?
Let's meet halfway: use rclone sync to get the cloud up to date with your local data.
And, when out of alpha, you can also use rclone bisync.
For my off site backup I have a computer which syncs everything to local storage. I guess I don't defend against malicious intent or "delete it all" software bugs but that tends to end up as en exercise in insanity.
For example, every single line of code is simply a scratchpad if I don't sync it to a remote git instance.
Your point remains, but for cloud sync, though.
Buy/build a NAS, fill it with SSD's, run raid10, run fast networking, and then back the NAS up to the cloud, use the NAS for hot data. That way, when a drive fails, you swap in a new one, and forget about it, no downtime, no BS.
If your choice is between cloud and local disk, you've already failed.
Not as much of an issue with HDD's, but, you also need many HDD's to achieve the real world performance of even a single SSD and that gets expensive in terms of drive cost and power usage.
I guess YMMV based on how fancy a setup you need.
Speaking second-hand here; my brother is an Apple devotee and a music buff. For many years, he collected recordings of live music shows. They were rare if not entirely transient (in that it would be impossible to reacquire them if lost), and he had a good library of them. Stored on his iMac. Then (as he tells the story) one day iTunes decided to replace all of his rare recordings with the equivalent studio songs. They were gone, and the support personnel he contacted told him they could never be recovered.
He still uses Apple computers, but I don't think that wound will ever fully heal.
Apple users have an abusive relationship with the brand. They forgive everything.
But not going back to Windows.
I’ve bought the best laptops money could get for some employees, System76. The fans are like whistles in the office: Impossible to work with them.
There’s nothing that works right with PCs.
Or he could just not have pressed Save.
Like so many, many people have done through the lifetime of computing.
a) Your brother should have taken backups of his MP3s as he was advised.
b) He pressed the button to replace the songs because there is no evidence, anywhere of this happening automatically. And common sense suggests this is the case because there would be a lot more uproar if 10's of millions of users songs were suddenly disappearing at once.
If you're an aspiring "influencer", producing "content", then go ahead and use bobblehead platforms designed deeply and pervasively to think of the user as a consumer of content, or a captive property of the developer.
If you're a hacker or a software engineer, you probably want to be investing in platforms intended for people like yourself. Which, right now, pretty much means Linux or a BSD.