From Cyberdog to dataless files: a brief history of iCloud

This got me when I was in the middle of nowhere abroad airgapped. The file I wanted wasn't there.

Switched to OneDrive and android now. Keep offline works.

I was astounded by how bad the books app on iOS was. I just want to open a book to read that I have a local copy of, but I need an internet connection to redownload it. If I close the app for a moment it needs to be downloaded again. If your on a rate limited connection it means you might have yo wait more than ten minuets to acres a book that’s right there in your downloads folder already.

> From what I remember, it was just barely functional

Cyberdog had instantaneous indexed search for local text, which combined really well with tech mailing lists that were really popular back in the day.

That didn't become a part of the feature set on Windows until Windows Vista shipped a decade later.

I remember using Cyberdog some in the late 90s, and what I saw most often is that it’d work fine for a while (as long as all the sites you visited were text and images with nothing fancy), and then start stalling when loading pages. Of course that might’ve just been the flakiness of the 28k dialup connection the house was on back then, but Netscape 3.x and 4.0.1 from the same era didn’t exhibit that behavior as often.

Cyberdog also included a handful of apps that were annoying to install on the Mac in 1996: contacts, e-mail, FTP, newsgroups, etc.

Dataless files are interesting! They are a special type of file with a bit set (and an xattr) in the file system and no actual file contents. They all live inside a “sync root” in APFS, where each sync root belongs to a specific file provider like Box, Dropbox, iCloud Drive etc. Whenever the kernel detects access to one of these files it will synchronously “upcall” into the current user context and XPC directly to fileproviderd which will in turn communicate with an installed file provider to download and provide the file (changing it from a dataless file into a normal file). This is called “materializing” the file. It will also emit an event using the FSEvents API.

fileproviderd (a launch agent on macOS and a daemon on iOS) watches the FSEvents stream and saves the events in its database. fileproviderd maintains its own sync engine for each provider on the system and will forward events to/from each provider.

Dataless files are an interesting example of Apple using its vertical integration to create an end to end solution that others can’t quite make. Since they own the filesystem, kernel, and user space daemons they can make something that works like this.

This is part of Apple’s years-long effort to kick 3rd parties out of the kernel. As apart of the M1 transition, kernel extensions were no longer aloud and file provider apps were using kernel extensions to intercept file system events at a low level before. Now that they are no longer able to do that, Apple has provided this replacement.

Yeah, I’m wondering that too. That was the hook I was interested in.

It also forgot to discuss Cyberdog, aside from mentioning its name. What is it? I kept rereading the first paragraphs thinking I’d missed it.