ExtensionKit | Hacker News Reader

ExtensionKit

developer.apple.com

141 pointsnateb202286 comments

Before ExtensionKit you could only make app extensions to extend the functionality of Apple provided systems. Things like authentication, file sync and badging in the Finder, Xcode text editors, or Final Cut Pro editing extensions. With ExtensionKit you can make your own extension points and let people develop extensions for your apps on all Apple platforms.

The code editor Chime just released extension support in their 2.0 release if you are on macOS 13. They have a lot of good details on how the implemented the feature on their blog. https://www.chimehq.com/blog/extensionkit-intro

Application extensions outside the Apple ecosystem have been used on Apple computers for decades. It was never limited to Apple provided systems, whatever that means.

This is incorrect. In the past apps like Photoshop have allowed plugins which differ from App Extensions by running inside the host app's process. App Extensions are run in a separate process and don't have access to the entitlements of the host application, and can't directly access its data. This tends to mean that extensions are much better behaved than plug-ins.

This. It is a different technical model that produces a different user experience. These are closer to widgets than to IPC.

So, basically, OS support for the kind of things that Chrome and Firefox have been doing for a while?

Having been (a minor) part of the Firefox effort, that was a real pain to implement, so sounds like a very useful component.

Applications have been using out-of-process sandboxes for years. But that's not necessarily a sign of quality, since this comes with a performance cost.

It's nice when extensions don't crash the host application, but the mitigations aren't free. It's interesting that Swift's stable ABI will drastically lower that cost, since you usually need some kind of serialization (although people have been using mmap to share data between host/extension process for awhile too to avoid the serialization penalty)

(author of that quoted blog post here). You are correct, there's nothing new about apps offering plugins/extension systems. What's new here is: this works across sandboxed apps, support for remote (out-of-process) views, and the extension permission and discovery system is entirely managed by Apple.

Like I mentioned in another comment most of this has been practice for awhile. The big one is discovery, but there have been techniques to fix that (like standard install locations). AU (v2, v3) is a good example.

ExtensionKit is for the new facility to extend the UI. Non-UI extensions only need ExtensionFoundation.

For example extensions on macOS, go to system preferences and search for extensions. They're listed by affording context (e.g., Finder, Photos Editing). Mine include pdf conversion, cropping photos, etc. You can create your own via Automator.

The docs imply that before Ventura (13.0), developers had to create a property list to declare NSExtension, but now you can do it in Swift. However, the TextTransformer sample has the usual melange of metadata, and there's no deployment information in the API documentation.

Ideally, a simple swift script would be sufficient :)

- https://github.com/insidegui/TextTransformer

To offer another take at a summary:

Apple has offered App Extensions for several years, in both UI-serving and non-UI contexts. App extensions are secondary services, bundled within apps, to provide some service. These extensions are invoked in their own process space, typically in the containing application's sandbox.

Examples include:

- The contents of the action/share menu in apps, which for example lets you share photos onto social media

- Audio Units

- Photo plugins

- Password manager autofill

- Content blockers

- Custom keyboards

- Filesystem providers to have a cloud storage show up in the Files app

- Finder synchronization for cloud-synchronized services like OneDrive

- All app integrations to extend Siri

ExtensionKit (and the underlying ExtensionFoundation) lets a host application define its own extension points, which other applications can provide. I do not believe you can define n-to-m extensions with this interface however; I can create a language server for an iOS text editor and put it in an app, but I can't do that once and support _every_ text editor.

This is the functionality that enables Live Activities in iOS 16.1. For example showing the score for an ongoing sports game on the Lock Screen.

That's a different API https://developer.apple.com/documentation/activitykit

Did the headline make anyone else think we were going to see system extensions again?

Sample code (not from Apple): https://iosexample.com/apple-extensionfoundation-extensionki...

> However, Apple’s documentation lacks crucial information on how to use these new APIs (FB10140097), and there were no WWDC sessions or sample code available in the weeks following the keynote. > Thanks to some trial and error, and some help from other developers, I was able to put together this sample code, demonstrating how one can use ExtensionFoundation/ExtensionKit to define custom extension points for their Mac apps.

Christ

Yes, not a good look for Apple.

It consistently amazes me how terrible Apple's documentation is. Not just incomplete, severely incomplete.

Is there any context, e.g. a press release, for people unfamiliar with the apple dev environment?

This Apple ExtensionKit documentation page opens with "Extensions are executable code bundles" but doesn't provide any executable code bundles such as examples, doesn't provide any way for a developer to ask a question, and doesn't show how a developer learns to create an extension.

This misses opportunities to reach developers-- and the tech is actually really compelling. I'm experimenting with it in the BoldContacts app, which conceptually extends the Contacts app to give disabled people ways to contact their families and caregivers.

Apple's documentation group needs a new leader ASAP who can emphasize documentation usability in concert with real-world developers. Apple wants its platform to continue being successful, and this goal can be helped by a documentation team that creates examples, questions-and-answers, and tutorials.

Totally agree.

Apple’s API documentation is nearly useless because it usually doesn’t give me any info that I can’t already infer from my IDE (which of course is XCode, sigh).

I usually don’t need a list of classes and methods, I need to know how the pieces fit together to do something useful.

They say a picture is worth a thousand words. Well, an example is worth a thousand method signatures.

Yes! To add to this, I also don’t want useful information buried in videos. I’ve run across things where the information I want is likely in a WWDC video that is linked from somewhere in the documentation full of method signature. I don’t want to have to watch a video to get what I could find much faster in some static form. It always confuses me when people see videos as a useful alternative to well written documentation: who on earth has time to sit through hours of videos just to figure out how to use an API? Transcripts help, but I find that they’re still not a replacement for well documented examples and walkthroughs.

I’ve been working on adding a watch companion app to an iOS app and its been maddening. The documentation boils down to: you can send strings back and forth. The method to sync app state doesn’t work in the simulator. It appears the rest is on my own to discover. I’m convinced its impossible to develop a watch app only using the simulator and docs.

> … doesn't provide any way for a developer to ask a question, and doesn't show how a developer learns to create an extension

Is this because programming that targets poorly documented APIs is taught at select universities, and Apple wishes for these new grads to have an upper hand?

I think this is by design. Apple wants to weed out non-loyal developers, so only the ones who truly love the platform build software for it. Unfortunately, it drives developers to use cross platform tech instead, which apple hates

> doesn't provide any way for a developer to ask a question

If you ask questions on the developer.apple.com forums, Apple engineers often answer them directly. At least that has been my experience.

They really love the Kit suffix, eh?

Why not? No different to the unix lib prefix

Consistent naming patterns help devs navigate the hierarchy of libraries/frameworks/APIs available.

CoreFoo is going to be a low level library for dealing with Foo. FooKit is going to be a high level way to create gui apps that deal with Foo. Likely FooKit uses CoreFoo under the hood.

And HomeKit is a product for IOT. Not 100% logical naming

It's a NeXT legacy, and it's a mighty fine and nice sounding one, so why not?

They inherited "AppKit" and "Foundation" from Next and they have been running with that naming convention ever since.

I do not understand what this actually does, has anyone got a better example? :)

I wish Apple docs would include examples. I'm not an Apple developer but I've played around with SwiftUI and they have fantastic examples.

I'm with you. Is there a tangible app example?

https://developer.apple.com/app-extensions/

This allows you to expose either a view or a function in your app to be shown/executed in another app.

How is this different from bundles/dylibs?

Dylibs is for including 3rd party code at build time, maybe a client to consume an API.

ExtensionKit allows you to expose realtime UI or information, coming from an installed app instead of a 3rd party service.

That way you don't have to expose your API or build an SDK. You just have functions in your app that can be invoked.

Also the feature is only provided if the user has that app installed. So the experience is curated.

> Dylibs is for including 3rd party code at build time

Nope. You can decide how and where to load and call entirely at runtime. Check out manpages for dlopen, dlsym.

I don't know anything about this new feature of Apple's, but knowing the way they've been going for the last decade and change I would guess the unique thing is around sandboxing, out of process calls, etc.

Practically speaking, that's still basically an SDK. It's all over XPC, but the interface still needs to be defined.

I hate to be that guy, but isn't this an Android feature?

As iOS user I couldn’t care less if it was.

Extension points have existed for much longer than Android has. The Mac has had them since at least system 6 (1988). Photoshop has had them since version 2.0 (1991). This isn't even new on iOS, since its been supported since 2015 in the form of AudioUnit plugins.

This is not correct. That's not an Extension Point. That's a plugin interface. Extension Points are a type of plug-in, but have the specific feature of not running in the host application's address space meaning that many types of bugs and security attacks are not possible. For example, if an Extension Point dereferences NULL, it will crash, but the host app can restart it and reconnect without losing any data. If a Photoshop plug-in dereferences NULL, Photoshop crashes.

Ah, yeah, that's an interesting distinction I hadn't thought of. Any idea what the first example of that type of extension was? My gut says that'd still been done long before android

IPC has been a systems feature for a while, but discovery is a higher level thing needed for this sort of plugin interface. COM and CORBA are ancestors I know about.

not any more than a filesystem is an "Android feature" because Android also has a filesystem API.

Such APIs have been around since the 90s, if not earlier. MS OLE, OpenDoc, Classic Mac's "Publish and Subscribe" API, I'm pretty sure OpenStep had something similar (which would be Apple's IP today, and predates Android by half a millenium).

so will I finally be able to open a link in one app that opens in another app and not in safari?

You can do that since iOS 14 (sept 2021) https://support.apple.com/en-us/HT211336

What I want (as a user, not a developer) is to be able to force an app to open links in Safari, not the app's built-in webview, and to force links to certain URLs to open in the app of my choosing, not Safari (in that case).

This is really, really basic stuff. Why don't we have this capability?

Ahh.. I see. Yeah, that's structurally unlikely to happen any time soon on iOS. "open a link" is not a shared idea across apps. In one app that'll be a webview, where the OS could get involved. In another app it'll just be a button - the OS doesn't know if that button is perceived by the user as a "link" or not.

So I think it'd have to be an app-review policy, and I suspect they'd have a LOT of pushback if they tried to enforce something like that.

They could have structured the OS so that each transition to a new view was a request to the operating system to open a URL, and the OS handled the routing to the browser, this app, a different app - but without that kind of a structure I don't think this very doable.

> What I want (as a user, not a developer) is to be able to force an app to open links in Safari, not the app's built-in webview

In some cases apps use the browser to render the app's UX, via hooks added to the browser to invoke native code (e.g. to expose native platform functionality that is not web accessible). This web content may only exist within the app sandbox's filesystem, or may be externally hosted.

But from a systems API standpoint, this is no different from Facebook opening links in their own internal browser so they can glean metrics on what links you are following.

So what you are talking about is actually a store policy (e.g. apps without the browser entitlement must open third-party web content via this other mechanism).

> and to force links to certain URLs to open in the app of my choosing, not Safari (in that case).

For HTTPS in general, IIRC the user can select from a list of apps with the browser entitlement.

Apps can already opt into their own HTTPS URL support for several years via Universal Links - but they have to get the domain owner to allow it to prevent abuse. They've been able to take over URI schemes outside of a reserved list since the App Store launched.

The android equivalent system (App Links IIRC) allows non-affiliated apps to take over certain HTTPS URL function, but the user has to enable it in settings - and will get a chooser if multiple applications can support the interface.

Not really. You can do that for web, or email, or maps, but for the most part - and especially with third party apps for things like Reddit, or Twitter - you're forced into opening links in Safari. You MAY be given the option to open the link in an app once opening in Safari but in my experience it doesn't always work.

It's fundamentally a worse experience than it is on Android, in every way.

I still don't get it. Can anyone provide an example how this could work in practice? Is there something like this in Android already?

It’s already used in many apps. For example, in the imessage app, you can tap the apps icon to show a drawer of all the extensions which are available. For example you have an extension of photos app available which lets you send photos in iMessage.

I don’t know if this can be implemented but I wish it was. I have the Word app on iOS. Now, a citation manager like Zotero could allow importing references into Word through this API. You click a button in Word and that brings up a dialog for selecting references stored in Zotero app.

One example that I use often — the Photos app on Mac has a built-in editing interface, but also supports extensions that allow third-party apps (e.g. Affinity Photo and Pixelmator) to take over that whole UI and present their editing UI inside the Photos app.

One potential use of for allowing "plugins" in an app (the host app exposes the extension points for others to hook into), where other apps can integrate with that app, within that host app's own views. This is different than existing extensions, which basically only shuffle data mostly one way, and always outside of the app that it is initiated from.

This framework is cross-platform, so it would offer such a plugin architecture on both macOS and iOS.

There's no hints that Apple would offer this kind of integration within their own apps however.

At first glance, looks like the apple version of COM/COM+/OLE.

Is this a new form of IPC ? https://en.wikipedia.org/wiki/Inter-process_communication

If not, what IPC is this using behind the hood ?

I'd like to take this moment to call out how atrocious Apple developer documentation is. This is honestly embarrassing. I would reject contributions at my company that shipped APIs/headers with this caliber of documentation. People who do it regularly get bad performance reviews.

I don't know if it's because they actually hate developers, or they're just bad developers themselves.

---

More on topic, this is an interesting approach to a tricky problem that exploits one of Swift's strengths (a stable ABI) and Apple's tight integration across the system (standardized app installation/structure/store/etc). You expose some functions as an extension/plugin API and other apps can just call them. Neat! IPC kind of sucks!

What could use some more discussion is how if you have two apps running and calling eachother (or being called from eachother), how does that affect the threading model? Are calls scheduled to happen later on the main UI thread, or are they actually coming from a separate process and you need to worry about synchronization if that call affects any kind of state?

They have a ton of different products and APIs and they just don't document them all. Even worse, much of the documentation they do have is hidden inside hard-to-search WWDC session videos or, ugh, Slack conversations with Apple engineers.

External developers have complained for years. [1]

[1]: https://www.caseyliss.com/2020/11/10/on-apples-pisspoor-docu...

At this point I'm starting to believe that Apple begrudgingly accepts that people outside that UFO that landed in Cupertino should be granted the privilege to develop applications for their hardware.

You'd think with nearly a trillion in cash, they'd be able to hire some full-time technical writers.

The only reason I was able to figure out how to use ExtensionKit at all was because of a chance encounter with an Apple engineer during WWDC that provided some needed information.

I'm sorry to disappoint, but ExtensionKit/Foundation do not make use of any Swift features in the way you describe. It's all just IPC (via XPC), so much of it is useable from ObjC, or even C!

Also, this does not provide a direct app-to-app communication channel. The extensions themselves must be separate executables and run within their own sandbox. I think the extension could communication with its containing app, but the system is not set up to do that. All your scheduling questions are really around how XPC works. The view itself is basically an image within your hosting app, so communication is entirely async to the other process.

> I think the extension could communication with its containing app, but the system is not set up to do that.

and on iOS, the containing app is likely not running (in a GUI context).

Typically, the extension is its own small program bundled within the app. The app can be run by an end user, while the extension can only be invoked via the extension mechanism.

Same thing on macOS. The containing app does not need to be running for the extensions to be active. You are right that the extension bundle can only be used by this mechanism, but interestingly the containing app can use its own extensions (though this may not make sense in many cases).

That's sad that it has a serialization layer. It makes sense, but it's not ideal or more performant than existing techniques.

I'm more interested in the extension handling after that IPC layer, it's usually non trivial to integrate the call into an existing app's threading model.

Still lacks proper documentation (like, vital details still missing). So obviously not a serious feature yet.

This fact has been reported multiple times before[1][2][3], and it's frustrating that a company that claims having over 1000 engineers, and touts its $6B campus at every opportunity in every one of its keynotes has, evidently, not enough resources for proper documentation.

[1]https://news.ycombinator.com/item?id=33411242 [2]https://news.ycombinator.com/item?id=33410300 [3]https://news.ycombinator.com/item?id=33410329

I object to that. Some of Apple's most serious features are missing vital documentation details!

Exactly. All this lack of vital info, from a company that offers APIs for developers to use (consume), is disgraceful. Especially when they claim they have over 1000 engineers, and keep showing off their $6B+ campus at every opportunity in their keynotes. They clearly have the resources to hire people to do proper documentation, but refuse to (no doubt because "Courage").

>Still lacks proper documentation (like, vital details still missing). So obviously not a serious feature yet.

Those two things are completely orthogonal tho... an API/tool/service can even be used to build billion dollar businesses and not have proper documentation...

As a total noob studying Swift and SwiftUI documentation I was wondering about this. But I couldn't believe this huge company would neglect their developer community like that. But as I delved deeper I found everything is beautiful on a surface level and beyond that suddenly there are no standards.

This fact has been reported multiple times before

That seems like a really good reason not to post it again as a toplevel comment.

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

Maybe there is a market for complete, comprehensive documentation by a 3rd party? Something similar to "definitive" programming books of the past, but web-based / searchable and maybe importable into IDEs.

I believe there already are plenty (objc.io, Swift by Sundell, Hacking w/ Swift, many more), where they're both technical and tutorial style info.

But I still feel that, as paying developers (membership fees, and giving out 15-30% of sales to Apple), we should get all the basic info needed without relying on third parties to get it.

The dream of COM is alive in Cupertino!

Not really? The example for Messages seems very specific; closer to an API for Messages.app than a generic process interop framework.

OpenDoc would be more relevant to Apple: https://en.wikipedia.org/wiki/OpenDoc

They at least decided to go with a sane language instead of C++

Also is it really COM like? it doesn't look like it

EDIT:

https://www.chimehq.com/blog/extensionkit-intro

https://news.ycombinator.com/item?id=33410067

COM is language agnostic.

Language agnostic in that COM lets you use the C++ object model in any language you want.

Not really except in that QueryInterface returns a pointer to a struct of function pointers. It doesn't place a requirement on the layout of the object (for example you can use a fat pointer for a COM object, which is incompatible with the C++ object model)

On top of the sibling comment, as of WinRT, COM has been extended with a new ABI, based on additional base interface (IInspectable), replacing TLB libraries with .NET metadata, and support for generics (.NET style).

So it isn't the C++ object model per se.

As someone who programmed COM back in the day for C++ applications, I can tell you definitively that you got no benefits over other languages outside of brittle macros and non-standardized Visual C++ compiler magics.

Especially once you crossed over into Office/VB land, and everything was just IDispatch, Variants, BSTRs and SafeArrays.

COM has been alive for quite some time, not only does Taligent and Copland ideas precede it (the IDL is lously based on DCE/RPC), similar ideas are used in the IOKit, DriverKit and XPC.

Reply on news.ycombinator.com