Explaining iOS 8’s extensions: Opening the platform while keeping it secure | Hacker News Reader

Explaining iOS 8’s extensions: Opening the platform while keeping it secure

arstechnica.com

81 pointsbitserf26 comments

If I'm understanding the article correctly, the only extensions iOS allows are keyboards, document pickers, photo editors, sharing services, widget things, and "action extensions." Assuming the action extensions aren't as catch-all as their name, this seems significantly more limiting than Android's model, where any app can arbitrarily define an intent (e.g. "com.example.calculatething") and send it with arbitrary data to any app that defines itself as catching that intent. These built-in extension types seem fairly encompassing now, but they don't leave much room to expand to future ideas, unlike Android's system.

Can any iOS developers clarify the situation? I've long been an Android fan (I've even written a few apps), but iOS 8 (and its integrations with Yosemite) are very tempting.

Action extensions are essentially share extensions with a return value. They may or may not present UI, and then manipulate content before returning it to the running application. For example, a Bing translate action can be run on a webpage.

Extensions cannot be enumerated or run programmatically using public APIs. They can only be presented after user interaction in a presented UIActivityViewController.

Hm, that doesn't sound too bad. Do you say a certain extension "implements" a certain type of action or something? E.g. the Bing translate implements "com.apple.action.translate.web" and thus when Safari asks for actions that implement "com.apple.action.translate.web" Bing translate appears to the user, or Zxing implements "org.moreactions.scan.barcode" so then when the user taps "find by barcode" in a shopping app the Zxing app is presented to them as a choice?

They're not quite as specific as "com.apple.action.translate.web". I believe it's more like "com.apple.web" - any action that can transform a website will include that in it's extension predicate.

I guess my primary question was whether apps can define their own actions -- can they, or is there an Apple-controlled set?

Action extensions can declare their support for any UTI (uniform type identifier), so yes - apps can effectively define their own actions. There is an 'Apple-controlled set' in the sense that the item being provided must have a UTI.

It's a process. iOS 4 opened the limited multitasking APIs, and iOS 7 allowed any app to refresh in the background. I predict the same for extensions, except on an accelerated schedule.

Exactly. Plus, I think Apple is (quite reasonably) concerned about the potential privacy and security risks posed by malicious extensions, and so is intentionally minimizing the available surface area, and some aspects of functionality in order to mitigate threats.

The mechanism underlying extension points (XPC, remote views), is pretty general, I would be surprised if the initial use cases were the end of it.

ActiveX is pretty much a poster child for what happens when you rush this kind of feature, so, hopefully the assorted security vulnerabilities will not occur with Extensions.

I think it doesn't solve a security problem as much as an UX problem. In Android, you can activate a context (a file, a photo, a document) after you've chosen an action from a, sometimes endless, list of possible actions. In iOS it's inverted. You'll get a context within an app and can then choose a possible action. Also, there is no option for a default action, which is a huge plus, because in Android that confuses the user more than it helps him. See also this link that was posted a while ago on HN http://jacksonfish.com/2014/05/22/a-user-experience-designer....

As for security: iOS warns you that a keyboard extension may send your input to a remote server, but it doesn't prevent it. I assume it's the same for the other extension points.

While I welcome the new keyboard extensions, they could've done better. A keyboard extension can send your input to a remote server but isn't allowed to set the text cursor or mark/copy/paste text. That's a huge drawback to me. The most effective mobile keyboard I've ever used was on my jailbroken iPhone, where you could move the cursor or mark text by swiping over the keyboard. But YMMV.

I think it does prevent sending keyboard strokes by having two levels of sandboxing, where you can either run heavily locked down with no network, or opt in to extra permissions including networking in which case your app will be extra carefully reviewed by apple + users will get huge warnings during install.

> I think it doesn't solve a security problem as much as an UX problem. In Android, you can activate a context (a file, a photo, a document) after you've chosen an action from a, sometimes endless, list of possible actions.

Note that Apple extensions do a different range of things than Android intents; they cover a lot of the same stuff as intents, but also some other things. In particular, they have plugin-like behaviour; you could, for instance, be writing an email, decide that you want to edit one of the images in the mail, and pop up an editor, which from the users point of view appears to be embedded in the app. The easy way would have been to implement this as a plugin; however, in that case it would live in the same sandbox, which would be dangerous. Android intents don't provide this functionality in the first place, so don't have to worry about it.

> As for security: iOS warns you that a keyboard extension may send your input to a remote server, but it doesn't prevent it.

See the documentation. (https://developer.apple.com/library/prerelease/ios/documenta... Table 11.1). There are two security models for keyboards. One type is allowed to talk to remote servers, and to communicate with its 'containing' app (that is the app it's installed with, not the app it's providing keyboard functionality to). Some keyboards need to be able to talk to servers for a full range of functionality. On installing this type, the user will be warned and asked if they wish to proceed. It hasn't been explicitly said, but in practice, this type will be subject to far stricter review.

The other type has no access to the network, or access to the containing app. This type is thus guaranteed not to leak data (barring bugs in the security model). This type is the default, but it restrains functionality enough that some people may opt for the other type.

I don't know if it's called intents (activities maybe?), but are Android's popup activities (I think) similar? An example is http://i.imgur.com/usXmLzn.png, which is Dropbox providing a file select to another app. Other than being full screen (I'm not sure if Android apps can do these activities full screen) is the concept about the same as that?

Similar enough, I think. Apple seem to have gone with the rather confusing approach of calling a whole lot of things extensions because they (presumably) work similarly, but aren't necessarily all that related in function.

For the usability, what I've seen so far in the WWDC Videos, I think iOS gets it right. Android's intents feel like the "Open with" menu on the desktop, only that it's always shown (unless you specify a default action, which many people I know, including myself, are uncertain about). The way iOS handles it seems much more suitable for touch devices. As you said, Android doesn't have to worry about sandboxing and its ramifications, that's why I deviated from the security topic.

> See the documentation. I admit I on glanced over the documentation. I dived right into implementing a custom keyboard and look around what's possible. That's when I got the warning about the remote communication. BTW, this was the keyboard tweak I was talking about: http://www.youtube.com/watch?v=R1caFI-hPkg

So... does this mean that I can finally click an address to open Google Maps?

Well that's unfortunate. It's one of the main reasons I'm switching back to android (that, and emulators).

Could any one who knows more about this more than I do answer me this. Would this extension framework allow an app to stream a file and have it played in another app. For instance, on my Android TV stick I can open my network share in ES File Browser and click on a file and have it play via MX Player. The ES Browser streams the files via samba from the server and then passes it to MX Player to play. Would the Apple extension system allow for this type of model.

I am just using video as an example. I guess in general I am wondering if the new Apple framework would allow the host app to continuously stream content to an extension. So, I guess another example would be like a website. If it's a dynamic website in spanish, for instance, and I would like to use Bing to convert it to english. Would it be possible to design an extension that would dynamically translate the site, or would it be more of a static deal, where once translated, you have to execute the extension again to translate additional dynamic content that may have appeared on the same page.

Also, apple claimed that they are going to expose GPU to apps, so you can now write apps that can offload calculations to the GPU. Does that mean that we might finally see GPU decoding and encoding for none apple video formats, or is that still a no-no?

It seems like that would be possible. The only restrictions that I can think of are the fact that each (iOS) extension needs its own app with significant functionality (so this extension would have to be provided by a file browser or something), and a extension will only run for as long as it takes to do whatever its supposed to do (so no background extensions). I think as long as you're watching the video the extension will be active, so that shouldn't be a problem. You might need to add support to the video player, but besides that I don't see why this wouldn't work

As for the GPU thing, I have no idea, but it seems like they're trying to be more open; I'd be kind of surprised if they said no.

> Also, apple claimed that they are going to expose GPU to apps, so you can now write apps that can offload calculations to the GPU. Does that mean that we might finally see GPU decoding and encoding for none apple video formats, or is that still a no-no?

Why wouldn't that be allowed, where it's feasible? Right now, there are lots of players which do software decoding of "non-Apple video formats" (I assume you're talking about h264? iOS doesn't support any of Apple's own video codecs, which are very old).

Am I wrong or is this stuff that is still under NDA?

It seems Apple has reduced the scope of the NDA this year. http://oleb.net/blog/2014/06/apple-lifted-beta-nda/

There was a blog post about this http://oleb.net/blog/2014/06/apple-lifted-beta-nda/

Further, Apple agrees that You will not be bound by the foregoing confidentiality terms with regard to technical information about pre-release Apple Software and services disclosed by Apple at WWDC (Apple’s Worldwide Developers Conference), except that You may not post screen shots, write public reviews or redistribute any pre-release Apple Software or services.

As was already said, Apple has limited the scope of the NDA considerably.

The NDA only applies to registered developers anyway, though. Additionally, this year anyone can watch all WWDC videos (discussing much if not all of what is mentioned in the linked article). The general public that can just access the page with the WWDC videos (without any login or any requirements to identify oneself or accept anything) is obviously not at all bound by any NDA at all.

Here are the videos, check them out: https://developer.apple.com/videos/wwdc/2014/ (sadly still requires the right software to watch)

I like this new more realistic and pragmatic approach to how they handle their NDA. In the past it was always a pain to listen to actual devs talk about the WWDC because they always felt like they had to be careful about what they said (though, to my knowledge, Apple never enforced that, but better safe than sorry, especially if you make your money on that platform). Meanwhile, everyone else was just talking about it freely, but exactly those people who most likely would have the most interesting things to say were silenced.

This is publicly available: https://developer.apple.com/library/prerelease/ios/documenta...

Apple's releasing more information than usual this year.

Reply on news.ycombinator.com