Apple's PR is notorious for cracking the whip, which means that the "inside story", if they give it to you, comes with a warning to the journalist to behave and be nice. Levy's piece is generous with flattery and cautious with criticism. He quotes Kaplan and Etzioni high and briefly in the piece, and spends the rest of it refuting them. Apple will give him another inside story down the road.
Apple has a big question to resolve for itself about the tools it's going to use to develop this. It can't go with Tensorflow, because TF is from Google. It's kind of at another turning point, like the one in the early 90s when it needed it's own operating system and Jobs convinced them to buy next and use what would become OSX.[0]
The most pointed question to ask is: What are they doing that's new? The use cases in the Levy story are neat, and I'm sure Apple is executing well, but they don't take my breath away. None of those applications make me think Apple is actually on the cutting edge. There's no mention of reinforcement learning, for example; there is no AlphaGo moment so far where the discipline leaps 10 years ahead. And the deeper question is: Is Apple's AI campaign impelled by the same vision that clearly drives Demis Hassabis and Larry Page?
We see what's new at Google by reading DeepMind and Google Brain papers. Everyone else is letting their AI people publish, which is a huge recruiting draw and leads to stronger teams. Who, among the top researchers, has joined Apple? Did they do it secretly? (This is plausible, and if someone knows the answer, please say...) The Turi team is strong, yes, but can they match DeepMind? If Apple hasn't built that team yet, what are they doing to change their approach?
Another key distinction between Apple and Google, which Levy points out, is their approach to data. Google crowdsources the gathering of data and sells it to advertisers; Apple is so strict about privacy that it doesn't even let itself see your data, let alone anyone else. I support Apple's stance, but I worry that this will have repercussions on the size and accuracy of the models it is able to build.
> “We keep some of the most sensitive things where the ML is occurring entirely local to the device,” Federighi says.
Apple says it's keeping the important data, and therefore the processing of that data, on the phone. Great, but you need many GPUs to train a large model in a reasonable amount of time, and you simply can't do that on a phone. Not yet. It's done in the cloud and on proprietary racks. So when he says they're keeping it on the phone, does he mean that some other encrypted form of it is shared on the cloud using differential privacy? Curious...
> "How big is this brain, the dynamic cache that enables machine learning on the iPhone? Somewhat to my surprise when I asked Apple, it provided the information: about 200 megabytes.."
Google's building models with billions of parameters that require much more than 200MB, and that are really, really good at scoring data. I have to believe either that a) Apple is not telling us everything, or b) they haven't figured out a way to bring their customers the most powerful AI yet. (And the answer could very well be c) that I don't understand what's going on...)
[0] If they have a JVM stack, they should consider ours: http://deeplearning4j.org/