Apple's AI Strategy in a Nutshell | Hacker News Reader

Apple's AI Strategy in a Nutshell

nextword.substack.com

43 pointsnextworddev55 comments

> It’s now clear that Apple knows how to train frontier model-quality models, but it’s simply choosing to lay low. Apple’s server-side models running in the Private Cloud Compute are apparently quite near the GPT-4o in terms of quality

I really don't see where this comes from. Apple has been deploying Siri for a decade. Despite this, Siri is still a steaming pile of cow dung. In few months, OpenAI built a working Siri, something that no-one at Apple was remotely close to achiving.

The fact that Apple signed a deal with OpenAI and includes GPT-4o as an alternative option is a clear sign that Apple server-side models are really not anywhere near GPT-4o. If they were, Apple wouldn't have signed this deal which is so unlike them.

To me, it really looks like Apple is late to the party in terms of LLM. They are betting that within a few years, high quality models will be commoditised and that having an ecosystem that leverages them properly will be the differentiator. Until then they are reluctantly incorporating the market leader in order not mis the train.

> something that no-one at Apple was remotely close to achiving.

Probably due to internal political wars. Apparently they had an internal lightweight version outperforming existing Siri, but the team never got anywhere with it: https://daringfireball.net/linked/2024/06/06/how-the-wall-st...

Apple's AI strategy in a nutshell:

Sell more devices by using privacy and security as an argument for making new features available only on new iPhones.

Yup, this was the second thing I thought that's happening. Also, I wasn't sure whether to upgrade from 13 to 16. However, the AI angle tends everything towards buying a new phone.

A very smart move from Apple.

> A very smart move from Apple.

Indeed. Very smart.

"Stay hungry. Stay foolish."

It’s misleading to not point out that Apple Intelligence is compatible with all devices with at least an M1 — which was announced in November 2020.

On the mobile phone side, it's compatible only with the latest iPhone 15 Pro (not even the standard iPhone 15).

The parent comment makes it sound like everybody has to buy new hardware to use AI. Plenty of users with 2-3 year old hardware will be able to use it. As well as all the existing iPhone 15 Pro owners.

Apple will be selling iPhone 15s for the next 3 months that won’t get the AI features they just announced. When you buy an iPhone, you generally get software updates giving you new features for a few years at least. What they did with AI is kind of shitty, given their track record.

On the other hand, if your phone can't run a model, it can't run a model, you can't ship people a new AI coprocessor to solder onto their iPhone's motherboard.

That’s true, but they’ve made a lot of noise about “private cloud compute” to run models too large for your device. They’re making the odd decision to not offer it as a subscription for devices less than a year old.

They’ve also certainly been working on LLMs for longer than the iPhone 15 and 14 have existed. They knew what it’d support before announcing those devices.

If you have a "Private Compute Cloud" setup anyway, you could give users the option to run the model there.

I mean, I’m glad not to have this AI stuff. I’m glad I have a buffer of iPhones that don’t support this AI when I will be forced to downgrade to a newer device from first SE.

If you don't trust Apple to disable AI when you ask them to, why use them?

I'd like to have a smarter Siri though. Guess I'm stuck with an idiot who can't even understand the word "bedroom" anymore (though it worked fine until a recent update)

> It’s misleading to [unnecessary nitpicking]

I fixed that for you in the parent. Thanks for pointing it out.

Am I crazy to think that the claim of private cloud compute running M chips is marketing fluff? Is their ARM based SoC really as performant as a GPU at the scale they need?

Does it need to be? When they can use their own SoC servers, they don't have buy hardware from other vendors or rent cloud compute capacity.

So it only needs to be so efficient that it will still cost less than those other options.

I think these days Apple Silicon is more of an umbrella term for the packaging of their CPUs, GPUs and NPUs.

If they wanted to, why wouldn't they make a modified, specialized, server rack version of their hardware?

Internally it’s Project ACDC – Apple Chips in Data Centers.

https://www.wsj.com/tech/ai/apple-is-developing-ai-chips-for...

https://archive.ph/p4iuu

No real details.

No one said they are M chips. These would be custom made for servers and AI inferencing. Also note their "ARM based SoC" has a GPU with 800 GB/sec bandwidth in the M2 Ultra.

Not that it's a real source, but TFA states:

>These data centers will also run completely on Apple’s M chips

That guy is just some random blogger and I wouldn't take that to literally mean they use the same chips as a Macbook. Apple only says they use "Apple Silicon". Common sense to assume it is designed for servers and AI inferencing.

> Apple is already a vertically integrated AI company, and deserve a higher valuation.

Why does the latter follow from the former? I guess the author has Apple stock.

The larger context is that Apple gets most of its revenue from iPhone sales and use which have seen tepid growth. This strongly indicates reductions in valuation.

The article mentions that Apple can use OpenAI responses for further training their own models. I don’t want to say that’s impossible but that means they really screwed OpenAI in this deal. OpenAI terms don’t allow training on responses

>But Apple can also collect data on how users utilize GPT-4o versus Apple’s models, and perform gap assessment. This becomes valuable training data for Apple.

I’d read that as: they know what queries users choose to route to OpenAI, so they can identify where their models are being perceived as less capable.

I don’t think the author is revealing non-public knowledge of the contracts.

That makes sense!

Those 4o responses are also sensitive user data. Not ok for model dev

While I agree I think Apple has a different sort of agreement. Big corporations can have different agreements between them.

On the contrary, it's worrying to wonder what Apple gave up to make OpenAI accept those terms.

Cash money, probably. Apple has a lot of it.

"Indie app publishers are screwed." - that was one of the first things I thought about.

I didn’t understand this. If an indie developer provides an app a user pays for why should the UX choice of the user matter?

Because these days a lot of the money a developer makes isn’t made up-front, it’s made through interactions in the app. Siri isn’t going to read you the ads in the apps it’s using behind the scenes.

Maybe this will cause a shift back towards up-front pricing, which personally I would welcome.

That is the only thing I could come up with up but I thought it was to cynical.

The problem is that Indie is not the right description of those developers. Those developers come in all shapes and sizes.

Apple has been laying the foundation for “agentic” use of apps for a long time. All of the functions that apps make available to Shortcuts today will be usable by Apple Intelligence. I wonder if they already had that use case in mind when they came out with Shortcuts?

> I wonder if they already had that use case in mind when they came out with Shortcuts?

They didn't come out with shortcuts. They bought an app called Workflow that was leagues ahead of anything Apple was providing on the automation front in iOS.

And Apple never knew what to do with Shortcuts. They bought the app in 2017 and didn't integrate Siri with it until 2022. And Shortcuts still remain limited, and barely usable.

"Also, Siri’s agentic features - if they work as advertised - can increase Apple’s leverage over App Publishers, because now the AI - not the user - is the entity opening and clicking on the apps."

This was really interesting to me. How does one develop an app for Siri (or an AI agent in general). Is there a standard way to communicate and expose the functionality of your app?

We're going to have to pay to market our apps to an AI now, aren't we...

Surprised at the mention of TPUs. Apple used Google cloud for its training?

The the term TPU a Google trademark? I'm understanding it to mean any ML focussed ASIC or supplemental SOC hardware.

Everyone else calls them NPU. Except for a few edgy companies going for LPU

TPU as best as I can see is always google. Either in cloud or their coral devices

I mean TPU's are generally the only off the shelf viable and reliable alternative to Nvidia GPU's for training and there are a lot of former Google staff at Apple that have experience with TPU's, so I wouldn't be surprised.

The article seems to be based on misinterpreted information:

> Apple’s model has extremely low latency (0.6 milliseconds to first token), outperforms similar sized Phi and Gemini models from Microsoft and Google.

From [1] it is 0.6 millisecond per prompt token so unless the prompt is one token the latency would be higher.

> Apple’s server-side models running in the Private Cloud Compute are apparently quite near the GPT-4o in terms of quality

The benchmarks from [1] never mentioned GPT-4o, the best model they compared to for the server model is GPT-4-0125. If their model almost matches GPT-4o they wouldn't need to integrate with ChatGPT.

[1]: https://machinelearning.apple.com/research/introducing-apple...

If you stream the answer, the first token time is roughly the per token time.

No, you have to feed the entire prompt token-by-token before getting response.

Oh, I see, I misread. Thought it meant 0.6ms per output token. Now I get that it’s saying “prompt token”, so if your prompt is 100 tokens, that’s 60ms.

That seems pretty fast. 1.6k tokens for a 1s time. Do other models compare to that? I’m not sure what the current top ranking for this metric looks like.

Latency here is weird. You are using the number as bandwidth in this calculation. Perhaps reporter doesn't really know what he's talking about.

Apple's move is interesting, despite being a lot more reserved than this article's fanboy take on the impact of the move. For me, it is at least another clear blow on the "AI is just hype" arguments I still see around here.

A yes, a "clear blow". Because we can definitely deduce it from... an over-produced video with sloppy AI-generated images and some heavily edited actions/responses from a product that will not be released until fall (and even then, not with all features).

What other "clear blows" have there been so far that this one is "yet another"?

If you’re in the right info bubble, every AI announcement is a clear blow ;)

Reply on news.ycombinator.com