1. You can't fit a XXX billion param model on an iPhone. It's just impossible. A mixed approach where most ML is local; but some are cloud, fits with their existing Siri approach.
2. OpenAI won't share model weights, but if there's one company they'd make an exception for, it'd be Apple. Certainly not their big models, but perhaps they have a distilled, SOTA 3-7B model that they're happy to license, especially with Apple's Secure Enclave and ML model encryption.
A mixed approach is a great point though and I think I could mostly be happy with that.
Wouldn’t memory be a much bigger bottleneck? You don’t really need a particularly fast a CPU/GPU to run most basic models that couldn’t even fit into the amount of memory that Apple is offering on their devices (especially if you still want to run other apps).
You misunderstand point 1. Three-digit billion parameter models like GPT-4 (or even 3.5) aren't going to fit on modern iPhones, not even close. Even Llama 70b requires 35GB of RAM at q4 quantization, and that's just for the model.
Compare that to the iPhone 15's 6GB and you'll see the problem. Apple isn't about to announce an iPhone with 6x as much RAM as any previous model. I'd be shocked if they even doubled it. Their local inference models are going to be tiny and limited, which is fine, but that means they have to go to the cloud to provide all the features people are expecting.
Microsoft has 'em [0]
0 - plenty of sources on this, but here's one https://x.com/elonmusk/status/1639138603371491329
Almost all Apple devices (including most Macs) have very low amounts of memory so Apple hadn’t really positioned themselves that well if their goal is running LLMs locally.
You need it to understand what you want to do within the limited scope of the phone.
Basically I want Apple's AI to do what Shortcuts is capable of today, but instead of a janky Scratch programming style, I want it to respond to speech.