Stable Diffusion sits at about 2GB for fp16, Whisper Medium at 1.53GB, LLAMA is 120GB.
Sure, Apple can ship an optimized model (<2-4GB) as part of the OS, but what if a capable app maker wants to ship one? Users will not be happy with an app sized at >1GB.
The problem with OpenAI's business model is that it's actually quite expensive for them to maintain centralised processing. With Apple, there are billions of very powerful computers deployed to users and these computers mostly stay idle apart from occasionally running some bloated JS to show a button ar something. If Apple manages to run a good enough model on device with acceptable performance and energy impact, then suddenly OpenAI and Microsoft will be just burning away money with no expectation of recouping if they provide the service for free, if they make it paid they will be making money in a niche.
There really isn’t that much difference between iPhone 12 and 14
If a nee one comes out with LLM Siri + hardware that makes it possible that would be a massive upgrade cycle.
They can announce an iPhone Pro Ultra model that comes with higher RAM and storage capacity along with a souped up Neural Engine, similar to how they do today with how there are differences between the iPhone and the iPhone Pro screen and camera.
Even better, they could bundle the base model for LLAMA with iOS and ship incremental model updates to those iPhone Ultra users (possibly on a monthly subscription).
Most interactions probably don’t require full power
Yeah, but if every individual is running decent chat-capable LLM, and businesses are running their own on their own devices, and those can communicated with each other, who needs to rely on Skynet?
I don't think you can pirate e.g. the Facebook app binary and make it your own.