I hope Apple gives us both a web API (that can be used on any platform at a low-ish cost, similar to OpenAI) and a smaller, private model that's optimally sized to run on Mac, iPad, and Phone hardware.
Seems like any AI stuff will be designed to add value to its hardware exclusively, probably in conjunction with its on-device Neural Engine (tm).
- Apple Pay JS
- MusicKit JS
- Sign in with Apple JS
- App Store Connect API (not really user facing, but still)
I think Apple will come up with some crazy hardware to run good quality LLMs.
you mean like the "neural engine" that has been present in their SoCs for nearly a decade? (this is also why M1/M2s can run LLMs at comparable speeds to desktop GPUs... and they weren't even designed with LLMs in mind yet)
if you use commodity GPUs, sure. if you use TPUs (which Apple is already building into their chips) the efficiency improvements are massive. seriously look at some Coral Edge TPUs and what they can do at power levels completely unheard of for GPUs. then look at how much faster M1/M2 Macs are than normal desktop GPUs for machine learning tasks because they have an onboard accelerator
apple is known to care a lot about stuff like this. like, a lot. they are pedantic as heck