I could definitely image Apple embedding a kind of LLM-optimized FPGA: slow to load (update) an LLM, but blazing fast at computing tokens.
Who needs memory when your model is set in silicon ?
Who needs memory when your model is set in silicon ?
on device llm gives apple the new "better camera" "better screen" race they need to keep people coming back for the latest.
for average users everything else is tapped out... screens, cameras wifi... all the core stuff is good enough now its hard to feel/see the difference model year to model year. embedded llm would let them ship something new and the on device ecosystem advantage is huge. especially as the gpt and claudes get ads and enshittified... the apple on device even if its less "capable" would be so compelling.