This is essentially what Windows Copilot seems to be attempting to do currently. There's obviously downsides to this approach from a security concern - it doesn't live on device, it's on the cloud.

I actually think Apple is incredibly positioned here. All of their devices have dedicated NPUs, if they can optimize an LLM to run locally on these in a way that's performant/accurate, the power could be incredible for a local AI that helps out. It'd likely be a huge sales driver as well - Pro devices might have faster/more accurate AI assistants. If it's purely local, the access to personal data becomes less of a concern. Super curious to see how this evolves.