You could possibly fine tune it for narrow domain tasks like they did with tiny-agent https://bair.berkeley.edu/blog/2024/05/29/tiny-agent/

I like the approach that Apple seems to be taking with fine tuned small models that handle routine tasks and then defer to larger off device models for things they can’t confidently do. I imagine you could construct a training set that contains examples that should produce low confidence answers where you could add an output that is essentially a “call for help” option so you could train it to choose that. Smaller models also means you could have more running in parallel and use another to route requests to the appropriate expert.

Reading emails, replying to emails, scheduling tasks, using apis for services.

Basically everything which doesn't need knowledge but actions.

"Tell my wife i'm late" and it will use some configured magic to talk to service xy and just does it.

Siri is very good in doing homeautomatistaion without the internet, the old google agent and alexa were absolutly not and i don't think they were ever available offline.

This basically gives you a local (local-first!) good working assistent

Local agent like siri that can do simple tasks, and route more complex requests.

It can be fine tuned for device related actions. In other words, with all the capabilities of your device applications or services, the small model can virtually have the same capabilities. It can always dispatch a user request in way of “natural language” to those applications, and orchestrate the applications. It can dispatch user requests beyond the device capabilities to a cloud model. This is powerful since it changes how you interact with your devices.

I tested the Google AI on my phone, I had the browser open and asked it to read the page to me and it responded that it does not have access to the internet.

So I would like an AI assistant that:

1 can understand english and my native language

2 that is aware that runs on Android(or KDE/Linux) and can understand commands like "open the Android Settings , Application section " or "read the page that is opened in the browser" or "read the text in the popup that is now opened". Basically to be integrated with the OS via public and open APIs. Big AI companies could compete on selling us better assistants especially for multi lingual people.

3 the model should be small , it should not know geography, history, music bands etc, for tasks where the user asks question there should be an option for the model to forward the question to a search engine or even an online LLM.

It could power simple agents like Siri under the hood. Helping with natural language understanding, intent classification, retrieval, and other agent tasks.

user cases are that of LLMs, from a mobile UI (so every AI use case there is), when you need privacy from big tech's AI APIs.

I'm just so amazed by statements like "LLMs can ONLY be used for autocomplete", like am I supposed to be impressed by the smirkiness?

Probably hacking foreign intelligence codes.