Web LLM runs the vicuna-7B LLM in the browser and it’s impressive

I think a real promise with these smaller models is in upgrading/replacing Siri/Alexa in mobile devices, especially when it's trained on your messages/habits/calendar/notes/shopping/music. A true virtual assistant you can say "I want to have a karaoke party, can you plan one for next Saturday?" and it creates a guest list, invites your friends it knows like karaoke (even the ones you forgot about!), puts it on your calendar, reminds your friends, sets up a delivery of the party supplies based on what it knows you have on-hand, handles rsvps, looks into pricing of having your place professionally cleaned, and creates a curated playlist. It might not be able to do all these things with its local capabilities, but it can know when to reach out to more powerful models to execute the more challenging tasks.

The rest of my article goes on to discuss how a worse model with access to tools - the ability to run and summarize searches, call out or calculators etc - might feasibly be more capable than ChatGPT on its own, even though it would be far less useful as a pure language model.