This is sick! We've built a similar product for LLM-powered browsing at https://zenfetch.com
Admittedly, we use cloud hosted providers since we found the quality to be much higher, though awesome to see local versions as well.
Admittedly, we use cloud hosted providers since we found the quality to be much higher, though awesome to see local versions as well.
I do agree that 3rd party LLM services (e.g. OpenAI) are much more powerful. For basic tasks (e.g. summarization, Q&A), local LLMs seem to perform decently.
1. Research across an industry 2. Generating new content 3. Connecting ideas / building networked thought
For that reason we were deterred from using local models. Hopefully the Mistral team and similar keep pushing out new developments to get comparable with GPT-4 levels