Hacker News Reader

Amazing work! This looks great.

Question: "no config setup" You mean the user doesn't have to pre-install Ollama or build llama.cpp or anything?

If you don't mind me asking - how are you accomplishing that in an offline desktop app?

Thank you! Yes the app has bundled built-in llama.cpp binary with it. The app just launches the executable - nothing fancy. We are a little llama.cpp wrapper :)

It doesn't require Ollama, but if you have existing Ollama, it works with Ollama's API compatible layer https://recurse.chat/faq/#openai-chat-completion-models.

Very cool you covered all bases. I'm going through this exact thing in an app, just moved it over from relying on ollama to using node-llama-cpp internally, but maybe I'll copy you and still support ollama too :D

have fun :) your users will probably let you know which way they want, or both

Reply on news.ycombinator.com