Hacker News Reader

Comment by CGamesPlay | Hacker News Reader

Possibly a strange question, but do you have plans to add online models to the app? Local models just aren't at the same level, but I would certainly appreciate a consistent chat interface that lets me switch between GPT/Claude/local models.

View on HN

You could try out Prompta [1], which I made for this use case. Initially created to use OpenAI as a desktop app, but can use any compatible API including Ollama if you want local completions.

[1]: https://github.com/iansinnott/prompta

This one doesn't seem to support system prompts, which are absolutely essential for getting useful output from LLMs.

Can you speak more to this? I get useful output from LLMs all the time, but never use system prompts. What am I missing?

Sure, I use one system prompt template to make ChatGPT be more concise. Compare these two: https://sharegpt.com/c/fEZKMIy vs https://sharegpt.com/c/S2lyYON

I use similar ones to get ChatGPT to be more thorough or diligent as well. From my limited experience with local models, this type of system prompting is even more important than with ChatGPT 4.

Is there a difference in using a system prompt and just pasting the "system prompt" part at the beginning of your message?

Haven't tested, but having it built-in is more convenient, and convenience is why I'm using these tools in the first place (as a replacement for StackOverflow, for example).

You can update the system prompt in the settings. Admittedly this is not mentioned in the README, but is customizable.

> the system prompt

There isn't a singular system prompt. It really does matter!

Copy the OpenAI playground, you'll thank yourself later

You use multiple system prompts in a single chat? What for?

Fair point, and it's not implemented that way currently. It's more like "custom instructions" but thanks for pointing that out. I haven't used multiple system prompts in the OpenAI playground either, so I hadn't given it much thought.

I've run into the same problem with deploying Gemini locally, it does not seem to support System Prompts. I've cheated around this by auto-prepending the system prompt to the user prompt, and then deleting it from the user-displayed prompt again.

Not strange at all! It's a very valid ask. The focus is local AI, but GPT-3.5/GPT-4 are actually included in the app (bring your own key), although customization is limited. Planning to expose some more customizability there including API base urls / model names.

Shameless plug: if you need multiple AI Service Provider, give BoltAI[0] a try. It’s native (not Electron), and supports multiple services: OpenAI, Azure OpenAI, OpenRouter, Mistral, Ollama…

It also allows you to interact with LLMs via multiple different interfaces: Chat UI, a context-aware called AI Command and an Inline mode.

[0]: https://boltai.com

https://recurse.chat/faq/#:~:text=We%20support%20Mistral%2C%...

...how did you highlight a specific sentence like that?

Looks like a Chromium-specific feature: https://web.dev/articles/text-fragments

Pretty cool. Doesn't work on Firefox.

It just worked on Safari on iOS. That’s pretty impressive.

https://caniuse.com/url-scroll-to-text-fragment -- yes Safari supports it

Reply on news.ycombinator.com