Hacker News Reader

Comment by dmos62 | Hacker News Reader

How do you use these models? If you don't mind sharing. I use GPT-4 as an alternative to googling, haven't yet found a reason to switch to something else. I'll for example use it to learn about the history, architecture, cultural context, etc of a place when I'm visiting. I've found it very ergonomic for that.

View on HN

I use them in my editor with my plugin https://github.com/David-Kunz/gen.nvim

Interesting use case, but the issue is wasting all this compute energy for prediction?

Can you explain what you mean by this question?

I host them here: https://app.lamini.ai/playground

You can play with them, tune them, and download the weights

It isn’t exactly the same as open source because weights != source code, but it is close in the sense that it is editable

IMO we just don’t have great tools for editing LLMs like we do for code, but they are getting better

Prompt engineering, RAG, and finetuning/tuning are effective for editing LLMs. They are getting easier and better tooling is starting to emerge

I’ve use lm studio. It’s not reached peak user friendliness, but it’s a nice enough GUI. You’ll need to fiddle with resource allocation settings and select an optimally quantized model for best performance. But you can do all that in the UI.

If you want to experiment Kobold.cpp is a great interface and goes a long distance to guarantee backwards compatibility of outdated model formats.

lm studio is an accessible simple way to use them. that said expecting them to be anywhere near as good as gpt-4 is going to lead to disappointment.

Reply on news.ycombinator.com