Hacker News Reader

I read, they don't merge PRs for Intel or AMD hardware, so it seems to be generally a bit of a shady project.

Just use Llamacpp directly

Could you expand on this, is there any disadvantage to continuing with ollama?

I use Ollama for prototyping and then move what I can to a vLLM set up

I think if you find Ollama useful, use it regardless of others say. I did give it a try, but found it lands in a weird place of "Meant for developers, marketed to non-developers", where llama.cpp sits on one extreme, and apps like LM Studio sits on the other extreme, Ollama landing somewhere in the middle.

I think the main point that turned me off was how they have their custom way of storing weights/metadata on disk, which makes it too complicated to share models between applications, I much prefer to be able to use the same weights across all applications I use, as some of them end up being like 50GB.

I ended up using llama.cpp directly (since I am a developer) for prototyping and recommending LM Studio for people who want to run local models but aren't developers.

But again, if you find Ollama useful, I don't think there is any reasons for dropping it immediately.

Reply on news.ycombinator.com