Hacker News Reader

Do the models run in a docker container? Probably one of my favorite things about Ollama.

disclaimer: I'm one of the maintainers working on Ollama.

I would love to hear how you are using Ollama. One of the upcoming releases will involve an official release of Ollama on Linux with CUDA support of the box. From there, we will publish Ollama Docker images to enable GPU support as well.

Thank you for your amazing work! I more or less use it as a llama cpp replacement cause I honestly can’t figure out a good prompt structure so I get bad results. On my M1 MBP, I use it for stuff that can’t go to GPT but can only use upto 13B. I also run a 34B model with CUBLAS enabled on my Linux server by modifying the go files mentioned in one of the issue tickets.

Sorry, I thought I have answered already. No, everything runs as one native-like (mix of native and webview) application.

Reply on news.ycombinator.com