Hacker News Reader

How are you running llama on the XTX? I have the same card and getting local LLMs running was always such a pain I've not tried in a while.

I'm running ollama on Ubuntu. I tried other distros first, but I ran into some issues. Pretty straightforward on Ubuntu.

I have only used nvidia cards, but in my experience kobold.cpp and ollama forks of llama.cpp can be easier to compile.