Hacker News Reader

Comment by dlcarrier | Hacker News Reader

I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.

View on HN

Two days sounds like a lot, both llama.cpp and openvino only takes a few minutes to compile on any decent modern cpu.

the full set of llama.cpp binaries builds in under 5 minutes with an unmodified build workflow straight from their github page on a literally ten year old dual xeon.

It also even easier to get working and integrate into your system in a sustainable manner with NixOS. Do it by hand or throw an LLM at it, it will get you a declarative patch for your NixOS config that brings llama-cpp into your config that you can review and add under version control (no random `make install` build artifacts contaminating your system, no wondering "what was it that I ran? what are all these files? how do I do the same with a newer version?" a couple months later). There's also likely some build cache where Nixoids have already build what you want.

I had a great experience with llama-cpp with Nvidia backend on NixOS.

(Sorry for being that guy.)

Does the A770 use the Xe driver? If so then it might work with the scripts that I've been using to build llama.cpp with SYCL support for the Arc Pro B70.

https://github.com/cptskippy/battlemage-llm-gateway

It's designed so that you can re-run the scripts to pull the latest updates. When Muse Glimmer was released the other day I just ran the 02 script to build the latest version of llama.cpp with support for it.

Thanks for the link. I'm running an Arch-based system, by the way, so I won't be able to directly use the scripts, which are expecting Debian's package manager, but I'll look at what they're doing and see if I can get something working.

I don't know how it works, but does Vulkan not work on Arc?

Reply on news.ycombinator.com