Hacker News Reader

It should work fine if you're on Apple silicon, everything else is unfortunately second-class citizen, because I don't have any other devices and it's currently just me working on it. But if you'd like to maintain the windows, linux or intel-based mac releases, ping me on discord and I'd be happy to discuss and to accept PRs.

View on HN

edit: [deleted]

Intel works in Ava, it's just slow, for some reason.

ah, got it. yeah the llama.cpp speeds I've seen are like 5-10 tokens/second on intel but I've been impressed with the relatively ancient hardware llama.cpp works on (2017 iMac with 8GB RAM at 5.5 t/s with a 7B model!).

it's ok, you can put the link back I don't mind the competition :)

yes, I know it definitely works, so it has to be some stupid mistake but it's impossible to reproduce it without actual intel mac :-/

I had enough trouble with supporting monterey, which also does not work 100% AFAIK

Reply on news.ycombinator.com