Hacker News Reader

I'll give it a try. I hope it's better than gpt4all, which I found out to be very buggy.

It should work fine if you're on Apple silicon, everything else is unfortunately second-class citizen, because I don't have any other devices and it's currently just me working on it. But if you'd like to maintain the windows, linux or intel-based mac releases, ping me on discord and I'd be happy to discuss and to accept PRs.

edit: [deleted]

Intel works in Ava, it's just slow, for some reason.

ah, got it. yeah the llama.cpp speeds I've seen are like 5-10 tokens/second on intel but I've been impressed with the relatively ancient hardware llama.cpp works on (2017 iMac with 8GB RAM at 5.5 t/s with a 7B model!).

it's ok, you can put the link back I don't mind the competition :)

yes, I know it definitely works, so it has to be some stupid mistake but it's impossible to reproduce it without actual intel mac :-/

I had enough trouble with supporting monterey, which also does not work 100% AFAIK

LM Studio works pretty well for me. https://lmstudio.ai/

LM Studio is great :) Also Faraday is great. But neither of them is open-source and this is going in a little different direction too (QuickTools, Workflows, there will be RAG and Vision support)

Reply on news.ycombinator.com