Hacker News Reader

For those of us who don't want to pay for Gemini tokens, what would be the best local LLM to use for this?

(A model that can run reasonably well in a ~24GB MacBook.)

I don’t have a good answer but would like one. I’ve got a not too old desktop with a 4090 and 64gb of ram, I’d like a local model also.

I haven't tried with Claudette but did evaluate https://github.com/zachahn/vomit and https://github.com/gvzdv/claudish-to-english earlier today. I ended up using claudish-to-english with a prompt derived from the vomit one and some of my existing instructions. I ran a few local models through a test harness to see how they did, and the gemma4 ones added bad behavior back in less than any others I tested.

So for the parent's Macbook question `gemma4-26b-mlx` should work well.

For you with 24 GB VRAM, `gemma4-26b-a4b`. I tried higher VRAM models and they slowed down while still doing just as well or slightly worse.

If someone else tests and finds a better performing model though, please update me here, I'd love to try it.

Reply on news.ycombinator.com