Hacker News Reader

Smaller, dumber models are faster than bigger, slower ones.

What model do you find fast enough and smart enough?

Not OP but I am finding the Qwen 2.5 32b distilled with DeepSeek R1 model to be a good speed/smartness ratio on the M4 Pro Mac Mini.

How much RAM?

It takes between 22GB-37GB depending on the context size etc. from what I've observed.

Thanks!

I'm running the same exact models.