Hacker News Reader

I’ve been running Orca 2 13B on M1 Pro with 32GB of RAM with LLM Studio and GPU acceleration quite nicely.

What kind of tokens/sec do you get?

Ollama runs 13b models just fine on my M2 Air with 16 GB