The amount of RAM is the bottleneck for LLM inference on Apple Silicon. With 8GB of RAM, 3B models StableLM 3B and smaller work well. 7B models nominally work, but there won't be much room left for running anything else. 16GB Macs are somewhat of a sweet spot, you can run 10.7B and 13B models, which are quite good.

Maybe at a small quantization, but you probably don’t have enough RAM.

Yeah, some of the bigger models won't work, but there are a bunch that will.