A Macbook Pro M1 would have the base M1 CPU while he was referring to the M1 Pro CPU in something like a Macbook Pro w/ M1 Pro. It's confusing naming by Apple.
right, fixed it
This is faster than running it on an RTX 4090 I think.
Nope a 4090 can do the 30b-4bit model at 20 tokens/s
I get 32 tokens/sec on a 4090 using GPTQ 4bit with streaming off, with the model 5x larger than that.
So nowhere close to the 4090, but plenty fast anyway.