16GB is not enough.
32GB is enough to run quantized Mixtral, which is the current best openly licensed model.
... but who knows what will emerge in the next 12 months?
I have 64GB and I'm regretting not shelling out for more.
Frustratingly you still have WAY more options for running interesting models on a Linux or Windows NVIDIA device, but I like Mac for a bunch of other reasons.
Maybe keep an eye out for M1 / M2 deals with high ram config? I've seen 64GB MBPs lately for <$2300 (slickdeals.net)
I think 32GB might be the best middle ground for my needs and budget constraints.
It's really a pity that you can't extend RAM in most Apple Silicon Macs and have to decide carefully upfront.
I currently run Mistral and a few mistral derivatives using Ollama with decent inference speed on a 2019 Intel Mac 32GB. So I assumed the new one with 32ish should do a better job.
I've tried vision model Llava as well, a bit more latency but works fine.
With Apple's own Mlx things might improve .
Bait aside, I'd love to read about how are you using those models. I'm mostly interested in code comprehension and meeting summarisation.
I'm going to bump up my usage of Mixtral a bit now to see how it feels for that kind of stuff.
Although for those napkin like ideas gpt4 (including the turbo variant) get costly quickly.