The same on Nvidia (various models) https://github.com/ggerganov/llama.cpp/issues/11474
[1] this is a the model: https://huggingface.co/unsloth/DeepSeek-R1-GGUF/tree/main/De...
I'm not sure I'm reading the results wrong or missing some vital context, but that sounds unlikely to me.
Where the a100 and other similar chips dominate is in training &c, which is mostly a question of flops.
I don't think they do.
From Wikipedia:
> the M2 Pro, M2 Max, and M2 Ultra have approximately 200 GB/s, 400 GB/s, and 800 GB/s respectively
From techpowerup:
> NVIDIA A100 SXM4 80 GB - Memory bandwidth - 2.04 TB/s
Seems to be a magnitude of difference, and that's just the bandwidth.