My first thought was how much RAM? Will it work on 64GB M1?
Edit: To add to this, I've had good luck getting solid output out of mixtral 8x7b at 3-bit, so that isn't small enough to completely kill the model's quality.
https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#qu...