I just use Ollama[1] - makes it incredibly easy to get going on MacOS, and can be run on linux/WSL also. RAM required will depend, but generally to run Mixtral at reasonable quantization levels (e.g. Q4) you're going to want 36GB or more.
No comments yet.