What is a decent tok/s?
Your best bet is to run a quantized 7b model using LMStudio or Ollamma on your M1 Mac, like neural chat v3.1 from Mistral/Intel.
Your best bet is to run a quantized 7b model using LMStudio or Ollamma on your M1 Mac, like neural chat v3.1 from Mistral/Intel.
No comments yet.