My perf sucks compared to yours. Added it to the post - same model averages 325 tok/s in processing prompts, and 34 tok/s in token generation. What am I doing wrong..?
I would try a different version of the model from HuggingFace while ensuring it's MLX. I'm also using LM Studio, not oMLX, and I've seen some threads like these:
https://www.reddit.com/r/LocalLLaMA/comments/1spuwir/omlx_10...