Jev-serve: Run latest qwen3.8, all LLM models, mlx-supported

github.com

2 pointsrreinold22 comments

Any recommendations for which of the models to try on macs with 16 GB of mem, and is this running well with any of the smaller models?

Guideline is (TOTAL_RAM - 5GB overhead) = max size, in billions of parameters. So 16 - 5 = 11, so <11B. Qwen 3.8 9B is a good fit