Hacker News Reader

This thing fly on Macbook M4 Max 128GB at over 100t/s, for small contexts, over 20t/s for large contexts. MLX 4bit quant.

View on HN

Is it good at using tools?

It would be nice having a fast local model that is good at using tools

All Qwen models are good at using tools, even the smaller 4B one. The 1.7B one gets confused easily

Thank you

Have you tried using them with something like Claude code or aider?

I’ve used it with Aider (32B and 30B, the previous 30B one, haven’t tried this fully nonthinking one yet) and 4B with home assistant. Both works great in terms of tool calling.

Like what type of tasks/tools are we talking about here, asking questions about the content from (PDF) documents or?

Reply on news.ycombinator.com