Hacker News Reader

Comment by milchek | Hacker News Reader

I tested briefly with a MacBook Pro m4 with 36gb. Run in LM Studio with open code as the frontend and it failed over and over on tool calls. Switched back to qwen. Anyone else on similar setup have better luck?

View on HN

I failed to run in LM Studio on M5 with 32gb at even half max context. Literally locked up computer and had to reboot.

Ran gemma-4-26B-A4B-it-GGUF:Q4_K_M just fine with llama.cpp though. First time in a long time that I have been impressed by a local model. Both speed (~38t/s) and quality are very nice.

Tool calls falling is a problem with the inference engine’s implementation and/or the quant. Update and try again in a few days.

This is how all open weight model launches go.

Haven't had time to try yet, but heard from others that they needed to update both the main and runtime versions for things to work.

Even with the latest version of LM Studio and the latest runtimes I find that tool use fails 100% of the time with the following error: Error rendering prompt with jinja template: "Cannot apply filter "upper" to type: UndefinedValue".

EDIT: The issue is addressed in LM Studio 0.4.9 (build 1), which auto-update wasn't picking up for me for some reason.

I googled it- supposed fixed template

https://github.com/ggml-org/llama.cpp/issues/21347#issuecomm...

Alas, this does not resolve the issue for me.

Yes same experience. Goes into loop mode where is sends same command again and again, till we kill it. This was Q_8 version on lmstudio

I can confirm that tool calls failed for me (Ubuntu server with charmbracelet/crush, if that matters)

Reply on news.ycombinator.com