Reduce or turn off thinking:
Granted, it's still much lower tokens/s than you'll get out of many MoE models.
Edit: Even set to medium or low there's still a lot of second guessing, less consistency, lower 'acceptable response' rate, and slower/more token churn vs gemma4:26b-a3b. I think gemma4 is just a better 'general purpose' model.
I don't usually rely on AI for much (I'm actually kind of anti-AI, although I follow stuff like this enthusiastically because of the rapid advancements, "average joe" access, and openness), however for what I asked? It was spot on.
The subject was a bit personal, so I won't share. I was just curious what AI would say about the situation and it definitely surprised me, especially since Qwen, while usually great on development/coding stuff, has shown weaknesses in other areas.
Definitely a solid release, and this one runs on my 4090 with minimal loss of quality!
Did I miss something, is it possible to have that model be reliable without thinking?
Ignore whatever nonsense qwen says about what settings to use. Turn off top-p and top-k. Turn on top-n-sigma and keep it at its defaults.
See repetitions that you don't like? Ignore the "repetition, frequency, presence" penalties and turn on DRY (don't repeat yourself).
Want more creativity? Temperature is okay but XTC (eXclude Top Choices) is better.
Llama.cpp quietly has had far advanced samplers compared to the shit stuff Qwen et al recommends for years.
I used the settings recommended on huggingface/unsloth's model page + whatever suggestion from various LLMs - didn't research too much myself which setting did what.