Hacker News Reader

Comment by cyanydeez | Hacker News Reader

I use exactly the setup you describe; it can handle 1-3 agents working if the agent work has IO delays. with MTP models, prefill is quite fast.

Not sure what damage you have, but no one waits 40 minutes per prompt or even a minute; The A3B model loads within 10 seconds and a simple response in opencode is maybe at most a minute, then catches up in 3-5second bursts depending on IO.

I put dynamic context pruning into opencode and tweaked it for 45k-85k context before it shrinks; this lets me get into 500k token sizes and fixed on medium sized github repos.

There's a chance this comment is a skill error:

1. USe llamacpp with a MTP model

2. Use reasoning-budget and reasoning-message

3. Tailor your agent to use the reasoning-message to use subagents and dyanmic compaction.

---

Now you have a reasonable coding agent for cloning, building and extending any github project I've seen so far.

View on HN

Are you on Strix Halo? If so, which llama.cpp backend are you using? ROCm on Strix Halo still seems incompatible with MTP models.

yes. I build a docker file from this: https://github.com/kyuz0/amd-strix-halo-toolboxes

It has all the backends you want. Just a script rebuilds it. Then I just put it in a docker compose and whenever we want to update it, we pull the repo and rebuild it.

I just a models.ini like: version = 1 ``` [*] n-gpu-layers = 999 threads = 28 flash-attn = on mlock = off mmap = off fit = off warmup = on jinja = true direct-io = on cache-prompt = true cache-reuse = 256 models-max = 1 models-autoload = 1 timeout = 600 ctx-checkpoints = 256 cache-ram = 0 mlock = on kv-unified = on no-context-shift = on swa-full = on

[Q36-35B-A3B] model = Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf mmproj = mmproj-BF16.gguf rope-scaling = yarn rope-scale = 1.14441 yarn-orig-ctx = 262144 override-kv = qwen35.context_length=int:1000000 ctx-size = 300000 reasoning = auto reasoning-budget = 421 reasoning-budget-message = you're reasoning too much; compress context and use subagents to investigate further spec-type = draft-mtp spec-draft-n-max = 3 batch-size = 16001 ubatch-size = 16001 cache-idle-slots = on temp = 0.6 top-p = 0.95 min-p = 0.00 top-k = 20 repeat-penalty = 1.1 presence-penalty = 0.0 ```

Using subagents, you keep the context size from bloating, and add tooling for the context compaction and it can keep looping. You probably want to clean up opencode's agent; back in the "Prompt Engineering" days there's a lot of unnecessary fluff. The goal of a prompt should be to get the model in the correct starting place, and not, try to give it everything it ever needs to do what you want it to do.

I'm able to use ROCm with MTP just fine (Qwen and Gemma).

I use Lemonade and have only the ROCm installed through it (ETA: their llama.cpp version is a fork, not sure what is different vs the main trunk).

Reply on news.ycombinator.com