Hacker News Reader

Comment by ricardobeat | Hacker News Reader

I had Ox Alpha working on coding tasks for a couple days non-stop, via OpenRouter and OpenCode Zen. Crush harness. It was able to complete tasks at a level that I'd put between Sonnet and Opus. It makes few mistakes, but is not that smart.

The main issue for me, is that it degraded into a doom loop several times. One of them was running the same bash command about a thousand times. The last model I've used that had this problem was Mimo 2.5, which is quite dated at this point. As a result of this, you cannot leave it unattended / not usable for agents.

View on HN

I couldn't get past all the network errors on OpenCode. Seemed smart enough, and was useful when I was low on usage on Claude, but beyond that, really hard for me to say whether it was Good or Bad.

I asked it to create a design system in Paper, and it actually did a fairly decent job when it wasn't getting network errors. I'd say it's much, much closer to good than bad.

Doom loops are as much of a model problem as it is deficiency of the harness. I have not seen any other open source harness that deals with them except the one I started because of this obvious gap.

See my other comment with examples where 0x Alpha is working non-stop on various projects with zero problems.

Pi.dev has an extension that does it. None of the models I use seem to have that failure mode so I haven't bothered throwing it in.

The failure mode I run into commonly is agents just stop sometimes. Even sending a "." Or something they start back up, but I haven't worked out exactly how to fix that generally in harness, bit unclear how to tell if they're done or just derped to a stop.

How do you recover from doom loops? Just send the same prompt again and pray that it works, or anything more sophisticated?

I'm not 100% sure what I'd do to detect this for shells specifically, maybe prompt the user, but my toy harness scolds refuses to re-read files because the previous iteration of GLM was room-looping on that. Clearly something that they need to work on.

detect, slice & dice, re-prompt

It feels like all these models are missing a long horizon orchestrator of some kind. Humans don't think in a single thread, they have thoughts that appear, pause and disappear.

The reason humans don't go into a doom loop is because of this ability of other thoughts to interject and say, no this is stupid.

IDK In my experience humans do go into doom loops.

I usually see doom loops when working with quants. Likely theyre trying to maximize the viability of a efficient model quant that can bw upgraded. Like cutting coke to get crack, quantiry over quality.

Nit pick; cutting (adulterating) cocaine doesn't produce crack cocaine.

It's hilarious when the HN crowd tries to fake street smarts.

If anything, the freebase is more potent by weight (albeit less water soluble than the salt form, so it's only really usable via vaporization).

Ox alpha at moments felt like it was quantized to hell. I think the last few days it might have improved.

> One of them was running the same bash command about a thousand times.

An amusing thought of returning to your workstation to find it as an obsidian block after it gets stuck executing "dd" thousand times.

glm models always had this doom loop issue. you can find reports of it on every version.

I noticed that this has a hard copyright rules...

Were you using the full model or a quantized version, and what harness/configuration were you using?

It sounds like you were using a quant model.

> Mimo 2.5, which is quite dated at this point

I know that AI is moving fast, but Mimo 2.5 literally came out four months ago. I literally had to double-check after I read this because it felt like just yesterday.

Also, I'm not sure what the industry standard is right now, but my (self-built) harness automatically exits with an error code whenever it detects similar tool calls being sent or when semantic repetition in the reasoning traces reaches a certain threshold. It's pretty easy to set that kind of thing up.

Reply on news.ycombinator.com