Hacker News Reader

Comment by dexwiz | Hacker News Reader

I think running a model unsupervised in anything above medium is a sucker move to burn more tokens. The high effort models can be great in limited context, but unsupervised they too often end up navel gazing. High doesn't always mean smarter, but it always burns more tokens. Medium and low seem to be decent for day to day tasks.

View on HN

I am not sure the supervision is the problem, it's more the way Claude is acting in general, taking initiative that could burn thousands of token in a second. Just today I ask it a small fix on a newly pulled project, it wrote 5 lines of code but then immediately started to run the full test suites, which itself runs in docker, with several images that I didn't pull yet so it started to download everything and by the time I click stop, it already filled the context windows with dozen of hundreds of useless lines... I find it very hard to fins the sweet spot in the prompt between "do not assume, do not take initiatives" and "inform me if there is a simpler way to do what I am asking or if there are some downsides I may not have thought about".

> I think running a model unsupervised in anything above medium is a sucker move to burn more tokens.

What about errors compounding and any on-the-go decisions being made being the wrong ones? That kills anything long form, no matter how good and detailed your initial plan is - there will always be something along the way.

The models would need a way to identify a difficult problem and apply max reasoning there themselves and cruise through everything else at a lower level.

That’s my experience too, going off the deep end with CoT doesn’t meaningfully improve output on a lot of tasks but it does incinerate tokens and take longer

My favorite part (codex thing) is how it runs all commands twice, every time.

Once on a sandbox, realizes nothing works in its sandbox, then again outside the sandbox.

Fucking hell.

I add commands that constantly fall into this rabbit hole to AGENTS.md alongside an instruction for the agent to request escalation immediately instead of failing first then retrying out of the sandbox. It’s a hack, but it works.

I've already granted it permissions to do everything outside the sandbox.

It still starts off in the sandbox anyway. :/

I just run it in dangerous / yolo mode in my own sandbox. Works great and no more nagging.

Reply on news.ycombinator.com