Hacker News Reader

Comment by bilbo-b-baggins | Hacker News Reader

Devin shows model thinking.

I’m pretty sure the big bois don’t do it because it would undermine “confidence”.

Seeing a model output “Oh I should just delete blah. Wait blah is a production service, I shouldn’t touch that. Maybe I can gain access to blah? Oh the aws cli isn’t signed in to blah. I see kubectl has access to blah though! Wait, I should ask user permission first.”

Yeaaaaah. Thinking tokens are fuckin’ wild.

View on HN

Idk I feel like the more likely answer is to prevent distillation. Having the thinking is definitely better UX (oftentimes, I don’t know if Codex is just hanging, which it often does, or working in silence).

Running some models locally and seeing these thinking tokens was quite the experience. I never saw an LLM so "unsure" about virtually everything.

I doubt most users would look at them if they were available. More likely they don’t want to stream distillation material.

I stay much more hands-on when I'm using models that display full reasoning traces. And I tend to get more things done as a result, because I know exactly when it thought of a good solution that it talked itself out of because of some invalid assumption.

Could be something very stupid like - "I don't have ffmpeg available here. Should I install it? No, I can't. I'll proceed doing something that will take me 100x more tokens and wall clock just to avoid adding a dependency." I can then just stop and say - you've got nix flake there, just add it.

That's impossible with western models. The only way is to ask why it did something stupid when it already spent 50% of your weekly quota and produced millions lines of slop.

Oh yes, I’m 100% with you. I wish they’d keep it. It’s better for users but perhaps untenable for the business.

Reply on news.ycombinator.com