Hacker News Reader

Comment by Wowfunhappy | Hacker News Reader

...the evidence, as best I can tell from the tweet, is that they asked Claude what effort level it was set to. But how would the model even know that?

Not convinced here.

View on HN

Then model can say it's Opus, but really it is some old Sonnet. This seems to be happening less often, but some weeks ago I had to give models some test problems to gauge whether I am getting Opus or something knee-capped.

The problem is that Anthropic seems to be getting away with selling one thing and delivering another. You pay for Opus, you get something else etc.

Saw this in npx ccusage@latest claude output. Had only used opus but showed sonnet. Can't remember if the jsonl retains which model is doing what, but meh

I don’t know if this is still the case but while using Copilot if you looked through the chain-of-thought output you would see it reasoning about a “budget”. i.e. “since I’m close to the session budget I should…”. So it could be possible

Effort level is actually controlled entirely by system prompt (as I understand it, the model is trained on that format but still), so actually this is a valid way to check I think

I know that's true for Qwen but I don't think most models work that way?

OpenAI models also work this way, as evidenced by full cache blowout when changing reasoning level. Every single open-weight model I've seen also works this way (your "reasoning_effort" argument just changes a small section of the system prompt in the chat template). I would have to see some evidence to believe Anthropic were doing anything different.

Reply on news.ycombinator.com