Hacker News Reader

Comment by acmnrs | Hacker News Reader

From the prompting guide<https://platform.claude.com/docs/en/build-with-claude/prompt...>:

> Claude Opus 5's default user-facing responses run longer than prior Opus models'.

The benchmarks do show Opus 5 as slightly more expensive than 4.8, although the scores are much higher.

This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their models. Fable's token efficiency made it seem like Anthropic would start following OpenAI's approach but that doesn't seem to have carried over to their other models.

View on HN

On a small, easily digestible task, I compared Fable to Opus and the cost of Fable was easily 2x despite being fewer tokens, and the output was not really better. Obviously, there are tasks where using Fable matters but honestly they're rather unusual. And for a lot of tasks I've found downgrading to Sonnet can be valuable because Fable and Opus are a lot more secretive about what they're doing, and it's impossible to "listen to them think" and stop them when they start making off-the-wall inferences/assumptions and going down bad paths.

I think in the long run tokens are probably the wrong thing; it's compute and cache memory that you need to be measuring, and when you look at it that way I suspect in most cases the models have pretty similar performance.

Yeah I mean if you prompt “hi” to both models, fable is going to be costlier. Most tasks are complex enough to use the higher intelligence.

This is part of why I switched to Grok 4.5

I don't need more powerful models, I need one that responds fast enough that my attention doesn't wander to other tasks. Grok 4.5 is so fast I can just use it in-band without swapping to other tasks.

Slower than Opus 4.8, which was already miserably slow, is indeed a step in the wrong direction.

for you, I am in general not constrained by the speed of models since I parallelize. For me autonomy and accuracy are paramount above all else.

I find that my orchestrating agent still needs to be fast to properly coordinate a fleet of slower models.

During post-training of opus 5, the last few days, opus was a real wreck. I had to swap in gpt 5.6 sol for my orchestrator and enable fast mode (1.5x speed) in order for it to keep up with work and communications from a handful of mostly 5.6 sol agents.

Also because interacting with a slow orchestrator is no fun, even when plenty of work is getting done in parallel in the background.

After actually using for most of a day now, it really seems not just slightly slower than 4.8, but way slower. Even relatively easy code refactoring tasks seem to take a while.

In my tests, it averages to much cheaper than Opus 4.8 on real tasks on account of being smarter and more token efficient.

I have a benchmark to build a game engine from a set of written instructions. It's a little tricky. Opus 4.8 did it in 470k tokens at a cost of $1.29 vs Opus 5 in 179k tokens for $0.33. (Fable 5 did it in 245k for $0.95)

Though if you really want to cut costs, Tencent's Hy3 model also got it right and did it in 283k tokens for $0.03

The final user-facing responses are usually a tiny fraction of the total tokens used over the course of a given conversation turn. When you're doing any real work, reasoning and tool uses constitute the overwhelming majority of the tokens in / out... not the final user-facing response.

> This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their models

Gemini also had modest increase before this - don't be surprised when OpenAI also has a "modest increase" with its next release. Cartel-like behaviour doesn't require direct communication when none of the participants are interested in participating in a margin-destroying price-war. All one needs to do is raise their price and watch how the competition react.

Such a scheme (and resulting high margins) would be imperilled by the existence of frontier open-weight models in the market, which may be why the reaction to Chinese models may be particularly shrill.

>Don't be surprised when OpenAI also has a "modest increase" with its next releases; cartel-like behaviour doesn't require overt coordination when none of the participants are interested in participating in a margin-destroying price-war.

No I will be surprised and I'll bet on the fact that prices will keep going down, just like it went ~50% down in the latest GPT 5.6 release.

If the price keeps going down, how are they ever going to be able to make back all the money they’re spending? If the price stays high, how are they going to undercut the cost of labor?

Margins?

But they also blew the cost of hardware up.

Profitability at 2024 prices would be easy, not at current ones

Hardware went up by 2-5x. Efficiency 100x.

This is a comment about user-facing responses, which are seldom the thing you're worried about when thinking about token efficiency.

> especially with OpenAI making so much progress with the efficiency of their models

To be fair though, Sol tends to go off the rails sometimes. It's much less reliable than Fable in its outputs. It tends to be overzealous in its research/changes.

Is that true? Sol responses are also longer than prior models.

Anecdata: my workflow has been working on the same personal projects for months now with Codex. I cannot anymore finish my daily/weekly code with 4.8 anymore.

I was dividing my work between Codex and DeepSeek. Now I barely use DeepSeek, or never because Codex quota is enough after Sol

I hit Codex limits (20x account, never using /fast) on Sol Medium in about 2.5 days

Which plan?

I've had the same experience but at the same time I also find Sol's answers in conversation longer.

It's a step in the wrong direction but also token efficiency has become a focus relatively recently (just the past few weeks it seems like the zeitgeist has turned it's attention to efficiency) while work on this model probably started many many months ago. I would expect to see models released that focus on token efficiency in 6-12 months

no infonat all about the default and recommended effort?

Almost completely disagree. Slightly more expensive, but significant better on a per-prompt basis? For non-trivial projects, the former is a small linear increase, the latter is a (somewhat-)exponential(-ish) cost/time/sanity savings.

Reply on news.ycombinator.com