Hacker News Reader

Comment by wilg | Hacker News Reader

Claude Opus and Fable are so bad compared to GPT-5.6-Sol it's ridiculous, their desktop client is worse, and the value is worse because OpenAI has been spamming discounts. They better get their shit together at Anthropic!

Edit: If you're mad I'm light on the details I have provided some in the replies.

View on HN

gpt-5.6-sol uses fewer tokens and OpenAI has been more generous with usage, but quality-wise I wouldn't say they are inferior to each other.

I have a Claude Code and OpenAI subscription so that I can use Opus/Fable/gpt-5.6 as I please, and the models are often catching things the other models missed. So much that I would significantly weaken my workflow if I dropped one subscription.

My best workflow at the moment is to create the initial plan with Fable (before review/revise-cycling with other models). From my own testing it seems slightly better at arriving at high-level ideal solutions after sweeping the whole project, projecting future needs, then coming up with good trade-offs like "by construction" correctness.

While mostly subjective, maybe the closest objectivity I have here is noticing fewer revision cycles needed with Fable-initialized plans.

Any model currently in 2026 is better than the ones of 2024. What is the quality difference really.

I mean i think if a developer has a good handle of the code the difference is marginal .

Unless we 100% offload the thinking to the model and act like a prompt manager. Maybe

For personal projects, I defer almost everything day-to-day to the models, so it gives me a good feel for what the models are good at.

Even then, it's kind of a wash these days between the sota models, and we're talking about maybe a 10% performance difference or something. But every once in a while there's the experience of one model spinning its wheels on a bug/repro/issue while another model comes in and one-shots the solution.

That make sense. Let me ask you this When the project reaches a level of complexity . Do you simply reach for better models or do you reengineer it or does the project scope stop at the egdge of the model's capabilities.

I am asking because in my personal projects after a while they becomes a giant messy ball of wires and i basically trust the model to untangle it for me , by the time it untangles properly, I run into my token limits.

Every time I try to use a lower tier model like Terra or Sonnet I regret it, so I just use the best one the first time to keep my sanity (it sort of works). Same with effort, you gotta just max it out (though I stopped using Ultra/Ultracode) and I never use fast because I'd rather work in parallel on more than on one thing faster almost always.

With Opus 5 / Fable / gpt-5.6 you can simply ask them to fan out subagents to look for ideal architectural simplifications and rank their findings by impact vs confidence.

You can swap out "architectural simplification" with performance opportunities, bugs, correctness, etc. I get the orchestrator agent to then itemize it all into a file where I can keep track of which ones I've implemented.

The results are pretty astounding. I run these right before my weekly limits reset for each subscription and the findings will dictate the secondary tasks I get done during the week.

It's definitely token-heavy. I'm on the $200/mo Claude Code sub and the $100/mo Codex sub.

But it's pretty clear to me that software engineering is more or less solved and all you need is enough patience + tokens to get what you want. I think 20 years of engineering experience more lets me save on tokens rather than unlock things nobody else can build.

An example of the scope of one of my AI-engineered projects is a iterm2/ghostty-like terminal app that implements its own pty session, parsing, rendering. It's almost 2000 commits right now.

That said, I have a specific workflow that isn't just a blind "ok now make it so a screen can be split into panes". I have a plan phase focused on coming up with ideal invariants and such. But I'm not sure anymore how much of that is useful vs just yoloing a solution and then paying technical debt in sweeps, like garbage collection.

> software engineering is more or less solved

But the example you give is of a terminal for which there are copius examples in open source code. How hard is it really for a pattern matching machine to do that?

If I was doing it I would start by forking an existing repo and I might even say then that "software engineering is more or less solved since the open source revolution"

But I don't work on things like that.

Pretty much every software problem we work on breaks down into steps that are solved in prior art.

Even if it weren't, if you're capable of explaining the context and constraints of your problem, then a modern LLM with effort=high will generally come up with a solution that's worth starting with because it's well-reasoned.

Moreover, you can start with the solution and then course-correct based on future information because refactoring is trivial with an LLM, yet human projects often ratchet into a local optimum because refactoring is too expensive.

I don't think "it's been built before" does as much work as it seems. I didn't fork a project. The LLMs reasoned about how to build the project from scratch using trade-offs that made sense for my needs, and they made reasoned, unsolicited deviations from kitty, xterm, and co, not just blindly doing what some ref impl did. Btw, it was still a lot of work because my project isn't just "kitty but swift".

Then the models went on to drive a well-reasoned incremental implementation of a system that lets me use the terminal running on my Macbook from my iPhone over tailscale with a decent scheme it came up with itself.

The point is that I don't really have to know how things work to build good software with modern models. LLMs can do things like read Linux source code and adversarially refine ideas such that the final idea is a good one. And my biggest influences on the project can be automated through the use of reusable markdown files.

Now, I'm at risk of downplaying all my years in software here, but I see the writing on the wall. It was only one year ago that I only trusted AI to do autocomplete.

In general, many of these stories of "one one shots it and the other spins on it" is down to trajectory divergence and can be achieved simply using another instance of the same model.

While just simply trying many times independently gets you the improvement that is due to pass@k vs 1, you can get huge improvements if on top of that, depending on your setting, you find a way to ensure some stochasticity by perturbing tool calls, etc and running multiple instances.

The general theme is, embrace the stochasticity rather than the leaky abstraction on top of it.

With modern LLMs, investing in this kind of harness tooling is much more fruitful than hoping for the best from the model.

While many basic instances of this are built in to the popular harnesses (much of cursors higher-than-usual success rate with older models was due to really excellent context mgmt), you can never beat one that is optimised for your particular codebase, infra and general setup.

Until last year or so, the context management needed varied too much at too coarse a level across different models and even model instances, but now they are all extremely robust in a much higher % of contexts and are thus way more amenable to developing context management tools for, without needing to do a research teams worth of evals.

Custom evals and harnesses are thus extremely high ROI now. We are finding companies needing to do less and less tweaks and getting much fewer regressions (you should have reg tests in ur evals) with every new usecase and every new model.

It can be really simple to start with: change your grep/rg that it uses to a script that does in effect "rg $@ | shuf".

More complex examples are: giving different subagents different tools, randomly failing tool calls, truncating file reads randomly, having a small model invent N possible failure modes causing a bug and appending that to N prompts and starting subagents from each - this all forces each to pursue different paths. $example_specific_to_your_company_setup is highest ROI though, since most companies actual failure modes are dominated by idiosyncratic API shapes and retrieval quirks that no usual harness will bother modelling.

Also important IMO to not assign any meaning or semantically interpret the CoT as an acceptance mechanism (it is ok to use it as a rejection mechanism e.g if you see it plotting a sandbox escape whether it eventually emits the exploit or not is not something you want to hedge). We have to resist the temptation and ensure we only interpret tool calls, codegen, etc in our evals and only think of the cot as "some output that pushes the conditional distribution" which may or may not semantically match the typical preceding tokens of the desired tool call.

Sure OpenAI has open source tools, and equivalent or better models, and lower prices, and they get along better with other agentic tools, but Anthropic has much better marketing and I guess that's what matters.

I don't know why this comment's getting down voted. It's absolutely true. I would also add that Anthropic seems to limit, compute, the more you use it because I'm on the Max subscription, and it just stops working for me.

I'll ask it to do something and it'll say, I tried, but I couldn't do it over and over again or some variation of.

But it doesn't do that at the start of my subscription, so...

> I don't know why this comment's getting down voted.

Because just calling something bad does not add a lot to the conversation. It's not thoughtful, interesting, or good.

Huh? Comparing model quality is something everyone is constantly doing here. If you need more specifics, it speaks in an incomprehensible way, stops working lazily, does a bad job and misses things constantly, and makes a lot more mistakes. I have both a $200/mo Codex and Claude subscription and swap between them when one runs out, and I'm always more miserable when I have to use Claude.

The Claude desktop app is also widely panned, as I mentioned, and for me this mainly is due to general UX and a poor remote control interface. Codex's connected machine support is top notch.

I also mentioned the value of the Codex resets!

The thing is, is it's not a subjective thing. It's objectively bad. I think that's what the story is or the appeal of the post. It's calling out, something that everyone else thinks is so great, but it's actually really pretty bad compared to the competition. Trust me, I used to want Claude to win, but it's not.

Because "$X is bad" is boring, especially when it comes to LLMs. I use Opus 5 and it's fine. It's more chatty and sometimes a little more difficult to steer than 4.x was, but I've found the code quality of 5 to be much higher.

Everyone has different experiences with these things (for example, I've never experienced what you describe), and "$X is bad" is not conducive to thoughtful discussion.

The issue isn't whether it's fine, it's that both Fable 5 and Opus 5 are worse to use for the job of programming than Codex-5.6-Sol in my experience. I am directly comparing these top tier models every week in my work, and many friends have left Claude in favor of Codex recently because of Claude's poor model quality.

One thing I do like about Claude is that the normal (non-Code) chat interface supports MCP, whereas ChatGPT basically does not.

It might be objectively true for you.

Opus 5 is fine for me and works better and faster on low and medium than higher effort on prior versions. Same as 5.6 Sol compared to 5.5 or 5.4.

This is just the cycle. If the OpenAI offerings are better and cheaper, people will shift there and then those discounts that made it such a deal will evaporate. Anthropic or someone else will entice people back with the discounts.

The whole thing is getting ridiculous.

Reply on news.ycombinator.com