Hacker News Reader

Comment by throwa356262 | Hacker News Reader

Question to HN:

How many of you actually need the SOTA level intelligence of Sol and Fable? What kind of tasks do you use it for, and what did you do 6 months ago when SOTA models were as intelligent as today's B team?

The other day I tried a 31B model from a Canadian company and felt this is good enough for 80-90% of my tasks.

(For the cuorious: North Mini Code. Free on OpenCode/Zen right now. No affiliation :) )

View on HN

For the particular task I'm working on (a mathematical task in validated numerics), even Sol has generally just repeatedly given up. I asked it for the main problems it could not solve, gave them to Fable, and Fable solves them, every time.

There is a clear hierarchy here and Fable is well and truly at the top for very tough tasks. It just sucks that Anthropic is so incompetent at delivering a cost-effective experience without the uncertainty of it getting pulled. It doesn't matter how strong their model is, customers will turn away.

For ordinary coding (making webapps, frontend and backend), the non-SOTA models do fine, but the SOTA models have better judgement and need me to intervene less often, allowing workflows like "Go and pitch me a solution to this problem, then go build it". 6 months ago, I would give lower-level tasks and read more of the code myself.

So I don't "need" the SOTA models, but they do help considerably. And I am also working on some other projects that may have been outside the reach of the earlier models entirely.

Subagents are a godsend. Frontier models for initial planning and spec writing, cheaper models for implementing from the spec, frontier models for review. Frontier models to help with research. The availability of cheaper models is great, since it gives more price points for different tasks.

Fable is the first one I found could bring brainstorming concepts closer to what I was striving for instead of adding in clutter and defocusing.

... and then it stopped having that ability to build on create concise, accurate summaries and expand on them. Same problem with GPT 5.6 Sol. I even tried and old prompt in the (more nerfed) recent version. The core initial thinking ability lowered, and I'm getting the same over-metacognition and epistemology as it tries to pull at threads in the brainstoriming process again.

The sort of thing Fable and Sol excel at are long horizon tasks. The sort of thing I have been using them for is migrating large numbers of repositories to new tooling simultaneously (adopting new linters, enabling dependabot automerge, rolling out mutation testing).

Some of that can be done mechanically or trivially, and Fable knows to write a script or deploy Sonnet for those instances, but other times there are complications that need to be overcome that need to be escalated. Then there are patterns that can be picked up in large migrations and fed into template repos or tooling.

I won't use Fable for everything, but if there is ground to be broken on a new concept, being able to build a prototype with Fable might be useful.

I also have some substantive migration tasks such as replacing a static front end with solidjs or moving from NLL to Polonius that I would like to use Fable for.

It certainly feels like over the last fortnight it has enabled a substantial amount of transformative change in my codebases.

For my needs, Opus does these kind of "long horizon tasks" well enough already, with additional tooling.

As long as you are chunking out the tasks generated from the plan, you can manually (or write an orchestrator to...) give the component tasks to agents that pass along inputs and outputs per the dependency graph derived from the plan.

You can write this plan yourself or review it with the agent. Chunking tasks out of the plan like this has the added benefit of being able to swap for a different model when the time comes (looking forward to Opus-level models I can run on my consumer card...)

Not really convinced using Fable and trusting the harness to orchestrate for you is worth the intelligence upgrade. An understanding of an high level implementation plan of your task is also necessary when working with colleagues who rightfully quiz you. Especially since, at least for my work, there isn't a lot that Opus struggles with.

I've been chunking out my plans like this for months, and even maintain a port of Get-Shit-Done https://github.com/LoganDark/get-shit-done to the Jujutsu version control system because that was the best way for me to make the most of Opus.

I don't need that with Fable. I can give Fable a task of any complexity and it will spend over 8 hours in a single turn if it needs to. And then the task will be finished.

For example, Fable is far better at helping me keep that port updated with upstream's agentic pace of development. With Opus I would be spending all day working through each merge step by step. With Fable I can safely nap through it.

I don't know exactly how to communicate that difference other than that Fable just seems to make better decisions more consistently and more reliably to the point where I feel confident trusting it a lot more.

For normal building work I use Opus to plan and GPT 5.6 Terra to build. The point is, these are not normal constrained building tasks.

Perhaps I should have had more faith in Opus's ability to complete tasks like these, but it just hasn't been my experience to date. Opus and 5.5 work great within constrained user story type tasks, but I just don't have confidence in them to complete a large unbounded task like: "find out why the shared dependabot automerge component isn't working on this one repo, then formulate a strategy to roll it out across these 90 other repos, ensuring that Python, Rust, Typescript, go, sensible, and GitHub actions are all correctly updated. Address any CI failures, codex and codescene review comments in your PRs, then merge on green."

> What kind of tasks do you use it for

There are some bugs that just seem to elude Opus and GPT 5.5. An example is an Elixir/LiveView app that I wrote for personal use that basically gives me a full tmux in my browser from the system it's running on (very useful for VMs or when on-the-go on mobile and need to run a quick command or two). On mobile the scrolling is complex and janky and even Fable struggled to figure it out, even with me spoon feeding direction.

> what did you do 6 months ago when SOTA models were as intelligent as today's B team?

The bugs didn't get fixed. I tried, oh did I try. New bugs (worse than the one fix) introduced, other things breaking, etc. I just lived with the bugs or fixed them myself.

Personally I apply it every so often to audit codebases but it’s mainly sonnet and Gemini for most tasks. That and a harness for an agent to keep iterating on a goal is perfect. I’m more concerned buy getting hooked on the 50% increase on all models, not just Fable.

Plannig and review. You want the best plan and when its done you want to catch all the bugs. Implementation can be almost anything.

Of course you can do the planning and reviewing by hand but sota models do a pretty great job, especially in review (please still read the code).

ml research, both brainstorming and running experiments

eg I'd throw a hypothesis at it in the evening, and overnight it would write the code, do a sanity check, start a run, monitor the metrics, identify and fix a bug, propose a new hypothesis based on the results, write the code and start the second experiment, etc

still not that good at generating ideas or drawing conclusions, but much better at criticism than opus

even the api price is not that expensive for this

Reply on news.ycombinator.com