The Harness Is the Thing | Hacker News Reader

The Harness Is the Thing

scott-fryxell.github.io

209 pointssfryxell124 comments

Reading this really makes me wish that I had a slightly better workflow. I'm really soley dependent of Fable to the point that I don't use other models, and I've already sort of hit a point where I'm running into usage limits every week. I am really living on borrowed time — when Anthropic finally collapses their 50% usage increase at the end of August, I'll definitely be forced to switch my workflow. When that happens, I have a hard time imagining that I'll be sticking with a single model on a single provider.

> Reading this really makes me wish that I had a slightly better workflow. I'm really soley dependent of Fable

Good thing then that you're fully in control of your own actions.

A gift granted to us by being a fully grown adult that is also likely registered to vote.

Meta: There is of course a way to put this less snarky, but that doesn't slap people in the face as hard as they need to be slapped in the face to maybe one day start remembering that they have agency.

> when Anthropic finally collapses their 50% usage increase at the end of August, I'll definitely be forced to switch my workflow

I cancelled my Anthropic subscription until they fix how their models write and it's no longer unbearably annoying and obnoxious. The concise output format is a step in the right direction but I need a few months away from them.

Kimi and GLM models on Max reasoning feel pretty close to Fable. That said, even at their most expensive plans, a single one of them might not always be enough, while getting both of them for a year gives you a nice discount and isn't insanely more expensive than Anthropic. The problem there is that they could still easily rugpull you with token limit changes later, I don't trust any of the big labs not to mess around with those for any length of time.

Also most harnesses let you choose models per sub-agent. Like I can use Fable for running the main session and just tell it to use Opus agents for implementation in Claude Code, same with the Kimi and GLM models inside of OpenCode and other harnesses. The only problem is that the UI for controlling sub-agents usually really sucks.

Yes both Opus and Fable’s comment style is truly unbearable. I like the code result but it’s so annoying seeing it reflect our conversation as code comments.

On the flip side, a lot of the reasoning and decision making might have happened in the conversation which is more ephemeral than a comment. A future LLM-age source code repository would store the conversation along with the commit, for future reference by humans or LLMs.

Just quit reading the code then duh ¯\_(ツ)_/¯

I only like talking to Opus 4.6 in its default form. I have some pretty aggressive prompting strategies ensuring my language guidance rules are front and center and it really helps with Opus 4.8+, something went wrong with those models.

Also, OMP has a solid subagent model and I like it with some tweaking.

I've been using GPT-5.6 Luna for everything at this point. It is more than enough to do all the work I need to do. You have to lead it by the nose, but if you know where you're going it works really, really well.

It seems there are two distinct groups, one which is doing relatively well with Luna, Deepseek Flash and similar models while other seem to be satisfied only with the state of the art.

Depends on how in-the-loop you want to be. I personally delegate all the writing of the code to agents, but I maintain a clear mental model of the architecture, which I come up with by iterating and prototyping with agents. I can use Luna for all of this, although I switch depending on the task. It's nice to be able to throw a 1,000 word rough spec at Fable and get a personal tool that works perfectly though. I expect that as the models get better, I'll continue to be in the loop, but operate at increasingly higher levels of abstraction.

Having worked with both, it seems like a control thing to me. Either you're cool with Fable spitting out tons of code you'll never read, or you're cool with Luna doing targeted work while you manage the main work thread.

(I'm in the second boat so long as I'm responsible for the code I PR)

I'm ok with not-Fable until I need anything design-related: a nice HTML page, LaTeX typesetting, UI design. Sol is especially incapable of doing anything sensible.

Is there any alternative model with design sensibilities?

Did you add some skill for this? I've been using a frontend design skill with Sol and it's reasonable (for my purposes). I don't remember where I got that skill.md though, it probably was from OAI's own blog about this a few months ago.

Kimi K3 is amazing

fable is good at it but I find it uses the same slop in different variants every time. Even the words it uses for the different alternatives it proposes are the same (cartography, atlas, bench, signal, etc). At least you develop a good nose for slop ;) it becomes a struggle to steer it off the same path every time.

GP kinda explains why: "You have to lead it by the nose".

If you're happy to "lead it by the nose" you can do well with a lot of very low end models.

If you want to kick off a "/goal run until [complex verification passes]" and let it run for a week with minimal intervention, then not so much.

You can make do with cheaper models for long running agentic runs too, but it tends to require a lot of extra scaffolding and additional review steps.

I'm happily in both depending on what it is. Even Sol / Fable cannot do some truly novel stuff and if you rely on it too heavily you get detached from the underlying systems to the point that it's both uncomfortable and detrimental.

Yeah, it's about, do you guide it, or does it guide you.

People who know what they want and know how to describe what they want with sufficient detail, and everyone else.

I’ve just been using it, with a chat interface. Basically, as a “consultant.”

For me, and my projects, it’s been great. It’s made an enormous difference.

I guess my workflow may seem “quaint,” to many folks, here, but the end results speak for themselves.

I suspect that one vocation that could get heavily impacted by AI, is the consulting business. That’s where many experienced people go, as they reach their career peak.

In my last project (just about to ship), ChatGPT replaced a whole bunch of services that would usually be supplied by external advisors.

But these are also services that I would normally not be able to afford, otherwise, and would just have to “make do” with. This release will have a level of polish that I have would never been able to achieve, unassisted by AI (I had originally used “on my own,” there, but the reality is, it actually was “on my own”).

The thing about consulting is that someone needs to verify the output, know what questions to ask in the first place, and provide a throat for the client to choke in the event of an issue. Also, professional insurance. I could be wrong but in the worlds I live in professionally, accountability is still a thing.

Yup. There's some places that we'll need that accountability. I suspect that this may be filled by folks that act as "LLM brokers," using AI in the background, while dealing with the legalities, in the foreground.

But in my case, it wasn't nearly so exotic. The LLM helped me to do a much better job, preparing the App Store presentation, Web support, privacy policies, budget prognostication, and app glossary.

I have just had an extremely complex app, pass App Review, in record time (from going into review, to approval). No niggles or bounces at all.

I'm thrilled.

I'd be curious to pick your brain about the process you went through for this. My mental gap here is not even knowing what to ask for a process like this.

Happy to do so. Probably not something a lot of folks here would find interesting, but I'm easy to contact, from my Web sites. People here, often do that.

Basically, my needs are different from others. I'm not working on the next NORAD upgrade, much of my work is open, and the more ChatGPT knows about me, and the app I'm designing, the better. One reason I chose it, was because of this "memory."

TL;DR: I feed it just about every scrap of information about my project as I can. Source files, documentation, screenshots, videos, information about the organization, information about the target demographic, etc.

With all that information, it gives me very useful advice.

It created a great tutorial. I usually write way too complicated ones. It did much better.

In the case of the App Store stuff, it helped me to choose the right privacy report, generated the privacy manifest, and helped me to compose all the copy on the storefront.

I'll probably be releasing the new app, soon. It's already passed review, but I want to make sure that everything is kosher, before releasing. It came together so quickly, that I have the luxury of time. I just need to release before (or as) iOS27 comes out.

When using LLM's to help write things like tutorials, how do you reduce the verbosity firehose?

Even though I use MD/skills and other methods, I have remind models that they are not paid by the word multiple times a day.

Oh, I don't really do much more than have feedback loops, where I review the output, then ask for things like less verbosity, or a lower-level vocabulary, etc. I like to apply this at each application, rather than make a blanket rule, as each page may have a different audience.

For example, I have made a tutorial, which is meant to be a "quick reference," from within the app (Use Safari to view the page), but I am also developing a "walkthrough," to show possible funders (we're an NPO). The walkthrough is a higher-level vocabulary than the tutorial. The LLM deals with stuff like making sure to keep the glossary consistent, etc., but I like to have the final say on the output.

I'm pretty sure that I can force the LLM to use certain levels of vocabulary, through the .md file that describes the default setup, but choose not to do it.

I am still in that "trust, but verify" stage of my relationship with LLMs.

Pro tip: use 5.6 sol, and tell IT to lead luna by the nose with subagents.

Why do you need to involve IT?

"it"

Same, all I need is speed to execute editing faster than I could in vim. Luna is cheap, intelligent enough, and fast enough I don’t lose the sense of flow.

I feel the same way with Cursor Composer 2.5.

Sol is basically at the same level, but with better usage. You're not dependent on Fable. You could switch the next minute and hardly experience quality differences.

My experience of Sol is that it's very instrumentally competent but is a complete terrorist that will cheerfully do unhinged things that "technically" solve a problem. For example, I asked it to verify something in our integration tests and it responded by grabbing kafka details out of config and trying to use them. When I pointed out that this was not it, it did the same thing with a database client instead. Conversely, I'm pretty sure Fable or even Opus would have done something consistent with what the rest of the codebase does (consume an API or point out that there's no API to access in this case).

I don't have Fable at work but I'd probably use it for actual code if I did because not having to spend time handholding the model on this stuff and getting useful code first try is very useful

What I do notice is that when it doesn't have the info it needs, or when I don't tell it that I'll handle part of the process, it will indeed do everything to get the job done. Also saw this but find this a great quality, because I never ask it to do anything that would involve unhinged things. Sol and other models as well have just gotten so damn good. They can get just about anything done and usally do it well too. Maybe I was inexperienced in prompting, but I remember doing much, much more handholding with late 2025 LLMs.

IMO it's kinda fun to mess around with these multi-model workflows. I've been using paseo to orchestrate everything.

Just plan extensively and let Sonnet do the exec.

And Opus5 aggressive audits.

Once it has exactly your coding conventions and access to other code to copy bespoke patterns, a strong idea for what to do, then you can let it do the work.

You do the wiring, it fills it in.

Coding was never the work.

Sonnet 5 is the worst model of 2026. Literally just turn effort slider down on Opus, it's smarter, faster and cheaper than whatever Sonnet is.

Beyond that, I find this whole plan and build thing to be a pointless waste of tokens. If your planner made a detailed enough plan, then the cost of executing that plan is a just one turn more of cached tokens, and minimal time.

Meanwhile: switching agents, reloading context and building from the plan will easily balloon your token use and time. And any emergent problem that the dumb executor finds will instantly wreck the implementation because they're not competent at solving it. And if your plan is so perfect that there's no edge case then you're wasting tokens because your planner was one turn away from finishing the project via cached tokens.

Completely disagree (except the Sonnet bit, yes, it's degrading).

"then the cost of executing that plan is a just one turn more of cached tokens, and minimal time."

This is just not true at all. There's a huge gap between 'figured out the hard stuff' and 'rock solid'.

Dependencies, integration, corner cases, docs, testing, unforeseen issues, a lot of back and forth auditing making sure things are really tight.

Audits get diminishing marginal returns, but you have to do them until they don't find anything, and that's usually a few cycles.

So aside from the fact there is 'a lot of labour' - part of the plan (maybe the most important part) is documenting most of the trip-up scenarios. If you ran an experiment or two in the background your agent will 'discover' a few key odd things, you back those into the plan.

I'm 100% certain that this pattern works because I (and others) use it very successfully.

Hint: save your main context by using sub-agents to do grunt work - even in impl phase - farm out anything directly implementable without a ton of background.

Also - make a skill so your Claude can call Codex and visa versa and maintain long-running sub agents of 'the other kind'.

An Opus with 1M context window executing on a 'plan' that a Codex 'sub-agent' is executing on - ad a different Opus sug-agent is auditing hard ... that 1M token window is dramatically extended to 'many millions of tokens'.

That can work within Anthropic/Codex Pro plans.

I'm sorry, but just because you achieve results you consider acceptable with this method doesn't mean everyone does.

I don't work where we can ship slop. I don't work where PRs can be merged based on what the agents say. I work where a human has to read and approve and own every single line of code. I work where the stakes are actually high, so the cost of not using the best tools in terms of human time are big. A single turn around in a PR costs more in human time than the difference between deepseek and fable in API costs.

So, when you admit "There's a huge gap between 'figured out the hard stuff' and 'rock solid'." but then claim that the cheapest/dumbest agent in your arsenal is your go-to for "rock solid", I have to question the quality of your results.

Personally, "using plan mode" is a very 2025 way of using these tools, and I wouldn't be surprised to see "plan mode" be removed from codex/claude code/et al.

Realistically, I'm using the best models to think about a domain and problem (Fable High+), and I'm using a cheap daily driver with an advisor pattern (Opus High + Fable) to iterate through POCs, and I'm using human review to guide design. None of that is "plan mode", it's actual engineering. Then we decompose the solution, we stack it, and we use only really strong agents to build, review and refine.

This obsession with cheap agents leads to low quality outcomes. "Rock solid" deserves the best tools, and the "plan" will never be good enough. I'm going to be sending fable xhigh and sol 56 xhigh et al at it in adversarial review, why the heck am I cheaping out on the actual implementation?

And finally: my time costs way more than any of this. Cheaper models are slower overall and when combined with re-work time, are dramatically slower. I'm costing my company hundreds in my time to save a few bucks on the API bills. Nonsense!

Yours was the casual dismissal; and based on a misunderstanding of what can be achieved.

Based your arbitrary dismissal and unwillingness to even try to consider new patterns with which you may be unfamiliar - it may be difficult to communicate with you.

I have the advantage of 'certainty' because I have the evidence over many projects / team members.

We ship near perfect code.

In addition to the hints above, we do this at least in part by explicitly anchoring and testing requirements into several aspects of the code, and ensuring that known 'weak spots' are managed.

The 'planning process' ensures the requirements are mechanically anchored and integrated into tests, that 'proportional' documentation is applied, and that module, library and project level documentation is perfect (and mechanically validated where possible), which FYI is what solves most of 'context problems'. (That's another hint, if you have extremely good docs, you don't need to load vast amounts of code).

Yes - I hear you that 'time matters' and that 'the stakes are high' - consider that you may be talking to people where the stakes are just as high, or higher - but more specifically, this is not about 'saving tokens' or cost so much as it is using the right level of model for the task.

Use the best models for background research and planning, use mediocre models for execution, and mid-high for auditing - in other words 'use the right model for the right work' - and in a certain methodology, dumber models are appropriate.

FYI this saves you the ugly 'Fable' problem which many are encountering as it burns though Max plans. Don't 'automate' with Fable, it's the wrong model for that.

I could go on, but consider that there are actually ways of organizing projects and orchestration that work well.

If you find yourself wishing for more Fable at the subscription rates, just buy another 20x account? I'm currently on 3, spaced out on Sunday, Wednesday and Friday. Just as one is running out the next resets. Problem solved!

I feel the value I get far exceeds $600/m. It's a straight expected value calculation for me and I'm happy to pay. I wish they'd make it easier though - just sell me a 100x account for $1k/m and save the messing around.

what kind of workflow are you using the requires that many tokens? I rarely hit my limit with the $100 a month plan and I use fable pretty frequently although it usually acts as an orchestrator while the actual work is done by lesser models usually.

Nothing weird or exotic - just frequent daily work in two reasonably large applications, with a lot of testing and follow-through. I've found that as codebase size goes up, token usage does too - perhaps quadratically, and a complex back end application just costs a lot to work in, probably because it needs to map out so many pathways and interdependencies.

I tend to work in 5-10 or so parallel streams at a time so that also multiplies token use as a function of time. Much less and I'm waiting for it too much, much more and I can't juggle effectively. My new big project is of course to try to take myself out of the equation further and increase the parallel streams dramatically; pretty hard to get that right so still manual for now.

Keep Fable, but have it orchestrate other agents running different models (could be Opus, Grok or Kimi K3).

This is my current workflow: https://x.com/sridca/status/2093354491965735091

I even let Fable choose the appropriate model for the task.

Boy you should try model routing, I dont think using Fable for all the tasks makes sense to me

Fable is pretty good at routing to Opus and Sonnet though, especially if you ask. also, sometimes just because it decides that's what it feels like doing at that point.

It reads like a flex. Online equivalent of a driver of a fancy car who has to make sure you know it.

The GPTs are just as good and I can trust many tasks to open models. Autonomous coding is best with frontier class models but steered coding is fine with even DeepSeek V4 Flash.

Us fable to help you with the transition.

> Single developer projects can build to the caliber and consistency of large development teams.

Yet the simple blog website static page saying that looks very weird and broken on the desktop firefox.

How large should be a development team to make proper margins in 2026?

I was a little "what a weird nitpick" then I opened it on FF and yeah... that's awful.

Normally I'd just switch into FF reader view when it's that badly done but that doesn't work either.

I looked at it on my out of date FF and it looks fine.

I looked at it on firefox and it looks fine. I wonder why it's off for some people

Looks fine in my Firefox. The text size is smaller than in Safari, but it’s fine.

Could this be an extension? I already have a <meta tag name="darkreader-lock">, but I'm not sure what extensions are common in firefox

Large development team... that is exactly the problem. The more effort and code put into it the worse it gets. Peak web was one webmaster per site, doing it by hand.

Looks fine on ff, and the same as other browsers. You sure the issue isn't in your end?

Well, who knows, but in brave it looks alright, and ff for me has extremely narrow text with disproportionately huge images. I see a few upvotes so I assume at least some other users have the same.

Direct Actionable feedback. Thank you. The text is trying to folow the seven to ten words per line rule.

This is my first blog post with graphics. so your comment helps me guage proportions

> weird and broken

Weird, yes. Broken, I don't think so. Seems to just be a matter of taste.

If it's not rendering as intended across all browsers, I'd call that broken. If it renders as intended and just look bad, that's taste.

It's hard to say since the gp didn't specify what they thought was broken.

The overall layout is fairly odd & weird - but it's the same in Chrome & Firefox. There is one technical bug with the main body font-size - it uses some invalid syntax (should be invalid in both Chrome & Firefox) & Chrome seems to be accepting it (against spec). The rule:

  font-size: clamp( 1.125rem, 1.125rem + (1.333rem - 1.125rem) * (100dvw - 24rem) / (80rem - 24rem), 1.333rem );

Firefox drops it & falls back to the default body font, making the article text slightly smaller. But it's definitely not a layout-breaking bug.

Whats wrong with the font size exactly? Im not a desktop to check it but its look fine? A bit unusual but I wouldn’t say a clamp with a bunch of math is invalid.

for me it's just a slight reduction in the font size due to firefox ignoring the above - which doesn't seem enough of a difference to warrant folk calling it outright "broken"

It's entirely possible I'm misreading the spec on clamp() but as far as I understood it, clamp() performs math on a bunch of comma separated fixed values, calc() handles arithmetic operators - this seems to be combining those without a calc invocation, but maybe I misread & overlooked clamp arithmetic operators. Firefox just says it's invalid, no detail on why.

This problem is easily solved. "Check that the blog looks the same in XYZ browsers".

I have LLMs align stuff for me all the time because I'm too lazy. It takes the screenshot, changes the code, code auto reloads and boom, done.

Sure. However I have a strange feeling when I read all AI praises and how productive everyone is while seeing more and more broken things everywhere. Sometimes from the same very productive people. Otherwise we might have said that maybe broken stuff goes from AI-deniers.

If everyone is doing the job of hundreds, extremely productive and everything is so easily fixed, why everything feels so slow and broken, even so basic things? That's kind of the point. I'd expect nearly perfect websites everywhere, especially from "productivity" people who mastered the flow.

> If everyone is doing the job of hundreds, extremely productive and everything is so easily fixed, why everything feels so slow and broken, even so basic things?

This doesn't make sense. There were slow and broken things before AI. This was true even when the things were made by hundreds or thousands of engineers.

If an organization doesn't care about making their app or site fast or correct, then unless they have a truly ludicrous amount of free manpower (more than is available now with AI, because review and architecture are still bottlenecks), it probably won't happen.

AI doesn't change this. AI doesn't change the priorities of an organization, it just changes how quickly and cheaply they can build. And every org has a point in their priority queue where things are no longer worth it to spend resources on.

If Venmo didn't care about making their login page work well with Firefox (which, in my experience, they don't) before AI, why would you expect it to be better with AI?

I think what you're seeing is A LOT of moving fast that wasn't possible before. When you move fast, you break things.

Where it would take 6 hours to make a first cut of a static site, you can now do that in 15 minutes. Obviously that's not going to be perfect.

Yes, something like this is a good skill for me to add to the critic phase. This is my first post with the blog redesign, and I will admit I am 100% using hacker news as QA.

Excellent feedback, Thank you.

please upvote then, people are downvoting because they don't like that it works.

Do large development teams care about Firefox?

I've been very happy with Luna but my approach is "many bite sized edits" for which models basically hit saturation a year ago.

(I also tried the "let a massive model make massive changes" approach and am still psychologically recovering from the experience. The codebase may never recover!)

Also, Luna and DSV4 Flash seem to be on par now except Luna is faster and cheaper?

FWIW I run a clinical analysis backend and directly compared Luna with DSv4 Flash 0731 - DS was a bit ahead, and cheaper even considering it used 40% more tokens.

The exposure to Deepseek made me question the valuation house of cards built on SOTA providers. There are more companies producing competitive and useful models than there are companies producing jet engines for airliners, and not for the lack of trying. China has been trying to make these engines for decades and so far failed (their flagship C919 airliner is using CFM, American/French), but it has produced at least three competitive model companies within 3 years even though they're handicapped by their hardware.

It's simply not that hard, and diminishing returns will, in fact, diminish.

You reminded me of this recent post, which I found very interesting:

Why jet engines aren't made in China

https://news.ycombinator.com/item?id=48740971

That's a very interesting point of comparison though. I wonder why that should be the case? Why is it so much harder to make a jet engine than a language model?

Maybe with LLMs the iteration times are lower? Or there's more public information about technique? Or are jet engines just intrinsically a harder problem?

What I've learned in last week is that a harness is basically a while loop.

In each iteration you make an LLM call, perform some work (e.g. tool call), augment the prompt (append or compact etc.)- not necessarily in that other- and continue.

Until an end condition is satisfied. Then you break out.

It's two, actually:

https://minimal-agent.com/

This example has one, but if you add a second one around their example code, and take user input, and there's a basic harness already.

That's the most basic version of a harness. But a harness is really about automating context delivery to the LLM based on your use-case.

A pluggable loop with events and hooks.

Author states, “Single developer projects can build to the caliber and consistency of large development teams.”

When I, as a single person, can produce a project in one month that would have taken a team of four people three months to produce, why would I care about token cost? I’m now spending $500/month instead of $40,000 month to get the same thing 3x faster. $500 for a project instead of $120,000. (Assumes my cost, $40k is the other three people)

It’s a no-brainer —- use frontier all the time.

"A robot that's a tenth of the speed is still cost effective when it's a hundredth of the price"

-- CPG Grey, Humans Need not Apply. Released over a decade ago.

https://www.youtube.com/watch?v=CMFj75kBQlU

Have you laid those people off? You still have people on the payroll. If each one of them uses a $500 subscription, you are now spending $42k/month, not $40k.

Now think about that he is counting it like:

He never had to hire anyone in the first place.

He didn't have to make any contracts, deal with screening, background checks, recruitment calls.

All that money he didn't spend he can spend on maxing out token usage.

No, but the 3 of us can now accomplish a lot more than we used to. We spend around $450 a month on hosted inference plus capital / utility costs of around $100 a month in local inference.

And our 2 non-technical staff are now busy designing apps, so when they hand off something to be productised, it’s far more complete than the old days of a few Figma drawings

Are you sure you are?

Try laying some people off just to be sure.

I'm very un-motivated to do so since then I'd have more work to do, and I'd be making less money.

My bet is we no longer have the benifit of the doubt. and so taking capital and burning hard is not going to break through into consumer as easily as it did. So I extend the arc of time, tell people I'm a business, focus on giving customer as much free as I can, (probably add the harness to my product), and over time build an affectionate connetion with customers and then remind them I'm a business.

> use frontier all the time

Agreed. You may think that the task is simple, but a brainfart of a dumb model overlooking something can cost you more in time and effort (if you relied on the output of the wrong code and now have to go back to regenerate), that it just doesn't make sense to use non-frontier for anything but hobby projects.

LoRA adaptive learning using open-weight models and your own reasoning traces is the thing. The big labs have a mammoth job ahead of them if they want to compete with running your own model - they will basically have to give every single user their own persistent virtual machine. When it's all said and done, I think their only really moat will be as inference/hardware providers. Stripe buying OpenRouter was a very smart bet.

Can you explain what you mean with "LoRA adaptive learning using open-weight models and your own reasoning traces"?

You train an addend to the weights rather than the weights themselves. You limit the dimensions[1] available in this addend to a small fraction of the dimensions of the weights, making training cheaper.

When you have a few example chats you want a model to emulate - say you made it from your proprietary data, you can train any open model on that data in this cheap way. You don't lose any quality versus not using lora since the models overall knowledge won't shift that much due to your data anyways, so it's a waste to make high dimensional updates.

However, only in some cases is it worth it and equal in quality to just making a good retrieval system and exposing it to claude code or whatever. If a retrieval system over the same data is very difficult, or if the data simply must be proprietary, then you should go for it.

[1] technically "rank", but I'm simplifying

Running an open model locally on your own machine, and having it augment-train itself using your own sessions as the training input.

One will lose the opportunity to develop domain understanding if they do not get into the weeds of thinking through the problem.

I use Claude plan mode to do relatively small changes and even then I find that if I actually try and think through the problem and solve it myself that I find good metaphors that will aid future work, and I will discover tangential issues which are then important to look at.

Do you explicitly ask for metaphors?

I wonder if we need a list of things to tell AI to stay sharp, like this one. Sometimes I tell the model existing design is stupid and then it explains reasoning to me.

I don't bother, because again, the value is in me thinking.

This whole "a single dev can do what teams could" has been true for decades lol.

Plenty of devs developed anything from operating systems, enterprise software or videogames on their own.

I still see no evidence that this has changed significantly.

This guy knows harnesses!! I'm curious to know what your thoughts are on dsh(deepseek harness), where everything is a plugin. I read an article where the author described it as a motherboar in the Mobo combo purchases we use to do to save money, with the processor(deepseek), being replaceable by other llms but shipped with it by default. Did you evaluate it as a possible replacement? If not, why not?? In my nondev brain, it looks revolutionary, the equivalent of the x86, to the SoTAs attempt at keeping us in a walled garden similar to apple consumers.

If you are using any of the agents without config, do yourself a favor and invest in any config library that implements SPACE[0]

It will save you a ton of time and you won't be tied to fable costs

[0] Search, Plan, Assert, Code, Evaluate. Obras superpowers, Matt pococks skills, and the nori high autonomy skillset all have this built in

I don't understand why no one has tried to make a harness without full shell access yet.

It would be so much safer.

Is that the harness' job? It seems to me the best place for sandboxing is at the OS level (i.e. running the harness inside a container with correct access configured).

I would argue that it is the harness's job... Certainly for any harness which is targeting an end user.

They don't sell cars without seat belts, and it is the car manufacturer who has to do it.

Also, small nitpick but technically a container doesn't give full isolation compared to something like a VM.

Ok that's fair if it's targeting a non-technical audience.

But I think this will eventually be a problem solved at the OS level in a more streamlined way. I.e. there will be fine-grained permissions you need to approve to give an agent access to the system.

You want both. The harness and the sandbox do different things.

The harness says "You have access to tool X, Y, and Z, but not A, B, C".

The sandbox says "If you try to use X to access a forbidden resource, I'll prevent you from reaching it".

I haven't tried this but sounds like https://github.com/cloudflare/cloudflare-os ?

That exist?

"permission": { "bash": "deny" }

Or something equivalent in any agentic editor of your choice.

What does that achieve if the agent can still run processes on the host system?

what do you mean? the agent can't do it anymore if there's no bash permission

sandboxing?

That's what I though. I've got a harness that uses landlock. Might not be perfect, but should be good enough for almost all cases.

We are talking about the process but not the result. I think the results will speak for themselves.

It is what I also keep telling everyone. The tools need to be shaped after your work and shouldn't precede it.

More agentic than this. The harness is the thing / Wherein I'll grant Claude access to Bing.

i doubt it. someday harnesses get complex enough that you start asking models to interpret and modify them.

the pattern of layering of deterministic, probabilistic, deterministic, probabilistic. it's a strange pattern but it seems somehow natural.

Mine is more granular. I start with a scoping brief before planning. But yeah, I'm all-in in this same thesis.

I use Claude Code, Codex and opencode pretty much interchangeably. I am currently using Claude more this month because (stupidly) I paid for Max ($100) since I have a large client project.

I generally use larger models to plan. All my generated Epics have similar structure. All my repos have similar structure (https://github.com/brainless/akar and https://github.com/brainless/daftprompt are recent examples).

I barely spend time or thought in making prompts. I have a simple text file with a few combinations. They refer all the common files (README, AGENTS, DEVELOP, etc.)

All reference software is cloned locally and the docs mention that. The prompt templates then boil down to research mode (write Epic) or worker mode (write software) or review mode (leave review notes in Epic). That's it.

Many of my harness experiments are about text manipulation, text search, graph on text. Because that is what LLMs are - text processing systems. Cut parts of prompts, cut parts of response, cut parts of user's intent. Join, break into epics/tasks, run with LLMs, repeat.

Software ate LLMs too after all

Always was and always will be.

Heh. Interesting article.

> Single developer projects can build to the caliber and consistency of large development teams.

This has always been true. Good developers, like truly good devs, could run rings around a team of mediocre devs. It’s a multiplier, a team of 10 1x devs will get dominated by a single 10x dev no matter how much AI they use. Nothing has changed here, if anything it benefits the good developers.

> At the moment my rig is supported by two subscriptions (Cursor, Claude) that I can augment with Pi as needed.

This conflicts subscriptions with an actual harness, doesn’t bode well for the rest of the article…

> Recently I learned about prewalk, Can Bölük's technique that uses frontier for the planning phase and first task, then hands off once the pattern is set.

This pattern has been known for years and is not attributable to a single person.

> Exploration leads to a plan formalized into an explicit DAG (directed acyclic graph) task list. Then a worker takes over, focusing on implementing the DAG one node at a time. Once complete, I bring in the critic to simplify and question what was implemented. Often this phase will push back enough that the worker phase is revisited. But once satisfied, the critic gives way to a promoter, which is my reminder that a job is not complete until you've properly communicated it to others.

I’ve tried all these complicated workflows. In the end the best way to use LLMs is to give it some instructions, take a look at the code, and then ask it for changes. At the end, ask it (in a fresh session) to review the changes for bugs or incorrect assumptions and architecture. Rinse and repeat. Anything more complex is over engineering.

None of the rest of the article seems particularly interesting. Just more busywork.

> HUMANIST SOFTWARE DEVELOPER

Uh huh?

Is the article slightly naive? Sure. It describes a relatively uncomplicated multi-agent system (which is good). But frontier model handing off a DAG rather than a raw markdown to a cheap model seems like a decent idea. Ultimately, this is about cost saving, not peak performance.

> But frontier model handing off a DAG rather than a raw markdown to a cheap model seems like a decent idea

It's not

> Anything more complex is overentineering.

For general workflows, I agree. Doing some DAG system is just a waste of time if you don't have a grounded verifier for every node. If you are going to human verify, why make a dag of subagents and waste time? Keep the dag in your head, the way we all did before LLMs. The dag in your head is also much better.

For specific workflows however, even this is under engineering. If your specific task or family of tasks can truly be decomposed into multiple verifiable subtasks then you should spend the effort to build that system. It doesn't matter if it's technically worse than the other method because it is basically automatic and incredibly cheap. For complex systems this is a lot of upfront work, but if it is done, then the return is completely outsized.

I painted it black and white, but you can of course compose everything.

People are doing too much.

The author's product (Realness) heavily promotes the fact that it works without the use of AI...

... but clearly AI was used in the development?

That seems deeply cynical.

Everyone now, more or less, has a harness, and the beauty is that you can afford to have your own version.

am I the only one here who is perfectly fine with deepseek and qwen? generally I do all the planning in my head and a few sentences describing what I want is enough for deepseek to save me hours of work. Hell I get away with flash for most of it and use pro for some reviews.

Reply on news.ycombinator.com