Code mode yields a 99.2% cost reduction in our systems | Hacker News Reader

Code mode yields a 99.2% cost reduction in our systems

agent-swarm.dev

62 pointstarasyarema69 comments

A maxim I’ve arrived at working w llm every day is “put deterministic things in code, non deterministic in llm”. I do wish the harnesses would be more helpful in this regard. For example I’ve seen tons of scheduled jobs that people wrote in Claude/copilot/etc that could easily have been scripts. They aren’t scripts because the author doesn’t know how to script and they stop at the point the llm does what they want. It isn’t hard to then tell the llm “make a script to do all of the tool calls this prompt needs and then pass the result back to the llm for this non-deterministic part”.

I’d expect the llm to write the script and run it.

The tooling can remain generic and token spending focused.

The way I try to illustrate this to my peers, in the context of automating with llms, is to "do as much of the deterministic work as possible before and after involving an agent". Tbf this is largely a restatement of your comment;

Ie instead of

"hey agent, use the api to get X, then do unpredictable mutation Y to it, and email it to me",

it should be;

"Scheduled task runs code to get X then place it in workspace. Task then starts agent in workspace and is prompted to do mutation Y to it. Post-flight code then sends final product."

The difference here being that the pre/post flight stuff is called deterministically, rather than called by an llm.

Preach!

Dedicated Infrastructure > automation > ad hoc scripts > non deterministic llms from a dependability perspective alone. Using an LLM to do something that prettier could have done is analogous to using a 3d printer to nail up a fence.

An absurd example is how harnesses like Codex, Claude code have no 'cut/paste this range of chars' affordance so if you want to move a block of code the LLM galaxy brain rewrites it in token space in the new file.

It ~can~ be done using standard bash utils but there just isn't a kind of standard pattern for 'move this text block via CLI tools' so the LLM and human both overlook it

Anecdotally, when I see coding agents preform this action- I see them using bash. IMO less tools is better, if the agent has a shell- so not having a dedicated cut/paste tool is good.

Worse, the content subtly mutates during that transformation. It's really rare to see that happen with anything affecting execution, but I see comments lost/created all the time.

I am still a bit surprised that this is not more widely used. Normal inference + tool calls is effectively composition via serialized natural language, whereas Code Mode / Programmatic Tool Calling is using purpose built languages (aka programming languages) for composition and concurrency. It somewhat feels obvious to me that the latter is way more token and latency efficient.

In the world of AI hype, everything is to be learned again and we are going to see many many breakthroughs like this. You can see this shift already in the tone of conversation in HN for example.

The basics of computer engineering is now novel approaches. We are discovering why programming languages exist, starting with a worst version of Cobal running on a very expensive and probabilistic VM.

We will shoehorn every square, rectangular, and all forms of peg through the AI hole.

to be fair, people are also discovering why vaccines, democracy and human decency are desirable traits; so it's not just programming in the age of LLMs.

There's clearly a paradox that happens when people have to face things they treat as essentially magic because the magic has wiped away the pain and suffering that borne them. So too will be another age when people have to relearn everything the LLMs do for us.

"You know, as impressive as these systems are, they sure are bafflingly stupid. I am wondering if perhaps there is more to human intelligence than technical mastery of human language."

Drew McDermott, 1976, "Artificial Intelligence Meets Natural Stupidity": https://dl.acm.org/doi/10.1145/1045339.1045340

  I wish to rail against a pervasive sloppiness in our thinking, the tendency to see in natural language a natural source of problems and solutions. Many researchers tend to talk as if an internal knowledge representation ought to be closely related to the "corresponding" English sentences; and that operations on the structure should resemble human conversation or "word problems". Because the fault here is a disregard for logic, it will be hard for my criticism to be logical and clear.

Note that 1976 had some cool AI, including theorem-provers and story-tellers (Knuth wrote a story-telling program, I think it's an exercise in TAoCP). Yet the underlying theory of intelligence was "intelligence is logical processing of symbolic information"; this is obviously inadequate, but the ideology of analytic philosophy is a helluva drug for a scientist. Likewise with the 21st century "intelligence is about correctly predicting things based on past information,"; this is obviously inadequate, but the ideology of big data technocracy is a helluva drug for a scientist.

TFA suggests that it already is:

> The one-line takeaway

> This already ships by default in every claude/codex/opencode session's system prompt

The article is so painful to read.

Yes and it’s also something that was already known and recommended. It’s not “code mode” or something revolutionary, it’s just instructing your LLM to feel free to write scripts to aid it in completing the task.

It’s very useful and I use it all the time, and in the end you can distill it into a reusable toolbox, somewhat similar to a collection of markdown files with project knowledge.

still most harnesses and AI systems do not use it, adopt it, or reference it enough...

Because it was probably written by an LLM.

Use a LLM to summarize it.

>Use a LLM to summarize it.

GIGO?

Relevant Slavoj Zizek talk: https://www.youtube.com/watch?v=qj4cxK-nOTw

Love this guy, you should read the jokes book he has, pure gold

username checks out? :D

+1. I think there's an interesting idea here but an AI slop writeup on an AI slop website is very tiring

We ultimately switched to this approach with great success for our own product. We stumbled on its success a little by chance.

We have a visual designer and for LLMs to interact with it we originally built a tool per manipulation operation type (e.g. add, edit, delete item - for various item types).

We actually already had a JavaScript API with corresponding .d.ts file for scripting inside our product and one of our clients asked that we also expose this as an MCP tool. I figured sure, should be quite quick and easy, and with Claude Code's help I managed to do it in a single afternoon.

We then found that the LLMs way preferred reaching for this tool, managing with it to get their tasks done with fewer mistakes along the way and in much smaller time frames.

After seeing this and doing some more validation (I've also read that Cloudflare article), we ditched the other tools completely and made a cheat sheet for the LLMs on how to use our API.

Because our API returns decent error messages including stack traces, even if the script fails the LLMs have no trouble making another script to fix their mistake and carrying on from where the error occurred.

In hindsight it really was hardly surprising as LLMs are already aiming to be as good as possible at coding and with JS being so popular I imagine it's particularly good at it.

yes, typed "sdk"s are key, and indeed they are so good at js

we did some iterations specially on the typing and error messages, as you mentioned, that help a lot in the reliability

Yeah, this has been my biggest contention with the models everywhere paradigm. I will definitely use a supervisor pattern and advisor pattern for simpler models when I just want to throw something together interactively with Fable and a few sub-agents. But for anything I am doing repeatedly, I built a deterministic orchestrator to run the steps and I have a hard rule that the skills that run from my repo can only ever be a thin shims calling to my central system so that rather than having agents enrich a person or call an API or do research they fire off predefined multi-step deterministic plays that just might have models for classification generation summarization and/or review.

In addition to the reliability and cost benefits you also then get shared capacity broker capabilities so if you're only able to enrich so many people or only capable of doing so many CI runs if that's all done deterministically you can have some intelligent orchestration in your main server to manage that limited capacity across all your agents that are trying to use it at the same time.

The problem is that when the LLM writes ad-hoc code how can you trust it?

For example in my Liveclip MCP server I'm working on (not released yet) I have table manipulation type tools

So for example a couple days ago Claude made a combined ranking score for my Youtube Shorts analytics -- it did all these tool calls in the same turn

First it normalized the % values into numbers

{ "destination_col": "G", "key": "cinemasleepstories_temp_2026071702", "pattern": "%", "replacement": "", "source": { "col": "E", "row_start": 1, "row_end": 6 } }

Then it made the composite score

{ "dest_col_start": "H", "dest_row_end": 6, "dest_row_start": 1, "expr": "round((likes+1)(followers+1)stayed, 1)", "source_key": "cinemasleepstories_temp_2026071702", "sources": [ { "col": "C", "name": "likes", "row_offset": 0 }, { "col": "D", "name": "followers", "row_offset": 0 }, { "col": "G", "name": "stayed", "row_offset": 0 } ] }

And then it set the new headers

{ "headers": { "G": "Stayed (numeric)", "H": "Composite Score" }, "key": "cinemasleepstories_temp_2026071702" }

If Claude just said "let me write some pandas against this CSV" the workflow would be a lot more iffy and generally uncomfortable/ephemeral

A python script is now more iffy than a LLM going through JSON?

Yeah, of course. The MCP server tools do use Python in the backend but just look at the affordances it provided in the args -- named columns, a fixed match/replace API, a way to specify the range of rows. The LLM is not gonna create a little library with these affordances in an ephemeral REPL it's just gonna YOLO some pandas code that you hope did what you wanted and addressed the right columns, had the right types, etc

I feel like I am having a fever dream. How is an LLM that is trained on endless examples of Python, a well defined language as well, going to generate worst code than some random JSON interface?

Have you ever tried analyzing a CSV in an LLM? Maybe then you'd get what I'm saying

Heck for a lot of questions the LLM won't even write Pandas and just try to eyeball the arithmetic and be like "Yeah March 3 to March 7 values add up to..."

And often -- without using code _openly_ or in thinking -- it is wrong because it is just a "guesstimate" at best due to the nature of how LLMs generate tokens.

The difference is a yolo’d script needs to be verified everytime it’s generated. So it makes sense to generate it and verify it once (a cli command or mcp call).

I think OP’s example is bad. As that (if I understand the example correctly as I don’t really know what liveclip is) sounds like something worth generating a script for once and reusing it. However, if you’re looking to book a flight, you would prefer an airline cli or mcp server or a nodejs script using puppeteer to automate the browser or even some airline sdk to buy a ticket?

What doesn’t make sense is generating a script to achieve a task without clear feedback loop for verifying the script correctness. In OPs example, there is no way for the LLM to very the correct final result since it’s unknown. It can “code review” the code looking for bugs, but bugs in scripts happen.

we have a way to see created scripts in a dashboard, and in fact they are not ad-hoc only, i.e. it can create "saved scripts" that it can re-use, iterate and so on.

Also not exec happens in a sandboxed env, which means it might not have access to the fs by default (it has to agent-fs, which is a layer we built on top of s3 for agents)

Yeah I guess the dichotomy between 'tools' and 'code mode' isn't really as clear especially if the tools themselves are intelligent and truncate long results and have start_range, end_range type arguments. So the gist is about 'what part of the output needs to reach the LLM' and a 'code mode' approach can help orchestrate that just like good tools can

exactly, I love how the cloudflare article puts it in fact

Mine is forced to always start with writing tests (unit, e2e, smoke), then the script (with eager/early fail modes). I have yet to see this approach to fail.

That's obviously a part of the broader picture Claude always runs inside bwrap sandbox, any work involving python/nodejs must happen in the dedicated devcontainer, standard operations are reused via Makefile, Ai can use dedicated account in the developed $thing for testing in live, hooks governing tool calling are brutal, but it works.

For me at least.

Hello all OG here. For anyone coming back to check the comments, thanks a lot for all of them and the feedback!

I did a re-write of the post to remove the sloppy parts and making more clear some of the estimates and assumptions we made in the post.

Also linked this thread in there, so that anyone getting to that post can see all the interesting pieces mentioned in here.

Till the next on HN!!1

I could be misreading this, but looking at the numbers, it seems that--despite the claim "we didn't estimate this"--basically everything in this is an estimate. It's a little hard to tell though, because the generated prose seems to occasionally contradict or repeat itself.

I read it like that: we haven't measured this YET until now, and now we did

thanks for the feedback!

it's really hard to measure some times. for example, once of the things we have seen is that the specific `workflow-triage` schedule we have have been running much faster and cheaper since we moved some parts of it to use scripts. From our analytics we saw times from ~5min before to ~1 or 2 min now. And in terms of costs we saw +50% reduction.

Why not 90%? because even if it uses the script, that's the "repeatable" part. we still have a reasoning around the output of it, and then deciding how to escalate, which removes the 40% reduction. Still 50% is nice for something we run daily.

Codex did this for me by itself when i asked it for some daily automations.

it just wrote scripts.. scripts are coming back

Cool but isn’t everyone already doing this? Surprised this is what I saw at top of HN this morning.

I did not expect it either tbh :D

But I believe it's not that common as it seems, only one actually trying to push this is executor.sh for example

Hi there!

We've been building https://github.com/desplega-ai/agent-swarm in the open for a while, and one of the things we wanted to do since the start is find how powerful it can be to push for a code mode like env for the swarm.

We tried it and we managed to reduce up +90% of token costs by using swarm scripts (the code mode variant we implemented) for some of our schedules.

This is game changer, as we are able to

1. Run those schedules faster 2. Cheaper 3. More reliably

Been thinking a lot now on how we should prompt and change the default templates to force the agents to build on top of this. If it can be re-used or it's a recurring thing -> scripts.

Thoughts? Have you seen this type of improvements in your setups?

Cheers,

    We didn't add this for the post. It's just what the swarm already does, and it's the same machinery behind Script Workflows.

    What we hadn't done is measure it against Anthropic's own yardstick, with our own production data.

You say here that "since the start" you wanted to do a "code mode like env".

But the LLM that wrote your blog post says your system was already doing this and you just hadn't measured it...? How is it that you managed a reduction in token costs by changing nothing?

And can you provide a human-written explanation of the experimental methodology that gets you this miraculous, literally unbelievable 99.2% reduction in tokens?

Hi, the blog post was written assisted with our own swarm, based on the experiment.

Indeed we had the concept of workflows from the start (like n8n), but the point of the scripts is type safe executions that the agents can write, with access to APIs, MCPs, etc.

E.g. this PR from ~10 days ago https://github.com/desplega-ai/agent-swarm/pull/934 was going in this direction, offering a way to defined type safe connections to able to use more the scripts instead of adding MCPs or tools.

e.g. offering a way to do `ctx.api.gmail.searchEmail` coming from an OpenAPI spec.

The whole point of this blog is the "dumb" realisation of the following:

If you have an agent (say Claude Tag, or some other fancy Slack bot) that you keep asking the same things (e.g. check DataDog error and correlate PRs on a timeframe) and the agent does that process agentically, you are wasting tokens.

Now, if you give a way for the agents to write these scripts (code mode) the next time a task like that happens then essentially you just run that and you get easily 90% cost reduction in THAT process. Not globally, obviously.

No offence but, “if you find yourself doing the same thing over and over again write a script for it” isn’t exactly novel thinking in the software field…this was the same process when it was a human doing it, we just measure that in time instead of tokens

yes, 100%

but it's not that simple. the whole point of this is that if you have an AI system that is in charge of this, what you need is a way for it to build those deterministic software based parts, while maintaining the AI routing and reasoning.

with time, our internal swarm has been building those "internal" software that is tedious and boring to build, where it makes sense.

e.g. we also have a "pages" thing (like claude arifacts) that it uses to join scripts, html and other components to build dashboards for us

There might be something valuable in this article, but if the you couldn't be bothered to write it yourself, why should we be bothered to read it?

This is based on a multi-day convo I had with our swarm. I might have not wrote it myself, but it's based on human interaction and thought process.

Any recommendations on how I could make it better next time, so you do not feel like wasting your time (which I get and feel, dont get me wrong!)

Indeed it's the HN variant of "I asked ChatGPT and it said...."

I wish there was a way to tag this type of the posts, but at least my HN client allows to tag the authors.

I'm sure there is some valuable information in the article, but the LLM-generated prose is unreadable.

Look at https://randomlabs.ai/blog (Slate), they're exactly that and you can play with it

LLM-assisted websites yield a 99.2% trust reduction in me

I get it, but you got it right. It was assisted, not just AI slop. Also nice that it's not 100% :D

codex CLI is already leveraging this internally. instead of performing raw MCP calls its internally using a js REPL to do that. elicitation is then propagated for approvals independently / it's interpreted as regular tool calls for operators etc.

so if i were to tell an agent to move the contents of a confluence doc into a file, it would do so without even reading the confluence page - in theory.

Yes, that's the right direction codex is taking 100%

Now our idea is to offer this to be harness agnostic (we support codex, claude code, pi, opencode, even devin). And you (as the one deploying AI systems) should be in control of the AI, the execution and the data. That's our goal

you lose the per-call approvals though. one script hitting ten tools is harder to gate than ten separate calls

yes, that is true.

in fact we are thinking a lot about this, not on a toll based, but rather on a role based.

giving access to specific tools to specific agents, how that works and evolves, and how that links w the humans using it

makes sense

Then the agent runtime should be happening in a sandbox, where policy is enforced by a gateway external to it. Bound the agent's autonomy based on of the affects the agent's actions. Approvals should be made into a contract before the agent runs.

I am once again asking for people to ask Claude to reformat their article text to ASD-STE100 so that I don't have to read slop

will do next time, thanks!

This quarter’s clankerisms: “Honest” and “vibes”.

The tech here is cool; good reminder to push for more scripts in general from the agentic tools.

thanks!

How’s the agent swarm working? Do you have like a sample startup you run using it?

We run it ourselves since nov last year (~20k tasks since then, note we are 2 people and bootstrapped).

Then we know a few companies using it w +1k tasks x day (~20/30 people actively using it daily). Then we have some anonymous telemetry too, we are seeing more adoption, but as it's a toggle we only know about the companies that talk w us directly / contribute to the repo.

The type of startups, we generally see tech adopting initially due to the technical deployment required (e.g. k8s), but then tech managers, product, marketing and ops starting to use it more. Connecting integrations, and the Slack integration is super good, works as if it was a peer.

I think ECREE applies. "Extraordinary claims require extraordinary evidence" Nothing is 99% effective. This reads like a marketing piece.

Could have been a one paragraph prompt and csv. Had to be a few pages worth of sloppy slop.

Token savings are great, but I wonder whether the biggest long-term benefit is latency reduction rather then API cost,Faster feedback loops can easily outweigh a few cents saved per request

Reply on news.ycombinator.com