It means that a LLM side tool (bash) can expose larger (and structured) outputs to Codemode than it normally does when that tool is executed straight to the LLM back as text.
16,526 karma · joined April 2, 2010
It means that a LLM side tool (bash) can expose larger (and structured) outputs to Codemode than it normally does when that tool is executed straight to the LLM back as text.
In a world where brain and hand are on different machines, getting the bash hands to reach back into the harness brain is something that requires a) putting tools in its hands that it does not know about b) are tricky to set up, usually involving some sort of socket based back channel.
I tried this quite a bit, by having pi be always there on the hands side, but it causes a lot of complexity and the LLMs really do not understand it well at all.
It is not from my perspective because Codemode runs in the brain, and bash necessarily runs where the hands are. So if the hands need to reach into the brain, I need to set up a communication layer from the hands to the brain.
{ "extensions": ["-builtin:codemode"] }
Is all that is needed to turn off codemode entirely. And -builtin:mcp would independently get completely rid of mcp.It becomes much crummier when hands and brain are on different machines.
Because the models are trained on JavaScript for code mode. You get away with way fewer instructions. They also want to be able to express concurrency and that works very well with the Promise global.
But a big reason is that code mode runs on the harness side so bash is a tricky target in particular.
That greatly depends on your agent design. If you give a user a full sandbox then you're going to be in a position where you probably need to snapshot it. But there are plenty of agent designs that are not using full VMs and for those the state story is way easier.
We tried so many things. At one point it pulls in so much more complexity. At one point we had half of automerge's proxy system in there. In the end we felt like this is a reasonable line to draw, but we will see!
The point of Pi is to be minimal but also follow what the models need. We were pretty outspoken that models need code execution, and that's why Pi to this day has a very small set of tools available. However as more and more training with these models abstracts even over toolcalls themselves with code mode and similar things, it requires changes to Pi.
Mario and I talked about this last week if you want to know our thinking: https://x.com/pidotdev/status/2104510506627121451
And yes, that's why there is no Jev tool in Pi either.
Codemode as a mechanism can expose non LLM functionality to the coding agent. In that sense, Pi does not have a tool for Jev or other classifiers. It just now makes it easier for the agent to utilize it in the same way as it's otherwise quite creative in using bash.
Codex in particular is using responses lite internally and relies on codemode for parallel tool calling. So codemode was a given.
Jev on the other hand is new but it's not the first type of model we had troubles with supporting in Pi and we looked at how to make that make sense. The internal pi-ai SDK supports image generation and classifier models, but without building an extension it was never possible for you to utilize it.
So there was a while functionality of Pi that few people used, because there were no obvious ways to hook it up with the coding agent. Codemode also allows us to close that gap.
And once you have codemode, modern MCP can work quite well if the servers cooperate.
Some bespoke slop :)
Codemode is a way for the LLM to orchestrate harness level tools. The reason this happening now, is because the models by the labs are increasingly trained on this. Codex for instance in responses lite requires codemode to even perform parallel tool calling.
If we fail to explain it, then we need to do a better job explaining it :)
This is not unique to ACs. Anything on the outside of a building requires your co-owners to agree.
Author here: maybe I wasn't too clear about this, but I have seen people direct their anger about about what AI does to their jobs at people who themselves are not in a situation to do anything about it (eg: line managers or leaderships in businesses who are trying to roll out AI based coding tools).