Comment by derdi | Hacker News Reader

Comment by derdi | Hacker News Reader

Parent Full thread

There is no good reason for running this in-process. The maximally inefficient Fibonacci function in the article is a good demonstration why: Call it with a slightly larger number, and your agent slows to a crawl, with no way to enforce a timeout. Call it with a slightly larger number yet, and you might bring your agent down entirely, with no way to recover.

There is a case to be made for a dynamically evolving "tool server", but it should be a separate process. That would be more flexible for other use cases too. For example, multiple independent agent processes could talk to one shared tool server. Like a blackboard system, more classic AI!

And if you really do want to evolve the agent itself: As the article observes, its entire state can be serialized. Nothing is gained from hanging on to one particular agent process. Serialize its state, ask the tool server to kill it, rewrite its code, then start the new version and replay the state.

Lately I've been adding a repl service to my docker compose configs so the agent can easily send fragments of code for execution in the project context without incurring the clojure startup costs each time.

So cool to watch the AI get into a tight learning loop when it has access to all the internal data structures.

Clearly, you’re not going to do it for just Fibonacci. And yes, you can always rewrite the agent code, quit, and restart to modify the agent. This is just like using an LLM to modify Pi and then restarting, which is done today. But being able to rewrite code in a live, running system is useful for all the reasons that Lispers have done it that way for 50+ years. In some cases, you can’t just serialize your state and reload it easily. Sometimes, for instance, you might not even have access to the whole state because some of it resides on a remote server.

> Sometimes, for instance, you might not even have access to the whole state because some of it resides on a remote server.

Not sure what you mean. The system I outlined is one where some "state" resides outside the process in a separate server. You don't need to serialize that, you just need to serialize the information to need to reconnect.

And my first point is even more relevant the more complex/distributed/brittle you make the whole thing: The more important it is for some specific process to stay alive no matter what, the less you want to live-slop code into it.

(Edit: Yes I'm aware of the live-patched space probe story. Human live-patching is not the same as letting an LLM try to one-shot the correct patch.)

Cool, you do you.

Reply on news.ycombinator.com