You can check out our super rough version here, been building it for the past two weeks: gateway.aci.dev
What I was talking about here is different though. My agent (Smith) has an inversion of control architecture where rather than running as a process on a system and directly calling tools on that system, it emits intents to a queue, and an executor service that watches that queue and analyzes those intents, validates them, schedules them and emits results back to an async queue the agent is watching. This is more secure and easier to scale. This architecture could be built out to support safe multiple agents simultaneously driving your desktop pretty easily (from a conceptual standpoint, it's a lot of work to make it robust). I would be totally down to collaborate with someone on how they could build a system like this on top of my architecture.
Very interesting! What kind of use cases are you using your agent (Smith) for? Is it primarily coding, or quite varied across the board?
The agent itself is designed to be very general, every trace action has hooks that can transform the payload using custom javascript, so you can totally change the agent's behavior dynamically, and the system prompts are all composed from handlebars templates that you can mix/match. The security model makes it great for enterprise deployment because instead of installing agent software on systems or giving agents limited shell access to hosts, you install a small secure binary that basically never changes on hosts, and a single orchestrator service can be a control plane for your entire enterprise. Then every action your agent takes is linked into the same reactive distributed system, so you can trigger other actions based on it besides just fulfillment of intent.
If yes, drop me a line, here or at manuel@kiessling.net