We built something similar, the ideas are strangely converging for production it feels like. But we used routing rather than direction agent invocation of agents. It gets exhausting to keep track of large number of concurrent agents. Best to keep a single orchestration agent that can route to agents based on capabilities.