Feels like a monolith would be better
why does the smallest script take seconds to even start then?
Of course, if you're invoking it on a remote cluster, there are many reasons it can talk longer, especially if the cluster has to scale to satisfy the request. But all those reasons are nothing to do with containers specifically - it's things like cluster capacity, node provisioning time, container pull time, network latency, etc. If you architect things properly, you can get the number below + network latency even for a remote cluster invocation.
$ time docker run ubuntu echo hello world
hello world
real 0m0.328s
user 0m0.011s
sys 0m0.010sFurthermore, the “docker” part wouldn’t even be the most resource wasteful if you consider the general computational costs of LLMs.
The selling point of MCP servers is that they are composable and plug in into any AI agent. A monolith doesn’t achieve that, unless I’m misunderstanding things.
What I find annoying is that it’s very unpredictable when exactly an LLM will actually invoke an MCP tool function. Different LLM providers’ models behave differently, and even within the same provider different models behave differently.
Eg it’s surprisingly difficult to get an AI agent to actually use a language server to retrieve relevant information about source code, and it’s even more difficult to figure out a prompt for all language server functions that works reliably across all models.
And I guess that’s because of the fuzzy nature of it all.
I’m waiting to see how this all matures, I have the highest expectations of Anthropic with this. OpenAI seems to be doing their own thing (although ChatGPT supposedly will come with MCP support soon). Google’s models appear to be the most eager to actually invoke MCP functions, but they invoke them way too much, in turn causing a lot of context to get wasted / token noise.