Why (Senior) Engineers Struggle to Build AI Agents

I don't know why you would say juniors handle this better. From what I've seen, juniors who ship fast often do so by ignoring domain complexity and testing against a single "happy path" prompt.

Seniors take longer because they are actually trying to understand the domain and cover edge cases. Speed isn't a metric of success if the feature is half-baked and breaks the moment a user deviates from the test case.

I like the closing:

> The transition from deterministic systems to probabilistic agents is uncomfortable. It requires us to trade certainty for semantic flexibility. We no longer know and own the exact execution path. We effectively hand over the control flow to a non-deterministic model and store our application state in natural language.

But I don't think it supports the overall point. After working on some AI products that shipped to retail, the issue wasn't that senior engineers struggled... it was more that the non-determinism made the products shitty.

As real-world prompts drift from the ones in your test environment, you end up with something that appears to work but collapses when customers use it. It's impossible to account for all of the variation the customer population throws into your agents so you end up dealing with a large failure %. Then you try to tune it to fix one customer and end up breaking others.

We found AI genuinely useful in cases where non-determinism was expected or desired, usually closer to the final output. For example, we built a game called ^MakeItReal where players draw an object and we use AI to turn it into a 3D mesh. This works because people are delighted when the output varies in surprising ways:

https://www.youtube.com/watch?v=BfQ6YOCxsRs

I think the issue is that senior engineers often have a better understanding of where a system will fail up front and that non determinism is inappropriate for most use cases we encounter.

In my experience junior engineers fail to understand (from lack of experience) that optimising their agent flow will be a gigantic time sink with N-dimensional edge cases the burden of which is pushed onto other parts of the org unintentionally.

Often building a 95% deterministic flow with 5% agent handling is much more reliable in production but takes more time up front (shocker). I've had to go on the same journey at least five times this year alone where my team have to convert "vibe coded" agents into production features and we've ended up pulling all but one. The investment it takes to actually get these things performing well often isn't worth it.

The article gives an example of agent friendly APIs:

    {
       "plan_id": "123",
       "text": "This plan looks good, but please focus on the US market."
    }

> By preserving the text, the downstream agent can read the feedback ("Approved, but focus on US market") and adjust its behavior dynamically.

I imagine it could be useful for systems to communicate using rich dialogue. But looking at the API, it struck me as a security risk. Couldn't a 'bad' agent try to adjust the behaviour of the downstream agent in a malicious way? Or am I out of touch - is this how it's usually done?

I recently had a brief brainstorming with someone building an AI data enrichment tool. I found myself suggesting the 'entities' in the tool would just be semi-structured json blobs; instead of normalising the data into some relational database schema.

I couldn't find good arguments against just slinging the data into a blob and getting AI Models to make sense of it.

Bringing determinism to a non deterministic world is basically the entire story of the human advancement. We require determinism.

AI is more of a new raw material instead of a new tool for humans. It’s our job to make it deterministic. “Pull the slot machine lever again” isn’t great advice (unless you work for a company that makes money off those lever pulls like the author).

The biggest downsides of working with people, like ambiguity when asking and imprecision when receiving an answer, but without the upsides like sharing food or gossiping about people you both know

Oh and the errors can compound dozens of times per minute (your favorite inference API isn’t fast enough for dozens per second), but I imagine that this is seen as efficiency.

People struggle to build AI agents because AI agents are new. There is no "ruby on rails" for AI agents.

Dosent just a whole lot of it boil down to agents being bad. Non deterministic. Dosent have the same references as humans. Cannot take simple clues. Stumbles on itself.

Calling a agent junior level is a disserive to juniors. At least humans more often realize they need more context, instead of just going ham.

"Why senior developers just don't get how blockchain is going to change the world economy"

Yeah no, this angle is offensive.

I use LLM's daily as a search engine and syntax help - but this bizarre meta-meta-meta now if you just abstract even more it'l work, maybe invent an entire universe, or hey why not invent a cluster of universes that'l turn off the lights in eastern europe so you can vibecode... no, no, no.

I don't think this crazy energy wasting hell is getting us anywhere useful when someone's going to wade through the bullshit and the energy needs will go exponential on an already strained energy infrastructure not to mention the state of climate.

I still believe LLM's are helpful but in the other direction, more focused, smaller scaled and with less abstraction and magic happening.