Welcome to the present.
Like many of us do not read the machine code generated by a C++ compiler, the code generated by an agent is similarly irrelevant and disposable, by now.
Welcome to the present.
Like many of us do not read the machine code generated by a C++ compiler, the code generated by an agent is similarly irrelevant and disposable, by now.
Reading a prompt but not reading the non-deterministic/non-reproducible LLM output is not comparable.
How do you know it does what you want it to do without reading it? Tests? How do you know what they test? Yes, clicking a button in a browser and getting the result you want satisfies most, but that only works on the most basic systems. Once a code base grows large enough, any one agent reading in its context wont understand the whole, and if no human does either, it becomes unworkable.
Nobody needs to understand the whole. They only need to understand each submodule, which is easy. And then, how they interact, in which case you only care about the API and contract garauntees, not the implementation.
Realistically, this is how pre-AI software engineering worked, too. Or how it should generally work. Nobody can read or understand a 10 million line codebase. So to ensure you don’t break shit, you need the submodules, orchestration, and tests.
Edit: ...and that the source code is useful in retaining enough context of the problem being solved. So, most people will not store the prompts, trusting that the source code provides context for the next iteration.
Isn't this exactly why OS projects are over-burdened by the firehose of contributions? The maintainers will want to read the code contributions, while those up-to-date with the latest models/agents/tools already trust their output to be above the average developer's (whatever that means in practice).
There's a good chance that if the agent cannot handle it, a human wouldn't be figuring it out either, without additional context. That context would be the sort of only-Joe-knows-how-it-works, so perhaps something worth addressing in any case.