So if you are having the agent write a script to call an endpoint and do something with the result. Have it write a script that does that and then ask it to run the script itself. It will then be able to close the loop by itself and iterate on the code until it does what you want (usually).
It is also very useful to ask the agent to write a test for something and then run that test to close the loop in the same way.
Its so fast to do these things. Seems dumb not to use an LLM to interpret an error message if you don’t immediately know the problem. And if it doesn’t work you can alway abort and do it yourself. The whole code gen and debugging process might take 45 seconds.
Sure its dumb if you’re writing some short simple function. And you need to be in the loop and understand all the changes you are making in a professional setting. But that just sounds like the basic workflow of coding with an LLM.