That said, while I am very, very impressed with GPT4 for lots of uses right now, so far it's just not clear to me that feeding it back into itself is fruitful at this point.
When I use GPT4 for coding, I give it MUCH higher level instructions than I use on a search engine, much closer to the actual problem I'm solving. But I'm still breaking the problem down into smaller problems; I need to read the output, fix errors or instruct it to fix errors, and then have it build more features on. It's similar with creative processes, brainstorming, and other writing.
These agents largely strike me as an attempt to replace this whole fact-checking/editing type routine with the LLM itself; but seeing as it's the thing the LLM is not yet good at, I'm not sure how much progress can be made there, vs just waiting for GPT5 and hoping it's another big leap in capabilities.