That's exactly what countless techniques related to chain of thought do.
People call it agentic AI but that's a word without a definition.
Needless to say better LLMs help with my work, the same way that a stronger horse makes plowing easier.
The difference is that unlike horse breeders, e.g. Anthropic and openAI, I want to get to the internal combustion engine and tractors.
People have found that even letting llms generate gibberish tokens produces better final outputs. Which isn't a surprise when you realise that the only way a llm can do computation is by outputting tokens.