It's not a given that the performance would suffer. For instance, you could use self-checking methods like cycle consistency or back translation in a sequence of prompts. Another option is to generate multiple answers and then use a voting system to pick the best one. This could actually boost the LLM's accuracy, although it would require more computation. In various tasks, there might be simpler methods for verifying the answer than initially generating it.

Then you have techniques like the Tree of Thoughts, which are particularly useful for tasks that require strategic planning and exploration. You just can't solve these in one single round of LLM interaction.

In real-world applications, developers often choose a series of prompts that enable either self-checking or error minimization. Alternatively, they can involve a human in the loop to guide the system's actions. The point is to design with the system's limitations in mind.

On a side note, if you're using vLLM, you can send up to 20 requests in parallel without incurring additional costs. The server batches these requests and uses key-value caching, so you get high token/s throughput. This allows you to resend previous outputs for free or run multiple queries on a large text segment. So, running many tasks doesn't necessarily slow things down if you manage it correctly.