You could achieve thinking though logical problems by adding chain of thought or tree of thought. Then the LLM will try to break stuff down into logic it’s already seen.
49 karma · joined April 10, 2023
The article about stack overflow mentions this little tidbit.
> That application handles around 6,000 requests per second, and around 2 billion requests per month.
To be able to handle loads of 1M requests per second, your going to need more than one server dispensing results. Especially since only so much data can be stored in memory without running into performance issues.
That said, these kinds of tools are an unique use case. Not all programs need them.