I do intend to perform a much bigger test around documents but curious to hear thoughts on why this might be the case.
I do intend to perform a much bigger test around documents but curious to hear thoughts on why this might be the case.
- Not provisioning enough resources and processes are dying (we run NLP models locally so the system isn't totally lightweight) - Access is not correctly configured, either while pulling in from the source or at the user level during query time.
Assuming nothing is wrong with the setup, it is most likely because of how we prompt the LLM. Questions that require reasoning or are more open ended, generally aren't "safe" for the LLM to answer. So the system is constrained on purpose. For example if someone were to ask: "How can I increase revenue by 30% next quarter". It's not safe for the system to just propose some actions and it's likely better to just search the documents and say there wasn't any answer in the docs (unless of course some doc explicitly states plans for increasing revenue).