Curious what the latency would be using OpenAI service vs using a hosted LLM like Llama2 on premise? GPT4 is slow and given the retrieval step (coming soon) across all of companies corpus' of data, it could be even slower as it is a sequential step. (Asking more as I am curious at this point)

Another question is does the latency even matter? Today, same employees ping their colleagues for answers and wait for hours till get a reply. GPT would be faster (and likely more accurate) in most of those cases.