A few days ago I prototyped an AI chatbot which has access to our product's documentation and so far whatever I throw at it, it answers pretty well, without hallucination. It uses GPT4o-mini and OpenSearch for hybrid search (with custom parsing and indexing). After answering a user's question, it also links to the articles where it found the information.

My conjectures are:

1) Their bot had bad retrieval.

2) Their bot had a subpar prompt.

3) Their bot had a subpar LLM.

4) Their documentation is a confusing mess.

5) I didn't test my AI chatbot well enough :)

Without the specifics, it's hard to draw conclusions from the article.