AI chatbots are banned from our docs for now

The lack of detail here makes this post pretty useless, though I guess I’m not surprised generic docs bots aren’t that great.

Without knowing any more detail than “We got in touch with a few docs bot services and set up demos that were trained on our docs and blog posts.” it is hard to generalize to RAG + chat in general. I’ve had very good results with a custom setup that uses Claude Haiku to narrow down the set of relevant docs for a question and then 3.5 Sonnet to answer it. The corpus is on the small side, so no vector embeddings or even text search are required — the trick is understanding the different kinds of docs involved (OpenAPI schemas, hand-written guides) and writing code that abbreviates them in an appropriate way for the retrieval/narrowing step to work well.

I also manually tuned the system prompts to get the kind of answers I want and avoid the ones I don’t. I imagine off-the-shelf solutions are mostly lacking this customization, and they kind of can’t add it, because if they do, you’d be wondering what the value-add is and why you don’t build the same thing yourself in a couple of days. I’m sure techniques will improve, and it’s possible that turnkey solutions will be decent eventually.

I also think the distinction between supervised and unsupervised is misapplied here at the end, even accepting the colloquial use of a technical term. A docs tool powered by a bunch of hand-written documents and a custom system prompt, with a person asking questions of it — that doesn’t sound very unsupervised.

They say that copilot is great but the chatbot was bad because it lacked context, missed nuance, and frequently gave correct-sounding answers that actually weren't great.

That's exactly my experience with copilot. For simple text transformation (see this pattern? Repeat it. See this comment? Make another. See how I'm doing this? Finish it) it's ok but when asked to write any amount of code beyond that, it only writes mediocre good-seeming code that when truly analyzed within the context of the codebase, the feature, the libraries in use, the framework and language... suffers from all the same problems. The number of times I've had to walk a junior through some bad react code that an AI wrote and ask them "what do you think this is doing" and have them shrug at me is ridiculous.

On the bright side, those of us at businesses that still care about quality have a lot of work on our plates cleaning up the AI slop getting pushed into code reviews, so yay job security

I think Discord is where these bots belong. For starters, it’s a context where people expect to chat. People are also accustomed to interacting with bots there, even before the rise of generative AI. And most importantly, unlike when it’s embedded on a website, in Discord the bot can be supervised, corrected by members of the community. You could probably even set up a system where the reactions of certain users, e.g. the project’s maintainers, get fed to the model as training data. I think this could work really well.

One obvious downside is that people may be more reluctant to ask embarrassing questions in public. Though, you could allow DMs to the bot to help with that.

A few days ago I prototyped an AI chatbot which has access to our product's documentation and so far whatever I throw at it, it answers pretty well, without hallucination. It uses GPT4o-mini and OpenSearch for hybrid search (with custom parsing and indexing). After answering a user's question, it also links to the articles where it found the information.

My conjectures are:

1) Their bot had bad retrieval.

2) Their bot had a subpar prompt.

3) Their bot had a subpar LLM.

4) Their documentation is a confusing mess.

5) I didn't test my AI chatbot well enough :)

Without the specifics, it's hard to draw conclusions from the article.

hybrid search is worth way more than the text generation

did the ai make a mistake? nah, the codebase it wrong