So you want to build your own open source chatbot

A good use for a chatbot would be a replacement/augmentation of documentation search and navigation.

Let's say you've got 200 pages of documentation on a product that needs to be well organized. You can spend weeks tracking how users interact with the page and working out a perfect layout of categories and subcategories, or you can fine tune an LLM on it and have it answer any query with both a direct problem-tailored answer and the actual pages of the doc where it sourced the answers from.

That way even if you don't even know what exact keywords to search for it should be able to give you an instant solution for almost anything even if the answer is a combination of like 8 different subpages in different categories that would've taken you an hour to find manually.

LLMs can make decisions and take actions so a talking FAQ is not necessarily the end game.

And some FAQs are so opaque and/or lengthy that even just a talking FAQ is very useful.

I totally agree with you here. The use of a chatbot becomes advantageous when the cost of delivering intelligent responses is prohibitive in terms of quality and / or speed.

> If I am trying to contact a business...

You and I both, but it sure does seem that the majority of their calls/interactions are not this way. So many people can't search/discover content on their own.

Worst part is that chatbots are usually sold with intention to replace humans, so there's not much hope for getting help from a real person, especially if business is at the state where owner kicked all developers and support out to cut costs, i.e. increase profit.

Docs can be hard to search and find the correct things. Even sometimes the answer is in there, solutions are sometimes to combine several answers.

For instance with Stripe. With the reference you don’t have a complete example of how to integrate it into express.js.

Using a vector search library & open ai or other llms you could make a very complete dev support tool.

I operate a 4fig/month micro SaaS.

We use a chat bot because we simply do not have the support staff to answer your questions.

So you get the bot --it's either that or nothing.

But what we do do, is monitor the bot logs. If a function is missing from the product or website, we add it so that future users can fully self-service.

It's important to note, users are free to cancel their account at any time and/or get a refund.

Have you never worked a support role? Users don't read shit.

Let's keep it real: The chatbot business is going to be great...

for businesses selling chatbots to other businesses.

Recently I contacted an app’s support LLM and it lied to me about a feature existing and even argued with me when I pointed out it was wrong, even saying things like “I didn’t say that”.

I’ve worked on a few support sites for companies over the years. In all my research I found >40% of customers never look for the answer before contacting support. That’s why you’ll see sites add a bunch of questions with recommendations to answers based on your description before you can contact support. Even AWS support does this.

Bots may be annoying but they can also save the company tons in custer support costs. I’m for it if the UX is good and I can quickly contact an agent if the bot can’t answer my question. This is assuming the bot won’t hallucinate and just tell me random fake facts.

> I have a question that their site wasn't able to answer, or I need to contact a representative to do something I can't do on the website

You sound like someone who has never worked in frontline support.

Having done some tech support before, you are an exception. The vast majority of things customers ask are along the lines of "how do I <thing explained on the faq page>" and "how do I <basic technical question that is not specific to the product, they just don't know how to use their computer>".

An LLM is basically perfect to answer these. It would be nice if there was improvements to detection that the bot can not directly answer the question.

given chatbots' tendency to confabulate, isn't that a risk for the product?

Agreed. In typical fashion engineers try to solve non-technical problems with more technology.

Sure, there are cases where a chat bot could replace a human or a well-written FAQ. But this navel-gazing overlooks the main reason support is so dreadful: because it’s designed to be.

Just take “call to cancel” as an example, and compare that to signing up or upselling which is technically more difficult problems. The point is to add friction for anything perceived as a short-term cost or loss. They know that a lot of people will give up or defer anything with friction. It’s the paradigm of nudging, or dark patterns. Look at eg the cookie banners, and how “reject all” is buried in most cases. Nudging allows a company to be compliant with the law, but evade the effect of it in aggregate, at the cost of your time and attention.

Chat bots is just another layer in the support maze.

FWIW, the post is about ChatGPT-style chatbots, not customer support chatbots (which I don't personally love, either).

A better Customer Service bot would be one that's trained on data other than what's publicly available. A support bot that has read and "understands" the code itself may be able to offer suggestions, or confidently determine there is in fact a bug, and either report it or fix it. Imagine a customer speaks to HelperBot and says the sorting is broken, and as a matter of fact it is. The sorting broke when the last change to SomeApp was shipped. Don't fear, HelperBot has rollback authority. "One second, I'll see if we can get that working for you..."

Working at SaaS companies I've seen countless "somewhat fluid" exchanges of information between customer -> support -> product -> developer -> support -> customer -> support -> dev, etc. The different modes of communication and long round trip times make things slow, bug reports take minimum of hours, up to weeks to absorb and resolve.

This is just one case but there are boxes drawn everywhere. Every level of intelligent organization within the society, including it's artifacts, has assumptions baked in. Now that the unit economics of applying `intelligence()` is being shifted by orders of magnitude, there's all sorts of stuff that's ripe for recrafting.

Disclaimer: Don't give HelperBot launch authority to offensive weapons etc. You know, make decisions consistent with a world line where the continuation of the civilization is pretty darn likely. Unless of course your project is to replace the current civilization, ... I don't know. Just don't do what Donny Don't does.

We have various "BIO" certified food. Maybe it is time for "human" certified companies. I'll pay more for my bank account if I can resolve problems with a human.

I'm actually looking forward to that future. Finally no more waiting queues and they are all the time friendly.

The biggest danger of AI is not that it becomes autonomous and escapes the hatch, it’s that humans put it everywhere in charge.

“Sorry judge, my whole plead was nonsense and I quoted law articles that didn’t even exist, but that’s just because I used ChatGPT” — actual lawyer who wasn’t even disbarred.

For what it's worth, LLM powered chatbots are quite different from the chatbots that were popular 5 years ago and often feel much more natural.

TBH, 99% of the clients never read the documentation, the faq, nor they search in google. Lazy users are wasting time and if a chatbot can filter a significant percent of them, then it would be a net gain for humanity. chatbots are not a silver bullet, but they can kill lots of unnecessary noise. all that we need is something that can answer basic questions asked by average users in a specific domain.

> I'm really not looking forward to the future where every business has chatbot support. They are already quite common and frustrating.

To be honest, I've had some good experience with some of them.

Amazon's comes to mind. I've been a customer for a long time, and I was shipped a faulty computer peripheral recently.

I briefly explained what the issue was to the chatbot and got an immediate response that a new order had been placed, that I should just keep what they originally sent me, there was no charge and it would be sent out priority.

And that was it. It arrived the next day and it worked fine.

Granted, it knew I was a long-time customer who has already spent a lot of money with them, but this was about as painless an experience as I can imagine. It sure beat clicking through multiple web pages of dialog options.

I also don‘t think a customer facing chat bot brings much value, but an internal, employee only chat bot could be really useful, depending on the organzization of course. The company in my last position was a rather big one with an insanely huge Confluence instance. I‘ve spent (wasted) so much time searching information there. Having a chat bot, trained on all that information, would‘ve been really useful, I think.

All current chatbots that I've dealt with have been terrible reimplementations of phone menus in text, completely unable to handle even a basic freeform question. Maybe the new wave based on LLMs will be significantly better, but I'm not holding out too much hope. Already with phone menus we get railroaded down paths convenient for the controlling entity, rather than being able to engage in a good-faith discussion.

the chatbot is better than the old IVR trees. i'd rather as a chatbot to cancel my subscription or re-send a receipt than "push 7 to continue"

Provide an RSS feed. The only thing that'll stop people following that is Google Safe Browsing. You'll still need to provide all the other methods for letting people follow you, but if you advertise your RSS feed, people might start using it.

(If you're feeling technical, you could set up an ActivityPub bridge, to let people follow you from social media too. If you're using Wordpress: https://wordpress.org/plugins/activitypub/)

Any idea how it was made? I'd like to do the same.

To be fair, if the alternative was "searching confluence", almost anything is better than that, whether it's a chatbot or a third party search engine slapped on top of your confluence data. Confluence's search is an absolute joke, and a bad one at that.

relative to offloading your ML stuff to some third-party API, using a VPS keeps things private and under your control.

explaining how to self-host on bare metal is not really within scope for an article on how to build a chatbot, and trying to pretend a VPS on google cloud is insecure is just silly.

GCP complies with various industry standards, regulations, and certifications that attest to its security and privacy controls. These certifications can give you added assurance that your data is being handled according to recognized standards. Here are some of the common certifications and standards you might look for:

ISO 27001: An internationally recognized standard for information security management systems (ISMS). GCP's compliance with this standard demonstrates its commitment to information security.

ISO 27017: Specific to cloud security, this certification focuses on the controls specific to cloud service providers.

ISO 27018: This standard is related to the protection of personally identifiable information (PII) in public clouds.

SOC 2: GCP's SOC 2 report can provide assurance about the controls they have in place related to security, availability, processing integrity, confidentiality, and privacy.

HIPAA: If you're dealing with healthcare information, you'll want to ensure that GCP is compliant with the Health Insurance Portability and Accountability Act (HIPAA).

GDPR: For operations in Europe or with European citizens' data, compliance with the General Data Protection Regulation (GDPR) is crucial.

FedRAMP: For U.S. government customers, GCP's Federal Risk and Authorization Management Program (FedRAMP) compliance might be essential.

PCI DSS: If you're handling credit card information, Payment Card Industry Data Security Standard (PCI DSS) compliance is crucial.

Ensure that the services you plan to use within GCP are covered by the relevant certifications for your industry or use case. These certifications are typically available on the Google Cloud website and can also be provided by Google's sales or support team if you need official documentation.

Does it use Intercom’s interface or Intercom’s AI to answer questions? There’s a huge difference.

Let me fix that for you.

“Here are ten reasons why I don't build my own chatbot, and why you shouldn't either.”