Lessons learned from building a commercial open-source bot development platform | Hacker News Reader

Lessons learned from building a commercial open-source bot development platform

livablesoftware.com

78 pointssoftmodeling35 comments

I'd be more impressed if the site had a chatbot to talk to. The actual site [1] is just one of those pages that wants your email.

This seems to be a generic problem with chatbot companies. I have yet to find one with a bot intelligent enough to answer basic questions about the chatbot product. Most chatbots are about as smart as "press 1 to pay bill, press 2 to return product..." with a weak natural language system on top. A basic web site is more useful.

[1] https://xatkit.com/

If you head to the examples page (https://xatkit.com/chatbot-examples/) you will see other examples of bots.

It's true the one in our home page is a bot to illustrate a simple scenario with some of the features that Xatkit includes (if you ask the bot for a demo)

    Hi there.
    I can help you get up to speed with a REST API
    Just give me the URL of the OpenAPI definition of your API

    > What is an example of an OpenAPI definition?

    Loading the API from . Please wait...
     Oops, I couldn't parse your definition
    Here is the error message:
    no protocol: Xatkit

What do you recommend that can hold an interesting conversation?

I don't think the goal of a bot is having an interesting conversation. The goal is to help you accomplish a task (get your answer, do something for you,...).

Clearly, in the above example, you didn't provide what the bot was expecting and since the bot is a research prototype, it had not enough training to identify that you were not providing an API file but asking a clarification question.

No, the bot owner should the dashboard to realize that a user (you) asked this question that made the bot failed and should prepare a new version of the bot that can answer it

Ask "where's an example", and they point you at something a customer did. Ask why the example is unimpressive, and they blame that customer.

In chatbot systems, the machinery doesn't do very much. It's anticipating almost all possible user responses that makes the thing work, if it does. All the engine really does is take the user's question and match it to some big FAQ file someone put in. Plus there's some kind of phone-tree like path to guide users through whatever the chatbot wants them to go. (Not where the user wants to go.)

In game development, this is called The Dev Team Thinks of Everything.[1]

[1] https://tvtropes.org/pmwiki/pmwiki.php/Main/DevelopersForesi...

"explaining why the world needs yet another bot solution"

I'd be interested to hear your take on this. The dozen or so prominent bot startups that popped up over the last 5 years nearly all ended with acquihires. Even Facebook, who once thought they could create an uberbot to do literally anything, have all but abandoned their bot aspirations.

To begin with, I think creating a general knowledge bot that can talk about everything is an interesting research concept but not a very useful one.

Then Xatkit aims to be a very flexible platform that helps you create chatbots on top of any platform you need, more than a concrete solution for a specific set of technologies. For instance, most current solutions are tied to a specific NLU Engine (being DialogFlow, IBM Watson or whatever). Xatkit tries to abstract from the concrete engine and let you define the bot in a more abstract way and deploy the bot on any of them based on your deployment preferences.

(this doesn't mean Xatkit provides right now all the connectors you may want, it just means that the architecture of Xatkit was especially designed with this goal in mind, it's open source so it's possible for you to create whatever you need)

The embodiment AI folks seem prescient, here.

How can chatbots ever mimic human interaction, without experiencing human-ness?

We parse incredibly complex dailog exceptions every day because "we made that mistake yesterday" (or "... when we were 5").

I've never had a good experience interacting with a chat bot. I find it insulting when a company I'm already giving money to insists I talk to a robot before I get to talk to a human being. Chat bots are just the latest version of listening to a dial in support phone menu and trying to figure out which option you need.

Chatbots, when done right, properly field answer your questions across a limited scope and domain, and allow for an easy exit to talk to a human. As a customer support platform with bots, my company helps businesses reduce their customer support load effectively using chatbots while making sure customers are able to talk to a person if they need to.

As a business, when you have customers asking the same question over and over (shipping questions, where is my order etc), it makes sense for a bot to handle that, and have your staff handle other more specific issues such as refunds, payment issues, or any other questions that the bot wasn't able to handle, or if the customer specifically hits the 'I want to talk to someone' button

My guess is the "bots" you've dealt with are dead-end bots that don't offer a way to speak to an actual person and are meant to make it as difficult as possible for you to actually reach someone. That's unfortunate - maybe it's a business decision (a poor one), or the chatbot just doesn't work very well and doesn't have a way to escalate.

Chatbots have gotten so much flak because a bunch of them try to use as much AI as possible to answer every possible question and businesses think chatbots makes needing an actual human support team unnecessary, which unfortunately is not the case.

But, as a business, when you have customers asking the same question over and over, doesn't that suggest a failing in how you're currently providing (or not providing) that information? Why is the solution a chat-bot and not adding to or improving the features of your website?

Well if you are able to discover that the customers are asking the same question over and over, that's already a good use of the chatbot (as a "discovery" tool). Then you can indeed make sure the info is more visible in the website and iterate again.

Having a chat bot and a good website work hand in hand. Sometimes users don't immediately find the page they are looking for, or simply prefer to ask the bot and then be directed to the right place, rather than investigating for themselves.

If you are looking for an integrated solution with many other features than only chat to build your support/sales/marketing pipeline take a look at https://github.com/chaskiq/chaskiq

And agree with you, being on development of a chat solution over the last two years I've realize that the AI sucks big time for chatbots no matter how hard we try they will never have the human touch. that's why I have avoided the NLP integration as much as I can. This lack of human touch is the reality of any chatbot that does NLP, sentiments analisis, etc.. they simply fail. I have tried xatkit and I could not establish a proper dialog either.

In that sense I've implemented in chaskiq some sort of path selector / decisional steps that you can compose for further routing assignment to human agents, and that works very well.

You forgot the bit where you also have to guess what your options are. Doubleplusungood if it's a voice chatbot that fails at recognizing your accent.

I get your point. Though for some people is the opposite (e.g. people that are shy or, don't speak good English and need to interact with an international company) and feel more confortable talking with a bot.

Also, for small companies, it may be the only option as they cannot be available 24/7.

Wouldn't a company improving their website be the more reasonable approach though? Rather than hiding the equivalent features behind a maze of keyword matching and sub-par natural language processing? A chat-bot is only going to be capable of supplying information and service that should be far more trivial to just expose on a website via simple forms, user-account features, and F&Q/knowledgebases.

In my experience, most tech-savvy people do not want to call or chat. They only resort to those options when they can't find a way to do what they need to on a company's website. There is another group that wants to talk to a real person regardless, but they usually prefer to call. I'm not aware of any group that wants, or has their needs best met, by talking to a chat-bot.

Sometimes, as a website owner, you may be wrong regarding the kind of information visitors are looking for. A bot can be also useful here (people not finding what they are looking for may ask the bot, the bot probably won't be able to answer but, if you're using a decent bot platform, you'll be able to easily get a report on what questions people are asking and then add this either to the bot or to the website if you prefer).

In this case doesn't your customer walk away frustrated at best, after a nonsensical conversation with a bot? That's been my experience every time. I ask a question, then the bot asks me questions that are irrelevant and make it very clear it doesn't understand what I'm saying and that I'm not talking to a real person.

That’s a reasonable point, but I would argue that the same could be accomplished (with a better user-experience) with something like a “Not finding what you’re looking for, click here” shown on the website in the appropriate places that leads to a contact form. When you end up talking with a chatbot you have no way of knowing if it doesn’t have access to the information you’re looking for, or if you just haven’t asked it using the right key-words or terms. That can make it a significantly more frustrating experience then just using a contact form and waiting a day for a response.

I've never had a good outcome (a solved problem) with a chat bot. The best I've gotten is the bot throwing up its virtual arms and pointing me at an FAQ page or giving me a phone number to call. It just feels like a smokescreen to distract or frustrate me.

If your English isn't good, I believe bots are much worse. You have a point for the very shy. Good customer service would probably be the other way around: a human pretending to be a bot to make the customer feel comfortable while also providing the best service.

A human pretending to be a bot is disrespectful to the person who has to pretend to be a robot for their job, and deceitful to the person who thinks they're talking to a computer. Who would want this dehumanizing situation?

It's deceitful (though that's an awfully strong word), but I don't believe it's disrespectful. They'd not be asked to pretend they are a bot because they are terrible people, but because of a anxiety disorder of the other person. Behaving differently than you normally would because of requirements of the recipient of your help is pretty normal.

I have yet to meet a person who would rather chat with a bot. I hardly believe that such a person even exists.

I have yet to meet a person who doesn't like cats. I hardly believe that such a person exists.

Have plenty of experience helping companies set up chat bots properly, and watching customers actually talk to bots and have their questions answered by bots. There are always some people that would automatically escalate to human support every time and not bother with the bot, but there's also a fairly large percentage of people who would go through with the bot.

What is a "fairly large percentage?"

They're simply too introverted!

Plenty of introverts would disagree with you

I'm an introvert myself, and I would sooner read the docs than call or chat. But, ultimately, when I need to have a conversation to solve my problem you bet your ass I don't want to talk to a bot. Let me know when they pass the turing test and maybe I'll reconsider.

Can we also dispense with the second-hand reassurances that "someone" would disagree, and replace it with first-hand testimony from someone who actually prefers the chat bot to chatting with a person?

Personally I have had many good experiences with chatbots, although they don't replace talking to a real human about complex issues.

Effective chatbots tend to triage requests and point users to further information, rather than fully solving the issue on their own.

When I spend too much time on R&D building something big and it doesn't work out the way I planned; the first thought is to turn it into a platform and sell it to other people who might find it useful. Then I remember that starting and running a company is exponentially harder than developing a killer app, so the next thought is to open-source and document it and hope people pay me as a consultant... It seems this thought process is universal.

Ok, so this is about chatbots. I thought it was about botnets.

Definitely not about botnets. It's about chatbots or "normal" bots (i.e. components that automate processes, like sending you a Slack message alerting you when somebody posts a comment in your WP site).

You read the article and thought it was about botnets? Or you didn't bother reading the article and thought the article was about botnets?

Reply on news.ycombinator.com