Challenges you’re going to face when building a chatbot

Thank you for your feedback. I do not think the author claims to "know it all". Also, this post is about what we've learned so far in this new experiment - definitely not about the top three problems you may encounter when building chatbots. Just as you said - sharing what we're learning and we hope this can be useful for other people.

I'm curious how this architecture (or any "AI" medical tool) handles regulatory compliance issues.

In this case I would think you'd want to display when the rules/facts used for recommendations were last updated, which would add extra fields to track. There are also some interesting QA problems to work through.

PM on Messenger Platform here. Great feedback!

For discovery, FB Messenger actually has more channels for bot discovery than any other platform, including:

- m.me links with referral tracking

- Messenger Codes

- Several plugins that allow easy opt-in via developer's website, including the ability to by default opt-in users on any form/flow on your website into your bot experience on Messenger.

- Customer matching (use phone numbers to send updates to customers and acquire them as a bot user)

- Messenger destination ads

- Sharing features that many have used to achieve incredible viral reach.

The power that developers have to promote their own bots using these existing channels is largely undertapped. I would encourage developers to see what they're building as a business first, and a bot second. The bot is just a facet/channel into what you offer (even if it is a primary one). If you don't have a reason to offer a product or service or publish content, you probably don't have a reason to have a bot.

Admittedly, we could do better at setting user expectations for bots and in distinguishing them in search results and in the FB main app, and maybe pushing them in more places.

Re: inflated expectations. Totally right. The "AI Agent"/NLP use case is just one possibility for a bot, yet it's the sexiest one that the media seems to like. It's the tip of the iceberg -- there are a ton of more mundane but incredibly useful bots that are succeeding on the platform.

If any bot developer has questions about this, or wants to brainstorm ways to get users, I am happy to help or bounce ideas!

Perhaps not exactly what you're talking about, but a bot has "changed my life", if you'd put it that way. I built an internet radio station with some friends, and we use an IRC bot to control it. I almost completely ditched any other way of consuming music - using the bot to search our library, request songs, interact with our music as a group - is much better than any other music interface I've used. I think most people just don't have the right use-cases - but it is possible and worth trying.

hi @vinvinc,

"there are plenty of more informative bot articles that would better serve the HN audience" could you share those articles ?

I saw a recent article at oreilly ideas : https://www.oreilly.com/ideas/designing-bots

And I am not sure how this became the top comment.

What? The very first point is discovery (which is fair) which just smoothly melds into discovery ON FACEBOOK? So we just assume bot discovery is best handed off to FB/GOOG/ etc?

So now FB, in addition to being the de facto destination where your personal data goes to be exploited, is also supposed to "get their shit together" to enable even more of the data collection?

Have you noticed how nobody from FB has actually ever engaged with anyone on HN around the ethical issues surrounding any of their practices?

Your last point is about how everyone involved in bots seem to assume we already know how bots should work. You make the same assumption about who should lead this effort. I would rather the bot technology takes longer to pan out than see a world where the giant corporations are also the sole operators of the bot marketplaces.

I agree with your most of your comment.

* Scalable architecture is, most of the time, a very-NOT-big-deal. Bots tend to be relatively lightweight apps, sending/receiving json data only. Even websocket Slack apps can now be replaced by HTTP API.

* Discovery is really un-solved at this time, especially by Facebook. Slack does a better job, but arguably has a smaller ecosystem of B2B bots to manage and promote.

However, I think the article is about a rather ambitious (more than usual) and interesting bot. Having to parse normal language symptoms into medical terms is 1) an interesting use case 2) hard to do well technically. So maybe they face challenges most bot dev don't face.

Design-wise, most people don't see how hard it is to design chatbot flows that hard not horrible and confusing. Most chatbots are horrible and confusing at the moment, because coding bots is easy, designing them is really hard.

I think the article make interesting points about their own specific case, but their situation is perhaps less general than they seem to think.

I have made it my mission to help with bot discovery. I have built a bot search engine that is a bot: m.me/qwazou I have close to 1000 bots discoverable via category searches and keyword searches.

Those are powered by about 5 main startups - a lot of them are actually people on the other end too, or they are during work hours.

Yes. It's kind of a gray pattern. Has someone coined a name for it?

The most frustrating part is when I go to a site, start watching the pitch video, and am interrupted 10-15s into the video by a chat bot asking me if I have any questions.

I don't know if I have questions yet. I'm watching the video. I have javascript enabled - you can tell that I'm watching the video. Leave me alone.

I'm okay with it provided they don't start prompting me with messages. that's intrusive. but I like being able to chat with a human on the other end to quickly make buying decisions.

For all the startups I've worked with, it's a real person (or persons) behind the chat, usually founders. The greeting is pre-set but all conversations are real.

How do founders have time for that? Not many customers use chat, but the ones that do are valuable. As pg says, all early stage startups should just be writing code and talking to users, so founders make it a high priority to reply quickly!

I actually love it when sites have chat solutions.

Usually it seems there are real people and it provides the fast respose of pgone together with the asynchronity of email. Fast but less awkward when kids are screaming in the background.

Ah good old mIRC bots. Same here man. Except mine were mIRC "mud" games.

Very quickly i wrote a very large game from scratch teaching myself all sorts of concepts. It wasn't impressive at all, but oddly i'm quite impressed that i was able to "learn as you go" a semi-advanced project with such a simple language.

I'm not likely to ever be considered a virtuoso or anything, quite ordinary infact, which then means that mIRC's scripting language and documentation must have been very friendly for a complete beginner to program a virtual environment with state, rooms, "ai" traveling between rooms, and etc. It was far more than a hello world, and all thanks to mIRC it seems.

Ah, the good ol' days.

There is lots of deja vu for me. I spent my days in 97-00 hosting Eggdrops on IRC (mainly IRCNet but some others as well).

Spent my days deep in TCL, accidentally discovering probably the single deadliest bug in Eggdrop with an accidental copy and paste (would cause all bots in the botnet to go into an infinite loop)

For the last two years I've been working on similar problems but for Messaging networks (Messenger, SMS, Intercom, Slack etc) in $currentstartup

If anyone has questions around bots, conversation management, integrations, NLP etc etc, feel free to ask.

I wrote my first chatbot for IRC, and then later on for AIM. Such great memories!

The first chapter of my book, "Building Voice-Enabled Apps with Alexa" covers several of these topics, including XDCC chatbots, Eliza, The Loebner Prize, and a lot more of chatbot history. Fun stuff!

"Building Voice-Enabled Apps with Alexa"

https://library.oreilly.com/book/9781939902443/building-voic...

Ditto, that was my entry to 'programming' - to begin with it consisted of copying a bunch Runescape stats scripts from Hawkee and running on my desktop. As it progressed I included some persistence for some feature, implemented by creating a bunch of global variables with the user's nick as the first part of it. I think I felt that storing to file was overcomplicating things, bearing in mind that I was pretty young at the time.

I do recall coming across some scaling issues, which I resolved by spinning up another instance of mIRC and running it with the name <botname>2. That seemed to work pretty well. 'v2' came next, wholly consisting of changing all of the scripts' colours to be homogeneous.

Native Norwegian speaker here. I've tried about ten different questions off the top of my head, and it's very much "God dag mann - økseskaft."

E.g. I asked "What is the interest rate on mortgages?" and it gave me basically what I asked for, with an example of total cost over 25 years. Then I asked "How much would that be every month?" and it gave me a link to applying for "avdragsfrihet", but with a text recommending against doing so. Apparently dividing by 300 is beyond its faculties.

Or I asked "How do I invest in index funds?" and it replied "I don't understand, try calling us".

Basically, I couldn't get an answer to any of my questions that I hadn't gotten if I'd just googled the exact same phrase. And it doesn't appear to understand follow-up questions at all.

I'm curious, do you have a question I can try that will give me something google won't answer given the same query?

I toyed around with that python DL-distance in cython and without much effort got it to work 3.5~4x faster. Have you looked into cython to optimize it? It certainly seems cheaper than adding CPU cores, even more so considering I'm quite new to cython myself (I just took the code and added types and a few fixes here and there to get it to work faster).

Interesting bot. I found it a bit confusing that the page says to "ask James" but the bot presents itself as "Banki".

Off-Topic: You've been on HN for 1337 days. You should celebrate that with a cake.

> you'll need to make a chatbot that is indistinguisable from a human over video chat

This may sound cynical, but if you could arrange a tableau, looking like someone is paralyzed in a hospital environment, speaking like Stephen Hawking, then you don't need any serious advances in robotics (IIRC uncanny valley still hasn't been crossed) to fool humans. But I imagine the rules won't allow that.

The first paragraph reminded me so much of jwz's diary from his Netscape days, which is one of the most important documents I've read on startups (and it taight me the brilliant term "impedance mismatch" for how it feels to hang out with drunk people when you aren't, which I last used in casual conversation yesterday)

https://www.jwz.org/gruntle/nscpdorm.html

That's a very good point indeed. In that particular case the diagnostic chatbot is a proof-of-concept to grab attention to the diagnosis+NLP API offered by Infermedica, and also serve as an example app.

Chatbots are a fancy name for when your API is accessible through some well-known IM service.

They come with all of the monetization issues of APIs, but they're surfaced in an environment where you have no pre-existing relationship with the client, and you can't even charge them for access.

This essentially limits the business model to where you give away the chatbot and make money some other way; or you recognize you have no business model and do it anyway.

Sadly most bots today are solutions looking for a problem.

Our customers (thankfully) are typically looking for ways to engage with their existing customer base better, and/or deliver their existing commercial services in a better way.

Actually this is right, so the information in the article is somewhat incorrect. Thanks for spotting this! There is however an issue with the ActiveMQ implementation: once a user is assigned to a consumer, it is hard to make ActiveMQ rebalance it. At least by default the load-balancing does not take place until a consumer is dead. In case of a chat bot, some users may be returning, while others may quickly go away, leading to drastic imbalances. Apollo does the balancing perfectly out-of-the-box.

Sounded like a perfect fit for Kafka.

I don't really understand the motivation behind chat UIs. Dan Grover says it better than I could here: http://dangrover.com/blog/2016/04/20/bots-wont-replace-apps....

Who is your target customer?

Someone who runs a website that needs a chatbot, to try and find an off the shelf chatbot?

Seems a tricky value prop. Most websites don't need or want chatbots. Those that do, how many want off the shelf but not free chatbots?

Isn't there a lot of competition in this space? It seems like your bots would need to be particularly intelligent or the interface would need to be particularly low friction.

It depends on what you mean by a "chatbot platform".

If it's a way of developing a single "chatbot" and having it work across multiple "platforms" (think slack, hipchat, irc, facebook, standalone, etc... maybe even including voice-specific platforms like Amazon echo and Google Home). I would LOVE that. However it's going to most likely suffer from the same issues as other "platform-independent systems" which is that you'll be developing to the lowest common denominator.

If it's just a way to make a chatbot via a GUI, I'm personally not interested at all.

Good overview, similar to the year in review conclusions here: https://www.oreilly.com/ideas/2016-bots-year-in-review

Just curious, do people already know about your product/company but they're just interacting with it in a different way than the chatbot? Or is the chatbot your main interface?

Have you tried the recently announced Newsfeed Ads for Messenger to drive traffic to your bot?

https://developers.facebook.com/blog/post/2016/09/12/new-mes...

I am the author of the article and also the designer of the NLP system. If you're asking about how the mentions of the medical concepts (as defined in the curated knowledge base) are extracted from user messages then here is a short explanation.

The bot uses an entity recognition engine for this. The engine is available via API [1]. The engine itself uses dictionary phrases from the knowledge base and a custom matching strategy that operates on dependency graphs, which allows to abstract from some surface details (e.g., understand that “pain in my left leg” is “pain in leg”). It also contains a modified version of the DepNeg algorithm [2] to detect negated mentions (“I don't have headache”, “no history of chest pain”).

[1] https://developer.infermedica.com/docs/nlp [2] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3392064/

Try AI.API. It is a chat bot framework

I'd like to know this too. I experimented with Pythons nltk library, but it's definitely a different world out there than simple programming.

Check out api.ai, for instance the demo in this video: https://www.youtube.com/watch?v=9SUAuy9OJg4

Check out https://github.com/rabidgremlin/Mutters

It uses ML to figure out the "intent" of what the user has said (using a document classifier) and named entity recognition (NER) to extract data from the text (date, location etc).

That is actually the easy part. The harder part is the conversation scripting and state handling. Mutters uses the Ink narration engine (originally designed for dialogue heavy games) for this.

It works pretty well. IMHO :)

I work for chatbot building platform: http://deeppixel.ai/. We use state-of-the-art ensemble of deep learning algorithms that try to understand the meaning of sentences by taking the whole sentence as well as bag of words/bag of ngrams. In addition to that, we use classic NLP techniques like POS tagging, constituency parsing, dependency parsing, Named Entity Recognition (NER), clustering.

There are custom-built bots that use tools like CoreNLP. They tend to not use any framework.

But, most NLP-powered bots are using machine learning based NLP APIs rather than CoreNLP. These tend to do two functions: intent determination and slot detection.

Intent determination means finding the overall meaning of a message. Slot detection means locating and extracting important terms (not just standard NER) within a message. Both happen based on examples provided by training data that is used to train a machine learning model.

The difference intent determination and keyword-detection is that ML based intent determination is much more resistant to different ways of saying the same thing.

If you search the ML research for intent determination and slot filling you will find various techniques to do it, from CRFs to CNNs to RNNs.

Example:

"I want to find a flight from London to New York" => Intent: find_flight, City.Departure: London, City.Arrival: New York

That set of information will then be passed to program code to determine the response.

(generalization:)

Right now, most code will generally map intents and conversation state to a branch of code and will have code that takes specific action based on the current conversation state. Whenever slots are detected, those get stored to conversation state. As the user types each message, the process repeats.

The logic is generally either structured as a tree or as something that resembles a state machine.

Examples are Microsoft's Bot Framework, Wit.ai's SDK, Init.ai's conversation logic, etc.

(the future)

The machine learning model can be adapted to predict the next message type if it's trained on sequences of messages. At Init.ai we do that, where in addition to classifying the current message and extracting slots, the machine learning model predicts who (which person or computer) will send the next message and what type it will be. That enables the bot developer to not have to write code and let the prediction system select text to send as appropriate.

In the future you'll see other things that make it easier, where the machine learning system starts using more of the conversational content to make more accurate sophisticated predictions of what actions to take and messages to send. For example, some new ML research is around automatic question-answering based on a dataset, and that could be used to make bots respond automatically based on information.

So over time it will get easier for developers as the ML progresses.

As an alternative to building it yourself you can rely on services that do this for you.

I've been using the microsoft bot framework together with the luis.ai (azure) service for natural language parsing. Works really well in my experience. It supports bimodal operation where first a regex is attempted and then as a fallback the sentence is passed to the NLP service to parse into action/entities.

I can really recommend the microsoft bot framework even without the natural language parsing. It plugs into a range of chat platforms, including sms and email, and you're free to build your bot with node or .net. I went with node and have been pretty happy with it so far.

Watson Conversation is a well established alternative: https://www.ibm.com/watson/developercloud/conversation.html

Quite a few depending on what level of control/functionality you are looking for.

Some are more developer/write your own code focussed, other more graphically focused.

In alphabetical order and not exhaustive.

API.AI (now Google) Chatfuel (YC) Converse.AI (disclaimer, $currentstartup) Flowxo Gupshup.io IBM Watson Engagement Advisor/Watson Meya.ai Microsoft Bot builder Motion.ai Wit.ai (Facebook)

AWS also has a suite of tools that between them you could use, check out Lex, Polly & Steps

All of these do one or more of the following:

NLP User management Messaging Platform Integrations Conversation State & Context Management Workflow Human Escalation Backend integrations (Helpdesk, CRM etc) Analytics

Pick your poison based on what you need :)

Smooch.io (disclaimer: my company) has a really rich API for programming conversations.

You can use the API to converse with people on web chat, iOS/Android in-app chat, SMS, Facebook, Telegram, Viber, Twitter DM, WeChat, LINE+++ all through one single set of APIs. You can even send rich structured messages that are automatically translated into the best representation on each platform.