Chatbots – Igniting Division of Labour in AI | Hacker News Reader

Chatbots – Igniting Division of Labour in AI

medium.com

38 pointspelumi16 comments

Do people actually prefer talking to bots instead of interacting with apps? Bots don't need to be installed usually, so that makes them lower barrier, but neither do websites.

They are useful when you want to query them with tasks that have a lot of degrees of freedom. These tasks may be easier to describe in human language where the user highlights the important parts forgoing the need to have a complicated GUI with lots of buttons. In these cases though, users don't know what the bot is capable off. GUIs are more discoverable because they present possible options with bad GUIs hiding even more possibilities in the submenus. This is one advantage over GUIs I would like to see bots explore.

In my experience however bots are not good at doing the right thing when queried with a task that has a lot of degrees of freedom. I own an Amazon Echo and at times I feel like I am reciting incantations to get it to do things I want from it; this is especially true of Alexa's "skills". If I have to memorize specific commands for the Echo to respond, it becomes a glorified voice operated command line to me. A verbose, inaccurate command line without a man page.

Websites don't need to be installed, but websites that aren't Facebook aren't where Facebook want their users to go.

I'm wondering whether this is really an Apps 2.0 play but in disguise because apps burnt all their user goodwill a long time ago - and they have a permission request thing too. I don't know anybody who intentionally uses any Facebook apps (or if they even still exist) and I doubt the current top tier companies have building one anywhere on their priority list.

This brings that functionality back to Facebook, not some other website or app, without the baggage associated with the old Facebook app stuff.

I think there is some cognitive load involved in context switching between many apps and websites, managing separate accounts and passwords with each, knowing their UI differences, etc.

Opening chats to 5 recipients and ordering stuff from them with free form text is a lot lower friction than installing 5 apps or navigating to 5 websites, meeting their password complexity requirements, reentering addresses and CC numbers, etc.

It's not good news for freedom or the open web, but it probably does lower friction for users to give you money.

I totally agree with you, not all tasks are best suited for bots. As your point suggests, we need an insane level of Natural Language Understanding handle and correctly execute tasks with many deegres of freedom.

I tried http://www.mitsuku.com/, it was interesting. It was able to do some math and respond to some general knowledge questions. It knows that a car is heavier than a book, or that fire is not wet, but water is. It can handle simple contextual questions based on past lines. Still a lot to go.

I like this idea a lot and have been thinking about this myself. Here are some questions I've been struggling with:

1. What would be the incentive for the bot developers to adhere to a standard that would basically encourage a user to download a meta bot rather than the bot developer's own bot? Especially once they have spent significant resources to develop and enhance their bot before any standards were in place. This would require refactoring at a level that a bot developer might not want to do. The meta bot might lose out on using such a bot, which might be really good at what it does.

2. Let's consider that once bots are created a meta-bot might be able to use the bot's NLP interface to issue queries. The response format might be so varied among the bots that the meta-bot would essentially need to allow a complete pass-through of the response. This would limit the meta-bot's ability to layer on better structure on the response or even perform it's own AI layering on top of it because the meta-bot would then need to be able to parse all these varied responses.

3. How would a meta-bot rank bots in the same domain? Maybe by scraping the review ratings on multiple BOT marketplaces, gathering performance metrics by issuing sample queries, etc. Once users start using the meta-bot there is lot of information the meta-bot can gather to influence it's ranking but the issue of cold ranking BOTs is very challenging.

4. Structuring NLP queries specific to BOTs will also be challenging because the meta-bot will need to keep track of various "flavors" of queries a BOT understands. e.g. 2 hotel booking BOTs could have been developed where one does a progressive Q&A to get to information it needs to make a booking where as another understands a specific command like "Buy 2 economy class tickets from San Francisco to San Diego for 15th May"

There are other such questions that make creating a meta-bot challenging.

Thanks for the thoughtful questions. Here are my thoughts on your questions

1. The incentive for a developer is to get better exposure. This is working with the assumption that the metabot has a larger userbase than individual chatbots. In general, the incentive is similar to what causes an online store to setup a 'store' on amazon's market place or retail sites to list on a price comparison engine.

2. As regards the second point, my thoughts when writing the was that it will infact be a direct pass through i.e. the metabot will not need to bother about the structure or nature of the requests/response but will instead focus on knowing about the bots and their capabilities. If it does this effectively, it theoretically pipes all requests for a specific task to the same bot to complete. This comes with it's own complexities as well.

3. To rank bots in the same doamin, an option is to guage the satisfaction of multiple users of the bots. For instance if a bot is faster to complete a flight reservation and the 'user' confirms to the metabot that it is satisfied with the service, that's a positive signal. This can be compared across multiple bots to measure the effectiveness of each bot.

4. Indeed, I agree it will be challenging. But I think the challenge is less of structuring the queries. If the meta-bot acts primarily as a pipe, its most important task is to identify who handles what and know the state of the conversation.

There are a lot of kinks to be worked out though...

Thanks for responding to my questions. I like this discussion so I hope you won't mind me continuing it a bit further 1. Your response is valid for a developer/team creating a bot once standards are in place. This is not the case right now so any bots created right now will need to be refactored.

3. Your response is valid when a meta-bot is tracking bot usage and user feedback on bots. That's what I would call a warm ranking. However cold ranking is when a meta-bot is just starting up and has no usage data about the bots.

4. Even working as a pipe the meta-bot would need to figure out how to pipe the query to a bot especially consider this scenario: The user has asked "Book me 2 tickets and a hotel in San Diego for May 15th". The meta-bot has determined this is a travel query and found that Bot A is highest ranked for travel queries. It pipes the query to Bot A. However Bot A does not understand how to process this query because it excels at processing queries like "Book me 2 economy tickets and a 3 to 4 star hotel for 3 nights in San Diego" and it received high satisfaction ratings when it was queried like that. However in this vague query scenario Bot B does a much better job but the meta-bot won't know about it.

This is fascinating. I have an Amazon Echo and ever since I got it when it was not widely available I've been fascinated with the potential of it and Amazon went in the direction that I predicted it would with that. However Amazon Echo still uses APIs to interact with services and provides a nice NLP interface to it. This is a friction point for getting new things in the Echo and I have been wondering how to remove this friction.

I think you may be overthinking 3 a bit. You only need to use the new bot enough to get enough info for the normal ranking method to work. Some simple solutions:

1. When a new bot is added, always use that bot first. This assumes that the user added it intentionally despite already having bots for that function.

2. If bots are ranked too closely, pick one at random.

3. If bots are ranked too closely, ask the user which to use. Have an option for setting that as default. If you have an Android phone this should feel familiar. This is how Android handles different apps that handle the same function (i.e. what app should open when you click the mailto link).

1. The idea of a meta-bot that the OP puts forward doesn't require user to add any bot. If user had to add every bot manually then it takes away a key advantage of a meta-bot. One bot to rule them all.

From what I (mis)understood, a meta-bot would internally have "friended" other bots and user only adds the meta-bot and interacts with the meta-bot which then sends the queries to its friended bots. Though now that I think about it I probably did misunderstand it. :)

Your description is what I had in mind. The user won't be the one to add the bots, the meta-bot has a strong knowledge of the 'chatbot community' and knows who is good at solving problems in different categories.

The proposition of allowing the user 'add' bots is also quite interesting but I've not given much thoughts to that.

If user is allowed to add bots then you take away one thing from the meta-bot. The task of Discovering relevant bots. At the same time you limit the meta-bot from helping you out with a task if a bot for that task has not been installed by the user.

The meta-bot could technically suggest what bots to install if it does not find an installed bot to perform a task. In the current world of bots I think this would be a more optimal approach rather than a true meta-bot to bot transparent communication.

Also take into consideration that a meta-bot should provide proper attribution to whichever bot it chooses to perform the task otherwise you open yourself up to copyright infringement or equivalent.

That's interesting. I guess I just assumed the user would have to add the bots. Otherwise how would they know what the meta-bot is capable of doing? Also, discovery of bots, especially ones that aren't "mainstream" (I'm thinking less utilitarian and more art/fun, here) or part of specific bot networks/marketplaces, would be harder. Allowing users to at least also add their own seems like it would be a useful feature.

If you want to take this offline I've updated my email address in my profile. Don't know if you can see it.

Yeah, we should indeed take this offline. I can't see your email, you can reach me on lumzor [at] gmail [dot] com.

I guess the meta-bot would use the NLP interface of the other bots. (You need natural language processing and generation anyhow, so you may as well use it as a bot-2-bot interface.)

I think your 4th point is more problematic, when you have a meta bot and the actual interface interacting, then you will get false positives, which in this case means bought/not-bought tickets.

Reply on news.ycombinator.com