We don’t know how to build conversational software yet | Hacker News Reader

We don’t know how to build conversational software yet

medium.com

59 pointsguifortaine26 comments

I think the bigger question is: why would I want to converse with software? Is that actually a good interface? People imagine the computer from star trek, but that was a plot device. The character was talking to the viewer with the computer as a proxy.

It's easy to imagine a large proportion of the population wanting to use natural language to achieve complex technical tasks. It is, after all, the same API that people use with each other. Someone could then describe what they want out of a spreadsheet instead of manually editing it.

What interface would you prefer?

I just built a very rudimentary version of this "speak to spreadsheet" idea:

https://github.com/diesendruck/ggspeak

Or for the demo video: https://www.youtube.com/watch?v=xmtrGxVvVyg

If you're interested in working on it together, get in touch.

>It's easy to imagine a large proportion of the population wanting to use natural language to achieve complex technical tasks. It is, after all, the same API that people use with each other.

True, and this api will be deprecated for those who want more efficient means of communication in the future and willing to work towards it.

>What interface would you prefer?

EM fluctuations in real time of the neurons in my body to machine code via trillions of nano sensors/transmitters/stimulators. Vocal cords and limbs need not apply… will be way more intuitive than "natural language".

the same API that people use with each other

Have you ever tried to do a "complex technical task" by directing an unskilled person over the phone? It's a tremendously frustrating experience.

Natural language alone isn't useful without your interlocutor having enough understanding of the problem domain to work out what you mean and help you. So a natural language interface to the spreadsheet would have to be on the level of "perform linear regression on these sales figures" in order to be more useful than mere speech-to-text.

Yes and no. We have a lot of ways of communicating with each other and talking is only one of many. For instance, if I want to formally request something I'd put it in writing -- I want there to be a reference document that all interested parties can go back to. Likewise if I need a building, we might have a conversation about what we want, but the most important communication is the blueprint. If I'm making a blueprint, it's a lot quicker to put pen to paper than it is to tell another person where to put the lines.

I think software is inventing a lot of new ways to communicate with each other, and that's a pretty awesome thing; I just think speech is a very inefficient and imprecise way to communicate things, and efficiency and precision are what you really want when dealing with a computer.

It's easy to think about interacting with a computer as a programmer, but "Computer, when does the next star wars movie come out?" is much easier for most people than "SELECT release_date FROM ...". Unstructured language is just super accessible to more people. Everyone knows it already! It's not just about conversation, but conversation towards tasks.

That and other examples like "Call me an Uber" [0] are already relatively simple tasks, and adding voice interaction doesn't really add much to it. Is this going to be the next wave of smart devices, instead of an app you talk to your washing machine?

Ok it's not necessarily a bad thing (especially in terms of accessibility - plus nobody even knows which setting they should use [1]), but I don't think this is what OP is hinting at. Say you are working in HR "How many staff took a greater than average sick days around holiday weekends?", for that task what does voice interaction provide over a report, which can even be automatically generated and delivered for when you arrive at 9:03am.

[0] https://www.kickstarter.com/projects/403524037/autonomous-de...

[1] https://i.ytimg.com/vi/P53Bde2yq5E/maxresdefault.jpg

Just keeping the star trek metaphor going (nerd alert!); I remember there's this episode where counselor troi asks the computer to "cross-reference" entries on darmok, or something like that (1980s version of google basically). The computer comes back with like 47 entries the computer reads back one by one and she gives up in disgust after like three! On the other hand, if that was on a screen that she could quickly just scan, 47 entries is nothing, and when you google stuff you're probably adding or removing search terms randomly to narrow down on exactly what you want.

I don't think there's anything wrong with having a voice interface, I just never see it being anything more than a secondary convenience.

That's exactly what a hypothetical conversational interface would do that a simple voice interface couldn't. It's the level 3 the article was talking about. If the computer responds "There are 47 entries," you might say "Do any of them contain the word tanagra?" or "Get me the most recent one."

I agree. That is the much bigger question. Texting with software has a huge novelty factor for many people, but it remains to be seen if it's actually more useful.

GUIs offer a lot of great short cuts that are simply faster to visually read and interact with than reading text and typing - or even saying - a response.

I think there are aspects of chat bots that really do offer a new and better user experience, but they're not necessarily the ones being highlighted in demos.

Asynchronous interaction with an interface for potentially long-running tasks is wonderful. I can tell an app to, say, find a list of hotels I might like in Argentina and then flip to another app knowing that I'll be notifieda few seconds later when the response is ready.

The way forward may be a hybrid interface, where bots can respond with messages that are little mini GUIs. There might be photos to scroll through and a few buttons to click. FB showed off a feature like this at F8 today.

Ultimately what would make a bot a really improved experience is if it knows you really well, but that doesn't imply a conversational interface. If you could say "make dinner reservations for tonight" and trust the bot would find the perfect place that would be awesome. That's hard. And it has nothing to do with chat bots per se, just a much better recommendation engine.

* level 1 — command & response

Functions?

* level 2 — hard-coded conversation flows

Scripts?

* level 3 — fuzzy/continuous/fluid state.

Threading?

I can't shake the feeling that deep learning is being shoehorned into a bot with a specific purpose (ShoeBot, etc). Talking to a ShoeBot, I would expect it to have predictable responses or state changes. I don't want it to suddenly infer that I'm looking for shoes for my wife because of an opaque model it's formed from an insufficient set of training data. Many applications of deep learning seem to be just a lazy way to have the machines do the work automagically.

> I don't want it to suddenly infer that I'm looking for shoes for my wife...

If the end goal is a human-like level of conversation, isn't this a desirable outcome? Inference and allusion are two elements that are used all the time during conversations between people -- with positive and negative consequences.

I guess it depends on the ultimate outcome you are seeking from the conversation.

Ultimate outcome is that I want to buy a good pair of shoes for myself :) I'm not particularly interested in the conversation unless the bot happened to be a domain expert (I find it unlikely that there is enough training data available to make it a credible domain expert..). The point was that I'm especially wary of its internal state incorrectly transitioning but now no one knows why due to the deep learning implementation.

Fair point, especially if the bot becomes biased in some way after that point and fails to adapt to your cues ("no, these shoes are for me, not my wife..").

As you say, if your only goal is ShoeBot this (scripts, functions, etc.) is a great way to go. One of the key points of deep learning approaches is that you have this abstract, powerful computational device that is trained to extract the necessary features and also perform the task jointly - and it just happens to be a ShoeBot due to the training data.

This usually results in improved performance along with "ease of use" in transitioning to new but related applications. A model just happens to be a ShoeBot when trained on specific data, but ostensibly a person or company could make ShoeBot, CarBot, ApartmentBot, etc with the exact same approach, given enough data. This is very different than a workflow of "craft tons of features for domain X, write custom scripts/conversation logic for domain X, etc.".

These choices between feature based approaches and "deep" techniques are tradeoffs reminiscent of "you aren't gonna need it" versus "room to scale", but in the ML algorithms you choose rather than the software stack/implementation.

It depends on things like how much data you have, how much compute you are willing to pay for, how many users you expect, and so on - but neither approach is necessarily wrong.

In general if deep learning approaches don't roundly beat feature engineered or hand crafted approaches, you don't have enough data or are trying to shoehorn (pardon the pun) a solution that doesn't fit. Right tool for the job and all that.

Like Google Photos, this might just be something I have to see to believe. However... Google Photos routinely categorizes pictures of my dog as Bear or Cat. My greatest concern is about being able to acquire the conversations necessary to train these niche applications beyond a laughable state.

I'd venture a bet that most of the advancement of levels 2 & 3 happen in private and/or are in use in areas you aren't already thinking of.

Have a look at the conversational examples Voicebox lists on their website: http://www.voicebox.com/technology/ (disclosure: I worked there for a while.)

The wide-open "talk about anything" software is still a few years out, I think. But, having a "human-like" conversation/interaction with a bot/AI already happens today.

Perhaps we forgot. Eliza bot written in PROLOG by Joseph Weizenbaum circa 1964 at MIT was quite up for the task. I am sure it is still about 90% more conversational than 80% of the current online English-speaking audience.

This mentions being able to build very simple faux-conversational interfaces within a hackathon. I've got one such hackathon coming up in a few months, could I go from zero knowledge to working proof-of-concept?

Eliza (for English) is close to a one-liner in any modern language. (If you don't count the templates and canned responses.)

I believe he was talking about using an existing tool to add a 'level 1 — command & response' style interface to whatever your project is. So yeah, using something like wit.ai you could likely integrate that style of interface with zero prior knowledge fairly quickly.

Should we come up with a language that's easy to teach for both computers and humans? A sort of generic human/machine DSL?

It was all done years ago ... https://en.wikipedia.org/wiki/AIML

If by done, you mean "a team designed a rigid flow-chart and rules-based engine where you can painfully map out every specific interaction in your system"... then yes!

surprising that 'theory of mind' doesn't appear in the article.

Reply on news.ycombinator.com