A Natural Language User Interface is just a User Interface | Hacker News Reader

A Natural Language User Interface is just a User Interface

medium.com

88 pointssyllogism16 comments

I'm waiting for the day where I can verbally say to my computer:

"No, please do not snap back the scrollbar when I move my mouse pointer farther away, keep scrolling!" or "No, bad computer! Do not select the entire URL if I single click, only when I double click!" or "It's a PDF! Don't give me a download box, render the PDF in the browser now!"

and the computer will understand what I mean, and update the GUI to work as I said.

RMS has this system: control-P! control-P foo control-s bar return

Works for him.

Here's a talk my colleague and I gave at O'Reilly Strata about natural language UIs: https://www.youtube.com/watch?v=lnV2JnNBM1I

We focus on NLUIs for the purposes of data querying. In many cases, natural language is concise and allows you to elide implied constraints.

For example, "Opportunities where I'm on the account team"

The SQL would be:

  SELECT * FROM OPPORTUNITY
  WHERE EXISTS
    (SELECT 1 FROM account_team_member atm
     JOIN opportunity ON opportunity.account_id = atm.account_id
     WHERE atm.user = me)

There's a correlated subquery in there. Even if you are a SQL expert, your time to query formulation is far higher if you use SQL than English. Drag and drop UIs are just a thin veneer over SQL, so a drag and drop UI requires understanding the underlying data model and join structure.

My argument here is that there are some key places where NLUIs really shine, especially when they are scrupulously combined with more traditional UIs.

This is indeed very nice :). I did my thesis on CCG, so this is all very dear to my heart. The paper I'm most proud of is CCG work too.[1] I have some unpublished follow-ups on light verb constructions I'd love to get released one day.

Btw, upshotdata.com returns 403 --- probably this isn't news to you, but just in case.

[1] http://www.aclweb.org/anthology/P10-1022

Your example has some contextual information contained in the overall structure of the sentence. There's also information in the text which wouldn't be contained in a vocal query. Your example might be answered by a query with the following link:

http://mars.nasa.gov/mer/mission/tm-opportunity/opportunity-...

Followed by the system registering that you are "on the account team."

I do agree that NLUIs can shine where there are constrained/specific interaction contexts.

My previous team studied allowing similar queries, and we didn't see anything so easy to interpret into a query. In testing, users typed in a lot of strings that couldn't be interpreted (by machine or human), or which were ambiguous.

It may be that people have gotten used to Google/Bing accepting crap queries and still showing a decent rezult. But, I suspect the main problem is they couldn't explain what they wanted to a human, at least not without considerable conversation.

I think the main 'magic' of NLUI is not the mapping. This person is projecting the model of how GUI works to NLUI.

A better way to think about NLUI is having another person have access to the real GUI - and you are merely commanding the person to achieve certain goals.

The thing about advanced NLUI is that it can convert really vague context sensitive commands to highly structured set of sub-routines that programmers will never be able to do easily.

'what should I major in ?'

'which university should I apply to'

'what headphones is best ?'

....

I find your examples sort of surprising? These don't seem so structured. Imagine a GUI that had three buttons:

Recommend University | Recommend Major | Recommend Headphones

Now...this set of capabilities is sort of an odd mix. But your application is going to be amazing if you can deliver on what your UI is promising here. If you can help someone make a better decision on their degree, that's life-changing. The UI that triggers that functionality seems quite beside the point.

The bot hype right now isn't about average UIs being replaced with text prompts of "What would you like to do today?", it's about large walled-garden ecosystems (Google, FB, Amazon, Apple, MS, Kik, etc.) funneling web searches and services through their own systems.

The article's most crucial point is that a natural language UI is just another interface to the limited set of functionality the application is capable of. But to the current big players in these game, these limitations don't apply: if the parsed commands fall through all of their many layers of specialty logic, they can always fall back to a generic web search.

By doing so, they will have provided the user with an interface that tries to understand what they want, while being no worse than if the user had typed the same thing into a search bar. And all the while, they have routed the user's intent through their AI, enabling better personalized recommendations, and tracking of click-through rates.

I'm glad someone wrote this article. "Chat-based" interaction is a lot like phone systems that ask you to describe what you're calling about, instead of giving a clear set of options. In my case it least, it usually ends in frustration, and falling back to the list of presets anyway. If I'm in a noisy area, or reception isn't perfect and the robot at the other end can't understand what I'm saying, then it's downright hopeless.

This was exactly my experience when I phoned my ISP and was met by an obviously recorded message that asked me what I want and then, apparently, switched to listening. I had no idea what to say to it! So I waited until it returned to the usual "wait to switch to an operator". Only a few calls later I learned from the human operator that I'm supposed to say something like "technical support".

(At the same time I have no difficulty and even prefer to call a local taxi using a voice menu based on my usual routes. These guys jump straight to business: "if your current address is X, press 0", "if you're going to Y, press 0", and then confirm; takes half a minute.)

The power of the chat interface is that you can include multiple bots and features in the same UI. No need for separate screens with a search field, filters and buttons. It's all in the same chat.

You're assuming that typing/talking to a bot is the easiest way to get things done. That's simply not true. We have custom UIs to interface with different domains because different domains work better/more efficiently with different UIs.

The following post has an analysis on this, with many great examples in the real world today http://dangrover.com/blog/2016/04/20/bots-wont-replace-apps....

The linked article is excellent -- really appreciated the history lesson. Thank you for the pointer.

The way I see it, this hype is a bridge to controlling software with voice. We're in a transition period and soon(?) our voice will be the UI. Remember that the best UI is no UI at all.

I'm stumped. Where have I seen before one of these Interfaces where users type a Line with a Command?

Reply on news.ycombinator.com