Hacker News Reader

This is true. I've read all the Watson Special Edition papers numerous times[1].

The original Watson work was very good, and adding neural-network based models as alternative question parsers and answer rankers is a clear way forward.

It's a pity IBM decided to use the Watson name for everthing they do in ML now.

[1] http://brenocon.com/watson_special_issue/

> there's a major misunderstanding of Watson which isn't helped by IBM's Marketing efforts

you got cause effect wrong, since it's a glorified search engine, the marketing team did all they could do pass this off as intelligence and to muddle the water around what Watson could and couldn't do. when sales target still were falling short, they started dumping everything under the Watson budget to prop up the dept.

I have seen this pattern repeated in many large companies. One big product is successful and builds a nice PR bubble. Executives pile on anything they had in the pipeline with a re-brand, diluting the new brand.

I wonder what the solution is? Stronger brand owners within the company who can defend and be the quality control of what gets associated? Or is it hopeless?

MS did this with .NET. They even tried to brand a version of Windows with it for a while. I heard at one point no on in MS could really tell you want .NET was anymore.

> The Jeopardy playing Watson (DeepQA pipeline) was a landmark success at Information Retrieval, its architecture is built largely on Apache UIMA and Lucene with proprietary code for scaling out (performance) and filtering & ranking. I'm not an expert on IR so I won't comment further. This is very different from Neural Nets that are all the rage in ML today.

The question I would like to ask is that if the architecture is built on UIMA and Lucence, what exactly did "Watson" do?

Apache UIMA came out of the efforts from Ferrucci and others from TJ Watson Research Lab.

"Watson's" secret sauce was the filtering and ranking, you get multiple results from the models, which ones do you pick?

There was significant engineering effort put into reducing the latency of getting answers out of the pipeline as well, you had to optimize given the constraints of latency, accuracy and confidence from the pipeline.

"Watson" was a Jeopardy playing bot.

The main trick behind Watson is to take the various systems (parsers, search, et al.) and hacks (constraints imposed by the rules of jeopardy) needed by a jeopardy playing bot and put them all together.

So, in some sense, you could say UIMA is what Watson did -- because it allowed a lot of flexibility for researchers to combine their efforts. Ranking and filtering becomes of ultimate importance in a system like that because at some point you have to make a decision. However, it is terribly reliant on the other modules at least getting somewhere in the ballpark -- and the ranking is also not, by itself, anything impressive.

So, it's an interesting case of how far you can get by just setting a single goal and slamming everything together -- but as it turns out, for every new domain you wish to apply something like that to, that magic ballpark is hard to reach without a significant amount of engineering & research effort to come up with new systems/hacks combined with a lot of relevant data. In other words, just like any other adhoc AIish system with a particular goal. Change the goal, change the system.

So, of course Watson was oversold, it was a PR and Sales effort from the beginning. Sort of like AlphaGo or DeepBlue -- you might be able to find one or two interesting ideas in the bowels of such a system -- but the system itself is not a generic one.

If you can find them, read the papers on Question Analysis and Answer Coercion. I am still impressed by how powerful these systems were, and how effective. UIMA is a framework for tying algorithms together, and experimenting quickly with different combinations.

(locked-down IEEE versions: http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=617771...)

There's a lot more there than just search. See http://brenocon.com/watson_special_issue/

IBM has definitely created some cool stuff at various times. The problem, though, is that IBM misrepresents themselves in publicly-facing marketing and with paying clients during implementation.