Ken Jennings on Watson
nydailynews.com
nydailynews.com
So my guess is that the daily news had more to do with that title than Ken. It doesn't seem like him, at any rate.
I know HN tends to dislike overtly dramatic headlines but copywriting is also an interesting art.
Nice image by Jennings. But I was surprised to hear him complain about the buzzer reaction time advantage. In an earlier interview with the Washington Post he discounted this, saying that the best human players anticipate the timing while Watson has to wait for the signal.
"On any given night, nearly all the contestants know nearly all the answers, so it's just a matter of who masters buzzer rhythm the best... But I wouldn't call this unfair...precise timing just happens to be one thing computers are better at than we humans. It's not like I think Watson should try buzzing in more erratically just to give homo sapiens a chance."
http://live.washingtonpost.com/jeopardy-ken-jennings.html?hp...
However, I think this would have been more interesting intellectually to isolate Watson's "thinking", with everything else on a level playing field.
You could pretty clearly see this if you were watching the game. Ken looked pretty bummed out every time he clearly knew the answer, but missed buzzing in by a few milliseconds or so.
Out of curiosity, how does Jeopardy work? Can you buzz in as soon as you know the answer, or do you have to wait for the host to finish?
Or, more precisely, you have to wait for the guy backstage who pushes his own button after the host has finished. If you push your button before he pushes his, you're locked out for something like half a second. His button is also connected to some light bulbs which are never shown on camera, which tell the contestants what's going on.
(But there are some human contestants who ignore the lights and try to time it anyway --- having practiced a lot at home, where they couldn't see them. One of them was a recent college champion, who was one of the pre-show speakers at a group viewing I was at for Monday's show.)
We already know that computers are good at the latter.
Everyone misses what should be knocked about Watson. The advantage isn't the buzzer per se. But rather the text message to get the question and the signal to indicate the question had been read.
The way it should work in the future is that Watson reads the question from the screen, listens to Alex, and sees the signal from the light.
Given the standard way Jeopardy text is written though, I suspect that all of these additional requirements won't change the outcome by much.
Watson is just a better Jeopardy player than humans. Are there other trivia games where humans are better? Probably so. But people don't watch these other games in prime time.
For that trivia is the wrong test. It's better to just give it paragraphs and ask it questions about the paragraph. And of course have increasing levels of difficult. More like the SAT reading comprehension test.
This would extract how it stores data brought in from structured/unstructed sources and the actual task of processing natural language.
I suspect it will be a fair bit worse at processing natural language than even the average person.
Looks like they must have overcome that issue.
As was pointed out by another commenter, if the game had been "everyone who knows the answer write it down and get credit", Watson probably wouldn't have won, and absolutely not by so much.
So I think Jennings is just cautioning us not to assume that AI has achieved human level language and information capabilities when really a totally unrelated and unimpressive other dimension of the game is the culprit.
If it had to do speech recognition for Alex Trebek, that would make it more difficult.