Hacker News Reader

Comment by cryptoz | Hacker News Reader

Watson has no clicker advantage. There is a human backstage that decides when Alex Trebek has finished speaking, and he lights up a board so the players know when they're allowed to click in. Watson gets the same signal and has to also press a physical button. There is a 300 ms delay penalty if you click in too early, and there was a question in tonight's show that seemed very much like Watson knew the answer but couldn't click in. He may have been estimating the clicker and getting it wrong.

While his mechanical finger might be a few ms faster than a human hand, it does not provide any significant advantage to him.

View on HN

Watson's advantage is clear, significant, and game-deciding. If you were familiar with the two human contestants you would know this. Barring an unlikely malfunction, Watson can't possibly incur the delay penalty from buzzing in early. Watson has no real-time audio or visual input, so it is certainly not anticipating the "clue finished" signal.

The instant before the "clue finished" signal is sent, Watson already knows whether or not it will attempt to buzz in. When the signal is sent, you've got a few dozen nanoseconds of wire propagation delay, plus the propagation delay Watson's "clue finished" signal-to-solenoid logic incurs, plus the delay of the solenoid itself. The final two steps almost certainly take less than 10 ms each.

Even with a very forgiving estimate of a 50 ms "clue finished" signal-to-buzz latency, it would be extremely unlikely for even the most anticipatory human to beat that. Granted, it does happen. In day 2 of the IBM challenge I counted 3 times when a confident and correct Watson was beaten to the buzzer by a human contestant.

These aren't two dim guys its playing against. For virtually every question asked, at least two contestants knew the answer. So no matter how good Watson is at knowing the answer (namely really really good), this isn't an advantage against a panel that is also really really good at knowing the answer. Watson's Jeopardy advantage must be elsewhere.

To quote Ken Jennings: "As Jeopardy devotees know, if you're trying to win on the show, the buzzer is all. On any given night, nearly all the contestants know nearly all the answers, so it's just a matter of who masters buzzer rhythm the best.

"Watson does have a big advantage in this regard, since it can knock out a microsecond-precise buzz every single time with little or no variation. Human reflexes can't compete with computer circuits in this regard. "

He goes on to say that the game should not be changed to account for this.

Human reaction time to visual stimuli is approximately 150 to 300 ms [1]. I don't know how Watson was configured, but even with the same visual cue I would guess it could press the button in a few milliseconds. That's a massive advantage, enough to win the buzz-in every time both a human and the machine know the answer by the time the light goes on.

[1] http://en.wikipedia.org/wiki/Reflex

Only if the contestant waits until the light goes on (what Watson has to do). If the human anticipates the moment the light will go on he can be _much_ faster. Or he can be too fast and get the delay - well, that's the risk.

The whole discussion looks fishy. Yesterday, the consensus seemed to be that Watson was at a disadvantage because of it's build in buzzer delay and the fact that it has to wait for the visual clue. Today, when Watson performed better, the consensus (at the moment) seems to be the opposite. So, the system hasn't changed, but the better Watson performs the more opinions are brought forward why this is an "unfair" contest.

There was no such consensus. I predicted (based on the practice videos that were released) that Watson would excel due to its buzzer reflex advantage. In fact, I never saw any predictions that Watson would have a disadvantage because of its buzzer system.

Ken Jennings recently said this, which sounds very much like the argument I've been making:

As Jeopardy devotees know, if you're trying to win on the show, the buzzer is all. On any given night, nearly all the contestants know nearly all the answers, so it's just a matter of who masters buzzer rhythm the best.

Watson does have a big advantage in this regard, since it can knock out a microsecond-precise buzz every single time with little or no variation. Human reflexes can't compete with computer circuits in this regard. But I wouldn't call this unfair...precise timing just happens to be one thing computers are better at than we humans. It's not like I think Watson should try buzzing in more erratically just to give homo sapiens a chance.

http://live.washingtonpost.com/jeopardy-ken-jennings.html?hp...

Reply on news.ycombinator.com