Hacker News Reader

Comment by klochner | Hacker News Reader

I'd like to see Roger go up against Watson, since both were "trained" for Jeopardy

My bet is that he'd lose. Beyond a certain level of knowledge, successful Jeopardy is all about hitting the buzzer first. Watson has a huge advantage in that respect because it gets the signal to buzz in electrically.

This.

The Watson show was a not a completely fair match-up since Watson had a much better average buzzer time than his human opponents.

If this was not the case, I would speculate a really good human opponent could beat Watson at least some of the time.

It's hard to say if that really makes it "unfair", since a lot of Ken Jennings' (and every other Jeopardy champion's) success was due to his reaction time with the buzzer. That's simply a huge part of the game.

On the other hand, nobody would dispute that a machine has faster reactions than a human, so maybe that makes it less interesting. Tough to figure out a great way to resolve that.

If you had data indicating what Ken Jennings' reaction time was, you could probably calibrate Watson to roughly match that.

Yes, but is that fair?

What is fair? Defining fair in terms of equal outcome is vacuous here.

For a bit I thought it would be more interesting to let the humans ring in as soon as they had read the question, but the problem is that today they would slaughter Watson, and three years from now Watson would slaughter them, and possibly without even updating its software.

I suppose the most fair thing to do would be to do away with the buzzer entirely. Let all players choose whether or not to answer all questions (that is, all three could get it right, all could get it wrong, one right, one abstain, one wrong, etc), and check the scores at the end. Without the buzzer advantage for Watson I suspect Jennings and Rutter still win... for another year or two, anyhow.

Having actually played against Watson, it certainly feels unfair when you've got a response you're 100% sure of on the tip of your tongue but you have to wait several long seconds for the question to be read. All the while you're desperately aware that Watson's reaction time is always 100 ms and yours ranges from 50-200 ms.

It is interesting to watch machines encroach on things we once considered exclusively human mental abilities. In the early days processors just multiplied large numbers faster than we could. Then they beat a grandmaster at chess and now a machine is definitely better at Jeopardy.

Actually, it should be harder for Watson to hit. Since Watson waits for the end of the question. A human can anticipate that.

The way it works in Jeopardy is that after the question is read, there's a guy who flicks a switch after which the betting window is open (and there's a visual cue so the players know when this happens). If you buzz before that you get briefly locked out.

Watson had a physical buzzer pusher. While he may have an advantage of reflexes, he did not do it fully electronically (obviously he 'electronically' activated the motor that pushed the button).

My method to eliminate the buzzer issue:

Allow both contenders to press the button within the first 2 seconds and, if they both buzz in within the 2 second window, the human answers first, because Watson could be trusted to "lock-in" it's answer.

Points are issued to whomever gets the answer right, allowing for both to receive or lose points on the same question.

One advantage Roger has is insane betting (which could backfire, obviously). I didn't watch Watson -- did it bet so aggressively on Daily Double questions?

Watson was able to analyze the current situation and make a rational bet on Daily Doubles. It analyzed it's comfort in the category the Daily Double was in, and then used a regression model to pick the optimal bet size, which factored in how winning or losing the Daily Double would change Watson's chances of winning or losing the game.

I'm not really sure if Watson's model goes as far to consider other players' Daily Double behavior, or simply examines other players' scores. It would make sense to try and look at the game theory aspect though, especially in an iterative-game scenario. If a player bets hyper-aggressively on Daily Doubles (and Final Jeopardy) and gets most of them right, you'd think Watson would want to take that into consideration.

Reply on news.ycombinator.com