Why Ken Jennings’s ‘Jeopardy’ Streak Is Nearly Impossible to Break

I'm gonna guess that they did solve it analytically, but for the sake of exposition and readers that might not know what a binomial distribution, they bring up the coin-flip as an illustrative example.

I played on a trivia team at the high school level, and most questions were structured like:

"This American architect, born in 1867, is known for works such as the Robie House, the Guggenheim Museum, and Fallingwater."

So they typically feed you information from least to most well known, and you can buzz in at any time. This rewards players who know the most trivia. Everyone who played knows Frank Lloyd Wright did Fallingwater, but a very good player would know his approximate date of birth and would buzz in immediately on hearing 1867.

I understand that this might make Jeopardy less fun for the audience, but I feel it would be a better test of trivia instead of buzzer timing.

I actually played Watson on the Jeopardy set they filmed the Watson episodes on. That "set" was just the auditorium here at IBM Research in NY, and they left the set intact for about six months after taping. Colleagues of mine hosted an academic conference in our building during that time.

Since Watson was on everyone's mind at that point (and we were literally looking at the set for it during people's talks), my colleagues asked the head systems researcher if he could give a special talk at the conference. He graciously accepted, and offered to do a full demonstration. There was only one volunteer from the rest of the conference - I think that if you're not a native English speaker, and not familiar with American pop culture and history, Jeopardy questions can be mystifying - so I gladly volunteered to play.

To answer the obvious question: I lost. Badly. In fact, not only did I lose, I lost a full-cheating version of Jeopardy where the audience (participants from the conference) shouted out answers, and I buzzed in using answers I liked. Not only that, but some of the answers that the audience was shouting were taken from Watson, looking at the readout it showed of candidate answers. (Some of the people were able to immediately recognize the correct answer, even though Watson was unsure of it. But keep in mind Watson came up with those answers.)

I would have lost to even a decent Jeopardy player, so losing to Watson is kind of overkill. But I think I learned a few things about playing Jeopardy, and how the human experience differs from Watson's experience.

I spent almost all of my cycles when the host was asking the question trying to figure out when he would stop talking. That is, I spent the entire question-asking period timing my buzz. I could not think of the answer and time my buzz simultaneously. The few times I thought I did know the answer, I did not know the answer when I buzzed in. I knew I knew the answer. The difference is that I did not have the answer in mind when buzzing, just that given a few seconds - which you get, by the basic protocols of the game - I could come up with the answer. I think other human players are the same: they spend the question-asking period timing their buzz, and are going by an intuitive feel for whether or not they know the answer. The actual answer recall does not happen until after they buzz in.

Watson is inverted. It only buzzes in when it is confident of its answer. In that sense, the game is not just about Watson winning the buzzer timing. Watson definitely could win every buzzer, but then it would lose the game; it's not accurate enough to get every question correct. So if Watson won every buzzer, it would answer incorrectly many times, and ultimately lose.

With that in mind, I hope it restores some of the romance. During the test run, the category was "Presidential Rhymes" with the clue "Barack's pack animals". Watson answered "Obama's llama's" immediately. Me and the other contestant just stared at each other, slack-jawed. I had seen it done on tv, but there was something far more immediate and impressive, since the head systems designer was ten feet from me, and the Power7 systems just a few rooms away.

So the computer's most effective weapon against Jennings was a 555 timer? That's cute.

Just checking my understanding.