Jeopardy Champion Ken Jennings Q&A about IBM Watson | Hacker News Reader

Jeopardy Champion Ken Jennings Q&A about IBM Watson

live.washingtonpost.com

274 pointsacangiano84 comments

I was actually surprised at how funny he is - many times I find the personality type that supports being so good at that type of thing tends to be lacking in social skills (broad generalization of course) but he comes off as humble, personable and funny. His answer to the Now do you know how people felt when they were competing against you? question was especially interesting to me.

At one point during his winning streak, he purposefully answered incorrectly so he could say "what is a hoe".

http://www.youtube.com/watch?v=zKA0K7bf7Rc

Well, I think it was purposefully anyways. (btw, he's a Mormon)

I watched that match back in high school (his streak corresponded with my own academic bowl team's season), and "What is a hoe?" was my response, too.

I'd never heard that semantic sense of "rake" before, and felt somewhat scandalized that "hoe" would be a response on national television (while in high school, I felt great affinity toward the puritans).

I have known that from this: http://en.wikipedia.org/wiki/A_Rake%27s_Progress

Does anyone else think "hoe" is actually a valid answer there, though? It's hard to tell from Ken's facial expression whether he "purposefully answered incorrectly" or just found himself amused at the situation.

Hoe is not a valid answer because the other word you are looking for is "ho" which is a shortened form of "whore".

Rake, on the other hand, is a homonym.

He gave an author talk at Google a few years back, and he was surprisingly charismatic and hilarious.

50 minute video: http://www.youtube.com/watch?v=6YMRVwdeHB4

Not being from the US (and not getting Jeopardy in my country), i've never heard of this guy. He's incredibly entertaining in this interview, definitely caught me off guard considering how serious and straight-faced he seemed throughout the Watson matches (Thanks, YouTube).

This interview is the funniest thing i've read today (3 hour work commute, I read LOTS.)

Oh, and did anyone else keep wanting to upvote his answers?

Ken Jennings got famous as being the longest-lasting Jeopardy contestant ever. He remained undefeated in Jeopardy for over 70 games, and won over 2 million dollars. He was very likeable on the show as well, especially given that we were able to enjoy so much of the show with him on it, which is unusual for contestants.

He's somewhat notorious in the IT industry because his day job (not sure if he still holds it) was a programmer, which at least endeared him to me.

Notorious?

As in the Duran Duran song

Notorious: publicly or generally known, as for a particular trait

It's an acceptable usage, though I freely concede I overuse it when 'famous' would be more easily understood.

Ah, OK. Given the negative connotation (at least for its more common usage) I was wondering if there was some Ken Jennings backlash out there that I'd missed.

You might be conflating notorious with "nefarious," which is used when someone is notorious for something bad.

I hate to be one that argues about definitions, but:

generally known and talked of; especially : widely and unfavorably known

http://www.merriam-webster.com/dictionary/notorious

You don't have to be famous to be nefarious.

flagrantly wicked or impious : evil

Origin: Latin nefarius, from nefas crime

http://www.merriam-webster.com/dictionary/nefarious

Ken is super-great. He was a fun and friendly guy when I met him at a book signing. I've become a regular reader of his blog, where he shares random trivia and his love of cinema, hipster bands and the 1980's era of Games Magazine.

And if you noticed he's a computer programmer and you're wondering, the answer is emacs.

Of course it's emacs. He can actually remember all of the commands, functions, variables, and key bindings.

(Before people get their flamethrowers out, I spend more time in emacs than any other piece of software I own.)

And the rest of your time in a text editor, right?

I spend so much time in it looking for the "optimal" way to do what I'm trying to do. I know there's a command for it somewhere!

That's too bad about emacs. I was really starting to like him. ;-)

> ..and you're wondering, the answer is emacs.

He blog doesn't say anything about him using emacs. Just out of curiosity, can you link to where there's any sort of discussion about his setup?

He mentioned it on his forums once, way back in 2007 when I was preparing to be on the show myself:

http://www.ken-jennings.com/messageboards/viewtopic.php?p=12...

(Tuesday Trivia is his weekly email trivia contest.)

I'll bite: were you on the show?

Yeah; I had a blast. At least that's my excuse for following Ken Jennings' forum so closely.

I ended up 72 victories short of Ken, though.

http://www.j-archive.com/showplayer.php?player_id=3965

For those wondering his blog is at http://ken-jennings.com/blog/

That's ok. I'm sure Watson prefers vi.

That's odd, I've always thought that given its college-bowl atmosphere, Jeopardy! is uniquely poised to attract people who are both erudite and of quick wit.

The guy who hacked Press Your Luck fits more your stereotype; given its flashing lights and general brouhaha, PYL seems more of a magnet for that sort than Jeopardy!. There are also quite a lot of that sort in The Price is Right fandom (yes, it has a fandom!).

His Twitter feed is worth a read too, if it's legit (I guess he's not notable enough to rate a checkmark): http://twitter.com/kenjennings

"@IBMWatson, I'm-a let you finish, but homo sapiens is one of the smartest species of all time."

I was curious, and looked through his blog - he references that twitter account as his own here:

http://ken-jennings.com/blog/?p=2437

If you are trivia geek like me, you should sign up for his Tuesday Trivia emails..

Send an email to tuesdaytrivia@ken-jennings.com to subscribe...

I love how he handles the, almost guaranteed, defeat with great grace and humor. For comparison, Kasparov was in tears. Not the same scenario of course, but I truly admire Kennings' attitude and acknowledgment of IBM's (and humanity's) accomplishments exemplified by Watson.

I've never heard of Kasparov actually crying after losing to Deep Blue. Reference?

I don't know about tears, as such, but the NOVA documentary about Watson included a video clip from the end of the famous Kasparov-Deep Blue game. It showed him angrily storming away from the table after being beaten; the way the video was cut, in combination with the narration, gave the impression that he was something of a sore loser. Note, however, that it was a very short clip, and there was probably a lot of context that didn't come through.

Maybe he was angry because he felt like he wasn't performing at the top of his game, and that if he did perform at the top of his game, he might have won. I'd feel pretty angry if that were the case.

Sure. But that's the point of commending someone for handling loss with grace and humor: it's contrary to their immediate emotion.

True, but it's my understanding that he asked for, and was denied, a rematch.

Here is an interesting eyewitness writeup: http://www.chessbase.com/columns/column.asp?pid=146

It says nothing about crying but portrays a very gloomy mood. It seems that Kasparov made avoidable mistakes and that really got to him.

Kasparov was pissed off about the human intervention to Deep Blue during the game. The scene where you see him jump out is where his frustration at knowing and realizing that humans had altered Deep Blue get the better of him.

He demanded a re-match, and IBM declined.

"""I AM PLAYING A PRIME-TIME GAME SHOW AGAINST A SUPER-ADVANCED ROBOT! This is the coolest thing I will every do in my life by a factor of a million. The future is here."""

My kids think I'm a little weird because I think this battle is so cool (I think the more you understand about the implementation, the cooler this is), but this statement pretty much sums it up.

I suspect you meant the word 'ever' not 'every' in your quote.

Ken Jennings is FUNNY.

    IBM: "There's a lot of you in Watson"
    Ken: "If it goes amuck and kills humanity and stuff so sorry lolz my bad!"

    Chat participant: "I read your first in practically a day, I loved it."
    Ken: "I'd like to thank my mom for taking part in the chat!"

Q: "There was an article in the Post the other day about trivia now being trivial because of the ability to look up everything in an instant..."

A: "...in a not-too-distant future where nobody knows their state capitals anymore, maybe trivia geeks will be revered for their even more fantastical-seeming abilities! We will be like gods to you, carried on litters to your feasts."

Or this one:

"Big congrats to @IBMWatson & the amazing team behind its Jeopardy success. Watson is the real deal. Now: rematch on Dancing with the Stars!"

Did you know it took 4 hours to film the episodes because Watson kept crashing? Seriously.

http://i.i.com.com/cnwk.1d/i/tim/2011/02/16/nova_610x363.png

I had no idea. Out of everything I've heard, including the buzzer response time advantage, this seems to be the one thing that isn't really fair. If Ken was a narcoleptic and fell asleep every couple of minutes, he wouldn't have been allowed to be a competitor.

I still value what they're doing and of course it's fun to see the crazy thing play and win at Jeopardy, but I think the fact that it crashes continuously should at least be made more visible to the viewing audience. "Yes, folks, it can beat Ken at Jeopardy, but we still have to press the reset button every once in a while just like you do at home." Skynet may be getting smarter, but it still has fundamental problems.

... no, I'm sort of shocked that's been so covered up. Several of the team members have been at my university giving presentations and talks on Watson (including one of the software engineering practices behind it) and I'm somewhere between shocked and appalled it wasn't mentioned.

According to cnet[1]: "Correction at 12:04 p.m.: After this story was published, we heard from PBS producer Michael Bicks that it was not, in fact, Watson that crashed during the show's taping. He would like to make clear the following: "I missblogged last night--It was not Watson, but the system that was the interface between Watson and the Jeopardy computer, completely separate from Watson, that crashed during the taping."

[1] http://news.cnet.com/8301-17852_3-20032244-71.html#ixzz1EAdT...

It's unfair that Watson is allowed to basically use it's robotic reaction to time ring in instantly when the light turns on, but all audio and video daily doubles are avoided because IBM didn't bother to add audio and video inputs.

If the show is catering to Watson's shortcomings, why couldn't they cater to human reaction time shortcomings?

Edit: punctuation

I don't think your understanding of the situation is accurate: http://live.washingtonpost.com/jeopardy-ken-jennings.html#qu...

Watson can't do this: it only buzzes once it has an answer in mind and a sufficiently high confidence interval.

Well, they could have programmed it any way they wanted to. It would have been smarter just buzz in right away given that they knew it would be right most of the time.

Regardless, Watson dominated because it was confident about an answer most of the time before the light turned on. If a human also wants to buzz in right when the light turns on, he or she will always be beaten by Watson. So, the game again boils down to who buzzes in first, regardless of confidence.

I must have misread your original post - reading it again it makes complete sense.

Q: I have already read plenty of doomsday reports for the blue-collar workforce that this technology could replace customer service representatives, in-patient counseling, bank tellers, cashiers, etc. Do you think Watson could replace Alex Trebek?

I'm starting to wonder what Watson could do with the database of Stack Overflow.

Enter CLIPPY

It looks like you're writing an algorithm that is O(N^2)! Would you like me to optimize it? Let me take a screenshot so Jon Skeet can laugh at you. click

PROGRAMMER frantically presses CLIPPY's close button

I think Watson would kick Clippy's *ss.

I think a broken Etch-a-Sketch would kick Clippy's arse.

Watson is just Jon Skeet typing into a terminal while he's blocking on network input.

  Three Words
  Industrial Strength Magnets.

Priceless.

The guy's pretty intelligent (duh) and funny. Fortunately, he seems to be accepting the challenge quite well.

Last thing we need after such an AI breakthrough is someone complaining about the fairness of the game.

Ken's answer was hillarious

"They give Brad and me bottled water during every commercial break. I think you can see where this is going if the machine starts to build a big lead."

I'm more interested in a Q&A with Watson about Ken Jennings

It'll probably read Ken's wikipedia page out loud.

"But I wouldn't call this unfair...precise timing just happens to be one thing computers are better at than we humans. It's not like I think Watson should try buzzing in more erratically just to give homo sapiens a chance."

Wouldn't it be better to build a distribution of champion level response times and have Watson draw randomly from that distribution? That would seem more fair.

The first rule of competitive game playing is that anything in the rules is... game.

Illustrating that the computer has the advantage over humans is the entire point. Competitive game play is not about 'fair,' in fact, the best players specifically try to make the game as unfair as possible, within the rules of the game.

We already know that computers have an advantage over human reaction time. We don't need Watson to tell us that, my laptop can press a button faster than I can. This is like requiring each player to run a mile before being able to answer the question... and then letting one guy drive a Corvette instead.

And if the challenge IBM set for itself was "can we build a machine that travels faster than a person", that would be perfectly legitimate too.

Of course, the impressive thing is that Watson can answer the questions at all.

The point being that at the highest levels of play, all of the contestants know most of the answers. This is a very important thing to remember. It means that even in all-human games, it's not "do you know the answer" but "can you ring in fast enough."

So it still is really quite impressive that reaction time matters at all to Watson, because it implies that it can compete at the highest levels.

But the rules have already been bent to help Watson in areas where it's weak:

• receives question (and answers) as coded text, rather than having to sight-read or listen to them

• no audio/video clues

• no categories that require an extra Trebek-explanation

It may even be receiving a direct signal when it's OK to ring in, rather than assessing Trebek's cadence and the same light-indicator as the human contestants rely upon.

So other 'balancing' changes shouldn't be out of the question.

I particularly like the idea that the penalty for a moment-too-early buzzing could be eliminated, and/or treating all buzzes in a small window around the traditional earliest-allowed moment as simultaneous. That still rewards quickness, without heavily advantaging the computer's ability to perfectly synchronize with the buzzing-period.

Once two buzzes are considered simultaneous, the contestant chosen to answer could rotate in round-robin fashion, or (for maximum drama) be the one furthest behind.

The easiest and fairest fix would be to hook Watson's buzzer up to a dataset of previous Tournament of Champion question-winning buzzer times. Or just use Ken and Brad's previous buzzer times for even more control. Pick a new human-achievable reaction time for every question.

AFAIK, Watson was just using the light signal to decide when to start buzzing in. Just the light, ignoring when Trebek was going to finish saying something.

Presumably a human could consistently beat Watson to the buzzer by predicting when the light would go on.

Not sure about 'consistently', though trying to predict the light seems a reasonable strategy to attempt.

There's a lock-out that applies if you buzz too early, and the variance of the 'beats' between Trebek's finish and the light, if that's a human-mediated process, may be far larger than Watson's light-fired reaction time.

Here's a crazy idea... what if we changed the rules to Jeopardy so that if the buzzer presses are within a 50ms window then randomly select a contestant.

I think this would be a good change for even all human games.

I think that's a great idea, allow for different pools of time after the buzzer, 0-50ms, 50-100ms, etc and if you end up in the same pool as another player there is a random draw for who gets to answer first.

Seems to me they should let people click their buzzers as soon as they want to, and then just take whoever clicked first. Having to wait for the question to finish being asked seems like it makes it a lot more random rather than skill-based.

It used to be this way, didn't it? It seems to me I remember people used to often ring in with an answer before Alex was done reading the question.

According to Wikipedia: "Before Trebek's second season, contestants were able to ring in at any time after the clue had been revealed, and a buzzer would sound whenever someone rang in. According to Trebek, the buzzer sound was "distracting to the viewers" and sometimes presented problems, as contestants would inadvertently ring in too soon, or ring in so quickly that by the time he finished reading the clue, the contestant's five-second limit had expired. He also said that, by not allowing anyone to ring in until the clue was finished, home viewers could play along more easily, and faster contestants would be less likely to dominate the game."

http://en.wikipedia.org/wiki/Jeopardy!#Ringing_in

There's a whole book on the concept of game balance, focused on arcade fighting games (Street Fighter), written by David Sirlin, who balanced Super Puzzle Fighter II Turbo HD Remix and Super Street Fighter II Turbo HD Remix.

http://www.sirlin.net/ptw

http://en.wikipedia.org/wiki/David_Sirlin

Particularly relevant is the section on 'scrubs', the people who arbitrarily ban certain techniques for being 'cheap' instead of coming up with ways to counter them:

http://www.sirlin.net/ptw-book/intermediates-guide.html

As he works explicitly in the world of game balance, he acknowledges that it must always be reasonably possible to counter all usable techniques, or else the game is flawed. He even advocates banning techniques that honestly can't be countered, as a way of turning bad (unbalanced) games into good (balanced) ones.

http://www.sirlin.net/ptw-book/what-should-be-banned.html

So I suppose the upshot is that Jeopardy!, as it is now, allows any reaction time physically possible given the limitations of the buzzer and the constraints of the enforced time limit (which penalizes you if you attempt to buzz in before Alex finishes, or for approx. 0.2 seconds after he finishes). You could imagine a Jeopardy! variant with different rules in that respect, but it would not be the same game.

Interestingly, Ken Jennings says about Watson what most people say about ordinary players:

   Watson sometimes takes some time to get acclimated to a category, so starting at the bottom gives me a chance to rack up some money before it gets confident.  In theory!

Supposedly that was Mr. Jennings' reasoning for jumping around during his streak, that the other players didn't have quite as long a time as him to consider the category.

Not sure how tongue in cheek that remark was, though.

I guess the difference is, Watson can easily retain the learning that is built up through a category, even if he hops around the board the entire round. For humans, the context switch is a lot more costly and a category change can catch you by surprise.

A lot of this advantage is from correctly hitting the buzzer in time. When my college roommate was on the show, he said very often the trick was not buzzing in early and getting locked out. It was apparent that most contestants knew the correct answer for most questions. I imagine Watson would not have buzzer jitters.

Great stuff. “An away game for humanity.” Worth the read!

Slightly off-topic: Is there anyway Watson could be used to fix Google's search results? /s

Ken Jennings is cool. He's a pretty funny guy, too.

To provide a better challenge for both the humans and the machine, you should be able to buzz in as soon as the question is visible. Watson no doubt is doing most of the work in the time it takes the humans to load up the question into their mind.

Wait a second, did anyone else catch this?:

* * * * * * * * * * * * * * * *

Quote: THE OUTCOME

My understanding is that the shows are taped and you obviously know the outcome. You made the recent comment on MSNBC that in order to win you had to play and bet recklessly. Do humans have an advantage in terms of betting (game theory) Your comment seems to be a tell that a human won, any guidance appreciated. – February 15, 2011 10:59 AM Permalink

A. KEN JENNINGS : I would say that Watson has the wagering edge--like you say, it's all game theory and math, and even a cheap PC is pretty good at doing math at high speeds. That said, a human player might be more willing to take risks that Watson is too smart to try. In the practice games I saw, betting big on Daily Doubles and Final Jeopardy seemed like the only way to cancel out Watson's big buzzer advantage.

* * * *

HE DIDN'T ANSWER THE QUESTION! Is this confirmation that Ken Jennings wins?

Reply on news.ycombinator.com