Playing Against Watson

In the course of my postings about IBM’s Watson project, to build a computer system that can compete on Jeopardy!, I’ve mentioned Stephen Baker’s blog a couple of times.  Mr. Baker is the author of a new book on the experiment, Final Jeopardy.  He has a new blog post about his experience in playing an informal test match against Watson last summer, and links to an article he wrote for the Huffington Post describing the experience.

He makes some good observations about the relative strengths and weaknesses of Watson compared to a human contestant.  On the plus side, Watson does extremely well on clues that are linked to specific facts, like names or dates.  This is not surprising, because those facts give Watson an “anchor” for its evaluation of potential answers.  (In another post, Baker has observed, I believe correctly, that Watson would be almost unbeatable if Jeopardy! had multiple choice questions.)   Watson also does not panic, or suffer momentary mental blanks.  (It does, however, crash.)  It does least well, relatively, with the Daily Doubles and Final Jeopardy (the last round in the game), because these tend to have more complicated clues, and also have the additional complication of wagering.  Baker also suggests that humans may have an edge in clues with few words, since they can probably finish reading more quickly than Watson can finish its evaluation.  (Note, however, that all contestants are prevented from “buzzing in” to answer until the show’s host, Alex Trebek, has finished reading the clue aloud.)

Of course, human contestants, at least, also have categories in which they are relatively strong or weak.   I would probably do reasonably well in science-related categories; on the other hand, Ken Jennings, who has won more Jeopardy! matches than anyone, and who will be playing against Watson in the challenge match, said on one show that he had a list of a few hundred favorite movies.  I am pretty sure I have never seen half that number of movies in my whole life, never mind counting favorites.

All of this conversation about Watson is undoubtedly music to IBM’s ears; getting publicity for this new technology is obviously one of their key objectives.  Nonetheless, it seems that they really have made some important advances in processing free-form questions.

Incidentally, Stephen Baker’s blog also has a post with a “Jeopardy! Party Kit” that you can download (it’s a PDF).  Obviously, this is intended as publicity for the book, but it’s kind of fun.