More on Watson’s Methods

Yesterday, I wrote about an article in the Wall Street Journal, by Yale computer science Professor David Gelernter, on the approach used by Watson, IBM’s computer system that plays Jeopardy!.   The Journal also has an excerpt from Stephen Baker’s new book, Final Jeopardy, which discusses some of the background of the project, and its early trials.  (I mentioned Mr. Baker’s book, and his blog, in an earlier post.)   As he reports, Watson is by no means infallible; the system sometimes misses an allusion that a person would catch, and sometimes it just comes up with wacky answers.  His account of some of Watson’s history and “growing pains” is also interesting.

The New York Times also has an article, by the novelist Richard Powers, on Watson.  Mr. Powers observes that, although the upcoming match between Watson and Jeopardy! champions Ken Jennings and Brad Rutter is undoubtedly a stunt designed to capture the public’s interest (it certainly seems to have captured the media’s), it is nonetheless work at the forefront of Artificial Intelligence research.

Open-domain question answering has long been one of the great holy grails of artificial intelligence. It is considerably harder to formalize than chess. It goes well beyond what search engines like Google do when they comb data for keywords. Google can give you 300,000 page matches for a search of the terms “greyhound,” “origin” and “African country,” which you can then comb through at your leisure to find what you need.

As I discussed in the post yesterday, Watson uses a massively parallel approach, on both the hardware and software levels, to try to come up with an answer.  The system decides whether to “buzz in” to answer based on a statistical measure of confidence that its answer is right.  Mr. Powers says that some people may discount this as a sort of gimmick.

This raises the question of whether Watson is really answering questions at all or is just noticing statistical correlations in vast amounts of data.

As he goes on to suggest, the question that is really raised is. what do we mean by really answering questions?   I don’t believe any of us is introspective enough to determine how our decision making process works, certainly not down to the “hardware” level.  Similar sorts of objections have been raised about the Turing test, proposed by the English mathematician Alan Turing as an operational test of a machine’s intelligence.  I have always found these objections to be vague in the extreme; if intelligence in playing Jeopardy! or the Turing test has some quality that transcends the ability to answer convincingly, I have never seen that quality described.

Regardless of whether Watson is a total flop, or wipes the floor with Ken and Brad, I think the project that built it can teach us a good deal about the problem of interpreting natural language.

 

NASDAQ Hack

Over the last decade or so, the nature of much malicious computer “hacking” has changed.  In the early days of the Internet, many attacks were motivated by a desire for thrills, or prestige; more recently, this hacking has become a more traditional and organized criminal activity, often motivated by money.  (There are also attacks motivated by ideology or politics, of course.)   So it is not surprising that the number of attacks targeted against financial institutions has risen.

According to a report published by the Wall Street Journal, some computer networks owned by NASDAQ (originally, the National Association of Security Dealers Automated Quotation system) have been under attack for most of the past year.  (The NASDAQ Stock Market is the largest US trading platform for stocks not listed on the New York Stock Exchange [NYSE].  It is the largest screen-based trading exchange in the US, listing 2800+ issues, and the largest in the world by trading volume.)  Apparently the attack did not affect the trading system core, but did affect peripheral Internet-accessible systems also run by NASDAQ.

Hackers have repeatedly penetrated the computer network of the company that runs the Nasdaq Stock Market during the past year, and federal investigators are trying to identify the perpetrators and their purpose, according to people familiar with the matter.

The exchange’s trading platform—the part of the system that executes trades—wasn’t compromised, these people said. However, it couldn’t be determined which other parts of Nasdaq’s computer network were accessed.

According to a follow-up story in the New York Times yesterday, the attacks first were discovered when NASDAQ staff noticed several suspicious files on their servers.  Apparently, the application that was directly affected was a bulletin-board sort of system for corporate managements.

The company said it had determined that a Web-based application on its servers called Directors Desk, on which corporations can store and share information, might have been affected. Nasdaq said the suspicious files “were immediately removed and at this point there is no evidence that any Directors Desk customer information was accessed or acquired by hackers.”

According to NASDAQ, the trading platform runs independently from the Internet — something that good security practice would certainly suggest.  Still, there is always a nagging fear that some out-of-the-way flaw may have been overlooked.  NASDAQ, the NYSE, and other exchanges are, reasonably, concerned about the effect incidents like this one might have on investors’ confidence, especially in an era when more and more trades are being executed via computer-based systems, as opposed to traditional exchange trading floors.

As you might have guessed, there is an investigation under way.