## An All-Purpose IQ Test

March 6, 2011

I’ve written here a couple of times about some research results that appear to demonstrate that some “dumb animals” (specifically, pigs and dogs) are considerably smarter than we may have thought.  We’ve also recently seen a computer system, IBM’s Watson, designed to compete successfully on the TV game show, Jeopardy!, and talked about the Turing test as a proposed method of defining an intelligent system.

An article in this week’s issue of The Economist discusses an interesting project to try to expand the scope of intelligence testing to species other than humans.

What is the IQ of a chimpanzee? Or a worm? Or a game-show-winning computer program? Or even an alien from the planet Zaarg who can learn any human language in a day, can beat chess grandmasters ten at a time and can instantly factor the products of large prime numbers? At the moment it is impossible to say.

The difficulty with existing tests is that they are all language-dependent.  Even the Turing test requires a dialog between the tester and the subjects.  (And IQ tests have also been accused of having intrinsic cultural biases, a claim I don’t intend to pursue here.)  But two academics, José Hernández-Orallo of the Polytechnic University of Valencia, in Spain, and David Dowe of Monash University, in Australia, think that they have come up with a method for assessing intelligence that does not depend on language.

Their idea is based on a concept from computer science called the Kolmogorov complexity, also sometimes called the stochastic complexity.  I will not attempt to give a formal definition here, but the complexity of a string is  the shortest description of the string possible in some specified description language.  For example, if we consider the following two strings of 9 decimal digits:

314159265

222222222

the second string can be described as “nine 2s”, while there is no obvious shorthand description of the first (which is the first nine significant digits of π).  Similarly, the complexity of an algorithm can be measured by the length of the shortest possible program to implement it.

The researchers idea is to measure the intelligence of a given animal, or machine, by measuring the Kolmogorov complexity of the most difficult problems it can solve.  The  problems used would be structured in a way familiar to students of behavioral psychology.

The actual tests would employ the well-honed methods of operant conditioning, developed initially on pigeons, in which the test subject has first to work out what is going on by trial and error.

As I’ve discussed before, many of the tests that have been used to assess intelligence in other animals, such as dogs, were originally developed to be used on (small) people.  The tests used here will be generated by a computer system.  This, it is hoped, will eliminate possible human biases, and also allow tests of arbitrary complexity to be constructed, including ones too difficult for people.

It sounds like a fascinating project.