The machine age

Watson will compete against “Jeopardy!” champions Ken Jennings and Brad Ritter tomorrow through Wednesday. (AP)


Forty years ago this December, President Nixon declared a war on cancer, pledging a “total national commitment” to conquering the disease. Fifty years ago this spring, President Kennedy declared a space race, promising to land a man safely on the moon before the end of the decade. And 54 years ago, Artificial Intelligence pioneer Herbert Simon declared that “there are now in the world machines that think” and predicted that a computer would be world chess champion within 10 years.

How have these bold efforts fared? We all know that Neil Armstrong made the giant leap for mankind in 1969, on schedule, and by the mid-1970s the sight of astronauts walking and driving on the moon became so routine that people impatiently started switching their TVs back to baseball. But to the layman it seems that efforts in cancer and artificial intelligence have failed; there have been no breakthroughs giving us the cure or the answer.

But the truth is more complex. On the one hand, the Apollo program truly was a spectacular success, but it also marked the end of progress — no human has traveled more than 400 miles away from Earth since 1975. Meanwhile, the war on cancer has made steady progress. In the last 20 years, death rates have decreased 21% in men and 12% in women, due to a combination of better diagnostic screening, reduced smoking and specific treatments for dozens of different cancers, particularly lymphoma, leukemia and testicular cancer. Those hoping for a single “cure” were disappointed because cancer turned out to be not a single problem but a complex arrangement of inter-related problems on which we continue to make incremental progress.

Artificial intelligence turned out to be more like cancer research than a moon shot. We don’t have HAL 9000, C-3PO, Commander Data, or the other androids imagined in the movies, but A.I. technology touches our lives many times every day, contributing hundreds of billions of dollars to the economy each year, and is improving steadily. Most of this appears behind the scenes, in applications like these:

* Spam filtering programs using A.I. learning and classification techniques correctly identify over 99.9% of the 200 billion spam e-mails sent each day.

* Your Android smartphone can recognize your speech and transcribe it into words quite accurately, despite your “New Yawk” accent and the honking cabby passing by on the street behind you.

* A.I. chess programs play at the level of top human champions (defeating the world champion 40 years after Simon’s prediction, not 10). IBM’s Watson computer will eagerly take on “Jeopardy!” champs starting tomorrow. In checkers, an A.I. program has achieved perfection — it can play flawlessly and it proved for the first time that checkers always results in a draw if both sides play correctly.

* With Google’s machine translation system, you type (or speak) a sentence in any of 58 languages and see (or hear) a translation into your desired language. The automated learning techniques in this project allow a new language to be added in two weeks of work rather than the two decades it previously took.

* Your Microsoft Kinect can recognize human motion and gestures well enough that you don’t need a video game controller anymore.

* Google’s prototype self-driving cars cruised a continuous 1,000 miles of winding highways and city streets without a single intervention from their human drivers. But autonomous vehicles are nothing new — unmanned drone aircraft have been used throughout the last decade in war zones, and modern airline autopilots are perfectly capable of flying a plane from liftoff to landing (but the human pilots usually prefer to do some work themselves).

* A.I. systems approve credit card transactions, insurance applications and claims and loan applications, while detecting fraud and calculating risk; they route billions of phone calls and Internet connections while relieving traffic jams and detecting suspicious behavior; they make stock trades better than humans (unfortunately we learned in 2008 that that wasn’t a high enough standard).

On the way to these achievements, the A.I. community learned several surprising lessons.

The things we thought were hard turned out to be easier. Early A.I. research concentrated on what seemed to be difficult intellectual tasks, such as playing grandmaster level chess, or proving theorems in integral calculus. But it turned out that these examples of logical thinking are actually not so difficult for a computer to duplicate; all it takes is a few well-defined rules and a lot of computing power. In contrast, tasks that we at first thought were easy turned out to be hard. A toddler (or a dog) can distinguish hundreds of objects (ball, bottle, blanket, mother, etc.) just by glancing at them, but it turned out to be very difficult to build a computer vision system that performs at this level.

Dealing with uncertainty turned out to be more important than thinking with logical precision. We think of a clever argument or solution to a problem as one that contains a series of irrefutable logical steps and are impressed when someone can come up with such a sequence. But this is exactly what computers do well. The hard part is dealing with uncertainty, and choosing a good answer from among many possibilities. The fundamental tools of A.I. shifted from Logic to Probability in the late 1980s, and fundamental progress in the theory of uncertain reasoning underlies many of the recent practical advances.

Learning turned out to be more important than knowing. In the 1960s and 1970s, many A.I. programs were known as “Expert Systems,” meaning that they were built by interviewing experts in the field (for example, expert physicians for a medical A.I. system) and encoding their knowledge into logical rules that the computer could follow. This approach turned out to be fragile, for several reasons. First, the supply of experts is sparse, and interviewing them is time-consuming. Second, sometimes they are expert at their craft but not expert at explaining how they do it. Third, the resulting systems were often unable to handle situations that went beyond what was anticipated at the time of the interviews.

Current systems are more likely to be built from examples than from logical rules. Don’t tell the computer how an expert solved a problem. Rather, give it lots of examples of what past problems have been like: Give it the features that describe the initial situation, a description of the solution used and a score indicating how well the solution worked. From that, the computer algorithm can learn what to do in similar situations (and sometimes in situations that are not so similar). This shift in focus is important because it is more robust, and because in today’s online world it is often much easier to gather lots of examples than to interview an expert.

The focus shifted from replacing humans to augmenting them. Another implication of the phrase “Expert System” is that it replaces an expert with a computer system. That approach made sense in the 1970s, when computers were rare and expensive and investment in computer technology focused on a limited number of high payoff applications. But today computers are everywhere and it makes more sense to think of aiding the performance of a human rather than replacing the human. That means that human and computer can each concentrate on what they do best. A good example is the web search engine, which uses A.I. (and other technology) to sort through billions of web pages to give you the most relevant pages for your query. It does this far better and faster than any human could manage. But the search engine still relies on the human to make the final judgment: which link to click on, and how to interpret the resulting page.

The partnership between human and machine is stronger than either one alone. As Werner von Braun said when he was asked what sort of computer should be put onboard in future space missions, “Man is the best computer we can put aboard a spacecraft, and the only one that can be mass produced with unskilled labor.” There is no need to replace humans; rather, we should think of what tools will make them more productive.

Let’s look at an example — automated translation from one language to another (for example, English to French) — to see how these lessons have been learned. In the 1960s, linguist Noam Chomsky correctly noted that there are an infinite number of English phrases; for example “big deal,” “really big deal,” “really, really big deal,” and so on. Therefore, the language could not be described by a finite list of examples, but only by a set of rules, called a grammar. The next two decades focused on trying to articulate these rules, but the quest proved elusive. Language is too vague, context-dependent, and creative to be captured with a definitive set of rules that strictly delimit the grammatical from the ungrammatical.

Around 1991, researchers at IBM resurrected an approach that had first been suggested in 1949: to treat translation in the same way that cryptographers broke secret codes in World War II, by counting the frequencies of letters and words and looking for patterns. This approach obviously oversimplifies the grandeur of language, and is doomed to be an incomplete theoretical model. But it turns out to be a good practical model. With the primitive computers of 1949 it was infeasible, but with modern computing power and the wealth of language examples available on the web it has proven to be the best technique available.

Here’s how it works.First collect examples of translated text. For example, a product brochure on my desk has the phrase “Installation: Additional instructions and troubleshooting assistance” under the heading “English” and the phrase “Installation: Instructions supplémentaires et assistance dépannage” under the heading “French.” Collect millions of phrase pairs like this. Then if anyone asks to translate exactly a phrase we have seen before, the computer can just look it up. With millions of known phrases, this will happen a fair percentage of the time.

But there will always be novel sentences that we have not seen before. For those we will have to assemble pieces that we have seen as parts of different phrases.

For example, when asked to translate “I need additional instructions” into French, the computer should recognize that we’ve seen “additional instructions” before and that it corresponds to “instructions supplémentaires.” It should also recognize that it has seen “I need” in other phrase pairs, and that it translates sometimes to “J’ai besoin d’,” sometimes to “J’ai besoin de,” and so on. Furthermore, it should realize that in all the French examples we have seen, “d’instructions” is more common than “de instructions.” Putting all these pieces together, we come up with the translation “J’ai besoin d’instructions supplémentaires.”

The whole process relies on statistics: on the counts of the number of examples we have seen. There are no explicit grammatical rules, but the system still implements the grammatical judgment that was known to the native speakers who created all the example phrases.

This approach may seem simplistic — all it is doing is counting up how often various combinations of words have been seen before. Of course, in a real system there are quite a few embellishments on this simple scheme, but the fact remains that it fundamentally relies on example. The key is that at some point the sheer number of examples passes a threshold and the system goes from performing terribly to being quite competent. This is related to the way in which a child learns a language. In both cases learning is involved, but the human brain is different from computer memory, in ways that we do not yet fully understand, so the analogy can only be taken so far. Still, it is striking that the team of Google engineers were able to build a translation system that handles 58 languages, even though some of the languages are spoken by no one on the team.



You can now use your phone to translate speech in real time, having a conversation that would not have been possible in the past. We’ll keep seeing ever more powerful and more intelligent applications of computer power becoming available to the general public.

Where is this all going? Will we reach a time when computers actually think?

Many computer scientists would say that the question is ill-posed, and that any answer to it wouldn’t mean much. After all, speakers of English have come to agree that “airplanes can fly,” using a verb that formerly was reserved only for birds, not for machines, and we generally agree that “submarines do not swim.” But these are facts about linguistic usage, not about aeronautical or underwater engineering. (In fact, speakers of Russian would say that submarines do swim.)

So computers will increase in their capabilities, and we will find ways to relate to them: as tools, advisors, workers, pets, companions,and other roles. But the words we use to describe what machines do, including whether we use the word “think,” has more to do with how we view ourselves than it does with the actual capabilities of the machines.

Peter Norvig is director of research for Google and the author of several books about artificial intelligence.


Watson will compete against “Jeopardy!” champions Ken Jennings and Brad Rutter tomorrow through Wednesday (7 p.m. on WABC, Channel 7). In a 15-question practice round in January, Watson won — but only barely. It was tied with Jennings until the final question.

Watson, which has been under development by IBM for more than five years, is not connected to the Internet. Rather, the software uses 3,000 core computer processors taking up the space of about eight refrigerators. Its hard drives are loaded with terabytes of books and information, which Watson searches with thousands of algorithms simultaneously. One of the biggest issues is determining what the question is really asking, translating a “natural language” query into something Watson can understand and find the appropriate answer.

IBM hopes to sell Watson to corporations — especially if it can impress Alex Trebek.

Share Selection