If any questions cross your mind, write them on these cards. They will be collected at the end of Dr. Shang's lecture by the ushers. And we'll have a panel discussion, and which we'll try to answer as many of these questions as possible after Dr. Milner's lecture this afternoon. Now, the correct time for Dr. Milner's lecture is 3.30, the time that's printed in the program. 3.30. It isn't easy after 20 years to keep coming up with good ideas for the Nobel conference. The Nobel conference is uniquely ambitious in its attempt to interpret science, to make it accessible to the general public, yet not to oversimplify it. This year's conference was initiated by three Gustavus faculty members, George Georgia Caracas of the Philosophy Department, John Hulty of Mathematics and Computer Science, and Mark Kruger of the Psychology Department. The idea was simple enough to live, and yet complex enough to grow. It grew into its present form thanks to the efforts of several faculty members from a variety of different disciplines. Karen Larson from Sociology and Anthropology, Jim Welsh from the Geology Department, Deborah Downsmeyers from English, and we got some extra help from Anne Walcott of the Political Science Department, Jim Nye and Howard Court reference librarians. That's fairly interdisciplinary, even for cognitive science. I want to take this opportunity to thank them for the energy and imagination that they contributed to this year's conference. And now I'll turn over the microphone to another of my colleagues from the Psychology Department, Mark Kruger, who will introduce our speaker. Well, I find myself playing a role of a prop in a sequence of events that I assume many of you will recognize. The introducer comes to the podium announces the speaker, who will be the one who tells you what you really are interested in. Now, I'm sure that many of you have not found yourself asking, what's he doing up there? Because you're recognizing my role as the introducer. But if you were to ask that question, we would have to wonder what kinds of information you would have to know in order to recognize this sequence of events, what perhaps I might have to remind you of in order to recognize my role as the person who simply enables the next speech. So without further ado, I will fulfill my role and let Dr. Shank, chairman of the Computer Center at Yale University, tell you what kinds of things either a computer or a human might have to know. Naturally, I have to tell you about this long list of accomplishments. But I think that his energy and creativity will speak for itself. And I'll allow you to fill in the blanks as you elaborate based on what you know about what it takes to get to this kind of position and to make the contributions that Dr. Shank has made to our understanding of how people and computers might operate sensibly in our world today. Dr. Shank? This is the first time I've ever felt apologetic about not having a Nobel Prize. I suppose it can't be helped. Nobel couldn't have anticipated a computer science, I suppose. A lot of times I've been approached with the rubric that the answer is artificial intelligence. What's the question? I think it's actually an interesting question to wonder about what artificial intelligence is about and why we keep hearing so much about it. One of the reasons we hear so much about it is the advent of the personal computer. Out there in the world are lots of companies born in garages who would like nothing better than for you to own a personal computer. They're not quite sure what you're going to do with these personal computers, but they're sure you should become computer literate.  I, for one, am not particularly interested in computers, which may seem like an odd thing for a chairman of a computer science department to say. I'm interested in people. And in fact, I'm interested in getting computers to the point where you don't have to be computer literate. That's what artificial intelligence is about, although it really didn't start out as a mission to make people not have to worry about dealing with computers. And in fact, those of us who work in AI, I don't think we really care about how people deal with computers. What we care about is, suppose we were going to have this machine that was really smart. Suppose 2001, the HAL computer or R2D2 of Star Wars really existed. Those movies are the kind of, people who go to those movies who squirm in their seats, not because they were afraid such things might exist, but because they can't figure out how they might build them. Those are AI people. And I, for one, watched 2001 at an impressionable age and said, gee, I wonder how we would do any of that. And I've been spending the last few years trying to figure out how to do it. I started working on the problem of language because it's fairly obvious how and other smart computers all speak. And they all understand everything that you say. Well, I wondered how it was that people could do all that understanding and speaking. I began to look carefully at language to try and see if I could answer that question. Language is really complicated. It isn't anywhere near as complicated as when you first look at it because we speak it so effortlessly and you all are sitting there presumably understanding me. But no computer is capable of doing that, not today. And although we have made a lot of strides in artificial intelligence, what's happened more is that the media has gotten interested in artificial intelligence and has made people believe that we've solved all the problems. We haven't, but we're pretty good at identifying some of the problems. So let me identify some problems for you. Take my favorite word, take. And I will tell you some sentences. John took the bus, John took the candy, John took a punch, John took it easy, John took a nap, John took a letter, John took his temperature, John took his time. And now, with your first line of computer program, you all know how to program I assume, right into your computer, the dictionary definition that you'd like your computer to have for the word take. When you take a bus, is that the same as when you take candy? When you take candy, you own it and are about to eat it. Not true of buses. When you take a bus, you ride on it. When you take a nap, you don't actually take a nap at all. There is no such thing as a nap, you just sleep. When you take it easy, you haven't taken it anywhere, there is no it, it doesn't refer to anything. When you take a punch, you haven't done something you wanted to do at all presumably. When you take your temperature, I don't want to start on that one. If you think this is just true of verbs, it isn't. For example, consider the word hand. John needs a hand, give John a hand. John asked his wife for her hand. John raised his hand, John is an old hand. John had a hand in the cookie jar. John had a hand in the robbery. Look at those two, they look so similar. Yet in one case, John is probably a small child. In the other case, John is accomplice to theft. They look similar. Where did I figure out that John was a child and John had his hand in the cookie jar? I don't know. Sounds right though, doesn't it? See that's the stuff that's hard about language. Oh one last thing, hand has the delightful property of also being a verb as in hand it over. So what do you teach the computer about hand? Is it a noun? Is it a verb? When should you think of small children? Well the problem we've had is that language is much more than the sum of its parts. We would like very much to be able to teach the machine all the words in the English language. So early stages of computer science people put in dictionaries. That seemed like a reasonable thing to do. But dictionaries don't tell you very much about the solutions to the kinds of things we're doing in these sentences that I'm talking about. Furthermore, the English language and in fact every other language has a delightful property of allowing you to say so much more than the words imply. For example, consider these few sentences together. Each one apart and you will see. I want you to think about in your own mind what else you're thinking about that might also be true when I read these sentences. That is to say process these sentences but do what we do in AI which is kind of think about it a little bit as you do them. John hit Mary. John hit his child. John hit his teacher. John hit a policeman. John hit Muhammad Ali.  Do you feel that way when he hits Muhammad Ali? You worry about John. Well, we got a problem here. How are we going to teach the machine to worry about John? What's different in that sentence? Does syntactic structure is identical? Seems to be the same thing. Yet there's so much else going on. Continue what I'm talking about.  John went to a restaurant. He ordered fish. He left a small tip for the waitress. What did John eat? Fish, right? When did I say that? Who served him the fish? The waitress, right? When did I say that? Was he happy with the service? No. Why? He left a small tip. Where does it say that? Said he left a small tip. He doesn't say anything about anything else. What do we to learn from all this? What we learned from this is as we began to look at language, we began to realize that the problem in teaching computers to understand language wasn't language at all. The problem in teaching computers to understand language was knowledge. We have to teach the machine what we know about the world. I was introduced by reference to scripts which is a theory I have invented some years ago and sort of half abandoned since. But what I'm talking about now is something that we call a script. What's a script? A script is a set of little things you know about life that are so mundane and boring that you can't imagine talking about them and therefore don't, but expect that your hearer will in fact know the script so that we can refer to it together. So when I talk about a restaurant, we have to teach a machine, if it's gonna understand restaurant stories, about restaurant scripts.  Somebody brings you the food, you pay that person, you may leave that person a tip, all the little things that I'm telling you. And in fact, if you start writing down everything you know about restaurants, you discover that you know a great deal about restaurants. So what I just described trivializes the process. There's in fact a lot to know about restaurants. Well now that we teach the machine everything we know about restaurants, how will it do? Will it understand? Well first let's see if the slides actually work. Can you read that? Okay, I can read it. What that is, is input and output to a computer program that we wrote about 10 years ago that processed stories from the newspaper and it processed these stories by use of scripts. This is a story about a car accident that had appeared in the New Haven Register and what it has in it is the ability to, the program has the ability to read the stories according to the scripts and I'll read you the story. It says, Friday evening a car swerved off route 69, the vehicle struck a tree, the passenger in New Jersey man was killed, David Hall 27 was pronounced dead at the scene by Dana Blanchard, medical examiner, Frank Miller, 32 of 593 Foxon Road, the driver was taken to Milford Hospital by Flanagan Amulance, he was treated and released, no charges were made, patrolman, Robin and Afrio investigated the accident. And underneath that it says, bigger letters I hope you can see, the output of the computer. The computer would read that story and summarize that story, so it summarized it as, an automobile hit a tree near Highway 69 four days ago, David Hall age 27, residents in New Jersey, the passenger died, Frank Miller age 32, residents at 593 Foxon Road in New Haven, Connecticut the driver was slightly injured, the police department did not file charges. And what you see below that is a Spanish summary, because when the program is busy understanding these stories it's building a mental representation of the story and once it does that it's taken the English out and so it begins to put the language back, it can put it back in Spanish just the same and you can see that in fact it has understood it a different way because it's not just spitting the words back at you, you get the car swerved off Route 69 and then that's paraphrased as an automobile hit a tree, the program in fact is not as good at speaking English as it is at understanding it. And then you can ask the program questions in English. So you can say, was anyone killed and it said, yes, David Hall died, was anyone hurt? Yes, Frank Miller was slightly injured and if you look in the story you'll see that it doesn't even say that Frank Miller was slightly injured. What it says was, Frank Miller, the driver was taken to Milford Hospital by Flanagan Ambulance, he was treated and released.  And so it infers that he was slightly injured. All right, put the lights back on. Well, so we were able to build programs like that and we built a program that simulated an US Senator by using scripts and would answer questions the way a Senator would answer them. It had a right wing script and a left wing script and we could put either one in. We had a program that you read the UPI Newswire and was reading stories according to scripts all the time. And it was summarized and answered questions and we were proceeding on happily and merrily. Two things happened that are worth mentioning. The first is best mentioned by an example so I'll give you the example and the second will be the subject of the rest of the talk. The example I like to give and I think it's really a problem in artificial intelligence, we show our results. It's easy for me to come up and say, see our program reads newspapers and for you to go home and say, you know there's a program that can sit around and read newspaper and understand everything there is. And in fact, we understood car accidents and a few other things, but not everything that there was. The leap between what we understood and everything was a big leap. And the best way to describe the leap and to understand a little bit about the nature of artificial intelligence is to look at the failures rather than the successes. So my favorite example of a failure was done also in my laboratory, was a program which in fact succeeded in the end. Its mission was to make up stories of the kind you might make up to your child. And so it invented a small world, world of living and talking animals who ran around and had a social society and interacted. We gave the animal some goals and some plans and some scripts and said, go ahead and behave. And stories came out. I'm gonna read you some of the stories that came out that didn't work so well. There's a good way to understand the kind of things we have to do in AI. One day, Joe Baer was hungry. He asked his friend Irving Bird where some honey was. Irving told him there was a beehive in the oak tree. Joe threatened to hit Irving if he didn't tell him where some honey was. Our program was smart enough to generate that and in fact it took some intelligence but it didn't understand what it generated and therefore didn't know that it had told itself the answer to where the honey was. So we put that information in. What we had to do is we forgot to tell it that beehives contain honey. It's easy, we did it. Next story. One day, Joe Baer was hungry. He asked his friend Irving Bird where some honey was. Irving told him there was a beehive in the oak tree. Joe walked to the oak tree. He ate the beehive. It's easy enough to fix these things but we do start to wonder about are we gonna keep on fixing where are we gonna run into trouble. Here's another example of the kind of trouble you might run into. Henry Ant was thirsty. He walked over to the river bank where his good friend Bill Bird was sitting. Henry slipped and fell in the river. He wasn't able to call for help. He drowned. That's actually an all right story. It isn't the story the program intended to tell because one of its characters died in the middle of the story. How do we fix that? Well the problem is that people in this world ants drown because they don't have any friends or ability to get them out and because in fact we hadn't represented drowning quite correctly. So this time we figured we'll represent drowning correctly, we'll have gravity pulling an animal down and we'll have an opportunity for friends to help you out if you call for help and so on. Here's the next story that came out. Henry Ant was thirsty. He walked over to the river bank where his good friend Bill Bird was sitting. Henry slipped and fell in the river. Gravity drowned. It didn't have any friends and couldn't ask for help. It was firmly located in the river according to our representation. Once upon a time there was a dishonest fox and a vain crow. We tried to get the programs to understand and generate asaps fables.  He noticed he was holding a piece of cheese. He became hungry and swallowed the cheese. The fox walked over to the crow at the end. Some of you will recognize the story we were trying to generate, but we didn't put in all the knowledge that's necessary to get it to come out right.  It's a good rule. But we had to change that somehow. We changed as follows.  When up came Bill Fox. Bill saw the cheese and was hungry. He said, Henry, I like your singing very much. Won't you please sing for me? Henry flattered by the compliment began to sing. The cheese fell to the ground. Bill Fox saw the cheese on the ground and was very hungry. He had previously been hungry. Now he sees it again and he's very hungry. He became sick. As we have a rule in there that says, if you're very, very hungry and don't get fed for a certain period of time, you get sick. Henry Crow saw the cheese on the ground and he became hungry. Now we had a little rule in there that if you want something, you've got to go through a certain ritual of deciding who owns it and offering him to pay them for it or bargain them for it or whatever depending upon your relationship with the character who owns it. We ran into problems here because Henry Crow saw the cheese on the ground and he became hungry, but he knew that he owned the cheese. He felt pretty honest with himself, so he decided not to trick himself into giving up the cheese. He wasn't trying to deceive himself either nor did he feel competitive with himself but he remembered that he was also in a position of dominance over himself so he refused to give himself the cheese. He couldn't think of a good reason why he should give himself the cheese because if he did that he'd lose the cheese. So he offered to bring himself a worm. If he'd give himself the cheese, that sounded okay, but he didn't know where any worms were. So he said to himself, Henry, do you know where any worms are? This is a definition of an infinite loop in case you're interested in computer science. All right, so the first point that I want to be able to make to you is that computers have to have a lot of knowledge. Now we in fact can sit down and try and stuff all that knowledge into a machine and in fact people in AI try and do that. But here's the problem I have with that. We had our little programs that understood car accident stories and earthquake stories and stories about war and a lot of things for which you could write mundane scripts. And we used to demonstrate these programs to incoming visitors. We had a particular 10 stories or so that we showed over and over and every time somebody came. And after about the 150th time of seeing one of these stories be processed and understood, I asked myself the question, I mean, I was bored, right? I mean, I'm seeing this story demonstrated over and over again. Why isn't the machine bored?  See, it didn't to me. It seemed to me that the machine should have been really sick of this story and should have said, by God, if I have to read about that earthquake in Italy one more time, I'm gonna get sick. But it didn't do that. In fact, it didn't even know it had read that story before.  Might be able to trick you into reading the same story twice, but by the third time, you'd have had enough. So what was wrong with our programs? Well, they didn't really remember, did they? And what's more importantly, more important, not only didn't they remember, but in a serious sense, they didn't understand. Now, this is in a form of sort of open self-criticism. What I was concerned about was that if understanding was to be able to simulate behavior, better to say, to understand a story means to paraphrase it or answer questions about it or translate it or something. And it also, you have to simulate the behavior that you didn't want to simulate in the first place, that you hadn't intended to behavior to do, but happens also to be the case. Hence, the problem that I'm fascinated with to this day, the problem of reminding. Suppose the following happens. Suppose you've been, imagine the following situation. You've been to McDonald's many, many times, but I find out that you've never been to Burger King. And I say, boy, you really missed something. You're in for a treat. I'm gonna take you to Burger King. So we pack off and drive off and we go into Burger King and we stand online and we order our hamburgers and put them on our trays and sit down. And you say to me, you know, this reminds me of McDonald's. Well, I discover an interesting thing while you're saying that. You've understood Burger King in a profound way. They're elements of the same sort. And you need to understand things in terms of a particular mental framework. Sometimes this is useful and sometimes it's gonna get in the way. I had an experience just about three, four days ago, which I've been pondering ever since, which I'll share with you. My father, who was a fairly old man by this time, decided he'd like to visit his boyhood home, which happened to be in a town called Mountain Dale, which is in New York, a very, very small town. So we drove to Mountain Dale to visit the house that he grew up in. And the people who were in that house weren't kind enough to show us around the house. And my father asked the woman who was showing us around the house, he said, but where do you work? And I could see why he was thinking that because this town was a very small town and he lived in it in 1918 or something. And he didn't really understand where you could get a job in Mountain Dale. And she said, well, I work in Monticello, which is a bigger town, maybe 15 miles away. And he said, Monticello, how could you possibly work in Monticello and live in Mountain Dale? And my father's not an intelligent man. And in fact, he had just been on the freeway that connects Mountain Dale with Monticello, not more than about five minutes earlier. And it's all of 10 miles away. And he couldn't understand how you could possibly make this trip. Why? Because the only way he ever could get from Mountain Dale to Monticello was by horse. This was not a very populated, well-built up area. And he had a horse and nobody was very much money and nobody had a car. And my father was in a sort of frame of reference at that time, which allowed him to look at the world in 1918 glasses. He couldn't see it in 1984 glasses, even though he'd just been in 1984, not five minutes earlier. But right then, he was imagining and understanding the world in 1918 terms. He was asking about where the brook was and where the barn was and where the trees were and what the birds were and various things that were interested in him that were from that frame of reference. You see, when we understand something, we understand something in terms of what you might call a domain, some limited framework of understanding. And we bring in that domain. What does that domain contain? It contains a set of expectations about the world, a set of beliefs that certain things will occur in certain orders. The insight in scripts was that we have these insights about restaurants or car accidents, about expectations about what will happen in the world. The failure of scripts was these things change over time. Now, it's possible to get back into a 1918 head, so to speak, but it's also important to be able to understand that you should change the script that you have and have to change the script that you have all the time. Once upon a time, I went to General Motors and I ate in the company cafeteria. And in the company cafeteria, I was a guest speaker, I ate in the company cafeteria. And it turns out, what you have to do there is you have to, you can't order from the waitress, you can't just talk, you have to write their order down on a check. And I filed this away somewhere in my head. And maybe a year later, I went to Bell Laboratories and the same thing happened to me. And I was reminded of the experience at General Motors. Now, you might say, so what? Of course, it happens to me all the time, but never happened to my computer programs. Not once did they ever say, you know, this story about this earthquake in Italy reminds me of that story about that earthquake in Yugoslavia, or in fact reminds me of a plane crash in Yugoslavia, that the ability to draw generalizations from what you read is critically important. And one way or another, it became clear to me that what was most important from our programs in terms of understanding was lacking. And what that was was an ability to make new generalizations or to put it another way, an ability to learn. You see, I got smarter as a result of Bell Labs and General Motors experience. I changed my restaurant script. I put a little codicell in it. It says, whenever you go to a company cafeteria, don't expect to talk to the waitress, expect to write it down. Now that expectation may turn out to be wrong too. What happens in life is that we come up with all kinds of expectations and we confuse them. One way or another, we have these expectations, see which one is right at the right time, we bring it in, we see if we can get it right. That's how we learn. Try to crystallize that for you some. Consider the following story. This is my very favorite story in the whole world. It's not very interesting, but it's my favorite story anyway. It's called The Stake in the Haircut. The ex described how his wife would never make his steak as rare as he liked it. When this was told to Y, it reminded him of a time 30 years earlier when he tried to get his haircut in England and the barber just wouldn't cut it as short as he wanted it. Now, why is that my favorite story? Well, first of, it's a true story. I've been collecting reminders for a long time now. It's my favorite story because the two things, the remindee and the remindand, if you like, don't seem to have a lot to do with each other. Furthermore, one took place 30 years earlier than the other. So now realize what had to have happened. We had to have this man, what I called Y, we've had to have him having an experience in his mind for 30 years sitting there waiting to be reminded of, sitting there quietly. And in fact, I asked this particular guy, have you ever thought about that? He said, no, I haven't thought about that in 30 years. Okay, now I ask you the question, and this is in fact the essence to me in any case of what artificial intelligence is really about. I asked you the question, can you build me an algorithm that would replicate that behavior? And what would it look like? Well, I'm gonna try and do that for you in a sketchy form right now. Imagine what's going on. X is saying, my steak is never cooked as rare enough as I'd like it by my wife. Y is now in the position of trying to understand that sentence. What does he do to understand that sentence? Well, first thing he does is he builds, it brings in what we call a knowledge structure. That is to say, some piece of information with expectations of a kind I've been talking about. Well, what one would work here? That's a critical question because if you build in the, bring in the wife cooking steak knowledge structure, if that was the one you thought was living in the head, you would never get reminded of the haircut. You have to build in and bring in a knowledge structure that's general enough to contain other information. In other words, if you process something with a very narrow domain or a very narrow script, you'll never see the broader implications. And what makes people intelligent is that they can see the broader implications. So what he had to have been using here is something which I might call somebody provide service for somebody else knowledge structure. What would the expectations be in a somebody provide service for somebody else knowledge structure? Well, it would be something of the order of, you ask somebody to do something, if they feel that it's appropriate that you asked, and if they feel that they're capable of doing it, they do it. Okay, let's assume that's what it looks like. That's a reasonable knowledge structure and people have that in their heads. And a lot of what AI is about is trying to find out what kinds of structures people do have in their heads. So people have that structure in their heads. Let's assume why I had that structure in his head. Still, how did he get to this particular structure? Well, what he had to have been doing is he had to have been asking himself, why? He had to have been trying to explain what I call the expectation failure. The most important thing in learning is failure. It's a little message for those students out there who fail from time to time. It's a good thing because when you fail something, make a mistake, do something wrong, have a bad expectation, you try and correct it. You try and explain the failure. So let's assume why we're sitting here trying to explain the failure. What's the failure? The failure is that he expects this wife to do what she can do, was asked to do because it was both appropriate to ask her and because she was capable of doing it. It's not hard to cook steak rare, after all. He had to construct an explanation. Why couldn't she do it? He constructed one. And in fact, you can see which one it is. The explanation that he constructed is the wife of X must have believed that it was inappropriate to do anything that extreme. Now how do I know that's the explanation he constructed? Because that explanation is also the explanation of his haircut story, right? Listen to the haircut story. He tried to get his haircut in England 30 years ago and the barber wouldn't cut it as short as he wanted it, right? In fact, the explanation of that, we'd assume in Y's mind, there could be many explanations, but Y had seemingly explained it with, he must have thought it was inappropriate to do it that extreme. Okay, now what does that tell you about human memory? It tells you that there's this little flag waving around in this man's head for 30 years saying the following. If you ever run into anything else where somebody doesn't do something because it's too extreme, even though they could have done it, call me. Now why would you call them? Why is, what's the evolutionary value of that kind of index into memory? And the answer is because I, as a processor, would like to compare the next version of expectation failure that I have with my previous expectation failure so I can rectify the expectation failure so I don't fail next time. In other words, this guy would like to stop believing whatever it is he believes that's an error. How do we construct new representations in our mind? How do we construct new knowledge structures? And the answer is by finding two instances where expectations have failed in identical ways, where the explanation is identical, comparing these two guys, and seeing if in fact they have something in common that causes you to want to create a new structure. Think for a moment about Burger King and McDonald's and imagine that you've never been to either of them, you've only been to fancy restaurants. The first time you walk into either one of those restaurants, you should have a number of expectation failures if anybody told you they were restaurants. They don't behave like restaurants, let alone I'm not even making a comment on the food here. The reason they don't behave like restaurants, among other things, is you stand up while you're ordering, you pay before you get your food, and then you have to bring your own food over the table. They behave like cafeterias, in fact. And you should be reminded of a cafeteria. Well, in fact, what we've been discovering is that understanding means being reminded of the thing in memory that is closest to the experience you are currently processing. As a public lecturer for years, I used to be infuriated with somebody, there's always somebody in the audience who would raise his hand and say, isn't what you're saying just like the work of X? I'd be mad, I mean, I wasn't like the work of X, how could they not see that? Now, I say, ha, you've just proved my theory. You understood what I said in terms of the thing closest in your mind to it. And that, in fact, is how we understand. It's how we remember jokes, even. I'll give you a story, remind somebody of a joke. X's daughter was diving for sand dollars. X pointed out where there are great many sand dollars. But X's daughter continued to dive where she was. X asked why. She said the water was shallower where she was diving. Everybody reminded of the joke yet? This reminded X of the joke about the drunk who was searching for his ring under the lamppost because the light was better there even though he'd lost the ring elsewhere. Now, why are jokes relevant? Well, jokes are often built on expectation failures. So what's going on in memory is that we are processing new information in terms of old information. So to get back to my problem with my story understanding programs, any story understanding program that understood an earthquake story without looking at all the other earthquake stories it knew to get the expectations and without comparing any expectation failures that might have occurred between the two would have been doing a poor job or to put it another way, this program in reading about 27 different earthquake stories in Italy all which took place in the same day at the same time should come to the conclusion there were a great many earthquakes taking place in Italy at least on that day or else be bored. In this particular case, you would expect to be bored. All right, let me tell you another story. I was, I gave a lecture in Puerto Rico not too long ago and as you might imagine, I took a vacation afterwards. I was walking along the beach the first day in Puerto Rico, I had arrived at the hotel at night and I took a little walk in the dusk and there was a sign on the beach and it said, no swimming at this point, water's too rough and I got upset because I'd gone to this nice hotel or this nice beach and you couldn't swim in the ocean. Wasn't, didn't pretend for a good time. The next day I was walking along and people were swimming in the ocean happily at this point and I walked along to a walk again and I came upon a new sign and this sign said, don't go beyond this point dangerous on the beach. I said, what is it with this place? I couldn't figure out what was dangerous about it except that they meant it wasn't hotel property anymore and they were like real Puerto Ricans on the other side or something. And all of a sudden I was reminded of something. Now this story works a lot better in Connecticut or states near Connecticut than it does here just because you're missing a piece of information which I'll tell you. In Connecticut there are signs on the road that say, I've never seen them in any of the state, road legally closed, pass at your own risk. That's not so funny except that they are on all the highways. And you can be on a, in a state 95 Connecticut turnpike in a traffic jam four lanes blocked in every direction at five o'clock and big signs that say road legally closed. Now people in Connecticut find these signs rather humorous and after a while don't notice them but when you first arrive in Connecticut you tend to think you shouldn't go on the road because it says that. Well, one of the things that I began to ask myself was why does Connecticut put up these signs? The only answer I could come up with is if in fact there was an accident sometime they would like to say it wasn't our fault. So they put up this sign and said see we had a sign too bad. Now why was I reminded of this? That seems obvious why I was reminded of it. Because that's what they were doing in Puerto Rico. They were putting up a sign to cover themselves. They didn't want to take the chance in case somebody drowned. See, we had a sign.  We had a sign. So you should follow the sign. Okay. Was it a reasonable thing for me to get reminded? And the answer is sure. As a functioning intelligent human being I should get reminded of such things. Now the question is how would I get a machine to get reminded in that same situation and what was the real role of the reminding in my own personal life? The role of the reminding is to be able to help you make the appropriate generalization. This slide is a little bigger if we cut the lights you'll hopefully be able to read it. This is what I'll roughly call the explanation process. What I'm going to suggest is that the most important thing we do as understanders is to find anomalies in our own lives. That is to say we look around us and we see what's wrong, what's strange, what doesn't fit. We try and predict the actions of the people we associate with. We try and predict the actions of the institutions that we have to deal with. And we try and predict the actions of the physical world that surround us. And what we are are constantly like children in a way that isn't really childlike is we're constantly taking these expectations and trying to modify them. Trying to say, is that one right? Is that one right? Is this what's happening? What's going on there? We have little questions that we ask ourselves all the time and when we find that we don't know the answer, when things are anomalous, when things don't make sense, when someone did something, I can't figure out why they did it, right? What makes something anomalous? First, something is anomalous if it isn't something you would do, right? Somebody else does something you wouldn't do and that's anomalous. That's wrong, how come? The second thing is people, you know certain individuals and you know their patterns of behavior, things are anomalous if it isn't what the person would ordinarily do. Then we go up another level. We ask ourselves questions like, well, I see this behavior that this person is doing and I don't know, I wouldn't do it and I know he doesn't usually do it but maybe it has some result that would be good. What would be the result of this action? When we can't figure out a result that's good, we say, well, maybe it's part of some pattern of behavior that he's involved in ordinarily that I don't know about. And if we can't find one of those, then we say, well, maybe it's part of some greater plan that he has. And if we can't find one of those, we say, well, maybe it's part of some goal that he's trying to achieve and if we can't find one of those, maybe he believes something different about the world than I believe and if we can't find one of those, it's anomalous. We don't know why it happened and what we do is we try and go into the situation of trying to find the kind of explanation that will solve the problem for us. I'll throw it back to that in a moment. In the explanation process, therefore, and what I'm going to argue is that the explanation process is the key to artificial intelligence. You can't have an intelligent machine unless it's capable of explaining its own actions and explaining why it's doing what it's doing. In the explanation process, what we're looking for is generalizations that are both inclusive and instructive. We want generalizations that include the outer behavior. So consider my hotel and Connecticut road sign example. I have to ask myself the question, why is this hotel doing this? And I postulate an answer. Maybe they're just trying to protect their cells legally. So that's my explanation. Now, I want to find the right level to generalize over. Right level to generalize over is very important. This is true in expectation failure, whether it's successful failure or unsuccessful failure. Considering, imagine that you've bought a stock on the stock market and it goes up immediately. Now that's expectation failure of a pleasant sort. No one really expects it to go up immediately. And the first thing you want to do at that point is generalize over the right parameters. Well, what are the right parameters? You should always buy it on Tuesday. You should always buy computer stocks. You should always buy from a certain broker. You should always buy stocks valued under, they're under 20, the price earnings ratio under 10. And what is it? There are a lot of parameters available. And what we try and do as learners is to establish which parameters are most relevant. So what I had to do in the hotel example is I had to say the most relevant parameter here is protection of one's own legal liability through signs. Now, I want to know, protection of who's legal liability through signs? Should I put signs up all over my house? Don't come near my house? I'm protected? Where does this idea stop? That's what the number nine is. The verification of number eight, the breadth of application. You get reminded, you try to get yourself reminded, such that you can build in an example that will give you an approximation of the verification, the breadth of verification. So with the state of Connecticut does the same thing as Hotel does, then I could say something like, large institutions put up signs and I should distrust them. At this point, I should get reminded of something else. Stop signs, large institutions put those signs up and I have to say, are they to protect themselves or to protect me? Ultimately, one would assume that stop signs should not be ignored because what I'm gonna come up with in the end is a rule inside my own head which says, when you see a large institution putting up a sign that might have the possibility of doing legal protection for itself, ignore it. And in fact, that's what you have to do or else you wouldn't be able to go swimming in the ocean in Puerto Rico in this particular hotel. Oh, what kinds of explanations are there? This is my favorite question I ask it of students all the time. Why haven't you finished your PhD thesis? Here are some explanations. You get excuses, that's one kind of explanation, not a very interesting kind, because I had to celebrate my wife's birthday last night. Or you also get alternative beliefs. That is to say, I can explain to you that, oh, you thought I believed that? Fool, okay? Because I don't think it needs work. Or alternatively, I could say, oh, because you thought I wanted a PhD. You see, I didn't want that. Laws of physics. Frequently, the reason we don't understand something is because of a physical law that we didn't know. So this is a silly one in this particular case but you could ask yourself, why did the car skid, for example, in the snow? And you would want to learn the law of physics or some naive approximation of it that would allow you to prevent it next time. So failures, it's very important to explain them correctly. In this particular case, the explanation isn't that interesting. Institutionalized rules. Institutions have rules. In this particular case, well Yale doesn't require a PhD anymore, or a thesis anymore for a PhD. That might be a rule I don't know, in which case it's a legitimate explanation. Rules of thumb. I've discovered that not writing it gets your professor to the point where he's willing to sign anything. In fact, rules of thumb are often good explanations. Why did, when you went out with the boss's daughter and you lost your job? Should you explain that in some way? And the answer is often that there are rules of thumb, rules about life that it's important to learn. How are we gonna learn them? And this seems a little trivial. Why should I be worrying about this? But that's in fact what going on in an intelligent person. An intelligent person is acquiring his own personal set of rules of thumb for operating in the universe. And we cannot expect computers to be intelligent if they don't do that very same thing. So they have to be examining every aspect of their own predictive mechanisms and their own experiences to be able to grow their new experiences from their old experiences. New facts, of course, are explanations. In fact, it's finished, what do is an explanation.  Why does a doctor drive a big Mercedes for appearances? But it's legitimate to ask that question of him. We also explain things because of misunderstanding of people's plans. So you can say, oh, well, not writing. Every other week is the best way to stay top on top of your writing skills. In other words, this is part of an overall plan I have that you didn't know about. We can also say that this is part of a goal that I had that you didn't know I had. In this particular case, you see, I don't really want a PhD, you misunderstood me. Role themes, we expect people to act in a certain style in a certain way. And I'm expecting this person to act as a student. And he says, in fact, that he's not a student. Scripts, which is standardized patterns of behavior. And you can say, you don't perform all possible scripts, and this is one script you don't perform. Something called Delta Agency, which means that the reason you don't do something is because someone else is doing it for you. In this case, you say, well, I'm not writing, my wife is writing it for me. Lack of alternative plan, not working seemed the best thing to do at the time, or it's not a particularly, you can often say, I didn't do something because of some other plan. Or mystical laws I have in there. Lama Dhamma says that that which is not finished is finished first or something silly that you can make up. People have all kinds of strange beliefs. You can hit the lights again if you want. Okay, so what is it that I'm really trying to say here? What I'm trying to say is that it's fairly important when you hear about computer's understanding, when you hear about the nature of understanding and the nature of AI, to be able to make some distinctions. And I'd like to make a distinction of, instead of talking about computer's understanding, I'd like to talk about three kinds of understanding. I'd like to talk about an understanding spectrum, making sense on the one hand, something I'll call cognitive understanding as a middle point, and something I'll call complete empathy as an endpoint. Computers can at this moment make sense of what they read. So for example, it's possible to type in information from a UPI wire and get a summary of a newspaper story or translate that speech into another language or something of that variety. We can manipulate the symbols such that we are making sense of what we see. So computers are at that stage of making sense. But what I want to argue is that their understanding isn't an all or none affair. When you ask somebody, can computers understand today, the first thing you want to be able to answer is, well, yes, they can make sense of information. But between making sense of information and complete empathy is a large gap. Where complete empathy involves things like, well, can men and women really understand each other ever? Or can two men and wife understand each other? It's an open question. People ask that sort of thing. Well, if you're willing to ask that sort of thing with that particular word understand, how can you use computer understanding in the same breadth? What you have to begin to realize is that computers will never understand at that level of depth, simply because they won't be people. They won't have had the same experiences. What I'm suggesting is that you cannot put inside to a machine all the rules there are about all the knowledge there is in life. So long and hopeless task. What in fact has to happen is a machine has to grow its own experiences. It has to grow its own rules. It has to have expectations and they have to fail. It has to modify them. It has to replace its structure with rest from restaurant and replace it by fast food restaurant, cafeteria, fancy restaurants, eating on an airplane. And it has to get further and further and further diversified. It has to have more and more experiences so that each one of these experiences becomes a little micro world in which it lives. It has to be able to make the mistake that my father made by being in a 1918 frame of mind so that it couldn't see the 1984 frame of mind around him at that moment. It has to be able to do that because it has to be able to ask questions of detail within the domain of which it's an expert. But what I'm arguing is that expertise cannot be just put into the machine. It has to be grown by the machine. And because it's going to be grown by the machine, it will grow differently than people. It will not behave exactly the way a person will because it'll never be hungry. It'll never be tired. It'll never be wet. It won't have the kinds of experiences that we have. It won't be disappointed in its fellow man. It won't have the kinds of experiences that we have. And we have to stop anthropomorphizing machines. Machines will of course understand. They will understand in the level of making sense because they do right now. How about the level of what I've been calling cognitive understanding? Cognitive understanding is the ability to add more to what you've read. So for example, suppose I input a set of stories about airplane crashes and the circumstances and data surrounding them. And instead of paraphrasing, summarizing, and answering questions, it draws a new insight, a new conclusion about why the crash may have occurred. How could we build a program to be smart enough to do that? One way is to have it very involved within the data of that particular domain, to have lots of information about airplanes, to have been looking at each one of these things and not forgetting it, to be able to go through and in a profound way understand this information. That's a level I call cognitive understanding. And that level of cognitive understanding is not there yet, but it's achievable. It's certainly possible that machines will be able to self-reflect in the sense of being able to look at what they've done and come up with some new conclusion. Complete empathy, which is, I argue, the endpoint of the spectrum is a little hard. Let me give you an input-output pair that I made up for complete empathy. I was very upset by your actions last night. I thought you might have been. It was a lot like the way you treated me last week. But I meant you no harm.  He meant no harm either. I see what you mean. I thought you might. There's no friend like an old friend. Now, what I'm arguing here is for this kind of silly conversation that we've all had many times with people we're close to who really know us and understand why we do what we do, we would not expect to have that kind of conversation with the machine. Why? We're unlikely to have developed a personal relationship with the machine. And in fact, machines are unlikely to develop personal relationships that one would assume. Therefore, they're less likely to be people. And we run into problems of trying to make them seem like they are people. Therefore, what are the right tests? Now, I argue that the ultimate test of whether a computer understands ought not be the Turing test, which has been the test that's around in the literature for some time, where Turing is trying to propose that what we should do is understand the difference between a man and a woman behind a table and then replace one by a computer and have the computer be able to do just as well. I argue that, in fact, you can't even, I don't know if you could tell the difference between a man and a woman behind the table, but what I'm really arguing is that men and women have trouble understanding each other and differentiating a computer, which is completely different to either of them, seems far less likely. What I'd like to see is something I call the explanation game. I'd like to be able to ask a question of a machine and have it explain why it came to a conclusion that it came to and what it imagines will be the consequences of its decisions. Essentially, there are different kinds of explanations that you can come up with. For example, there are failure explanations. We are concerned with explaining failure, explaining why what you expected to happen didn't happen. That's an important kind of explanation. There is a coherency explanation, where what you're trying to do is explain how things fit together. Coherency explanations fit within the level of making sense. They explain how A is connected to B. Computers can do that today. Failure explanations are a little further apart. They are the kinds of explanations where what we have to do is understand why something didn't happen the way we did and construct a new invention of what might make sense. We do this kind of thing all the time. I was in a store in Vermont the other day and this store was a sign that said closed on Sunday. This coming Sunday, they were always open on Sundays and they said closed on Sunday. And I wondered why were they closed? It was the middle of their busy season. Why were they closed on Sunday? And the answer, I asked the question of the person who was working in the store and got in fact what was an answer that said oh yes, that's an explanation. I can recognize it when I saw it, although I couldn't construct it for myself because I didn't have enough of the facts. The explanation was that in fact, this store is part of a national chain and the national chain, all the stores were gonna be closed that Sunday for inventory. I then asked myself the question of well, why do they have to be closed for inventory? And what I'm really proposing is that what intelligence is all about is the ability to ask questions of oneself constantly and answer them constantly. And that we won't really expect to see machines that are actually literally intelligent. Remember the term artificial intelligent? It was supposed to refer to real intelligence. Well, what makes a person really intelligent? What makes a person really intelligent is that he's capable of having a new insight of looking at something and coming up with a new conclusion, a new way of looking at the world. The only way we can expect the machines will ever be able to do that is that they've been doing it all along in a very mundane way perhaps in just growing their knowledge about restaurants to the point where they discover something profound about restaurants. It's the sense of discovery which is the mission of artificial intelligence. And the thought that I'd like to leave you with in terms of how you should understand computers and how you should understand artificial intelligence is this. Computers are something which I think will impact on your lives tremendously as we get more and more adapted artificial intelligence. They are not something you should feel an obligation to understand any more than you feel an obligation to understand your automobile. They are something which you should be able to use if people in AI are helpful. What the intelligent intellectual type wants to understand about artificial intelligence is not what great gizmos are on the horizon, not what kinds of things we can find tomorrow in our backyard. What you wanna understand is that artificial intelligence is modern day epistemology. It's modern day philosophy in general. It's a new way, a new quest to find out more about ourselves. Thank you. We will reconvene at 3.30. Work on your questions. Work on your questions.


My fellowship here at the Stedman Clinic in Stedman-Phillipon Research Institute has been really an amazing experience. It's been one of the most impactful years, both for myself and my family, you know, personally, professionally. I feel like I've grown a lot as a clinician, as a surgeon. You know, we have the opportunity to work with a lot of great people here and I've had really, you know, great mentors both, you know, at work and also, you know, people I can lean on for advice in my personal life as well. So it's been a really great year. The surgeons, the PAs, the ATCs, everyone that works around here, they're really all patient-centered and I really do feel like we provide some of the best care in the world here at Stedman. That's something I want to bring with me and institute into my practice that no matter how busy I get, you have to always remind yourself, you know, why you're here, that you're here to take care of the patient and that's been one of my favorite things about my time here. This year has been the culmination of a long road of undergrad business school working for a few years in administration at a veterans hospital and on to residency and finally here at a fellowship. So as the year winds down, you know, here at Stedman, it's a great opportunity to reflect on my time here. It's too many people to thank individually, but obviously all the faculty, the staff here, my phenomenal family, my wife, Pauline, and my two young boys. You know, my wife's been incredibly supportive through many, many years now and I couldn't have completed all this training without her and we're looking forward to many more great years together as our journey continues.


Well, good morning, everybody, and thank you for that kind gesture, Mark. And welcome especially to those who traveled, in some cases, from far away to join us at this important meeting. I thought I would just say a few things to set the context of what really has brought us here, and I can't help but just sort of think back on recognizing that when the Genome Project ended in 2003, NHGRI put together a strategic plan that sort of articulated a vision for what we thought was going to happen in genomics in the coming, you know, five to ten years. And I think it has caught all of us by surprise at the pace at which technological developments and strategic insights were gained in a fashion that allowed genomics to be thought about in a fashion that truly was going to be applied to medical care. And so the kind of imagery represented here, I think, has come onto the scene faster than any of us anticipated, and that's a good thing. So it was very timely that several years ago when we kicked off another strategic, and the reason we kicked off another strategic planning process was because of this acceleration in developments that had occurred faster than we really could have predicted. And so through a process of multiple meetings, workshops, websites, conference calls, et cetera, we did our usual collecting lots of information and got tremendous amounts of input in various areas, but the main message of this, of course, was that the time was absolutely here to be far more ambitious in trying to chart a course to apply genomics to medical care and to really see the realization of genomic medicine. And so, of course, I think many of you, all of you hopefully are familiar with the strategic plan that came out in the 10th anniversary issue of nature commemorating the 10th, the earlier publication of the draft sequence, the human genome. And if you want to know more, I'm not going to talk about the strategic plan or the strategic planning process. We have a website that details much of it and catalogs all the details if you're interested. Of course, the organizing aspect of this entire strategic plan is now built around a framework that is portrayed in Figure 2 of that strategic plan, and the most important aspect of it is these five domains of research activity across the top, which sort of chart a course from the more basic side of activities of understanding how genomes are put together to understanding how they work, but most importantly, especially for this kind of workshop, then learning about disease, genetic variants that underlie disease, and then using that information to advance the science of medicine and eventually improve healthcare. And much of these activities towards the right are in fact all practically towards the right size figure are new for NHGRI and in many ways new to the field of genomics, and that's where the challenge becomes. Now the thing, of course, that is driving this, if you ask me what is the one thing that drives this strategic vision, this figure, much of what NHGRI is thinking about, I think the field of genomics is thinking about, of course, it is this due to this remarkable technological advances that have yielded rapid reductions in the cost of genome sequencing, and with that are coming studies that previously were unimaginable, but now are absolutely right here on our doorstep. In fact, in the case of NHGRI, we have a, as you know, a large-scale sequencing program, and that this year is the year we are in the process of renewing that, and we have, we will soon be describing or announcing sort of the results of that renewal process, but importantly, as we put out the program, or the RFA's describing this, it was, again, part of a planning process that led us, and our advisory process that led us to sort of seeing opportunities that we wanted to catalyze that would, again, start to bring genome sequencing closer and closer to the bedside, and as you will hear about in the coming days, indeed, we will be doing exactly that in the kind of sequencing program that we will now be leading over the next four or five years. What I will tell you for NHGRI's sequencing program, and I guarantee it is mirrored by other funding agencies, certainly the Wellcome Trust, the Sanger Institute, and others, is it is going to be ambitious and audacious, as always, but with these kinds of reductions in genome sequencing, we're going to be sequencing not thousands of individuals. In fact, I think we're going to be sequencing not tens of thousands of individuals, but in reality, we're going to be sequencing hundreds of thousands of individuals, and with that, and much of that, although not all, but much that is going to be part of clinical research studies at major medical centers and research hospitals around the world, as we try to chart a course that will eventually get us closer and closer to using that kind of information as part of medical care. And that's the challenge that lies before us. Finally, if you want to know a little bit more about some of our thinking coming out of NHGRI, about some of these more clinical applications extending beyond our strategic plan, Terry Minolio and I recently were asked to write a commentary that was published earlier this year in Cell, and it's just shown there that describes a little bit more how we see how the clinical impact coming out of these basic discoveries and genomics are going to come to pass. So with all that genome sequence data being generated increasingly around clinical research studies, what we heard, anticipating that, loud and clear, where there were a number of things that needed to be done, and what we heard during the strategic planning process were several things that we actually, I referred to as no-brainers, things we absolutely needed to move on as soon as the strategic plan was published and we wanted to act on. And we actually laid those out in one of our text boxes. We called it Imperatives for Genomic Medicine, the slang for it, I use, or no-brainers. And one of those no-brainers is right there. It's about the fourth or so it went down, where we talked about, and because we heard loud and clear the need for practical systems for clinical genomics informatics. And we needed new models for capturing and displaying these variants and their phenotypic consequences. These need to be developed and incorporated into practical systems that make information available to patients and their healthcare providers. And that very much reflected language we heard over and over again during the strategic planning process. And I will tell you we continue to hear over and over again at workshops, conference calls, various ways we're interacting with individuals, even since the strategic plan came out in February. Now, this is new territory for us. It's sort of out here thinking about these rightward domains, the more clinically oriented ones. But we are prepared now to start planting the seeds that will, especially to start the process that when the coming decade will yield the kinds of clinical genomic informatics systems that will be practical, will be robust, and will be exactly what the community is asking for. And so the reason we are here is to have you advise us and others, such as the Welcome Trust, about the creation of such clinical genomic information systems. We recognize these are highly interrelated with electronic health records, highly interrelated with the development of appropriate tools that healthcare professionals are going to be able to use routinely as part of clinical care. And all this gets very complicated very quick, but it's exactly the reason why we wanted to convene a group of very smart people, such as all of you, to help give us and others advice. So that is the purpose of this workshop. I want to really stress that this workshop is very much a collaboration between NHGRI on behalf of NIH, and by the way, it won't just be NHGRI and NIH, and we have multiple representatives from other institutes. Because obviously this is an issue that affects many different institutes, and I'm quite certain, and we already know that we will be heavily in working with other NIH institutes as we move forward in this area. But we also recognize this is like many things in genomics, would be best handled dealing with other types of collaboration, such as the Wellcome Trust. And we appreciate their co-sponsorship and support for this workshop as well. The goal is to consider processes, databases, and other resources needed to identify clinically relevant variants, to decide whether they are actionable and for what action they should be, and to provide information for clinical use. So all that sounds simple. You can reduce it to one slide. I'm quite sure we're going to be really having to put our heads together to think about the best blueprint to develop for that over the next couple of days. So that's what I wanted to say as an overview. What I want to just tell you a few more logistical things, I want to be the one to immediately thank a number of people on the planning committee who have put in a tremendous amount of work. In particular, I should start by saying thanking Rex Chisholm and Mark Williams who are co-chairs of this planning committee, and the membership of which is here, and I know it's also printed in your book. But I should also give a special thanks to Aaron Ramos, who's really been the NHGRI staff member who has just led the charge in keeping all this coordinated and worried about every last little detail. And she's made the promise to me that if this meeting goes well, she won't have her baby because that baby's due anytime. But if discussions start going in a bad way, she's going to start to pretend like she's going into labor to get them back on track. So in any case, but Aaron has just been a hero in this, and all of us should make sure to stop by at some point and thank her because these meetings are not easy to put together. And she's been just terrific at doing that. And the last logistical thing I wanted to say is that we are videotaping this. You'll see a camera back there. We are videotaping this, and it's not being videocast live, but we are planning. There's a lot of interest in this meeting, a lot of interest in this topic. And so we are going to videotape this, and then we will be posting this in the coming weeks and days on our website. So for people who can't be here or might be interested in hearing some of the talks in particular, you'll be able to point them to that. And you'll be able to find it on our website, and we can send around the information when it is available. So I will stop there, and again, thank you all for coming. I enjoy hearing about this for the next couple of days. And with that, I'm going to turn this over to Tim Hubbard of the Sanger, Wellcome Trust Sanger Institute, who's going to also add his own introductory remarks and comments, and again, I really want to stress, this meeting is very much a partnership between NIH and the Wellcome Trust. Tim. Thank you. OK, so I guess I want to welcome you on behalf of the Wellcome Trust and try and get out of...