Okay, today we'll talk about attention. You'll see quite a bit of the compelling demonstrations. So we are bombarded, our bodies are bombarded with huge amount of information. People have tried to estimate how much information it is in terms of bits, and it's very difficult to do because it depends on the exact representation used by the nervous system, and of course it depends on the input, you know, is it just a blue screen or is each point independently, you know, is the input very rich one, or is it just a blue screen? But if you do such estimates for, for example, the retina, you could argue that there's only all of a million wires, a million axons, they spike on the average between one and 10 spikes per second, and when people do estimates of independent information, they come up estimates like one bit per spike. So this is really, you know, very much rough back of the envelope type of calculation. So then you end up with things like, you know, a couple of millions, tens of millions of bits per second coming up the optic nerve. Now the vast majority of that, the vast bulk of that, we are not aware. We only perceive very selective aspects of the visual environment. That's what the lesson today is about. And you can ask, why do we do that? And the general answer is because there's no way that a system like ours could in real time, given the computational resources it has, deal with all the information in real time. It just couldn't do that. So what evolution came up with the system and seems to be quite old because there's evidence even that flies have already attentional selection that it came up with a system where it focuses on one or few relevant items or relevant objects in the world and that you only attend to those and that the rest you neglect, the rest they just don't happen. Or if they do happen, they might influence you but in ways that is not directly consciously accessible. And we'll show some of that. Also we'll talk about the link between attention and consciousness. Historically this goes back to the 19th century when attention and consciousness was real for the first investigator using what we would recognize today as modern scientific methods, that by definition almost most psychologists define conscious sort of attention and consciousness to be more or less anonymous. In other words, what you're conscious of you attend to. I think that's probably a mistake. I think that's probably there's a tight relationship very often what you attend to and what you're conscious of is the same but there can be conditions and I'll show you some of those when you can be conscious of something without attending. Because that raises the issue what exactly do you mean by attention? The first one we need to distinguish attention from more general use. So when I use the word attention, I mean selective attention. And today I mean almost exclusively visual selective attention. There are other forms of attention in the somewhat of sensory domain. Like I mean I don't right now attend to the pressure that my body weight exerts on my soles of my feet but I can perfectly well attend to them. Or you can attend, you can have a mate but I'd something on the back of your skin on the back you can perfectly well attend to that. Or same thing in the auditory domain. But we'll talk only about visual domain because once again it's by far the best explored. So I'd like to distinguish it from alertness of vigilance which sometimes also called attention. Those are more general arousal system. We mentioned them in lecture five. They pertain to one of these enabling systems that you need to be alert, you need to be vigilant. If you're not vigilant, if you're half asleep or drowsy then you will be unable to attend to stimuli but that's a much more global phenomena. So I mean when you're in the state that you are currently in and now you can attend or you can for some choose to attend to my face or my voice. So that means at the same time you're neglecting vast amounts of other things out there. Of course as we all know, attention can be supremely captivating. So this is one of my favorite authors. He's the guy who wrote, you know, the mountaineer who wrote into Sinea and he has his collection of essays on climbing where he makes this statement that's really very much apropos. By and by your attention becomes so intensely focused that you no longer notice the raw knuckles, the cramping thighs, the strain of maintaining nonstop concentration. This is on a big wall climb. A trance like state settles over your efforts. The climb becomes a clear dream. Hours slide by like minutes. The crude guilt and clutter of day to day existence. All of it is temporarily forgotten, crowded from your thoughts by an overpowering clarity of purpose and by the seriousness of the task at hand. Now this is difficult to reproduce in the lab. Although we try very hard. So in a laboratory, but this is all, you know, every time you do something really intense like climbing or anything else of, you know, where you really have to pay attention because the price of not paying attention could be severe, you can forget everything else or you fail to notice everything else around you. But you can reproduce some of those things in a lab as I'll show you. One of the most common definition of attention, it's difficult to define rigorously, but as a working definition comes from William James. He thought of the father of American psychology. He taught at Harvard for many years in, I don't know, 1880, 1890, 1900, and he was a brother of William Henry James, the novelist, the American European novelist. So in his, what is monumental book, The Principle of Psychology? And there he says, everyone knows what attention is. It is the taking possession by the mind in clear and vivid form of one out of what seems several simultaneously possible objects or trains of thought. This is now very important. It implies withdrawal from something in order to deal effectively with others. So almost by definition, when you attend to something, you have to neglect something else. Otherwise, the words really have no meaning. Now today I'll talk about purely at the psychological level. Next week I'll talk about what some of this means at the neurological level. There's been a lot of studies at the level of functional brain imaging in humans and at the level of single celled in monkeys. Where does the attentional bottleneck arise? Okay, so just to, I'll play a little game here in you. So in all of these illusions that I'll show you in the next, you should never say loud what it is. You should just raise your hand or say, I've noticed it. I've seen it because otherwise you're gonna ruin it for your buddies. So I'll do this little trick. So these are six cards. And what you should do, we should attend to one of them. It is important that you attend to one and particularly look at the details of one card. Pick any of them and just really notice the pattern, the layout, the details of the layout or the orientation of the various suites, et cetera. One of these cards. How many numbers they are. If it says seven, the card you pick, make sure they are actually seven. So you each have one card. Okay, now I'm gonna play a trick on you and I took out the card. Didn't I? Yes, no, maybe so. Well, what happened here? Yes? Yeah, yes. It works very well if you have a single person because instead of you can pretend that you're actually looking at the patterns of the eye movements and trying to see what they do. Yeah, so I mean, what I did, this is the original six and this is the five and none of them are identical. Now if you really do that, also maybe you did it too long. You know, if I only give you brief times, essentially. Well, I mean, who noticed this? Be honest, who noticed this that I changed all six? You all of them? Well, yeah, yes, of course. Yeah, but that's how you think about it after the fact. Okay, it didn't work very well. Okay, you're just too clever. Okay, so let me show you something else. Okay, so here what you have, this all relates to a phenomenon called change blindness. So this is one of these phenomena that reveals to you that you think you see everything is false, it's an illusion. So here, there are two images and one of them is doctored. So this was first done by three people, Oregon rinsing a clock at the Nissan Center in Cambridge, next to MIT. The two images are identical except one critical detail. Up, up. Now just raise your hand if you see it, if you see it, like the other ones, you're gonna ruin it for your friends. Okay, so I see there like four or five of you. Okay, from this case, it's here, the background. Now, once you've seen it, these are not subtle changes. You might remember, I certainly, as a kid, they were in newspapers, you had those, you had two pictures next to each other and there were like 10 changes that you had to notice. And some of these changes were tiny. These changes can be fairly sizeable. I mean, in terms of fractional of images, probably includes, I don't know, 5% of the image. Again, if you see it, just raise your hand, don't say what the change is. Now, the bad news is, do you wanna be told the bad news? Does anybody wanna hear the bad news about this? The bad news is that your action time is directly coupled linearly to your IQ. Inversely to your IQ. Now I'm joking, it's a joke, it's a joke, it's a joke. Don't feel bad, it's a joke.  But I mean, so most of you seen it?  Look at that horizon. So the changes of such a character, the once you see it, it's impossible not to see it. So those two images are ruined for you because if we do this in a year from now, you'll very quickly see it. I think I have one last one. The different ways you can achieve this, so this is called change blind as the inability, the fact that you're blind to changes in the fairly last changes in the image. The different ways to do it, so here what you do, you have an image blank, an image two, which is the doctored, and then blank again. So you need this blank, so there's always a blank between those two images, you need that blank because if you don't have that, you would see it immediately. Because what happens then, your system is highly sensitive to transient. In fact, this is one way how astronomers detect, for example, a satellite or moon or something, or an asteroid, they take two pictures and rapidly super, they take one on day one, the other one on day two, exactly the same part of the sky, and then they rapidly subtract them and that's how you can pick up, how you can pick up the satellite or the asteroid, or the meteorite. So in order to avoid that, to avoid this big transient, if I just would have image one and image two without having a delay between, you would immediately see that. So what they added, they added this blank frame. You can also do another way you can do it, you can joke it to eye movements. So in other words, I'm looking at this image and eye tracker, looks at my eye movements and every time I move my eyes, you change, for example, you make this change, or you make different changes. To another observer, where the changes are not joke to his or her eye movements, it's obviously because of this signal that's induced by changing abruptly from one image to the other image. But to me, because I move my eyes, and remember we talked about saccharic suppression and while I move my eyes, my vision is sort of severely reduced. So if you do it while I move my eyes, I will not be able to see the change. It's quite dramatic if you see it. A third way of doing it, you can do it using so-called mud splash techniques. Okay, because, and everybody knows him, so we found I did one, these are my dogs at home. This is nowhere else published, so nobody else has seen this yet. Again, this is a very big change. This is probably the biggest change of them all. Okay, let's see, it's a carpet there. Now, of course, this is the basis for many magic shows.   What you have, almost always, you have sort of the magician and then he has this beautiful bikini cloud assistant. She distracts you, and in the meantime, he just makes a very quick movement. These people are all, most magicians in my experience, the ones I know are all magicians since very early childhood.   Now, Dan Simmons, he's now at Harvard, he did this at Cornell. Wait, okay, so I should, let me. Okay, what you're gonna see is two things. This is the, here the argument was okay, this is all fine, this is all in the lab, somewhat artificial. In real life, these things don't happen because in real life, I cannot immediately change, for example, my yellow shirt for a green shirt. You know, the claim would be unless you attend to my shirt, you wouldn't see that, but of course, it's gonna take me a lot of time to change my shirt.  Did the following experiment. So he had a student, this is a student, approached a subject, she doesn't know if she's a subject. So he approached her and has sort of a map and says, I need to go to James Hall, please tell me where James Hall is. And then the lady, the subject, explains to him how to get there and then you see what happens. I notice she's gonna look at him there, she looks at him, directly in the eyes. Again, I mean, if you want, just compare the people, how they look before and afterwards, sort of the blue and the yellow helmet, the same yellow helmet, but different sweater. Here's another case, again, he's the subject, he doesn't know about it. I mean, look at him, he doesn't look too dissimilar from the other guy, he was about to replace him. But again, he looks him straight at, he looks straight at him, numerous times. Now, half of the time, subjects notice, at least half of the time, subject claimed they noticed something when they were debriefed afterwards. And half the time, there were cases like this where obviously the person had no idea. Now, this is not, I mean, one shouldn't, one should be a little bit cautious here, that's not to say that if this guy is replaced by, let's say, somebody looks totally different, let's say if a woman is replaced by a man, you would probably notice, or if a white guy is replaced by a black guy, you would probably notice. But the idea is that you don't, this guy doesn't individuate him to a terrible high level of resolution. He looks like a student, sort of, the idea in his brain just says a student, white student, 30 years old, approaches me, and then, you know, and your brain deals with that, and then when another roughly the same age, roughly similar looking guy replaces him, you don't notice a difference. Of course, you perfectly can if you choose to attend to him. Yeah, now the, this, this, oh, let's see what's happening here. Let's see what happens here. Let's see what is, okay. Did you notice something? Yeah, now I shouldn't have put the title there, I just realized.  And so usually there's so many on it, on there's a job of some person on any big movie to make sure that these discontinuity errors don't occur. So focusing on what's typical, you do indoor filming one day, and the outdoor filming a different day, and then of course in the real movie, you have the actor walking in from outside into the inside of the room, or vice versa. You have to make sure, and there has to be a person who makes sure of that, that the person's dressed exactly the same way in this different way on these different days. It's not on one day, for example, I think it happens in Blade Runner once, that he comes in from the rain and he's wet, but then once he steps inside the room, that shot is taken on a different day, his hair is suddenly miraculously dry. Now you don't tend to notice any of these things unless you obsess and you watch the same movie over and over again, or unless your tension is drawn to it. But we're incredible, we're just, well, incredible good, we're incredible bad at seeing a lot of these changes, particularly if there's a compelling story, if there's something that draws our attention. And the claim is this happens very often in life, which is this, all of this research really makes you extremely skeptical when it comes to witnesses, right? I mean, when you then sort of listen or read court transcripts, when there was usually a very emotional, traumatic event, a rape or another violent crime, it was dark, you only saw the perpetrator for very brief time, you've got to be extremely skeptical because of sort of the attention lapses that people have, plus our amazing ability to do post-fuctum reconstruction. After the fact to reconstruct, this is probably what happens. And then I see a guy who I think might fit the bill and then sort of I'm 100% convinced that that's actually the guy. And I'm not overly lying, but it's sort of top-down influences and backfilling and filling in these stories after the fact. Now, you're really bad when it comes to noticing small changes. You're really bad. So what happens here, as we talk, this changes. So again, it's another of these illusions by Dan Simmons. Things are changing. Did anybody notice a change here? It's a very big change. Okay, let's try it again. Well, let's go to this one. Wait, there's no change here. Ha! Sorry, why is there no change? I'm not sure the movie's running. The previous one, there was a change. Okay, so this one, yeah, this one is working. So here there is a change. It's very dramatic. You mean you looked up and then up again? Right? Maybe we can go back. I notice this here. Well, let's just go outside here. Did you see it? It's the old sort there. So why are these? Okay, let's try this one. Yeah? There, now it's moving. I don't know, this is the Patrick's raising the question whether is this attention or is this memory in this case? In the previous cases, I mean, the change is so rapid that it's very unlikely to be memory. Here, it's a good point. Of course, they're closely linked. Did anybody saw the change? We can do it once again as a quick time. These are subtle things. I mean, these are very nasty to see. You see it here? These some are all in all I find less compelling because they are so small and so slow, these change. I find the other ones I find much, much more compelling, the ones that occur in a fraction of a second. What do we learn from these change blindness experiments? What we learn is, okay, so A, in the general statement is that we think we see everything at once, but that's an illusion. And of course, we have to explain the illusion. We have to explain why we have this very compelling feeling. Open eyes, of course, I see everything.  It's very vivid. I have access to all that information out there. Now, one possible explanation is, well, that's exactly the point. The information is out there, so when you actually need the information, when my brain needs to have to make sure what's actually out there, I'm just gonna look. Why should I sort of make use of very expensive memory, expensive in terms of neural hardware, that sort of paints a picture of everything in my head when I don't need it because I can always look outside. In the real world, I know things don't disappear within a fraction of a second. So that's sort of one argument that says that I use the world as outside memory and I don't need to have sort of replicate that memory inside my head. It's just waste. But it still doesn't explain why at the same time we still have this feeling that we see everything else. And then you can, of course, you can do experiments sort of what determines how fast you see a change or whether you see a change. And that depends on probably where your eye is looking. If your eyes happen to look exactly at the look, if you attend, it's more important, attending. If you happen to attend to the location where the thing is changing, then you can very rapidly see it. Or if it's very salient. I mean, if I had, if there's a single image, let's say there's a single person on the screen, everything else is just a homogeneous background and my head is changing or my shirt is changing, that you're gonna see instantly. But if many different things compete for your attention, if there are many different objects, different people, sort of different colors, et cetera, they're all very vivid, like in some of those scenes I showed you earlier, then they all compete for attention. And then, you know, you'll have to go for one object to the next. So a big lesson and a big sort of theme underlying, particularly once we talk about the neuronal correlates of attention, is the theme of competition. But ultimately what we have here with competition is inside your head, crudely spoken, you have different coalition of neurons, different group of neurons that code for the different things outside there. And they're not independent. I look at you, I cannot, there's no way, well, it just doesn't happen. And then we can ask why it doesn't happen. But it's not that for each of you, there's sort of a coalition in my head that codes you. Why doesn't this happen? Well, because the neurons, particularly in the higher part of the visual field, they all overlap. And so it's impossible within the same brain area to have different independent coalition of neurons that represent sort of all of you. So instead what I do, most of them get squashed at any given point in time. And I only focus on one or two at a given point in time. I only focus on one or two, maybe three objects, a very small number of objects. The neuronal representation for the other things gets squashed for some fraction of the time. And then I shift my attention to somewhere else. And then the previous things that I attended, their neuronal representation is squashed. And I facilitate the coalition of the assemblies corresponding to other objects. And I can do this using two strategy which psychology is called overt shift of attention and covert shift of attention. So overt shift of attention are eye movements. We do them roughly 100,000 times a day as often as your heart beats. And because we have this very highly inhomogeneous receptor density with lots of receptors at the fovea and very little in the surround, that's sort of one way how I can access information, high fidelity information by moving my eyes to where I wanna see, where I wanna look. But then independently, and you should try that, you should try it once.  And then you can perfectly well sort of attend to something else, right? Just, I'm proud. But you can, you know, you can move some, you know, you can move the other thing around. You can perfectly attend to this other finger even though you keep on looking here.  That's exactly what you're doing. You're looking here with your fovea, but you can divorce the covert shift of attention. You can sort of attend to over here. Although it looks like I'm looking over there. Now, that's used quite a bit in the lab, but that's actually a very unnatural condition. Usually what happens in life, of course, when something's interesting there, I look over there. Unless there are, you know, social or other, unless there are reasons or constraints, why I may not be, you know, why it's not a good idea to look over there for all sorts of reasons. And then sort of, I can use my covert shift of attention to attend over there. But usually what I find interesting is what I tend to look for. But in a lab, they can be dissociated and they often are dissociated. Now, even more dramatic form, well, another form, which is not exactly the same, is called inattentional blindness. So before I talked about changed blindness, the claim with changed blindness is also something that occurs in normal life, as I mentioned. And might, in fact, there was an article in the Yorker a year or two ago, which made the argument, based on these psychological experiments, that a number of accidents that occur in real life where you have, you know, a person that's no alcohol, no foul play, no drugs or anything else, there's no, you know, the hardware, the car, the physical plant all work perfectly, but the person just failed to see another car that was in plain view, and ran into this other car. And there are a number of these otherwise unexplainable accidents, you know, there's no bad weather, nothing like that. And the argument is, there could be these inattentional blindness or changed blindness, where for whatever reason, the person wasn't attending, and therefore just failed to see it. There are now a number of studies that people do, in fact, using, I'll come back to it, using dual task condition, where you do two things at once. Of course, particularly where you do a visual task and an auditory task, as in a cell phone. And if you read those, then you do like me, I don't, I avoid using the cell phone now on the road, because what you can see is there's a significant dramatic interference, your error rate goes up by two or three times when you both drive, when you try to attend visually, and at the same time you try to have a telephone conversation. And the claim is it's not so much the mechanics of it, it's not so much because that people can usually learn fairly quickly, so in other words, you can dial, you can drive, and you can dial, unless it's very complicated, but something like that, dialing, you can do. But it's not the mechanics, it's not the holding, unless of course the phone falls down, you have to sort of grope for it down there, that's bad. But leaving those things aside, the idea that it's really the cognitive, the engagement that you're engaged with another person, particularly if the conversation is sort of, engages your attention, because for whatever reason, that really withdraws attention from the road. And then if people, at least in a lab condition, they're much worse at detecting, for example, stop signs or detecting cars that suddenly appear. It's a significant effect, it's like a factor two or three, and certainly should give all of us pause. And of course we know from Accident Report that a significant number of accidents involved these days involve phone calls made just prior to the accident. And of course we all know, we constantly see people on the phone in front of us who just seem to be asleep. You know, misread, green lights and things like that. So that's something for our own life that we should be concerned about, and probably not do. Or if it's really engaging, you should definitely pull off the road.  Okay, now this is inattentional blindness. In all the previous experiments today, in fact almost everything in this lecture, and in fact almost all psychology experiments are the following ilk. You get a subject and you show the subject, let's see in visual psychophysics, you show the subject to stimuli, or different stimuli, and then you do an experiment hundreds of times. And typically the first, depending exactly on which experiment you do, the first trial is typically you throw away because you say, well, that's learning. So for example, when you even do simple sensory tasks, like it's a very fine line, is it tilted a little bit to the left, a little bit to the right, early on most of those trials you throw away because people rapidly improve for all sorts of reasons. They understand what's being demanded of them, they understand the mechanics of it, or also their visual thresholds improve, something in their brain actually improves. Now, so by the time you take the real data from these people, they are highly knowledgeable about what's going to happen because they've been trained dozens or hundreds or sometimes thousands of times. And so they have an expectation, they know what to expect. And this expectation is probably very important in forming representations in the brain. And if you don't have those representations, you might totally miss certain things. In other words, if things happen totally unexpectedly, in particular, if they happen very briefly, if there's a signal that's present only relatively briefly, you might totally miss it. So let me see whether this works. So I'll try to give you a cartoon version of this. So this was done at, for these people, McEnrog at Berkeley and somewhere on the East Coast. And they did the following. They are subjects. They only did eight trials. And they didn't take sort of eight trials after the subject learn. They just got a person in, did eight trials, and then thank you very much. Now, so obviously you can only do this at a school like Berkeley where you have lots of students taking your class for credit. So the task they were told to do is there's a cross that's going to appear. And you have to tell me, is the vertical arm longer than the horizontal or the horizontal arm longer than the vertical? Okay, and then it's immediately masked. So I'll show you this very briefly and then I'll immediately mask it. Now, of course, I can't do this very well because it's a PowerPoint and then I don't control the timing of the LCD but this is just sort of to get the idea across. What did you see? Did you, what, who went why? Which arm was longer? What? That's horizontal. I guess in Australia it's different because it's. Anyhow, so I mean, did anybody see anything? Hello? Yes? Well, you should tell me what you see. Okay, let's do that again. So you should fix it always at the cross. Well, I mean, it's supposed to be faster. It actually works reasonable well. What did you see? What? Horizontal. Did you see anything else? Okay, so let's do this slowly. So first, that was a stimulus. I actually don't know which one is longer but it doesn't really matter. That's just to engage your attention. There was nothing else. So they did this three times. On the fourth trial, they did this, something like this. I mean, there were different versions of it but essentially what you do, you tell people, okay, there's gonna be this cross and I'll give you three trials. So on the fourth trial, totally unexpected. I'm not telling you anything. I'm not warning you, there might be something else. I flash up something else. And I don't do anything, I just record your answer. And then for the fifth time, again, I show you just the cross. On the sixth trial, I show you something else again. Now that you might be warned, oh, sometimes he's gonna flash something else. And on the last trial, on the eighth trial, I tell you ahead of time, there's something else. I want you to still do the task. Is this one or this arm longer? But I want you to tell me what else you saw. And then people, so A, at the end, people could all, when they're being warned, even though this was very brief presentation, people could always describe what they saw. And so typically what they saw, what they projected in there was sort of a hexagonal or square or triangle. Sometimes they were one shape, sometimes two or three and sometimes they're different orientations. And 20% of people had what's called inattentional blindness. They didn't see this at all. Okay, they didn't see it at all. Then they did a variant of this experiment. So in the original experiment, they did the following. You fixate it here. And then very briefly, this cross would come up. And on the fourth trial, here I would, I would, for example, do this and I ask you, well, I wouldn't ask you ahead of time. But at the end of the experiment, I would ask you, did you see anything? And so 20% of people didn't see anything at all. They were just totally blind to this. And the other 80% did see, like some of you saw, yeah, there was something there. People usually could never tell the shape, interestingly. Now they did a variant of this experiment, which is much more powerful, but unnatural, which I probably explained why it's even more powerful. So here you fixate, you fixate here. Then I ask, okay, I'm gonna flash this cross. Again, you have to say, is this arm longer? I mean, it's a vertical longer than a horizontal, or the other way around. Away from fixation. So you look here, and I flash the cross over here, and do this three times on the first trial, right at where you're looking. I put, for example, you know, three triangles here. And now the number of people who failed to see is went up to 80%. So in other words, 80% of the people, four out of five, failed to see a stimulus that was right where they're looking. That was by itself highly visible, that was not crowded, was isolated, single, highly visible stimulus. And like I said, if I warn you, everybody sees it. This is not presented so brief that you don't see it anymore. So this is really quite remarkable, and pertains to this phenomenal, inattentional blindness, and I think the real lesson here is if you don't expect something, then you might totally miss it because you have not built, either you haven't built up the representations at all, or because you don't expect these representations right now, sort of, you know, they're just not used, they're just not activated. And so surely this argues for relatively fast learning process that in the first couple of trials, your brain, your visual brain, rapidly builds up some expectation of what to expect. And then the same signal gives, the same input signal gives rise to much more powerful, much stronger neuronal signal that you can then pick up. I always notice this when we do first psychophysical experiments and we flash things very briefly, there's folks on a stimulus known as hypoxudy, which is two bars, they're very close together and the lower bar can either be to the left or to the right of the upper bar, this is called hypoxudy for reasons we'll go into next year in the vision class. And when you do things like this, for the first time, you hardly see anything at all. And, you know, it's only when I sort of, you know, when, this is my experience and also when we show two subjects, you have to show the same stimulus, you know, a couple of times or 10 times or 20 times, then people begin to see it. And so there's a, I think there's a very important lesson there. Now, all of this is very difficult to reproduce in a lab because like I said, you have to take people who are totally naive and who haven't had this particular stimulus before.  And of course, it's even more tricky to do in animals because the only way you can get animals to do any of these experiments like monkeys, you can get them to do a lot of these experiments, but first you have to train them for several months or half a year in order for them to understand because you can't, you know, you can't tell them what to do, of course. So you have to tell them indirectly by shaping and by teaching them and that takes a lot of time. So by the time a monkey is trained, you know, literally he has seen the stimulus hundreds of hours, thousands of times.  And the lesson is if you've never seen it, you might literally not see it. Although it's, okay, so let's see, this is another change blindness. Let me see whether this works. Okay, here what you should see, there are two teams of people and they have a ball, two balls, and you should count how often they pass a ball. Okay, hello. Just count how often they pass a ball to each other. Just keep a silent tally of that. Now, can you lift your hand? Who noticed something strange? I mean, did you all see the man in the gorilla suit? Okay, I also saw it the first time. I find, but it's claimed. So by people who've, this is the first time I show this in class, I have to show this in a general public audience. It's claimed that up to two-thirds of people who've never seen this movie before failed to see the gorilla. I find this difficult to believe, but let's see. Yes, count. Yeah, I know. What? Ah, excellent. You get extra credit. Wait, who didn't see it, just the two of you? Well, no, wait, how many people did not see it? Okay, so there are a number of you. Okay, good. So you also didn't see it? Did you see the gorilla? Okay. Okay, good. So I think there were... I mean, that's the point. Good. Okay, but that's not the answer. No.  You can find the answer by doing either the white or the black. Just to clarify.  Oh, okay. I said the white. Me, some of these girls, I don't know. Some of these girls, if you do a white, I think it's a much better question. Maybe there are other things about that that are not related to the rest of the movie. It's like they're not very disciplined enough to do what you're asking them to. No, but Leila said half of them still miss this. So I mean, if you think of it, this is really amazing, right? I mean, this is really scary. I mean, this is not rapid. This is not sort of one of these unintentional blindness where I flashed for a millisecond. This probably takes, I don't know, like five or six or seven seconds in full view. This guy walks across it. I mean, I just find it so amazing.  Yeah, but the guy with the gorillas, he crosses it, right?  These are all excuses. I'm sorry. No, no, I mean, that's a point. That's a point. You would somehow attempt only two, but actually there are two balls, right? So it's a little bit more tricky. Well, you could, but... Well, okay. This is another, I can't show it to you. I don't have the movie. I just have the picture. So this was done, this is a little bit scary, and this pertains to the point I just made about the accident, that this was done in a simulator at NASA Ames up in San Francisco, and they had, in simulators they had, this was done with four or six, not a very large number, professional pilots, while they were landing there seven for seven. They superimposed while they were approaching. You know, here you can clearly see they're almost there. They superimposed this silhouette of a second plane. Now, this should cause them immediately to abort, right? Because they're about to land on top of this plane with disastrous consequences if this would be in the real world. Half of the pilots totally failed to, this was just briefly flashed. This wasn't kept long, it was just briefly flashed. Half of the pilots didn't at all notice it. In fact, there was recently an accident in, well, a year ago in Taipei, right? Where this, what was it? It was a seven for seven crash. Burning crash, everybody dead. Were they landed or they started on a runway that was closed? Was highly visible, was clearly marked, and nobody knows why. I haven't seen the report, I just read it in the old times. What? But the point, what I read, it was highly marked and was very visible. So this might happen, this might, I mean, now that I know these things, I do, I pay a little bit more attention also to these things at the metal level, and I do notice that once or twice when I'm driving I just totally fail to see something that was clearly there. So our visual systems are good on average. I mean, statistically they're pretty good, otherwise we'd still be stuck in the slime, but they're not perfect. So what we can learn from inintentional blindness is a critical role of expectation. If we don't have expectation, we don't expect that we may not see it even though it's in front of our eyes. So in the literature on consciousness, the Grand Illusion is now known. The claim is that sort of everything's illusionary, in fact consciousness itself is illusionary, just as these other phenomenas are, and this is known as the Grand Illusion, it's essentially based on experiments such as these. Your inability to see many things, like do you see an obvious mistake here? Only you did, only a single person, right? The illusion of seeing.  Recently I was interviewed for some archives and I was asked to review the transcript, the video transcript of it, it was Francis Crick talking about our stuff, about consciousness, the work I'm extremely familiar with, which I think explains this following phenomena, that when I saw him on tape talking about other stuff, and then I got a transcript from the secretary, and I read it, and I thought, gee, this is really shorty transcript, I mean, if this is the transcript sort of they make in courtrooms, it's really bad, because if you read the transcripts, there were errors, and there was pauses, and there were incomplete sentences, there were repetitions, it just really read bad, read really bad, while if I listened to him, it made perfect sense, and it was a beautiful pose. And then what I did, I actually took the transcript, and word for word, I listened to it, and the transcript turned out to be perfect, and what happened was, again, it was neat, it was filling in, it was blindness, but now in the auditorium, I just didn't hear the drop senses, I didn't hear the repetitions, I filled in things that he left out, because I'm so familiar with him as a person, as well as with the material. So that's sort of same phenomena in the auditory domain. Okay, so the point here is, I mean, we have severe capacity limitations, at any given point in time, we only see a few things. I mentioned this idea that the world's external memory, so unalying vision, and this is true for any perception, including audition, including olfaction, including pain, pain is no different. All of these are active processes, that I don't, the naive empiricist view is that there's a world out there, it gets mapped onto my sensory surfaces, my auditory, my visual, my somatosensory, my tactile receptors, or my pain receptors, and then I just sort of, there's an internal world that I passively reconstruct out of the external world, sort of this naive realistic view. Now, we know that that's wrong, so we partly from here, what we do, what we do consciously, unconsciously for the most part, we select only a very small subset of things that we choose to attend to and that we choose to perceive, and we choose that either volunteer now or talk more about it, we choose it partly unconsciously with sort of automatic gates that we have very little influence over, and then of course we can also choose to attend by our own, we have a second mechanism, selectional mechanism called willful or focal or top-down attention that I can choose to attend almost anything. That's a very powerful mechanism, but it's slower and takes time and effort. And the bottom line is that perception is an active process. You can think of perception, the construction of a description, that some of it's almost a definition, that perception is a construction of a description of the outside world in my head. And this also pertains to pain, pain is no different, people think, oh, it's pain is fundamentally different, but it's the same thing. Therefore, you can also have all these dissociation and illusions of pain that the same physical stimulus under one condition might be highly painful and bad and sort of you have a stronger version to it while the same physical stimulus on a different condition. For example, when you're playing a game or when you're climbing or something, you don't even notice, it's the same dissociation. And so there's a big literature that I'm not really aware of. I mean, I know it's out there, I've read one review where they, so one standard way of testing ADHD, attention deficit hyperkinetic disorder is to use attentional experiments but usually vigilance where they put you for 20 minutes or half an hour in this context and the various signals that come and you have to detect them. And ADHD kids usually do do worse. We don't know the underlying condition. All people know that partly they have difficulty, I mean, that's almost by definition, they have difficulty attending for long times to stimulate. I don't know really anything more about it. We don't really know that, I do know. We don't really know the underlying neuronal conditions. We just know it's one of the symptoms of ADHD. No, so I guess Patrick asked that question already. No, we don't really, I mean, I think that becomes more relevant for these slow changes, for very fast things, you know, when there's only one second interval between the first image and the second, it's unlikely sort of to be a memory, a conventional memory, a conventional working memory deficit because, you know, there's just very little time for you to forget things. Okay, so I think that's the end of these nice sexy illusions. So now we'll have some more sort of graphic material. Less graphic material. So the standard way that attention is studied in the lab these days is started off using very simple stimuli, probably because when this sort of was at its height, people didn't have computer memories and they didn't have the hardware nor the software to display realistic images, so they used these very impoverished scenes. The typical experiment is you fix it here, then at the location where you fix it, you get a cue that tells you that's sort of 80% correct, so 80% of the time over here will be a little faint target and 20% of the time the target will be over here and your mission in this experiment is just to push a button as quickly as you can when you see this faint light. Okay, so and like I said, what they do, the experiment list fools you, some fraction of the time, the stimulus actually doesn't appear over here but appears down here and so now you can compare the time, you know, how good are you when the stimulus was there versus the stimulus was down there? And so you can see if you plot the reaction time for these very simple stimuli, there's a difference, albeit it's a small one but for reaction times are like 300, 400, 500 milliseconds, there's a small difference that if you don't attend, you're much slower than if you do attend. So that would suggest one of the advantages of attending to something is that you're faster, you're faster at seeing it. Now of course, all these conditions is different from these unintentional blinds. Here you always know what to expect, you're doing hundreds of these trials. If you know perfectly well, it's only gonna be faint light and the fade line can only appear here or appear here which is a mention is very different from having no idea what goes on. So now these conditions, this is called the Pozner paradigm after Mike Pozner who pioneered this. This is a standard way how you can also dissociate eye movements from intentional movements because your eyes all the time are fixated here. You always fix it here and then you do this task. So this is always where you dissociate your eye movement from your intentional movements. Or they use different, no, no, so this is fixed by the experimentalist. So they use different, if it's 50% then there's no advantage for you, right? If it appears 50% here, 50% time there, then there's no advantage why you should, there's no reason why you should attend. If it's 100% then you wanna have a certain number of trials, not too many because then the subject is frustrated and doesn't attend but you wanna have some trials where he attends there and if you don't queue it, the stimulus is actually there. I can't see if you don't queue at all. Is it closer to the valid queue or the invalid queue? It's somewhere roughly in the middle. They also did that. If you don't queue at all, if you just attend here, then it's somewhere in the middle. Now these attention delays are probably much bigger if you use more complicated stimuli. These are highly impoverished stimuli, right? There's nothing there except a very faint light. Any other questions? In this experiment, so what is the subject? Oh, so in these experiments you're told exactly what goes on and you do probably at first 10, 20, 30, 40 trials where you just do training and then you throw all of that away. And then once the subject has stable performance then you start counting their performance. So as I mentioned here, it's all, you know perfectly well what's going on. There's nothing unexpected here in that sense. And that's what almost all the experiments in psychology there, the stimulus is always expected. It's much more difficult for statistical reasons, right? Because if it's unexpected, I can only really do it one time. And so that's extremely difficult to collect any significant statistical meaningful responses. Now a very popular paradigm is due to Anne Treisman who is now at Princeton. Her husband recently got the Nobel Prize then, Kahneman in economics, economics. So Anne Treisman pioneered this so-called visual search paradigm. So there's several ways you can study attention. One is as in this Posner trial that's very impoverished. The other one, which is a bit more closer to real life but still pretty impoverished, used to be amazingly popular, it's now somewhat less so, is using these search paradigms. Well, you're given arrays like for this array or this array or this array. And this flashed on a screen and then either with or without eye movements are different ways you can do this experiment.  For example, if you detect an odd man out, boom. Now here immediately you'll see there's something different there. Boom, here immediately you'll see there's an odd man out there, right? This guy is clearly different than all the other guys. He's red, everybody else is green. This guy is sort of horizontal, all the other guys are vertical. Now here the odd man out is much more difficult to detect, right? You know what it is? Green horizontal. Yeah, green horizontal. But here you have to do what's called a conjunctive search. So here this is called pop out. In pop out the idea is that something pops out. Well, okay, so A from a phenomenological point of view, let me show you this here. So this is how usually one popular way of doing the experiment. You vary the number of distractors for one particular experiment. So for example, I show you this experiment. I always ask you, is there is another red bar present? And half the time there's a red bar present, half the time there's no red bar present. And what are vary the number of distractors? So here they're now four, eight, I don't know, like 12 distractors and I vary that number between one and, I don't know, whatever, 15 distractors here. Okay, so here I put the number of distractors and here for example I put your reaction time. It's one way. You're supposed to give a speeded response, push the button, present or not present, as soon as you can. And then you see sort of, for example, here I put two experiments here, either here searching for a diagonal among these other diagonals or for searching a cause among else. Then this guy here, in fact, this is the performance for six subjects. This was done by Rufian van Rulen here in our lab. Okay, so on average this is the performance, average of all trials. So here this is called pop out. This, the slope here is almost zero. It's not quite zero, it's five milliseconds per item, but it's very, very flat. So this is called pop out. The interpretation is, here your target is so different from all the distractor, it immediately pops out, relatively independent of the number of distractor present. Now this trial, this case here, results in this search slope. These are called search slopes because you have to search for the target. Here the target is a T among L. And this roughly, the year now you have a slope that's nine times bigger, 10 times bigger, and all of magnitude different, bigger. Here it takes you 42 milliseconds for each new item that's added. It takes you 42 milliseconds additional time to correctly respond whether the target is present or not present. This is something else. This also gives rise to these slopes. Here you have to look, you know, the targets are either black, white, or white, black. Yeah, so these sort of targets give rise to serial search and these sort of targets among these distractor give rise to parallel search. Furthermore, that's not shown here. If you split the data, if you separately plot the data for all the data when the target was present and compared to all the trials when no target was present, you see the slopes differ almost exactly by fact of two. So this is average, but if you plot it, I think it was something like 30 milliseconds. So if you plot all the trials here, this task, where the target was actually present, and you see how long do people take that, it gives rise to a slope of roughly 30 milliseconds. Well, if you plot all the trials when no target was present, it takes you, the slope is 60 milliseconds per item. Can anybody tell me why that may be? Why does it take you twice as long in one case and in the other? Yeah, well, essentially you have to check, if you don't know anything, you have to check every item, right? So if these are randomly, if the target is randomly distributed, then on average, you have to check half the item. And if it's not present, of course, the only way to say is to scan every one of them, which is why it's twice as long. Okay, so this sort of, okay, so here it's idealized, pop out this is serial search. This was done like in the 70s and early 80s. And this, together with the rise of computers, particular of parallel machines at the time, gave rise, I mean, to mass parallel machines like the connection machine, gave rise to the idea to very simple and very powerful theory, we ask you to describe it briefly in homework, that's now is believed to be wrong, called feature integration theory. This is by Anne Treisman and her student. So the argument was, well, what happens, this is evidence for serial search light. There's been this metaphor in psychology, goes back to at least William James and for all we know might originate with Leonardo Da Vinci who also talked about these things, that the metaphor here is a search light, right, for attention in the visual domain, that essentially you have the search light in your head and you can illuminate things out in the world or at least the stimuli that you perceive, you can illuminate them and you attend to what's sort of within the spotlight of attention. Now, like any metaphor, it's powerful, it can also be misleading because a search light, of course, for example, if I move a search light from here to here, you know, it's gonna illuminate everything in its path. Also it implies a circular shape and yeah, and also it implies a constant diameter, all those things are wrong. So we know that based on your expectation on what stimulus is, you can expect that you can attend to something over a large distance, a large space, or you can attend it over a small space. Like if I train you on a task, if I do a task and the target might appear anywhere here or the target appears anywhere, you know from experience it only appears somewhere here, you can change your focus of attention and you can show in some experiments that it takes time to change the focus of attention. Again, you can think of it like a search light and you can of course change it by changing the optics of it, but that takes time. Like what it appears to be that it's not true like a search light that when you know, I tend to hear that everything, all the spots in between are lit up. People have done some experiments there. It's probably more like a stage light. At a stage light what happens, I'm here and then maybe there's another actor over there in the dark and then the stage light is switched off and it's turned on over there. So that's sort of more the metaphor. But the basic claim of feature integration theory is that what goes on is a following. That you have this attentional search light and that these cases, it's not really needed or the attentional search light immediately gets drawn to this stimulus because this stimulus differs in a single dimension. Here all you have to look is orientation and then you can define the target by orientation or by the fact here it's a plus among else. While over here the idea is what you need, you need to have the search light that very laboriously goes from one object to the other. It has to say, okay, is this one white black? Is this one white black? Is this one, yes, this is. So then I can terminate it. And that's a very powerful metaphor and so the theory is very powerful or has had quite influence in psychology. And the idea is it's a conjunct, it's called feature integration because you need attention, why? Because here I need to combine two stimuli. I need to combine stimulus along two dimensions. Folks, I need to combine, well I sort of, I need to carry this phase relationship. Is it black white?  Or in these stimulus over here I need to combine information along two dimension. I need to know about color and about orientation. So the target is defined as a conjunction of green and horizontal. Well here the target is only defined by red and here it's only defined by horizontal. This is called the feature integration theory, FIT. Now notice sometimes there can be a tiny difference between serial and parallel search. So here notice it's always the same, all objects here, the three objects, the L's, pluses and T's. They all consist of just two lines. They're always the same length. The only difference is here, here you'll have a plus among L's and here you have a T among L's. So the only difference is the pluses is and the T is this, right?  So this target is very difficult to find but this target is trivial, it's very easy to find. So that's really, it's quite difficult to explain which is also the Achilles' heel of feature integration theory. So today people, although it has been sort of seen its day feature integration theory because it's very difficult to explain for instance why, as I mentioned before, why the only difference between here and here that this should be parallel, this pops out the plus while this one sort of needs serial attention. Particularly problematic is the fact that the search slopes can differ widely. Depends exactly on which stimuli you use. So these, depending on the exact stimulus, I can essentially get any slope between five milliseconds and 150 milliseconds per item. Depending exactly on the stimuli I used and you know, how different, for example, here the orientations are very different. Well, what happens if I don't make them like this, if I make them like this, if I make them progressively smaller, for instance, then the degree in general, the degree of difficulty, of course, I mean, as you can imagine, well, it goes up dramatically. Also, it turns out not all conjunctive searches are required serial time. So the feature integration theory says, well, as long as I need to combine input from two different maps in cortex, like a map for color and orientation, I can only do that using attention, that takes time. But then people showed, for instance, that if you combine motion and depth, you can perfectly well do that in parallel. In other words, if you have things that move in different planes, they can move here or here or here, and they can move to the left or to the right, that thing you can perfectly well determine in parallel. You can determine that something moves here in this direction at the same time as you can attend to stimulus over there. And also it turns out that some conjunctive searches, some searches defined by triple conjunction can proceed in parallel. So certain things that have three dimensions can pop out. So the new models now, and it's a heavy research topic for those of you looking for synthesis, now models that emphasize competition that don't have this explicit serial nature, that emphasize competition that you have these neural networks, they compete each other, and essentially the slope, and how big the slope is in the search task really depends on the signal to noise. Depends how different say that how easy it is to pick up the target among the distractors. It's not an all in none thing, it really depends if it's very easy, then you have very flat search lights, the more difficult it is, the more steeper the search lights. So these models are much closer to the data than sort of this model. This model has a virtue, it's very elegant, but doesn't pertain to the brain, unfortunately. Okay, so there at the same time, this is work that I did when I was still at MIT, working with Schimanoelmann, a very powerful and popular way of looking at, modeling this is using a different mechanism, a different related mechanism called saliency. So I briefly alluded to it already and I'll talk about it a little bit more, or maybe I should first talk, oh no, let me do this, that you can understand attentions operating having two components, there's a bottom-up form of attention and top-down form of attention. The bottom-up form of attention is essentially inherent in the image, it's inherent in the input. Certain things I inherently attract attention, they're salient, they're conspicuous. So for instance, if everything else is stationary, if everything else is and I move, I will inherently attract attention because I'm a moving object among stationary theme. If this yellow on the background of this red tie, my black, this is attentional attractive. If you have something that's onto this, everything else in this neighborhood is onto this, this will be attentional attractive, salient. And then superimpose, you have a second mechanism where which sort of you control as it were, which is volitional controllable and takes time and sort of is much more powerful. So I can look in this crowd, I can look, if a friend of mine see, I can scan each of you with my eye movements or with attentional, with attentional COVID attentional movements. It's, I can direct this attentional spotlight at a place in the image, I can direct it at a particular feature, like I can enhance everything that's red, or I can direct it at an entire object. Now how do I model the saliency mechanism? So the idea we had roughly 20 years ago is that, well, wouldn't it be elegant if somewhere in the brain there's a single map, one or multiple maps, that encodes things not in terms of red or not in terms of horizontal or color or depth, but encodes things how interesting they are. So the claim is at the time we made there's a topographic map somewhere in the brain in the various locations like the superecoliculus, the frontal eye fields, areas like LIP and the parietal lobe, and that these neurons see a code not for red, among green, I mean not for color or motion, but the code for the saliency. And then the idea is that you have a saliency map, you have another network that tells you, okay, currently the most salient location is this, you better look at that, because that's a salient location. Then you attend to that location, you read out the properties of the neurons at that location in the various feature maps, and then how do you endow this mechanism now with dynamics? How do you go on to the next step? Well, you just inhibit that location in the saliency map. So it was the most powerful location, it's the most interesting location, after a while let's say 200 milliseconds you inhibit it, and then your winner ticker will automatically shift to the next most salient location, we'll inspect that, then you inhibit that location and immediately go to the next third most salient location. So subsequently this sort of inhibition of a return has been found psychophysically and called IOR in addition of return. So DS, you briefly attended to one location, for the next fraction of a second or a second you're much less likely to return to that location to attend to it again or look to it again, both covertly and overtly for a second or so, that makes sense, right? If you wanna inspect it, display, you don't wanna return, you know, if I looked at that just a while ago, I don't wanna immediately wanna return to it, that's very wasteful. Now you can implement this, it's been implemented a couple of years ago by Laurel E.T., who's now a professor at USC, you can do beautiful movies. You can do this, I'll just show you it too, you can do this on all sorts of stimuli, you can mimic a lot of psychophysical experiments and you can use it on real stimuli. So the idea that you have, that you filter that the way he did it, he filters, he represents the visual world in lots of different maps, in this case 42 maps. In the meantime, there are many more maps that operate in color space at different spatial scales at very high fidelity, at very high spatial resolution and at very coarse spatial resolution. It does it for color, for intensity, for orientation. Now sort of, we've added motion channels.  Finally, combine all of them. You have these center surround operation where essentially you say, okay, I'm looking at the output of a red-green cell at this spatial scale and I'm subtracting it from a red-green cell at this spatial scale. So the idea is, you know, what salient, as I said, is something, if something is very different from its surround. So you can do that by sort of looking at nonclassical receptor field of neurons. Then all that gets combined in the saliency map and then the algorithm just looks at what's the most salient location. Okay, let's read out its content and then let's go to the next salient location. Let's go to the next salient location by automatically inhibiting that. So if you do that, this is what he did for his PhD. Okay, maybe we should slow, I'll show it again. So what you see here on the left is always a display. And on the right is a saliency map and this is the spotlight of attention. So it wanders around. Let's just do that again. It's probably a little bit too fast. So just, this is all for one setting. It really works remarkably well. So these are the, those treatment tasks. Just pop out. Here you don't have pop-outs. Here you have to laboriously go from one to the next. It works great for natural scenes, posters, images, noisy images. And this current work in the lab that applies this, for example, to motion. Here's a piece of an underwater video at Monterey Bay where the aquarium there is trying to classify, they have thousands of hours of underwater video imagery and they want to make a complete list of all the life in the ocean at various depths in Monterey Bay. It's very deep. And so they have these automatic submersible that swim around there. And right now they have a team of nine women who very laborsally take each image and annotate it. It's incredible boring and, you know, it's not a fairly exciting task. And so here the idea is that you have this saliency model and it really works remarkably well. That sort of picks up what salient, in particular these weak elongated fish-like, colony-like creatures, and then reads that information out and gives it to an object, it misses this one, object recognition scheme. I mean, the only point I'm trying to make is that these safety-based schemes work remarkably well for real images and are now being applied to all sorts of situations. Let's finish here. I mentioned this already, top-down attention doesn't only work, you cannot only attend, you can attend to space, as opposed to expand, but not only to space, you can also attend to features. You can, for example, if you know, in fact Melissa did her PhD on this and we'll talk about it next week when we talk about neural hardware, you can, if I know the target is red, for example, if it's a kid of the crowd of kids and I know my daughter's a red dress on, I can sort of bias red. And what you can show them psychophysically and also now in the image are that you're more likely to attend to red things that this bias itself helps you to know the red is sort of more salient and you're more likely to look at red things than at non-red things. Furthermore, you can also attend to an entire object or so people have done nice experiments when you have two overlapping objects, they're overlapped. This is a little bit like the gorilla example, when you have two objects, sort of two complicated letters that overlap. You can perfectly well attend to one without knowing what the other stimulus is, although physically they sort of overlap. And physically, in terms of the visual input, both strike your retina, but you only attend to one or to the other. There's some nice experiment. And you can even show that you attend to the entire object. So in other words, attention is a very general purpose module and it's going to depend on which area you're looking at in the brain. Attention ultimately, as I mentioned already, relates to neuronal competition and that competition is going to take very different forms in different areas. And it's going to affect, just as a preview, it's going to affect almost every area in the brain with the exception of retina. There's no evidence that, because probably there are no fibers, well, there's no fibers that go back from the brain into the retina, there's no evidence that any of this affects the attention but any other structures is modulated to larger, to smaller extent by attention. So that you can think, so this just summarizes again what I said, that there are at least these two very crude forms of attention, I mean, looking at this from a very sort of high point of view, very far away from the data, you can summarize and this was done already by William James as two forms of attention, bottom up and top down. This operates throughout the visual field, this can be very focal. This acts sort of automatically more or less all the time, that this is something that you learned early on with visual experience and now it's present all the time and you can't shut it off. It's very difficult to shut it off. So I think this is one reason why you have in the Army, I think when you have an instructor who sort of stands next to a recruit and screams at the recruit and the recruit sort of has to stare, ramrod, straight ahead. I think what you have to learn there is to use your frontal eye field to inhibit your superior colliculus to avoid looking over at that screaming face. And we know if you go to a bar, a sports bar and there's a TV sort of flickering, it's very attention grabbing. And again, that tells you that this thing, the saliency bottom up operates all the time in the visual field and you actually have to prevent, have to sort of inhibit it in order to avoid moving your eye there. Again, top down is much more general, it's much more specific. This one, I haven't discussed the evidence, this one seems to be transient, while this one is sustained, although it costs an effort, you can intend many seconds or you can attend many minutes to something but at a cost, right? That's why paying attention is difficult. Actually, it's an interesting question. Why is it difficult? I mean, metabolically, why is it so difficult for us to attend to something for a long time? I mean, what is the exact price? Why is it so expensive? I think it has to do because you constantly, it's a cost of inhibition when you attend. Same thing when you're trying to pursue a single shot of train, it's very difficult to do it for a long time because we know we constantly get distracted, right? There's a random thought, oh, I forgot to do this or that or something else, constantly get distracted. So when you really concentrate on something, I think you constantly have to prune away all the other intervening ideas that sort of interrupt you on that. I think it's what makes it metabolic expensive. Anyhow, this doesn't depend on task. This bottom top down depends totally on task. It depends on the task instruction. To give you one instruction, look for red among green or look for tea or something while it doesn't influence salience. This one, again, is not un-evolutional. Control this one is. This one, so now in the last five minutes, I wanted to talk again to pick up again the relationship between consciousness and awareness and attention. This one doesn't necessarily guarantee access to attention. So you can perfectly well attend to my voice by somebody jumping over there up and down. That person at this moment might be much more attentional attractive but you willfully choose to attend to me. Now that, as I said, that might take you some effort because you have to sort of suppress the neural representation corresponding to the person jumping up and down there but you can perfectly well do that. I know of no case, it might be possible. I just know of no evidence right now that you can attend to something without being conscious of it. Okay, let's finish with that. These two forms, I'll leave you with these two forms of attention.


Thank you. Okay, so I am here to talk about bias auditing. Auditing, artificial intelligence systems is now all the rage, but we began working on this topic some time ago with a lot of people. We have tried to amass diversity of people, so here are the people who are permanent in our work team. These are people, but we actually have had hands-on workshops with diverse people from extremely different, no, not extremely different, but quite different backgrounds, so people working in nutrition, journalists and data scientists as well to get the grasp about how bias auditing could really, really work. So I'm going to talk a little bit about what bias is and then how we have approached it. So bias, as per the definition of the Oxford languages, is an inclination of our prejudice for or against one person or group in a way considered to be unfair. And how does this translate into our everyday life? Usually in our everyday life this translates in the form of stereotypes. Stereotypes are subtle but very, very useful ways of packaging bias, and I am going to give you an example just so we are sure that we are in the same page. For example, would you consider hiring Sylvester Stallone for your film? You probably, maybe not, maybe you would, but probably what you would not is consider hiring them, hiring him as a screenwriter. Although he was shortlisted for the Oscar as best screenwriter for Rocky. I'm not very much into this kind of film. So this is an unfair situation for him because you are not giving him the same possibilities as other people for this job. Even though he is a white male coming from a central country and strongly recognized professionally. So imagine what is left for the rest of people who are not so well positioned. So this is how stereotype hurts. And in language technologies we have lots and lots and lots of stereotypes that produce unfair situations for people. For example, if you type the poor people are, this search engine tends to complete your search because it assumes that you will probably look for what most of the people have been looking for or have been writing about this. And I am going to give the completion in Spanish because in English it has been word railed already and probably in Spanish it has as well by this time. And you can see that some of the auto completions that this mainstream search engine gives for los pobres son are things like los pobres son responsables de su propia pobreza. Which means the poor are responsible for their own property which if you are poor and you are using this tends to have an effect on how you are to perceive yourself. Which is not so positive. And you have more auto completions in the same line. It's not just one error. It's a systematic error. And that is what we call bias in data science, artificial diligence or the marketing name that you choose for this is systematic error, non-homogeneous error. So this is something that we want to audit systems for. We want to audit systems and see if they have this kind of systematic behavior that discriminates against some group. And in some cases these biases are huge like gender bias. Gender bias is something that we all know that is pervasive in most cultures. And we kind of have a consensus on how to model it. And we have resources to do that. And so we kind of have been advancing in how to measure that. But some others are not so huge. They are more subtle and they are difficult to perceive, to formalize, to transform into something that can be used systematically by anyone. And most of them are culture dependent. And this means that they are also language dependent. So you cannot reuse what has been done in one language for other languages. You cannot reuse what has been done in the same language for one culture to another culture within the same language. So this means that it is critical who defines bias. Because only people with bias experience can define bias. So only people with experiences in discrimination can get to define bias. And experiences in discrimination come in all colors, not only in white. But we have been having a huge problem with this. Although bias auditing has been advancing in academia. So it's a hot topic in conferences. And there are many, many papers and toolkits and frameworks and proposals about how to measure bias in language technologies in particular, but in general also in images and whatever. The very big problem that we found when working with this is that these approaches are very technical. Are highly technical. That not only that they require you to be able to program, some of them do not exactly require programming skills, but they require that you manage mathematical concepts. And they want you to install complex software. And they resort to terms that are very, very, very technical. So people with discrimination experiences have to climb over these very steep technical barriers to be able to express their discrimination experience. And that is suboptimal, highly suboptimal. So what we did was a tool to facilitate that people with discrimination experience, and when I say discrimination experience, I'm talking about not only that they have suffered discrimination or that they consistently suffer discrimination, but also that they have studied it or worked with that. That these people can express their knowledge without the technical barriers. So we, in this tool, we resort to intuitive concepts and we have extensively tested that these concepts are actually intuitive for people who are not in data science. And intuitive concepts like, okay, you think that the technology that you're using is biased. How do you see that? Well, you can see that, like actually see that this way.   Okay, sorry, sorry, sorry, sorry. And you can see that they are far in space, yeah?  They are almost both at this line, while masculine, fat, and masculine, ugly are much more separated.  And this is something quite intuitive. Underlying this, there are vectors, there are distances, there's a cosine and principal components and embeddings, and neural networks and all the blah, blah, blah, technical things that you want. But this can be understood. Actually, this is not something that we came up with. This is something that people working in the social perception of eating habits intuitively knew, and they checked it in our tool and they found this. This is a finding that these people found about the underlying artifacts in language technologies that actually deployed. So how does this translate in discriminative behavior of technologies?  And when you have the fat in feminine plural, they are disgusting, they can get pregnant, which are less positive complications. And to do this, we did not need to resort to any of these concepts that have been mentioning and that populate all the literature about bias auditing in academia and available frameworks. So we are happy to be working with the intuitive approach because we find this is useful and productive for people to express their intuitions about bias. But this, as you can see, is a little bit anecdotical, and you need to be more systematic in the way that you audit bias. So this is why we provide something like this. Yet this is more systematic than that can be collapsed into a number, a figure that you can report to the competent authority or whomever. This is, for example, to the right you have feminine, to the left you have masculine, and you can see that verbs like argument or think or ask are closer to the masculine meaning than to the feminine meaning, while knitting, sewing, and dancing are closer to the feminine. And this kind of systematization can be achieved by just listing words that represent the feminine meaning, like woman, she, mother, girl, whatever words, the people with the experience know that represent the meaning that they consider as feminine, the same for the masculine, and then check whether words that should be in the middle are in the middle or not. With the typical example of these are professions, you have a representation of feminine with she, blah, blah, blah, a representation of masculine with he, man, whatever, and then you have professions, and you will say, okay, but you have then, for example, professor feminine, and then professor masculine, and they are feminine masculine, but you can see things like nurse feminine, which is very much into the feminine side, nurse masculine, which is the least masculine of the masculine professions, and this is clearly biased, and this is quantifiable, and this is reportable, this is systematic, and this all available in this tool that is graphical, and you do not need to know that this is measured using cosine distance with woman, whatever, yeah? So what is critical here is not that you know about vectorial space and distances and similarities, but about word lists representing the biases, the stereotypes that you're experiencing or that you have experience with, and for that, the only thing that you know is experts, experts have a hierarchical role in auditing systems if you do not have technical barriers, yeah? When we were working with this in the first hands-on workshops, the feedback that we got from experts was that this was very limited, working with words was limited, because words are ambiguous, we knew that, yeah, I'm a linguist myself, we knew that from the start, but this was the first thing that we had, so words are ambiguous, they are inherently vague, they need the context to specify their meaning, and then there's also multi-word expressions, things that you express with more than one word, and then there are non-binary semantics, like age, you do not have old and young, you also have child and young and teenage and middle-aged, so it's inadequate to express that with this kind of diagram, isn't it? You don't want that. We also have problems with non-marked meanings, like we said, okay, we want to check about indigenous, but then what is the opposite of indigenous? Non-indigenous? No, it's not, okay? So to address that, we worked with language models, we worked with language models, language models are all the rage now with chatGPT, but this was before that, when language models already existed because they have been existing for some time now, and what we did was work with sentences, people express the biases, the possible biases in sentences, and if your language model is not biased, it should give equal probability, it should prefer all sentences equally, and if it is biased, it means that it is more likely to say one thing than another. For example, here you see the Bolivians cannot manage their money, the Argentinians cannot manage their money, the Germans cannot manage their money, guess which of these sentences the language model is more likely to produce, like GPT. GPT is more likely to say the Bolivians cannot manage their money than the Germans cannot manage their money. This is a very nice way for people to express their experience of discrimination, their experience of stereotypes, and it is quite easy to measure if a given artifact, a given language model, like the one that's this underlying chatGPT or Lama, or whichever language model you like, has a preference to say one thing or the other. So now we are quite happy with our instruments, but we are people working in computer science, I am a linguist, but I have been working in computer science for a long time now, so my imagination is beneath the standard that any standard that you can get to, so we have only been working with gender and race, there are many, many, many more biases to be studied, so we need experts to create these resources, word lists, sentence lists, and we are expecting, we are working now toward people using this or any other approach, but we are very confident that our tool is useful to create these lists that represent the stereotypes that they experience so that these lists can be readily used for people working with artifact, these artifact, these language technologies to have a first audit of these, of their technologies to assess what next steps need to be addressed with the results of your audit. So we encourage you to create these word lists, contact us, we are working towards having these approaches more widely used and to have experts in the hierarchical central role that they deserve in the development of audits for language technologies.


Absentee ballots are now available to eligible voters in Crowan County who want to cast a ballot prior to election day on November 8th. Crowan County has 40,000 voters. Because of the population difference during the fall and winter months, Crowan County's estimating that 10 to 15,000 of those voters will be sending absentee ballots this year. To better accommodate those travelers, Crowan County gives voters several voting options. There are several options available to voters. They can come in person to their local county office here at the courthouse. We're open Monday through Friday from 8 to 5 and they can vote in person here. They can fill out their application, receive their ballot, cast their ballot and leave their ballot here with us. Or they can also choose to have a ballot sent to their residents where they can vote in the privacy of their own home. Voters who want to find out more about absentee ballot applications can visit the county's website. If you've enjoyed this segment of Lakeland News, please consider making a tax-deductible contribution to Lakeland Public Television.


Hi, my name is Ellie Chang. The spelling is a little confusing. I do admire when people just go for it, but it is pronounced Ellie. And today, I'm here to talk about the glorification of generalization in tech. So glorification of generalization, it sounds like I might be saying that being a generalist is bad, but that is not at all what I'm saying. I think there's a place for generalists, there's a place for specialists, or a mix in the workplace. Do whatever, do whatever works for you. But what I'm not so much a fan of is glorifying generalization, because what you're saying in this case is, this is the way to work. Every other way of working is inferior, and that I'm not so much down with. So ultimately, today's talk is about perspective, having some perspective about trends in our industry. Now, generalists have been around for a really long time. You've probably heard of the Jack of All Trades, and trades there is the important part, because the Jack of All Trades does more than one line of work, and maybe isn't particularly a standout in any of those lines of work, but does multiple lines of work anyway. More recently, we've had something called the T-shaped professional, so-called because they have one particular specialization that they spend a lot of years diving into, and becoming an expert in, and then after they master one specialization, they go and branch out across the org, and learn about other departments, and those areas of knowledge for them are a little shallower, and that's the top of the T. This year, we have something that's called the full-stack employee. Not to be confused with full-stack engineer, which is just focused on coding, or full-stack designer, or full-stack marketer. The full-stack employee is a term coined by Chris Messina, and it's the ultimate generalist, the generalist-generalist. They're not limited to one line of work, two lines of work, three lines of work. There's really no limit to how many lines of work they're in, and the full-stack employee does everything from marketing, data analysis, engineering, design, you name it, they've got it covered, but they're not just limited to their skills. As Chris describes it in the article he wrote, full-stack employees have an insatiable appetite for new ideas, best practices, and ways to be more productive and happy. They're curious about the world, what makes it work, and how to make their mark on it. It's this aspect above others that defines and separates the full-stack employee from previous generations. So you've got someone's skills, and then apparently you've got characteristics which ranks above their skills. Chris also goes on to say, being a full-stack employee isn't necessarily easier or harder, it's different. With always-on connectedness via their mobile devices, the boundaries between work and non-work blur, creating an implicit expectation that there's never a time too late for a Slack chat or Google Hangout. So there's skills, characteristics, and now you're throwing lifestyle into the mix. And you might be wondering, well, what's the big deal if someone just wrote an article? But this guy isn't a nobody, he's in Silicon Valley, he's the inventor of the hashtag. He's also a well-known open source advocate, and long before he wrote this article that became insanely popular, people were already in the tech industry thinking about full-stack. They just didn't have maybe the vocabulary for it. So when this article came out, people loved it. I mean, people who identified as full-stack, employers who want to hire full-stack people, and apparently universities who are teaching people to be more diverse in their MBAs. But who doesn't this idea benefit? Who doesn't necessarily fit into the box of knowing how to do everything, wanting to do everything, and having the time to do everything? Well, probably a lot of us. And in general, that's a good thing, but when this idea gets popular, then we all look bad in comparison. So then the full-stack idealization is really problematic. And let's talk about how specifically how this idea is changing things in the workplace. How does it change hiring, for example? Before, we would talk about skills. Someone would look at your resume, think about what skills you have versus what skills the company needs. But now, you don't have to get specific about skills at all because full-stack employees are expected to have all the skills or be able to learn them right away. So instead, we're just getting specific about the type of person you are. So really, we're not even pretending it's a meritocracy anymore. It's not about your work. And it's not really even culture fit because they're not looking at your values or how well you get along with the team. They're looking at how much do you think and behave in this very specific way that was described earlier. And when you throw in stuff like working all the time, that's not a culture thing. That's just a really stringent restriction to put on people. So it's not really that great when you're looking for a job. What about career development, though? What if you're in the job already? How does this change the conversation there or your day-to-day work? Well, whereas before, you might expect that the company will help pay for you to get better and better in your career.  And if you don't invest in yourself, why should the company? Were you really a true full-stack employee? And in terms of corporate mobility, you're not expected to conveniently focus on moving up anymore because you're supposed to focus on moving horizontally. You're looking to add more and more skills to your stack, so to speak. So promotions, that's not a thing you have to concentrate on.  And that's a misunderstanding of what managers do. Managers aren't the people doing all the things of their team members. Managers have teams that they then coordinate to get something done by a deadline. So the most important thing about being a manager is being able to manage. Thank you. And really, the top skill you have to have as a manager as a good manager, you have to be able to manage. But as a manager, in general, the most important skill you have is knowing how to navigate the political hierarchy of your company, AKA getting the right people to like you. I mean, we've probably all had bad managers. They certainly didn't get there by their skills or anything other than hierarchy navigation. So this whole concept of the full-stack employee is taking all the power that employees have and saying, hey, companies, take more of that power. Employees get to be expected to work more and more and expect less and less in terms of money, in terms of time, in terms of investment on the parts of employers. So what is it then that we're really promoting other than becoming a full-stack employee, of course? Well, we're saying that you should be an independent hero rather than a team player. You should be the person who does it all because you know it all. But the best and the highest performing teams aren't high performing because of individual success. Ultimately, how well a company does depends on the success of the team as a whole. And studies have shown that when you put people individually versus in teams, the people that are in the best teams are not necessarily the highest performing individuals. So it's really about being able to work together and being able to collaborate really effectively. And if you are in this mindset that you have to do everything yourself and you are not good at setting boundaries, what you're really doing is creating a recipe for yourself to burn out. Plus, this whole concept that you can even be a person that knows everything is pretty insulting. I mean, the name itself says full stack, but what they really mean is multi-stack because how is it possible that one person can know everything there is to know in a company? I mean, they're not advocating that you spend years and years studying each discipline and each line of work. No, they're advocating that you just like have a passing knowledge of the thing and so you can do it. And that's pretty insulting to specialists who do take the time to study their field. We're also promoting the idea that work is a virtue. So by virtue, I mean, virtues in life are courage, honesty, kindness. If you put work up there with them, what are you saying? You're saying if you're not working so much, working all the time mentally and physically, then you're failing as a person, morally. And that's a pretty gross kind of statement to say. Some of you will probably have seen Office Space and will be very familiar with the scene I'm about to describe. But for those of you who aren't familiar or just don't remember what's happening here, Jennifer Aniston is playing a waitress and in here, her boss is confronting her about the flare, the buttons that she's wearing and he goes, we need to talk about your flare. And she looks down and she says, I thought I was wearing 15. He goes, yes, Brian over there is wearing 37 pieces of flare. And she goes, I thought 15 was the requirement. He says, 15 is the minimum. Do you wanna be a person who only does the minimum? And so she says, well, so more than. And he just gives her a look and says, people come to Chachkis for the attitude.  And he wants to create this atmosphere in which she's competing with all the Bryans of the world both on and off the clock and constantly having flare offs about who can wear more. And that's just because for all the tech skills that you're getting, people are thinking that tech is work and work is life.  I mean, it might be relevant to more than one industry, but primarily the tech industry. They're not advocating that you're learning printing skills or that you become a lumberjack. So your focus is actually really, really singular. And it's one thing to love your job and to work really hard. And it's another thing to not have anything outside your job. I mean, we weren't here, we weren't built as robots sent here with like one purpose. We're human beings. We're complex. We have other interests. And to really force us to restrict ourselves to one thing I think is pretty damaging.  One of the things is why is the focus on quantity and not quality? I mean, why is having more skills better? Why can't you just have one skill that you're really good at? And why is this set of traits unique to being a full stack employee? I mean, can you not be curious about the world outside of work? Can you not be curious about one line of work? Why do you have to be curious about everything? And why do you have to have an insatiable appetite for productivity as it relates to your employer as opposed to your life if you really want to be productive, wouldn't you want to get things done by a certain time rather than having the always on be part of your life? And why is the full stack employee even being defined by more than what they're doing? Like for all the descriptions that they're talking about that make a full stack employee, the companies are asking that it's about you being. You being their idealistic employee rather than what you're doing tangibly at work. And really what makes this career path the way to go? You can do a lot of different things. This isn't necessarily the highest paying career path. This isn't even necessarily the most interesting career path because we are all interested in different things as individuals. So why then do we keep pushing this idea that you should be a generalist in tech? Ultimately, you can be really successful in a lot of different shapes and forms, whether that's at work, outside of work, in tech, outside of tech. So rather than following the latest trend of what's popular right now, you can think about what's right for you as a person. What are you interested in? What do you like doing? What can you reasonably do and make money at and decide yourself what's the best path in work and life? Or in other words, do you? So my talk originally came from an article I wrote on Model View Culture in which I'm much more eloquent than I am today. If you wanna read it, there's a shorthand link to it. And now you should feel free to tweet at me as much as you like, good or bad things. And thank you very much.