Good morning. It's it's really nice to be here in those were just fabulous opening comments and I sort of couldn't be teed up much better so as mentioned my training is as an economist and mathematician and When I started doing my research I got really interested in this field of what was called complex systems Which is really starting in the 1990s and complex systems are things like economies the brain ecosystems things like that and there's a group of us Mathematicians physicists the economist theoretical biologist who are studying complex systems and one of the things we found In our research was that diversity makes these systems more innovative more robust more interesting has been what I was mentioning and There was sort of this disconnect as he as he mentioned introduction when I would talk to organizations and businesses when you mentioned diversity There was sort of a feel-good social justice Angle to this as opposed to sort of a bottom-line performance angle So what I want to do today is I want to talk about Research some of it by own but most of it done by sort of a group of scholars That really talks about really just sort of the bottom-line benefits of diversity So the outline is I'm going to start out by talking about complexity a little bit I want to formally mean define what we mean by complexity because if You go to a decent restaurant. They'll decide they'll define the pasta is complex So I want to sort of make sure we nail that down Then what I want to do is I'm going to talk about two specific tasks that are relevant to the bank one is prediction and the other is problem-solving and what we'll do is we'll talk about why diversity actually is as important as ability in those contexts and then I'll There's a little bit of data and then talk about how we may be practice Leveraging diversity and then we'll hopefully have some time for some questions So to frame this a little bit and just sort of juxtapose The bottom-line notion of diversity with a sort of social justice notion I want to use a quote by Wendell Berry which says we have been wrong We must change our lives so that it's possible to live by the contrary assumption that what is good for the world is good for us So why do I begin with this quote? I begin with this quote because I think when we talk about diversity and inclusion We tend to think of this as the right thing to do And what I'm going to do is argue that it's not only the right thing to do But it's also the better thing to do and that's why we have to sort of change our minds So a wonderful quote with regard to this is by Astro Teller who runs Google X which is Google's sort of innovative out there You know self-driving car space and he says look people throw on the word diversity like it's a tip at a restaurant But what you really want if you're trying to be a cutting-edge innovative place is people who think in different ways Right, so that's gonna be the key to think about this And I have a new book coming out in August called the diversity bonus and the reason I put this out There's I have a previous book called the difference Which is a book about sort of how diversity makes things better and this was a book written when I was a professor at Caltech And it's a very mathematical book and I was at the IMF a couple years ago and somebody introduced me And I said this is a really good airplane book And I was sort of surprised that he said that and then he paused and he said if you're flying to Singapore Because it's really kind of tough sledding right because it wasn't really written as a business book This book is written more as a book for practitioners and it'll come out in August But my take on this and the way I come at this is much more like the way a standard economist would do it and what I do is I I come at this by sort of writing simple models to try and organize how we think about things and I think the theory is incredibly important in this particular space because It's not the case that all diversity necessarily makes us better This is something that has to be managed and understood we have to think about How do we get the right types of diversity in the room and how do we manage those things? So if we don't have a theory if we don't have an understanding of how diversity makes us better We're gonna end up with what my sons have two teenage sons They describe as a dog's breakfast, which is basically just dump a bunch of stuff in the bowl and hope somebody will eat it So let's talk quickly about complexity. So they get global financial networks They're much more connected than they were in the past if you look at how we make policy decisions in banks So this is a Chart from the World Bank But what it shows is it shows the different US banks prior to the bailout and these numbers that you see between things like this 456 and this 94 And this is Lehman Brothers not Lehman sisters here, right? These the size of the numbers show sort of how correlated stock prices are in the tails And so a big number here means if something bad is happening to AIG that means something bad will happen to Merrill Lynch so when you looked at sort of This network what you see is it's not just that we can draw lines between these different financial institutions It's that some of these lines are stronger than others and when we think about a particular financial You know entity we can't just think in terms of its own balance sheet We have to think in terms of how it affects something else So when you talk about complexity what we mean is we mean things that are connected in sort of deep ways So a couple years ago is involved in a project to try and figure out what's causing obesity and we solved it There it is. It's the simple solution now if you really obviously you can't read this That's on purpose if you drill down what you see there's different colors in here So for media social economic food activity infrastructure So everything from lack of sidewalks to large Coca-Cola's everything you can think of is sort of in this graph But the point is is that no one we look at the big picture. No one could possibly understand obesity We now have so much knowledge so much information We recognize there's so much complexity in the world that one person can't do anything anymore, right? If you really want to sort of solve obesity understand it you need teams of experts in order to do it So when we say something's complex, what we mean is that it's between ordered and random. It's not simple It's not completely random or another way we put it is to we say that it's deep It's something that's difficult to explain engineer, right or predict or evolve, right? So the brain is very difficult to explain. It was extremely difficult to evolve Right, and if you tried to engineer a brain that would be difficult So we would therefore say the brain is complex, right? We wouldn't say a coffee table is complex Because it's none of those things So if you look at economic phenomena, some of them are complex and some of them aren't so oil production isn't that complex, right? It pretty much grows in a linear fashion with economic growth oil prices because you can have inventories You can have tankers just cycling outside of ports waiting for the prices to go up or down, right? Those are complex So what we want to do is we want to say some things are easy some things are hard and some of the things that are hard that we want to use diversity So here's what I'm going to do and take two tasks The first one's going to be predicting or forecasting second one's going to be problem solving and we'll talk about why diversity is beneficial in each So I want to start with then I was mentioning group think I want to start with a book written by a friend of mine called the Wisdom of crowds how many have seen this book anyone okay, so they came out about a decade ago and Jim So wiki who writes this book he begins with this sort of amazing story of the 1906 west of England fat stock and poultry exhibition There's 787 people who guess the weight of a steer and the winner gets some prize like a cake or a pie or something Right, so there's all these guesses about the way to the steering the average guess is 1197 pounds Take 500 kilos The steer weighs 1198 pounds, which is amazing and this is a great way to begin a book But it's just an anecdote So what we'd like to do is move from anecdote to a real science of understanding like what is it that made this crowd smart? You can prove the following theorem You can show that if people are making numerical predictions that the crowd's error Equals the average error the people in the crowd well that makes sense You think the crowd is about as good as the average person in it But it turns out the crowd is actually better than the average person in it The crowd area equals the average error minus the diversity now. This is a real theorem. This isn't like a Management book theorem that like teamwork equals thoroughness plus effort plus ability plus meaning or something now This is like real math. Okay, so this is it's like the Pythagorean theorem. This is a mathematical identity It's always true any time you have a group of people make numerical predictions This will be true. So this C is the crowd's prediction. This is the truth This is just the square difference between the crowd and the truth This is the average of the people in the group So the crowd error you might think equals the average, but it doesn't the crowd There is actually less than the average and the amount by which it's less is these differences in the prediction So in Goulton's case, here's the actual data You know Goulton's been dead for a hundred years. So he happily shared the data, right? people don't always do that and So the crowd is off by less than a pound the average error Was about 70 pounds because these are squared errors So the point was these people weren't savants who could just guess the weight of a steer They weren't some sort of like geniuses from the west of England The reason the crowd got it right is they were diverse But this is the general thing if a problem is easy. This is where complexity comes in if the problem is easy Average error is going to be small, right? So the crowd can be smart without diversity But if the problem is hard Average error has to be big so the way you create a smart crowd is by having the crowd be diverse. Okay So when I talked to I spend a lot of time talking to the New York Fed and recently I've been doing some things with Some larger companies on Wall Street and one of the main things I try and get across is when they put together a portfolio of assets They think about having diversity, right? That's just one of the key things, right? You want a diverse portfolio of assets But when I have a diverse portfolio of assets, the reason I do it is to minimize risk The return I get on that portfolio is equal to the average return on those investments, right? So if one investment like the yellow investment does really well and the blue investment does badly, I get the average of those two investments With a group of predictors, I just showed you the theorem you actually get better than the average Right, that's a that's a mathematical fact. It's a mathematical fact that a portfolio you get the average payoff It's a mathematical fact with a group of predictors. You get better than the average So as diverse as your portfolio is in terms of stocks Your group of predictors should actually be more diverse, right? Because you're getting this sort of bonus hence the reason for the title of my book you get a bonus from this diversity Right, so diversity actually makes you better Okay, so that was people guessing the weight of steers a hundred years ago So you might say Interesting, but maybe not kind of where we're at now in an age of big data and hyper sophisticated models So let me jump to the completely other extreme. Let me jump to sort of what Was the most sophisticated case of collective prediction sort of this is most sophisticated contest ever run this is in 2006 Netflix Corporation released all of their data on The people who watch their movies So if you go to Netflix and you watch a show it'll give you a it'll give you sort of a gas in terms of how much You're gonna like that show. So if I type in avatar It'll say five stars and the reason it says five stars is because I have two teenage sons And this is they sort of have figured out this is what my family probably likes and it'll say five stars if you know Then was a French intellectual, right? So if he types in avastar, it may say two stars, right? Because it would be beneath him maybe to watch a show if Anybody so far as I know anyone types in rush hour three is a horrible movie It'll say one star, right? So that's a really easy movie to predict So here it is six years of data half million users 18,000 movies Largest data set ever publicly released and after they released it the US Supreme Court said you cannot release data like this So it may end up being the largest data set ever publicly released It's no it's no longer legal to do this and what they said is if anybody can beat their internal program So Netflix had this internal algorithm to predict movies. They said if anybody can beat us by 10 percent We'll give you a million dollars So what happens is and there's here's where diversity comes in So what happens is all these teams and this is these are the sort of the best and brightest Computer scientists machine learning people in the world They construct these predictive models where movies have attributes and then people have weights across those attributes What do I mean? So an attribute might be how much money didn't make in the box office, right? How many weeks was in the theater? Is it an action movie? How many days has it been available on CD also anything you can think of and Where diversity comes in is these teams had to sit around and think about what are the attributes of a movie? What are the things that might matter and then of those things that are matter Which ones can we scrape off the web because you've got to be able to you can't like code these in by hand for 18,000 movies So you got to have some way of like using reviews or using available data to scrape all this information on the med We haven't throw it into your model So the early leader in this is Belkorn. This is going to be the star of the show this guy Dr. Robert Bell who's the chief data scientist at AT&T Research Park and their team their team was called Belkorn They had 50 dimensions for each movie So he had a whole team of people working on this and this is kind of amazing So when you go home tonight try and think of 50 dimensions for a movie, right? So this is pretty impressive their best model could be to buy 6.8 percent Which is pretty good. They could so they 10% they win the million dollars six dirt 6.8 percent Now what they did which is interesting is they would like have a whole bunch of different models Where like these are the different data and they would sort of give different models different pieces of data And then to kind of average them now the reason they did this is you want to sort of Avoid overfitting and also one of the things sort of state-of-the-art in machine learning isn't to create one Supermodel that it's great a whole ensemble of models that you then average up because otherwise you end up Overfitting the data and finding patterns that don't really exist The other thing you do is you use this technique that's called boosting which is really interesting So the first thing you do is called bagging where you just sort of give give these algorithms different data So that they have so that they're diverse kind of like giving people different life experiences It's sort of like making computers like people the second thing they do is these use this technique called boosting Which attempts to create diversity. So here's what happens suppose like I've blue things are good and Positive ratings and red things are these negatives are negative ratings So you might have one model that says everything over here is good everything over here is bad and another model It says everything over here is good everything over here is bad what you see is like these things with the pluses, right? you're sort of getting wrong and So what you'd like to do then is like put more weight on these things because you're sort of confused one model saying they're negative one Model saying they're positive and so what you do then is you say okay? I'd like to get these ones right and these ones right and so then basically put more weight on these particular occurrences And you train something to figure that out. So what these boosting algorithms do is they actually train algorithms to be diverse relative to the other algorithms So what bell core does then instead of just having one model they actually create a hundred and seven different models None of them are as good as the best model, but by combining these models they can get to eight point four percent What's making them better? What's making them better is the fact that the models are diverse, right? They're systematically making diverse models So the story's fascinating. This is AT&T research park and if you read the media reports on this They're completely misleading because they're talking about these people trying to win a million dollars Okay, this is AT&T research park dr. Bell his budget is in the hundreds of millions of dollars This had nothing to do with winning a million dollars. They said everything to do with winning Right, this was a whole bunch of people competing to see who is the best data scientist? You know who's to has the best data science team in the world. This wasn't about the million dollars They were giving the million dollars to charity.  It's been great that you've been doing this and you've got a lot of publicity from you winning the contest But it's over you've got to stop doing this because you've got to go you have this other thing that we like to call a job and We would like you to go back to your job So he says fine fine. I'll just win this thing You know cue the chariots of fire soundtrack I'm just gonna win it right now and his boss is like, okay How exactly are you gonna do that because you've been doing this for two years and you haven't got to ten percent? How will you do it now? And he said well easy. I'm just gonna bring in some people who aren't as smart as me And his boss is like, well, who would that be and he said well anybody right because like my team is the best We've figured that out. I just need to bring in some people who are diverse Right, and we've got this theorem that says if you can bring in people who aren't as good as him or diverse They'll do better Right, so they bring in these Austrians called big chaos who aren't very good at figuring out attributes for movies But they're really good at figuring out how you weight the different models So they bring that them in they almost get to ten percent right and they do this fancy thing Or you remember I showed you that simple summation sign They take derivatives and stuff in order to figure out what these weights are using something called a Crestman weight function They still don't get to ten percent of so then they look at this team in third place fourth place And they actually take this Canadian team in sixth place who they bring in and these Canadians Weren't that good at waiting, but they're really good at figuring out funky attributes to movies Okay, so they had some these behavioral insights That never occurred to the people at Bell Labs, and this is the big one Okay, so this is a Will Ferrell movie and this is a movie called snakes on a plane I don't know if anybody's ever seen either of these movies, but here's the thing about these movies that the Canadians figured out They're a lot like margaritas And that is like right after you've had one it seems pretty good If you wait until the next morning, it doesn't seem so good So it turns out there's a whole bunch of movies that if you rate them right after you watch them You give them five stars if you wait a couple days You're like oh that was a stupid movie, and you give it two stars So there was these sort of this snakes on a plane Will Ferrell category that the Canadians figured out that had a huge effect Right, and so the same movie could get five stars or one star depending on when you rate it So they put all these teams together these three teams together now they have 800 variables for each movie, which is insane Right their best model is now 8.4 percent So there's just this bonus from this diversity because now they've got all these different variables They can get to 8.4 percent when they combine the models they get to 10 percent so they win right So they've got two different types of bonuses here one bonus is just more variables to choose from the other bonuses You combine a bunch of models so they win except for they don't win This is what's so great because the contest rules say once you get to 10 percent the contest ends in 30 days Their reaction was pure panic at this point. The reason why is here's what they knew There were 23 teams from 30 countries that they had been beating for two years But prior to this if you're an economist these other teams had no reason to work together now when there's 30 days to go They've got no reason not to so they all send all their data to Berkeley and Penn these are countries from all over the world and They combine 48 of these models hundreds and hundreds of models are sent they combine 48 of them And they take the lead this group called the ensemble takes the lead with a day to go Right, they eventually Both submit and it's a tie Belcore officially beats him on the fifth decimal point, but the contest only goes to four decimal points So the tiebreaker is who submitted first Belcore submitted 22 minutes sooner So fortunately the New Jersey public school has got the money All right, which is great, but here's the beauty of this whole contest the ensemble The people combining these models were graduate students So if you open up a thesaurus and you look up graduate student the first synonym is snarky person Right, so these graduate students on the ensembles web page when this is over they say here are the final results It's official Belcore takes second That's literally what their web page said and they're absolutely right because let's think through it I've got two models that are equally good Right so the same average error, but they're wildly diverse Right one is combining 48 models and around the globe One is complaining combining 17 models using 800 variables from three teams, so they're wildly different So if I have two models that are equally good that are diverse what has to be better Combining them Has to be it's a mathematical fact. It's like five squared plus 12 squared equals 13 squared if I make a right triangle Right this just has to be true So the challenge here when we think about Leveraging diversity is thinking about how do we get people who think differently in the room and then it's been almost mentioning How do we make them feel safe so that they share their different models because if we do we're collectively going to do better, right? So this isn't a Feel-good thing. This is just a mathematical fact now You could say these are people picking movies, right? And this is Neil Hunt the chief product officer of Netflix because this kind of cascaded on them They thought initially this was going to be we're going to find the person who's the best data scientist And instead diversity broke out, right? They didn't anticipate this at all He's like wow this turned out to be all about combining diverse ideas Not in some like touchy-feely sort of diversity is a happy thing But like the only way we could solve this problem is by putting diverse minds to it so if you look this is data from our Central Bank the Federal Reserve System and this is a regression where they compare We have something called the Federal Open Market Committee and this staff forecast if you take the 600 economists to make individual Forecast and you're on a regression on inflation just a simple OLS regression And then you say how much weight should be put on the average of the staff forecast or how much weight should we put in the FOMC Forecast do you get a weight of one on the staff forecast than effectively zero in the FMOC if you get unemployment You get a weight of one or point nine seven on the staff and minus point oh three in the FMOC on growth You actually get a little bit more weight in the FMOC that on the staff The point here is is that you've got all these in-house staff forecast They're just put in a book and they kind of read them you'd actually do a lot better just averaging the staff forecast So there's Using technology right and this was done by Christina Romer who is Obama's chief economic advisor Because she sort of believed that they were just ignoring a lot of information Okay, if you look at economic forecast in the European Union in the United States over a 40-year period So this is every base. This is 28,000 forecast by professional economists including the ones that the ECB puts out, right? But you're sort of team of economists the crowd so we know it's true The crowd has to do better than the average economist. They do 21% better Across all the state 21% better Okay, I mean mathematically. It's got to be better, but it doesn't have to be 21% better And here's the data this again. This is sort of some of these studies. There's 40 economists predicting some of these studies There's 12 economists predicting Here's a benchmark of one for just an average economist So one an average economist has an error of one if you just take two average economists you do 8% better And if you take three random economists, do you do 12% better? Here's the best economist to date Here's after the fact if you knew the best to come so let's say here's the best to date She gets 11% better if you average in the second best economist to date with the best economist to date you do 17% better Now this should be somewhat counter-intuitive right because it's like silly is the best economist I'm second best you would think let's not even listen to him Right and that would be the normal way you'd think about it But the reality is if my model is different than hers you should listen to me Because on a high-dimensional complex hard problem if we have different models that diversity helps you out Right and what you see is you see increases sort of all the way down to five six and seven, okay What about real people investing money? Well, this is a study of every single mutual fund This is an unpublished paper, but I refereed it seems right so I'll put the graph up But that's why the names aren't up here But hopefully this will become public information in a couple weeks If you look at the number of managers on mutual funds in the United States So this is 10,000 funds over a 20-year period They do 60 basis points better than funds run by one person and you can think well If that's true funds should no longer be run by one person and guess what that's true So 30 years ago 75% of funds United States were run by one person now 75% of funds are run by teams of people Right why 60 basis points, right? That's not very complicated math Okay, so the point is if you want it if you want to make an accurate prediction What you want to do is you want diverse crowds, okay? Let's go to problem-solving. This will be much quicker So here's a simple test suppose I've got a really hard problem and problem-solving difference in prediction and prediction I'm trying to come up with a numerical value for something estimate some future thing in problem solving I'm trying to find some solution to a problem like maybe some way to route my trucks Some way to design a computer chip, but I've got a bunch of variables that I'm choosing among and I want to try and find something That has really high value So what you can do is create a bunch of agents. This is what economists call people, right and Think of them as having diverse perspectives Which are sort of representations of this problem and then they've got sets of heuristics Which are methods they have of trying to solve them and all these agents are smart So now I create a group of the 20 best agents the best individuals and I compare them to a Random 20 agents all the agents are smart So I'm taking the very best ones versus a random one and I have them just kind of work collectively right on this problem So I can think of this as the following I've got one group of people who are really really good and one group of people who some of them are good and some of them who aren't that good right and What you'd think in this setting is that the group of people who are good should be better than the group of people Some of whom are good and some of them aren't that good it turns out though If you sort of do the math on this that the diverse group almost always outperforms the other group if you sort of Reasonable size groups like groups of size 10 or 20 The group of high ability does better if you have groups of size one because that means you've got the best person against a random person Right, but once you start having in 10 or 20 people this flips And here's why thinking of people in terms of ability on the problem kind of misses the point Right, that's kind of your score. What you wanted is you want to ask what tools do they have? Well, suppose this is an economic problem Maybe that all of these people have econ PhDs and they've taken a lot of statistics and mathematics So the third person doesn't know anything that the second person and the first person doesn't know So if I got this alpha group if I think in terms of toolboxes and Ted of scores I've got a whole bunch of people with that exact same skill set Right Where's this diverse group? I may have someone who's actually studied political science Somebody knows linguistics someone who's taken sociology someone in psychology So Andy Haldine who's the chief economist of the Bank of England? He was in Ann Arbor a few months ago and he was talking about how the Bank of England is now hiring Psychologists and I'm like wow is it 2017 already, right? I mean the point is right you want to think about really enlarging right who you bring in to discuss these policy things And Andy's great and he's really been a leader in trying to think about how do you diversify? The skill sets of the people who work in your research team Now the thing is that model this toolbox model is a simplification When there's a famous quote by George box that says all models are wrong And so I'd the paper the monologue just showed you where diverse groups do better than random groups was written by myself And Lou Hong is a mathematical economist at Loyola University in Chicago So I was visiting Cornell and there's a guy John Kleinberg is a canal who's much smarter than I'm he's a Certified genius by the MacArthur Foundation, right? He's won a genius award. He's a brilliant guy So he decided to look back at the my paper with Lou and rewrite a different model Sort of like if a statistician wrote the model so he went out on a pretend I'm a statistician right the same model so our model was we've got perspectives and heuristics We have people sort of coming up with representations of the world, right? So what some of our agents might have used Cartesian coordinates like x and y and some would use polar coordinates right with the theta and r stuff like that and What we showed is that like a people who sort of use represent problems the same way and solve it the same way gets stuck at the same points It's the formula. We call that sort of getting stuck at the same local optima So in our case what had to be true is the problem solvers had to be smart You had to be drawing from a diverse set of people and the key condition was the problem had to be hard So when we figured out what was true it was Smart people who are diverse hard problem, right? So John's model what she wrote with my Thorego as a grad student of his they said here's our model's totally different We're gonna assume that people are just distributions of solutions. So if I ask Celia What do you think she just dumps her ideas on the table? I dump my ideas on the table and then we sort of like Combine them or do something with them and come up with some best team idea And so the team value is then gonna be some sort of function of all the things we dump on the table That's their model So you can think of them this sort of a function of the following sort so you could have a linear function or an averaging function where you just sort of like average the ideas if that's the case if it's a Simple problem then the best team consists of the best people But as soon as it becomes nonlinear as soon as you do something interesting with the ideas that throw in the table Then the best team doesn't consist of the best individuals. So they get the exact same result that we get Completely different model But the exact same result and the exact same result is as soon as the problem becomes complex The best team doesn't consist of the best individuals Why because the best individuals tend to be similar and what you really want on hard problems is diversity Now what they go on to show and this is why Kleinberg is a genius and I'm not is that Because he uses words like sub-modular and super modular that just means nonlinear But what they basically show is this is that not only does the best team not consist of the best people on that problem There's no test that you can come up with Such that the people who score highest on that test Are the best team? So there's no single hiring criteria you can use That determines the best team Now there's a third model that Lu Hong has that basically shows This holds generically for any function that's nonlinear in a broader class of problems So as soon as it becomes nonlinear There's no test Right what you can't use the same test and then pick the best people what you need to do is you need to have diverse criteria Okay, so again this portfolio of asset thing doesn't make any sense Right because portfolio we get the average and the groups of problem solvers. We actually typically get better than the best Right, so if I'm in a room and I have crazy ideas most of the time You can ignore them most of the time and if everyone's in a lot of a great idea Then you get to take my great idea that rare moment when I have the great idea, right? I don't get averaged in otherwise. I would be like investing in Kodak Right where you'd lose all your money because my average return might be very very low Right, but with a group of problem solvers, you don't care about average return You care about the best and you care about how can we combine and interweave best answers And so again if you think just sort of at a heuristic level What that suggests is you really want to have a lot of diversity in the room Okay, let me do a quick aside on identity diversity except I've been talking entirely about cognitive diversity and see we talk about some of this as well But how does identity diversity relate to this right because when we talk about diversity we think about where I started We think about things like gender race You know culture sexual orientation that sort of stuff and here's the funky thing and this is work by Catherine Phillips This is a survey article, but if you survey hundreds and hundreds of articles What you find is diverse groups Based on identity are actually more creative or innovative than groups that are homogeneous, right and Catherine and I are good friends and so we like to joke about this She says she's basically when I just spent the last 30 minutes telling you is kind of obvious That if you have cognitive diversity you should do better, but what's less obvious is that identity diversity actually makes you better and The story behind this kind of has to go as follows right it has to be the case that identity diversity somehow maps to Cognitive diversity which leads to better outcomes. So the operative thing has to be cognitive diversity. The question is how does identity diversity? Get you to this cognitive diversity. This isn't very complicated either, but what you've got it is you've got to kind of unpack it So the stories the mathematical models aka stories that I've told you about how diversity makes you better Boiled down to cognitive things like different information sets different knowledge different heuristics different representation different models right different ways of thinking These things are identity diversity gender race age sexual orientation ethnicities social class neighborhood right religion these sorts of things So the question is how do these things map to those things or do these things map to those things and that's a very complicated map but people used to sort of say well There must be sort of differences these things must map to those things So in my opinion the best evidence that these things map to these things is that Google knows where you search right, so they know what information you grab Google can tell you these things with an error rate of less than 1% So if you go to Google and ask who am I they will say to me You're a 55 year old white guy who grew up poor in rural Michigan and you're Protestant And I'm like that's right They and they know all sorts of other things that they won't tell me that I wish I knew about myself Right so the point is right that Who we are affects the information we gather the knowledge we acquire that sort of stuff and like and that can be predicted with a high degree of accuracy So if you can predict this from this it's incontrovertible that these things influence those things The other last thing and I'll talk very quickly about some data and then some actions is there's indirect effects So this is some work by Sheen Levine where they created groups of people trading and this looks at sort of market efficiency And these were homogeneous groups and these are diverse groups And one of the things that you find is you get more group think and homogeneous groups And you get in diverse groups because of the fact in diverse groups Because they're sort of more questioning and less trust which we see as negative things Actually can sometimes be positive things because people will be saying wait a minute why are other people doing this, right? But in order for them to actually question it in this case it works and actually because it's no market You can bet against other people you've got to have sort of an open inclusive environment Okay, so I've given you theory. Let me talk quickly data and then quickly practices So the data is kind of amazing. So this is in computer science average number of authors for papers gone from one and a half to five Medicine it's gone from one and a half to six Brian Uzi has looked at this in gory detail. He says here's why If you look at team authored papers versus solo authored papers, they're four and a half times as likely to get a hundred citations This by the way is on a data set of 18 million papers. This is every paper ever published So teams are if a hundred citations is like a threshold for an important paper Which is what most universities use you're four and a half times as likely to get one of those if you do it as a team If you unpack this, this is a baseline paper. What causes these team author papers to be better This is the baseline paper. This is if the bonus you get if you cite other important work This is the bonus you get if you cite works that haven't been cited together in the past very often So if I cite psychology and economics together This is kind of a diversity bonus and this bonus is the bonus you get is if you cite these papers and these kind of papers So good papers and diverse papers you get this bonus plus this bonus plus that bonus And again, this is over sort of the 18 million papers Richard Freeman has looked at a smaller set much smaller It's only 11 million papers So he's science and he finds which is interesting the number of email addresses the number of references and the number of past papers All increase your citations and the impact factor the journal it's in and homophily Which is do you work with people who come from the same ethnic group based on last name? They code last name hurts you So Richard says and it's absolutely right the really good way to write a b-plus paper is to work from someone Basically who's at your same university you cite the same stuff over and over you haven't read many papers And that person's from the same ethnic group with you which is what unfortunately most of us do Right the really good papers are working with people other universities different ethnic groups and citing lots of stuff if you go back and unpack in a different way This is Melissa Schilling from NYU. It turns out this is an odds ratio So adding authors each time you add an author actually cost you in 20% 20% in terms of citations But this 15 means you get a 15 times the chance of writing one of these papers with a hundred citations This atypical connect is the same thing that Uzi get it if you connect papers that hadn't sort of been connected in the past So it's really driving this is not more authors.  It's not more authors. It's more ideas more authors actually hurt you Accounting for everything else. It's the more ideas that helps you Lada Adamic has looked at this with patents What you're seeing here is this is the paper data This is the patent data. This is on this axis is sort of how good the patent is or are good the papers This is the top. This is the top 1%. This is the bottom 60%. This is a measure of proximity Which is how similar the authors are who worked on this in terms of their research What you get is that again in the B plus area are all these patents and all these papers that are high proximity which means that there's not a lot of diversity in the things you're citing and The very best ones and the very worst ones are diverse And this is the management challenge and this is something that if you look at the management literature It says the same thing if you look across hundreds and hundreds of study the best teams are diverse The worst teams are diverse the homogeneous teams are the B pluses, right? And this is why people no longer just talk about diversity. They talk about diversity and Inclusion right because it's about practice. So how does the practice work really quickly? Then we'll have some like 10 minutes per question So first thing you want to do you want to think about casting wide net So my friend Sheen Levine says he has this wonderful phrase he uses called the siren call of sameness like he who looks like me is smart Right, you've really got to sort of avoid that Google has worked really hard at this Google gets over three million job applicants a year Which is a lot right and so because they get so many job applicants They've got really good data on this and what they found is that problem-solving abilities sort of the main criteria They use for hiring and they look for sort of different ways that people solve problems People tend to think that Google only hires from like elite the elite institutions Well here's Ivy League Cornell's the highest ranked Ivy League, which is actually part public They're seventh Harvard is 10th and Penn is 19th the public schools like California, Michigan right each Wisconsin, Washington, Illinois, San Jose State, Berkeley They actually have much more from public schools from the elite privates, right? But the the main thing to notice is they're hiring from tons and tons of schools and that's because they want this diversity When you think about it as a manager though once you've got you want to think about Inclusion is starting really on Sunday night because the key message I have in some sense is that diversity helps you on these hard problems so I don't feel like so in My world everybody is a sort of outlook calendar, right? So you sort of look in your week and you say good This is what's ahead of me for the week What you need to do I think is think about okay, what are the decisions or what are the meetings? I have during this week where these are really hard problems, right and on the meetings that are really hard problems You want to think about okay? Who should I have at that meeting? How should I prep those people at me at those meetings? Right because it shouldn't necessarily be it's harder. It's costlier. It takes more time to manage a diverse group So you don't want to have a diverse group if you're sitting around deciding what food you're gonna have at the reception Because that's just gonna eat up a lot of time. No pun intended, right? But what you do want to do is have a diverse group when you're thinking about Really complicated policy decisions, but you also have to have some forethought in terms of thinking about who should be at those heart at those those meetings All the evidence suggests in order to make people feel safe in order to really credit inclusive environment You've got to make agendas available beforehand and actually make sure people had time to read the materials beforehand Especially because people who come from less safe places or don't feel as secure or aren't in the majority group Need to make sure that they're informed before the meeting starts Some organizations and I've got the companies at the bottom that do this just to sort of give them credit for this at a place Like Boeing when they have a meeting of a new meeting of a group the person in charge of the meeting will explain Why each person is at the meeting and if you can't explain why I'm at the meeting then maybe I shouldn't be there All right, it's that because the answer shouldn't be Scott looks like me. So I wanted him at the meeting, right? That's not a good reason a Lot of places like Ford and AB in bed which I've worked with which have a culture of doing things fairly quickly What they'll do is they'll do relatively fast direct reports This is how they sort of like leverage diversity without wasting time They sort of speed it up the direct reports and then people can just basically raise a flag and say here's where I'm having trouble Right, here's where I'm getting stuck and then and you focus everybody's attention on the things that are hard Places like Google the New York Fed does this to some extent. They'll require that everybody speaks at a meeting Because you're not there as an observer now This can get a little bit into sort of culture of fear But the thing is once people sort of accept it right then it can kind of work And then the last point I want to bring out is that in this space of diversity I think this is the hardest thing to work on is that Efforts to be inclusive Can also sometimes not take people as seriously as they should so this is Kim Scott And if you can read her name at the bottom, she's this wonderful book out where she talks about radical canner She tells this amazing story about she was working at Google and she presented to Larry and Sergei And and they said your unit is doing great and they're gonna give her unit all this extra money to do more stuff And she walks out of there and she's like all excited, you know, everything went great in her boss at the time Who was Cheryl Sandberg who obviously everybody knows right that? CEO Facebook and Cheryl chases her down and says You said I'm a lot at the meeting and Kim's like whatever and Cheryl's like no no no you said I'm a lot Kim's like, yeah, okay fine, whatever and then Cheryl's grabs her by the shoulders and says You're gonna think you're facing a glass ceiling, but you're gonna face an um ceiling And I'm gonna send you to an um coach on Monday I know a really good um coach and you're going to the um coach, right? So Kim wrote this book on going to the um coach and she has this great two-by-two chart This is just brilliance right so she says look when you think about managing you have to care about people personally And you also have to challenge people directly if you challenge people, but they don't think you care about them You're in this obnoxious aggression box But the danger with inclusion is if you care if you create this inclusive environment You're caring about people personally, but you're not challenging them you end up in the ruinous empathy box That's where you don't tell the person right that they've got an um ceiling So we've had I was I was working with a company and they had someone who had been this was a minority female This is in the United States and she wasn't coming to work on time right, so her value they're thinking of promoting her and her evaluations were all great except for in one category right come to work on if she wasn't coming to work on time and They almost weren't gonna promote it, but then they decided well Let's ask her about this let's at least give her the benefit of the doubt and it turned out You know she was managing West Coast accounts Working on the East Coast and her boss had said doesn't matter if you show up on time her first boss but then Her other bosses never you know she kept never showing up on time for seven years and the other bosses Nobody ever called her on it, and she's like you're kidding me. I'll get it promoted if I show up on time no problem Right, I mean she's handing billions of dollars in accounts But she had been in the ruinous empathy box Everyone was so happy to have her there that no one was challenging her directly about showing up and then her performance was suffering Right, so what you have to do I think in order to have a really effective inclusive policy Is recognize that these diversity bonuses these benefits and diversity come from challenging people's ideas, right? deeply engaging their ideas But the only way you get the ideas to engage them is by caring about people personally I'd be creating a place where people feel safe and included and then you end up in this sort of what she calls the radical Candor box, which is I think where this good stuff can happen. All right. Thank you very very much And I think we have about 10 minutes Okay, so you have 10 minutes and we have a microphone


Good afternoon, everyone, and welcome to the afternoon session, which will continue a discussion around the theme of secure and inclusive cities. But we have a special treat this afternoon because we're going to talk not just about the problem and providing an analysis of the problem, but to actually talk about solutions, interventions that are proven and that work, and how we can take these interventions to scale, how we can learn from these interventions, what is generalizable from these interventions across context and geography.


racism. There's nothing easy about this topic. However, this year, with the heightened rhetorical pressures of a presidential campaign, we as concerned citizenry have become more aware of the implicit biases and systemic racism in our society. This July, as we were celebrating our nation's 240th birthday, in the space of 72 hours, two black men were shot dead by police officers. Alton Sterling, a 37-year-old father of five, and Philando Castile, a 32-year-old elementary school cafeteria supervisor. An unarmed Alton Sterling was fatally shot on July 3rd in Baton Rouge. Philando Castile was pulled over for a broken taillight in Minnesota. He was carrying a permitted handgun and was attempting to show his license to carry when he was fatally shot on July 6th. These deaths in close succession were a wake-up call. I think that many of us are concerned about these incidents from Trayvon Martin to Michael Brown to Tamir Rice, but seldom we find time to do more than stand at a vigil with a candle in our own silent memorial. I know personally I can no longer be silent. I'm ready to ask the tough questions and listen to the heartfelt answers. I will do my best not to get defensive. I will do my best to stop myself from explaining that I'm not like other white people. Because no matter how enlightened I may think I am on any number of issues, if I do not share my thoughts, compassion, and concern, I'm no better than the racist, sexist elements of our society. So today I start a new venture here at Davis Media Access. I will attempt to ask the tough questions, not the gotcha questions that we've come numb to, the heartbreaking questions that I hope will lead to increased understanding of difficult subjects. Because the more we know about each other, the better we understand our history, the more difficult it is to look at any societal problem as an outsider. As my guest today frequently says, if we continue to deny we have a problem, we will never have the will to solve it. Joining me in this journey and as my guest today, Sandy Holman is the founder and director of the Culture Co-op in United in Unity. Organizations created to assist those working with diversity and equity. Sandy has committed herself to advocacy for children and education. She particularly focuses on the importance of providing cultural relevant and responsive services and materials, as well as increasing cultural competency. Her 30 plus years of experience as an educator, counselor, diversity equity specialist, and author have given her a practitioner's insight. One that goes well beyond theoretical research into the challenges communities and organizations face each day. By practicing what she calls active research, where social theory is matched with practical application, Sandy's innovative approach to a variety of difficult issues has had a tremendous impact on many communities, including our own. Her books, We All Have a Heritage, Grandma Says Our Hair Has Flare and Grandpa Is Everything Black Bad, are expanding the way people view literature for children and being used in schools at all levels of education, including training programs for educators. Sandy is truly inspirational and committed in all of her endeavors. She continues to create quality multi-cultural literature, provide diversity and equity training for youth and adults, and promote literacy around the country. So with Sandy's help, let's get to the heart of the matter. Sandy, welcome. Thank you for having me today, Andrea. And we're here to talk about a difficult subject. We're going to start in the beginning, or at least the beginning of our country, with the historical policies and constructs that were set, some of them 400 years ago, and that lead us to where we are today with the racial tensions that we find becoming ubiquitous, unfortunately, on our feeds on Facebook, Twitter, Instagram, whatever you're following, you're barraged with a video, a soundbite, of yet another wife, yet another mother watching as her African-American son, husband, brother, is shot down. Yeah, absolutely. And so it's not an easy subject. There's not easy solutions, but we have to delve into it. And so let's start from the top. Where would you start when you're educating a group of people on the beginning of our racist history as Americans? So first of all, I want to say that the information I'm going to be sharing is primarily from my life experience, research experts from around the country who've spent their life studying this and lay people who are living this. And I think that's really important to know. And when I say lay people, I mean people of every ethnic cultural background that you can think of. But we are truly as a country and really a world, but as a country, and we'll focus on that, facing a critical time, a critical mass. So what people need to understand is the issues that we're seeing play out in all our major institutions, be it economic, educational, the criminal justice system, healthcare, politics, media, and so forth. Education, again, just to name a few core ones, are things that have been a part of us since the beginning of our country and rather a continuum and not something new.  So now you have this visual evidence to back up what people have been honestly saying for years in most of their lives as I think about my dad and his grandfather and his father before him. So if we're going to really even have a chance to change these dynamics, we have to be stewards of studying history and that is something that a lot of us have not done. I mean, we can all remember being young and not even wanted to go to a history class and actually most of the classes that teach history wouldn't talk about these matters anyway, but we really have to look at the foundations of how this country started.  It was already a country with indigenous peoples who had a history and a way of living and who actually were very helpful to us when we first came over here. But with white colonists coming over and selling in this country, a whole new dynamic started. So our country is primarily based on the institution of slavery. It was based on laws, actual laws, practices, constructs, historical ways of being that made it okay to enslave another human being. That to this day is still having devastating consequences. So in those laws and in those structures, we actually put that into the fabric of our country, how things were going to operate, how someone like you at that time would see someone like me.  I mean, it actually even varied amongst the states. But once you were identified as someone who was a slave, you got certain edicts attracted to you, put on you, meaning you had no rights. You were property of somebody else. You could not get educated. You were much more likely to be punished and even killed for doing what we may say wrong things, like at that time escaping was a wrong thing. You were more likely to essentially be treated worse than the animals that lived in the day. You were treated as an actual product. And what people often don't want to understand just laying down that that foundation, even right there, which is still manifesting in how we see blacks and people of color and people in vulnerable communities or people who are low income, it's still playing out. So it wasn't like something happened 400 years ago. Right. And then it was done. And why can't we get why can't they get past this? Why can't they get I never had a slave. Yes, I never had a slave. And you and you're exactly right. I will say to my white brothers and sisters who will say I didn't do slavery. I never had a slave. What you need to realize, though, is that even though you didn't do those things, you are benefiting from a racialized society that was set back then to give you entrenched and ongoing white affirmative action. So you know, even as we talk about things like affirmative action, white people in our country have an entrenched, ongoing and persistent affirmative action of privilege based on who they are that is steeped back in our beginnings. So in education from the beginning, you know, people talk about the education gap, which I think is disingenuous because it's more of an opportunity gap. In education, we set in structure who could get educated and that was not blacks or people of color or poor. Often beaten for learning to read. Yep. Often beaten are worse for learning to read. We could not, like you said, learn to read, which is foundational for growing your mind and understanding even what's happening to you in the criminal justice system in which, by the way, what we see currently in our policing right now really didn't get its formal starts until the early 1900s. So and I want to talk about that for a second. Use it as an example. So slave patrols in the south and what they called night watches in the north are the precursors to what we see as our police forces today. So back in South Carolina, which is considered one of the first police forces because they had some rules and structure to it and everything back in the 1700s or early 1800s. They had slave patrols, which were set up specifically to control slaves and to recapture slaves who had went away. And so all of those moors and values that were set in with the sole purpose of keeping slavery institutionalized and controlling a group of people who had dark skin, who were from Africa, who were black, started in the slave patrols of the south and what they called the night watches of the north. In England, the night watches were a vestige of England where you would literally be in a place and you didn't go out and patrol. You were there so that if something went on or something happened, people would come to you as the night watch person and say we need your help. Where it was different in the south where they literally went out and patrolled what they called almost like the beat because they were assigned geographical areas, which is why the beat is still used as a word today. So people need to understand the culture that we're seeing in the police departments for example. It's not like we just have one or two bad cops and it's not really the right discussion to have actually. It's more of a culture that was laid that cops are stepping into that was based on a historical practice of subjugating African and black people and then protecting the resources of that time and even still today to some degree of the rich and those who are in power and in leadership. How we structured those slave patrols is how we started our police forces in a lot of those night watches that we had in the north and other parts of the country. Eventually they became like what we did in the south which was patrolling people not looking for white people, not looking necessarily for white women, but looking specifically for black people and other people who were considered non-humans in our society. That culture is still here today and that culture into police departments to kind of give an example of a historical institution was based on violence, fear, murder and no accountability to your actions for subjugating people that we saw less than human. So it is no surprise that today that culture is still into it. I would say that police officers even well-intentioned police officers and I have police officers in my family. I don't think every single one of them is bad. However, all of them are in a risky situation because they're walking into a vulnerable culture that was steep historically in the institution of slavery and controlling people. In the north that was steeped in controlling immigrants and that included Europeans and all that too. Oh yeah, Italians. My family was in that. Absolutely. So we really saw police forces galvanize around immigrants actually in the north who later also wanted to control African Americans too, but when immigrants were when they were concerned about immigrants getting out of control, protesting, asking for their rights, they started these forces to subdue them and to keep them down. Originally that's what was going on more so in the north and that included like I said immigrants from Europe all over whoever the working class was at that time. But eventually those two concepts kind of merged and then when you think about us creating a police department or a force we call police departments to now back then which incorporated a high number of Ku Klux Klan. Most people don't realize our earlier forces had a huge members and numbers of the Ku Klux Klan. Again with the ideal of controlling a group of people, predominantly black people. So it's not shocking to see the relationship of today. Right. So I really would encourage all of us to go back and study this history which is readily available in all kinds of places to really start to look at the origins of this so that we can understand no it's not that African Americans are doing more crime we know this in fact it's just that you're more likely to go to jail or be persecuted or be profiled doing the same kind of crime as a white person would be doing because of the way our system has been set up historically. No it's not that we are dumber or we don't have the capacity to earn money but there's actually been structures set up economically that tell you even where you can live and one of the biggest predictors of your success and you're thriving in an equitable society is where you live and we are extremely a segregated society and we'll we can talk a little bit about why that is too because of things laws and practices like redlining which is and then the denial of African Americans after World War II to get money to buy houses when there was this big housing influx and whites being encouraged to move out and subsidized to move out to what we call the suburbs today and which is why you see you see well which is why and the government actually encouraged this and they actually punished housing people who didn't have in their covenants that they would would not allow blacks to move into these were actual laws and policies. They're still communities in Yolo County believe it or not many many places have done what they can and have gone and and rewritten these bylaws and founding articles but there are still there are still neighborhoods in the county we live in here in Davis California that have in their founding articles statements about no black people no African people no people of Negro descent because those were the words that were used at the time when they were writing these things. Yeah and what people don't understand is since we were set up as an oppressive system and by and large we still are structured as an oppressive system. We there are certain constructs that are acting on our lives and some of us understand a little bit of it but most of us don't even know it because we're not taught about this we don't talk about it so for example some of the traits of oppressive systems are things like a certain group controlling all the wealth conquer and divide with your minority right and I think we have a graphic that we could put up while Sandy's going through this to go through the oppressive systems because it is a lot to take in there it is it's on the screen so go back I didn't mean to interrupt you but let's go ahead so so again historically people need to understand that we were set up with oppressive structures based on white supremacy meaning that whites were better and knew more and a lot of other implications than anybody else and it's it's it's okay to acknowledge that truth was love that was something that was just the way it is yeah and was in many places of our country right so some of those constructs again are accumulating all the wealth and having control over all major institutions using fear violence and intimidation to keep groups suppressed creating divisions against groups that really should be working together to challenge the status quo so that they're in a spin cycle of not ever really being able to get out of their situation right using distracting distraction techniques and chaos things like you know pouring alcohol change into communities poor food quality health care so people are so focused on just their survival they can't even begin to think about and postulate what issues are impacting their life adversely which allows the status quo to blame them for their situation that's the really right egregious and sickening aspect of this and I think we see that even in modern day examples with these shootings that that take place and I mentioned two that happened in July there've been there've been unfortunately several others since then and I didn't even bring the number with me today because it's a little hard to keep up with number one because it's not always reported there's not a universal structure for how to report it there is universal structure for how to report if you have gonorrhea if you go to god if you go to a clinic a doctor anywhere in this country I don't care how far back in the sticks you are and they run a test for gonorrhea the cdc will know about it the center for disease control there is not a universal structure over our policing organizations there's not a universal reporting system for what happens when there's an officer involved shooting and how that is reported each individual jurisdiction is able to set its own reporting requirements and structures so on that note because you're bringing up a very critical point it's voluntary for police departments all over the country to share their data and information with the fbi and the other problem is is it's not really a good idea as far as accountability to have police policing themselves and doing their own interviews and reviews after incidents happen so because we don't report these things by and large I mean what police department wants to report that yeah we have a we we're seeing a a trend here where we're profiling people are we seem to be disproportionately targeting black and brown people most people don't want to call that out about themselves some have been courageous to do that but most of them don't so because we don't require reporting and because there's not an outside accountability force to make sure that report reporting is accurate then we we struggle to see the real picture it's bad enough what we're witnessing with our own eyes as far as what's happening out in our communities but it's much more complicated than that so those are some of the basic things that we'll talk about in solutions but I wanted to give you an example using using the criminal justice system as an example historically of of how this stuff is a continuum of practice policy steeped in oppressive system systematic targeting and white supremacy that has continued on to the day now police forces are perceived as something that helps keep under desirable by a lot of the populations under control and they don't even question why that is right which explains why a lot of white people feel like their police is a good thing and they're to protect them and why often African-Americans or Latinos have a totally different experience so allowing our communities to be segregated again which is steeped in historical practice and by and large we are the most segregated that we've ever been in fact people associate when you say suburbia with white people in our city with black people or brown people right and it makes it easier when you have people put in different places to act out on your oppressive practices because they're all concentrated in one area so for example blacks and browns are more likely to be policed for those minor infractions like a broken tail light or are having people in their community driving around at a heart looking for people who are black and brown who are out of those urban places and have moved into suburban spaces oh my god drive a nice car our driver nice car and so it's very concerning because for those and there are some of those unfortunately who are vested in keeping the status quo and impressive structures and systemic isms in place it makes it really easy to subjugate and oppress groups of people I you know if you live in an urban area you're more likely to live near a toxic dump site you're more likely to live near a liquor store you're more likely not to be able to access good quality food your stores the food quality is going to be poor and the cost of it's going to be higher you're more likely not to have adequate health care to take care of yourself and you're going to be tend to go only when it's an emergency or education for your children and education is deplorable and these are social determinants of health and a person's ability to just survive and thrive because when we think about equity all equity is is a just and inclusive society where everyone has the right to survive I like to say survive and thrive other people would just say to succeed and be successful in their life so all of those communities historically are structured with a compounded institutional impact all those institutions impacting them negatively to keep people from succeeding and then we blame them for it and say if you would work harder which I always thought was funny because the labor and wealth of this country is built on the backs in the institution of slavery and I don't know any harder work that you could come up with and then other groups of people too as they came into the country but the wealth of this country and the world people need to understand this is built on the back of slaves and the Rockefellers and all those folks all those old monies are tied back into these systems of operation no you're you're right on and we're we're quickly coming up to the end of our first episode here but I want to end with a quote if I may and and there are people who may watch this who may find it hard to believe that I'm quoting this person the quote is a great nation does not hide its history it faces its flaws and corrects them this museum tells the truth that a country founded on the promise of liberty held millions in chains that was George W. Bush at the opening of the National Museum of African American History and Culture just September 24th of this year 2016 and I was impressed that at least his speech writers wrote a nice quote and I think he really did believe it Laura serves on the board and this shows that this can be a bipartisan issue it does not have to be a partisan issue we the more people we include in our society the stronger our society will be so I will say goodbye for now this is the heart of the matter and tune in for the next episode