Please join in a moment of centering silence so we can be fully present with each other this morning and now Let's get musically present by turning to the words for our in-gathering hymn, which you'll find inside your order of service Morning everybody and a happy. Hello on this Sun-filled Sunday at First Unitarian Society where independent thinkers gather in a safe nurturing environment to explore issues of social Spiritual and ethical significance as we try to make a difference in this world I'm Steve Goldberg a proud, but slightly overweight member of this congregation and it is my particular pleasure to extend the special welcomes to any Guests Visitors or newcomers if this is your first time at First Unitarian Society I think you'll find that this is indeed a special place and if you would like to learn more about our special buildings We conduct guided tours after every service Just meet over here by the windows after the service and we will take care of you speaking of taking care of each other This is a perfect time to silence those pesky electronic devices that might get in your way this morning And this announcement goes for those of you watching or listening at home. Thank you for taking care of that And if you are accompanied this morning by a youngster, and you think that youngster might prefer to See and hear the service from a more private space. We offer a couple options One is our child Haven located in the back corner of the auditorium And the other is a group of comfortable seating just outside the main doorway in the commons from which you and your Youngster can see and hear the service As is the case every weekend our service is brought to us by a wonderful group of people we call these people Volunteers we are grateful for their service And I'm going to announce their names. They deserve our thanks and appreciation. So please Give them a handshake or a hug a high five when you have a chance to thank them later We thank mark Schultz who's handling the sound system today. Thank you mark and smiley who is our lay minister and who also Provided the flowers that you see behind me in memory of patricia leonardy Our greeter upstairs kyle Kern. Thank you kyle Thank you to ellison brooks and dug hill for being our ushers And also the chip quad a and gene hills for handling the Hospitality and the coffee a little bit later on today and richard miller for serving as our tour guide just a couple announcements One is that our elves need our help. Did you know that our elves need our help? Who would think elves needed help because they're the ones providing help in this case They need help with our f us family-to-family holiday giving program Which provides gifts and food cards to more than 130 families in need Throughout dame county to help make their holidays a little bit brighter so next weekend you'll have a chance to sign up and match with a wish that will be on the list indicating the kinds of tasks and Support that these families might need you can spare just a couple hours in the next month or so to help those elves We would appreciate it take a look in the red floors bulletin and also come with your eyes open next week To sign up for a volunteer opportunity The second announcement also includes reference to the holidays. This holiday is Thanksgiving In a few weeks our potluck dinner on Thanksgiving Thursday right here At first Unitarian Society starting at 230 and that's when dinner will be served You'll have an opportunity to bring your favorite Thanksgiving dish to share with others Small donation to help cover some of the costs and a lot of fun If you'd like to get involved in helping with arrangements or get more details check with Donna Kanglossi She's arranging this and you can also see information about the Thanksgiving dinner At one of the information tables in the Commons during the break Two more brief announcements One is very important to those of you who might have made the mistake of parking on University Bay Drive this morning in the 900 block unless you want to contribute $40 to the revenue of the village of Sherwood Hills You might want to move your car Something that FUS is trying to negotiate with the village, but meanwhile the fine stands $40 for parking at University Bay Drive today The other announcement is a little more pleasant and that is it's only a hundred and fifty three days until the next cabaret so Those are the announcements for the morning Please sit back lean forward and enjoy today's service. I know it will touch your heart Stir your spirit and trigger one or two new thoughts. We're very glad you're here We continue with these words from an unknown author and titled a caregiver's morning prayer As I begin this new day, let me have the courage and the strength to face what lies ahead Let me practice tolerance and patience. Let me reveal empathy and love. Let me offer sustenance and hope And let me have the wisdom to know when I need to step forward and when I need to step back I Ask of this so that when I come to the end of the day I might rest having known that my time and my energy have been wanted and Have been needed and yes, I am asking all of this for my own sake And I'm also asking for the sake of another I Invite you to rise in body or in spirit for the lighting of our chalice and Our words of affirmation are responsive in nature this morning So if you will join your voices in reading the bolded sections We would come to this house in an attitude of openness Not knowing quite what will happen yet daring to receive a new idea a new experience Sustenance for our minds and hearts we would come in an attitude of humility Knowing how much we need one another and how alone we can be in the world how vulnerable if we face life solely by ourselves We would come in the spirit of joy seeking Reaffirmation and renewal of life of love and of hope And now I invite you to turn to your neighbor on this fine November day exchange with Emma warm greeting And as you settle in I invite you to join me in the spirit of meditation from many places and many conditions of the spirit we come We come here seeking a center for our lives and a sense of greater wholeness We come from dry places where words and knowledge seem broken into brittle fragments that do not cohere and from overfilled places Or information abounds But there's no real depth of understanding We come from hard places where feelings are dulled and From lonely hollow places where meaning seems thin Here in this caring and supportive community At this time of quiet reflection We come to be emptied of all the clatter in the confusion of the information that we once thought was all sufficient emptied and Then filled with the spirit that flows in and among us And that can be for us a reliable source of solace and of insight in this time of quiet Let us center our spirits Ground our being That we might find that power that already lies latent within us power for love power for creativity power for hope and power for transformation Given this reprieve from daily pressures may we learn once again to appreciate that every inch of space is a miracle Every instant a wonder an opening for new opportunity at this very moment then May our hearts be open to compassion Our minds be open to wisdom and our spirits open to grace Let us continue on in a moment or two more of silence Blessed be that I'm in Now I invite you to rise in body or in spirit as we sing hymn number 326 And at this time if there are children who are still in the auditorium They can go for the classes or they can remain here if you like first of our readings comes from James Miller and Susan Kutchall Their slender volume the art of being a healing presence being present is Not a complicated matter All you do is wake up You open your eyes look around come to your ordinary senses and once you are awake You simply stay that way as long as you can You wake up where you are taking in all that is around you Sometimes you survey the entire panorama But other times you can focus on the details Paying attention to lines shapes colors textures following your natural curiosity. You just open up to as much as possible And you wake up not just to your surroundings, but to what is going on in those surroundings Is their movement? Where? What's sort of a movement? Is there stillness? What's the quality of that stillness? Are there sounds how many sounds? What kind? What do your senses gather and convey to you? Being really present includes being awake to yourself. What's happening within you while all these things are happening around you?  Another person or persons they may be present so you you awaken to them as well Who are they? What are they like? What are they doing? What's going on between the two of you? When you are completely present you stay awake to each unfolding moment You give yourself fully to the present instant letting go of what was and choosing not to anticipate what will be You don't do presence. You are presence It's as simple as that and yet It's not necessarily easy for in today's world. Most of us are busy too busy We overfill our days with activities until these activities overfill us The multitasking attitude that we fall into in everyday moments and workaday situations Can interfere with simple presence We unthinkingly presume that these moments are so ordinary that we just take them for granted We see a tree. We see the sky a particular person so many times day after day That we forget We forget how truly amazing a tree is how truly spectacular the sky can be the unique miracle of another human being We lose sight of the Extraordinary as it lies hidden in what we call the ordinary And once we allow that to happen It becomes difficult to reawaken To consciously discover the wonderment and the vitality that's all around us And yet the more we practice being completely present the more natural it becomes The more we wake up to what is around us and within us The more we are inclined To stay that way the second reading from Mary Oliver's collection Dog songs which are prose poems about the various dogs that have shared her life with her and her partner and so in this first one there is a Imaginary conversation taking place between Mary and her dog Ricky and other characters who appear are a dog that she once had now deceased by the name of Percy and Her partner Mary's partner Ann Got that straight So Mary Oliver begins Ricky Can you explain how it is that Ann and I can talk with you as we did with Percy too? And we all understand each other Isn't that a kind of a miracle? It's no miracle says Ricky. It's actually simple When you or Ann talk I listen when I talk you listen as you did with Percy Well, of course we listen. No, no, I mean really listen Here's a story and you don't have to visit too many houses to find it One person is talking and the other isn't really listening someone can look like they are But they're actually thinking about something that they want to say next Or their minds are just wandering Or they're looking at these little boxes that people hold in their hands these days And so people get discouraged and After a while they quit trying And the very quiet people you may have noticed They're often the sad people Ricky you've really thought about this, haven't you so we can talk together because We really listen and that's because yes because We care it is a pleasure to welcome Wes and Jess back for our Sunday services and We're getting used to this fine music. You're spoiling us folks One of the busiest Intersections on the west side of Madison is at University Avenue and Midvale Boulevard During the morning and the evening commute period traffic can back up for a couple of blocks in either direction And the stop lights on University Avenue feature arrows for those Intending to make a left-hand turn either on to Midvale or for eastbound traffic into the Village of Shorewood Hills The latter turn arrow unfortunately is a very short duration Allowing only three or four cars at most to execute that turn before the light changes Because I live in the Village of Shorewood Hills. I am a frequent visitor to that intersection On most occasions, I'm able to beat the light But occasionally there will be a car in front of me whose driver is simply not paying attention And I suspect that they are busy checking their smartphone or sending a text message What they are not doing is watching for the green turn arrow to illuminate Thus leaving both them and those stranded behind them frustrated for yet another cycle of the lights Such behavior is of course highly inconsiderate and yet it is becoming more and more common Despite repeated admonitions from transportation officials to leave the cell phone alone while you're operating a motor vehicle Now I don't believe that people intend to cause inconvenience to others when they are inattentive They just don't appreciate that attention is at a minimum A responsibility It's a courtesy that we owe to others and at other times it can be a precious gift That we all have the ability and the opportunity to bestow And I understand of course that this can be a challenge in a society that competes for our attention every waking hour Writing last spring in the New York Times Matthew Crawford shared his own discomfort with today's Capitalists who as he puts it seem determined to dig up and monetize every bit of private headspace by appropriating our collective attention Crawford describes his experience in major metropolitan airports where fields of view that haven't been claimed for commerce are getting Fewer and fewer and fewer one cannot ride the moving walkway at O'Hare without being distracted by advertisements for Lincoln financial group that had been applied to the handrail televisions tuned to CNN are ubiquitous almost impossible to avoid and thus there is less and less space left for travelers to attend to and possibly conceivably to engage with one another What is lost Crawford says is the public space that is required for Sociability the kind that depends on people not being self-enclosed an Airport lounge. He continues once felt rich with possibilities for these spontaneous encounters Even if one did not converse our attention was free to a light on one another and to linger there We encountered one another in person if only in silence And so the author goes on to suggest that we may need to begin treating our attention in the same way as we do our air our water and our soil It is a resource to be protected from the predations of those who would expropriate it for the sole purpose of making a buck off of us We have the right Not to be addressed to be shielded not from those who would address us face to face But from those who never show their face Yes, attention is a precious and limited resource and we do well to shepherd it to direct it toward those activities objects and individuals that are most worthy and deserving It is after all the rare person who does not crave attention I have the feeling that in today's world where it's increasingly hard to get it for free We often resort to paying for it. Do we suffer from a deficit of attention from our friends and our families? Well, there's always counselors Fitness trainers personal coaches who for a fee will step in and fill the void now to be sure these people also have expertise to offer But also a reliably human touch that is in itself of considerable merit Francis Weller an authority on traditional and indigenous methods for handling grief and loss argues that for thousands of years human beings were Emotionally and spiritually nourished by the members of a closely connected community We were communal people and people would gather around the fire at night They would listen to the elders tell their stories They would sing together share meals in the evening share their dreams in the morning And they would support each other in times of loss and bereavement And according to Weller those communities satisfied what he calls primary human needs But today because those needs often go unmet People turn to other devices to satisfy them But they satisfy them indirectly through what he calls secondary Satisfactions status well privilege or more problematically addictive behaviors We need the attention of a group Weller insists especially when we are hurting Especially when we are under stress But we also need each of us to grant attention as a participant in a reciprocal self-sustaining process And this is I would argue what many of us look for when we enter the doors of a faith community and Attentiveness that we try to maintain and share with one another in this privileged commercial-free environment Now truthfully we don't always succeed in our efforts Because many of us have simply gotten out of the habit Or we unconsciously repeat those inattentive patterns of behavior that we have acquired out there in the larger world But here at least we recognize the importance of attention and we do our best to practice it What we are speaking here about here is an attitude of the heart Pope Francis wrote in his latest in sickly oladito say One which approaches life with serene attentiveness Which is capable of being fully present to someone without thinking about what comes next Which accepts each moment as a gift to be lived to the full And for inspiration and instruction in this discipline I can do no better than to observe our dog Sasha For her as for Mary Oliver's dogs Ricky and Percy Attentiveness seems to be a natural attribute one that dogs developed over thousands of years of partnership with human beings At least some of the more recent literature on dogs Speculates that it was this very ability that endeared them initially to human beings Dogs study us. They look for cues in our eyes our hands our postures in our tone of voice In order to give us what we want and for us to give them what they need and Nothing seems to harm a dog emotionally more than to be neglected to be ignored That's when you notice that pitiful hang dog expression on their faces Now I don't want to press the comparison of human and canine attention too far because we face challenges in this respect that dogs Simply do not as creatures of the modern age. We are fated to spend a good bit of our time Up here preoccupied with thoughts and daydreams and storylines That neither dogs nor earlier generations of humans had to cope with We don't have to have our cell phones close at hand to suddenly find that our attention has wavered That we've tuned out a conversation or that we've missed the green turn light According to Kay Lindahl the author of a book entitled the Sacred Art of Listening While we do on average spend about 45 percent of our waking hours listening The average person is also distracted or inattentive for three quarters of that process 22 seconds 22 seconds That's about how long Without making a concerted effort that most of us can hold our attention steady Being attentive takes practice As one manager of a large business told Lindahl I've always prepared myself to be able to speak I've never prepared myself to be able to listen But the benefits of paying attention are considerable Whether one is the object of that attention or the person doing the attending for instance in studies that have been done of Effective salesmanship those men and women enjoy notable success Who take the time and are able to listen carefully to customers as they explain their needs Rather than push their product on an unfamiliar prospect the smart salesperson steps back Asks questions draws the person out and therefore Convinces that person that this the salesman this is someone who wants to understand me and to satisfy me Now of course that can be overdone Attentiveness must always be accompanied by a certain sensitivity We want our presence and our interest to be affirmed, but not scrutinized When interest is too intense Makes the other person uncomfortable as if they were being examined under a microscope So a healthy interaction always involves respect for those personal boundaries Alan Wofelt is a noted grief counselor and he underscores this point when he notes that in Companioning those who have suffered a major loss or someone who is actively dying We need to let that person call the shots Respect their need to be alone part of the time Allow them to express their thoughts and feelings, but don't force the issue if they seem to be reticent don't try to raise the other person's spirits by doling out cliches But just listen and then provide assurances of your own care and your own deep concern for their well-being Wolfelt writes give your person your friend permission to express feelings without fear of criticism Learn from that friend don't instruct don't set expectations about he or how he or she should respond And thus think of yourself as someone who walks with Not behind not in front of that grieving or that dying person walking side-by-side It's hard to underestimate the advantages that the act of consciously attending confers a Number of years ago the psychologist and family therapist John Gottman decided to observe newlyweds in a clinical setting what he called his love lab He wanted to see Gottman wanted to see if he could isolate certain behaviors Negative or positive that would help him to understand why it is that some marriages succeed and why some of them fail the most meaningful gesture between couples that he discovered was turn toward In other words when one party to that relationship wants to share something The other party quickly drops whatever it was he or she was doing and redirects their attention turning toward their partner Couples who were good at doing this Gottman labeled masters But those who evinced irritation or resentment over having been interrupted He called those people or those couples disasters Couples who were still together after six years of marriage were very adept at turning toward each other 87% of the time To illustrate the difference he offers the example of a spouse who comes home bursting with excitement Anxious to share some great news promotion at work Entrance into a prestigious graduate program the partner Might respond half-heartedly Glancing at the watch Shutting down the conversation with it. That's nice dear By contrast the masters practiced what Gottman calls active constructive responding If the partner reacted to the good news in this way active Constructive responding then he or she would stop what they were doing immediately turn toward the partner and say that's great Congratulations, when did you find out? Did you get a call and? According to Gottman's findings the only salient difference between couples who were still together and those who broke up After six years was this active constructive? Responding the parties paid attention to each other and thus showed that they prioritized the relationship between Couples in families and among friends the dividends of paying attention are typically shared Both sides benefit the person being attended to feels validated and Despite the effort that might be involved the person doing the attending experiences a deeper than usual sense of well-being as The renowned Harvard psychiatrist George Valiant found after tracking a large sample of Harvard graduates for better than three decades Sustainable happiness for these men and women Correlated much more closely with meaningful connection to others than with any measure of personal success Sometimes the rewards can be quite tangible Rachel Naomi Raymond is a pioneer in the field of integrative medicine She tells the story of a colleague who was providing health care on an Arizona Indian reservation Well, one day a native woman well in her 90s was brought into the clinic by her daughter The physician Raymond Raymond Rachel's friend was named Elizabeth and Elizabeth took on this woman She did everything that she could from a medical standpoint. She addressed her urinary tract infection She managed her diabetes. She reversed her incipient heart failure. She also ordered any number of lab tests Mobilized social service agencies to help with the daughter and even located an agency that would provide some extra financial assistance for the family Well, the old woman died many months later at about the age of 96 and after that the physician Elizabeth received a call from a researcher at the University of Arizona and This researcher was planning on writing a book on traditional American Indian medicine and He had become aware of this woman a very important practitioner who had recently passed away This lady had received the lineage from her forebears was one of a handful in the Southwest Who had kept alive these ancient ways of healing and the researcher had first contacted this woman's family But the family had referred him to Elizabeth because well, she too was a healer The two women knew each other well the daughter said and so she presumed that Elizabeth and her mother had discussed these matters during their visits Not so much Despite her solicitous and impeccable professional approach to the woman's medical issues Elizabeth had never inquired into her background And because Elizabeth had never asked the old woman had never shared Native American women are typically kind of reticent about that. I have never forgotten this Elizabeth said ruefully I think of her sitting there all of these months watching me shuffling my papers tracking my lab data And knowing what she knew just sitting there What was going through her mind? I've been so busy with my numbers and with my tests what I would give now for just an hour with her to ask her all Of my unanswered questions to get her perspective on suffering and loss and illness and death or maybe maybe Just to ask her blessing Raymond says with respect to this instance you have to be present to win And by winning of course, she is referring not to any particular opportunity But winning at life in general because attention is in the end a form of love as well as of respect and In fact attention can serve as a fulcrum for love can even generate love In her book all joy and no fun at the paradox of modern parenthood Jennifer senior disputes the classical notion that we care for our children because we love them Parental love. She says it's about time. It's about engagement. It's not just about biology We love our children because we care for them. Both assertions are probably true And taking care of kids can be as any parent knows a thankless task at times But sometimes it is thankless because a part of us wants to be somewhere else Part of us wants to be doing something else And so we refuse to give our full attention to the child that's standing there before us And if at some point in time We do discover that love and attention go together We will also understand that the one and the other are reciprocal and they are contingent upon each other May we always be cognizant of that association? as parents with our partners with our pets With one another And when we're waiting at a green turn signal light on University Avenue blessed be and amen and Now it is time for the giving and the receiving of the offering and as you can see it'll be shared 5050 with dry hooch Which is an organization that assists veterans with their substance abuse problems And there is a table out in the comments where you can get more information. Please be generous We gather each week as a community of memory and of hope and to this time and this place We bring our whole and sometimes our broken selves We carry with us the joys and sorrows of the recent past and seek here a place where they might be received and celebrated and shared It is with deep sadness that we announced the passing of Patricia Leonardo Patricia passed away on Friday afternoon due to complications of multiple myloma We will miss her bright spirit her creativity her deep and immense generosity and love We are holding her partner Linda and the many leonardies in our hearts and the service Has been planned for December 19th at 1 p.m. Across the parking lot in the landmark auditorium. Our hearts are also With this and Rupp Nitschke as they recover from a recent automobile accident This had very successful surgery on Thursday and is being released this afternoon We send our prayers and our strength for healing to both of our friends And then on a more joyful note. We extend our congratulations to last year's ministerial intern here at FUS Sasha Ostrom Sasha was ordained at the Unitary and Fellowship of Raleigh, North Carolina yesterday morning at 11 o'clock and I had the privilege of preaching the ordination sermon for her and My only regret is is that the plane was two and a half hours late into Madison, so I'm a little jet lag this morning But Sasha was absolutely thrilled our shawl ministry bestowed upon her a lovely shawl with a chalice on it in honor of her ordination and We again send to her our very best wishes and congratulations And so in addition to those mentioned we would acknowledge any Unexpressed joys or sorrows that remain among us as a community We hold those with equal concern in our hearts let us now sit silently for just a moment or two in the spirit of empathy and of hope And so by virtue of our brief time together today may our burdens be lightened and our joys expand These rise in body or in spirit for our closing hymn in the teal hymnals number one zero one two in peace deeply regard each other Truly listen to one another Speak what each of you must speak with candor and compassion and be ready at any moment to disarm your heart and Always live as if the reign of love had begun May it be so please be seated for the post loop


Okay, so welcome to this fourth and yet in this HMI Data AI and Society series of seminars. We're delighted to have with us today, I'm Clinton Castro, I'm from Florida International University. Let's start by acknowledging the traditional custodians of country throughout Australia, and the continuing connection to culture, community, land, sea and sky, and paying my respects to Elders past, present and future. I'm recording this in Michalago, which is Narago lands. Sorry, my respects to the Narago people in particular. So this seminar by Clinton Clinton's an assistant professor of philosophy at Florida International University. We went after a PhD at the University of Wisconsin, Madison with where we're supervised by Mike Titlebaum, a close friend of the ANU. Clinton is going to be talking about just machines. And as usual, we're going to keep the introduction short and sharp. So Clinton, if you'd like to share your screen with your slides, we'll get started. Sure. Thank you very much for the introduction and for the invitation. I'm very excited to be with y'all today. And this will be my first, this will be my first go at a zoom presentation. So hopefully it all goes smoothly. Fingers are crossed. Okay, so I guess I could just jump into my next slide. I may just as a heads up, I may talk a little bit quickly because I do want to get through everything on time and I have a little bit to get quite a bit to get through. So I hope I don't go too quickly. I'll try to be articulate, but also I will be trying to stay on schedule. So it might go a little quickly. Hopefully not too quickly. Okay. So my broad topic, my really broad topic today, most broadly speaking is fairness as it applies to predictive scores, which I understand as mathematical expressions used to describe or predict a group or individuals attributes or future behavior. Just some quick examples. I think a lot of us might be familiar with this, but here's some examples of how they've been used. Famously criminal justice, for example, whether to grant bail, predictive policing and figuring out where to send officers or who to target. Education is an example we'll talk about a little bit more today. Namely, one example being determining effectiveness of teachers. And then finally, here's a really big set of examples. This isn't the only other context, but it's a big one, the market. So predictive scores have been used to determine consumers age, ethnicity, gender, frequency of purchasing general apparel, television usage, job security, allegiance to buying main brand or generic drugs, likelihood of moving to another merchant, likelihood of smoking, and maybe one of the most famous examples, likelihood of being pregnant. But there are hundreds of hundreds of more scores, maybe even thousands in the context of the market. Now, so I'll be talking about those today and fairness as it applies to them. More narrowly, this talk actually has two goals. I usually try to stick to one, but I'm shooting for two today. One is I really, one motivation behind this paper is I have a lot, and I'm sure we all do have some really smart friends, maybe friends that we think are smarter than ourselves that have not jumped in on this topic yet. And I take it as not only do I have friends that haven't jumped in on this yet, but I think very highly of, but also lots of philosophers haven't jumped into this topic yet, or these issues. And so one motivation of this paper is actually to provide something of overview of machine learning and fairness in machine learning, so as to invite other philosophers and other academics into this, this topic. I'm hoping that what I have to say isn't so basic to this redundant for a lot of the people here today. I am hoping I have something for everyone. But if some of this is redundant, especially given our interdisciplinary crowd, I would love to hear feedback on whether at all this is a good way of trying to summarize sort of a high level overview of the, I think, kind of key concepts of machine learning and fairness in machine learning. And I should mention I've switched over to use the word machine learning. The reason that's appearing here is because that's a common tool for developing predictive scores. And then my second goal is to make a diagnosis. And this actually is served very well by giving an overview of things first. And this will kind of come up later in the talk.  And then what I want to say is why have we been able to find counter examples to all these proposals. And I want to kind of go up beyond the local counter examples and kind of give up general diagnosis like what's what seems to be like the larger problem here. And so I'll be offering kind of a diagnosis of why we have not landed on a measure yet of fairness in predictive scoring for machine learning. So that's kind of like the basic, those are my goals. Here's the plan. I'll work through some machine learning and bias machine learning and how bias can kind of work its way into machine learning. I'll work through as I said some fairness measures and their discontents. I'll offer a diagnosis of kind of why these haven't been working. And then some really broad general lessons maybe even so broad that they're like more like invitations for kind of further comment and discussion. So some lessons from what I think is the correct diagnosis of the situation. Okay, so here's an example I want to talk to you really quickly. We'll talk through sort of three examples that'll be touched stones throughout the paper. Here's the only I think non-controversial one. This is sort of my model for machine learning. So I call the example Frenchies and Boston Terriers to anyone who doesn't know Frenchies and Boston Terriers are breeds of dogs that looks at least somewhat similar. They're both short have short hair have some noses and they're the kind of the reason I could imagine people confusing. And I can and this is somewhat contrived example of course but we can imagine that you have trouble telling Frenchies and Boston Terriers apart and you want to get better at this and so you develop you want to develop a system for telling the difference. One way you might do this is you might go to the local dog park with an expert make some notes on the dogs they classify as Boston Terriers and Frenchies look for some patterns in your notes and use those patterns as a guide for future sorting. Like I said a little contrived but hopefully like it's somewhat intuitive model of how you might solve this problem that you have. Now, that isn't all that different from what a computer does when it uses machine learning to mind data sets to find patterns. So we can, if you can kind of get how that that that process works as nice to describe, you can imagine doing that in a computer assisted way, filling out some spreadsheets maybe looking for some patterns and and you're off to the races.  Okay, the method I just talked through is something that we can call classification. And the next thing that I want to talk about is sort of, I'll do two things quickly in two steps. I want to break down the process I just described into some steps, three steps, and then I want to talk about how each step bias can kind of slip in to the process inadvertently. And I'll raise the question, well, how do we detect bias, there's ethical concerns, I gave all these examples machine is learning is used in all sorts of really important aspects of people's lives. And that motivates the kind of discussion of these fairness measures. So I'll look at classification really quickly in a bit more detail. First, so I'm slowing things down just a tad bit. The first step or one of the first steps in the process I walked through is choosing what to take notes on some very quick terminology, when you choose what to take notes on your engagement what will be what's called feature selection. And when you choose which labels to use for the thing that you care about Boston terrier Frenchy, you're defining your target variables the thing that you care to learn about.  When you do this you're engaged in data collection, and you'll use this data to train yourself so this will be your training data. And here's a quick example, I think people can see my cursor, people see my cursor. Good. This is like maybe an example of what your notes might look like. Moving on. Then you look for some patterns and I think if we kind of put this on a graph we could see it maybe doesn't jump off the page here but here you can see that there's kind of a pattern going right where I'm using dots for Boston terriers and X's for Frenchies. And if we wanted to be a little bit more circumspect we might say, we'll draw two lines. One vertical line to separate as many X's from dots as we can and one horizontal line to do the same. And now we might have a rule. Supposing you have access to this data, you might say if if a dog's attributes put it in what I've labeled as quadrant D, you can go ahead and call it a Frenchy. And if that rule works for you, you're done. Kind of depends on how accurate you want to be, but maybe if you were no better than guessing and you just wanted to improve a little bit, maybe for the sake of say targeted advertising. Then maybe maybe you're done here. I guess one thing I'll say really quickly while this slide is close is obviously we could do a lot better. We could have sort of gone gone a little more fancy and drawn like maybe a line here kind of or maybe we could have tracked more more data points. That might have been more work than we cared to do at the moment. Like I said, maybe this is good enough. But we could get better by making things a little bit more complicated and as simple as my example is that's when you kind of see the need for computers, right. So we could spot those those those patterns that I just put up there. But if we had more dimensions, more data points, more complex relationships, maybe not. So the basic thought there is that classification can get difficult and complicated very quickly. This is why machine learning has made its way into many aspects of our lives. Especially that that that's part of why another reason is that computers are cheap data is cheap story data is cheap now. And so like this is why we see like I said in my example with the market that we see kind of consumer scores everywhere. Okay, so now hopefully we have kind of a grip on how that process works. Now let me just kind of talk about three ways in which bias can kind of slip in that each of the three steps identified. So at step one, by a choice is what to take notes on. So here's another example we talked to Boston terriers and the French ease. Here's another example will return to a couple times I'm calling it hiring teachers. And I'll just read the case really quickly. You're hiring teachers and you want to know which ones will be effective. You decide to look at your past hires to see which teachers have added the most value to their students education measured using test scores. You find that teachers that went to certain colleges added more value so you give more teachers that went to those schools interviews. And that might look fine, but we can suppose, and this is going to be true in a lot of cases that the following correlations hold minority and non minority teachers tend to go to different colleges. Minority teachers gravitate toward schools with higher portions of minority students, the higher the portion of minority students the poor the funding for the school poor funding causes lower test scores. And now just by kind of from the outset before we even really started we might have geared things up so that we're going to end up with what's going to be an unfair scoring system. I'm going to look at step two and I'll use that to introduce a second case. So at step two we can introduce bias by how we're collecting our data. So here's another example. This one we're going to really talk about in some detail. And I'll call it pre trial release. You're deciding who to release while awaiting trial. You only want to release people who will not re offend while awaiting trial. You have data about past defendants that suggests that the number of past arrests a defendant has is positively correlated with future arrests. You use this to construct a rule that recommends only releasing those with you previous arrests. And again, somewhat predictively, we can end up with a problem. If we suppose certain correlations hold, which they do in a lot of cases. For example, if we suppose that there are many crimes that are committed by white and black people at similar rates, but that the black people get arrested for disproportionately often. So this is like an example would be drug drug crimes in the United States, I think is a I think pretty uncontested example of what I'm talking about there. If, if, if we're using the pre trial release system, the way I describe, and we have this background fact, we will again have a system that just by tracking the data is not going to look fair and that's just because the data, the data collection has not been a good process. Okay. So lastly, this will be my last example will actually turn to an old return to an old example, but this is like the last way in which might have biases seep into the process. And that's by just choices of how to balance false positive and false negatives, or basically like what our standards are for accuracy. And so I have this familiar slide about the Frenchies and the Boston terriers. And we can see here if we use that rule, anything that's in quadrant D just call it a Frenchie otherwise maybe call it a Boston terrier is going to have the effect of falsely accusing. At least some Frenchies of being Boston terriers. It will also never accuse a Boston terrier of being a Frenchie. And that might not be a big deal. In this case, it depends on maybe what we're using the score for. But we can imagine that it would be a huge deal if if this is being used for hiring or detention or whatever else. One thing I didn't say previously but now that we have this again that is going to be important and maybe a good transition to the next thing I'll talk about is we could imagine that if we had two graphs.  Maybe there would be a clean line separating all Frenchies and Boston terriers and by introducing that that third data point, maybe we could have no inaccuracy and kind of not have this problem. Which this is going to this is like maybe now that we have the slide in front of us it might be helpful to see that that maybe by doing something as simple as just having two graphs one for males and females, maybe the kind of ambiguous cases in the middle would just go away, and we wouldn't have any inaccuracies. So hopefully that's all clear. Hopefully it's all not too simple or familiar to everybody here. I had to worry about that. But I guess we'll have to just kind of soldier on. Okay, so now I'm going to talk about so hopefully now you at least if no one was worried about this or maybe you're now worried. Oh, these things are being used in for all sorts of purposes we should care that they they're fair. In non simple examples, how might we tell. Well, I'm going to work through three proposals. The first one is less quantitative. And it, and it goes like this. So one, one claim is what here's what here's something important about fairness and machine learning. So the principle anti classification, don't take inputs about as to whether someone belongs to a protected class. So don't ask for people's race ethnicity gender don't use those in your scores. Now obviously, and now we're on to the counter examples, and I'll do this for each one that we'll talk about I'll say why I think it's not sufficient and why it's not necessary. So it doesn't seem to me that I think, well, maybe everyone will agree that satisfying anti classification is sufficient for being fair. Why, well, we can already see this if you bought either the hiring teacher or pre trial release case if either of those aren't fair. And the way that I set them up was that they don't ask for for, we can imagine at least that they don't ask for race or gender or anything else. Then we can see that satisfying anti classification isn't going to be enough for being fair. But maybe we can say okay well maybe it's maybe it's necessary maybe it's not sufficient but it is necessary. And here's why I don't think it'll be necessary. But at least some contexts women tend to reoffend less often than men. Taking gender as an input then can prevent a score from problematically over estimating root of is recidivism among female defendants so you kind of can kind of recall the the the the graph example, and how we could maybe improve our accuracy by by tracking gender for the dogs. Here, we could say, we can kind of apply the intuition here and see that at least in some cases doing the same thing with people might allow us to avoid being an inaccurate ways that might seem unfair. Because if we blind ourselves to that data, we might over estimate recidivism among women which would feel unfair to them, presumably. Before I move on to the next slide I should say really quickly, I haven't marbled in my references into these slides, I have a reference site at the end. And a lot of this discussion comes from several computer science papers with with similar authors there's a couple. There's like a group of people who read a bunch of papers together. I'll just kind of introduce those at the end and call back to them. I think where these definitions are coming from. Okay, so moving on to classification parody, or moving on to the next two measures. And I'll be presenting the next to the last two in tandem. Let me backtrack a second, I'm going to present the next to in tandem, because they seem very similar. And also these two have been more, I think at the heart of more controversies the last one. The first of the two is something called classification parody. Classification parody tells us to make classification errors equal across groups defined by protective attributes. And basically what that means is it shouldn't turn out that in my scoring. I tend to be less accurate about men and women, or women and men, or black defendants, as opposed to white defendants, I should have kind of error rates that are equal across those groups should sound intuitive enough. I think we can hopefully see why that sounds like attractive. Calibration in contrast, which can sound very similar to parody, but we'll tease them apart. That's kind of why I'm starting with them side by side as I'll tease them apart first. Calibration in contrast tells us to make outcomes, for example, re arrest among people we identified as high risk of reoffending, probabilistically independent of protected protected attributes, given one's predictive score. In simpler terms, it's telling us that, let's suppose I judge somebody as 50% accurate. If I'm good at what I'm doing, then there's about a 50% chance, sorry, if I judge someone is about 50% likely to commit a crime, and I'm doing my job well, there should be about a 50% chance that they're going to go on to commit a crime. Knowing how I've kind of performed in the past, you shouldn't upon learning someone's score and their gender deviate from that 50%. What calibration asks us is basically to, it asks us to how our scores mean the same things for each person across protected attributes. Now, I think that classification parody and calibration can sound very similar, but they're not and I have a graphic that I think should really help see why this is the case. But first I have a note on my slide that another thing that people might know who follow this is, it's been discovered that actually you cannot satisfy both of them in many, in many cases at the same time so you have to choose between them. So if you like one, you might have to choose against the other is the kind of a result that has kind of come out recently. Okay, so here's a graphic. This is inspired by some of those computer science papers I talked about. I've created this image for my own purposes.  I thought to kind of communicate the intuition it would be easy to just kind of make some hypothetical numbers. So these are made up numbers that do track a real phenomenon, but I've kind of smoothed them out so that it's easier to kind of follow the important point. So if we want to see, if anybody wants to see why calibration and classification parody say different things, this graph can help us see it. So what I have here is basically the some statistics on the performance of a hypothetical pretrial risk assessment tool. Each of these bars represents a different category. So here on the left side. The defendants that were the black defendants that identified as low risk, and we can see that it judged 100 defendants as low risk 100 black defendants as low risk. And that 80% of those defendants did not go on to commit a crime or to be arrested. So it's pretty accurate. And that's the same thing for the recipes we have high risk and black defendants low risk, white defendants high risk, white defendants. And one thing that we can see immediately is that for the low risk black defendants and the low risk white defendants the they they went on only 20% of the time to commit to be re arrested. So low risk means low risk whether you're black or white it means exactly the same thing. The same thing is also true of the high risk categories even though it's a little harder to see, because this system judged more black defendants as high risk. Only 40%, only 40% of them did not go on to, sorry, only 20% of them did not go on to commit to be re arrested. And that's, that's also true of the white defendants 10 out of 50 versus 40 out of 200. So we can see that low risk means low risk, high risk means high risk, in exactly the same way, whether the defendant is black or white, so it's calibrated. But it doesn't satisfy parity. And we can see this if we pay attention to again these 40 black defendants that were identified as high risk but did not go on to commit a crime. So all those misclassified it sort of identified them as people who would go on to be arrested that these people were not re arrested. And we could see that 40 out of 300 total defendants were identified were misclassified as high risk. That reduces to two out of 15. Whereas in the case of the white defendants, even though it's calibrated and all of the statistics for each defender the same. Out of 150 total were misclassified, which reduces to one out of 15. So even though the system is calibrated. It's twice as likely to identify black defendants as hybrid misclassified black defendants as high risk. And that's just a way of showing how classification parity and calibration can come apart. Sorry, I don't know if you guys can hear that we're having some thunder. Hopefully the internet doesn't go out. Okay, so now we'll go on to my counter examples and hopefully I haven't taken too much time with this. I have a little bit to go through, but I think we can end in close to on time. So, again, classification parity tells us to make classification error equal across groups defined by protective attributes. Quickly, let's see why this isn't necessary for fairness. Recall the fact about men and women and reoffense a calibrated score will behave a lot like our hypothetical pretrial risk system. So back like this one. Except now, it will be twice as likely to identify men as misidentified men as as as reoffend as high risk. But if we kind of massage the context enough, this might not seem this might not seem unfair. Maybe for the men it first of all wasn't through uneven data collection. Maybe this was actually a true feature of the men in the kind of system that we're talking about. And further, if we can massage the case even more by saying like, it could also be the case that this is like a nuance system that we're working with. And being identified as high risk doesn't mean you're going to be treated harshly. Maybe it means that you'll get social services or something like this, right? So there are ways of maybe deviating from parity in ways that are that don't strike this as unfair, right? We could do things to compensate for this and maybe it's the fairest thing we can do. So alternative being if we calibrated the system, we would be overestimating recidivism among women and maybe detaining them without need. Okay, so here's why let's move on to the sufficiency question. Here's why it's not sufficient for fairness. Well, I just said that forcing parity in this case would mean systematically overestimating reoffense among women, which I think would seem unfair. So it seems that classification parity, satisfied classification parity shouldn't be necessary or sufficient for being fair. If you guys, if you share my intuitions. Okay, so moving on to calibration really quickly, which again asks us to make the same score mean the same thing for everybody for each person. Now we can just kind of go through this quickly. Here's why I don't think that's sufficient for fairness. We can imagine a case like pretrial release where any inequality in the scores are driven by arrest rates that don't track reoffense. So basically, if the case I walked you through where black defendants were twice as likely to be misidentified as high risk and maybe therefore detain. If that struck you as unfair, we could see calibration at odds with fairness as at least as a, as we could see attention in the claim that satisfying calibration is sufficient for fairness. But again, we can ask whether it's necessary for fairness. And I don't think that it is either because now this one imagines requires a little bit of imagination, but we can imagine a situation where we try to adjust for the problem in that graph by forcing parity to at least some extent and maybe that's the best we can do. So maybe being identified high risk for black defendant means something slightly different from being identified as high risk for white defendant, because we're sort of over compensating and forcing fairness, forcing parity on the system in name of fairness. That might not be feasible in the real world for all sorts of reasons, but at least maybe we can kind of see how we might deviate from calibration in pursuit of fairness to compensate, for example, uneven data collection or some other sort of issue. Okay, so that's kind of, that's kind of like maybe the meat of the kind of the talk is just kind of working through a bunch of those fairness measures and some counter examples. Now I want to make like a little bit of a diagnosis beyond the local counter examples, why have we ran into some issues. The terminology here isn't perfect to offer a diagnosis I want to introduce a distinction. I'm going to deviate from the terminology here but I couldn't resist because this is a, I think a great story maybe people I think I assume a lot of philosophers have heard it before. But if not, let me introduce a distinction by way of telling a little story from the history of philosophy of possibly a possible history, a story from the history of philosophy. Well, Sidney Morgan Besser, at least allegedly, I think some people have said this is maybe not true but Sidney Morgan Besser once attended a protest where everyone was hit beat up by the police. When asked if he was treated unjustly or unfairly he said unjustly yes unfairly no, he elaborated an officer hit me on the head with his night stick, which was unjust, but they were doing it to everybody so it wasn't unfair. He describes a nice distinction he's using the words justice and fairness I'm going to use slightly different terminology, but hopefully that gives you an intuitive grip on on the on these two concepts and introduce this terminology and these definitions by the way come from that rat hooker. So what Morgan Besser or what hunt in the Morgan Besser story is identifying as fairness all called formal fairness, the equal and impartial application of rules. So the officers treatment of the protesters in the story was formally fair. There was a rule hit each protester on the head of the night stick and they did that. But of course, that doesn't actually seem fair to us. While the officers may have been formally fair. They are not what I will say. They were not, as I will say, substantively fair or as hunt says, and Morgan Besser unjust where substantive fairness is the proportional satisfaction of a certain subset of applicable moral reasons such as dessert agreements needs inside constraints. So in the story. Again, if we want to call that unfair it would be substantively unfair not formally unfair. Okay, what does that have to do with anything that I had to talk about today. Well, let me say a few more things. So first distinguish a classification rule. For example, if a suspected if a suspected Frenchies height and weight put it in quadrant D predict Frenchie separate that classification rule from what I'll call I don't know if there's a word for this. But I'll call the ultimate rule or you maybe even could use the word maximum. That the classification rule attempts to satisfy. So the rule in our case was label Frenchies as Frenchies or something like that. The measures of fairness that we considered. Tell us whether a classification rule, which attempts to satisfy some ultimate rule by giving instructions about what to do results in the unequal or part or partial application of its ultimate rule. Right, so I think one way here's my claim of understanding where all of those rules we were looking at were up to is they were trying to give us conditions for formal fairness. But I think many of our complaints about fairness and machine learning have to do with whether classification or ultimate rules are substantively fair. So I think a lot of our examples were grading on our intuitions. Not because there was an issue with formal fairness but because there was an issue with substantive fairness. And so that's my diagnosis is why are these why are these rules not working out. It's because we're kind of chasing up with the wrong concept. Or some white claim. So, like I said, I promised some really big somewhat vague lessons, but here's sort of sort of three follow up thoughts. And then I guess we can kind of open it up for comments and hopefully I don't think I went too badly over time to my. I'm glad about that I was worried. So first, one quick lesson is maybe we need to think less about formal fairness and isolation from substantive fairness. I think we can see that maybe there are a plurality of possible candidates for measures of formal fairness. And I think it would be helpful to think more about what substantive fairness requires, and also to think that maybe it requires different kinds of formal fairness in different situations maybe classification parody. So we've seen that classification parody seems like a good objective in some contexts and parody and others, and maybe there's room to think about sort of when which kind of formal fairness is desirable. Another lesson is I think we can use formal fairness as heuristics for substantive fairness. What we saw in the example where we try to what we saw when there were when we saw that there was conflict between classification parody and calibration is that when those both can't be satisfied. There's some issue with an inequality between protected classes protected groups. Now that might not settle the score as to whether or not there is in fact anything unfair going on, but it's definitely a red flag. And so the formal measures here could be used as heuristics not arbiters but for heuristics for detecting unfairness right. When you can't satisfy both you are in that situation. And finally, I think you need a better understand better our understanding of substantive substantive fairness as it relates to predictive scoring, and that's like a really really really big lesson maybe if you call a lesson. But that's maybe, I mean I think it's an important point. I think sometimes, at least in practice maybe sometimes in theory as well. You get this vision of the kind of like the technological context or the machine learning context as like this context, where you can kind of think about things in a certain kind of way and maybe come up with some principles of fairness or whatever. But there is no such context. As we started in the beginning of the talk. These scores are not are using all sorts of different contexts criminal justice, hiring the market. And I think it's it behooves us to think really hard about what substantive fairness calls for as it relates to predictive scoring, with an understanding that that's a huge project that's just more or less the project of asking the question of how people should be treated, because it's not like machine, I don't know this point is coming across, but it's not like machine learning is like this one little context where we can ask narrow ethical questions. It's being used to do all sorts of things in all sorts of different ways to all sorts of different people. Which means that it's a big ethical project to figure out how to do this correctly and that we should resist thinking that there's going to be one circumspect mathematical rule that's going to tell us the right answer here. If that was the case, we would have finished ethics a while ago, I think. So thank you very much. Thank you very much for your time. I'm sorry that I went over a little bit and as promised really quickly. Here are some papers that I relied on pretty heavily for this presentation. And these three in the middle, the Corbett Davies papers are the ones where I got a lot of my examples and my definitions. And this one in the middle is a very helpful one where I kind of that inspired that graph that I use it for me has been very helpful. I think it's like one of the most helpful images I've come across. So with that said, thanks. And I guess we'll turn it over to questions or discussion. Thank you very much. Thank you. That was fantastic. So the way we're going to do Q&A is panelists will be able to raise their hands. I'll sort of call on people trying to get a sort of distribution of questions across our various disciplines. If you're in the audience, you can put a question up by writing into the Q&A section, and I will relay that to Clinton and integrate into the discussion. Okay, so we'll start with a question from Atisa. Okay, thanks a lot, Clinton. So I was wondering if like I wanted to push you basically to a little bit connect your discussion to the discussions of fairness and justice in philosophy. So for example, when you give this counter examples about necessary and sufficient conditions to fairness, I was wondering what do you mean by fairness there? Are you just using an intuitive conception of fairness or are you using like a theory of justice or theory of fairness? And the reason is that I became a little bit like a susceptible that maybe some of these problems are not really problems with formal fairness in the context of machine learning. It's just because we don't have a theory of fairness that is going to be applicable in all kinds of contexts. And when you look into the history of theories of justice, there are like 10 different kinds of theories and each of them has their own benefits. Some of them are very ideal. Some of them are not ideal. So I was wondering whether you see some nice analogy between these problems that arise for fairness in machine learning to problems that arise for choosing which theory of justice or fairness actually is the good one in philosophy and law. Right, thank you. Thanks for the question. Let me see. Yeah, so as you see, I sort of tried to resist a little bit the justice word for partially for this reason in the way that I was setting this up. And I guess that what the short answer to the question of whether I have a theory of justice or fairness in the back of my mind, I don't. Because I wanted to make kind of a general, a really general point that might look attractive from kind of any point of view on this. So I don't I'm not coming from a place where I have like a subtle deal on this, and I think that there is, I think it's fair to ask and I think a reasonable next step to think hard about different trade off between different conceptions of fairness and justice. So, I haven't, I haven't thought through. I haven't thought thought through that yes, I don't have a view on it. And I could, I, I guess one other thing I should say is, is that I beyond just using I guess fairness in an intuitive way I meant to use it in a somewhat broad way, obviously broader than formal fairness. But again, I think I'm just going to repeat myself here. I was hoping to make a point for myself and maybe this is just for me. The thing that I kind of brought in at the end of my diagnosis was illuminating to me to figure out why things have maybe gone awry. And I wanted to make that point again in the general in a general way that didn't kind of have any commitments with what kind of fairness or justice might amount to but I could see how that methodologically might be a problem as well. And if you have any, if you have any suggestions, I would be very curious to hear them the specific ways in which you were skeptical or any connections that you see. So I don't know if I don't know if you if you do or if we're not doing follow ups. Maybe we'll do follow ups on the slack just because I've got a bit of a bit of a cue to go through. Yes, but please do do that. So next question is from Damian Clifford. One of our lawyers. Thanks very much for that. I suppose the separation between formal and substantive fairness, I was wondering, to what extent that can actually also be classified as a distinction between equality and equity.  Is it actually ever achievable.  And then to what extent that actually results in just declaring certain applications and machine learning and predictive tools. Unfair de facto from the outside because they're always going to be. I don't know, not equitable. Yeah, so I think I mean, to give a really satisfying answer to this question, I'd have to think through the last one a little bit more carefully and maybe have an answer so maybe having a theory would be would be nice here. But I actually don't. I mean, I'm, I'm open to being wrong about this I'm open to be wrong about everything I said. But I actually don't think that I'm sort of setting the bar too high, or kind of setting out an unachievable goal. And I would say more other. I mean we have to look at this, the cases kind of case by case and side by side, I guess. But it seems that there are, I guess, reasonable demands that people could have for how they'll be treated by sort of automated systems. And those demands can be sensitive to what's reasonable and what's kind of a possible. So, I guess like in the example of the pretrial risk stuff. For example, I sort of in my examples had some references to how people are going to be treated for example. And so I think that when we're using things like a pretrial risk tool. We do need to be careful about where the biases are lying and stuff like that but also like what the system that they're part of looks like and how people are being treated. And on balance the whole the thing taken as a whole can be in my in my opinion substantively fair. In ways that wouldn't be super impossible to achieve. So maybe to put a little bit more meat on that. So for example, if we know that we're going to have to trade off between classification parody and calibration. Okay, maybe that's a fact and we have to figure out what to do about that. But also like if we look at the whole system, we can say, Oh, we're going to be inaccurate no matter what we do here in some ways. And maybe that'll inform whether we send everybody home. And maybe give some of them ankle monitors or ask them to check back in versus detaining all the ones that we think are high risk for example. I don't know if this is like coming out as articulately as I would like it to. But I guess, um, like I said at the end, when we treat someone through one of these kind of automated systems. What we're doing is we're just treating someone another we're just giving someone some treatment. And presumably there are ethical ways of treating people. And that's kind of not much more than what I'm talking about here and time out like maybe a subset of ethical concerns. But just meeting those demands, and it's going to be kind of harder to give more than one liner there but I think it's I think it's doable. I don't think it's like at the outset saying it's going to be a super high standard. In fact, it might be a very reasonable standard and in part because if you look at the context, I think that the what people can demand in any situation is going to be sensitive to kind of as feasible as possible. I hope that doesn't sound like empty and trivial. I hope that kind of gets to the heart of the question. Next question is going to come from Michael, PhD studying computer science. Great. Thanks. So thanks for that talk, Clinton. I think this is still in line with what you've been talking about already, but when, when do you think there are cases, like what, how would you figure out if there are cases when just the use of predictive scoring itself violates like substantive fairness. I think people have said already about the use of criminal recidivism scores. And I noticed that I saw in your field people profile that you have some work about consumer scoring data so that's sort of a kind of something I'd be interested in hearing more about. Yeah, so let me see. So, well I guess is the question just sort of, are there, what are the cases where it just can't be used is it sort of kind of. Yeah, yeah, I was like, I think I love to hear what you think those cases are or there may be some criteria. Yeah. Yeah, so I mean I guess I guess I have some thoughts about that. The overarching thought would be, I don't know if I would want to say, I wouldn't want to commit that there's like one context where it couldn't shouldn't be used. But I would want to say that it kind of depends more on how it's being used. So I guess. I mean this is going to be, I wish I could say something more interesting but like, I think that you know you should be certain that you're reliable. And that the extent to which you're being reliable and accurate that that's sensitive to the stakes involved right so just for an example right. I think there's a good way to use these for hiring and recidivism maybe not, but it's not whether you use them but it's how and if you're going to use them for those kinds of situations you should have a high standards for being accurate. And also I think that. So I think should be should be accurate. Sorry, I don't know if you guys can hear this there's a big thunderstorm right now. My lights just went out so hopefully again, I don't lose everybody. It's just kind of distracting. I also think that you, you should be sensitive to other things that people might might reasonably demand, like some level of transparency and other things like that but I think personally I don't know if I have any kind of hard and fast rules. Because more the more I think about this the more I think it's like, what should judges be doing when they do this what should hires be doing when they do this and those those kind of local and we do have to do these things and maybe as the computers can improve accuracy I know that with a lot of the, with a criminal justice they can improve consistency a lot, which could seem like very important. And then there's kind of a question is, at what cost of that come and how can we achieve that consistency without trading it against something else that's more important. And how can we kind of. Yeah, so I guess hopefully it gives you kind of an insight at least how I've been thinking about it. Yeah, that's great. Thank you. The next question is from Will Bateman in our project but then I'll go to one from the Q&A after that. Thanks, and thanks Clinton that was a really fabulous presentation. My question and I haven't sort of worked out all of its implications. The question is really about the kind of broader impact of being able to quantify and really identify reasoning processes that involve some what we would understand as discriminatory patterns of thought that involve protected attributes, race, sex, age, disability, etc. And I come at this from the perspective of working on a paper with machine learner and a philosopher where we're thinking about the dilemma that designers of ethical systems are placed in because if they work on historically biased data sets then the automatic response will often be a type of disparate impact discrimination to use US legal terminology. But if they try and avoid that by tweaking the decision criteria by reference to protected attributes they embroil themselves in liability for disparate treatment. And the thing that's really come out strongly from thinking through this dilemma and how you'd fix it from a law reform perspective is that the broader implication of being able to really clearly quantify and identify exactly how exactly the weight that you're according and each step of your decision making process to a protected attribute. It has these broader implications for the way that we design and think about discrimination law full stop. Because the way that we've gotten around the black box of the human brain in terms of discrimination or liability for discriminatory behavior is by saying, look, we can't work out exactly how prejudiced you were. And we can't work out whether you were discriminating on the basis of protected attributes in a type of good faith way or if you were doing it because you're just racist. And so we will just implement a blanket rule that says any reasoning process whatsoever which includes any of these protected attributes is ex facie without anything more unlawful and discriminatory. And it just it seems to me that there's this and this is my I'm asking you I suppose to reflect on my now very long setup. It seems to me that there's this broader implication if you can actually start to really clearly quantify and identify exactly the way that you're reasoning in respect of these attributes and in respect of non protected attributes kind of legitimate attributes that aren't protected that might impact the way that we design our laws more generally. Because we might be able to or at least we might be able to say look for automated systems we have a different rule where it's not just any involve any engagement with a protected attribute in your reasoning process which involves liability. It's whether or not you're doing you're including that particular attribute for a kind of legitimate purpose or you're doing your best. Not to be prejudiced and not to be discriminatory which has a whole set of other really complicated evaluative conclusions but so yeah that that's my sort of before 10 am coffee ramble. I'd love to hear your thoughts on it because I think you thought really deeply about it. Yeah, it's interesting question I. So I guess like I take it that the kind of the core of the question is, should there be maybe different rules for the automated decision systems with respect to maybe blanket bands on using protected attributes as part of your reasoning or something like that. Yes that's thanks that that's one one core that is one core the other core is do we have to rethink discrimination law more generally if you start to get to a point with with different types of ML systems where you can actually really clearly identify the weightings given to different types of attributes in a very broad set of applications and so you're starting to use these systems very broadly. And displacing a lot of independent human that you're displacing a lot of black boxes with a lot of really clearly identified algorithms clearly specified algorithm. Yeah, so I think it's really interesting that somebody who's like not a lawyer I get a little bit uneasy with some of this stuff because I don't understand the law perfectly well I but let me think through the first the first question. And I think it bears on the second of whether there should be a different rule for this for the algorithms I think I think you want to touch on this. So like, given the caveat I just made I'm not sure how to think about this, but I guess one benefit of these systems is you could give you could show. I mean I think that there's going to be a really hard question of even if it's going to improve fairness or whatever it is going to improve one goal in one way, if including race and gender. Presume that including race or gender or whatever will will have some improvement. There's like this further question of whether of how to make the kind of more holistic decision of whether to start rigging things up that way you might you might worry about all sorts of things. Like reifying some ideas, for example, but one benefit I guess it's I think it's going to be really messy business but I think you already said this one benefit of this could be that you could be completely clear about what the reasoning is and show that it's being used in a certain kind of way, right and kind of lay out explicitly that these are the rules this is what's going to happen. Not maybe perfectly deterministically, but you can at least think about that in a way where you're not going to have to worry about anything implicit in the reasoning that's going on there. And you can also, I guess you could also run some tests on it to make sure that it's not going to have sort of implicit problems. But beyond that I don't I don't really have a good theory about how to think about this and I get really, I get really kind of queasy talking about the law and and kind of policy solutions because I know that that is a that's sort of outside of what I know how to do as a philosopher, and it's kind of a it's true I just I just recognized as a really tricky business. Let me just cut in there. So, so first lesson is you guys should keep talking on on the slack and you can sort of avail yourself of the of the folks in our project and to know who have experienced Damien also would be a great person to talk to as well as well. But let me just try and get to a question from our audience. So Jessica court has asked, and I'll just pass this straight through to you. Could we treat machine learning bias in the same way as cognitive bias in real world implementation on human systems. So, for example, you might say, look, it says there's a 60% chance of recidivism, but we can also see that the person is in low socio economic status and black. So be aware of the bias and making a final decision. So do you think we can sort of compensate for the problems with respect to formal fairness by sort of judiciously using these systems in order to make a decision that also is a human decision. Um, I think that there might be some hope for that and I and I'm pretty sure I can't remember exactly which one but one of the computer science papers identified by Corbett Davies and company. I think gets into some of this and one of their recommendations actually mean I don't this is quite a recommendation. I'd have to look at the paper again. And, and I might invite Jessica do that too. But I think that one of the recommendations is I gave this contrived example of how we might force parody to try to achieve fairness in one of the cases where that we have a calibrated system that looks unfair for various reasons. And one of their recommendations is you could probably do it that way. I mean you might be able to kind of break things up that way but it would be better to let the system run calibrated and then make a judgment outside of it. I think I think some more some further context. And so I do think that that that there's at least some one good reason to have a human on the loop is maybe that they could we could say like what there is this system that tends to be better than people but it does systematically make some kinds of errors. And so we can consult its outputs. And then we can look at the broader context and make maybe some further judgment calls and hopefully improve improve accuracy that way. And, and, and I mean I guess on that note to there are other reasons good reasons to have humans on the loop with these kind of automated cases because we've seen them kind of. They can be really spectacular certain things but they can also kind of make mistakes that only make mistake only seem like reasonable mistakes from their bizarre point of view but humans can kind of spot certain kinds of mistakes as kind of obviously incorrect. So I think that I think oftentimes having a human kind of on the loop in the loop is a good idea for various reasons but then I guess you get into the tricky business of like whether people are going to over correct or not and I think that's also going to be very difficult. It depends on what that person's kind of training and skills are I guess this is a good point to move us over to the slack. So I'm going to track down there's a great paper that showed the judges responding to predictive scores were more likely to override them when they went against their prior prejudice.  So I'll put that one up on the slack and Matthew Phillips had a great point at the end there as well about the consequences for individual judges of going against a predicted score where if the score proves to be sort of confirmed that that might end up being something that goes against their record so they might want to protect. But we can continue that kind of chat over on the slack for now let's all thank Clinton.


Now, we have the next talk, Introduction to Network Security by No Master. No Master is part of the chaos door here, and also is active member of the fry from community, and he likes to take a practical approach to things. And so I think we will have a very practical introduction to network security now. No Master. Well, thank you. This is my very first talk at the congress. I'm so excited. I hope I'm not dissolving myself somewhere in the middle. Also, I'm glad you're all here. I've been asked to give some introduction that is probably more for the beginners. So I hope that you can enjoy what I'm speaking about. That's neither the problem that you don't get what I'm talking about, or that you have all heard of that before. So let's see where we can take this, yeah? So my idea was like practical, as you said. How can I get you some idea about what the actual issue about network security is? At Noc, we get questioned a lot about, is this network really secure to connect to? And the usual answer is the network is in itself not secure. It's your system, your laptop, that should be secure. But what is actually a realistic thing that can happen here? So that's what I'm wanting to show you. I have brought to you three examples for that. I hope that gives you some insight. So first, before we start, we need some mindset for that. The network stack we have, here's a totally oversamplified version of that. It relies on different layers, as you may have heard. So every computer has some medium access control address. That's the MAC address. It's not about Apple computers. We have the internet protocol address, which is additional to that. And on top of these, we have the application that uses some TCP or UDP socket and the protocol defined for that. You know that, huh? OK, so what really matters to us is the two layers here, because this is the network that you're using. This is the protocols that are involved with that. The other stuff is on the systems, the client on the server. And this is where the data goes. Of course, what we want to accomplish here is that no one can look into the application other than the client and the server. So your privacy is compromised when third parties have access to that. You probably see the S here in the end. We will talk about that later. So first, that we know what we're talking about, the easy examples here. IP address. You have seen that, IPv4, which is typically your home routers address, maybe. And IPv6, I have brought to you a very simple example for that, which is just a longer version, usually written down hexadecimal. So that's the IP address. And then we have the MAC address. This is a very unique address that every computer has on the network. So this is usually borne in on your device. You just start it up, it has the MAC address, and uses that MAC address to communicate on the network, to, for example, to get an IP address. And this is where the bad things can happen then. So my first example is ARP spoofing. Who has heard of ARP spoofing? So no point in explaining that. But I guess the rows in the back didn't raise their hands, so I'll get to you then. I usually like using whiteboards. I don't have one here, so I used my computer last night to make some whiteboard notes. I hope you can read that. My handwriting is really bad. So what we use here for the resolution of ARP addresses, of MAC addresses to IP addresses is the ARP protocol. As you know, as I told you, the MAC address is burned in on your device. And the IP address, which your computer uses to communicate on the internet, is then assigned to the computer. We need this address to be able to route a packet through to your computer. It needs to travel probably all over the world and reach its destination. So this is why we have totally two different address spaces. One which just gives your computer a unique address, which is always used, which is the MAC address. And the other one, the IP address, which is always individual for the location you are. Smart people know that the IPv4 address I showed you is not really unique, but we are simplifying things here. So on the local network, your router has to give the network packet to your client, to your computer. And to be able to do that, each one of these needs to know the MAC address of the computer that has the IP address for which the packet is destined. In this case here, the client asks for the IP address of the router. It sends a broadcast frame to the network, which is an ARP frame, and asks for who has the IP address. And the router answers, the IP address is at my MAC address. So what happens here, if you look closely, is that the router sends a packet. It says the IP address is at this MAC address. You get an idea? So what happens if someone tries to steal your traffic? They can send a reply saying, someone asked for the MAC address of the router, and the MAC address is that address. And what they do is they send a false MAC address, which is their address, and not really the router's, through into the network, so that your client learns the address and says, OK, well, then the router has the MAC address this and that, and sends its packets to you, to the attacker. You're looking at this from attacker's perspective. So then the problem is, for the client, you don't see on the client anything happening other than the usual data flow is commencing. The client sends the traffic to the attacker, which just forwards the packets to the router and back. So it's able, then, if it has compromised also the address of the client to observe any traffic that happens on the network, transparently, by just piping it through. And looking at it, and I wrote observing, of course, if you get hold of the packet, you can also modify that. So this is a very old-school type of network attack. And you get now the impression how that works. Hopefully, you can then get also an idea how to prevent that. What we have on the networks nowadays is switches that have security features. But you must be aware that on a usual network, this is not always the case. So every network you encounter is probably vulnerable to that kind of attack. So the next thing would be then the DNS hijacking. This attack is on a somewhat different layer of the protocol. It uses the DNS system. The DNS system is, of course, for the resolution of a host name to an IP address. So we have another layer here. It's not the IP address to the MAC address, but the host name to the IP address. That enables us to do a similar attack, but not only on our local network, but also on every host on the internet. What we then do is we try to get hold of the IP address of the DNS server. Usually, the client asks the DNS server for its address, for the real server it tries to connect to. In this case, an A record, which is the IP address. And the DNS server returns the IP address of the server in question. Then the client connects to the server and gets an answer back. Obviously, if we can get hold somehow of the DNS server's address, we can do then the same thing as before. We can return not our MAC address because we're not on the same network. We can return our IP address, which the client connects to. It can be on any host on the internet. And then the client sends, we return the IP address of ourselves instead. And the client sends the traffic to us. And we then, again, transparently forward that traffic to the server. And we see every request happening there. In that case, we don't observe the traffic coming back from the server to the client. We should also attack the server for that. But always keep in mind that the payload of the traffic is not very interesting for an attacker, maybe. They just want to know which address you're really requesting. So very much of the privacy issues come from the metadata of the packets. Not with the metadata. In that case, it's the additional data. So which host name are we requesting? You can imagine some host names are worse than others to disclose. So that would be two examples on the usual networks. What we also have at the Congress, what we get questions about is how about the access points. That is a more complicated thing here. Of course, when we connect to a computer to a switch, we see that we just took a cable and just plugged it into a switch. So the computer is connected to that switch. Very similar the wireless system works. The engineers designed wireless traffic, the Wi-Fi system, to work as a wireless Ethernet. So it works as if I would plug in my computer into a switch, but otherwise it connects to the access point. So what I have on my computer is a system that chooses the access point to connect to and does that automatically. So it works just like if I had an assistant that when I move my computer, it takes the cable from one switch and plugs into another so that I don't have to care about that. It just makes sure that the connection stays up. When we think about this like that, we can see that there are issues. What we need to know first is that the network is identified by an SSID. This just means that I know which group of switches I want to connect to. On this conference is the 35C3 network. On your home you have probably chosen your own cool network name for that, but you may not have thought about that at your home you have one access point, otherwise we have here hundreds of them, and your computer changes the connection to the access point all the time. So what we have to prevent that your neighbors are connecting to your network, we have these encryption protocols. In the past there's been WEP, which is just broken and obsolete. So we now have the WPA protocol. And for that we use a pre-shared key, the Wi-Fi password. So the usual thing you go to your neighbor, to your friends, to a cafe, and you ask for them what is the Wi-Fi password, you just use that. One issue here is that it's only for preventing other parties from using your network. Everyone who has hold of the pre-shared key is able to connect not only to your network, but potentially able to decrypt all the traffic that is running through it. Then another problem is that I can say I place another access point, which has the same SSID as the currently existing network, but it's not an official access point, but my own. What then happens is that the client just connects to that access point, for example, because the reception of the signal is just stronger, you can move it towards the attack computer, and then just the same beacon with the same SSID as the attack network to the client. The client connects to the access point and sends its traffic through to it, and of course the way back. You could do that on the network here, which strongly asks you not to do that. Just one reason is we don't want too many access points running around here with too many channels to use. The other thing is, of course, you shouldn't attack other people, computers. Just explaining here how that works so that you get an idea how to prevent that. At this Congress, we have a system that is used in enterprise environments to prevent you from doing that, which uses another layer of encryption. It works like that the access point needs a certificate to make sure that there is an official access point you connect to. What happens then is that in the beacon, there is some encryption stuff, and your computer can check out whether the access point is really valid. When you set up with the Android app your wireless connection here, there is some Android app in the Play Store. This one generates a configuration in which a certificate is written down. Only the official access point here are in hold of the key for the connections then. So your computer, which is in that case your smartphone, can make sure that the access point is official. On your laptop, you have to configure it by hand. I would like you to try that out. Go forward and look into the wiki. There are details how to configure that. There's some handshake protocols, some encryption tunnel, there's a domain name, a fingerprint for the certificate, and you can say, please make sure that the certificate is valid. Only then, this attack doesn't work anymore. At your home, you probably don't have these enterprise authentication system, which needs another server, which needs to be configured and running, and usual home networks don't have that. In the future, with WPA3, there comes an extension which introduces that kind of protection. So, what can you do about that? I would recommend to you to make sure that you have a basic knowledge about protocol definition and the standards. Just look at the internet. There are RFCs, the Request for Comments, protocol documents you can read, how that stuff really works. And you see, when you read really how it works and you get your understanding of it, you also get an imagination of everything that can go wrong there. So, also, from the other side approaching that, read through the internet. There is plenty of documentation, not only more precisely and more clearly explaining what I did to you here, but also all kinds of different other attacks. And also, these give you a broader imagination of how stuff works. This also gives you an insight how to really protect yourself. And I mean, if you only want to do that, as I said, use HTTPS, which is a secure protocol for the application. So, the application clients make sure it connects to a valid server and encrypts all the traffic between that. So, the most of your traffic is not observable, even if these attacks are in place. And that is very important. We have, fortunately, a big increase of encrypted HTTP traffic in the past thanks to the Let's Encrypt organization, and we're always very grateful for that. Yeah, and if you want, you can try that out. Build your own WPA-EAP, Enterprise Authentication Protocol System. But that is, of course, also only limited. What you can do more practical even is, just with your laptop on your Congress network, look at the traffic, type Zoodo TCP, this that, and you just see the packets flowing in and out from your computer. You can check out the route packets actually are taking with the trace route software, the simple trace route protocol, or the more sophisticated program MTR is helping with that. And you can learn a lot by using a program like Skapey. I can recommend that to you. It's a simple Python shell that enables you to forge packets. You can say, like, I want a packet that has this source address, destination address and looks like that. Please be always respectful to others. You'll be excellent to each other. Use these tools to learn for yourself and to learn how traffic between your system and maybe your other system or your friend's system is working. Maybe you can pair with another person here with that computer and try that out. Yeah, so I hope I could spark your interest. And I guess my time's over now. So no real Q and A, I'm sorry. Thank you. Thank you for this really good introduction. And you were exactly in time, but sorry, no time left for Q and A, but I'm sure they can all find you and ask questions. Yeah, I'll be over there at ChaosDorf if you want. Okay, so let's have a nice warm thank you. Thank you, applause for no master again.