Welcome to the United Nations Peace. Some of you have been here for a very long time before. We're very glad to have you here. Others from across the street, thank you all very much for coming. Others who have been here before, welcome back. This is a great event for us. We're very pleased to be able to be here for this book launch. On speaking their peace, personal stories, too often, we in the peace building community get wrapped up in the projects that we're doing or the places that we're going or the bureaucracy here in Washington. And we miss the important stories. And the important stories are the ones we're going to hear about here today. So this is a great opportunity for us to tell you a little bit about and also for you to ask questions and talk to the authors and editors and people who have put this book together. So this is a great opportunity for us. The book has a lot of characters. But also, there are a lot of characters who put this event together. So let me just run through these real quickly. Bearded green from growing 40s press, health guide publishing process, and Nigel Queenie, the editor who provided valuable guidance for shaping this book. I've always recognized David Swack, who recently retired after 25 years here at the Interpeace. David was supported by the creation of this book. And so we have David to thank for putting this project forward. The pictures that you see here and out in the lobby here are the product of Fitz, Bill Fitzmaner, in the back, who is still taking pictures. But thanks to you on your way out. I'd also like to recognize DJ Simhan and Bethany McGahn from his last rule of law team who developed a website for Speaking of Peace. We encourage you to take a look at this site, which is at www.speakingofpeace.org. Take a look. You can see additional content and upload your own video of what this means to you. So this is an interactive one. Accompanying this book are a number of discussion guides for high school students and college students. And we're very glad that Anne-Marie is here and she has something to do with these guides. As well as book clubs, all of which you can download for free off of the website. Many thanks to those who developed these guides, including Kay Spencer, the last rule of law team, Professor Kristi Warren from the College of William & Mary Law School. Now, we have these people waiting to be interviewed. So I'm just going to introduce them very quickly. First of all, of course, is Collette Roberts. Collette Roberts. Collette Roberts. The main author and I guess editor, author editor of this book. Author, editor. Boom. Nigel Zettler. Nigel Zettler. She, of course, everybody knows in this room, acting vice president for government law and society here at USIP. And today, also, we have Kitam Akatani, where Kitam's here. Very good, Kitam. A program officer here at USIP. Many of you know her. She currently works on Iraq and Libya. She's also a member of the team that helped collect the interviews that you will be hearing about. Nazla Amagwish is also here. Thank you, Nazla, for being here. One of our panelists has worked with us extensively in the past on Libya programs and has also contributed to the book. And the last person we have on the panel here is Biola Ganger. We're very glad to have Biola here. Biola is actually a step up from what we were going to have. We're going to have an NPR reporter, but she was sent to Baltimore for other duties. And so, better than that, we have Biola. So, thank you very much. And with that, I'm going to turn this over to Biola to handle this thing. So, thank you. Please welcome the panel. Thank you. Thank you, Bill. A lot of pressure there. I also want to welcome the online podcast. And anyone can participate in the discussion and submit questions via the hashtag speak peace. And I would encourage as many questions as robust a conversation as we can have today. Thank you very much. And I want to start with as provocative a question as I can think of. Your book is about conflict zones, war and peace, dynamics that go into it. So much of that is determined at the highest levels of government and politics. Why, what does it matter? How much, what ordinary people think and say in these societies, how much power do they really have? Yeah, well that's actually what drove the creation of the book. I think a lot of us work, and I see a number of you in the room who work at different organizations or the State Department, NGOs. And oftentimes we're working at that higher level, so to speak. Whether we're assisting with policy development or designing projects from afar, they're going to be implemented. And I think after doing this for many years and a lot of my colleagues, you get to that point where you think maybe we don't know everything. You're seeing that changes don't happen no matter the best efforts at those higher levels. And so the driver of the book was to try to find out what is that wisdom? Let's not just assume that regular people don't have a voice or don't have the answers. And maybe we're just swimming at this technical level and maybe really the answers are at a deeper level. So I think for me it was why not ask. And I think that's where often the power can happen is the voices from the individuals because they're living it every day. They have the wisdom every day and maybe that can help guide us in a better way. Did you see any examples, and this would be a question for any of you, examples where the people that you were speaking with, which is 80 people in 11 countries, did exercise some influence in their communities that made a difference? Definitely. And I think actually I'm thinking of a specific person in Iraq as well as in Libya perhaps Gautam could give the example of one of the mothers. Sure. Well, first of all, thank you so much for this opportunity. Actually part of the objectives behind having this book is the speed of development at the local level is always the speed at the local level is always higher than the speed at the policymaking level. So no matter what you try to cope with the speed at the local level, it's very hard to keep up with. So answering your question, there was actually a couple of examples which you can find in this book. Some women in Iraq, they were the victim of al-Qaeda invasion to Iraq and some of them they lost everything or what they have like few members of their family that they have to fight for. And the members of their families were the light of hope that made them look around in all the darkness they lived through and they tried to look for alternative solutions to change their life and to influence their community, not just to influence their remaining members of the family but also to contribute to the community as a first step and then contribute to the international initiatives that was there in Iraq, hoping that their voices and their action can reach out to decision-making positions in policy processes in Iraq and they can put their fingerprint at that level. So definitely we had a few examples but I will not reveal who they were but you can read through the book. Can you give us just a taste of one example of that? Well, I interviewed a lady. She lived in the... and she actually is still living in the triangle of death in Baghdad, actually south Baghdad and that was one of the horrible locations where you can live, especially under Al Qaeda control but after losing her husband and after tearing the community around her, she decided to take a step further and she decided to create a way of life. She was living a very wealthy life but all of a sudden she lost everything after killing her husband. So she started actually working from her house and she started producing some food and non-food items and she started selling them undercover because her dignity was... I mean, she heard deeply to be in that position but she said, I'm going to take this path than taking the path of revenge and losing the remaining of my family and also I'm going to create my own way of living and instead of having others sponsoring me and driving me and reshaping my life the way they want even her brothers and her relatives. So she started creating like a food line and non-food line items and then she expand to other widows who've been through the same situation as hers and she started bringing them together in her house and when I asked her why you had this idea in mind she said, we just need to talk to each other. Nobody will understand us better than we understand each other. So I realized the moment when I needed someone to listen to my... to my suffering and to see my tears without, you know without any, I would say, underestimating look like I'm a widow now, I'm not a widow. Don't look at me and don't categorize me in that window but I'm someone who got hurt in a community that I have nothing to do with. I have no control on violence or conflict in my country but it happened to be a member from this community and then she started bringing women together and then she took further by accepting the initiatives by local organizations in Iraq, by women groups and she actually took the lead to convince all widows and all other victims in her neighborhood to be part of that initiative and to turn their life from the victimhood cycle and to transform their roles into peacebuilder and she actually... one of the symbols that touched me so much and made me rethink about my position and my role as first as Iraqi citizen, second as a worker or as employee within international umbrella and how much responsibility that puts over my shoulder. So that's just one example. Thank you. Thanks. Do you have an example of where you saw evidence of some of these ordinary people that you've talked to that you interviewed taking control, more control than they might have before the crisis, for example? Thank you for this question. I think the situation in Libya in general was unique because the timing issue was have conversation about it was after the revolution directly people have high expectation and have different feelings about how they perceive conflict and how they understand conflict. So I think that explained a lot about if they really were able to make change in that time. I mean, most of the people that interviewed the book they really have significant role in terms of supporting the revolution either supporting the others or trying to contribute to the society during that crisis. Did they have significant roles beforehand or did they step up when the revolution began and they felt like there was a place? So there is two cases where there are two people they're coming from outside the region both of them they live one of them the woman in France and the other in Canada and he's Canadian citizen it was so interesting to see those people just live their home where they have all the facilities and good education and good chances and they just come back to Libya and the passion for both of them to do something for their country was very obvious. So I think they decide to be part of the change and decide to have different role where they put themselves on the risk. So for example, Yusuf he's a Canadian citizen he was in Benghazi and he witnessed the main military pace when they fall down in Benghazi by the revolution and he was part of this demonstration and his cousin like he was really injured very badly during the protest and he witnessed the blood and witnessed the violence. So all that different dynamics and the way how he witnessed violence and witnessed the conflict in Benghazi city was really interesting to see like young person just decide to come and fight either way to just participate. He had the decision to go back to Canada any time because he had Canadian passport. Did he become a fighter? What was his role? He wasn't a fighter, he was like more activist like supporting the revolutionary being part of this huge protest trying to... Also he helped a lot the United Nations through Egypt and the border like he tried to help the people there and bring humanitarian aid and supplies and all that stuff. Collette, what period of time over what period of time did these interviews take place and what changes did you all see over that period of time in the perspective of the people caught up in this maelstrom? It was interesting we were just talking about this before the project started in 2009 and I think we were all a little bit naive and optimistic and we thought we'll be done with this in two years. That was 2009, it's 2015. So it was a seven-year process and that was for a number of reasons. One, we were originally going to stick with the countries this was before the Arab Spring, this was before Yemen, before Libya, before Myanmar's transition. So even the countries that we started with almost doubled during that period of time. And then also we were looking at countries that had been further away from the conflict when you look at Peru or Nicaragua were two decades past the conflict. Then we were looking at Nepal and Kosovo that were closer and then obviously with Libya and Yemen and Iraq the story is still being written with many, many changes. So I think the most challenging part was Libya and Yemen and Iraq because as I said those stories are still being written and when you look at Peru and Nepal even with the earthquake and the damage done there there's still fairly stable countries in Nicaragua but these are three that we still don't know where that's going to end. So there was the Yemen, Libya and Iraq where when they were interviewed especially with Libya or Yemen we thought and those in the country had a lot of hope and even with Iraq that okay we have a plateau we can now start moving forward we can now start rebuilding and just as we were writing this with the, I mean I think Nigel was going to pull his hair out and we had to keep changing the country profiles because it kept changing we finally had to just say stop we can't keep this as a day by day below of what's going to happen so it was just very different because you're dealing with fresh conflicts and those that have had some stability and time behind them. And in those where the conflicts were evolving so rapidly during the time that you were working what changes did you see in the perspectives of the people you were talking to? That was very difficult even with keeping in touch with some of the people we had interviewed because there were then new security concerns where the thought was do we need we kept reaching out to them do we need to change our identity does this need to be confidential does this now put you at risk and then I think when we were talking with them three months ago there was just this level of despair and this level of all the high hopes especially in Libya or even what a semblance of stability was starting in Iraq then it's a whole new issue with ISIS in the occupied areas and Yemen that's now fully embroiled in a war there was just this level of despair in Libya because it was so high Yemen the interesting thing is it was a little bit different we were getting emails saying yes it's difficult now but this is the course of history in Yemen and this is what we expect ups and downs so it was very different and sometimes we would have an assumption of something would be one way and then it would be completely different but it was kind of a roller coaster I think for a lot of the people working with it because we know a lot of them now and in contact with them and so you're feeling what they're feeling what they're going through and you say that the Yemenis most recently have been more sort of not saying one but more understanding that this is part of a cycle and part of the transition I guess it's I don't want to speak for everybody I would say with some of the youth there was frustration because they were part of their version of the Arab Spring and they had high hopes and then they see it devolved into this political externally internally driven conflict I think that very greatly affected them on the other hand those who from the older generation there was one colleague that we'd worked with and I would email and say oh I'm so sorry what's going on what can I do and it would be oh my dear Colette you just need to understand and put this in perspective these are the cycles of conflict so I don't want to generalize but I think that sense of wishing things could be better but some were a little bit more philosophical than others and I can't speak for the whole situation What changes did either the two of you National and Qatar detect over the period of years of doing these interviews I mean even me as a person as an interviewer in that process is completely different the way how I felt about the interview and the feelings and the emotional and the perspective is completely different from now so that's really the beauty of the book because it shows how people affected by conflict and how sometimes the narrative it's very small because they're really involved and being part of the conflict and try to work in daily I mean for me I was like wow maybe after two days it would be perfect and everything would be fine I didn't actually get the sense things would be I know Libya needs a lot of work but I wasn't actually expecting what's going to happen today and I think part of it is the education and the experience but I think most of it because I was part of this movement I was working there with the civil society I was just involved it's hard to me to see the big picture and I can see that also from the interviewer when I just go back now remember that all their expectation was so high because they are being part of that and they suffer a lot under the dictatorship so their expectation is very high now we get rid of the dictatorship now we can rebuild our country we need accountability we need justice we need all that dreams and stuff but one of those just interviewer had a different perspective her name is Selwa and she was so interesting to read her narrative in the book because she highlighted she was an observer more than participate in what's happening so she decided to come from France to Benghazi and just work in exhibit there she's an artist so and the way how she perceived conflict it was completely different she shows really a lot of you know knowledge about the trauma issues and how we can change the perspective of the people one of the things she decided to paint the car the military car like a big trunk with pink and just you know just try to and the reaction of the audience was so so interesting to people happy with that color and something completely different from all these dark colors and you know these you know so she was really able to change the narrative and try to encourage the others to look to the conflict from different perspective and that was interesting to me which goes back to your question too about what can an individual do yeah yeah and that I mean when you looked at things like that you can see the impact it didn't change the course of the conflict so to speak but it had an impact that can be built upon and grown and impact on the perspective of those in and around and also she knows her capacity so she's just all the time she was emphasizing during the interview I'm just only one person I'm individual I can't change but what I can do this is a small exhibit I can encourage the others you know and bring colors and play with arts part and you know to inspire who come to visit this exhibit so she was really specific about what she can do is she still there as far as you know I unfortunately I don't know I hate to be honest but yeah now what did you hear from the people you spoke with about the interaction between the general populace in these countries in these communities and the elected leaders or the political leaders whatever form the political leaders at the time took which varied I think in some of these environments because we take for granted I think in sort of western democratic systems there's an imperative for political leaders community leaders to be in constant contact with their constituents that's not necessarily the case or at least the dynamic is not the same in post-authoritarian environments what did you hear about what kind of contact and influence the populace the general populace had on elected leaders or on the community leaders one common theme that came out was this sense of disconnect and a consense of picking up what Nezha said of wanting whatever it is of elected officials government to make changes to be accountable to have justice and there was this sense of urgency for example in Nepal urgency after their change of government that they wanted that next government to be better I mean it was just this common theme of disconnect with the citizens feeling that things would be better if the politicians came together and found a way forward and to help build consensus rather than divide and being very nervous and concerned that that division or the politics of in many places political violence it wasn't just division of words or ideas but violence was going to make it difficult to either build a piece or to sustain the piece so it was this sense of disconnect I mean it was very difficult especially that part I mean especially in the preparatory phase to convene people to like to come and share their stories like for instance the some of the mothers that I met with they were not educated they were not educated in Baghdad yet but that doesn't mean they don't have a voice that doesn't mean they cannot contribute and they don't have the same platform as lawyers can do and contribute to the rule of law status in Iraq so being in that process they realized this fact like these are one of the channels they can reach out to legislative authority or executive authority and tell them this is our evaluation of the policies that you have and I think from my experience through like participate in in the interviews I think the people we met with they have and forgive me I'm going to be very frank here they have a huge wisdom and they have a very deep wisdom than so many people we met at the decision maker they were criticizing some processes in a very objective way in a very constructive way I haven't met with any of the people we have interviewed and they just criticize the system without giving me some solutions around it so it was I mean for them and for us it was a very good like process also to hear their evaluation and to hear their ideas and creative thoughts and feed it back to some of the decision making channels and some of the legislative authority in Iraq not necessarily all of the recommendations being taken to consideration but many of them at least when we communicated things to civil society organization they had a strong foot in advocating for many of the suggestions and the criticism we've been communicating to them so I think that's what we what we have experienced in Iraq through this process and that was a good point because a lot of the people that we were interviewing it was natural for them to be cautious what are you going to do with it why do you want to hear my story and many of them were used to people coming and asking different things and not seeing where it went and so the connection there was as soon as they heard that their story not parsed into some sort of research paper not but their story or their view would appear and that it would be conveyed to people who could make a decision that was huge and you'd see kind of their eyes light up and then the second part was is this going to be shared with others and other countries because perhaps if somebody from Iraq or Libya or Nepal could share their story and what they went through was somebody else maybe it could help them get through it or even better help them work within their country so the conflict or violence could be stopped to begin with I think those two driving factors and then there was an openness to communicate when they didn't when it wasn't feeling like it was going to just parse this into a footnote in some research document naturally in Libya as the transition evolved and after most of the fighting ended during the law unfortunately as it turned out to be what did you find there about connections between the leaders in whatever form they took and ordinary citizens how much understanding was there about the connection and what form it needed to take I think because the audience were selected randomly and we meant to choose different cities so it's also interesting to see the dynamics in different cities during that time so Benghazi because there was the first city pulled down and the expectation was so high and also in the beginning there was really strong connection between the government the political decisions and between the citizen and I remember because I worked for a short time as a head of public engagement unit and was my role to hold those seminars and workshops and try to introduce the people who represent the government to the civil society it was really big potential to have this really wonderful dynamic democratic interaction between the citizen and between the decision maker and this is actually the first time happens in the Libyan history after the Dafi regime so it was really it was significant experience where we felt like there is potential to have this regular feedback the way how we did it people criticize and ask questions and was life and everybody can hear it and can participate in that was really public event by all the meanings but that didn't continue for many reasons and when Tripoli liberated that time things changed and everybody went to stay in Tripoli and then the connection start you know the people start lose connection especially in the east and also in Tripoli to be honest there is during that time there wasn't that strong understanding and what the leader could do was the election process was very successful because again the people have very high expectation but they also include the political leaders they don't know exactly what their role because also they were in that first time in this position where they need to be leader and political and I'm not finding excuse for them but I'm just trying to show how the people perceive that those leaders and how the leaders don't think about actually the citizen perspective anymore after the election process so there is no kind of you know regular connection and I think also part of it I also blame the citizen because I think when you grow up in dictatorship and I include myself when you grow up in dictatorship the perspective all the time you just waiting for the government what they can afford to the people and all the time you're playing them you know instead of also being part of the responsibilities and I think that one of the lessons as a Libyan we need to learn how really we need to be also responsible for our communities and try to think about creative way how we can work together as you know even support the government or be part of the change positively instead of just criticizing because what's happened when you have this very high expectation now when you have the narrative with the people what are happening today they are completely angry and they just criticize all the time because there is no connection because there is high expectation because there is no citizen education about their rights and their obligations over the new country and how to build country so I think there is many reasons about that but it's again I think there is this book is so important because I think it will be one of the lessons we learn we don't learn from the history and if we example the French Revolution and we see the people during that time how they have high expectation about the revolution and what's happening next we don't learn about this lessons and I hope this book will help the people to understand those stories and how that affect their life and their expectation about the next you spoke to some war fighters in several of these countries Nicaragua Nepal, Liberia Kosovo what did you hear from them that most surprised you these were former fighters I think there was one saying that it was retired general from Nepal and he said something that struck me and it was at the end of the day whether we did right or we did wrong will depend upon the outcome and we judged accordingly and so I think what he meant is that a lot of times people say something is a just war or we had to go to war to make the revolution and make the change or we should never have gone because it was just a political war and there was no hope and at the end of the day out of all the war fighters it was what Nezha said it was their viewpoint at that time but that changed over time so for example in Peru there was one who had fought on arms in Peru who had fought on the rebel side and when she was young it was just it was important nothing was going to change unless you do a revolution but now she's my age she's 50 and she looks back and she feels like that nothing happened the way it was supposed to and all they did was swap one government for another government that isn't responsive yet one the most because they were still fresh off of it and things went well it had to be done the Serbs were not going to make a change and this is the right thing to do so it was just interesting and then the rebel the Maoist rebels these young guys it was still in the ether of their revolution or their insurgency and it was all indoctrinated in that and it was just parroting all of these things of justifying it so I think at the end of the day I look at it is it depends upon what happens and what period of time and it's not static interesting did you have any connections with former soldiers in either of the settings where you two worked Katam? not as much you must have certainly in Libya and what did you hear anything from them about their perspective that surprised you? you mean now or during the at any time I know one of the I know one of the Libyan who fought against Gaddafi and he was really passionate and he wanted to be part of the change and one thing he mentioned to me was very touchable like my life changed he felt like he's stuck in this cycle of violence people actually when they start fight the ordinary people when they start fight they fight for their freedom for justice for all these beautiful rights even that means yes maybe killing the enemy but this was actually the purpose and they don't know even how to use the weapons I mean Yusuf when he read the story the protest and his cousin you know fell down when he was injured he didn't even know how to use he see the people like just holding the weapons and he didn't know even how to use it the people felt like they need to do that because there's purpose without that and their passion to liberate their country but the problem now most of those people who are being part of this liberation process they are now still exist in those different groups because they didn't they couldn't get themselves out of that traumatized the circle of violence so I think there is a lot of trauma including the stories and a lot of pain and wound and and it's so and I think it's so interesting to when we read the stories we try to think about the personal perspective you know and just see how that affect their lives and the way how they look to others and look to themselves and the sad things most of them they don't know they really suffer from this you know trauma where they can't really change they can't change it because they don't have that technical help to go through that process in the wake of all of this turmoil and the meltdown of the Arab Spring I occasionally hear people say that this idea of democracy just doesn't translate in other places and it's not really relevant and helpful to the general populace and that it's this you know thing that intellectuals are construct and think is a great thing on an ideal basis but in practical reality it doesn't really work for the majority of people based on what you have heard from these people you've interviewed and with other interactions that you've had in your years of working what would you say to that? I'd definitely be curious to see what Tom and Nejla say I think sometimes we conflate democracy, governance with other things that are going on and so I without a doubt there are certain themes that have transcended all the countries and it's the notion of justice it's the notion of wanting accountability of having secure lives of being able to send their children to school now people will wrap these things up in democracy however we define that or election processes or however elections equals democracy that is a separate issue but the root of it with the people we interviewed across the countries is these very basic constructs of as I mentioned justice security living peacefully sending your child to school being able to make a living and so I think that is really what the common denominator is but this issue of democracy as you said and I've heard pundits say the same things it'd be interesting to see what Tom and Nejla say about that specific dynamic and can democratic systems deliver these types of things I think it's an ongoing issue and it's a million dollar question for the people is what is the definition of democracy if we're looking for the academic political definition of democracy it's completely different than the definition that the people looking for at the local level at the provincial level at the national level people I don't think many especially of the conflicted zones now and I'm going to speak on Iraq for them nothing have been changed like even when changing the dictator regime yet their voices haven't been heard and nothing have been done for them to make to improve their life the previances is the same injustice for them even worse the in equal services even getting worse so that's what make flip the coin maybe of Arab Spring in different countries because it's not just about changing faces at the legislative authority and executive authority ministries and so on it's about reaching out to people it's about hearing people it's about reflecting the voice of people so I think unfortunately not much have been changed especially at the grass root level I resonate with I think the challenge is how to change the mentality for the people and the way how they especially the political decision like how they really be aware about how they include the others in the process and try to have approach different level of communication instead of trying to enforce or fight especially now Libya with its chaos and different group and between them the sad things about Libya the expectation was about when you write the book it was about rule of law accountability justice now the demands we just want safety now for my family for example they just want to make sure they live day by day without being targeted by random shot comes from gun or car or whatever so that's you know it's the way how people now see the situation they don't have any high expectation anymore and to be honest in Libya they don't care anymore about what's going there they just want to make sure they live safety and they at least have the basic needs and can just you know provide these you know even the education before they want high education now they just want the school be open because you know the school was closed for a year until now so it's really sad to see how the even the expectation and how the demands completely change because people now they can't see anymore they just want safety and you can see the number of refugees now outside the country and yeah so I think the only thing I can say you know listen to the voice of the people in a way without make any interpretation and just understand the motivation behind these stories you know I'm not finding excuse for violence but violence sometimes you know the motivation under the violence if you understand the motivation and the causes behind that and why they being part of this you know one of them they see 350 bodies front of him and they collect those bodies so how you what's what you gonna wait from that that person if he was witnessing all that violence so if he I think that the beauty of the stories because when you hear the story from there that will explain a lot of what's happening even today we just have a few minutes before we want to open it up to the audience so I want to ask two questions quickly Colette when you're if you've been in touch with anyone in Nepal and I think you have because you've worked there pretty intensively in the past what are you hearing right now from them about what effect this earthquake this tragedy will have on Nepal's political chaos coming out of this once they have a chance to think about that that element of it right now it's so fresh it's not even to that point I mean the interesting thing talking about where people are and what they could think about because the aftershocks are still happening they're still sleeping in the car sleeping in tents and there's still that anxiety in the kind of fight-flight dynamic and the only thing that's coming out right now is the hope that they're not going to have looting in problems that that will hold together and then they hope that people just give them time because they're in that mode of just trying to survive right now and international community is starting to pour in chaos and not coordinated so it almost felt a little bit like deja vu and so the hope is be patient let us get it together and not overwhelm us or get too far ahead of us so it's just this interesting voice that we often heard coming from the stories in the book too let me ask you all as professionals in peace building conflict resolution was there anything you heard in the course of these interviews that changed specifically in concretely what you do or how you do it in this field? Do you want to start? I'll let you start. Well for me personally I realized that the fact in the in the conflict locations that I had my interviews in is that dealing with conflict doesn't need huge resources dealing with conflict doesn't need and doesn't require high academic level of technical support dealing with conflict needs reach out to people needs to find out the missing cycle where is the chain is getting broken and we need to do our best in finding alternative to create that missing cycle to bring the chain together and that's very much I think that my that was the lesson learned through these interviews because with the differences of the stories with the differences of the situation conditions and the level of sorrow the level of pain and the local perception or the interviewee perception rule of law and how can we promote rule of law in Iraq I learned the key thing is to bring the decision maker to the to the ground level we need to we need really to build that bridge even if I mean it's gonna be long trip it's gonna be a long path but we have to walk it and the more people tends to walk with us through this path I think the faster we gonna build that bridge so that's that's what's actually I'm carrying carrying on through the projects I'm working on this is a fact I have to always stick to I don't need huge budget to build peace in certain countries in certain communities I just need to be realistic I just need to look what's there available as resources that I can start investing with before like investing on different directions that might not be realistic and well designed enough so that's thank you Nashla do you have any I think yes I think you know the element of trust so important when you have this communication with others people when they trust you respect you and then the communication will be easy I think the conflict tools and analysis tools is very helpful to have as backup to think about the conflict it's good actually to to provide space where we can hear different voices and different narratives so I think that's where the academic tools maybe will be helpful but from the personal level I think listen with respect and without judgment and and provide safe space where that person really can contribute and tell his story the way he want to say it we don't have to change the words we don't have to change the way how explain it and just hear it and give them the agency to describe his story I think this is the power I think this is the way how you encourage the others to express themselves and speak about it thank you Colette I don't want to repeat what they were saying but at the end of the day I think after so many years of the technical things the theories the books all the things that we do do do we go to pass laws get elections do do do run run run and if we do all that run fast enough then everything is going to be okay and I think at the end of the day what this taught me and I actually kind of had to force me through this is to pull back that kind of curtain or that veil and then just to embrace the human part and to just be and to be present and to listen and to listen not just with your intellect but your eyes I mean sometimes I didn't understand the language but I could feel it and it's to get into that part it's uncomfortable and it makes you feel vulnerable at the time but I think that's where you make the connections and things happen and I think that's the answer there's so much we're being pushed in our field or in anything to do something what are you doing, what are you doing and sometimes you feel like you just do something so you can check the do box when all the change that I've seen happen is through transformation of individuals whether it's within the government and as Katam said talking to a government person and then if I would like shut my eyes sometimes and listen some of the military people talked with I wouldn't have known that they were compared to a politician educated or not because when they are talking from their human side and not their job side you really get this depth so anyway to the extent that we can focus more on that but it's difficult in this world of doing and doing quickly thank you we'd like to open it up now to the audience and I would also again repeat that everyone can participate in this including those watching the webcast via the twitter hashtag speak peace and invite you to submit questions for our panelists and the first one is from esteemed David Smock thank you very much and if you could use the microphone please I want to pick up on the points the last points that the three panelists made and emphasize what I found to be the principal value of this book I had the opportunity to read it both in manuscript form and in book form the last 10 days or so and that is the value of the stories Renata Stubner who is my long time colleague in the religion and peace building program and lived through the Bosnian war often said people in international agencies and that includes USIP don't understand what people go through and they experience these conflicts and this book tells the stories stories not for drawing generalizations not for drawing lessons out of them but just listening to what people went through and what their perspectives are on these conflicts and I think that's to me was the great value of the book and something that needs to be lifted up thank you questions questions yes please hi my name is Andrea I'm a grad student at Georgetown and I have two questions for the panelists the first one is so did you try to interview more women than men or did you ever think of the gender factor and the second one is like when you heard those stories didn't you want to do something about it like didn't you have moments when you were like I need to do something else not only write these stories like I don't know if you got these feelings and then suddenly or how do you manage how do you shut yourself down and don't feel it too much I don't know thank you thank you these are very amazing questions first of all I'm going to speak about Iraq we try to balance and we try to open up to all victims of conflicts or all those who contribute to promote rule of law in Iraq not necessarily to be victims but maybe a player so we try to reach out to as many people as we could but there is there is an imitation like people you might know but they might not be open enough to share their stories and there is other people who are very open and look at this as a huge opportunity for them to share their stories I remember one of the widows from the triangle of death she told me are you serious is this my story going to be read in Europe and America I told her that's exactly what's going to happen but there is no pressure if you want to share it with other women you have the space now if you don't then we completely respect your role on your other question what we have done and there is a struggle between you as someone who's doing the interview and someone who work with internationals who can actually do something about these these issues so what we have done at least in Iraq we communicated some of what we managed to identify as common issues and we have communicated them to our partners our civil society the civil society organizations we're working with and we have communicated some of the other identified issues to either medium level decision makers or high level decision makers and we haven't just conveyed what we managed to identify as issues at the grassroots but we also conveyed their recommendations how to navigate and finding alternative solutions at least at the short term and then think of the long term processes that has to be amended and really contribute to promoting rule of law in Iraq so that's how we we benefit from this the process of the interview to contribute to the bigger picture in Iraq I just wanted to pick up on something that David mentioned that was the biggest stress that I personally had with this book because I thought okay if we're going to be talking about it especially in walls here like USIP or elsewhere it's going to be so what's the answer what is the conclusion what are the recommendations and it's like just read the book I don't want to judge it you know because it's bigger than that and it's but so many times I feel this compelling need in Washington elsewhere to get the final point and that's why I love what Nezla just said it's about the stories and you bring that to home and sometimes that's where I just want to hold it up and say here's the answer read their stories it's not up to me to parse through or dictate but people to read and make their own judgment I bet 10 people could read it and come up with 10, 20, 30 different things and that's okay and that's what we want it's difficult to sit here and not want to you know parse through the answers when it seems like a driving force and I think if I understood correctly part of your question is also how do you deal how do you protect yourself when you're working in these kind of environments Nezla would you like to address that because you have been there to say the least huge idea I've been traumatized by those stories to be honest because as a Libyan I didn't witness within my family a lot of violence during dictatorship to be honest but I was part of the society and I heard a lot of violence and a lot of people who used to know as a neighbor or friend had been spent all their life in the prison or lost their families so just hearing that from those people with a lot of emotions and tears one of them there was like just crying shut down and he couldn't even continue the conversation and just witness all this emotion with the conversation it's really it's a lot to deal with and holding these stories in your shoulders and just thinking about it and process it it's really painful and I think as a peace builder in the field or interview we have also we need to take care about ourselves to help the others and I think this is one of the lessons I learned after this interview you need to be aware about yourself and your emotions because that will be really affected you and affect the others if you can't really handle this conversation I was like crying a lot I was like thinking about it a lot but I think learning no more about the trauma issues I think we need to have place where we can't exercise think about ourselves take care of ourselves even spiritually or do some exercise take a long walk or share your feelings with others all these type of things will be helpful to deal with traumatized issues because it's really painful and heavy Colette, did you want to add something? It's a very important question and I think sometimes we don't even realize that we're affected until something happens and you're not functioning in a certain way and I think a lot of times people do it in different ways and I think a lot of people I've worked with they do it by shutting down but then how can you be present and do the work and I remember when I was in Peru and I was in Lima and everything was fine and we were supposed to fly to Ayacucho the next day and after about five days when everyone in Lima said where are you going next I say Ayacucho and they would just look at me and after a while I started to get really anxious and dreading this trip and Ayacucho was the scene of atrocities from both the shining path and the government it's just horrific the time had not changed in 20 years and I remember right before my colleagues said come on we're going to go talk to somebody and it was a man who's Peruvian but he's like they call him the Peruvian Buddha and he studied in India and he kept asking me what are you worried about I said I'm worried about the stories why are you worrying about the stories and we walked through this whole thing he said you're just there to be present to listen and it was as if you talk about the stuff coming off your shoulders all of a sudden all the weight went off my shoulders I had to do both, I had to fix it listen and fix it and he's like who do you think you are there's a lot there some of it's ego, some of it's just being naive and once that came off he said you're just to listen and then my colleague who spoke Spanish and you're there to translate so she can listen and then we just showed up and I think that was the turning point of being able to do that not taking it on and then also doing some self-care people do different things to work through that yes please I wonder if you could talk a little bit more about how you built the relationships, how you built the trust and then also when it came into play how you provided for their safety and security and sharing, thank you Collette do you want to tell them to start I think having these interviews in Iraq was very tough because the sponsoring organization is United States Institute of Peace so we have the first part the name of the organization is the United States and when we had these interviews was like after Al Qaeda and after all the security deterioration across the country and all the damages so it was very tough because the perception and the perception of the Iraqi community against the US is completely different maybe than when we had the interviews in Libya so first I had a responsibility to build the relation with them through reliable channels so they can actually trust me and they can come to the process second I had to to provide maybe double effort to gain their trust although I was Iraqi but they cannot I mean I cannot prevent them from seeing my USIP hat and you know like in Iraq unfortunately within the employees of international organizations we've been accused to be for foreign for foreigners so it wasn't an easy thing but we went through reliable channels we went through our partnership through our partners local partners from the Iraqi civil society organizations who already well known for those victims or for those interviewees and also part of the people we have interviewed they were our partners so we already built over years we built that relation and we built that trust the good thing about USIP I mean one of the reasons why USIP has gained good credit in Iraq is that we haven't left Iraq even during the tough timing so that adds up the credibility of USIP in Iraq also the impartial role that USIP and the neutral role that USIP played in Iraq that adds up but also that was in a preparation for the interview but during the interview there was a second part of the relation building and trust building which is when you open up and when you be honest about who you are and what is the goal of this book and where these stories are going and what we're going to do with these stories and why at the first phase we reach out to those people not someone else and also you have to be open up sometimes and maybe encourage them by sharing your personal stories and that's what happened with me in many cases especially with the people who doesn't know me before so I had to open up and I have to talk about the pain I went through just as a as a local person and what difficulties I have witnessed and how I cope how I managed to cope so that was also part of my responsibility through the interview is not just to not just to pull words out of people but also to help them navigate through the sorrow and the pain they've been through so that's how we managed to do it in Iraq. Collette, did you want to talk about the building trust and what are some ways of doing that? It was similar in all the places because it was important to us to have this as a sharing experience that this wasn't us coming in to extract in some way but it had to be a sharing experience and in many situations it changed us as we write in the book there are a number of people that would be going to a country and we just say could you do an interview here can you handle this and they came out finding themselves transformed or seeing the conflict or seeing it differently so it's just very important for us to have this to be a book of stories and to be a sharing experience and to build that trust and many times especially in Libya and a few other places it also meant multiple days and some folks especially in Peru and other places they processed it in a different way and so it's a different approach so you had to be very you had to go with the flow and where people were, you had to meet them where they were and in Libya there were a number of where it was days because we'd have to stop and you don't just leave and then we came back we came back and then we followed up with them and there was one in particular the youth where I was very worried after the end and I'm a mother and you know he's young and I just wanted to hug him and I'm like I don't know if that's you know what do you do in this environment because he was in so much pain but at the end of the day we started communicating by email and he just kept sending emails and I'm so thankful that I was able to say this I've never been able to do it and I got a load off my chest and it and he felt better I think that's the only time I felt okay it's okay because you're always worrying about the reaction Thank you. Any other questions? Yes? Kind of convenient because I already have the mic I'm stepping over here so I don't get a reverb from the speaker so I thought about this I know that this book took place internationally but you know now a lot of things are happening in Ferguson in Baltimore kind of violence from the people from I guess a perspective of your book what could people readers here take from your book that could maybe help them through exactly what maybe needs to be done or kind of how these kind of things can impact them on a local level I just wanted to know if you guys might be able to answer that question Call it? Say a few overarching comments and then turn over to Katam because we were just talking about this that since a lot of the challenges here we do a lot of work and there's a lot of discussion of people in the book you know police and civilians disconnect between the government and violence and demonstrations and when a lot of this is going on in our country we were getting even in Libya we were getting questions about this and wanting to know and one time we were in Libya it's when our government shut down and we were getting questions like how does that happen and what is happening there so it really caused us to reflect on our own situation here and what are the things that we've learned overseas that could be used in our own country and what do we share as people who've been through that that means we're not any different and that's a challenge everybody makes all these divisions whether it's divisions by country divisions within a country and what could we share that is more equal but I wanted to turn over to Katam because we were just talking about this and some of the things that might come from the book that will help us parse through these things actually this book wasn't meant to be for a specific country it was about the story itself it was about the experience itself it was about the lesson learned out of this experience and each story has its own contribution how can we improve things whether if we wanted to mitigate conflict or if we wanted to really contribute to policymaking or some other stories talked about tolerance and so on I mean we can go on and on the list that this book touched upon but I would say if you want to learn about the definition of rule of law then I mean everywhere in the world the book you need to read if you want to learn about countering radicalism and countering violent extremism then this book is what you need to read and if you want to actually learn about how to bridge between the community and decision making figures and how can they benefit from other experiences this is the book that you need to read this this book is not meant for specific context is not meant for specific race or specific religion or specific component of the community global wise it meant to be for everyone so people from different borders from different colors from different ethnicity and nationality and religion so that's what we need to learn how can we make experiences over Europe, over Middle East over Far East of Asia applicable to anywhere in the world then you need to to get the answer of your specific question by reading what they said about it and learning from their experience so I think we're unfortunately almost out of time and I just want to wrap it up with a few reminders and then I want to ask one last question for Colette and the reminders are for those of you who are here in the room there is a coffee tea reception just outside afterwards Colette will be here to sign books that are available for sale in the lobby and the proceeds go to the VIP to help us continue our work I also want to encourage you in the room next door there is a videotape system set up so that you can speak your piece with a 30 second reflection on what peace means to you and also again encourage you to check out the website for the book which is www.SpeakingTheirPeace.org Colette I'd like to just to wrap this up with you got a forward for this book from his Holiness the Dalai Lama how did you manage that and he talks about universal responsibility how does that tie in to these stories his forward came in and it was one of those moments because of what he was talking about the interdependence and how everyone has a universal responsibility to make peace and what was interesting is which what Nezla just said at the end of the day it's not just the government to make it right it's not just one group to make it right at the end of the day it's everybody and everybody has responsibility in picking up what you mentioned Viola what can somebody do when the big decisions are made at the higher level and so what he was impressing upon everybody no matter who you are where you sit what your role is everybody can play a part and it's when everybody plays that part then things can begin to change and then the other part was just how interdependent because it's in everyone's best interest to take their role because what happens this world is becoming so global there's things that we know there's impact because of the way the world is working now that what happens in Iraq affects us what happens here affects Iraq what happens in Libya so even if in the old days as he says you wanted to be separate you can't anymore so it's in everyone's interest so he was kind of looking at it in both ways Thank you very much unfortunately we are out of time and we invite you to continue the conversation afterwards both on the website on Twitter with the hashtag speak peace and on the website for the book Thank you very much


I would love to know what your most memorable or funniest moment from making it was. Stepping into the tomb for the first time. I grew up loving adventure films and like Indiana Jones and The Mummy and read a lot about Egyptian mythology. Me and actually Walton Goggins, my colleague, had spoken so much about that and the nice thing is that we didn't use that much green screen. Instead we kind of had all these incredible build sets. So we kind of were just like jumping down. We had a very difficult time being serious the first day because we just walked around these big rooms and looked at all the details. So what are you actually seeing on set when you go into those two? They actually used the far end of the studio so it felt like a huge cave. It looked like it was rocks going up all the way and you could like go down big hallways. It was this, you know, we could find sarcophagus. You could find inscriptions on the wall and paintings.  So yeah, I was pretty blown away by the detail work that I put in. Were you able to steal anything? Steal anything, mementos. Any art defects that I brought back home? No, sadly. I did bring the clapper with me from set as a little token. That's a good souvenir. Yeah, I thought so. I'll put it, you know, next to my TV or something. It looked a little bit cool. The film sees you doing some incredible action stunts. I would love to know what your fitness training routine was like. Yeah, it was a lot. Even I was like not really aware what needs to be put in because I wanted to. I had looked at a lot of athletes, women and all the stunt girls that I've met in the film industry of the last few years. And I was like, now when after meeting the director, they want a much more realistic tone to this film. I really wanted to be plausible that Lara can do these things. So I was like, I need to have more of a physique as these women have. And I kind of looked up to them and I met my trainer, Magnus, and he helped me put on about 10, 12 pounds of muscle. And Sandy goes away quickly again, but it was a lot about what you eat. You need to eat a lot. I trained every morning and I did like archery, climbing, MMA, boxing. Did that for about three, four months leading up before shoot and then throughout the shoot as well. You're not keeping up the MMA then. The boxing, I had a incredibly cool girl called Rose who trained me quite a lot. The rock climbing too, that I really enjoyed. I even went and bought myself a pair of climbing shoes that I've been enjoying using a few times since I finished the job. So yeah.


All right. Thank you everybody for joining today's webinar on launching your career as an AIML product manager. I'm super excited to be able to talk about the subject with you through product school. And thank you so much for having me on today as a speaker. Please feel free to use a chat along the way to ask any questions that you have. I'd be happy to get back to them at the end of the discussion. So without further ado, I'd like to give a brief intro about myself and then walk you through what we're going to be talking about today. So a little bit about me. My name is Chin Mai. I'm currently an AIML product manager at Microsoft within the Teams team on the engineering side. And I focus on making features intelligent and improving the quality of experience that we provide teams users with using AI and machine learning. Previously, I held roles at J&J, Shopify in the Ontario public sector and at Microsoft as well. So just a quick walkthrough of the agenda and the topics that we're going to be covering today. The first one is just a refresher on what the typical PM skills are. We're going to review the foundations of being a strong, proud person. Next is understanding what AIML is and where it fits into the bigger domain. The next is the skills required for an AI or ML product role and how it differs from maybe a traditional product role. Fourth is the responsibilities of an AIML product manager. And again, doing a little bit of a compare and contrast between typical product and what AIML product means or is. And then finally, the future of the AIML product domain. Really, what will this space look like in the coming time? What are the areas that people should be focusing on? You as professionals, I'm guessing are attending this webinar because you know how nascent the space is and how much growth there is still yet to come. So super excited to talk about this. So just going into the refresher. At the end of the day, the first thing that's important before thinking about being a good AI or machine learning product manager is learning how to be a good product manager first. The ML techniques are the add on the things that you develop over time and you can bring data science skills in but at the same time being customer obsessed and understanding the skill set of what makes a good product person. Even from a stated AI product manager or published by the product led alliance, everyone that was surveyed over index on being a good PM first. So what does being a good PM mean? Like what are the core skills that differentiate one PM from another, or maybe not so good from a great. So there's a couple of topics. One is translating those unmet needs into customer or crisp technical requirements. Writing both communication skills in both verbal and written are key to succeeding as a product manager. Because you're always talking with two stakeholders communicating things. Always explaining concepts that are inside your head to others and being able to do that as succinct and Chris play is extremely important. Second is understanding and empathizing with customer pain and customer obsession and these are terms that I'm sure you've heard before, but really what problem are we trying to solve. Why does it matter and what's the pain point that we're trying to address for the customer. And is this actually something that they need to have solved or is this something that it would be a nice to have, but it's not really solving the core problems. Always asking and having an internal dialogue with that. The third is balancing the use of data with product, customer and product, marketing, customer intuition to drive decisions. And where I say product, marketing, customer is so important because really it's at the nexus of all three where you have to influence others to be able to have them follow you in your vision to get what you want. Or what is in your mind done as a product manager. Fourth is running effective meetings and this is one of the things that people don't think about when they think of our product people. But running, making sure that whatever meeting you set up with whoever stakeholder is involved knows why they're there. And there's a very clear agenda provided at the onset of the meeting so everybody comes out with clear outcomes. There's so many meetings and in fact there's a project that I'll take you through that I'm working on today. Is how many there are so many meetings that we have in our day to day that just don't have a purpose for being there. Meeting shouldn't be there for meetings sake. Finally is influencing the authority. This one is a key item that you'll see over and over and over again, well, whether it's in public blog posts or whether it's in historical books about the art of product management. As a product owner, you do not have direct authority over any of the teams that you work with. They will follow you because it makes sense or your plan makes sense but not because you can tell them to do so. And that's a key skill to learn because it in fact, as an AML PM this only ratchets up and it becomes even more important because you're working with even more diverse stakeholders. So people are going to, I don't want to take away from this conversation to be go focus on data science skills and just focus on that alone. The customer centricity portion is extremely important. Again, from that same report that I mentioned earlier, 84% of all survey respondents said yes to customer centricity being the first thing to focus on. The other and the know are saying that yes, customer success and centricity is important, but it's not the only thing in that machine learning and understanding how the ML product lifecycle works is also key to success. But as I can, as you're seeing, there's still a huge margin of people that believe understanding your customer's pain points and building a solution from that versus just using the latest and greatest model isn't going to really cut it. So let's take a look at how understanding how AI fits into the broader world today. And I want to start us off with a quote, and I'll be using some of these really from AI ML product leaders along the way. And Peter Skomorowch, I'm not sure if I'm pronouncing his name right, but he really is an authority in the AI and product ML space. And one of his quotes that I really enjoyed was companies understand how to apply machine learning will be best positioned to scale and win their respective markets over the next decade. But the key word I want to highlight here is that understand how to apply machine learning and that too to solve customer problems in a new and novel way that haven't been done before. So where does AI sit in? Why even talk about AI right now? Why are we here? So this really goes back to the fourth industrial revolution model. And because we're past the steam electricity and now we're actually even past the computing in revolution where all the apps that needed to really get built, the core apps that we use today have been built. There's a ton of data flowing from them. And that data is now useful. It's extremely useful. It's what people refer to as the new oil. And I take a little bit of, I have a little bit of a qualms of that statement, but I can go into that a little bit later. But really, the next revolution is how do you make intelligent products and features for customers? The things that we expect from things like Google's autocomplete or the auto sentence generation that we see in Gmail. These are becoming de facto. It's not enough to just ship the next big iPhone because even that's just iterative. Really, it's about how do you make an iPhone or maybe how do you make the next hardware device? Or how do you make the next piece of software that you're going to ship more intelligent than the previous version? And by intelligence, I mean, how do you predict what the user is going to need and adjust the software to those requirements on the fly? That's really where the magic really happens. And so AI is cool. And it is a little buzz wordy right now, but I want to break that down here. And artificial intelligence really breaks down into, there's a lot of different areas, but the two areas that will be the focus of this discussion are machine learning and deep learning. So within the artificial intelligence broad spectrum, it enables machines to mimic human behavior. And that definition really is encompassing two areas. One is artificial narrow intelligence, which we are in today, and artificial general intelligence, which is what you see in movies like iRobot or what is described as a machine taking over and making autonomous decisions. And we're at this point in time, at this very time, we're not there yet, but we're getting there. But that's a topic for another day. Machine learning is the subset of an AI of an artificial narrow intelligence AI technique, which uses statistical methods to enable machines to improve with experience. And that is really the focus of a lot of the machine learning use case or the AI use cases that are being discussed in companies today. There's a lot of structured data and some of these terms may be new to you. So I'm happy to go into them and chat if you'd like. But structured data are all the things that you see like an Excel spreadsheet or a table or in databases, and it's organized with columns and rows. Unstructured data are things like audio, video, and text, and things of that nature where you don't have a specific row or column associated with that kind of data point. And finally, within machine learning, there's a subset called deep learning. And that's the thing that kind of astounds most people because that's where you're building neural networks.  And yes, there is some element of truth to that. But truly, it's really a series of layers and activators. And happy to go into that again in the chat, prefer some sense of links out for people to learn more. But I don't want to go too deeply into the theory of deep learning here. But just know that it's used to take unstructured data. Let's say even as a classifier, say you might have noticed that in Silicon Valley, there was a classifier that one of the characters built to say hot dog or no hot dog. And that's really how that's the most simplest use of deep learning is to classify images and text in real life. So what going straight into what is the ML product development lifecycle look like? So scoping is where the team comes together to understand the problems that they face with respect to the AI products vision and try to forecast the effort that will be involved. Experimentation is around like any other product development lifecycle, the team will need to build a prototype or an MVP. But instead of building a bare bones functional product, the team will need to select the right approach to utilize the data that they have. Or choose a mathematical model that seems to show the best outcomes. Around training, the bulk of the work in this phase is with shaping the data required to train a pre-trained model. This might be a surprise to many of you, but an AI ML team's job is not to reinvent the wheel with a brand new model. We're often taking pre-trained models, shaping the data that fits into them and then trying to get the best outcome possible. Finally, serving. So now is the moment of truth. The model will be tested in production with real data flowing into it. Many things could happen, but primarily they fall into two buckets. One is either the model provides the result it's supposed to, the confidence level similar to what the team saw during the training phase, or it doesn't fit because of the differences between the real world data that the model is getting and the training data that's used. If not, the team will need to go back to the drawing board and I'll go into some of these terms now. So what are the basics of building an ML ready data set? So data set is think of it as really 90 to 95% of the work required for any AI machine learning project. And this is true of any data science project as well. Classical machine learning has two types of data sets. One is has pre-categorized or numerical target values. And I'll go into this later and or the data is not labeled in any way. And that further breaks down into supervised or unsupervised learning. Supervised is when you have a label available for every row or record in your data set. So you can either predict the category to say, let's say I want, I have qualities about a sock and I want to divide them by color. Or I want to predict the number to say, hey, given all of these real estate indicators, I wanted to predict the pricing of this specific house or row or record of that house. Unsupervised is broken down into clustering, dimensional reduction or association. And I don't want to go again too deeply into some of these things, but this is a high level primer of what things that you should look for as you're beginning to learn more about the subject area. So there's a couple of terms that you might see along the way that I want to simplify for you. ML and statistics are really one and the same. The concepts and the fundamentals come from statistics for machine learning. And in statistics, some of the things that you might have heard are dependent response or output variable, and that's the label or target. That's the thing that you are trying to predict. Independent, explanatory or input variables are input data used to make the prediction. So those are, if you take a real estate housing example, all the factors that influence pricing would be the input data. So that is what machine learning engineers or people in the ML space refer to as a feature. Data transformation is cleaning and reshaping and preparing the data to make sure that it's usable. Data is dirty and that's why I took issue with data as in the oil because there's so much effort that goes into cleaning, understanding and basically cutting the data. And a lot of the data that we have, a vast majority of it isn't actually useful on day one for any machine learning tasks. So that is what we refer to as feature engineering. And finally, there's variable or subset selection. Not every variable matters equally. There's correlation of variable to targets, so of a feature to targets, and that's why using these terms. So you want to pick the data that's most useful given the computational constraints that you have or even the timing constraints that you have. So that's something that we refer to as feature selection. So I want to make sure that everybody understands what a good ML ready data set looks like and what in this case is supervised data set. So think of it as your typical CSV or Excel file. You have rows and observations from top to down. You have a target variable, so a target outcome and that's what we refer to as supervised learning. Where in this case, we are trying to predict that what grade the student will get based on all of these different features that are available to us like instructor, lab available, teaching assistant, overall GPA, gender and prerequisite grade. Now, the model will determine what features matter most in predicting that grade and that's ultimately what we want to do with this data set here. Now, you can't really go forward in AI ML without understanding the CRISP-DM framework and this is really the bread and butter of modern data analysts, data scientists and AI ML product managers. AI projects require a feedback loop in both the product development process and the AI products themselves because AI products are inherently research based. Experimentation and iterative development are necessary. Unlike traditional software development in which inputs and results are often deterministic, the AI development lifecycle is probabilistic. We don't actually know what will happen. This requires several important modifications to show how projects are set up and executed regardless of the project management framework. The data understanding takes a huge portion of any AI ML product lifecycle. But completing it and completing it well is one of the strongest indicators of future success. A product manager often needs to balance the investment of resources against the risks of moving forward without fully understanding the data that's available to them. Acquiring data is also extremely difficult, especially in regulated industries. Once we get the data and once it's obtained, we as AI product managers aren't the ones maybe doing the research ourselves. But our goal is to be able to guide data scientists, analysts and domain experts towards evaluating the data in a product-centric way and to inform meaningful experiment design. The goal is to make sure that you have a measurable signal for what data exists, the solid insights into that data's relevance and a clear vision of where to concentrate your efforts in designing features. Data wrangling in feature engineering is the next thing, is one of the most difficult and important phases of every AI project. It's generally accepted that during a typical product development lifecycle, 80% of a data scientist's time is spent in feature engineering. Trends in tools in auto ML and deep learning have really helped in reducing the time, skills and effort involved, but not the actual product itself. Building a superior feature pipeline or model architecture will always be worthwhile. AI product managers should make sure that project plans account for time, effort and people need it. Finally, the modeling phase is where people get most excited because that's where you put the data in and you get a result and it's magic. But it's actually very frustrating and sometimes very difficult to predict what you will actually get from the data that you're feeding it. It's iterative. You're often as an AI ML PM and often the team that you're working with will go in and out between the data understanding and modeling and that will have to churn a bit between the two to make sure that the outcome that we want, the prediction that we want, makes sense for the users that we want to ship it to. Finally, unlike traditional software engineering products, AI product managers have to be involved in the build process. Eng managers are usually responsible for making sure that all the components of a software product are properly compiled to binaries and organizing build scripts, but many DevOps processes make that part easier, but they were developed for traditional software products. These tools do not exist today for ML AI product managers and product teams. The skills needed and the tools needed need time to mature and they're usually high-touched and they're very custom to each project. Finally, monitoring is a huge part because the prediction that you train the model with may not be the prediction that you get with real-world data and there's often a huge disparity between what you think the world is and what really it is. It's a reality versus training data. Having alerts set up in the system to inform you when the model's results are drifting past a margin of error are extremely important for allowing site reliability teams or engineers to be able to take action on that model and make changes. That's where you as an AIML PM are really the subject matter and domain expert for your model that you've shipped and deployed. Just quickly recapping ML product development breaks down very much as a typical PM job with a couple of differences really in the design piece. So understanding what data you have, what data is available, what you need to get and then figuring out what the model's outputs, design and proposed UI for those predicted results will be. And ultimately it's not just you ship the model and that's done. You have to continuously build a pipeline to monitor the impact of what the model is doing in the real world and how you're going to adjust for changes on the fly. So what are the skills required then? How does this all translate into you as a professional learning, breaking into the world of AIML PMing? If you're already in it, this is an amazing slide. Like this is a great deck for you. But if you're not in it, what do you need to do? What are the gaps? So really it's understanding how ML integrates into products as a whole. And I think Clemens really summed this up well where he says, but while ML grows more important, few PMs know how to integrate it into their own products. So skills that matter most. So once you have the base level or fundamental understanding of how PMing happens, the next thing is really having a working knowledge of how machine learning development and deployment works and having hands-on experience with data and modeling technology. So having built a model from scratch from start to finish, having a predicted result, understanding what the steps are in the intermediate is going to give you an intuition for the things that your data scientists teams are struggling with. As a software engineer or as an AIML product manager, your job is not to build the models themselves, but be an authority or an expert on how they work at least. It's like asking a software engineer or asking a regular product manager to understand what their product is and does. Our product is the data that goes into it and the models that come out of it. So there are a couple of ML systems that I want to go through very briefly at a very high level. But often on an effort to time scale, improving existing features takes the least effort in time, even in an ML way. And of course, creating a new product is on the upper end then to the right. So there's a couple of different systems that are often you'll see that are integrated into the most popular products that you use today. One is the recommender ranking systems that provide a list, a rank list of documents from a corpus, something like these documents can be anything from apps in the app store to movies on Netflix. It can even be your favorite item prediction on Amazon or any e-commerce site of your choice. Event or action prediction models predict the likelihood of an event or user action. For example, at Google, machine learning is used to predict a click. So then showing you videos that you're actually likely to click. Classification models are classify arbitrary objects into known classes like categorizing emails as spam or not spam. Generative models generate output in a form similar to the input that they were trained with, like translation models that turn text in one language to text into another language. Finally, clustering is a form of unsupervised learning that you saw earlier in which similar objects are grouped together like segmenting users into different groups. So on the thing that people sometimes miss and don't realize until they're actually on the job is how important the integration path is for your model into the product itself. Where is the prediction going to be displayed? What is the actual outcome that you want for users? And how are they going to make it actionable? So in the improving an existing feature, really, if you're trying to build a better recommendation system, ultimately your goal is to make sure that the items that are recommended in your shopping cart, for example, in Amazon are more relevant to users. Enabling new features. Google implemented a reverse image search product or feature that allows you to, instead of searching with text, actually copy and paste an image into the Google search bar and then it'll show you results from that. Now that's something brand new because that means interpreting an image, turning it to text, querying it, and then displaying the results to users. And finally, an example of enabling a new product is providing insights, for example, in Microsoft Teams through Viva Insights to enterprise users about what trends or patterns they're seeing in their organization, and then making recommendations on things that they can change to improve the culture. And that can be everything from productivity to how meetings are done, to employee burnout. And so now you're enabling a new product line that never existed. So a small case study I want to take you through, and this is something a personal project that I've been working on at Microsoft during my time, is trying to understand how we can make meetings more effective and inclusive. Microsoft Teams, we use the telemetry that we collect from each call to predict whether a meeting was effective or ineffective based on what's happening in real time in the call. We're not listening to the conversations, but we're using all the surrounding data that we get to understand what changes can an organizer or an organization make to improve meeting culture in their respective domain. So we first identified product improvements that we can make. We then understand what signals we should use to train the model. These signals are the circles that you're seeing, and these are the signals that we're actually getting from each team's meeting. Then we're understanding how to define the experience itself and what will be surfaced. So for example, using Teams or an admin page to surface the insights that we're going to build on this. Next is helping to analyze the outcomes of different models. And then the fifth step is planning and managing a launch strategy. So asking questions like of myself and of the team should be launched internally or should we launch to a subset of users? What is our test group going to be? Is our data set generalizable? Are the insights that we have, are we confident that these insights are going to have the intended impact of moving meeting effectiveness from the left to the right and actually making an impact there? And how are we going to be measured? How are we going to measure that? And this is where the whole metrics aspect comes right back in. The core key and fundamental skills are still as valid in this project as there are in any other. And finally, you have to sanity check the recommendations. We have to ensure that the model actually yields good results on the test data that we're getting it, giving it. This feature should only trigger when the models perform and the confidence is quite high. We don't want to start showing insights or showing behaviors or showing actions that organizers can take without really having clarity or confidence that the insights will work as we expect them to. So how do you get ready for an AI or machine learning PM interview? And this is the question that most people have. It breaks down into three areas. One is product sense, which I'm pretty sure you're aware of. The second is statistics. And this is where it kind of differs because no traditional PM really focuses so heavily on statistics. But having an intuition helps a ton, especially with understanding P values, confidence intervals, hypothesis testing, and sampling techniques. And finally, setting success metrics. This becomes even more paramount. You have to understand what the model's output is in production and be able to say and conclude if leadership teams come to you and say, hey, is this model working as we intended? You have to think broadly. And MISI is a great framework to do that. A mutually exclusive, collectively exhausted. I'll post a link to that framework in the chat later today. And of course, mapping user actions to metrics and providing context on why the metrics matter. All of these are really the building blocks for any strong performance, strong performing product person interview. So I've gone a lot into what the background is and how, yeah, it's in and what the life is, but what are the actual role and responsibilities as an AI and MLPM? So I think to remember, always in the back of your mind, is that AI or machine learning is a hammer. It is not the hammer that should be used for every nail. It's only one of many tools in your toolkit. And it's not good or bad. It's not a magic solution. If and only if so, if it seems like the best tool to solve the problem, then it should be used. So the things that matter most on the job are really bridging the language of data science and product development and having the ability to solve and prioritize user problems. It's funny because the ability to deliver AI powered specifications to data science teams are important and they're one of the top three, but they're not the most important. A lot of, again, goes back to being a very strong written and verbal communicator and even more so in such a technical domain. Key responsibilities usually include deciding on what the problem you're trying to solve is and for whom it matters. Always understanding, evaluating the data pipelines and ensuring that they're maintained throughout the AI product lifecycle, orchestrating cross-functional teams from teams like data engineering, research science, data science, machine learning engineering, and software engineering. So it's a pretty exhaustive list. Next is deciding on the key interfaces and designs and user interfaces and experiences of feature engineering. Next is actually integrating the model and the server architecture of the existing software products. Another one is working with ML engineers and data scientists on tech, stack design, and decision-making. The seventh is shipping the AI product and managing after the release. And finally is coordinating with engineering, infrastructure, and site reliability teams to ensure that all shipped features can be supported at scale. This is extremely key because we need to ensure alignment at every step of the process. So common challenges that you're likely to face will almost always be poor data quality and lack of data availability, which is ironic because apparently we have so much data anyway, but why is that so problematic? Because a lot of the data that's been collected through legacy systems isn't usable for AI and machine learning systems. Garbage in, garbage out. You're going to hear this over and over again. But as a data as a data and AI and machine learning PM, you are now empowered to make sure that whatever features you build, the data that's being collected from them is cleaned at the time of collection or as clean as possible. You don't need to collect data that is just going to sit there and not actually provide any value for additional features that you want to build in the future. So what's the future of AI? And I'm going to touch on this very briefly. And I think Pete does a great job of telling that story is at the end of the day, we want to ensure that AI systems are simple. We don't want to increase complexity. We want to increase simplicity. And it takes something special to be able to say, I'm going to actually take something that seems very nuanced like AI and machine learning and dumb it down and build a feature that people love to use because it's so smart. So AI product managers, you might have seen the UX tech business then diagram that was floating around for many years, but really it's changing over to data and AI fluency, buy-in and alignment, and use case familiarity. There's a couple of themes that are going to come up, like bots using for communicating product launches or recording standard meetings or prompting PMs to assess the performance of changes weeks after launch. And a lot of this will be accomplished by the things that we're seeing and the models that we're deploying to production will have bots and services enabled to ensure that we're always getting alerts back on how well our bottle is doing over time and if it's being adjusted for data drops. Transparency is another key theme that's going to come up. Customers aren't happy with being told that there's some sort of black box model that's going to take this data and output this result. They want to know how it's being done and really the result that it's putting out should be trusted. And finally, consumer psychology. Product managers need to understand that their end users care differently about they have different changing habits and AI and machine learning can help capture those trends in a more generic way. So the coming time there's a couple of themes that I also want to discuss here. So today we're focusing on automation, communication and collaboration in the different areas of decision intelligence, everyday AI, AI engineering, autonomous systems and responsible AI. But what customers again care most about is are the results being shown to me relevant to me in context? Can I trust the results that are being shown? And are there experiences between who's building product, the teams that are working on making sure that it's running the way it should be and the actual output that any customers or any end user is seeing? End users are no longer happy with knowing that hey, my data is going somewhere, it's being stored in some server halfway across the world and it's now being used by this giant tech conglomerate to make decisions or to sell me things that I really didn't really need and I didn't really want or push ideas that can be disruptive. So transparency is going to be a key theme in every AI or ML feature or product that is going to be built in the coming time. Thank you so much for your time today. Please feel free to use the chat to continue the conversation. I'll be there and happy to answer any questions. Thank you so much.