Great, so welcome everyone, thanks for joining us in the interest of time and being strategically scheduled before many an important match today. We are going to get started and hopefully keep to time for all of us. I'm Nicole Goldin, I'm the director of the Youth Prosperity and Security Initiative here at CSIS, which we launched a year and a half ago in partnership with the International Youth Foundation. And I am very delighted to be joined today by a three very distinguished guests. We have Paul Teeple, who is the director of Aganar and Sports for Development at Partners for the Americas. You have their full or lengthly impressive bios on stage. Of course, to his right, we have Brianna Scurri, we're delighted to have with us two-time Gold Medal Olympic champion, World Cup champion, among other impressive talents, and is also currently a sports envoy with the U.S. Department of State, which we'll hear more about, I'm sure. And to her right, we have, delighted to have my friend Owista IU, who is the director of South Asia programs at Seeds of Peace, and also the author, which she'll tell us more about of her experiences in the Kabul Girl Soccer Club. Like many of you, I don't know how our guests had to drink several cups of tea last night in order to get my voice back in tact for today's session. But I'm very glad that you're all here to talk about the serious side of youth and soccer. You know, we're having this conversation today amidst the World Cup, of course. If you weren't aware, it's also Olympic Day, June 23rd, so keep that in mind. But, you know, more seriously, we're also having this event amidst the global youth unemployment crisis or challenge, depending on how you look at it, you know, recurring and flaring conflicts and crises around the world, and really seeing the rise of soccer and sport more generally as a platform, as a tool in diplomacy and promoting peace and importantly in sort of youth development and skills generation. So those are some of the things we're hoping to scratch the surface of. As always, we'll probably ask more questions and we'll answer. But with that, let's pick it off. And I'm going to ask the three of you the same question, and I assume we're probably going to hear three slightly different answers, which is why should we think seriously about youth and soccer? Paul, I'm going to go to you first. Okay, great. First of all, thank you for inviting me and thank you all for being here today. Why should we think about youth and soccer? It's passion, and young people are passionate and want to be passionate about what they do and what they learn. There are so many good things that we can learn from soccer. And let's just take last night's game as disappointed as it was for many of us. The U.S. suffered a terrible lapse in concentration, which led to a goal. We need to learn for us, which Aganar, as we like to say, is a youth employment program wrapped inside of a soccer ball. We focus on youth employment and we use the lessons of soccer to help young people learn the job skills that they need, both the life skills, employability skills and the hard skills that they need to get a job from lessons from the field, and both the interactive activities and classroom activities. So you have to concentrate on what you do. You have to dig deep within yourself and find a way to come back. You have to play the full 90 or 95 minutes. You can't take off a minute before the horn blows or the whistle blows. And you have to be ready for that one moment when the light shines on you that you have to step up, and she knows better than I do, she's the expert about stepping up on the greatest stage in the world. These are things that young people, we have a tremendous dropout program, a dropout problem in Latin America and the Caribbean. And I'm looking at some statistics in Guatemala and in Honduras. If you look at the 20, the poorest 20% of the population, less than 10% of those people finish, young people finish high school. And 40% say that they drop out because they're bored, because they don't find school worthwhile or interesting. So we need to make education interesting. And what's more interesting than soccer, what's brought us all here today, what got us all dressed in our national colors for the last two weeks. It's our love of soccer and these things that unite us. And so what I think we can do is use soccer as this uniting force and build on this excitement. But then once we peel it back, we can see that there are many, many, many lessons that from playing soccer, from just trying soccer. A young girl for the first time who learns to receive a ball and to make a pass. What that means in her life and what that gives her the confidence to do in the rest of her life. There's just so many things that we can build on that make it relevant for us. That's great, and a great segue to a list I'm gonna come to you next. Tell us why we should think seriously about youth and soccer. Sure, and I'll answer that question just kind of giving the brief history of women's soccer in Afghanistan, something that I saw from non-existing ten years ago to being a very strong program today. So for those of you who don't know, I started a girl soccer program about ten years ago for a group of Afghan girls, actually ten years ago this month. I brought a girl soccer team from Kabul to the states. And it was an idea that I actually thought of many years before that, being an Afghan national, wanting to be back to a homeland that I loved so much and I grew up watching Brianna and Julie Fowdy and Mia Hamm win many World Cups and many gold medals. And much to their credit and much to the support of Title IX. For me, soccer as an African-American girl was very much viewed as a gender neutral sport within my mindset to see a girl playing soccer. For me, it was no different than seeing a boy playing soccer. And I think a sport like ice hockey, something that I did play, is very different and I couldn't say that for ice hockey, for example. And so when starting this team, I was very naive. I had a very American mindset, didn't really understand the culture and the context of what soccer meant for a young girl in Afghanistan to play. And when I was recruiting girls for this first camp that I was holding, I couldn't find girls in Kabul who were playing soccer. And I really didn't question it. I was 23, 24 at the time, very naive and very idealistic. And I really didn't understand at that time why I couldn't find these girls. So the girls came here, they were trained, they went back. And it wasn't until they went home that I really understood what I'd done. And I naively ended up putting these young girls in a position of power, but also vulnerability. Because they were the only girls who were playing soccer in Afghanistan at that time, not because there was no youth soccer program after 30 years of war, which is what I attributed that to. But because girls just didn't play soccer. And what I thought of as being a very gender neutral sport with my American context was far from the case. In the case of Afghanistan and many other countries since then that I've worked in, whether it was Morocco or India, Pakistan as well. And it was really then that I understood the politics that soccer entails. And I think it's important to understand that idea. I think it's an American we take for granted sometimes that we've had world championship women's teams that really have broken that stereotype. And we don't think twice about what it means to be a girl playing a sport. But in a country like Afghanistan it means a lot. And so when these girls went back they were really up against men, even who were sometimes three times their age who had never seen a girl kick a ball. And they were finally speaking up against it, not because they wanted to really push the gender norms for any reason. Because other than the fact that they love the sport, that they came to really understand and feel passionate about. And so they ended up really igniting these conversations off the pitch too. It wasn't just about the right to play soccer. But what it meant in the context of Afghanistan when women's rights was so hampered for the past 20 years before that date. And so I think soccer and sports in general when it comes to a country like that says a lot about what it means for these girls to push that barrier. And just to give you some ideas of what issues they were struggling against and the expectations that were put on them. But I think soccer really ended up having them challenge some things when it came to their private life and their public life. What it means to be a young girl at home who's responsible for errands and what it means to be a girl who would rather not do those errands and play soccer for an hour. And a lot of times there were battles at home really between parents and brothers and between siblings. Whether it was another girl too who really couldn't understand and value why their sister wanted to play soccer within this family. The question of marriage. Marriage is still very much at a young age for these young girls. And so the Federation, even when I was in Kabul told me why should we invest in these girls when by the time they turn 18 or 20 they'll get married. And we've just spent four years investing time, energy and resources to them rather than on the men's team. And so that was certainly something that they're up against all the time. Again, that leads to a length of access to the sport and travel. When these girls started traveling around the world later on, what it meant for them to be a single girl who was traveling with a team, even if it was other young girls, what it meant to travel away from your family. And lastly, what was interesting is gender roles when it came to sexuality. I remember one time in Morocco being in a meeting with a coach. And he actually was quoting back scriptures of the Quran. Because there was rumors that girls who were playing sports and specifically soccer were considered lesbians within this community. And so this coach was really nervous about what that meant for this team that he was representing, but also that the girls are representing. And I found that to be very fascinating. I certainly understand the context of why he was doing that. I might not agree with it necessarily, but as a Muslim also, I really understood what that meant in a culture that's so conservative to have these questions of sexuality come up. And I'll just leave you with a few points and then we could leave on. But I think what ended up happening with these girls was pretty phenomenal for a number of reasons. And it wasn't just these young girls alone that were really carrying that torch. But this program that started ten years ago with no girls were playing soccer has now grown into a national football team that the Federation supports. Because they finally understood the value of having these girls play. FIFA certainly also played a role by mandating that a women's team be formed in any country that now is FIFA supported, similar to our title nine mandate. But I also saw Skatistan and these other programs for young girls, whether it's cricket, just really blossom in the past five, six years alone. So I think these girls, I think it's safe to say really ended up spearheading this effort. And I'll just leave a few, I think, points in terms of just what I think helped become a driver for that. And I think part of it is advocacy. I think whether it was the girls or other officials and adults who really became advocates for them. I think awareness really also was a big driving force to make that social shift happen. Success, certainly, once the girls are becoming successful, whether it was nationally or internationally, I think then men and others in the community were able to finally justify why there was a soccer team or any sports team. Access, whether it was the lack of access in the beginning and then increased access as a result of that success also became a major driving point to help the sport or other sports stick. And lastly, funding. FIFA did and does still support the women's team through their own funding in order to really help promote sports for young girls in Afghanistan and to promote soccer. And so I think all these driving factors combined really helped make this initiative that I started as a very naive, you know, 23-year-old into something that really shaped the landscape for women's sports 10 years later in Afghanistan at least. Thank you. So lots to come back to already between Paul and Uesta. But before that, Brianna, so we've heard about the skills piece we've heard about, you know, the promoting rights awareness within the South, within the community as a player, both, you know, growing up here in the U.S. And as I said, competing globally. What are your thoughts on why should we be taking this seriously? Well, my answer is in two parts. Let me first address the part about youth and soccer. The one thing that's common of the sport of soccer that is uncommon with most of the sport is the countries other than the U.S., it's part of their culture. It's part of the fabric of that country, whether it's the boys that are playing and girls aren't playing or both are playing. You can go to Guatemala and you see soccer played there.  You see soccer played there. You go to Europe. There's somebody playing soccer. And that's part of the culture. The sport of soccer is part of the culture internationally. And we're actually U.S. are a bit late to the party. So we're running as fast as we can to literally catch up with the skill level of every other country. And that's why the sport of soccer is so great as a symbol and as an introduction because our country and all these other countries have that in common. We have an understanding of the game of soccer. Therefore, we have somewhat of an understanding with each other. That's where the common ground is. So when you go to other countries that may not have had baseball before, you have to teach them baseball. Go to other countries that didn't have American football and you have to teach them American football. Whereas in soccer, you go over there, we can all kick around whether we're from Russia or from Argentina or Canada and all have an understanding of each other without even knowing each other. So that's the beauty of soccer in the international community and in the U.S. In terms of the spread and who's playing and who's not in Afghanistan in particular, being someone who started with the women's national team when I was 23, played in the 1995 Women's World Cup. I didn't know as someone who was growing up in soccer, playing in high school and college, that there was even a women's national team and I was in the sport. And I didn't find out that there was a women's national team that there was a women's world cup until 1993, two years after it had occurred. I mean, Michelle Akers, Julie Fowdy, Mia Hamm all played and won the 1991 Women's World Cup in China and I'd been someone in the sport at a high level, close to possibly being able to make the national team had no idea, no idea. So Michelle Akers tells a great story. She said, we went over to China and played in the Women's World Cup in one and the only person that agreed us when we got back to the airport was the equipment manager to come pick up the gear. I mean, the Federation didn't really acknowledge it because they were so focused on the men's side and just the response of the community and society of the US had no idea that these women had done this amazing thing. And then 1995, Women's World Cup, it was a little bit better, you know, a little bit more visible. We were playing in Sweden and we ended up getting third, unfortunately, that time. A few more people started to notice and then in 1996, Olympic Games, women's soccer, the first time women's soccer was in the Olympics, we went full medal the first time. Usually there's a period where there's an experimental period and then the next Olympic Games, you're full medal. We went full medal and played in front of 76,000 people in Athens, Georgia. That's when people started to sit up and take notice, but as a women's national team, we had to fight for two massage therapists. We had to fight for childcare. I mean, because we are, you know, the athletes and the women that are having the children that are expected to have one type of societal role to having to fight and let the federation know and everybody else know that yes, we can play a sport professionally and still be moms and do both well. And so Joy Fawcett, Karla Overbeck were two of the very first higher visible women that played a professional sport and had children at the same time and brought their kids to Portugal and Italy and Canada and Asia and were mothers on the pitch and off the pitch. And so we had to fight. And I think everything that is great that becomes something bigger and everybody understands that it's more important than just them, you have to claw and you have to fight for these things. And you see the end product like the 1999 World Cup, Rose Bowl, 90,000 people, holy cow, this is amazing. That took a lot of work leading up to that point. Took people deciding to get in there and make a difference and change something in order to get to that point. And so that's the beauty of soccer but our women's national team who had already won a World Cup, we had to fight as well. But coming back to that, I mean, like you said, it's about the fight and it's also about the individual. I mean, to what Paul was saying about some of those skills that you gain growing up. So in the same day for a second, we're reflecting back to when you're in growing up as a player, whether it's, I'm sure, from time you were small, but as you've gone into sort of high school and even beyond, how did sort of playing the sport, how do you think it affected whether it's your work in school or your ability to then kind of be on this international stage and be successful on the pitch as well as off? Right, well for me, I always gravitated towards the team sports when I was younger. I didn't start playing soccer till I was 12 which is like way late for a lot of the kids nowadays. And my family didn't have a lot of money. So I knew that in order to even go to college eventually, I was gonna have to earn a scholarship. And fortunately, I was gifted athletically because of my parents and they were also incredibly supportive. And the things that I learned playing soccer, I mean teamwork, commitment, leadership, these are all things that have been woven into me and I've learned and become part of me through the sport of soccer, but they also become a part of someone that plays basketball as well. But I think the key thing for me was I was given an opportunity. I was given a chance and a choice to play a sport and express myself that way. And that wouldn't have happened necessarily without other people coming before me and saying you know what, this is important for young girls to have this opportunity and I was grateful to be able to have it. Paul, and we're throughout Latin America and the Caribbean and I'm sure many of the participants in your programs similar to the Afghanistan or other South Asia contacts are facing adversity in one way or another. How do you get them engaged? What are some of the particular challenges that you face in not only bringing both girls and boys into the program but keeping them there, having them be successful sort of in the long run and then implementing the program in general? What are some of the sort of risks you see? Well it's a couple of interesting things. When we started this program nine years ago, one of the things people said, well what about girls? Girls don't play soccer, what about girls? And we said just give us a chance, girls will show up. Because we knew from our own experience here that girls just wanted an opportunity to play and if you, so one of the things you need to do is make sure you have a safe space and that girls know that they're welcome within your program.  This is pretty cool, I didn't do so well in school, but I think I can stick with this. And so there was a hook related, but the hook is also the job and I think that's important for us to remember. The things that make it tough as we go through, we work in some of the most violent communities in the world, we work in some dangerous communities in San Pedro Sula Honduras, which is the most violent city in the world right now and it's not in a war zone. And so young people have to walk different paths every day to get to class. They need to pair up in twos and threes and fours. Young girls face a lot of, they have to walk through a gauntlet of comments and they have to convince their parents that if they're gonna be in a soccer program, they're not gonna become a lesbian, that it's just a lot of things, a lot of things that they go through and then when they play, they get some of the boys aren't so kind to them at first. And so we have to work on having this safe environment and having strong women in the program who show the way and work with the young men to also demonstrate respect back to the girls. And what we found a part of that is that, there's a real deficit in opportunities for girls, but young boys who are the people in Latin America who are killing themselves at the highest rates really need to learn from girls and really need this practice. And so we play a game where you have to hold hands with your teammate. And just think of all of the implications of a boy and a girl holding hands and the snickering that can go on for a thousand different reasons. And so, but what does it mean? They learn, they learn. And sometimes the girl is better and it really shows up the guy and that really upsets them and it's just, it's great, it's great. But are sometimes the guy won't pass the ball to the girl and somebody says, hey, you know, she's your teammate. Why aren't you being respectful to her? Or she's afraid of the ball and what she learns over time is that, oh, I touched it, I completed the pass. And that becomes a metaphor for something bigger that she can do in life. And so, the violence in Latin America is so extreme now that I think that's still our big challenge and young people who just, they're dealing with violence in the homes and violence in the communities and they cross gang lines to get there. But what I think we need to remember is that in spite of this, in spite of all the bad things that people say about young people, there are tremendous young people out there and tremendous young people who dropped out of school in the second and fourth and eighth grade and tremendous young people who have suffered all sorts of hell. And they're smart, their academic achievement is not an indication of their intelligence and they, you know, they just want a chance and when you give them a ball and you give them a chance, they can, they need to learn themselves, they have to be able to understand that they have hope and that they can aspire for something else. I talked to someone last week about the immigration crisis. Now, and the young people come to the United States and he said that one of the big problems with this crisis and the gang problem is that young people do not believe that they can hope, that they do not see that there's a better future. So we have to use soccer to do this and scoring a goal is like, you can do this, what else can we do? You talked about scoring a goal and it made me start thinking a little bit about, you know, what is the success in soccer programs? Because what I'm hearing is it's not only about what the young men, young women, girls and boys get out of it themselves, but for say a community in Afghanistan or one of the other countries you're working in, what does it mean from a community perspective and how do we, how are we and how should we be measuring success? Yeah, I mean I think Afghanistan is so early on in the phase of sports and development so I think success really is just about numbers rather than it is about any kind of championships one. I think the U.S. is very different. We know that's how we measure success, but I think it's really about just getting the numbers up there. I think, you know, Paul talked about some of the issues that he faced with getting girls on the pitch and you know, in Afghanistan it really oftentimes is just having conversations door to door with parents and really getting the permission for their daughters to play. And so I think numbers, whenever I've talked about success it's always comes down to that. You know, and I know that's always not always the best and safest metric or measurement of success but I think that's what it really is there. And I think to be able to say that there's so many federations now that have started women's programs I think is also a benchmark of success and a positive number that you can give out that really demonstrates where the country is today versus where it was 10 years ago when no federation or maybe two federations had a women's program to now have over 20. I think that number of success. Paul says is what, getting a job? Getting a job. Getting a job. And so for us, our latest analysis that about 75% of our graduates were getting jobs within a year or going back to school. So for us, scoring a goal is getting a job going back to school or starting your own business. Brianna, when you think about the work that you're doing now, what does success to you look like in terms of young people and their own role in themselves and in their communities and societies? I think originally for me, success meant, if I can make the save, if we can win the game, if we can win the championship. But it wasn't until five, 10, 15 years after the Olympics in 96, after the Women's World Cup in 99 that I realized success meant something completely different that it meant, in fact, inspiration. So many parents and players that were 11, 12, 13 years old when we had the World Cup who were inspired by what we were doing and went on to create amazing programs or be a part of soccer program or teach their kid how to play or were just like touched somewhere in their bodies and their souls by what we were doing and all we were doing was just trying to win the game. And so for me, the measurement of success is how the sport exponentially grew after our success on the pitch grew off the pitch. And like you said, all of a sudden, two federations that had a women's program, now it's 20. And I believe that there's a direct correlation with our team's success and showing that we could put people in the seats and watch a women's only event and we could seat 78,000 and 67,000 and 77,000 at these tournaments and that showed people okay, that may have taken a lot of work for the U.S. to do but they did do it. At the very least, let's try. Let's try to give girls a chance. Let's open ourselves up to giving them that opportunity not expecting necessarily for them to play for their country but giving them a chance, giving them a choice and that to me is success. So on that note, what's getting in the way? And I ask that, we're here sort of in the room, clearly the interested parties. We're here at CSIS, the development, foreign policy, obviously think tank, national security and in the global development and even arguably diplomacy context, I mean sports is to a certain degree a cute thing, a nice thing, an add-on, right? What is getting in the way of really scaling of these kinds of programs and this kind of work? I mean, you mentioned, you know, Wista in your opening comments, what you saw as being required for success and is it funding, is it political will? Is it the right environment? Is it the right partnerships with the private sector? I mean, as you're sort of out there thinking about how to do more of this, what's getting in the way? And is it, do we not know enough about what's working and how to do that? Wista, come to your first. Yeah, I mean, I think I left here Paul's answer to your mind or mine. I think I've struggled with the idea of scaling as being, especially when it comes to development, as being the only benchmark for a program is successful because I don't think scaling is always a solution. I think what I've seen in the sports and development in the realm over the past 10 years since I first got involved with someone who was very outside of that sphere is that I think it's become a very mainstream type of activity to add on to things. And I think there are those, like what Paul's doing certainly, it makes sense, but I've seen just organizations that have no sports focus at it because USAID now is supporting it. And so I think I'm nervous about saying scaling is really any kind of success within our field. And I would say maybe sometimes what's holding it back is because so many people are doing it that it's somewhat been watered down. So those who really are the experts and really know what they're doing, like Paul's organization that has 10 years worth of a history underneath their belts are not the ones who maybe have the access to the right that funding. And I know you do, but maybe they don't have the right access to the funding who are, I don't want to say worthy, but who have really demonstrated the impact of their work. And so I think there are organizations that are receiving large amounts of funding who haven't demonstrated that impact. And I think for me demonstrating impact is more important than scaling. So I know that's a very loaded response that I'd love to hear both of your perspectives on that. Paul, I'll tee you up to go next. Okay, I'll put this one, two things. One of the things when you look at scale versus quality, the quality I think of all of our programs comes down to the quality of the facilitator or the coach. And so how we find the ability to scale has to come from investing and training, supervising, guiding, and accompanying those great teachers, facilitators. Those are, they're the real heroes in our program. They're the ones who go into the communities in Ciudad Juarez and Guatemala City and the favelas in Rio. And I get to come to nice things like this, but those people are amazing. So how do we, basically, how do we replicate those people and how do we pay them? Because they can't live off of good intentions either. And so that's one thing. The other is, and so I think we need funding to scale those people, to train them. And then the other part is we need more evidence and it's a really important thing that IDV and USAID and State Department through Sports United all have offices dedicated to sport for development. And other institutions are taking it very seriously. But the evidence, and so we're working with USAID right now on an impact evaluation in Guatemala and Honduras to try to look at it and say what are the secret elements, what's the value added of sport within our model versus a very similar youth employment training program. It's really hard to do that, but a lot of people are very interested in what the results are and it's gonna take a while for us to get to those results. But our hope is that once we get to that, we're gonna be able to say to people, this is the value added and even if it's 10% or 15%, that's 10 or 15% on the lowest level investment you can ever have, a soccer ball and a piece, it doesn't even have to be a piece of grass, it's a piece of dirt, rocks. We're young people play.  These are low, the real investment needs to go back to those teachers and providing maybe a snack for the young people, but we need that evidence and for people, I like to talk about the magic of sport, but I need to be able to give numbers instead of talking about magic. I think what is important to remember is when you're a trailblazer, you're naturally gonna have more friction to go against compared to anybody that comes behind you. So I think what we're seeing right now is, honestly, the trail is now being blazed by different organizations who began 10 years ago or 15 years ago. And if you look at the history of a movement, for example, these things take decades sometimes to become what they are. For example, the NFL right now, there was a time in the 60s and the 70s where it was a hobby or just as something to do and watch. And now it's like the huge juggernaut business and it's bringing in billions of dollars in TV and fans and revenue and all that. And that took time and so will this. However, there are people that care and people that are willing to share stories and do the work and roll up the sleeves and start hacking away and blaze those trails for other people to come behind. And naturally you're gonna have some people that come behind that aren't gonna be as successful. But at least with those individuals and those organizations, you see what does and doesn't work. And maybe they had an element to their program that was good in one way and the different element that wasn't. So the one that comes behind there adopts the one that works and alters the one that didn't. And then before you know it, you have several different organizations and companies wanting to invest and funding and people are making change and literally changing the world one person at a time and giving the youth of this whole world an opportunity to let me kick the soccer ball around instead of doing something else over here that might be a detriment to that child. Just being able to have more than one choice on the table for me really is a huge difference. I am up here now today even playing on the national team because of after school activities that were available to me when I was 12. If those activities weren't there, I wouldn't be here. And it was literally a piece of paper and an opportunity that was given to me that I took hold of and ran like crazy with it. And you'll be surprised in the 10 years from now, 20 years from now, you'll hear stories of the kids that you guys helped that ended up being so much more than they thought they could be because of it. That's amazing. And before I come to you all for your questions to start getting them in your head. Branna, you mentioned talking about what works and what doesn't. And I think in the development context we're often very leery of talking about where something didn't work out so well or talking about what didn't work. So before we go to the audience, is there, you know, Paul and it was, is there any particular, Branna, if you have any additional thoughts, you know, something specific that is, you know, to be aware of something that didn't work that we sort of learned from. You know, again, we want our young people to learn from mistakes, right, as we go forward. Is there any, you know, particular thing in doing this work that we can take away is what didn't work or what did you do better? There's so many. It's hard to narrow it down. I'm gonna look over to my colleagues because they're like, well, why didn't you say this? Why didn't you say that? One of the things we're struggling with right now is we've got a couple of, most of our programs are after-school programs or you come into it voluntarily. And we're working now with some programs in schools. And so one of the things that is a challenge is that there's a difference between the profile of the young person who has been told by their teachers they need to go into the Agana program versus the young person who says, I wanna go into the Agana program. And we're needing, we're trying to figure out how to motivate those young people who've been assigned to us in the ways that the young people who come to us voluntarily are motivated. And that's a little bit of a struggle right now for us. I mean, this is a hard question for me to answer because I did make a very naive decision 10 years ago but I would say, you know, it's a standard-friendly development program whether it's sports or non-sports, just really understand the context of what you're doing. I mean, fortunately, 10 years later, I can say that it became a successful program through a very, you know, through I think a lot of different factors that just happened to coalesce. But I didn't know that it would be as successful as it became locally, even without my hand in it. So I think just understand the context then. And to be honest, had I known that soccer was so gendered as a sport, I don't know if I would have chosen it. You know, 10 years ago, I probably would have been safer and picked basketball or volleyball, which were historically female sports. So by default, I ended up really like changing the status quo and really put these girls in a position where they were changing it. But I would say just understand the context and sometimes it's not, it might not work out in this way. And maybe there are different avenues too that are safer and maybe more sustainable too. Branna, any particular sort of words of caution, so to speak, from either your own personal experience or sort of what you've seen, what you do now in thinking about soccer programming? I honestly consider myself someone who sees the glass half full.  So that's considered a bad thing as a goalkeeper, but hey, guess what, it's gonna happen. So you take it with the bad, you take the things that didn't seem to quite work out and you learn from it. And I think part of the thing that I've noticed that people tend to lose as they become adults is understanding that the mistakes are actually part of how you learn. And that's part of the path as well. And the mistakes are just as important as successes. And so now, with the two of you, starting these programs, like I said earlier, you are showing people, okay, I did this and this with my group and the next person will come behind and say, okay, I wanna take that on. And I think it's amazing that you chose soccer. I know it was probably incredibly dangerous for those girls, but I mean, they are heroes. And they may not understand that now, but some day they will. And people who come behind and other girls now probably feel, hey, me too. And just if you're able to change that mentality of a young child, you're already halfway there. Hey, me too. That's all it takes really. Well, hey, me too. I'm sure some of you have hey, me too questions. But before we do, I did mention that today is Olympic Day and Paul mentioned the work of IDV and we're very lucky to have our friend, Frederic Iglesias with us today. He is a living Olympian and currently also working at the IDV, leading many of their sports for development and programs. So I'm gonna ask him for his sort of initial kind of, you know, few comments on what he's heard and if you get to ask the first question as a fellow Olympian in the room, we've got a mic for you. Thanks. Thanks for the invite and for this wonderful panel. I just want to comment a few things. Brianna make a good point on her experience educating people in the U.S. since soccer was not a big sport now, it seems like it is. In some countries in Latin America, like Bolivia or El Salvador, we have the opposite experience. In my case, for example, I ran 800 meters and every four years during the Olympics, everybody has high expectations and Bolivia or El Salvador never got a medal in the history of both countries and several other countries in Latin America or just got one in the whole history. So we have different problems talking as myself, talking at the IDP, we just closed, we just finished a project in Bolivia, in El Alto for gender equality. And one of the components that we had in that project was the education in self-steam. I think it's a big step because if you don't believe in yourself as a person, it's difficult to reach different levels. So when you ask about the measuring success, I mean, we all have a common language today, but it's a good point that you rise on table. At the IDP, we consider monitoring and evaluation from the inception of our programs. I'm sure you consider the same, but it's important to measure that success. In this particular case in El Alto, Bolivia, we would like to increase the participation of people in their communities and maybe someday as policymakers. My last comment, we interviewed the forward of the national female team in Bolivia, a person like you, a national hero. And she said that among the 20 clubs, female clubs, there was just one only female head coach. This is something that we need to change eventually. Thank you. Thanks so much. I mean, the point about coaching, I think is an interesting one. Brianna Divany, sort of thoughts on that. And then we'll take a few more questions. Yeah, we are the national team. We've had male and female coaches and we were, Anson Dorrance was the original head coach of our team and then we had April Heinrichs and then we went to Greg Ryan and now it's Tom Sermonni and now it's back to Jill Ellis. So we've gone back and forth with that, but I think females at positions of command and power as coaches, the CEOs, as running major organizations, it's all changing. And I think once again, those things take time, but somebody's gotta be first. And sometimes that friction of being the first or the second or the third takes time. And it's time now to accelerate hopefully that change and it would be nice to see more women's head coaches either in the national team arena or in the higher level club team level as well. But I would say in the next 10 years you'll probably see a female head coach at some major club in the world at some point. Paul, just to say back to you on this one because you made the interesting, I think good analogy about between the importance of what we know of the importance of the role of teachers in education quality. Where, do you have trouble finding sort of the right mix of coaches, is there, what is the coaching gap, if you will, or is there one in this context? Well, we look for people to, we have a manual that we follow. It's a 100 hour course that focuses on teamwork, communication, discipline, respect, to focus on results and continual self-improvement. And it's a mixture of classroom and field activities. And so in some programs, depending on the size of the program, we have people who dedicate the field activities and others dedicated to classrooms, but in the small programs, it's one person who does it all. We struggled with saying, oh well, some people say, well, we gotta find soccer coaches. We gotta find people who are certified PE teachers. And we found out that those folks were not always the best ones because a soccer coach is kind of trained to be a soccer coach. And to oftentimes, their reward is a trophy. And for us, the reward is that self-esteem, that person stepping up and recognizing that they've got something, they're different. And that's sort of the person who was in the worst shape on day one, if this young boy or young girl who raises their head to ask a question after three or six weeks, that's a gollazo, you know? And so we want people who do that. So we found success finding people with social work background, with adult education background, hiring people who come from the same communities to relate to them. And what we found, like in Sudat Juarez, a couple of cases, where people just have terrible stories, backstories, and that's holding them back. And it's that facilitator from that community who works with them, who works with them. And this is the magic of the ball that makes them connect. And eventually that young person feels the, as we say in Spanish, the confianza to say what it is that's holding them back. And now they're opening it up and they can sort of start dealing with the problems and this young person can then start to blossom. But it's that. And we have a lot of training and then facilitators need to know that just like a great coach, maybe your coach can push you in one way but can't push me a ham in the same way. Because you're different people and you respond differently. And those facilitators need to understand what's going on. Or maybe this girl came to class today and I have a good, you know, the fact that she's here in a few minutes late, she had to go through hell last night at home. And so I need to, in the way I work with her or this young boy who's so angry, I have to, I'm not gonna call him out. It's a little mistake. So I did make a mistake calling out a young boy because without knowing him. And good facilitators need to know the stories of everybody so they can really motivate them. In the Muslim context, was this an issue in sort of, again, not only getting the participants, but the coaches and the right facilitators? Sure. I mean, I think there's still very few female coaches. And so just as an example, the national team in Afghanistan, the first national team was run by a male coach because he was a former national team player who knows the sport very well. And so it was the best position to coach a national team. And that did cause issues at home with the girls sometimes when their parents were very reluctant to send their daughter to the pitch. And so the way the Federation came about addressing that issue was to invite parents to practices, just at least see the interaction between the players and the coach and the nothing inappropriate was happening. So I think we, you know, many of these Muslim countries are up against that too, is that the best coach is oftentimes a male because there hasn't been infrastructure to develop a female coach through the system. But I think the federal, like at least in Afghanistan, there's been training of female coaches in order to down the road address that issue. But it is an issue today still at times. All right. Oh my gosh. All right, many a hand. We're gonna do this as we often say, well, Bankstahl, we're gonna take three questions and this woman right here with the, take three questions. Again, if you can tell us your name where you're from and please keep it brief to a question so we can really try and get as many folks' comments as we can. Thank you for excellent presentation. My name is Rogan and I represent an amazing group of girls in Tibet, in India that have formed the first Tibet Women's Football Club and they are inspired by people like you. They've come up against almost every issue you've brought up and so my question is, how do you see cultural and national identity working with or against the fight for gender equality because they are not represented by country. They are Tibetans, they don't have a country so it's very difficult for them to find people that will play them because playing them recognizes that they are Tibetan. So they play a lot of Indian teams right now but they are at the same time making huge headway in terms of the Tibetan community and gender rights. So I'd love for your comments on that. Great, thanks, Tim. Thank you, Tim Nurse for making sense. I want to follow up on the question of coaching and also how we've been talking a lot about work on the pitch but also the whole concept of soccer is so universal that it's very effective outside of the soccer field as well and so we've done a lot of work on life skills and using the whole metaphor of coaching and working on the field and how that can develop the soft skills necessary for employment. And what we find is that the coach is such a critical component in that we want to get away from trainers who are telling you to do something to the idea of a coach who will help you practice and develop skills and then be, quote, off the pitch during the game when you're out there on your job or in your employment. And so it's kind of a comment just how these concepts are really applicable outside of the soccer field as well and I think very appropriate for youth in general. But then also a question for Paul. In our work we found that the soft skills that we're developing through the life skill programs are very effective in general but in particular for more informal jobs around entrepreneurship because of the multitasking aspect and so just curious in your work are you finding that the impact is greater for self-employment in the informal economy or for formal employment? Good question. We're gonna take one more in this round. My friend right here with her handbag. Hi, my name is Lauren Key. I'm interning for RTI International. As a female soccer player, I found as I was growing up that having female role models, not even who are coaches, just older players was really helpful and just for inspiration. And I was wondering if either of your programs had like a mentorship program with possibly like a graduate of your program coming back and saying, oh look like I got a job and this is all due to the confidence I gained from this program, just to inspire the girls. Just wondering if you had any aspect of that. Great, so three good questions and we'll give you a chance to answer one or all of them.  Okay, so on life skills, our numbers on entrepreneurship are really small compared to getting a job or going back to school. And I think it's because we've been focused mostly in urban areas and our training has been mostly focused on learning a skill for a formal job. Yeah, but I totally agree with you that those same skills, teamwork, communication, a focus on results are tremendous entrepreneurial skills. And I think we have some more rural programs that have a higher emphasis on entrepreneurship. And, but we're not seeing that the skills are any better or any worse, that they're universally good, however we use them. And to what you said about this, we had a program in Brazil called Vinsadoras. It was specifically for girls and one of the most powerful things that happened was that girls who went through the program, a number of them served as interns. And when girls would call in and say, can I apply and tell me about this program? And when a girl answered the phone and said, I went through the program, you could just feel the tension leave the young girl who's calling in. And they're saying, yes, you can do it. I did it and here I am. And I'll come talk to your mom or whatever. And it's, you're absolutely right that those examples of other young people make a huge difference. Was there any thoughts on sort of cultural national identity and or sort of the mentoring aspects of the program? At least the mentoring aspect, I think what really helped in Afghanistan was the media and really them being able to showcase female athletes. And I noticed these are very anecdotal stories, but you know, I would see some times where girls, which helped to the practices we were holding with the Federation and would recognize some of the other national team players because they were featured on TV. So I think the media can really play a huge factor in that in countries that are developing new programs. In terms of the cultural and national identity, I'd love to talk to you more, maybe offline, but just, I just have a few questions in terms of being able to answer that. I lived in India actually for three years. I was working for Cease of Peace and still am as director of South Asia programs. I know that the Tibet issues always really a very sensitive topic so I'm not really well versed able to answer it, I was a 30 now and then to have a go on non-record. But yeah, I'd love to ask you a few more questions and we'll talk a bit later with you. Brian, any thoughts of mentoring? Older, younger players? I wanted to kind of speak to her questions. Like I said earlier, our national team, even though we had won a World Cup championship, we still had to go to the table and negotiate with the Federation on getting somewhere close to equality of what the men's team had even though we had already won a World Cup championship. And even though both teams or the Federation was receiving USOC grant money that is supposed to be used equally, we still had to fight for that stuff. Now granted, we didn't have our lives literally at stake, although you could say that our financial livelihood was in fact at stake because there was a time when we got $5 per day per diem and the men got $35 to $40 per day and we had to somehow go to the table and explain how yes, we in fact also should have $35 if you don't mind. And it seemed at the time very irrational that we would even have to try to fight for two massage therapists and these types of things. Now like I said, we never had to fight for our lives but you're fighting the good fight. What you're doing is providing an opportunity and you're going to have humongous obstacles to overcome but if you're hard in the right place which it seems to be with the program you're doing, I don't understand necessarily or know or even proclaim to know the political or national implications of what you're trying to do and how that affects the girls and you but it seems that every time you have to begin something that changes, there's gonna be that friction and it's gonna be that resistance and it's gonna be literally people that think it's ridiculous what you're doing and as long as you think it's good and can find a way and have people who maybe have come through to help you figure out how to go and to guide you, then you'll get there eventually and if not you then somebody behind you will. So keep up, keep it up. I'm gonna take two more quickly. I'm the woman right here. Again, if we can just keep it brief. Good afternoon, ladies and gentlemen. Nico, thank you so much for the youth event and launching the youth initiative. Thank you so much for your presentation. My name is Rosemary Seguero. I'm the president of Hope for Tomorrow. We focus on young people and women in here locally here in DC but I'm initially from Kenya. Then my question is how do you look at, first I was part of Prestendopama Initiative. I did the review for the young people. It was a great review but something funny we never had sports and entertainment, sports looking at soccer. How do you look at the young people especially young girls and boys who are with disability like Tef. My son is Tef and he's a very good soccer player. We have young girls in Kenya who are Tef but they are very wonderful soccer players. How do you include them into these programs of soccer and why they are good players and how do we work with you in this effect of soccer? The other thing, when is the next women soccer, the Wild Cup Soccer so that we can all attend again. Thank you so much. Thank you all so much and just a quick note, Brianna, you said that success is inspiration. You've inspired me and many others and you continue so thank you for that. Huge supporter of women's soccer. Our season ticket holder, The Washington Spirit. They could use more fans. Nice. Thank you. Quick question about the coaching program. I wonder if, so I'm Suzanne Petroni. I'm with the International Center for Research on Women. I wonder if you specifically tackle gender norms in the curricula for coaches and facilitators. It seems that there's a tremendous opportunity, particularly given the gender challenges in both Central America and Afghanistan. We at ICRW have a program in India that specifically works with coaches as mentors to change gender norms and cricket in India. Be happy to share some of that with you. We'd love to hear more. All right, we're gonna take those two on gender norms in coaching. I don't know if you have any thoughts on that and also on any thoughts on how to include disabilities in sports programs. I'll take the gender norm question. I think for us, yes, we do address issues of gender and we talk a lot with our facilitators about, first of all, establishing a safe and equal place, trying to recognize where boys come from and where girls come from and having a respectful environment while we're there. I think we can approve it as you ask the question. I'm thinking to myself, yeah, we could probably go a lot deeper into it. We have a five day training of trainers and it's packed. People are always complaining that we go from eight in the morning to about eight at night and they're worn out and we need to, so I'm not sure how we do that any longer without spending a whole lot more money. But it's really important and our activities are important. So again, to go back to holding hands with your teammate and what you learn or playing a game with no rules or we talked to our facilitators about how you need, we give them the flexibility to change the rules as the game goes along, as the game goes along in order to balance the field. So that if boys aren't passing the ball to girls, then put a rule in that you can't pass it from a boy to a boy or the next goal has to be scored by a girl or only a pair that's holding a boy girl pair or a girl girl pair that's holding hands can score a goal, force this. But it's not always the boys, it's sometimes the girls who need to speak up and they need to learn that they have the right and the confidence to claim their space. And so our trainers, facilitators need to work within those subtleties. I mean, in terms of disability, I think in sport, I think unfortunately sometimes when you have a country that's still on the development stage, it becomes a very secondary point of focus for a federation when you're first working with the men's team and then maybe have a women's division. I think that the needs are very different, they're drastically different sometimes and the resources might not be there to really support it properly. So I think I've seen not as much support in that realm, in the countries that I've worked in at least. So I really can't speak to what's been done in some others to really develop it, but at least that's what I've seen within the countries that I work in because when you have an Olympic athlete or a pro-Olympic athlete who may or may not be elite, I mean they just require different equipment that might cost thousands of dollars that a federation can't really support. I mean in terms of just the question about gender and coaches I know I don't think there's a really considered effort from what I've seen in Lake and Afghanistan to really address that specifically. What I can't speak to is I know Magic Bus for example a friend of mine has done a lot of work with them and I'm happy to put you in touch with him but he's done efforts, like made efforts to really change the rules towards being able to address it. So one thing that he told me about that he's done with Magic Bus in India is have like a magic minute with soccer is that boys who are passing to the girls they're applying mixed and so what they do they did is institute a new rule that said there's a magic minute, you don't know when it is during this game but if a girl scores your goal is actually worth three versus one. If a boy scores it's worth one but then they were just seeing exponentially that girls were having access to the ball a lot more because the boys were like we have no idea when the minute's gonna be or we can get two extra points. So I think there's ways of doing that and obviously not at the elite level when you can't change people's rules but when you have youth programs should really help address that and I'm happy to put you in touch with him because he does a lot of great work in terms of really changing and adapting rules to address certain issues. Okay, one, I'm sorry, Brianna, go ahead. Just addressing gender equality in soccer with coaching I could probably say maybe at least half of the players that have come through on the national team either assisted coach or head coach of a major collegiate soccer team in this country. Carla Overbeck is the assistant coach at Duke. April Heinrichs came through the national team. She was a head coach at Maryland and University of Virginia. Jill Ellis who is now the head coach of the US national team was the coach of UCLA and there's several other players that have either come through the college ranks and played at a really high level or were in the national team pool or play on the national team that now became coaches so these things take time but it does happen. You need to be able to train these women to understand the game of soccer but then also to be able to relate to students. Another great example is Karen Gabara. She is the head coach at Navy and has been for I think 13 or 14 years now and she came up through the national team and has been coaching at that university for a very long time and her role is multifaceted because she's training kids to go on to military careers and trains them as individuals and as an athlete and also as someone who fights for freedom and she does incredibly well there. So I mean it can happen and it is happening. There's several examples of female coaches in the college ranks. It's beginning to really start to impact and it is happening. One other comment just on the disabled that small scale kind of more community level examples are where differently abled participants are brought into non-player roles but as part of the team or whether it's as a manager or are brought into a program to kind of support the team another way and sort of grow with the team and participate in some of the other aspects off the field so to speak that they do the team. So I've seen that work in some at the community and local level but with that I hope you'll please join me in thanking Paul, Brianna and Owissa as well as for Drake for his thoughtful comments and all of you for your great questions. Thanks for joining us today. We'll look forward to seeing you again another event soon.  So good luck, thanks again.


How do cryptocurrency charts work? Reading cryptocurrency charts is like checking the weather conditions before heading out. It gives you insights into the market climate for smarter decision-making. Think of each line and candle on the chart as a weather indicator like humidity, wind, and temperature. Just as a weather forecast guides your choice of clothing, technical analysts use cryptocurrency charts to predict potential market movements and make informed decisions. While it won't predict the future with absolute certainty, understanding crypto charts empowers traders to navigate the financial climate, much like preparing for different weather scenarios. Think of cryptocurrency charts as your financial weather forecast, essential for a successful journey in the market. Click the link in the comments to learn more about


We have started Olivia, your meeting. Hi, everybody. Welcome for this new Jenkins infrastructure meeting. We have a few incident to report, so we definitely have to be totaled today. So first, let me share the notes for today. As last week, we are going to use ICMD. So here are the news. If you cannot edit the news, feel free to request access. So let's start. The first thing that I want to talk is the incident that happened yesterday. We get the Jenkins that I own. So as a reminder, get the Jenkins that I own. It's a mirror engine that redirects every request to download an artifact from the Jenkins to a mirror located to the closest mirror from your location, which means that from Europe, you're redirected from to Europe, mirrors, and so on. What happened around 6, around 5 PM UTC? For some reason, the network storage, so the Azure file storage, mounted into that mirror bit service stopped responding. We got error, say, quota exceeded. And so because we could not communicate with the network file storage, basically mirror bits had no idea about which file can be distributed, can be distributed to which mirror. So that's basically what happened. So the fallback, the current fallback, the way it was configured is if that get the Jenkins that I was not working, it fallback to a service that was using the same network file storage. So basically, we just sent way too much request to the Azure file storage, which was problematic at the time. So it took us around two hours, two, three hours to understand the issue. The good thing is that was the same issue that affected us back in November 2020. So we had a rough idea of what to do, where to look at. Several people were involved. That outage, so the first step was to redirect the traffic to a node machine that we named package, Jenkins, and IO, which has every file. So that machine has the same content that what is located on the Azure file storage. So the idea was just to redirect the traffic to a different machine. So we could put that service back and track until we understand what happened. So the service was done. So yesterday evening, Europe time, everything was fine. Yes, sorry. So the redirect was that redirect of get.jankins.io traffic? So it was a DNS change to switch from? Yeah, exactly. So we didn't fix the get.jankins.io. We just redirected traffic to a location that was working. The temporary solution was fine, but that machine is not able to handle the loads that happened the week hours. So it was definitely just a temporary solution until we understand what was happening. So on my side, what I did yesterday was to open a connection using PowerShell with the Azure account to list every open file connection with that Azure file storage. And there is a hard limit to 2,000. And what I identify is there was in one session, we almost filled that limit. So we opened like 1,998 connection around, I'm not sure, Damian, because we investigated with Damian this afternoon around 4 PM, I think, UTC. Yes, 4 PM UTC. It's when we started to see the first peaks. And so once the Azure file storage was not working correctly, then we saw a lot of side issues like CPU usage when crazy inside the nodes, memory increased. Obviously, because the service was not able to answer requests, the number of requests increased as well. So we can clearly identify huge peaks that happened during that time. Until now, we could not identify the root cause of, I mean, who basically opened those files? Was it an issue on Azure? Was it an issue on our side? Did we have one process that just opened every connection in one time? Yeah, it is not something that we could identify right now. We have identified a private IP, but we are missing some observability data to be able to know, because the IP is one of the Kubernetes nodes, and we don't know. We are missing information to conclude, did we cause the Azure file system issue by making too much request, or was it an Azure issue that caused the request to pile and to go and raise condition on whatever application and keeping the file under open? And this we cannot conclude on one or the other, and we miss data to be able to conclude. So that, yeah, that was the main issue. Any question on this? No, so if we are still on that specific issue, I think what was really nice to notice based on our learning back in November, first, we put in place a status page in November. So we could communicate about this issue, and people were able to quickly open the incident on the Jenkins infraslash status kit repository. So we were able to communicate about the incident when it starts, when it closed. That was the first thing, first thing, sorry. Some people ask why the monitoring and the status page did not show that the Jenkins that I was on. The root cause of that is because the way the container is working, the service starts and read in the directory. So read the net or the Azure file storage content into a directory. So the service was working, but could not read the data in a specific directory. So our int check, because we only monitor the route, so get the Jenkins that I you, our int check told us that the service was up and running. But when we try to access a specific file, obviously that was not working. So that was the thing. So we have to improve on monitoring to monitor a specific file. It says slash time slash Jenkins or whatever of file we monitor. Ideally, that file need to be small so we don't put pressure on the network. But we have to improve the monitoring. Something that we saw in November and we saw the same pattern this time was some requests were passing and others not. So we were able to download some specific files, but other were returning five or three hours. In November, we had the same issue where we could access every files except those under the directory please. I didn't understand why at that time the problem resolved by itself. And we saw the same issue yesterday. Some requests were passing, others not. Get the Jenkins that I you was up and running, but so yeah, that's one of the thing. So yeah, I'm not sure what we'll do is we'll just improve the monitoring to monitor a specific file, but this will just reduce, I mean, this will just help us to detect the issue. But in this case, we didn't get the monitoring notification. The second thing that we monitor, that we check was since now while, we monitor that we can download the latest Jenkins version from package at Jenkins.io. So we have a monitoring check for that. And how it works, we query, we put the Jenkins.org to see what's the latest version for the weekly data test. We relieve the retrieved that version and then we query get the Jenkins.io. If the data check is failing for 30 minutes, then it trigger alerts and then we are notified by page activity. What happened here is because the service was not failing for two hours, some requests were passing and others not. So when we look at the metadata dashboard, we clearly see that's half of the request was were working and others not. So that's why we didn't get notified by the monitoring. So we had a look to think about things here, but we could not really, I mean, yeah, that was a tough issue to detect. So what I have to, another thing that we have to improve here, I clean up the open connection. So I just had to connect with Azure console and run a bunch of comments. Obviously in my case, that was easy because the comments that are run back in November were still in my Porsche history. So I just re-execute the same comments, but I should put the documentation in a runbook. So the next time, if it happened again, someone else can do the same. So this is something that we work with Damien to be sure that someone else can run the same comment again in the future. But yeah, any question? So if you have, yeah, I just put here the kind of access that are done on that network storage just to give you an idea. Why you can mount the same network, the same Azure file storage into multiple containers? You can write from multiple containers, you can read from multiple containers. You still have a limit on the number of open files you can have at the same time. And just to give you an idea, we have monitoring check that tests if you can access some specific location on the container. You have the Apache who have an Apache that can return content. We have mirror bits which scan on a regular basis from every containers. We have data doc monitoring we have. So that's, it's really difficult to really have to have a clear understanding of where those access, those file access came from. But yeah, we are still investigating. The next topic that I want to mention, the next outage, I mean, that was not really an outage. So any question before we move on? No, another issue that happened last week. So we wanted to improve the way we deliver check-ins at IO website by directly to only rely on home charts. And we face an interesting challenge here. We put branch protection on the key trip repository that contains check-ins at IO website. So we put branch protection. So we always use pull request to introduce change. And because the new workflow implies committing to the branch, we could not identify a way to say we want to keep the brand protection but only allow a specific bot to modify that specific file.  One of them was to just remove the branch protection. So now we just allow the right person to directly commit to master. Or there were also suggestions about keeping the branch protection but automatically open a pull request and automatically close that pull request. So I think I have a proposal here because there were at least six different way of implementing that workflow based on changing some bits, connecting with the meeting. This means that we don't have a consensus right now. So that could be a nice ground to start writing an EOP, which is same thing as the DOP. It's something we haven't done since a long time. So infrastructure enhancement proposal where the goal will be to state the goals, list the solution with each point and cons and then discuss based on that just to be sure that everything is clear for everyone and have a specific meeting and decision. And then we can jump to implementation because it appears that the two tries at implementing it were missing a tiny parts where we discover that maybe we have to go in a consensus or to act somewhere. So that's my proposal here that we start a specific discussion with a written process first, where we list the solution just to avoid a risk of a meeting where we might not understand or see all the parts. What do you think? I would be really happy to work on that documents because four years ago, I work on the same for the current implementation and the current way to deploy things on our websites. And I mean, every other website like Shavadawkin API and PluginSense on and in four years, a lot of things evolve. So yeah, I would be really glad to re-evaluate my assumption that we're done four years ago to propose something different. It was mentioned that maybe IOP could be merged into the GEP. I don't really care. The goal is that we get started on writing the proposal there. If we see that we can do that exercise as a community team for one or two important topic, then we can raise the discussion of should we go to GEP? But the goal is to learn to work before running here. So that's the goal of the proposal of staying on the IOP that hasn't been updated since a lot of time. So let's see how we behave as a community team. And then we can then see if we have to merge to GEP where I will say the writers and readers of GEP are more at ease with that process than we are today. That sounds a good idea. And I would even go one step further since we are working again on documentation git repository to put a lot of documentation. I'm just wondering if we could just put the IOP document there as well. So we have just regroup that the IOP document, the meeting notes, outage, maintenance, and document and so on in one location. Olivia, I'm not sure I understood the last thing that you said, could you? So the idea was place the IOP in a different location than the earlier IOP repository. Yeah. Close to the, okay. So right now we have, so we have a git repository named documentation that was created a long time ago. And the idea was to have a public documentation where we document everything related to infrastructure. Last week we had a discussion in this meeting about should we put ACME documents in Jenkins LA website or in a different location. We agreed that we would push in a different git repository, which is the documentation. And so since we collect the notes for the meetings and the upgrade plan and every other things, including one books, I was just suggesting to move the IOP documents in the same git repository. So we just have a bigger repository with more content. That sounds great to me. Okay. Thanks for the clarity. I'm going to move the front slash documentation. Sounds good. Those were the two most important of the peak that I wanted to talk today. Damien, you were mentioning that you wanted to bring the website to the peak and the issues we have with release at CI in front of CI. So I guess that's the right time. Yep. So as I mentioned before, the recordings were mentioning it for everyone. Yesterday during the incident, while we were investigating the Azure console elements, we saw an alert on the Kubernetes, which was only warning when we run the diagnostic and troubleshoot integrated on the Azure UI. And that warning was about the fact that the Kubernetes control plane of our IKS cluster named public gates is thresholds. That means that we are making too much requests at some times. So I'm having a bad moment with language. Natural threshold is the correct word. It's throttled. That's with a T. So we are throttled. So that means that some of our requests are queued before being sent to avoid having big workload on the API control plane. And these requests come from different sources, mostly all our N file process, which takes care of doing the GitOps operation to the Kubernetes cluster, but also from all the Jenkins instances that are spawning pods. Because when a pipeline is running inside a pod template, it run WebSocket command from a Kubernetes client inside Jenkins to run the kubectl exec command in charge of the pipeline steps SH or B80, depending on Linux or Windows. So these requests are also being sent directly to the Kubernetes API control plane. So we don't know who the culprit is, but this will explain the WebSocket issue we see because WebSocket not only is used between Jenkins and its agents, but also between Jenkins and container that are not the default GNLP container or the given pod when the pipeline is run. For each SH, there is a kubectl exec, which involves one WebSocket. And most of the errors we saw were mostly related to this and are correlated in time with the peaks. So we will have to push this further, maybe monitoring the amount of requests or the peak of builds. I don't know which direction to go from there. That's also something we saw yesterday. And that's all I don't want to have. We need more data to prove that it's related to that. What monitoring can we get out of Azure? Can we ever to see when we're exceeding the thresholds or the quotas on the last? Yeah, there is some monitoring currently integrated in Azure. So we could maybe start with this point. I don't know how to extract that information continuously from Jenkins though. There are way to have a groovy script to run that will print instantaneous usage of the current open connection from Jenkins. However, I saw on the graphs, it looks like that Azure is able to determine the user agent of each request because I saw some requests that were coming from my own web browser with the operating system, the web browser and the kind of client. So maybe that one could help because I suppose the user agent of the Kubernetes client in Jenkins is different than a web browser or a M file apply from a go along. What would be also nice is to identify all the potential limits that we have to use AKS. Because that's a common issue in Azure. Whatever service you use, you have the limits. Like the number of CPU you can deploy in one region and stuff like that. The number of files you can open on the Azure file storage. At the same time. And so maybe we have a limit that we need to identify. And also that could be worth it to check with Jenkins and Jenkins 6 communities and maybe Jenkins user or any variation because I'm sure we are not alone. I mean, we don't make so much request. So is it because Azure AKS is meeting some issues? Is it because of the Kubernetes version? Do we have other user with the same issues on those are Kubernetes kind cluster? Because I mean, it's not uncommon. And we don't do anything that is exotic. We run pipelines on pods and the pod template might have two or sometimes three containers. That's not a lot. So even though there are technical limitation that's also subject from the user experience in Jenkins when using the Kubernetes plugin. Yeah, I'm wondering whether, so I know on Jenkins X, there's a recommendation to turn down Kubernetes external secrets from Poland because that can be quite heavy in terms of listing secrets and looking for changes. But I don't think we're running that on that, thinking of that intro cluster. My, I'm guessing that it's probably all related to the fact that, the fact that we run a full deploy or a full kind of shell. Maybe, maybe, so often. Maybe it'd be nice to reach out to our elastic friends because I know they work on Jenkins observability, so. Yeah, and so right now the first step will be what I'd say the checking the existing monitoring on Azure and see the breakdown between different clients and see which one is emitting a bunch of peaks. And then from there we could go further. It's interesting to investigate and that maybe Damien, you will have to look at, you will get something. Interested. Anyone interested to pair with me is welcome from the team or from outside. Yep, you're right. I can tell you. Sounds great. So one quick add-on. In the action, I move on the action points the run box, I'm gonna polish the writing. The goal is to write run box about to fix some elements that could have helped us yesterday during the incident. The goal is we tried as much as possible to not wake up Olivier who already had the knowledge from the previous one. So we partly failed and partly succeeded because team was able to provide a fallback solution. So I've identified at least two procedures that must be written at all costs. How to fall, what team and Mark did yesterday how to have a temporarily fallback to be sure that user can still download file for one full day slower without the mirror capability but still they can download. And how to identify and fix the Azure file storage. Identify we were able to follow Microsoft online procedure but we were missing some point about PowerShell script that Olivier mentioned earlier. This being should be a run book.  Only the main dots we can still do ourselves during an incident, the line between the dots but we need, there was some missing element that could have helped and maybe could have helped the fixing without bothering Olivier. So now that we all have the knowledge and understanding of what happened because since it happened already in November that's the second time. That means we will have another issue with Azure file storage. So if we have a run book for that, it's a no brainer. Anyone from the team can fix it and that should also shrunk the time of the issue. Sounds perfect. And again, we'll put those, those do run book in the Jenkins for documentation, we'll be starting. Is it worth one more action item to investigate or suggest or discuss ways to detect that style of failure? The failure mode was rather strange in terms of flapping and, you know, it was on again, off again, some here, some there. I'm not sure that failure detection is ultimately possible for that kind of failure but is it worth? So. Yeah. To be honest, identifying this issue that happened yesterday is definitely challenging because as you said, as you said, that was flapping. When we look at the monitoring, sometimes the check were passing, sometimes that's an even worse. We have 400 gigabytes of files. Some files were accessible, others not. And in November, those files were under the director plugins. This time it was not necessarily the same. What do you think Olivier may be putting, I don't know if it's technically possible but the amount of file under open on Azure file storage where a pretty good indicator that something went wrong because it's the quota we reached. And once the quota was reached, then everything got out. So do you think it's possible to have a routine that only that's a measure at least the amount of open file under on that specific volume. So if it reached, let's say it's 2000. So if it reached, let's say 8600, then we have an alert that say, okay, maybe you should check and with the run book with the process. So a human still have to operate but the human would not have to go through the which is the issue on the Kubernetes implication. We know that the issue was specifically on the Azure file storage there. And not on the Azure. So first thing first, everything is possible depending on which time we want to invest in there. Practically, I'm not sure if it's worthwhile because Python, the checks are published to DataDoc. They're written in Python. I don't have any documentation. So we could probably use the Python SDK for that. But those are definitely not information. I don't think those information are available as is in DataDoc. And so we definitely have to collect those and publish them to DataDoc ourselves. Would you be okay, Olivier? I think there's some interest there. If I ask DataDoc, do they already have a built-in monitor somewhere that would check Azure file storage issues? Because I would expect this to be a common thing they've already implemented. And all we would need to do is find out how they, what they did. So we definitely have storage information. Storage, file counts. I have to do a check. I mean, we have information like the math, egress, egress, and stuff like that. But we have the latency, but we don't have the information that... Yeah, I have to look at. Okay, I have to... Thanks. So we are running out of time. So I propose to finish the meeting here. But before we do, I just want to highlight the fact that because of the new workflow, we are going to have one document permitting. So I put at the end of this document the link to the next week meeting. So if you have anything you want to put to the agenda, feel free to add that information there. And so yeah, we use the next document next week. Thanks for your time. Have a great day. Bye-bye.