uth to local youth so what are we saying are we saying that radio can actually change people's lives can it actually have an impact on people well let's have a look at children as a result of the Boko Haram insurgency as you know thousands hundreds of thousands of children left without schooling but not only were they left without education but they were displaced but there was no possibility of building new schools for them or training new teachers for them to be taught so had nothing been done there would have been a whole generation of uneducated youth who would be sitting targets as Boko Haram recruits so one solution was to educate these children over the radio a simple solution easy to implement and one which can cover large areas and it could be done so without major construction costs and without to teacher training costs it could also be implemented very quickly so this is a project that was run by the American University of Nigeria and USAID and they used radio and mobile technology to raise the appeal of western education this project taught basic english literacy and numeracy and it used songs and radio dramas in the local language using characters that little children could relate to and they were also synchronized there we go they're also synchronized activity workbooks and they encouraged children to engage with the songs and the stories that they were listening to and so radio was used to encourage children to imagine to learn and to engage but these these programs these lessons more importantly were broadcast by a relatively new radio station Dandalkura which broadcast in the late Chad area where as we were hearing hearing before Boko Haram had a monopoly to spread its extremist narrative there was little competition for credible information and yet this is a gap that's now been filled filled by radio by this radio station this education program was broadcast over 18 months and over 20 000 children who otherwise would not have been reached have now received basic education and this is through the radio the project was so popular and successful that it was continued for a further six months by another local radio station after the initial funding had had run out so how else can radio impact people's lives how else has it been used to in this fight against extremism well simply by providing information by news programs people can receive an alternative message that it enables them to question messages that they previously had been receiving give audiences programs and then bring on experts to debate that subject to debate the subject so i'm currently working on a large project with studio kalanga which is a radio station which a radio studio run by the swiss-based media development organization fondatia kondel and it is using this approach to broadcast independent information programs over it broadcasts from the capital of mejair using satellite and then it broadcasts them to its network of community and commercial radio stations who's then rebroadcast them using fm networks so it then brings on experts moderate religious leaders community leaders people in positions of trust who can be believable to discuss this to debate the subject so what is importantly though is that these programs are broadcast multiple times in multiple languages throughout the country which means that these alternative messages can reach the population what's more we're using the prevailing radio culture to take this further the culture of listening clubs is widespread in many many countries and this is they enable the ordinary person to take part in public life people will sit around an ordinary radio set outside their houses in the streets and wherever they drink tea and they discuss the radio program they discuss it together it's a socializing medium and if it is used and if radio is used and used well to broadcast positive messages if it is targeted correctly in the correct language to the correct people it can make a difference it can have a ripple effect in mejair for example which is affected by the conflict which is spilling over its borders studio kalangu and my own project we're trying to work to contribute to empowering women it recognizes not only the gender inequality of this poverty struck country but it also recognizes that women play an essential role as policy shapers educators community builders activists and so on and it's an important role not just in developing the country but in fighting extremism as well because it's widely accepted that the capacity of women to spot and react to extremism to react to extremism in their families ranges greatly depending on levels of education social awareness and geographic remoteness those who don't have as much formal education struggle to recognize warning signs they may perceive changes in their children as simply then becoming more religious and this may be considered a good change so an obvious solution is to make and keep women informed and listening groups are an ideal solution for this so that this isn't quite so easy when women aren't allowed to come to mix listening groups for the very reason that they're mixed so what we did over the summer was set up a network of women's only listening groups in using community radio so community radio would broadcast independent information and then women come to these women's only associations in their hundreds and receive information discuss information about warning signs but also at the things like health education and so on and they pass on this information and the ripple effect continues and then the community builds get stronger and the resilience is to extremism is built so the methods and the techniques of using the radio continue so radio has a role an important role and one which isn't going to go away just because of the growth in the internet there's a picture of my ladies my women's only listening group but in short we need to ensure that radio and particularly community radio can thrive and can continue to push out these alternative messages to all forms to all effective groups and these are alternative messages to extremist ones so radio may be a traditional medium but it's a socializing one it's sociable and it can bring people together and can do so in a positive manner so radio is not past its prime it must be used and it mustn't be disregarded thank you thank you so much i'd like to now welcome our second speaker to the stage dr Pamina Furchow is a assistant professor of conflict analysis and resolution at george mason university her research interests include political violence transitional justice especially victim reparations reconciliation and peace building in particular she's very interested in the study of international accompaniment of local communities affected by mass violence her specific focus is on the role of concept formation in the measurement and evaluation of external interventions and how local people can be included in these processes please help me welcome to the stage dr Pamina Furchow i'm a professor at george mason university um and actually also a former senior Jennings Randolph fellow here at usip so it's it's really wonderful to be back i'm going to talk to you today about a project that i've been leading for about six years called the everyday peace indicators project and this approach was developed by myself and my colleague Roger McGinty from Durham University now in the uk in order to address our concerns about measurement validity and concept formation uh in traditional approaches to measuring a difficult to define concepts such as reconciliation governance peace justice and violent extremism uh the usual way to measure these kinds of concepts is to start with experts or scholars or in the cases of evaluation which some of you may be more familiar with program managers usually sitting in washington dc or london or capital city far away from the communities where the research is carried out or the evaluation is carried out many of you may be doing that yourselves and very familiar with that process so just to run through the process experts come up with their own definitions for these difficult to define concepts like peace or violent extremism in this case usually based on broad theories that are meant to be applied in many different contexts around the world and they use this theoretical definition to develop different signs or indicators to measure whether there is more or less peace or whatever social phenomenon they're studying so violent extremism for example in the places that they're studying they might use these indicators to develop a survey or then come up later with focus group questions or an interview questions these ways of measuring difficult to define concepts typically don't change from one country or another or from a region to another definitely not from a village to another for example a survey in columbia would ask the same questions in most cases in bogota as it would in in cartagena so why is this problematic well outsiders may not fully capture the concept in the way that it is being lived or understood on the ground therefore leading to concerns about measurement validity or whether or not that tool is is valid they might not fully capture what the concept means at the community level so at the village level or at the neighborhood level or reflect how the people on the ground understand it outside experts will have difficulty framing questions and indicators in a language that can be locally understood so extra processes need to be taken in order for people to understand what they're talking about so communities really are best place to identify their own indicators of the social phenomena they experience since they're already doing this in their everyday lives in other words people are already using their own everyday indicators to determine whether or not they are at peace or reconciled or there's more or less violent extremism for example so why not gather those indicators and analyze them for policy use so instead of letting experts decide what these concepts mean in different parts of the world the everyday indicators approach starts off by asking people at the village level or the neighborhood level how they define them and then it uses their answers to come up with locally specific ways of defining and measuring these concepts this process is also useful for planning and evaluation purposes of peace building related work the process that we've developed involves four key steps first we develop the indicators through a series of focus groups by asking people what signs they look to in their daily lives to determine whether they and their communities are at peace so I'm speaking to individuals about in focus group environments to really get the tangible signs what are those signs that they use in their daily lives to determine whether they are more or less at peace or whether there is more or less violent extremism for example these indicators are highly localized and contextual here are some examples of indicators from past studies so for example in Atiaki Uganda can the Boda Boda cyclist ride to certain areas as you can see it's very context specific very almost anecdotal in some cases but it's it's a local experience and this one when you dig a little bit deeper you can see that it has more layers to it because Boda Boda cyclists who in northern Uganda are the taxi drivers right they often ex-combatants and so they're plugged into post-conflict networks and clearly it makes sense that people are looking to them to decide where they can go and where they can't go right and you know we've we've done this study in other places as well in Afghanistan with USIP we did a study on violent extremism indicators where I mean the lists and lists of indicators but an interesting one that I saw was and sort of related to radio was TV antennas whether or not there were TV antennas in a village allowed people to have a better idea if there were a lot of TV antennas in a village allowed people to understand if there was more or less violent extremism in that in that particular context so as you can see very contextual again it's important to stress that what we're gathering are indicators that people are already using in their daily lives or what we call indigenous technical knowledge so people are not inventing these indicators for us nor are we asking people to you know create indicators for a particular project for example right we are sourcing indigenous technical knowledge so these are indicators that people already use in their daily lives and what we are doing is not creating new indicators we are sourcing them and then analyzing them and using them for policy guidance and then going back to the process the indicators are then verified through a participatory process that involves the community and this ensures that the indicators are as representative as possible of the community and also narrows down the list of available indicators so that we first of all have a manageable list of indicators but also you know oftentimes in focus groups there'll be someone who comes up with a one-off indicator but this process really narrows down the indicators to and also creates a list that's meaningful for the for the community and representative of the community these indicators can then be analyzed and also be coded into categories in order to help with planning and design of the projects and so the individual indicators can be analyzed or can be used for policy guidance but also we also code the indicators into categories because there can be the problem rate that you maybe are not seeing the forest for the trees and so in order to be able to or sorry the trees for the forest anyway anyway in in order to be able to see a little bit of a broader overview we can code them into categories and we usually do that through process tracing of the actual indicators so we create categories that are inductive rather than deductive but usually those categories end up shaking out to be policy areas of interest anyway and and so so then we can say for example that one area is particularly concerned with issues to do with human rights whereas another one is more interested in security or is defining peace more in terms of development or whatever so that you can zoom out a little bit because sometimes it is almost too localized right so it's important to stress that the difference between the everyday indicators and regular indicators is that the indicators themselves can already tell us a lot about what peace means to people on the ground and and therefore can be independently analyzed to guide programming so they're the indicators themselves are analyzable but then the indicators also provide the basis for questions that allow us to understand more about that concept in a community and sorry I there we go I was pressing the wrong button so the indicators are can provide the the basis for surveys but also for questions or existing data you can populate the questions with whatever data is available where we typically populate them into surveys because there isn't a lot of data available at the very local level and so one way is to survey the communities over time with the indicators in order to understand whether they are at peace according to their own indicators and how that changes over time and that's particularly also useful in evaluation the everyday peace indicators approach is concerned with rigor validity and reliability but it really also prioritizes local voices over external voices in the development and evaluation of programs and in measurement in general therefore we consider ourselves an approach to produce participatory numbers and statistics that work with communities to produce policy guidance in a language and a format that policymakers can use so the idea being that we are galvanizing communities to really help work with us to produce numbers but include them in the process of creating those measurement tools where they're usually excluded in that process they may be sources of information but they're excluded in the process and so we're including them into that process recognizing that policymakers often have a need for numbers so if you'd like to learn more about the project we have a website which you can check out everydaypeaceindicators.org and you can also pick up my new book on amazon which is here and the book engages with the academic debates on concept formation and makes an argument for participatory numbers and the everyday peace indicators approach in particular and it then goes on to make arguments about local level peace building effectiveness using the everyday peace indicators in a quasi-experimental design in villages in Colombia and Uganda. So thank you very much. Thank you again final speaker for this TED Talks series is USIP's own Dr. Maria Stefan who directs our program on nonviolent action here at the institute it focuses on applied research training and education and informing policies and practice related to civil resistance nonviolent action and the roles in transforming violent conflict and advancing just peace. She was formerly the non-resident fellow senior fellow at the Atlantic Council where she co-led the future of authoritarianism project previously Stefan was also the lead foreign affairs officer at the US State Department's Bureau of Conflict and Stabilization Operations where she worked on both policy and operations for Afghanistan and Syria. She's also the editor of civilian jihad nonviolent struggle democratization and governance in the Middle East as the co-editor of is authoritarianism staging a comeback and is the co-author of why civil resistance works the strategic logic of nonviolent conflict. Please help me welcome to the stage Dr. Maria Stefan. Last TED Talk of the day my name is Maria Stefan and I direct the program on nonviolent action here at the US Institute of Peace and I have spent most of my professional life working with writing about and otherwise seeking to support activists and organizers who are leading nonviolent movements around the world. So for the next few minutes I will discuss the role of civil resistance otherwise known as nonviolent action or people power in preventing violent extremism. So let me offer two examples to set the stage. Four years ago in Burkina Faso which we heard about this morning which has been a victim of violent extremist attacks a grassroots anti-corruption movement mobilized the population to prevent the abuse of power by its president. The Ballet Citoyenne or citizens broom movement which used the broom as a symbol of sweeping away political corruption was led by a group of reggae artists and helped mobilize the population after president Kampwari attempted to manipulate the constitution and extend his mandate. Ballet Citoyenne organized protests musically inspired street theater boycotts sit-ins other nonviolent tactics to generate pressure which ultimately forced the president to step down from power. More recently this past March in Afghanistan after a gruesome Taliban attack in Helmand province that killed scores of civilians. Ordinary Afghans from Helmand began to organize protests denouncing the violence and demanding peace talks. The protests spread to other parts of the country and culminated in a 300 mile march from Helmand to Kabul in the scorching hot sun and in the midst of dust storms. The marchers incidentally spoke with Taliban fighters along the way and when they arrived in Kabul the marchers organized sit-ins in front of Afghan government offices in foreign embassies. This nascent movement has brought Afghans together across tribal divisions. It has involved men and women young and old to up the ante for a negotiated peace. Both of these examples feature ordinary people who were fed up with violence and repression and who decided to take action to do something. They use civil resistance which is a method of struggle that involves tactics of protests and persuasion like vigils rallies marches methods of non-cooperation like civil disobedience boycott strikes and methods of non-violent intervention like sit-ins and the building of parallel structures and institutions and they use these methods to disrupt the status quo and to shift power dynamics in their societies without the use of violence. Civil resistance has been used in every region of the world in all cultures and by adherence of all the world's major religions including Islam. It's been used to advance women's rights in Pakistan to end civil war in Liberia and to challenge authoritarianism in Tunisia and corruption in the Philippines. Nonviolent movements are rooted in communities and they are led by those people who are most affected by injustices including and notably youth. Leaders of extremist groups often claim that violence is necessary to challenge tyranny and oppression that it is the only meaningful way to resist violent kleptocratic and exclusionary systems. However we know that this is not true. A few years ago my colleague Erika Chenoweth and I collected and analyzed data on close to 330 major violent and nonviolent campaigns over the past century. We intentionally chose the tough cases that involve challenging mostly repressive authoritarian regimes willing and able to use violence to stay in power. We found somewhat counterintuitively that the nonviolent campaigns were twice as successful as their armed counterparts. The primary factor in determining the outcomes of these campaigns was the size and diversity of participation. Significantly more people can participate in nonviolent resistance compared to arm struggle or terrorism and when large numbers of people withdraw their consent and cooperation from an oppressive system or an armed group and they engaged in organized non-cooperation the system can no longer wield effective control so you can win without violence. But civil resistance is not simply about protesting and stubbornly saying no. It is also about building positive alternatives and saying yes. Erika and I found in the research that nonviolent resisting resistance campaigns contribute significantly to democratization even in places where the campaigns failed to achieve their goals. This finding has been backed by seven or eight other independent studies and this is because the inclusive participatory nature of nonviolent movements is conducive to building social trust and building systems grounded in a respect for human rights. Not only is nonviolent resistance an effective means of resisting the injustices that fuel radicalization it is also empowering. There is a reason why it is called people power. We know that people join extremist groups to be part of something bigger than themselves. To make sacrifices for what they believe in one of ISIS's main messages for recruiting youth was simply join us and help build an ideal society where you will always belong. Joining a nonviolent movement provides many of the same psychosocial benefits as these groups. I've heard from countless activists around the world that their participation in nonviolent action has offered them a sense of agency of being part of a community and a sense of purpose. Movements like Balei Citoyen and Burkina Faso and the Ana Taban or I Am Tired Youth-led Movement in South Sudan like many others around the world have used humor and satire as we've seen symbols rituals and slogans that have helped build a sense of cohesion and solidarity that inspires and attracts others to join the cause. Silver resistance is life affirming and its creative potential is boundless. So there are positive effects of nonviolent action at the level of the individual but also at the community level where there is evidence that nonviolent organizing contributes to resilience. Michael Markusa conducted a fascinating comparative study of various villages in Tunisia. He found that those villages with history of nonviolent organizing experienced lower levels of violent extremist recruitment compared to those with no history of nonviolent organizing. Other research by Lauren Van Meter, Jonas Claes found that communities that focused on communities in Kenya found that those that featured high levels of self-organizing and that built autonomous governance structures were less likely to experience extremist infiltration compared to those where such organizing did not exist. So in summary, nonviolent resistance offers a propulsive in effective way of representing grievances and of resisting injustices. It offers many of the same psychosocial benefits as joining terrorist groups and it helps build community resilience. Now let me conclude by offering a few ideas for practitioners and policymakers. First, by now it should be a truism that investing in locally driven community led approaches to preventing violent extremism is both sensible and strategic. Policymakers should amplify the work of grassroots movement leaders, center them in decision-making processes and ensure that their voices are featured prominently in national and global policy forums. Secondly, governments, international organizations and NGOs can use the carrots and sticks at their disposal to help keep civic space open around the world in places where it is closing. They can use diplomatic, economic and military to military tools to mitigate violence targeting nonviolent activists and to curb the predatory repressive practices of other governments and their security forces that fuel violent extremism. Finally, outside actors cannot create movements nor should they but they can support the transfer of skills and knowledge about how to wage nonviolent struggle effectively. They can provide the convening spaces to help civic actors in fragile and conflict affected states to plan strategically and to learn from other activists. They can help influential civic leaders tell their nations often buried histories of nonviolent resistance through educational and media channels, something that organizations like ours in the international center on nonviolent conflict focus on. Equipping aggrieved oppressed people with knowledge and skills about how to organize in their communities and build powerful movements to advance rights and freedoms is a critical ingredient in the fight against violent extremism. Thank you very much. Thank you so much and that concludes our session. We are going to once again move out to the Great Hall for another breakout discussion. I hope that this afternoon will be just as fruitful as this morning as we share our impressions and we also contribute to what we're trying to do in terms of analyzing what we need more of what we'd need less of going forward. So please feel free to use both the doors and we'll come back here at 345 for our last salon style discussion. Thanks so much. So I have the honor of introducing our panelists. So all the way to my right is Dr. Todd C. Helmuss who's a senior behavioral scientist at the Rand Corporation. He specializes in terrorism, strategic communications and social media. His work focuses on improving U.S. efforts to counter militant recruitment and decrease popular support for terrorism and insurgency. He has examined the networks of ISIS supporters and opponents on Twitter and identified ways to enlist key influencers in support of U.S. strategic communications. He has identified approaches to assess CVE campaigns as well. He's worked closely with U.S. special operation forces in Afghanistan where he served as an advisor to U.S. commanders and led studies on U.S. efforts to train the Afghan special security forces. In 2008 he also served in Baghdad as an advisor to the multinational forces of Iraq. He received his PhD in clinical psychology from Wayne State University. Next is Cheryl Frank who joined the Institute for Security Studies in 2009 as the director of the Pretoria Office. She's currently the head of Transnational Threats and International Crime Program in Pretoria. Before joining the ISS, Cheryl was executive director at Child Rights Organization, RAPCAN, director of the Criminal Justice Initiative at the Open Society Foundation for South Africa, research and program director at APCOF and researcher at the Institute of Criminology University of Cape Town. She began her career as a social worker with the National Institute for Crime and the Rehabilitation of Offenders. Cheryl has a Bachelor of Social Science, Social Work degree from the University of Natal, and an MBA from the University of Cape Town, South Africa. And sitting next to me is Dr. Matthew Leavitt who is the Fromer Wexler Fellow and the director of the Washington Institute for Near East Policies, Jeanette and Eli Reinhard program on counter-terrorism and intelligence. Dr. Leavitt has written extensively on terrorism, on countering violent extremism, illicit finance, sanctions, the Middle East, Arab-Israeli peace negotiations with articles appearing in peer-reviewed journals, policy magazines, the press including Wall Street Journal, Washington Post, Foreign Affairs, Foreign Policy, and numerous other publications. So I have very esteemed colleagues joining us on the stage today and this is probably just a snapshot of many of their accolades. So throughout the day today I hope we have helped sharpen why we think it was a pretty good idea back in September of 2015 for international stakeholders to launch the Resolve Network. The idea of connecting, capturing, and curating locally informed research and violent extremism and how it can promote effective policy and practice is a worthy endeavor. Almost three years after its inception and with a commitment to clarify its mission and its practice, this year's Resolve Forum is designed to push the envelope a little bit further on addressing what are our knowledge gaps on violent extremism as well as the gaps in between research and how it influences policymaking. I'm hoping that our panel here can unpack some of the very real experiences of experts well versed in the CVE ecosystem on research to policy conversations and hopefully what we can do about improving our community of practice going forward. So we're going to start I'm just going to have a little bit of an informal opening and ask each of the panelists to give opening remarks and then we'll go to a couple questions. So we'll start with Todd please. All right well thank you. I was intrigued when you sent the email describing what some of the topics you wanted to address noting that part of the question is like what are the challenges in seamlessly integrating research into practice and I find it a I'd laugh a little bit because I've never had good luck seamlessly integrating research into practice and it's so hard on multiple levels. Some ways by design and just some ways just with the difficultness of the topic we're looking at I mean number one there's in general a disconnect between policymakers and academics. From my experience academics oftentimes struggle to understand what the policymakers need, what their actual research needs are and even at Rand where we oftentimes work very closely with the policymakers at least the shop that's you know funding whatever work we're doing even then it's really hard to get a good understanding of what the policymaker wants what is their decision point and how are you trying to inform that just CVE research in general is just a very messy process getting access to subjects is hard the research that we do rarely has like gives you a hundred percent variance answer on anything so pretty much anybody can criticize most research on terrorism because there's loopholes and flaws and and everything so in some sense as researchers really just need to they're trying to provide part of an answer rarely do you ever provide a full or a comprehensive answer and oftentimes there's not much on the back end so as researchers finish their research products up oftentimes at least from my experience we spend all our money doing the research we have spent very little time and money and effort interacting with the policymakers to push our products we hope that by providing a hundred page report on the website that somehow will magically influence people and i'm shocked that it doesn't and and the policy process is hard right there's a lot of stakeholders involved in all sorts of decisions and rarely does the researcher get a hundred percent vote on anything they probably shouldn't right there's a lot of stakeholders to include the general public that has a say in these things so anyway i think it's really hard to have this seamless integration i think there's ways to address all of those factors but it's a it's a bit of a slog and i can't think of any home runs that i've had in my work i'm waiting for the day but i trust that over time by answering questions please answering part of the question continually we know that starts to get a good chunk of it addressed so and charo what are your thoughts on a similar question yeah you know if if you would have listened carefully to lianne's description of my bio is that you know my entire career has been moving around all these spaces of being in practice as a social worker and then running an NGO working in research institutes at universities and now my kind of hybrid job of doing research and then capacity building and technical assistance in africa and and my experiences are in africa and then and then also acting as an advocate for certain human rights issues and and particularly in the child rights field so you know i've been maneuvering around the space trying to find my way and find where is it that we can make the the most significant change is it in practice is it in research is it in influencing policy so my entire career is the search for has been a search for these sorts of answers but i think what you know the main thing that i struggle with currently is these three pieces of things which is one is evidence the other piece is policy and what is in policy and the other pieces actual actual practices and the huge gaping holes between all those things and how they they relate to each other so and how they communicate with each other and how one may have something stick on another such as in africa we have this i mean i'm south african so even and my career started in south africa we used to call this this problem implementitis is that we had the most fantastic policies straight after apartheid and we had the most brilliant pieces we have brilliant constitution we had all these brilliant pieces of legislation supported by the world um and we just couldn't make it happen in many places in in and with very disastrous results for children for families for women um which is the field that i worked in at the time so um this implementitis problem and in between policy and practice at the time and then um and then as a research as i developed noticing the the massive gaping hole again between evidence and the other other two things policy and practice again and the sort of triangle of things that was confused and really difficult to figure out how they linked to each other so um i remember and and i'll i'll just end my opening comments with this as i remember um we were preparing as a group as an alliance of organizations to go to the south african parliament to have south africa's first juvenile justice legislation passed so we'd been working on this for years and we're taking this to parliament and like a good research as i suppose i went to poke around in the data on you know how does um how much do parliamentarians actually listen to evidence because we were preparing submissions to present to parliament around these issues um and um i i sort of was depressed for about three or four days after that because there was a lot of evidence coming out of the uk um particularly in in terms of their overseas development assistance um analyses about how little it was happening and i see the same things over and over again i looked again yesterday when preparing for lian's comments and noted exactly the same problem so um so it's really a question i think again of communication issue between these three spheres of existence and is it a practical communication issue i wonder um and it it's what you were talking about is is it the 18 page 80 page report on the on the web on the website that is the communication problem and is that part of the problem or is it a you know a bigger dynamic that we have to address and and i suspect it is the latter and over to you matt so first of all i think that on one of the uh measures of success uh you should use as how many times it's been downloaded not how many times it's been read i personally got to page 80 but not past that um if anybody wants my business card i'm happy to give it that's one of my measures of success how often do i have to replenish my my business cards and you laugh but that's a little bit how we tend to measure some of these programs um and uh at the washington institute i lead a cv working group uh with some wonderful people involved um and uh we talk about these things and then i've also led now three different uh bipartisan study groups uh three with different reports that we published they're all available on our website and you can download them from my metrics i'd appreciate that very much um and in that we we uh brought together we're non-partisan think tank but we brought together democrats and republicans put together the support and then we kind of walked it into policy makers offices and i want to give you a little bit of feedback from that uh the first point is that uh in this country we don't have a political system that allows for failure okay so uh i give our european counterparts for example tremendous kudos for trying and failing and trying again uh lots of people like to beat up on on the brits for their various iterations of their contest strategy fair enough but they have tried and continue trying and don't always succeed and i'd like to see a little bit more of that in this country but we don't have an either side of the aisle we don't have a political system that tolerates much of that so i walked into uh various uh house and senate committees and met with members and staff with democrats and republicans from our task force and we briefed them on a whole bunch of these different issues and ultimately they'd come down and they'd say well why should i fund any of these programs if i don't know that they're gonna work and we said you won't know what will work until you fund something and you build metrics and evaluation into it and you can limit it and then then we'll have something more to talk about they only saw it from the political perspective of you're asking me in a very very sensitive area we're talking about terrorism here right to to take a risky stance um and if you can imagine how risky it is at the front end when you're talking about off-ramping people who haven't necessarily committed any crimes yet but seem to be inclined to go down a wrong path you can imagine how much more sensitive it is for an air where we are going to need to spend a lot more time and effort on the back end as people uh finish serving their terms in prison for example and come out of prison and as you heard this morning there are no not no effective there are no reentry programs to help people uh be able to to reenter society which brings me to my next point very often policymakers in this country on both sides of the aisle come at this through a very ideological lens from both sides of the aisle um and that is to say there are many people who feel uncomfortable putting in place as several people have put it to me a social welfare programs for terrorists that that's not how we deal with terrorists right our our purpose here is not to embrace terrorists before after we have other ways to deal with them um that is extremely short-sighted I said to one person um you know um we have programs for rehabilitation for people who carry all kinds of really heinous horrible crimes this the t-word becomes so emotional and so political and so ideological that somehow we put it in a completely different context every administration that comes uh into the White House um has a period of policy review that's always the case the policy review for uh what I still call CVE policy but the Trump administration now calls terrorism prevention has taken until now and the pendulum is just now beginning to come back into the larger middle after what was a very ideological discussion mostly about radical Islamism when that is not what this is actually about and that's only if anything a small small piece of it and so I think that there is an opportunity now to have some of the discussions that have come up over the course of today and that may surprise some people but I think that that's good the downside is that for many people this is an ideological issue for example some members of the task force that we put together really didn't like it when we wanted to talk about domestic terrorism white supremacist violence uh that conversation has changed post charlottesville but at the time people said to me and these are smart smart great people we've had those problems in the united states for years that's not new what's new is the radical islamist terrorism that's coming from abroad so we have this ideological piece too I think we need to think of this also however not only from the inside the beltway federal and legislative perspective we need to think about this at a local and state level not only because I think we are seeing now that there is more headway in cve programs domestically here in the united states and I should say that's that's my area of focus more cve domestically than internationally um but because ultimately these are phenomena that are happening in communities and the communities are those that are best placed to have a sense of what they need you know people in within the beltway here talk about the need to give dollars to federal state and local law enforcement to deal with terrorism but if you go out and you talk to local law enforcement across the country most of those jurisdictions they want to deal with opioid problems they want to deal with a high murder rate it's not that they're soft on terrorism that's just not something that's part of their regular lives and if you come say and you need to spend 15 20 25 30 percent of your budget on counter terrorism that's that's out of left field for them so i spoke to some of the people who lead the core initiative here in maryland some of whom she and abat and aguaire and others have been here i don't know if they're still here i think shan had to go pick up her kids at school to talk about how some of these issues of metrics and evaluation fit into how they actually run programs in communities and one of the things i heard is that there is mine phd fatigue enough with you phd's coming and telling us all the different boxes we have to check when we are running programs in communities we have to deal with the communities themselves and we have to deal with the local and state government mostly law enforcement with which we need to interact the local and state government doesn't want to know all about the phd isms what they want to know is how can i fit this into existing programs because you're not giving me more money to run these things all right budgets are limited how can i fit this into things i'm doing already and when you go to the communities they don't want you to come with a bunch of big terminalities and big ideas and tell them we've studied it here's what you're experiencing what they want is to be asked and so you know the collective impact methodology where you come in and you say hey we've done some studies and we have a sense of what's going on but we'd like your opinion here's here's what we think is this what you're experiencing in community is is there something different and there you get you get basic buy-in and the last thing which is an area of tremendous disconnect with federal government with legislature etc is that across the board i haven't spoken to every program there's across the country but every program i have spoken to unanimously says it is tremendously unhelpful to put this into terms of terrorism or any type of ideology to the extent we can put this in terms of public safety to the extent we can put this in terms of violence prevention which ultimately is what this is about we will inherently get much more community buy-in and a lot of people here in the beltway don't like that and you will notice that under this administration this effort in most not all but most federal agencies and departments is now referred to as terrorism prevention which i think is undermining the ability to run exactly the same type of programs we're talking about in communities who don't want it to only be about the big t-word ultimately what we're talking about is violence prevention and public safety so you all have given incredibly important insights i'm going to pick up on the last one for the question so one of the topics we've been trying to grapple with today is how do we get more research or expert or analytical findings to actually influence policy what are some of the incentive structures or disincentive structures that we should be advocating for and so maybe i'll hone in just on matt's last point if focusing on the word terrorism is actually a part is not necessarily helpful what are ways in which we can be as the research community as the community of experts be pushing forward on ways to do what is more helpful started a local and state level the fixation on on these terms tends to be one that is within the beltway here so for example and by the way there are parts of government for example the Department of Justice has programs for evaluating programs and they're doing great work but for example in massachusetts the state department of health and human services has partnered with academics at the state level to help drive some of their programs a lot of state money there might be some federal money but this is being done at a more local level so it doesn't all have to come out of washington for it to be effective and for it to then be available to be used by academics and others to then have conversations with policymakers about how this can inform smart policymaking i'm going to reflect just slightly on that which is um so the the name of my title and many titles of of those doing programs in this room and around in this community is countering violent extremism which i have a completely unscientific sample size that everybody hates this term so yet it is the organizing principle on this phenomenon i think there are a variety of different reasons as to why this term is um alienating to communities it is not that helpful in actually describing the phenomenon it's not the the most popular um term but it is our organizing principle and so to take what you were just saying Matt but maybe reflect on it a little bit further um if what we're trying to do is solve the challenge hopefully well what can we do about the way in which we're conceptualizing it and from the research perspective are there ways to either study this or to give analysis or empirically based study about why we're sometimes cutting off our nose despite our face even when the goals are good but sometimes the ways and means may be challenged what wait see that last part again um all right the last part was cutting off our nose despite our face um and so this was the idea that the goals of cve are quite good but perhaps the way in which we're describing it um are yeah i think well so i think i'll know you can do research at many different levels right so um for me oftentimes i'm doing research for very specific policy shops within the department of defense or department of state not trying to influence like broad u.s policy on all cve issues um which you know pretty cool matt's had a chance to do this right great bipartisan work for bipartisan committees we has a chance to do that but it's a so it's a very different type of topic i think i think ultimately um knowing your target audience who are you doing this work for um how are they going to use that work being really integrated with the implementer or the policy maker so you can provide them the product they need it's really easy to provide research that misses the mark on many different levels so i could do a study that for a specific policy shop but my recommendations are very broad and generic well that's not going to help the policy shop i could do a work for broad for broad policy goals but i'm really too in the weeds and sometimes the recommendations i'm making aren't even feasible on any real political level so i think part of it is knowing the politics what it will be acceptable what's not acceptable you can shoot for the moon but the moon's not always going to get you where you want to go um and then you know spending time and money and allocation to try and integrate those findings at the back end of it rather than just post your 100 page report up so i mean really finding ways of talking to people and communicating the results of what you're doing maybe that means using the word cv e maybe it doesn't but there's many different types of questions that researchers need to deal with when they try and think about is someone going to actually use what i'm doing um i i just have come out of you know really being embedded in thinking about practice when it comes to cv e pve whatever we want to call it and uh in certainly in west africa where we we've been looking at lots of pve cv e projects many don't call themselves that simply because that is uh it's a means for um raising funds for what you think really needs to be done which is peace building or conflict prevention so it's a very much alliance with that experience i think in africa our bigger problem is that there aren't resources being thrown at all those other things so violence prevention all of those other public safety issues so there's a limited part of money for what needs to be done um there's a very interesting observation that came out of some of the work we've been doing recently around um somebody commenting that uh and i think this was a comment from niger is that people don't ask us and that aligns with your comments people don't ask us what we they think need we think needs to be done we're dealing with a whole range of intergroup conflict we're dealing with religious christians versus muslims over many years maybe we're dealing with a whole range of ethnic difficulties between groups nomadic groups and settled groups we're dealing with a whole range of criminal violence violence in homes we're dealing with a mix of issues and the and here you come along with some money and it's focused on violent extremes right so now it's that little bit that we have to find our way through and all those complexity of issues may relate to violent extremism as we say it and seem to know what we think we mean when we say it but um so it's this question of you know what is going on those those communities and that's where i think the research matters and and and the research and and i'm particularly talking about the practice of pve now on cve and and changing things for people on the ground is that the research that you've done matters because we get an understanding of what what people need what the issues are we don't assume that there's going to be violence on those campuses um we find out about it we develop programs we um you know this entire sort of continuum of activities that we need to do to build an evidence base um that's the sort of thing we should be arguing for when we talk we're talking to our donors and we have a lot more influence on donors sometimes than others um and and possibly naming and shaming those donors that are not um and that are not really supportive of this agenda of generating evidence that is useful for everybody um so you know we're going to be raising it at our side event at umga next week um where we release this report but one of the things we keep talking about is that is is this a real thing we're talking about is this pve something different is it is it what you're talking about is it peace building so if it is something different then let's figure out what is different about it and then hone in on those issues um so um i think that the the the evidence building continuum um really we need to be uh i i think batting away at that a lot more when it comes to what we regularly say and i see emily smiling there because it is something that we talk about a lot and we say um i remember a conversation with emily um earlier emily from russi and we was we were saying we were supportive skeptics of the idea but that as researchers we do have to be skeptical at these early stages when we really do not understand what this thing is about but that as that develops but it doesn't develop unless you invest in the research and the evaluation so um yeah i'll stop there and imagine how difficult that and it is to convince policymakers when we are skeptics ourselves which we need to be just two quick comments on on on the cv terminology issue one i think we did it ourselves a tremendous disservice by failing early on to distinguish between cve and pve they are different things they include different things failing to do that led to a situation where everything was cve and we compounded that by making cve the sexy thing and giving money to cve and everybody wanted to be in cve and people described things they're already doing a cve to get in on the cve money and suddenly cve was everything from you know off ramping to building a playground in a disadvantaged community which is a misnomer of an example a straw man but you get the point the second point is this and several of you in the room have heard me say it many before and you're gonna laugh i say it so often that my former research assistant got me a mug which sits on my desk which says hater's gonna hate the situation is such that there are people who are anti cve because they're anti cve there are people who are have been convinced that cve is a cover for spying and i get that but there are also people who are against it because they're just not interested in this because they're they're they're not trying to be a part of the solution i've been doing this long enough uh where i can spend a few minutes listing off you all the different terminology as we've used for this in the past many people may not remember that cve was the vanilla term that was intended to be the least controversial and as we're not dealing with extremism that you know you can think whatever you want but have you acted on it so it's only violent extremism many people myself included at the time complained look if we're only dealing with violent extremism then we're a dollar short and a day late but it was meant to be more comforting and cve has become a term that is yes hated by a whole lot of people except state department which still has to use the term for congressionally mandated financial reasons um i still use the term too because i don't think anything that and that would come up as an alternative has been better and i especially don't like the term terrorism prevention again because it focuses wholly on terrorism and frankly if you're one of those people who was convinced that cve was just a cover for police and counter terrorism now you think that you were right you've been vindicated and good cve certainly pve is not at all a cover for intelligence or law enforcement for me that really begs the question though because from a analysis point of view from a research point of view how are we able to to adequately research certain situations if the way in which we're supposed to be doing it is for something that communities and others may find so abrasive and i ask that because one of our speakers earlier this afternoon is really talking about everyday peace indicators and how do we get this real indigenous understanding of what what looks like a safe community what looks like a secure community to you community members and if we as researchers are trying to showcase to policymakers some of those findings what what the positive trend lines look like from an analytical perspective how do we do that under the moniker of a term that that is really difficult so i mean from my perspective we've dealt with this a little bit ran a few years ago produced a report that whether you like it or not a lot of people hate it and and it dealt with sort of quote the moderate muslim issue and it sort of seemed to articulate what a moderate muslim was and was not and so it was perceived as identifying good muslim bad muslim and so with that history in comes todd helmas to go to the same people that are really upset at ran for doing this project and i hate to articulate that i'm not that guy i think part of it is just relationship building communities certainly as any community does they have their they have their perceptions and some of those perceptions are right and some are inherently some are wrong and i think it's just a matter of building a relationship with them to overcome those issues trying to address their concerns head on like hey listen i know it i know i know what you're talking about and that's not me that's we're looking at something very different um and help me help me help you and let me carry water for you because part of our job is to is to do that to doing good research requires doing interviews and talking to people and understanding what their concern issues and concerns are and i think if you can show you can listen then the terminology drops drops away very quickly i mean no one's gonna i don't very few people are going to continue to carry that water forever once they get to know you once they get to address your concerns especially if you're doing it sort of a real legitimate type of relationship perspective yeah i think that to the extent that you can build relationships you can do a lot of good things and if you can demonstrate that what some people refer to as cve let me explain to you how we implement that and the way if you can explain for example at a local level that you're implementing that through things like community policing and violence prevention the types of things that parents care about for their children you're not i'm not coming to have a conversation with you because i think you're part of the tourism problem because this community whatever this community is is generating terrorists or could generate terrorists then you can overcome some of that it's difficult to overcome all of it because people are going to have to get people in the room in the first place and it gets to some of the metrics that that we tend to get it's not that they're bad metrics they're just not necessarily truly cve metrics maybe they're pve metrics things like over a period of time taking a poll of people who participate in these community engagements that get to things like what type of level of comfort you have speaking with people from local government or if you had a problem would you approach someone or would you know who to approach and we can improve those things through community engagements of different kinds uh whether that is a true cve metric is a conversation it's also true though that if you can improve those types of metrics you can get to a place where you can then talk about other things that can be more sensitive and can get to more cve issues i'm gonna pivot us slightly and ask a little bit more of a specific question if um if you all might share some examples from your experience of when you have seen um research actually um either change somebody's mind change a policy change a practice um if you've actually seen kind of have any anecdotes or examples of that kind of evidence-based shift i think there are small examples um and this is in the days before um alternative facts and facts and not facts and and so on you know it's this it relates to the earlier conversation i was having about um you know trying to convince parliamentarians in the south african parliament to pass juvenile justice legislation that was that was pretty liberal um and liberal in the sense of treating kids through diversion you know pretrial diversion a lot of alternative sentencing etc but we managed to use evidence to to argue for something that was quite controversial at the moment anything to do with sex is controversial but um young sexual offenders particularly um the sexual offenses committed by young offenders was a big issue and we managed to argue them around purely based on international research and local research on this um but i think that um there are several examples also of practice driving policy and not the other way around um and these are two also two examples from south africa and again my juvenile justice example criminal justice examples uh but um you know in order to enable um diversion to be considered an acceptable part of a juvenile justice system this means pretrial diversion kids who have committed offenses admitted to the committing those offenses mostly nonviolent offenses being allowed to not be prosecuted based on an agreement to do some programs or to apologize or whatever so pretrial diversion was never part of the system we used um in south africa and and luckily being part of a national organization used the the idea that that prosecutors had discretion over these things and convinced each director of public prosecutions of nine different provinces that this was okay to try and let's try it in a few courts but the time it actually came to legislating over it we had it operating in 20 different jurisdictions and really had data on the fact that those kids who you know a minor number of them five percent of something reoffended so um practice drove and and similarly south africa implemented um alternative sentences um community based sentencing around community service community service orders as an alternative to imprisonment um as a sentence um and it was in practice long before it was actually in legislation um so sometimes um if it's you know these are just lucky circumstances because we were able to have little loopholes to work with uh but also is that i think that um we need to think upside down as well a little bit because the successful examples have come from practice driving policy uh and not waiting to get are the problem in africa is it takes so long to to to make policy that you might as well try and put things into practice first get it done and maybe the policy will come along later here in the united states policy making is quick and smooth so uh in 2007 we came out or the report uh arguing for population centric counter insurgency in iraq and uh lo and behold the following year there's a population centric counter insurgency strategy so i myself thought i must have had something to do with this because you know the report got some press the bright people must have seen it certainly general patreus read it um finally when i was doing my advisory work in africa in iraq had the chance to to meet patreus and became quite obvious he'd never read my report and and and but to this day my brother-in-law thinks that i was behind the surge um so i mean even when you think you have an effect you really don't know the policy process can take like a long time to follow through i know there's at least a couple instances when we seem to have good luck and these happen to be in the military side and if you get a chance to do military research it's kind of nice because a decision maker the decision making process especially in a deployed environment is pretty straightforward the commander wants to do something the commander generally can do something so if he likes what you do there's options and if you get to know this system you get to know who is reading and who can use the stuff you're doing so we did work on uh survey opinion survey research and build on a operational template that the special operations folks were doing that i've heard from multiple sources um helped to validate that operational approach because it just demonstrated that wasn't upsetting the populations like people thought they were so that's a rare chance i get the feed got the feedback from the policy makers and i saw the subsequent reports citing our work like okay so that made a difference um and that's so i think the lesson for me there's a lot of value in doing evaluation work if you can do evaluation work on ongoing efforts ongoing operations ongoing cv programs there's such a need for that there's such a hunger for results there that is almost guaranteed that if you do it well you're gonna you're gonna help make a difference either by showing that the program is worth doing or not worth doing the other piece that we did was recognizing that our audience was not necessarily the commander it was the guys on the ground the e-fives um the special operations teams working in very forward deployed areas and so we did a report looking at best practices like interviewed number of folks how can you best run these types of operations in the future um the commander had no use to the commander um but we found that folks were transitioning that around for pre-deployment training over and over and over again and years later this piece that was never published just a pdf document had really made the rounds so there again it was like thinking our target audience of in that case knowing the target audience was going to be guys who were really motivated to learn something before going um to Afghanistan um and that they would be uh suitable audiences for this and I think not that you can just do a go-to research for like special operations guys going down range but know who your audience is how are they gonna use it um and you can frame your report in ways that can be most useful to them and it doesn't always have to be a decision maker or congress it could be others that could learn from it and then the outcomes are much softer there's not like a major change that happens after a briefing like oh okay Todd now we're going to do all of that stuff that never happens um it's much more softer than that and you got to look for the cues you got to let me leave work at the end of the day and assume that everybody's read my product and is taking my advice other it's hard to hard to write it i give you i give you three examples two positive one not uh first uh you you had a judge in Minneapolis who decided based on the research of a German researcher uh who started his work on on hate groups and moved into uh terrorism uh on levels of radicalization uh to start in his sentence saying to include alternative dispositions uh this was controversial it gets back into my earlier comment about how uh we don't have a political system that uh is willing to take a lot of risk um but the other thing that was interesting about this and remains an issue today is that um this has not been done across the board so this is still one one judge uh and if you talk to prosecutors and different jurisdictions across the country uh people are left feeling like well you know someone in Minneapolis got an alternative disposition in a halfway house for a year and then the lifetime supervision maybe uh but in another jurisdiction is going to get 15 to 20 years so it's not necessarily consistent across the board but you had real research that a real impact on not just policy but on people's lives and led is continuing to lead to a real policy discussion uh another is um the uh debate as to what the biggest nature of the threat is here in the united states and and whether that includes uh what we'd widely uh broadly refer to as as domestic terrorism white supremacists etc etc um ADL and some other groups came out with a series of studies that I think really quite definitively demonstrated that the number of incidents attacks thwarted attacks etc uh number of people killed or injured by just about every metric um there was a greater threat from what you would describe as domestic terrorists here in the united states not to say that we don't have to be concerned by international terrorism we obviously do but that if you leave politics at the door um the reality in the ground was a little different an area where uh research hasn't really had quite as much of an impact and that's because it's high high policy uh is on the debate as to whether or not um um immigration and illegal immigration separate or together are a major counterterrorism problem are the majority of incidents of terrorist attacks or plots in the united states by people who came here from abroad legally or illegally um there are some in this administration who push that line very very hard uh I've written about it several times now pushing against it and there's a tremendous amount of very very strong data the data unanimous that this is not the case uh that this is not an immigration problem immigration is a third rail hot potato right now and therefore it hasn't had uh the research hasn't had the impact but that's because of the of the specific and unique circumstances of this particular hot potato issue let's see many many times in fact I had uh an article come out about it um after the uh vehicular attack in Manhattan on Halloween um the article came out the morning that I was speaking on one of the opening panels at an nctc conference on what we were then referring to as as cv e um and got pulled aside by a whole bunch of people saying well okay not necessarily uh with a thumbs up but your thing is getting a lot of attention which was great that's why I wrote it to not necessarily make people happy but to but to prompt that discussion got a lot of attention in the broader community sure uh the broader and the twitterverse community is not the same as high policy uh and uh with some critical exceptions um and uh I don't know if that policymaker saw it but I do think these voices contribute to high policy if not now then maybe eventually um so even if there's not like that direct policy result I mean to be able to change the conversation to be able to influence the conversation to be able to be a part of the conversation that's evolving I think is a real special opportunity to to be a part of that um and so I think those are those are wins even if you're still waiting for the high policy to change because I think um the wins are going in that direction if not now then later I think these are all interesting um points for us all to consider as the future of research and you know research for research sake research for influencing policy research for influencing practice and all the different permutations in between so um you have all mentioned in some way shape or form monitoring and evaluation and I'd love to kind of pull the thread a little bit more as to how monitoring and evaluation findings can be influential and if there are meaningful benefits between kind of um research for more research for research sake than research for in the monitoring evaluation vein and if maybe we can pull a little bit more nuance in those type of findings and how how they might be stratified meaningfully or if not stratified then then used in an ecosystem where we're trying to take multiple pieces of information to make better decisions. Look I when Jesse spoke this morning he put a slide up there quoting uh General Nagata from NCTC um that line was from an event that I hosted I was sitting on the day of them as he said you know uh to something to effect of you know we still don't know what drives these things and I uh I said to General Nagata after the fact that I disagree um we don't know everything but we actually we didn't actually know a lot. Again I've been doing this for long enough that you know that can tell you how many rooms the size we can fill with these studies classified and unclassified both that we've been doing over the years about all these issues. What we need are not more studies what we need is programs that are being evaluated to see if they are actually having the type of impacts we want um and we can put those programs in place because we have a decent understanding of the very very very broad waterfront of issues that can lead to radicalization from grievances to ideology and everything in between. There are now a handful of programs that have gone through some pretty good am I any and that's great but we need more so that when people like me go to members of Congress saying hey we need to put in place programs like these and they say well how will I know they're going to work why should I fund them in the first place you can say well because we've we've done some things we've measured some things here's what we can tell you from what we can tell works in certain circumstances we need that evidence base we need that ammunition to be able to get the ball rolling the most difficult thing is getting it rolling at all but I think we've done that now again there's there's there's a good program at the Department of Justice that has been funding evaluation of programs in different parts of the country it's not like we're starting at ground zero but we need a lot more of metrics and evaluations of actual programs not another study saying the role of ideology or the role of grievance and another pyramid and okay we've got plenty of these okay now tell me if the different types of programs we put in place to address different touch points on that waterfront or on that pyramid are having the intended effect I agree that the the the real space to to for us to focus now is on producing good M&E and that means producing good research right at the front end of programs and being able to measure the effects of those but I think that we're not appropriately structured and funded in order to do that yet and that's part of the advocacy we need to do around the way M&E is funded for example the duration of funding you know and the timelines required to achieve some of these things you know the violence prevention people will tell you the timelines there are sometimes 15 years and intergenerational depending on what you're trying to do so the timelines are not aligned with the actual funding available and and so there are a number of technical problems in terms of program design what's how evaluation is funded and and then and then whether you're producing monitoring and evaluation data so the second piece of this is of course communicating it and communicating it to who for what reason so let's just say we want to scale up this program and we want to do it somewhere else we want to take the principles and maybe try them somewhere else and I hate the idea of replication of programs but especially in the context I come from but you know if you want to take some principles and try them elsewhere why why we're doing the M&E is to show that maybe the program works or some principles work but I think the the biggest problem and it's a bizarre silly thing that we keep doing is communicating badly and and not communicating in a way that people can really understand the value of what we've done or the lack of value of what we've done and and what has worked and what hasn't worked and and packaging that communication in a way that people can really understand it so the hundred page report is required and and it certainly is required it sits on our websites because we do need the technical work to be done however you know the repackaging of that information we actually treat some of that information now as we've reviewed our entire communications approach as an organization the last four years is actually tweeting out research findings from reports and having those be retweeted by policymakers yay I mean if that happens you know so doing that as as a means and then also reaching policymakers in Africa is actually about getting your stuff into the media and getting you and and social media now so so putting those short little stories into the media trying to get people to interview you and to cultivate journalists and so on because policymakers are not listening to us as researchers they may be reading the newspaper and in it is the paper and radio and television so um yeah let me leave it to that I'll just I'll just a very short in time I'll just add in I think the tide is turned on this I think there's an increasing understanding that evaluations are needed for these types of things they've been sort of missing in the past I think there's value on doing evaluations on many different levels if you are running your own CVE program there's no reason all you shouldn't collect your own data to to see whether or not you're producing the type of effect you're doing there's many benefits to it it's much more than just being able to go to your funder and say ah I see I'm working give me more money it's I mean the evaluations are really critical for programmers to improve their own programming are there certain types or audience certain types of their audience that are responding well not responding well if they're not responding well you can figure it out early and change your program so they do respond well so it's it's not just to be able to have a up or down vote on is it good or bad it's to actually improve your programming and so that's really valuable um but of course um I think there's real value in also doing real academically sound research on this we're we're this is a new year for me we're doing a lot of this type of work in fact we're doing several clinical trials on CVE programming so um uh which is a sort of a different level it's a different price tag to that of course then handing out some questionnaires to your participants it's not cheap it's expensive to do but I think if an implementer is like spending if a government agency is spending a lot of money on a program um then they should need to spend a chunk of a chunk of money on an evaluation and it probably should be an independent evaluation so I'm gonna stay on that last point um how I think that I'm just so in agreement that the M&E movement has been really really useful on the idea that programs have to be evaluated and has to be built in from the outset and that um that we need to be doing more of this I wonder how we do that in um ecosystems that are so atomized by the stowed pipe efforts that happen whereas you only can impact from um a plethora of different efforts all at once and so to me some of the work for the academic community and for more external research is to look at an entire system while some of the M&E is going to be project level program level um but it kind of gets to your point Cheryl how do we do this on the time horizon where you can actually see something and I was at a lunch uh earlier um last I think it was the this week or last week where somebody said um it's very difficult to know whether you're in a trend or a cycle when you're in it because cycles look like trends until you're out of it and it struck me as something that could be you know very applicable to a lot of our work how do we get that kind of meta level of analysis how do we maximize what can be an outside look at research and combine lots of program level or project level inputs together to see what's actually happening because there is an idea that you can have many successful programs but not actually be seeing a successful ecosystem change or impact change because external environmental factors are changing at a more rapid pace than any of your interventions are addressing so can we talk a little bit about how we can get to that kind of meta level from the researching perspective I'll do my two cents on that as I to me that's like it makes it too hard to pose the issue like that that assumes that your outcome about the dependent measure of your evaluation is less terrorism more terrorism yeah can't do it you're probably not going to be able to do that evaluation um you're never going to be able to randomize enough people to find out if some people actually committed terrorist attacks and some didn't um even showing changes in attitudes do it if you can but it can be hard so to our view the really the key goal is interim outcomes um what what no program just blindly yes our goal is to reduce terrorism and that's our program that's nobody has that program the program or always have near-term interventions is it to help socially stabilize at-risk youth give them social networks is it to provide meaningful job opportunities to people coming out of prison so they have alternative actions to do rather than resort to a life of crime or maybe even violent extremism so in generally cv programs try to do something that get that is an interim objective to their big panacea outcome and that's where the evaluations I think need to occur um that is uh and those can be done in a short-term horizon so if you have a program that seeks to work with at-risk youth to give them social outlets does it increase social outlets you can test that in a very short period of time and if it doesn't then your program is not going to work um and it's probably not going to have the long-term outcome you think it will have because your long-term outcome is dependent on the short-term outcome so I think that's sort of the key goal is to is to shorten the limit your goals and objectives on your evaluation can I also just say that you know the um yeah it's a difficult question but you know other fields have been doing this for a long time um criminal violence prevention is a field that exists for more than 40 years they have 40 years of evidence of learning of techniques and strategies and um in order to to try and figure out contribution attribution all of that technical stuff which is boring but does need to be I mean we do need to engage with it as if we want to do M&E I think that those 40 years for them and it really does uh we can I mean they can demonstrate some evidence-based practices that have worked in Japan as well as they've they've worked under similar circumstances in New York City and they've worked under similar circumstances somewhere in Sweden and uh so you know they they've come really far in terms of their evidence space and this building of evidence-based practices that may work in your area for a particular thing so um I I don't think it's impossible to do any of this but we're we're at such an early stage in this in this field that it's really difficult to to try to pull it apart one of their biggest problems and and I have colleagues who work in this field at the moment is this business of scaling things up right so you may have a fantastic project that has shown that you can actually reduce terrorism in all violent extremism somewhere in the world and and it's really the the the issue is how do you scale things up unless you can get government to legislate and fund it fully you know in one go and that that process is something that that field is still struggling with um notwithstanding the massive amounts of evidence that may exist around that so again it's that conversation between what is this complication between evidence and policy and and and why is it and it's different all over all over the world every context has its own issues um and it's mostly the politicians of the that are the problem not us but really it is it is a big puzzle and we have to figure it out for ourselves in each individual case I think yeah I can only add that um to me what the biggest part of the problem is everything that you both said makes all the sense in the world except if you're a policymaker right the policymaker isn't interested in what's going to happen in the nearer they want to know how was this my goal actually is to defeat terrorism I can't tell you how often still to this day I get asked well well when is terrorism going to end anyway yeah exactly how many terrorists have you stopped um look you know when I was the deputy assistant secretary for intelligence at treasury in a different area in terror finance issues we had similar problems with uh finding methodologically sound uh um information to demonstrate that the monies we were uh getting were actually stemming the flow of funds and uh improving the uh security of the international financial system and so we did uh the methodologically unsound and politically wise thing of declassifying a small number of incidents and and stringing them into a testimony uh not methodologically sound but people would be able to go forward and say look here here are some successes we've had and so the problem is is not thought that you're wrong you're absolutely right it's just the the the when you translate this into trying to affect policymaker's decision making um that's the barrier because they want to know something beyond that and then you know your your point about criminal justice absolutely but because terrorism has been put in its own box you know I get told all the time look oh but but crime is something different gangs is something different all these things that we could learn from suicide prevention is something different I'm not saying that terrorism is exact same thing as gangs or suicide prevention or or or uh crime but there are things from each of those areas that we can learn from there's a lot of study that's been done in each of these areas that is useful to us and frequently policymakers don't want to make use for that because for political reasons uh this terrorism has seen as something different and and I think it's just a matter of an insufficient amount of resilience we usually talk about societal resilience I think there's a whole political and policy resilience that is lacking that is having a really negative impact on our ability to leverage the research that's been done already in a Venn diagram kind of way what this is separate but there's a piece of it that's relevant and we are not able to use that because so many people insist that all those other things are something completely different well this is the moment where I get to uh pet the resolve network on the back because during our discussions breakout sessions that all of you were a part of one of the um areas on that kind of sticky notes uh framework where what are other areas that we should be learning from as the CVE community of practice and I know at the tables that I got a chance to sit with uh there were a lot of different areas that I think we can learn a lot from and if we are committed to elevating the empiricism the rigor and the knowledge base on this problem set we have to be learning from other um areas not in the this is the exact same thing but there may be some extrapolatable lessons and also there may be some non extrapolatable lessons things that are absolutely different and we can be learning from that as well those are no less important yeah so with that um we are out of time I'd love to thank my colleagues and co-panelists for this last discussion so please join me in thanking them thank you thank you thank you all at the end of our day thank you to all of you who have made it to the end of the day we are at our closing remarks section and I am just delighted to introduce our keynote speaker for our closing remarks uh Mr. Pete Morocco serves as the deputy assistant secretary and senior bureau official for the Bureau of Conflict and Stabilization Operations CSO at the State Department many of you have heard us refer to CSO throughout the day and our incredible partnerships um in their um support for the resolve network I'd also like to give a special thanks to Mr. Vasu Vaitla sitting in the middle who is our main point of contact on a daily and weekly basis and I'd be remiss if I didn't thank him for his consistent support for the resolve network um Mr. Morocco before assuming his current position oversaw the Bureau's Europe Eurasian Near Eastern Western Hemisphere and Analysis Planning Program and Learning Office he oversees CSO's leadership on atrocities prevention including the atrocities prevention board an interagency committee consisting of US officials from the National Security Council the departments of state defense justice and the intelligence community and others he has an illustrative career before coming to his current position and we are delighted to invite him here today so please thank you Leigh Ann for that nice introduction and I'd like to thank the president of USIP Nancy Lindborg for having me here today um they tell me I've only got about an hour and 10 minutes to speak before the reception starts so kidding I definitely would not listen to myself talk for an hour so I'll get right to it uh the subject that we're talking about today is near and dear to my heart the work that the Bureau does the Bureau of Conflict and Stabilization Operations it's this is exactly what we focus on is how we can take evidence and put it into action and have impact so this is this is a meaningful and relevant discussion to a lot of the policy and a lot of the decisions that we're making today this year's focus on innovation is important it's exciting to see the resolve network discussing actionable research our mission is to anticipate prevent and respond to conflict this includes early detection of violent extremism and our focus is evidence-based analysis and expeditionary diplomacy and when I say expeditionary diplomacy I'm talking about the Secretary of State saying we want to get our swagger back he wants to see us back in the field and that's exactly what we're doing but we can't just go there we actually have to take something valuable with us so CSO we are nimble we measure for impact we're the State Department's surge capacity and conflict zones and fragile states our methods are rigorous and proven our reports are based in evidence and action and not personal inflection we don't do advocacy and we always arrive with an exit strategy this afternoon I'm happy to report that this is exactly what our focus is on and I heard a couple of interesting remarks there at the beginning was research for impact I can tell you for instance we don't do literature reviews anymore and I know that in the past there's been a lot of questions about do you do research for the sake of research or research for the sake of having an effect and I can tell you unequivocally that we are there to have impact so let's talk about the big picture and I think the big news that we have right now is that we have a new government-wide stabilization strategy launched in May 2017 and recently completed it's called the stabilization assistance review I'm seeing a few heads nod we affectionately call this the SAR for those of you who are addicted to government acronyms in the past many humanitarian and stabilization efforts morphed into needs-based development assistance this was sometimes with no discernible national security priority across the board we're being much more selective about our engagements more targeted in our approach and much more disciplined in our spending the US government cannot serve as a world's policeman and we are not nation building not every conflict or failed state warrants us intervention and there are some cases however where in other states internal conflict and instability threatens our national security but if it's not a national security priority if we don't have reliability and effectiveness of the local partners if we don't have security of the operating environment if we don't have the opportunity for burden sharing with our partners if there's not potential for progress towards predetermined political objectives we're not doing it rather than maximum dollars out the door we must focus on targeted approaches to sustainable assistance where there's an issue in security sector reform we train the trainer and where the issue is promoting defections or countering violent extremism we identify locally legitimate actors to do this local context is critical culturally sensitive approaches are necessary for long-term stability we must avoid inadvertently exacerbating conflict and having unintended consequences and we also as they mentioned earlier have to monitor for our progress in these environments innovation to this theme I'm actually very excited about something that we're doing at CSO we're launching a new hub for forecasting conflict trends we provide data visualization to the secretary that tracks events and pinpoints risk we look at media trends hate speech fake news election violence extremist ideologies political turmoil and malign influence we can layer this with monitoring where there are resources or or people from other countries that are entering these spaces we look at the drivers of conflict for early warning we spot for leaders and losers we look for affinity dependence and dominance this technology platform allows the bureau to connect to a diverse range of sources that includes partners outside the US government and they're willing they're willing enabled to to bring us vital data that populates our models and enhances our forecasting this hub captures emerging trends in violent extremism and disseminates information very quickly we need to be creative and nimble and use today's technology for early action to prevent conflict and atrocities understanding trends online can prevent early warning for impending conflict the hub will draw insight from social media and open sources but we can't do this on our own we have to learn from you we have to learn from other people in the community innovation in government is not just leveraging new technology it's how we partner with the private sector and partners like resolve it's also innovating our missions and recognizing how we need to improve and modernize at all times and we're happy to announce that we are in fact making solutions more inclusive president trump signed the women peace and security act into law in 2017 making the united states the very first country to enact legislation on this very important issue it's now us policy to promote the meaningful participation of women in all aspects of overseas conflict prevention management and resolution so let me tell you what this looks like in 2018 this is not mere politics this is policy it is practical and here's a few examples women in many of these communities they have a much stronger concept of what civil society looked like they have a lot of information that a lot of the security sector does not directly they understand where the risk is and who to watch for where the at-risk youth are they have valuable information and contributions and they often are aware of tools that are needed to deter radical elements but they don't have access or they don't they're not in power in many places many of these women they can't go to the police and we need to find ways to facilitate that so one of the things we're doing is encouraging the communities to have more meaningful roles for women in the security sector giving them direct roles with the police force and I'll give you another example that I just saw recently in Kosovo that really troubled me they're in many small towns in Kosovo right now you have little girls that are prohibited from playing soccer they're not able to play sports and this is not something that is tradition it's not a part of their culture this is something new it's from external influence and the government is extremely unsure how to handle it and as well the the legitimate Islamic community has recognized that this is not acceptable and they don't understand why this is happening so how do we carefully address sensitive issues like this at the clash of religion and culture in the Balkans we actively engage with regional partners to counter violent extremism and it's becoming a source of strength for reconciliation between neighbors who are often at odds this crucial moment for the Balkans as they work towards a European future is highlighted by a few major events the Prespa agreement recently between Macedonia and and Greece is a historic achievement and we're going to be supporting the referendum on September 30th success in this area is going to mean huge strides for the region and intensified normalization talks between Serbia and Kosovo could also give us lots of great new opportunities but it also comes with some great risk also upcoming elections on October 7th in Bosnia and Herzegovina could lead to an extended period of protracted political uncertainty my bureau is working very closely with European partners to prevent a political crisis these dynamics change to present opportunity that is is sometimes I think confusing to many of the people that are there and it also it also gives some radicals I guess some excitement too fighters that are returning from Syria this is one of my chief concerns and there's three that I particularly highlight in the Balkans the fighters that are returning are finding new ways to radicalize online and in prison and in many cases we don't have the legal mechanisms in these countries to deal with these these individuals appropriately unresolved ethnic conflicts are presenting opportunity for these extremists to flourish in the area and we can't be timid about depriving them the opportunity to do this house mosques or paragemots are showing up with significant external funding promoting radicalism fostering friction with no connection to the local culture at all we will begin by convening a forum of mayors in November CVE coordinators researchers civil society actors women religious leaders and others from around the region and they're not only going to discuss CVE best practices but they're also going to immediately deploy action-oriented programs at the local level and resolve is critical for our strategy and this is what we're going to be relying on them for one development of the research basis to provide the local policymakers actionable programs three train the trainers to build local research capacity in the region the resolve network is a prime example of how we can achieve evidence-based research and empower local partners to sustain their efforts against violent extremism by grounding policy and local level evidence the resolve network can facilitate action grounded in local realities this is essential for more effective and sustainable solutions to preventing further radicalization and developing the capacity of local partners as I mentioned is critical we we've seen how one-off activities are just dabbling and something can actually come with great consequences can be counterproductive short-lived or or much worse our best chance of sustainability is in building capacity of local actors to be the frontline partners on countering violent extremism by connecting the right partners with the best research our strategy is to build for impact and success and we're excited that resolve is going to be joining us in this effort in closing when we talk about fragile states we're talking about states who have broken the social contract with their citizens or in some cases they have never truly established what that is and what that responsibility should be cso's analysis is highlighting global trends and violent extremism that that threaten these fragile states and thus threaten our national security we have to improve our understanding at the local level and ensure local empowerment and this will help people solve their own problems where they are we have a lot of work to do ensuring american security and extending american prosperity will require tenacity but this also requires the evidence that we've been talking about today and because we needed to have confidence in our policy and not just hope i look forward to partnering with the resolve network in this i look forward to hearing more from many of you at the reception today and i appreciate you having me here today thank you that is the end of our day we are delighted to invite you to a reception uh in the great hall um no in the international women's common which is beyond the great hall um over toward the other side of the building there will be plenty of people that are able to direct you to the correct location we hope that you can stay to those of you who have spent the whole day with us we really appreciate it we hope that everybody is leaving here having learned something new having met somebody new having figured out a way in which you can bring more research to bear on this common challenge of addressing violent extremism and i'd like to give a big thanks to everyone who made this day possible particularly all the members of our resolve and cve teams everyone here have just been incredible all day today and every day frankly um i'd also like to give a big thanks to everyone here who's worked in av our um official photographer fits and and many others here at us ip who have made today possible so please let us all go to the reception and i look forward to chatting with many more of you throughout the rest of the evening thank you