Good morning, good afternoon and good evening to some of you who are in different parts of the world. I would like to thank you for joining us today. Without making it too long, let me introduce the director of BDT, Doreen Bogdan. So she can give us the opening remarks and then probably after that, Nick will take over and we'll have a discussion on NCS. Thank you. Doreen, the floor is yours. I think you are muted. Yeah. Can someone unmute Doreen please? Okay, thank you for that. I was also muted Orhan. So good morning and good afternoon. Good evening everyone. I heard we have some early risers with us today. We heard some colleagues waking up at three in the morning. So a special good morning to you. I'm very pleased to be joining all of you today to discuss this very important subject of cybersecurity, which is an integral part of our digital strategies. We have all had cyber incidents, whether as institutions, organizations or even as individuals as cyber crimes and cyber attacks have become commonplace, plaguing online users. And this of course requires a collective responsibility for all of us to see how we can really make cyberspace more secure and also increase the confidence of the users to this critical resource. As we have our eyes on bringing the unconnected 3.6 billion people online, we must also give equal attention to areas that may hinder the meaningful and effective use of the web. Digital is the future of development. It cuts across every sector of the economy, and we must make sure that it is secure. Many countries that have embraced the digital transformation journey are launching strategies and initiatives to improve connectivity and to find ways to leverage the benefits of ICTs and to increase efficiency. I think we all know that that COVID-19 has really put the spotlight on connectivity, as well as the need for resilient digital infrastructure. It has also been an accelerant for digital transformation. And today, we really find ourselves at a critical point, trying to leverage the transformational power of ICTs for economic growth and social development, while at the same time rapidly evolving cyber risks are threatening the confidentiality, the integrity and the availability of ICT infrastructure and services. The most trust and confidence in the use of ICTs are eroding due to cyber insecurity. And of course these concerns have really been highlighted throughout this, this COVID pandemic. Cyber crime damages are projected to it to exceed a staggering $6 trillion in 2021. Institutions, tech companies, hospitals, government agencies, and just about every other sector are investing in cybersecurity infrastructure to protect their business practices and the millions of customers that trust them with their data. And from our perspective, this is why comprehensive national cybersecurity strategies are so important. And this is a point that was also well noted in the UN Secretary General's digital cooperation roadmap that was launched just a few months ago, where he highlighted the importance of public of trust security and stability. To reap the benefits and manage the challenges of digitization, countries need to be, countries need to be focused on the importance of ICT enabled infrastructure with comprehensive national cybersecurity strategies. These strategies seek to respond to cyber risks by coordinating actions for prevention, preparation, response, and incident recovery between government authorities and other stakeholders. And at the ITU, as many of you know, we engage in ongoing cybersecurity projects, activities, and regular interactions in many countries to raise awareness and to help build the needed skills. These activities aim to really instigate strategic reflection into national cybersecurity activities and outcomes, including support to national policymakers to help them develop, establish, and also to implement national cybersecurity strategies and capacities. The COVID crisis has also prompted us to further broaden this initial vision and the reach of our national cybersecurity support to also include digital training, as well as formative activities on cybersecurity governance at the national level. And I wanted to highlight very quickly a couple of examples. To improve readiness and incident response capabilities of developing countries we conduct annual cyber drill exercises and technical trainings. Some of you may have been joining into our global cyber drill that's been that's been running over the past several weeks. We also tried to help countries develop similar capabilities at the national level when when possible. And I also wanted to take this opportunity to highlight some of the work that we're doing in the space of child online protection, and trying to work to mitigate online harms, especially for the most vulnerable amongst us that is our children and our youth. And on the time that the UNSG launched the digital cooperation roadmap, we launched our latest child online protection guidelines. And we will be having a session later today to share those guidelines in a regional context in the Americas region. And of course our work in the space of national cybersecurity strategies which is what we're going to talk about today. In September, we, working together with 20 international partners from the private and public sectors from academia from civil society, we started the process to update our second edition of the guide to developing a national cybersecurity strategy. The guide is really intended to trigger strategic discussions, and also to help national leaders and policymakers to develop and implement their national cybersecurity strategies. It's comprised of a set of principles based on a range of experiences, knowledge and expertise of stakeholders. The second edition update, I think could not be more timely to address the new challenges, many challenges that we have seen throughout this, this COVID pandemic. And when we look back to the first edition of the guide in 2018, I would say we're very encouraged to see the number of cybersecurity strategies worldwide have significantly increased. When we first did our global cybersecurity index back in in 2018 our data showed us that only 76 countries had adopted a national cybersecurity strategy. Our more recent data shows us that 120 countries have these strategies in place or under development, which I think is very encouraging. And with the ITU with our hybrid membership base, which comprises 193 member states are 900 members from industry, regional pan governmental bodies, academia civil society and research institutes. I think we bring we bring a unique platform to the for for collaboration. We consider multi stakeholder partnerships as part of our DNA. It is the basis virtually of all of our work both at headquarters as well as in the field. And I simply want to invite all of you to continue to collaborate with us to ensure that the online world is safe for current and future users. And to do that, we really need to advance real and implementable solutions in a partnership way. And as the UN Secretary General stated when he launched the roadmap back in June, it is important for all of us to redouble our efforts to better harness the potential of digital technologies while mitigating the harm that they may cause. And with that, ladies and gentlemen, I just want to thank you for the opportunity to address you today. And I wish you a successful webinar. Thank you very much or Han back to you. Thank you, Doreen. Thank you for this incredible remarks. And now I would like to pass the floor to Nick, but before Nick takes over. I think Andrea Righorn is not connected yet maybe let's see if he connects in meantime, thank you. All right. Thank you and thank you or Han and thank you Doreen for those opening remarks as well. I am Nick Espinoza, I am the chief security fanatic of security fanatics here in Chicago, in the United States I'm also the official spokesperson for the COVID-19 cyber threat coalition, and the creators of the five laws of cybersecurity. So I'm very happy to be here with ITU moderating this wonderful discussion with some of the top cybersecurity experts around the globe. I think everybody here is going to agree that that adopting and developing a national cybersecurity strategy for any country is of paramount importance here. That is obviously just just beyond critical in this day and age with the proliferation of criminal hacking and everything else. With that, I'm going to go over a few housekeeping items before we start actually with the panel and have them introduce themselves for us and as Orhan mentioned we are waiting for Andrea and hopefully he will be here so somebody just please send me a message in chat when Andrea shows up because I see a lot of people on my screen right now, and I want to make sure I don't miss him so with that, please make sure that your microphones are muted. The ITU has somebody that will go through and check that out for you but make sure that you are keeping those muted because obviously we want to give the full attention to whoever is speaking on the panel at the time. You know photos or screenshots are going to be taken throughout the session and we request that the speakers and the moderator myself basically keep our cameras on and all of that during the session so the ITU is able to get some really good screenshots as well. Also, please use chat only for raising questions related to the topic. You know that that we are talking about this will actually obviously help us moderate what is essentially going to be a large meeting with a lot of people the one thing that we are not going to be straying into is politics of any country, or organization primarily because I think everybody here can agree that security is essentially agnostic to the political system we all need it. And so let's make sure that we are we are making sure of that. We're also going to be using a platform called mural, which is actually very interesting it's an online visual collaborative workspace. If you guys want to show that on the screen right now it you. And basically we're going to be gathering main recommendations that are spurring from this discussion and a link to the mural page will shortly be posted in the chat, and then you can obviously use the chat platform to make any. There it is to make any comments or questions and all of that it moderators will be monitoring audience input throughout the discussions as myself as well will be looking for questions that I can be inserting. I know absolutely into everything as the speakers are talking and all of that so I think we're going to have a pretty good time. There is going to be a recording of mural and everything else is going to be compiled actually into a recommendation paper on empowering women in cyber security which is absolutely a wonderful thing and finally the chat and video recording and this is being recorded you should be able to see that in the top left of your zoom window there may be used for it reports materials and other things so just heads up on that. So that's that's where it at and that's where we're at now. That said let's start introducing our panel and apparently Andrea is is on now so thank you for that Chris and Andrea why don't you. And for the record if I pronounce anybody's name wrong please correct me. I apologize in advance. And we'll go from there but Andrea why don't you introduce yourself first so we know you're here and just basically give your basic bio and thank you. Thank you very much and I'm very sorry I joined late but this is you know welcome to the crazy world of zoom calls you know all day. It's like you know an airplane if it takes off you know the first flight late and then so 32 years in cybersecurity. I've just known one thing in my life. And that's you know cyber security got passionate about big problems big challenges and it looks like you know governments are in the best position. I've spent many years both for the advisory site spent a few years in Busan and Hamilton, but then I also work for governments. I serve my prime minister as head of cyber cyber security, supposing many initiatives in particular with United Nations and it you on this topic where I'm very passionate so here I am to contribute to this great discussion. And thank you and thank you. It's, this is just going to be so fascinating. And just by order of people I see on my screen. Sam would you would you care to go next for me. Yes, thank you Nick good morning I hope everyone can hear me. I'm Sam Visner. I'm honored by the way to be here today and I thank the International Telecommunications Union for inviting me thank you very much. I am as I said Sam Visner I'm the director of the National Cyber Security federally funded research and development center in the United States. My company MITRE which is a nonprofit corporation manages the National Cyber Security Center of Excellence for the National Institute of Standards and Technology. I'm also a professor of cyber security policy operations and technology at Georgetown University. I've served in the US government to the Department of Defense, and I've also managed a couple of cyber security businesses. So I've been in government I've been in the private sector. And now in the nonprofit sector, working cyber security for several decades. So again, thank you, Nick, and thank you ITU for inviting me. And thank you thank you and as you mentioned when we first met organizing this it's it's great to have the band back together it's good to see you again. Thank you very much and somebody that I just met recently and also next on my screen would be Chris Chris can you please introduce yourself as well. Thank you. Again, Chris Gibson, I am the executive director of first, which is the form of incident response and security teams. Been in this game, probably 20 plus years a bit like Sam, very similar to Sam I started in the private sector. I spent three years. I built and ran CERT UK the UK's first formerly chartered national incident response team so so very focused on result, you know, building that to fix and to comply with the national cybersecurity strategy that the UK had at the time. I've been moved on a little bit and spent some time working as a CISO. And now I'm in the nonprofit sector in first so I'm all about building collaboration coordination across the world amongst incident response teams. Pleasure to be here. Thank you very much for inviting me. Thank you for being here as well. And I appreciate it. So, the next one up. I believe I am pronouncing that correctly. If not, please correct me. Are you here. Yes, I'm here. Good morning, everybody, if you pronounce it very well. Thank you, Nick. Thank you. My name is like Villa Guineotiana. I'm head of cyber and new technology unit at the United Nations Office of counterterrorism. United Nations Counterterrorist Center. I recently joined it this year right prior to that worked for five years in a private sector as an international consultant helping countries around the world to develop national cybersecurity industries. I worked in Southeast Asia, in Europe, in South America, in Africa. And before that, I worked for the government almost 20 years in different capacities, mostly national security sector. I had a successful career in the military. That's my background. And thank you for inviting me to participate in this very interesting and relevant discussion. And thank you for being here. And I think it will be both interesting and relevant as you say, just given the landscape today. With that, Irfan, are you here as well. I'm Irfan Hemani. I currently work for the UK government in our Ministry of Digital, Culture, Media and Support. I'm head of the team that looks at drafting of the new cybersecurity strategy along with our international work and skills policy. I started my career in the private sector actually at the point like one of my co-panelists where I worked on information, security and technology, largely after the Sarbanes box of the act in the early 2000s, kind of security and technology, auditing and monitoring to the front. So I started my career in the area around 15, 16 years ago and I've done various things since then. Thank you. Thank you very much Irfan. And also next up would be Pratima. Pratima, are you here? Hello. Can you hear me? Yes. Hi. Hi. Hello. Thank you so much, Mr. Nate. Good morning. Good afternoon and good evening. I am Pratima Pradham. And I work as a senior ICT officer at Bhutan Cert, a computer incident response team, not emergency response team. I've been working with the government of Bhutan for about eight years and I have joined BT Cert only in the beginning of this year. That's because I completed master's degree in cyber security last year in 2019. Before joining BT Cert, I was working with a division that takes care of application development and management. At present I look after the development and implementation of national cyber security strategy. I would like to thank ITU for giving BT Cert this opportunity to share our experiences. And I would also like to thank on my behalf for the privilege given to me among the experts. Thank you. And thank you. And thank you. And just as a quick reminder to our panelists, because we have interpretation going on, I should have mentioned this in the housekeeping. Let's make sure that we are not speaking as fast. That will be my problem as I am usually a quick talker, you know, and I have a radio show and there's no dead air. So, so by virtue of that, thank you all. We will eventually be joined by Martin Koyabi as well of a Commonwealth telecommunication organization. And at that point Martin I would love for you to introduce yourself but I'd love to dive in to the basically to the questions right now and I'm actually going to start where I started the introductions with Andrea and I think this is actually a really good place to start because there has been a proliferation of national cyber security strategies worldwide. And according to the ITU repository more than 110 companies countries excuse me have adopted a national cybersecurity strategy Doreen also mentioned that as well now given the overall rise of cybersecurity development as a public policy phenomenon. What countries are rather why should countries or why do countries start creating national cybersecurity strategies. So, it looks like if you don't have a cybersecurity strategy, you are not dealing seriously with the phenomenon or with the topic. And from a certain perspective, this is true. I mean, and we need to clarify what a strategy is. This is not a document where you just describe your high level objectives. Strategy is your real strategy. What do you intend to do as a country as a government to protect your national interest that being economic, the economic development of the country, protecting the business protecting the citizens, the freedom of the citizens. A strategy can also be not public. Some countries decided to have a strategy and simply they decided not to have a visible strategy, something that can be read and commented by other countries but even those countries, they have a very specific strategy. So that's the first reason why this is gaining popularity. There are also some initiatives globally that are asking governments to become vocal about their strategies in particular looking at indexes to understand what a country is doing on on on cyber security. And one of the first questions that governments are asked to respond is, do you have a strategy. Now, I would this make a distinction between the strategy with a small s and the strategy with a capital s. That means not just a document where you know you talk about the importance of protecting critical infrastructure where you state that international cooperation is important blah blah blah. And of course, you know, this is a good first step from the strategy with a capital s that is how a government is going to play its fundamental role in orchestrating the protection of the national interest in cyber space. And I think that's a really good point to in the sense of quantifying exactly what a strategy should be, you know, for for a country, it's a huge huge problem and I think we've all seen organizations even at the national level that basically might have something written down but not necessarily executed upon and while it's great on paper, it really doesn't help us when when we're under under threat and so by virtue of that I think when we're talking about strategies and Chris I'm going to pivot to you real quick. There are important goals to be basically included at the strategic and policy level to ensure countries readiness resilience and capability to recover as well as obviously quickly restore services, after they have some kind of incident and just given your relationship and your work with first. I think that's a, that's a really good question to ask you. So I guess the first thing to think about when you're when you're creating a strategy and I was peripherally on I was involved on the sidelines in the creation of the present UK strategy is really making sure you join all the dots right across both government but private sector academia and so on this is not something government can solve on its own it's not something private sector can solve on its own. So what you need that strategy needs to encompass both of those to work out how to improve the life you know how to improve your, your, your resilience going forward, and how to make your country a more resilient space for for cyber security for cyber and so on. Measuring is difficult. We've all had this challenge for years and in all aspects of cybersecurity. It's always been a challenge to prove whether you're doing a good job if you see more incidents is that because you're better at detecting them or because you're more vulnerable. If you, if you fix more incidents more quickly is that because you're not seeing the difficult incidents you're only looking at the easy ones. So it's always been a challenge. One of the measures that that I've heard used when I was in the civil service in the UK was it was sort of a. It's sort of it's almost an intangible is, is there someone with a grip on this is there someone personal one area that actually sees the whole picture and can get back together and understand that and report that up the line. So when you have a major incident are 10 different bits of government running around in 10 different directions to do stuff because if they are, that's really bad. You know what you need is, is that coherent solid strategy that everybody understands how they play in this how they work on this, how they interact together. But I think to be fair more importantly, how do you get that single view of the, of what's going on you know single single point of truth single statement of truth going up the ladder into into senior management you could call it you know senior politicians whatever, so that they can understand this. There's also, how do you measure, you know, how do you, how do you, how do you push that into public sector and private sector so, you know, if you go to someone and this was a comment that was made by the head of the Bank of England at the time, he said he didn't know who to go to when there was a cyber incident. Does he go to law enforcement, does he go to the intelligence services in the UK.  There are too many moving parts it's and it is I mean that is a challenge in our world, but a good cyber strategy will bring those all together into a quite a simple coherent state that everybody can understand and work with. Yeah, and I think that's a really good point. And I think what it essentially brings up the point that you're making right here I think really leads into the gap analysis. You know that we all essentially have to do at a national level, you know, to make sure that we do know who we should be calling that we do have those contingency plans in place, and everything else. And also FYI on a complete aside, just an audience member had requested in terms of interpretation translation that we should all slow down myself I am the most guilty of that. There is no doubt of that, but just as a quick reminder but to stay to stay on gaps, Sam just real quick, what gaps exist in international cyber governance, and how implementing monitoring and evaluation at the strategic level can help governments basically filling these gaps and understanding their own their own gap analysis. Nick thank you for that question. And as you know I live in Washington DC. So having thank you for that question I'm first going to answer a different question and then I'll come back to that question. Please, I want people to distinguish carefully between policy and strategy. A policy is what a country intends to do what interest it intends to pursue a strategy is how that is done. And one thing I urge every country to have is a policy and strategy architecture. The group of stakeholders from government from industry from academia from nonprofits from every sector who help define and understand what the country's policy will be with some guidance from the country's leadership. That has to be translated to a strategy architecture which is how want to choose the policy. And that strategy architecture has two great values. The first is that it provides a mechanism for broad representation from all the sectors that are involved. And secondly, it also couples the strategy to the resources capabilities and responsibilities of each sector and each stakeholder, because cybersecurity is a whole of nation problem. It includes government. It includes critical infrastructure. It includes manufacturing. It includes the private sector. It includes the economy. It includes civil society. So it is important to have an architecture that provides a means for not only their representation, but for each of them understanding and implementing their, their role in terms of international governance, there are several problems. The second problem I think is the overall lack of a common framework for planning and I think a common framework for planning is something which if all countries shared would make it easier a for countries to understand what each other is doing and be for countries to work together. A framework like cyber crime prevention and prosecution, incident response, resilient operations, risk management and resourcing, policy and standards, civil law, regulation, accountability, public awareness and a culture of cybersecurity and a cybersecurity is if that or another framework, though that I think is a very good framework were shared by countries, it would provide better international governance and better international cooperation. My last point is this, we need better international transparency. We need international mechanisms to identify if countries or criminal groups are developing offensive capabilities and testing them and using them. Such that countries can be held more to account. I look for an example at the comprehensive test ban treaty organization, which built an international monitoring capability that allows for the detection of nuclear weapons testing and development. And it not only does a good job technically in my opinion, but it also does a good job from an international governance perspective. So over time, moving to some shared understanding of how to detect elicit cyber activities and to hold particularly cyber criminals to account. I think is something that ought to be done. And I do think there have been useful steps in this direction. I would urge that more attention be given to them. Thank you, Nick. Thank you, thank you. And I think what you're saying really underscores the need to have that collaborative, you know, international, basically coalition that is essentially going to come together and, and really, really tackle this problem which I think is just a huge huge problem that the entire globe has and with that I'm going to pivot to act V-Lay real quick, because the global outreach of or the rather the global reach of cyber criminals. You know, there's no real single national law enforcement apparatus that can legally pursue malicious actors, even though we have things like Interpol and some others what are the existing international regional channels to bring together law enforcement, private and academia to build and share those resources, things like strategic information and threat intelligence to essentially identify and counter cyber criminals. What are the typical minimum requirements, do you think for basically a country to essentially engage and build partnerships in this manner. Thank you, Nick for this question. Well, regarding international collaboration so everything starts at the national level so countries engagement and international collaboration is defined by its policy strategic approach. Internationally, as you mentioned Interpol and Europol are those channels that information regarding cyber crime activities can be exchanged, joint investigations could be conducted. So in terms, there are different initiatives to bring private sector together there's a global for counter terrorist forum where industry comes together to look at the terrorist activities online and look for the solutions how to counter that. There are efforts to help member states to exchange information in prosecuting criminal offenses and counter terrorism offenses. So one of those initiatives was a joint counter terrorist executive directorate and you know DC publication on practical guide for evidence of electronic evidence from internet service providers and in different jurisdictions because we see this is a problem when you need information from from Facebook from Amazon to prosecute crimes and cyber crimes. This guide helps a member states to approach that in a more effective manner know how to request information request the evidence that is outside the jurisdiction. So these are the efforts internationally. Because there are various initiatives and in terms of collaborations of incident response team. At first and bilateral cooperation that is happening but more effort is needed in that regard, and ways to explore how to make this cooperation more effective is also needed. Thank you Nick. Thank you. Thank you and I think that's a really good point, especially when we're talking about the massive online platforms that the world simply seems to use like Facebook. You know, outside of cyber crime we've seen we've simply seen issues with this on the privacy side. Europe, for example, has the GDP are the United States doesn't have anything like that. Which makes obviously a disparity in law, which I think obviously translates as you say into, you know, into the cyber crime side of things. But I think this is actually a really good place to pivot to Martin Koyabi with the Commonwealth telecommunication organization because obviously, the Commonwealth is a rather large organization across the globe and Martin if you could please introduce yourself. I'd appreciate it and then I've got a question for you. Yeah, thank you, Nick, and also for the panelists who have just come before me. My name is Martin Koyabi. I work for the Commonwealth telecommunication organization. My main role is to look at the technical support and consultancy across the 54 or 53 countries depending on who is in or who is out. We do look at issues ICT related and so far in relation to this particular webinar, we've been looking at cyber security cyber crime and cyber standards across some of these 54 nations back to you Nick. Thank you, thank you and welcome, welcome. Because obviously the Commonwealth countries, as you say, there's quite a few of them, but like other countries, they have a need for implementing a national cyber security strategy. However, being a Commonwealth company can also country excuse me can also pose some unique challenges. Can you tell us about your experience with national cyber security strategies in this vein. Yeah, thanks for that again, and I know there are people on this panel who might have had experience. Back to the point. I think I agree with Andrea and salmon. Can I do it in about. I agree with the other panelists when they talked about what a strategy is and I think some put it very clearly. It's what you the actions that you take I look at it as a map that says okay you're going from point A to B. What do you want to achieve along the way. And are you going to stop. Are you going to have some fuel. So you plan what you're going to do along the way. In short terms, there are very many challenges that many of these countries do face, but I'll just go through them in, you know, in the short time that I have one is the issue around as pointed out earlier, the support within the national support for having a strategy. This is very, very important because the multi step all the partnership that has been evangelized all along is a very important component to make sure that you have support from different sectors, as put up by one of the panelists to therefore the support by the government to make sure that you have a cyber security strategy is a very important aspect of it. And when you have a strategy. As Andrea said, it's, it's the strategy is the action it's the people it's what you do. But the point is that we reach a point where you have a document which is a strategic document, but you don't know what to do in the next step. One challenge has always been, what is the priority, how do you prioritize the actions that you need to take that have been identified in the strategy. And prior to that, there's also the issue around gaps that you've identified so there has to be a process of how you identify the gaps, and we've seen efforts from the likes of the CMM model, which has been used in a number of countries to make sure that you can be able to identify in a strategy and then be able to implement them. So the issue around having a strategy and how you go from a strategy to the implementation is also a very tricky aspect so we need to think about prioritization. That's number one. The second issue also is the issue around the sustaining and how do you fund those tasks that have been identified. Many of the countries that we've dealt with have that particular problem of the funding, which is never discussed during the formulation of a study. We never call in the finance office or we never call in the finance ministry, but we call them after a strategy has already been put in place. So the funding is a big, big issue that needs to be considered. But moving forward, we are now seeing another trend that also is helping this particular process whereby donor agencies, funding agencies, developing partners, the development sector are trying to come together before advancing their assistance in whichever shape or form to this country. And we've seen that through the global forum for cyber experts. You know, that's another forum that the GFC forum is a good forum because what it does is that it puts all these people together in order to prioritize how to approach a specific problem that has been identified in some of the developing countries. But I think there's no doubt we have challenges, especially in a number of areas in these developing countries. One is the area of legal, the issues of policy and regulation that is a challenge that needs to be addressed. We have challenges when it comes to capacity development, how much capacity of the understanding of cybersecurity as an issue. That's another challenge. And I think in terms of just the resilience, which again, the panelist talked about Chris talked about a little bit earlier, but the resilience of this particular infrastructure in these countries. That is something that needs to be looked at. And then of course, cooperation, both regionally, nationally and internationally, as talked about by my colleague there, that is another issue that needs to be looked at. The financial aspect, how do you sustain? How do you make sure that these strategies are implemented? And how do we monitor and develop them? So those are the challenges that are there. But again, as we've said earlier, efforts such as the one that the ITU has put forward where consortiums and organizations with experience come together to come up with a guide who can be a good contribution. And it is a good contribution towards guiding some of these countries to understand what sort of areas to look at, what sort of issues to prioritize and how to go about them, which is very, very important. Back to you, Nick. Thank you. And thank you. And thank you. And I think that's an excellent answer. And I think it really all comes back to what Sam was talking about regarding that gap analysis that I think all countries developing or otherwise really have to face. And I think one of the other issues that we are going to have with this, I think, is trust and confidence in implementing as well or implementation as well. And so Irfan, I'm going to turn to you on this because it's been reported that only 34% of managers actually have confidence in their team's ability to detect and respond to cyber threats. Now, this is due essentially to a general shortage of cybersecurity skills and professionals globally. You know, I know we have that problem here in the United States, for example, 69% of managers state that their team are essentially understaffed, and almost half of them have said that university graduates in cybersecurity are not prepared for the job challenges that they're going to face. I've actually seen that firsthand with some of my work that I've done here with universities. Now, what is the role of basically national CIRTs in addressing this gap? What are the essential skills and knowledge an incident response team must have, and how can these be developed just anywhere on the globe? So Nick, thank you for asking those questions. I think they're really important and they get to the crux of the issue. I think we can talk a lot about the different parts of cybersecurity. You know, the governance is important. International cooperation is important. I think when it comes down to it, as you and others will know, the work is done by people, even when you are talking about technology or smart technologies or AI in the defense of the digital economy. They're put together by people. So I really can't stress enough how crucial skills are to a secure digital economy. Now, you mentioned a few gaps there. In the UK, we have a similar kind of level of confidence in the ability of people that are training for these jobs, but also just the size of the gap. So in the UK, we had something like 650,000 businesses that stated that they had a cybersecurity gap, and we have 120,000 cybersecurity posts that are difficult to fill. Now, it's amazing to think that that's the case when there is an economy in the state that it is in, and this was similar 10 years ago when we were also in a downturn. So we really need to think about why is it that these jobs, which are high quality, high paying jobs, aren't attracting people into them. I think there's a few things here. You mentioned the role of C-Certs in the skills training of people into this profession. I think it's broader than that. And we were talking and a few people mentioned the importance of strategy. Dr. Karbi mentioned it just now and it was spoken about right at the start that strategy is a piece of paper that sits on a shelf. So the strategy is really there about it to get the various, you know, one of the things it does is it brings various stakeholders together. The C-Certs are one of those. Industry is another. The training sector is another and the education sector is another. And what a strategy can really do is bring those different strands together to actually, you know, get the desired result. I think one of the issues that we see in the cyber skills sector is that it's really hard to navigate for businesses and for trainees and for employees. There's no single standard of what level you get to after a few years of training and I think the massive need over the last 15 years for cyber skills has led to a plethora of qualifications that, you know, it's difficult to navigate how to go from one to the other to the other. And so actually putting together a pathway for the cyber career and the cyber sector is really important in allowing people to navigate those jobs. So, you know, there is also, and I touched on this earlier, how do you get more people thinking that cyber is a sector that they want to be in? I think the majority of many people look at the sector and say it's very technical. It's for a certain demographic or a certain profile of person. And that's not me. Actually, those of us that have worked in cyber have seen a whole range of different people with different backgrounds working there. So, you know, one is to make sure we get enough people into that talent pool to start with. Another one is to make sure that that talent pool knows where to go after the training. And then another one is to actually have the standards to make sure that what people are being trained at, trained on is the right thing. And I think those are all really good points, you know, in that sense that we do have a limited pool in cyber security of those that are, I think, very interested in going into the field. But to your greater point though, as we are training and developing, you know, these individuals and putting them out into the workforce, whether it's the private or the public sector, I think it really might scream or rather really might underscore the need for organizational structure. And so Sam, I'd like to know your thoughts on this, but what organizational structure or structures like bodies, agencies, etc. Should countries be putting into place to ensure that their national cyber security strategy implementation monitoring and evaluation and everything else is constructively performed. Obviously, if we have a bit of a gap, if you will, and let's say the education, the training and the experience of generation of cybersecurity professionals coming out of college and or university and going into the workforce, then obviously, you know, we need to make sure that we are properly channeling them to ensure that we are ensuring our national cybersecurity strategies. Nick, thank you. That is an excellent question. In my view, one of the big gaps we have in strategy is accountability for what we do. Not only does the strategy stay we will do something, but it also says who will do it, and with what resources and when it will be done. And if possible, what metrics will be used to determine if it has been done, and if additional actions and additional resources are necessary to accomplish that part of the strategy. Because of workforce development, I think this is a national issue. And it's a national issue both to develop people to help people find the right opportunities to sustain an educational process that continues to develop people, and that develops people not just for technology, but for operations, and for the continued development of policy and strategy. In the US, we have a program called the national initiative for cyber education, or nice, which is an effort, one that continues to evolve that provides for government national government guidance, but also is implemented at the state and the local level. I'm not saying that this is the model that every other country should use. I'm not saying, however, is that particularly in workforce development. It's important that we develop people, but also make sure they have some place to work. I think Chris made that point that we have a gap of people and yet we're not necessarily finding them the right work. And at the same time, we also have to make sure that people have current skills that if we move to cloud, if we move to 5G, if we move to IOT, if we move to smart cities and smart infrastructure, that we understand the challenges of securing new systems and technologies and infrastructures. And we continue in our workforce development efforts to ensure that people understand those challenges and have the right skills. So I would say that overall, there is uneven development around the world in cyber workforce development in particular, and in coupling that development to a country's national cyber policy and national cyber strategy and strategy. If it assigns resources and responsibility and timelines and metrics can help close that gap. Yeah, and I think that's, that's a really good point, just in terms of developing that that national cyber security and we actually have I think something that kind of dovetails with that rather well a question from the audience Vanessa from Brazil asks, metrics are an essential component of the strategies can you share good practices of metrics that you have identified in national strategies and I'm thinking at the lay just given your background you might be a good one to answer that question for us. Well, this is a really tricky question regarding the metrics and as Chris mentioned at the beginning so what are we measuring and are we looking at the number. Yes, metrics need to be measurable, first of all, so are we looking at the number of a cyber incident and we have if we have them more more of cyber incident, does it mean that we are better at our ability to detect them or does it mean that our strategy is not working. This is also a big question at the UN where we manage our UN global counter terrorist program on cyber and new technologies which assists our member states in building capacities in three particular areas. So, and the question is always there so how we measure the success of our capacity building programs and it can be applied the same to national cyber security strategies. What are what are the good metrics to see that. So, first, of course when developing metrics one has to think about what they are trying to achieve in terms of policy and cyber security strategy and how to measure that the number of cyber incidents could be, could be a metric, but what does it show does it show that we are better at detecting them or does it show that our strategy is not working and you know, or the international environment is getting more and more toxic and nations are not capable to do that. I was thinking about, you know, how to measure, for example, the effectiveness of law enforcement and prosecuting cybercrime so one of the metrics that could be used is to, to compare the actual cases that have been started to investigate cybercrime and actual prosecution, number of prosecutions successful prosecutions of those cases because in this regard it can show that we started the investigation and we were able to prosecute it bring it to court evidence was submitted and was acceptable. So we have good electronic evidence collection capabilities digital forensic capabilities maybe sharing of information between incident response teams and cybercrime units that also helped so that could be a metric to to do in some cases for example, certain elements of national cyber security strategy are important like having all of the government exercises, for example, so the metric could be you know the number of exercises helped and the number of lessons identified. And also help to achieve a nation better preparedness and better resilience because cyber exercises they bring stakeholders together at the same table to run different scenarios and that could be a good metric but actually the metrics are tricky and it's difficult to develop there are some good practices available how to develop them and what one that I'm aware of is an ESAS guide to development of cyber security strategies. But I cannot give a very definite and clear answer what is a good metric. Thank you. Thank you. Thank you. And yeah and it's actually it's very interesting and Andrea actually just posted in chat in as a response to Vanessa as well and his last statement at something that we should be essentially joining forces on as an international community to I think create those standards is something that is you know that's very important as well and Chris I just saw you, you know chimed in as well but I think though this actually dovetails when we are talking about resilience and all of that and Andrea, I'd love for you to answer this question, you know, kind of in that vein just picking up where where Vanessa's question left off because in order for national cybersecurity strategy to essentially add value to the development of national cybersecurity strength and resilience. It's essential to elaborate a roadmap, you know, for its implementation, you know, we've talked about the gap analysis, and all of that at some point we we start planning for for the implementation and so what are some of the most important elements in terms of responsibilities priorities requirements etc to be taken into account when elaborating on that that implementation plan. So we go back to some business points. I mean it was very clear before saying you know you have the policy you have the strategy, but then you know the most important thing is how you translate those principles and actions into reality. So first of all, a great metric to measure you know a strategy is do you have a plan, and is that plan clear enough I mean identifying which are the actors involved, the timelines, the budgets, and talking about actors or capabilities. If there's a clear identification of the stakeholders, you know, I think that national cybersecurity or cybersecurity in general because it's very difficult, you know, to create a boundary a national boundary around cybersecurity is like a complex biological system is not protecting the system, you know, from viruses, you know, we get in contact with viruses every day and we have an immune system that should be responding to that. And it's the same for national or in general you know cybersecurity related to national interests, you know, you need to allow a flourish economy, you need to allow digital services to be as interoperable as possible, but at the same way you know protecting them. The most difficult. Now without using medical examples because I'm not a doctor, you know, I understand a little bit of music is like having you know the government should be playing the role of an orchestrator. First, in order to orchestrate, you need a group of musicians in a group of players, and the players are not only the government agencies. Every time we are asked by a government to come and support us. We find out that the government most of the times they don't have a dialogue and intense dialogue with the private sector owners and most of the times they own, they manage and they deliver the national digital service or the national essential services. First is recognizing the players. Second, there are players that are not national. I mean, what's the influence to national security of big global players like the social media, or the new, the new media platforms. I mean, they're becoming closer and closer to an international organization or a government with its own governance and they need to be recognized as somehow relevant stakeholders. Third, you need to have a score, you know, okay, I have this beautiful group of talented musicians and then what we're going to do. That's the plan, but the plan, you know, needs to be based on the music you want to hear. And that's how at the end, you will judge the execution, you know, those are the metrics that now I suspect. It's more than a suspect. I'm sure that most of the metrics I've seen are very tactical. It's like measuring cholesterol. Are we sure do we have real evidence that high cholesterol will lead to death. So there are some behavior, some practices that we think might help national security, but we don't know yet. That's where in my written comment in the chat, I'm suggesting that we need common frameworks because we need to merge those compliance based all fashion metrics approaches to the new approaches that are based on massive data, artificial intelligence. Let me see, you know, what I observe in real cyber security and compare it, you know, with the final results. That's where we need international projects where we can provide a safe and secure environment that will not threaten, you know, freedom of speech and privacy and personal security, but on the other side will help us as governments, you know, to improve. Let me finish with the last comment. It's true that every government has a different strategy. But if you look at most of the strategies and for the work I do, I very often end up doing benchmarks comparing the different strategies. 25% of the objectives are exactly the same. Sometimes there are very relevant differences, but let's forget those differences. Let's work together on the common objectives. That's where governments might work together to find common solutions. Yeah, and I think that just really underscores and I love your analogy of the musicians because it takes all of us working together to make a concert. You know, and so I think that's, that's a really good point. Now, Martin, I know you wanted to jump in here real quick for a minute or two and, you know, add your thoughts onto that particular question so Martin, if you, if you would. Yeah, thank you, Nick. I was just listening to Andrea very carefully and I know Andrea means well it is true you need data for to, in order to know where you are but let me just take you two steps back. It also depends on the stage of the country. If a country for example is at this initial stages and we are calling it this ground zero, no strategy available only policy or even none of those. Then I think the builder is dependent on what you would consider the core factors. So for example, you would advise a country that look, you've got to look at your instruments of law, you've got to look at your policies, you've got to look at the regulation in order to allow you to have a strategy that has a focal point, because if you run it as an agency, you can't even start formulating that strategy in the first place. So the priorities are dependent on the stage of the country, where it is, and if there's more data to go by, then you could use that data to fall back to that data to give you the data of the priorities that are required. So for example, if a country wants to have a set as a measure to make sure that they can be able to protect its infrastructure, then that could also get a priority, it could be prioritized based on the impact that you will have if something else went wrong. So therefore, the issue about the risks and the value of that risk, if things go wrong, could also help and shape how you determine the priorities. But some of them are very, very much interlinked because they are more or less related to the legislation, with the policies that are in place, and even the mandate of the agencies that are supposed to conduct that. So I think that should bring that perspective. I mean, to point it depends on the stage and the information that you have moving toward in terms of prioritization. Thank you. Yeah, and I think that's obviously a really good point, because some countries are going to have a massive amount of data collection others are not, you know, we have an entire path, you know that we take from from basics all the way to maturity that that essentially I think is going to dictate where a country is and I think this is actually a really good time to talk to Pratima Bhutan as well because you know I think she's her country Bhutan is going along this path right now and so Pratima I'd love to know what have been some of the main challenges that your country has encountered while developing its national cybersecurity strategy and what mechanisms has your country put into place to ensure that the national cybersecurity strategy implementation is going to be successful. Thank you so much for the question. Actually, the challenges that we have we have faced and still we are facing are almost answered by all the panelists. So thank you so much to all the panelists. I'm enjoying the talk. In my answer, let me just explain you about my country Bhutan because I think most of the participants in this virtual room may not have heard about Bhutan or may not have known about Bhutan at all. So Bhutan is a small landlocked country in between a quiz between China and India, and we have a population merely 700,000 people will be soon graduating from the list developed country to a developing country and a lot of credit goes to it. Hi to you. Thank you. I city was introduced two decades ago and so far we have progressed so much in I city adoption, like in terms of mobile banking we have got less transactions, number of 3G and 4G network mobile subscriptions has increased a lot which almost equals to the size of our population. Most government services are offered online. Their hydropower sector is one of the highest economic generating sector of our country and they are also adopting the high end operation operation technologies like God. And even because of the pandemic, most of the health and education services are also transitioning into online digital platform. The third Bhutan come to instant response team was established four and a half years ago in 2016, and the cyber security journey began with the readiness assessment conducted by ITU in 2012. We function under the department of IT and telecom within the government of the town. And what we are mandated to provide both the reactive as well as proactive cyber security services to the entire nation. The initiation of the development of cyber security strategy. It happened to us the end of 2018 through ITU support, Dr Marco get to the country and provided us a rough draft initial draft. However, the first draft of NCS, the actual finalization of the NCS was being able to complete only this year, a few months ago. So we are at the moment we are awaiting public consultation. And after that, the NCS, a national services strategy document will be submitted to the cabinet of the town for the approval. And as the eargap demonstrates, we did face quite a number of challenges in the process. Among many other difficulties, the most difficult we're on explaining the importance of cyber security and to the necessity of cyber security strategy. Like our leaders, mostly they are non IT background and in a country where adoption of ITT is a working progress, educating cyber security is another bigger challenge. And we are in the management to perceive that the cyber security is a problem with technology when it is actually more than that. The other challenge was the support and buying from stakeholders. NCS caters to all sectors, in fact, to the entire country and for it to for NCS to come as the comprehensive and inclusive we needed involvement and collaboration from all sectors. So we are challenging in the beginning to bring everyone on both. First problem, the second was also in very difficult to come into one consensus. The other main challenge was the because being us very small country and also we are, we are very young democracy, we have limited fiber security capacity and capability. Very few, we can count from in our fingers we have very few number of cyber security professionals and NCS requires a leader who has a complete understanding of the entire ICD ecosystem. At the start there were difficulties in getting a clear direction of the strategy, like prioritization of activities, being inexperienced we felt all the domains of cyber security were important, because there were cases, there are cyber security incidents were happening. The awareness was another, another domain, and then technical development of technical skills was the other. So there was lots of confusion in the beginning. Coming back to your part to the second question. We also feel that we have we have come a long way. So we are hoping that once the NCS document gets approved by the cabinet, we feel that implementation would be smoothly, though it's easy as a pandemic. The quality has a high level I city steering committee which is composed of top level management from both the public and private sector, and that committee is shared by the Prime Minister of the time. So our plan is that the implementation of NCS will be stayed by this body. We have secured budget roughly around 7 million USD, which is less, but if we converted the equivalent is milton 50 million. That budget is for identifying critical information infrastructure we haven't done at the moment. So that's in in in the plan. The budget is also for conducting cyber security awareness and also in developing capacity and capability within the country. The other. We are also we are trying to ensure that the strategy will be successful because we have identified stakeholders and their responsibilities. Now in the plan we have identified a creation of three groups, like the legal group will be composed of stakeholders from LA law enforcement agencies who will be looking after the cyber security legislation of the time. The other group is the child online protection group will be composed of education sector. They will come up with ISPs and CSO, so they will come up with the COP guidelines. And the third group is the technical group which will be composed of people from experts, mostly technical experts from the critical sectors, as well as from academia and other technical experts. The first staff that we have identified for now is the development of baseline security specific to CIA agencies as well as other non CIA agencies and SME small and medium enterprises. Other monitoring will be the plan is the monitoring of NCS implementation will be conducted by BTCET monthly. And if any issue arises that will be escalated to the high level steering committee. So we feel that we are really hoping high that once the NCS gets approved we will kickstart with implementation by formation of this group. That's all that I have to say for now. We, we haven't implemented so I think more lessons will be learned once we start implementing so that's all for now. Thank you. And thank you and thank you and I think that's just a really interesting insight for Tima into Bhutan and and where they're at and also, you know, lessons that you learn right as as you develop the strategy and you begin that implementation process and I think that actually goes rather well with a question from the audience, just overall in general, regarding essentially motivation and encouragement and Chris I think just given your background and first I think this would be really interesting to hear your response to. Thanks again for Tima. But the question from Audi from the audience is one of the most important things explained in the discussion was how to encourage the involvement of all stakeholders, government private sector, etc, to implement a defined strategy. Can you explain Chris how the best way to foster engagement for all of these stakeholders is. Really, really good question and I'll try and explain it, both from sort of a first point of view and from some of my previous experience. So, first believes in is a, you know, it's always all about collaboration and helping people build a better world but that requires people to have bought in in the first place so we're already talking about people who want to do good stuff and now they're trying to join an organization that allows them to talk to other people with a similar mindset and with a similar similar desire to improve things. I think as we look at this in a national Caesar realm where you're you're in a country and you're trying to encourage academia and private business and government and civil society and so on, to do the same thing. It's a different space, and it's something the UK government has wrestled with over the years and I'm sure a family will will jump in on this at some point. When I was part of that world which is, I think I left that world about four years ago. Originally we had tried very much on the let's encourage people to do good things let's let's build an environment where they can join. Let's build an environment they can share information, but we're not. We're very much sort of using a carrot lesser and not so much of a stick. There was a big theory that or a big you know worry that if we regulated things. It would become a sort of a tick box environment you know have you got a firewall yes well having a firewall doesn't make you secure it just means you've got a box.  So, so for a long time there was this we can't regulate but we will encourage through national things and there was a lot of work done on helping businesses see an improvement to their bottom line so so cyber insurance. So you know being able to ensure some of your losses if you if the company was good it should get cheaper insurance there for it there's a, there's a bottom line hit to that companies profitability, which hopefully incentivizes them to do things.  That's a very niche area to work in it doesn't really help in a lot of areas. You know I look at lots of sectors in the UK, where you know the medical sector that our national health service hugely important hugely amazing gang of people.  That's a real challenge to to work through and I get that that's difficult. So, for a long time we were we we we we held off using any form of stick on this. Just as towards the end of my time in government the GDPR sort of Hove interview, British government obviously part of the year at the time was very interested in how that worked. And we saw that as a very natural way to encourage people to do better. Because it actually meant that if they did badly, then there was there were punishments, you know there were fines there were there were hits to bottom lines etc. They also put the power sort of back where it should be which is the British government's view is it's a business is problem to be cyber secure and run a secure business and be resilient and and and be cyber safe and whatever. And if they're not then the customer should move and go somewhere else. So again you're trying to drive that bottom line figure that works quite well in business I still you know it's still a challenge in in say the national health service it's still a challenge in in charity sectors and so on, but it is very difficult to encourage people to do this properly, but there has been a bit of a sea change away from that. Let's just hope people are going to be good into actually we're going to start forcing them we're going to start driving them to be good through the regulation side of the world, but very much based on best practices, very much based on you common standards and so on rather than you should have a firewall, you know, you should be resilient or you should be secure to the level that everybody would expect and if you can't prove that then we'll give you a bigger fine. VA just got fine for you know some issues they had a couple of years ago they're one of the first to be hit with a reasonably big fine as a 20 million pound fine. So, so that some of those are coming through. It certainly was something that when I worked in the private sector, it focused the board's view very closely the thought that they could lose 4% of their global turnover, potentially, is makes a board certainly but again that's very focused on private sector into, you know, companies that with businesses and so on. How do you encourage society to be more secure is a real challenge. We tried to do that by lots of awareness lots of explanations lots of collaboration sharing and help and changing what was the previous perception of government was the government was only interested in taking information from the private sector. So that's one of the stuff we when we when we built cert our model was that unless we couldn't we would always share back with the private sector. So trying to genuinely build a partnership with the private sector rather than just being a receptacle of information. So building that trust through working together I think was was the other piece. And I actually think that's a really good point because I in this day and age data really can't move one way, especially when we're talking about things like threatened intelligence. You know, and I also can't tell you how many times I've walked into, including massive organizations that say well we've got an anti virus in a firewall, we do cyber security, you know, it's a it's a very common misconception I think the world over and it's something that really needs to be addressed but Sam, I want to ask you this question though because, you know, I think, just to expand on what Chris was talking about here. What is the added value of engaging society in the implementation of a national security strategy and how can this engagement and obviously future ones as well be more inclusive more collaborative in terms of needs orientations and perspectives from essentially a wide variety of cybersecurity stakeholders both public and private. Thank you Nick for that question. There are several things that can be done and several advantages that can be gained. First, there is a need to engage all of society. If people don't practice good cyber security individually, and if they don't have good knowledge about what to do, why to do it and why it is important. It may not happen. There is a discouraging set of anecdotes. And as we say the plural of anecdote does not equal data, but in this case perhaps it does that a lot of old exploits continue to work, because people don't take enough care, and the organizations for which they don't work, don't take enough care. I mean some ransomware attacks against cities using ransomware that has been around for a long time, and yet it continues to work, even though mitigation is possible. So good education among stakeholders at the enterprise level, the government level, the corporate level, the individual working unit level and the individual level, I think is important. This is another reason why I think metrics can be important, because when people say, so what. So there's a problem doesn't really affect my company doesn't really affect my government ministry. The answer is yes and here's the evidence. We talked earlier about metrics, there are generally two kinds inputs and Chris Gibson, you mentioned one, you know, do we have a firewall. What we are doing. The other are outputs what results we're getting an input is how much am I spending for, you know, a kilogram of cybersecurity the output is, and what did and what benefit did I get metrics are slowly, slowly becoming available. And the improvement in metrics is an area where I urge more international cooperation. For example, the cybersecurity insurance industry is beginning slowly to understand what cybersecurity losses are occurring, and they need to understand because they can't understand what cybersecurity insurance if they don't. We are beginning to get some data, both in the US and internationally and globally about the effects of cybersecurity ransomware of how much is being held at ransom and what companies are paying. The enforcement works globally together, transparently, we can improve the quality the quantity and quality of data relating to ransomware, both in terms of what is happening and how much is is being paid. If we have that information, and we disseminate that information, we can do a better job of explaining to people what they have at stake and what they need, and what they need to do about it. Let me make one other point. Also explaining to people what to do could be could benefit from having a good common structure, whether or not it's the cybersecurity maturity model, or the cybersecurity framework from the National Institute of Standards and Technology here in the states, but having a logical way of structuring a program that is easy to understand and commonly understood would be another way of helping people de mystify a very complex subject. This does not have to be mysterious. It can be explained clearly, good metrics, good international cooperation to develop those metrics and common frameworks to develop cybersecurity programs can help. And I think those are, those are all really good points in the sense that having a framework like NIST, for example 800 which you mentioned or, or the new CMMC for the US government. Those kinds of things I think do indeed de mystified for a lot of organizations and give them that foundational roadmap that framework that they can start snapping in, you know their, their, their execution and implementation of a project on I think that's a really good point. And we actually have. I think a question that dovetails, essentially pretty good with that from the public, and I'd love to hear your thoughts on this. And the question is, according to the accountability of the cybersecurity strategy. Who do you think should play a role in ensuring that accountability that cyber strategy is implemented as expected. You know just if, if Sam is talking about the frameworks that we're going to use for the guidelines. How are we assuring the accountability of who's basically going to be executing that for us. Yes, thank you for this question a very good question it relates to the governance mechanism within the country so there's no what no one answer who is accountable for that. At the end of the day it's probably the government if it if it's approved at the government level. It could be the parliament. If it is approved as a leg as a piece of legislation. But in terms of accountability it is very important to establish the mechanism of governance of the cybersecurity strategy. And monitoring and evaluation mechanism so if a country has a cybersecurity national cybersecurity strategy together with the implementation plan. So at the lowest level, a lot of public agencies, maybe private actors would be responsible and accountable for the implementations of a particular pieces of elements of national cybersecurity strategy that they were assigned to or agreed to implement. At the high level, some countries choose to have some kind of interagency steering mechanisms where they discuss the ways the progress achieved what has been done what other delays, whether there's a need to revise a plan of action, because it is not not working. So, to sum it up, each country has different strategic frameworks and in terms of accountability, there's should be a specific agreement and understanding within the government who is accountable for the overall implementation of the strategy and of course it comes together with the resources. So who is assigned the resources for the implementation of a national cybersecurity strategy, because as we started at the beginning and I think Andrea mentioned so if there are no resources financial resources attributed to that how you can implement implement those actions. So, yeah, so in short, these are my thoughts and thank you for the question. Thank you. And I think those are really good points. And I think it also actually interestingly enough dovetails with another question from the audience which just wonderful questions please feel free to keep them coming. We'll try to get them going but we're going to actually pivot a little bit and I think this dovetails with your answer at Ville but Irfan I'd love to hear your thoughts on this, because one of the things that we've talked about. I think in the last few years in the cybersecurity community, I know, either when I'm on stage I'm getting questions on this or if I'm in the audience somebody is talking about it and that is the supply chain. And one of the questions that we have from the audience is, many countries are including their national cybersecurity principles aspects related to the development of supply chain controls and homegrown technologies to be developed in critical services. So, how important is it to include such an aspect for developing countries, and what are the best practices in this direction. I think that's a great question. I think it's a really good question. It's a really important one and and the timing couldn't have been better. I think the panel represents a number of different countries, all of which have very different views on this so I don't think there is an answer on this yet I think everyone is trying to do their own thing. And this is still happening across the world. I would say this is this has become really important in the last couple of years as we've become a bit more aware of what supply chain security means for not just critical national infrastructure, but for businesses. So, some of this will depend on, you know, how reliant you are on external technologies, what you consider to be a trusted supplier, and what those actual vulnerabilities are. I remember that these are not kind of esoteric concepts that we're talking about. We are talking about disruption of services. So, supply chain is one way to look at risk and that's a risk set of risk across the economy. But of course, you know, your cybersecurity strategy can't be reliant on one thing or another. So, supply chain might be one one area, but you know, managing risk and being able to recover from cyber threats is an important one as well. You know, having the skills and industrial base in the country to be able to rebuild after an attack is important as well. And just because you have certain protocols over supply chain does not mean that you are risk free and I think that that in itself is a risk relying on, you know, at the silver bullet of something that's high profile at the moment and take your attention off other things.  So, we talk about the importance of cybersecurity, particularly post COVID, because we're much more reliant on technologies now than we were seven months ago. I think what the last seven months has also told us is that our technologies are quite resilient. There's very few stories around the world of entire sectors of industry switching off. People have largely been able to continue their work. They've largely been able to adapt to new circumstances, and it is because our technology is resilient. So there's a trade off here on how much you want to encourage growth and how much you want to, you know, make sure you have all of your bases covered and that's why we talk about cyber in terms of risk management, because they are there are trade off and this is about prioritizing. Right, and I think that's that's an excellent answer. And I think it covers a lot of the basis. You know, in my own experience, one of the things that I see where supply chain assurances lacking is in contingency planning, you know, for for that exact event. I think the world really needs to start focusing on on building that awareness and supply chain into the contingencies with that but I also think that that and Andrea I think this is would be a great question for you here but let's talk and pivot real quick to ICT because given the pervasive nature of information and communication technology or ICT throughout the world, an ongoing issue for policymakers is essentially how to best define a cybersecurity strategy that works for the benefit of the government of industry as as you know was just mentioned not to mention civil society as well so how can governments elaborate approaches on this at basically the strategic level to balance, you know, accepted norms of a country with the opportunities presented by the internet which I think really dovetails on the onto the supply chain issue. Yeah, and it's connected to the statement I made before that's, you know, identifying borders in internet or cyberspace or ICT infrastructures now it's, it's impossible. So first of all, I don't have the solution I don't have the recipe I can only help governments, you know, identifying and better understanding all these variables. First is understanding that there's not national ICT I know that many governments are in particular some large governments are focusing the attention on technical ICT sovereignty, but the reality is that we live in a global world even in a connected world, when even some large countries or very large countries they struggle to deal with the dynamics of the ICT market if they work in isolation. If we look you know what's going on in 5G, even you know the largest countries are you know struggling to deal with the problem by themselves and they're looking to build you know, political alliances or alignments to try to push what they think it's you know, in their eyes the right move to do. So, I think it's important for most governments, if not all to recognize that in when dealing with technology, there are no borders and they're not only the technology vendors, there are multiple dimensions that overlap with very different boundaries. We have international regulators because sometimes you know a problem might be solved not by a technological vendor, but by a standardization body, or by an international organization, or by a group of companies that are heavy users of those technologies and they might drive, you know, changes in that specific technology. Things are becoming worse now with IoT. Why? Because the number you know of connections is exponentially increasing and you can define in a precise way a subset of the global digital ecosystem. Again, I'm not giving the solution because I don't have it. I can only recommend, you know, first of all, not to do damages to stop and understand the complexity of the ecosystem and all the players involved. Second, engage the as many actors as possible, not only the national actors. Sometimes or most of the time the national actors have limited tools and that's where governments working together makes a lot of sense because you know when dealing with global problems like you know security of core interoperable infrastructures, it's not something that a single government can manage by itself, even the European Union, you know, 27 countries together, they struggle, you know, they have of course more power than the single member states. But yet, if they try to find the problem or just a problem by themselves, we're really seeing the results, you know, it's not enough. Right. And, and I think that's, that's a really good point that you make it. It really does at the end of the day come down to that collaboration that communication between different governments different entities and all of that and I think, you know, there are different forms I think of communication that we need to really address here because one is, again that collaboration globally that we need to have on this but the other one, or the other part of this and Chris I'm looking to you and first to answer this question. Is communication during a crisis or during an incident because when cyber attacks occur there's a wide variety of stakeholders involved in managing the crisis including things like operators civil society governmental agencies, and on and on and on and so how can communication during a crisis be improved how can coordination between different stakeholders be beneficial for the overall capability to essentially deal with and absorb incidents. I think that's a really, really good question. And it's something we struggled with all the time in incidents with with who gets the message out.  So again, I'll draw back on my experience with insert UK we were the designated national incident management organization. And that changed from sort of the previous world of physical contingencies and challenges was that in those days a lot of that was government controlled and as we someone pointed out earlier, you know, in the UK, 85% of the critical national infrastructure is not owned by the government the government doesn't understand how it works. The government has no control over it and frankly if you were one of those companies you don't really want government people coming in to help you fix it because the government doesn't know it well enough to do so. So we want we had to work out a way of bridging both the government side of information and obviously there are various sources of information, both within the within the UK government but also, you know, governmental relations with other countries around the world, but also bring in the private sector piece into an environment that up until then have been solely government, you know the, the UK is crisis management construct essentially you may have heard it's called I think all cobra it's a it's briefing room in the cabinet office is a very government thing there's no private sector in there I don't know that they've ever been in there. And that was a real challenge based off of oddly enough the Olympics when when Britain had the Olympics in 2012. When they came up, they realized there that this was going to be a challenge and they needed to mailed both government and private sector and so on together, and they built what they called the opt the Olympic cyber coordination team I think it was. And that was the model we use. So what we did as as the search was, although we called ourselves a search to be fair in the strict interpretation of what a search is, we weren't. We were an information sharing an analysis center with a coordinating role as well. That's what we did.  This is what we know now. This is that we've taken all the data from all of these separate bits of government and private sector, and we built with one report. This is what we're doing. And that's so critical in a crisis to have that one single voice, telling you what the challenges is. Now, the government construct that is not so much to fix the problem, although they want to. The government challenge really is to fix the outcomes of that problem.  Government obviously wants to turn it back on, but what they're really interested in is how that is affecting the simple population is that causing, you know, shops to close traffic lights to fail people to die in hospitals and so on. And that's what government is trying to solve more than how do we fix the problem to turn the power station back on obviously in that case it's fairly binary but but there's a lot more about managing the outcomes of these things and again at that point having that single voice going in telling what the problem is really critical. So we did that essentially through a lot of awareness, a lot of talking to people a lot of making sure that that role was absolutely, you know, black and white defined this is what our role is.  So we would manage those incidents very strongly to make sure that we weren't getting multiple messages. I'm always open for a debate about what the message is, but what I don't want is multiple people talking into into the that single point you know that government construct with what we think it's this and we think it's this. You know at some point you have to make that clear. The second thing to do is to do a lot of exercises.  And every time people will see the benefits of that that very structured information gathering and reporting structure, and that again almost came down to awareness again so as part of what we did. We reached out certainly to the critical national infrastructure and over time much more out to the to the supply chain to explain what our role was why we were here we had a collaboration platform that we could use to talk to them and to gather that information, but it is. It sounds very boring but it is really you have to be structured in doing this you cannot just have someone turn up and and wing it in these cases. Cyber moves too fast scales too fast. It's too international. None of these things are helped by having confused messages at the top. Right, and I completely agree. I, one of the things about cybersecurity, unlike I would say standard technology is that we oftentimes have to pivot on a dime. I like to say we don't we never know when that 15 year old kid is going to hack Google and now the entire game has been changed by virtue of nobody thinking of, you know this particular, you know methodology before. You know, but I also think it speaks to, you know what you were saying about you know the government just doesn't necessarily understand, you know, about let's say things like like becoming an internet service provider or how that communication works it's dealing with its sector and so Sam, I actually want to pivot to you on this speaking of pivoting because you've obviously worked with probably one of the largest and most confederated and complex governments, you know on the planet which is the United States which, you know I think in and of itself has a lot of different challenges when you have so many moving parts and aspects to a national government any national government and so Sam what are some of the main challenges and obstacles preventing governments from ensuring effective monitoring and evaluation for the implementation of a national cybersecurity strategy. Thank you Nick. Great question. There are several challenges a national government will have. And one of those challenges is to get reliable information about who is doing what. What it is being done, how effective it is. One lesson that we have learned is that having good industry and technology specific information sharing and analysis. Organizations or information sharing and analysis centers can be useful. It brings together an industry, energy, or transportation, or financial services or healthcare, so that they share information. It also provides a mechanism for them to share information with government and for them to receive from the government information about current threats and what works. Now this isn't perfect. Government is not always in a position to share the most sensitive information. And sometimes industry is worried about sharing information with government that might include personally identifiable information and we have in in our country had to contend with these and very difficult issues. On the other hand, I think we're doing better. We are seeing that that information sharing is approving in specific industries and across industries that are working on things like IOT and smart infrastructure. And that sharing is also leading us to identify some of the gaps that have to be that have to be that have to be closed. Sometimes we don't know enough about the vulnerability of systems that use AI, or the vulnerability of systems with large attack surfaces, because IOT devices, perhaps as many as 1 million IOT devices per square kilometer. I want to, however, focus my answer a little bit more on developing countries, countries that are only now building a an advanced information and communications technology infrastructure, their own cyber ecosystem, and that may not have had in the past the resources necessary to build up a cyber security ecosystem to support their IT ecosystem. I would recommend that these companies that these countries that developing countries pay special attention. They develop new IT infrastructures and new smart cities infrastructures and make those investments or have access to their investments. They should ask, what information is on what what information is collected in these infrastructures, who secures that infrastructure. The governance of that security going to be managed. Asking these questions early can help put in place the policy and strategy architectures that developing countries also need. And in doing so, they can avoid some of the gaps that perhaps the industrial countries experienced because we didn't look at this soon enough. So Nick, I hope that answer is helpful to you. It is, it is, and I think it actually really starts to dovetail into what is going to be the last question in our discussion just due to time limits in terms of actionable items and things that we can start implementing and really looking at as well and so today you get the honor of having the very last question that I'm going to be giving here today and so I essentially want to talk about effective policing in cyberspace because it's driven by information. So when an incident occurs and in some way shape or form that information must be promptly accessible to law enforcement, you know, and other organizations that needed so to what extent, you know, basically have nations developed communication sharing mechanisms between cybersecurity bodies such as certs socks national cybersecurity agencies and law enforcement and in terms of actionable what are best practices. Do you think in this direction how does this improve the overall cyber resilience and you get the final word so thank you. Nick and I'm very honored to have this final word in our very interesting discussion about policing and cyber domain so incident response teams they work on preventing and mitigating cyber incidents. Law enforcement works on reducing the number of threat actors that incident response teams do not care that much. But sharing of the information between the two is a very crucial aspect in helping both CSERs and law enforcement to achieve their their objectives because information can feed to one organization works and to the others. So CSERs they don't do not have the powers of law enforcement vis-à-vis private actors and regarding attacks of the criminal nature have an important role to play in investigation and insecure especially in securing electronic evidence as they are the first responders when something in the big incident is happening. And they're dealing with a lot of information it is very important that in terms of the prosecution of a crime, a crime, not just an incident. Evidence is not deleted and it is maintained in line with the with the standards that will be later admissible to court. So in terms of sharing information across the country it is getting better. Law enforcement organizations work with incident response teams and vice versa. But as I mentioned at the beginning legal and policy context play an important role how it is implemented on a national level. So, especially important is how a nation shares evidence with other countries. So the nation or the country is engaged in international cooperation which organizations it cooperates, which countries it cooperates and shares information. What are data protection standards because a lot of information after the adoption of GDPI within the European Union is attributable as a personal identifiable information and so it's have a lot of that information and have a regulation whether it can be shared. Another important element to think for nations when developing a mechanism for information sharing between law enforcement and c sets is how cybercrime is defined. So what is the scope and the mandate to share that information. So these are I think important elements for the nations to think and when developing a regulation policies or national cybersecurity strategies to think you know whether it should be a voluntary or whether the nation should put a requirement for the c set to report suspicious activities to the law enforcement when discovered during the incident so there is one question for a nation to think about. Do search will will your national or sectorial incident response team will play a role in criminal investigation. For example, by providing technical experience because they are most technically experienced people at the national level in courts and supporting evidence collections what will be the rules for that. And from the law enforcement side, it is also very important to make sure that it is not a one way street. So how law enforcement will ensure the secrecy investigation and will help see certain in their work that they are doing. And, and yeah I think these are the bullets I wanted to mention upon which I think are important in devising policies and strategies and enforcing a cooperation and information sharing between law enforcement agencies and c sets. But to build that, I believe the most important element is trust how each organization trust each other. So transparency, knowing how information will be used. If it is shared and and the need to enhance that cooperation if we want to address and reduce the number of threat actors on the internet is very important so that would, that would be my answer and thank you for for the question and final question. Thank you, and I think that is a great way to end it and real quick we're going to take an extra 10 minutes just to wrap this up so interpreters. If you could stay on for another 10 minutes or so. We're just going to give every panelist a quick word real quick but but thank you to everybody for attending. Obviously thank you to the panelists just some excellent conversations. This is definitely going to be the highlight of my week I'm sure there's no doubt of that and make sure that everybody you check out mural as well you can see it here shared on the screen but you know there's a lot of really good points that are that are being made on that and the last thing I want to do, you know as we're wrapping up here is essentially to, to give every panelist the ability to tell them, or rather tell you how you can find them in the world whether that's linked in or Twitter or whatever that is. And so with that, Andrea, if you would start us, how can people find you if they want to contact you or get more information from you. So I'm very active in the international community. So if you go on, you know, LinkedIn, you know, you find me, you find my name on many ITU projects and initiatives or in the chat I will drop, you know my email address of course you know I work for a private company, but I'm doing a lot of work for international organizations, you know contributing to the development of new strategies, new initiatives inside the security so always very happy to connect with other people as I said I truly believe in joining Thank you. Thank you. Yeah, I just your answers are absolutely fantastic and I appreciate your time. Thank you. With that, Sam, how can people get a hold of you if they want to learn more or ask you a question. Thank you, Nick. I'm easy to find my email address is s, the ISN ER at MITRE MIT RE.org. I'm also active on LinkedIn. One can look at MITRE's web page and also at Georgetown University for cybersecurity activities in which I'm, I'm involved. And if you are a member of the Council on Foreign Relations as I am. I also post occasionally to the Council on Foreign Relations members wall. So as I said I'm pretty easy to find I'm active on social media, and I'm active as well in the print media regarding social cybersecurity. Thank you. And thank you. Thank you. Chris, why don't you go next how can we learn more about you. How can we learn more about first. It's very simple. It's www.first.org. My email address is Chris at first.org so very easy to find. I'm on LinkedIn as well. I'm on a bunch of places but those two will find me very quickly. I don't think I'll ever remember that for the record but Chris at first. Fair enough. All right. How can we get a hold of you or learn more about you. Yeah. I'm active also on LinkedIn. So now we'll be moving to New York in two weeks. So if the participants are from the member states. I can be contacted through their permanent missions. And also if there's a capacity building assistance from the Office of Counterterrorism in terms of preventing and countering terrorist use of new technologies and cyber. So permanent missions are also the channel to request that capacity building assistance from the UN. Thank you. Thank you. And I believe Irfan actually dropped due to connection issues so Irfan if you're here speak up. If not we'll move on so Pratima let's go to you if people want to learn more about you and Bhutan and everything else where where can they find you. I'm in LinkedIn as well. But I think what should anybody know about me. I will drop my email ID here. Yes, LinkedIn and my email ID. Sorry that's not. Yes. Thank you. Thank you so much everyone. And thank you. Thank you. And Martin, if everybody wants to learn about what you're doing or follow you. Where can we find you. Any other person I'm very active on LinkedIn. I will drop an email at the chat along the chat where you want to get a hold of me. We do a lot of work within the Commonwealth I'm sure some of the members of the line of members of the interaction with different platforms. Sorry, I do quite a bit with the idea as usual, and also with many Commonwealth countries that we do a lot of development work within the UK. So I'll do that and thank you so much for the invite and for the opportunity. Thank you and thank you. And I do believe that wraps us up. Most of the panelists have put their email addresses or other forms. You know in in the chat you can find me at Nick a ESP on Twitter, or slash Nick Espinoza on LinkedIn and I put my email in the chat as well so please feel free to drop me a line if you need anything from me as well. And as this wraps us up, I'd like to thank the ITU for basically putting this together so we could discuss national cybersecurity strategy want probably one of the most important things that the world is addressing and tackling today and so once again thanks to everybody that participated thanks to everybody that that followed along that asked us questions, and we'll contact us in the future. We really appreciate it and I hope everybody has a great day wherever you are. Thank you.