With the objective, and this is different than other nations, but with the objectives of building capacity in science and technology and engineering and elevating the capabilities when it comes to design and manufacturing for systems. As everybody knows, the UAE is a natural resource dependent economy. And science and technology for us is the future for when we're talking about a post-oil economy. It is where we want the dynamo of our economy to actually base that off. And hence, the space program is actually there to develop that foundation. And for us to be able to create that sort of monumental shift to attract more people into STEM, what better way to do it than to have a mission that explores another planet. And that has created a movement not only within those that are within the engineering and the science fields in the UAE, but it has created quite a movement in children at schools and change their perception of what they can do, not only attracting them to STEM, but it's expanded their horizon when it comes to what change they would like to have on the world, how they foresee the future, and what impact them as individuals would like to do. And this is one of the main drivers of the Emirates Mars mission. It's the first mission out of the region to explore another planet, and it's meant to study the planet and provide new data, scientific data to the science community around the world. So not only is it there to impact the country, it is also there to increase understanding on the planet Mars. And so this is the Hope Mars probe mission. Yes. And in some ways it's sort of a case study for our new age of space exploration. Can you just tell us some of the collaborations that are involved here in this mission in addition to your own team? So within the team itself, our initial approach was capacity development, as I've said. And the timeline that we've gotten is quite different than other missions. Other missions usually get about 10 years to launch a spacecraft to Mars. We had about six years to launch a spacecraft to Mars for several reasons. One reason being we would like to innovate the way programs are approached for planetary exploration missions. And this is what a lot of new space entrants are actually doing because there's... So heritage is great, but with heritage comes a lot of baggage of failed missions that is incurred into the design and development process of space. New entrants of space can look at it from a different perspective because that baggage doesn't exist from decades ago. And the approach for international collaboration was actually to go to research institutions and work with them because their primary objective is to generate knowledge and to create new knowledge and to share knowledge. And therefore we're currently working in partnership with the University of Colorado. They have got a lab out of there called the Laboratory for Atmospheric and Space Physics last. And also we're working with scientists to develop our scientific capabilities when it comes especially to physics and research about the planetary atmosphere with the Arizona State University and also with the University of California Berkeley at the Space Science Lab SSL. And your launch will be out of Japan next year with a private space? Absolutely. So our launch will be using Mitsubishi heavy instruments. It's out of Japan. So if you're talking about international mission, our endeavors have always been international. And one of our primary when it comes to Earth observation, one of our primary partners of South Korea. So this is sort of giving a perspective of the non-typical sort of space nations that come to people's minds like ESA and NASA. It's actually global players, so many global players that are currently entering the space race that are supporting the development. Yeah. And so it's not because of this whole ecosystem that's developed, it makes it accessible to beyond the traditional space powers. Alice, I want to bring you in UK Space Agency. Can you talk us through your focus there? Yeah, sure. I mean, fantastic to hear of the ambition coming from the UAE Space Program. I mean, it's really, really inspiring. And I think in the UK, we're also very ambitious in our space programs. I will probably highlight three areas of focus at the moment. So we have recently announced our ambition to enable launch from UK soil. So we hope to do that in the early 2020s. And that is in recognition largely of the subject today, the new space age. We recognize that missions are changing. They're not all these great big institutional birds anymore. We have a whole range of missions and increasingly they're getting smaller in size and that opens up a new opportunity for launch. So that's one area that we're really excited about. I think also exploration, I share the excitement there. For most of us, efforts on exploration have been focused on the International Space Station for the last 20 years or so. And I think that's been an absolute marvel of international cooperation. You know, 103 countries have been involved. And we'll come back to that model in a few minutes, yeah. So, but now, right, now we've got a blank sheet of paper. So fantastic. And we're really excited about that. Again, we're really excited about what the new space age can offer to that. We're particularly interested in what new commercial models can be applied to exploration. We're used to thinking of our space missions as being incredibly cutting edge and therefore largely the domain of government funding to drive forward science and technology. And whilst that is still true, there are other areas where our space programs are quite mature and we can look at different procurement models. And then I think the third area, we've recently announced a 150 million program with developing countries. So we have this program, we're engaged with 29 different countries, targeting 10 different sustainable development goals. And there's some fantastic real world benefits coming from those programs. And to my mind, that's a really interesting interpretation of the new space age as well. I was on a panel this morning talking about oceans. And with communities that we're not always speaking with and highlighting a project that we have in Malaysia where they're concerned about the mangrove coastline and the deterioration there. And just highlighting the fact that from space, you can pick up those signals. So the deterioration is coming from oil spills, from boats illegally flushing out oils. That signature from space is really clear. So using our space technology, we're really able to bring those down to earth benefits on earth. So I hate to ask this right away, but I do want to ask you about Brexit because you are the UK space agency and have worked as part of your European space agency. How does the UK space agency anticipate being innovative, competitive, post-EU? Yeah, absolutely. When I listen, space is an international endeavour. It will always be an international endeavour. We will always cooperate internationally for the delivery of missions and we will always cooperate internationally for the safety of the operating environment. We will continue to be a very strong participant in the European space agency. We're looking forward to the next ministerial meeting, that point at which we make new commitments to new programs at the end of this year. Bear in mind the last time we had one such meeting, it was at the end of 2016. We had already had our referendum and at that meeting, we were the number one contributor to all of the European space agency applications program. So telecommunications, navigation, earth observation. I think that's a really strong signal. Rick, I want to pull you in and talk a little bit about what Lockheed Martin is doing. Maybe we can focus on the Orion program because you're building a spacecraft in Cape Canaveral, I think, right at the moment. Absolutely. At the Cape, very exciting now. Orion is getting down that final integration at the Operations and Checkout Center there at the Cape, getting ready for next year's launch. And when reinforced, both at Allison Cerro said, it is an international endeavour. So what happened about a month ago is Europe delivered the service module that will be integrated with Orion and eventually fly. This will be the final test flight that will not have astronauts to validate the safety and the effectiveness of design and make sure we got everything right. We'll go out beyond the moon coming back. We're getting pretty excited about that and seeing that. And building on Cerro said about STEM because we're all struggling with driving kids and stuff and filling engineering roles and STEM roles. But that motivates seeing a launch, seeing that much more than maybe kids sitting and texting on the couch. But given that aspiration and what they can see and do, and I think that's a global perspective and I think every country, Alice mentioned 103 countries. We know it doubled. I can keep the count. But the pace of innovation right now and this new space age is huge, right? And that's what's key. Yeah, and so I want to we're going to get into some of the specific policy and other questions that you all are anticipating as we get into this next phase. Before we do, I just want to ask if you would also characterize what this kind of where we're at and how to how we all should think about say the next five or 10 years in terms of key developments and and Rick, just to continue with you, Orion is is meant to be a manned lunar mission as a as a sort of building block for the development of a lunar economy. And this is something that if all goes well, we can anticipate these sort of missions within a few years from now, right? That's correct. So, you know, I know the, you know, the US government and administration with NASA is focused on helping develop that lay the foundation, not just for the funding of Orion, which it starts with the moon and there's a lunar gateway, but ultimately have a safe travel to Mars. And we kind of not to just play around with words, but you know, people are talking about a race. We like competitive spirit, but where's the endpoint? Mars is not the end. That's a stepping stone. We want to explore there's too much to go look at out there. I mean, in our lifetimes. Yeah, I love to be in lifetime. But we're going to do it safely, right? Step by step. But yeah, I think in our lifetimes, you'll see not just going back to the moon, but staying. Yeah, and it's going to have to be multinational and international. And real exciting about develop those technologies, then we can go further out and be safer to get folks out and back safely. Yeah. So I want to ask you similarly, how would you what do you anticipate in terms of key developments? For now, probably an increase in international collaboration, because exploration cannot be sustained, like everybody said here, not be sustained by any one nation could be it can't be sustained by any one entity. And there's a lot of opportunity for actually new entities to come into the playing field. And that's the next direction next stepping point when it comes to exploration in general. And there's new business cases actually to be made. There's companies that that you were talking earlier, maybe if you can elaborate on that on the lunar economy, there's a lot of things that could happen to make status planter exploration viable. And there's a lot of roles and businesses that have not been created nor have they been defined. And that's the direction that a new space age could take. Yeah, Alice, what's your view? Yeah, I mean, very much echoing the comments that made already. And I think, you know, we will see an increase in international cooperation. You know, but we will still have competition. And it's quite interesting, because the competition is obviously is key to driving the pace, right driving the pace of our innovation. So we have to find a model where we can get that competition. But we can also get cooperation. And that will be particularly true of things like, you know, the lunar economy, you know, we're going to need, we need standards, we're going to need to enable interoperability. I'm sorry, everyone thought they were going to hear about exciting news page. I'm talking about standards about policy.  I want to dig into some of these things which so we have international cooperation, which is great. But we also there are some messier components of this and just to start at a very basic level, the oversight governance of the use of space. And in particular, you know, 2018, we saw a launch that wasn't a registered launch out of India. We have lots of questions about, you know, space junk to use the technical term and how and how we keep it this these areas safe and well regulated. How concerned are you all about the the state of the oversight in some of these areas? And what's needed next? Go ahead. Okay, sir. So we just talked about how many countries are coming in. There's over 900 startup companies Yeah, that I know of, probably through the US and some countries, and we probably don't have a full account. Yeah, you see. So the innovation pace and everything is is way out probably in front of the regulatory environment. So one of the panels and groups we've been with that the World Economic Forum is focusing on sustainability. So in the past, it was so bad, we didn't worry about we put up lots of small satellites didn't worry what happened left them up there. And right now we have to do it at a voluntary basis. But we need if you take a small satellite up, cubes at research, we need to orbit those. Yeah, and right now we'll probably set up some kind of voluntary framework. But at some point, you know, this is going to have to be a government entities will have to figure out what are the right policies. And deorbiting means that that you get them to burn up or come back a little burn up in the atmosphere as they come back down because, you know, you're moving a high velocity everything up there is very dangerous at that debris for for human travel. Yeah. Alice, what about you're probably involved. Exactly. I think it's fascinating. I think we will, you know, in the next few years, we will see a fair bit of regulatory reform. I think we have to appreciate that, you know, the regulation of the space environment has been pretty effective, actually. I mean, there have been incidents, but it has been pretty effective to date. But those structures were conceived at a time when the space sector was vastly different. You know, there was many, many fewer players, big satellites, the risk of collision was much lower. Now we've got nearly 2000 active satellites, we've got hundreds of thousands of pieces of space junk. So we have to find a new way of operating and just building on Rick's point, actually, I think it is interesting and that's why it's been a discussion here, actually, at WEF, trying to understand how we will keep the safety of the operating environment. And there is a real appetite from industry to take a lead here. So whilst the regulatory responsibility will ultimately always fall to government, there is also a real recognition in the industry and they are incentivized to find safer operations because at the end of the day, if there is a collision in space, then everyone stands to lose. And is there a consensus approach at this point? We're developing one. Okay, so stay tuned, Sarah. It's just important just to add as a caveat, any regulations that would need to come to place should take into consideration new entrants into space and not make it over, not make it beneficial to those that already have access to space because they have the technology to deorbit, etc. And on the other hand, handicap new entrants into space. So as has regulations been before, this is a new interesting time for plantar exploration. And this is something to look at when we're talking about regulations. Yeah, that's a comment. I agree with you a lot. We have to be careful with that. One thing we talked about in the forum there is for instance, I know NASA and I'm probably you said they actually put online how to do that. And it's a pretty simple system like in a deorbit case that you'd be anybody, any country can download at no cost. Go do that very simple system. So and I think, you know, as a community, we just have to make sure we don't we don't want to shut down that innovation and that participation. I agree 100% with you. Yeah. So there's another there's another so that's a basic framework. There's another group of issues that we're going to start confronting with some of these lunar missions, especially so the Moon is obviously a big focus for exploration now, especially with the confirmation of water on the surface.  Alice, you're laughing. I think your description of relatively ambiguous is very generous. That's very, very extremely ambiguous. I think so. I think so. And you know, none of us have the answer today. Yeah. My sense is, you know, where we're talking about, you know, any kind of interference with the moon, be it mining, be it resource, what have you, where we are doing that in the interests of all of mankind, where we are doing that in international collaborations as a stepping stone to enable further exploration activities, we may be able to find a consensus. Yeah. I think the difficulty will come is where those activities are being carried out solely for economic gain. Yeah. And that's a different... And my understanding is currently the basic governing principle is pretty vague and it's basically the principle of non-interference. So if there is an activity that's going on, another party can't move your probe or that sort of thing, which obviously creates complications as you are. Some regulations, so some of the regulatory frameworks, what they're currently looking at is to adopt something similar to international waters that we have here. That's discussions that are ongoing. I'm not sure where they went, but I think they're on one of the UN platforms on how to do it. But like she said, the ambiguous is very generous of when you're trying to explain that. If you have access to space, you can literally go there and do anything. And particularly as we get into things like a lunar economy, Rick, as you were talking about, where there are specific resources that are being cited. And some people already have identified specific real estate on the moon that has continuous solar power, effectively continuous the peaks of eternal light as being a potential places where there could be a serious land grab going on. Yeah, we have to build that economy and I think what we'll face in, as Al said, to these issues. I mean, if you think about a lot of these policies and government agreements were put in place a long time ago before we were going to have such a robust exploration. So as we move out and more people move out, it's going to have to be picked up by the agreements reached across government entities and regulators to establish that. Is the ISS, International Space Station, any model in this regard? Alice, you've obviously interacted with them. I think inevitably, there must be best practice that we can learn from the ISS, but I think we should also recognize we are at a different point in space exploration. Those models were conceived 20, 30 years ago. And non-commercial contacts, largely. The sector looks very... Limited number of public administrations involved in negotiations. Absolutely. I mean, I see certainly ambiguity is much closer to earth as well. So some of the really exciting things I think happening around innovations for in-orbit servicing, for instance, are really exciting, but our existing structures don't enable that new market opportunity, don't enable that new capability.  It's not designed to enable some of these new innovations, which I think will be very powerful for our space programs. So it needs everything. So in orbit servicing means that you're able to change your orbits and actually service the equipment that's up there. Absolutely. Or extend the life of existing missions. Yeah. Well, not too complicated. When we start doing more travel and more going out to space, you know, there'll be a big push to manufacture in space or on the moon with additive. That's why we hear people talking about asteroid mining. Not to bring back, but we need resources to go do that. That's going to open up a whole field of complicated. And is the ambiguity currently an obstacle to commercial interest starting to activate against the opportunities that you're talking about? Or there's still a little bit of time? I think there's enough time. I mean, we clearly, anything we do or, you know, we're working on one of the contracts NASA let that's a come, you know, how to take commercial capability to the lunar surface. I mean, they will, they will oversee that we make sure we comply and, you know, we always want to comply with all the regulatory framework. Yeah. But again, you know, as more people come in, it's like cited here with serenalist, it's going to be more complicated as we move forward. You can project forward 10 years and more capabilities. There'll have to be some new regulatory standards established. So right now it's things that nations actually take on to themselves where they say that just for the benefit of humankind, you're not going to do it. And whether or not there are signatories on some, on some of the agreements. For example, he's not a signatory on planetary protection, but our spacecraft has all the planetary protection standards and it's getting the necessary certifications from planetary protection at NASA and at JAXA. So at two, at two major space agencies, although we're not signatories. So right now we're at a point where we're relying more and more on the good of, of the players that are entering into space that they don't, that this is a valuable resource to humanity and it's, and we're pretty small in the larger scale of things and we are there, we are then not to do any harm to it or damage future prospects basically. So we're currently relying on the good will. I want to switch to another area which is about bad will or bad faith or conflict because this is given some of the military applications and, and potential conflicts. This is another area that, that we're going to have to grapple with as well. We're going to come in, in a few minutes we'll, we'll work through some of these issues and then come out to the room to pull you all in. So if you have questions we'll be coming to you, be coming to you in a few minutes. So how do we, how do we approach the space to make sure it's not a zone of conflict and tension and you know the issues of geopolitical aggression that are being discussed in every single other room in here practically. What's the, how are we going to do that? Alice, you want to start? I think the start point is you know we must never lose sight of the fact that actually it is in everyone's collective interests to maintain that safety of the operating environment. I mean it's, it's such an unusual sector in that respect where you know the aggressor can equally be the victim of his own actions. You know it's, it's not normal. So we really have to never forget to reinforce that. And I think, I think that will be at the heart of whatever measures we take. I do think, is that just to, on that point, is that different than the earth where, where it's you know arguably in no one's interest to, to initiate an armed conflict but yet it happens constantly. Yeah I think it is. I mean I, so I use the analogy you can think of orbits as like ring roads around major cities but if you have an accident on a ring road on a motorway okay that's bad but ultimately you can you know remove those damaged vehicles and you can reopen the road in space. There's none of that. You know you can't get the vehicles away and you can't stop more vehicles piling in from behind. So I do think it's at a different scale. Of course conflict is, is bad news. You know wherever it happens but I think the, the damage that you will do to the space opera, operating environment is very much affecting every operator. Now I think, I think what's interesting is as we see an increasing number of players, as we see an increasing number of active satellites and pieces of debris we will have to turn our minds again to how we manage you know the space situational awareness, how we manage access to that information. I mean that's, that data exists and it has been very much in a closed system with the US military and there's been very good arguments to do that but I think we have to take this point of inflection and ask ourselves is it still right that it is that closed system and should we be working through international collaborations to find a way where all operators can get access to the information they need for safe operations. Yeah, Ricky we're gonna No I'd say look uh you know I mean I also have very good points there with the bell on that we talk about space but there's a $370 billion economy or industry sitting around that that's here you disrupt space you disrupt you know your navigation system down here if you could use GPS the ability to do you know if you damage a weather satellite to protect life and property and get warnings out it is interrelated to you know how the entire you know global economic system is what is the $370 billion that's the estimate of the current size of the space economy if you will okay and that's primarily satellite communications and well the total wrap up yeah everything that you'd say is space between launch and everything else yeah by the little aside you know everybody's you know we see launch all the time it's really yeah it's really you know public sees it it's a inversely proportion it's only 3% of that total value 60% of that is in terrestrial systems and processing and the rest in space but the new space age if you listen to Morgan Stanley by 2040s to be $1 trillion yeah so to get there we don't want anything happening in space that would cost the debris that would damage our assets or anything because that would impact that valuation just to stay on you for one second and then maybe we'll pulse area but how confident are you that the conflicts here won't actually extend and the rivalries won't extend to there you have the two principal the leads on the space agents on the space station you have the US and Russia which are in some sort of new cold war you have China's own ambitions and I find it hard to imagine that the rivalries here won't actually spill into some of these areas particularly given the ambiguity so maybe Rick and then Alice well I'll let Alice probably comment the most on that I'm not an expert in this area but I think governments will have to talk about how do they manage this as more and more people go up there we are going to have to set up systems and have certain levels of security agreements we're going to have to make our satellites and things more resilient and capable because just getting more crowded up there there's even risk of some incident not even nefarious purposes just something happening or colliding or whatever so we're going to have to deal with this Alice mentioned kind of the space management of the things you know look at the look at the early days of the airline industry right you didn't you didn't worry about flying when there weren't too many planes in the air and then we had to create an entire air traffic traffic so and I think that's why the current US administration was leaning and put that in our department of commerce so that could be shared and work cooperatively with other governments so we can understand that but they I don't know if anybody has derived the policies yet of how you manage that do you do an altitude differentiation or a orbit regime they're all studying this now to say what makes sense for the future Sarah just to get your perspective on this there are all these conflicts you're a party to this arena but a different sort of power do you have confidence that there won't be a militarization of space and extension of the conflicts that are great apart from the examples that they gave our lives depend on space assets whether we know it or not yeah no one from the moment that they wake up throughout a 24-hour day does not use something that is in space and it's something that is very essential to our lives and this is something that is I think understood across the globe yeah and a basic example is every time you take money from a automated cash machine it's actually using GPS which is your phones cameras here television access to information access to weather just waking up in the morning knowing how to get dressed and getting out that's because there is weather satellites out there covering all the earth and I can only point to historic examples where this hasn't happened so International Space Station is a very good example today there has been so many conflicts with different countries that are that are part of that and it has ebbed and flowed but the International Space Station has continued to provide research and scientific outcomes regardless of politics and at the height of the Cold War the US and the USSR collaborated on a joint mission in space when they were both actually competing with each other on the space race so just due to historic evidence it is not to the benefit of any nation to go down that path yeah Alice and then we're gonna open it up to the room really just to re-emphasize what Sarah was saying actually because I think you know none of us have a crystal ball and it is evident that there's a lot of isolationist behavior on earth at the moment but we have a fantastic track record you know it was the late 70s it was the height of the Cold War and we had over 70 countries collaborating on a global scientific experiment which was to lay the foundations for the first global weather forecasting capability you know sensitive satellite technology built by US, Russia, China, Japan and Europe I mean that's extraordinary I think we can take some comfort from history here so maybe space will be the exception to humanity's conflicts so space should actually be a very good learning venue on how to do good politics yeah and look as we start doing human travel in space that's a dangerous endeavor yeah and we have to cooperate and then you have if you get someone in distress other nations are going to have to help each other out to the end is get you know protect the life of the astronaut and get them back home there are a bunch of Hollywood films about some of the scenarios that you're talking about so we'll see how it plays out I want to open the discussion up to the room here we have a microphone if you could stand up we'll bring you a microphone right here that would be great if you could state your name and if you have any organization affiliation sure yeah I'm Jaydeep I'm a global shaper from India I work on electrifying villages in the Himalayas interesting thing is the European Space Agency contacted us because they said now we have satellites with which we can monitor which areas of the world now don't have access to energy and so we are collaborating with them on a pilot project in India so just two points one is around regulation that you talked about and the way I think about regulation is once you put in regulation and often regulation is always catching up with innovation so suddenly you have drawback effects of putting in the regulation now imagine when people are flying hot air balloons suddenly you had Wright brothers had you regulated the Wright brothers to come up with that innovation they would not have been doing the things that they ended up doing which resulted in the airplanes coming into the picture so how do you see regulation sort of not playing the catch-up game and also not limiting innovation in that sense the second thing is if left to the governments every government after the five year or four year mandate the new government comes in they say okay we are not going to invest into space and I was reading this book which is about beyond space where they talked how the Obama administration curtailed on the space funding and now the Trump administration has sort of opened it so I'm not sure you know if you leave it to the governments how that innovation is going to come about the space age but for the private sector for the Elon Musk and everyone if you do not make it incentive enough to explore the space why would they do that so the entire debate about colonizing Mars or moon there has to be some economic incentive for the private sector to get into so okay so there are two parts of it Alice do you want to speak for the public public sector and then Rick maybe the second part of it the economic incentives would be something you'd have there okay yeah just to take the regulation for it it's an excellent question actually and it's very well observed that you know we've been here before history has taught us before and you're absolutely right over regulation is not the answer and today we absolutely see that regulation reform is not at pace with innovation and I think to my mind the answer is the answer is the same for many tech sectors actually you know we have to be careful not to be too special in space because all these other sectors are having the same challenges you know if you look at fintech you look at AI and the secret I think is firstly governments don't do anything in isolation right governments work very closely with the industry there should be a no surprises policy government should not suddenly introduce in any form of regulation that has not been long anticipated by the private sector that enables both sides to it gives government assurance on being able to you know take appropriate account of safety but it also gives the industry time to innovate and evolve and make sure that we're all operating within the same regime I think inevitably regulatory regime reform will all be slightly behind innovation but I think the key is that government and industry are working very closely together and there is a no surprises no surprises and rick the economic incentives maybe I'm going to build a little bit Al said too is like we always talk about either or government or commercial and it's really has to be both it's an end function because governments invest and for a while build infrastructure and take down risk think computers think internet everything else so that's going on now but you're seeing the agency now push to open this up for commercial commercial is going to need a way to generate a return right whether it's resources whether it's some other capability or it won't hold now there's a lot of activity we mentioned 900 companies I think space angels estimate there's 18 billion in the last several years into startups 3 billion in 2018 alone they're not investing if they're not seeing something that might be there and I want to comment one more things I always get asked a question well you know we want to go to governments want to do commercial now not whatever we do commercial now GPS is a great example it starts a government system for a location but that sort of GPS and of course now as partners in Galileo and others that's engendered tens if not hundreds of billions of dollars of startup industry and everything else there's a great example of how that works together now the first time we go out so let's say go to a lunar environment that's going to be some different models come up different business models people are going to figure out to get a rate of return and the specific commercial activities that people are focused on right now are as you mentioned resources so natural resources mining asteroids the moon wherever and space tourism right isn't aren't those the two primary current ones people are thinking about yeah that and others there's infrastructure ones like I mentioned you know how do you do added manufacturing in space yeah might be you could argue the derivative of you know of the actual effort yeah whatever yeah why don't we let's take another question now there's a question in the front here maybe in someone before okay do you have anybody else who wants to thank you hi I'm Ishten Alson from Quartz2 I kind of like your optimism that all the politics we have in the on a weren't spread to space but I think one of the things we're talking about this week a lot is how governments are changing and they are becoming much more nationalist than even if you don't have a nationalist head of state if you have domestic issues you're forced to focus inwards I mean how can you be so certain that as we go forward that none of those conflicts do do spread over and your point that the US has the data and it needs to be opened up you know right down in the middle of a right now in the middle of a government shutdown how can you be sure that these issues will prevail over domestic problems that we have at the moment thank you you um yeah go ahead well I mean yield to them on the being probably better expert in answering this but look I you know practically speaking whenever we have gone off and explore to come together many people there's always been some level conflict we've had to set up security protocols to deal with that that's going to have but Alice mentioned earlier very well I haven't heard the highway analogy with that for but you know the consequence of that space is dramatic and it'll it'll interfere with any expansion of commerce any expansion of use of space so you know I'm optimistic to get you know though we always have our challenges and things that you know governments have to come together for the good you know of the earth you know there's a lot of talk about let's go live on Mars yeah um it'll be a long time since until there's a golf course on Mars I'm just saying so probably some other things before they get to your golf course but uh but you know and you watch all the discoveries that are going on now and NASA and ESEM plants we haven't found one like Earth yet so it's a pretty good planet we want to use space to make sure we monitor it and protect it and do everything else and stuff as we go out and explore and I think that that will drive everyone to understand that this has to be multinational as we go forward Sarah, do you want to it's just it's the same echo the risk of not collaborating and not working together found far outweighs any nationalistic approach when it comes to space you lose a lot if you don't collaborate you incur additional unnecessary costs if you don't collaborate so would it make any sense for any country they don't want to collaborate to build another navigation system when you've got already three that exist around Earth that are very it requires a lot of satellites for example to have a navigation network doesn't make any sense same thing with communications and so on it just doesn't make an economic sense to do it alone yeah I'm very much built on the point Sarah made earlier which is around awareness actually and when people understand the societal dependence that we have on space already that is a very very powerful you know convener for cooperation I mean this I don't know if anyone else was lucky enough to see the preview of the David Attenborough film earlier in the week where he had these really dramatic images of ice sheets collapsing you know it was very very emotive actually and I think it was just last month that NASA released the you know 25 year data sets on the ice sheets you know it's such a critical input into our understanding of climate change and the oceans I've really enjoyed this week participating in those discussions about oceans discussions about how we are going to protect our water resources and I think as that message gets out as that awareness increases and you know through our program in particular working with developing countries we're very often working with those governments you know providing data on you know sea surface heights wave heights storm surges in sea surface temperatures and with that evidence base then you can start putting in place adequate coastal defenses for instance so you know we really are across the world all dependent on these technologies so I think that awareness piece is very important even if we go down to actually exploration exploration is not driven by government sitting on a desk and agreeing to it exploration is driven by scientist curiosity yeah and they drive a lot of the international collaboration where prior to it becoming a national objective the scientists have already done their job created the necessary collaborations know who they will work on know who's going to develop the instrument prior to a mission being formulated and it's those relationships that are actually that actually outweigh in exploration at least they're probably less likely to be nationalist for example science is a collaborative effort that has always been that way we have time for one or two more questions and then we'll come back to the panel and finish up up in the front here we have a question a comment something a contribution hi I'm Anjara Jackson I'm a physician from the United States two questions one is you mentioned that there are about 900 upstarts in this arena what exactly is their business and then my other comment a question is even with the best of intentions and cooperation once you colonize have commercial enterprises conflicts going to develop so who's going to what will be the laws of the land and who's going to enforce that yeah great I'll start with the 900 company yeah okay go ahead 900 what are they doing they're across the board so there's many that are doing small satellites you know from you know remote sensing and from imaging such as planet labs if you're familiar with them there's other companies doing radar sands other companies do atmospheric sensing and different things like that even some small weather that are trying to miniaturize you know the there's a lot of a big boom area sixty some percent of that market sits in the terrestrial but data management and processing and all that getting really unique because once you bring that together and put that in the cloud then we can apply AI and other machine kind of learning techniques to really better our understanding of many disparate systems in those areas so even for spacecrafts development in general no one entity develops everything that that is in the spacecraft so there's a lot of companies that are actually subcontracts are part of it that would specialize for example on the processing capabilities on board a satellite so that's a company by itself that that that develops just that form of application so even prior to launch even prior to the entire device you have a lot of players just going into building one spacecraft regardless of what entity is is doing the overall system so that could come back that to build on that that's very true for instance in for for Lockheed Martin space sixty cents of the whole of about a dollar of the whole program goes into our supply chain as a matter of fact sitting here we're processing right now a satellite for Arab side the Kingdom of Saudi Arabia it's down at French Guiana getting ready to go on an area on launch one of the major elements was built by McDonald Debt while they're out of Canada and if you follow the chain I will guarantee you there's at least 15 countries that we acquired pieces for so space is already pretty collaborative pretty multi-nation however you want to define that just in countries is pretty normal it's just if you want to develop any space system you'd at least go through a handful of countries the second question was about how do how do how to deal with conflict what's the who has jurisdiction how are these conflicts resolved before we get to that I just want to follow up on this question about the 800 companies we haven't spent a lot of time talking about the blue origins and the SpaceX and the Virgin Galactic and the private space operators who are have been focused at least initially on space tourism and launching satellites and and what their impact has been on the speed and the and opportunities in the you know in this field and Sarah you may have a sense of that given given kind of where you sit so first off just to take a background a lot of these companies are supported by by the government especially on the first side not not a lot of people declare that but a lot of these companies would not exist without government funding one primarily to alleviate the risk that is to get into there just having the the comfort of having a guaranteed contract once once you're once you develop whatever it is be it a launch vehicle or otherwise does alleviate a lot of the risk in starting those businesses and regardless of their overall goal whatever those companies are they are developing them with commercial eye and a commercial eye when it comes to that so we will still see government play when it comes to funding for space but at the same time we will see more and more entrance from the private sector in areas where the risk has been alleviated and there has been an economy created out of it yeah Alice do you think that the these entrants have accelerated the opportunities or yeah absolutely but I think Sarah's observation is really spot on as well that actually about the role of government I mean you know to put it very simplistically you can think you know there's there's the effort that needs to go into the early R&D then when you've got you know your idea you're accelerated then there's the effort that needs to go into really commercializing that product and then there's an effort around the you know gaining access to a new market opportunity and I very much see the government intervention at both ends of that scale so where it is you know blue skies high risk cutting edge technology I think it is largely the preserve of governments to support that activity where you're trying to commercialize something that you've already developed I think there we look to the private sector and we need to you know support the awareness of the benefits of space to enable that commercialization to happen successfully and then again at the end I think government has a role in in in regulating whether there's a new market opportunities making sure there's a level playing field making sure it's enabling yeah so the second question if I'm characterizing it properly was around conflict and conflict being inevitable if you assume that how what's how is it resolved yeah gosh I knew that I'd be a rich and you figure this out I knew that I suppose I can only reflect back on the earlier answer I think I think we will see you know mining for exploration purposes in space and I think where we can coalesce around a common goal you know that common goal being furthering the knowledge of mankind then we should be able to find frameworks for cooperation I think where there's an activity which is purely around commercial gain economic gain it will always be more difficult so I I certainly hope that we build upon the very strong international corporations we have already to make sure that we're really focused on that furthering the knowledge of mankind just to add on that I think the discussion currently is premature because we have a lot of unknowns when it comes to a lot of these businesses so the technological capabilities are still not economically viable to make it a to make a worthy business case out of it and due to these other so many unknowns I think it's very premature to put any regulations or conflict resolution mechanisms in place without us better understanding the the scene that's going to happen in the future it is something that people are currently exploring but it's not something that has still found a value out of okay we've time maybe for one more one more question can you wait for a microphone and you can stand up to everyone hi my name is Nikhil I just wanted to follow up on Sarah's point about making sure that the regulation does not handicap the new entrance to this club and that's because of just one very it may be trivial but an illustration from this conversation where the question around the startups in the US was characterized by you know how are they accelerating innovation and that about you know the unauthorized launch and the launch in India last year being unauthorized when frankly the unauthorized part was between swarm technologies and the US government you know the so I just feel that you know the way in which some of this conversation actually happens the burden of regulation actually comes around much harder on the new entrance and I'm wondering you know what's the way in which the global community can actually have a conversation about what are the rules of you know whatever minimal regulation that's required yeah I think your question is one that we've been kind of trying to grapple down throughout this discussion which is what is what are the forums what are the baselines for actually creating a level playing field as Sarah noted not just among among nations and and the various players how would you all how would you all kind of summarize kind of where you where you see the best solution to this I'll start by just I don't know if I had the best solution but I'd say look we're talking about a new space age for a new space age you exist we need a dynamic marketplace with many companies all through a very robust supply chain interacting so we need to bring people in with that said it has to be sustainable their existing processes and are not that difficult to go get approval if you're gonna fly whatever so people know about it because of that traffic pattern problem I think there's NASA probably has it I haven't checked that particular issue or someone to give the government agency can put that in a website or something but these are established until more more robust processes come online let's say downstream but the US government's policy that I know has been and always and through NASA and NASA administrators they want to sponsor and they want to engender a very robust commercial sector to propagate to the future now there's gonna have to be some rules of the road so we don't harm each other as we go through that would be how I'd start with that so even on an international venue there is an international venue convened by the UN and I can't remember its exact name the United Nations Office for Outer Space Exploration or the Peaceful Use for Outer Space excuse me for not knowing the exact name but it's based out of the UN headquarters in Vienna sorry the UN office is out of Vienna and they do convene with major space players so current space players are members and then there are other actors can become observers off that and that's where a lot of the perhaps agreements that a lot of the nations basically are there and the right people within the nations that are that are to set regulations are also sitting on the table and this is something that has benefited the UAE being an an observed so since we started we were an observer status and then transition to becoming members of it but that has benefited us for having a seat for dialogue and more and more nations that are that want to become players in this in the space field could utilize that form for them Alice? Yeah Sarah is absolutely right you know it's the UN copies that's that grouping that is the legal cornerstone if you like of all our regulatory activities I mean and it has many many strengths you know not least inclusivity where it struggles I think is the issue we were talking about before about keeping pace with you know innovation and because of the nature of that construct as well you know the the challenge is all always around finding the lowest common denominator the lowest area where everyone can agree and because of that sometimes the thresholds are quite low yeah so that's the gap that we need to fill and I think again we shouldn't underestimate the contribution of the private sector to do that we've heard already that the the multinational nature of supply change the integration of the industry globally already and I think we really need to tap into that it's why I'm very I'm very pleased that we're having this discussion through the World Economic Forum as well because I see that as a really you know adding a huge amount of value in being able to get that industry voice heard so you all so we're we're just about out of time but each of you has some milestones ahead and I wanted just in closing sort of recap so that people can focus on those Alice maybe we'll start with you you talked about launches out of the UK yeah in the coming decade what's the what's the time frame we should be looking for yeah absolutely so we've stated ambition is the early 2020s okay early 2020s you know that's within five years sounds sounds roughly that would be early 2020s come on that's okay it's close it's close I also I would say you know the second ambition as well is really to make sure that you know through our spending processes over the next couple of years as well we can also really embed the the huge range of applications that there are for our space service as well I mean and that's incidentally why a second initiative we've taken through WEF actually is to generate a a prize you know an x prize type construct where we will be looking for those latest novel solutions for using space technology for sustainable development goals I hope that that's something that we see an increasing trend over the coming years yeah and Sarah so you have your Mars probe which is scheduled for launch next year have you have you thought beyond that yes so the WE does have quite so we've launched our first home developed spacecraft ended last year yeah 2020 is the mark for us launching our spacecraft to Mars in July to arrive on Mars to arrive to Mars orbit in first quarter of 2021 the WE has also from from perspective of an overall collaborative international efforts has also sort of put its hat in the ring saying that that we're we're going to be part of the over exploration mission by launching a long arching Mars destination Mars that that is moving forward again our Earth observation program is continuing there's also a program for putting an astronaut on the international space station and destination Mars means the station Mars is about the same endeavor that everybody's looking into colonizing Mars but more importantly we're focusing on areas that are important for us in the WE just because of our environment so we're looking at if we're if we're going to send astronauts to colonize Mars and live there they'll need to have a sustainable source of water and food and for us in the UAE water is is quite scarce and food is very hard to grow we're going to have to wrap up Rick Orion you have a scheduled non man launch end of the year but we're in a fast low we have three commercial launches during the year we're going to get the second GPS three up in May June timeframe do Orion and it is a robust that having business right now I do want to mention Mitsubishi though I see some of the audience here we talk a lot about a lot of these other launch systems we've been doing but don't forget Mitsubishi has now their H3 vehicle they're launching which will help the whole system you know bring launch costs down have access to space great okay we're going to wrap it up here a lot of these questions we're getting at early which is probably a good sign before their scuffles on the lunar surface over the peaks of eternal light and other things and thank you for this great discussion to to sort of tee up these discussions that obviously you're going to need to intensify in the in the coming years so Alice Sarah and Rick thank you thanks for having us thank you


Well, obviously this is a really big deal for me and my team because it was about a year ago We decided to become entrepreneurs and start doing digital news So to receive recognition from an organization like Enba really makes a difference It validates the choices that we've made it validates the kind of work that we've done and it really makes a difference to us So thank you Enba As far as Enba is concerned This was a particular event and exchange for media as an organization that I always had a very good relationship with while I was in television as well So it's amazing to have to be able to continue that relationship as I've transitioned to being an entrepreneur I thank Enba for recognizing digital news and for creating a category for digital news I hope that this category then grows into a whole IP on its own and We can build a brand new sort of recognition and relationship for all of digital news through Enba Thank you so much to exchange for media, Mr. Raghav Batra and to the entire jury for this award. Thank you


Well, I guess welcome all I'm I'm mark seager with HP cloud services and I'm a performance kind of guy and I do a lot of performance monitoring and visualization and such And I'd like to talk to you for the next hour 45 minutes whatever about some of the things that I've been doing in our own HP environment for Swiss monitoring and Basically, I'd like to first start out and tell you what problem we're trying to solve because I think that's often a good a good starting point I'd like to talk to you a little bit about this open-source tool. I developed a number of years ago called collectal and Then jump in and start talking about what we're doing monitoring swift and glance environments and also a little bit about what we're doing monitoring Nova environments So that said There's this interesting Conflicting problem goal statements requirements, whatever you want to call them At least I think they're conflicting and it's also it's also kind of a religious topic the notion of centralized monitor centralized monitoring versus, you know, node-centric monitoring and The way I see it is if you want to do centralized monitoring It's certainly it's certainly a great idea, but the problem is if you have thousands of Machines that you want to monitor and you want to Collect a lot of data you either, you know, it's one of these things You know, you get a lot of machines you get a lot of data You collect it frequently, you know now pick two or maybe even only pick one because you just can't do it All at the same time So you have these centralized monitor folks who want to be able to see what's going on in the entire environment and At the same time you have these other people who are interested in getting as much data as they can as frequently they can and They're two very different problem statements. There's there's no real intersection that I know of So in reality what you wind up having to do from a central monitoring perspective is You have to pick a subset of data that you want to monitor pick a monitoring frequency that's not too fast and Collect the subset and Then the dilemma is if you're a If you're in the support organization, you're trying to support a situation where the system crashed or you're having some performance issues Now is the time that you want some really fine grain data You want to see what's happening every five seconds or every ten seconds or every one second But the centralized monitoring solutions only collecting data maybe every one minute or every two minutes It may not be collecting the level of detail of data that you need to solve your problems And this this is what I think creates the conflicting problem statement I mean I'm I would guess that there really is no single solution where you can take a tool put it somewhere and do both But I would claim the solutions obvious if you need if you have two separate types of requirements You have two separate kinds of tools you pick one that's going to run centrally And it's going to monitor a small set of data every minute or two And you take a whole a different tool and you put it on all the individual nodes You let it collect all its data as frequently as one as much as it wants but that kind of has problems in itself because First of all is the data going to cross-correlate if you're collecting If you're collecting all those data centrally or the numbers that you're getting going to match the data Is you're collecting locally because it's certainly possible that the two tools? Oops, sorry about that The two tools may actually be Monitoring data differently even though it claims to be monitoring it the same for example I've seen tools that have reported system that have reported CPU load some tools include the IO wait Counter as part of the CPU load other tools don't so if your centralized monitoring tool does report it Your local monitoring tool doesn't report it. You go to correlate the data. It doesn't match that. That's just one example But then the other thing is what if you want to do some customizations and you want to collect some custom data centrally So you modify your central tool to collect the central the data locally But now your remote tools don't collect the data and again you have a you have kind of a mismatch So what we've kind of done in HP? We've kind of chosen a hybrid model and what that hybrid model says is you have a centralized monitoring tool and it collects a lot of the data Centrally and we have a second tool that monitors the data remotely But the remote tool has the ability to feed the central tool with the subset of the data periodically and As a result at least when you want to add some custom data to collect in the node level space That data will correlate exactly with the data you're collecting centrally because they're both collecting the same data from the same source That might make a little more sense is in the next slide or two But the thing is kind of interesting and this is totally a this is totally a coincidence Our centralized monitoring tool is called collect D. That's what a D at the end And our other monitoring tool is called collectel with an L at the end There's absolutely no relationship between the two of them and they just happen to show a lot of letters in common So what I want to do is I want to talk to you just for a couple of minutes about collectel because that's what I do and Also kind of give you an idea of the kind of information that we can get when it comes to fine-grained data So it has been around for probably a dozen years or more It's an open-source tool and it has its roots in HPC. That's high-performance computing So from the very beginning collectel had been designed to work on multi thousand node clusters collecting data at a very low At a very low Overhead so typically under a percent of a CPU sometimes under a tenth of a percent of a CPU depending on how you have it configured And one of its features is it coordinates its samples across the entire cluster to the microsecond or From Linux's perspective as close you can get to a microsecond so that when you're trying to Track down some kind of a cluster-wide event If you look at the samples on one machine and you see something happen that 1103 in five seconds You can find another node and then you'll see exactly what happened 1103 in five seconds on that note as well So that's that's that's really pretty important when you're trying to figure out what would hit, you know cross cross machine events One of the features that collectel has is it can actually generate data that you can immediately Plot some tools will collect data And then you have to like write a little script to analyze the data and reformat it if you want to generate plots Where's collective you can do it automatically, you know, you'll see a little later where that really is a handy feature The other thing it can do is it can actually collect data and send it over a socket To a remote monitoring station at the same time So that's another feature that it has that can help that can help address this kind of problem And it also has a built-in API so that you can add add features to it for whatever it is that you happen to be interested in That may not that may not currently be native to it And again, we'll talk about that a little bit as well And then there's a few utilities that you can use with collectel to it to extend its capability beyond a single node So real real quick, I don't want to spend a lot of time on this But collectel has this notion of summary data and detailed data and the thing that's important about this is summary data We'll talk about say CPU. So we're going to give you a summary of the CPU load, which means it's the Average load across all your CPUs You got to be careful because if you're monitoring an eight-node CPU and it tells you that you have a 12% load You might be fooled into thinking oh, I'm only using 12% of my system when in fact one of your CPUs Maybe at 100% and you're actually blocking so it's one of those things you got to be a little careful with The other thing it'll do it'll it'll tell you your aggregate network bandwidth So if you have two or three or four nicks, it'll add up all the bandwidth and report it as a single number It'll do the same thing with discs as you have multiple discs It'll add up all the disc I out reported as a single number and well on the surface You might think that's kind of a dumb idea because I want to look at individual discs I don't want to look at it. I don't want to look at just the total discs oftentimes in many instances It's a single device that's generating the load so by looking at the totals Without knowing which individual device you're interested in you can get a relatively good idea of what's going on But again, sometimes you want the individual sometimes you don't but the other thing that the other thing that this gives you Which I personally believe is pretty important as If you look at that top display, we're actually looking at the CPU disc and network load in a single line So that means when you're running this tool You can literally look vertically down a column and see change and to me That's one of the most important things about monitoring is you want to identify Anomalies abnormal behaviors spikes would have you and when you're displaying data on multiple lines where each line may represent a different a different type of device it's really hard to spot change and and That's the that's the hallmark I think of collectal and what I call brief and in brief mode you get one line per sample And that that's the key to keeping the back of your mind Verbose mode says okay. I can't fit this all on one line.  You see how much time was spent in nice mode? You see how much time is being spent on processing system interrupts? and And a couple of other per a couple of other parameters for for CPU similar things for discs similar things for network so that's that's kind of a Real-high-level example of some of the collectable data Detail mode simply says okay. You want to see the individual CPUs or the individual discs here. They are it's a lot harder to read You may not want to look at it in this particular format, but if you need it, it's there All right shift gears. Let's talk about Swift and Glance Currently there's really no good way to get to get additional information From what I've seen about what's going on inside Swift or Glance. They do happen to write a Transaction record in the API log every time someone does a get or a put or a delete There's a little record written into this log and it tells you what time to get occurred the the object size How many milliseconds it took to do the transfer? So that's kind of a cool thing So basically what happened was I wrote a little I wrote a little script That runs as a demon and it tails one of the log files It tails the log file on the Swift node or the Glance node and the reason I keep talking about Swift and Glance is Since their log files are almost identical. It makes it real easy to do the same thing for both of them instead of just one so it tails the log file and What it does is it it? writes a rolling counter into a static file so Every second it says here's how many gets I did Second later. Here's how many gets I did here's how many gets I did and it's not it's not one of these things where I've done So many get since the last time you looked it's a rolling counter and this is the one thing that a lot of a lot of surprisingly smart people who do Monitoring stuff have not grasped the concept of a rolling counter and the whole notion with a rolling counter says As many people who want to read the counter can they can read it anytime they want And then they can read it again whenever they want and just subtract it to and divide by the interval It's really easy and it guarantees that that you know multiple people can read the counter at the same time You never clear it. It just simply keeps incrementing forever When it hits a maximum value it goes back down to zero So if you read one sample and you read the second sample and you get a negative number You add two to the 32nd and now you got now you got the difference so this little utility reads the logs counts the gets to puts the deletes, etc And it writes them to this little it writes them to this little file that looks like this It's extremely ugly that nobody would ever want to read But it's in perfect format for a computer to read for a program to read it and parse it and do what you want So this gives you the ability to Then take this data and visualize it and the tool that I used to visualize it with this collector So I wrote a little plug-in that reads the collect that reads this file generates counts of gets and puts at whatever interval you want and Gives you the ability to display it like I showed you on that previous slide with this brief mode and this verbose mode and that sort of thing and Then there's this other capability that collect all has that says I can I can send data to a remote machine Or I can even write it to another file at a different frequency that I'm collecting it That's the key at a different frequency So what we have going on here is Collectal is actually running once a second collecting this data and logging it and doing everything it has to do and then every minute It says oh, yeah collect D. Here's what happened in the last minute So you wind up with kind of both scenarios. You're logging it locally once a second You're logging it remotely once a minute the people managing the environment centrally can take a look at This high level This high-level information and if they see something weird happened at a certain time of day They could go to the individual nodes and look at the fine grain data to see every second what was going on So here's what you wind up seeing and again I don't know how well this displays toward the back of the room but what we're kind of looking at here is that What you have in the very top display we can see the CPU load the network load and The glance load or the or the swift load and we're actually able to see how many how many How many kilobytes and how many kilobytes it gets and puts per second there were and how many Operations there were i.e. the total gets puts deletes etc And the and if there were any errors that occurred the next level down is the verbose display Which as I said before it gives the ability to display more stuff and fill up the whole line And now we're actually able to see how many gets puts deletes posts etc. There were This display down here the one second from the bottom that one's kind of cool because this can tell us of your of your gets how many of the gets This is the network banned with one. So this is actually this is actually telling you How many of those objects were you able to transfer? Between zero and ten megabytes a second between ten and twenty megabytes a second between thirty megabytes a second You can you can get a feel for what kind of load Swift is putting on your network or a glance and in the bottom display actually says of the object sizes How many object sizes were between zero and between zero and one megabyte one megabyte and ten megabytes? Ten megabytes and a hundred megabytes a hundred megabytes and a gigabyte So actually you can go back at some point in time and say jeez if there was a problem at a certain point in time What kind of transfers were in flight were these large objects small objects how much bandwidth were they using? Okay, a little bit of a gear shift again. I want to talk about Nova Same kind of scenario. I wrote a collective plug-in that Runs on the Nova server, so it's not running in the VM. It's running on the server and this guy is looking at VMs and the thing that's kind of interesting is If you look if you look at the command line that started the VM in the command line it Tells you the Nova instance ID Which is kind of neat and it also tells you the Mac address of the virtual Nick Well given the Mac address of the virtual Nick you can look in some of the System tables and figure out what its network name is like the net underscore 13 or 18 or what have you and then you can go inside collect them and say collect oh What's the bandwidth on? This particular virtual network and then in this little plug, and I wrote you can kind of put it all together So what we're looking at? Stop that so what we're doing what we're looking at here is this is a machine that has five Virtual machines running on it and for each virtual machine. We can monitor the CPU load the This is this these two columns with the BCK that kind of stands for Bach, which is our own Block storage that we can allow VMs to mount so you can monitor the block storage you can monitor the the local disk activity You can monitor the network activity and you can tack on to the end of this the instance ID and then using like Nova Manage you can then figure out who the user is it's associated with this and It gives you a really nice view of what's going on on all the VMs on all your Nova on all your Nova servers And again this data here in our case is being collected every five seconds, but we're also able to send it Remotely up to collect D so that it can display this information once a minute So again you get the coarse grain at the operation center You get the fine grain on the individual nodes in case somebody needs to drill into that data some more so that said there's still a couple of missing pieces here and basically what it is is I I wrote some more software that kind of says I still want to Visualize some stuff centrally Based on all the stuff that I'm collecting all these glance nodes and and Swift nodes So let's use the cloud to monitor the cloud and and what I really mean by that is There's a bunch of calculations that you can do on the individual nodes and then remotely and parallel Copy that data up centrally and if you're only doing text and if you're not dealing with databases and stuff The this can be pretty fast So the oh and the other piece that I'm also doing is Actually rendering some of this collect all data into plots in Parallel on each of the machines so what this all turns out to is when you pull it all back You kind of wind up with this what I'll call a crude but useful on Interface and again, I apologize if people in the back or having a hard time seeing this But basically if you look in that upper left hand box What we're seeing is a summary by day. So the the labels didn't I left that off here when I when I made the slide but we're basically looking at a week's worth of Of proxy operations and you can see that we're doing on the order of some 20 million operations a day and If and and everything on this upper left hand display is a hyperlink So if you were to click on that link For like the 167 e the e was an experiment of mine that said when I see errors I'll put an e next to it and of course I forgot the number one rule of Larging large environments everything's always going to be in an error state So it doesn't really provide a whole lot of useful information But if you click on that link here's what you see and what you see in this link is you get to see the 16 proxy servers that contributed to that number of I Guess that was 10 million operations that day and you can actually see over here the name of each machine How many operations it did how many gets there where how many puts there were how many deletes there were and again? Visually you can start looking for some anomalies and things and you know for example According to this on proxy 3. There were 6 that's 6 million gets on proxy 3 And none of the other proxies did that many gets you know, I don't know why it just happened to happen that particular day One of the other things that you can do with this display each one of those node names is a hyperlink Actually, I don't think I think Not yet. We'll get to that one But if you back up here again, this display is actually a lot wider than would fit on the screen So I've broken into two displays So what's to the right of this display that you're looking at is a further breakdown of How many 401 errors there were how many 402 errors there were all the different error codes and how many different errors there were as Well as this is the get-and-put network bandwidth about how many objects were I'm not the band with the object sizes So that first column that says get one is telling me how many objects were on between Ten megabytes and a hundred megabytes and the get two is how many were between ten megabytes and a hundred hundred megabytes and a gigabyte Etc. If you were to then Click on one of the CD all these error codes if you click on one of the error codes You can actually go into the error log and see the exact error messages that generated those error codes so back to These node names up at the top that I said were hyperlinks if you click on one of those node names You then get this display that breaks out the gets the puts and deletes by hour so now we can start drilling down a little deeper and for example looking at this display you can see at five o'clock at five o'clock there were four hundred and four posts and All the other ones were single digits or or barely single digits. So again All all interesting data that might be worth exploring in more detail Okay, a little more of a gear shift I mentioned some of these utilities that go with collect all and one of the utilities is this thing called call plot So what I was saying before was every day or every night call plot goes and Generates a whole bunch of plots from the day before and uploads them all to the central server and What that means is you can statically display these plots and By statically displaying the plots you can look at hundreds of plots all at the same time Very very quickly if you have to render them when you want to display them it gets a lot more expensive and a lot slower So again this little interface that I built which is very crude turns out that like this a AZ1 and a AZ1 2 and 3 are our production servers and then the rest of these cist test ones cist test to R&D These are all our test clusters So you can pick which which type of data you want to look at and then say I want to look at Bach data database data Glance data Swift data, etc and then you a little click on that little lower left hand button and It'll say okay, you wanted to look at this data What kind of plots do you want to look at and it just has a whole bunch of different links to a whole bunch of different plots? and since and since we've been talking about Looking at glance and Swift Operation counters which in this case are the gets the puts the deletes, etc. I Highlighted those two is the kind of plots you could look at and when you actually click on one of those buttons up pop these plots and They're very very detailed you really can't tell it from the display, but each plot has 86,000 data points on it because it's one a second for an entire day and I Guess PNG is a very impressive Technology because each one of these images is only 10 kilobytes I'm still not sure how they fit all those all them pixels into that one little image But that's how I I checked it multiple times and they're really only 10 kilobytes So these these these plots render very very fast. So again in The back if you really can't see what's going on here We've got a lot of different colors on the plots and one color is the gets and one color is the puts and one color is the deletes That's the top three plots, but when you when you Bring up the web page. There's really like 16 or 20 plots on a single page And you can very easily scroll up and down to compare what's going on and The plot on the bottom is actually showing me my put Rates of objects greater than a megabyte greater than 10 megabytes greater than a hundred megabytes So if you could envision a plot That's showing this kind of data as well as CPU load or network traffic or whatever Or disc load you could you could then start saying things like you know jeez. Here's this spike You know, I can see this spike right before 1600 and you know that spike goes straight up through the top and It's like whatever happened, you know, someone did a get no, I'm sorry they did a put on a large object and It probably crossed those three different servers because I can see the three different spikes on those three different servers Okay, another little bit of a shift gear There's another tool that goes with this environment and it's called call mux and call mux stands for collect a multiplexer and It has an interesting property and the property that it has is it will allow you to run Collectal on multiple machines of your choosing It'll take all the output from collect all and Display it in a single display much like top Okay, but the big difference is instead of looking at top processes top processes You're looking at top anything So anything that collectal can report you can display in this form and you can sort it by any column of your choice So in this particular case Again, we're looking at Swift data So we can actually look at the gets the puts the deletes Across all the proxy servers sorted by any particular column and That'll and that can help you identify proxies that may be that may be a little hotter than other Proxies that are you know getting more more their load than they should or you might identify proxy servers They're getting less of the load than they should or if this was a Swift object server You could actually say I want to look at the disks on these devices and you can sort them by which disks have long long wait times or big I oq's or long service times and again you can look at them across dozens or even hundreds of nodes all at the same time and By the way, this is part of you know the collectal open source stuff. There's nothing. There's nothing Cloud specific except in this case I'm showing you how it works with some of these plugins that I that I wrote that that work with Swift and glance and There's a second form that actually See that the good thing about the top guy is it lets you look at the top activity At any point, you know every every second when it refreshes the biggest problem with a tool like top Whether anybody's really noticed this or not when you run top It'll tell you the top the top process at this second Then the next second I'll show you the top process and it's like wait No, was that the same process or a different process or the process is changing every second There's the same process always the top process and you can't really tell because there's no history with top It's always instantaneous. Well the second form of call mux Gives the ability to say, okay, I know I can't display everything Historically, but I want to pick one or two or three data items and I wanted to display those Every second on every machine. So this display over here at the bottom is Saying I want to look at six machines and two elements I want to look at gets and puts and now every second it displays a new sample So now again, we're into that whole business about being able to spot anomalies with vertical columns Every row represents a complete sample across all the machines that you're looking at And again in this case, we're only looking at six machines. So it's not particularly exciting A few years ago I was uh, this was back when I was doing high performance computing I was at a customer of ours who um, who had a 2000 node cluster and they had Four 40 inch monitors or 50 inch monitors I mean it was like mega wide and when I saw that I said wow, I gotta run call mux on this So I ran call mux on it and I was monitoring. I think it was 192 compute nodes So I want to show you a picture of it and I want to caution you before I bring up the next screen You're not going to be able to read it But that's okay. I can't read it either But what you can do I hope is shape recognition Because it makes it really obvious. I think what's going on If you look at this display right here Without even being able to read stuff You can see those guys on the left aren't doing anything. Oh, they're all zeros You know the guys over on the right you can see where it gets a little a little lighter and a little You can see a burst of a lot of cpu traffic. You can see, you know, this solid band Where it says very busy. I don't know what the numbers are but you can see they're all pretty busy And then you can also see in the in the right hand side of the middle monitor It's like wow things are really erratic here. I don't know what the hell's going on But this is some of the kinds of stuff that you can do with call mux And displaying things as a single line Sometimes it'll help, you know, you get to see properties that you couldn't see before So again, this kind of takes it all back to this whole notion I think I hope of central monitoring versus local monitoring Centralized monitoring you really don't care about this type of data that often unless there's a real problem But having this data available locally can really provide a lot of insight into what it is you're trying to look at So that's kind of that's kind of this dual Schizophrenic model that we're using at least with some of our data So does anybody have any questions or No, no, I I do text I mean I mean numbers Yeah No, that's that's that's from the nick If the the one thing that that may or may not be obvious With with node level monitoring is almost everything is available in slash proc If you've ever looked at slash proc, there's like a ton of data in there So they'll so like if you do If you do what is it like if config? You'll you'll see some high-level counters and other stuff all that comes out of slash proc And that's the same data the collector was reading. Okay. Good. Thank Yeah, no, that's a good. Yeah, that yeah heat maps is something I've only just scratched the surface on one of my colleagues had done a lot of taking some Collectile data and sticking them to heat maps and you uh, you see some very interesting patterns arise that you didn't normally see Yes, sir Is this on I guess so This would be a good point a time to point out that uh, swift has stats d metrics emission as well So A lot of the similar data is available through that avenue as well And where does that data come from? It assuming you configure the proxy servers to send it They're emitted as stats d udp packets So you're still you're still responsible for running a stats d server So you can there's a number of ways to deploy that you can go with a central server We do one per node And then aggregate As a second step how often does this send out the data? I mean it's configurable Oh, sorry the the data is sent out real time the udp packets and then you configure your stats d server to sweep through and Yeah aggregate and flush so it's it's the It's the collect aggregate flush cycle as opposed to rolling to roll encounters Once the data gets into whatever Stats d stats d is upstreaming into whether it's graphite Which is sort of oh, okay Dude, actually, there's also a collective interface to graphite So you could send it to help out collect all have collect I'll send it to graphite I guess the only I guess the only thought that comes to mind is what the whole udp thing is if you're a lot If you're if you're not listening then you're gonna miss something Right, right It does have that property But anyway, it's uh, it's in the code base and Okay, if yous Okay, cool So congratulations. It's it's wonderful work I was wondering how much of what you're planning to contribute back to the community Excuse me. I was wondering how much of what you've contributed what you've created here You're willing to put back into the community. Oh, I If you were one of my earlier talks, um, I'm looking into this. I'm hoping that I'm not going to have a I'm hoping Not to have a lot of issues trying to get things forward A lot of the stuff that I showed you was already open source The one challenge I might have with the community Actually, this is as good a time as any to ask it. I I'll make a statement. Then maybe I have to hide under the table. Um, it's all written in pearl When um when I first started doing all this stuff That that was the that was the language my group used So all of collectal and all these plugins and all everything is all is all based in pearl and um Unfortunately, collectal doesn't know how to talk to python So all the plugins have to be written in pearl as well But depending on how amenable the community is To uh, non, you know non python stuff It's it's not all that it's not all that complicated But it's certainly something I would like to look into making available to the community so Perhaps we can get there. Alrighty. Thank you. Mm-hmm Anybody else have anything Okay, well, I guess um, I'm not I'm I might be a couple minutes early here Yeah, shame on me