the State of Clean Energy. I'm your host, Mitch Ewan. Our underwriter is the Hawaii Energy Policy Forum, which is a program of the Hawaii Natural Energy Institute. I'm very pleased to welcome our guest and my very good friend, Dave Donald, who is the founder of Hawaiian Blue Volcanic Coffee. Today, Dave's going to talk to us about a specialty coffee adventure at Hall Farms in Waimea on the island of Hawaii. Dave, welcome to the show. Why coffee and why Waimea? How did you get started? Well, Aloha. Thanks, Mitch, for having me on your show. Appreciate it. Why Waimea? Well, I moved to Waimea, so that's where I was living and where I was living. It was a small little plant, two feet tall, and I wasn't sure what it was because I wasn't that familiar with coffee at the time. It was back in 2016, and I asked someone about the plant, and they said, yeah, that's coffee. And I said, how long has it been growing there? And they said, oh, about 25 years. And I said, well, isn't it kind of small? And they said, yeah, well, coffee doesn't seem to grow well in Waimea. And that just challenged me. And six months later, I managed to get it up to around six feet and the flowering and the fruiting for the first time in 25 years took off from there. Pretty awesome. So let's look at the first slide. And this is your awesome logo. I love it. Talk to us about that. Essentially, coffee flowers. So really, you know, Waimea is a bit of a desert. It's considered a desert yet you were able to grow coffee trees there. Tell us about that. So there's a wet side and a dry side to Waimea. And on the dry side, there's some farm lots called La La Mido farm lots right next door to Parker Ranch headquarters. And it's definitely considered desert because it rains less than 10 inches a year. That's the standard. And we got about eight inches rain a year. But La La Mido farm lots has a well established irrigation system designed for the farm lots back in the 1950s. And so with that, there are many successful farms here growing various sorts of leafy green vegetables, some flower farms, tomatoes, things like that. And I was just given the opportunity to take some of the seeds from the mother tree and plant them on the farm. And they took off and they're growing really well here. So if you look at pull up slide three, this is what Waimea is supposed to look like. Horses, trees and beautiful hills. But I don't see any coffee trees there. They're close by. They're not in that shot. But I think you've got some other shots of the coffee trees. I do. So you pulled off a miracle basically. Nobody thought you could ever do that. Coffee just doesn't grow in Waimea, right? That's what folks said.  But on the dry side, you won't see any. Okay. So one of your differentiators that I understand it is like you're getting into green energy and your whole value chain is going to be based on renewable energy. So let's have that next slide. Slide four. And that looks like a pretty impressive system, space age. So Paul's farm, Howard, who you've had on a previous show, he's set up a very substantial solar array on top of his 100 foot long barn. And he's got those blue eye on battery racks storing all the energy that he's manufacturing with panels. And that's what's powering the entire farm, including the container that I'm in, which is the coffee lab you'll see in a few minutes. And so there's no utility poles on this property on this farm. Everything is coming from the sun. And he's got some intention of installing a hydrogen electrolyzer production facility and fueling station in the near future. But in the meantime, what we're really interested in, we're interested in trying to understand how coffee grows and how we can grow the highest quality coffee. And so in order to do that, we're looking at different methods of farming, regenerative agriculture, and ways that we can produce with a carbon neutral result, or even a carbon negative result. So that goes back to all the energy that we use, and it goes to the style of farming that we practice. And even with the hydrogen, we'll be using that eventually to roast coffee instead of using natural gas or propane. So going that way in the route you're going, that helps keep any kind of contaminants and things that could affect the taste of the coffee out of the system. Do you like to comment on that, Dave? Yeah, well, coffee is a complex drink, more complex than I ever realized. And they say in wine, there's maybe a thousand different flavor descriptors. In coffee, apparently, there's as many as 10,000. And so we're researching right now the health of the soil and the health of the plants and how the plant's proximity to other plants and other trees affects the flavor of the coffee. And so there's a few good films available to watch on the intelligence of trees and how they interact, and on the mycorrhizae in the earth, and how there's... It's interesting, we have the World Wide Web, they call it the Wood Wide Web. It's a layer of mycorrhizae that is just millimeters underneath the surface of the earth. And it carries the communication between all trees and plants. And so we're researching that. And we're doing experiments right now. We have the ability to harvest from a particular tree and process it all the way to roasted coffee and compare that to the flavor of the next tree or the next tree or the next tree. And so we're refining our research and we're developing the best flavor profile that we feel is desirable. Let's have the next slide up. I want to show Dave's toy for the boy. There it is, guys. An electric tractor and no exhaust fumes. Talk about your new tractor, Dave. I know it hasn't arrived yet, but... Well, we're looking forward to this. There's one on Oahu already, and we'll have the first one on Hawaii Island. And yeah, it's it again, you know, we'll be able to charge it from power created by the Sun.  And what I like about it is that I won't have to wear these guys while driving it. And I won't have to breathe in diesel fumes. And it's just a nice clean... And in any kind of operation you want to use with a tractor, we don't till the land. We're a no-till operation because it's an orchard. But there are different things that we have to do. And occasionally we have to spray. And we spray USDA organic chemicals like insecticidal soap or something, or neem oil in particular. That's one of our favorites. And so the machinery that applies that onto the trees typically is operated by a gas powered motor. In this case, it'll be PTO driven right off an electric tractor. So again, carbon neutral. And because we're not tilling the land, we're creating a carbon sink with the land and the trees. So eventually we will be carbon negative. And you're not drowning the trees in diesel exhaust either or oil or all the other things that are stewed up by a fossil fuel engine. Exactly. And we don't have that cost. We don't have that cost to buy fuel. Right, exactly. So let's go on to the next slide because nobody knew it actually snowed in Waimea. When the flowers blossom in Kona, they call it Kona snow. And so now that we have a coffee farm in Waimea, we call it Waimea snow. There you go. Great. Next slide. So standing guard over all of this is... We have some spectacular views. We're debatable whether we're on the slopes of Kohala Mountains or Mount Akaya because we're right sort of in between. But we have this spectacular view of the top of Mount Akaya when it snow caps. And I think that shot shows the flowers blossoming and the snow on top of Mount Akaya. So looking to the future, next slide, you have a pretty aggressive seedling program for new trees. So tell us a little bit about some of the challenges and what your plans are there, Dave. Currently we have around 3,000 seedlings that are getting ready to plant. And interestingly, there's two issues that coffee farmers in the state are up against right now.  And the other is CLR, coffee leaf rust, which I believe is a fungus. And so these are prevalent on all islands and on a lot of farms. They're pretty devastating. Because we're so isolated here, we knock on wood, don't have any of those issues right now. But we do have more sort of traditional or more familiar insects like mealybutt or greenscale. And so typically we'll use neem oil, and which is a really effective organic product to use to treat them. So we're developing these seedlings using neem cake and neem oil treatments to get them as healthy as possible before we plant them. Okay, so let's have a look at Dave's fantastic lab. Next slide, there you go. Dave, this is beautiful. So tell us about your lab. Well, we have a wet mill outside where we process, and inside we're setting up a dry mill. We have some machinery. There's probably a shot there about a green bean sorter. It's a computer driven artificial intelligence green bean sorter. And we have some ability to dry and isolate the trees, as I said earlier. But once all of that's done, we want to drink the coffee. So we need a lab where we can haul a small amount of the parchment and roast a small amount and cup it. And so we do that in this lab, and we have a little cupping line outside. You'll see. Okay, hang on. Tell us about cupping. This sounds like some millennial kind of a thing. No, it's the process of identifying and, you know, desirable flavor descriptors and scoring. And so, you know, we're in the fourth wave of coffee. The first wave began 100 years ago and was essentially World War II, where they just mass produced coffee and they shipped it out to the troops to keep them away. And then the wave two was in the 60s when you started to see coffee shops and a few coffee shops started to become their own roasters. And then wave three and now wave four is this specialty coffee level. So just like in the wine industry, you can buy a $10 bottle of wine or a $10,000 bottle of wine. In the coffee industry, you can buy a cup of coffee for a few dollars or you can buy a cup of coffee for $100. And so there's a specialty coffee association and they've established a scoring system and they've established courses to train people to cup coffee where they're called sensory experts and cue graders. And they look at the coffee and they taste it and they smell it and it's a whole process. There's a lot of loud noises and a lot of slurping and sipping and spitting and they score the coffee. But you have farms like the Lamastice farm down in Boquette Panama who's selling his green bean for $1,200 a pound. Well, that opens up a whole new... Oh, sorry. Go ahead.  I was leaving. Yeah, okay. So that opens up the the idea of high value agricultural products. So why don't you tell us about your thoughts on that, Dave? Right. So farming in general is endangered. A lot of the farmers are retiring across the country and in Hawaii and their sons and daughters are not up to follow them because it's hard work and unless the crop is an extremely valuable one, there's not a lot of return. And often farms are sold off as subdivisions and unfortunate because it's valuable land because it's high in nutrition to grow. So we're sort of rethinking the whole industry and something called high value agricultural product or HVAP is the acronym is what is attractive to some folks and certainly myself because we've been scored as a specialty coffee and we're selling our coffee for a high price and we're working hard to push the score higher and higher and higher to get a greater price over the seasons. So there's other high value agricultural products. You know, one that comes to mind is wasabi for instance. That's an HVAP. So there's a number of them and so that's it's the wave of the future and how do you do this profitably. One of the ways is by pursuing a sort of a carbon neutral way of farming using solar, hydrogen, etc. You know, to produce your crop. You're producing the purity of your product has a great influence on how it tastes and how it's valued by people that are buying it. So next slide. Let's have another view of your lab looking. I love your shiny floors, Dave. I can eat off that floor. It's just epoxy. Okay, so next slide. So the lab huller, you know, this is a mini version. A large machine would be able to do hundreds and hundreds of pounds of parchment taking the husk off the parchment per hour. But this is just a very small hulling machine that's designed for lab and it does, you know, like half a pound and takes a couple of minutes. And that's what I was talking about earlier. The purpose of the lab is to, you know, prepare coffee from samples and see, you know, taste how this tree compares to that tree or this grove compares to that grove or this month compares to that month's harvest. So we're doing all kinds of tests. Let's flash up the lab roaster because that's a companion piece. Yeah, again, just a mini, mini roaster. Roasters are designed for specific quantities. So big roasters are usually five to 30 pounds. But if we're all we're roasting is a quarter pound or a half pound, we need a small machine like this. Okay. So let's have a look at what a cherry looks like. And what is a cherry anyway, Dave?  I'm not sure which way it goes, but anyhow, they produce this very beautiful and fragrant white flower that lasts only around 24 hours. And the flower falls off and then gradually over time, you start to see a green berry that starts to form. And as it grows and matures, it starts to turn color. It goes through various stages of yellow and orange, then finally red. And when it is ripe to pick, the color will be quite a deep red, very much like a cherry. And when you see a tree covered in ripe coffee cherries, hence the name cherries, and then you pick them when they're that beautiful deep burgundy red. Sounds good. So let's look at the various stages of coffee being processing. In this shot, you're probably looking at some cherry pulp in a tray. And then I think there's a bag of parchment. And so underneath the red pulp, there's the first mucilage which you ferment off, and then it remains parchment. And this is sort of a hard shell that covers the green bean. So that has to be hulled. And after it's hulled, you get the green bean and then it's roasted. But to get to that stage, it's quite a long time. It takes at least three weeks to dry the parchment. And then you want to store the parchment for at least three months to let it rest before you hull it. And then once you hull it, you have a shorter time to roast it. And once it's roasted, it should be drunk within three weeks. Wow, it's quite the process. Let's have a look at the next slide. So at the end of the day, after all that time, it ends up in a bag. Yeah, that's a sample bag. We have a very simple looking product. Right. And roll the drums for the next slide. Well, we were fortunate to have an opportunity at the last moment. We realized we had an opportunity to enter the statewide cupping competition. There's two major contests in Hawaii every year. And one is the Kona coffee farmers. And we're not in Kona. We're in Waimea. So we can't enter that one. But we were able to enter the statewide competition. And we managed to swing first place in the Hawaii district. So very grateful for that. And next year, we intend to do even better. Awesome. Okay, I think you want to reach out to a few people that helped you along this path? Well, I wouldn't have been able to do any of this without Howard and Pat Hall. They've been phenomenal in extremely generous and kind and helpful right from the beginning. And so we're growing the coffee on their farm. And they've installed this really fantastic solar and battery powered operation. And Andrea Calabada from CETAR has just been so generous with her time. University of Hawaii. I got to get a plug in from the University of Hawaii. University of Hawaii College of Tropical Agriculture Research and Human Resources. And Andrea is the coffee expert for this island and probably for the state. And she's just wonderful. And we've also been helped tremendously by Pacific Coffee Research, Brittany Horn, and Brian Webb, who's our roaster. Brittany's been our roaster as well. And Brian's been our roaster. And he has the Optimist Roastery in Cafe. And I forget who else is on the list because it's not in front of me. But I think the most important person is my wife, Ann Lee, who's my partner and Ann's a scientist. And so she spearheads all the experimentation in growing the seedlings and helping me diagnose issues with the trees and understand what kind of treatments we need to apply and how to improve everything. So we're fortunate because the coffee tree that we originally started with was a hybrid developed by Professor University of Hawaii Helo Campus. And it's a hybrid of Jamaican Blue Mountain and Guatemalan, which is what Kona Coffee is. So everything we're growing are cakey from that original mother tree who's named after my maternal grandmother, Irene. That's great. So you're going to start calling each of your trees by a specific name? I think that might be a good idea. It's better than row three tree number 45. So I think I'm going to have to come up with names for everyone. Yeah, exactly. So let's have a look at the, I think we're on our last slide now. Okay, so your website I understand is under construction still. Yeah, I mean, it's still brewing, right Dave? Still brewing. Yeah, it's steeping right now. It's, it'll be up and running within a few weeks. I just, I have to go over some of the, some of the wording, some of the writing and stuff, but it's, it's essentially designed, but you know, a little bit of tweaking before the big launch. Okay, we're almost at the end of the show Dave. So I want to give you a chance not to be interrupted by me. Do you have any closing thoughts? And maybe for other people that want to get in the business or what has been some of your major challenges? And do you have any idea what kind of solutions are required in the agricultural sector to help people like yourself be more and be successful? Well, I wouldn't have been able to have done any of this without the help of the University of Hawaii, CTAHR. I mean, I can't, I can't state that strongly enough that without the help of all of the researchers and scientists at CTAHR and all the knowledge they share with us freely. Yeah, I was just going to say, I mean, they don't charge you for that. That's like a service. Yeah, the only cost to me is my, my gas to get down to South Kona from Waimea, but you know, it's all free of charge. And they even come and visit the farm. If you ask for them to come and visit the farm and they'll spend several hours in the farm, you know, looking at every individual tree, they'll help you with analyzing the soil. You can bring soil samples to the extension offices. And within a week or so, you get a complete nutritional report of what's in the soil and what you need and what you have too much of specific to the crop you're growing. So what's an extension office? Well, there's, on this island, there's five extension of CTAHR. So they're, they're experimental farms and extension offices. And so at the office, they're, they're manned by all of these researchers and scientists. And in the, in the instance of, of coffee, that, that office is located in South Kona. And so that, because I'm growing coffee, that's where I go. But the Waimea district has an extension office and they have an experimental farm growing tea, for instance. Volcano has an extension office and they grow berries and some root vegetables. So it's all appropriate to the region. And so you just, you can show up there and, you know, ask questions and they're extremely helpful. They, they, you know, they have a display of all the services and courses that they offer. And you can sign up and, and take courses and seminars. And now of course, because of the pandemic, we've been doing a lot of that through Zoom or video. Like we are now. Yeah. So Dave, we're out of time. Okay. We're going to leave it there. So thank you so much for coming on the show and giving us all your insight on, on your Hawaiian coffee and all your adventures and your successes. And we wish you success going forward, of course. So thank you so much, Dave. Thank you very much, Commander. Okay. I like that part. So aloha everyone. This has been Mitchell in on Hawaii, the state of clean energy. I'll be back in two weeks with another action pack show. So everybody aloha.


Hi everyone and welcome to our real conversation in English. I'm Liz Wade and this is Adam Navas. Hello. Hi, Adam. And today we are going to have a conversation about our program called Music in War. And this is kind of a complicated program, but if you have not had a chance to listen to it, there are a couple different ways you can do that. You can go to our website at www.spotlightenglish.com. You can listen to it on our podcast where you can find that almost anywhere that we have or that you can find podcasts. And also you can go to YouTube at Spotlight English One as our channel. And so you can check that out there, follow that along. And of course, if you are already on YouTube, you might notice right underneath this video there's a Join button. And we encourage you to press that Join button. That doesn't make you pay right away, but it does give you some information about becoming a member of this channel and all of the resources that you are able to get as a member, including free PDF scripts, a few of those a month, additional video content, your name in every video, and some other really cool things as well. So we really encourage you to click that Join button. And also if you're on YouTube, please just take a little second right now to hit that Like button. And that really helps us spread Spotlight English on YouTube. So now that that housekeeping is done, so, right, Adam, that's what that's the housekeeping thing that we're doing. The little bits that we do every time. Yes. Now that that housekeeping is done and you know where to find that program, take a listen and then join us in this conversation. So Adam. Yes. In this program, I always feel like when I hear a program like this, it is so sad. And sometimes they're really difficult to listen to because you're really talking about war. And one of the things that really hit me in this program is that it wasn't a one time thing where music in war was an important thing. This program goes all the way from World War II to the 1990s to the late or mid-2010s. And I imagine that it could continue on into many generations. And that is, yeah, that is really something that I noticed in this program. Yeah. I noticed a similar thing. So I had two things. I have never lived through a war and I'm very appreciative of that. That affected our area. And for any of our listeners who have our hearts, of course, go out to you. And if you're listening to this and you're actively in a war zone or a conflict zone of some kind, we hope and pray that you are safe. But the second thing that I thought about this program, which really walks through just one example of music in war and music as a form of protest, it reminded me of all the other examples where music or poetry or other kinds of creative painting murals or things are an act of protest or an act against war. And I just think sometimes you think of art as fun or entertainment. And I don't think I spend enough time thinking about art or music specifically as a way to say something important to protest. And of course, there's a long tradition of that. Right. So I was going to add sometimes people think art is not important, right? So not only is it just for fun or beauty, but it's just it's not important. And clearly here, it has been important. I think for me at the beginning of this program, when it talks about this composer who walks through the ruins in Dresden, Germany, and is inspired to make the song, what is the song called? I'm forgetting the name of it, Albinonis Adagio. Oh, OK. So this this composer is inspired by this, these ruins to make this song. And then then the program goes into the 1990s, into the former Yugoslavia, into that war in Sarajevo. And there's a siege and Vydran Smilovic, that is a very easy to remember name, he is in that siege. We should pause and talk about is siege. Well, first of all, it's a very hard word to say, but do people or siege, I think both are good. I think both are OK. So a siege is like a war that isn't active, but is active. So if you surround a city and stop all people going in or going out, you've kind of created a siege environment. So it's it's war, but it's not like there's there's some fighting, but it's it's defined by that kind of locked in can't nothing's moving. No people often don't have food. They often don't have medical supplies. It's very hard to live in those kind of conditions. Right. So the story concentrates on this bomb attack in Sarajevo, where people are waiting in line at a bakery and this bomb is set off in that line and 22 people are killed. And the Vdron Smilovic Smilovic is in that city and he decides to as a form of protest, but also as remembrance. He plays his cello, which is a very big stringed instrument.  He plays his cello for 22 days, and that is remembering each day is for a person who was killed in that bombing. And I, yeah, I think he only played that Adagio, right? Yeah, I'm not. I think that's right. I think that was what he was saying. Yeah. So, yeah, I thought that was that was, you know, when you when you hear something over and over again, it can maybe be frustrating, like, you know, like, but this is a way to use that frustration, I think, to like remember, like, you know, if you're getting if you walk by the same place and he's playing his cello and it is the same thing every day, it reminds you of that of that constant loss, right? Right. Because of course, it's very easy to forget. It's very easy, especially when something so painful happens, you want to kind of move on and say you're all right and not not be in that place of pain. But it sometimes it's important to remember sometimes it's important to because we can't we can't move on, we even if we convince ourselves that we can we have to just be in that painful place. I don't like to be there and I don't think you do either. But it's it's necessary to give honor to the people who've died, but also to call people to account, you know, to say like, this isn't right, even if there's no one person who you can find to say, Oh, what you did was wrong. If it's just it's just to, you know, I think there's a line in it in the program that says it presented something beautiful in the middle of death and destruction. It was a song of hope. And I really like that. I like I like thinking about it didn't change the situation, but it did offer hope. And I think sometimes that does change situations if you have hope. Yeah. Yeah, that is that is an interesting thought. Like it's a symbol that that not everything is lost, right? That there's still beauty. There's still hope, which can feel it can feel hollow when you're in that situation. So you might feel like there's no hope. And I think even if you, I mean, obviously this is a if you have been in war, that's a different feeling. But I think even if you have had a crisis in your life or something very difficult that you've gone through, it can be hard to remember that there still is beauty in life. So yeah, that is a that's a good thing to remember. Well and we can all make art and we can all express ourselves. I think that there are times in all our lives where we find ourselves not knowing what to do or not able to make things happen at the speed or we don't have total control, right? But we can sing, we can dance, we can create something with our hands and our minds. And I think that the act of creation is sometimes a very human thing, a very we may have been told we're not creative or we're not artistic. But just the simple act of even sketching something with a pen and paper can help express and help get something, get that pain out of you and offer you that kind of hope. I don't know if I would say if you're watching this and you have found a way, if you have a way of expressing yourself, if you have either had to process something hard or maybe you consider yourself an artist, tell us about it in the comments. I would love to hear about that. I know that I like to, I often keep a journal. I'm not a musical person, but I like to do that. I don't know, Liz, do you do anything that you would say that gives you hope or that you'd think of as creative or centering or something like that? I wouldn't have to think about that. I wouldn't say that I'm a very artistic person, although I do really like creating art. I like making things with my hands and I do think that that is, it's healing for me in a way that just thinking about things isn't. But I do always think that I should journal more because that actually is very helpful. We have a program about that I think about journaling or handwriting is actually a good way to get your feelings out and work through those. I think that's a really beautiful thought that even if you are in a period of suffering, and I think many of us, some of us in war who may be watching this, some of us who are just in pain, but everyone who is watching this has been affected by COVID and that has changed everyone's lives. I think it's a really beautiful thought to say if that is affecting you, make some art. Put some beauty into the world. Even if it's not the most beautiful beauty, even if it's not the most beautiful poetry or words, put some out and see how it makes you feel. I would really love to hear about that in the comments if you have done that. Even if it's just with your phone, that's okay. But I mean, take a picture and share it or draw something or just figure out a way that you can add a little hope to the world. Yeah, I love it. And I think that's a really good and positive note for us to end on. I think so. Yeah, so leave us a comment below. Please press like on this video if you liked it. Check out the program Music in War if you haven't already. Check us out on our website so that you can follow along with programs and listen to them at the same time. That's www.spotlightenglish.com. Make sure to check us out on Twitter, Instagram, and YouTube. And thanks for joining us. Until next time, listen, watch, practice, learn. Spotlight out.


Let me first thank the panel. It's the first panel that started this summit, but also the last panel. It always leaves us so much things to reflect upon, and I want to thank them. And it was a great discussion. In fact, it sort of made me alter a bit of my comments, but I hope you'll bear with me, because it was really very rich, and I think gives us a good direction going forward. But in a moment, I'll write his Excellency Abdullah Bintuk, Secretary General of the UAE Cabinet, to share his reflections on the 2019 annual meeting of Global Future Councils. But let me first, on behalf of the World Economic Forum, thank his Excellency, Muhammad al-Gurgawi, Minister of Cabinet Affairs, and the future of the UAE, and in my head, his great team, and the government of the United Arab Emirates for the commitment, partnership, and hospitality extended to the Global Future Councils. So thank you. Let me see, they've already started, but I'll speak to this in a moment, but let me first pick up some on the discussion that prior to this closing here. Suzanne, I always enjoy listening to you, and certainly I learned a lot in the panel, but you started with this notion about focusing on human intelligence. And if I'm not mistaken, I know in some schools of thought that what separates us as a species is that as humans, we actually can imagine a future, not only imagine that future, but we plan for it. I think there are quite a few evolutionary biologists who say that's what makes us different, that we can imagine a future and we can actually go about and plan for it. So I took that as a really key thing for us to reflect upon, but you may also recall the panel chaired at the very beginning by my colleague Neil Menon at the start of this meeting. His panel underscored the need for every council member to commit to thinking and doing if we are to have a collective impact. It's good to remind ourselves of that as we close this meeting. And another important reminder from them was to make every effort to engage the communities and constituents not present here with us in Dubai, nor that will be with us in Davos. And I think that's also a very important thing to reflect upon. So my remarks will focus on those two elements. What each of us can do in the month ahead and how we can engage the rest of the world in our efforts every day. The strategic context for this is this aspiration. You can look this way, but we as a world economic forum and it's not about our annual meeting, it's us as an institution, we want to be this platform to integrate all the efforts and enable the efforts of all the actors, NGOs, business, civil society in achieving the mission to improve the state of the world. And here in this context, I think in the past 48 hours, I think we've identified really six imperatives that flow from the brainstorming sessions of the 38 councils and over 700 experts assembled here today. These are imperatives from them. There'll be some initiatives, but I think more importantly, I hope there's some important new narratives that emerge. If I may, based on what I heard from the panel, I think there's one theory of change. If you want to change a system, I think there's one way, it's first if you can identify or reorient its purpose. All right, that's how I think we could deal with the complexity of these systems is we can make the system do something else. I hear in these corridors, people talking about, well, the healthcare system shouldn't be about illness, it should be about wellness. I hear when I read the newspapers that trade should not be about trade, but maybe it's about employment and other things, but it's changing. People are debating the purpose of things and I think we shouldn't lose sight of that and the importance that you have. I agree, there's no simple solution, but maybe there's a better understanding that the work of these councils can help and help us rethink the purpose of a lot of these systems. And there, what are some of these imperatives that that I know that it's going from, I would rather have, I prefer the slide to say from Dubai through Davos because it extends beyond Davos. It's going to inform the work of the forum all of our constituents for the year ahead and years ahead, but there are these imperatives and the reality is, is that, and I love this what the panel was sharing with us with test practices because there are no technical solutions to these. There are adaptive challenges and we're going to have to feel our way through this. I would love that there are best practices, but the reality is they're test practices and we will have to look at these, but the key is I think each of these six imperatives I think speak to the work of not just one but multiple councils here and we want to engage with you around that. And again, one factor of the future success of each council will be in developing initiatives linked to these imperatives, yes, but more importantly your impact will also depend on developing compelling narratives, right, that engage other forum constituents that in other communities and the public at large, you have I think with your expertise and insight, you have one more creative challenge and that is to build that narrative that brings us all together, it makes us feel part of this.  Narratives were part of it, we can actually make the ending together. And in this context I thought it was important to sort of think about the road ahead and again through doubles and perhaps reflect a bit on, or at least elaborate a bit on the two digital platforms that can help each of your councils to share knowledge with other experts and communities to engage citizens worldwide around your initiatives and your narratives around these imperatives. Boy that's a mouthful, but work with me here. So what is that road ahead? And from today through 2020, we have a number of actually citing convenings because we have to bring people together. And but in this day and age, I think we have to leverage the digital technologies that enable that in ways that we can be together in many more ways on many more issues without losing sight of what the goal is and the change and transform these systems. So we know that your council managers and has talked to you about some of the things you can do and deliver with us, working with us on webinars and some other items. But let me talk to you about how we hope to share that knowledge. That was my, remember I said I want to talk about two things, sharing that knowledge. You should know that since launching a public version of our strategic intelligence platform here, here in Dubai two years ago at this meeting, we have more than 220,000 public users of these platforms. There are everyday 6,000 active users and on average 500 new subscribers each day. This doesn't include the network of 6,000 experts that are part of the forum network here in terms of different domains over 140. But they're everyday literally hundreds of people are engaging, thousands of people are learning from you. So we look forward to your council's future webinars and transformation maps and analytics and white papers to help us engage more of the forum communities and partners and constituents. We, that's the positive thing about these technologies. We have the means to really educate and reach out to many more people. Now, the other piece I want to close on is really about engaging the public at large. Again, the people that are not here, that are vital to us, to making, to really changing the world. They are the world, right? We have an excellent public engagement team led by my great colleague, Jim Monk, that publishes everyday 365 days a year, 20 to 25 blog posts, four new social videos, 50 Facebook posts, 120 tweets, I can keep going on and on and on. They work pretty hard, small team, but highly effective. And today is the results of your collaboration with us and their efforts, we have 60 million followers of the forum across major social media platforms engaging the public in multiple languages. And that's also very important. Their daily engagement of that 60 million followership has really led to over 200 billion, I'm sorry, two billion video views a year. I was gonna say 200 million, but it's two billion video views a year. And 600 million people come to our websites each year to learn from you. And that keeps growing and we hope to grow that with you, that engagement, that public engagement with you. And what's remarkable is I just said, well, that's those great numbers. Can you tell me a little bit more? And I learned that most of these are younger people, they're between the ages of 20 and 30 years old, 50 sort of gender distribution, more women seem to be active and follow us in social media, but that's also very positive because it's more representative of that, the real world. But I also asked Doug a little further, I said, well, what happened like yesterday? Well, I said, well, you really wanna know? And I, yeah, I really would like to know how I work here. And they said, well, we had over a thousand, a hundred thousand page views on blogs linked to your councils, just yesterday alone. And these are deep engagements.  And so the public is highly interested in the work of the Global Future Councils. So we look forward to your future contributions in this regard. Now, I'd like to, I want to close my remarks by also thanking the council managers and fellows for all their efforts today. So thank you. And also on behalf of the world I come from, I want to thank every council chair and member for your trust and commitment. You know, the panel talked about collaboration. Well, your continued collaboration in the days and months ahead will make the world a better place for future generations. And here I'm not speaking as a member of the Banji Board, I'm also speaking as a citizen, as a parent. You're really, it is always inspiring and encouraging to know that we have in this room and outside this room so many people who are thinking of a better future, but also planning and helping us plan for one. So thank you very much again. Now, Your Excellency, the floor is yours. Thank you. I'm coming to an end to the closing ceremony today. And I've realized, you guys are, after 48 hours of thinking, there's about 30,000 hours spent of having you experts, stakeholders and 41 councils, over 1,000 conversation in the past 48 hours. So thank you so much for thinking for the future of the world. The question is, Excellency Mohammed Al-Gurgawi raised yesterday on how do we address what's three and three ways. One, the inequality of the future, trust in the future and the wisdom of the future have been great anchors in the various council meetings. I've listened to your disagreeing, to your agreeing, to your sharing, challenging and questioning each other and the pursuit of the future and better future of humanity. It gives me hope, truly does. We're not just talking about shaping the future for this event. We should pause for a moment to understand the scale of what has been achieved. Global cooperation has underpinned growth, new ideas and moved 1 billion people out of poverty over the last 70 years. This was a highlight from Annabel Gonzalez, the non-residents and your fellow of Peterson Institute for International Economics. Yet, scale of challenges, problems and opportunities that lie ahead are even greater. The world's need seven trillion worth of infrastructure investment in the next 10 years to boost economic prosperity. This was according to Danny Qua, the Dean of Professor of Economics in the Lee Kuan Yew School of Public Policy. I agree. Our education system are still similar to 100 years ago. Our health system are similar to 50 years ago. And many of our mental models are many centuries old. The scale. The scale is a challenge to reinvent both new systems and mindset that serves the future. There's 133 million jobs will be created. These jobs have a new nature, the nature of AI, advanced technologies and we need our education system to adapt faster. We need to move at the speed and the scale. Otherwise, we risk the widening of the inequality of the future, as His Excellency Mohamed Al-Gigawi talked about yesterday. Before we can scale new solution to address the scale of the problems, we must first scale our collaboration with the stakeholder in both quantity and quality. We need to redesign governance system to cope with challenges, future legislations and policies needed, co-develop, co-created with stakeholders, innovators, business sector, other governments, NGOs and the civil society. And now, let me shed some light on a few council outcomes. The Global Future Council on China will explore emerging models of Chinese leadership. It will develop case studies to illustrate how business leaders inside and outside China can bring an entrepreneurial approach to solve global challenges. The Global Future Councils on Cities and Urbanization. They took a moment to think about the future cities and our organizations and they will focus on working with cities to find and leverage scale levers. They will establish initiatives to advance city capacity and building programs and many more. The Global Future Council on Development Finance will explore innovative financing mechanism, adaptive scale. The Global Future Council on Geopolitics is planning to promote regional and cross-regional mechanisms for confidence building dispute resolution and threat management. The Global Future Council on Russia will act as an advisory board or a body by providing recommendation on the design and adaptation of artificial intelligence strategies at the national level. The council as well will work on several projects, including the country AR Redidence Index. AI is a major toolkit for best urban AI practices. The last one, which I was a member of last year, was a Global Future Council on Agile Governance, which I was, they will build on the last year work on providing guidance on the best agile governance tools for addressing different policy questions and create platform for communities, practitioners to increase the speed of learning between our nations. And with that focus on increasing the speed and scale of learning, I'm excited to announce the one of the outcomes from the government of the UAE side. And based on the last cabinet decree by His Highness Sheikh Mohammed Barash Al Maktoum, appointing a UAE ambassador for the Fourth Industrial Revolution, which is first on a global level. This role is designed to support our nation's role to grow from a global hub to a global bridge. The fourth IR ambassador is elevating the technology diplomacy to cross cutting priority in UAE foreign policy. Ambassador Role has a global mandate to spearhead new global partnerships on behalf of the UAE government and service of a better future. The role is an owner of another question, the fourth question. How might we scale the future? Not in 30 to 50 years, but in three to five years. And finally, a sincere and humble and grateful thank you to the World Economic Forum, especially Professor Klaus Schwab, Borje Brende, who was with us yesterday, Lee Howell, and the whole of World Economic Forum family, who with their energy, eagerness and support help us think and inspire and challenge the state to school for a better future of humanity. I look forward to us as a collective, taking conversation and outcomes to these council meetings to Davos. Thank you very much. With that, I'd like to formally announce the close of the 2019 Annual Media Global Future Councils and wish everyone safe travels, but also looking forward to collaborating through the rest of this year and through 2020. Thank you again.


I'm going to do for other architectures maybe other kernels, I don't know yet. We'll have to see what other people interested in doing that, and scaling up from the better and better of kernel testing and basic root of S testing on development boards, scaling up to see what else we can do and what else can be done with this software. So the slides are in the Git-anx for DevConf share, the URL is on there, ac mae'n gobi'r documenta i gael ei fod yn cael ei hunain y llwyddiad, felly mae'n rhaid i gael ei ddweud o gael o'r ffordd, o'r cyffredinol, o'r cwmysgu a'r teimlo i'r ddweud.  Yn ni ddau'r gwahoshio ar y cwmdeithasol, mae'r ddiannod CIGs mae'n gwahosio roedd cymryd Garchanaeth Llanaro Cymryd Llanaro, ddyn nhw'n ryml yw'r peth. 103 dyfais online at the moment. We have a staging instance, which has another 40 of. There are various developers who have their own setups to try to expand the range of the devices available. It's based currently around a lot of OS deployment testing. That's not just Debian based, a lot of the images are still Ubuntu based. We do a lot of open embedded work. We've recently started to get a lot more work from Yocto. We've asked and sought involvement from people doing Fedora testing, and so far we've only had a little bit of interest. We're not sure what the problem is with that, it's maybe just the Fedora approach to ARM or the particular way the images get built. But if there's any interest in other operating systems outside that, we are more than willing to take it on. The principal thing we've been doing so far is a lot of kernel boot testing. Has this particular kernel build improved over the last build? You start off with a simple boot test, but then you start to track performance data.  We've recently added multiple node, so multi node allows you to have a single test that runs across multiple devices. Those devices can then be synchronised purely over the serial connection that all these devices share. They can raise their own network, they can do whatever they need to do with their own switches and their own environments, and you can have multi OS deployment as well, so you could have an OE box talking to a Debian box. It's trivial. The main thing that we need to talk about with Lava with a lot of people and a lot of conferences is the idea that we don't write the tests, we don't define or prescribe what tests you can run. So people often ask, well, what tests can Lava run? Whatever you can think of.  We have our own tests that try and stress Lava itself, but those are largely irrelevant to the people who actually want the data on the systems because quite often we are testing with a static kernel build and the static root of S and they are submitting jobs that are a new build every time. We do track results over a long period of time. One of the main features is you can go back right through the database, track a particular series of builds and watch how it actually performs. So if I go back to here, that's a simple graph for my local box. It doesn't go back that far in time. There are other ones on the main servers that go back a lot further. So you're just tracking the pass and fail of a particular series of tests. You can see the various things were going on there. At one point or other, my own installation being a development box wasn't performing as well as it should do for productions of the results dipped. Those summary reports are created by users. We put the facility there for people to write those reports, but the reports are written by the people who want the data. So the data can be exported. That main report there is basically a simple overview and a front-end to your data, but you can export the data for further analysis, combine them in new ways and work out what actually happens with the rest of it. The actual deployments you can do, what Lava can do, can be expanded by particular hardware. We've got ideas for, we call it the NMP, it's still in final testing, but it allows you to put a relay between critical parts of the device and the device setup. So it allows us to switch the SD card externally as controlled by the test itself. So suddenly your SD card goes away. You can switch away the network. You can switch off the SATA. You can switch the SATA to a different connection. That's not just useful in terms of being aggressive with the test and seeing how the kernel behaves, but it's also useful to allow us to test bootloaders because you can put a bootloader on one SD card, switch it over so that the device can see it, boot the device from that SD card. Oh, it's bricked. Switch the SD card back to the previous image and boot it again and it's fine. A lot of the automation comes down to being able to automatically recover boards. There will be situations, these are dev boards, there will be situations where it's managed to fry when the controller is on the board. There will be various issues. We've got one particular board can lock the SD controller. In a completely invisible and transparent and undetectable way. We can try and work out in advance. Okay, there are things you can possibly do to indicate that this particular board of that type is suffering this kind of problem. You try and put some diagnostics into the test in advance to see whether you're actually going to like to hit that problem and compensate and work around the problems of particular boards. So what does lava currently support? There's the arm bias, not surprising for instance where we're actually starting from, and Lenaro is concentrated on Arm V7. So we don't look at Arm V5, Arm V6, or Arm V7 are the main pieces of hardware that that's the majority of the lab. Arm V8 is then coming in with new hardware and emulations and models. But the majority of the boards that are available for testing currently are Arm V7. We've got support for where the boards themselves are boarded running a virtual arm system on their physical arm hardware. And that allows you to actually test both sides. You can run tests on the physical hardware whilst it's running a VM. You can run tests inside the VM. And if you've got the right hooks you can actually then communicate across and through the VM. Or you can actually communicate with another node that's set up to interrogate it in another way. There are actually the six systems in the lab at the same time, not just for actual lava deployment but actually as test devices. There are situations where people need that so those devices can be available. Via emulation we can do any of the other architectures that QEMI supports. We haven't got a lot of any physical hardware in the Cambridge lab but with lava endebving you can easily install that and set it up with your own architectures. A very useful feature is the idea of dummy devices and these can be a simple S-troute so an isolated truth environment or SSH and that allows you to connect to a device that hasn't actually had to have a deployment. It's set in there pre-configured and it may be serving a lot of different jobs and you actually have jobs that connect into it and do certain things with it. Contention on that box is your problem as the testwriter. That's not up to lava to try and sort out. If you've got two jobs going in on SSH and both of them try to run deep package sorry, you have to sort that out yourself. A lot of the builds coming through are CI builds for kernel builds. We still do a lot of Android testing and then we are working towards increasing support for testing bootloaders. Obviously U-boot was one that we started with inevitably with on V7 dev boards. UEFI is the next big change and the next big bootloader we need to test and then Grubb is something else that we are looking to support. We don't have explicit support for Grubb in lava at the moment but it's on the development line. We are currently only testing Linux kernels. There may well be hidden assumptions in how the structure works for other kernels. We don't know yet. We will have to find out when the community tells us. Test jobs are written basically using shell and markup. You can address and execute any utility binary anything you can actually build or download or put onto that system. Most of them if you define the right parameters for the kernel you can have a nice networking interface and you can pull down whatever you need. It's up to you, you set it up in the test. We get results based on whether the deployment itself worked, then the test that you wrote and then there's parsers to work out whether there's parcel fails and you can do measurements as well. A lot of the time you're running someone else's test suite LTP or some kind of Python unit test type thing you often have to write your own parsers. Sometimes you can do that in the YAML with a bit of Python Regex code. Sometimes it's actually a lot easier to write a custom script in whatever language you prefer, whatever language you can actually ensure is executable on that platform and run the test inside that custom script. Then the custom script just outputs the test results in the format that is easier to understand. What Lava does is it gets hold of whatever the deployment is going to be with this image, the tables and we overlay a basic set of data and shell scripts. There's a bit more data and shell scripts if you're actually doing it with multiple nodes, but generally you've got a common interface there that's based on on Puzzic Shell compatibility but we don't define what you can run and we like to think that we are making no assumptions of what the device is capable of. Sometimes that lets us down because there are ways where you sometimes have to make some kind of assumption of what the device is able to do and sometimes we do push back at the device manufacturers and say now look, the way that you've actually asked us to test this board and the way it actually boots is not going to work. We need to have something that's a bit more sane and a bit more reliable because we are automated after all. You can't have a board that needs a lot of manual intervention to get the thing to boot in the first place. We're booting it three or four times per test a lot of the times or even more. We sometimes have to push back at the manufacturers and say now look give us a board that we can reasonably automate but other than that and as I said we work hard to make sure that there's an automated way of recovering from a bricked device or what would otherwise be a bricked device without any kind of intervention from the lab admins. They're busy enough as it is. So now we're reaching out to Debian. We've been using Ubuntu to actually run the infrastructure for a while. We've now got it all migrated over to Debian. All the packages are in Debian. We're running the packages as they are in Debian. We are looking to test the RMP kernel. I've been working with a number of people in Debian already on exactly how to do that. So we're looking at where's the overlap between the boot proposal we can access elsewhere the available boards in Lava and the DDPs that are defined in the RMP kernel. I've got some links there for the various information. That's the summary of the boards we've found so far that can easily be tested for the RMP kernel. I've put in a request for extended support for the arm deal because that should add another couple of devices to that list. What we're trying to achieve with the RMP is make sure that not only does the RMP kernel boot but that hardware that is available on the board is actually operational under the RMP kernel. We know that for some of those boards on that list various components, various pieces of hardware are not operational after you've booted the RMP kernel. So this gives us a way of producing repeated tests of a RMP at a time to try and track this and see whether we can improve it without causing regressions elsewhere. Those are just the main devices that we've got at the moment. We're obviously open to having more devices added to the lab or to a different lab and having access. Some of those devices on there are not physically located in Cambridge but we would still be able to use those within Devian. So the particular challenges with RMP we need to be able to put the modules into the init-RAMFS. We can test with the init-RD that the init-RAMFS tools actually make but that is then using K-Lipsy. So we'd like to be able to use other init-RAMFSs with the RMP modules put in and we've got a G-Lipsy environment and you can do more testing. We need to decide on which DTPs we can actually support in the RMP kernel and put the data out there and let people work out what boards they can reliably use. At the moment it's a bit hit or miss as to whether the RMP kernel supports your board. Let's get the data, let's get the test results and find out what's going on. We can submit these jobs over XML-RPC to a variety of existing instances and to a variety of other tests on each one including testing the Debian installer on ARM. There are ways of preceding that and making it work without interaction so we can test that through. That will generally mean that we need to start testing with dual media so we can actually have a booted media and a deployment media so we're looking at some of the boards that support Sata for that. This comes back to what we were talking about in the first talk about what Lava can do in Debian in conjunction with PyU Parts and CIDebin.net and the archive rebuilds. Lava is an opportunity for Debian to start testing outside the idea of just the one package. It allows you to start testing combinations of packages. It allows you to start testing a distribution against a different distribution whether that's a different suite or a completely different distribution. We can test Debian against an open embedded. We can do a whole range of upgrade tests. We can actually work out whether this actually works in a multiple node environment. You have a client on the server with physically separate devices. We didn't expect that necessarily to work right up to where the client does it recover. You can do testing across multiple architectures and as I said earlier across or through the virtualisation barriers depending on what kind of support you compile into the test. The images themselves or the deployments that we support we can always make better tools for those. We can make use of existing tools and we can also increase their availability. Then these the upgrades so that we can actually work out can you go from squeeze all the way up to SID with this particular set of packages?  Can you throw some dirty images with random configuration changes and does it still work? Because you can define all that. You haven't got to just say I just want a basic to bootstrap in each case. You can build the image you want put in whatever dirty configuration changes you want and throw the images into Lava and see what you get. Very useful tip when you're actually developing tests is hacking sessions. Basically you install open SID server. Lava has support for accepting your public key as a parameter and your IRC nickname. It'll boot the device you've chosen with the image you've specified start of open SSH and then notify you on IRC saying that the session is available at this URL in a private message and you just put that into a terminal connect and you're in. So you're then in on the test image that you've chosen on a device that you've selected of a particular type and you're in a full Lava session you've got all of the Lava helpers and scripts and overlays there in front of you in your path and you can see what's going on. But that's only what we've thought of. That's what we've thought of within the Lava team as to what we can offer Debian via Lava structures. Come back to us when you've played with Lava and see what you can do install it one of your own boxes see what it can do see what it can connect up and come back to us with what LC can actually do. Now, after we've done the work on the packaging and Tony will know all about this because we've done all the planning for the refactoring. So this is intended to make it much, much easier to extend and develop inside Lava. So if you're looking at what Lava doesn't support this kind of device, how can I get it to support this particular moment this particular feature? We're not going to make it a lot easier in the code base to actually work your way through that because Lava has developed organically and now the refactoring is urgent. So we're looking at model of components much more identifiable sections of where things go making sure that actions are important and diagnostics making it much clearer and much more obvious and much more common that when Lava spots an error not only is Lava able to say yes that was sorry that was a Lava error philobargon will work on it that was an infrastructure error because a network switch has probably failed something or someone's switched it off or that was a job error you've made a typo in one of the scripts or it was actually genuinely a test failure so working through those kinds of situations and then when we've got Lava errors or device errors something going unexpectedly wrong what can we do whilst we've still got the connection to the device in that particular mode can we actually get reports of data, get it into the logs get it into the actual test results and say well there's what we've tried to work out even if it's just what I have config saying what's root dash n saying what's n map saying just get the data out there so that people have a chance of working out what the state was at the time we don't do enough of that yet the refactoring will also concentrate on allowing you to simulate the job in advance so you'll be able to run through your test definition and see all of the actions all of the primitives, all of the actions then the commands of that thing would actually do on the actual device you can follow it through step by step then we get more of the data coming back at the moment we have the logs and then we have the result set, the result bundle we don't actually have enough data to come back into the results so that's what we're going to fix with that and allowing people to override the one of the common problems is that as lava developed there were a lot of devices that had quite slow reactions to some of the operations so some of the timeouts got longer and longer and longer some of the some of the snow walls some of the v-expresses they took such a long turn to get through all the different firmwares different stages in boot that the timeout for a failed boot got too long and it became a default rather than actually being customised to particular boards so we're going to fix that and make sure that for devices now that boot much more quickly if it fails the boot you're not sitting there for five minutes after it's failed the boot to get the message for lava at the timeout and think actually that didn't boot right, sitting at the back there quietly working on the video is Andy who's working on the idea of putting lava into hardware so it's an open hardware design that will be based on Debbie and Jesse and the idea with this is to allow you to have lava in your bag at a demo at a conference already set up support up to six devices or one little box and go five five volt input and you'll be able to just connect up the devices you'll have a PDU inside the box that'll be controllable remotely you'll have a network switch inside the box you'll have serial connections and the serial server again remotely accessible, remotely controllable and you can turn the serial off completely during a hard reset there are devices there's a QB2 where there's a design floor in the hardware and if you leave the serial connected when the device is hard reset it locks the boot loader in a bad state and it can't actually get back into the test image so that was one of the boards that led us to this situation where we think we have to actually have an easier way of getting our developers set up with a lab on the desk and then working through and actually that can be really useful for conferences and every developer working on lava or just one of the test stuff you can actually put these up as a full rack as a full lab or you can have them as a dedicated unit with a little board inside a nice little starter drive and you've got a whole lab in one unit now let's just show you I'll come back over here so currently this is what lava actually looks like so you can see it's using a local file this test took actually took about 47 seconds so it uses a local image that was built with VMD bootstrap which is a nice little tool that Lars originally brought and which Antonio and I have improved after that you get a nice checksum to make sure that we've downloaded is what you think we've downloaded there's the overlay we come through and then pass that, do the overlay pass it down to Qimium in this case because this is a KVM test normal boot output also output all tracked there's your network address and network information it's a kernel boot time 5 seconds that's automatically tracked and other stuff like that will be tracked across the board you can see here lava, KVM or one so that's where the overlay stuff actually lives nice little check to make sure that there's actually some available space inside the KVM before he's started doing stuff and then the test runner doing stuff like dump the IF config in the output there it is that passed and then get some routing information do a test ping make sure that the actual ethernet is sane and working see if you can install the package and it passed it came down to the end and you've got the result bundle the result bundle is just a way of collecting all the different results you get from the lava tests and your own tests so you can see here the lava test results we deployed we arrived at the test shell the results and the the job came back as complete and then the user tests work out something like that so the ping test actually failed at that point that's mainly because my laptop is not bridged because just because so when the KVM came up it didn't have a it came up behind a knotted address so this is on staging which is our test instance a lot more devices a lot more jobs running so if you look at well just looking through that are there any let's do one of these foundation models let's see what that looks like when we go through so again we work out through the foundation model you're probably better off explaining this than I am but it's it's modelling an arm v8 right in saying it's not actually an emulation is it it's it models the entire CPU this one is running on x86 and modelling an arm v8 so that one so you get various aberrations then when you're expecting various facilities to work it's expecting real hardware and they're not there so you can record that and you can work on what's going on and we're back into a normal test so the point of showing that is that despite the disparity in the actual hardware there you get a very common very similar interface if you go back to a real piece of hardware let's go for a panda that's what the that's what the submission actually looks like we're currently using JSON we are looking at using YAML for future deployments once the refactoring is in place but you can see there that's where the image is obtained from you can see where parameters are inserted into the the YAML file that is actually in the test definition in the repository and there's where you submit results and you're saying that's a panda device type and it's given a job name and a default timeout which each action will use as their timeout unless it has a specific timeout just like the lava test shell so if we look at the complete log this time we have to try and connect to a real device it's not a KVM or an emulation on a model we've got a real bit harder so this is where the serial camera comes in so this is the stick of these for C04 this is a telnet interface our nice familiar U-boot drives us all nuts the panda is using a partitioned SD card so we boot into what lava calls currently the master image this is one of the ways that we ensure that the test image doesn't make our panda unusable for the next test in the line so this one isn't testing bootloaders as such at the moment because it's relying on the same bootloader but you're just allowing a test image to be deployed onto the third and fifth partitions on that SD card so we check that the master image coming up is sane in case something has managed to break it in the previous test and then we get hold of the test image so the panda is doing a lot of this work we then send the modified tar balls with our overlay back to the panda write it out onto the SD card sort out the boot partition check that we've still got a sane system and then we go down for a reboot so this is now expecting to come back up into the test image at this point we are able to control uboot lava uses pxpect a lot so we are interrogating the serial output at each point and lava is stopping uboot and say what? I'm going to set this command now I'm going to set that command and that's how when you reboot into the test image lava is able to make sure that you go into partition 3 because it's telling it you load the test u image from partition 3 not partition 1 where it would have come from to get into the master image so there's uboot of the test image and the other partitions with this own middle interface and now you're in the narrow test so you're inside a test image similar sort of stuff you work through and you've got the same interface again with your test runner inside the image you've defined you're passing in information about which test are passing and failing and that's quite a long test it's one of our functional test but it's a bit longer than some of the others so you can see this test image was a ubuntu raring image so this is one of the ones actually that will cause trouble if I choose to resubmit it because ubuntu have now taken raring off their mirrors and you can't see raring so these kinds of things are things that you need to think about as a test writer what am I basing this on and then in the years to come we've come back actually I want to rerun that test back from there if you're relying on third party sources for the updates and the other bits you're bringing in if you want to run these things long far ahead in the future put the stuff in your own repository take a snapshot of it, keep it so you can run that test in the future this test suffers from that problem that it's an old test that I wouldn't be able to get hold of that data there we go, failed to fetch doesn't exist anymore 404 because there's not a problem it's just one of the tests didn't run so it goes on tries to work out what other tests it can run and reboots into the master image once in the master image yep again checks that it's half sane gathers up the result data from the test image because the test image is on the same SD card so the master image can easily just read the data around and then pass it back to the server you can see there direct update failed, direct install failed curlftp failed simply because the information wasn't there that's the indication of how you use the measurements and units just a little demonstration of that in this particular test and the result bundle is there because we had multiple test definitions inside one test you've got multiple sets of results you can do it so that you have an automatic reboot between each of these sets of tests or you can combine them and have all your tests running in sequence in the sequence you've defined in your own test they're not resorted or that you define everything about how the test actually runs quickly look up this is what a test definition actually looks like currently we're working on this part isn't likely to change with the refactoring the submission format may well change but the structure here is not likely to change much you're defining a little bit of metadata maybe it's not necessarily deterministic we don't rely on the fact that you've marked the OS you want to it's just a clue for the result pausing same with the devices that's that device list is there mainly to help other test writers work out whether they thought of this test supporting their devices it doesn't actually preclude someone trying it on another device and you start with the basic set of run steps that's a very simple if you go up a little bit further this one has installed dependencies so you can start with someone's image and then think right now I actually need to add stuff if it is a Ubuntu or Debian based test image then it's simple to actually install them as packages I know that Fred's test image over there didn't have these things installed I'm going to add them in for this particular test and then you just go through on each one and do a bit more testing so you can see we're calling there a check IP SH script which is part of the git repository that this YAML file lives in because the test definition is cloned directly into your test image and you can run anything that is in your git repo that's where you do all your scripting and do all your definition of things that don't quite fit into a single line of YAML if you start needing to use a lot of pipes and redirects and maybe a bit of bread jacks pass in that goes into a script of whatever language you want to write to in yourself whatever your preferred language is or whatever the test image can support you can compile it from C as long as the test image has a compiler of course some of them don't just before Daniel let me show you the we've written quite a bit of documentation it is all available every time we install Lava you get all this documentation on each of the instances and it is the documentation for that version that you've currently installed naturally a lot of that documentation will be written by the developers so it may well be something that you need to actually come back to us and help us improve Patch is welcome well I wrote a lot of this documentation but I definitely know what needs patching right so that's the link there for the main documentation on the main lab server the staging instance is there if you've got questions after the talk come to us on the mailing list or on the hashlunarolava of TCC channel and there's our gitrepos and you can come to us directly on bugslunarol.org with an account or you can just use the debinvts and you the install command to go and have your packages Daniel so actually is this working? so the first question I had I think you actually answered which was whether there was a way to program how to acquire the serial connection because you stopped saying it's a serial connection it's a serial connection and at one point service that comes under the device type configuration so lava will work out or will assume that pandas have a particular way of being connected and you pass that connection command on a per device basis so you say that panda6 is on securities for port 17 is that generic enough to for example do IPMI connections and that kind of thing? yes there's a whole range of different ones that people are using it is just a connection command so it's the entire command including the whether you use telnet or whether you use screen or minicom or whatever you actually need to use to actually get onto that board so my second question is you mentioned the hacking mode and that sounds like a really really useful thing to have does that work on systems that don't have networks? I'm not sure how you could actually get a concession serial connection the problem of the serial connection is that lava has a requirement that we have access so that we have control of the serial connection because we are running P-expects all the time so if the with the refactoring there will certainly be ways of having secondary connections onto boards which could allow if the board supports a second serial or something like that or some other way of doing it I'm not sure I'll really export the serial connection to a PTY and then let an incoming SSH session to the server talk to it it'd be really useful patches impressive ok third question the lava overlays are they static or are they per test run? there is data that is test run dependent so what do you do if you can't repack the image say for example that what you're trying to deploy to a system is a signed lump we'd have to look at that in terms of that is going to be one of the things we need to look at with refactoring currently we don't actually have a way of doing that if we receive an image we will currently break it up into bits and deploy it onto partitions but there will certainly be need for how we should one of the plans will be that we will deploy it as signed we will use whatever we can structure-wise to verify that it was signed then we'll just blap-lover it you're almost making me forget my last question no you have made me forget it does anyone else want this otherwise I'll remember it in about another 30 seconds oh yeah the tests that you showed that indicated that you could say oh I require OpenSSH server or I require NTP are they declarative enough that for systems which you can't install packages on like say an android system or an otherwise arbitrarily static system you can say ah I can run these tests because the system is exporting these tags in some manner no that would be down to the test writer so for example we get this problem with open embedded open embedded is a problem in that unlike any other distruder that's out there it does not identify itself there's no tag there's nothing in an open embedded image that says I am open embedded so you have to look for everybody else and get to the end and think right it must open embedded because there's nobody else we recognise or even an LSB release no not necessarily it only has LSB release if you compile LSB release into your open embedded image it's too much control so yeah we can't necessarily be accurate enough in working out exactly what system you got on there to work out yes you've got that available available or whatever and it's not really Lava's job so you gave us the test image you gave us the test definition catch if you break it you get to keep both bits and I want you to marry these test technicians to these images we will Lava will do that and marry them together but if it breaks it breaks and a final thought on the packaging thing if Lava were to provide a way to specify pseudo package names and I don't mean the Debian pseudo package then you might find that's going to make the fedora people slightly less unlikely to not come on board because at the moment you're saying this depends on these Debian package names effectively well the test definition might be different but if you can have more test definitions that are generic across distributions then you might get somewhere more but then it's easy for someone to come across with a Lava installer script that is based on yum it is extendable this isn't hard code somewhere in the depths of the python code of Lava it's there in a shell script that's copied onto the board so the Debian and Ubuntu images have a little shell script that is Lava installed packages which calls out to get Antoni, is there a question? So how these dependencies work is that you have both besides having drivers for each type of device you also have something similar for types of test images so you have Debian, Fedora so when your test definitions say it depends on full it will install full in the right way in Debian or Fedora and then for open embedded where you can actually install stuff then your test images needs to have that stuff already installed and then the dependency installation step will just say give a warning you depend on these packages but they don't know how to install I'm assuming your image already has them and then in the case where you have packages with different names across distributions you can specify package names for Debian package names for Fedora package names for full bar that you need so it is possible to keep generic test definitions that you can be used on every single image that's fine thank you