I'm Rusty Komori and this is Beyond the Lines on Think Tech Hawaii. I was the head coach of the Punahou Boys varsity tennis team for 22 years and we were fortunate to win 22 consecutive state championships. This show is based on my books Beyond the Lines and Beyond the Game and it's about leadership, character, and creating a superior culture of excellence. My special guest today was the all-star baseball pitcher for the University of Hawaii baseball team and he was drafted by the Seattle Mariners and he's currently the general manager of ESPN Honolulu. He is Matt Apana and today we are going beyond sports. Hey Matt, welcome to the show. Hey Rusty, how's it going? It's an honor to be part of your show. Thank you for having me. Matt you've been you've had such an incredible career and we're so happy that you're the general manager of ESPN Honolulu but I want to go back to the beginnings of Matt. When did you start baseball and what is it about baseball that you love so much? You know that's a good question. You know I think it probably all started from my great-grandfather watching sports all the time and I just took the baseball playing baseball on the streets in southern California and it just grew on me and you know my dad and my mom obviously my family they supported me. I ended up going to Dodger baseball camps at the age of eight and yeah and then you know when we moved to Hawaii I just felt like there was so much opportunity here locally and going to Roosevelt it just I had some great guidance and great coaches and they just inspired me and one thing just led to the next so very very fortunate and yeah and like I said it just it became a passion. It really did for me. And Matt I remember when you played at UH and I mean you were awesome as our pitcher and what was some of the best things that you enjoyed playing at the University of Hawaii? No Russia I think one of the best things about playing at the University of Hawaii was it was about the people of Hawaii pitching in front of family and friends and all of our great fans that we have here. It was very motivating to get out and be able to pitch in front of huge crowds and at the same time become successful. You know you just really wanted to do so well for for not only my family but also for just the whole state of Hawaii. Then you guys seem to have some really great camaraderie and I know that you played you know for coach Les Murakami and he oh he's the legendary long time UH baseball coach but what are some things that made coach Les so great as a coach? Well you know it's funny because when coach Les first came to watch me pitch when I was at Roosevelt High School we were playing pepper and one of my teammates actually let go of the bat and it flew in the air and coach Les had the duck and we're all like oh my god coach Les you almost hit coach Les but to my point I think coach Les was just a huge icon for baseball for our baseball community in the state of Hawaii. One of the best things that coach Les did that now I realize is he brought this a lot of people together. There was he got he had a lot of support behind him behind the whole program in general and I think at the end of the day when I look back at coach he was a he was a big father figure. I get a little emotional on this because he said if you pitch for Hawaii you will be successful in Hawaii as a business leader or as a as a person that works for a company in Hawaii and that was so super true on on on what he said but to his credit he was a great planner he was a great visionary as a person as well so I really appreciated his guidance throughout the years of my younger lives. Yeah I've only heard some great things from you know past players that that had played on the University of Hawaii baseball team and and Matt I want to ask you how did it feel to be drafted by the Seattle Mariners? It was probably one of the biggest moments in my in my whole baseball career. You know I was hoping to go earlier but it didn't work out and when I got the phone call that night saying they're going to give me a call in the morning I was like great I'm looking forward to it sure enough their promise what they what they said they delivered and it was something that I wanted to do and move on and I was I was very you know very again very fortunate very blessed to have an opportunity to play at another level of baseball. What was the culture like with the Seattle Mariners organization at that time? Yeah the culture was actually really good um that's you know from that 90 say 95 98 years when they had King Griffey Jr. Randy Johnson, Edgar Martinez, Jay Buehner they really established a strong winning culture at that time and I think a lot of that had stemmed from their minor league organization they had a bunch of great leaders and coaches and more or less this people person in the organization you felt it was a big ohana in that sense and you know obviously the coaches had tremendous amount of experience I was able to learn so much more about just the game of baseball and and apply more my attitude and work ethic to how far I went in the organization as well. So Matt what do you see as the biggest differences when you know you're transitioning from the college level to the professional level? Right I think it's really about growing up it's really about learning how to work harder um establishing obviously a great attitude you know setting goals some things I think when we're younger we don't really see that until we start getting in the mix of it it's very competitive I mean here here I am at you know barely six feet where you're going up against competition pitchers that you know are six five throw 98 miles an hour you know and I was throwing the ball in the low 90s at that time but what you have to do is you have to outwork people and you have to want you have to want to go somewhere and I think that's what I learned the biggest change was the drive that I established and the confidence that I that grew upon me I started to learn how to trust myself to be better at what I what I needed to do to be successful and and from that point on I really grew as a baseball player as a pitcher just as a person in general. So I like hearing these insights from you Matt and what what big life-changing event happened to you while you were playing professional baseball in Taiwan? Yeah you know we taught we touched upon it a little bit it was actually I had just finished my career with the Mariners unfortunately ended with an arm injury but I went on and I played baseball in Italy in that 98 season and then I actually went back to Italy to play in 99 and for unfortunate reasons I I left just because it wasn't really working out things have changed a little bit and I actually went to play independent ball in Schomburg Illinois from that point I actually got an opportunity under Hadeo Nomo's agents to go and play in Taiwan so when I got to Taiwan it was early August and the season in Taiwan doesn't end until mid to late October but it was in September and it was to this point at 1.44 in the morning and we were watching the Exorcist 3 now sitting on our couch and the couch was rattling or shaking and we thought you know who's trying to scare who but it ended up being we ended up being in a 7.6 earthquake massive earthquake and that really scared me I didn't know if I was going to live or not because the walls were crackling our our our building was um you know who knows if it was on the verge of falling or not but good thing it held strong but as a team we ended up going to the epic center as as we do you know being professional baseball players in Taiwan to give back to the community and to help these people who were no longer living in their homes or condos because it was there was complete devastation from from that 7.6 earthquake and having to experience that was very eye-opening and I share this story a lot because as we walked in you see buildings devastated flattened on the ground and here we are in our uniform tops carrying boxes of food for these people and when you get there you see people laid out on grasses and lahala mats and and then you see the military reserves I suppose that's who they were you know carrying dead body bags as well putting them in containers it was it was a it was a sight that I'll that I'll never forget um but it also was a very big life changing moment because you just felt how important life and family are to you especially when you're experiencing that at that time and I like hearing how how you and your teammates were stepping up to really help you know with you know giving food giving water to people in need there and I guess that's that's the positive thing is you can see the good in humanity when some bad disaster happens at times right yeah right I mean you don't think about it too but you know the the the source of water you can no longer drink that water because of disease you know people who knows how many you know people were were fatally hurt or just or had passed at that time and then you think about all that disease that gets carried through so you got a limit on on food as well as as as water and so you really almost have to fend for yourself and find whatever you know leftover food or or water you have in storage yeah things that you don't really think about until it actually happens and I didn't really think about it at this time as well because Taiwan is an island and you know we had tsunami watchers as well I mean I could just only imagine if a tsunami came and here I was or here we were just trying to get out into an open area in the street so nothing could come falling down on us geez so Matt I want to talk with you about ESPN Honolulu tell tell me about what you do with ESPN Honolulu yeah um you know I'm the general manager of the station um you know we have two we have two stations here we have ESPN Honolulu we have CBS 1500 so I oversee those two stations and in a staff of about 13 people also I oversee our sales staff as well and so I do tend to wear different hats and um and yeah I just think here at ESPN you know we have a very close knit sort of ohana as well and um and we do a lot of a lot of things a lot of big things for the type of staff that we have I would like to say that we you know we have a pretty good culture that that's surrounded by just people who who back and support one another up and work very hard at it so Matt you know social media has really changed how the regular tv news broadcasting is done as well as the newspapers how how has social media affected radio yeah I think you know that's a good question recipe because I think with ESPN Honolulu we're in a good situation just because we're so unique in the media market just being sports alone you know we have a great partnership with the university of Hawaii where we broadcast all the all the all the games all the major games you know obviously university Hawaii is the biggest show in town but I I think what we've done is we've pivoted and we've adapted to to enhance and to grow our our platforms in digital you know and I think with that we we're starting to really if that's if maybe if I back up that's an area where we've really started to pay attention to and I can only expect us to continue to grow and do really well with our followings the good thing too that I like about what we do with digital is we can take a digital podcast and edit it and also play it on air on radio so people can listen to that as well so we have a lot of flexibility doing what we do and it's a good mix of for our people who advertise with us how we go about creating engagement for our local fans here it just gives us a lot of flexibility just because what we do as a radio station but now also what we're capable of doing in the type of resources that are continuing to evolve as we move forward because obviously we want to get better at what we do it just it just creates that much more enhancement with an impact as a whole station well I I love listening to the sports animals radio show with Gary Dickman and Chris Hart now why do you think those two make such a great combination you know that's everyone everyone wonders about those two guys you know I think what they do is the biggest strength is you know they're they're they're like they're I like to call them like a married couple you know and you got Chris who's a lot looser on the on the sports side and I think he honestly does that purposely is to create that engagement and and that excitement and then you got Gary who's all about sports and stats and and tends to combat everything Chris says or does and so I think what that does is there's a yin and a yang yin and a yang that that's there that you know opposites definitely attract in this situation um yeah and they've learned to obviously learn to work with each other very well have grown together and and sometimes I guess my job is learning how to you know be able to make sure that they're they're getting along all the time all the time as well but I think what they've done though is they've done a good job of being consistent they've they've maintained their style their type of show during that that pm drive time and I think it's really paid off for them over the long haul yeah like you said consistency I'm they're so good at what they do and and also Bobby Curran I I really enjoy listening to Bobby Curran and why did why is he so successful yeah well Bobby I think he he really took off when he first got into radio and he's really established himself and been recognized as one of the the top sports casters in the in the state of Hawaii you know and and I think with Bobby you know he's always again it becomes what he delivers and and he's all about the local people and about delivering content and I think for his sake that has what's made him and built him to be as strong as he has been he's very influential he knows a lot of people and what his strength to me really is is his relationship building I think which has really helped him establish himself in the market as who he is um with his show and as a play-by-play analyst um broadcaster for both football and basketball oh without a doubt and he's so professional and um really really love Bobby Curran and Matt you and I met a few weeks ago for lunch and we you know we could talk forever and you have both of my books and what are some some principles that that really stood out to you in my books you know I gotta admit I think all principles and I'm honest being honest with you and I know I talked to you a little bit about this before the show those all keys are so critical to developing a champion um or just a great overall person as a leader whether it's in sports or business you know um and as I was reading each of those keys I reflect back as I was reading your book through my playing days and what I had encountered you know the environment you know having a pitch in high altitude cold weather extremely hot weather and or or extremely humid humid weather you and if you don't have the right mindset at play at a professional level you can get eaten up very quickly and so as a young professional baseball player sometimes those first goal rounds are not the most successful but you have to go back and and reflect back and look at you know your performance and figure out how to make adjustments you know and so next time when you're in adverse situations you know how to handle yourself a lot better and really it comes down to is just having the right mindset you know it's not having really that 93 mile an hour fastball you can still beat someone with an 88 mile an hour fastball you just have to be a lot smarter you have to be prepared but I loved all the all the all the components that you talked about obviously passion was another one you know and I fully believe that if you're going to be successful in whether it's in sports or in the type of business career that you have you have to have passion for what you do you know and I've seen it even here where it's it hasn't worked well for certain people because money is more important than success and I think once you can establish how to become successful I'm a true believer just I can baseball if you work hard you plan hard the the numbers are going to come the championships the stats are going to come well in this case revenue for the station will come good things are going to happen to those who work hard and have a good attitude about it instead of just thinking about this one thing and sometimes that's you know maybe that just that that that paycheck at the end of every two weeks I like that you like the book and and how you can see that those eight keys are so necessary for not just achieving success but to sustain success and Matt you and I have been on you know teams before whether it be in sports or business and we know what the leader or coach does that was good or not so good what do you feel some of the best leaders or some of the best coaches do you know I think one is is is they build a culture of having the right mindset you know they're able to explain it to people or to a team to get everyone on the same page have everyone have the same mission vision also preparation having a plan is critical as well you know and and I'm and I'm a believer on discipline as well but doing it in the right ways because it teaches you a lot about just in life in general you know but it also teaches you how to have the right mindset and I like to use the word mental toughness you know trying to establish mental toughness is critical especially when you're in adverse situations or even in the job environment when you know you're having to put out fires and you know you got to learn how to step back and not be so reactive as a leader and be able to understand and show more empathy towards you know your your players or your coworkers and and I think if you can show that caring attitude and leadership in that sense I think you can go a long ways and I think it's really about the foundation then about you know how much skill set you really have obviously skill set is important but if you don't have the right attitude the right work ethic mindset all that that we talk about it's hard it's hard to be successful I completely agree with you 100 Matt and how would you describe your leadership style as a general manager and that's a tough question to ask because I you know I mean I'm probably and it's probably my weakest point here is I don't really like to talk a lot about myself but no but I tend to look at things I don't like to micromanage for one you know and obviously I'm going to tell you that I'm always looking to get better you know and I think it's my work ethic and my attitude and my caringness towards people is what really gives me the strength to gain that respect from everyone else because I do my best to try to respect and to give people that autonomy especially our department leaders that opportunity to be successful and and to do what they need to do where I can just be there to provide that support and that guidance for them and it and it really comes down to communication I mean but again right we can all get better at communicating even myself you know there'll be moments during that day that you know maybe I go back and I reflect and man I wasn't a very good leader or communicated very well today or gosh you know I didn't really handle myself because I let someone you know kind of get to my nerves a little bit but I think as you can learn to make adjustments as a leader especially in adverse situations and and you continue to want to grow I think that's what strength makes you a stronger leader in whatever you do well you know the greatest leaders are always learning and that's what makes them even greater and Matt that's why you're a great leader I you have that awareness about you know always you have a constant striving for excellence and I love that about you and you as busy as you are you're also coaching some little kids in baseball what is it about what what do you like best about coaching these kids in baseball yeah I have a nine-year-old and a five-year-old and and as they're getting deeper in the baseball gosh I just only can imagine how much busier my schedule gets but what I really like about it is that I'm able to give back not only to my sons but to their teammates a mind a mindset that will help them become more successful I'm I'm always big on learning from failure and I think what happens is you see a lot of parents that yell at their kids for making mistakes instead of helping them understand why they made that mistake and at the at the end of the day if my son makes a mistake we're going to talk about it and we're going to be better from it because obviously no one wants to make mistakes and I think out of that whole thing it's not about really the fundamentals of baseball yes fundamentals are important but it's really about getting the right mindset for these youth so when they go on and become whether they play baseball or any other sport or just you know go on to college and and be successful in their careers they're able to understand and you know having that right mindset it's okay it's not it's it's okay to not be perfect you know as long as you keep pushing yourself forward right and so I think that's the biggest thing that I want to get across at the moment my nine-year-old is actually doing pretty good to be honest with you but it's my five-year-old that's going to give me challenges because one thing I do like is he hates to lose which I love but at the same time he's he loses his composure a little bit but again right he's only five years old but it's really that those little teaching moments that I love the most because it's going to help him just be a better person overall when it comes to you know you win with grace you lose with grace right totally agree and and Matt you know as besides that devastating earthquake that you experienced and besides coven what's a what's a big adversity or challenge that you've dealt with that you overcame in your life yeah the biggest thing was probably my arm injury well and um you know as I reflected back on some of the information that I was that I was putting together for you um you know I was nearly ready to get called up into big leagues in 96 and it was actually in um in 97 was when I hurt my shoulder and I tried to pitch through it but it just I just I just couldn't sustain it and and when I went down that was devastating mentally for me and it in effects it affected me for a while even after my playing days but um being able to come back through that whole process was hard because you just weren't the same physically and mentally you did what it just never felt right and so going through that whole situation but then being able to come back and still having that opportunity to play for a little bit longer and then having that experience to go to Taiwan and Italy to play professionally um that was that was a that was a huge comeback for me but at the same time when it all ended it stayed with me for a while because of my passion and love and all the hard work that I put into it um and so I look back to even after that and I reflect as much as maybe my early 30s it affected me for a good maybe four or five six years you know um but it made me a better person at the end of the day and I when I look back at it it was all meant to be for some reason and maybe that's why I'm in this in this place now you know so um there's always yet you look at those adverse situations and there's always something that's going to come out of it in the long run in a positive way I can totally see that how the physical and the mental part really affected you but Matt you know you you are a great leader and an even greater person and I want to thank you for taking time to be on the show today thank you Rusty I really enjoyed the time with you and I'm sure we can probably continue to the talk for another 30 30 minutes to another hour here but I really do appreciate your time and very fortunate to be a part of your show thanks Matt and thank you for watching Beyond the Lines on Think Tech Hawaii for more information please visit RustyKamori.com and my books are available on Amazon and Barnes and Noble I hope that Matt and I will inspire you to create your own superior culture of excellence and to find your greatness and help others find theirs aloha


From around the globe, it's theCUBE with digital coverage of AWS Public Sector Partner Awards brought to you by Amazon Web Services. Hi, and welcome to a special production of theCUBE. We're talking to the Amazon Web Services Public Sector, their partner awards program, I'm your host, Stu Miniman. And we're digging in on education. It's one of the sectors, of course, public sector looks at nonprofits, looks at the government sectors and education. And of course, when we talk about remote learning is such a huge important topic, especially right now in 2020 with the global pandemic. So happy to welcome to the program. We have two guests. First of all, we're representing the award-winning company Mohamed Hawke. He is the co-founder and senior vice president of architecture and engineering with E-Lumen. And joining is one of his customers, Damien Doyle, who is the associate vice president of enterprise infrastructure solutions at the University of Maryland, Baltimore County or UMBC as it's known. Gentlemen, thanks so much for joining us. Thank you, thanks for having us. All right, first of all, Mohamed, congratulations, as I said in my intro, such an important topic. I have two children that are dealing with remote learning. I have lots of friends that work in higher education and know in the technology space. So your company is the 2020 AWS Public Sector award winner for best remote learning. I'm sure there is a space that has a lot of competition and of course, leveraging public cloud is a great way to be able to ramp this sort of thing up rather fast. Give us a little bit, you know, you are the co-founder so would love to hear a little bit of the origin story, your background and tell us about what differentiated the E-Lumen. Sure, E-Lumen, we provide managed products and services around end user compute with a focus on education for providing access to applications and other technology resources, course content, course applications in the public cloud so that users are able to use whatever device they have wherever they are and have access to those applications that are required for completing that coursework. They can be in, you know, at home, in their dorms, at a corner coffee shop on the side of a mountain in the Middle East, wherever they may be, but leveling that playing field so that they can have access to any of the demanding applications on any device is what our goal is, is to make sure that we're not having technology be a barrier to their learning. Fantastic. Damian, if we could turn to you then at UMBC, maybe if you could give our audience a thumbnail of the university and I have some idea of the challenge that was put in front of you when you talk about E-Learning, but maybe you could give us a little bit of the pre-COVID and what you were faced and what you were looking at when it came to dealing with the current situation. Sure, be happy to. So UMBC is a mid-sized public institution. We're sort of suburban, about 14,000 students and we have undergrad, graduate and doctoral programs and we have a heavy focus on a lot of the STEM disciplines. And so pre-COVID, very based in collaborative environments, active learning, but hands-on. So a lot of our programs really do have a lot of that and we leverage technology very heavily, even if it's in, whether it's in engineering, biology, any of those kinds of programs. As you said, the challenge became how do you very quickly pivot into an entirely online model when you sort of scattershot all of your students and you don't really have a great sense of what they're gonna have access to and the abilities and connectivity they're gonna have. So this kind of thing was really critical for us as we made that transition. Excellent. Mohamed, were you working with UMBC before the current move to go remote, give us a little bit about the relationship and how that started? I believe actually that the pandemic was the impetus to kind of drive this forward. Damien and his team reached out to Illumin looking for a solution that would allow them to kind of have students access the applications that they would normally would have access to in their physical computer labs, but with the change and not having to access to those labs anymore, needed a remote learning solution, a remote access solution for being able to access those, high compute, high graphics, memory-intensive applications through the cloud, taking into account the fact that students won't have the highest end computer laptop. They'll probably be working on a Chromebook or a lower end machine, but need that compute power. And then we had to kind of provide a solution pretty quickly because it was, schools were shutting down essentially, physically shutting down and needing to continue on with their coursework. Yeah, Damien, I'd like to understand from your side, can you share with us a little bit the timeframes? How fast did you go from, oh my gosh, we need this, we need proposals, we need to roll this out, and we need to have students and teachers back up and running? Well, I think the one thing from our side, we had already known of Illumin and we had been looking at that pre-COVID. We knew we needed a product that provided us this kind of agility and really gave the students some better access to the computing tools that they needed. So once we identified that, the thing that was amazing to me is, is we moved from our existing system over to production, Illumin in, it was about two and a half weeks sort of start to finish. And to get all the images, to get all the technology running, tested and everything up and running in two and a half weeks for a full solution for a campus is was pretty amazing. And that was one of the real benefits we saw as going to the cloud. We also looked at this outside of COVID as something that really provided a major benefit to the students so that they could work from anywhere at any time, rather than be sort of tethered to that physical lab. Well, I'm glad you raised that. So if you could Damien a little bit, help us understand, how much are you using cloud before? And it sounds like you believe that, in the, I guess if we say post COVID world, you will probably have some hybrid model, would that be fair to say? Yeah, I think before we did have a different solution that was still cloud based, it was part of our business continuity. So we still had some semblance of a virtual computing solution in the cloud, but it wasn't that extensive. And a lot of our individual programs, chemical engineering, geography and others were using physical labs that the students would sort of schedule times and be able to work in as part of their coursework. Coming out of this, we fully expect if we're going an extended period of time where students are able to access these materials and these demanding software packages at any time from any kind of device, coming out of COVID, they're not gonna wanna go back to that model where they're asked, they have to get permission and go in in limited hours into a physical lab and sit there. This is going to be the expectation going forward, is that they have this kind of access and this kind of flexibility from now on. Yeah, this is, I mean, they've gotten a taste essentially and so, they see how easy it is to complete their coursework without actually having to trek across campus into a lab and kind of fight with the population to find a seat. This basically will become an expectation of an offering. Yeah, Mohammed, what I'd love if you could drill in a little bit for us there. Architecturally speaking, of course the cloud is built to be able to scale and move fast. So if you need capacity and need to scale up fast, that's great. If in the future you still wanna leverage this solution but you can scale down, that should be possible. So maybe give us a little bit of how AWS architecturally supports what you're doing and just from a pricing solution standpoint, how you'll be able to support the customer in today's environment and however that path goes down the road, you'll be able to support that too. Right, I mean, so with the AWS cloud, we're able to, as you said, scale up or down as demand is needed, but we've taken that even a little bit further where we're scaling based off of student scheduling. So if we've got a course that we know that is running from 10 a.m. to 11 a.m., prior to that course starting, we'll scale the environment up so that it's available for those students if it's more of an in-course lab session and then spin things back down after the course is done so that we don't have those many, many machines sitting there running and burning the hours and running up the bill. Physical environment, once you've installed it, it's there, it's always running, you cannot do that, but with the power of the cloud, we're able to go up and down. We're able to take things, scale things down off hours if we look at the patterns for student usage, off hours, overnight, take things down because you don't need those machines sitting there running all the time. And this is one of the biggest differentiators. So many times in higher ed, we struggle to have to explain to companies and vendors and providers what our needs are and how we're very different from corporations and other verticals. With the Lumen solution and the capabilities in AWS, we're really having this tailor to our student schedules, to the class schedules, and that kind of flexibility makes the product economically viable for us, but it also means that we don't get nearly the kind of pushback from the academic side because it is really tailored to meet their needs versus just something we're kind of shoehorning in. So that makes a huge difference in terms of adoption and the way it's perceived from a marketing and acceptance standpoint. Yeah, Damian, I'm curious, once you did that initial rollout, how much of an on-ramp is there for both the education, the educator side as well as the student side, and you talked about having some flexibility as to how and when students use things, that sounds great, but do you have to change office hours or the hours that the staff are leveraging that? I'm just trying to understand the ripple effect of what you're doing. No, it's a fair point. We have done fairly extensive training. The students picked it up very quickly. What we, with students, if there's a tool that they can use to do their work more effectively, they're going to use it, whether it's something we provide or something they find through other means. But what we've done is reached out to all of our faculty that were training, that were teaching in our physical labs and tried to work with them to understand what this solution is, how they can sort of rethink some of their classes and a couple of our departments have actually taken an approach of rather than sit everybody in a virtual lab the same way they would sit people in a physical lab, they're moving some of this to more asynchronous so that the students can sort of work at their own pace and sort of rethink how they structure some of those classes because of the flexibility being provided. But it does take a lot of training from the instructional side and some rethinking of this. But the end solution is something that reaches the students where they are and the way they want to learn which is a really powerful thing we're always trying to do. Excellent. Mohamed, I'm wondering just broadly learnings that you have from what's been happening. Obviously, I'm sure you've been quite busy in responding to things. What's been the impact on your business? How has AWS been as a partner to be able to support the needs of what you're doing? Well, as you can imagine, things have just really blown up in terms of demand and being able to, again, through the power of the cloud just being able to scale up and rapid deployment. As we spoke about earlier, this deployment was two and a half weeks from start to finish. Being able to do that with AWS tools have been critical in moving things forward. Excellent. Damien, it's back to you on this. Obviously, if you had had more time to be able to plan this out, there might be some things that you would do differently. But what have your learnings been with this? And if you've been talking to your peers, any advice that you would give as you've moved through this rapid acceleration of the move to remote? Certainly, I think we would have certainly done some things differently, but we had been talking about this move for three or four months ahead of COVID. So for us, it wasn't quite as rushed as the actual deployment wound up being. I think the big thing is having a vendor and having a partner where you can understand all the options. So the good and bad of the cloud is there's a hundred different ways to do almost anything you want to accomplish and taking the time to understand what the different features and the ramifications of how you deploy and how you think through that. For us, we deployed one way because we could do it very quickly. And then we took the rest of the semester and part of the summer to do some more thorough evaluations to really ask our constituents, do you like this method or do you like some of the other possibilities and see which user experience they liked more? And then we're able to work with Illumin and they've been able to be very nimble in adjusting the services to meet what we've gotten our feedback on. So I think if I had to do it again, I would have done that testing ahead of time, but that's a very minor thing. These are really sort of small tweaks to just make life a little easier, not fundamental differences in what we're providing. Yeah, Damien, one last question if I could. Sorry, Mohamed, just I'm curious from the financial standpoint, how much you felt that you understood what costs would be and some of the levers as to what you were using and the impact there. We've seen great maturation over the last handful of years as to transparency and understanding how cloud actually is billed. But just curious if you have any final comments on the financial piece of things, seeing that it probably wasn't something that was in your budget for the last quarter. It wasn't, that's very true, but we also knew that it was essential. So what we realized was we didn't know how often a lot of our physical labs and these classes were being used. So we knew there was going to be some unknowns. We'd move to this, we'd have to see what adoption was, but being able to get the reporting out and working with Mohamed and others to really start customizing in the cloud, that's the beauty of it is we recognize, we saw some really fascinating patterns where during the week people would use this sort of as you'd expect, but on the weekends, it was in the evenings. Nobody's logging on Saturday or Sunday morning, but boy at eight PM, there's a good bit of usage. So we could tailor and do some of that off hours work and really slows things down. Having that visibility has made the economic piece much more viable and really being able to tweak the computing power with two different needs of the different classes. So it's actually been fairly easy to understand, but it was a ramp up where we had to sort of guess it first and then understand our own processes, but that's more sort of the, if you don't have good data coming in, it's hard to get it out. Excellent, Mohamed, I want to let you kind of give your lessons learned. Obviously it's a technology space you've been in and it's just been an acceleration of some of the things you're working on. So lessons learned, advice you would give to other companies, other universities in education, no facilities out there. Right, and this is again speaking to the power of the cloud, some of the biggest lessons learned here is you don't necessarily need to get it right the first time. As Damian was saying, we went back, kind of analyzed what we were seeing and after the initial deployment, took a look at the actual usage and kind of adjusted based off of that, according to that, taking in feedback from faculty members on how they were using the system and tweaking the presentation or tweaking applications on the backend for accommodating those needs. That's the power of the cloud, being able to adjust on the fly. You're not, you don't have to be committed to every single bit there and being able to change it on the fly is just something that is kind of natural in the cloud these days. Excellent. Well, thank you both so much for joining us. Damian, thank you for joining and moving forward, sharing your story. Wish you the best of luck going forward and Mohammed, big congratulations on winning super important category, especially here in 2020. Congratulations to you and the team. Thank you. Yeah, thank you. All right, stay tuned for more coverage here from the AWS Public Sector. It's their partner awards program. I'm Stu Miniman and thank you for watching theCUBE.


Hello, I'm Secretary of State Kim Wyman from Washington State and I want to first say thank you for inviting me to be part of DEF CON 2020 and share a little bit about Washington's experience with vote by mail and the challenges that we have with election security and how we've tried to overcome them and how we're preparing for the upcoming August primary and November general elections here in Washington State. So I'm going to real quick share my screen and walk you through kind of what Washington State has done to prepare for the 2020 election cycle and how we are dealing with all of the things that are coming at us in 2020. So first and foremost I'm pretty certain each one of you has been very aware of Russians trying to hack into the election system in the 2016 elections and I've been an election official for many years and have watched this kind of progression that we've had in the election community for election security and in the 1990s we instituted cutting-edge technology like using the driver's license information to register voters and put election night results online and from that moment to today election security has been something we've been aware of and have worked really hard to try to make sure our systems are secure and I think right up until 2016 most election officials felt pretty good about the level of security that we had in our industry and today what I want to do is share with you kind of what a Washington State's election system looks like how we inspire public confidence in our process and what our kind of triad of election security is and then talk a little bit about the impacts of COVID-19 on our election system and quite frankly the nation's election system. So I got into election administration in the 1990s, early 1990s and I can tell you when I started in this field my goal was to become an elections expert to learn and understand how the election process works and to be able to conduct elections in a secure and accessible manner and then the 2000s that evolved because suddenly the demands on our system were so great that we had to rely on technology to be able to produce election results quickly and to serve our voters well by providing a lot of information on the internet so we now suddenly not only had to become an elections expert but also had to be an IT expert and understand at least how systems work and how to use technology in a way to help us oversee our election systems and about the time I started feeling like I knew what I was doing in the IT world and I say that because I'm just dangerous enough to know how to use technology but certainly not to any of the levels that any of you are but by 2016 it became very apparent that not only did we need to be election experts and understand IT systems but we needed to know how to secure them and cybersecurity for the last four years has certainly been front and center in election administration so let me start kind of with the the basics of why I think the United States election system is very secure just because of the decentralized nature of it so when people think of elections I think oftentimes they think of the national that there's some sort of national election administration when in fact it's a really decentralized network made up of 10,000 election officials nationwide who like me are either appointed or are elected to office they take an oath to uphold the constitution and the constitution and laws in their state and I think the overall strength of this system is that no single person or group has total control of the overall operation well and so when you you get down to election systems in each individual state and I bear with me here I'm having a challenge operating my own PowerPoint system you have the same similar threads we have a decentralized system here in Washington and each state is going to be different and unique and Washington is certainly one of five vote by mail states but I think you can look at the the individual makeup of each state and know that this decentralization is actually happening within the states as well so here in Washington we have the secretary of state myself my office is the overseer of elections in Washington and I'm the chief election officer I have the rulemaking authority to put into put into effect rules that adapt our laws that our legislature passes so this is how counties implement the day-to-day operations on the ground we manage a statewide voter registration database and oversee the election initiative and referendum process as well as do election night reporting we do voter outreach and education and we certify and train our election officials as well as the election tabulation systems that are used in each of our 39 counties our system is decentralized in that 39 county election officials are the chief election officials in their county and they are elected they oversee the voter registration processing and the management of elections in their counties so they're the ones who are actually issuing ballots and operating the tabulation systems and any kind of other tech like ballot sorting equipment or the actual facilities that process ballots they also provide voter outreach and education to their voters and manage all of that information usually through social media and their web pages now I imagine most of you are aware that in 2016 the federal government designated elections as critical infrastructure and this is significant because it put us on the same national security level as the power grid or the banking system and so what it has done in our state it is really helped us reinforce what I like to think of as our triad of security for Washington and our vote by mail system I forgot to mention that earlier in 2011 Washington state became a vote by mail state and at that point every single one of our counties had had to conduct elections entirely by mail and mail out a ballot to each of their voters so this process didn't happen overnight we really started back in the 1990s and in allowing any voter to be a permanent absentee voter which meant they received a ballot every election and that evolved to the early 2000s when we had the closest governor's race in the country's history and following that and following a lot of things that went wrong both at the polling place and in the absentee balloting we determined that we couldn't do both elections well and the legislature allowed counties to move to vote by mail voluntarily in 2005 many of the counties in the state did move to vote by mail but it still took our state five years to completely become a vote by mail state and that's because it took time to get the equipment and facilities to really provide for processes and procedures that we could conduct elections well so which is what leads us to the triad of security for Washington vote by mail elections so there's three real pieces to this this election model it's the physical security the logical security things you would just do innately and the digital security that we provide so let me start by just talking through kind of the physical security elements that are inherent in a vote by mail election first and foremost I think the strength is that we are we rely on paper ballots even the ballots that are issued in a vote center and voters use to use an assistive device to vote them all of those ballots are paper ballots that are printed by those machines that the voter actually does mark and our process so we can always go back to those paper ballots and do a risk limiting audit or a full recount if we need to to really inspire that confidence that voters need to have in the election results we have ballot drop boxes here in Washington state in addition to the the blue mail boxes provided by the usps these ballot drop boxes are throughout the state and they're managed by our county election officials they we have about 470 of them statewide and they open at the beginning of the voting period which is 20 days before election day anytime they are emptied they are emptied by two employees of the county and I know that sometimes they try to have that be a democrat and a republican when when possible but they are staffed by they are they are staffed that are employed by the county and take a note to uphold the laws and the procedures and policies for the county so they are held accountable for that processing they seal they lock and seal those boxes and they do that with individual numbers numbered seals that are actually logged and kept track of two so pretty much all of the steps I'm going to talk about are what we use to build an electronic and physical audit trail that we have in place in case we ever get into a recount or a situation where anyone is calling into question our processes we can go back and show which employees had access to which ballots at any given time and we have dual control environments throughout this so it's it's just really important that those audit logs and dual controls are in place to inspire confidence again this is true throughout all of our election processing facilities as well we have secured storage for ballots and ballot materials that involve physical locks in some of our counties it may be keycard access servers the server room for example for ballot tabulation equipment is in a secured room and has very limited access and again is log or keycard controlled so we have a record of anyone who has access to it and then we have security systems and keycard access throughout these facilities as well to just have that physical layer of security on top of any cyber security elements that we have in place the second I guess group of security measures are those sort of logical ones things that just make sense of course the first one would be that our tabulation systems aren't connected to the internet and I know this is something that came under fire in the last few years nationally but our county election officials have made sure that their tabulation systems not only are air gapped from the internet in every way but there's no modems operating or even installed in those systems and that any connection that needs to be made with the vendor or with with election officials is done onsite in that air gapped environment we we have a real love of transparency in elections and certainly that was true in any processing facilities or courthouses across our state that all of the activities are observable we we like it when it's bipartisan when we have democrats and republicans overseeing that process we actually asked the local county parties to appoint observers to come in and represent each of the parties if that can happen and of course in the COVID-19 environment that's becoming a bit of a problem for just space reasons our county courthouses oftentimes do not have a lot of extra space in the smaller counties so a lot of counties have used their security money to install cameras that run 24 7 during the voting period during that entire 20 days and then the 20 days or so after when we're certifying the election so that people can watch it online from safety and comfort at their home but also have that transparency so you can show that we're not we're not trying to hide anything or do anything uh kind of under the table um ballot reconciliation is a really important part as well um this would be you know counting the number of ballots that are received each day in the mail and then through the process reconciling them and being able to show whether a ballot was counted that was issued to a voter or not and if it wasn't counted why it wasn't and the only people that can reject a ballot are our bipartisan canvassing boards uh we also have um election night reporting and we have failsafe measures and and checks and balances that my CISO told me to to say is out of band communication verification and what we do on election night is counties are uploading their results to our state election night reporting system and we compile those results for offices that go across county lines and we take that initial file that they send us and then we verify it by a fax or an email that is sent by the county as well separately to just make sure that the information that we're putting up online is accurate and is the way that the county's reported it in the first place so all of these systems you know this logical element of our election security is really just trying to create that sense of transparency and provide a lot of checks and balances both physical and electronic that makes sense and are easy to explain to the average voter so that they can understand what it is we're doing of course the final leg of that uh that try that uh stool would be the digital security and for us we we really started working on this probably five ten years ago by modernizing our election system and now we have a highly segmented network system that employs multi factor authentication for all of our users we have um really tried to to increase and expand our user training and um outreach to just make them aware of cyber security issues and things that they need to do to make sure our overall system is secure and we've done robust testing and that happens on an ongoing basis and as as all of you are aware you know it's it's a constant battle to stay ahead of of hackers and people trying to get into our system so I think just that user awareness has increased in the last four years on a level on a scale that we just weren't doing before 2016 we're currently part of the national sensor network and this is not only the state system but counties across the state are part of it as well and we we built and have created a dedicated um election security operation center that is in my office but it also supports the 39 county election offices because what we found is that many of the smaller and medium sized counties are lucky if they have an IT professional for their entire county let alone in their their elections division and let alone a cyber security expert so um this partnership has also helped us have good communication and training with those election officials and do assessments in their counties that help them strengthen their system as well we also played an active role in the critical infrastructure build out of the election sector so these are all kind of the the key parts of of our system and I think that critical infrastructure designation also allowed us to create partnerships with our own state SISO and CIO but also our Washington National Guard who is a very strong cyber security team and they've come in and done assessments and monitor our system and and give us feedback and ways to improve our system they've also helped us do tabletop exercises and trainings for our users which again just continues to raise the awareness level and prepare to solve the level of our election officials we've worked with them also to build out our continuity of operation plans which have obviously a digital component to in the security of our system but also how are we going to not only detect and and defend our system or protect and defend our system but how are we going to detect an intrusion and recover and respond in case that does happen we've also worked with the state auditor's office department of homeland security and the FBI with similar threats to those activities so we've been focusing on all of this for the last four years pretty intently and as if that wasn't enough in early in the early part of this year we had to throw in a global pandemic because you know why not so what Washington was in the middle of our presidential primary when COVID-19 began to spike and we started having the first deaths in the country actually for COVID-19 and it did affect and impact our March 10th presidential primary we were on track to have turnout of about 60% and the weekend right before election day was when COVID really started spiking and the governor started to issue stay-at-home orders so we ended up with a turnout of 49% and I think we're running about the third highest in the country for presidential primary turnout and a lot of that has to do with vote by mail because our our voters were still able to drop ballots off and didn't have to go into a polling place or vote center on election day we've seen of course across the country presidential primaries canceled or postponed there has been incredible interest nationally on vote by mail and I've given many many talks and been on panels and and talk to media about our experience here in Washington and what I can tell you is that that our state is well we're implementing a primary right now our regular primary is on August 4th so we'll be in the middle of of the we're right in the middle of processing ballots for that election and certifying it and of course we're gearing up for the general election and my colleagues across the country are doing the same in their states trying to expand absentee voting or vote by mail as best they can because congress hasn't allocated money and as if that wasn't enough you know the usps is in financial distress and congress has just allocated an additional 10 billion dollars of potential loans so we're hoping that they they remain stable between now and november i think they're going to but the usps is a critical element for elections this year across the country because so many elections offices are expanding voting by mail to give you kind of perspective of how significant this is if you look at this map the states in blue are the states at the end of 2019 we're already vote by mail including washington oregon utah colorado and hawaii as you can see as you move east in the country fewer and fewer states either are vote by mail or even have access to absentee voting and so the grayer or the wider the states as you move east those are going to be the states that have the hardest time moving to expanded absentee usage and vote by mail because they don't have the systems in place high-speed ballot sorters envelope openers even just the physical space to socially distance and process those ballots as efficiently as we can here in washington or the other four states that do vote by mail elections so um we're we're trying to help those states gear up and they're working very hard to expand that capacity and capability and i think you're going to see a hybrid model across the country including my state here in washington of um voting at home and voting um in person because even here in washington one to two percent of our voters will vote in person so so it's going to be a heavy lift but election officials are working hard to make sure it's going to happen and in that environment of course we introduce the politics and uh it's bipartisan i like to talk about these things in terms of bipartisanship but we have republicans on on the right certainly the president and senator mcconnell and some of their associates who are concerned about voter fraud and on the left we have members of congress like senator widen and senator klobuchar who are really trying to expand vote by mail access and voting at home and and unfortunately out of the white house and department of justice we're hearing about voter fraud on a level that is implying and his point blank said that there's going to be rampant fraud and vote rigging and you know this is the environment that election officials are operating in and trying to inspire confidence in and so i say that because um we're seeing success stories already in in the primaries that have been held in the last month or so i think kentucky is a great example of um the innovation that that election officials are trying to bring to the covid environment and make voting the same experience with a lot of other persons all of them um so you can see in these pictures these are things that happen on election day in kentucky they have a huge supportage as we have seen across the country uh election workers because many of our election workers are senior citizens for the last month for 2019 so they have a lot of life is making these mega polling centers and i think this was right in their uh right in the next room i see in that next room right here in noxington uh i believe it's where these pictures are uh when they made massive uh noxenters before they come and socially distance and receive their votes and they were able to process these as well um the other thing of course is i was talking about is election officials they have a lot of offices throughout the state that are that are service daily by a lot of offices so um you know what we're really trying to do right now here in what i know we're doing here in washington i think what we're trying to do across the country is provide voters options and make sure that those are secure options that i can believe in and try to inspire confidence in elections and so i think with the the biggest thing i'm concerned with is all of the arts and charter about voters in russia if you don't have either russia vote i mean complete poll site voting um is uh really dealing with the misinformation and disinformation that are out there um not only from the president or from members of congress or whoever but you know from foreign actors like russia uh we have a national movement or national movement national hashtag trusted info 2020 and what we're really trying to do is drive um voters to trusted sources like secretaries of state offices or county election departments or local election departments that are going to provide info that you can believe in and know is true and uh you know different jurisdictions have different other hashtags and taglines like ours is the vote starts with you and really trying to empower voters to um to to know where to go to get information on social media and uh online so some of the security measures and practices we've implemented here in washington state i'm sorry that we can't be together in person but i i guess i want to leave me all with uh a sense uh for all of the work that you're doing to try to find the vulnerabilities in election cyber security and giving us feedback so we can make the system better i i think as i looked back a few years ago to uh to see early back on voting measures uh where you know you are pointing out some pretty bad things and i think that it was it was difficult for election officials like myself to hear and are a major reaction to push back and say we're wrong and what i love about datacom now and what i understand more is that we're to work together and we need to um help us find those vulnerabilities and give us some help in the insecure election system because at the end of all of this no matter where we are in the political spectrum no matter what job we have in this process we need to inspire confidence in the public that our elections are accessible and secure and that the election results are fair and accurate and we look forward to working with you in the future i hope in 2021 that that you'll invite me back to uh to death gone and we can have these conversations in person but feel free to reach out to me uh the Secretary of State's office and uh here in Washington uh my email address because this is certain fashion probably put to the script it was kim dot wyman that's k-o-m dot w-y-m-a-n m-a-n at s-o-s dot law dot gov s-o-s dot w-a-n dot g-o-v so thank you so much for giving me time at the conference i hope you have a great conference remotely and we hopefully can be all together next year so take care


So I'm Marie from the OpusDec Foundation. I'm joined by PTL's Steve Baker who's going to review orchestration, codename HEAT, and David Lyle who's going to review Dashboard, codename Horizon. Today our speakers will each have about a half hour to walk through their latest project updates for you, followed by about five minutes of your questions, give or take. And if you have any additional questions, we'll see if we have time at the end as well. A little back there. So let's get going. And I'm going to turn it over to Steve. I'm going to, hold on a second, I'm going to turn this into a presentation mode for you, Steve, and then you can get going. Okay, so I'm going to do the project update for the OpusDec orchestration program which includes the HEAT project. I'm Steve Baker. I'm the orchestration PTL for the Icehouse Cycle. So we just go to the first slide. There's an overview of the Havana Release code activity. We had 42 blueprints implemented, almost 300 bugs fixed, over 800 code commits from 64 people, and almost 4,000 code reviews. So this gives a pretty good indication of how active this project has been in the Havana Cycle, even though it's quite a young project. Now if you just go to the next slide, there's a vendor breakdown of contributions from the Grizzly Cycle. Now as you can see, it's pretty obvious here that HEAT was started out as a project sponsored by Red Hat. The intention was always that it wouldn't be this way forever. So if we go to the next slide, we have the same chart. This is sourced from Stacolotix by the way. So this is the Havana chart. And as we can see, the contributions from Red Hat has gone down to 61%. We've got some major contributions from the likes of IBM, Space and HP, and also quite a long tail of contributions from many other companies. And while the contribution from Red Hat in relative terms has gone down significantly, in absolute terms it's actually gone up. So this is an indication that the project is growing and it also has quite a large diversity of contributors from a number of vendors. So I think this indicates that the project is from an open source culture point of view is in a really healthy position. So now I'm going to move on to cover some of the features that were in the Havana release. First up, we've got concurrent resource operations. Previously when you launched a stack, each resource was created serially. And that was even when there was no dependency between any given resource. So for some stacks, it would take a long time to launch a stack. But now whenever there's no direct dependency between any given resource, the resources are created in parallel. This can have a huge performance improvement for some stacks. Next we've got the provider and environment abstractions. So this gives you a way of changing the way a stack behaves without actually modifying the template. Instead you provide a separate environment which specifies different behaviors for a given resource type. And this environment can be specified at the cloud operator level or at the individual user level. So a cloud operator might want to provide some different resource types that provide slightly different behaviors from the standard resource types. Or they might want to override the default behavior of a core resource type to integrate some particular behavior of their cloud. And similarly when any user launches a stack, they can also provide an optional environment which can give those same overrides to different resource types. Now when you override a resource type or crash a new resource type, you're actually authoring it as a stack template itself. So for example you could override the compute resource type. As a stack, give the resource type some different defaults, add some behaviors. So it's quite a powerful abstraction. But only for solving certain problems. Next we have full salometer integration for metrics monitoring and alarms. So previously metrics and alarms were implemented as a lightweight compatibility layer for the CloudWatch API. That still exists. But now we have a full integration with a salometer. So any stacks which need to trigger actions on alarms such as auto scaling can now be driven from metrics that salometer collects. It's possible to push custom metrics from instances to salometer. And it's possible to configure a heat stack to perform actions based on salometer alarms. We have some new actions stacks suspend and resume. So now an entire stack can be suspended. And for the resources which support it, whenever a stack is suspended those resources will go into a suspended state. Whatever that means for that particular resource type. So for example an over server will go into a suspended state. And when the stack is resumed then the server will be resumed as well. Finally we have a standalone mode. It's actually possible to deploy heat outside of a cloud but still be able to launch stacks on any single arbitrary cloud. This has a couple of obvious use cases. One is just for development for heat developers. You don't have to have your own full cloud locally just to develop heat. But another obvious case is if you want to have your own local heat deployment which deploys to an open stack that doesn't have heat support yet or just any external open stack you can do that with standalone mode. So if we go to the next slide we've got the usual collection of new resources in each new release. We've got a sender volume attachment. There's some new resources for Neutron including load balancing, firewall and VPN as a service. There's a new native nova server resource which is great because now it's possible to write templates without having to specify AWS EC2 instance resources whenever you want to nova compute. The nova server resource also exposes anything in the any API option that the nova create API call supports. So there's a full transparency of all nova features. And finally we've got a collection of racks based cloud specific resources that have been put in the contrib directory of the heat repository. It's expected over time that these implementations will get smaller and smaller as racks based converges on a more standard open stack architecture. So if we go to the next slide we've got some preview features that landed in Havana but will continue to mature throughout the ice house cycle. First up we've got initial support for the hot template language. So this is a new native template format which is related to but is starting to diverge from the cloud formation template format that we started off with. It's based on YAML. We have the flexibility to clean things up, make a template format that's more author friendly and it lets us add any new features that we want without being held back by the cloud formation format. So we never set any expectations that it would be complete in Havana but we've made some progress in Havana and we have a hope that by the ice house release the hot template format will be ready for offering production templates. So also we have initial integration with Keystone trusts. Now currently a heat stack needs to make API calls which are not triggered by the user throughout the life cycle of the stack and obviously each API call needs a valid token to call it with. So currently the only way of achieving this is by actually storing the user credentials with the launch stack and using those credentials to create a token whenever we need one. This is not ideal but with Keystone trusts instead of storing the credentials we can store a trust ID and whenever we need a token to do an authenticated operation we can turn that trust into a token and use that for the operation. So this is part of a range of changes we'll be making which require new Keystone features and that work will continue in the ice house cycle. So I'm going to start talking about some planned features for the ice house release. Some of these have already landed but as with any open source development cycle we will attempt to land these features but it depends entirely on how the development and review cycle goes. So if there's a particular feature that's important to you then we would strongly encourage you to get involved in this development process. So as usual in every release we have a collection of new resources. Currently what's been in development and landed so far is a Trove resource a service, a Savana resource for the data processing and some new neutron resources such as a network gateway. So next slide. The order scaling API. Currently our order scaling implementation just implements the cloud formation order scaling within HEAT itself. What we plan to do is create a brand new API which can be used with or without HEAT stacks. So if you're not currently deploying your applications as HEAT stacks but you still want to incorporate order scaling into your architecture then it will be possible to do that. You'll still have to deploy HEAT but you have no obligation to directly launch HEAT stacks. Now the order scaling API will be flexible enough to scale more than just single compute resources. You can scale compute plus associated resources around it and there does need to be some way of representing those resources and how they relate to each other and how they expose their parameters to be integrated with your application. So the obvious way to represent that little collection of resources that get scaled is with a HEAT template snippet. So when you interact with the order scaling API you will be specifying HEAT templates in some way. Now once this order scaling API exists there will be some HEAT resources created that consume that API just like any other HEAT resource that consumes any other API. And from that point it will be possible to author templates that pretty much make no reference whatsoever to cloud formation. The order scaling resources have been probably the most important resources that have yet to have an open stack native implementation. So this is quite a large milestone for us to be able to author completely native implementations that include order scaling. So going on to the next slide. There's a hot software configuration. This is also quite a large change. These blueprints have a number of goals. They include being able to integrate any configuration management tool that's commonly used to configure software on running compute resources. And by configuration tool I mean tools like Puppet and Chef and SaltStack. To add a new configuration tool it just requires a small Hock to be authored and that Hock needs to be delivered to a running instance somehow. Ideally in a built image, in a golden image rather than deploying it at boot time. But you could deploy it at boot time as well. Other aims of hot software configuration is to provide a composability mechanism so that these configuration scripts can remain in their own configuration files or be invoked via URL. So this will quite radically change the way that templates are authored and the way that you write templates which do complex software configuration. Next slide. Management API. There will be a new API that is only used by cloud operators. It will give them some operations that an operator would need to manage a heat deployment so that all running stacks can be queried and viewed and potentially manipulated. Once the API exists we'll incrementally add to it over time based on whatever operators need. So next slide. Got the heat multi-engine scale out. So our API has always scaled out because it's just an API process. It's stateless so we just use normal load balancing techniques to do that. But we couldn't do that with the heat engine because the heat engine does maintain some state during long running operations such as stack create or stack update or stack delete. So until now we've been limited to having a single heat engine process for a given stack deployment. The current implementation requires a locking based solution. That may change in the future but it's lock based for now. The default lock is just based on a database table but there is a plugin system so that an operator could choose to deploy a different distributed lock such as ZooKeeper. But it does scale out of a heat engine now. So next slide. So stack convergence and failed update recovery. Now there are cases where the real world resources for a stack can get out of sync with what heat believes the state should be. This can happen due to transient failures or due to manual intervention of resources that heat wasn't aware of. And when this happens it would be nice to be able to bring those real world resources back into line with what heat believes it should be. And this is what convergence is all about. So convergence will introspect the real world resources, compare them to what heat thinks they should be and then come up with an action plan to run the appropriate API operations to bring that state back in line. And in a way related to this is the failed update recovery. Currently when you do a stack update and that update fails then the stack is stuck in a failed update state and you really have no choice but to create a new stack. But updates could fail for many reasons. There could be a template authoring error or there could be a transient cloud error. So really you need to be able to run update as many times as you want to until you get it into a state that you're happy with. So that will be possible now. So next slide, stack, abandon and adopt. These are two related features that also have uses by themselves. So stack abandon, it's a light stack delete but it does not delete the underlying real world cloud resources. It leaves them in a created state. So one use case where you would want to use it abandoned by itself is if you wanted to use heat to deploy a resource, to deploy a collection of resources but you don't want to use heat to manage the resource lifecycle as a whole. So you could create that stack and then abandon it immediately and those resources still exist but on their own. Now when you abandon a stack it also gives you a packet of information that describes the resources in that stack. And you can actually use that packet of information to pass it to the adopt call. So adopt will take that packet and it will create a stack but it won't create any resources. It will just attach to the resources that are specified in that packet of data. So as you can see abandon and adopt can be used together. You could abandon a stack on one heat instance such as a local standalone and then adopt it on another heat deployment such as the heat that comes with the cloud that uses that stack that uses those resources. Actually adopt can also be used by itself. If you manually author that packet of information you could in some circumstances adopt the resources that you specify. So resources that are created manually could then be adopted by a heat stack and from that point on the heat will manage the lifecycle of those resources. So next slide. It will be possible to launch all stacks without needing admin privileges. So currently some stacks, the resources for some stacks also create users to allow authenticated API operations to occur within the scope of that resource. And to create a user currently you need to launch the stack with an admin privileges user. And this has been a real problem for us for a long time now and we have a plan now for solving this issue. So it will now be possible to launch any heat stack with a conventional user. Then finally V2 API. We have a plan to have a new V2 API. It gives us a chance to clean up a few things such as having the tenant ID in the URL and for not requiring the stack ID and the stack name to be in the path for stack operations. It also gives us a chance to have a better handling of request scoping and policy for authorizing stack operations. So that's all I have at the moment. I'd be more than happy to answer any questions now. I see a question here. Question is we have noticed there is limited support for sentos and heat. Is there a plan to add more support in Icehouse? So I guess we need to find out what we mean by support for sentos. Is that for deploying heat on sentos or for using sentos as a guest? To answer your question with the announcement that Red Hat will be participating in the sentos community, we would expect that support for sentos will be very good in the near future, both as a guest and for installing heat on sentos via the RDO distribution. But as long as sentos has clouded in it, it should integrate well with the new software configuration that we're planning for this cycle. Any additional questions for Steve? The line is open if you want to ask or you can put them in chat. Okay, so just in the last couple of days there's been some interest in using windows as a heat guest. There are a couple of projects out there that are pertinent. There's CloudBase in it, which is a port of cloud in it that also works on windows. There's the Murano project, which among other things is an application catalogue for deploying complex windows applications that uses heat under the hood. So there is some activity there right now. I expect the way that a hot software configuration would integrate with windows is that the windows image would need cloud base in it. And it would need some kind of software configuration hook, which lets you specify configuration operations in PowerShell or whatever other windows configuration options there are. So I'm not personally working on that, but there is interest in activity in that area at the moment. Great. And if anyone else has additional questions, you can put them in the chat box or I think there's Meeting Burner sends an email afterwards. You can send them back to me as well and I'll get them back to Steve. Great. Well thank you Steve. I appreciate it. Thank you. I'm going to turn it over. Yes, good job. I'm going to turn it over to David. David, let me get your presentation up. Okay. So David, when you're ready. Okay. I'll set. Hello. My name is David Lyle. I'm the Verizon program PTO and this is the OpenStack dashboard project update. Slide please. So we completed the manner release. We made a lot of progress. We had 41 completed blueprints with a total of 406 commits and 6282 bugs. Next slide please. So that comes from a community of 104 contributors which I'm really proud of. It's up from 58 in Grizzly so we've got quite a bit of community growth and those commits come from 34 different companies as well as some unaffiliated contributors. And there's a lot. It's not just one company dominating the commit to any longer. Red Hat, HP, NEC, Mirantis, United Stack, MetaCloud, today Nebula is still in there as well. So very diverse contribution base which again to me is a great indication of the health of the community. Again, and also 217 reviews. A lot of reviews by a lot of various reviewers. And so that certainly is helping our project along. Next slide. So what have we accomplished in Havana? We got our first pass at heat integration as heat came out of incubation and graduated from incubation in the cycle before Havana. Then we added heat integration in Havana and that included stack creation. You can actually import the stacks definition from the URL. Go ahead and launch that stack and then view the topology as it builds as well as when it's up. And then be able to do some resource inspection once that stack is up and created. Solometer also came out of incubation in the cycle before Havana. So we graduated. So in Havana Cycle we added the preliminary support for Solometer. So the initial support is just in the admin panel and it's just created doing cross-cloud querying for certain pieces of utilization. The plan in the future is to get a more full-featured integration with Solometer. But this was the first pass at it. We also added Keystone B3 API support. So this is a fairly large effort to add domains, groups, and roll the cred for those to our identity model management as well as integrating that with existing projects and users. So now you can have a multi-domain setup, log into an individual domain, create additional domains, assign roles to end groups, assign users to groups, roles to those groups, et cetera. So it's a fairly full-featured Keystone B3 API implementation in Horizon. We improved Neutron feature support. The key things there were the VPN as a service and firewall as a service support. There are fairly rich panels in Horizon now to support those features in Neutron as well as we moved to a better parity for security group and quota between Neutron and Nova Network. So now you can actually, before for that, you couldn't manage security groups or quotas in Neutron from Horizon and now you can. And the other feature that was added is an interactive network topology. So in the previous release we had a static network topology which is truly beneficial. It's much better to be able to visualize your network topology from a visualization rather than from trying to sort these, sort it out from tables. But in the Vano release we added the ability to not only look at it but interact with it here. So you can go ahead and launch instances, create a new network, create a router at it and see that getting added here and see how that relationship looks. So other highlights from the Vano release. The big one as far as I'm concerned is multi-region support. So before Vano we only, Horizon could only see one region. And so for larger installations, multi-region support is absolutely necessary so now we can manage services across diverse regions. This helps Horizon become more than a token UI. It's intended to be a true UI to manage OpenStack. And then trying to keep up with and have a feature support. So we added a bunch of features there, editable default quotas, past words, availability zone support. So when you're launching instance you can specify where the availability zones are going to go or which availability zone that's going to be in and see what availability zone your particular instance is running in. Resizing instances, boot for volumes, improve boot for volume support and project for your support. And the last item that we added was trove integration. So although trove was still an incubation and at the end of the Vano release, they had the UI far enough along that we felt comfortable including it in the Havana release. We label it as experimental because it is not part of the official release for Havana. So there was the ability to turn it on and turn it off but with the integration that was provided in Havana you could create here a managed database as well as the backups of those databases. So what are our priorities for IFFAS? We have quite a list of items we'd like to get done. These are just the top ones. The first is role-based access control. So in Havana we added the start to this. We added a policy engine into Horizon to allow an implement it for the keystone elements inside of Horizon. Basically the idea here as with all the other policy engines in OpenStack is to be able to have finer grain access control. We want to be able to provide and enable certain actions based on what roles you have. So the idea now is, now that we have the engine, now that we have the keystone support, we'd like to enhance that nice house. We'd like to extend it throughout Horizon. And one of the benefits of that would be not only finer grain access control but hopefully condensing the project dashboard and the admin dashboard and just use role-based access control to define what the user sees. This would greatly simplify the code base, reduce some duplication, reduce some overhead, that sort of thing. This is a big item we'd like to kick out. Another thing we're looking to do is create a more extensible user interface layout. So currently we have the two tab dashboards side by side. It doesn't really give us a whole lot of room for expansion. One of the fundamental guiding principles for the Horizon project is we want to support extensibility. You can't expect what the upstream Horizon project, delivers, is going to be the final user interface for anybody pulling it down. They're going to want to add features to it. They're going to want to add new dashboards to it. We don't really have the space for that right now unless you have maybe a three-lettered name dashboard and one of them. So we're going to change the layout. It's going to be more of an accordion-style layout. Still going to be on the left-hand side of the screen. And we're going to move around the project selection in some context information. It should be more of a versatile user interface and certainly more extensible. Once we knock those first two items out, we're going to start getting some information architecture changes. So basically what that is is we're going to organize the data differently. The identity will be pulled out into its own dashboard. What is currently identity will be pulled out in its own dashboard. And again, once we have the role-based access control, a lot of what's in the project in Admin will get condensed. And there's more information on that on the OpenStack UX site. But basically we want to make it more sensible. We want to include better speedometer integration. So right now we have limited-kilometer support in the Admin panel, and it's by itself. It's its own panel inside the Admin dashboard. What we'd like to do is not only have that type of feature, but also be able to sprinkle graphs of spark lines throughout the other panels. So use the information from Solander to better inform the user. Better richer client interaction. So we've adopted AngularJS as a JavaScript platform inside Horizon. The steps are just trying to make some of the workflows a little more friendly and better validation between client and server side. One that's already been knocked out is Configure World dashboard loading. So the idea here is right now you have to do a bunch of config file changes to load dashboards or to change what dashboards load, what order they load in. What we'd like to be able to do is just drop, say, Company A pulls down Horizon. They want to add a dashboard that should be able to, or remove a dashboard that should be able to do that easily so they can either drop the dashboard in place, set a config file, and have it read in or not. Another thing we'd like to do is split Horizon from the OpenStack dashboard. There's a little disparity here, but Horizon is essentially the toolkit library that OpenStack dashboard uses. And so we'd like to, as it's a library and as other groups like to build off this library, we'd like to simplify things by splitting out the Horizon section of it, the actual toolkit side of it, and leave what's in OpenStack dashboard as the actual application that manages this OpenStack. And we think this will simplify, they'll simplify build and packaging, but it will also simplify flight for people who are trying to, for projects in incubation where they don't want to maybe pull down the whole entire OpenStack dashboard, then they just want to expand Horizon to implement user interface for new features. And last, we'd, on here, we'd like to have better tempest integration. So currently Horizon I believe has about one test in tempest, and it's basically just a sanity check. As Horizon utilizes most parts of the stack, we feel like it would be a great place to at least test the Python, let's say Python Keystone, Python sender client, all the clients to validate that there's no, not that no changes break backwards compatibility. And since we exercise all those fairly extensively, you know, this is not only for President Rayson of Horizon working, but also we just feel like it's a good integration point to make sure those changes don't happen that are detrimental to the stack. And that is all I have. Is there any questions? I see one over in the chat, I believe. So Keystone LDAP. Yeah, so does Keystone support LDAP integration? Yes, it does in the state now. They're currently working on better support for LDAP with federated backends. So if it were ready, we would like to have it included in the Ithous release for Horizon. Most likely this is probably going to slip to, at least as Horizon goes until early Juneau timeframe just so that that can stabilize. Other questions for David? You're going to have to air for too long, but feel free to ask your chat if there are any questions or the line is open as well. Going, going. Okay, well, that looks like that's it. Well, that's good, which will end a little bit early today. Thank you for joining everyone. Please check back for postings of this webinar and our speakers' presentations on OpenStack on the Foundation's YouTube channel, as well as our blog. I'll hopefully post those Friday or Monday. And thanks again to our speakers. Thank you, Steve, and thank you, David. And if there are no more questions, yes, appreciate your time. This concludes our webinar then. Take care. Bye-bye. Thanks so much. Thank you.


Nothing says summer like a hot day at the ballpark watching a game and this weekend there's plenty of chances as Bemidji's northern heat travel baseball teams host tournaments this weekend across four different age groups. We'll start with the 13-year-olds as the northern heat host Winnipeg long traveled weekend for them and Bemidji was not kind to their guests up nine to nothing in the second bases loaded. Bases loaded. Isaac Sievert's lines went to left center and that clears the bases in two batters later. Christian Haggoli smacks one through the 3-4 hole that scores Sievert's up 16 to nothing in the third now Haggoli at bat again and he delivers this time to the left side of the infield. Sievert scores again Bemidji cruises in this one 18 to 3 in four innings. If you enjoyed this segment of Lakeland news please consider making a tax deductible contribution to Lakeland PBS.


What's going on everyone Sam here with late in sports cars and I'm ripping five 2022 Panini father's day packs for John H. So good luck John see what we got in here for you buddy hopefully some nice stuff got rookie Spencer Torkelson there you go take a torque rookie then we've got Drake London rookie for USC got one Soto Joey Logano on the insert yeah hi as a nugget still funny to see that Julio Rodriguez the shorter printed rookie there nice I'm mattingly nice Donnie baseball and Patrick Mahomes Texas Tech very nice and last pack here good luck got John Morant and Josh Giddy bass rookie there you go nice one there that'll do it for your packs there John congrats on the cards we'll get them right out to you thanks as always buddy