Lakeland Public Television presents Currents with host Ray Gildow, sponsored by Niswa Tax Service, offering tax preparation for individuals and businesses across from the City Hall in Niswa and on the web at NiswaTax.com. Hello, everyone, and welcome to Lakeland Currents as we begin our tenth season. Hard to believe that it's gone that fast. And through all these ten years, our underwriter has stayed with us, Niswa Tax. We're very appreciative of that, and you probably have noticed if you've been watching Lakeland Currents that we actually shoot this show now both here in the Brainerd Studio, where I do it, and we do it also in the Bemidji Studio, where Beth Wesley is the host of Lakeland Currents there. And we are in the new night, Friday evenings, which is probably a little better night for us because it ties into other public relations programs, like Almanac. So we're happy to have a new time and a new day of the week, and we will be bumped from time to time by hockey. Hockey's taken over, you know, it's pretty important, I guess. So once in a while our programs will be delayed because of the hockey programs, but we're happy to be here, happy to be joining you again for our tenth season, and I'm very happy to introduce to you tonight the new superintendent of schools, Lane Larson in Brainerd. She comes from the Bagley area, so if some of our viewers are from Bagley, you know who Lane is already, but happy to have her here and talk about some of the things that are happening in the Brainerd School District. She's brand new, so she's learning the system and learning people, but welcome to the show, Lane. It's good to have you here. Thank you so much. It's an honor to be here. Tell us a little bit about your background, because I know you've spent a lot of time in Bagley, but what is your background? As you started, I began my career in Bagley, and I was a family and consumer science teacher for 16 years, and for about the next 10 years, I was in administration. I was an elementary principal, assistant principal, assistant superintendent, and all the way to superintendent, and so I was in Bagley for 26 years, and then my husband and I moved to Thief River Falls, where I was superintendent of schools there for seven years, and this past winter, a fortunate opportunity came about with the Brainerd School District, and I threw my hat in the ring, and here I am, so it's an honor to be here. I remember reading in the Chronicle of Higher Education back in the 90s that we're going to be seeing huge shortages of leadership, not only from higher ed colleges, but also from K through 12, and I think that's come to fruition, not only are we seeing leadership shortages, we're starting to see shortages in teachers, and to me there's nothing more important in America than education, and it's a shame to be seeing this happening, and one of the statistics I read last night, I did do a little preparation for this, the average superintendent in urban America is only lasting about three years and six or seven months on the job. That's an incredible turn over. That's an expensive process, you're losing a lot of experience in some of these situations, and it's creating turmoil in our educational system, don't you think, when we have to have that kind of turn over at the top? Yeah, I agree with that. I agree with you in the fact that I think that being an educator is the most wonderful and the most important job that we can ever have, and so it alarms me too, when we have such a teacher shortage, not only in Minnesota, but throughout the entire nation, and it goes all the way through to the administrative positions as well, and it is, because consistent leadership we know is something that's really important as far as building on the strengths of a district, and to continue to move forward and do what we need to do on behalf of our kids so that they can be competitive in a really global society right now. What do you think are some of the global reasons why there's such a big turn over at the top? Especially in urban America? Yeah, I think that there's just a number of challenges in education today, and I think that the jobs throughout education have become very difficult. There's a lot of diverse needs. I know even yesterday at Rotary, they were talking about areas like with mental health needs and such that we have. We have a lot more expectations. Our kids need to be very competitive in a global society like I talked about so that technology is a huge piece. I think the financial reimbursement to districts and living within the budgets that we're given make it extremely difficult. But I still have to come back to the fact that I think it's the most wonderful job that you could ever have. There's a lot of people out there, a lot of great teachers and administrators that work in every single way to make a difference in the lives of our kids, and I can't think of anything more rewarding to be doing than to be a superintendent of schools right now. I had your predecessor Bob Gross on last year and we talked about how often Americans are compared to European and Asian schools and how the test scores always don't look very good. And one of the things that Bob pointed out was that in many of these countries they teach to a certain level of standard and there are certain students who if they're not capable they don't even get put in the mix. They're not part of the group so to speak whereas we take care of all of our kids. We put them all into the groups whether they have special needs, special education issues and so that tends to skew those scores and makes people think we're not doing a very good job but one of the things Bob mentioned was the same thing you're mentioning is we have wonderful teachers who are so dedicated that we sometimes overlook that and it's sort of sad that they're getting lost in the shuffle of financial funding issues as we know the funding for education has been decreasing for really about 25 years K through 12 and higher ed and about 80 percent of our cost are people and so we have to have the people and when you walk into a district like Brainerd what are the things that you looked at here that you particularly liked? Well first of all I've had a great opportunity since I've been here in July to meet with a number of community citizens and to meet with my administrative team and many teachers and there's some common messages that I hear consistently about the Brainerd district. First of all that the teachers and the employees overall throughout the district are just top notch. They're top of the line and it's just so exciting. A second thing that is a common message throughout is that we have a very strong tradition of excellence in our school district just last week I was identified by the commissioner of education that three of our elementary schools are reward schools. Three of the top elementaries in the state of Minnesota and so we have that strong tradition of excellence that we get to build on. And the third thing that I hear overwhelmingly is community support. And I'm learning that as I meet people and as I experience being the superintendent here that people really love their school. They're very proud to be Brainerd warriors and really want to do everything they can to support and provide the best quality education that we possibly can for our kids because they are our future. Now when you said the elementary schools were reward schools what does that mean? Well there's different rankings based on the test scores and this only qualifies for our school district for schools that receive title funding and the reward schools are the top schools that are closing the achievement gaps within whether it's within ethnicity or if it's within special education, if it's within gender but one of the things that we noticed is that within these three schools that we were making huge strides in closing the achievement gap so that all our children are successful and they can live their purpose and so that we can help to give them what they need for their future. Now did you walk into a strategic plan that's in place currently for the district? Is there a strategic plan that the superintendents have been working through regarding building and that sort of thing? There's a number, as far as a systemic strategic plan that's one of the things that we're working on right now but one thing that I visited with my board and administrative team is the fact that there really are six areas that we really want to focus on and work with strategic planning. The first one I think has become very visible to the people in the Brainerd Lakes area and that is that the past year they've been working on a facility plan and they've worked through the Cunningham group and just last week we had a final product for what they after they've done their assessments what the needs are for the district and so now we get to move forward and condense it a little bit. We get to meet with more staff, meet with our students meet with the board again to talk about what are the priorities now, the must do items the should do items and so on and to communicate that with the public and to hear from the public as to what the needs are. So facilities certainly that have been out in the forefront of our district, but there's a number of other areas. Certainly we want to always close the achievement gap and to have high academic accountability and so that's something that we've been working on. The financial accountability to our constituents we owe that to our generous taxpayer to ensure that we're living within our budget. We always want to work on the communication within our district both internally and externally with our community to build on that and technology is a huge piece as well that we want to be looking at how can we put the technological needs of our students and our staff in their hands so that they can be competitive with a world that's going very very technological. I'm a graduate. I went through the high school here and I know a lot of the elementary buildings are really old. In fact I think if you look across Minnesota a lot of our elementary schools were built in the 30's and 40's and haven't really been upgraded much since that. And it's getting more and more difficult to get bonding bills passed to upgrade facilities but is the district growing with students or is it fairly stable or where are you at with enrollments? Our demographic study as far as the district certainly shows an increase in students and we are already this fall and the demographic study certainly shows that we're going to be continuing to grow as we move forward and so that is one of the issues that as we're faced with facility changes within the district that we will want to take into consideration. Also the age of the buildings, how technologically advanced it is and how we're providing the needed education the space needs and the educational adequacy needs that our students need to be competitive even as we move into the 22nd century you know being almost 17 years into this one that as we move forward what are our students going to need and our teachers going to need to make that positive difference. So are you seeing the greater need for facilities at the elementary level or at the high school level or middle school or what are your early studies revealing? The early studies certainly are that there need to be some updates at the high school and also in our elementary schools. Forest View Middle School for those who have had the opportunity to tour that building that was just finished I believe it was nine years ago just a lovely facility that is meeting our needs right now and the elementary schools they're having wonderful results but we really are busting at the seams in some of those areas so to look at what are the needs at that level and then the high school of course will we do in that area so we're excited to bring to look at the plan that Cunningham has and also to bring it forward and get more support ideas and from our staff and our students and bring that forward to the board so we're very excited about that opportunity one thing I should say when I was in Thief River Falls I did have an opportunity to work on a building bond project and to do a little over 70 million in project to the facilities there all staying on site that we had at that time and so I'm looking forward to going through this process again. Looking forward to it?  I really am. I guess so but I really am looking forward to it. I know I've done a number of shows with the University of Minnesota extension and we've looked at the population projections for the grander area and you're right I mean there's a lot of projections for growth some of it will be with senior citizens obviously but it's also going to have a lot of new young people coming to the community you really have to plan for that and that takes some strategic thinking and it also takes some courage by the board and the superintendent and the local community to make those investments. One of the sad areas that we have to deal with that we never dealt with 25 years ago was this whole issue of security. We are spending I know I was at the college and we started spending a lot of time and resources to make sure that our students were secure. Do you have a security plan that you use in the district for all your buildings and how do you approach that here in Braydard? Well as we move forward with the facility projects that we'll be looking at one of the major probably the major goal as we move forward will be the safety and security of our buildings. Now some of our newer buildings some of the buildings have extensive safety and security systems in the district right now and some of them aren't as comprehensive as other ones and so we will be looking at some kind of a uniform safety and security measure because truly the number one goal of the Board of Education is to ensure the safety and security of all of our learners and our staff and to provide that environment that is conducive to learning and that kids can come and really do their best work and the Board is committed to that. I don't think people who don't work in these environments understand how complicated that is. We're public facilities we're trying to be welcoming to people to come into our buildings and yet we have to be very cautious and careful sadly because of what's happened around our country but I know that's a very big challenge. This school district has a really good reputation for preparing kids for education after high school I don't mean that they all have to go to a four year college because they certainly don't but I think that's one of the real rich traditions of the Brainerd School District that you've walked into and I think you'll appreciate that. What's your management style? Do you try to get out into all the buildings on a regular basis or how do you operate? Is it as visible as I can within the buildings? At the activities outside of school and in the community? To me that's part of the key of this whole piece. You talked about one topic that I think is so neat. Just the other night I was at a district advisory committee meeting and I was saying that I couldn't believe how comprehensive it was the number of people from the community that were engaged in what we were providing in our district and we have new legislation through Minnesota within the last couple of years called World's Best Workforce. They talked about just exactly what you're talking about first of all what are the standards how are we meeting the standards in our districts and taking those standards to ensure that our students have the work-ready skills that they need to get good jobs when they leave Brainerd High School. We have the CLC or another two-year college that they have the skills to be successful there and if they move on to a four-year graduate programs beyond that are we providing the opportunities for them to be successful. That's one of the areas that I get very excited about is that I believe that our staff takes it very seriously, our board takes it very seriously and our curriculum and assessment committee takes it very seriously and we're ensuring what our kids need to be successful and to live their purpose in the future. That's I think very, very critical piece. And again the community support I just I couldn't get over how many community members cared and showed up for that meeting. There's a lot of manufacturing companies in the area or just businesses that are welcoming to students to come in and sort of shadow what's going on in the places so that's you're absolutely right that's really cool. The businesses really that collaboration now between the school district, the college, the business community and our communities is really what it's going to take to move us forward. We have to work together in a collaborative manner and I'm thrilled about that. Excited to be doing that. What is your view of the funding that's going to be going on in the next biennium? Do you have a sense? I mean I know that superintendents work together in the state on these issues but I suppose without knowing who's going to win in the elections it's hard to get a real handle on what you think might be happening for funding for school. But do you get a good sense that there's a lot of support to continue to fund education in Minnesota in the ways that we needed to fund it to keep it going in a healthy manner? Well you talked about it that we really have seen a huge decrease in funding. You talked about for 25 years but 10 for sure even more than that. School districts were just behind where we need to be as far as growth and so on. I do believe that there's a lot of support that people value education highly. Clearly I believe it's about a third of the state budget goes into education itself. And so it's clear that certainly our taxpayers value education. It's just there's X amount of dollars to go around and how do you distribute those dollars with everything with all the needs that we have within the state and within the nation. And so we keep plugging away, we keep working at it. Our legislators, I believe the legislators here are very good as far as being able to visit with them and to share our concerns and they hear us and I appreciate that when they go to the legislature. A superintendent's job can be pretty lonely at the top. I have a number of friends who have been superintendents. Do you build in a support network for yourself to have that kind of support when you need it because you do have periods that are really challenging as you know because you've been a superintendent a couple of school districts. How do you manage that? I agree with you that it is quite lonely a lot of times and there's lots of reasons for that but I'm really fortunate in that I have an awesome husband and I have two just incredible daughters that have a strong family system and great friends that are great support systems that really love me for being lean and accept that. Professionally I have a lot of colleagues as well. It's a different relationship but I feel very supported and primarily from my family and my friends but yeah, it can be hard. Yeah, you bet it can be hard. I don't think most people think about that but you're torn in the directions of the board and the elementary schools and athletics and all those things. It's hard to have all those hats and wear them and be at all those different places but do you have a strong interest in athletics? Very much so and so that's part of what we do is we just attend a lot of things and we meet a lot of people that way which is fun and the other thing is that there's a lot of athletes in the area that are in the same kind of in the same boat and we become very good friends and like if it's some kind of a work related issue you can give just like a scenario or something kind of brainstorm a little bit and that's really a helpful support system as well. The other thing that I really believe that is strong about Brainerd is we have a really strong and healthy board of education and administrative team that I feel really comfortable with and I believe that we are just a really strong team that want to move forward on behalf of our children and to and all of our learners in our district and for our whole community to continue to be a state leader. You and I were both at the same rotary meeting yesterday where we heard about one in four Crowley County members has mental health issues. That was absolutely amazing to me. I didn't really realize that's significant which also means you have mental health issues with students. Do you have enough help in the school districts to help deal with those kinds of issues? I know having been a classroom teacher myself sometimes when you have issues with two or three students, you just don't have time in the classroom to deal with them and I know over the past X number of years there's been many cuts made to counselors. How is Brainerd faring in that area? I agree with that. That really is an area that I believe we need to work on is providing that support to children and to their families and to our staff because there are a higher much greater need even when I started being a superintendent 11 years ago. There's a greater need than there used to be. I think providing more counselors more homeschool interventionists mental health support throughout our communities is really an important piece and I believe that the members from ESUNCHI yesterday when they brought it up that it really was kind of a rude awakening to everyone about the needs that we have within not only our state but within our nation and our world. I don't know if people understand how that can I don't mean this the wrong way but it can pull down a teacher's time with the rest of the kids because it takes so much of their time if they don't have that technical support for those types of students in their classrooms it makes it very very challenging for teachers and boy my hat's off to the teachers in the classroom because that is such a hard job. How are you dealing with many bilingual issues in Brainerd? We have some and then we have the E-R-E-L-L support in department and so we have some of that and throughout my career in education we've dealt with that quite a lot and we're seeing more of that. We just have a couple minutes left here but you talked about technology what are some of the new things that are happening in the Brainerd district with technology? I know you have a website but could you just briefly touch on some of the things we're doing? Some really exciting things are happening within our technology department right now that is really moving forward. We are just going to be showcasing our new website probably in October we're getting a new communications director here shortly within the district we had all new laptop computers put in the hands of our teachers just within this past year we have many of our classrooms have iPads computers, we have carts for computers we're really updating the computer technology within our district what we need to make sure and to ensure is the fact that we have the staff development to help support our teachers as we move in that direction because a lot of our, even our curriculums and such are coming out on technology to save money regarding books and such and so we need to make sure that we have the professional development and the tools available to our teachers so they can do the work that they want to do with technology. Will the students have any opportunities for online learning or that will be something you're doing? We already have some of it and we'll continue to do that as we move forward and so one of the things that I know is a big initiative in school districts right now is one-to-one technology as far as being able to save money with textbooks and those kinds of things and I would guess that we'll be moving in that direction out of time probably in slow increments but it certainly is something that our kids will have as they move into college and move forward so So were your communications person will that be the primary responsibility for them is to work with the website and those issues on the web? Yep and doing all the things that we want you know one of the things that we want to do is brainer like I said has such a awesome history of excellence and we need to tell the people about all the wonderful things that are going on all the wonderful initiatives I know I've been throughout the schools just again this last week and every day I just get amazed by all the great things that are happening we had forest day last week and went through the school forest and all the education that's taking place there we had Rachel's challenge this week and what a wonderful opportunity that was for our kids to do some work with anti-bullying just every time I turn around I'm learning about the balloon program and how the education foundation has been so support of it helping us and so we really want to showcase we're doing great work here in Brainerd and we're very proud of it and we want people to know how great it is Well we're out of time but you're not out of time I want to wish you the very best in your job and you've got a great attitude and I think you're going to be a real plus for the Brainerd community and thank you You've been watching Lakeland Currents where we're talking about what you're talking about I'm Marie Gildow, so long until next time


Live from Las Vegas, it's theCUBE. Covering Edge 2016, brought to you by IBM. Now, here are your hoes, Dave Vellante and Stu Miniman. We're back, this is theCUBE, the worldwide leader in live tech coverage. Check out ibmgo.com. Donna Dillenberger is here, she's IBM fellow. Cube alum, Donna, fresh off the keynote. Great to see you again, thanks for coming on. Hi Dave, hi Stu. So really enjoyed your topic this year. We were just talking off camera, we were talking last year about sort of bringing analytics and transaction systems together to solve some of the big challenges in the world. But today you talked about open source and you talked a lot about blockchain. I want to sort of share that with our audience. Well, if I want to start with, you had 10, I think 10, at least 10 anyway, open source projects that IBM is heavily involved in. Talk about the importance of open source to IBM. Yeah, I mean, so, you know, of those 10 projects, some of them are, you know, no JS, open container, Eclipse, open power, open mainframe. Spark. Spark. Linux, open stack. So, you know, everyone has their own expertise, every company develops a core set of competencies. So for example, IBM, we're really good at like scalability and security. But when we partner that with other people who donate their expertise in analytics, their expertise in Docker containers, I mean, that's like the best date, right?  There's nothing that sharpens minds like other minds, right? And to have other people with such great skills, right? It not only brings IBM people to a better level of technical expertise, but just that collaboration, it really is, we're better together than separate. Okay, and then you also talked about, well, let's talk about blockchain. You know, it started with Bitcoin. Yeah. And I remember seeing Warren Buffett on CNBC some day talking about how Bitcoin would fail. And I remember thinking, well, Bitcoin is more than currency. You know, maybe the currency will fail, but the technology implications, which I didn't fully understand at a time and don't claim to fully now, but I think IBM was early on in understanding the technology implications of Bitcoin and blockchain. What were they and what led you to hyperledger? Yeah, so, you know, Bitcoin started in 2009 as a result of the Great Recession, right? There were a bunch of people, they wanted to build this cryptocurrency that would be free of like central governments and banks, right? And to do that, they thought, okay, we can't even rely on, you know, computer systems, right? We have to ensure that even that Bitcoin would be hosted in distributed computer systems that no one owned, right? And that led to the creation of a protocol that could run on distributed computers. And even if one computer fails, then the Bitcoin network would last. If a set of computers get compromised then the data would also still have integrity. And so that led to the creation of the blockchain protocol, right? So Bitcoin uses the blockchain protocol to use, you know, distributed systems to record financial transactions. But now that you have a distributed protocol that could record financial transactions, you could also use that distributed protocol to record the provenance of your food, you know? So for example, you know, if we're interested in buying this food product, we want to make sure that, you know, some of us, right? The eggs came from hens that could like, you know, walk around freely or that the ingredients weren't mixed with like glue or something, right? I mean, right now we hope that it's not mixed with glue, but now with blockchain, right? You could have these internet of thing devices. You could have these sensors put in the soil to measure the toxicity of the soil. You could have these IoT devices in shipping containers to measure the freshness of like lettuce.  And then all that data it's put on the blockchain, you can't change it, right? So even if you wanted a tamper with it and say, okay, well, you know, this factory isn't as clean as we thought it was, you can't, you know? It measures the level of toxicity. So now there's that level of transparency, right? And that ability to provide that data in a distributed way so that even like one owner can't compromise it, that actually leads to more trust, right? So yeah. So some of the characteristics help us unpack this. So it's immutable. I just mentioned that. I think I heard it's encrypted. It's distributed. So it's got provenance. There's no third party verification required. Right. I tweeted one time that it's virtually unhackable and somebody said everything's unhackable. I said, okay. If more than 50% of the participants, I guess, collaborate to hack it, it could be unhacked. It could be hacked, but if you and I have a transaction, we have to agree. Correct. Hey, let's hack our own system. So it's conceptually, virtually unhackable. Is it not? That's the goal to make it unhackable, right? So, you know, the Bitcoin, you know, Bitcoin was the first one and, you know, the Bitcoin development of their use of blockchain has some weaknesses, right? And that's why, you know, IBM with other companies came up with a different implementation of the blockchain. The Linux Foundation Hyperledger, you know, blockchain, and the Hyperledger blockchain is extremely different from Bitcoin-derived blockchains, right? Bitcoin-derived blockchains uses things like proof of work, right? And there, you know, when data gets committed, all of the computer nodes have to solve a complex math problem. They have to, you know, hash that into a value that starts with like what they call a nonce number, right? To provide some randomness in it. But then this is the weird thing, right? The computer system that first comes up with the solution to that complex math problem and creates that hash value, that's the data source that all the other Bitcoin-derived blockchains has to follow. And that is hackable, right? As proven by, you know, the breaches that were in the news, you know, there's somewhere where $60 million got stolen, $3 million got stolen. Hyperledger, you know, does not have that requirement for solving complex math problems and proof of work. We have a set of validators, and it's more, as you said, if we are part of transactions, you know, just Dave and I, then Dave has to validate that, you know, okay, I gave Dave $50, right? And Dave says, yes, you know, truly, Donna gave $50, but we don't really care what Stu says, right? Stu is not part of this transaction. So why should Stu have any say of whether, you know, this data is valid or not? You know, we keep Stu out, right? So the Hyperledger blockchain provides that flexibility so that only the participants who are part of the transaction, they are the ones that know whether that data is valid or not. And you gave an example of IBM supply chain, you know, for some, I guess, some hardware components, 4,000 suppliers.  It takes a lot of time.  What do you mean you don't see my invoice, you know? Yeah, I sent it, I sent it again. So on average it takes 44 days to resolve. So you've implemented the system with things like proof of delivery and proof of payment, et cetera. What has been the business impact to IBM? Yeah, I mean, the goal, I mean, the reason why we put this on blockchain is to reduce the dispute, the amount of time it takes to resolve a dispute, which is 44 days. And then when we reduce the amount of time it takes to resolve a dispute, then we don't require that much money, you know, that we have to, like, put on hold in our supply chain. And then it also creates better relationships with our partners and suppliers as well. And that, you know, has been the goal of, you know, the IBM global finance blockchain, yeah. Donna, so many of the challenges that we're facing in IT these days seem to fit right in your area of expertise. You know, it's all about distributed architectures. So you mentioned that, you know, IBM really has strengthened scalability and security.  You know, IoT is from, you know, edge to core and all over the place. You know, how was that knowledge of building distributed systems, getting spread in IBM and working with your partners and across the industry? Yeah, yeah. So, you know, when all was started as technical people, right, we, you know, we started by building a particular component, right? We programmed or we built a computer board, right? And we had this sense of craftsmanship, right? Because we own this widget. We own this, you know, these 20,000 lines of code, right? But it almost goes back to Dave's question about open source. The real power is like when we combine all these things, right? So if we just created little sensors on our own, right? And we never had that data, you know, we never looked at that data through analytics, then we couldn't find as much from putting these sensors in soil or putting these sensors in like airplane wheels or like cars, right? But then you combine that with analytics and then you could just, you could find all these patterns. But then, you know, you get this problem, okay, well then how could you trust the data, right? Because the data sent from IoT devices, you know, to some storage system or a laptop, it could get hacked in the middle, right? And that's when we have the blockchain protocol, right? It gets into, you know, it gets into a blockchain node, right? Other IoT sensors in the soil also sends the same data, right, so we have corroboration, right? And so now you have the pedigree of the data and it's combining all these three, right?  You need like distributed systems to be able to, you know, safeguard the data and provide the power. So it's weird that, you know, it's almost like baking, right? You have all these ingredients and when you put them all in, you get a great cake, right? So it's much, it's really good to put them together. I'm curious your thoughts about some of the hyperscale players, you know, people like Google, Amazon, you know, Facebook and Apple, they all build, you know, very distributed architectures, but they're not nearly as open as a company like IBM, you know, is that a misperception or, you know, could that be a limit to some of our growth of things like IoT or going to require interaction with some of these type of players? You know, it's almost like, and maybe this is just a statement at this moment in time, but all roads lead to open source, right? I mean, we were talking about like history, right? So, I mean, just look back even before blockchain, right? The open source UNIX systems, right? You had Solaris, you had like AT&T, right? BSD, right? And then you had Linux and we all knew at the beginning, Linux was not as powerful, right? As all these other, you know, UNIX versions, but Linux came to be the most adopted, right? And you know, one reason is because it's free, but the other reason is because you have this whole community donating code to make Linux, you know, to make sure it scales, to make sure it's secure, to make sure it works with like little phones and large servers, right? That type of like, that type of density, you start out as a proprietary system, you know, the Googles or whatever companies, but even then, you know, even the Googles, right? They open source their code, other people donate and it makes it even better, right? I mean, if you really want to make your product better, you know, just open it up to the light of day of open source, right? People are going to read it, they're going to say, that doesn't work. This is a much better way to do it. And so what? We're grown-ups, we take criticism, right? But it'll make it better. And then we'll say, okay, you put an improvement, but that really is really bad. Here's a better way to do it, because we're always trying to like one up each other. That's a human thing. So Donna, it seems like everybody's a software company these days, so I've been thinking about Hyperledger. What types of companies are contributing to the project? What kinds of things are they doing? Where are they coming from? Is it different industries? It's not just the technology industry, obviously. Yeah, that's what really surprised me. You know, I thought, okay, we're going to open, you know, Hyperledger is going to be an open source blockchain, it's only going to be blockchain companies donating. But that's not true, it's industries that are writing blockchain applications. Financial companies are donating code to make sure that we satisfy their financial protocol requirements, right? Manufacturing companies are donating code and to make sure that we support like these manufacturing standards, healthcare companies, right? So in fact, the fewest numbers of companies that are donating blockchain code are actually pure blockchain people. It's the users of blockchain that are donating code. And that really surprises me.  And it's good because they guide the shape of it, right?  So it's, it's, it's interesting. And IBM's making these services available in the cloud. Yeah.  Can you share with us what you've done there? Yeah, yeah, so, you know, there's something called FIPS 142 level four, Federal Information Processing Standard 140-2 level four, right? So it's a set of guidelines for any software or hardware manufacturer providing cryptographic algorithms. It has to, has this behavior, it has to protect your data in such a way to be qualified for FIPS 140 level two. Your crypto algorithms have to pass, you know, these set of criteria. There's four levels of FIPS 142. Level one is the weakest and level four is the strongest. The existing computer systems out there that provide crypto algorithms, you'll find levels one and two. If, you know, the industry really requires it, you'll have specialized companies provide these hardware modules that you could network attach that gives you the level three. And IBM Z systems, the Linux one servers, the only one that has the strongest level level four. In fact, it's not something that's network attached so that it could be hackable by network intruders. It's built into the system. The reason why we have to qualify for FIPS 140, you know, dash two level four is because if you're providing crypto algorithms for any type of regulated industry, you know, financial, healthcare, insurance, many governments require that level of certification. So we had to do it for all these others, for all the sensitive data and then we said, you know, people are putting this type of data on the blockchain, let's make sure that we, this vault we built also protects, you know, blockchain. So given the inherent security attributes of blockchain, there are obviously security applications and use cases. I know IBM's working on some, MIT's got Enigma. Can you share with us kind of what you think the potential is for blockchain as a new security paradigm? Now that's very interesting, right? So that will be really interesting. Like two years from now, we come back and we see if this takes hold, right? You know, at the heart of all these endeavors is this idea of what is identity, right? So, you know, I trust that you're Dave, right? And you trust that I'm Donna, right? We see each other, right? But what if we weren't sitting physically across from each other? What if we couldn't even Skype each other? How would you ensure that this other person that you couldn't even see is Donna, right? And then, you know, so you have like crude levels of like authentication, user ID passwords, those are hackable, right? Then, you know, the things coming up in the industry now is multi-factor authentication, not just one password, but okay, hey, we'll send you some, we'll send something to your cell phone and only you as the owner of the cell phone could see it, you know, we'll send it to your email system, right? But it's all to try to prove that you are who you are, right? And blockchain, since it is, once you put a record on the blockchain, you can't change it. There's a set of documents, right? That people, governments, corporations are coalescing around that if you have, for example, this driver's license, you have this in the USA, this social security number, you have this passport number, right?  In other countries, you know, they might say, okay, you also have to supply like your last three postal addresses or something, but you know, the answers to those questions, I mean, we do it every time we sign up for a website, we have to put like five to 10 personal questions, right? I mean, that is so inefficient, right? But if that information could be on a blockchain somewhere secured and protected, right? And then you provide something called zero knowledge proof where, all right, you know, I'm Donna and I've provided two out of the five criteria, right? And when that blockchain identification service says, okay, Donna answered these questions, Dave, you could really believe this is Donna without Dave ever having to know what are the answers to my personal questions, right? That is the substance of this collaboration. It is, how do you prove who you are and not have to answer the same questions like a hundred times over? The black box validates it and I then, as a consequence, trust that you are Donna, so. Yeah, yeah, that's very interesting, right? And you think that's a couple of years, we'll start to see commercial use cases for that type of technology? You know, imagine that, imagine how small, smaller your wallet's going to be, right? No more driver's license, no more social security card, no more like whatever card, right?  Boy, that's really good, yeah. Well Donna, it's great to see you face to face and I really appreciate your time. Thanks for coming on theCUBE. All right, thank you. All right, keep right there, buddy, Stu and I will be back with our next guest. This is theCUBE, we're live from IBM Edge, hashtag IBM Edge.


All right. Hello. Welcome to benchmarking Sahara-based big data as a service. My name is Trevor McKay from Red Hat. On the schedule, this was actually listed as Matt Fairley. If you know Matt, he sends his regrets. He couldn't make it, so I'm filling in for him. I'm here with my colleagues, Jidong Yu, and Wei Ting, Chen from Intel. We put this thing together. So I'll be doing introductions, and then Jidong will be covering the performance data. We're going to start with why Sahara. Sahara has been around for a few cycles now. It's big data on OpenStack. We'll talk a little bit about what the motivation is for having that as a service. We'll go into details. A little bit about the architecture and features. Then we'll cover deployment considerations. From a performance perspective, what do you need to think about when you're putting these things together? Containers, bare metal, VMs, where your data is located, those types of things. They'll show some results that they came up with from Intel. Then we'll talk a little bit about where Sahara might go and specifically what we're going to try to do for the Liberty Cycle. Let's see. Big data is everywhere these days. It's really not sort of a niche thing anymore. Everybody's doing it. My new friend, Asim, here from Cisco has extensive experience in that. You can talk to him about use cases. If you're working at Walmart or Yahoo or Facebook or Twitter or you're doing genomics like the SwiftStack guys we're talking about with Fred Hutch, there's just piles and piles of data. The CERN guys write petabytes, terabytes, exabytes of data all the time. Someone you work with is probably running workloads on something like Amazon or Azure or Google. It's just a reality. This stuff isn't going away. There's a few reasons to run workloads there. One is just virtualization, the benefits of OpenStack. We all know that. You also want these modern features. You want storage, you want database as a service, you want elastic map reduce type facilities from Amazon. When you migrate to OpenStack, getting the core services isn't necessarily just enough. We love Neutron, we love Cinder, we love Glance, we love all that stuff. But it's the Defcore and it's really, really important, right? Essential. But on top of that, you start to need to add applications. So newer projects like Sahara and Trove start to cross that boundary between just core services and applications out of the box so that you can do things at the application level and you don't have to write them yourselves. So this is sort of Sahara's reason to exist, is to make that possible for you. And of course, as the bottom line says, writing the applications is complex enough without having to manage all the infrastructure. Let's see. Okay, so big data analysis is tough. You have to acquire the data. There's a lot of it that can be challenging. You have to organize it. Anybody who's worked in ETL knows that that can be hard, just managing this stuff and shipping it around. Then you have to write your analysis, figure out what questions you want to answer, and you've got to take your output and do something with it, right? Present it to somebody so that they have an answer for, you know, some customer informatics question that they were trying to answer. And again, all that complexity is just in the domain space itself. That doesn't include the tooling and infrastructure you need to do it. So, this is why we made Sahara. And some headline features here in case you don't know what it is. Sahara gives you repeatable provisioning capabilities. So, you define cluster topologies, what you want them to look like. You store it in a database.  So, once your cluster is up, now you want to run jobs. And this is what EDP is about. That stands for Elastic Data Processing. That's basically your job management system. So, you might have bursty workloads. You might need, you know, various kinds of storage. So, we have things like cluster scaling. We've got integration with different storage types. We integrate with Swift. Hadoop compatible file systems. We also pay attention to security for you. And things like service, like anti-affinity and high availability considerations so that once you have this up, your cluster doesn't go away or become, you know, unserviceable. So, these are all the things Sahara gives you. Here's a quick look at the architecture. So, this giant blob in the middle is the main Sahara engine. As most things in OpenStack, it has a client that talks through a REST API. We have a page in Horizon. Within the main Sahara body, that DAL box down there at the bottom is the interface to the Sahara's local database where it tracks state and those kinds of things. It has provisioning logic in it which talks through heat to build you clusters and uses Nova to spawn instances, sender to set up non-ephemeral storage. We get our images from Glantz. We use Swift to store input and output data sets. And then, of course, there's the vendor plug-in piece. Vendors may be a misnomer because there's actually a collection of grad students from South America who are doing awesome work. So, you don't have to be a vendor to write a plug-in for Sahara. So, if you want to write one, just write one. So, we should maybe change that name. I don't know what else to call it, though. But the vendor plug-ins have the logic to spawn a cluster of a particular type and configure it at runtime. So, you can see from this that, actually, Sahara is in the big tent model of OpenStack. We're really a consumer. A large majority of the OpenStack services and use them ourselves. I forgot to mention there is a raffle at the end of this for this awesome biometrics gear that will measure your heart rate when you come to exciting talks like this. So, stick around. You don't want to miss the raffle, okay? Let's see. So, how do plug-ins work? We have plug-ins for particular processing engines. So, there's an HDP plug-in for a Hortonworks data platform. There's a Cloudera plug-in for the CDH stuff. We have a Vanilla Apache Hadoop plug-in. We've got one for Spark. Spark standalone deployment. We've got one for Storm. We really, any processing framework that you want to add, you can. I was just talking to somebody today who has a new storage system from their company and they're thinking about writing their own plug-in to be able to use their storage with Sahara. So, obviously, to vendors, this is a way to integrate with OpenStack and get accessibility to a user base who's consuming OpenStack.  They may have various applications to run on their stack. One of them probably is going to be big data processing, so it becomes one-stop shopping. Let's see. Oh, MapR, I forgot to mention. That's relatively new. I think maybe last cycle it was being added. Storm is brand new. And, of course, in downstream distributions like RDO from Red Hat or Mirantis OpenStack, those vendors will certify particular plug-ins, right? And they'll do all the QA, give you reference architecture so that you know when you go and deploy this thing it's going to work and I don't have to worry about it. Let's see how we're doing on time here. Oh, okay. Great, perfect. Okay, so now I'm going to hand this over to Zhidong and he's going to go over the performance findings with you. Good afternoon. My name is Zhidong Yu and I work at Intel. We are committed to cloud and big data and we kind of believe for big data moving to cloud is inevitable trend, so that's why we work on this Sahara project. So when you start deploying a Sahara-based solution, there are a number of things you should consider. In this section we will go through those considerations. It's kind of like a functionality benchmarking. Then followed by this section we will show a little bit of performance testing we have done and the results. So it's the performance benchmarking. Then based on what we have seen in the functionality and the performance benchmarking, we are going to talk a little bit about our invention of the future of Sahara. Then finally we will have a summary and a cut reaction. So the first thing you need to think about is the storage architecture. So there are basically two major different ways to provide the storage in Sahara. The first thing is you can let the tenant to prevent the storage by themselves. By default Sahara can help deploy the HDFS components within the virtual machine. So the good thing for this model is everything is almost the same as the traditional bare-made model where the computer and storage are co-located together. But the problem with this is that since when you upgrade it on top of cloud you are actually expecting an on-demand service. That means you will launch the virtual cluster and terminate it from time to time. So with this model you have to import the data from somewhere else because you can really go on the data processing. And you also need to possess the data back to somewhere before you can terminate the cluster. That's not exactly the users want to see. So instead of this there's another option the tenant could prevent the storage in another set of virtual machines like the scenario 3 in the chart. So with this scenario the storage VMs could persist longer than the computing clusters. So that's kind of fixed the problem we are talking about in the first scenario. In those two scenarios the storage is actually a disk attached to the virtual machine. So the disk could be either a ephemeral disk or a signal volume. With ephemeral disk the good thing is the computing and the underlying storage actually is co-located on the same physical machine. So there is still a chance for Hadoop to achieve data locality. But if you use the signal as the volume back end depends on how you implement the signal service. You may use LGM which is co-located with the computer because the network removed a safe implementation or something. So in that case you may lose the possibility to achieve data locality. Another way to provide storage is admin could deploy the storage system in advance. So in that way the storage is actually logically disaggregated from the computing tasks. Tenants will only run the computing tasks within the virtual machine. But this implementation could be either physically co-located with the computing or totally a network remote solution. So in the diagram here the scenario number two we deployed the HDFS in the computing of the host level. The HDFS system is external to the virtual machine but with proper consideration it's still possible to achieve local data access. But in other scenario the scenario number four the virtual machines are accessing data in a network remote swift. So in that case all the data access will go to the network. There is no data locality. By the way in both mode since the virtual machine needs to talk to the outside of the VM so the network is very important. In our testing we found one of the new trends feature DVR is very helpful for this. So our opinion is that in disaggregated storage system has much more values than a build-in I mean within the virtual machine storage system. So in disaggregated storage system in this mode there is no data silos or the virtual Hadoop cluster can share the data in the same system so there is more opportunity to find the potential new business usage or something. And another thing is with this mode Sahara may be able to leverage the OpenStack Manila service. So that means Sahara may be able to call the Manila API on behalf of the tenants and let Manila handle all the storage provisions since Sahara is just a consumer of the Manila services. Then this approach also can create more opportunity to the vendor so they can build some advanced solutions. For example create an in-memory overlay file system within the computer nodes like Taqian or something. So this is about the storage architecture. Then the next thing is the compute engine. Basically we have three choices, virtual machine, container and bare meta. We all know about the pros and cons for virtual machine. For container we know it's lightweight. Provening is very fast but the security is kind of a problem because it's weaker than virtual machine. The other thing is currently the Nova Docker driver is still not in the upstream. So Sahara actually cannot support the container very well. The last option is the bare meta. I know this morning there is a panel discussing using Sahara and bare meta together. For me the good thing for bare meta is it has a very good best performance in all these three options. But the problem is that we have tried to deploy Ironic in our lab but since Ironic is less mature than both VM and container the other thing is the resource utilization efficiency is still a problem. Every single virtual cluster is still monopolizing a single physical cluster so there is no resource sharing at all and the migration is also a problem. The provision is also slower than container. My point is that my opinion is the container seems to be the most promising technology for running big data on top of cloud but unfortunately currently Sahara doesn't support the Nova container very well so there are many things ahead. On the other hand there is another problem common to all those three options which is even with Sahara the tenants still need to concern about how to determine the appropriate size for the cluster and what kind of flavors should be used with this cluster. So they can choose to use a large flavor with less nodes they can also use smaller flavors with more nodes. So it's not easy for tenants to make this decision. In many cases the tenants are just developer or data scientists they just want to use the services. They don't want to need too much about the underlying complexities. The third thing in the consideration is the data processing API. By default when you finish a cluster provision you can use it as usual tenant could just access it into the virtual machines and consume the service as usual. If the specific distribution provides a high-level API like OZ tenants could use that as usual. But in Sahara we recommend to use the EDP API. EDP is designed to be an abstraction layer for users to consume the underlying service. So ideally EDP should be vendor-neutral and plug-in agnostic means you can always switch among the Sahara plugins like CDH or HDEP. But unfortunately currently EDP is still under development. We have implemented a lot of features but we could add more like more job types. And I believe a few hours ago in the self-knowledge there was a session talking about adding more features to the EDP. So there's another option. You can use a third-party abstraction layer like the Cask CDAP. Cask CDAP is an abstraction layer to make the big data developers live much easier than before. In theory those components can be used to replace EDP or anything that can do such kind of job. So unfortunately Sahara currently does not support the third-party abstractions. So as a summary this is a matrix showing all the things you need to consider about. On the bottom there are storage and computer. There are a few options. Then the next one is the distributions. There are many options but we did not talk about anything here because it's totally up to the consumers or the users. On the top is the data processing API. Currently we have the traditional way, the EDP and potentially we could support third-party APIs. So with this in mind we are trying to back to this page we did a few performance testing to compare the options for the storage and computer layer. The top layer is totally about the functionality so we did not do any performance testing. So we just used a four-node cluster to compare a few configurations. So the first one is we compared the ephemeral disk performance with the bare method disk performance. So basically we see the only left of this chart is the read performance, right is the write performance. We see 1.3x read overhead and 2.1x write overhead. So we understood in the clouding context the disk access pattern totally changes, right, because in OpenStack a typical configuration is that you have a bunch of disks, you create a rate and have a logic volume, then the logic volume to the Nova instance store, then every virtual machine will have its own image file from the instance store. The access pattern is totally different from the bare method scenario. However, our characterization shows that only 10% of the overhead are from the access pattern change. So all of the overhead are still from virtualization like the IO and memory efficiency. So we have already done a few tuning here, not very extensive, but the performance is still not good. So we believe if you want to achieve good performance, heavy tuning is necessary, is required. We compared another scenario. We moved the HDFS from the virtual machine to the host level. The good thing is that we can see the performance for the write case improved significantly. For the read case it's still higher, but we know the reason. The reason for the improvement in write cases and the first thing is the virtualization overhead is removed. Now the HDFS can access all the discuss in the physical mode. I mentioned that the Neutron DVR feature helped optimize the network path. So that's another reason for the performance improvement. However, the reason for the write overhead is that we further to enable the location awareness feature for Hadoop in cloud environment called HVE. This is a feature developed by VMware in Hadoop to support their vSphere product. Actually Sahara can benefit from that feature as well, but somehow we tried to enable this feature, but somehow it didn't work in our environment. So if we can make this work, we expect to see a better performance for the reader case as well. Then we compare this with the performance. Same as before, this overhead is higher than the ephemeral disk case. I believe we know the reason. The reason is this location awareness feature for Swift is not enabled. We tried to enable this, but somehow we run into errors. So in Sahara, if this feature can be turned on, then we should see similar or even better performance than the internal HDFS. So the conclusion from last page and this page is that if we can move the data from the virtual machine to the outside, we have a chance to achieve better performance. And at the same time as I said before, now we have a centralized storage. It may bring more business opportunities. So the last test case is compared by a radar versus container versus a VM. I just said Sahara doesn't support a container very well, so we actually did a few hackings to make the container work. We use the Novadocker driver. So the performance shows the virtual machine has 2X container, has 1.4X. One thing I need to call out is that we have done a few tunings for the VM case, but everything with the container cases out of box, so it's still much better than the virtual machine case. So we think container is very promising in terms of performance compared to VM. So based on what we've seen in the functionality and the performance benchmarking, here are a few things in our mind that we expect to see in Sahara. This is not a roadmap. This is just something we think Sahara would be better to support this. The first thing is we need an architecture to allow disaggregated computing storage. We want Sahara to support more storage back end. If possible, Sahara could be integrated with Manila. The second thing is we need better support for container and Belmada. I believe Belmada is already in the roadmap. There are talks in this summit, but for container, we should consider if Nova is the only option, or maybe we should use a Magnum. The third thing is the EDP. As the abstraction layer for Sahara, it needs more improvement like data connector workflow, policy engine, SLA, auto-scale auto-tune. So ideally, as I said before, the user don't want to concern about anything about the underlying infrastructure, like how many nodes, what kind of VM flavor. Ideally, this abstraction layer should take care of this. Even like auto-scale, when the load is high, this layer could add in more nodes to the cluster, something like that. Finally, I think Sahara should offer a broader when integration opportunities, not just the big data engines like Cloud Air and Hortonworks. This is because a complete big data stack may have many options at every layer. You have the storage at the bottom, then you have the data platform, and you also may have a few data analytics layer components on top of that. So there are many vendors at each layer. I think Sahara should be open these opportunities to all the vendors. See that as an example here. Another example like SAS analytics products. Okay. So this is pretty much my talk then. Trevor, we'll continue to talk about the currently low map for this Sahara liberty and then conclude with a summary. Okay, so quick note. How many questions do you think we have out there? Show of hands. People want to ask questions? Haven't thought of it yet? Because I can talk fast or slow. All right. Looks like maybe we have a couple. Okay, so let's see. I went back too far. There we go. So that last slide wasn't a road map. This is a road map. Okay, and these are the highlights for the cycle. So big things we're going to be doing. HDP is one of our major vendor plugins. It's going to be ported to HDP 2.2. If you're familiar with Hortonworks in Umbari, you know that there are a lot of improvements there. We're going to be working on HA for the CDH and HDP plugins. That's HA on the cluster level. So there's HA for Sahara itself. There's also HA within the cluster, within the Hadoop services. That's what that's about. We are using heat as our primary provisioning engine now. We used to have a direct engine. We had heat as well. Now the direct engine is being deprecated. Heat is the new heat. That's what you should use. And we're going to add features to it. Also plan to bring Sahara to the Python OpenStack client. One-stop shopping client. So that will be great. Bare metal clusters. We've already talked about several times this summit. So we're going to be working on integration with ironic security enhancements are very much on our mind because we realize that that's important. So there will be a lot of work there. And also a number of things in EDP land. So job scheduling, repeatable jobs, future jobs, coordinated workflows. We were just talking about that today in a panel. So you can ultimately, someday, we hope to have things like directed acyclic graphs with dependencies from one job feeding into another so you can develop more complex things. Log retrieval for simple debugging. Very simple feature, but very helpful when something goes wrong with your job. You want to know why. And also, if you've used EDP before, you know that some of the jobs, the argument passing and parameters that you can pass to a job are somewhat fixed for some of the job types. So we have a model figured out for making that much more general and much more flexible so you can run all kinds of things. Okay, so our call to action. We did a lot in Kilo. We have real customers out there using this stuff, running real workloads. We're beginning to get all kinds of customer feedback and asking us to help them out. So this is something that you can really use. It's maturing. Let's see. Honestly, I'm not sure. Well, I guess that's a summary. I'm not sure about the thought behind the second point. It's a true statement. Bare metal is still preferable for performance. There was an interesting note on that on the panel earlier today and it was a developer from Cloudera and he noted something which I think is true. The performance characteristics are going to be sort of dependent on your workload. So bare metal is going to be more important for people who are doing real-time analysis on-demand kind of stuff, queries against data, trying to figure something out and that's where your performance is going to matter more. If you're running batch stuff in the background overnight, it's not going to matter as much. So there's still room. VMs might be slower than bare metal. Okay, fine, does that mean if you're running Sahara, you have to go out and buy a whole bunch of new hardware and use Ironic and run stuff on bare metal? Well, no, it depends on your workload. So there's a performance difference and tuning is important but it also is going to depend somewhat on who you are and what you're doing. This is probably the major point at the end here. We're a smallish team. I don't know, I haven't added them up. Maybe what, 20 contributors, something like that, actively, really active people. We can certainly use more. We come up with more awesome ideas than we have people to do, right? So as someone else said this week, if you're looking for a project, come join ours. I guarantee you we'll give you something to do. Let's see, we have our, let's see, we're doing, oh great, five minutes for questions. I didn't put it on the slide. Our meetup is tomorrow in room 218. That's 218. We'll be there all day long. Please come, we love visitors.  So come hang out with us, tell us what you think. Ask us questions. It's great. He hung out there all day today. So I guess that's it. And if you have questions, please use the mic. If you don't, I'll have to repeat them. Questions from anybody, for either of us. Andrew Lazar from Mirantis. And I was making a performance test of Sahara two summits before. And during my testing, it is really huge difference in performance parameters depending on configuration. And in your slides, just the results, can you share more details? What was host cache parameter? Which case do you mean? This one or this one? For example, for ephemeral storage use case. This one, we use a DFS IO. Actually, I compared our results with... Because tests, speed is very sensitive to, for example, chunk size. So you will get completely different results for small chunks and for big chunks. So tests are very sensitive to host page file configuration. Because if you enable... In this case, we enabled the huge page and we did see performance improvement with the huge page enabled. I think our case is a little bit different from the result you guys published last year. Of course, we used a 1TB depth size, but you use the DD as the workload level. We use the DFS IO. We use a much smaller depth size, only 64GB. But I compared the results. It's very similar. You guys reported a 2X performance overhead. And what drivers do you use? Drivers of QM or... Discuss? No, I mean... QM drivers... QEMU? What kind of... Because we saw... You mean, what IO or QEMU? Yeah, virtual IO drivers. I think it's... 2.7 or 1.7. Because we see... Yeah, QEMU, but Andrew's question is the driver mode for the storage. Is it a virtual or QEMU? Yeah, version of driver. Because we see a huge difference between 1.7 and 2.7. So it's... What driver was used for? It depends on what is the default in OpenStack NOVA. So for this part, we did not do much tuning. We only tuned a few OS level like huge pages. We tried to make the performance look better, but on the other hand, we realized maybe we just needed to show the out-of-box performance that the majority of the customer will see. So we don't want to do too much tuning or too extensive tuning. Thank you. So, hi. So during this week, I've heard two conflicting things. So some people say, oh, compute nodes have local storage, almost always. And then other people say compute nodes always use network storage. And I'm saying that when you look at this problem, so you are saying that the overhead is mostly because of IO. Do you see a solution that makes this optimal in both cases? It's more like a curiosity. I don't really have a question. Well, you know, the data here, we compared a few cases. We did not really compare the local storage versus the remote storage. So currently, we don't have data points to show how bad or how good if you put all the storage on the network remote side. But do you think the overheads would be the same if you're comparing network IO and disk IO so you have the same... Oh, no. I would expect the worst performance if you use the totally separated, safe or glassed FS as the storage back-end. Because after all, all the data traffic goes through the network. But on the other hand, it's a typical deployment for most open stack. People like to use the safe with the universal back-end for both glass, nova and single. If you deploy it safe or glass together with the computer, they may bring other issues like resource contention. You will not get a determined performance. So that's one thing we plan to do in the future. We will compare the performance. In those cases, the disks are all collocated with the computer. We will compare those cases with other cases like a network remote safe or glassed FS. Right, so you don't believe that the overheads, the IO itself is just the usage of the disk. So I think we're over time. So we'd love to answer your questions anyway. So come on up and we'll answer them. But I guess that's it for that official talk.