Welcome to Staten Energy Man here on Think Tech, Hawaii. Another really gorgeous day in Hawaii. Gotta love it. Glad I grew up here. Glad I live here. Glad I'm gonna stay here. But I'm planning to move to the Big Island. That'll be good too. Anyway, it's almost fall, believe it or not. We're ending up August, rolling into September. Kids are going back to school. Not hard finding parking today. Right before a three-day weekend. So it's actually pretty nice here in downtown Honolulu. We're going to talk a little bit today about a subject near and dear to me. But before we get started on that, just a little public service announcement. Staten Energy Man has been on every Friday for the last four years. And they're going to start moving us to different schedules. So Staten Energy Man is going to start being on Tuesday at three in the afternoon streaming live here from Pioneer Plaza in beautiful downtown Honolulu. That'll start on the 17th of September. So we're still good for today and for next Friday. And I think one more Friday. It'll be a Friday the 13th will be my last Friday show. That's a real scary one. And then after that we're starting on Tuesday. So we'll catch you then. For today's show, we're talking about sustainability and actually how we really start moving to sustainability. And I'd like to go back a couple months ago. I had a great guest on my show and get a huge impact on how I think about things and actually validated a lot more about what I think about in terms of energy and sustainability and decarbonizing the planet and things like that. His name is Professor Nate Hogan. And he runs a program, I think it's out of Minnesota, where he talks about energy and he talks about making choices. And it's really important that you understand where he's coming from. He uses a term called energy blindness. And I think the quickest way to define energy blindness is a lot of people know how much their electric bill is every month, roughly. But almost nobody knows how much power they use every month out of the electric bill. In other words, they can tell you that their electric bill is 180 or 250 or 300 bucks a month. They can't tell you how many kilowatt hours they use on any given day. And for the last at least six or seven years, I've been looking at my energy and it's pretty consistent for my house. And that's important because you have to have a concept not of the dollars because whether you're on a different rate schedule or whether the price of electricity was up or down, that'll change your electric bill. But how much energy do you use and how do you use it? I mean, you sit there and really focus on saving energy, which is the first step in all of the sustainability models that we do is you try and be more efficient with the energy that you have and then go for the solar panels or the wind turbines or whatever you're going to put in. And so being energy aware and having a good understanding of energy and not being blind to it is critical. So I'll give you a really crystal clear example. My house uses on average between 20 and 22 kilowatt hours a day. I have an electric water heater. I have an electric oven and I have an electric dryer. I don't have central air conditioning. I just I have fans in the house. That's the profile of my house. It's not insulated. It's single wall. I put gecko roof on in white so that it reflects as much heat as I can. And that's it's a real basic house here in Hawaii. What does that really mean? Well, if you know how many kilowatt hours of electricity you use, not how many dollars, but how many kilowatt hours, then you can actually start doing analysis. And you don't need a rocket science degree or a calculator to figure this stuff out. For example, the typical truck that we used to convert for the Air Force had 28 kilowatt hours worth of battery on board. And it would take that truck about 30 miles. Now, if that was a car, it could probably go about 65 or 70 miles on the same 28 kilowatt hours.  In fact, if the energy in my house is only 20, 22, 21 kilowatt hours, then your your car could probably go 50 miles on that. Your car would use a lot of energy if it was an electric car. But we don't think about that. We think about electric being good, and it is much more efficient, much cleaner than using diesel and gasoline to power your car. But you have to be aware of exactly what we're talking about. You also have to be aware where the power is coming from. So let's start off by talking a little bit about transportation. You have cars nowadays, and everybody asks me, Stan, if hydrogen vehicles are so great, how come we're not driving all of them right now? And the answer is fairly simple. We spent about 100 years working on transportation infrastructure based on oil, based on a very cheap, also fuel, easy to get to, easy to pull out of the ground, easy to refine, dirt cheap. And we're still using it, but the price is creeping up. It's impacted also by international issues. When we have OPEC running things, like we did back in the 70s, man, there were gas lines here in Hawaii and across the US, because that fuel is controlled by a certain group of people, and they could use it to leverage other countries. It's the same way they're currently. We're leveraging our economic strength with other countries. They balance our trade with those countries again and get it more fair. Other countries do the same thing, but hydrogen, if you think about it, no country owns hydrogen. Every country has hydrogen. We could do it. But then we take away that ability for one country to leverage over another country. So maybe that's one reason why we don't do it. Another reason, we've spent so much money on infrastructure for gas and oil. You can't compete. That infrastructure has been bought and paid for for a hundred years, but we're still using it. But it doesn't drive the cost of oil up any. It's already a sunk cost. But to do hydrogen, we'd have to start from scratch or pulling hydrogen out of water or methane or whatever, whatever we could pull hydrogen out of almost anything. It's the most common element in the universe. The bottom line is, though, if you want to start moving the right direction in transportation, you have to look at hybrid technology. But what is hybrid technology on car? Hybrid technology is an electric drive train being provided electricity by either stored energy and batteries, produced energy by a generator being run by a gas engine or a diesel engine, or hydrogen, or you could even use a flywheel, I guess, in a car. You could use nuclear power in a car. As long as there's something providing electricity for that hybrid drive train, you can have a variety of power sources. So right now, most prevalent choices are electricity from the grid, not necessarily clean.  Now you're doing clean energy, clean from solar, and clean stored in batteries, put it in your car, you're good to go. But if you can't, if you don't have it, like you live in an apartment, you can't put solar on your little patio or a deny, not enough to drive your stuff and hard to wire that down to your car, sitting 24 stories down. So what do you do? Well, you get a hybrid vehicle. Now, hybrid vehicles have smaller engines that are matched with a generator.  And it's very inefficient at the low end. And it's fairly efficient at the high end. And there's a sweet spot somewhere in the middle there, where it's got its peak efficiency. And we're almost never there. But a hybrid's different. The gas engine in the hybrid is designed to work at its most efficient point, or not at all. It's either charging the batteries and giving power to the electric drive train, or it's off. And that's why hybrid cars get like 40 to 50 better gas mileage than a regular gasoline car.  And then you have other power sources. Like I said, I don't think we'll see nuclear for a while. Although I have nuclear engineer friends that say it's the way to go. And the new power is safe, clean, everything else. I don't think we're ready for that, at least not socially. But even hydrogen, fairly simple. You take oxygen, oxygen from the air, hydrogen, put them together. It makes water heat electricity. And it completes a regular eco cycle, just like you'd want. But the key, again, is we need to start looking at hybrid transportation. Most of the car companies out today have hybrid technology. They don't advertise it a lot. But you can find every auto manufacturer makes hybrid drive trains. And they're there, need to look for them. And I would say if you're going to make a vehicle choice in the near future, look for it.  Eventually, the entire transportation sector will be electric. So what does that mean in the bigger picture? That means we're going to have a lot of electric power requirements. And right now, we have grids that are struggling to maintain our current requirements and absorb more energy from solar and wind and other things. And try and balance them all together. And I can tell you right now that the current grid is just not there to handle it. What I'd like to do right now is show a short video that talks about how we probably ought to start looking at microgrids in the future and how that impact will affect transportation as we go forward for the next 20 to 30 years. There are over 300 million people in our country. And the vast majority rely on large scale centralized power grids for their energy. But the infrastructure is aging and it is vulnerable. Natural disasters, cyber attacks and other threats can leave large swaths of the country without power. Fortunately, there is an alternative. A renewable energy microgrid represents a different path for the future. Renewable microgrids generate power from sources like solar, wind, hydrogen, waste to energy and geothermal. That power can be stored within the localized system using technologies such as advanced batteries, hydrogen, flywheels, pumped hydro and others. These microgrids can provide reliable and efficient energy transmission, especially to critical facilities like hospitals, airports and military bases. Unlike our current large scale systems, microgrids eliminate single points of failure and are therefore more resilient to disasters, threats and power outages. Our current energy infrastructure loses a lot of money. Grid outages cost up to $33 billion annually. They are expensive to build, expand and maintain. And they're inefficient, losing more than half of the initial energy to factors such as line loss, spending reserves and theft. Microgrids solve these issues and greatly reduce transmission loss and maximize efficiency. They also reduce carbon emissions and eliminate imported fuel costs, keeping money within our local economy and even create new local industries and jobs based on clean, renewable energy. Our energy grid was built over 100 years ago. When energy needs were simple, with the increased complexities of energy demands, power sources and transportation, now our old grid struggled to keep up. We required new ways to generate, store and deliver energy. Renewable energy microgrids are a potential long term solution that will provide safe, clean, reliable and efficient energy for generations to come. So that's just a little picture of kind of where we're at in the grid now and where I think we should be going. That video was put together by Hcat, used to work for Hcat. And it's a beautiful video, well done, actually won some awards here locally in the advertising world for their animation. But the message is the important part. We have to be mindful of the fact that the grid of today has been stretched to its limit. And here in Hawaii, we have a pretty straightforward grid, all run by one company. That one company takes power from a coal facility, oil burning power plants and H power, which is basically a big incinerator. They try and also incorporate as much solar and wind as they can. And those things bring in some instability into that old archaic grid. So we're going to take a quick break here. We'll be back in 60 seconds and we'll talk about that grid and maybe what the future of a grid should look like that would also incorporate the solution to clean transportation. Aloha. My name is Wendy Lowe and I want you to join me as we take our health back. On my show, all we do is talk about things in everyday life, in Hawaii or abroad. I have guests on board that will just talk about different aspects of health in every way. Whether it's medical health, nutritional health, diabetic health, you name it, we'll talk about it. Even financial health will even have some of the Miss Hawaii's on board. And all the different topics that I feel will make your health and your lifestyle a lot better. So come join me. I welcome you to take your health back. Mahalo. Aloha. My name is Mark Shklav. I am the host of Think Tech Hawaii's Law Across the Sea program. My program airs every other Monday at one o'clock on Think Tech Hawaii. Most of my programs deal with my own life and law experience. Recently, I interviewed Alex Gempell, who I have known for over 30 years, about his voyage across the sea as a lawyer from Tokyo to Hawaii. Those are the type of stories that I like to bring and like to talk about. Human stories about law and life. Aloha. Hey, welcome back to Stand the Energy Man here on Think Tech Hawaii. We're talking about the grid and what it should really look like in the future. Now, the scary thing for me is that Hawaii already has a mandate to be completely carbon-free on our grid by 2045. Basically, no fossil fuel in our grid by 2045. And the electric company says they can make it. They might even make it early. But currently, the only thing I'm aware of, and that's looking at things they have running through the Public Utilities Commission and projects they have going on with the university and other areas, is battery. And I can tell you that, doing the math, batteries are not going to be the solution. They're part of the solution. They're definitely part of the solution. The batteries even in cars are important. They're the things that can give you a quick reaction, your quick boost, just like ultra-capacitors.  But for long-term energy storage, for massive scale and massive power like we have in our grid, batteries by themselves just aren't going to hack it. They're way too expensive. They're way too big and heavy. They use way too much what we call rare earths, which are right now controlled by the Chinese for the most part. And also, hazmat. Lithium cobalt technology is very hazardous. It's flammable. It has thermal runaway issues. And if you haven't seen lately, Tesla's been sued for some of their wall units that they're putting.  You've got to kind of use the batteries wisely and use them properly and keep them in balance. Use the right technology. There's better technologies than just lithium cobalt. And cobalt itself is more rare than lithium on earth. And we'll probably run out of yet long before we run out of lithium. So we need to start looking at better battery technology. But for certain, there's no way that you can replace all of the huge amount of energy stored in oil and do it with just battery. Can't be done. So what's the magic on the grid? Why would we need to change the entire way the grid looks? And I'm not saying you have to pull up every wire and knock down every utility pole, but you need to reorganize the grid into smaller grids for a couple of reasons. Number one is survivability. When you have one big grid that's all totally interconnected and can't sever itself without falling down, you have a big problem. Because if you're attacked, the whole grid goes down. And if you've ever been through a large area-wide power outage like here on Oahu, the whole island or in the northeast when the northeast US goes down, the power doesn't come back in an hour or two. It comes back in two or three days. And that's assuming they just had an interruption in power, not a physical actual attack on their system where transformers and substations were knocked down. It may be from a storm where utility lines go down, they have to replace utility lines from an ice storm or a windstorm or a hurricane. But the reality is, if you can isolate those grids into smaller packages we call microgrids, the solution to balancing power and providing power is much simpler. The utility has several big problems, whether it's Hawaiian Electric or a big mainland grid. The first is that they owe the public clean, reliable power. You've got to have it. That's part of the contract. If you had brownouts all the time or if your power went out every day for an hour or two hours, you wouldn't even want to pay your electric bill.  And you just can't run a company nowadays and have that kind of service. They do everything they can and spend as much money as they need to to give you reliable power. And that's really tough to do when you have industrial customers, you have high density customers like in downtown Honolulu, and then you have people way out in the country that are at the end of a power line that may be only shared with two or three other people for a couple miles. It's a tough grid to balance. What makes it tougher is when you start bringing in wind and solar. Everybody thinks wind and solar are going to be great. Why? Because you don't cost anything. The sun shines for free and the wind blows for free. We don't have to buy any oil. Great. And it is. It's a great way to go. The problem is equipment that turns it into power still costs you money. You have to invest in it. We have to do that investment to eventually get to the point where we have stable prices for our power and the power starts coming down. The normal return on investment here in Hawaii for like if you took your house and pulled it off the grid is between seven and 10 years. You could put solar on your house and in seven to 10 years you'd have all your equipment paid off and the solar should work for at least 20 years and hopefully your batteries as well if you have battery backup systems. That means the last 10 years you should be getting free electricity and that should be our goal. But we're always faced with that price up front and that's why power purchase agreements are so good. But on the grid all that solar and all that wind when you're having to push it into and mix it in with the rest of the grid, it stabilizes the grid. The electric company hates that. When you hit about 25 to 28 percent of solar and wind on your grid it's too destabilizing to manage for the electric company. That's why here in Honolulu we're pretty much maxed out. On the neighbor islands the neighbor islands can actually do a lot more in terms of solar and wind because their grids are simpler. They're smaller. They have less customers. They're actually easier to control and they can actually absorb and make modifications to their system to take the wind drying off or the clouds coming over and covering the solar. So microgrids are important for a couple reasons. To help you control all that solar and wind and make it a little bit more controllable if you use the solar and want to store energy you can make hydrogen with any excess power instead of shutting down your solar or curtailing the power as they say. You can start making hydrogen and store energy and hydrogen and do it right there right there in your community and then turn around and put it in a fuel cell and put the power back into your community right there in your community so that you could use the power at night. Why would you do hydrogen? Number one the technology for making power from hydrogen is getting cheaper and cheaper and cheaper. The technology for splitting water into hydrogen is getting cheaper and cheaper. It's clean. There's no carbon if you have solar and wind making the hydrogen. You have no carbon from beginning to end. Zero and that hydrogen can be used not just for putting power back on the grid in your microgrid at night. It can also be used for your car, drive your car. It can also be used to cook with. You can use it just like natural gas and cook with it. So if you look at taking the grid and sizing it right sizing it you have a lot of ways that you'll not only get cleaner but get more efficient and more economical. You don't have to have power lines stretching across the coals or long distances to push power from Kahi Power Plant and Waianae all the way to Waimanalo when you have microgrids that can make their own power, produce the power that produce enough for the community, store it and put it back in the community at night.  You lose power. You don't have to have as big a spinning reserve because your smaller grid is more predictable just like on the neighbor islands. You know what your variations are going to be. You don't have to have a generator running inefficiently for no reason other than to cover a load. You can use other technology like flywheels and capacitors and batteries to take up those shock loads and you don't have to run a generator for what we call spinning reserve. There's huge technical reasons that a microgrid makes so much sense. The other reason is that when it comes to a hurricane or even a even a kinetic attack, an enemy attack on our country, we know it's almost impossible unless you're using nuclear weapons to wipe out an entire region of a country and if that happens, that's more of a problem than just your grid. But when you do have an attack on a grid, currently it can wipe out a whole sector. But microgrids, being that they're smaller and they're totally separated, they can be connected but they can also separate themselves, those grids can survive. They can survive a hurricane. Even the hurricanes we've had hit Hawaii, they don't take out the whole state. They take out one coastline or one half of an island because that's where the major winds hit. They'll take out a couple major transmission lines, bring trees down and things like that. But if you have microgrids all separated, we could have a major hurricane hit Honolulu and we could have virtually three-fourths of the states still have power, still making ice, still making, you know, power and providing energy back to the people that don't have power. That's what we should be looking at. It's a survivability piece. And of course, it all gets back to sustainability. Here in Hawaii, particularly, we don't have oil. We don't have natural gas other than the methane that we get off our landfill and off our wastewater treatment plants and Hawaii gas is already tapping into a lot of that and that's their plan for the future. They're already tapping into whatever methane we have here in Hawaii.  For a coal plant that's burning coal and for oil turbines that are making electricity with oil. Huge, huge carbon issue and huge waste of money. Here in Hawaii, there's just absolutely no reason why we couldn't survive just on solar and wind. But we have more than solar and wind. We have geothermal, not just on the Big Island, on Maui, on Oahu. There's geothermal. And there's technology now where you don't have to use the deep drilling geothermal that we're used to. You can actually use a lower grade of heat and still generate a lot of electricity. There was a project, most people don't remember it, but on the Big Island there's a project called SOPAG and they had a engine that ran off differential heat. They had solar concentrators that were heating oil to 500 degrees Fahrenheit. That's huge. That's a quite spoiling point. You could run steam generators from that. We could even use solar thermal energy, not solar panels, but solar thermal energy to generate heat. So there's no reason why we can't all start moving towards a cleaner, more sustainable environment. And we need to understand that electricity is going to incorporate encompass not just our grid and what we use at home, but our vehicles as well. And we need to understand that our grid needs to change in dramatic ways, but not necessarily huge infrastructure changes, but structure changes and how we do it. And if we can get there and we can do that, we'll be way ahead in the state of Hawaii and we'll be way ahead in the USA. So that's going to wrap it up here for Stan the Energy Man. I appreciate your time today. I appreciate you listening to me kind of ramble on, but this is important stuff. You know, climate change isn't going to solve by itself and you're part of the solution. So make yourself part of the solution by becoming more energy aware, get rid of that energy blindness like Dr. Hogan says, and start making the change yourself. You be the change. Aloha, until next Friday.


Who is Polo G?  Not only has rapper Polo G been leveling up in the hip hop scene lately, he's also been upgrading his real estate. He had bought his mom a home in Atlanta back in February and now had the money to treat himself to a nearly $5 million dollar villi-style mansion in the Chatsworth area of Los Angeles. Today we'll check out the young rapper's impressive starter home which is full of some crazy features. In these videos we don't reveal any addresses and even though I've done a house tour of my own place, please do not show up at any private residences because it's not safe for anyone. Taurus Tramani Barlett, better known to the world as Polo G, is a rapper, singer, songwriter and more who rose to fame in 2018 with a couple of hit singles. After two albums, this year he released the single Rapstar, which was his first number one hit on the chart. In addition to Polo's success in the music industry, the 22 year old has also launched a basketball team in his hometown of Chicago to help keep youth out of trouble, made the 2021 Forbes 30 under 30 list and even launched his own clothing line. All things considered, his estimated net worth is at about $7 million dollars and he is clearly doing well after treating both his mom and himself to beautiful new properties. Hey everyone, I'm Cara the Vampire Slayer and I'm bringing you another exclusive house tour here on Famous Entertainment, this one featuring the homes of rapper Polo G. If you haven't already, make sure you subscribe and ring that bell for notifications because we post brand new content daily. As always, you can follow me on Instagram to chat and let me know who you want to see featured next. Now let's get into this video. In recent news, Polo G just celebrated his success by purchasing a starter home and by far one that would exceed any expectations in the form of a mega Mediterranean style mansion costing him $4.9 million dollars. I think it's rare to find any 22 year old doing so well to purchase this and an impressive Atlanta house for his mom. Located in the Chatsworth area of Los Angeles, his latest acquisitions stood in almost two acres of land in the Santa Susana Mountains. Chatsworth is a suburban neighborhood in L.A. which once upon a time was home to Native Americans who left caves containing rock art. Polo's house is next to famous neighbors like DJ Mustard and Swaley, more specifically in the 24 hour, guard gated community of Indian Falls estate. Built in 2018, the rapper's new mansion has so much potential whether it'll act as a party pad, family home, or private retreat. Inside, it stands over 10,800 square feet of space with six bedrooms and eight baths throughout as well as luxury bonuses like porcelain tile floors and beans and coves ceiling. If you want easy access to the resort style grounds, there are plenty of walls of French doors that create an indoor-outdoor feel. Walking into Polo's new mega home, giant wrought iron doors greet you and take you into the rotunda entry with soaring ceilings overhead. Of course, the dramatic ambiance is made even better with a sweeping staircase and fancy artwork. The wooden staircase leads to a loft-like space on the upper floor that overlooks the entryway and near the foyer, there are grandly sized living areas including a light-filled great room. Here, there are double height ceilings with large windows, a fireplace, and plush velvet couches. Plenty of options for the rap star to hang out. Also on the main level of the mega mansion, there's a formal living room with custom wine room and bar, formal dining room, games room or billiards room and guest suite. Polo G is only in his early 20s after all, so an impressive home like this looks perfect to throw some wild get-togethers out with famous friends. I don't know if the rapper cooks, but the center of the mansion has got to be the pro chef's kitchen with sprawling center islands and top-of-the-line Viking appliances. The upgraded space has granite counters, an onyx backsplash, and some cute and stylish bar chairs. Moving to the upstairs of Polo G's new villa, there are six bedrooms, including what we can best assume is his master-treat, fit for a king. The master bedroom boasts a walk-in closet with custom built-ins to organize any designer wardrobe and an en suite with double shower and a separate circular jetted tub in a rotunda-shaped corner of the room. I wonder if that soft-looking furry rug comes with the shower too because it's a nice little touch. If that's not enough, Polo can pop out onto the private balcony and take in the gorgeous views and the grounds of his new home. Outside on the grounds, the rear of his mansion is lined with plenty of covered patios and outdoor dining spaces, leading to the pool and other amenities. The yard is perfect for entertaining with stone terraces and the massive swimmer's pool with spa, while elsewhere there's a lit tennis court and a barbecue station. I hope that Polo G has a car collection or is planning on starting one because a definite feature of this mansion is the oversized garage, which looks huge from any angle and has room for about 14 cars. Let's not forget that before buying himself a lavish pad, Polo made sure that his mom was taking care of. Earlier this year in February 2021, the rapper took to Twitter saying, Oh well, I just bought my mama a crib bigger than mine, WTF. His mom, Stacia Mack, who looks younger than ever, by the way, showed off the home proudly on Instagram, explaining the promise he made to her at the start of his rap journey as well. Her caption read, He promised he'd create a life for our family free of struggle and or lack, she wrote, expressing her approval of him delivering on that promise. Today, my son purchased my dream home. I showed him the home and he didn't flinch. His response was what you need. The same appreciative is an understatement. Thank you, son. Thank you for being a man of your word. Thank you for giving me things that at times I wasn't able to provide you or myself. I love you and I'm eternally grateful. While we don't know exact details of the home, we do know it's located in Atlanta, Georgia and boasts a spacious kitchen with marble island and luxury details. There's also a sweeping carpeted staircase in the double height entry, which Stacia loves posing on. All right guys, so that wraps it up for our polo G house tour. We checked out his brand new villa style mansion in Chatsworth, LA and more. So what did you guys think? What was your favorite amenity at the rapper's estate? Which feature did you like better? The resort style backyard or the great room with double height ceiling? I think I love the resort style backyard in pull-bast, but it's hard to choose since his entire home is impressive. I've also been reading all your comments on the channel and I'm gonna be shouting you guys out. This one was from our real housewives of Beverly Hills, Doree Kemsley's house tour. I really like Doree on real housewives of Beverly Hills. She's pretty, I loved her accent and she has a great style. Amazing how much she looks like a blonde Kim Kardashian in the beginning of this. Nicely done as always, thank you. Glad you liked the video, Heidi. I think she looks amazing too, as well as her style. And on our Erica Jane house tour, Elvis Humphrey Road. Do a Lisa Vanderpump house tour among all the housewives. She has had the best and lavish home. I actually just filming that one and couldn't agree more, so keep your eye out. If you guys want me to shout you out, be sure to drop me a comment about Poloji's homes down in the comments. Thanks for watching. Don't forget to follow me on Instagram and I will see you all in the next video. Bye.


Yeah, I'll go ahead and get started. It's about 2 p.m.. So Charles Crouch was going to present with us today But he had a family member issue a health issue so he's not gonna be able to make it So it'll just be myself. I'm Steven Dake, and this is Danny and Hansen want to say a little bit about yourself Yeah, sure. So Danny and Hansen software engineer with with Cisco and been working with OpenStack since Diablo and have been focused on containers for Just about the last year. Yeah, and I originated heat about four years ago the heat project in OpenStack And now I work on containers pretty much full-time. I work on Kola Magnum Quaran Magnum ptL of Kola So that's what I'm interested in now and focusing on I'm really excited to talk to you about what we've done with Kola and that's what this talk is about So if you want to know what you're so there's there's no more real integrator releases in OpenStack Because of the big tent model. I don't know how familiar everybody is with the big tent model But basically what it allows is for everybody to kind of do their own thing in OpenStack and Compete with different projects that do similar things which is kind of an interesting concept and What that means is that there will be more projects going into OpenStack over time In my opinion You know a technical committee has has oversight over what projects enter the OpenStack get namespace But I think they're gonna be pretty loose around what they allow Just as long as there's an active community maintaining the software so if we look at the last release kilo there was four new projects that were added to the OpenStack namespace and There were 12 projects prior to that So if you look at they do the math like 30 40% growth, I think we're gonna see that every cycle So that's that's hard for distros and deployers to deal with because it's a lot of churn a lot of new code A lot of stuff to integrate and test It's just really difficult so it makes deployment difficult because you have to manage the deployments and Managing those deployments means you have to either some kind of puppet System or ansible system or chef system to manage your deployments today That's how pretty much everybody does things and as you add these new services because of the big tent What do you do? How do you manage those services? Does your distro provide those services? so it becomes more challenging to manage the deployment of OpenStack as time progresses because of the big tent and So this part's really important and I'm gonna get a lot into detail about this later during the presentation But we convert Cola Cola is a project we use to implement containers in OpenStack We convert everything into an atomic unit that does one thing does it well It includes the environment and by converting it into an atomic unit. It can be easily managed and deployed So that's what we really do in Cola So we heal the deployment pain that everybody has We don't actually do deployment yet, but we might Other projects do deployment. We did just do container content at the moment But we might tackle that in the future. So I'm on it So I saw I talked about the cola does simplified deployment. Well, how does it do simplified deployment, right? Because we we have a set of key value pairs. That's what you deploy you deploy a container with key value pairs You maybe you've got an API service and maybe you've got a conductor service And maybe those are the two services that you've got to deploy for your new service that you want to add to your system All you have to do is deploy those with key value pairs now to do this with puppet and chef puppet Chef have key value pairs just like Docker does with an environment variables The thing is puppet goes out or chef goes out and changes a whole bunch of stuff on the system and configures everything and That's not really ideal. Whereas in Cola, we don't we don't have that problem We we simplify the develop the deployment because we just deployed as one unit Instead of a bunch of stuff all over the place just one thing instead of a bunch of things So think about that for a moment and multiply that by the 12 services that are in the kilo release Times the four new services that were added to kilo and multiply that out by the growth rate of open stack because of the big tent So we simplify deployment if you don't buy my argument. I can't convince you Apologize for that. So simplifies ongoing ops operations is difficult right because You have to manage these key value pairs and keep them up to date For whatever version of open stack you're running We tackle that problem as well The way we tackle that is we stay on top of our container Content, but I think other projects do this well, too. I think the puppet Modules do this well in open stack. So I think that's really positive This is not something that's unique to cola, but this is something we do well now when you go from Kind of developing your software and then you want to deploy If you look at how things are done today Typically somebody deploys a whole bunch of software on a system and then they got to manage all that software on a system We deploy each little atomic unit their container of microservices. We put them put open stack in a microservice We put our apis in one and we put Maybe the engine or conductor or whatever it is and another for each service we've got about 40 microservices implemented and maybe 30 something like that and By doing that You can you can break apart those microservices into different teams managing each part individually This is a key key thing with microservices This is like the number one reason why you want microservices because you can break apart those 30 people all working on something separate Into separate things and it doesn't matter anymore that this has to integrate exactly with this So if you look at the puppet model, you've got puppet It's got you get a set of deployment scripts for puppet It's all got to work together with microservices. You can deploy different versions different Different from source or from our PMs or from Debian You get that whole model One of the really cool things about containers is a really reliable bass They just rock and we're gonna show that in our demo. Hopefully I feel comfortable about that. Can you move on to the next slide? Okay, so we leverage the power of containers. How do we do this? So we use container best practices that the Docker community is defined So what are what are these? How do what our container best practices? I'll give you some examples So we use data volumes instead of bind mounts if you know anything about Docker You know that it can mount data on the host operating system It can also mount a container that contains data So all of our persistent data things like the database and the glance information All that's the Nova compute VMs. All those are stored in data containers Those data containers can be backed up and restored individually of the rest of the system and the data container is not persistent It's not running it you launch it once and then it creates storage on the system And then that storage is permanent Unless you delete the container now if you came along and delete the container You would lose your persistence just like if you deleted your database So there's not really any more risk of damage your system. You can still back up and restore Now why do you want this because it gives a mutability so containers should never change except for the environment passed into a container and How that environment configures the container that's immutability now immutability is like this big computer science nerdy word Which is basically means we don't change anything. We don't change the packages. We change the configuration a little bit But it's consistent and the cool thing about immutability is that it makes an imperative system Which open stack is imperative. It's you've got a set of instructions. They're their conditionals They make decisions and you have a different tree that outputs. That's an imperative system Cola and containers if you use if you have a mutable system become Declarative so you get one of two outputs either the container creates or the container does not create those are the two results you get So that's a very critical feature and that's what you get by using Docker best practices. So that's very important I What else is there? Let's get one. I think also is Is you know the microservices approach where we're running a single service for the most part and in the containers so So that would be another best practice and right on the last best I know there was one more item potency. So that means a container can start and restart Without changing the System or implementation so we want item potency in the containers and we have that today The item potency is very important for us. So those three properties of a container Those are best practices and the way we get those is through data volumes We don't use a main bind mounting unless we're absolutely required to do so Some of the things we do with cola Some people wouldn't like for example, we use this thing this fly calling that host Which means we turn off the network namespace isolation of the host But we're running open stack on bare metal. That's our goal running open stack on bare metal without any other services We want open stack by itself. So that's okay for us We turn off sometimes we turn off the PID namespace for Libvert for example. So VMs can be reconnected to when you upgrade. This is another thing we do That's kind of weird that most people wouldn't like we had to add this Docker compose, which is one of our tools our orchestration tools But it's really cool. It's a nice feature Finally so container orchestration our orchestration today is a shell script Not ideal not really not ideal Monday. We had a session with the Ansible easy as well the day and Our answer will collaboration day and we kind of talked about what the different ideas were and I think the community has kind of agreed that as well as a way to deploy and Containers are a way to do that What we're not sorted out on is exactly what the best practices are for that to occur So there'll be more in the future. I think at the at the next design cycle We'll see exactly what this container orchestration is, but we're just getting started there We just kind of have a shell script. We've got an all-in-one Ansible system. That's about it Okay, this is architecture diagram, and I gotta get down so I can Talk a little bit about this so infrastructure engineering These are the guys that are we're girls ladies that are writing the software. They do infrastructure engineering They submit their code to Garrett And it goes into a CI system. It's reviewed the CI system reviews the code Then the CD system once it's reviewed and approved the CD system will Feed the output of the build For our open-stack containers that are based on COLA into a registry So it goes into a local registry that runs on in the environment of the system. It also Launches Ansible. This is kind of like a future thing. That's what we think we want to do in the future the launch is Ansible which contacts Docker and Contacts compose and basically launches our system multi-node with HA. This is our basic architecture Now you see some blue and green differentiation The green is stuff we do so we're going to maybe do some ansible scripts. I'm not really sure We definitely do compose stuff now and we do the open-stack service containers all the blue stuff It's this open source that everybody can leverage Okay Time check 213. Yeah Okay deployment flexibility, so this quote is great about DevStack I hate DevStack because it destroys my machine. I think if you're an open-stack developer, you've learned to accept this You get deployment flexibility with open-stack or with COLA where you don't have to have your whole system destroyed You have to install compose you have to install Docker those two things you have to install It doesn't install like a million dependencies containers Package the environment together with the software and this is like a key thing They're like an atomic unit that can be upgraded and downgraded. So slide performance okay, so Charles Crouch did these numbers. I haven't verified them myself.  So just take these with the grain of salt But I think what it showed no, no, yeah, I think what it shows is that COLA It starts up faster. This is how long it takes to get to a working environment call It takes nine minutes just that takes 14 minutes pack stack takes 44 minutes. That's kind of you know, it's faster Next slide Okay, so Docker is really green It's young, you know, it was released in June of last year. I really like Docker. I think it's cool technology It's not new. I it's not really a new idea But some of the ideas they've done if you compose other people's ideas. So it makes it powerful The first version of Docker I would recommend using is Docker 1.6 Every other version of Docker has been broken in some various form In fact, we recommend using Docker 1.6 with COLA because Docker 1.5 will lock up Okay, Docker uses threads. This is kind of I don't know an anti-pattern at least in my mind But it is what it is so Docker's it's young but the community rocks I worked with community like for six weeks on COLA upstream I talked to him day in and day out and they're really responsive They are the type of community you want in an open source project. So, you know, Docker's green and young They're on the ball COLA we're like totally way younger than Docker. We're like six months old So there's not really not much to say there other than we're missing services We need to do more work to kind of finish a job mostly cross-platform so we run on Ubuntu or rel and you can use or not rel CentOS and you can use our containers are based on CentOS Now the problem is when we did that we use CentOS on Ubuntu There's kernel bugs with like PAM With odd with the auditing system. So we had to like hack around that So so like the kernel is not totally consistent between different versions of distros because it's a different version of the kernel Even though it's mostly the same upstream a little different So not all the containers work exactly the same that said COLA does work perfectly on either Ubuntu or Fedora And I assume it would work on rel and it does work on CentOS Different deployment models new bugs. So, you know, we're doing something new. We're gonna create new bugs So it's pretty straightforward next slide. So not ready for production. We've got no high availability We do have plans in the community to resolve this with the blueprint that Daniel has worked on Operationally immature. We have no way to deploy COLA operationally. That's a problem for us And then the reason it's promised because we can't do future development Because we need to be able to do be be able to deploy on multiple nodes To be able to do HA to be able to test the HA works correctly. So we need to have that deployment in there To do that to do HA and to do multi node We need those things in order to continue our development really hard to audit. How do you tell what's going on in 30 containers? I think this is a solvable problem, but we haven't solved it yet So we don't really have a configuration management tool more have like a key value pair, which is fine works for us I think it's pretty solid. I think it's a good model. It's pretty simple The downside of it is there's a limited set of configuration options in what you can configure It's not like you can go set every configuration option to whatever you like I think this only really matters at least now for the neutron container the other containers Probably work with a kind of opinionated deployment model Okay, this is Daniel's part the wide demonstration now This is such a great diagram here Daniel drew this on it and he texted it to me and this is how I should set up my home network to get neutron to work This is it right here and this works so you can copy it at home and Daniel I'll let you describe it a bit and then you can get to the demo Yeah, so what you see here in the in the diagram really just kind of focus on where you see cola host and so in the demo and What we focus on right now is just getting cola working on a single host and This is Steve's home lab environment if you look at the cola host We have two interfaces one interface is used for management. The other interface is used by neutron to connect to The external or public network doesn't have an IP address that is kind of where your floating IP addresses come from What where neutron bridges between the tenant network and and the public network? Let's actually jump into a system Yeah, I was gonna say maybe make it how about I do this Make the lettering a little bit bigger. Is that okay? Can everyone see I think bigger would be better, okay? That's good. Let's see here All right, so here's a big iron. So this is you know the single system that you saw in the diagram and This is our our base Operating system that we're gonna go ahead and deploy all of our containers on top of and so just kind of a quick recap is Traditional open-stack deployments, right? You've got different options from deploying on bare metal, which you know if we started installing The different open-stack packages and configuring the files and so forth then we're installing on bare metal Or you may install on a virtual machine and that's where cola comes in We're trying to just change the game here and actually leverage containers install these And run these open-stack services in containers This is a fedora 21 system, but long-term Where we want to focus is more of the micro os types of operating systems. So coral s Atomic and so forth Let's see here See we've got no containers running. So basically on this operating system. We installed docker docker compose and We're ready to start using it, okay Let's take a look at a couple things here So this this gen n is just a wrapper script around docker compose so docker compose is one of the tools that we're using and It's a basic composition tool, right? So so surprising by docker compose, but it allows us to define very simply What groupings of containers that we want this system to run So it's just a nice abstraction layer and then we've built this gen n tool That wraps around docker compose so we could simply for development purposes start all the cola containers stop all the cola containers kill the cola containers and what it I'm sorry the gen n tool actually creates all Of our environmental variables, right? So Steve talked about really the only interface as a user you have is A bunch of environmental variables that feed into the containers. So when you start a container Using docker compose or using this wrapper utility. It's calling an open stack dot em v file That's the entire list of your environment variables. Okay So let's go ahead and instead of generating them. I've already generated them So let's just take a quick look at the output of the open stack em v and again Here's all the environment variables with with the settings that were created from from the tool and Then we have another tool our cola tool and this Is a tool that actually wraps docker compose So if you look at this tool in the source code, you'll see it's just basically wrapping Docker compose so that we can start stop do whatever we want with all the cola containers that make up the environment so Let's go ahead and kick that off and It's going to go ahead and start Creating all of our containers We'll give that a second here and all the containers Will be created and then the next step will be to actually configure the open stack environment because although all the services Open stack services are up and running Within these containers. Nothing's really configured or such as there's no glance images There's no neutron networking so on and so forth And so the next thing we would do is again built a simple utility that goes in and sets up a basic configuration for For development purposes Sure, so let's just take a look really quick and and verify things are up and working. So let me source my credential file and I could do a keystone and point list Glance image list got no no glance images Nova key pair list neutron extension list So we see that we're able to talk to a neutron server Let's do an agent list and make sure that our neutron agents are up and functioning and one thing just to point out too is right now The the plug-in or the agent that we're using for L2 is is a Linux bridge agent I believe there's a work underway for supporting open V switch as well But we see all of our agents are alive as well so so far so good, right, so let's go ahead and Run the init run once tool again this tool Does some basic things downloads a glance image starts configuring our neutron network subnet router attaches that router to the gateway modifies some Nova quotas because we start actually spawning and in some of the demo scripts that we have quite a few Nova instances, so we need to modify the quotas and Give it a second. It's done. All right, so now we could actually do a neutron net list and see That we've got a network or a neutron router list So we've got a router right, so let's go into our demo heat and take a look at what's here and We've got a launch script and basically what it's doing is simply just Creating a heat a heat stack and the heat stack is made up of a Resource group and a resource group is a special type of resource within heat that produces X number of Identical resources that are defined within a particular resource So we called the resource stake and we said a count of 10 of these resources Well, let's see what this stake is all about right so stake is going ahead and creating Nova instance neutron port and a Neutron floating IP so very simple right let's spawn 10 of these Nova instances and then attach them to a neutron network and and also give that instance a floating IP So let's go ahead and launch it and it will take a few minutes to actually start spawning the the instances We can do a novelist and see if any instances are starting to spawn You see that they're in a build state So it's gonna take a few minutes for for them to build out when they do build out What I'll do is is actually get into one of the instances So you could see that it's not a bunch of black magic that stuff's actually working now One thing I'd like to point out if you could show us to the docker ps. It's real quick These are the open-sack services running in containers. It's not bare metal with puppet or chef or anything like that These are the containers that are providing all the services that Daniel is demonstrating. Yeah, and it's cool from a development standpoint That I can build test and then just blow away the containers Rebuild test. I mean this this demo right here. I've probably rebuilt literally 30 times in the last few days Just trying to prepare for For today and so, you know, that's pretty awesome to not have to mess with vagrant You know not have to actually deploy on bare metal and then you know rebuild my system and and so that From a development standpoint is pretty sweet. Let's see. They're still in a building state. So it may take another couple minutes here But we see that some of the instances are actually starting to get their IPs and speaking of this so Let's see if you can elaborate on it But there's there's a bug that we ran into with with Nova where Nova lists Nova show when you spawn Tons of of instances using heat actually doesn't display the networking information. Yes Let's see here, so let's do a pseudo So you see here, let's see Nova compute for example So I could do a pseudo Docker logs and then the name of the container or you could specify The idea of the container and there you go It's gonna spit out all the logs here another thing that you can do is actually jump into the container and Then you know tail your bar log Nova compute and see the logs there as well Let's see if any instances have become active Yeah, so we do have some instances that are active and here's one, you know with an IP. So should be pretty easy I could go ahead and ping the public IP SSH Into it and then even like curl Google so great now. We've got everything up and running. I've got an operational open stack environment pretty awesome but one of the you know, one of the things that's very attractive about Docker and just the whole container approach is what Steve talked about is, you know, the immutable server in the sense that We've got these services that are immutable for each of the different open stack services and Now if I want to go ahead and upgrade a service I can go ahead and go into my container update a package make a change whatever it is Rebuild that that image give it a different tag Let's say instead of ice house. It's now kilo and then I could go ahead and very simply take down one container and Bring the new container with a new tag up if things don't go well with that upgrade I then go ahead and shut that new one down. That's broken and just bring the old one back up So basically talking about upgrades how easy it can be to upgrade open stack services And also because these services are now split up into these atomic units. I can upgrade them individually or you know groups of different services So that's that's You know, it's very interesting from an operational standpoint and let's actually take Take a quick look. So if we go into Docker, we could do Nova compute Nova Libvert And here's basically the start script for for Nova Libvert and I'm gonna do a pretend upgrade and just do an echo this is an upgrade and Then so he just pasted it at a hundred times. I think or something Yeah, just so it's easier to see when we when we take a look at the logs again And again in a reality that would be I'm going in and changing a package and I would probably go ahead and tag it with a You know test or just some kind of unique tag so that I still have my old Nova Libvert image And I have a new one and then I could start playing around and testing back and forth and making sure that Nothing's broken So now what I'm going to do is just simply Take down the Nova compute Nova Libvert Nova compute data Containers Steve mentioned earlier about that concept of Using data containers instead of doing doing union mounts to the host and That's here's one example of us doing that Let me actually do it. Okay so recreating the the containers And in doing so and that recreate it then goes and says oh, let me get This new image that you just built with all the this is an upgrade echo So what just happened here is an error with docker 1.6 and the PID equals host operation and the LVM driver This has been fixed in master By using the overlay FS driver this error It's hard to see this thing Driver device my bird failed that's okay because we can just repeat the operation and it'll go through But you know doctor 1.7. I think will be where we like Cola and docker together So just quick recap. I just showed you how easy it is to to go ahead and Get an open stack environment up and running on top of containers Configure that open stack environment and then actually, you know walking you through how to to spawn a bunch of VMs using heat as well as the last piece right here is doing that upgrade So if you need to upgrade live vert You know how easy it is to go ahead and modify something to the image rebuild it and then Take those running containers and have the you know one or more of those containers use that new image What was that last command you just ran? Dr. Logs Dr. Logs, right? Okay Good any questions I Can't really see because I'm blinded by the lights So if you just want to step up to the mic and ask away that would be great I have a question regarding the tagging in this demo. You're showing the container can be tagged So that you can roll back and roll forward a particular service. What if I want to? Identify my open stack cluster Having a specific set of services with a specific set of multiple tags So how do I tag all the containers together to one song single thing? That's a really good question and I think the answer is we haven't solved that Identifying you're talking about version management of an entire cloud, right? And I don't think anybody's really solved that even the puppet guys I don't think anybody saw that very well Maybe some of the distros have solved it in a decent way because they only allow to deploy from their latest version But yeah, we don't solve it here. Yeah, and I think with the microservice approach It's almost like the reverse of that right so, you know now it's instead of this open stack cloud it's really just a collection of services and and If I want to upgrade one service or the other or back out of that upgrade because it fails Well, then we're just using them the tags associated to those images, right? Yeah, the main concern is about the compatibility between the two services of like we upgrade one of the service But it's not working or compatible with the other one So once we find out the compatible set how do we retain that set information? That's where I was yeah I mean I would just see that as like a separate tool that's able to go ahead and soon as you say Okay, this is the environment that I want that could potentially go out there and look at all the you know the commit the commits or the tags of you know of your containers and images and so forth and and then Kind of create a snapshot of the environment from there, but that's not part of a not part of the project I get a question. I get a question. I'm not sure if you already covered. How are you restarting the services if the container dies? If the container dies we use restart always the Docker feature. Okay, so you simply start always yeah Have you tried out using it like the upstart or system D? No with the with the a flag We haven't tried it I think restart always works pretty well the problem with that is if Docker fails then you need to restart Docker, right? Okay, okay. I have a quick question.  Solution or is it's going to be integrated with triple low? It started out under the triple umbrella So I'm just curious, you know, you know, what is what is your vision of this project? Where do you see this? That's a great question. So will it be integrated with triple? I think the answer is yes I can't commit to that because I don't involve myself in the triple low community because I don't know anything about it I think it's very likely Will we expand beyond our current kind of objectives? Single-node container content probably to actually finish the job because we need to be able to do ha do ha We need to do multi-node and I don't want to wait on triple o to deploy our containers before we actually have ha I want to have a j ready for triple o prior to that so Yeah, and to that point. I mean just a couple things on on the roadmap, you know completing the service implementation right now We are lacking some services for example sender I think that's going to be completed here soon But just generally there's there's services that that we don't have that we'd like and and with the new big-tent model There'll be more and more services that we need to keep up with on the plus side though is that These container images are really easy to create so you spend a little time look at the code and how You know how what the containers look like and it's it's a very simple to go ahead and create the containers for new services Steve touched on high availability. I put together a spec out there. There's some feedback that I need to review and hopefully we can Actually move forward with with ha here Installing from source right now. We're installing from rdo packages, but we understand some operators Don't want to wait around for you know package for packages and actually build their own packages based on source Again ansible playbooks right now We have a sink we have one ansible playbook there to do a single all-in-one node pretty much just for test dev kind of purposes But to expand that potentially for our multi node Purposes But again, you know cola simplifies the new big tent You know with all these services that are going to be coming within open stack It's much easier to create You know images for each of these services within Docker than it is to You know to go about it in other fashions You know and just to kind of get straight to the point is you know, we could really use You know your feedback or or you know getting involved in the project You know, I'm not sure if anyone's kind of picked up on this whole kind of container You know messaging that's going around the summit But you know, this is a project that we're getting a lot of interest I would say our communities what doubled in the last six weeks So we'd love to get more people join The the project and there's no shortage of work. So you could do some good work Do some good work on the project we have to conclude there because we're out of time. Thank you. Yes