All right. Hello everyone who is on the line. Good afternoon, good evening or good morning. Depending on where you're joining us from today, welcome to Engineering for Change or E4C for short. Today we're pleased to bring you this month's installment of E4C's 2018 Off Grid Energy webinar series focusing on why the power grid is in everywhere and particularly the role of grid extension and electricity access. My name is Yana Aranda and I am the president of Engineering for Change. I'll be your moderator for today's webinar. The webinar you're participating in will be archived on our webinars page and on our YouTube channel. Both of these links are included on the slide in front of you. E4C members will receive invitations to upcoming webinars directly. If you have any questions, comments or recommendations for future topics and speakers, please feel free to contact the E4C webinar team, series team at webinars at engineeringforchange.org. Also, if you're following us on Twitter today, please join the conversation with our dedicated hashtag, hashtag E4C webinars. Before we move on to presenters, I'd like to tell you a bit about Engineering for Change. E4C is a knowledge organization and a global community of more than 1 million engineers, designers and development practitioners who are leveraging technology to solve quality of life challenges faced by underserved communities. Some of those challenges may include access to clean water and sanitation, sustainable energy, improved agriculture and more. We invite you to become a member. E4C membership is free and provides access to our news and salt leadership, insights on hundreds of essential technologies in our Solutions Library, professional development resources and current opportunities, such as jobs, fellowships, funding calls and more. E4C members also enjoy a unique user experience based on their site behavior and engagement. Essentially, the more you interact with the E4C site, the better we will be able to serve you resources aligned to your interests. For more information, please check out our website and sign up. So, today's webinar is the second in the Off-Grid Energy series that we are sharing with you over the course of 2018 and 2019. Additional topics covered in the series are drawn from the book titled, Off-Grid Electrical Systems in Developing Countries, authored by our presenter, Dr. Henry Louie. The future webinars in this series are listed on this slide and will also be announced via our newsletter. The previous webinar we had in this series is already available on our site as a recording, so if you missed it, you are welcome to go back and listen to it. That is the one on energy access and requirements of rural communities. E4C members will receive the information about all the webinars in this series directly in their inbox. So, definitely a good motivation to sign up. For reference, you can find examples of off-grid energy products like the Mobisol Solar Home System in the E4C Solutions Library. There, you can learn about technical performance, compliance with standards, academic research, and user provision models of the system. All the information around these solutions is sourced by E4C's research fellows and reviewed by our community of experts, and it's available to E4C members free of charge. There's a link showing where the Mobisol Solar Home System is available on our platform on this slide. Now, as always, we'd like to make sure that we all understand how to use this platform, so a few housekeeping items. Let's practice using the WebEx platform by answering a question of where you are in the world. In the chat window, which is located at the bottom right of your screen, please type in your location. If the chat is not open in your screen, try clicking the chat icon at the bottom of the screen, which is about in the middle of the slide. So, I'll go ahead and get us started. You should be seeing my location pop up. All right, we have folks coming in here. So, I'm in Brooklyn. We have folks in Minnesota and Nashville. I do see some folks answering this in the Q&A window. So, we have folks from Los Angeles and Nepal. Do try to use the chat window, as this is the dedicated window that is going to be for sharing remarks during the webinar. As well as sharing any technical questions, you can include them there or just send a private chat to the Engineering for Change admin. During the webinar, please use the Q&A window, which is located below the chat, to type in your questions for the presenter. That way, we can keep track of those. Again, if you don't see the Q&A window, just click the Q&A icon at the bottom of your screen around the middle of the slide. So, welcome to everybody. Wow, we have a really diverse audience today from across the United States, along with Nepal, Ghana, Puerto Rico, also part of the state, Tokyo, Indonesia, the South, Phoenix, Arizona, also part of the state. Very cool. Thank you so much for joining us, everyone. We're so excited to have you here. If you are listening to the audio broadcast and you encounter any trouble, try hitting stop and then start. You may also want to try opening web effects up in a different browser. Now, E4C webinars qualify engineers for one professional development hour or PDH. To request your PDH, please follow the instructions at the top of the E4C professional development page after the presentation. The URL is listed here. All right, so we will try to keep today's webinar short as it's a little bit more digestible, approximately 45 minutes. So hopefully, you'll get everything you need in that time. And now, a little bit about our presenter. For those of you who don't know him, Dr. Henry Louise, an associate professor at Fort Francis Wood Endowed Research Chair in the Department of Electrical and Computer Engineering at Seattle University. His research areas include electricity access in developing communities, renewable energy, and appropriate technology. He's the president and co-founder of Kilowatt for Humanities, a nonprofit organization providing electricity access and business opportunities in Sub-Saharan Africa. Dr. Louise served as a Fulbright scholar to Kapa Belt University in Kichwe Zambia. He is recognized as a distinguished lecturer of the IEEE and is an associate editor of the Journal of Energy for Sustainable Development. He's the author of the book, Off-Quality Electrical Systems in Developing Countries, as I mentioned earlier, which was published by Springer Nature and we are benefiting from his insights today. With that, I'm going to turn it over to Dr. Louise. Okay, thank you. I hope you can hear me fine wherever you might be around the world. So today we're talking about grid extension. I was giving a presentation a few years ago on some of the work my nonprofit does in off-grid electricity access in Zambia, and I was describing the statistics of electricity access that a billion people don't have access to electricity around the world. And I was interrupted with the question from the audience and somebody from the audience just said, well, why not? Why is it that a billion people don't have access to electricity? And it struck me at that moment that I think a lot of us practitioners that are in the off-grid space don't really think about why we are trying to solve the problem we are trying to solve. You know, why is it that after more than 100 years of power engineering development, the grid hasn't reached everywhere? And so that's really the motivation for today's talk. We're going to understand why that's the case. We're going to go through a few ways to compare on an off-grid solution and so forth. So let's go to the next slide. So just a background about me real quick. I teach at Seattle University. We're a Jesuit University located in the United States. My nonprofit organization that I work with is called Kill Lottery Humanity. And in the past I've worked for IEEE Smart Village and I've been a Fulbright scholar. So the book that today's webinar is based off is called Off-Grid Electrical Systems in Developing Countries. You can get it on Amazon, hard copy, electronic copy through the publisher Springer's website. And there's also a link on my website if you're interested. So today's talk is really on the third chapter of the book. The previous webinar is covered the first two. So we'll be talking about grid extension and enhancement. And you see that there's a lot of other content in this book. We'll go into far more detail in the book than I can cover in the next 30 minutes. So if you really want to get into the details, then you only can download the appropriate chapter in the book. So why do we care about grid extension? It's a good question. And I think the answer is that it is by far the most common way that people gain access to electricity. If you look at in terms of investment, if you look at it in terms of the number of people connected, grid extension is really the dominant mode. At least presently it's a dominant mode of providing high-tier electricity access. In many cases it can be faster and more cost-effective than off-grid systems at providing high-quality electricity access. So more than just off-grid lighting, but providing electricity access in a way that can power stoves and washing machines and so forth. And then I think even if you don't care about grid extension, but you care about off-grid, many of the technical concepts and economic concepts of grid extension actually do apply to at least larger off-grid systems like mini-grids that might have their own distribution system. So the learning outcomes for today, we're going to be talking about grid extension and its role in universal electricity access. We're going to talk about the different factors that might make grid extension preferable over off-grid. We're going to try to understand some of the basic considerations when we design distribution systems. And then we're going to talk about the economics of grid extension. So really there's two paths to providing electricity to a community that is presently off-grid. There are off-grid solutions like solar lanterns, solar home systems, mini-grids. We talked about those somewhat in the last webinar. And then there's the on-grid approach. And in the on-grid approach, which is the subject of today's webinar, focuses on grid extension and grid enhancement. So when we talk about on-grid electrification, we're talking about the providing electricity access by extending or enhancing the existing national grid. So literally we're taking the existing electrical infrastructure and we're extending it to reach people that presently aren't connected. And I think there's two important characteristics of this approach. The first is that we rely on the existing fleet of power plants that the country has. So we're providing electricity through the energy produced by hydrodams or coal-fired power plants, for example. And the reason why this is so important is the energy provided by those power plants is usually far cheaper than what a small off-grid system can provide because there's economies of scale. So you can more efficiently produce electricity with a larger power plant than a smaller one. So by connecting a community to the electrical grid, you capture some of that efficiency and production. The other advantage of on-grid electrification is that there's no storage required. The power system in the national grid is built to supply electricity 24 hours a day, seven days a week. And so you don't need to have batteries. So these are two big advantages of on-grid electrification. I also want to describe briefly the difference between grid extension and grid enhancement. A grid extension is basically extending distribution lines to get to rural areas. Grid enhancement is upgrading the existing system. It could also include providing electricity access to people that are underneath the grid. So the grid doesn't really extend, they just get a connection to it. Although these are two different approaches, I'm going to just universally refer to it as grid extension, just out of convenience. So I'm talking about grid extension or grid enhancement when I say grid extension. So we'll talk about the national grid next, just in case you're not a power engineer. So the national grid is a collection of power plants, distribution lines, substations, and end users. And the flow of power is typically from the power plant all the way to the end user. So we're going to kind of go through each one of these elements and I'm just going to briefly describe this grid at which we're going to extend into rural communities. So it begins at the power plant. These can be hydro plants, they can be coal-fired power plants, natural gas power plants, etc. Here we produce megawatts, tens of megawatts, hundreds of megawatts, or even thousands of megawatts of power. And most of these power plants are located far away from where people live. And so what we need to do is we need to increase the voltage using the step up transformer. Let's go to the next slide please. Yeah, so we use transformers to increase the voltage to a few hundred thousand volts. And the reason why we do this is simply because it's more efficient to send power at high voltage and low current than at low voltage and high current. So we connect the substation to the distribution line, or excuse me, to the transmission lines. Which would be the next, thank you. And these transmission lines are the large ones that you might see from your car as you drive down to the interstate. They're quite tall. They're usually a metal lattice structure. And they have very, very large insulators because the voltages are quite high. The transmission lines can be hundreds or even thousands of kilometers long. And their goal is to bring the power from the power plants to the load centers, the cities and the industrial areas so that power can be consumed. There they enter the substation where we reduce the voltage so we perform the opposite transformation. We step the voltage down to a safer level somewhere between 11 and 33,000 volts where we connect it to a distribution line. Now distribution lines are the lines that you see running through neighborhoods. They're on smaller poles and they are at a much lower voltage. So these are the ones that the distribution lines are a real critical part of grid extension. And the distribution lines then terminate at distribution transformers where the voltage is again reduced, this time to maybe 100 to 400 volts and they provide power to people's houses. So grid extension then is connecting to this existing national grid, either at a substation or to a distribution line and connecting homes and rural communities. So typically the grid extension is organized by a government agency called a Rural Electrification Authority or an REA. So what REAs do is they come up with a master plan for rural electrification within that country. They might set some strategies for implementation of projects. In some cases they actually solicit funds from international organizations to fund rural electrification. I would say that if you are working with an organization that is planning on doing any sort of rural electrification that you consult the country's REA just to make sure that your project is synchronized and coherent with their plans. The last thing you want to do is invest $50,000 or so in a mini grid only to find the grid arrives six months later and electrifies the town that you invested all that money in. And then another important thing that REAs do is they identify electrification priorities. So an REA might pick a community to electrify because it has one or more of these characteristics. Communities that have a high potential for electricity consumption are ones that would receive priority. Typically those that are closer to the grid versus further from the grid would be given higher priority. We look for areas that are densely populated that have potential for commerce industry or tourism. In addition we might look for communities that have some sort of social infrastructure like medical clinics, schools, government buildings and we might target those communities. And then finally we might target communities that have some sort of political or cultural significance. So let's consider a hypothetical community of Milwaukee. So this is just a hypothetical community and let's say that we're trying to figure out a way of providing electricity access to them. So Milwaukee is a little community with about a thousand or two thousand people. There's about 200 households and we're going to target providing about a kilowatt hour per household per day. And if you recall from the last lecture or the last webinar, a kilowatt hour per household per day is a reasonable amount of electricity consumption. And we'll assume that Milwaukee is about 25 kilometers from the nearest distribution line. So if we wanted to consider a grid extension approach to electrifying Milwaukee it would involve connecting to the existing distribution line. So you see the existing grid on the diagram on the left. We probably have to add a substation to connect the two lines. We build a new distribution line and then along the distribution line when we get to Milwaukee we would connect these distribution transformers which would reduce the voltage to about 400 volts and then we would connect the individual households from that. So on the right side of this slide that you see you see what this might look like in reality. So we have the three-phase distribution line coming into the top of the transformer and then coming out of the transformer we have the low voltage lines that would connect to people's houses. So it turns out that the distribution lines are really important when it comes to grid extension. They're the most costly aspect of grid extension. Here is just some numbers that I pulled from the Ghana National Electrification Scheme. And you can see that the cost of the distribution lines are somewhere between $20,000 and $30,000 a kilometer. So it's actually a very large amount. And for communities that are 5, 10, 15, 25, or even further away the distribution lines can be 80 or 90 percent of the total cost of it. So we're going to focus on distribution lines and talk about some of the design considerations. So for Milwaukee, like any other community that we might extend the grid to, when we design the distribution line one of the most important things that we think about is what size conductor to use. Like what's that cross-sectional area, how much material, how much aluminum do we need in those cables. One of the main considerations is the voltage drop. As we send power down that distribution line to the community the voltage encounters some impedance or the current will encounter impedance and the voltage will drop along the line. And we really want to design the line so that the community at the end of it has a high enough voltage so that their TVs and lights and so forth will function properly. So usually that means we want to have a voltage drop of no more than 5 or 10 percent along that line. If we find in our design that the voltage drop is too large then what we would do is we would pick larger more expensive conductors. So I'll show you a few examples of what this looks like on the next slide here. Yeah so on this slide we have three different scenarios. The top one you can imagine just being a base case. Here we're sending power from the substation to the community so point A to point B and the voltage is going to drop along that line and you can see that there's an acceptable voltage drop window that's sort of highlighted in that yellow rectangle. So in this situation the design is sufficient to deliver power at the proper voltage to the community at the end of the line. The middle diagrams here show a community that is further away so point B is further away from point A than the top diagram. And we can see that the line voltage now is actually lower than the acceptable voltage drop. So this would be a bad condition. We wouldn't want this so this is a poor design. And so our way around this would be to actually install larger conductors which is shown in the bottom figure so the distribution line is now more substantial and the voltage drop then is within the range. So what this shows is basically the further away a community is not only does the line need to be longer to reach it but the conductors in fact need to be bigger so that we don't have a voltage drop problem. So it becomes more and more expensive the further away you go. Let's move on to the next slide please. Now we also have losses along that line as the power flows down the line it encounters resistance and heat is generated and that heat actually can be so much that the line mechanically fails or it starts to sag. In addition those losses will increase with the length of the line and so again to counteract these effects we would use larger conductors. Larger conductors reduce the resistance and so it will reduce the losses along the line. And on the slide that you see now is just an example of how we would actually go about selecting the conductor. We would pick a conductor that meets the thermal limit so it doesn't overheat as well as one that is large enough so that we don't have a voltage drop issue. So let's assume now that we've we've designed our distribution line up to Milwaukee we would then start looking at the total cost of this great extension project and there's two components to the cost. There's the cost of the energy and the cost of the infrastructure and the cost of the infrastructure there are several different components the distribution line cost the low voltage line cost the cost of any transformers that we might need at the at the distribution level so by people's houses we have a cost to modify or install a new substation and then there's also the cost of wiring people's homes adding the meters adding the circuit breaker boxes inside people's houses and so forth. So these would be the total costs of the infrastructure itself. So after we've come up with their design we would consult some vendors or we consult a table that has the prices so here's just these are all hypothetical but we would look at our design and come up with a cost estimate and in this case it's about seven hundred and eleven thousand dollars to provide access to the the community of Milwaukee. This is actually surprisingly high if you think about it there's only two hundred homes that we are targeting to provide a grid extension electricity access to and the total cost per connection is over three thousand dollars so three thousand five hundred and fifty five really so it's quite a bit of money to to extend the grid to that community. So we can look at that cost in a few ways. We can imagine that you know the the distribution line that runs to Milwaukee which is 25 kilometers long maybe it passes a closer community that's only 10 kilometers away well we can repeat the cost estimate for that community and we would see that electrifying 200 homes there would only cost four hundred thousand dollars and then if we also think of maybe there's a community that's right underneath the grid to connect 200 homes might cost just a hundred thousand dollars so as we go further and further away from the existing grid it becomes a much more expensive proposition to provide access to electricity and I think maybe a more meaningful way of looking at this is in terms of the connections that could be made given that same budget so if we had a budget of seven hundred and eleven thousand dollars we might have the option of connecting 200 people in Milwaukee or close to four hundred people in maybe some community that's not quite as far out but if we focus entirely on people that were underneath the existing grid the grid passes over their heads but their their home isn't connected we could connect almost 1200 households so if we look at the per connection costs you can see that there's quite a range and so if you're the the rural electrification authority or if you're the government and your goal was to provide access to more people you know which option would you pick would you connect Milwaukee the community that's along the way or those that are under the grid and I think this underscores one of the reasons why the grid hasn't made it out into the rural areas is it's just you get more bang for your buck you connect more people per dollar if you start in areas that already have a distribution line already have access to electricity you simply connect the many homes that aren't already connected you might think that people in urban areas already have access to electricity but that's really not the case there's still many that that don't so in addition to that overnight infrastructure cost we're usually interested in converting that to an annual cost of infrastructure and I won't get into that into too much detail here it's covered more in the book but basically there's a well known financial equation that you can apply to come up with what the annual cost of any infrastructure project is so on the next slide here we've done that calculation for the grid extension project to Milwaukee and we see that for a 30 year lifespan of the project it's going to cost $43,000 each and every year over that 30 year period so that's another way of looking at it but it's also useful to look at it in terms of cost per connection and here we have 200 connections so it ends up being about $217 per home in Milwaukee each year that you would need to collect from in order to pay for that distribution line and $217 is quite a bit of money to a rural household but we haven't talked about the cost of energy yet so we we have to remember that the electricity comes from somewhere and there is a cost of having that power plant of its fuel its maintenance and so forth so there are a couple of ways of of putting a number on the cost of energy and one of them is called the levelized cost of energy and you can think of the levelized cost of energy as the cost of energy that you the price that you'd have to sell a kilowatt hour for in order to break even on your power plant so you're not making a profit but you are covering all of your costs we cover this in more detail in the book but for now let's let's assume that the cost for Milwaukee the levelized cost of energy in the country that Milwaukee is then is is 15 cents a kilowatt hour so then we can do a quick calculation to figure out what the energy cost for Milwaukee is it's simply 15 cents a kilowatt hour times 365 kilowatt hours a year times the number of homes which is 200 and we get the cost then of providing electricity to Milwaukee which is $10,950 in fact it's going to be a bit higher than this because we've we've ignored losses for the sake of simplicity but we'll just we'll just assume that this figure here is correct so the energy costs when added to the infrastructure costs and again this is on an annual basis it ends up being a 54,335 dollars per year so in other words that's how much money it costs the utility or the government to provide the the community of Milwaukee energy and the amount of 365 kilowatt hours per household per year so we can look at this a few different ways we can look at it as the the cost per connection each year and it ends up being about 271 dollars or the cost per kilowatt hour supplied which ends up being closer to 74 cents a kilowatt hour and so this begs a very reasonable question is well are these costs appropriate is this okay could a rural community afford this can they afford to pay 74 cents a kilowatt hour for their electricity can they afford a 271 dollar a year fee for their electricity access and most likely the the answer is no remember we have a target of 5 percent a year no more than 5 percent a year spent on electricity costs for these rural communities and almost surely 271 dollars a year is greater than 5 percent of that that household income so the reality of grid extension is and this is true not just in developing countries but and for example the United States is that when you extend the power grid to rural areas it's often a loss making proposition for the utility in other words they they're going to lose money by doing this the distribution line cost is just that high so what that means is the rural electrification is subsidized by the government so the the community might pay one rate for electricity but it's going to actually be below what it costs the government to provide that same amount of electricity so governments tend to do this either to spur development or maybe there's some political reasons or maybe because they they get a grant to to do so but we always need to remember that the government is actually losing money on each kilowatt hour of electricity they provide and so there's this this incentive or disincentive to connect people in rural areas when the government starts to subsidize electricity it actually can do a disservice it can have like sort of an opposite effect it can discourage international investment in electrical infrastructure but it also affects off-grid entrepreneurs those that might want to privately install a mini grid and sell electricity they're not capturing that subsidy in most countries and so you see a very very large difference between the electricity rate that is charged for mini grid operate for mini grid customers versus those that are connected to the grid and most of the time the rural communities don't really understand why there's such a large difference so what about off-grid we see that extending the grid to Muassi is can only really be done if it's subsidized at least from an economic standpoint but what about off-grid well we can look at these two figures the 271 dollars and the 74 cents a kilowatt hour and that serves as a benchmark for an off-grid solution so if we can come up with an off-grid system that can provide the same amount of energy for less than 270 dollars a year per household then you can say that this off-grid solution is is a favorable outcome versus an on-grid solution similarly if it can provide it at a cost of less than 74 cents a kilowatt hour then an off-grid solution might be cost competitive and so you would pursue that instead of extending the grid more generally we can look at the decision to provide an on-grid versus off-grid solution as the graph that's shown on this slide so the grid extension costs we know will increase as the distance between the existing grid and the community also increases so communities that are further away of course have higher grid extension costs the cost for an off-grid system however is going to be pretty much the same no matter if that off-grid system is is installed right next to the distribution line or 100 kilometers away so it's it's going to be just a horizontal line now there's going to be a point where these two lines intersect the grid extension cost line and the off-grid system cost line and to the left of that point grid extension is going to be economically favorable and to the right the off-grid system is going to be favorable so you can see that communities that are closer to the grid you're more likely to extend the grid to them at least from an economic standpoint it's more sensible and as we go further and further from the grid off-grid systems in fact make more sense so to summarize that point here on the next slide when we think about communities that should be connected via grid extension we think of communities that are near the existing grid as well as having high consumption the more energy that the community uses the more favorable grid extension will be and this is because again energy from the grid is generally cheaper than energy from an off-grid system so for communities that are far from the grid who might only need lighting or electricity for cell phone charging those are perfect communities for off-grid solutions because the consumption is low and the grid extension cost would be simply too high to add the infrastructure to reaching them so to kind of summarize the point here we might consider grid extension to a community because first of all it can be more expensive that are less expensive I should say than an off-grid solution in terms of infrastructure and in terms of energy but of course that's not absolute it depends on several factors most importantly the distance between the community and the existing grid we might do grid extension because it's familiar so even even if the community is far away well we have you know most countries have lots of trained power engineers you know how to build distribution lines and substations so it's a familiar exercise in a way the utility knows exactly what they're getting into because it's it's something that they've done before there's also existing codes and regulations and oversight for grid extension and there's a less that's less so for off-grid systems and then the grid can offer a higher quality electricity access here than an off-grid solution and I say that sort of with a caveat and that in many countries the the grid connected power is very unreliable the voltage varies widely throughout the day and in some cases off-grid is even better can offer better reliability so the other side of the coin is off-grid solutions so we might consider an off-grid solution if the wait for grid extension is too long maybe the community is close enough to the grid where grid extension is actually the more economically favorable outcome but perhaps it's going to be 10 years or longer before that that grid will realistically reach that community so if we don't want to wait then an off-grid solution might make sense we if the grid is unreliable then an off-grid solution might make sense and then again in certain scenarios off-grid systems can ultimately be less expensive than than on-grid solutions so let's move on to kind of wrap things up here so why isn't the grid everywhere well I think the line of thought goes like this well the grid isn't everywhere because governments and utilities they have limited budgets for infrastructure projects especially in many of these developing countries that struggle with electricity access and grid extension is an expensive proposition both in absolute terms and in cost per connection terms so to supply electricity to those 200 homes in Milwaukee it costs you know closer to a million dollars that's quite a bit of money for a smaller country for example so it's an expensive proposition and then if we do decide to invest in that infrastructure we're going to do it at a loss so that money we're not really going to recuperate and it's going to be subsidized then and that you could argue that the subsidy then is pulling money away from the government's bank account that could be used on other development goals like improved healthcare education sanitation etc so if you have to decide would your other people have more resources for electricity or for example better healthcare and then finally it's more efficient to more cost efficient anyway to provide access to electricity to to people that live very close to the grid or they live in a large large city and then fortunately they're not connected to the grid so connecting those people first is just a more efficient use of money so the grid really hasn't reached the rural areas because in many countries we're still working on the low hanging fruit we are still trying to connect urban centers people living in urban centers to the grid and we haven't made our way out to the the rural areas we don't have the money to do it and it's simply just not economical so I'll summarize then the main points of this this webinar grid extension is usually the least expensive way of providing electricity access provided that you're close to the grid that seems obvious now but oftentimes we don't really think of it in those terms when we compare off grid to on grid connection schemes off grid is really favorable for communities that are far from the grid that consume small amounts of energy that are dispersed and that are not prioritized for grid extension so the weight might be too long distribution lines are often the most expensive part of the system and we need to design them to have an acceptable voltage drop and to minimize losses but that those those designs that use larger conductors add to the cost of course and that when we consider any sort of grid extension or even off-grid approach we need to consider the infrastructure costs as well as the energy costs and I think the calculation that makes sense is the annual cost per connection or per unit energy described that way we have an apples to apples comparison in terms of what we're going to be talking about during next month's webinars we're going to focus on energy conversion technologies so off-grid systems you know how we how we power them and in particular we're going to cover in some detailed generator sets of PV systems micro hydropower wind energy and biomass systems so that's going to be in December and you can check out the E4C website for registration details so with that there's some references and I'm happy to field any questions that we might have in the remaining time thank you thank you so much Henry and for those of you who are watching the webinar my apologies for the back and forth on some of the slides we had a little challenge advancing the slide so I was trying to take cues from Henry as to when the transition was going to be and we might have been off on a few of them so hopefully that didn't cause anybody too much eye strain so right now it's an opportunity to ask any questions that you may have to Henry regarding the grid extension and open up the floor here if we have any questions it was quite a thorough webinar we do have a question we have so Ignatius if you don't mind putting your question into the chat but unlike the virtual salon that we recently had this will be a requires us to type all our questions so while we wait for questions to arrive perhaps Henry if you have some insight what has been some kind of patterns or trends that you've seen in terms of grid extension decisions by governments in consideration of off-grid projects or solutions that are happening kind of in parallel is there attention or is there is the government even consider what players are in country developing those solutions and deploying them as part of their overall equation in determining whether grid extension is reasonable yeah that's a that's a great question so it depends on the country and what we're seeing in East Africa Kenya Tanzania and Nigeria and a few other countries the governments are understanding that off-grid electric electrification using off-grid systems is gonna they see the potential for it and so what they're doing is they're working to develop regulations that provide those companies that wish to do off-grid systems they provide some clarity and consistency in how they're going to be treated they're describing and defining what's going to happen to a mini-grid when the grid eventually gets there you know who owns that asset and so forth so I think the my perspective is that the utilities the the governments are understanding they're seeing the potential of off-grid and you're going to see a lot more coordination in the upcoming years in the past they've virtually ignored you know any sort of off-grid system and and grid extension has really been their mindset and I think we're starting to get some momentum in the other way and so it's really exciting to see how this is going to shake out and what policies are going to spur off-grid development and which ones might not work so well so it's a very exciting time yeah indeed so we have a few questions that I've come in so for people to be under the grid at some point a grid was expanded even if it wasn't a quote unquote good idea at the time or it had a low err isn't this also a way of expanding and then having low hanging fruits and the low hanging fruit is this case is in reference to those under the grid homes or urban homes that were not connected yeah I mean that's a good point and the example that we did in this webinar was rather simplistic I mean hopefully the the utility would identify communities between the existing grid and and the community of Milwaukee and so it wouldn't just be one one community although certainly I've seen grid extension that are you know 10 20 kilometers that all they do is connect a health clinic they bypass all those homes yeah so it's very common to see distribution lines going over people's homes in rural areas and it's a little strange because the utilities will often require very very hefty connection fees so they plan on investing and bringing a distribution line to the community but they're going to charge you know 500 us dollars to connect to it and many homes just they can't do it so so they're simply not connected so that explains it so yeah a strategic approach could be you're going to build the distribution line and then the people will eventually save up enough money to be connected and you yeah so that might make sense in some areas yep yes in particular when we're talking about urban context and rapid urban population growth and sure and and that frame of mind also informal settlements within urban centers there's a question here that's kind of speaking that if population growth of factoring how difficult it has been to connect everyone would can you just stand a little bit on that particularly when we're talking about urban centers yeah absolutely I mean in you know the population the rate of growth of population in many countries in sub-Saharan Africa is just astronomical the amount of investment you need to make just to kind of keep even is is incredible so yeah utilities are struggling to connect people in formal as well as informal settlements it absolutely makes makes a difference especially if the electricity rate is subsidized again they have a they have sort of a disincentive to connect people if they lose money and it's not like these utilities are flesh with cash you know most of the time they are they are struggling to make their own payments to their suppliers so it's it's a real challenge it's a real challenge they can't afford to subsidize really everyone and every connection so they have to pick the ones that make the most sense and and then you know sort of work their way up the the tree from the low-hanging fruit so there was another very specific question and we have quite a few questions we'll try to tackle all these okay lcoe correct metric to compare off-grid and on-grid cost of power delivery to customers well i would say yes what you need to do though is you need to make sure you account for the infrastructure cost so the levelized cost of energy we usually do for particular power plants and we might ignore the the transmission and distribution costs so if you want to include compare on-grid to off-grid you would calculate the levelized cost of energy including the infrastructure cost so it's similar to what we did here in the webinar although albeit it was a little simplistic and we just sort of bestowed a levelized cost of energy of 15 cents a kilowatt hour but i'd say it's certainly a fair approach to do obviously you could come up with a more rigorous model but for the purposes of explaining the concept absolutely it makes sense so there's actually another follow-up question that is more of a case study kind of question so this particular listener had visited a village in rural Kenya that is powered by microgrid however he found that it was kind of curious that there was a 33 kilovolt wind up past less than 10 meters from the village so why do you think it might have been advised to sell microgrid in this location where a step-down transformer would have been much cheaper if we're looking at pure economics yeah so maybe you could kind of shed some light as to maybe the same yeah absolutely so this is a fantastic question and something that i've noticed as well i was actually just talking to somebody about this last night in zambia where microgrid was installed across the street from a 33 kv line so it's not just that case in kenya we get contacted my nonprofit by organizations that wants to do solar for a school or a medical clinic and then we always ask them well how far away is the the grid and in some cases the grid is you know this is a the grid is there and they just want don't want to pay an electricity bill and so in their mind they think oh if we have solar things will be cheaper and that's usually the exact opposite so why we see mini grids in places where the grid is there has to be at least i'd like to think that there's some other maybe non-obvious reason like the the distribution line itself was not designed to handle additional load so that could be one example so in other words they designed that distribution line based on a certain number of customers and they've reached that limit and they simply don't want to upgrade that line to accommodate the customer served by the mini grid that could be one possibility um i think another possibility is that maybe the the utility would connect the customers that the mini grid are connected to but the wait might be too long and they say okay we'll do it but we'll do it in you know 18 to 36 months and maybe for whatever reason they didn't the mini grid customers didn't want to wait that long another reason could be that the the utility is waiting for more customers before they will install a transformer so it costs a lot of money to install a transformer and if there's just a small number of customers connected to the microgrid they might have asked the utility to connect and the utility said well we'll do it but we need to have at least 50 customers so those are some possible reasons um but it could also just be poor planning by the microgrid implementer they didn't know the grid was coming or they didn't care that it was there yeah maybe they had a grant or something absolutely always always always check to see where the grid is and if the grid is coming before you do anything like anything like that before you install mini grid words to the wise here so this is i think uh there's somebody else who had asked a question regarding cost analysis and on-grid and off-grid systems and incorporating factors that might come from the level of electricity access and I think this particular case study was actually quite good in demonstrating some of those factors and then uh so one question is if a community is connected to an quote unquote unreliable national grid with large voltage fluctuations are there other devices required to protect equipment from over under voltage conditions absolutely you can get you can get protection devices that will protect against higher voltages lower voltages not so much or at least not in a I mean you can buy what's called a universal power supply which is basically your own little battery backup and when the grid goes away it will provide nice clean power so that's that's one technology that those are quite expensive in terms of larger infrastructure you know you can have transformers that adjust the voltage dynamically but again those are pretty expensive to do a lot of a lot of equipment can these days can handle a fluctuation of a voltage without without damage but lower quality inexpensive appliances and things that you might actually see in these in this this rural context there are probably going to be less likely to have that built-in robustness so it's yeah it's unfortunate and somebody probably could do a study at the cost of of those voltage fluctuations and like to get a better sense of what's the true economic burden on having to replace lights televisions and so forth due to damage for yeah and maybe that will make mini-grids even more appealing because you don't that could be a selling point that you don't have that large fluctuation great question yeah it's a very good point so there's a couple of questions they're going to end with and I think many of them actually set us up nicely for upcoming webinars one is are there any generating devices outside of those listed that are available for rural villages yeah great question so so the ones that I that we're going to talk about gensets PV micro hydra wind biomass I think those are the the main ones there are there could be some niche ones that might come up in very specific locations but those are them those are really the main ones and in fact I mean PV is is I'm sure the fastest growing there's probably more gensets out there than PV systems or it's going to be pretty close you don't see very much wind micro hydra really only works in places where you have that you know the water resource that's adequate and biomass is really just a different fuel for a genset so those are the main ones that you see human power generation things like that really I don't know that there's been a lot of success stories around around that tidal geothermal those are you know I mean I don't know that there's many of those systems that are out there for small scale yeah exactly so for those of you who are interested just as for reference point again Henry will be speaking about these in an upcoming webinar and if you want to learn in advance or just kind of explore what's out there we encourage you to check out the E4C solutions library we actually have a variety of examples of these particular technology categories gensets level take micro hydra wind energy and biomass systems that you can peruse to see what is out there and including human power and some options of and to kind of make your own initial assessments and we're going to wrap up with the last question because I completely lie to everybody here about our 45 because we have to service conversation so how do you feel about various energy delivery business models like solar lanterns working off for off-grid electrification do you think they are reliable enough again maybe something that's leading into upcoming webinars or past webinars but I'll let you tackle that one Henry has a closing question yeah I think most people would say that solar home system well solar lanterns in particular are great but they're not that shouldn't be that that's not the final goal I mean I think the final goal should really be grid like electricity access so enough electricity not to just power lights and recharge cell phones but to do some of those more important or at least higher quality electricity services like refrigeration and maybe even electric stoves so solar lanterns sure and high quality ones I mean I have one that I use here in my office actually believe it or not they last a really long time and they're quite reliable I don't think I've ever had this this thing go out when I've needed it but there's a lot of counterfeit ones on the market and so it's very easy for the impression that they're low for them not to be reliable I mean there's there's actually a basis in that and that a lot of counterfeit ones go out there and people have that perception yeah the energy access tiers is low and I think we we should aim higher as a community and as an industry so Henry would you be willing to detain just a couple more questions so we had sure yeah I mean just just yeah people want to keep talking yeah yeah you can also put my contact info slide in there and people can contact me on twitter or email yeah absolutely there you go so um what role do you see international utilities or energy companies playing in grid extension or enhancement oh uh gosh that I'm sure that varies widely from country to country because it you know most of these utilities are are owned by the government or their peristate or sort of quasi government so the government would have to uh to you know sort of welcome their investments there's been some rumors in some countries that the government is using their utilities as collateral for loans and things like that but I don't know how how true their how true that is I would I think that I think that these governments should actually welcome international investment in their power systems especially because you know they've many countries have just struggled to keep up with population growth that was the question earlier so international investment sure you need to you know when we talk about power grids I mean if you look at the United States for example there's so many different models of investor owned utilities versus municipal utilities you know versus you know special utility districts there's a lot of different ways that it can work I think that you know it's very possible that you know an international company could come in and inject some finance and some capital into a much needed power system I think certainly in the off-grid space we're seeing that actually happen where more and more off-grid companies are getting financed and they're able to install more mini grids and I mean it's still a very dynamic business landscape to see if these companies are truly sustainable and can generate profit in the long run but it's an exciting time to see what will happen and I think you know the more the more attention that this problem gets is really the better you know the we'll find solutions that that will work us towards universal electricity access for sure I love your optimism so and I agree I'm with you on that and finally last question I promise this time if you could share any perspectives about interconnected micro grids and by interconnected bed this means with utility main grids or amongst each other and what actions that entails from the utilities or regulators yeah okay so I actually I maybe have a more pessimistic approach or thought on this this fact so I mean strategically you know in the systems that I've worked with and I would I would put my mini grids in a place where the grid isn't coming anytime soon so that means that when I design it I'm maybe less worried about making it backwards compatible with the actual grid itself thinking that by the time the grid gets there you know maybe the inverter or some of the other components will need to be replaced anyway they'll be at the end of their service life I think also when we talk about mini grids in rural communities connecting with each other boy I don't see that happening anytime soon so I'm glad that people are thinking about it but there's just so much so many communities so far from each other that don't have a mini grid to begin with that worrying about how we're going to connect them is maybe a problem for you know 10 years down the road at least not not not next week not it's I don't think it's holding anything back I think it's a fantastic topic to study now sort of in the research phase but you know many of these communities are so rural that even to connect from one community to the next that is going to require quite a bit of investment and it might might not make sense to future proof them now knowing that it's going to cost more money so lower costs I think right now is maybe more important so and I'm sure I've talked to other folks that have a different view than I do on that so my caveat is I'm a little pessimistic about how rapidly grids will extend mini grids will extend and so forth well I think balanced with your extreme optimism on other side you come out neutral so I think we're all okay so I promise I was the last question that is last question I really do appreciate everybody participating and sharing these thoughtful questions these are really great and I hope they've been useful for you for those of you who are speaking we're going to we're going to be closing out I want to thank you all for attending and certainly I want to thank Henry for presenting and continuing to dedicate himself for another several webinars to really dig deep into these topics we're so excited for the one that's coming in December for those of you who are interested in PHHs please click on the link or grab that link that you see in front of you to apply for your PHH if we didn't tap with your questions feel free to email Henry directly or email us if you didn't catch his email and of course don't forget to become E4C members so that we can send you information in upcoming webinars thank you everyone thank you Henry and a good morning good afternoon or good evening so you would remember where you're from and we're looking forward to catching you on the next E4C webinar have a good one bye bye great


Welcome to Cloud Native Live, where we dive into the code behind Cloud Native. I'm your host today. My name is Whitney Lee. I'm a CNC app ambassador and I'm a developer advocate at VMware Tanzu. So as you may know, every week we bring new presenters to showcase how to work with Cloud Native technologies. We will build things, we will break things, and we'll answer your questions. Today we have Stephie Janis here with us to talk about how to supercharge your infrastructure management with Open Policy Agent. So just a reminder, this is an official live stream of the CNC app. So as such, it is subject to the CNC app code of conduct. So that means please don't add anything to the chats that would be in violation of that code of conduct. So be nice, be respectful of the other folks in the chat, be respectful of Stephie the presenter, and be respectful to me too, please. I'd appreciate that. Friends who are joining us live, please do say hi in the chat right now. Tell us where you're tuning in from. I love seeing it. I will be able to shout it out, but I do watch it and I love to know that people all over the globe are watching, are here with us right now. It's so amazing. So as always, if you have questions during the presentation, go ahead and put them in chat. Today we're not going to answer them right when you have them, we're going to save them for the end and then we'll do a Q&A part after Stephie's presentation. So with that, I'm going to hand it over to Stephie Giannis to kick off today's presentation. Great. Thank you very much for having me with me. It's amazing to be here again. So I'm Stephie Giannis and I'm the CTO and co-founder of Firefly. Today I'm going to talk about open policy agents, and I'll elaborate and explain why we need this. Before we begin, I'd like to explain a bit about my company, about Firefly. We built the first Cloud Asset Management Platform that reduced the Cloud complexity using infrastructure as code. What we do in Firefly is to scan the entire Cloud footprint. I mean, hyperscalers like AWS, Azure, Google, etc. We analyze also Kubernetes and SaaS applications, and Firefly analyzes also the infrastructure as code. All I see tools from Terraform, CloudFormation, and even Helm charts, etc. We continuously detect the discrepancy between the Cloud and the IAC. We can generate Terraform code, or IAC code, using AI for something that was created manually. We detect drifts and unexpected changes because we are integrated with the Cloud events. And we also govern the Cloud using Open Policy Agents. And that's why I decided to talk about this subject today. Great. So go ahead, Sydney. Oh, nothing, I'm just excited. Cool. So let's begin. And I would like to explain what is OPA and why do we need this? So Open Policy Agents is the first unified policy as code solution with the language called Breville, the clarity language called Breville. The idea behind the OPA is that we can check, we can build any rule that we want and check on the JSON file. I mean, if I have a list of people, and right now I'm building a rule to detect who is more than 30, I can detect from the list of the people exactly I can get the result. And in this session, I will show you many examples for Kubernetes, for AWS, et cetera. In general speaking, you can see Open Policy Agent, that's a mature, open source tool, also approved by CNCF, and many companies out there adapted this great open source tool for many other purposes. I told you about Firefly, that we adapted OPA for governing the cloud, for secops, they have a real ability, et cetera. But out there, there are solutions for authentication and authorization that they utilize OPA for this purpose. Or for network firewall rules. For example, we can detect if a network is allowed or denied, et cetera. So I would like to share a screen and start with a quick example. Thank you, Whitney, for that. Okay. So I brought here an example for a rule, as you can see, that check if we have someone that is older than 20. And of course that I will elaborate and explain exactly the structure of this rule. And I have two people, I have John Doe and Michael Smith. John Doe, you can see he's age is 30 and OPA is OPA and OPA Kubernetes, and Michael Smith is 18 and OPAs are serverless. So let's take the OPA rule that I've created here. And I would like to introduce you to the OPA Playground. I'm John and I'm sharing this screen. Okay, the OPA Playground, that's a great way to detect our rule, to analyze, to assess if we build our rule correctly. So I'm taking the OPA Regal rule here. Now let's take the two JSON files and I would like to show you the result. Or John Doe, you can see that it's valid equal to true. And for Michael Smith, you will see the negative, I mean valid equal to false. Great, let's talk about the structure here. You can see that in Open Policy Agent, I started with the package. Package means what I'm going to run. And then I built here a default variable called valid, that the default value is false and I override the default value if the age is more than 20. So that's a simple rule for that. And we can, and I will show you in this session may complicated rules for many purposes. But let's dive more technically and if somebody here has some questions or would like you to repeat or something that feel free to write on the chat. Cool, so first thing, I would run right now the OPA server and let me zoom in this. You can see that I'm running a Docker run file. Docker run for a port 8181. The server is already running. So one, let me run it again. But it's very easy to run OPA server. On top of that, we will add our regular rules because we will add more and more and more policies. And after running the OPA server, after loading the OPA rules, we would like to check JSON files for each rule to understand if a rules, if a JSON file is complied or not compliant. So here, and I will explain what we have here. I'm loading the first OPA rule. You can see that I have file called example Revo that that's exactly what I showed you on the left side. And then running a command to HTTP put to load the data from the example into the open policy agent, let's run it. Now you can see that it's loaded. And I will check right now John Smith and Michael Smith and John Doe. So the first one, you can see valid through. You can zoom in. So trust me that that's the result. The second one is a valid false and we run right now the OPA server on our local machine and I show you this because it will lead me, it will help me run the second parts of this talk. So just to explain, I'm reading the JSON file of each user, John Doe or Michael Smith and then I wrap the JSON file with input and I'm taking this and sending HTTP post for the OPA server. And then I'm folding the wrapped JSON and then I'm getting the result. So that's how it works. Super. Before continuing to the second part of this talk, does anyone have questions? I have a quick question. So policy seems really powerful. And so if you're trying to start policy into your whole system from scratch, that seems like maybe kind of a dangerous undertaking or something you need to do very delicately. So do you have advice around how to start if you don't have any policy right now, like how to, what you would do as your first like baby steps into getting it working? Yes, of course. First, we need to understand the data because here I built it to users and in the next phase of this talk, I will show you a real example from AWS or Kubernetes. So we need to understand the data and then let me show you the regular playground. So you can see that they always search input dot age, input dot something, dot something because I needed to deep dive into the JSON and this JSON object might be complicated. Here, for example, I have hobbies list of something, but if I would like to see if I have at least one hobby that contains a prerequest of the world, we need to dive into it, okay? Excellent. Super. Great, so we think I would like to continue with the second part of this talk and now I will show you more example, real life example for Kubernetes, for AWS, et cetera. Let's start with Kubernetes, CNCF, I know that most of the people here love Kubernetes. So what I did here is to take a Kubernetes deployment, I logged into my Cube CTL and run Cube CTL to get the configuration of the specific deployment. You can do it also with the lens, for instance. And as you can see, I have a JSON file and I will show you this soon with the deployment itself. Let's copy the JSON here. And a build for you two different rules, one for detecting CPU requests that all deployments contain the CPU request, mandatory CPU request, and the second one, readiness to a probe. Because I understand that that's important policy. So let's copy the readiness probe and let's place it here in the lab right on the rule and let's evaluate and you'll see here the result. In that case, it's not allowed. And if you need to dive into the rule itself, you can see that I'm using something that was created by the community. I'm importing a library. And if I'm going through this, you will see under Open Policy Agents, the library itself. And I don't recommend to use pre-created libraries. So we have here the default variable allow, starting with false and then I built here a function to detect if I have a readiness probe. As you can see, it runs through the configuration that they have in that case. If the deployment has a spec.template.spec.containers, this sign is really important for you to understand what is that because I might have a list of containers. And if I would like to check if at least one of the containers inside has readiness probe, so I need to use this sign. And in the end of the day, I run the rule here and I'm getting the results and the same also for the CPU quest. This one is an easier one, but let's use it. So you can see that I'm also importing a library that was created by the community. And I added here value equal to one for a default CPU request. And then I deep dive into the configuration and the policy that returns if the deployment has CPU request or not, if I'm running this, I get the result. And one of the pains that we face with is to write the rules. And you asked me a great question before. And I can tell you that I spent hours for building rules. That's why I really recommend to utilize AI to generate the rules for you. Yeah, and in the last session that I had here about AI AC, the open source tool that we have created at Firefly, I showed some examples for infrastructure as code. What I'm going to show you right now is that you can use AI AC, AI tool to generate the policies for you. So what you need to do, A, you should go to open AI, generate an API key. As you can see here under the platform, you can go to the settings, APIs and generate a new key. And then you need to add the API key into the terminal. It looks like that. I'm gonna ask you to slow down just a little bit. People, when you were doing the playground earlier, we got the question, will you be sharing the samples? So the examples you gave of OPA before we even got into AI. Can we see those examples or is there a way for people to access those later to try them out for themselves? Yeah, I will upload everything into a dedicated guide and I will try it with you, okay? Okay, so great. And then another question is, can you share the library again? The library, you mean this one, the AI AC? Maybe so, we zoom in on that so people can see what it is or share the link in the private chat. Let me elaborate, please, because everything that I've used here, all the rules that you hear, they are available for free for your firefly. I mean, what you need to do is to go to a firefly.ai. You can sign up for free and that will show you all of the rules. Under the policies and insights, we built many rules for you, for Phenops, SecOps, Relability, for AWS, Kubernetes, SAS application, et cetera. So if right now you would like to see a rule for Kubernetes deployment without CPU requests or CPU limits, you can see here the OPA rule and you can just sign up for free and use our platform. So enjoy for me. Yeah, we need to have any more, any other question or I can continue? Please continue. Some folks are asking the Zoom, so you're doing, I think, a pretty good job of being mindful of making sure that font isn't too small. Yeah. But just something to watch. Sure. Cool folks. So what I wanted to show you is AIAC, our first open source tool that actually generates infrastructures called or policy or whatever you want using AI. So I started with OpenAI that you need to generate the API key for OpenAI. And then I will give you a real example for policy. Let's copy this. AIAC gets OPA policy that enforces readiness probe at Kubernetes deployment. Now I will go back to the CLI. It's a bit hard to zoom in, so I'll copy to the text on the top. So I'm running the AIAC. Now it's generating this rule with AI. Takes a couple of seconds. And then you will see the rule that was generated by AI. You can see here the rule itself. And you can use it. OK. So you can see that this rule was fully generated by AI and that's the example that I showed you before. So what you can do is to run the CLI command for AIAC and then you can use an argument for output file. I mean, to save the generated AI policy to a dedicated file and then you can utilize this. I mean, you can add the policy to the OPA server and then you can run any configuration that you want to detect the policy. So before continuing, I just want to conclude this specific part. I'll show you some examples for Kubernetes. We took a configuration of the deployment and then we built two rules. Doesn't matter if we built it with AI or we just wrote the policy for you. But in the end of the day, I wanted to check if my Kubernetes deployment complies with those policies. Now I'm going to elaborate on AWS because here I brought a lot of examples for FinOps, for SecOps, for Adability, et cetera. So the first thing that we need to do is to scan the cloud. And this is not so easy because, as you know, in AWS specifically, you have more than 20 regions. So you need to walk through all regions and run the CLI command to list your EC2 instances. So what I did here is just an example and I will explain how we can actually do an automation for that. So I'm running the EC2 describe instances and I will show you the command here. One second. EC2 describe instances, minus, minus profile, minus, minus region. And then I'm getting a result. I'm getting a list of all my instances. I'm doing the same for volumes and for lambda function and I save each of them in a dedicated file. Instances or a lambdas or whatever it is. Then I built many rules and I will walk you through them. If we talk about the SecOps, sorry, about FinOps, how we can save money, how we can reduce the cloud cost. The first thing that we would like to detect is if we have any stopped instance for more than six months, clearly we pay for it. And I will show you here an example for that. You can see here on the right side, a list of instances. On the left side, I built a rule to detect this. So you can see that I built a function, stopped instances longer than six months that represents the instance ID. Then I'm diving into the configuration. I would like to check if I have any reservation with at least one instance, with at least state.name.stop. If I have something like that, so clearly the instance is stopped. And you can actually detect it here in the JSON. And then, one second, I'm taking a property called state transmission, a transition reason. And you can see that it's a time. So I parse this time using regex, extract it. And then I built a rule to detect if this number is older than six months. And if the answer is yes, I'm tracking the instance ID. So let's click here. Let's evaluate the rule. And you would see the instance ID that is open for more than six months. The same also for EBS volumes, GPT EBS volumes. You pay for them. It's redundant. You should upgrade them to GP3. So I built here a rule to detect if the volume type is GP2 and the state is in US. And if I'm evaluating the rule, you would see the resource ID, the volume ID that is GP2. By the way, you can find it in Firefly, as I mentioned, for free. And we also give you the recommendation. I mean, the CLI command to upgrade GP2 to GP3, this is really useful. But let's say I would like to show you more examples. Unattached disks, of course, it's something that you pay money for it. So it's redundant if I have EBS volumes, that the attachment length is 0, then clearly I would like to understand which EBS volume is that. And probably I will remove it. More use cases for lambda function without concurrent execution limit, I can tell you a real story from Firefly. In a mistake, somebody deployed a lambda function and it has not configured the execution. Then we paid $4,000 a day, like this. So I would recommend to detect this as well. And if we dive into the secOps area, so we can find if we have any EBS volumes unencrypted with disk, you can see here if the disk volume dot encryption equal to false. We can run this rule and understand which resources, which volumes do not comply. The same for public EC2 instances. And here I'll give you one more comment for how we can elaborate this rule. I wanted to check which resources, which EC2 instances has public IP. So I will run this rule. You will see the instance I think. But what happens if right now I would like to elaborate the rule for understanding which EC2 instance has public IP, but the security group is open at least for 22, for SSH. And this is more complicated because it's dependencies between rules. So what we can do is we can build one rule that we will pass at least of all the security groups. So we will ask which security group is open for 22. Then we'll get a shorter list, and we'll forward the list to the second rule, to this one. And then we can elaborate our rule and not just detect if I have public IP. Also we can check if I have any security group that is part of the group of those that are open. And I'll give you an example how you can build something like that. So I built here a rule to detect unsupported lambda functions. And here you can see a list of lambda functions like a Python 2.7 or Node.js 0.10 point something. And I built here a rule that runs, that walks through the lambda and check if the runtime is unsupported. And if the answer is yes, I'm returning the function ARM. So exactly like this approach, you can do something more complicated, something more sophisticated, and build a combination of two different rules. Great. You have many other examples here for SecOps. For instance, to detect public accessible assets like EKS with a public API or Astribucket that might be public or Astribucket without a server-side encryption, very easy to mitigate just to configure the KMS key. Or this one, one of the first controls in CI's benchmark to detect users without MFA. And if we talk about reliability, obviously one of the most important things is to see tags, to understand that all of our instances or resources generally have an owner tag. So here I am checking that if we have at least one tag, that the key has owner. And then I can run the Evaluate button and see which instances. And I'll give you more examples for this. If the EC2 instance has public network interfaces, it's also more than one network interface. If the count of this is more than one, it's a bad practice and we would like to get notification. And I built here a broad deal, more example, for autoscaling group with a single availability zone or Astribucket without a versioning, et cetera. So before continuing, I'm stopping for a moment because I showed you right now Kubernetes and AWS. How you can leverage OpenPolicyAgent for governing your Kubernetes, your, you can create policies for Phenops, for SecOps, for availability, for everything that you want on top of your AWS and also for your SDLC. And I would walk you very briefly through that, but if you want to check that our Git that we defined some processes for our Git, we can utilize OpenPolicyAgent for this mission to detect that we avoid push to master. I mean that only by creating a pull request we'll be able to merge the code or test coverage. I mean that we can block CICD if the test coverage is less than 80%. For example, we can go ahead. What does the acronym SDLC stand for, please? Sure, SDLC is software development lifecycle. Okay, thank you. Sorry, that's complicated. Thank you for asking. We can also build a rule for the RBAC, role-based access control. I mean to define the groups and users on top of the Git. And if we talk about the Git configuration itself, we can define the code owner. And today that's mandatory. It's something that you should do to configure who can be the owner, who is the owner, the authorized code reviewer that can touch your, that can approve new pull requests. Or the repository visibility, if it's public or private. And if we have any, this, for example, is mandatory for SOP2 compliance. You must connect your pull request with JIRA. Using Open Policy Agent, you can do it. And I'd like to hear a question from the audience before I continue to the third part of the talk. I think I know what your answer is gonna be, but the question we have right now is, do you have any example for GCP with FinOps since you've been trying everything AWS? Sure, yeah. Sure, it's the same. GCP and AWS, of course, it's two different animals, two different type of sparrows, but you can find, for example, the unattached disk in your virtual machine, VM instance, or you can have a VM instance stopped for more than a month, or you can create a VM instance that is not utilized properly. Exactly what you can do. And if we talk about BigQuery, we can build a lot of policies regarding that to make sure that we utilize BigQuery correctly. Or if we talk about databases and let's talk about security, I would like to check if my database is encrypted. So it doesn't matter if it's Azure, Google or AWS, the whole idea behind is the same. So basically the rules will be different, but the concept is the same. Yeah, where can someone find the community written OPA rules? Because one thing I'm learning from your presentation is the possibility of what you can do with this is just so vast, it's so wildly vast. It's a little overwhelming for someone just hearing about it for the first time in a good way, like all the possibility it's exciting, the air is electric. But I think a great place to start would be rules that community has made. Where can we find those? Yeah, so there are some community, for instance, in Reddit. In Reddit, there is a huge community for OPA. You can see many people have questions. So I actually, sometimes you can find some Git repository, public Git repositories with the examples for OPA policy agent depends on your purpose. If you're talking about the cloud and IPaSkate has continuous governance, you can use Firefly to get least of all of those policies. Excellent. Great. Shall we continue to the next step? Sure. Okay, amazing. Cool folks. So right now, what I'm going to do is to actually block CICD using Open Policy Agents. I brought here telephone code that I built in the last session of CNCF and I'll tell you about this. And I will run the telephone plan command to see which resources are going to be changed. And then I built an OPA rule that detects the resources, the provisioned resources, the resources are going to be changed. And I'm checking if they have tag owner and tag environment, mandatory tag owner and tag environment. And if we are going to change a resource that does not comply with that, we will block the CICD. So before continuing and I'll explain all of this, I'm taking you back to the session that I have made in a CNCF a few weeks ago, with you, Whitney, by the way, that we talked about migration from cloud formation to telephone. And we used AAC to generate the telephone code. So here I built telephone code, fully operated telephone code, and I would walk it through the code itself. And we also generated a CICD, in that case, GitHub, GitHub Piping, GitHub Action, sorry. This one is the repository. This one is the repository, CNCF webinar and here under the actions, we will see here that we have a deployment that was generated by AAC. Here that's the telephone plan. All the resources that are going to be changed. But in the last session, we talked mostly about security. How I can take this and run security checkers like Bridgecore to stop the CICD, to block the CICD if we have any misconfiguration. So I'm taking the same idea, but instead of focusing specifically on check-off and security, right now I will show you how we can utilize the OPA for that. So let's begin very easily with terraform units. So give me one second and I will show you the telephone code. We have here a couple of services. If you will run terraform apply, it will create web application with some video. Then you have the provider, the latest one in terraform. You have the variables here and you have the outputs here. Here in the correct folder, I'm running terraform unit. In order to be able to run the terraform, we must run the terraform unit and then we will run the terraform plan. Next few seconds. Here terraform successfully initialize. Great. And now let's run the terraform plan. The terraform, let's start with the title. Next few seconds. You will see a couple of resources that are going to be added. 25 resources are going to be added. Zero to change, zero to destroy. So what I've done here is that I wanted to analyze only this and this. I mean, if I have something new, of course that I would like to block the CCD, or if I'm changing an existing resource that does not have the mandatory tags, I will block as well. So let's take the terraform plan and I will show you something that you should know.  So OPA works with a lot of different tools. So you talked about cloud, you talked about Kubernetes, you talked about GitHub and now this is terraform. Is it, what's the limit? It's like everything? That's the first unified policies code. Yes, it's everything. Everything in policies code. You can use OPA really for everything. Yeah. Fascinating. Please, definitely. So I shared here before that I ran the terraform plan command and right now I can use the output attribute and that will save a binary file called tfplan. Okay, if I try to open it, you won't understand nothing because it's a binary data. But that's why I would like to introduce you to a command called terraform show. Terraform show actually read this binary data and extract it into JSON file that we can do something with that. So if I'm running the command down terraform show minus no color and the relevant tfplan, you'll see a huge terraform, sorry, a huge JSON file for the resources are going to be changed. Then I beautify here and I save the data into an output.json. I mean that right now in this file, I have the provisioning. What is going to be changed if I will apply my code? And I will edit here inside the regular playground. You can see that takes one second. You can see here that the modules here, the resources changed. If, for example, everything is aligned and I'm just trying the plan without changes, this list will be empty and then my rule will pass. But I have created here an OPA rule and I will elaborate on that because it's important for me to explain about it. But it reads, okay. So first I call the package set. Zoom in please. Zoom in please. Thank you. Thank you very much. Good. So first I created here a package CICD. Then I took the input and I changed it to tfplan just for being easier for me to understand. I built a generic function for as key and I'm utilizing this function here to detect if a list of tags contain the key. Then what I'm doing here is to run for each resource change. I mean only provisioned resources. I extract them and I set them a variable called resources. And now I'm creating a new variable called violations because for each resource from this list, I would like to check if the resource has the mandatory key, the owner or end. And if I don't have one, at least if I don't have both of them, so I have here a list of a bunch of resources. Then I'm extracting that address. Address means the location in the telephone code for those resources. And I'm creating a deny rule with explanation here. For instance, it's a medium and I added the policy description. It's a platform, you can do whatever you want with that. Let's run the rule and you would see the result. You can see that we have a deny. By the way, some of the resources here comply with the mandatory key. We assume it, yeah. Yeah, I know, cool. And I have a list of violator. As you can see, all of those resources, this is the resource type, dot resource name, that's a unique in Terraform. And then we can find the list of them that will block the CICD. Now I'm going back to the code. After explaining that I am taking the Terraform plan, JSON file, I'm running my rule on top of that. And right now, let me share the readme.md here. So I am adding the OPA rule to the OPA server as I showed you in the step one, step two of this talk. Then I'm running the following command. I would like to check and to get the result. I would like to get this result, the entire one. As you can see here. So I'm using the OPA server with my regular rule and getting the result. And I'm extracting the violator. So I'm creating a shell variable called violator that actually runs the, this one is the sending the HTTP request to the OPA server, okay. Exactly as I showed you before. And then I'm extracting the result.violator. I mean to extract this. So I will have in my shell list of violators in a variable called violator. Let's run this, let's equal, we have a list of results as then. I'm extracting the length, I would like to understand if I have any violator like that. So I also created a variable here in the shell script called length that checked the length of result cell. We can see 14 results as now I'm, I'll explain what I did here. If the length is more than zero, then it's not valid in creating the violators. Else let's run Terraform apply. So if I'm running this, I will walk you line by line. So it's a, the length is zero and then I'm equal. I think that it's not valid. The violator, else, Terraform apply, finish. We will see not valid and we block the CI CD. And you can go back to the previous top that I had here to get the step four of the CI CD using a CI explain how you can utilize it. And you can add one more step for OPA exactly as I showed here right now. And that's it for today. If my key takeaways is that you should use OPA will help you streamline your DevOps processes. You should, the most painful part of that is to understand which resources you have and then you can use some open source tool or you can use Firefly to fetch the resources for you because we do scan the entire footprint. And if you would like to build your own rules, you can also utilize AI to do it for you. So we didn't, did we see very much AI? And you showed AI very briefly at the beginning, yes? Or what AI did we see today? You see an open source tool that we have built. Yeah, you see, that is based on the JetChat GPT but we did the prompt engineering over there that you'll be able to get a more accurate result through that and using that, it generates the OPA rule for you so you save a lot of time by putting it back in English. So are there risks when you're, again, like I still see policy writing with, like it's important to keep your system safe but it's also like so powerful that you need to be careful with it too. So like what checks and balances are there when you're using AI to write your policy to make sure you're not accidentally making it incorrectly so that too much or too little gets through? Oh, that's a great question. Fast, don't put any sensitive information, personal information, PII, sensitive data, etc because you don't want your private keys to be exposed. So always take your terraform declaration of your prompt and make sure that you remove all of those stuff. And I'll give you an advice, it's a bit more sophisticated when talking about JetGPT. They have a declaration of something, definition, sorry, for something called temperature. It's a number between zero to one. Zero means no thinking, I mean no creativity and generating code use the temperature of zero. If you would like JetGPT to tell us a story, of course we need more creativity so the number would be closer to one. So yes, I encourage you to use zero temperature. That's fascinating. I didn't know about that, I love it. So then, what I'd like to hear, I'm interested in the AI part of this. And what's a real world example of someone using AI to improve their costs? Oh, I'll give you an example from one of our castlemen. They've been- Okay, great. We've been castlemen for five years but I really use it, it's really, really, really- I just walked right into that one. Yeah, amazing. So one of our castlemen use the AI AC, we call it the CHI, policies called AI to generate policies using AI and using that created the rule to detect GP tools for and there are so many things related to EC2 instances that are stopped, et cetera. And they generated AI, the castlemen saved almost one million dollars like this because we detected all of the EC tools and all of the GP2VS volumes and they removed all of them. Yeah, so that's why I encourage you to use the AI. That's great, it finally all clicked for me. I wasn't quite understanding the AI piece but the AI is what's going through your existing infrastructure which is probably vast and finding all of those instances that you can then write rules about. Is that correct? Yeah. And I'll just came together for me. That's super cool. I loved, if anyone has any questions, please do ask them, otherwise I'm just gonna keep talking so watch out. But one thing, so I do this show called Enlightening, you can see my lightboard behind me and people come on and they teach me about different tools. And I love also hosting this show because I did an episode on OPA so I kind of understand it from a theoretical level but my show doesn't have any hands-on coding. So then here I get to see what it's really like to use it and I've been enjoying this very much. So anyway, one thing I loved is like seeing how, I liked that you explained how rules get chained together and using the output of one as the input for the next one to do more complicated rules. That was really cool. I'm just telling you what I like now. I don't know that there's a question in here. Maybe there is, we'll see. I also really liked seeing the rules. I liked how human readable it was. Like, so some of that was you just doing a good job commenting and naming your functions in good ways but also just in general, Rego was maybe a little more usable than intuitive than what I expected it to be, than what its reputation is. So I thought that was cool. Yeah, definitely. And of course, if you can use the AI to elaborate the prompt and then for example, that's exactly what we do in the AIC. You can use chat GPT, but it will give you just the rule itself. So you need to train to give more context. For instance, RMLs, let's add comments and let's make it a human readable rule. And using that, you get an OPI rule that looks like you build it. That's cool. That's awesome. And it's so necessary because I think part of the benefit of policy as code two is like the people who are running the policy maybe didn't write the policy. So when something doesn't pass, it's very important for them to understand why it didn't pass. And it's a learning opportunity too for the people who make future EC2 instances to know or what to, like the Kubernetes examples you used at the beginning that like it has to have a compute limit mentioned and it has to have a readiness probe. So once they go through the experience of having it fail and understanding why it failed, then the next time they try to create a resource they'll know to put those checks in there. So I like that it's teaching the users a teachable moment. Yep. And there are many more examples. I just brought two, but there are bunch of examples specifically for Kubernetes for the deployment for the horizontal pod, the auto scaler, the HPA that you should configure. So one thing and not another about KEDA if those folks who use KEDA or a Prometheus configuration you can use a CRD of Prometheus. You can configure it properly using Open Policy Agent. Much, much more. We can talk about Open Policy Agent for hours. And then we can... For days, for years, I think. Like that's like a, that is like a really, it's a thing I learned before in the episode that OPA works with a lot of different tools. But now like today it's like really sinking in like how vast the landscape of like possible OPA. And then while you tell me when you chain together different OPA rules can you be chaining them together between different tools altogether? Yeah. Yeah, that's a super interesting question because the OPA rule is specific for the context. I mean, if I have a JSON model and the JSON doesn't look the same so we need to write two different rules. But is also a new technology for cloud specific right now called the cross plane. The debt, the share, you have heard about it? Yeah, I know a lot about cross plane, yeah. So for the audience, cross plane is a really interesting open source tool for infrastructures called cloud agnostic. Instead of building a few different resources like S3 bucket for AWS or block storage for Azure or GCS for Google, I can build the Kubernetes CRD for storage and decide that I'm going to deploy it on AWS. And then I'm building a generic resource, something that will be able to migrate between one cloud to another. And then on top of that, the OPA regular rule will be almost the same. Yeah, that's interesting. Cross plane just like OPA is just can be used in so many ways across so many different technologies that it is very, it's like exciting to see all the different things that people come up to do with it. But with cross plane, the way I like to explain cross plane generally is that basically you could take any kubra, any excuse me, any API in the whole, in the universe and then you can make it into, you can interact with that API via Kubernetes with cross plane. So it brings that API into Kubernetes. And then you can use that control loop, you can integrate it with the rest of your system. It's super cool. That's great. I'm looking at this specifically from the cloud perspective because I'm a cloud client. So that's why I'm a cloud client, yeah. And that's, I think the most popular use case for it right now too is to use it to provision and manage cloud infrastructure. I have a question here from Akash. Let's see if we can get it on the screen. Oops. Can I use Azure open AI? Azure open AI, of course you can do. I don't know if you mean for using AAC using Azure AI, we added the support for that. And by the way, in Firefly, we also utilize open AI in our private instance in Vine. So I really recommend using Azure open AI and it's great. Awesome. Well, I think we're wrapping up today. Is there anything else that you wanna say before we say goodbye? And that's something specifically that I already sentenced for in each target and finish with is that while coding your telephone code, while coding your infrastructure, always keep your secrets safe. I mean, don't push secrets to the GIF. And it's not related specifically to this stuff, but I would think it's kind of safe. We should just end every single episode that way always. It's just good advice. Yeah, excellent. Well, this has been an absolute pleasure. Thank you for showing us about open policy agent and specifically how we can use AI to really supercharge how you can enforce policy across your entire cloud estate. That was amazing. Okay, I'm gonna read the ending strip now if you feel ready. Okay, I have a script here. Okay, thank you. I would have said thank you anyway, by the way, regardless of whether it was on my script. But anyway, thank you everyone for joining today's episode of Cloud Native Live. Thank you so much, Steffi Janus, teaching us about supercharging your infrastructure management with open policy agents. The chats, y'all were amazing, like always. I loved the questions and the interaction from you. And I love seeing where all y'all are from. I'm from Texas, if you can't tell me putting all the y'alls in there. Here at Cloud Native Live, we bring you the latest cloud native code on Tuesdays and Wednesdays. So thanks for joining us today and we have another episode tomorrow at the same bat time and same bat channel. And thank you everyone who watches the recording and we'll see you again soon. Goodbye friends.