This is stock number HA153A, a 2014 Jeep Grand Cherokee limited. This vehicle has a 3.6 liter Pentastar V6 motor. From this HD video, you will be able to tell that this Jeep is extremely clean inside and out. It has the projector lamp headlamps, the projector lamp fog lamps, billet silver is the color. We shoot all of our videos in 1080p so if you have HD capabilities on your computer tablet or smartphone device, turn them on right now because it's like you're right here looking at the vehicle with me. Goodyear Fortera 26560R20 tires and these tires look like they have about 60 to 70% of the tread left on them. Billet silver, always a good looking color. Seven slotted grille, no dents or dings in the hood. Your side rim is in absolutely perfect condition, no scuffs or scrapes on that. We take these videos so if you are far away or even if you're close by and you just cannot make the trip down, you can still see the vehicle, hear the vehicle, and have confidence in what you're looking at before you even get here. We sell these vehicles all over the country so if you are watching this, you can have the confidence that you're dealing with the right vehicle and the right people. On this side, absolutely perfect, I didn't see any dents or dings. The back rim is in excellent condition as well, no scuffs or scrapes on that. Back tires have just as much tread as the front tires. We want to make sure that you get the most accurate representation of the vehicle as possible so that when you get here, there's absolutely no surprises. You can see it has the dual rear exhaust, the towing package factory, four pin and seven pin backup sensors. The rear gate is in really nice condition as well. It is a power lift gate that goes up like so. Back storage area is very, very clean. It does have the factory subwoofer back here and it has the full size spare tire. Has the power rear gate so you press that button. It'll beep at you three times and then it'll come on down. It has the LED license plate lights, LED taillights, and as we go down this side of the vehicle just as clean as the passenger side, I didn't see any dents or dings down this side of the vehicle either. Very nice condition, chrome trim mirrors with the built in directional signals. These are also heated mirrors. Just wanted to show you one thing here. This does have the capitalist fuel fill so really nice feature. You're never going to get gas on your hands again.  There are no rips or tears on these seats. This car has never been smoked in. It was purchased brand new here and traded in on a brand new one. Factory floor mats, lumbar on the driver's seat, and memory driver's seat as well.  You can see that this vehicle has 72,156 miles. You get the 7-inch TFT display. It's all configurable for the corners and it also has your digital spinometer. The heated leather wrapped steering wheel, no rips or tears or scuffs on that, cruise controls on that side, Bluetooth and information center controls on that side. It has the 8.4 touchscreen radio with the factory navigation system. Also is where your backup camera shows up. Tool climate control. You can also do your heated seats and heated steering wheel right there and then you can control the climate control up here as well. It has an AUG's USB and SD jack including the Bluetooth and you have downhill assist control, four-wheel drive low, and then you have the Quadra Track 2 system which gives you the snow, sand, auto mud, and rock features. There is a neutral setting so you could tow this flat-toeing. Passenger side seat, very clean as well, no rips or tears on that. Like I said, this car has never been smoked in and is very, very clean inside and out. We have the power sunroof up here with the home link, power rear gate, and then your 911 and assist buttons are right there. Those are features of the vehicle. You never have to pay a subscription for those. Take a quick look at the back seats and then we will check out under the hood. Back seats are just as clean as the front seats, no rips or tears back here. Does have the latch child safety system. The second row is heated as well. There are your heated seat buttons along with a 115 volt, 150 watt plug-in and two USB jacks. Let's take a quick look under the hood here. Glad my camera refocused. Under the hood we have the 3.6 liter award-winning Pentastar V6 motor. Engine bay is very clean, runs very smooth. This Jeep has been fully safety and inspected by our Jeep certified mechanics. Has a fresh oil and filter change. All the fluids have been checked and topped off. The vehicle has been gone through mechanically 100% and it is 100% ready to go. To see more pictures of this Jeep or one of our other 400 new and used cars, trucks, SUVs, minivans, wranglers, you name it, we got it.  Videos of every single used vehicle that we have all at SummitAuto.com. Thank you so much for checking out the video. If you like this one, why not make it yours? Give us a call right now,<PHONE_NUMBER>. Ask for one of our sales associates to make this Jeep Grand Cherokee yours today. Once again, that number is<PHONE_NUMBER>. Our sales associates are standing by to take your call. If you like the video and want to check out more, go to www.youtube.com. SummitAuto, remember to like, subscribe and share on those videos and this one. In fact, in a second you'll see a link to subscribe to our YouTube channel on your left. A link to more used inventory videos like this one on your right. If you haven't been to our website on the bottom link to this Jeep on our website, click those. Check us out. We really look forward to helping you with this super clean.


I use thermal electric generator tiles so it harnesses this thing called the seabag effect. Your hands contact the tiles which generates an electric current which powers these lights. In a lot of the developing world there's harmful microorganisms in the water. These can cause all kinds of sicknesses.  I think that water is such a precious commodity and with global climate change occurring right now access to water is going to be more difficult to obtain. The thing that I thought I could have the most impact on was providing clean water for people. As a young person, Yash was always very curious. Yash as a child he was very talkative. He always started off with I have an idea. I originally wanted to do explosive ordnance disposal but I couldn't do it because I'm colorblind and you can't cut the red wire if you don't know which one is the red wire. I want to fight. I want to do good even if I'm not part of the military. That's still something that I like I strive to do. I've wanted to do something with water for a long time. The original invention was a water bottle that purifies water using the heat of your hand as a power source. You have to hold like a water bottle anyways so as you hold it it'll power the purifier and then ultraviolet light is what purifies the water by killing microorganisms in the water. So you just hold it for a few minutes and then your water is purified. Without water the max you're gonna live depends on the climate you're in is three days. If you're like a marine on a mission you can definitely use a clean bottle to like get water from like any stream really any body of water you come across it'd be drinkable after that. Our gear is what brings us success. My fight is kind of an internal one. I'm gonna be living in this world for the next 70, 80, 90 years like I want it to be a world that I want to live in. Even just helping out like one or two people that's like enough positive change to like make the world a better place.


To share, you know, Washington, we're so filled with kind of sour news these days. I had the privilege of sitting next to the Afghanistan Ambassador, Ambassador Hakimi tonight, and he was just sharing with me the excitement that he felt personally about the election that we've had in Afghanistan. Seven million people voted in this election. Thirty-five percent of them were women.  Here in this country, you know, if the line's too long, coming behind the polling station, we turn around and leave. You know, in this country, people stayed for hours risking their lives so that they could have a vote in their own future. And I don't remember who it was that said it. It says, we don't yet know who won the election because there might be a runoff, but we'd know who lost, and that was the Taliban. And I think we should just, with a little bit of applause, congratulate the Afghanistan. Carla, let me ask you to come up and get our program going for real. And I just want to say thanks to you and Big Brzezinski for co-chairing the advisory board. And thank you very much, Carla. We are so fortunate to have John Hamry in charge of CSIS. And I hope all of you have enjoyed your dinner conversation and the meeting outside of what I regard as a beautiful vista, as much as I. But we're privileged tonight to have this evening with us, Norway's Minister of Foreign Affairs, the Honorable Borga Brendi. Minister Brendi has had a long and distinguished career in government, having previously served as Norway's Minister of Trade and Industry, a position to which I relate very strongly, and the Ministry of the Environment. He's also held very important positions outside of the government, including as Managing Director of the World Economic Forum, Secretary-General of the Norwegian Red Cross and Chairman of the UN Commission on Sustainable Development. Our topic this evening and for our advisory board is the geopolitics of energy. As you all know, Norway is an important producer and exporter of oil and natural gas. It occupies a strategically important geopolitical position with respect to its neighbors, Russia, and the European Union and its proximity to ongoing developments in the Arctic. Minister Brendi has graciously offered to provide us with a Norwegian perspective on geopolitical geopolitics of energy. And I might say this is not a new topic for him before his appointment as Minister of Foreign Affairs. He served on the board of directors of Norway's Stat Oil. A leading international oil and gas company. And before I invite the minister to take over the podium, let me remind each and every one of you that his comments are on the record as is the discussion following his remarks. And so, Minister, we're honored and privileged to have you here at CSIS and we look forward to hearing what you can talk about. Thank you so much, Carla Hills, for that great introduction. My respect for former USTRs are very high. We know that dealing with trade issues are not only important, but they're quite complicated. And we have just come through a period of the great recession, but we have, as an important part of getting out of it, we have used the WTO that we built together as a security measure of not turning into protectionism that we did in the 30s. So, Excellencies, ladies and gentlemen, it's great for me to be here. I'm very honored. Also, thank you to my great friend, John, where are you? There, you're over there. Thank you for inviting me. Mapping the global energy landscape is vital for understanding the world we live in. In order to grasp modern geopolitics, we need to examine the ways we produce and consume energy. It is currently, it went in Europe that we cannot not mention. It was mentioned the situation in Ukraine. And I think those have reminded us about important values we share. Russia's actions and calculations belong to a century far removed from ours. We are also seeing that the sanctions that have been introduced has had a psychological effect. We have seen the depreciation of the ruble, the impact also on the stock market. These dramatic events have also reinforced existing concerns about energy security in Europe, which is something I will address in a moment. In a situation where global energy demand is expected to increase by 35% over the next 20 years, the interplay between energy, power and politics will continue to affect foreign policy making. Economic factors play a decisive role in shaping our world. This was effective when Adam Smith and Karl Marx could agree on. Energy supply and demand are extremely important in this context. A journey through the geopolitics of energy it takes us west, east and south, and not least up high north. The global map of economic and political power is shifting and the energy landscape is changing along with it. China is about to become the world's biggest importer of oil. India is forecast to be the biggest importer of coal by 2020. This development comes as no surprise. What is surprising, however, is the second feature of the new energy landscape. The American Revolution, this time an energy revolution. It is astonishing to see how US crude oil production has increased by 65% and gas production by 34% in just 10 years. After decades of dependency on the Middle East, who would have predicted that the US will soon be second only to Saudi Arabia in terms of oil production and second only to Russia in terms of gas production? I do look forward, John, to the launch of the CSIS study tomorrow. I'm sure your discussion will provide interesting insights into the geopolitical consequences of the unconventional oil and gas and prove valuable for US foreign policy decision makers. Even though the shale boom is bound to have geopolitical consequences, I still believe that the Middle East will remain a region of geopolitical significance. When the oil consumption and demand will increase by 35%, it will, of course, change where the demand is coming from when it comes to the Middle East. It will come, for example, then from China. Given the stakes involved, discussions on the geopolitics of energy easily end up focusing on the potential for conflict. Without ignoring such concerns, I would like to propose a somewhat different perspective. The Norwegian approach to energy security is quite simple. We are a reliable and predictable provider of energy and will remain so. We believe the role of politicians should be to provide the predictable regulatory framework necessary to allow companies to operate on a sound commercial basis. Making politics out of the energy markets will profit neither consumers nor consumers. Companies need predictable and functioning markets in order to finance the huge investments necessary for future energy production. Norway is the third largest exporter of gas globally. We are also in the major league when it comes to oil exports. We are currently number seven. And we also made sure when we became a major energy producer to secure the governmental rent to the people of Norway. So we have been running a tax system and maybe as a board member of Stadthalle, I wasn't always that thrilled. But 80% marginal taxes on oil and gas companies. We do have in Norway, but there is still a huge interest from a lot of companies to drill and be at the Norwegian continental shelf. And we made sure that the governmental rent and we know it belongs to the people of Norway. And we have built up our sovereign wealth fund. It's the largest sovereign wealth fund in the world close to 900 billion US dollars and own around 2% of all listed companies globally. We also have decided that is only the real interest of the sovereign wealth fund that can be allocated to our state budget. And we would like to see those contributions from the sovereign wealth fund invested in future orientated sectors like education, R&D and also infrastructure. Norway is also the second largest producer of hydro power on pair with India. And Norway is the second largest exporter of gas to Europe after Russia. More than 70% of Norwegian gas resources remain to be produced. We are maintaining a high level of exploration activities. We will continue to secure predictable deliveries as the best contribution we can make to energy security. In addition, we strongly emphasize energy saving and efficiency and are working to develop technological solutions for combating climate change. Exporters and importers depend on each other and we fully understand that consumers want to diversify their sources. When the First Lord of the Admiralty made the historic decision to convert British battleships from being cold-fired to oil-fired in 1911 he also stated that the Royal Navy could not rely on deliveries from one oil field or one country. Safety and certainty in oil lie in variety and variety alone, he said. This was not the only time Winston Churchill recognized a fundamental truth before most people. Some of us think this is also food for thoughts in the current situation in Europe. The European Union has started an ambitious project to find ways of improving the continent's energy security. There can be no quick fixes to the issue of energy security. It takes a long time to change energy systems and corresponding energy flows. We believe that gas should continue to be a key component in Europe's energy mix. Gas emits 50% less CO2 than coal when burned. Would you live in an increasingly globalized world? In most cases, win-win solutions trump zero-sum thinking. Since 1970, the value of world trade has increased eight times. We still live in a world of competition but national interests are better served by cooperation than confrontation. We enter into trade agreements and bind ourselves to common sets of rules because we benefit from doing so. Even in authoritarian states, there is a growing realization that old-fashioned zero-sum thinking runs contrary to national interests. The road to prosperity is true innovation and inviting other countries to engage in voluntary trade in order for trade to flourish, stable and friendly relations between neighbors are needed. Even if power is currently moving east and south, the world has not been turned upside down. We are entering a world where the US and China are more influential than others, but we are not heading for a repetition of the bipolar world of the Cold War. The US and China are too interdependent for such a comparison to be helpful. The world economy, as I mentioned, has survived the crisis in 2008 because leader resisted the temptation to resort to protectionism and nationalism. They stuck to the rules and kept trading. The prosperity of nations is best served by agreed rules, trade agreements and predictable investment climates. Energy markets are no exception. That is why the Norwegian government wants our oil and gas companies to operate within predictable legal frameworks and base their investments on commercial calculations. There are several examples of how energy production can be impeded by unstable conditions and due state interference, poor governments and conflict. The massive increase in energy demand, especially in East Asia, India and Middle East, will compel nations to find new ways of cooperating. China's energy requirements will affect its future foreign policy decisions. Beijing needs a stable international system to protect its energy imports. Some speculate that the current crisis in Europe may accelerate Russia's plan to build pipeline eastwards. Such projects will still take a long time to complete and will not necessarily lead to a fundamental reduction of Russia's interest in the European markets. As power is shifting towards the east and the south, as I mentioned, geopolitical eyes are also looking north. It is estimated that around 20% of the world's undiscovered global petroleum resources are to be found in the Arctic. The high north is characterized by low tension and we must ensure that it remains a region of peaceful cooperation and respect for international law with actors using a win-win perspective when formulating strategies. When they became observers in the Arctic Council, China, India, Japan, South Korea and Singapore agreed to the rules of the game in the high north, including the law of the sea and the rights of Arctic coastal states. You may have read press reports about race to the north or a scramble for resources. Many of you will know that this is misleading. As I mentioned, on the Norwegian continental shelf, international companies can compete for licenses on equal terms as they can in the North Sea. Of course, there is competition among companies and we have representatives, fine representatives of many of you here. That is the way the market economy works. The Arctic Council provides the political framework needed for dealing with challenges in the North. Establishing a set of agreed rules and following them promotes the prosperity of nations and it strengthens energy security. Changes to the global energy landscape will not change our basic approach. My government will continue to be a stable supplier of energy and to provide a predictable framework that offers companies incentives to increase their production. Thank you. An enormously clear and very valuable presentation and we appreciate it and thank you warmly. We particularly thank you even more for your generosity and time and willingness to answer a few questions. So we'll keep the minister who is, I think you must have some jet lag. He just got in yesterday, but we'll take a few questions. Over here at the Daisy table. Would you state your name so that the minister knows who is speaking? Well, thank you minister. I'm Nelmini Rubin. I'm with the House Foreign Affairs Committee here in Washington, D.C. and I think I actually may have met you at the World Economic Forum many years ago. I wanted to ask you, given your perspective and your spot in the world, what do you think the United States should do differently to better support European energy security? We'll take a few questions and then take them as a group. Is there another question from the floor? Oh, you're so shy. Why do you take it? Yes, please. Larissa Courtney. I wanted to ask the Prime Minister what Norway can do to help Ukraine in its current plight with Russia? These are either energy needs. Any others? All right, Mr. Yes. There's the mic. Oh, sorry. As we look at the developing situation with Russia, the Ukraine natural gas, do you feel that Norway can and should play a critical strategic role in supplying Europe with more natural gas and hydrocarbons so that they're not so dependent on Russia? Great. Thank you. Let me give a couple of remarks on Ukraine first. It is a very serious situation in Ukraine these days. There is a tough situation where it is interesting to see where Mr. Putin and Russia is heading. I hope that there will be no further military intervention either in the east or in the south. I think that will end in a very dramatic situation. We should not forget that the Ukrainian army is also 140,000 soldiers, and I think the development will not be the same if there is military intervention in the eastern part, will not play out the same way as it did in Crimea. I think that is an understatement. We are seeing that a lot of gas, Russian gas, is running through Ukraine. Ukraine is also a great consumer and buyer of Russian gas. If there was a situation developing where this gas was hindered in a way, of course Norway is ready to take it, it's part of the responsibility, but natural gas, as I mentioned earlier on, needs also the infrastructure to be exported. Norway is almost running close to the capacity when it comes to exporting natural gas to Europe with the current infrastructure. So maybe for a short term we could increase by 10%, but that's what the capacity is when it comes to pipes. In the future there is more natural gas available. As I said, still 70% of the known resources are still there to be utilized, and we will be predictable if there is interest for Norway stepping up in that respect. We, of course, will take that responsibility. But there are no short term, there are no quick fixes. I think this confirms that Europe needs to diversify and Europe needs to pay more attention to energy security. On the other side I know that Russia is also very much relying on the European gas market. 30% of the European gas comes from Russia, and it's a very important revenue for Russia. And where else can Russia then export this gas? As I mentioned, they can, of course, in the future build pipes for China, but it's costly, takes time, and in a situation where Russian economy is growing less than 1%, and used to grow 5%, and the psychological impact of the sanctions are no felt. I'm reluctant to believe that Russia is not interested to continue to be an energy exporter to Europe. And we know that a lot of the legitimacy of the current regime in Russia has and will come from the middle class that has developed. And also Mr. Putin has put a lot of effort into securing pensions and all this. So for a pensioner in Murmans that has seen the pensions developing okay the last 10 years, I think that depends also on economic growth in Russia to continue this in the future. And I think then the gas is an important part of it. On the US side, how can the US contribute? I think in the future there will be interesting to see how the US will deal with the LNG. And there will probably then be a discussion both on the Hill and in the administration if there will be an opening up for export of LNG. I think this is a discussion that will take place also in the US. I think that we are all very, very grateful for the Minister's time. He is a real expert, not only on energy but on global affairs. And we wish you well on the rest of your trip as you move on from here. I hope that you enjoy Houston. And as much as we've enjoyed having you here. So thank you so much. Thank you. Colleagues we're going to have a very full day tomorrow. We'll start at 9 o'clock. Coffee will be available at 9 o'clock maybe a little bit earlier. The program is going to start at 9.15. We're going to start with a little brief presentation that summarizes the results of our study. And then we will turn to Fatah Barol who is going to be with us. Fatah is the chief thinker at the International Energy Agency. We're delighted to have him here as a good friend. In the afternoon we'll have a round table session on the geopolitical implications on a geographical basis. And then we will have a dialogue. Oh Dan Poneman, Deputy Secretary of Energy will be with us. And then we will have a dialogue between John Hess and Zbig Brzezinski. So we'll hear what the two different perspectives on the same world. And it'll be a fascinating discussion. So please join us tomorrow at 9 o'clock. And thank you all for coming tonight. We'll see you tomorrow.


Well welcome everybody. I'm Josh Burkus. I work for Red Hat in the OSAS, which is always known as the Community Department, where I get to work on Kubernetes, which is truly awesome. And if you know me from earlier times, you looked at my CV, you know the way that I got into Kubernetes was actually moving Postgres to Kubernetes. And so this has been my sort of personal project for a while. Of course, what you're about to see is not just my personal project, or even primarily my personal project. The lion's share of the work is done by the engineering team at Zolando. We're specifically the database team at Zolando. We started collaborating two years ago on the Petroni project. Nobody from that database team could be here at KubeCon, which is why I'm on this podium by myself. But I'll tell you all about it, and I'm sure they will be at KubeCon Europe talking about things. So my goal and really the goal of the Petroni project from the beginning is to make deploying PostgresQL on Kubernetes, or frankly, to make deploying PostgresQL in any form boring. That is, you shouldn't be boasting about deploying PostgresQL on your internal cloud, or on your public cloud, because it should be something that everyone does with a single command. And that's our goal. Now, what is what boring comes from is, if you saw Matt essay posted an article a couple of days ago talking about how, despite being 27 years old at this point, I think, PostgresQL is like one of the hot hip databases to use with all of your new technology tools. And one of the reasons why people like using PostgresQL even with new languages and new servers and new technologies is that it is boring. And what he means by boring is, it is not something that you have to think about. As in, you deploy Postgres, and then you save your data to Postgres, and if Postgres is your data is saved, it's saved, and if Postgres is your data is replicated, it's replicated, and you don't really have to think about whether that's true or not, or about starting strategies, or about all kinds of other stuff that you have to with some of the more interesting scale out databases. Now, it's a good statement. It's not hip, and it takes a bit to set up, but after you're done, you have a reliable workhorse.  It takes a bit to set up, and that's what this project is about, and Kubernetes is a big help with this. So I'm going to explain a few things here. I'm going to explain the Petroni project and the tool in general for anybody who hasn't seen one of these, one of the earlier presentations for earlier versions of the tool. Explain what we changed recently for anybody who has seen my previous presentations, and then actually show you the sort of Kubernetes native version of it, and demo that as well as a brief demo of the Zalando Postgres operator, and then talk a little bit about another Postgres operator. Now, for anybody who has seen, you know, this before or read any of my stuff about Petroni, et cetera, what's changed since like the last KubeCon is a lot tighter Kubernetes integration in order to just rely more on Kubernetes for all of the infrastructure things, and I'll show you that. The second thing is that we have an operator now, which is fairly alpha, but there's an operator out there for anybody who's familiar with the operator pattern, and again, I will show you that. And then the third thing, of course, is that Petroni Kubernetes, et cetera, is now being used in production in a couple of different places. And so as a result, we've learned a lot by having it deployed in production and relied on which improvements have gone into Petroni. So some terminology. So Postgres, PostgresQL, that's the database we're talking about. The one that's been around, again, since the 80s, and yet continues to add new things. Petroni is a high availability demon service that runs inside a container. Let's explain a little bit more about that. And Spilo is a distribution that wraps Petroni and PostgresQL and some other common utilities in a container to be your container of Postgres goodness. So you'll see those words again in this presentation, and that's what the three different sort of things are. So particularly I'm going to talk a lot about Petroni, which is the high availability demon. So this does a few things. You run it in your container as PID1. It controls whether or not Postgres is running in that container. It automates failover and replication for your cluster of whatever defined size. And it also supplies a management API for the things that you can't do through the normal Postgres interface. Now question of, you know, what's the idea? What's the design behind this? What are sort of specifically, what do we think, why we're doing Petroni and not using some other systems and, you know, what are our goals? And there's three basic ones. So we're looking for simplicity, usability and availability. And what I mean by simplicity is we're really trying to, the majority of people, right, us people who are Postgres geeks, we think of Postgres as this huge constellation of services, you know, that does all these different things and you've got foreign data wrappers and you've got monitoring tools and you've got all these other things.  For everyone else, Postgres is one thing. It's an endpoint and they want that to be one endpoint and not 16 different things. And so our goal, partly with Petroni and Spilo, is to make Postgres one endpoint and not 16 different things. To, you know, rely on Kubernetes as much as possible so that the Petroni installed does not have to be complicated, that instead we're simply relying on the tools that ship with or tools that we can enable with Kubernetes. And to supply a set of defaults that are what we believe just work for most users instead of requiring anybody to go through any kind of a complicated configuration for a simple database that's going to support, say, a web application. For usability, we've got a couple of things. One is, there are some things in terms of controlling Postgres QL that you can't do through the port 5, 4, 3, 2 SQL interface. And that includes things like changing security settings, starting and stopping, initializing replication, et cetera. And so we supply port access to that with a restful interface. Some configuration and plugins in order to support a variety of cloud environments, like, for example, there's a whole set of plugins for different backup tools because for continuous backup to cold storage, just in case you lose your entire cluster, so you still have data. And then the other thing is that, of course, unlike a lot of public cloud providers, we want you to be able to run something that places no limitation on what Postgres plugins or configuration options you're going to run. You know, because maybe you want some obscure plugin for doing, I don't know, biological data on Postgres that you can't get off of Amazon RDS. We want you to be able to install that. So, and then, of course, availability, we're talking about fully automated replication and failover. That's it. It should just work. You've got three Postgres nodes, one of them is the master at any given time. If the master goes away, you get a new master. So now let's talk about a little bit about what's changed recently. And by change recently, I mean this is still a feature branch. So our Petroni design started out actually before Kubernetes was really usable as anything, but we had at CD. And so we designed it where you spin up an at CD cluster to provide a single source of truth. I think everybody here knows what at CD is and what it does. Yes. Yeah. At least by this point in the conference you ought to the and to supply a single sort of truth, run that at CD, separate at CD cluster on our Kubernetes. And then the Postgres nodes communicate to an update the at CD cluster. And then at the same time, the Kubernetes cluster maintains things like I should have three nodes in this particular Postgres set or five nodes or whatever. And they have this storage, et cetera. So and that works pretty well. And that's actually what's in production in several places. However, we're looking at this and we're saying, you know what, that's still more complicated than it needs to be. Why do I need this separate at CD when I already have a single search of truth, which is called the Kubernetes API? So what we've done recently is get rid of that separate at CD as an option. There's actually always going to be reasons why you want to run a separate at CD for some clusters. But again, simple defaults. And as a default, you know, instead of having a separate DCS, everything goes to the Kubernetes API. So, you know, leader elections go through the Kubernetes API to some other structures and everything else. So this is as simple as possible because the idea is now I can deploy a high availability Postgres cluster that consists of three and only three pods instead of needing several different services. So now, because most people in the room have not seen any Petronia presentations before, let me go over basically how this works in terms of high availability with an animation and then I'll go ahead and actually demo it in action. So the basic idea here is we have our container with Postgres inside. We've got the Petronia demon running in the container as PID one. And then what happens is we have Kubernetes.  And then we've got a bunch up and running that are empty currently because we're deploying a new cluster. So what happens is they all communicate with Kubernetes API and they say we want to have a leader election by updating actually a config map and an endpoint. And then they have that leader election and one of the nodes wins. That node becomes the master at which point replication, the other two nodes start replicating from that master. Now, things can happen to the master, particularly we're in a container cloud environment. If nothing else, we can run out of resources in that node and Kubernetes can decide that it needs to migrate that container, right? So if something happens to the master, what happens is the master key in the config map times out and then the remaining replicas will do another leader election. And one of them will win and become the new master. And the other one will start replication from that new master. And this will all happen in the background. And then, of course, Kubernetes will notice that it does not have enough nodes in its stateful set and bring up another postgres, which will then replicate from the existing master, making sure that you have constant this. And then via Kubernetes services and end points, we can make sure that there is always a connection to the master available. So let me actually go a little bit more into detail in that failover because it is a little more complicated than just making a call to the API. So what happens is the master vanishes and it's got this time out. We've got a time stamp on it and so we know the master hasn't checked in in a while. So then the replicas who are checking in realize the master is not there and they grab a lock. Currently, what we are doing for that is we are actually having them redefine the master endpoint because in the way that Kubernetes currently works, that's an operation that only one of them can win. The winner then checks the current replication status to make sure that it didn't already have broken replication before the master went down. If replication status is okay, we'll update the config map indicating that it is the master and then the other replicas who were not able to grab the endpoint will remaster off of that master and you will have a new cluster. So it's a two-stage failover which can affect you if you have a really crappy network because if your network support is really crappy, then it's going to be a little slow. So let's actually do that. Actually, let me show you what's in that before I do this because it creates very fast. So we've got here a sample manifest. This is just a slight modification of the one given as an example in the Kubernetes branch of the Petroni repo. So you can take a look at this yourself later. But basically we just have, we define a cluster name in there. We have labeled applications. We're telling it to do three replicas. Obviously for high availability purposes, I recommend to always at least three because there's a lot of things that can cause two to fail. Then we have our container here. I have a slightly modified image because this is a feature branch and I've been debugging, but it will be merged before the end of the year. Then we just have some definition of the container and everything else and your usual volumes. Then we define a whole bunch of environment variables that are getting passed through to the Petroni daemon in order to configure the behavior of postgres. So the usual things. Password, ports, if we were going to put in any kind of performance modifications from the postgres defaults, like how much memory it uses, et cetera, we would also pass them here. And I've got this commented out because I've got it running in Necube and there's no actual volumes. Then we have to define a couple of other things. We actually wanted to find a separate endpoint. If people are not familiar with endpoints because you might not be. Normally when you create a Kubernetes service and you give that service a selector, then in the background you create something called an endpoint that actually allows things to connect. However, you can create a service without a selector in which case there is no endpoint and you can define your own endpoint. Most of the time people use this to connect to services outside of Kubernetes. But in our case we're using this to redefine the connection specifically to the master node. So that we update only the endpoint. And then we create this sort of headless service with no selector. Now I also want a separate service that's going to load balance to all nodes for read-only connections. So I create that service as well. And then of course we have to have passwords and secrets. So you can see this here. We've got this extra label called role. I'm going to actually shrink this just a little bit. And you can see first a master comes up and then replicas come up. That's very interesting. Sorry about that. That is a remnant of running through this demo earlier. So we've got a master in two replicas right there. I'll show you this actually in the console because it's actually kind of in easier way. So we've got our stateful set of Petroni demo right here. And then again here are all of the nodes. These are actually sort of fun if you look at the we send all of the activity here to the log output. So you get the leader talking about being the leader and you get the replicas talking about being replicas. Which is your main log output. We also funnel of course the Postgres log output to this. In the process of this we create a couple of config maps. And those should also not be there. OK. One of which actually has the configuration of the cluster which would have any non-default items. I don't have a lot of non-default settings in here. One of the non-default settings I have in there is. There's a tool called PG Rewind that can allow us to bring individual Postgres into sync if they've gotten a little ahead during a failover circumstance. Postgres or application details. The more important thing is this leader config map actually gives you one of the several ways we offer to get information about who's the current leader. So this is a lot more interesting. So I show you the failover. So it's killed the current master. And as you can see when network latency is negligible, as it is when you're running this on Minicube, it picks up the fact that the master has gone really fast. When it's slower than that, it's slower than that specifically because of network round trip time to actually sort of detect that things are gone. And obviously when you do things like when you lose physical nodes from your network, it can take a little bit longer for network to time out. And then Kubernetes will spin up a new one and we will have a replica again. And let me show you those end points here. So we've got two end points. One is the master only end point which is pointed at whichever one is currently the master and then we have our load balancing end point that gets created. Now, one of the things that you can actually say here is, wow, that's nifty and that's really super useful. But among other things, if we want to actually, among other things, doing all this pass and configuration via environment variables in Kubernetes manifests is really irritating. And really doesn't help me when I need to actually manage dozens or hundreds of Postgres clusters. So it would be nice if there was another way. Well, there it is. And that's called operators. So how many people here are familiar with the operator pattern and the whole operator idea? Yeah. So I mean, CoroS created the sort of operator idea around a database at CD. And it turns out that it's actually useful for databases in general, probably for other things where I've seen it used as databases. So there's a few reasons why you want to use an operator. One is it makes it easier to keep documented and to get repo the clusters that you're supposed to have, because you reduce the information to the information that's distinct about each cluster rather than having a lot of scaffolding in with your cluster configuration files. For that matter, if you have divided up teams, you can actually pass a much simpler structure set of information to define the cluster so that people don't have to understand Kubernetes YAML structure and how to define Kubernetes objects and stuff and have the opportunity to screw up your Kubernetes cluster in the process. And then the other thing is an operator is actually an active thing. It is a thing that runs on your Kubernetes cluster. And that's important because databases require scheduled work. Like if you have continuous backup running, they have to take a snapshot periodically. Sometimes you want to run other forms of maintenance and that sort of thing. So simply a stateful deployment, deployment, you're not done. And the big thing is, hey, I'm effectively creating this a manifest anyway. So if I'm doing that, why not have that run everything? Why don't I just create a manifest and have everything else be automatic? So I'm going to show you one operator. So like I said, Spilo is the sort of full packaging of Petroni plus the other stuff. So there's an operator for Spilo, which is what Zalando currently uses in their staging cluster. It hasn't gone to the production cluster yet. And what you do for that is you install the operator, you create a manifest for each cluster. If you want to modify clusters, you do that by modifying the manifest. So let's actually do that, although it'll take me a minute because apparently I have a lot of remnants of the previous uninstall. So pardon me, why delete stuff left over from the previous run of this because otherwise it will refuse to build it. Okay. So I need to install a config map for the operator itself because the operator has a configuration. And then I need to actually create a service account for it because this is something that's required for creating operators. And then I need to create the operator. And then you'll see the operator logs in, registers itself. That's the Spilo operator there. So once I've created an operator, I can, by simply defining a manifest and passing the manifest to the operator, tell it to create a cluster. So this is what a manifest looks like for an operator. You know, it's still YAML, but it is not, you don't have to understand the Kubernetes infrastructure to be able to write these, which is really important if you're going to have, say, development teams creating databases for their own applications. Instead, you define some users, you define some Postgres settings. I don't really have a lot in this particular one because I'm trying to show you a simple one. But you can define a whole bunch of Postgres settings within that manifest. So let's actually go down here. And now, no, no, I'm actually going to do, hold on a moment. I still think I have remnants left over from other stuff. So let me spin up a new cluster. Hey, there we go. Yeah. One of the outstanding bugs, if you go to the Postgres operator thing, you will see that there's an outstanding issue for it failing to clean up when we delete clusters. So there is actually an issue there. It's being worked on. But we go ahead and install that. Why are we not seeing the spiral roll? Oh, there we are. So again, like the bare patrony, one of them gets selected, whoops, no, that's not what I wanted. One of them gets selected as the master, then the other two should replicate from it in just a second. The, if we actually refresh, so we've got now our stateful set running our three nodes for that. And the same, some of the same config maps that you would actually see through doing the patrony. It's the same thing, but again, you've simplified it even further. And this has a tremendous advantage that you can now hand this off to development or testing teams who can deploy Postgres clusters as they need them without needing to understand how Kubernetes works. Now, spiral operator is not the only operator out there. There's another whole, another project for Postgres and Kubernetes that comes from Crunchy Data. Hi there. Sorry, we have one of the Crunchy Data staff here that comes from Crunchy Data. They have their own operator that's been out for a little bit longer, so it's a little bit more mature. Their architecture is very different from, their architecture is very different from the patrony architecture. They were very much coming from the enterprise side of things. And so they were really looking at having a very full service Postgres offering with a lot of tooling, which is what their thing, the Crunchy Container Suite, is what it's called in a little bit of.  Design of our operator, the spiral operator is extremely manifest driven and there really aren't like other ways to interact with it. The Crunchy Postgres operator is very command and control driven. And what I mean by command and control is there's a client tool called PGO that talks to the operator. And so if you're using the Crunchy operator, the main way to actually deploy things is to use this client to specifically communicate with the operator. And they have a whole demo and everything else around that. And again, it's one of those sort of trade-offs in terms of how you want to do things. There are advantages both ways. Ooh, that image did not pop in. Sorry. I had your whole, I had the whole architecture diagram of the container suite. Container suite is a lot of things in it. It's also, like I said, it's also a very different setup for high availability Postgres. And I am a little bit too far forward. It's also a very different setup for high availability Postgres than the Petroni setup. Quick comparison, I was actually going through and trying at both operators and going through this. There's some of that, I mean the biggest thing is that the SPILO operator is still kind of alpha because it was created recently. Things move fast, we'll say that forever. Sort of different ways looking in there. One of the main reasons why it's still kind of alpha is we haven't really written any documentation for it, which will change. There's not, yeah, there's not a specific CLI for the SPILO operator. There very much is one for the Crunchy operator, a very sophisticated CLI. Depends on whether CLIs are not. Crunchy is got, is OpenShift compatible now. SPILO is not really both support rolling upgrades of Postgres, which is another important thing that the operator does for you because you can do that by specifying a lot of things in your upgrade strategy for new containers in a Kubernetes deployment, but you'd rather not have to write that since the operator already knows what to do. So, some future work plans because it's active. Number one is obviously merge the Kubernetes native branch into the main Petroni. Second thing is to actually right now the operator integrate the operator and the Kubernetes native branch, which currently currently the operator uses the older version of Petroni. I'm working on OpenShift compatibility because I work for Red Hat and I have to work on OpenShift compatibility. Besides, there's lots of people who use OpenShift and use Postgres on OpenShift. So, the Prometheus integration is a big interest of mine because I like metrics, but I also want to package some other things up for it like Powwow, which is this Postgres performance analysis tool, and then one of the other things that's come up immediately is support for more complex replication typologies. Other things that have come out, some other much more hypothetical things that have come out partly from being here at the conference. Obviously, Istio integration, in order to route traffic, have more sophisticated routing of traffic to the Postgres nodes. The, there actually is a command line tool for Petroni that's used for a few things and right now it's completely separate from the operator and maybe it shouldn't be. We're discussing that. Also, I was interested in Brendan's metaparticle lock CRD, so we're going to look at that and potentially having something that's defined lock operator rather than overloading endpoints to perform that function for us. Support for logical, the new logical replication is obviously a big sort of hypothetical thing and whether or not we could customize the workloads API and maybe be less reliant on operator custom controller behavior. Now there are a few things that we run into in Kubernetes that are still limitations that make life hard for us. There really aren't very many. I mean, mind you, I've been working with a team that does stateful set for quite a while and so there really aren't very many because a lot of the things that I had on my issue list got done. But multi-data center, multi-availability zone cluster is hard to do with stateful sets right now. I should, you know, Kubernetes have a built-in leader election thing that's been discussed before. I'll discuss again. Upgrading stateful sets pretty much requires you to do an operator right now because the stateful set has no intelligence about what order it should upgrade things in and it could. So again, up for discussion and then it would be really nice to be able to do more extending of kube control so that we could stop having all of these separate CLIs, one per operator that's completely separate from kube control. So, but if you're interested in this and there's something going to use, we really would be happy to have more contributors. Open project. The Petroni. I'll admit that we do most of our communication via GitHub issues. It's not a mailing list or anything so if you have a question or something like that, file an issue or find one of us on the Postgres IRC and let's make Postgres boring. I'll take questions in just a second. Research links and now questions. Go ahead. To do. Okay, we're not going to pass this around. So the question was what's it written in? The bulk of so Petroni is in Python and the operator is in Go because operators aren't Go. The more questions. Okay, so the question was the replication is with replication lag. Yeah, the default is asynchronous replication. Which means indeed there is replication lag and if you have an unexpected master failure then you will lose some transactions if they're in flight. However, Petroni is completely compatible with you turning synchronous replication on. There's examples in the documentation of what that would look like and so if you want to run it with synchronous replication then you can turn that on. And particularly if you're going to do that I would recommend using Postgres 10 and running it with synchronous quorum replication so that you don't get replica failure blocking your your right transactions. So there's another question up here. Okay, so right this question was what you missed most was user management and creating users via config maps. The operator if you modified the manifest has a section of the manifest devoted to users and if you operate the man if you modify the manifest it should create the users if you're using the operator. If you're using Petroni without the operator then you can either create the users when you create the cluster because that goes into Petroni config or if you want to create a user later on obviously log in a super user and create the other user but you know which would happen in deployment time but if you're trying to do it in a sort of get repo managed way then the operator is the way to go. Let me just see if it was someone else at a quite some yeah. What about storage? How do you specify storage? Well the same way honestly that you'd specify it in well for Petroni in general you're specifying storage just through the Kubernetes API right? Right. The there's a section in the operator example where you tell it the storage selector and that's the limit of the discrimination it has. Yeah yeah. Okay we're over time so further questions will have to be taken out in the hallway. Thank you very much.