Code green think tech Hawaii. Welcome to our small army of viewers this afternoon. We have a wonderful guest in the form of Melanie Islam who is with Mason architects, and she is a lead certified professional and does some really exciting work that we will hear about very momentarily. But first, I had a bit of a shock couple of days ago, and I wondered what in the world, we green advocates had been smoking, because a conservative friend of mine said Howard you've got to listen to this fellow here. And turns out he's a financial advisor, and he loves well he doesn't love oil he says green PV's storage wind. That's all very fine, but oil is our only way out he didn't quite say out of what, and he was really convincing. I said, what, again, what have I been smoking around here but then I went back and listened again. And as an energy professional, I started put punching holes in his argument until it looked like just pure Swiss cheese. But he made a point two points that I had not thought of before, namely, if we're going to power the whole world with photovoltaics plus wind plus storage. We need gazillions and gazillions of PV panels, and gazillions of great big batteries, and each of those wind machines takes a heck of a lot of material and it turns out that the material needed for all three products includes a lot of rare earth metals. And a lot of those metals come from nations that don't like us so much. What are we going to do, what are we going to do, that got me thinking efficiency. That's my Kuliana PV wind storage. That's not enough we need to shift gears into what I'm terming hyper efficiency look for all the other avenues to get us to 100% clean energy. And the first thing that came to mind was building and home restoration. There's a heck of a lot of energy inherent in each building think of all that concrete all that metal all that glass. In the case of my home you can see above there that's a heck of a lot of wood that went into this residence here. So that was a cornerstone and then just to back it up this morning I saw where the European Union is going to make restoring buildings a capstone of their European new green deal. All of which sets up miss Melanie Islam because guess what she specializes in restoration of homes and buildings, and she does this beautiful work. And on that cherry note. Why don't you take it away miss Islam. Thank you Howard so much. Hi everyone. My name is Melanie Islam. I'm an architect sustainability director at Mason, formerly known as a stock tech. So like Howard said we're an architecture historic consulting practice. We've been practicing in Hawaii Honolulu for nearly 30 years. So we have threw a lot of feet in the ground a lot of experience. But you know Howard you brought up an excellent point you talked about all that energy that is living in our existing buildings, you know, we're starting to sort of talk about this conversation. So we're starting to sort of look at embodied carbon, you know, as architects, you know, the AIA just updated sort of our code of ethics, but more importantly, sort of updated our charge and our responsibility to the environment. And one of the big things is how do we more succinctly tie in with our work reduction of carbon and looking at sort of the team energy goals. And in fact, looking at existing building is one of the big things that we need to start doing. We I think all know I think everyone in this audience that that building basically sort of admit about 40% sort of emissions right in the larger picture. And how do we sort of bring that down. We don't often have the opportunity to sort of build all the time so we have to look at how do we preserve our existing building stock. Some of those historic treasures that will never be able to replace and how do we have modernize that and make that more efficient. So I'm just really happy to be here here to sort of share some of the work that we're doing at Mason. That's what I call currently in construction and some upcoming things that are in our community. So, Howard, how do you want to, how do you want to do this. I would like to. Oh, first I want to point out that. Yeah. Another advantage of doing restoration or reuse is creation. Guess exactly like, yeah. Hawaii and the unemployment rate is bad at about 8%. Last time I looked our unemployment rate was it had come much better than in March or April, but it's still about 8%. And a lot of those who could be searching for a new career other than the washing dishes and wiki key or whatever, and restoration involves skilled labor. You need to get training and you have the once you get that job, it's a well paying job. So there's another advantage to new building. So why don't you agree. Yeah, your projects. Sure. Yeah. So I thought I'd share a couple of the work that is currently going on in a couple of projects that I'm working on that deal with existing building review, adaptation of those buildings. And it's also secondly, how to be sort of deconstruct those buildings and sort of extend the life of the materials and then. So the first one that I wanted to share is a project that we've been sort of working on on coconut island. So I think a lot of people know what coconut island is it was sort of in the middle of honey oh hey bae. We've been fortunate enough to sort of work on one of the first structures there for the university and for HIV Hawaii Institute of Marine Biology. It's over 30,000 square feet. It's actually two structures. It was originally one of the first lab buildings there, but what like many of sort of entitlement work and how long these changes over time. The building itself fits in an FMA so so we're moving the structure was basically not possible, but we need sort of programmatic needs for the university. So we were hired to look at how do we modernize the structure that we did was sort of brought the building down to its bones were essentially retaining a lot of existing shell and the roof all that sort of body carbon that lives in the masonry that lives sort of in that sort of light gauge steel that structural sort of steel assembly we're preserving that and we're not modernizing the building by bringing it up to code. So we had to fully grow out the building. But the really important thing is the university as a whole sort of working towards a larger energy efficiency, sort of target right energy efficiency reduction things by 2030, 2035 they sort of have all these sort of incremental timelines. And our intent with this project was to ensure that the work that we do at HIV, what sort of means those targets. So as an existing structure we have to look creatively at how do we insulate the building how do we sort of beef up the thermal performance at the walls at the windows and as labs they consume a lot of energy. How do you sort of bring the energy footprint down and then how do we couple and add to the renewable technology to that. So I think a lot of you guys are sort of seeing the slides the top one there is really what the building looks like gutted. So essentially we created a new palette to work from. And then we sort of created a bunch of new sort of laboratory buildings we have two types of labs in the structure, one that fully is naturally ventilated and sort of daily that really helps to sort of bring down the energy footprint. We're, you know, practicing a lot of the tenants that were sort of brought up in our new 2015 I co adoption, which was the passive ventilation through the space use the feeling found to the mood that cool there through. So that's all the ground floor and that really works nicely with the program because that's a fully sort of wet water, water lab that where you're, you know, scientists lab assistants are bringing in all these sort of what sample so this space needs to be durable, but it needs to be efficient, it needs to be nicely ventilated and daily. This image which is like on the top. It's on my top light might maybe find your top left, but that's of your more typical sort of laboratory building that has those really sort of tight air change changes, which are like four to six, you know, their handle changes so that requires a lot of energy, and also sort of has the traditional sort of human that kind of pulls a lot of energy as well too. So this building what we did was we'd be set to installation on the roof. We put it in sort of installation and thermal resistance in the wall cavity, we'd be steps of the window assembly. And then we sort of modeled you know we we kind of sort of study the daylight in the building and we situated sort of window glazing performance as needed. So, overall, we brought down the building used to about 20%, which was above the baseline, and then adding our renewables brought efficiency up to 40% as well too. So in all, it's a really great I think story of how do you use this in structures that can be modernized, but yet can still meet the code and through the larger sort of intense of our state. So this is a really great project that has a lot of great points to it, national ventilation efficiency, daylighting renewables. The other project I wanted to talk about is, you know, you Howard you sort of talked about single family residents, right, they are everywhere. Single family residents is something that we're all going to be sort of impacted by but we can most most important sort of connect with the next project is it's not a reuse project. We would call it in the traditional sense, it's a more of a deconstruction and we sort of built new built new with intention of being energy efficient, and sort of bridging together sort of the contents of what the client why in terms of a contemporary lifestyle but you have a larger goal. So this home, it's your traditional single wall construction home in Hawaii. One story has a combination of Redwood, you know, single wall plan plus a combination of mainstream. So we were really concerned and the client was really concerned about what their footprint would be. So what we did was we worked with reuse Hawaii and we actually deconstructed the home and that was a great thing because it allowed us to divert about 70 to 80% of the construction rate. We have landfill sort of issues where we're not able to sort of bring the waste there or not able to sort of look at reuse. So doing this allowed us to take something that would traditionally probably go in the landfill bring it to Hawaii and have someone else sort of upcycle and reuse it. So that in itself is a good way of how do you sort of salvage material salvage or that embodied carbon and energy in it for someone else to reuse. So with that then it gave us sort of a new place to sort of rebuild in the structure. So we built a new structure to dwelling sort of has an 80 attached to it as well so it looks and starts to address some of the housing issues in Hawaii. But the home again sort of designed to be as efficient as possible we use natural ventilation and sort of daylight at the baseline. The home has cross ventilation throughout a lot of windows but a nice sort of balance into wall ratio. So we added a 35 kilowatts or the PV array to sort of both structures to allow us to have some renewable efficiencies in there. We also plumbed portions of the house to have additional PV put in for that so that the owners can start to sort of see how they're using the home and sort of add array as needed. So those are like some of the works that we're doing at Mason to sort of just address the synergy between efficiency renewables and existing sort of building stock. What else are we doing. How I know we talked about a couple other things. Oh yeah, really great project. I think many of you guys know, you know recently sort of in the news by WC for hers was in the news a lot for the great social impact work that it does with WCCC. So we may have been really fortunate to have a great partnership with the by WCA and sort of work with them at their first facility and look at ways of, I think, more importantly, how do we how does the facility support the mission and purpose of a nonprofit. So it actually first started off with sort of volunteer work that I was sort of doing with the YWC sort of three years ago we worked a lot with sort of with the women that come from WCCC and sort of lived at the Y and envisioned and talked stories with them of what they saw the future to be like at the facility. And when it pointed out, you know, and I think a lot of these things are coming up in conversation about social equity and access right. What it pointed out is that what privileges are taken away from women or men when they get in custody and one of the first things that they want back. And we found that gathering was important, having sort of ability to connect with people and spaces to provide that, but also providing spaces that allow women to sort of build their independence and build sort of that social structure that was deconstructed when they were incarcerated. And one of those first things starts with food. I think food is a piece that connects with all of us writing allows us to bridge connections make conversations. So we started with food and we identify that there was a need to have cooking there for her has always historically been sort of like a lodging facility for women. But it doesn't have any cooking facilities that you know it just just sort of has a serving kitchen so a lot of the food that comes in and out of that facility. So our efforts were through restaurants and nonprofit organizations that bring lunch, breakfast and dinner meals. So our first thing of work was really to provide what we're calling the home kitchen where women can build upon their independence. And they can cook their own meals. And so, sort of a couple of that we worked with a great suite of engineers are sort of all donating our services pro bono to look at then what is the water efficiency in the space. And using opportunities from the cares act funding which they receive and also Hawaii energy energy relief ramp, which is afforded to nonprofits to upgrade lighting efficiency throughout the space. And really just tie in some of the things that we do in all of our sort of private and public practice, but bring it down to a level where it's accessible for everyone. And that everyone knows that, you know, getting access to good quality design good quality products and efficiency is not something of privilege. It is equitable and accessible to everyone. So this is just a great project that we've been working on with providing a home kitchen, we're benefiting a couple of their dining and multi-purpose spaces, some of the living room gathering spaces. We're also looking at the roof repair. So we're adding insulation on so we can improve the thermal comfort of the total facility. So we're doing sort of like smell, but I think impactful work to the envelope and to them for the building. So that's, that's just one other project that I thought I'd share that feels to an existing structure looks at efficiency and whatnot. So, yeah, that's our work coward. Thank you. Thank you. One important point you are indirectly making is technology is improving hand over fist. There's something a law called hates law. It's named after the electrical engineer, and it says that 18 months, the cost of LED lights shall drop by 25% while the efficacy, the efficiency of the lamp shall increase by something like 50% and there is an uproar that. Yeah. Yeah. Yeah. And so I'll give you a little quiz. I needed some lamps. Oh, gosh. Okay. It's a lamp that's giving me illumination right over here. This is an old what you used to look it looks like an incandescent lamp but these days it's an early and it's just a plain old screw in nothing fancy about it. Guess what the C was is 85 up 93 100 higher 116.6 lumens per what we measure miles per gallon in in lights and this is just plain the shelf thing. Oh, here's an you're too, too young to remember incandescent lights. I know incandescent lights. I had that in the house I grew up in. Yeah. Yeah. What is the, what is the efficacy of your typical incandescent light. I don't know 40. I have I should know this is this really bad 2015 10 15 lumens per what 106 lumens per what is an increase of seven fold. Yeah. Yeah, for sure. Yeah. The Latin word for white hot heat is incandescent just what that light is called the incandescent light. Yep. The state senator measure the heat of 100 watt incandescent and he got 465 degrees. Okay. And what at what temperature did you use to bake cookies. 300. I think so I don't bake. I'm thinking like what I cook. Yeah. Energy codes are all building codes. Turn over or get renewed every three years. Why? Because it's you talking about here. Technology is so, so, so rapidly now. So that that's why we I can see where you keep up with all the codes. I love the code. Yeah. And just regarding fernhurst, you know, it did. It benefits all of us when you're helping disadvantaged people be they men or women, because they don't have the skills that really basic skills that most of us grew up with. Yeah. You know, I, so I've actually, I'm just been involved. I've been volunteering for the Y. I think long before for her, but I found, you know, that statement is true. But I also found interestingly, like someone like myself, right? You just get into some hard time to easily also be incarcerated. Those stories actually have the most impactful with me, someone that could be my kin to be in my cohort and my age group. I think it's like all these things that happen, but I think the main point is that, you know, good design access to efficiency, integration of all these principles that we're talking about and knowing that you have an impact and a role in making a positive impact to environment is afforded to everyone. It is not a privilege. It is not something of entitlement. We can all do be part of it. I'm just a matter of, you know, how do we break down those barriers and start those conversations? Yeah. And speaking of energy efficiency and this. Yeah. We were living in a COVID age. And putting on. Webinars or hosting webinars. About. UVC ultraviolet C. Lighting. Yeah. And they, they, that, that spectrum actually zaps the COVID. It disables them. So. Yeah. The question is how to get, get it safely in there. But one of the things I have found in doing considerable research. This ties into what you're doing too.  This ties into what you're doing to your. You're promoting the natural ventilation whenever possible. That's one of the most effective. Means of avoiding the COVID is being outdoors. Because all of this air circulation. All over the place. behind somebody who has the COVID up there, some of those little wrestles are going to reach you, but there's something called dose. And mostly them are flying all over the place. So your dosage is very, very low and your immune system can handle it. So you're talking about natural ventilation coming through an interior space. That is also a very, very effective anti COVID measure. Yeah, for sure.  So we're going to install a couple of them into the facility and see how they work. Great, great mindset. Thank you for bringing that out. Yeah. Yeah, yeah. And yeah, speaking of fans, I'm, I'm the energy codes guy, as you may know, and I'm introducing into the next generation of codes is mandatory ceiling fans, even for, this is in residence, even for centrally air conditioned homes, because the typical planes is, is centrally air conditioned.  And again, you've got this great air circulation and you're reducing your COVID exposure. For sure. Yeah, exactly. That's great. I look forward to seeing that and hearing the conversations related behind that, in our community. Yeah, we always get pushback from the builders. Howard, that's kind of fun. I know. I know. I know. I know. We need to take bold actions though, Howard, in order to sort of really sort of reduce our carbon footprint. You know, we can't do these incremental little changes. We need to take bold actions. That's why I'm going to, yeah. Yeah, you're speaking my language here. Yeah, I thought I shared systems. Yeah, we've just got a minute or two left. Do you have any events that you would like to announce? Yes, I would. I just want to take this time. So, coupled with just the work I do, I also sit on the board for you to see Hawaii. It's a U.S. Green Building Council that administers sort of the lead program, a bunch of other services programs from Green Building Services. And so, we have upcoming a town hall with our CEO, Mahesh Rabantaman, on October 5th. And so, it'd be a really great opportunity for our community to, we're intending to have an engaging session. It's going to be a conversation really to talk about the future of U.S. UBC, how it's working in Hawaii, and how can the role of Green Building help to sort of advance our economy and sort of rebuild in this post-pandemic time. So, I welcome you all to sort of join. There's a registration link on the image above. We also have the next day, some training to really sort of, we were talking earlier about workforce training. We're going to be doing training on Lee Daryl and Lee Carbon. So, how do we sort of bring, looking at how do we bring our sort of building usage down and also a couple of that are kind of footprint. So, that also sort of has a registration link. That's going to be sort of a session where people that are professionals that new CEUs, you can get three hours of CEU credits. So, we look forward to having that on October 6th, on the 8 o'clock in the morning. And the last thing that's really kind of personal to me, is Hydrogen Day. That's also coming out this week on October 8th. I think personal, because I have a hydrogen vehicle, and I did that intentionally sort of start to look at how can we support alternative technology to kind of, you know, be in parallel with pure electric vehicles to kind of get to a clean energy world. So, we're kicking off our first sort of Hydrogen Day on the 8th. It's intended to be a really fun, sort of morning session. I think it's headed by Blue Pan and I think even also the State Energy Office. So, yeah, tune in. And then, yeah, but those are the three events that are coming up next week. So, so hopefully you guys can join if any that are of interest. And did you by any chance include a slide that includes your contact information? Oh, yeah, I think it's like one of the last ones. There we go. Is that it? Okay, because we've got about 20 seconds left. So there is Melanie's contact information. She would love to hear from you and keep up the wonderful, wonderful work, Melanie. Thank you Howard. On a tidal wave. It's a small wave now. The whole renovation thing turned into a wonderful tsunami. And on that, we hope so, yeah. We must bid on farewell from Sink to be Howard Wigg. Say, Fawn farewell, Melanie. Bye-bye. Bye.


My name is Wayne Kelly, I'm a lecturer here at QUT. I'm going to be talking about the problem of parallelization. So I first want to define the scope of the particular part of that issue that I'm trying to address. Don't explain what's wrong with current approaches for dealing with that particular sub-problem and then describe a novel approach that I'm trying to follow to solve in a different way and a tool that I'm developing to support that technique. So parallelization, there's two real scenarios here. If we want to write a parallel program. Now sometimes we start off with no program, green field, and we start off from the very beginning intending to write a parallel program. So from day one, we're thinking in terms of threads or processes, and we're thinking in a parallel way. There are many parallel applications developed in that way. But equally, there are other applications where we start off with a sequential program either because it's a legacy code that's existed for many years and now we want to parallelize it to improve its performance. Well, sometimes simply it's an easier path. We first want to create a simple sequential program, get it developed and debunked, and then worry about optimizing it and parallelizing it. So let's assume we're starting with a sequential program and we have the task of parallelizing it. Again, within that scenario, we have two options. Sometimes we need to fundamentally change parts of the program, change the algorithms. The sequential algorithm is just not suited to parallelism. We have to choose a fundamentally different data structure and different way of doing things. Now, that sort of algorithm change is beyond any sort of automated tool support. It really requires human and intellect to make that sort of change. So I'm not addressing that scenario where a fundamentally different algorithm is required. What I'm considering instead is where we're going from a sequential program and making it into a parallel program simply by exploiting so-called inherent parallelism. And let me explain what I mean by inherent parallelism. Here's some trivial examples. Example on the left, we see here, just one moment, so I can point a little more easily. You can see here this third statement on the left-hand side. It depends on the first computation and the second computation having completed. But there is no dependencies between this first statement and the second one. So we could, if we wanted to, execute them in parallel. A more interesting example is perhaps where we have a loop where the original program is to do it sequentially one step at a time. But in this example, since all we're doing is initializing each element of an array, there's no reason why we can't do all that in parallel. I'm not fundamentally changing the algorithm. I'm just exploiting parallelism that existed in the original program. So that's what I'm fundamentally trying to do. Now, you might ask in those previous examples, well, you could parallelize those two assignment statements, but would that really lead to a speed up? So let me first explain my overall view of the world, the parallelism world. I view the process of parallelizing the application as three sequential steps. You first decide which parts of the program could you run the program in parallel, could run in parallel, if you say chose, and still get the same result as the original program. Now, that requires analysis of dependencies, which I'll talk more about in the next few slides. Having decided which parts could be run in parallel, you then have to ask yourself the question, which of those are actually worthwhile running in parallel? The overheads of synchronization and granularity, depending on the architecture, may not make it worthwhile. So that requires performance analysis profiling to decide which parts worth parallelizing. Then finally, having decided which parts to parallelize, you then actually need to do it. So you need to create your threads, you might do it manually, or you might use OpenMP or MPI or whatever your favorite threading language or libraries. A lot of the talks we've talked about heard today, various languages for doing parallel programming, they really address the third question, how do you actually express the parallelism? They don't really help you address the first question of how do you take a sequential program and work out which parts are actually parallel and which parts aren't. So my research is all about that first question. Now, you might say, well, that's trivia, I write the program and I know this loop is parallel, or it's not. Well, you may not have written the program, but it's very large or complex, and even if you write it yourself, there may be some subtle little data structure using somewhere which does introduce the dependence between one iteration and another. So it's very hard for a large complex program to convince yourself that what you're parallelizing is actually safe to do so. Now, this issue of parallelizing, it applies equal, all the problems and issues, applies equally well whether it be some sort of automatic parallelizing compiler, a tool trying to parallelize, or a human trying to do it manually. It's hard for both of those entities to perform this problem. Caracom pilers aren't smart enough to do all the fine-grained static analysis that's required, and humans don't do this very well either. It's very time-consuming, requires great skill, and it's error-prone. So we really don't have a solution to this problem at the moment, there's problems on both sides. So let's consider the sorts of dependence analysis we need to do to decide whether code to be run in parallel. There are two different types of dependencies that exist. This is parallelism 101, I'm sorry for boring those, ready for me with these basic ideas, but we've got a broad audience here, so I thought I'd first better define these ideas because the rest of the material relies on an understanding of data dependencies. We firstly have control dependencies. So here I have a loop that might and might be a thousand iterations which I'm trying to parallelize. But if we look inside the loop, we have this if statement here, which says if some condition is true, then we wanna break out of this loop early. So we can't parallelize this loop that has a thousand iterations because we might find after five iterations we need to stop not actually do the other iterations and doing so would produce an incorrect result. So control dependencies can get in the way of parallelization. I'm however, so I do consider those, but it turns out the data dependencies are the harder ones to analyze. And we have three different types of data dependencies, the obvious. Can you have a question? Yes. So on the control dependency, wouldn't it be a case of the ratio of the size of the array versus the processing? Because if you had enough processing, then it might be worthwhile to just put the output somewhere else. Sure. And then when you get done, copy where it's supposed to go. So there are workarounds for dealing with these control dependencies. The point is, in the presence of them, you can't just trivially parallelize stuff. So that's how it was true of data dependencies. It may be data dependencies in their way of working around them, but it means simply in its current form at least, it can't be trivially parallelized. So for data dependencies of flow dependence, in this case here we've got some statement that writes some value and then later on some instruction which is reading it. We need to preserve that order. Otherwise we won't be reading the correct value in the second statement. So that's referred to as a flow dependence and we have similarly output dependencies and anti-dependencies which result from a right followed by another right. You need to keep them in the same order or when we're reading some old value before it gets overwritten by some subsequent instruction. We need to preserve those ordering constraints, otherwise we might get a different result. Well, let's say it is sufficient if we preserve all these dependencies, then we are guaranteed for it to produce the same result. Now that was trivial in that previous slide because we have lots of very simple A and B, just integer variables. You can tell A is different to B. They contain simple values. We get to arrays as we might find in a scientific application that's typically run on these high-performance computers. We ask, can we parallelize the i-loop? Can we parallelize the inner j-loop? We have to ask ourselves, are there any inter-iteration dependencies? So any iteration of one loop that reads some value that's read by one of the earlier iterations. Now, automating parallelizing compilers can address this sort of numeric kernel by asking, does there exist iterations i and j for the reads and the writes such that various constraints exist? And you can solve this sort of problem. It's an NP-complete problem, but you can still solve it and you can get a yes or no answer. So there are automatic parallelizing compilers that use these sorts of techniques. Now, the fact that it's NP-complete is an issue, but that's not really the main problem. The main problem is that this sort of analysis only works for all of these loop bounds and these index expressions are very simple, whether they're affine. When they start getting to be more complicated expressions, the compiler can't analyze them anymore. So if the compiler can't analyze them, it says, I give up, maybe there is a dependence we better not parallelize this loop. So they must be conservative. It gets even worse when we move beyond four-trans scientific codes which deal primarily with arrays to modern RO type languages, C, C++, Java, where we have pointers and references. You might have two different methods being called here, one on object A, one on object B, but we don't really know whether A and B might be the one and the same object. And to actually track whether they may be aliases or not, we have to analyze the entire program, which it's very complicated when we have very complex inter-procedural data flow where these references are being passed all around our program. It's complicated by these pointers and references.  You don't know which actual implementation might get called unless you know the subtype involved.  You call some method, but the actual implementation of that method may not even exist at this point. It may be developed by some third party that'll be plugged in dynamically at runtime. So all of these make it very hard to do static analysis that allows to prove that a particular loop or part of code can be run in parallel, which is why most of the majority of the techniques to do automatic parallelizing compilers have been restricted to Fortran type languages where many of these complications don't exist. But even in the Fortran world, the problems are not solved. We can't still parallelize many Fortran programs. The programs are still too complicated to do precise enough analysis to prove that various parts of code are parallelizable, even though they are actually parallelizable. Now, so when we do any sort of static analysis, it's conservative if it's inexact, which means we need to overestimate the dependencies that might actually exist, which means that we're underestimating the amount of parallelism that we can actually exploit. So for really large object-oriented complex programs, if you try to do any sort of static analysis on it, most times it would say, well, there's a little loop here and a little loop here can parallelize, but all of the really interesting stuff, the outer level, coarse-brained parallelism, it's just gonna be too complicated to analyze. So it will say, no, I can't parallelize it and you've got virtually no parallelization from an automatic tool these days. So the question is, how much parallelism are we missing? There's been a lot of work parallelizing scientific codes, Fortran codes for supercomputers, but what about your everyday business application written in Java? How much parallelism is in these sorts of applications that we're not exploiting today? We don't really have a good feel for that issue. So I'm trying a different tact since we can never be precise and perfect with static analysis ahead of time trying to predict what will happen. Why don't we wait to run time and measure what actually does happen? Because at run time, all that uncertainty about whether this pointer might be an alias to this pointer over here, which method gets called at run time, we know all of those answers. We know for each instruction, as it executes, it's actually what memory locations is at read and write. We know exactly what methods get called. We know what branches are taken, which exceptions get thrown. So the proposal then is to instrument the code to not just do what it originally did, but also keep track of all of these memory reads and writes. So it's actually measure which data dependencies do actually exist or materialize at run time. So the basic idea as the program runs, we've got some instruction A, an instance. So I'll pick up a machine code instruction or IL level instruction. It executes and it's going to read or write various memory locations. So let's say this instruction here reads this particular memory location here. Later on as the program executes, then to some other instruction B that also happens to read that same memory location. So if we can actually record that fact at run time, we can infer that there was a data dependence from instruction A to instruction B. No doubt about it, it definitely exists. I've shown it here in this program run, it definitely exists. Now the observant among you will realize there are obviously problems with this approach. It may not find all the dependencies. It only works for this particular data input. If we give the program some different data, it may find some additional data dependencies. So we can certainly run the program time and time again, giving it different data sets, and we'll find more and more data dependencies. But we can never be certain we've found all of them. So I can't use this technique as a way of saying, well, based on the output from this tool, you can definitely parallelize this loop. It can point me in that direction, it would can tell the programmer, hey, this looks like a section of code that appears to be parallel. It can also tell the programmer, hey, this loop here definitely cannot be parallelized, at least in its current form. Maybe by transformation, you can change it. And it can tell you not just that it can't be parallelized, but it can tell you precisely why. It can tell you exactly what data dependencies exist at the moment that would need to be removed in order to facilitate parallelization. So it's interesting therefore to look at the relationship between static analysis and dynamic analysis. Static analysis gives you an overestimate of the data dependencies and an underestimate of the parallelism. And for runtime analysis, it's the other way around. We underestimate the dependencies and possibly overestimate the parallelism. Now the truth obviously lies somewhere in between. So if we use a combination of static analysis and dynamic analysis, we can squeeze the actual true amount of parallelism somewhere in between. Okay, so to support this kind of approach, this is a very simplistic crude description. I'm leaving out all of the optimizations. But the basic approach is to create a node for each instruction as it executes that might do a read or write. We then keep track of two dictionaries or hash tables if you like, which map from the memory location to whichever instruction most recently wrote to that memory location. That allows us to compute flow and output dependencies. If I also want to compute anti-dependencies, I not only need to record who wrote it most recently, but a list of all the instructions that have read that location since it was last written. And of course, if it's a garbage collected language, then we need to worry about that because we're simply tracking war memory addresses, but the garbage collected could come along and move that object from here to here. Now logically it's still the same object, but the actual physical addresses have changed or it could be garbage collected and that memory now means something else. So we need to be aware of that as well. Now if we want to determine whether a particular loop is paralysable, we can't just have this big collection of instructions. We need to know that this instruction belongs to a particular loop. So we create nodes not just for the instructions, but also for method iterations and for method calls. So what we're doing at runtime is keeping this dynamic call graph effectively. So as the program runs, we might have method A calls method B and inside of method B we've got a loop and in loop iteration one, inside of that loop we call method C. Method C has a loop inside of it and inside of the fifth iteration of that loop, then we execute some instruction which happens to write to this memory address here. Somewhere later on in the program, some other instruction executing in some different call chain also happens to read that value that was written over here. So there's a dependence between those two instructions that in order to actually make this meaningful and to present it to the programmer, we need to lift it back up, we need to follow this tree back up to its greatest common ancestor up here. And we can infer that there's a data dependence from this loop iteration to this loop iteration which tells me that this loop that exists back in method B cannot be paralysed because there's a inter iteration dependence between them. So the actual reads and writes are happening way down low in the call chain that we lift them back up to find the level at which the actual dependence exists at a single method level. We can then present that to the programmer. The programmer considers one loop at a time and it'll say yes, well I couldn't find any dependencies between loop iterations or if I did, I visualized them to the programmer via a graphic error. So I say there's a dependence from this statement here to this function called down here. And if I wanna know precisely how it arose, I can click on the arrow here and it'll give me the two call chains corresponding to the source and the destination of the data dependence. So we're trying to integrate this sort of visualization into an IDE in the tool that I'm developing and also support for refactoring. But it tells me that it isn't paralysable, perhaps I can do some sort of renaming or various common refactoring which will eliminate dependencies. So those sorts of refactoring will be built into the IDE as well. So what else can we use? We can determine which loops appear to be parallel and for those that don't, we can tell precisely why. So the example I've described before, we consider all types of data and control dependencies, flow, output and anti-dependencies. But if I consider only the flow dependencies, the writes followed by a read that sort of true genuine dependencies, if I leave out the output dependencies and the anti-dependencies, I get an even higher degree of potential parallelism. And that's the parallelism that I might potentially get if I can perform transformations that eliminate output and anti-dependencies. So there's things like loop privatization, variable renaming that can eliminate output dependencies and anti-dependencies. So by considering just the flow dependencies, then I can get a higher degree of parallelism. I can also, this will give me the amount of parallelism and say, this loop's parallel, this loop's not. But there are other types of parallelism, not just loop parallelism. There's parallelism you might get from recursion or from task-level parallelism. So if I want to get some sort of measure of total parallelism in the program, at least from a theoretical point of view, I can measure, in terms of instructions, the critical path. Let's say this instruction can't execute until this one is executed, which can't execute until this one is executed. So if I take the longest path through the program, that's how long it will take to execute on a processor with infinite number of processors. Divide that by the total amount of work, the total number of instructions, that ratio gives me a theoretical measure of the amount of parallelism. So what's this tool about? Well, it certainly can help programmers guide them in parallelizing specific applications. I've got their program, they're trying to parallelize, this tool can help them with that. From a research point of view, I'm perhaps more interested in an alternative use, where I'm not trying to parallelize a particular application, but I just pump in every open source program, Java, C++, whatever that I can find, general purpose, all sorts of genres, and simply measure how much parallelism do I typically find? Is it in loops? Is it recursion? Is it map-reduced parallelism? Is it coarse-grained? Is it fine-grained? Does it fit the patterns that we already know, or do we need to develop new analysis and new parallelization techniques? So I'm using this as, I'm hoping to use this as a meta-research tool, that not just helps parallelize existing programs, but will measure the actual true parallelism in a whole range of different programs, which will help us better understand that task of parallelization generally. So we have a prototype implementation, which we've developed, a number of different front-ends and back-ends. The front-end is a part that does the instrumentation, the collecting of the data, and the front-end is, or the second part, is the visualization, the IDE, for presenting the information to the programmer and allowing the refactoring of the program. So my prototypes, the first one we've developed is for the .NET platform, which uses the mono-open-source .NET implementation. It's actually built into the JIT. So we take ILL code, and in the jitting process, we add the extra native code instructions to do the instrumentation. I've also started work on a front-end for instrumenting native code using the Intel PIN tool for doing instrumentation. Both of those different instrumentation phases will produce the same form of XML output file that has record all the dependencies which were measured at runtime, and then we can then feed that into some sort of IDE. We're working on both a visual studio, one at the moment, and potentially an Eclipse one in the future. That's it for me. If we've got time, I'm happy to take questions. Thank you very much.


Welcome to yet another devlog episode of me complaining about the floating panel that I implemented doesn't work as well as it should. Why am I still milking this floating panel thingy because it's still giving me pain and hours to debug? So what I'm going to focus on right now, what I have focused on is to make sure that when you maximize a window as an example, let me actually finish the animation with all the patterns. Thank you to all the patterns, by the way. But when you maximize a window, as you know, the floating panel deflates and expands to fit the space. And like this, you see, like this, it becomes bigger. And people have complained that it's too thick, fine. Okay, originally it was supposed to be a little thinner, it was supposed to move down a bit and not expand vertically, but it would still have been more thickness. Why did we change that? Well, basically the issue is if I have a thing like this, and another like this, well, these two windows are not detected as maximized. They're just one on the side of, you know, they're on the sides, which means that the floating panel doesn't actually, you know, become thicker, it's still floating. And you can see that there is a nice margin all around the panel like this. The only way to deflate it a bit thinner was to actually give up on this margin, which means that these two windows would have touched the margin, even though it was the panel, even though it was still floating, which we decided was not good enough. So, so, so, so I decided to try to tackle the issue a different way. And what I'm trying to do is actually make it deflate without gaining any additional thickness, just going down, which is what most people request. And as much as it is a fair request, there is a reason why I didn't even attempt to implement it the first time. And that is, it's a mess. It's a mess, it's a mess. Why did I do this? Why? You convinced me to try this out and I'm just wasting time. So, let me go through it. Firstly, my big issue is that you see that it is working maximizing a window that does work now. But of course, I don't develop on this system. I have a plasma shell that I built myself with all my testing changes. But I don't know why. I have no clue why. But when I turn that up, as I'm going to do now, sorry, just a second. Well, it just doesn't work for some reason. For some reason, when I maximize my window, it doesn't work. I don't know why. I don't know. It maximized windows are not detected as maximized. And why should I even try to debug that? I'm trying to work on something else entirely. So what I did, I just added a timer, as you can see, that maximized the panel and then they maximize it every few seconds. So I just can test this out. And as you can see, it's working. You might look at this and say, wow, you did it. I did it. That looks cool. It goes down. It becomes opaque. It works. What's the issue? Well, let's try to maximize something. Well, what happens is it kind of works in this case. But you can see that the application that is actually touching the panel resizes to, obviously, and it does not resize in a pretty way. Now, all applications actually resize slightly different. Let me pull up, like, all in as an example. This happens. That's bad. Like, I think we can agree that this doesn't look good. But there is to say that in theory, in theory, the defloting shouldn't, in theory, happen every few seconds as I have with it with the timer. But it should happen when you maximize a window like this. So what we could hope for is to make the animation fast enough that the length of the animation of the application, you know, maximizing actually covers like heights, the fact that this animation of the panel floating and defloting would mess with application. Because if they both happen at the same time, you don't notice that one is messed up. That's my hope. How can I, I mean, can't I just test it? And no, because for some reason, my plasma setup cannot currently recognize whether a window is maximized or not, which is annoying. Thankfully, thankfully, I received a package. Let me actually bring that up. Okay, so I was saying, look at here, look at this little computer. It's not little. It's actually super pretty. This is the kiddie's limb book. And it was sent to me as a review unit by a limb book. Thank you so much as limb book, by the way. And of course, I'm going to do a review of this device. And I am immediately going to put it at work, because this is a machine that is for now untouched by me, which means that it still works, which means that I can try out my patch that for some reason doesn't work on that computer. But I can try it out if whether it works on this computer. This computer should still work. So my current plan is to build KDA plasma on this limb book, try out my patch on this limb book, and decide whether it makes sense to even attempt something like that. Because yeah, it has some holes. It's not just about animation. You also get this flow that if you have like dolphin hop and white, come on, this is this computer so messed up. I don't even get the, it crashed. Beautiful. Beautiful. Thank you. Okay. The fact that the panel moves down also means that the click targets are moving by a few pixels, sure, but they are moving. So in theory, that's already not too good of an approach. But I can understand that maybe the visual benefit that you get from it, that is the panel not getting any thicker is worth it. So I'm trying that. So if it works, we can discuss whether it should be on by default. And I need this one to try it out. If it doesn't work, I give up on it. No, I can still try to make it a bit thinner, but it still has to get a bit thicker when you defloated. That's necessary, which brings me to the second question that I often get asked. And that is, can't you add an option to never defloat the panel? No, no, I'm not going to do that. Stop asking, please. I've already said I don't want to do that. And that is because it doesn't really fix anything. Does it? I mean, floating panels with a maximized window out of the box look bad. They just look bad. This is not how it's supposed to look like, like this. And if the issue is that the panel thickens too much, then disabling the defloating entirely, not even by default, it's not the solution. It's a bad workaround. And it's also one more option to have in the UI, which in theory the panel should have been like a simple thing and now has a lot of options. I don't want that to happen. It's also one more code path to maintain. And again, the panel was supposed to be something simple and it's getting more complex by the version. So I don't want to make the panel any more complex. I just want to make it more simple and defloating like this kind of work, kind of works. It adds some code. Yes, not too much though, so it's visible. We just have to hope that it works better than it looks. And let me just send that by saying that I do all of this for like three in my free time at least. So if you have like some spare cash to hand me, I've got donations link and that would be super much very appreciated. And you get some special content kind of I'm restructuring that part of the channel. So thank you everybody. And I'm just gonna end this video by saying that yes, I'm very happy that this camera does green screen better. I've set up the lights better and it's more sharp. And my face looks better than before. But I don't want to do to every day I have to take this off. Oh, it's so annoying. You can live with a bit of like you can live with this. I don't look bad, right? I don't, right?


Thank you for the flowers if you want to follow the slides on your Notebook, there is this URL tiny URL.com slash modern CD And this will kind of move you forward automatically as we move forward here Alright Yeah, have you ever compared other professions with software development really I mean I'm talking about construction plumbers electricians and Doctors so You know what I mean is do doctors wash their hands when they do before they do a surgery on you Do you want that or would you pay extra if they do? Electricians today at a grounding phase to every installation or just the ones that are paid better plumbers, you know when they fix a toilet When they mount a toilet They usually use a rubber seal on for every toilet don't they and And then in construction. I mean bridges. This is the Example Yeah, they usually there is an architect that does the bridge and there he hires or she hires structural engineers That do stress analysis and calculations so that that really works out and doesn't break so imagine the Golden Gate Bridge Breaking down with a lot of traffic on it. I mean, it's horrible. So So what about software development? Do we do that? So Does everyone of us write tests I mean seriously who writes tests for every piece of code Well, I mean what you you think that's fine everything every valuable piece of code if you think the code is not worth anything Probably don't need but who who does it? I mean who does it really? Yeah, more than I would have guessed and Well the modern about the The continuous delivery here in this talk Let me just get my Beamer on yeah This show yeah, so the modern is actually less modern and you would think it's more about how would Yeah, how should you work today? so and so that it's actually working out for tomorrow and Nothing bad happens like Bridges that break down or so So it's not only about Kubernetes and hyper technologies and so All the technologies you'll probably notice But keep in mind that the the actual point of this presentation of this talk is that you should do your software properly Right. My name is Peter Bidner I'm a developer of people companies in code. I run painless software and From a day job. I'm DevOps engineer at visioner gay in Zurich And I do a couple of open source of free software projects really so there is a choice of them here on the slide and Yes Well, I love to to Help everyone run their software smoothly and this is also what is continuous delivery about so let's go to the basics This is the definition It's a set of practices and principles and software engineering aimed at building testing and releasing software safely faster more frequently and in a sustainable way and there's more to it It is the goal of continuous delivery To put the release schedules out of the hands of it and into the hands of business So the one that pays you I mean I'm not talking about your boss Your customer the the feature sponsors They are meant to press the release button and you are meant to prepare everything for that and there are some other terms that are While used Confusing Lee in a similar way that I might and that I want to mention so especially continuous deployment So this is popular It's a popular term today used by Amazon GitLab Atlassian unfortunately in the wrong way I believe because continuous delivery is actually a concept and the other two are techniques, so you use a technique of Integration and deployment within the concept of continuous delivery So and actually automatically releasing is much much easier than giving control out of your hands Think about that. And if you want to read more, it's just humble who wrote the book in 2010 about it Yeah, and what's the modern then about here? Specifically, it's about what everyone does today. It's about Come on probably batteries are empty It's about immutable infrastructure. That's containers. It's about container orchestration. That's Kubernetes open shift stock of swarm and so on It's about version control automation So all things you know already you may also use on a day-to-day basis. This is about CI CD pipelines And it's about cloud native application which are applications that are resilient on scale So resilient means when they fall over they stand up again by themselves So after a tech for example, and when there is a heavy load, you can scale and you can launch multiple copies of them And they will share the load And there is more about modern also another aspect Modern the modern world is so complex That we need help so there are various offering offerings that help us with with getting our software in a cloud know Yeah cloud services like an AWS by Google by a Microsoft Asia, there's open shift, of course so there's all the Kubernetes stack and Yeah, and and not everything is a choice here. So some is a lock-in I mean the the borders are blurred but There is a there's that's a bit of a problem with being locked in and we are locked in when we use a an API Appropriatory API so we are locked in when we use something that is only available on AWS for example And why is this a problem? Because it doesn't allow us to move fast So imagine one day you find out well They raise their prices and you you don't like to pay the higher price tag. I mean, okay They're best in town, but yeah, let's switch over to Asia or Google cloud They have a similar thing How long will it take you? It's probably a project of weeks and months But actually it should take you if you do it right a couple of what I'm probably not saying minutes But maybe hours and that's it so That should be so this is the this is the today's situation so Do we have a choice? Can we change this and I would say yes and There is a very boring answer to this Some people may know him this is Uncle Bob. He is telling for years that we should do clean code so we should do proper software development and I would add to that we should deploy from from commit one to production Because that's where you have your that's why your time is missing in the end before you're releasing to production You know so scary be bang All right, so let's jump directly in a demo. So I want to show you how this can be done So I actually Showed a demo with two things. This is on not showing here This is on Let me actually Do the I need to make this simpler. Otherwise, I can't go fast Where is it? I'm sorry about that I need to make that works out. Okay, so I Show a deployment on a Puyo. I mean, this is just an an open shift Platform that that I run that vision runs the company I work for and this is just just open shift so Yeah, what I've already prepared is I've I have Yeah, kind of made this a free trial and I created two Two projects as three project projects actually and I'm going to use. Oh, no, that was the wrong button I'm going to use a cookie cutter for generating a a project a jungle project So I'm going to copy this code here to get started with cookie cutter and Then Yeah, here we are So, yeah, I've downloaded this already So who is me, it's me And this is the Euro demo and a demo for Delivery sorry, I'm not so fast like other people and And we're going to do that with good lab.com. We have I have prepared a namespace there get left see I Deploy to a Puyo for the moment Because that's supported and then I use Django and Postgres We run all the linting and test for Python 3.0 7 and be a BDD and As some century thing. All right. Voila. So This has been created. If you don't see this, this is the important part. So this is what happened You're a Python demo and yes, really there is a repository already initialized and It removed minus V and it's there as promised. So it says I can create the project here Let's create it here But that work out Well, it's probably on the wrong Well, let's do it like this Get lab Puyo and Yeah, we need to create that one So creating project. Oh, it's actually your Python demo Yeah There was the Demo for talk Delivery so everyone knows about it make it public so everyone can see what we're doing Yeah, so that's it. So we know that we can don't need we don't need to look up with the instruction where Because we know when we push there That should work out right Okay, let's take a look Yeah, that's cool at worked out. So there's the first commit and Yeah, we need to do some setup of the app Puyo Instance so the OpenShift Cluster so we need we have three we have actually three accounts here. That's the CI process is explained here down here And We need to run those setup procedures that are described Can be a concisely here and the read me As a service account that we have created Everyone who does cover need to snows or OpenShift knows what this means and then we give them the permissions And oops, sorry wrong button and Yeah, now we need the token here And we need this token because we do the current integration This is nice Where is it operations? Kubernetes and There is no cluster configured yet So we add a new an existing one We add This is the URL of a Puyo and there is the service token This is managed by a vision so we don't need to manage this with GitLab Yeah, okay, are we done? Let's check well last step We need to provide Access to up to the service of the service account to from from the other Namespaces so we have this this was the the last one was for the development namespace and now we have and before we had the integration namespace so yeah, and That's it Let's check what happened in the meantime so we have a Pipeline running on GitLab that was configured and It looks like this. It has the checks. So this is the linting and we run tests the one that we configured and Then there will the bill the image will be built so the jungle application will be built Into an application image and it will be deployed later So this all goes directly with with a single commit Of course, we have prepared this and you can can do this too or you can use something like the cookie cutter That is that is there Good so let's go back to the where is the Presentation actually Don't need that. Yeah so Let's talk about what is in there. Oh, that's what I wanted to show actually so There is this tree structure here so I can show this what the structure actually is so we have the application here the application code to the typical jungle application and then we have the deployment stuff in a single Folder this is these are yaml files. This is kubernetes Open shift the kubernetes yaml files and then we have The Docker compose for local development we have the tests for local development and Also, of course for the pipeline and we have for local development. We have talks at talks configuration So we have a few things here running. You can you know as usual you can use that. What is it? Let's do the safety that looks nice So the same thing we run so actually when we run just talks we run everything so the same thing that runs on Here Runs Well the same thing that runs here runs identically on my local computer So I'm not doing anything special, you know, and I'll show you why and why this is important So and of course it behaves the same way it will behave on on the pipeline So all of the tests at the linting past and there was nothing built yet But we'll see that later So let's take a look at what that is all about. So it's actually I made this the setup in I What I used our kind of Responsibility responsibility layers. So it kind of looks like the boat here For a reason. So there's a layer for the application logic. There is There is one for development. There is one that is concerned just for about deployment So this is actual deployment to some environments and then there's the automations automation stuff So the the pipeline mainly so in detail What is it about so? What you should care about is that you When you do it in a modern way, you just do it for one environment don't think in environments like this was traditionally done Do you think in 12-factor apps? So you read everything out from the environment logically? But don't design for different environments designed for features you build for features and what does it mean? when you build for features it looks like this that Okay, you get a century DSN and when this is defined and not non you configure it is and The same thing for the debug when debug is true. You do something That you don't do when debug is false Obviously, yeah But you don't design for I do debug only on in the integration and development you if you want to turn it on in integration it should be possible and So you build for features and you compose those features In environments, so you compose that for the environments Then the development Layer and this is actually about The tooling so how you How you do proper software development? How do you do professional software development? I may go back to the application this here should actually just work like you did software development 15 years ago So it's just I'm doing some hacking and I get and do pip install Django and then I do create You know part managed by a new project and they're not start hacking it install everything locally So it should work like this So each this makes you independent and then for development you this is a separate layer you have What do you have here yeah, you have the talks configuration that helps you doing the tooling with Yeah with all your Python stuff so without having it to install the thing separately, you know this helps You only need to install talks then you have of course you have the test because Professional software element you do that you have all kinds of tests acceptance tests unit tests and For easy development you may have something like a Docker compose Which of course then goes into the deployment at a deployment configuration that so the Docker file We'll see that later Yeah, but make it easy your standard practices So no make file or something if if there is a docker compose file every developer knows that yeah, you write docker compose Up docker compose build and it just works. You know you do this like this. Yeah, it's darker compose But I actually may do the build Also put this and demonized because I want to do the migrations later Yeah, so Don't write lots of instructions comprehensive instructions It should be simple and user friendly, you know, so when something is logic It's large it must be logic and not in the documentation the week you're not even in the read me The setup wasn't super logic that I mean Kubernetes is complex And then you have the deployment kind of responsibility layer Which is actually just concerned about everything that is About deploying to environments and of course you're a local machine or you're when you develop in containers. It's also an environment So that is where you by the way would turn on the bug for example but you call it also turn on sentry and that should be Completely at your disposal and nuts, you know kind of made by a decision made by design for environments and so there is the dockerfiling here everything that belongs to the dog file the you whiskey any and so for for the the web server for the application server and Or a guinea-corner whatever you want that and and this is important that this is a separate file so you can expect inspected like you would inspect it as a you whiskey hero Micro whiskey hero you would look into that don't put it in a YAML file Don't mingle it into YAML files for example and the same for the web server Don't mingle that nginx configuration into the YAML files You can do that technically, but it's like PHP developers mix HTML with PHP code. Don't do that It's difficult in Django. It's almost impossible. So don't do that in YAML Okay, and for the secrets, of course, you should put them in in your repository. That's the new thing but of course Not in plaintext Because if you don't put them in the repository, you can deploy it. You can have complicated things like a vault But the simple thing is you seal the secrets so you you Encrypt them with a public key and only the The cluster has the private key to decrypt them and you have a kind of Kubernetes operator there that decrypts it all right, and then the automation that's the simple part keep it simple and Let the the CI service do what you would do manually don't Install things in the YAML, you know, let's take a look at that So you don't do kind of install things here you have an image that has talks installed and that you know That does some things here you say, okay the The test image the test actually that's just a template so the test thing has a talks image and I test for pi a 3.7 And it's not installing anything. It's all there. It also speeds up your builds and It should also be nicely readable. So, you know, this is it's pilot is checking here You should be able to tell a story with that This here is deploying to integration only for Changes on the master branch. So when you merge a major quest into master for example into the main branch This deploys to production and only when you push attack So you can tell a story and guess what when do we deploy to the development environment only when we open a merger quest? So this is nice to read and that's how it should be Tell a story and make it as simple as possible. Okay, so and there are some some some other Yeah terms that or Considerations I would like you to follow. So it's the asset when you hear asset don't hear us, you know, don't hear the usual thing Here as simple as possible I'll show you why and Deploy deploy early deploy often and that means deploy from Comet one and directly to production So by the way, what do we do? Let's check what happened In the meantime, yeah, there's already Deployed so we can see here the build Yeah, that that's building. That's just Docker, you know, you know that I don't need to show this So and but by the way here the test for example that Looks like just it looks on your console on your local machine. Yeah and then for the deployment after building the image the image was pushed to The Kubernetes cluster to open shift actually because it has a registry And then for deployment. Yeah, this looks a little bit more complicated But these are just OC commands a single command. So you have one image that does everything and the good thing is here This is nicely integrated. Let's take a look where this ended up There we go click on a link and there we go Well, it's stuck for some reason. Yeah, this is the usual demo God Yeah Let's check back later Usually this is because Django needs to wait for the database but this Is actually being a little bit law All right, so Let's take a look at the process by the way So we said Merge requests you remember they end up on development When you open a When you merge a merger quest so you make a change on master this ends up on integration So I pushed directly on master for the first for the first commit. So that's that's fine then Afterwards it should be protected that would be good so you can only do merge requests and then So for triggering a deployment You just push a tag so you do something like it Good tag one OSA and did you say get push tax and then the tag is pushed and What happens is that? We get a thing Running the initial commits that will deploy to production the image is already built so we don't need to build it again. So By the way What was that darker compose? Exek Application oh, yeah Um, what is it? Manage hi Migrate right so that is for jungle. Yeah, that's cool And then I can do everything that you know and when you know how to work with dr. Compose Logs, I think logs minus F right Yes, it does something that was because we had no migrations and when and dad is actually So when you stop this Dr. Compose Down the compose up You can see this directly Yeah, and that's easy. I can't I have the link here. I can click on it. So this is just saying Not found because this is this is actually there is nothing here, but this is actually running This is actually a running up the jungle application because we have the I mean We don't have it from the front page yet So, yeah, so we can see the request here and this is actually a nice trick that you can do this like this and Yeah We can see this We can see this trick In the dr. Compose file So we are not even running the web server We are actually doing what you would do 15 years ago or 10 years ago when jungle was born you would just run Python managed by the run server And then you can click on this and it's actually that this is running inside the container But I can still access it because we have configured it correctly like this So but it should be nice and easy to use Good, then where did we stop?  Okay. It's not worth anything because you don't have a front page. Yeah, but you iterate You improve you add monitoring you add you add actual tests because the the first test the dummy tests and so on and then I Mean just the usual, you know crying of of us That do agile do test-driven development do a prayer program make pair programming First, you know do that first because you get more things done when you have two heads doing things and when and try to to Tackle the test-driven thing try to try to write the test first a single line of Code in the test that must fail a single line of code in your code in your application That makes the test pass back and forth back and forth only a single line. That's how you do it And you you have to do this goat It's on the workshop on some pie kind of so this is super when you when you experience how good this feels You'll never gonna write code without test again and free your software. So this is more for you know general Consideration for projects the less code you have to maintain internally The less burden you have in the end of okay open source free software projects are a burden of a different kind But you are it you force yourself to make it nice You can't just leave You know your code base without test because this will harm your reputation for example and When you have no secrets or when you have no closed source or you have very little closed source You don't you have very little security holes ideally and hopefully when you make this popular then You you have people looking at the code for security holes Yeah, and that is what Robert C. Martin says the only way to go fast is to go well and I know people don't believe this. So let's look at this Whether he says this for real Tell your friends in technology that the only way to go fast is to go well Okay, if you don't believe me believe him that was basically it So if you want to try out Cookie cutter you can scan a QR code or click on all one of those logos and We are open now for for questions But I may suggest before we start the questions because we are all Python is right who is a Pythonista Hands up So and then you know the oaths that we have to speak so speak with me Beautiful is better than ugly Explicit is better than implicit Simple is better than complex Complex is better than complicated and people are leaving because they get scared flat is better than nested sparse is better than dense a readability counts special cases Aren't special enough to break the rules although practicality beats purity Errors should never pass silently unless explicitly silence and in the face of Be good to refuse the temptation to guess There should be one and only one obvious way to do it. Although that way may not be obvious at first sight Take some time Now it's better than never although never is often better than right now if The implementation is hard to explain It's a bad idea It is one if the implant implementation is easy to explain it may be a good idea Continuous delivery is the honking great idea. If you deploy to production from comment one, let's do it I start today Python Thank you Peter and especially for your live demo very interesting and I think we have time for some questions You talk briefly about secrets. My question is how do you inject the private key to the container? yes That's a fair question because I talk about secrets and I Actually expect that people say why Why to commit to deploy to production? You don't do this. Anyway, maybe someone else will ask it So the typical solution you do with Kubernetes cluster is you run an operator so you you you run practically a service in the in the In your cloud instance that has the the secret key in there, but there are other ways. I mean this Yeah, I'm really not a text like I'm test expert. I'm not a Kubernetes expert. I'm not a Docker expert. I'm not I'm not I'm just telling you what you should do because Uncle Bob says it So but it works. I mean we do it Yeah, yeah, yeah, and the people are struggling with the vaults And and when what's what's are down? They are the single point of failure, but man as everything when the cluster is down It's a single point of failure another question I've seen that you deploy the merge request code to Development environment. So how do you handle if there are several merge requests open in parallel that all have to be refute Yes, so the typical answer is skitlab review apps I've not done this here for simplicity because it really depends on how big your team is And and also, you know, how much you want to spend because you know The concepts are nice to explain it. But in the end there are business people that say Okay, you need more RAM. You need more CPU. This costs us 50 francs more a month. No, no, no, no, no, no, we don't we don't do that So the easy solution is to only have one instance. So and then it all it's all about discipline So if you have two three developers, they say, okay, now I'm pushing my merge request I'm opening merge request and Yeah, it's not the ideal way But it saves money for for the customers and that we have Such customers and for the others that want the cool solution We do review apps and you can find this in the git lab CI documentation Thank you another question I Have a question So you mentioned cookie cutter and I already heard in other talks and do you recommend you use it every day for creating the template? Well, I'm in DevOps. So I'm not creating the templates. It's for me. It's mainly Yeah, I'm trying to help bootstrap others. I mean What we also do what what I plan to do is have the creation of the different combinations automated But I'm not using it on a daily basis if for me it's more like, you know How do I put the the things that I would that we see every day and and you know ideas evolve and we Find out new things. For example, we have we have some kind of git pool now in the in the build step And this is weird, but it but it it makes your your build faster when when you cash the image layers So I need a place to put that information somewhere So I put it in the cookie cutter and and there are other tools. I know some There's some Tom some something that I that I started with but cookie cutter is popular and yeah It has some defects because you can't upgrade really the code you have you have created. Yeah Thank you other question Hi How do I prevent getting locked into my CD tool in this case kid lap? Excuse me. I couldn't hear the first part. So you've chosen to use kid lap. Yes Now you're essentially locked into kid lap because that's all very specific Configurations, how do I avoid that? Yeah, so the the point is you have seen I've presented those layers and The the kid lap is practically only the should I go back? kid lap is only this Automation layer, so it's the last layer here so my ideas to So what what would we create what I create is? in an automation so a a Bitbucket pipeline configuration or a circle CI configuration or a Travis CI configuration that fits with the deployment for example that Runs all the commands that work that are designed and that are compatible with the deployment layer So that's the only thing I have to do and and for for building.  Thanks so much Other question. Yeah Hi So I'm just trying to understand so okay. This is Python conference, but are you? Like proposing all those things with what you said so they should work in like language and technology agnostic way or Is this somehow very tied to Python and docker and so on why I'm asking this question is because When you talk about production for everybody production is a different thing Yeah, so right now as I understand it's just a docker container which is deployed somewhere, right? This is the end product of this Yes, so I mean other people have Different types of production. Yeah, for example, they release. I don't know RPM packages that being packages which then have to be Manually installed and so on so No, you don't have this if you do if you do docker if you do containers You don't have this because you can run a you can you can install everything in your containers. So everything is in your Docker file here. So you do what you do you I mean you're talking about traditional Deployments, correct Yes, and no so my point is docker has also security concerns. So that's why well, that's not everyone You can use it for testing sometimes but not for deployments. Yeah, deployments have can have different procedure I mean to production. Yeah, they can have completely different procedure from for example, what you do development but if you if you go through the talks through the slides again and try to understand how I mean I mean is It's really when you work with containers It's completely isolated. That's that's the whole point of containers that you don't have to worry about What does my target? Environment look like you don't have to worry about you don't even have to worry about Processors and all that stuff because in the end you you don't even have to worry about when this happens with Kubernetes because you only say Hey, Kubernetes. Look, there is an image. You push it somewhere now actually for to Kubernetes. You only say here is the configuration and then Kubernetes has a concept that is called Eventual consistency so it does not immediately deploy it deploys when it thinks it's right So when the time has come so kind of you know, so it kind of Fires up your container and when that works it shuts down the the older one So you don't have to worry about this and the solution is containers But this is not a my talk that's that explains that this is just why we use containers Okay The is this Is this okay for you that I answered this way? Can you give me the mic back, please? So in some companies the installation procedures are much more difficult than than this due to security concerns and due to machines being in some remote locations and so on so and Yeah, but it doesn't work as simple as this but let's all for local development. It's it's okay You can you can know no not at all local local is just one environment You know that's a point and everyone needs to understand this. This is critical. There is no local environment There's no production. That's why I say production because hopefully some people would jump up and say no We are not deploying to production. This is dangerous. It's not You know, it's just one environment and you know, the point is you can give if you have security concerns You give the darker containers to the people that deal with security and they will make it secure That's the solution So it's not I'm not it's not much about my talk I'm taking this for granted because the industry now works like that if you and your company don't work like that again You should catch up. You really should Thank You Peter we are Free and for this session and thank you for your time and we can proceed with the other look. Thank you