a new way of looking at things that we in the performance team love. So, Dario. Okay. Thank you. Thanks very much. Thanks for having me here. I'm really thrilled. So, what I'm going to do is that I'm going to bring basically two viewpoints. I've been working on academia for the last 15 years or so, and then last year I moved to Huawei. So, I basically have industrial viewpoints with university mindset. So, today what they're going to do is that I'm going to talk about metrics and models for web, and there's a little longer subtitle that I'm not going to comment here. We're going to see throughout things of all. So, of course, this work would be not possible with number of people, if they are in alphabetical order. Two are actually in the room. One is Gila, the book from Wikimedia Foundation. The other is Flavia from Telecom Polytech. So, thanks to them, we also can discuss more interesting thing now. So, just to set about what we are focusing on, I mean, I'm not a web developer, so I'm going to have a completely different focus, and right now I'm working on equipment vendor, so I will have a very much lower layer focus. So, no matter what you're working on, if you're a browser maker, if you're a CSP content service provider, if you're an internet service provider or an equipment vendor, what you care about is that the user are happy, right? So, offer quality of experience is a common goal. Of course, if something goes bad, you want to be able to detect it fast, if possible, if you want to be able to forecast before things go bad, and if you're good at forecasting, you could also try to prevent things from going bad, in order for your user not to churn. So, detecting quality of the gradation, quality of experience, the gradation is important. Now, how do you detect quality of experience? How do you define it? Well, typically we need to have a good idea of how the user, if they are happy or not, and then try to correlate some of the telemetry. So, like for instance, the boomerang is collecting a lot of telemetry, and we'll try to correlate that with the user quality of experience. Now, if you're taking the point of view of equipment vendor of internet service provider, well, you're gonna have a little bit harder time because you're not in the browser. So, you don't have all the rich telemetry, and encryption now is really gonna be painful, because you're only gonna see stream of encrypted traffic. Still, we want to do something, because otherwise your user will churn, if the user will churn, the equipment vendor will not be able to sell equipment, and so there's a loss of money as well. So, it's important to get a hand on what's the quality of experience, and user quality of experience is basically affected by a lot of things, including, for instance, the context, so where is the user, work, other places. If he's a pessimist guy, or if it's an old lady, probably they don't have in excess only the same perception of delay. And there's, of course, system influence factor. If you are down-signing the building, ground two, or minus two level floor, probably your signal is not very good, so we have slow performance. So, in order to factor all those five, be an engineer, of course you're gonna ask the user, but you're gonna try to infer these things from looking from the system perspective. So, system perspective starts from the lower layer, the network, so over there you will be able to measure some quality of service indication. These will in turn affect application performance, application metrics, application QoS metrics, like the one that Boomerang was reporting, or other, like web page test, are reporting to use some telemetry. And from that, it will have an influence on the way in which the user are experiencing the browsing behavior. And so, what you're gonna do is that you're gonna be able to measure some of these metrics, like from an end-to-end view point, what is the latency, what is the bandwidth, what is the pocket loss, or point-to-point, what is the Wi-Fi quality. Of course, that doesn't make sense if you look at the throughput of a single connection because you want to put them all together in order to be able to tell meaningful metrics from a session view point. Session means, for instance, if you're looking at a web application, it's gonna be a page load time or speed index that we're gonna see later. There are also session metrics that are correlating measures about multiple sessions. So, for instance, engagements, so measuring if you're staying on the website for long means that you typically would are happy with the quality of experience they're serving. And of course, you can go and readily ask the user how it feels about the service you're giving him. So that they can ask many users, poll around the room, you get five stars, and then you do the average, it's your opinion score, you can also ask a different thing. And of course, if you know about the device type, if you have a cheap phone or if you have a high-end phone, maybe your expectations are different, maybe the phones are also rendering differently. So, all of that, of course, is very complex. So today, we're gonna focus on a subset of it. In particular, we're gonna look at the web, that's the web dev room. So, we're gonna look into performance metrics page load time, speed index, try to see how this correlates with the mean opinion score, your user, other user feedback. And of course, we're also gonna adopt the viewpoints of the lower-layer carriers, where they're only gonna be able to measure some weak signals. They don't see anything about the middle layer because it's quick, HTTPS or whatever, all that kind of encryption. And so, they either want to try to, from the network QoS, you learn something about the application QoS or make a big step and go to the quality of experience of the user. So, that's basically the agenda for today. So, they're gonna delve into four different aspects, data collection, so the modeling parts, the metric parts, and then again, some method that allows you to go from row to a pop. So, we have a pot. If you're from the network, you need to start with your method, learning something which is metrics about the browser can easily measure. You need to learn the metrics that are useful. So, for doing that, you need to couple two things. You need to couple measurement involving the user, asking user whether they're happy or not, and building models that, based on your metric, are hopefully able to extract this information from automatically collected one. So, in the agenda today, we're gonna work this top down, so we're gonna start with the data collection. So, data collection, typically, what you do is that you build up some crowdsourcing campaign. They have a huge cost, and they are no perfect campaign. In the last years, we have been doing three type of different things. We've been asking user, what is the mean opinion score? So, write your experience from one to five. We've also been asking user, when do you think that the page was finished, or what is your user-perceived page load time? Or, seeing two pages at the same time, which page did you think it finished at first? So, to get a little bit of an idea of how the user are perceived in the web. And finally, with Wikipedia, living collaboration with Wikipedia, we started asking the user whether they are satisfied with the experience they have while browsing Wikipedia. So, of course, there's no personal solution. In the first data set, we were doing lab experiments. This means that we were having a few panel of people that were typically volunteer, close to 150 to 150 people, recruited universities. So, you have very specific class of population. It will definitely not fit the grammar's behavior. The good side is that we were using real servers, real protocols, we were able to control the conditions, but the number of web pages, of course, is not as completely representative of the internet. So, then you can do something else, stepping up by moving to crowd sourcing. So, you have, for instance, Amazon Medical Turk, Mechanical Turk, so you can leverage a large pool of people. But over there, you need to, you cannot let them access a web server, so you will typically put videos of the web page rendering process. So, this is not really exactly like browsing. You reach a larger audience, but these are also interested in getting paid for the task. So, you need to filter out a lot of people that are just there to make money. So, last thing that we did with Wikipedia is very interesting because we are actually, we are polling the user, so there's one billion pages visit monthly, roughly, and a tiny fraction of that is gonna be polled for performance metric, and a tiny fraction of that is gonna also be polled for binary feedback. It's likely more than binary feedback about whether they were happy or not. So, this is good because you're gonna poll users that are in the real service, from the service they like, the service they use typically. The downside is that you have a huge heterogeneity. Remember, on top of my head, that we were polling on 65,000 people. They were looking at 42,000 different Wikipedia pages, 3,000 networks of 250 devices, and 45 rosers. So, there's a lot of heterogeneity, and so building a single model is not necessarily trivial. When I'm putting the icon there is that the data set are available, so if people are interested, if there are people that are doing research on that, like we were saying before, sharing tools is important, sharing performance evaluation is important. Sharing the data is even more important because it allows you to replicate and see whether the performance that are reported are true or not. So, now that you got the data, okay, cool, what do we do? Well, basically, we're gonna have a way to go from the data, so our Y, to find some function that based on some of the things that we are able to measure, like our Incognita Y, plug into a formula F, it's gonna be able to tell us what is magically if you want the user performance. So, here, by each, typically, people use a single scalar metric to generate the page load time. The function has been predetermined by an expert, and there are typically two approaches that are being used. One is IQ X hypothesis we are using on an exponential model, and here, with a logarithmic model, which is tied to the Weber-Fetchener law, which is a psycho-behavioral model that tells that the human response to a stimulus is logarithmic-related. And this is, for instance, used by a standard, so what you do is you do a lot of measurement, all the points here are different answer from the different user, and then you do a fitting, and here, the fitting, we can be happy with that. Now, there are limits because, typically, there are a lot of metrics, although telemetry that is made from browsers, and so here we are only using a single metric, so you can go one step further, and instead of picking a single metric that you like and a single function that you like, and although the fitting seems nice, you could do something which is machine learning driven. So, basically, having a factor of input features and having an automated way to select what is the optimal fitting of the function by minimizing some error. So, here, the trick is that whenever you select a very specific machine learning algorithm, you're implicitly selecting which are the type of function that you will be able to learn, and here you see that you have a slight gain with respect to the typical models that you have here by considering more metrics. Of course, there are different models that are available. We're not gonna delve in the detail of that, just to say that for me, there's still some room for improvement from going to the feature that we have to the user experience, but still you have a good and quite high correlation. So, this brings us to the metric. What are the metrics that we can work on? So, in order to be quite clear about everything, we're gonna have a very small animation about how is the web page loading process after you go and click on a link. So, we start something that you're gonna start downloading, and at some point, you will have an event that is gonna be fired by the browser, document object model. So, at this point, you know the structural page, and you can start putting things around, and so you have a visual progress of the page that increases from zero to upward. Then you keep downloading more things until at some point, which is called typically above the fold, all the visible portion of the page has been downloaded and shown to the screen. That's called the ATF, and your visual progress is increasing. And you can represent here your visual progress as a function X of t that is growing from zero to one, where one is basically everything that needed to be rendered for the page to be visually complete is finished. So, X of t, of course, you can also do something a little bit more fancy. So, basically here, the integral of the residual of this function is the gray shaded area above the curve, and this gray shaded area above the curve is what Google defined as speed index. So, we're gonna come into that in a moment. And then, of course, I mean, you can keep downloading more content that is not necessarily available and immediately visible, but it's gonna be available when you scroll. And that's when all the content is loaded, it's typically the page load time. So, now we have two type of metrics. So, one are the time instant metrics. So, you have, for instance, time to the first byte, DOM, time to the first paint, about default page load time. These are very specific time, which are important to somebody. And then you have something else, which is the integral form of it, which is basically looking at all the area above the curve. So, why this thing intuitively is important, imagine that you have two realizations of two pages that have exactly the same page load time. So, they finish exactly the same time, but this one shows half of the content very fast, and this one shows half of the content almost much more later, right? So, in which of the two you would be happier in this one. So, whenever the area above the curve is smaller, then it's better, and it's faster. So, one additional comment is that, given that you are integrating something that is a dimensional, and integrating over time, also the area above the curve is a time in dimension. So, physically, if you are an engineer, you would think that of a time, is a time unit of measure, and you can think it as a virtual time that is explicitating how fast was the rendering process. Now, you can define a family of metric like this, and depending on what you put is x of t, you're gonna have the speed index. If you're looking, for instance, at the difference in the histogram that were shown, so the colors on the page, you have room speed index that is measuring the areas that each of the different objects that are drawn on the page are gonna put, and they're gonna compare it with the amount of rectangular should have been drawn at the end. You can look at SSEM, PSSI, perceptual speed index using SSEM metric, which are much more advanced. So, all of that is very good because is visual progress, but they're out on site. So, for instance, you can only measure them in browsers, and some of them are actually processing intensive. So, if you need to do SSEM, there's a lot of computation you need to do. So, some years ago, we were proposing to do, as a proxy of this more advanced metric, they were very simple inputs, like object index or bytes. Just looking at the bytes that are coming, you would get a pretty decent idea of what is coming to your browser if it's coming fast or not. We're gonna see a little bit later if it's work or not. Good side is that you can do it in layer three in the network, is correlated with speed index, doesn't necessarily is good for creative experience. So, that's a question that you need to address. And I'm not gonna go into these kind of details, but you can also have affecting, for instance, the cut of all the integral in order to optimize some of those metrics. But I'm not gonna go into these details. So, now, if you are in the browser or if you are a content service provider, what you have is that you have a pretty good picture of everything that is happening. You have per domain, the vision of all the different objects, also the type, if they are images or not, CSS, whatever. And you can reconstruct this feature with quite accuracy.  And the only thing that you're gonna read is that, okay, this is a packet, this is a packet full packet size, MTU, and this is a smaller one. So, what do you make out of it in order to extrapolate from this? So, again, I'm not gonna go into a lot of details. I'll rather gonna show you why this thing would work, but basically the idea is if you are familiar with machine learning, you need to perform some amount, some really simple amount of signal processing in order to make your input to be homogeneous. We are using supervised technique. So, supervised technique means that we need to have exactly the same input. And then, different models that you are using, like extremum gradient boosting, which is an assembled method based on trees or 1D convolutional neural network, what we do is that we present them with a lot of samples. And we tell, look, this sample, and we also explain them, for instance, had this above the fold value. We build another model, providing the same example, and providing what is the page note time or the speed index of any metric that you are interested, and we provide many samples to train a model, and we test it over previously unseen cases. To give an intuition why this should work, so here we have the webpage rendering, so this is basically the user. Here is what we see in the browser, where every burst is gonna be one object and we have one color per different domain. Actually, we're presenting only the top three domains and the others we are using the same color, otherwise the picture would be really, really too colored. And here is what you see from the network. So we're gonna have one packet, a little bit more, we are aggregating packets in 10 milliseconds, and then you're gonna see one color per IP server. So when I'm starting, if I click on the right place, you see that, okay, now this is a Chinese webpage, so it starts late, at some point, you see those fingers are progressing, here there was a big object, this big object has been a lot in multiple packets, same thing here, the green packets correspond to this big object, and you see that these curves are slightly different, but you see that there is some similarity, right? They are not completely different. And indeed, if you systematically perform this experiment, this was just one example to show you how these things look like for real, then you can go and make an experiment where you're monitoring the network, so you're taking the real encrypted traffic, you are monitoring the browser, so you have the ground truth, so you have the above the fold, whatever metric you are interested in, and you can repeat this process and try to see extrapolate some accuracy number. So here is the only accuracy picture that I'm gonna show. This is reporting the absolute error in milliseconds, actually this is the median, and this is the 23th percentile, and this is the 75 percentile, so this is basically, in the 75 per case, your error is gonna be much lower than this, and in the median case, it's gonna be this one. And you can see here we have two different approach, one is, even without machine learning, I'm not gonna explain why, the colors before in the picture had a mathematical interpretation, but I didn't want to bring it up today, it's not a point, but with an algorithm based on that, we can have already something that is gonna learn only a single function, which is the byte index, and we can approximate the byte index learned from the network with the application byte index that we learned in the browser. And that one has a six percent error. On top of that, this is without machine learning, so it's a very simple online algorithm. On top of that, you can add machine learning and you can compensate for this error, so you can reach a lower error, and then you can learn, generalize to any metric. So we're learning the page load time, the object index, the speed index, or room speed index, the DOM, if you're interested in learning the DOM, with these kind of errors. So we did test with orange on a number of pages that we were never seen before, in a number of networks we were not seen before, and these are the accuracy estimated, indeed, in those settings, so it's a pretty good portable. And, okay, not to make an advertisement, but given that the algorithm works, we are also porting it into Huawei products. Now, there was one catch that I didn't talk in this talk, due to lack of time, is that we are also able to handle multi-session. So if I go back here, we see that there are a lot of packets coming from a lot of different flows, but you need before to be able to isolate the flows that are gonna go to the same session. So this is something that you need in order to be able to apply your machine learning technique, and this is also something that is done, but we just didn't talk for lack of time now. So basically after, okay, now it's where we stand, so where we could go to go further. So I'm gonna just talk about three couple of free ideas. So for people that are familiar with machine learning, unfortunately, in the web QoE domain, we are still at expert-driven feature engineers. So basically we have somebody that is defining speed index and why should we define speed index? Seems are very natural and very bright idea, but we have no clue whether this is really a proxy for quality of experience. So a better approach, I'm not saying more explainable, so it's less intuitive, would take raw input, raw sensory data from the user and try to do what? To learn the features by the learning process. The learning process is gonna, in the neural network through bake propagation, is gonna create some features that are the most relevant in order to find and explain why the user voted a given score, okay? So that's definitely not interpretable, it's more versatile. The downside is it requires a lot of sample, okay? So here what we did was taking packets and learning any of this function. Similarly, we could use these inputs and trying to learn functions which are user happiness. Of course, getting the data is difficult because you would want to be as less intrusive as possible, so if you need to put sensors like this, maybe you're affecting the user experience. And other thing is that maybe, okay, you can leverage, so I know people that are working on happiness recognition through future recognition, but over there if you're happy, it may be for the content of the message that you receive or the page that you're visiting and not whether the experience in your loading that page was happy, was good. So it's quite difficult to get the sensory part working. Second thing is that I was speaking about single model and actually we did single models because they are easy deployment, but of course, the wide web web is really, really large. So for instance, Wikipedia is not image-intensive and you will have other websites that are mostly done by images or video. So all kind of single model fits all. So of course, to increase accuracy, you should go per page. Here just an example picture where you have black line is one average model and these are all the points that you're getting and of course if you have many per page model, they're gonna have a better fit. Now the problem is that inherently this process is not scalable. So how to make it scalable? Well by prioritizing things that are more important. For instance, if you have top 100 web page, you can build reliably models for the top 100 pages that are more frequently visited by people. Then you can have a second approach in which we cluster the top 1 million web pages. For instance, here you see number of clusters out of which 24 pages were extracted and inside each of these clusters, there are thousands of pages. This cluster have similarity in term of the number of domains, the number of objects and the size of the page. So there are higher chances that if you build models that are accurate for pages in this class, then you're gonna also be able to cover more accurately the top 1 million. And then of course, okay, for the rest, the top 1 billion pages, you're gonna use a single average model and pray it will work. But at least you're gonna already have in a better operational point in the accuracy versus scalability trade-off. Then final comment, which is a community comment. If you are working in this space, the first thing you need is data. So keep collecting and sharing data is very important. So I'm very happy that finally, we're working with Wikimedia, we were able to release a data set in a properly anonymized form that was protecting successfully the privacy of people and also letting people doing the research in order to build models better than the one we built. So you need to take into account that when you go to the supermarket, you already find this machine, right? And they're asking you, are you happy or not? And you click on it and you don't think even about it. When you're calling over Skype or Facebook at the end of a call, there's something that is calling, asking you, write your call. Also my phone started asking me, did you find this suggestion useful? So to have binary feedback from him and this is them from Wikimedia. So what you would you gain for keeping this data collection is two things. One, you have, until your model will not be good, you really have some information from the user. So you already know if something happens that is users of your service that are telling you directly and you don't need to go over Twitter and try to understand if the user are complaining about your service through other channel. Second, this continuous stream of data is gonna also be able to make your model better or if there's anything that changes, next protocol. So we had the HTTP tool and now it's gonna be HTTP-free sooner over quick. Maybe your model need to be retrained. So you will need to have this kind of data. And if the user population is large enough they're also limited on site. Only it is a risk of annoying users if you are having a small, if you leverage small panels. So without, so this is basically a talk that is based on these resources here. I put the different papers. They put also the icons for the different data set. Some of the implementation that we release and everything is accessible from here in this page. There are things that are not out yet. So more will come. So for with all this, I think that I'm done. So I would like to thank you for listening so far. And if you have any question, please go ahead. If you shout, I can also repeat the question. So like, yeah, okay. So there are studies also. So the question is, what if I'm able to break the key? So why are you doing things about the encrypted stuff where you can break the encryption? So government guys, so there are two answers. So if you are having the, if you're able to decrypt probably you're not interested in web performance. So you're breaking this in order to look at different information. Second, there was a study using X telling you what is the fraction of in data. You got proxies, for instance. In some of the institutions, you have a proxy and you're delegating and you're accepting the key and in a PC that is managed by your organization, indeed, you have a proxy so for which it is not necessarily useful. Now in GDPR, this is pretty serious now. So definitely if you are Huawei, there is gonna be twice as more concern as if you are a regular vendor right now in this moment. And so of course, I mean, having that your devices are totally not interested in looking in the payload because they don't need it, okay? It's much more important. So here basically what we are doing is that we are leveraging very weak signals that are intrinsically in the timing information that comes in packets. Much like, I mean Debussy was saying that the music is the silence between the notes, right? So here in somehow, we are waiting the information that we see even if you are not listening to the notes, not looking at the content to try to reconstruct the signals. The thing that I was showing today was not for the government, was more for internet service provider and well, equipment vendor, but if you go up to the Chrome browser, for instance, they still, the missing link is between the layer seven and the layer eight, the user. So how do you, can you ensure that, for instance, there will be a talk on normalization of timing APIs? So you want to normalize something that is relevant for the user, right? So, and this is the part where indeed it fits from going normalizing that from level seven point of view. And if it's relevant, we can learn it also from layer three without breaking an encryption key. Don't know if it clarifies or, yeah. Okay, so that's a very, yeah. So that's a very good question. So actually it's seasonality, so basically things that are non-stationary over time and particular seasonality means that there's periodicity is something that we look out, we extensively look out in the data sets. For instance, with Wikipedia, we have measurement of months worth of studies. So we were expecting to find day, night effect, weekend, weekdays effect. We didn't find any about the happiness of the user. It was amazingly stationary during the period. So this is documented in WWP and we also extended that I don't know what this non-seasonality. We were expecting it. The data now is available, so, yeah. I think that is much, yeah. Mostly the size difference or the difference in timing between packets? Okay, so that's basically, I went very, very fast. Can you repeat the question? Yeah, I'm gonna repeat the question when I come to here. So basically the question is, okay, what is the magic? How can you learn from the packets? And actually, we're not learning directly from the packets because every webpage has a different number of packets and our supervised method, which are regression method, needs to have a fixed input. So what we are doing is basically that we are chopping the time into a regular interval of times and what happens is that basically you are sampling periodically a signal. You're sampling periodically this signal when you are every SOF and every delta T, you are looking at the packets that come, put an integral there and you're basically sampling this curve here, right? So this is the way in which we get the input, which is by just accumulating over a small period of fixed time arrivals, packets belong to the same session. And that's what makes the input. So there's the basic signal processing level amount of feature engineering to normalize your input to be able to fade it to a neural network. And okay, five minutes left, so I was too fast. So you can, sorry guys. You have a question? Okay. Cool, I can ask directly in the microphone. Do you have an estimate of how many data points we collected on Wikipedia in like a typical week during the study? How many data points we collected in a typical week during the study on Wikipedia? You are in a typical week. So now I know that basically finger change a bit. So I think I have some backup slides. So too many backup slides. So this is okay, about the stationarity, you get your picture there, which is here. So I know that we were collecting the 62,000 data points during the first period, which was basically a first test case in which we were, so if I remember correctly, web performance timing are triggered once every 10,000 page visit in Wikipedia. And out of those, we were sampling one over 1,000 at the beginning. At the end of that, we step up a little bit. So you step up a little bit the sampling, but this is basically over this time period of time. So hidden is the fact that we basically issued the sample, the query to 1.4 million people and only 62K replied. So because people are, they can willingly or not, except to click on those or not. So the numbers per week, I don't have them in my mind because you are mostly focusing on, can we get a breakdown of how the user happy? And in this case, in Wikipedia, 85% of the users are consistently happy with no seasonality and no correlation with surveillance. Okay, okay, so we can go back, this is slide one, which is, so you're here, you want to know if things break or not.  Now, what Huawei do as a business is basically selling boxes. To operators and operators, what they do is that they sell pipe capacity to their customer, which are the user and from time to time, they have problems because the service doesn't work and the people will complain to the ASP, but actually it's not the ASP, the problem. Maybe the content service provider, maybe the DNS, maybe the BGP. So over there, basically, there's a need for troubleshooting tools in order to be able to tell, oh yeah, it's our problem. So it's our network that is down, so we're gonna fix it or look, guys, everything that we have on our side is okay, but there are a lot of problems on that website everywhere. In order to be able to say so, you need to know what is the typical page load time of your user or detecting whether this is changing. So this is why, indeed, before I was working more on the, if you want layer seven kind of aspects, and there the question was, okay, we have the speed index, okay, we have the above the fold, but nobody tried to compare whether this was really relevant for the user. So this is where we started involving users. And now this bit about, okay, and then I'm working for an equipment vendor, so am I able to do the same things, but from a more challenging viewpoint, which is starting from completing crypto traffic. Just for, I mean, this is research, so it's fun. But then, given that I'm no longer in university and I'm in, well, there's also business model behind, because basically if you are able to detect whether there is a problem, then you can fix it, and then it will not have user churn and so you're not losing money, right? So the same thing for the content provider, why they are optimizing? Because there are ads, except on Wikipedia, so there are, in Wikipedia there's a donation, but if you are Google, if you are being from Microsoft or Facebook, you're showing ads. And this is the way you get money. So if your webpage is low, they were studied by Google, by Bing, they were showing that for any amount of milliseconds you add from hundreds milliseconds, you have a loss in the number of people that are gonna go to the server, click on the ads, and so you have a losses of revenue. And if you multiply 2% loss by 1.2 billion people visiting, that's big numbers. So same thing, but from encrypted pipe from the network guys, few points. Okay, so thanks a lot for, thank you very much.


Assistant Professor, Department of Mechanical Engineering, Volchian Institute of Technology, Solapur. Today, we are going to study properties of pure substance. At the end of this session, students will be able to use MOLLEAR diagram. Students will be able to calculate the heat required for weight, dry, and superheated stream for given conditions. These are the contents we are going to discuss today. That is, MOLLEAR diagram. What are the concepts required to solve the numericals? MOLLEAR diagram, it is the representation of the condition of the steam on enthalpy versus entropy diagram. And this diagram is divided into two parts. This will be the, this is the line called as the saturated liquid line. Below the saturated liquid line is the weight mixture. And above this, this is the superheated. And this line is represented as the quality of x1, means the dryness fraction. If you are going to measure at any point on this line, that is 1. And if you are supposed to draw the parallel lines to this, these are having the constant dryness fraction lines, means at this, this line will represent x of 1. This line will represent x of 0.9. This line will represent x of 0.8. These lines are representing the constant temperature lines. And these lines are representing constant pressure lines. That is the use of what you can say, MOLLEAR diagram to find for the given conditions. If two conditions are known to you, you can find the remaining conditions. Suppose, for example, if you are having pressure and the temperature. If, let us say pressure is given as 1 bar and temperature is given as 220. Now, from this 220, you are going to draw this and whatever this pressure line is there, at which point it is going to make, that is the given point, you can find out the remaining parameters. These are nothing but the enthalpy, entropy, or specific volume. Means if you know two properties, then you can find out the remaining properties for the given MOLLEAR diagram. That is the use of MOLLEAR chart or MOLLEAR diagram. Now, what are the concepts required to solve the numericals? First one that is important one that is dryness fraction that is denoted by x, which is calculated by, it is the ratio of how many, what is the mass of the steam divided by the total mass of the given mixture. And that is calculated by M S divided by M S plus M W. For weight, the value is in between 0 to 1. For dry, the value is 1. And for superheated, that is also 1. Weight steam, this is the formula that you are going to use to solve the numerical. That is total heat contained by the weight steam for per unit mass. That is equal to H F plus X H F G where H F is the sensible heat content of that liquid. H F G is the latent heat of the liquid at a given pressure and X is the dryness fraction. For the weight steam, the value of that X is in between 0 to 1. Moving towards the next, that is saturated steam. For the saturated steam, same thing is there. Only the value of that X will become 1 so that the formula will become as total heat content by the saturated steam per unit mass is equal to H F plus H F G where X is 1. Moving towards the next, that is superheated steam. Means you are heating that steam above the saturated steam. That the formula is given by H F plus H F G which is also known as H G plus M C P delta is superheated. Where that C P is the specific heat of saturated, specific heat of the steam. And delta T is called as the degree of superheat. Where that delta T is calculated as T superheated that is the temperature of superheated steam minus saturation temperature with respect to the given pressure. These formulas are required to solve the numericals. Let us see the numericals. What amount of heat would be supplied to produce 4.4 K G of steam? Means M is given as 4.4 at a pressure 6 bar. Pressure is 6 bar and at 250 degree Celsius. Means the output temperature of the steam must be 250 degree Celsius and the pressure is 6 bar. From water at 30 degree Celsius. Means initially water is available at 30 degree Celsius. And they have given tech specific heat for superheated steam as 2.2 kilo joule per K G Kelvin. Now first thing from the steam table we are going to find out the values of H F, H F G at 6 bar. At 6 bar from the steam table H F is 670.4 kilo joule. H F G 2805 kilo joule per K G. T saturation temperature is 158.8. T steam is equal to 250 degree Celsius. Now I have a question for all of you. What will be the type of steam? Is that the output of the steam is wet or dry or superheated? Pause this moment for a while and think on that. The answer is superheated. Why superheated? Because the output temperature of the steam is 200 degree Celsius. And the saturation temperature with respect to given pressure given pressure is 6 bar. And for a 6 bar the saturation temperature is 158.8. As the temperature of the saturation steam is greater than the sorry as the temperature of the steam is greater than the saturation temperature. Therefore, definitely the quality of the steam is superheated means the value of X is one. A total heat contained by one cage of superheated steam is given by the equation of superheated. That is H F plus H F G plus C P delta T. C P by putting the values C P as 2.2 we will get 2956 kilo joule per K G. That is for per unit. Already the water is at 30 degree Celsius. Water this H F is there that is from 0 degree Celsius. Now I have to subtract this sensible heat. Therefore, per unit sensible heat associated with one cage of water will be given as MCP delta T simply. M is 1, 4.18 that is the specific heat of water. And delta T is 30 minus 0. The total will be 125.4. Therefore, net quantity of heat to be supplied per K G will be this minus this you will get 2830.6. That is for per unit mass, but the total quantity is given as 4.44 K G. Therefore, the final value that is total quantity of heat to be supplied to water is 4.44 multiplied by this per unit mass you will get 124.546 kilo joule. Moving towards the next numerical that is determine the mass of 0.15 meter cube of weight steam at a pressure of 4 bar and dryness fraction 0.8. Also calculate the heat of 1 meter cube of steam. Now, first thing at 4 bar what will be the values of H F H F G that we are going to find out by using the steam table. These are the values. Now, density is calculated by 1 upon X V G where V G is the specific volume and X will be the dryness fraction. As they have mentioned that is weight steam and then they have given the value that is pointed. Therefore, we will get the density 2.7056. Now, once you know the density if you multiply it by that is mass you will get the total mass that is 0.4058. Total heat contained by 1 meter cube of steam which is mass of 2.705 kg will be given by 2.7056 multiplied by H where H is the total heat content of 1 kg that already you have calculated. And as they have given that the quality of the steam is weight therefore, H F plus X H F G by putting the values you will get the total heat contained by 1 meter cube of steam that value is 6252.9. These are the references. Thank you.


The narcissist wants to hurt you as much as possible. Narcissists always hurt the people who are closest to them, the people that they always have access to, but they treat other people very differently to how they treat you, people who aren't even helping them, people who aren't doing anything for them, but they will get better treatment than you do. They will get the narcissist's kindness and generosity. The narcissist will do whatever they can for them while having a composed and self-assured manner, while being self-controlled, which may confuse you. It may make you wonder how can they be so polite and courteous with other people when they act in a silly way and seemingly without sense when they're with you. They're offensively impolite and bad-mannered. They're disrespectful and inconsiderate. They're passive aggressive, but then they act so differently with strangers with people they don't even know, people who aren't doing anything for them, which may make you wonder why wouldn't they do that for someone who is doing something for them, but this is how narcissists are. They hurt the people who are closest to them, the people who they always have access to, but this should reveal to you how insincere and deceitful they are. This should reveal to you their shortcomings. It should reveal to you that something is not right because there's such an extreme contrast between how they are with you and how they are with other people. There's such an extreme contrast of behavior. If they treated other people the same way they treated you, it would make more sense. You could argue that it was just their personality, but when you see them going out of their way to accommodate strangers and they're so welcoming to visitors and guests, but they treat you like you're nothing. When you're doing all of these things for them, it doesn't add up, it doesn't make any sense. It just makes you wonder why they don't act that way with you, but this is how you should know that it's all an act because what you're receiving is who they really are. They can't be fake for long periods of time with someone who is always around them, which is why you might see them being kind on some occasions, while in another moment they will be completely out of control. Their behavior is very inconsistent, which should reveal to you that it's all an act. They're deceiving people, which is how when you try to tell other people about how they really are, no one believes you because they don't know them. Those other people don't live with the narcissist. The narcissist is very cautious to avoid other people seeing how they are. They will go out of their way to do things for other people. They will help people for free, but then they will make a strenuous effort to argue with you about getting a chocolate at the store while they're doing all of these things for other people. They're going out of their way. They're cooking food for them while they have this disinterested and selfless concern for your well-being. But when you go to these other people and try to reveal how the narcissist treats you, there's no way that they're going to believe you because they've already witnessed all of these kind and selfless acts from the narcissist. So they're not going to believe that the narcissist is so unrestricted and unpleasant. No one will believe that all of these things are being done to you because they've already taken action to limit any damaging effects that might otherwise incur. They've already managed the perceptions of other people because they're making preparations for the real abuse which when it finally gets to that point there's not going to be anyone on your side. There's not going to be anyone who is able to identify with you what understands you. No one's going to save you from danger or difficulty and that is when they will begin their smear campaign.  They will deliberately humiliate you in public. Whenever you seem to be happy they will start an argument. They will do something to make you angry or upset especially if you have a reason to celebrate. They will tear you down. They will bring up things that you thought were already dealt with. They will blow things up out of proportion because they just can't stand to see you being happy. They can't stand to see you enjoying something. They will deliberately go out of their way to bring disturbance, confusion and uncertainty to you. It's done consciously and intentionally which is why they will often start arguments on special occasions or when you're trying to sleep at night. They will argue with you for hours about something pointless. They don't want you to have any peace because they're very envious and jealous which is why everything becomes a competition where they try to defeat you or establish superiority over you. They always find a reason to complain or criticise something and they do it in such a persistent and irritating way. They're never happy. They never feel positive about anything. They're miserable 24 hours a day which is why they always have to try to make you feel the same way. They can't let you be happy while they have to deal with it in a turmoil while they have to deal with all of these things that they don't like and you're happy and having a good time and a peace with yourself. They have to bring you down to their level which is when they start fault finding. They have to find something wrong with you or your situation. It may be difficult for you as a normal person to understand this but it is all done deliberately. They will make stuff up.  It makes them feel important even though there was never any real issue that bothered them but they will use it to make you put in work to make you try to make things right when you were never wrong anyway. They just managed to convince you that what you did was so disturbing. It's just a test so that they can see what you're willing to tolerate so that they can see what they can get away with and they will use it to justify their actions while you're trying to correct the situation because they put on this act like they're so upset with what you did when really they're just trying to make you feel guilty because that is another way that they gain power over you.  When it may not have been an insecurity of yours to begin with they just use delicacy and skill to get you to respond in the way that you did so that they can give you their design plan of what they want you to do so that they can really begin to program you and how they want you to act to behave for them even though you didn't do anything wrong they will exploit it they will take advantage of the situation so that they can get you to comply with their demands because they just want to control you but the moment that you agree that you did something wrong they will then expect better from you in their minds they believe it gives them permission to step over your boundaries until you are allowing yourself to be controlled by them until you are following their orders and you don't see it coming until it's too late until they become more and more abusive it gets out of control because they can't help themselves they have an intense and selfish desire for power and control and they're never satisfied they always want more which is why they end up overdoing it because they have an appetite that is impossible to satisfy if they had more control over their behavior they would probably have more control over you but because their behavior is so extreme it eventually ends up pushing people away because people are unable to remain undamaged or unaffected by it but it manages to lock people down for a certain amount of time which is why they're so arrogant because their manipulation has been successful in the past they manage to control the minds of other people before you they've done this so many times and it always results in their victim not functioning properly or having financial difficulties the narcissist gets what they want from you and then they leave they have to leave at some point because they know they're causing severe damage to you while they're doing these things to you they're not planning to stay with you they know they can't stay with you after they've messed you up you're not going to serve any purpose for them they just have to keep getting as much out of you as possible while they still can for as long as you're willing to tolerate it until they beat the very ego of your soul to a pulp until you're left broken and senseless where they sense of overwhelming disruption and emotion and then they're done they're not intending to be with you forever you may have children but they just see them as investments because they know that at some point they go into experience a period of difficulties and hardship so they need to have something they can recourse to when they're in difficulty but everything they did to you was deliberate it was done for their own selfish needs for their own survival but they don't like it when you leave because they don't see it as though you're leaving off of your own free will at an independent decision making they see it as though someone else has come along and taken you away from them they don't see it as though you're capable to leave on your own so they assume that someone else is stolen their property because they just see you as an object and in their minds they're trying to protect it they're trying to prevent someone else from taking you away because they don't think you're capable of going off in your own but once they've done their work on you they're very territorial over you in their minds you belong to them you're their project you're their experiment so they don't want to see someone else taking you away but if you're on your own and you're miserable they will leave you there they won't have any concern for you but once you start to wake up once you become more wiser and more knowledgeable and you start to figure things out then they get very jealous they get very possessive and controlling because they feel like someone or something is taking you away from them and then they want to cause disruption they want to intervene they try to get you back they try to do things for you but it's not because they care about you it's not because they have any concern for you they just don't want to see someone else taking what they believe to be their possession after everything they have done to secure you and keep you under their control because from the moment they met you it was all a game you were led to believe that they cared about you you were led to believe that they wanted the best for you but it was all an illusion they never took the steps to make it a reality because they never intended to make it a reality but at the end of the relationship you may experience conflicting attitudes and beliefs because you think that the character they displayed to you in the beginning was real to break the cognitive dissonance you need to understand that everything they displayed to you was an act they were just mirroring you they pretended to like what you liked and wanted but they were never about any of that they weren't about anything they're never going to make anyone happy they're never going to be a good husband or wife for anyone they're never going to make anyone proud of them because they're condemned they're never going to change they're not going to magically become what you always wanted them to be that's never going to happen it's never going to be a reality it's never going to be possible you can change and improve your life but they're always going to be this way thank you for watching i hope this video resonates with you please like comment share and subscribe if you were delighted to meet my paypal link is in the video description coach the inquiries you can email me at coaching at nark survivor the kuru uk thank you for watching and i'll talk to you soon


Appropriate because we're about to talk about refactoring infrastructure code and like another song from that movie we are stepping into the danger zone. So let's go ahead and get started. I'm Nell Chambrale Harrington and I'm a software engineer at Chef. Although I currently live in Seattle like Jill, I also used to live in Southern Illinois in O'Fallon Illinois just across the border from St. Louis. I'm glad we have some Southern Illinois representation here. So if you're free to tweet me at Nell Chambrale at any time or email me at nchambrale.io. Now in my work at Chef I have refactored a good deal of infrastructure code, some good, some bad and I would like to share those lessons with you today. So let's first talk about why you might want to refactor your infrastructure code. You might find yourself needing to refactor when you need to add a feature and it's impossible with the code in the state it's currently in. When you need to fix a bug, when you need to improve the design of the code to make it easier or even possible to work with or when you need to optimize resource usage. This can include things like memory, hard disk drive space and in the example we're about to go through we're going to optimize our use of AWS resources. So it would be one thing for me to tell you about how to refactor but today I would like to show you a real life example. In this example we're going to refactor a Terraform config. How many people here have used Terraform? All right we've got a few hands up there. Terraform is a delightful provisioning tool that allows you to provision infrastructure whether that's VMs, containers and more. And I like to say that using Terraform makes me feel like the character Elsa from the movie Frozen. I just go shoo and all of a sudden I have a AWS cluster built. Now the problem is that changing your Terraform code or not doing it carefully, it can go just as wrong as the ice magic goes in that movie. So today the Terraform config we're going to refactor is a real one. And it's at github.com slash nelscham rel slash supermarket hyphen Terraform. If you'd like to go ahead and follow along. Now when I refactor code, whether it's application code or infrastructure code, I like to start out by mapping the major components of the code. And I'm a visual learner so I prefer doing this in a visual way. So this is the first part of our Terraform config. This is the main file called supermarketcluster.tf. And in this file, I'm using certain variables needed to when I spin up my infrastructure. So in this case I'm spinning up AWS instances. So I need to include things like my AWS access key and my secret key in order to spin things up in the correct account. And I define these variables in a file called variables.tf. That's the second major file within my Terraform config. And finally I provide the value for these variables in a config file called Terraform.tf bars. So these three files are the core files within this Terraform config. Now some of you who have used Terraform with AWS might be asking, wait, shouldn't I use an AWS credentials file rather than passing my credentials in line? And the answer to that is you certainly can and that's even considered the best practice. But for the sake of this example in this presentation today, I'm gonna go ahead and pass them in line. So let's take a closer look at supermarketcluster.tf. Now this file has way more code than I can show on a slide. There's nearly 300 lines in it. So instead of showing that code, again, let's go through a visual representation of what it does. So when I run Terraform apply with this config from my workstation, it's first going to create an AWS security group. Then it adds a rule to allow SSH access into any instance that I attach the security group to. Then it spins up a second security group and adds another rule to it. This rule will allow traffic to flow from within my AWS instances out into the public internet. Then it spins up an EC2 instance and configures it to be a Chef server. The next component is another EC2 instance and it configures this to be a supermarket server. Now supermarket is another Chef product that can be used in conjunction with a Chef server. It provides an easy to navigate artifact repository for sharing Chef cookbooks and tools. So the next thing it's going to do is configure supermarket to use that Chef server for authentication and authorization. And then it's going to configure the Chef server to be aware of the supermarket server. And I can do all of this from one Terraform config. Finally, it's going to configure the user's workstation who's running that Terraform apply command to use the newly spun up Chef server whenever it needs to use the Chef server. And it's going to configure it to use our newly spun up supermarket. And that's the end of the config. So the whole point of this config and spoiler alert, the one that we're going to be refactoring is one that I wrote. It was actually my first Terraform config I ever wrote. The point of this is to spin up a complete cluster for development and testing for Chef supermarket. So let's pose a hypothetical. Let's say management says that we are using too many AWS security groups in our AWS VPCs or virtual private cloud. So the default limit for a VPC is 500 security groups. And yes, you can increase that limit. But again, for the sake of the example, let's say our company does not want us to increase it. So we need to meet this hypothetical to change this config to create only one security group rather than two. So this is the why of why we need to refactor. I always like having a specific reason that I'm going to refactor code. Now let's go over how to refactor. And there are two common approaches to refactoring, whether it's application code or infrastructure code, according to the book Working with Legacy Code. And the first is what's called edit and pray. This involves making a change, often a major change, and deploying it, and just hoping and praying that it works. I've sometimes heard this referred to as PDD or prayer driven development. The second approach is called cover and modify. And this one is harder, but it's much more effective and much less risky. This involves covering the section of code you want to change with automated tests and then making the changes. So last year, I presented this approach on a talk on refactoring application code. And during Q&A, someone grabbed the mic and said, and I quote, this is a bad approach. You should just look at the code and change the code and know what you're doing. Now my reaction to that was I don't have that much faith in myself, to make changes cowboy style and have it not break anything. In fact, I don't have that much faith in anyone. And this is because confidence in code without tests is false confidence. I don't trust a human, thank you. I don't trust a human, least of all myself, to know all the ins and outs of how my code currently works and all the unexpected side effects of the code, which other parts of the code might be depending on, and that it will keep that functionality after I make changes. And this is because what the code is intended to do is much less important than what it actually does. The only way to know what my code actually does is to actually execute it. That is the ultimate source of truth. And the beauty of automated tests is they allow me to execute my code automatically. So when I present this approach, sometimes people immediately dismiss it as being impractical. If they have a giant legacy code mess, how in the world are they supposed to get the whole thing covered in tests? They ask me, are they supposed to suspend new work until they can add tests for everything in the code base? And the answer to this is unequivocally no. Even if it were possible, it would be very impractical to try to cover a whole code base in tests and all its associated templates and other support files as it is right now. And frankly, good luck selling that to management. So the point of adding tests, the reason we add them when we are refactoring, is to not make things worse, to preserve the current functionality of the code at the very least, and to start making the code better here and now. So the next question is, how can we test Terraform? And to do this, I'm going to harness three open source tools. Now the first is Test Kitchen. How many people here have used Test Kitchen? All right, we've got a number of hands up. So Test Kitchen is a delightful tool that allows us to spin up a VM or container, configure it with our Terraform config, and then run tests on it to verify that our code did what we expected it to do. So if you want to find out more about Test Kitchen, head on over to kitchen.ci. Now Test Kitchen was originally written for use with Chef Cookbooks, but it can also be used with Puppet Manifests, Ansible Playbooks, and more. In order to use it with Terraform, we're going to use a special provisioner called Kitchen Terraform. This just came out in the last month. Kitchen Terraform comes from a company called New Context. So we need this provisioner to not only be able to provision new AWS EC2 instances with Terraform, but also to be able to SSH into those instances and run our tests. And you can find out more about it at github.com slash newcontext slash kitchenterraform, and I'll be tweeting that link out after my presentation. So in order to use both Test Kitchen and Kitchen Terraform, we need to configure it through this .kitchen.yaml file in our configurations repository. So the first thing we need to do is configure the driver. This defines where we want our instances to be spun up. In this case, we're saying we want them to be spun up through Terraform, and the Terraform config itself will say we want to spin them up in AWS. Second, we define our provisioner. This is what will actually create the instances. So conveniently, our provisioner is also called Terraform, and we also needed to tell the provisioner which file contains the values for the variables our Terraform config uses. So in this case, remember, we're defining those values in a file called terraform.tfrs, so we tell our provisioner that's where it should find those values. And in order for Kitchen Terraform to run tests on our instances we spin up in AWS, it needs to be able to SSH into them. In order to SSH into it, it needs access to an SSH key. So in this transport section, we provided a path to that key on our workstation to use to SSH into those instances once they are provisioned. And the rest of this is some additional Test Kitchen boilerplate, so if you'd like to learn more about that, head on over to kitchen.ci. And finally, the framework we're going to use to write our tests is called InSpec, which is similar to server spec. So to find out more about InSpec, head on over to chef.io.inspec. So now that we have our tools, we're going to go ahead and start from a clean slate. We need to figure out the bare minimum we need for our tests to run first. So let's open up our config file, and what I like to do is to just comment out the entire thing and rebuild it piece by piece. When I'm refactoring application code, I use this similar approach when I'm adding tests for a method I want to change. So at this point, with everything commented out, if I ran Terraform apply from my workstation, nothing would happen. So this is a good starting point for us, and let's figure out what the bare minimum is we need for our tests. So first, we need a provider. In this case, it's AWS. So let's go ahead and uncomment out that provider resource in our Terraform config. And along with our AWS provider, we also need actual AWS EC2 instances to run our test on, including both that Chef server. So let's take a look at the Chef server resource within our config, and it's pretty standard. We're spending up an EC2 instance, we're passing in the AMI, the instance type, and the key name as variables. So let's go ahead and uncomment this out, so it actually fires. And at this point, if I ran Terraform apply, I would expect it to spin up an EC2 instance. So the next EC2 instance we need to spin up is our supermarket server. So let's take a look at that resource within our Terraform config, and it looks very similar to the Chef server resource. So let's go ahead and uncomment it out. So now, with this uncommented out, when I run Terraform apply from my workstation, I will expect it to spin up those two EC2 instances. But, we also need at least one security group in order to spin up our EC2 instances. Now, we could use the default security group in our AWS account, but I don't like doing that in tests. And the reason is that default group is out of the control of my tests. It would be easier for someone to change it, and then suddenly have my tests failed in an unexpected way. So I'd much rather define my security group in the code. So let's head on over to the section of the code that defines our security groups, and let's uncomment that section that will spin up our allow SSH security group. So at this point, if I ran Terraform apply, I would expect it to spin up that security group and my two EC2 instances, and then assign them to that security group. Now there's one more thing we need to do. Remember that kitchen Terraform needs to be able to SSH into our instances in order to run tests on it. So we need a security group rule in addition to that security group to allow SSH access. So again, we head on over to our security group resource, and let's uncomment out that AWS security group rule. So that's gonna allow SSH traffic into our instances through port 22. So at this point, if I ran Terraform apply, I would expect it to spin up that security group and that allow SSH rule and those two EC2 instances and then assign them to that security group. So now this is the bare minimum we need to create our test cluster. So let's go ahead and do that. And through, if we're using test kitchen, we're gonna do that through the kitchen converge command. And this is a lot like running Terraform apply, but we're running that through test kitchen so we have all the benefits of the test kitchen framework. Now the first time I run this, I'm going to get back an error. This is gonna tell me, it initially doesn't look that helpful, but it's gonna tell me to view the log file at dot kitchen slash log slash kitchen dot log. So let's go ahead and take a look at that file. And when I open it up, my eye is drawn to this line, unknown resource AWS security group allow egress referenced. It looks like we are telling our EC2 instances to use a security group. We're not actually creating at this time. And this is because at this point with what we have uncommented out, we need our EC2 resources to reference only the one security group that we have uncommented out at this time. So let's take a look at that Chef server resource again and looking at it, you can see that we're assigning it to two security groups. So let's change that and only assign it to the allow SSH security group because that's the only one we have uncommented at this time. Then we head on over to the supermarket server and we do the same thing. We change it from assigning it to two security groups to assigning it to one security group. So at this point, I'm gonna run kitchen converge again and this time it's going to complete. And yes, if I were really running this from a terminal, that kitchen converge would take much longer, but for the sake of time in this presentation, I decided to use a little bit of magic. So at this point, remember we have our security group with our allow SSH security group rule and our two instances when we run Terraform apply or when we run kitchen converge. So now we're at the point that we can actually write some tests and to do this through kitchen Terraform, we first need to define a test group. So let's open up our .kitchen.yaml file and the first thing we need to do is define the verifier that test kitchen will use to run the tests. And in this case, it's also called Terraform and then we define our first group. Got a little on my head of myself. We define our first group, which I'm gonna call the default group and then we define the test files that will run as part of that group. So I'm saying my file is gonna be called security groups and I'm gonna go ahead and create that in just a moment. Finally, let's skip over that hostname's attribute and let's look at this username's attribute. I've got a bunch of lemmings in the back row all of a sudden. Sorry, it's so low. This username is Ubuntu and that's the username that kitchen Terraform is going to use to SSH into the instances. When I spin up an Ubuntu instance in AWS, I need to use the Ubuntu username, at least at first, to SSH into it. And then let's look at this AWS hostname's variable. This is the output variable we will use to capture the hostnames of our two EC2 instances. We need to know their hostnames in order to SSH into them with that Ubuntu username. So let's go ahead and create that output variable and we do this through the outputs.tf file in Terraform. So I'm going to define my variable, which is AWS hostnames, and then I define the value for that variable, which in this case is the public domain name of the Chef server and the public domain name of the supermarket server as a string separated by a comma. So it's one thing for me to show you the code. Let's look at another visual of how this will work. So when we run our Terraform config from our workstation, it's gonna spin up those AWS resources. Then it will capture the public DNS value of the two EC2 instances in that variable and then it will pass that variable to kitchen Terraform and then kitchen Terraform will use that variable, AWS hostnames to SSH into our instances and then it will run tests on those instances. So before we try this again, let's go ahead and destroy our current test infrastructure to give us a clean slate and to make sure we have a value for that output variable and then we're gonna run kitchen converge again. So when this runs successfully, we now have a test infrastructure we can run tests on. So we can start thinking about writing those tests. Now it's good to start out writing tests just for the portion of the code that you want to change. So remember, our hypothetical was that we need to condense from using two security groups in our code to using only one security group. So now that we have already have one defined that second security group resource is still commented out. And before we uncomment it, let's go ahead and write a test. So the test I'm gonna write using InSpec is I'm gonna tell InSpec when it runs the command pinggoogle.com on one of my instances, it should receive the output one package transmitted, one received. This means that my instance can send traffic out to the public internet. So after I write this test, I run it by running kitchen verify and the first time I run it, I'm gonna get a failure. This is gonna tell me that the output was not one package transmitted, one received, it was one package transmitted, zero received. So this tells me our instance in its current state cannot send traffic to the public internet. And this is good. It's good to have a failure first because this means our test will fail when we expect it to. So let's go ahead and make it pass. And we do that by heading over to that security group that allow egress security group and uncommenting out both the security group and the security group rule. So at this point, if I run terraform apply, I expect it to spin up both those security groups and both those EC2 instances. So now we need to call the security group from our Chef server. So let's head on back into that resource and we're gonna change it from using just that allow SSH security group to again using both security groups. Then we go ahead and do this in our supermarket server. We do the same thing, we change it from only assigning it to that one security group to assigning it to the two security groups. So once again, we're gonna run kitchen destroy and kitchen converge to give us fresh test infrastructure to work with, then run our test using kitchen verify. And this time when I run the test, I get it shows me that the test passes. So now this section of the code that second security group is covered by tests and this means we can go ahead and make a change and be confident that it will work. So let's go ahead and condense these two security groups into one security group. And we do that by deleting the second security group that allow egress security group and then changing that security group rule that allows traffic out to using the allow SSH security group rather than the security group we just deleted. So at this point, our instances should only use one security group. And that tells us we need to go back into our Chef server resource, change it back from using those two security groups to using that one security group. When you're refactoring the test, you're gonna do a lot of changing things back and forth. Then we head on into our supermarket server and do the same thing. We're only gonna use that one security group now. At this point, if we were to run Terraform apply from a workstation, we would expect it to spin up both those EC2 instances, one security group with both security group rules and assign that security group to those two EC2 instances. So now let's run kitchen destroy and kitchen converge followed by kitchen verify to run my test again and my test passes. So we have now changed functionality without breaking our code. But let's not stop here. Let's go ahead and improve the design of this code to make it easier to work with both now and in the future. And we're gonna do that by moving our security group code into a module. And the benefits of moving it into a module are that a Terraform module is a self-contained package. It's a reusable component. We could theoretically use it in other Terraform configs. And it's a way to improve organization of our code, make it easier to find what we need, when we need it and to fix it. So let's go ahead and run kitchen destroy. And then we're going to make a directory for our new security group module. I'm gonna call it security groups. And then I'm gonna create a file within that directory called main.tf. And this is going to be the main config of my security groups module. So once I create this file, I'm gonna copy and paste the security group code into it from my previous supermarket cluster.tf. So now we need to connect this module from the main config. And in order to do that, we first need to know what variables the module needs passed into it. So looking at our security group code, when I see a variable that starts with this var prefix, that means it needs to be passed into the module from outside of it. So we definitely need to pass this one into our module. Now when we look further down the config and we see a variable like this, this one refers to a resource within the module's config. It will be determined when that module runs. So since this value is coming from within the module's config file, when I practiced this talk, someone told me I should make a joke here about how the variable is coming from inside the module. It does not need to be passed in from outside of it. So again, looking at this visually, we have our supermarket cluster.tf, which uses the variables. We define those in variables.tf. We define the values in the terraform.tf var's file. And in our module, we use those var, we also define variables in the variables.tf file for that module. And let's go ahead and create that file now. So we're gonna declare, we're going to use a variable called username. And again, looking at the visual, that main.tf is going to use the variables that are defined in variables.tf. So effectively what we're doing is we're passing variables from the supermarket cluster.tf to our main.tf in within our module. In order to do that from the supermarket cluster.tf file, we're gonna define a module resource, tell it the source of that resource, which, or source of that module, which in this case is a subdirectory within our directory. And we are going to tell it that we're gonna pass it the username variable. So when I run Kitchen Converge, testing these changes, I'm gonna get an error at first. So I'm glad I checked this before deploying this out into a production environment. And looking at that default Ubuntu log again, I'm gonna take a look and my eyes drawn to this line, which is unknown resource awssecuritygroup.allow ssh. The problem is that our supermarketcluster.tf file is trying to access that resource, but that resource now lives in a module. It can't access it directly. So the next thing we need to do is create an output.tf file within our module and pass that value back to the config. So I'm gonna create that file, output.tf. I'm gonna name that variable, name that value, sgname for securitygroupname. And then the value for that variable is the awssecuritygroup.allow sshname. So finally, we need to use this output in our supermarketcluster config. So again, we're gonna change it from using that securitygroup, referring to the securitygroup resource itself, to referring to the sgname output from that module. Then we're gonna go ahead and do the same thing from our supermarket server. Change it to use the securitygroup name that will be output by our module. We're gonna run kitchenconverge again. When that succeeds, we're going to run kitchenverify. And when I run my test, my test passes. So at this point, we have one improved our resource usage and the code's organization and design with minimal risk. We can be confident when we deploy this that it will do what we expect it to do. So infrastructure code must be maintained and refactored just like application code. Even more so because infrastructure code involves so many moving pieces, as you saw from all those visuals. When refactoring code, always cover it with tests first. That's what I'm hoping you can take away from this presentation and refactor one small piece at a time. Thank you.