Something I've just learned is that I can actually detach these windows if I drag them into the file area. So that's kind of nice because now I can switch between windows and I can have one very large and not everything is so large and that will make it easier to look at plots and things. I've also made a few small updates so if you want to just use the version control icon and pull branches you will have the latest version of the EDA introduction. What did we learn about quantile plots? What interesting things did you find? So I had a number of discussions about what do these quantile plots actually mean. How can we understand what we're doing here? If this was a quick post-prandial quiz, could you write down meaning of a quantile plot and get away with it? So essentially what we're doing with these quantile plots is we're comparing the shape of a distribution. So it's relatively easy if we analyze quantitatively to compare means and variances and standard deviations and ranges and everything. But there's more subtle information encoded in the shape of the histogram, in the shape of the distribution. And that's what we do with the quantile plot. Essentially we build a histogram along the quantiles and then we compare how these histograms match up against each other. So we get rid of any variances that are simply due to shifts. We get rid of variances that are simply due to scaling. And we basically look at the shape. And if the shapes are very similar, then the quantile plots will fall on a straight line. If the shapes are different, then the quantile plots will differ. And I think what many of you who have compared controls against the LP estimated data have seen is that there's a break in the data. So the parts on the left-hand side are kind of linear. And then around the value of minus 10, usually it shifts, it drops down, or there's a curve there, or stuff is different. The way I would interpret this is that with the values of the small enrichments that we're seeing, the controls and the stimulated distributions are essentially the same. So there are localized differences, but essentially the differences in the individual genes correspond to noise. So there's no big difference there. But there are significant differences in the higher enrichment sections. And that's where things actually happen. So whenever you have distributions, or plots, or lines, or regressions that seem to have two lines, you always have to think about probably there's more than one population in my sample here. And in this case, the two populations that would come to mind are the regulated and the non-regulated genes. The ones that correspond to an induction with LPS, and the ones that do not respond to such an induction. Now, if we know that and we can say, well, all over all, there seems to be a breakpoint between the regulated ones and the non-regulated ones around the value of minus 10, we can use this information for subsetting the data. And we could then just pull out all the values where genes at some point fall below minus 10 or not, and thus increase the contrast between signal and noise in our data set analysis. There's nothing we knew about that before we looked at the data. So this is a completely unbiased exploration of the data, looking at some of the distributional properties, i.e., essentially the shape of the histogram, and then from that, figuring out that probably there are two subsets of data there and how we can pull them apart and use that for further analysis. So this is why QQ plots are interesting. It'll probably take a little while to wrap your mind around what that means. Again, the best thing to do is remember that we're basically looking at the shapes of distributions here. Think of how you could use something like that in your own data at home, for example, by then comparing distributions from your own data against normal distributions with QQ norm or with between controls and experiments. And ultimately, it'll click and it'll work, and you'll be as confused as now but on a higher level, as we say. Now I'd like to shift gears, still stay with exploration of data, and look at a few scatter plots with flow cytometry data. So not to go too deep into scatter plots, there are loads of examples in the plotting reference, but I'd like to just create some visuals before we move on to completely other things. This is a table, gvhd.txt in the package. It's a rather large table. And this is tab separated values, including a header row. And I got this many years ago from Sorab Shah, I believe. This is graft versus host disease. So flow cytometry of graft versus host disease. This time we read this with read table, graft versus host disease.txt. Header equals true because there is a header line. And that's what this looks like. So I'm not entirely sure what these channels are. This column here is fit C labeled CD4 surface antigen. This column here is CD8 labeled probably FICO erythrine. This would be CD3 labeled per Cp. Another different wavelength through R04. And this is probably aloficocyanine. Yeah, exactly. So different fluorescent dyes. How do they color the cells? How do we get the dye specifically onto the cells? The cells have different fluorescent colors? Antibodies. So we chemically conjugate the dye to an antibody which specifically recognizes CD8 or CD3 or CD4. Note that we have two columns for CD8 here. What can we do with that? Exactly. We should expect that these two channels correlate very well with each other. One value is high. In one channel, it should also be high in the other channel. That's a good internal control of a dye effect during that labeling. OK, so let's play with that data. For example, we could extract only the CD3 positive cells. If we look at a histogram, what would we consider to be CD3 positive? You know, there's a distribution of labeling events for the purposes of the tutorial. We've always used a number of 280. That's about here, breaking it around the middle. And then basically subset our data and say, graph versus host disease CD3 positive is a data frame of our original data where column five of our original data is greater than 280, so subset. And of that, we want the columns 3 to 6. So now we have the four dye-label channels here in columns 1, 2, 3, and 4. And only some of the rows are used. How many rows do we still have? How many elements in this? How can we tell? How many cells are still in this data set? Can we do DIM? I don't know. Let's try. What does this mean? DIM. It's the dimensions of our object. So this is a two-dimensional object. And it has 2,089 in one dimension and 4 in the other dimension. We've already seen that. We have four columns. So that would be the columns in 2,089. Any other way? Can we not click somewhere? Let's click somewhere. Data here. If we click on there, and we'll look at it, and it tells us 2,089. Is there another way? We can do structure, data frame of that. Can we have another way? Why am I harping on this? This is because if I do DIM, I think this is completely correct. Looking it up and using structure basically has that information as a side effect. If I want to write it into code, I would like to make more explicit what I'm actually doing. And thus I would use the command n row to tell me that. Why do I need that? Well, sometimes I need to iterate over it row by row by row. And then I often put 4i in 1 colon n row of whatever the object is. I think that's the most explicit way to document what that number is, what it does, and where it comes from. There's a Cognate n call that does the same thing for columns. OK, so let's plot it. So here we plot all the rows and two columns. So what's the x and what's the y in this? So if you think about this as two columns of the entire thing, then the first column is implicitly used as an x value and the second column is implicitly used as a y value. And that's our scatter plot of Kraft versus host disease, CD4 versus CD8. Incidentally, we just mentioned that CD8 against CD8 would be a good internal control, and it should be highly correlated. So that would be column 2 against column 4. Let's have a quick look. This is different. This is columns 2, 3, and 4. OK, well, since we have it now, if we give the plot function more than one column of a data frame, more than two columns of a data frame, it shows us all of the combination plots. So I mistyped here instead of 2 and 4, i.e. C2 comma 4, I typed 2 colon 4, which expands to 2, 3, 4. So now I'm seeing a scatter plot of column 2 against column 3, 2 against 4, 3 against 2, 3 against 3, against 4, and so on. So the one that I was looking at is 2 and 4. This would be this one or that one. This is 2 against 4, and this is 4 against 2. Same difference. And what do we notice? Are they well correlated? Not really, right? And it's kind of weird. It's not just noise. So apparently some subsets of these cells can be easily labeled by both fluorophores. And some subsets of these cells don't, especially in the middle intensities. There's something weird going on. As the signal of CD8APC increases, the signal for CD8BPE doesn't seem to increase as much. So the truth of the number of receptors is somewhere between these extremes of plots. And that's what you have to keep in mind when you look at data like that, this way. That's what I meant. Interesting. So there's a lot of overlap in these plots. And I would just like to demonstrate a few ways to work with data that is very dense and to find basically what the underlying distributions of elements are like here. There's so much overlap that we can't really appreciate the fine structure. Now, one way to look at this is Hexbin. So Hexbin is kind of like a two-dimensional analog to histograms. So instead of having histograms all on one line, it puts histograms into the plane. And each single histogram has a hexagonal shape so that the plane is completely tiled. And the elements that fall into each bin are approximately equally distant from each other. And there's a function here, Hexbin, which we can feed our graph versus host disease. We can specify how many bins we want. And we can specify the colors that are being used. So the Hexbin actually basically specifies a data object. And the plot function then knows what to do with that data object based on the class that is assigned to it. So this is a Hexbin plot, useful for very dense data. So now looking at this, I think we can appreciate very clearly that there seem to be at least four different populations of cells in our graph versus host disease that we probably need to consider differently. It's the same way of like we looked at our QQ plot and determined that probably there's more than one population in the genes. And we should analyze them separately, thus that would be basically the same with these that's demonstrated with the Hexbins. Yeah? So in the column. Right. So there's a lot of information on colors in the plot reference. But essentially what happens is that the function color ramp palette returns as its value a function. And that function generates the colors. So let me show you what I'm talking about here. So this is a very bright yellow color, much red, much green, lots of blue. And this is a kind of a dark blue color. And these are some intermediate colors. So basically this specifies fixed points in a possible gradient of colors. And the function color ramp palette creates a function that can be assigned to the variable. So if I have, if I assign the output of color ramp palette to the variable f call, then I can use f call to create color values that will lie on that ramp. So if I need five color values, it will give me five color values starting from the first one I specified, going to the last one I specified, and having something in between. So notice I only specified four, but I can get five colors here. If I need more than five color values, i.e. in this case, I need 20 color values, if call will give me 20 color values. Again, starting from the beginning, ending at the end, and going smoothly in between. So this is how we can end up with as many color values as we want, smoothly varying. Now, if I would have changed this to something more purple, I would get a different color ramp. So I can either get very smooth or very different. The use of color in plotting needs some care and thought. So what we can do is we can color either in a quantitative sense, i.e. one extreme of values is, say, light, and the other extreme of values is dark. Often we use things like heat colors, so black for low values, and going via red and yellow all the way up to white for very high values to emulate the emission spectrum of iron or something. But we can also use colors for categorical differences. So if we have apples and oranges, we would color them green and orange, for example, because that is a good mnemonic for us to identify categories of elements in our data plots. So these are different ways of using colors, so it shouldn't be arbitrary. But there's a fair bit of information and some examples in the plot reference script. OK, so Hexbin is one version of doing this. The other version of plotting and dealing with overlap is something called smooth scatter. This basically smooths the points and gives us this kind of a cloudy, low-resolution appearance, kind of reflecting the fact that there's error associated with every single of these data values and then giving us dark colors where there's a lot of overlap. Somewhat similar is if we vary the colors themselves by density. So if a data point falls into a very dense region, we would give it a darker color. And if it falls into a very non-dense region, if the next neighbors are all far away on the scatter plot, we could give it a lighter color. And this is by using the dense calls function on the two data columns as the color argument in a plot. PCH is the plotting character that we use. CX is the character expansion. So the plotting characters we use can be circles and squares and triangles and daggers and stars and different things. And these are identified by individual numbers. Again, refer to the plotting reference. Now, PCH 16 is a way to plot things so we can specify the elements to be filled with a particular color. And CX, the character expansion, specifies whether the plotted symbol is going to be large or small. So if we plot this here, it kind of looks like the scatter plot, but more defined. If we make CX smaller, we get smaller dots. And so you can also appreciate the internal structure of data sets that overlap. And this trellis plot, which is implicit on this kind of lattice, we've accidentally stumbled in before. So that's what I basically wanted to cover as a first introduction to looking into data. We've talked about plotting data in various ways, defining some statistical measures, some statistics like mean values and variances in order to characterize our data, how to use quantile analysis to analyze the distributions of our data points and compare them. But to get a little more quantitative than that, one of the most frequent types of analysis we do are regression analysis. So let's spend some time on regression analysis. So we'll talk a bit about correlations. We've mentioned the word correlation before, but let's have a closer look at what correlations mean. We'll talk a little bit about linear and nonlinear regression. Unless there's huge demand, I'm actually not going to explicitly go over nonlinear correlation. But the principles should be clear from linear correlations. And there's code for how to specify functions and how to take functions through nonlinear least square fit. We'll talk about that later. And of course, we should be able to compute or not compute them. So here's a scenario. These are points on a scatter plot. And that's a very noisy scatter plot. But nevertheless, even though it's a noisy scatter plot, we kind of see that there's a trend that if values are small on the x-axis, they're kind of also small on the y-axis. And conversely, if they're large on the x-axis, they're also kind of large on the y-axis. So some of the information that we store in the x and y is actually redundant. Because some of the information of x can be recomputed and then give a value of y, or can be used to predict values of y. So this is what we mean when we say data is correlated. i.e. a part of the data depends on another dimension of the data. Now if we assume that just for example, the x values could be the height of people that we've measured in meters. So we have small ones and tall ones. And the y value could be their weight in kilos. So we have small ones that are rather light and tall ones that are more stout or in the middle here. And the question is, how do we quantify it? To what degree these are correlated? If we quantify that, what does that even mean? And how do we get the parameters that can define this correspondence? And that's where we get into correlations. So correlations don't imply causality. That's another question. In r, the function core correlation measures a correlation coefficient. And the values can go from minus 1 to 1 and 0 means no correlation. So these correlations are parts of models. And linear regression is one of the possible statistical models that we can apply to data analysis. Now for a statistician, a model is not what you perhaps might think it is. So model does not mean we have a kind of a quantitative understanding of nature, and we can plug in some values into that quantitative understanding, and it will spit out the correct results. A statistician's model is simply a statistical function that creates data that has properties which come as close as possible to the properties of the observed data. So it's a black box. But it is supposed to have the same properties statistically as the observed data. So what we do here in modeling is we specify a model. In this case here, it would be that there's a linear model that we can apply. We estimate the parameters of that model. So for a linear model, the parameters would be the slope and the intercept of the y-axis. And then we ask, well, is the model any good? Does it actually explain our data well? If it is not good, we either re-estimate the parameters or we specify a different model. But if it's adequate, we can use the model, for example, for prediction. Now, linear regression assumes a particular model. And the model is that there is some offset plus some proportional factor here, basically a multiplicative factor plus some errors. And these different components of the model go by different names that you will encounter in the literature, things like independent valuable or predictor valuable, regressor, exposure variable, and maybe input variable. And the result is the dependent variable. So there are two parameters that you can independently fit. One is the regression coefficient, i.e., something that characterizes the slope. And one is the intercept. So the implicit assumption here is that our whole system only has two variables that are of interest. Of course, there's also errors or noise, but we would like to try to exclude that. One variable is a response and one is a predictor. We don't need to adjust for confounding or other between-subject variation, i.e., confounding means if we have two variables that change in the same direction due to a similar effect. So one example of confounding is it's well-known. Actually, I don't even know. Do we have storks in North America? Do we? I've never actually realized that in Canada. Yeah, anyway. So in Germany, there used to be lots of storks. And as you know, that Germany is also one of the first world countries with declining fertility rates. So fertility rates are declining. And it also appears that the number of storks are declining. So there's a very clear correlation there between the number of storks and the number of babies that are appearing in the households. Does this mean that we're validating the idea that babies are brought by storks? Not necessarily. So is it just spurious? It's not necessarily spurious either. The underlying realization is that both of these effects are caused by industrialization or affluence. As people become richer and are perhaps better embedded in a social network and have old age security, it becomes less important to have many, many, many children to feed you porridge once your teeth fall out. And at the same time, industrialization also reduces the wet meadows, which produce lots of frogs, which are good food for the storks. And thus, the habitat of the stork is reduced. So this industrialization is a confounding factor which acts both on the number of storks as well as on the fertility rates in society. And both of these effects are influenced in the same direction. So if we analyze them one against the other, we will find a correlation. But there's no causality implied. However, there's a relationship which is determined through that so-called confounding factor. We could adjust for that confounding factor by simply measuring how much of an effect the confounding factor has, and then specifically removing that confounding factor effect from our original data sets, and then asking whether they're still correlated. And usually, we will find that they're then no longer correlated. So this is an adjustment that we would need for confounding factors, which we can do with basically regression analysis that we would do beforehand. So it's part of cleaning up your data before you analyze it. But that's the assumption. Any kind of cleaning up of the data that needs to be done is already done before we throw our data into the linear regression analysis. Another assumption is that, well, it's actually linear. It's not quadratic. It's not synocoidal or anything else, not exponential, but linear. If it's not linear, it can be fit. Anything can be fit. But the fit will not be meaningful. The correlation coefficient, the variance should be constant, and especially it should be independent of the size of the value. So we shouldn't have larger variance if the values are large. The errors should be independent of each other. And for proper statistical inference, the errors or the noise should actually be normally distributed. So even a simple statistical regression, like linear regression, has a number of implicit assumptions that we make. Fortunately, most of these assumptions are not bad. So we can live with them. Now, once we do linear regression, we estimate the parameters. And then we need to ask ourselves, well, how good is the model in the first place? Can we believe our parameters? How much of our data would then be covered by a prediction? And one of the reasons why linear regression is so important is because it has an analytic solution. What does that mean, an analytic solution? And what's the opposite? Have you ever heard that term? Comes up a lot in computation. Well-behaved problems may have analytic solutions. Messy problems have numeric solutions if they have solutions at all. So analytic solutions are solutions that can be solved in closed form with an algebraic formula. Numeric solutions don't have closed form solutions. So we need to start trying things or simulating things. Somehow running an optimization algorithm that will find the solution by slowly, slowly, slowly iterating itself towards a correct solution. That's numerical. These things are often slow. They often get stuck in local minima. There's a number of other issues. And nonlinear least squares fit, i.e. fitting fundamentally different models, like logistic regressions or trigonometric functions, have these properties. Linear regression, on the other hand, is well-behaved. There's an analytical solution which we can compute. This means it's fast. It's efficient. We can apply it to very large data sets. And it's a good assumption for a first kind of look. So let's study this in code. Mohamed? Yeah? Is that something that we measure, or is this something that we measure? There are new statistical methods that actually try to work with quantitative estimations of error terms and get some interesting results. For our practical purposes, what we do is we estimate the slope and the intercept and then treat these as the residuals, i.e. the differences from the predicted value and the actually observed value. But that these are residuals and not true effects and the data is, again, an assumption that perhaps can be improved upon. OK, so this is a different R project. The project is R EDA regression. You download it in the usual way with the URL of the repository, github.com, hughin, R EDA regression. R underscore EDA capitals hyphen regression. And once that is loaded, type in it. So we'll have some stuff. Anybody stuck at that point? Wave your hands. Put up a red post it. No, everybody got it. Wonderful. So in any kind of modeling analysis, it's really important to be able to generate synthetic data. You're going to try to estimate parameters of an unknown distribution according to some model. And how would you know that your estimation is correct? How would you know that the functions that you're using and the procedures that you're using actually have the capability to find the right parameters? Well, what you do is you generate some synthetic data for which you already know the parameters. And then you try to recover the known parameters. And if you can get something that's close to the known parameters, you may have some confidence that your workflow is valid. If you admit to do that, it's just a wild guess. It may be right for the wrong reason. It may be entirely wrong. And if you publish results based on that, you're in a good way to have a conversation with the dean about your recent retraction. Uncomfortable. What? The model? The one is to use the new. Would it require that? Unfortunately not. It's just pretty terrible. But what people do more and more frequently is they take all of their code that they've used in the analysis of the data. They post it on GitHub, and then everybody can start reproducing the data if they're interested. That's the modern way to work. Make your reviewers feel confident that they can trust your data because you apply the utmost care and generate it. Now, here's a small function that generates synthetic height and weight samples according to a function that I've essentially have just made up. It takes the parameter n, the number of variables or random numbers that we would like to retrieve, just like our norm has that parameter, and that tells us how many numbers it should generate. It has a parameter hi, height minimum of 1.5 meters, just to constrain it. A number ha, height maximum of 2.3 meters. I think that's kind of reasonable. It has a scaling factor of 40 that kind of seemed to give sensible numbers to me. We set a seed value here to have things reproducible. If I would be doing this for a function that I would reuse more often, I would allow the option to turn off the seed or to pass the seed as a parameter into the function, not to always get the same values. Then I make a matrix, something like a two-dimensional vector, with n rows and two columns. And then I generate a column of heights in the interval defined by hi and ha. And that uses the function r uniform. So this is not run if this is r uniform. Oh, OK, so that's not defined. So let's say seven values between 20 and 23. So this gives me values. And where the values for the normal distribution were, of course, distributed according to that bell curve, the values of the uniform distribution are just uniform within a specified interval. So basically, this is random numbers in the interval. And then I generate a column of weights with a linear model. And the linear model is simply q, i.e. this number 40 times what we find in the column of heights and adding one. And after we've computed that, we also add noise to the data. Otherwise, we would have a perfect linear correlation, but adding noise to that gives us some variability. And then we can create an object simply by saying this function, return me 50 values with the default parameters. And we can plot that in the same way we used scatter plots before. If you have a very good visual memory, you will notice that this is strikingly similar to the plot that I have shown in the PowerPoint slides for reasons that are probably self-explanatory. OK. Now, what about the coefficient of correlation? That's easy to calculate. It's just a function c, o, r with the first column and the second column. So that calculates the correlation between height and weight. Bingo, here we are, 0.54. Yes. What does that mean? Good, bad, high, low. What should we expect? Not sure. I'm not sure. I don't know. This is a number. We often have these procedures that give us numbers. So we need some frame of reference to interpret this number. What's a high correlation? What's a very high correlation? How does that look like?  Let's ask, what kind of correlation coefficient do we even get if we take a perfect correlation or a less than perfect correlation? And for this, I take 50 random values according to the normal distribution. And I set a little constant here. And then I define the y values that correspond to the x values as the product of this constant plus 1 minus r times another set of random values. So what this means essentially is that in this case, I have 99% of my data has a perfect linear correlation. And 1% of my data is random noise. In this case, 80% of my data has a perfect correlation. And 20% is random noise. And in this case, 0.1% of my data is correlated and 99% of my data is random noise. And we can plot that and then see what the correlation coefficients are. So this is the first case here. Noise is only 1%. Correlation coefficient is 0.9999577. So very close to 1. If we have a correlation coefficient like that, that's kind of what the data looks like. Very, very much everything falling on a single line. Here it gets scattered out a little more. This is a correlation coefficient of 0.97. You already have significant scattering, 20% randomness, but still r value of 0.97. 60%. In this case, the r value drops down to 0.57. So this is the kind of a data cloud. This is not our height and weight, even though it kind of looks similar. But that's approximately what that looks like. Yeah, remember, our correlation was 0.54. That correlation is 0.57. So they kind of look similar. So correlation coefficients of 0.5 or even 0.4 are still considered quite significant. Matters become a lot less trustworthy if the correlation gets less. And in the case of just 1%, so this is essentially the random correlation, and the coefficient here is minus 0.04. So very close to zero. So that's the kind of intuition we can form about the actual value of the correlation coefficient and what degree of scatter or what degree of noise that represents. So up to around 0.5, it's pretty good. Now remember, we said that the relation is not necessarily linear. In biological data, we often have sigmoidal functions, or we have exponential functions, or distributions that are normal in one degree. But since the values can never be negative, they're censored. And then there's a skewness in the tail. So what happens if that's not linear? So let's have a look at some correlation coefficients of functions that are not linear. And I'm doing this with data that is from a uniform distribution between minus 1 and 1. And this mixing factor here that I've used above of 80% or 40%, I'm setting that to 0. 0.9. So the actual correlation now is 90% functional and 10% noise. Now in the linear case, it looks like that. So that's a linear case at that level. And our correlation coefficient is 0.98. So what happens if it's a periodic thing? I.e., we're looking at correlations of time series in cell cycles. So this is going through one cell cycle, periodic. Again, 90% functional, 10% noise. Now the correlation coefficient is virtually 0. The coefficient of correlation tells us that this data is not correlated. The scatter plot tells us that's actually not true. It's not linearly correlated. But finding the right function would allow us to make a very good prediction of what the data are. What about polynomial? So here I'm using an x squared function, parabola. Same problem. Very nice correlation with a parabola. Very, very poor coefficient of correlation. Exponential. The values get very high, so the noise kind of gets suppressed. Here we actually get a reasonable coefficient of correlation. Why? Well, if we draw a regression line here, we could kind of fit it in here. And then there would be outlier values that wouldn't contribute as strongly. And so we could only looking at the number kind of believe that there's something there. But of course, the fit, having this exponential data and then fitting it with a linear function is wrong. Circular. Data on a circle here. Again, virtually a relationship of 0. So that's the caveat here. The correlation coefficient picks out things that are linear correlations quite well. But if your underlying model is not a linear correlation, then of course the correlation coefficient may be entirely meaningless. So we have to be careful. This is why EDA is so important. Just using canned procedures and calculating numbers may be very, very misleading. It's really important to look at your data, look at the scatter plots, and try to determine whether your statistical model is actually correct. That said, calculating a linear regression in R is absolutely trivial. All we need is the tilde operator. So note that this is a tilde. This is the character that is at the left top of your keyboard. This is not a hyphen. So look carefully. Linear model of height weight column 2 against height weight column 1. And that gives us the parameters. Intercept minus 2.8 and a slope of 42. Is that any good? So remember, we worked with synthetic data. What did we put into the data? 40. That was the slope. And the intercept? Plus 1. So is it good? I think it's pretty good. You know, looking at that degree of scatter that we do have here, getting the parameters right to very high reliability is actually quite surprising. So we re-plot that. Now let's plot a regression line. So we use this app line again. Only this case, we don't specify vertical or horizontal, but we feed it a regression model. And app line then is able to take the parameters from the regression model and plot the correct regression line. Here it is. This is the regression line. What's this line? What does it represent? Intuitively, we say, OK, it kind of characterizes where the data would be in the abstract case in the ideal case. But mathematically, why do we put the line into this spot? The sum of the distance is 40 plus sum. Almost. The square of the sum. Not the square of the sum. The sum of the squares. So this line minimizes the number that you get when you take the distance of each point from the line and square that and add all of these together. So this is why it's called at least squares fit. So it's a fit that generates a mathematical formula where that distance is minimized. Now, residuals are these difference values. And we can easily calculate the residuals to look at our plots. We can also easily calculate the idealized values with the function fitted. So fitted again takes the linear formula and knows how to extract the values from the linear formula and put that into a vector. So we can then use what we calculated for residuals and for fits to plot the differences. And we do that by plotting these as segments. So these are now unconnected line segments. That's another way to put additional information onto a plot. A segment needs four points. x1, y1, x2, y2. So x1 are the values we find on the x-axis of height, the first column in our data. x2, or y, is the second line. So taking these together, it says our segment starts at one of the data points. That's the coordinates of one of the data points. We find that in these two values. And our segment goes to the same point on the x-axis, but a point on the y-axis that we find in fit, i.e. in the idealized value. So taking together, this will draw a line from the data value down to the corresponding point on the regression line. So this is a plot that shows us the residuals. And the square of the length of these red line is what's being minimized. Then we can plot fit against residuals. This is this. What's that good for? So this is the fitted value. This is the residual value. If we look at the correlation, we see that fit against residual is extremely close to zero. Well, that's not surprising, because the model was built in such a way that that should be minimized and should be as close to zero as numerically possible. And that's successful. So ideally, the correlation between fit and residual should be zero. What would it mean if there is remaining correlation in fit and residual? No, it would mean that we can come up with a linear model that is better than the one we have. But by the definition, the one we have is already the best one. So we could then adjust our linear model to further reduce fit against residual, which is kind of contradictory to the mathematics behind this, which is already optimal. But there's another way in which fit against residual could come out systematically different. And that's if the linear model is no good. If the linear model is not able to remove any kind of a systematic effect in the residuals, it means we really have to change our linear model. So that's one way to see whether the linear model is adequate. We plot fit against residual, and only if the linear model is adequate will we see that this is now a random and uncorrelated data setting. Then the linear model is good. Isn't that the same? Isn't fit plus residual the observed value? So the observed value is residual minus fit. So it should come out the same. We can try that in the coffee break. But I think there's no different information there, because fit plus residual gives the observed value. There's no independent observation. OK, now in order to further characterize whether this is any good and what we can do with the parameters, there are two aspects of our linear model that we need to discuss. And one is prediction limits, and another one is confidence limits. Prediction limits tell us boundaries on future observations. And so they say how well the model is expected to accommodate new data. If we plot prediction boundaries on our plot, we can tell where in our plot new data points that are not in our observed population could possibly fall. Whereas confidence limits give us boundaries on the adequate model. So they characterize how well the model is expected to accommodate the data. So if we go back here, the confidence limits would tell us the possible range of slopes and intersets that we can have on that red line, whereas the prediction intervals would tell us the possible range of where points could fall. Now when both are calculated with predict, one is calculated with predict interval equals p, and one is calculated with interval equals pc. And it gives us the fitted values that we've seen before, and a lower and an upper bound. So for the first observation, which is in our data set, we have the fitted value, the one that would, I don't know which one the first value is. Let's assume it's this one. Oh, actually I can tell it's 60.57. So actually probably this one here, or this one. It should, there's a lower and an upper bound for the model at that spot. The lower bound is that the fitted point could lie around 51, i.e. down here. And the upper bound is it could lie around 69, i.e. here. So these are possible bounds on the fitted intervals. So basically this then tells us where this could lie. Now all we need to do to plot these boundaries is to take the numbers we find in lower and in upper and plot them on our plot here. But that's not as easy as just saying lines of these columns here. Because the values are not ordered. So if we simply do a command and do lines from point to point to point, the lines would jump all over. And that's not what we're looking for. The lines should go from the smallest to the largest. And in order to plot them from the smallest to the largest, we have to use order again, as we've done before. i.e. we need to sort our data frame in ascending or descending. So we should recompute pp and pc in sorted order. And then we can plot them. First, we plot the data. It's now plotted in an ordered way, which we can't tell. It's the same data. But now we start with this, then that, then that, and so on. And we can use the function mat lines to plot the upper and lower bounds. So these are the upper and lower bounds of the fit, i.e. within the confidence interval, which I think by default is 0.05 of probability, i.e. something that is still significant. A line that somehow lies in this field here would explain the data equally well as this line here. So we can say our model lies somewhere in this bound here. The mean of where it could lie is this gray line. But it means that there's a lot of noise, and other lines could explain the data equally well. And in terms of the prediction intervals, we have this here. And this says, again, within our confidence limits, a new data point would fall in between these two lines. We would be surprised if it lies outside. That would kind of violate our assumptions about how well the model performs in this case. However, if we have large numbers of data points, we are almost certain to find some that lie outside some of the time. Warning, predictions on current data refer to future responses. So these are not predictions about the current data. So it's a prediction about something that's not observed in the data. Why is that there as a warning? Honestly, I don't know. Maybe it's just a reminder about how statistics works in principle, but it's kind of odd. I should look it up. OK, now, this is what I would consider a complete regression plot, i.e. You show the data. We can actually see what the data looks like. Here it's not a whole lot of data, but if it's more than that, if it's very much data, we've looked at scatter plots with density colors or with hex spin or whatever, all of which you can use to create a plot about lots of data and the density of the data. That's still informative. You show the regression line. That's the result of what our best belief is, what the correct model here is. You show the confidence intervals for the model, which will be broad or narrow, depending on how well our data actually is explained by the model. And you show the prediction limits, i.e. If we have new data, where would we expect this to lie in? So with this plot, we're not just showing our analysis of the data, but we're documenting the quality of the analysis. The difference between this and simply showing a regression line is the same difference if we have a bar plot without the whiskers on top that show the significance of the lines. So basically, think of this as a two-dimensional or a regression analysis equivalent of such a bar plot. So we have about 10 minutes before we break for a coffee break. How about you take our data and produce one regression line plot for a good regression plot? Yeah, let's just take any of the cell lines, take the controls, compare them to the LPS stimulus, and produce a scatter plot from that, and produce a plot of this type on top of that. So basically, all you need to do is to copy some of this code, throw out the comments because you don't need them. And change the labels of the R objects that we're using here and the column numbers. If you're really lazy or smart, which is in computing the same thing, you would overwrite HW column 1 with a control and overwrite HW2, HW1 with a control, HW2 with an LPS plot, then you don't even need to change the code, right? You don't understand what I just proposed? Or you could easily reproduce it. That's the wonderful thing about the script. But yes, indeed, I'm implying that. And once again, if it doesn't work, if you're stuck, if I've been confusing you all afternoon and you don't know what to do, then show us the red posted. Sorry to interrupt, but I might have missed this. But what is the significance of ordering a vector before entering it into prediction? Remember, we went to, if they're unordered, and then if we use the similar one, we'll start with the first, go to the second, go to the third. That's not going to work. If we order them, it will go correct. Sorry, that was an obvious place from the start. No, it wasn't obvious. Yeah, I'll save it. So you don't really start on the call. No, just very briefly, my solution for this very simply would be, since we have a new project, we need to reload the data. So we load the data from that path. The two dots mean go one level up to the parent directory. We specify the directory in which we want to descend down, which is the R EDA introduction. And from that, we pull the R data for the LPS data. Then we make a matrix with the number of rows equal to the number of rows in LPS data and the number of columns equals two. And we place the two columns that we're interested in. In this case, I'm using macrophage control against macrophage lipopolysaccharide. And we place that into column one and column two of HW. I'm using HW, as I said before, because I'm lazy. I can then just use the code that I've used previously to do everything. Now, plotting the prediction and confidence limits first requires us to order it, i.e. creating this ordering vector, smallest first, next smallest second, next smallest third, and so on. And then we rearrange the data in HW according to this ordering vector. So now everything goes nicely in terms of all the values go from left to right, not all over the place. And since they all go from left to right, we can consistently draw a curve from them. So we compute Pp and Pc in that sorted order, i.e. based on the resorted values here with intervals of confidence and prediction. And then we plot it. You can use a standard scatter plot. I've used one of the density plots here just to integrate something we've done previously using these density colors, which shows me that most values in that data set are very small, and only a small number of values then become larger and larger. And I specify the label of the x-axis, mf.control, the label of the y-axis, mf.lps, the y-limits to lie in the range that my prediction gives me. If I wouldn't be doing that, then the y-axis limits would be bounded here by the highest value of the actual point. But the limits that I want to plot can go beyond the highest point. So if I want to include them on my plot and not have them truncated, or have them truncated at the bottom here, I need to redefine the y-axis limits with some expression like that. So the range between the maximum range of the highest and lowest in my prediction intervals. Plotting character is simply a filled circle. Character expansion is one, i.e. keep the circles relatively small, and that gives me this plot. And then I had to change nothing with these lines. It's exactly the same command for the prediction boundaries and the model boundaries. And you see that in the correlation between induced and non-induced data, the linear correlation is very highly defined, but there's also a lot of noise. And there's quite a number of points that fall significantly away from that line. And these would potentially be points of interest to look for actual regulation in response to the stimulus. So one final question here is maybe, why do we see such structure here? What does that structure mean? Where does that come from? Why does it look gridded? Any idea? Whenever you see structure in data, you'll know you'll have to explain it.  Well, basically what these gaps in the data mean is there's values here, but nothing in between. How can that be? Well, because of the limited numerical precision we had in our data. So many, many of these data points, simply depending on the numerical precision, we are not using the entire range of the x-axis here. That's all that means, nor of the y-axis.


Good afternoon. Welcome to this session on sustainable scaling of Kubernetes workloads with in-place pod resize using predictive AI. I am Vinay, I work for eBay Cloud, where I'm helping build EBB powered Kubernetes networking at global scale for large clusters with a lot of pods. Haran, please introduce yourself. Hi everyone, I'm Haran. I'm currently a PhD student at the UIUC and I'm working on microservices and the cloud resource management with machine learning. Thank you, Haran. So the agenda for today will start by describing the overprovisioning problem and look at the environmental impact and the dollar cost of overprovisioning. We touch upon our roadmap and outline why this is important for eBay and quickly recap the in-place pod resize feature. We'll also look at the cluster auto scaler use case and see how in-place pod resize can help improve it and we then switch gears and Haran will walk us through the current state of AI in the cloud. We'll take a look at the role that reinforcement learning can play in auto scaling and see how AI can help us go from being reactive to being proactive with auto scaling and to conclude, we'll look at how RL training is done and review evaluation results. So accurately estimating what resources your pod needs is a very hard problem. Those of you who containerized your apps in Kubernetes may have come to the resources section of the YAML and wondered, gee, how much CPU do I need? How much memory do I need for my pod or how much storage? And this makes it hard and it's a challenging problem and for various reasons. The Java apps that use CPU, there is, excuse me, for example, Java apps use more CPU at startup time and consume a fraction of the initial CPU during normal runtimes. So if you use fixed limit for a guaranteed QoS class pod, it means choosing between a slow startup or over provisioning. You could get the over provisioning, you could get the provisioning right, but still be subject to load shocks due to external factors such as other pods going down or the load balance or misbehaving. Your code may take slower paths more often due to varying nature of the requests and the services that you depend on may experience outages causing your service to get backed up with requests. And if you do everything right, if you get the provisioning right, you profile your code with real traffic and use VPA recommender, perfectly tune the load balancer and HPA. Well, as of today, K8s does not allow you to mutate your pod resources out of the box. So in order to deal with changes. So there's not a lot that you can control. And over provisioning comes with a cost. First, there is the environmental impact. Steven Mojrat did a case study two years ago when cryptocurrency media was in full swing. He estimated that a single data center consumed the power equivalent of 50,000 homes. That crypto energy hunger is now being replaced by AI energy needs. And AI workloads tend to be compute intensive and network intensive. Data centers also have significant cooling needs. They require a lot of water. And then there is the CO2 emissions impact and there is a noise and electronic waste that comes with it. So why should we care? It's simple because there is no Planet B. And then of course, there is a dollar cost to over provisioning. J.Chapel in our blog estimated that 26.6 billion dollars were wasted cloud spending in 2021. And he found that 40% of IS instances were over allocated, which tallied to 8.7 billion dollars in overspend. In another report, a vendor named Cast AI estimated that 37% of computer resources allocated were never used. And last year, a company named Strongforce did a survey where people responded that 47% of the cloud waste was from over provisioning. They also felt that majority of the respondents also felt that Kubernetes complexity was a contributing factor. The theme is that about half the resources are wasted. And in a nutshell, this is an expensive problem and this affects the bottom line for companies. And in the end, it impacts consumer wallets. But it also means that we have an opportunity here to do better. Sustainability has been an important business goal at eBay. When your business is about finding new purpose for once loved but abandoned items, reuse and sustainability comes naturally. Specifically at the data center level, that $550 shipping seals the deal for me. Specifically at the data center level, the goal is exclusive renewable energy use. HPA with predictive AI to better estimate the replication needs is an ongoing effort at this time. And next year, there are plans to take up VPA to right size pods and containers. And this is where in place pod resize is an important piece of the puzzle as it avoids workload disruption due to vertical scaling and the overhead of scheduling new pods and starting them up. Eventually, we want to get to, sorry, we eventually want to get to back here. We want to get to the multi-dimensional pod auto scaling. Now in place pod resize up until earlier this year you could not edit the resources given to your pod. You had to restart your workload if you wanted to change its resources. And then finally, we merged, finally we merged the pull request in Kubernetes 127 that enables us to use, enables us to resize pod without disruption. I have a blog that you can visit to learn more about this feature and how to use it. I presented a detailed design about this in a talk last year. The field names in the API have changed slightly since but the core design that was presented remains the same. And if you're interested in the gory details, there is a link to the KEP, KEP stands for Kubernetes Enhancement Proposal. That's our design document. One application of in place pod resize with sustainability benefits came from a recent blog post by Peter Mankowski. He noticed that Java apps needed a lot more CPU during startup time than when doing regular work as I mentioned earlier. So if you have a guaranteed QoS class pod with hard CPU limits optimized for the runtime need, it would result in long startup times and the alternative is over provisioning. In this use case, he resizes the CPU limits lower after the app startup phase was complete. And this means a job can start and finish faster and we can power down unneeded nodes sooner and AKA it helps you become energy conscious and cost conscious. So cluster on a scaler mainly does two things for us. Number one, it scales up clusters when pods are pending due to insufficient resources and it scales down clusters by removing underutilized nodes. Consider this scenario. You have Kubernetes cluster and a couple of nodes in there and they are at capacity running pods. A couple of new pods show up. They are pending due to lack of resources and they will remain pending until some pods finish and free up some room. Cluster auto scaler sees this and calls into the cloud provider API to allocate a new node. Then the scheduler can then assign these pending pods to that new node. The issue with this is that cluster auto scaler only considers the pod resource requests. It does not take into account the resource utilization of running pods before allocating new nodes. This also means that your cluster could be very underutilized yet you end up bringing new nodes online.  Let's have a quick look at how cluster auto scaler works today. I'm gonna use, this is using QBDM cloud provider and I'm gonna play a prerecorded demo for this part. I apologize if people in the back find it hard to see but there is an uploaded video for this as well. This is a demo of cluster auto scaling with QBDM as cloud provider. In this demo, we have a simple cluster with two nodes, a master and a node called node one. And we have two pods that are running on node one. Taking a closer look at node one, we see that pod one has requested one CPU and pod two has requested 500 mili CPUs. We also note that this node has two CPUs allocatable and a capacity of two CPUs. These pods are idle though. And if we wanna schedule one more pod that request one more CPU, it's not gonna be able to schedule because there's not enough room in the cluster even though the cluster is underutilized. Let's start the pod. When we create the pod, we see that one more pod is up in the API and it's pending. And it will remain pending until there's more room in the cluster. Let's take a look at the reason. We do a key describe pod on this pod and in the events we see it's failed scheduling and the reason is insufficient CPU. In order to get it for CPU, we will add a new node to the cluster using QBDM cloud provider. With this YAML file, we can deploy the cloud provider and this cloud provider is gonna listen for requests to scale up the cluster on the local host address at port 8086. So now that we have deployed the pod, the cluster, the QBDM cloud provider is up and running. And if we look at the logs, we see that it's listening at the local host address port 8086. Next, we start the cluster auto scaler. We tell the cluster auto scaler how to reach the cloud provider by providing this config file, which just points to the address to the cloud provider that we just started. So now we start the cluster auto scaler. It's going to connect to the API and see that there is a pod pending and it's gonna request a scale up of the cluster with the QBDM cloud provider. There is the request for scale up and the QBDM cloud provider has added a new node. This node is up and running and that will be reflected in the API and the scheduler will see that a new node is up and running shortly. And when it does that, it will look to schedule this pending pod. And there we go. The pending pod is now up and running. This concludes the demo of QBDM as cloud provider for cluster auto scaling. Okay, I'm glad that didn't crash. So what you saw is just a vanilla cluster auto scaler. We allocated new node and then we schedule this pod on that new node node two, even though node one is underutilized and that's what motivated the stock today and sets the basis for the next demo we're gonna show in a little bit. Now let's look at what we can do differently with in place pod resize. What we have now is the ability to quickly resize the pod without disruption. So that means pod disruption budgets are not an issue. We can make a small tweak to the cluster auto scaler logic where we instead of immediately requesting a new node from the cloud provider, we check to see if current pods can be resized down. And if we can do that and we can create some more room, then we end up scheduling the pod without firing up new nodes and we save some money and we become more sustainable. What does this look like in terms of the code change? Well, it's still a very simple tweak. What we do is we arm the cluster auto scaler with what I call as a pod smusher. The difference is that we check that in this case the pods that are underutilized, we shrink them before we scale up the cluster. Now, what does this look like? For this, we'll switch to a live demo because I'm feeling adventurous. Let's see how that goes. So you have the same setup as before. You have two nodes here, the master and node one. We have a few pods running. There is the QBADM cloud provider pod that's standing by to receive requests to scale up the cluster if need be. And there are those are pods, pod1 and pod2, which are underutilized. They're tailing the null device, so they're really doing nothing. If you look at the node, we see that pod1 has requested one CPU and pod2 has requested 500 CPUs in a node that has an allocatable of two CPUs and a capacity of two CPUs. These pods are great candidates to be resized down and that's what we will do in this case. You can see here that pod1 is also requesting, has a request of one CPU and allocation of one CPU by the Qubelet when we do describe the pod and get its container status allocated resources, a new feature that was added in in place pod resize. Next, if you want to schedule one more pod as before, which requests one more CPU, it won't be able to schedule just as before. So let's do that. I'm gonna create this pod in the API and it's showing up in the API. It's spending as before and the reason it's spending is, I'm gonna do a kdescribe pod and in the events, we see that it has failed scheduling and the reason is insufficient CPU.  Instead of running our vanilla cluster auto scaler, we are gonna run this podsMusher cluster auto scaler which is hard-coded to resize down pod1 to 200 milli CPUs. So let's hit it. So now it's gonna connect to the API and there we go. It has detected that a pod is spending but before resizing the cluster, it checked that can smush pod1 to 200 milli CPUs and we'll look at this again. Now we are at 200 milli CPUs for pod1 and we see that one more pod has now been scheduled but this time it's scheduled on node one because we created room by shrinking the pod and we did not need to allocate node two. So thus, we save some money. So that's a demo for the second case with in-place pod resize. Now, does this mean we are done? Well, not quite. There are many ways this could go wrong. Let's look at a couple of ways things can play out in ways you do not expect. You just took away memory from a pod that's known to get room killed during spikes and a spike is about to occur or that idle CPU you repurposed for degrades the shopping experience for your site users later when a bad job starts. We can speculate all day about the different ways in which things could go wrong but the real question to ask is, can we as humans come up with a smarter set of heuristics? Sure we can, but if we had infinite time but then is there a better way than that? Well, it turns out there is. Making recommendations based on a large set of parameters in a reasonable amount of time in a reasonable amount of time is a job best suited for, keep my things up. Making recommendations based on a large set of parameters in a reasonable amount of time is a task best suited for AI. So let's hear from Haran how AI can help. Take it away Haran. Thank you Nave, oops. So let's stack back a little bit looking at the cloud platforms and you may find that general resource management is actually everywhere in cloud platforms. Workload out of scaling is one example and others include job scheduling, VM or container placement, congestion control, et cetera. And such problems have been around for a long time both in theory and in practice but yet remain significantly challenging. Currently most are relying on human engineered heuristics. On the other hand, we have learning based approaches such as reinforcement learning which allows us to use deep neural networks to express the complex dynamics with raw and noisy signals and to express the decision-making policies. Learning based approaches are available because we have abundant data generated in more than a cloud platforms. Examples include monitoring data, system metrics, application performance metrics and those are there due to the improvement of observability tools. So let's look at what people do today for those system management tasks. There are two main categories, human engineering and reinforcement learning based approaches representing the learning based solutions. But actually at a higher level, they share similarities. So here I make a side-by-side comparison. On the left-hand side, we have human driven engineering. People usually start with a simple system model based on for example, Q-theory and they need to manually produce some heuristics or parameters to make it work. And of course we need to test and tune those parameters with extensive profiling based on like until relatively good heuristics are found and due for the average case. And whenever there's any changes to the applications or the cloud platforms, we need to redo these steps again and again. On the other hand, there is opportunity for using learning based solutions such as reinforcement learning where an artificial agent is created to interact with the environment. And the system management task is usually formulated as a Markov decision process, essentially a sequential decision-making process. And the agent starts from a random policy. This policy maps from the states to the actions and is usually parameterized by neural networks. At each time step, it obtains states, make an action and then gets the reward indicating how good the action is given the current context. And this policy will be optimized based on the reward. As you can see, this is also like a loop and it will continue until convergence or there's no improvement when updating the model parameters. There are two main reasons why learning based approaches such as IO is suitable for cloud system management. The first reason is that it provides a systematic framework for automatic retraining to reduce repeated human-driven profiling and tuning. And secondly, it reduces costly optimization or search to constant time which makes it scalable to the large state action space in dynamic cloud environment for heterogeneous applications. As a primer, IO is an approach that falls in between supervised learning and unsupervised learning. It doesn't require any labeled data but needs a reward. An agent interacts with the environment in a step-by-step manner. At each time step T, it's gonna get a state ST, make an action AT and then receive the reward RT. As I mentioned, the reward here serves as the feedback like the loss function which directs the IO policy or model training. And the goal of our agent training is to maximize the expected cumulative reward in any T-step trajectory. So let's look at how we formulate the workload auto-skilling task as an IO problem. In the Kubernetes cluster, application workloads are usually deployed as puzzle containers to continuously meet application as a load and achieve high resource utilization. The IO-based auto-skiller is responsible for auto-skilling like in a vertical dimension such as resizing the container regarding the CPU and memory limit and horizontal skating to adjust the number of containers. With IO, we get rid of human-driven application profiling and parameter tuning in heuristic-based approaches. For example, in threshold-based auto-skilling, the optimal threshold for CPU utilization without violating the application performance as a load actually varies across different applications or platforms. And IO automates policy learning with a systematic and dynamic feedback control loop. To support IO training and inference in Kubernetes, we have a multi-dimensional pod auto-skiller or MPA. The design of MPA actually follows the similar style of VPA that separates skating recommendation from actuation. By doing so, it supports customized plug-and-play multi-dimensional auto-skillers such as IO. First, we have application deployments and metric servers in the Kubernetes cluster. And then MPA recommenders guess the measurements from the metric servers and guess the skating recommendations from either the IO agent from our case or the traditional VPA or HPA controllers. The recommender then sets the skating configurations in the MPA API. And then the updater operator executes the horizontal or vertical skating recommendation configuration updates. Here, we took the open source implementation firm which has been published in OSDI 2020 as the IO-based auto-skiller, which is a proactive approach, meaning that the IO agent decides how to auto-scale to react to the perceived states, such as the current utilization or the current application performance metrics. But we can make it a proactive auto-skilling approach by prepending a predictor following the way that deep scaling is doing which has been published in SOCC last year. Instead of taking the current measurement like the time series data, the predictor forecasts the next-time windows time series data on utilization and loads and then pass it to the IO agent to make the decision. For the IO agent, it's really just trained to make resource reprobation in decisions directly from experience. And it is optimized for the end-to-end objectives. What does that mean? As I mentioned, the reward function in IO acts as the loss function to point the IO agent to the right direction. And the reward function, in our case, is defined as this function. It basically consists of two parts. The first part means to mitigate SLO violations fast. In this case, SMT is the SLO maintenance at time T and is defined as the SLO latency divided by the current latency. A lower value means worse performance degradation and we give it like a penalty by having lower reward. The second part is to avoid over-provisioning. It's defined as the resource usage at time T divided by the assigned resource limit or allocation. A higher value means high resource utilization efficiency and the less over-provisioning. And this aligns with our objectives in an end-to-end manner. And we did evaluations on microservices deployed on Kubernetes. Overall, we found that IO-based auto skater reduces the SLO violation mitigation time by up to nine times, compared with baseline Kubernetes auto skaters. Breaking it up, it reduces the average tail latency by up to 11 times. While reducing the overall requested CPU limit by up to 62%, in the meantime, it reduces the number of drop to request or time out request in the microservice applications by up to eight times. So to summarize our talk today, we saw that cloud computing comes with a significant amount of environmental impact and dollar cost. Thankfully, in-place pod-resize feature helps us drive towards the goal of multi-dimensional pod auto skating. We also look at how reinforcement learning helps us further improve the efficiency, which has played a very promising role in Kubernetes auto skating. It saves us from the laborer's work and it also helps us from being reactive to proactive. Our next steps were to drive the in-place pod-resize feature to JA and realize cost saving and reduce the cotton footprint via holistic auto-skate. And we could use community to help with this. Here is a list of references whose work helped us put together this talk, including several papers I mentioned in the talk. And we would like to hear back from you so that we can learn what we could do better. Please scan this QR code. It will take you to the place where you can leave us feedback. And with that, we will conclude this talk and open this session for Q&A. Thank you. So if you have any questions, please come up to the mic over there and we can answer them to you. Really neat stuff, appreciate it. Could you back up one slide so we can get the references? I have uploaded the slides to scan so you can get it from there too. Hi, this is Abhishek from IPM Research. Hi, Aura. So I have two questions. We covered a little bit about AI workload use case. So the first question is, GPUs currently are expressed in Kubernetes as fixed integer quantities. How would scale down work in that aspect? And the second question is again regarding the AI workload use case. Most of these workloads have gang scheduling semantics. So will this technology help in resizing gangs of pods that are related to a single application? Currently, let me take the first question there. The GPU workloads, yes, they are currently, they're showing up as extended resources. And the spec that we have today for in-place pod resize only covers CPU and memory. It does not cover extended resources today. And that was mainly a decision to keep the scope in check and it's a large project as it is. A future cap is welcome that can scale up and scale down GPUs. I don't know if that could benefit from in-place pod resize kind of, whether it depends on whether you want it in units of one or you want it smaller than that. But I can see that potentially being one of the things that come in extended resources. You want to scale the number of GPUs that you have for your pod up and down. Yeah, that could be there. And regarding the second question, let me think, the scheduling approach that we have today with in-place pod resize doesn't really, it's not even assisting, it just observes. And this has come up, scheduler could potentially come up as something that can assist. What it does today is that it steers new parts that are coming away from nodes that are requesting resize. So that way resize gets a little bit of priority. As far as gang scheduling, whether we're gonna take that into the scope of in-place pod resize, I don't see that happening anytime soon, but if you have a strong proposal of requirements that are kept in mind, that's totally welcome. The community would love to see that and we want to get community review.  You need to change the API, you need to change the scheduler, cobalt, all critical components of Kubernetes. So they will go through thorough reviews. Thanks, Vinay. Thanks for the talk, really interesting. I was curious when you're doing your reinforcement learning, are you training that against a live cluster or do you have some sort of simulation environment that you're doing the training in or what does that look like? Yeah, thanks for the question. So the question is, during the reinforcement learning agent training, are we using the live cluster or simulator? So actually in these experiments, we are using a live cluster, creating the, first deploy the microservice benchmarks on the cluster and then deploy the workload generator to drive the microservices running to serve the request. And then the reinforcement learning agent training is actually happening by interacting with the MPA, like setting scaling recommendations and then receive the feedback. Yes. Cool, thanks. Thank you. Hello, thank you for the session. I'm Feben and I'm from a healthcare domain. So with not auto scaling, will not-selector taints and tolerations will be considered while this auto scaling is performed? Let me take the first part of the question. I think there are two part answers to this. The first one is vertical scaling, in place vertical scaling. That is after scheduling. So taints and tolerations don't really come into play here. With regards to multi-dimensional part auto scaling, the author of that cap is standing right next to me and I'll let him answer that question. So, could you repeat the question? Is it? Okay, so while you do the not auto scaling, yes. Will the not-selector pod affinity rules and taints and tolerations, will that be taken into consideration? Mainly when you scale it down. Scaling down. When you remove a pod, sorry, when you remove a node from the cluster. Yeah, I'm trying to understand. So you're asking when we are scaling down the resource limit? Yeah, so let me tell you this. Two or more than one pod an application has only two pods and you are scaling the node down. Yes. These you only have two nodes with a certain not-selector and the pods can run only on those two nodes. Oh, okay, okay. Yeah. I see, so you're asking when there's a constraint that the pods can only run it on those two nodes but there's no capacity on those two nodes? What do you do with that? No, the capacity is there on the two nodes. Actually one of the nodes is underutilized so you don't have that node to be on the cluster and you decide to scale it down, remove the node from the cluster. Oh, so let me take that question. So the current cluster auto-scaler, the way it works, I think it takes into consideration if a pod can be evicted. If a pod cannot be evicted, we won't get a scale down. But frankly, I think the current cluster auto-scaler where the auto-scaling community is looking to replace it with the carpenter and let's see if that gives us more features. Thank you. Correct, yeah. No, it's a node. But they said the node also, right? You had the node? I think pod disruption budget will stop the scan and then pay before. I wanted to know what all things are taken into consideration. No. No, yeah, it just changes the sizing. Thank you. Thank you for your tremendous work. As someone operating solely on-prem, I've been looking at this proposal for three years and every time it hasn't met the release milestone, it was like Christmas taking back from me. So my question is you are performing reinforced learning on actual good data, but in practice, we are then operating in environments with microservices where the reward function and the metrics of reward function could be heavily impacted not only by the performance of the service itself, but rather by the performance of dependencies like databases or performance of other services that are dependencies for this one, upstream and downstream performances. So what are your ideas on how we can incorporate the filtering of the latency or other reward targets that could smooth out the impact of not the decisions made by the reinforced learner autoscaler itself but around the environment? Is it clear? So let me repeat the question. So you're saying in microservices, there could be always third-party services like databases and how can we isolate the root cause for isolate violations? Yeah, so how can we in the end of the day stop the autoscaler to trigger any kind of excessive upscaling or downscaling which is not actually correlated with the decisions made previously because there is noise in this kind of environment. Yeah, that's a good question. So actually we, like if you check out our paper there, like the number eight here, so actually we treat the microservices like a graph. So we first use tracing tools like a Jagger to first nail down what's the root cause candidate for those SLO violations. So if it's not the microservice A, then we are gonna only focus on the root cause for the SLO violations. And then our agent only focus on that particular microservice component but not on all the microservices at the same time. So by doing that, we are not like excessively scaling up some peripheral, some microservices but only targeting the root cause candidates. I have extra questions but I think I first need to check the paper here. Yeah, I feel free to shoot me questions. Thank you. Okay, we are out of time. So I think we need to stop now. We can certainly hang around outside and then take more questions if that's okay. All right, thank you guys. Thank you very much for coming.


Welcome everybody. Today the ECB released the financial stability review. We now have with us the Vice President, Vito Constancio, and we're going to ask him a few questions about this. Mr. Constancio, thank you very much for joining us. You looked at the sustainability, at the current situation, the economic situation in the Euro area. What are the key risks that you found? Well, from the point of view of financial stability, our general indicators that we use to measure systemic risk have improved yet again. And indeed, since 2012, they show these improvements. There is nevertheless, both in Europe and in other parts of the world, a disconnect between the situation of financial markets and what is going on in the real economy. And indeed, the situation of low growth, low inflation and high unemployment is underlying all the risks that emerge in the financial sector. And actually, that would be my first question. This first key risk that you identified, this abrupt reversal of the global search for yield, why is that such a risk and who exactly is exposed? Well, the risk is, of course, when that happens, it means that the prices of financial assets goes down, which means that the investors face capital losses as a result. And that may create vulnerabilities in their balance sheets. That's the risk. And of course, we look to the institutions that hold most of those securities, and banks have been reducing their balance sheets also in what regards securities, so that they are not the institutions that face the more significant risks from that possible development. It's more other institutions like pension funds, insurance companies and investment funds. And just going back to the banks where you said it's actually more investment funds now who hold these types of securities, but actually as a second key risk, you identified the weak bank profitability. And is that not something after the comprehensive assessment people could think, oh, that's being taken care of, but what are your worries there specifically? Well, of course, the comprehensive assessment was essential to take care of the capital position of the banks or the robustness of their balance sheets. They are now in a much better place and will be in a much better place. But the question is that at the same time, due mostly to the weak economic situation, their profitability is suffering. But it's also the result of the fact that the weak economy increases the non-performing loans. And so they have to face costs with the provisions. And these will tend to improve now that the comprehensive assessment has ended. They don't have to be so protective. And we hope that their situation in terms of profitability will improve next year. And as a third large risk or major risk, you've identified the sustainability of sovereign debt. Now, some countries have reduced their fiscal deficits. Why is this still such a concern for you and sovereign bond yields have also stabilized? So what are your concerns here? Well, as we point out in our financial stability review, that third risk has a much weaker probability than the other two. So it's less important. It's more distant. It's a possibility, of course. And again, it is linked with the situation of weak growth. Because if indeed there is no growth and inflation is also low, it is more difficult to sustain the debt. That is true about sovereigns. It's true also about private indebted families or firms. And that's a concern because the continuation, the possibility of a continuation of weak growth in Europe can indeed raise again concerns about the sustainability of the debt. And we've mentioned sort of the three key risks that the financial stability review has identified. Is there maybe something else that is also a cause for concern and that possibly causes you the one or the other sleepless night? Well, I would quote the underlying economic situation in particular in the Euro area. Because indeed we face now a low nominal growth, which creates all sorts of effects that I already mentioned related to the financial risks that we see. And this situation accompanied by iron employment is really, for me, the major cause of concern. Because we have to find ways to get out of this situation and have some growth again. Otherwise, the economic and social fabric of Europe will suffer and we already see signs of that. So it's vital that all authorities in Europe assume their own responsibilities in what they have to do, what are their competencies, what are the policies they are responsible for in order to have a joint effort that can indeed overcome these environments that is putting problems and concerns to the whole European project. Thank you very much Mr. Konstantin for your time and thank you very much for watching us.