So, in the last class we had taken an example where there was an existing drug with a cure rate say 0.6 and there is a new drug which is proposed and that is tested on a sample of 20 patients and the question to verify or the hypothesis to verify whether the new drug has a better success rate than 0.6. So, the so called null hypothesis H naught is that P which is the unknown probability of success for the new drug is less than 0.6 less than or equal to 0.6 that means, the new drug is not better and the alternate hypothesis is that the new drug is better. So, to overturn the existing status quo which is a null hypothesis you need strong evidence and so, the chance of making that type 1 error which is of rejecting the null hypothesis mistakenly that should be controlled because we do not want to unnecessarily change to a new drug which is not significantly better. So, the test statistic is x the number of successes out of 20 and the suggested test region is that if x is significantly more than 12, 12 is equal to the old drug. So, if it is significantly more than that then we are not likely to make a mistake of rejecting the existing drug very casually. So, we will we will reduce our chance of making an error of judgment in reducing the in rejecting the null hypothesis. So, the suggested rejection regions are of the type x is greater than or 15, 18, 14 like that. So, we can plot the rejection possibilities suppose p is equal to 0.6, 0.7, 0.8 like that. In fact, we can plot it for p less than 0.6 also. So, that turns out to be this sort of curve. So, this curve plots the rejection probability suppose p is equal to some value. So, even if p is a small value like 0.3, 0.4 by chance you could get 18 successes out of 20 that is possible. It is a very small probability, but it is non-zero right. So, you see what I am saying right. So, p is say 0.3 that means it is actually only 30 percent chance that a given patient is going to be cured. If we take 20 patients then actually we expect only about on average 6 patients to be cured, but by chance it could happen that a very large number of patients is cured like 15 or something. It is possible binomial probabilities will give you that. So, there is a small chance that even for p equal to 0.3, we will reject the null hypothesis meaning we will mistakenly say that the new drug is better because we get more than 14 or more than 15 successes. So, that probability is what is being plotted here. So, the probability of rejection of the null hypothesis is for different values of p. So, as p increases these probabilities of rejection of the null hypothesis they increase and asymptotically go to 1. As p is small they all go to 0. So, it is asymptotically going to 0 on one side asymptotically going to 1 on the other side. It is an increasing curve it is a continuous curve. So, it is going to be this a shape thing. So, that you can show actually analytically one can show that this is going to be there. So, what you should see is the nature of this curve as the rejection region changes. So, as x is 14, 15, 18 the rejection criterion grows tighter and tighter. So, the curve moves to the right that means for the same value of p the rejection probability is less because the criterion is stronger and stronger. If I want to reject only if I get 18 or more then I am for the same value of p I am less likely to reject it than for you know x greater than or equal to 15 or. So, for different rejection regions defined by x greater than or equal to 14 x greater than or equal to 15 like that the different curves can be plotted. So, those curves are called the operating characteristic or rather this is the power curve the 1 minus this is the operating characteristic. So, this power curve is the rejection probability which is this a shaped curve which as a rejection region becomes tighter and tighter moves to the right. So, this is what I had told you verbally and which I have added in the notes put up that this is the way the curves behave. So, you should check your intuition and your understanding of this by looking at this qualitative nature of the curve. So, any questions about this part? So, this was the binomial probability hypothesis test regarding success probability of a given experiment. So, where you repeat that experiment and you expect a certain behavior based on that behavior you define a rejection region and based on that one can compute the type 1 error which is the probability of rejecting it rejecting the null hypothesis mistakenly which is always possible, but we want to put a bound on that. So, that is the type 1 error which we want to control in the design of the test. So, any questions about this? Yes? No, no, no. See it is small p only it is the unknown value of p for different values we can imagine what will x be ok we can see for a given value of p say 0.7 what will x can take on different values with different probabilities x is the number of successes out of 20 with p equal to 0.7 the chance of an individual success. So, one can compute the binomial probability distribution for the chance that one will get 15, 16, 17, 18, 19, 20 successes that cumulative probability is plotted for a given value of p ok that tells me how likely am I to reject the null hypothesis for a given value of p. So, for example, for p equal to 0.6 which is the extreme case where the new drug is actually same as the old drug it is no better actually then also I will get some value of alpha which is this probability of rejection and that is the type 1 error that means even though the new drug is no better I am still going to reject the null hypothesis with that probability. So, that it whatever it is it may be 0.05 or whatever the value is in this or 0.1 say that means with a 10 percent chance even though the new drug is as good as the old drug no better I am going to conclude that the new drug is better than the old drug and therefore, reject the null hypothesis that the old drug is actually. So, that chance is what is being plotted here. So, based on a judgment we can then see whether this test is good enough to discriminate the new drug from the old drug. So, if I want to be really sure that I accept the new drug only when it is really better then I make the rejection region tighter in this case may be make it x greater or equal to 18 in which case for a given value of p there is only a small chance that I will make a erroneous conclusion. So, as you can see for the blue curve on the extreme right is for x greater or equal to 18. So, for p equal to 0.6 there is only very small probability that I will make a mistake and accept I mean reject the null hypothesis. So, the in this case my type 1 error will be very very small. So, I am very conservative I am not going to throw out the old drug unless I have really strong evidence. So, that depends on the judgment. So, for example, you know if I am the drug controller of India and I you know it is already established in all sorts of public distribution system and people have confidence in it and so on. I do not want to just based on a few trials just conclude that something is better. So, I say you know I want to I have only a 1 percent chance of making an error of that type. So, I my a significance level alpha of the type 1 error is you know 1 percent then I will appropriately decide the rejection region for the test. So, rejection remain remember is for rejection null hypothesis null hypothesis in this case is that p is equal to p is less than or equal to 0.6. That means, the new drug is no better than the old drug ok. So, please just go write down the binomial expression and verify that you can even plot it actually you can use Sylab or you can you yourself plot the nature of this curve and verify that it is of this type and qualitative nature of this curve you should be able to reconcile with these points ok that all the curves are asymptotically of this type they are all continuous and the curve shifts right for different types of regions and what is the interpretation of the type 1 error and type 2 error in this curve. So, this you should try to see the other thing which we did was testing for means of normal populations with known variances. So, supposing we so, this again 2 minute recap of what we did last time and then I want to continue supposing we have a sample of size n with a known variance. That means, the causes of variation are diagnosed from earlier experiments because they are more to do with environmental factors whereas, the mean is the particular setting in a given run of the process let us say. So, let us say I am testing you know the dimension of a manufactured product through setting of the equipment. The variability in the dimension is because of you know inherent characteristics of the machine and the environmental conditions and other structural features, but the mean the setting is operator dependent day to day it changes it depends on the tool and so on. So, that is what I want to control from day to day variance also I am interested in, but let us say that is more stable over a period of time it depends on the process which I have selected. So, that does not change. So, for known variance, but unknown mean. So, there is a certain context in which these type of hypothesis tests are done. So, you might wonder what is this known variance, but unknown mean, but so this is an example of what I am saying that variance is the characteristic of a process once it is designed and it is in many cases the causes of variability are statistically known and they are not something that we can control or something that is we take it for granted and we quantify it and it is known as sigma square. The mean of the process is something that it is our control and we would like to set it, but because of differences in the either the skill level or the particular setting on that day it could vary. Now, when we observe something we do not know whether it is because the mean has shifted or the mean is different from what we think it is or it is because of the variance. So, supposing somebody tells me that this set of values comes from mean 50 and variance 5 and therefore, there is a certain spread around 50. I do not know if that is the case or it is really mean 52 and variance something because I will see some variability around 50. Some of the values will be more, some will be less, but you know maybe the mean has actually shifted. So, I would like to be going to distinguish these two cases where the set of values that I see is actually because of shift in the mean rather than the inherent variability which is sigma square which I already know. So, the way the question is posed is the null hypothesis is that the mean is mu naught which is some known value which is told from before and h 1 is that the alternate hypothesis is the mean has shifted to some other value mu naught, mu naught is a specified constant. So, what you know if you observe a sample of size n then and if you take the sample mean which is x 1 plus x 2 plus x n divided by n, you would expect that value to be close to mu naught if indeed the null hypothesis is true. If the sample mean is different from mu naught then you would suspect that the process has shifted that is the relation mean is not mu naught, but something else ok. So, you look at the sample mean x bar and you look at mu naught if they are very close to each other you say well it looks like the mean is really mu naught if they are different then you say that look because you know we know that if I take a sample of size n and I let that sample size go to infinity or take a large number of samples if the mean is really mu naught then I expect that x bar to be very close to mu naught right because the laws of sampling that you know laws of large numbers I expect that if I take larger and larger samples although there is a variance mu sigma square if I take a large enough sample the sample mean will be close to the true mean. So, if the true mean is indeed mu naught then the sample mean will be close to the true mean. So, x bar I expected to be mu naught. So, x bar minus mu naught in absolute value will tell me whether it is really the true mean is in fact mu naught or something else. So, x bar will not be exactly equal to mu naught, but it should be close to mu naught. So, the question we are trying to ask is how far away from mu naught should x bar be before we start to get worried. So, if x bar is only slightly away from mu naught we will say ok this is because of sampling I have only taken n samples after all and there is inherent variable variability sigma square. So, x bar cannot be expected to be exactly equal to mu naught it will be close to mu naught. In fact, we can write a distribution right of x bar if the sample is normal then x bar also got normal distribution with a known parameter. So, we know that there is going to be some variability in x bar you see what I am saying right. So, I am drawing samples from normal distribution with mean mu naught let us say it is really mu naught it is it is no different from mu naught and the variance is sigma square. So, if I draw sample I am not going to get value mu naught I am going to get something centered around mu naught, but could be more or less if I take 2 3 4 n samples and I take the mean of that then I expect to get close to mu naught because some will be more than mu naught some will be less than mu naught averaging out it should come close to mu naught right. So, but still it may not be exactly equal to mu naught by chance you know if I take a sample of 10 values may be 6 are slightly more and 4 are slightly less. So, I get on average slightly more it could happen. So, how much is that slightly more before I should start get worried start to get worried that the mean is actually shifted it is not mu naught but it is actually mu naught plus something. So, that is the question is the question clear at least ok. So, what we see is that. So, this is what I said that why is this type of experiment done. So, the critical region for rejection of the null hypothesis. So, we want to at some point we want to conclude that the null hypothesis is not true that means, mu is not equal to mu naught the process should be stopped the operator should be told to take measurements and calibrate the process again reset the machine and so on. Now, you realize that that is going to be an expensive and time consuming and unnecessary cost for quality I mean you sort of stop the stop the line so to speak. So, you do not want to do it very you know just based on some hunch or something. So, you want to do it based on some quantitative criterion. So, the criterion is that if x bar which is the sample mean of a sample of n if that x bar is different from mu naught significantly that means x bar minus mu naught is bigger than c in absolute x bar minus mu naught could be more or less I mean could be greater or less than 0, but significantly away from 0 by at least c then we say that the mean has shifted ok. So, what is that c that is that is what is to be designed. So, that c is very small then we are going to stop the process every now and then and you know we are going to incur that cost so to speak. So, if c is very small then you know I am willing to take a chance that the process is actually shifted when it has not. So, it is like that cry wolf story where you know you raise false alarms and say that the mean has shifted and actually go and see it and nothing has happened it mean is in fact mu naught what you see is inherent variability, but if c is very large then you are very very conservative you do not you do not reject the null hypothesis till the mean has significantly shifted. So, definitely you do not make that error of stopping the process unnecessarily, but you may make the other type of error which is that you know the mean has shifted, but you are not you are not reacting to it ok. So, there is there is a balance which you have to strike the way of doing it traditionally is that you you put a bound on the type 1 error and within that you make the acceptance region I mean the rejection region as tight as possible. So, you say that I want to mistakenly stop the process no more than 5 percent of the time if this if this thing happens repeatedly I will be wrong some of the time. So, I say that I do not want to be wrong more than 5 percent of the time. So, then c is chosen appropriately. So, for that the quantitative basis for that is that x bar is of a known distribution x bar which is a sample mean is has a known distribution. So, we know that x bar is normally distributed in random variable with mean mu and variance sigma square by n. So, the statement that x bar minus mu is greater than in absolute value greater than c with a certain probability equal to alpha which is my type 1 error that can be written in terms of the distribution of the standard normal random variable because we know that x bar has a known distribution. So, x bar has distribution normally distributed with mean mu and variance sigma square by n. So, x bar minus mu has got you know mean 0 and variance sigma square by n. So, x bar minus mu divided by sigma square by n has got variance 1 and mean 0. So, we can write it in terms of the standard normal random variable for which we have tables and all those things. So, x bar is normal with mean mu and sigma mu naught and variance sigma square by n. So, we can construct a standard normal random variable. So, from that we know what is the chance that standard normal random variable will be greater than c. Of course, since we are taking absolute value it could be greater than c or less than c in absolute value. So, we write it as absolute value of z is greater than c. So, we put that alpha or our significance level on both sides of the error. So, this is this turns out to be the criterion. So, c is chosen such that probability that the standard normal random variable is greater than c root n by sigma that probability is equal to alpha by 2. So, in this alpha is known sigma is known n is known the sample size is known c is chosen. So, that this is true. So, that will give me a test which will make a type 1 error no more than alpha percent of the time is that ok. So, in this in this setting we know what is the n our sample size we know what is alpha which is our significance level that is our subjective level of comfort and we know what is sigma from prior experiments we have to select c so that this is true. So, you can compute this and you turns out to be c is equal to sigma z alpha by 2 divided by root n. So, small z alpha by 2 is the point in the distribution of the standard normal random variable. So, that alpha by 2 is the area on the right hand side of the curve. So, the cumulative distribution that standard normal random variable. So, that the area to the right of that is alpha by 2. So, we can write it in terms of the original statistic. So, here is the example that supposing I see signals drawn from what is supposed to be mu is equal to 8 and variance equal to 4 sigma square equal to 4 and by chance I take 5 5 signals because each signal takes me time to measure and takes me effort to measure. So, I observe 5 signals and sample mean turns out to be 9.5 where the true mean is. So, somebody is saying that you know we are in fact nowadays we are talking about you know these radiations from the cell phone towers and things like that. So, somebody claims that in appropriate units it is 8, but you know there are variations. So, the variance is 4 I take 5 signals and take the average and it turns out to be 9.5 then I start to get worried that you know maybe it is not 8, maybe it is actually 9 or more that person is trying to convince me that it is actually 8, but you know there is variability of to the extent of 4 the variance. So, that therefore, you have seen 9.5. So, should I believe that person or not that is the query. So, I have taken 5 samples they could be anything, but the mean is 9.5 and I have to accept or reject the claim that the true mean is 8 with variance 4. So, is the variance enough to explain what I observe? Of course, it is possible with mean 8 and variance 4 by chance I could get 5 readings all above 8 and the mean could be 9.5 of course, it is possible, but you know maybe it is a bit unlikely. In fact, supposing I get x bar equal to 10, 11, 12 it is still possible, but more and more unlikely. So, when is that time when I should really get worried that is the nature of the question. So, here the null hypothesis is mu naught equal to 8, alternate hypothesis is mu naught not equal to 8 and we chose say 5 percent level of significance. We construct that test statistic and here in this case we compute the test statistic and we check that 9.5 mean from 5 samples is ok, I mean it could happen. So, that is what this computation tells me. So, H naught is not rejected. So, that person is ok in claiming that the mean is actually 8, but so if I got the same 9.5 with a 100 readings instead of 5 readings then I would say look this is stretching things too far. I could get 9.5 even though the mean is 8, but it is highly unlikely. So, you can see that as n changes for the same mean reading the test would be rejected here it is accepted is that ok. So, please see this example and similar examples are there in the text book. So, the p value is what I said last time also that this is not very important it is just that it is a language used by practitioners. So, that that significance level you know need not be told upfront. So, it sort of tells you you know supposing I get a very small p value then you know it is very unlikely that the null hypothesis is true. So, this p value is something that it is used in the practitioners world just to say. So, it is it is the sort of highest alpha value which would be accepted. So, the p value of the test is the chance of rejection of the null hypothesis. So, that means if the p value is in this case in is 0.5776 then you know any confidence level below this would be would be ok for the test. That means somebody who is likely to make even a 50 percent error would would use this data to reject the null hypothesis. That is to to lose I mean. So, we would not have that much faith in our test. So, if the p value is large it means the null hypothesis likely to be true if the p value is small it means the null hypothesis likely to be false. So, without having to specify the the confidence level upfront ok. So, now going ahead this was still what I did up to last time. So, if I want to test now a one sided hypothesis that means the the often you know we are interested in questions not whether mu is equal to mu naught or mu not equal to mu naught not not that type of question, but whether mu is say greater than mu naught. So, supposing we are interested in these type of questions. So, so for example, supposing somebody comes and tells you that the the amount of preservatives in this food item is less than or equal to some value you are ok with that versus the the conclusion that it is more than that. Now, if it is something less than that then you are ok. So, you you are not you are not so concerned about exactly pinning it down to a value mu naught you are interested in just looking at the conclusions mu less than or equal to mu naught versus mu greater than mu naught ok. So, again you will do the same thing you will check x bar minus mu naught, but in this case supposing x bar minus mu naught is positive mu naught then get worried. So, reject h naught this is h naught x bar minus mu naught is less than 0 then ok. So, earlier we had you know looked at the absolute value of x bar minus mu naught here we are only interested in x bar minus mu naught in sign. So, x bar minus mu naught greater than 0 that is the one that we want to use to reject the null hypothesis we do not want to use either greater than 0 or less than 0 we want to use only greater than 0. So, it is a one sided test. So, the test has to be appropriately designed so that we only look at that probability. So, in the earlier case we had we had said that x bar should be equal to mu naught. So, x bar can be different from mu naught either on one side or the other side. So, the cumulative probability is that happening had to be controlled here it is only one side the probability has to be controlled ok. So, here the test statistic is x bar minus mu. So, we reject this. So, actually this absolute value should not be here. So, that is the that is the point here. So, this this test this this t if it is greater than or equal to minus z alpha then we reject it otherwise we accept it. So, the the summary of this is given in in Ross's book actually that h naught if it is the equality hypothesis that mu is equal to mu naught then the null hypothesis mu mu naught equal to mu naught then this is the part that we have done where the test statistic is this and where we reject the null hypothesis if the absolute value of the test statistic is greater than z alpha by 2. So, there we we could be greater or less with with some probability. So, that alpha by 2 shows up in in the case of one sided hypothesis that is mu less than or equal to mu naught or mu greater than or equal to mu naught either it is a sort of symmetric argument then the appropriate null hypothesis is mu is less than or equal to mu naught then the alternate hypothesis is mu greater than mu naught then the test statistic is the same but there we look at just the value of the test statistic and it should be greater than z alpha. So, z alpha is the the point in the standard normal where the the area to the right of it is equal to alpha the significance level. So, with that probability we will be in that rejection region and therefore, make a mistake. So, that is what we want to control. So, in in all these computations the the sample is of size n drawn from a sample drawn from a population with mean mu and variance sigma square where sigma square is known and the test statistic is derived from x bar which is the sample mean and because we know that the sample mean has got a distribution with mean mu and variance sigma square by n we are able to we are able to construct the test statistic standardize normal random variable test statistic which is root n by sigma into x bar minus mu. So, x bar minus mu x bar minus mu naught divide by sigma by root n is standard normal. So, that is the test statistic and if if that is some extreme value then we reject the null hypothesis ok. So, just to give an example of this. So, people are claiming that you know average life span is increasing. So, sometime back say in in some some community the average life span was 70, somebody is claiming that it is more than 70 now ok. So, they have taken 100 sample and at the time of death what was the age of the person and computed that it it is 71.8. So, is that enough evidence to conclude that the average life span has actually gone up. So, we know from a long history that there is variability in life span there is a variability and that variability is standard deviation 8.9 ok. So, in a sample of 100 I have observed a mean life of 71.8 is that explained by inherent variability of 8.9 years over 70 life span or does it mean that actually the life span average has gone up from 70 to something more than 70. So, the null hypothesis is that mu is equal to 70 and the alternate hypothesis is mu greater than 70. What we have observed is 100 sample mean sample mean equal to 71.8 sigma from past experience is known to be 8.9. So, do we have enough evidence to overturn the null hypothesis which is that then that the average life span is not increased it is equal to 70 ok. So, we construct the test statistic which is x bar minus mu naught divided by sigma by root n. So, that is in this case 71.8 minus 70 divided by sigma is 8.9 and the sample size is 100. So, root 100. So, that turns out to be whatever it is and so that is 2.02 I think in this case. So, we look at what is the chance that we will get this 2.02 from a standard normal random variable from a standard I mean from mean 0 we see that that is quite unlikely. So, we actually compute that z subscript 0.05 and that is 1.645. So, this is in fact to the right of that. So, we reject the null hypothesis. So, actually the p value for this which is computed here is 0.0217. So, actually we it is it is even at a 0.02 percent level of significance which is that you know chance of making a mistake is only 0.02 this data would have been convincing enough ok. So, that is the type of argument that we do in this hypothesis testing. So, an extension of this which you would have seen in the case of confidence intervals also is the case of unknown variance. So, this is a natural extension where we look at some data which is assumed to be normally distributed and we have a hypothesis about the mean, but in this case we do not know the variance. We do not know the variance up front then what do we do? Well, we use the same data to estimate the variance I mean that is really all that we can do. So, in the earlier case we had assumed that the variance is known from a long history of past data in this case the variance is not known. So, of course we have to assume something about the variance because there is inherent variability that much we know. So, the natural device left to us is to estimate the variance from the given population. So, we just the rejection region when the variance was known is x bar minus mu naught divided by sigma by root n and whether that was greater than or equal to some value derived from the confidence level and significance level. In this case we do not know the variance. So, we estimate it. So, what is the estimate of the variance from a sample? It is the so called sample variance which is this quantity s square this you have seen right. So, if I have a sample of n then the estimate of the variance population variance is this so called sample variance which is this quantity divided by n minus 1. You would have seen this before. So, please just revise your knowledge of this and I mean we have discussed this I mean I remember there has been some discussion on this why this is n minus 1 and not n. I hope you have noted this point and basically it is because of the fact that if I am using the same population to estimate the variance then my estimate of the mean so that I can do this variance calculation is drawn from the same set of values. So, it is it tends to be biased towards that set of values which I have drawn. So, if I divide by n I sort of underestimate the true variance. So, that correction is to made I have to I have to divide by n minus 1. Of course, as n goes large it does not make too much of a difference really speaking, but it turns out that this quantity divided by n minus 1 is the unbiased estimator for the population variance. Is that ok? So, we just use the same logic x bar minus mu naught divided by s instead of sigma square we have this s sample variance divided by root n, but in this case this quantity is not a normally distributed one it has got this so called t distribution. So, everybody is familiar with this t distribution right. So, this t distribution with n minus 1 degrees of freedom. So, this test statistic x bar minus mu naught divided by s by root n for a sample of size n is a known distribution it is not the normal distribution, but something close to the normal distribution it has got the same symmetric shape, but it is slightly slightly different as n grows large it approaches the normal distribution. So, for large n you can even just use normal distribution it is ok, but for small size samples this may be worth worrying about. So, that t distribution is there. So, x bar minus mu naught divided by s by root n is again a known distribution the only thing is that for different values of n this has got a different distribution with n minus 1 degrees of freedom. So, this if you have to have the t tables for different values of n which is called the degrees of freedom. So, that using those tables we know the distribution of this quantity. So, we know the probabilities that this quantity is greater or equal to something. So, using this we can quantify our error in coming to a conclusion. Hypothesis testing we come to a certain conclusion either we reject the null hypothesis or we accept it. Because we are doing it based on this known distribution I mean a quantity with a known distribution we know what is the chance of that happening that value taking on a certain that random variable taking on a certain value we know the probability of that. So, therefore, we know the chance of making a mistake even when the null hypothesis is true. So, that is the type 1 error right. So, the t distribution with n minus 1 degrees of freedom is used to quantify the rejection region and its probabilities. So, here we know that you know from the t distribution tables with given degrees of freedom we can define t alpha n minus 1 such that probability that something is greater than the given I mean t value is I mean standardized t statistic is greater than or equal to t alpha minus 1 that probability is equal to alpha which is the chance of making that mistake. So, for the test with level of significance alpha t should be large enough. So, that this quantities is greater than t alpha by 2 n minus 1 with equal to alpha. So, that t small t is chosen with so that this is true. So, it is the same type of logic that we use earlier this is the t distribution with n minus 1 degrees of freedom this can take on extreme values even when you know this is supposed to be centered at mu and with a certain shape of this probability curve it can take on extreme values it can lie in this region or in this region and since this is symmetric thing this probability is alpha by 2 this probability is alpha by 2. So, even though the null hypothesis is true mu is equal to mu naught we could still get extreme values and therefore, reject the null hypothesis. So, even though mu is equal to mu naught by bad luck we could wind up here or here suppose mu is not equal to mu naught mu is here then this these values are quite likely, but even with mu equal to mu naught we could wind up far away from mu naught that can happen with some probability. So, I put those probabilities on both sides and therefore, then I defend my rejection region. So, the rejection region is in terms of the test statistic root n x bar minus mu naught divided by s or in other words x bar minus mu naught divided by its variance which is s by root n only since this variance is estimated from the sample itself this has got a t distribution. So, if this quantity in the absolute value is large then I reject that means, x naught I mean mu naught equal to mu naught because otherwise I expect x bar to be equal to mu naught. So, it is the same thing as what we did earlier except that here we are using s to estimate sigma square and so, we are making that making use of the t distribution instead of the normal standard normal distribution. So, the rejection region is that this absolute value of this is bigger than t alpha by 2 for the one sided test is the same thing, but now we will put we just look at x bar minus mu naught absolute value for the one sided case where we are testing where mu is greater than mu naught versus mu less than or equal to mu naught. Then the rejection region is x bar minus mu naught divided by s by root n is greater than t alpha with n minus 1 degrees of freedom. So, here is the example an agency has published figures in the annual number of kilowatt hours expended by various home appliances and it is claimed that the average is 46 kilowatt hours. I take a random sample of 12 and you get an average of 42 and you know that has got a standard deviation the sample that I have taken has got a standard deviation of 11.9 then is that enough to conclude that the vacuum cleaners that is the devices expand on average less than 46. So, here is a one sided test where somebody is claiming that these devices are energy efficient and they are consuming less than this thing, but inherently there is some variability. So, is this enough evidence for me to accept the claim. So, the null hypothesis is that mu is 46 or greater than or equal to 46 and the alternate hypothesis is that it is less than 46. So, I have got x bar as 42, but it is from a small sample of size 12 and there is an inherent variability of something which I do not know, but from the same sample I can estimate that it is variable. So, I estimate sigma square from the sample. You got the question right. So, the default claim is that 46 is the energy consumption. I take a sample of 12 and I find an average of 42 and in that sample itself there is a variability. Some are more than 46, some are less than 46, average is 42 and from my sample itself I know that there is inherent variability and the standard deviation of that is 11.9. So, given that there is so much variability and I have taken only small sample of 12, is it enough to conclude that it is indeed the mean consumption is indeed less than 46. That is the question. So, I construct the statistics for statistic 42 minus 46 divided by 11.9 which is a standard deviation of the sample divided by root 12 which is the size of the sample that turns out to be minus 1.16. The table T value for 0.5, 0.05 that is 5 percent level of significance for 11 degrees of freedom is minus 1.796. So, in this case the null hypothesis is not rejected because the inherent variability is so much that I could have got 42 just by chance. So, it is not enough evidence that it is significantly less than 46. So, in this case I would not reject the null hypothesis because if I want to make only 5 percent error that I am going to accept the null I mean reject the null hypothesis then this is not strong enough evidence. So, the thing to do in this is you know just try the examples in the book and try one or two more examples and convince yourself that what is really happening here and for the same example it would be a good thing if you can play around with the parameters of interest. That means, if the same data was given to you, but not from 12 readings, but from 100 readings then what conclusion you come. If the same data was presented to you, but your significance level was not 5 percent, but 1 percent or 10 percent then what conclusion you would come to. If the same data was given, but the sigma was different then what conclusion you come to if sigma is more or less. If you try these things and you are comfortable with it then the job is done. So, in this example for a if what would happen if the significance level was not 5 percent, but 1 percent or 20 percent or something. If the significance level is increased that supposing I make it 50 percent significance level is 50 percent which is not really meaningful, but just for argument sake that means I am willing to make a 50 percent chance that I am in error then this looks like good enough evidence I am below 42 I am below 46 anyway. So, I would accept it, but if I want to be conservative and not throughout the null hypothesis unnecessarily then in this case because the sigma why have I rejected it why have I continued to accept the null hypothesis even though the mean is less than 46 it is because the variance is so high that it well could be a 46, but with variance that even with the size of sample even with the sample of size 12 I got something less than 46 it is to this extent. So, if the variance was not so high if the variance was not so high this would be fairly convincing evidence I have taken 12 samples I have got 42 instead of 46 and the variance is not that high then I would accept it I mean I would accept the claim that the mean is less than 46, but in this case the variance is so high 11.9 that this could well be by chance the mean is not really shifted that at least I am not able to conclude the mean as shifted is that ok anyway. So, the other part which is there in the book is section 8.4 which is testing the equality of means of two normal populations you people have done this in confidence intervals right that you have two populations and you want to test the means whether they are equal or not or you know the difference of means write a confidence interval for that anyway this is section 8.4 is actually quite straight forward it is this sort of situation I have two populations say x 1 to x n and y 1 to y m and I am again you know we have to distinguish the cases of known variance and unknown variance the case of known variance you know I am doing testing in different conditions where the variance is supposedly known only the means are different. So, again I am testing two different sets of data and the question is whether they are resulting essentially the same mean or different mean. So, it is like saying one process is better than the other, but for both of them I am gathering data ok. So, supposing somebody says that the performance of this equipment is better than the performance of that equipment. So, under standardized test condition where the variability is the same and known ok. So, I gather data regarding the first set of equipment performance and I gather data regarding the second set of equipment that performance. So, the x 1 to x n is the data for the first set and y 1 to y m m could be different from n because you know the conditions could be different. So, I am able to gather say 10 data points for the first set and 8 data points from the second set because each experiment costs time and money ok. So, I will x 1 to x n has got inherent variability they are slightly different and they have got some mean and they have got the known variance ok y 1 to y m again has got some variability it has got some mean and has got some variance. So, I can compute x bar and I can compute y bar ok. So, if mu x which is the real mean of the x process is the same as the real mean of the y process then you know I expect x bar and y bar to be the same ok. So, I can construct the test statistic is the question clear to you I mean the answer is ok, but is the question clear to you that the unknown mean of the first process is mu x and the unknown mean of the second process is mu y. So, the null hypothesis is that these two are the same ok. So, what do I do I estimate mu x by this is what I expect and y bar is mu y. So, if these two are equal I expect x bar minus y bar to be something close to 0 ok. So, you can you can show that I mean x bar minus y bar is actually normally distributed with mu x minus mu mu y and sort of weighted of the two populations if this is equal to 0 then x bar minus y bar should be 0 mean and with some variance. So, I compute x bar and y bar from actual data and see whether it is close to 0 if it is different from 0 then mu x is not likely to be equal to mu y. So, I construct a test statistic. So, the test statistic this is the test statistic x bar minus y bar which is expected to be with mean 0 and with some variance that variance some getting is a combined thing from two samples it is sort of weighted some of that. So, this quantity in absolute value if it is large then I get worried. So, in absolute value if this is bigger than z alpha by 2 then I conclude that mu x is not equal to mu y ok. So, it is similar to what we did earlier except that here you know you are looking at the difference of these two means which you expect to be 0. So, x bar minus y bar if it is equal to 0 then you know you have sort of you are fairly confident at mu x equal to mu y because x bar is an estimate of mu x y bar is an estimate of mu y. So, if x if mu x is not equal to mu y then you expect x bar to be different from y bar. So, x bar and y bar if they are significantly different then you will concluded mu x is not equal to mu y, but in the actual data that you get x bar and y bar could be slightly different and could be explained by the inherent variance. So, that variance is sigma x square and sigma y square and drawn from samples of size n and size m. So, turns out that this is the expression for scaling that x bar minus y bar quantity to explain the inherent variance. So, divide by the inherent variance that quantity is a standardized normal random variable. So, it turns out that this quantity written here is a standard normal 0 1 random variable. So, this will have some inherent variability if this is large then the thing that I have seen is not explained by the inherent variability it is in fact, because of mu x not equal to mu y. If this is small then you know what I have seen is quite likely from the inherent variability. So, mu x is in fact equal to mu y, but there are small differences in x bar and y bar. So, do not enough for me to get worried about it is that ok. So, if mu x is equal to mu y then x bar will be very close to y bar that is fine. If mu x equal to mu y even then x bar could be slightly different from y bar to some extent as given by this standard normal variable with this with zero mean and this variability. So, this quantifies how much of variability I can expect by chance not because mu x is not equal to mu y, but by chance. If mu x is not equal to mu mu y then this quantity is going to be large anyway because x bar itself is going to be different from y bar. So, x bar is an estimate of mu x y bar is an estimate of y mu y. So, then this quantity is going to be large even if mu x equal to mu y there is going to be some variability. So, this tells you how much is that variability that I can expect by chance. This guides me how to come to a conclusion to separate the statistical part of the thing from the shift of the mean. So, this also examples are there in the book very standard type of examples and again the same thing if these means are known and the means are not I mean it is a variance are known and variance are unknown. So, this is in section 8.4 of the book please see that as well. So, about hypothesis testing this is all that I wanted to do. So, the next 2 classes will be on regression models which is sections 9.1 to 9.3 and 9.5 in process book. So, the next 2 classes will be on regression models. Thanks.


The world has long been looking to the enormous potential of Africa as it emerges from a deeper colonial past and from various social difficulties that have occurred since. Africa has incredible potential in terms of food production and it's essential that it gets that right as economies continue to develop. One of the things that contributes greatly to that is having a good science base both within institutes like ILRI and within the national governments themselves. Over the years ILRI has trained more than 8,000 young scientists, many from Africa, many others from other developing countries. This has made a substantial contribution to building a science base within the developing world and ILRI is justifiably proud of that. That continues. There are now many, currently many young graduates and post-graduates training at ILRI and they will cede out to various national institutions. In addition, there's the BECCA initiative which seeks to promote bioscience development in East and Central Africa, which is again enormously important and can use ILRI as an intellectual, if not a technical home. This type of activity delivers enormous benefit for the dollars spent, some of which can be measured very directly, some of which cannot be that directly measured, but there is no doubt that as Africa continues to develop and becomes a food powerhouse, this will be a major factor. At the moment, Africa is still importing a large amount of food. That is costing an excess of $40 billion a year. That can and should change and there is no reason why that cannot happen with well-organized national strategies based in evidence.


Welcome to the 2016 Luka Soil NCAA Superstakes big final tonight the open classic and the winner was Morgan Cromer on made of metal with the 223 congratulations Morgan. Amazing. This is your first major win in the Will Rogers you seem very emotional when it's set in. Yeah it is it's it's unbelievable I mean I you know I've been coming here since I was a kid you know and and so to be able to win one is pretty amazing. So you made you had three make it to the final how did that feel coming into it what was your preparation. I just made me feel confident you know made me feel confident that I was doing the right thing you know I was working in the right way and and and that I had him on target and they just you know they felt great last week when I showed him at home or two weeks ago and they all made the finals out there so I knew you know what I mean I just knew they were on target and I felt confident coming down here and that's that's a big part of it. Did you have a feeling it was going to be made of metal that was going to be your top horse. Well she Andrew Coates has been helping me all week out there and he's staying out there and he said he said I think this is made of metals week so he called it.  Well you know I just kept telling myself that I was happy with whatever happened I mean it you know it's it's one of those things where you can never be disappointed with being second but you know I yeah I always just try to not be overly happy until it's all the way done but you know I mean we all know that's such a great mare and you know and he's phenomenal so it's like you you know you feel if you're second to somebody like that it's it's it's really great too especially here you know. Yeah it must have been the longest two heads of your life. It was and luckily I had another one to show in this her so I kind of had to focus but then after that I was a mess yeah they were making videos over there of me you know like trying to tear the fence down. So take us through your run cow by cow. The first cow I actually wanted to cut another cow that walked off over into the herd and that cow that we like just walked out and got started and like I said about that mare she can be a little wild off the start so you know as soon as she started right I was more confident to use her you know and and then the second cow was a white cow that we really really liked all along and it just happened to be there you know it's one of those things that usually when it's gonna work those cows kind of walk to the top and and that white cow was really good and it tested her but in a good way you know made her get smart and it made her it made her get you know really really hit the ground and then that third cow was just an option cow we knew it but we it was just a black cow that we needed to finish on and and she felt a little tired there at the end but she never quit trying you know I kept kicking her a little harder because she was she was kind of getting tired but she really really tried so she's now just with this win made over a hundred thousand I think I think so too yeah that's awesome and you've had this horse pretty much the whole way through all the way through I traded for her as a yearling so does that make it more special yeah yeah it does especially when I've been telling you know I've been telling the lady that owns her Judy I've been like this is she's good I just need you know and like I said she's had a few little issues here and there and so we haven't been consistent on her and now we've been consistent and she feels great okay now for your success you I think made over 260,000 last year just in this three months you've now hit over 160,000 is this gonna be the year of Morgan I hope so but now it feels great and and you know it just I think it's one of the things and everybody knows when you get on you know when you get on a roll and you're confident and you're working your horse is good it just you know it just keep rolling with it because it'll change but you just got to go with it when you got it you know I do know there's gonna be some Californians partying tonight because we put up a Facebook post when you made the three horses to the final and the response was absolutely overwhelming you've got a real support they're out there yeah for sure they're great and and I wouldn't change that for the world I love it out there and it's my home and you know I mean the Pacific Coast is is so awesome they're all support each other and and and that's what I love I mean I've got more text messages and everything from you know from out there it's been great well congratulations on a job well done thank you very much thank you


All right, thanks everybody for coming my name is Russ Lindsay. I'm director of solutions architecture for stack velocity and we're going to do a quick little conversation today about getting your hardware ready for open stack and Yeah, Sean O'Connor solutions architect for Mirantis so a couple quick things What we want to do is we want to you know go over the hardware planning hardware management and tools associated with getting ready for open stack as well as monitoring once you're deployed so kind of a High-level approach to to the discussion. We're going to cover things Not in enough detail to really to really cover every any one of them as well as we probably should but you know what we're not going to be doing is we're not going to be getting into a deep dive about bare metal provisioning or You know any any recommendations of particular tool chains or a recommendation of you know OEM versus ODM hardware We're we're trying to do this strictly as informative as we can without Injecting any personal opinions or that kind of stuff So a couple questions that you want to talk think about when you're planning your workloads is Are planning your deployments is you want to think about what your workloads going to be biggest thing you can that is a factor in what your hardware setup looks like is is what you're going to do with it So you want to really make sure that you've spent adequate time planning What resources are going to be necessary for for your hardware to to achieve your goals so? Also budget is a big factor in that for most people we've got You know you can spend as much or as little as you want on any particular hardware deployment especially OpenStack, so just want to be cognizant of that and then Deficient or if you if you care about efficiency Some people are you know green initiatives or you've got limitations of power based on your data center location So those are some things to think about also We'll also cover open source tools and then some of the pros and cons between open source and or I'm sorry I'm cloud optimized or OEM style hardware versus ODM OEM hardware excuse So big thing about processors Again, we're gonna cover these at a real high level, but selecting processors Biggest thing is is think about what you're gonna get for your budget You've got you've got a wide variety of processors to choose from a lot of number of cores available out there Anywhere, you know the high-end core usage from AMD Is an option depending on if you've if you don't care about power consumption. It's a good way to go but most people of these days in the open-stack world are settling into Intel processors today and The the big thing is is to think about Your clock speed versus your core count So if you are if you're really the biggest thing with open-stack to me is how many cores you've got to work with So typical recommendation from my standpoint is to choose maybe the lowest clock speed of a particular processor core range So, you know, whatever pick pick a core processor and choose the lowest clock speed associated with that Memory we're gonna just quickly touch on the fact that you really You want to look at optimizing your memory memory is a key factor in the performance of open-stack Maybe even more so than a lot of other other applications. So you want to you know, take a look at your at your At your memory count for what your VMs are gonna need or so if you basically if you calculate out your cores in your And your memory for each instance Figure out what your total amount of usage is for your machines You want to make sure that you're running that at about 60% of your total memory capacity You want to have that extra capacity to help your system operate? Efficiently because the last thing you want to do is starve your operating system If you're if you end up needing to use more memory for operating systems In an underlying infrastructure than you have for your for your core or for your For your instances essentially what you're gonna end up doing is Starving one of the other and having to start caching things to disk and it's gonna hurt your performance a lot The thing is just go ahead. I Personally yes, probably that's that's that's a common practice for some people to do it's It's dangerous because if you end up in a situation where you completely disable it you know That it can be unpredictable if you end up if you if you didn't plan accordingly and you're in your over utilizing And Don't plan for over subscription. I know a lot of people recommend it Smaller deployments of OpenStack. It's fine. If you're you're trying it out. You want to see what you can do with it But you know if you're really looking at it to use it for a production environment Over-subscription can really can really cripple you if you just it's too unpredictable Network card selection. I'm not gonna really spend much time on this But just make sure that when you're looking at network cards that they are that their drivers are compatible with OpenStack There's hardware there, you know things have improved a lot with OpenStack over the last few years There's a lot of vendors are making drivers now But you know, you'll still run into things occasionally where it's not supported. So Biggest thing is just make sure you check that out hard drive controllers is It's a it's kind of Pandora's box of questions, right? Do you do you use RAID? Do you use HPA? Do you?  Caching there's there's a ton of things that can be covered here but essentially really what I guess from my perspective the key takeaway from this is The only time I would ever use RAID in OpenStack I'm gonna make that blanket statement is Is when you are mirroring your boot drives other than that I would use HPAs across the board for all my storage whether it's you know in in the same server or it's attached through JBod But you know direct-attach one-to-one ratios is by far the best performance ratio We'll take a look at hard drives for a second basically the key takeaway for hard drives is You're you're looking for in most cases with OpenStack. You're looking for volume of storage. So As a mass, you know for the mass storage portion of your deployments I would I would stay with SATA, you know stick with stick with SATA sass has got a little bit better performance But it's really not as as important to you as as a as a storage capacity that you get Kind of the bank for your buck out of your out of your SATA drives You know for for performance increasing you're better off to use more SSDs for caching purposes You can do your journaling on SSDs and then you know use that to write out to your storage media Just as a kind of a rule of thumb Generally speaking the rec recommendations we make is somewhere between One SSD for journaling to every four to seven drives of spinning disk You'll end up with a little bit better performance if you stay on the lower end of that But the sweet spot on price is generally you know around six to one The last thing I've got on there is consumer grade versus enterprise grade There's there's a lot of discussion around this going on in the community on and off Everybody thinks that you can go down to Best Buy and buy a bunch of drives and stick them in your machine And they'll work fine and you know and you know you can save a bunch of money that way Well, it's true. You can but there's a there's a big bud associated with that and that's that's reliability So if you're you know, you've got a couple of machines in your back room that you're trying stuff out on You know it's something that you can try and get away with probably if you're deploying stuff at scale in a data center even Even at the rack level the last thing you want to do is is do that and go have to be out there replacing drives on a constant basis One exception to that maybe In the instance if you are doing a lot of a lot of reads and not necessarily a lot of writes I have seen some studies that show decent performance out of switching to some Consumer grade SSDs that you can get you can get decent life out of them that way as long as you're not writing a lot And the performance is good. So as a cost-benefit ratio. It's it is an option Now we're kind of gonna get into the nitty-gritty of what we really wanted to talk about today And that's what does it really look like from a process standpoint to get your hardware deployed? There's some of these things are Everybody takes them for granted rack and stack and cabling is a prime example You've got you know Oh, you throw is throw your stuff in the rack and plug some cables in and you know You end up with a pile of spaghetti and sure it works as long as you don't ever have to maintain it So big thing that you really want to think about is Trying to make sure that when you're deploying stuff whether you do it yourself whether you do it through a service There's a lot of companies out there who will do rack rack integration services for you You want to make sure that when you're setting up your racks that everything is clearly labeled easy to follow and neatly organized Especially labeling your cables, you know I can tell you from personal experience when you go into look at somebody's rack And there's no labels on any of the cables and you're trying to track one end of one black cable and a bundle this big to Where it goes on the switch on the other end. It's not a fun thing to try to do So, you know, it's it's something you just want to pay attention and plan It's it's not a hard thing to sit down and write it out make a spreadsheet Plan where stuff's going and put a label on each end of the cable Even if it's just one on this end and one on that end and two on this end and two on that end It makes your life a lot easier when it comes down to trying to maintain it I'm not really gonna talk a lot about airflow, but this really comes into play. I think more so if you're looking at Trying to adopt a white box type technology versus an OEM technology OEMs have had a lot of time to plan, test, build machines to, you know, to suit a wide variety of use cases in there And they've spent a lot of time optimizing and making sure that everything works properly When you go the other end of the spectrum you kind of You kind of get all ends of the spectrum, right? You've got you've got people who've spent a lot of time planning and have done a really good job about managing things like airflow You've got people who haven't touched it at all And you know you open the box and you see the bare motherboard and there's no air ducts or to know nothing So essentially what I'm saying is just take a look at that You know open it up at least make sure that you know in your first your first golden sample that you get from somebody that you've got An air duct something looks like it's managing your airflow to help improve your performance in your cooling process Otherwise you can you leave a lot of money on the table, especially at the data center operation level And then really from a from a deployment standpoint firmware and BIOS Management is a really big thing and it's a thing that a lot of people get hung up on when when you Set up OpenStack. You're setting it up on your laptop and a bunch of VMs It's much different than when you're trying to do something at scale or you know even with just a few machines At you know at the rack level you've got to make sure that everything is configured the same every time You've got to make sure that you've Managed all of your drivers correctly all of your All of your firmware correctly all of your bios settings correctly everything always has to match if not you can end up with a lot of Unpredictable performance out of your machines and potentially chasing failures around in a circle trying to figure out what's different on this box versus that box so There there are some Well now there really aren't there's not a lot of good stuff out there for for this process There are some there's some tools that are available to to do provisioning and that sort of thing But really what it comes down to is when you're when you're originally laying out your layouts your designs You want to make sure that you use you stabilize on versions So if you pick version XYZ of your raid controllers firmware You need to make sure that every one of them has the same firmware on it and it's not always cut and dry you could order you could order raid controllers or knit cards from a vendor and You could get a shipment of 2,500 of them in and it's all in the same huge crate and 10 of them are different than the knees and 20 of them are different than that And so what you end up with is really a kind of a conundrum of trying to figure out how to make sure at all That's managed In my experience best way to do that is as you're going through your process of provisioning and burn in Just flash them you you can do one of two things you can do you can do a quick check to make sure that the version is right Or you can just do a blanket right It doesn't really matter as long as as long as you make sure that you are Verifying that those are the same on every machine And you know keep even if it's just a spreadsheet keep track of it Make sure that it's always the same and if you're buying machines from a particular vendor or through a particular systems integrator Make sure that they have that list that they're following it You you know every time you need to be checking when stuff shows up at your location You're verifying, you know at least spot checking that whoever integrated your machines for you Adjusted those firmware versions and things so that they're right Bios is another big one for that, you know There's a lot of people out there are still going through and manually changing bios on machines to try to make sure that Everything matches up. It's not the right way to go You'll you'll end up human errors is the the cause of a lot of people's woes so You know the best the best thing you can really do is just Get your golden sample get it set up You know dump a bin file off and flash that to every machine as it comes through when you're doing your process of testing and Burnin if it'll just save you a bunch of headache. You'll know every one of them is always the same and then That kind of brings me to the topic of burnin Which is also an interesting subject that a lot of people struggle with and they're not quite sure What the right answer is and the reason is is because there's as many options out there as there are people in the room There's people do this stuff all different ways all different methodologies My personal choice if you're gonna do it yourself is to utilize open source tools You know much like we're doing with the rest of our community support those guys You know Google released a great tool called Google stress. It's now called stress app test Does a really good job of stressing your your system and it dumps out a decent amount of log files tells you what went wrong You know at least at least gives you an insight into what's happening during that burning process while it's stressing all your equipment and then from From a hard drive perspective. There's there's a number of things out there FiO is my personal favorite probably it allows you to establish workloads of rewrite to the machines Again, it's open source tool you can use to to You know configure each drive to write and read a particular amount for a prescribed period of time There's a little ones out there thrash and some others But you know do do a little bit of research dig around see which one you like best But there are there are a few things out there that are key Biggest thing is the overall arcing test which you know, which the Google stress or stress app test covers most of and then You know key component testing So you want to also do things with your network test to make sure that you've got good connectivity across between your machines so Let's see Lin pack I don't really think of some others Anyway, there's there's a lot of stuff out there and if you guys have questions feel free to reach out to me later And I can get you a list and then The last thing probably most important thing to think about is managing your systems from from remote location it's it's It's one thing if you've got your machine sitting there in front of you and you can walk up your console And you got a keyboard and a mouse hooked up and you're able to sit down and work on some things That's generally not the case for most of us who are working things from a professional level We've got a one rack or ten racks or a hundred racks or a thousand racks in a data center somewhere and those things are They're usually a long way from wherever you're sitting, right? So you've got to you got to make sure ahead of time that you've got ways to deal with those machines from that remote location whether that is You know setting up DHCP server with pixie so that you can You can load images bootstrap things from a remote location Whether it's you know static IPs and you're going to manage it all manually There there's a lot of ways to go about it But the biggest thing is is make sure that you have addressed accessing those machines from a distance You need to be able to to get to the networks first where they're located and then you need to be able to communicate with them And that's where you can use tools like IPMI IPMI tool is a primary tool used in most of the large data centers in the world today it anybody who's using a Non-proprietary system at least it's it's a standard that was established by Intel that can be used for both system monitoring and power management Fan speed control there's a lot of different things you can do with it really it's essentially it's your it's your out of band communication Window into what's happening with your systems? So if you haven't started using IPMI, it's probably a good time to start if you're going to think about moving away from anything That's proprietary. It's a it's a universally accepted protocol that's used on Everything from Dell to HP to white box hardware You can you can use the same interface for everything and it it really helps you Solidify a platform across your entire organization that can be used no matter what brand of hardware you choose to choose to deploy The last thing on this list is really kind of it's it's it's becoming more important and It's as as things like open compute hardware become available There's there's the there's a trend towards optimizing hardware to remove cost of operations And in doing that some things are removed like video cards or you know in base level video off the motherboard stuff like that So what you want to make sure is that? You understand how to how to communicate with those machines at the local server level If you walk up to a machine that doesn't have have any video capabilities at all It's pretty hard to work on it when you're standing in front of it. So think about think about Setting up your toolkit for either serial over land where you can do remote Management of the machines from another machine a laptop what have you or or serial cable connections just so that you've you've got a process in place to to Work on that if that's the kind of hardware you choose to adopt And at that point I'm going to hand it over to you. Yeah Thanks everybody so again, I'm Sean O'Connor I I'm gonna take a look at two tools The rest was saying there's really not a magic bullet out there for managing Those those four things firmware BIOS raid Those there's really not software to find cabling so we don't have to worry about that but So I'm looking at crowbar, and we're gonna look at the ironic Python agent So a little bit about crowbar It started version one by Dell as an open stack installer. It was open sourced in 2011, and it's actually still maintained by Susie version 2 Was completely re-architected broader focus. So the idea was you could deploy heterogeneous operating systems You and and you can install into any hardware with the idea that you would be able to update firmware Configure raid provision and do all this kind of task I Dell stopped active deployment in April of 2014, but it's still community maintained If you guys are familiar with Rob Hirschfeld in his blog He's been for a long time mr. Crowbar at Dell and now he's kind of mr. Crowbar on his own And I would absolutely if you want to dig deeper Suggest his log. I kind of look at it as an outside in solution So it's not really focused on an open stack. It wants to be able to deploy open stack could do Lots of scale applications and it wants to work with a variety of hardware If you look at what it's doing today, it's somewhat of a limited matrix of hardware It's still very Dell centric. It takes advantage of a lot of the Dell tools WS manuscripts and etc So here's a kind of a you know logical architecture diagram It's a lot of the the original tools were built in chef You've got this concept of jigs where you can take other tools and shim the man So you can actually integrate with puppet and it also has the script jig which SSH is in to do a lot of the hardware management And I an operating system deployment so the counterposition of that is ironic python agent So this was developed by rackspace they They did a really great session on it in Atlanta And again, they've done a lot of great blogs to dig deeper into it So it's it's core of their their on-metal project What it does is it replaces the pixie tftp image with a with a python agent with an api so We're seeing some in a second. It also has an ironic neutron plug-in So it's the goal of that is to actually be able to go out and configure the physical switches So a lot of those virtual concepts plugging in port setting VLANs term of printing VLANs etc are covered by that This is more of an inside-out solution. So it's it's born and bred inside of an open-stack integrated project They're focused on a specific ocp platform. So they have a predictable hardware Footprint that they're deploying to And it's it's really somewhat more of a focused solution, but it's very targeted So every year you can look at the Actually easier to read over here than on the screen So you've got the ironic pixie deploy driver on the left and then you can see what the the python agent is doing So we're on the left. You can see you're doing DHCP requests and tftp across and deploying that image Then connecting back over ice scuzzy and doing a DD copy to bring the to configure the image or to configure the bare metal server So you're actually beginning provisioning at the very top So they're taking that first tftp request and actually pushing a boot image agent And from there they have HTTP. They've got an ipixie interface So they've got a lot less primitive tools. They've raised it up a level. They can do more in the agent And then from there you You actually can reduce some of those reboots. So instead of having a tftp image wait for that to boot up Do some push the image and then reboot again You can actually do a lot more of that in one reboot and then hand it off to your end user I don't know about you guys my servers. I've never said that they boot too quickly. So being able to take that out It's definitely a benefit So moving off to compute NONDA networking So I talked to a lot of customers who are coming into open stack and somewhat new to it and There's a lot of confusion about this like there's a lot of networks. So, you know, we've got you got cluster management You've got storage, you know private networking public networking Pixie networks, there's IPMI networks. So, you know, how do we take care of all of that? How do we make sure everything's plumbed correctly and how do we verify that? So we're not going to get real deep in the weeds on network design here But a couple of options you want to look at are you doing layer 2 or layer 3 at the top of rack? Layer 3 Definitely a great network design But you need to consider that you may have a lot of services that are layer 2 adjacent or that are expecting layer 2 adjacency And that you're wanting to spread those across racks. So you need to be able to consider how you're going to Be able to communicate with services between racks So M lag multi-tracy lag, you know as far as getting away from spanning tree at top of rack you know pretty much all of the Switches in some form or fashion have a multi-chassis lag there, so you can bond ports between two upstream switches Equal cost multi-pathing in L3 You know, you've got a few different options whether you're doing L2 or L3 OSPF or or trill depending on your your switches And then VLANs are you going to be doing GRE or you're going to be tagging all of these interfaces? And and what are those VLANs going to look like? How are you going to map those from the virtual networks to the physical at which network? So you can all combine which ones are tagged which ones are not tagged Hint pixies typically untagged And then if you are doing bonding, how are you going to deal with that with with pixie traffic? Are you going to bond it or are you going to separate out the pixie interface or do your switches support? Some type of force-up feature to be able to pick a primary interface to pixie boot off of And then top of rack aggregation, so there's a leaf and spine architecture So instead of having two large core switches actually, let's go ahead and come over to here So here's you know kind of three pretty common network designs on the left. You have a fully Layer two network with M lags You've got your services distributed across all of your racks. You've got VLANs And then you have two large core switches at the top It's great for about a thousand, you know in the thousands of nodes. You definitely will run into scaling issues So you move to the to the right and you've got a full layer three equal cost multi pathing with leaf spine switches So you've got your top of racks and you have all of those distributed out to multiple distributed aggregation switches at the top Again, your services are isolated by racks So you can scale to 10,000 nodes, but again, you've got to consider how those services are and then you can look at VXLan So now you you have the ability to run layer two over layer three. So now you can have your services sitting anywhere And scale Basically you get larger scaling and then you also can get away from your service isolation by rack So once once your nodes are up, how do you monitor your hardware nodes? Nagios is kind of the grand daddy in the open-source space for this Version one was released in 1999. So at this point, it's fairly mature. I Can run agentless or with an agent most of the functionalities in the agent So you've got a plug-in architecture, so it's very extensible So you've got plugins for for database monitoring application monitoring You can configure network equipment with SNMP or and monitor by with SNMP You've got IP my polling and then you can feed all of that data into display tools or graphing tools kind of the Next generation and we're definitely not going to get into the religious debate about Zabix versus Nagios But I think I would say it's fairly mature. It's newer version one was 2004 It handles auto discovery and auto registration of your nodes and it builds in aggregate graphing capability It has a sequel DB backend and it's a little more resource hungry than than Nagios So next is log management. So all these hardware nodes are generating lots and lots of logs How do we aggregate that and how do we act on it? So Splunk again, they kind of invented this space. So you have to mention them Their own marketing they called it Google for your logs So if you're not familiar it grabs all those logs it indexes them you can search through those logs They have an entire app infrastructure. So you can add application aware search and functionality They've released a SAS version Splunk storm But it's closed-source and there is a pretty extreme or a pretty large cost that comes along with that What a lot of our customers are looking at now An open-source alternative to Splunk is the elk stack. So that is a stack of three projects Elasticsearch does indexing in search log stash collects and manages all the logs and then the visualization is done with Kibana Any kind of real production or deployment at scale you want to separate each of those projects into Dedicated hardware because they can be a little resource intensive and That's what I've got on kind of the the software tools to manage. I think we're about out of time as well So any questions? So we looked at foreman. I see most all of the tools foreman included have have raid and a Lot of those functions on the roadmap. I didn't see where they have Like it where they have the tools actually developed at this point And that's why I left them off. But yeah, I mean Our fuel product a lot of them are looking at different things in terms of, you know, raid network verification network configuration It just tends to be something that is looking forward more than it's actually delivering it at least on a limited set of hardware today Yes, I didn't hear what kind of platform Yes So I've seen the same thing where if you're managing Dell systems, you're gonna use the Dell bios tools HP you're gonna use the HP tools And then the bios manufacturers as well between the different versions, yeah So I believe that they package they take the actual packages and then script them So you have to actually build an update or you know a kind of a tool chain for each hardware type And then they orchestrate around it. Yeah, that's something that we we're actually having a conversation about this earlier today There's there still needs to be support from each individual vendor at this point for that Because the problem is that you run into and it's not necessarily a problem But the the proprietary nature of it comes from the fact that you know bios for example You might all start with an AMI bios, but then depending on what you're doing with your hardware You may make customizations In fact, you pretty much always make customizations to that bios to support what you're specifically trying to tune it to do And so that can break the generic tool for for being able to manage it So what what we were discussing is how The the crowbar initiative they're kind of working towards being able to Have those tools all incorporated in one place So they're gonna work with the vendors to to work on each individual piece and be able to include it so that it's available to You seamlessly in the in the process Yeah, I mean my opinion. I think having the tool set integrated with ironic It's someone gets community focus on it and potentially could build like a back-end architecture or a plug-in architecture Like we see with neutron and sender and things like that and have the vendors You know kind of put it the onus on them and actually get them contributing and building those tools into a common platform Not that I've seen Anything else? All right, great. Well, thank you everybody. I appreciate you listening and let us know if you have any questions


There's something wrong, I don't know. Oh, is it? OK. So the last talk of the session is essentially optimal robust secret sharing with maximal corruptions by Alison Bichon, Valerio Pastro, Ramiro and Raya Moran, and Daniel Vich, and Valerio will give the talk. Thanks for the intro. So I'll talk about robust secret sharing against as many corrupt players as possible. So what is a secret sharing scheme? I'm going to talk about threshold secret sharing. A secret sharing scheme is a couple of algorithms.  Another algorithm, the reconstruction algorithm, takes the shares and reconstructs the secret. We want two properties, one of them is T privacy, meaning that any set of T shares reveals no information about the secret. And we also want T plus 1 reconstruction, meaning that as long as you add one share, you get the secret back, meaning T plus 1 shares completely defined the secret. So the textbook example is Shamir's secret sharing. In order to share, you just sample a uniform polynomial of degree T, which evaluates to the secret at the origin. And the shares are nothing but the evaluations of this polynomial at public points. In order to reconstruct, you can do polynomial interpolation or read Solomon decoding. What about robust secret sharing? So just add the extra property that I'm going to describe now. Suppose that T shares are selected by an adversary who can then corrupt them arbitrarily. Even in this scenario, we want the reconstruction to be correct up to some error probability. So what is known about robust secret sharing? So let's say that K is the security parameter. So that error delta is 2 to the minus K. The secret size is m, and n is the number of players. So here, I'm going to plot the share size against the number of corruptions. So it is known that with this honest majority, you can have robust secret sharing. So more than half of corruptions, it's not possible. Since we have a threshold structure, it is not possible to have a share size smaller than secret size. If you think about it, the threshold property tells you that T shares give no information about the secret. But if you add one, that one share will completely determine the secret. So that one share has to be as big as the secret, at least. Positive results, we have that Shamir becomes a robust secret sharing when the number of corruptions is small enough, because you can think about it as a read Solomon encoding, and it supports up to a third of errors. So what happens here? There is another positive result by Robin Benoit. This result is for the case of maximal corruptions. It also expands to fewer corruptions, but I'll just focus on this. And it has overhead k times n in terms of the share size. Another, OK, a negative result is a lower bound that says that in the case of maximal corruptions, you have to have an overhead of at least security parameter. Another positive result by Sevalasfer and others, it's an elegant refinement of Robin Benoit that shrinks the overhead to OK plus n. So here, what happens is this. You can combine a ramp secret sharing by Kramer, Demgore, and others with Shamir, and you can get a robust secret sharing for a constant fraction of corruptions that achieves essentially optimal share size. And the only thing left is this. So what we do is essentially fill the gap, and we have a scheme that achieves all tilde k overhead. OK, so I'm going to focus a little bit on the Robin Benoit scheme just to give you an idea of how it works and then move to our scheme. So the Robin Benoit scheme uses two building blocks. Shamir secret sharing to achieve privacy and message authentication codes to achieve robustness. So OK, here, just a notational thing. If I have a key and you have a message tag pair that authenticates with my key, I'm going to draw a green arrow between me and you. If your message tag pair is invalid with my key, I'm going to draw a red arrow. So yeah, just keep that in mind. So as I said, Robin Benoit, it uses Shamir first, and then this is what happens for authentications. Let's focus on two players, i and j. Let's sample a key zji for player j, and we give player i a tag via this key on this share. It means that player j authenticates player zji's share. So we do this for all j and for all i. OK, so this is how you share. How do you reconstruct? So there's going to be three types of players. Honest players, they're t plus one of them. They don't touch their shares, so they're exactly how the dealer defined them. And then the corrupt players are split into two. There are players who don't touch their Shamir share and those who instead corrupt it. So let's see how the authentication graph is colored. Edges from honest players to honest players are all green, of course. So if I'm an honest guy, I'll have at least t income in good edges. If I'm one of the red players, which I call active, well, all the honest guys are going to accuse me. So I'll have, at most, t minus one income in good edges, namely the ones that come from my other corrupt friends. If I'm the last type of player, which I call passive, well, I'll get at least t income in good edges because my Shamir share is valid. So one way you can reconstruct is, well, let's find a subset of h union p and use the Shamir portion to reconstruct. So if you can identify h union p, you're in good shape. So how do you do that? Think about it as a graph game. You're given a graph with these coloring rules and you need to find h union p. One way to do that is just select those players that have at least t income in good edges. That's it. So the high level idea is Shamir's secret sharing gives you privacy. And then the authentication graphs gives you robustness. And essentially, you're compiling this into a graph game. In order to reconstruct, you solve the graph game and retrieve h union p and then use standard Shamir to reconstruct the secret. We're going to use a similar approach for our scheme. And in this talk, I'll just talk about how to recover h union p from the graph. There is a lot in the paper about how to define the graph. So please read the paper. Good. So how does this graph interact with the overhead? In the Rabin-Banor case, the graph is complete and the max are long. Long max means that the overhead depends linearly on k. And the complete graph means that the overhead depends linearly on n because each vertex stores n edges. In the refinement by Sevalos and others, they managed to shrink the overhead by having short max. The graph is still complete, so you see this end here. So what do we do? We use a sparse graph and also short max. So OK, what's our graph game? So it looks like this, the dealer samples graph. And the adversary adaptively corrupts and decides which of the parties are active and passive. And then he has the ability to change the outgoing edges from the active players arbitrarily. We call this property zero. What about the coloring properties? Honest and passive guys authenticate each other. Honest guys accuse active guys. Active guys accuse honest guys. And notice that the edges that are going from H are hidden to the adversary. So that's another interesting and important property. OK, so let's play the game and retrieve HUNP. So OK, I'm the adversary. I'm corrupting T players, deciding these are active, these are passive. So property zero tells me I can change the active. The outgoing edges from the active players, so I'll do that. Then the coloring properties kick in. So honest and passive authenticate each other. Honest, accuse active. Active, accuse honest. And everything else that you see, black, can be colored arbitrarily. So how do we recover HUNP? Well, it turns out that HUNP, with this setting, is the largest self-consistent set. Self-consistent meaning players in this set only authenticate each other. So essentially, we have a reconstruction algorithm, more or less. But let's see why that's the case. So property one tells me that honest and passive, the set of honest and passive guys is self-consistent. So that's promising. What about other sets that are as large? Well, they must contain some active guys and some honest guys. And the probability that the set is self-consistent is bounded by the probability that the honest component of the set is disconnected from the active component of that set by property two. You can evaluate that probability by using the uniformity of the honest edges. And this number here is a very low quantity that survives a union bound. So HUNP is very likely the largest self-consistent set. Good. So we just solved the problem, essentially. But the algorithm to find the max self-consistent set may be inefficient. So how do you make it efficient? So the way we do it is just by essentially having a very short list of candidate sets. And the way we achieve this short list is by using two algorithms that I call budget and bisection. The budget algorithm is effective if the number of passive players is large. The bisection algorithm is effective in the complement case where the number of passive players is small. So a very high-level idea about how to prune in the case of large P. We define a budget of possible accusations for the active players. But just rejecting everybody who complains too much. Then since A is small, we have that active players are accused by more than a half of their incoming edges. This is because there are more than a half honest players. And conversely, if you are an honest player, you don't get many accusations because there are not too many active players. So well, this is an average, by the way. And you can define a quantity that stays in between these two and use the gap to just kill everybody who's above this threshold. So if you're accused too much, you're out. And this is more or less how the scheme works in the budget algorithm. What about in the other case when there are just a small number of passive players? So this is very interesting, I think. We just forget about bad edges, first of all. And then notice that the cut honest players and corrupt players has just a few cross edges. This is by property 2 and 3. So you could find an algorithm that bisects the graph into two sets of about the same size. And there is an actual algorithm by RECA that has a logarithmic amount of cross edges more than the minimal cut for sets of these size. This means that, well, since we have this property, we know that the number of minimum times log n is bounded by this. And this can be set up to be a constant fraction. This means that, essentially, one of these two sets is very honest. And it turns out that it's honest enough that there is a simple method to expand it to the whole set of honest union p. So essentially, we are almost done. So if you have the five properties that I told you at the beginning, you just run the budget and the bisection algorithm in parallel, get the full list of candidate sets, pick the max self-consistent set, and use Shamir to reconstruct from the Shamir portion. And so this turns out to be, essentially, what we do for the graph part. There are tons of other questions that you may ask, for example, how to define the graph. And the way we do it is by using list decoding, universal hashing, and private max. In a, I would say, list decoding and universal hashing are used to define something that is called robust distributed storage that really helps you for defining the graph in a robust way. What's the size of the shares in this instantiation? If you do it naively, it turns out to be with an overhead of k squared. But this can be optimized to be OK by just parallel repetition with lower security. So yeah, that's, I would say, pretty much it. So thanks. We have time for questions. Is this in the model adaptive, or is it just static? It's adaptive. If you settle for non-efficient reconstruction, is there a way to get rid of the tilde? To get rid of the? Of the polylog factors, depending on k. Not that I am aware of. Yeah. Any other question? No question? You're using a random graph in order to do your analysis. I have a gut feeling that if you use a constant degree expender, it might be a good substitute for the random graph, and it might give you some advantages. Have you considered looking at expender graphs? Yeah, I think so we looked into them, but I think the properties are more or less the same for us. So it doesn't quite give you an advantage. I would say it's, yeah, if you pick a random graph, it's fine, and I think probably an expender works. Most random graphs are expenders, but the expenders might give you something provable, whereas in your case, there is a certain probability that your construction will not work. You had some probability of error if you choose a random graph which has bad properties, and that's the main advantage I would like to have from the expender. Guaranteed expansion properties. But the graph can't be public, so there has to be some randomization. You'll have to choose from a sufficiently large collection of possible expenders the one you want. Then each one of them will have some good properties. I think that's probably a good point, yeah. OK, so let's find the speaker again. So this concludes the session, and now there are some announcements. Yes, wait, before you leave, please. So it is a quick announcement for the RAM session. RAM session is taking place today. This is your wonderful RAM session chair together with me this year. The RAM session will take place in this building. There was announced yesterday a small shift of time because of the IACR board meeting, so please check on the website when exactly everything will take place. What is a RAM session? A RAM session is about flash talks. Everybody can submit its own talk, so we have a submission system open. Please do submit your contribution. Flash talks means it's about two to three minutes. There are some constraints that are advertised on the website, so please check the details there. It can be scientific. It doesn't have to be scientific. Important is that it's entertaining to everybody. As I said, there's a couple of time constraints. Check the website for this, and there's also some technical constraints, so have a look and come tonight. Thank you.