To our session 27 on Quality Control and Improvement with Minitab, I am Professor Indrajit Mukherjee from Sheresh J. Mehta School of Management, Tiety Bombay. So, previous session what we are doing is that we are discussing about the multiple regression and multicollinearity problems like that. So, we will take another examples to again emphasize on the importance of how to tackle multicollinearity like that. So, multicollinearity what we have explained is that relationship between X variables that exist and that can distort the relationship between what is and we are not able to generalize the equations which can be used in reality for prediction like that. So, we need to deal with this and the variation inflation factor that we have to calculate and try to see that it is not more than 5 and if it is more than 5 then we have to adopt some of the means to deal with that and how to deal with that that we have mentioned over here that means, highly correlated variables one of them will we can take and other ones can be eliminated. And we can attempt stepwise regression method which takes care of which can suggest that which are the variables to be taken and we can reconform the multicollinearity exist or not in the final models like that. And we can also see best subset regression where multiple options exist because we do not want to stop only considering what stepwise regression gives because sometimes what happens say alternate model is easy to control in real life that means those variables are easy to control which is suggested. And if it is fine if we have to sacrifice some amount of r square values or something like that that is not a constraint for us in production processes, but variables which are very difficult to control and stepwise regression gives those variables in the final equation then it becomes difficult for us to change that one if we adopt that one. So, in that case it becomes difficult. So, best subset regression gives you some more options to select the variables like that. Then appropriate methods like partially square regression principle component regression all these things can be adopted. So, we will take one more examples where we have variables like yield is the y characteristics and this depends on time, velocity and temperature. These are the variable X that was considered to be significant or potential variables and we want to get a model out of this regression model out of this. All are continuous variable over here. So, in this case and when we did the correlation analysis of this correlation analysis what we observe is that velocity is highly correlated with time variables over here and then we can see that yield is highly correlated with time velocity over here. So, but this p value will indicate whether which of the variable is highly correlated. So, yield is one of the variables. So, this seems to be significant over here velocity is also significant over here, but temperature does not seem to be significantly influencing the yields over here. And what we can see is that temperature with time is not significant, temperature with velocity is also not significant. So, velocity with time is highly correlated this with time velocity and time is highly correlated over here. So, this correlation matrix gives you some preliminary information what is expected means when we when we run the regression analysis what is expected if all these variables. So, what we what we can see is that time and velocity are two variables which are highly correlated. So, one we have to adopt over here. So, how do we select which one to adopt over here? So, maybe when we do a trial and error basis if you have to select the variables the policy may be that because yield is the correlation coefficient is 9454 time and for velocity is 914 maybe this variable we should select like that time may be the only variable and replace the velocity with the time like that. So, yield has to be regressed with time and if temperature is significant then we will include that one. So, anyhow this is the suggested one of the guidelines. So, which is highly correlated with the y variable we can select that one. And we want to see that what happens if we select stepwise regression what happens like that. So, this is the data set C 11 to C 20 this is the data set that we are having and we want to we want to implement this. So, what we have done is that basic statistics. So, over here we have gone to correlation and then we have identified which are the variables over here we want to understand the relationship. So, yield is the first variable time velocity and temperature and in options what we have given Pearson correlation we want to see and then in graphs what we want is that correlation with p values and if I click ok and results what we have mentioned is we can also see pair wise correlation table and if you click ok this is the this is the diagram that I have shown in the PPD slides like that. So, this shows relationship between yield and other variables yield is highly related with time the p value is point near to 0 and velocity is also highly related with yield over here, but temperature does not seem to be significantly influencing over here. So, this is the relationship and time is having a high time is not having a correlation with temperature over here and velocity is having no correlation with temperature, but velocity is having a correlation with time over here. So, that is reflected over here temperature and time. So, this is highly 0.958 is the minus 0.958 is. So, this is so what we can do is that we can go to stepwise regression. So, I will use stepwise regression over here. So, fit regression model and here only thing I have to do is that I have to introduce stepwise regression over here and click ok and then validation we can we can trace validation as usual and also storage we can save the residual standard as residual over here and I click ok and let us try to see which are the variables we will go in the model and which will go out of the model ok. So, what happened is that it has identified time and temperature. So, initially we are thinking that it is not so highly correlated, but this is not so significant 0.05 just above the cross line that is 0.05 that is the cutoff over here. So, it is the analysis over here says that we have to take into consideration time and temperature that is the best model that is coming out of this and the r square adjusted value is 90 point something like that and 10 4 cross validation more or less close. So, this model seems to be adequate if I consider that alpha level of significance is 0.1 in that case we can also consider temperature as significant variable we may retain this one. So, let us try to see what about the normality distributions of this residual over here. So, last residual will be normality we can check and in this case 0.74. So, there is not much problem with the normality assumptions over here and we do not have any problem over here. So, in this case normality is not a constraint. So, in this case we can also see when I use base subset regression which is the best model it is giving. So, I have taken yield over here and we have taken all the three variables and if I click ok then what happens let us try to see and we can we can just paste it copy as a picture over here. So, then we can paste it in excel and try to see what happens ok. So, we can adopt we will try to see what happens if we select excel we are opening on excel sheet ok. So, let us let us do that one and we can paste it over here. So, when we are pasting it over here what what observations we have that we want to see. So, over here this is time variable over here this is temperature and this is the velocity over here with one variable what we are seeing mellow cp is about 4 r square adjusted is 88.5 when I take consider only time as the variables like that ok. But mellow cp is higher than the number of variables plus 1. So, this is not recommended second one is also not recommended with one variable third one what we are seeing time and temperature it is mellow cp is less than number of variables plus 1. So, 3 is 2 is less than 3. So, this seems to be adequate this one of the suggested model over here and we are getting r square value of 92 which is higher than the earlier one and what we are getting is that r square adjusted is also high over here and prediction is also more or less same what we are getting over here. Only thing is that when we adopt this one 10 foot cross validation also we can check ok. 5.9 is quite high. So, this goes away and last one is considering all models we want to deduce this one. So, last one does not we do not see the last one. So, mellow cp suggests that this is the one when time and temperature can be can be considered like that. So, so that is also suggested by stepwise regression what we have seen like that. So, in this case again I am doing this. So, if we select this one fit regression model over here and in this case time and temperature. So, I am removing this one and these are the two variables and stepwise regression I will not use now. So, in this case I have already identified the variables and I want to see all graphs whether it is and this residual can be standardized residual over here. So, if I click ok and validation we have already k foot cross validation we have given and if you click ok what happens is that this is the final equation minus 130.7 and plus time it is positively correlated and second one temperature is also positively correlated like that. So, now temperature is retained over here although the p value is not so significant over here we can also just eliminate and see whether that model performs in any way because we have a variable to enter and exit we have given a alpha value of 0.15. So, that is why this model it has come in the model when we have done stepwise regression, but we can retain this one because this is very close to 0.05 and we can we can see that normal distribution assumptions over here that seems to be satisfactory as all the points. So, this graphically when we see seems to be satisfactory points on the line. So, but we can cross check that one even the residual versus feed plot also does not show any abnormality or patterns like that. So, maybe Bruce Pagan test will also confirm this one and also there is no as such abnormalities in auto correlation what we observe some trend or something is not observed it is on both side of the 0 0 0 point like that. So, in this case what is expected is that it should confirm the normality and other assumptions over here. So, we can just check this one and the last row will be the last column will be the variables residuals that we have to check. So, in this case it is coming out to be 0.7 seems to be satisfactory. So, in this case you have to practically also think that whether to retain the last variables or to remove, but R square adjusted as has improved what was observed. So, R square adjusted as improved if I consider only one variable maybe R square adjusted will be low. So, in case I consider one variable as time as the only variable. So, in this case we can do that. So, we can we can just consider time as the only variables. So, feed regression model instead of temperature I will remove this one. So, just keep a note of this R square adjusted is around 90 this is around 85 R square predicted and 10 fold cross validation earlier was around 85. So, 90 and 85 approximately that is the range what we are getting and when we do this here you see R square adjusted is also lower than the earlier one. So, it depends on the practical sense of whether to adopt the variable or whether to remove the variable which is not significant like that that depends on the processing engineers or somebody who is knowledgeable about that whether to retain that one or eliminate that one and there is no lack of feed as such that is observed over here. So, this is lack of feed. So, linear model seems to be adequate so we can adopt this one. So, you have to think from practical aspects like that. So, multicollinearity problem when we are considering both the variables over here. So, if I consider both the variables and in that case we are not getting any multicollinearity issue over here. So, this time and temperature if we consider both of them and variation inflation factor is around 1.05 so which is quite satisfactory and so whether to include both the variables or not to include both the variables that is the judgment that you have to take, but I will suggest to retain this one because this is improving the R square adjusted and also 10 fold cross validation and R square predicted is also improving so we can retain this one ok. So, but there is no black and white scenarios in regression like that. So, it depends on the process engineers and then see the predictive behaviors and then we try to adopt whichever model is very close. Suggested model over here is conclude we can include time and temperature both the variables, but if you go by significance in that case temperature may be dropped and we stick to time only that is the only variable ok. So, this is one scenario and we can have in regression some other scenarios like that we can place this is the example I want to place over here maybe we will delete this one SCDOL over here. So, this is one example that we are taking. So, RPN type of cutting tool and surface finish this is the variable surface finish is Y and RPM is the continuous variable and there can be a categorical variable that means or type of the cutting tool. So, here the number is categorical it can take two values 302 and 416 like that. So, this is the two types of tool. So, this is categorical variable basically. So, this is no I cannot say 302 is greater than 416 like that. So, this cannot be arranged in ascending order descending order. So, this is like color different types of colors. So, this is categorical variable we have to treat and regression has an option to deal with categorical variable also. So, surface finish is Y over here. So, what we can do is that stat go to stat regression fit regression and fit regression model over here and instead of this variable so over here I can just use continuous variable and categorical predictor also we can include in the regression model over here. So, in this case what we will do is that we will we will incorporate the response as surface finish which is the actual variable and then continuous predictor over his RPN and then categorical predictor over here is will be type of cutting tool over here. So, in this case stepwise regression we can use and we can see which variables we will go in and which will go out. So, same significance level we have used and in models we have included both the variables and included the constant term also validation also we have taken cross validation. So, over here and results what we can do is that we can see all results over here. And here there is a options of Darwin Watson's statistic which has to be compared with tabulated value then we can we can use this one. So, I am not using because I can convert into R I can go to R and model that one and see Bridge Pagan test and also the DW test and corresponding P values. What I will do is that standardize residual I will save over here again and then graphically what we want to see normal plot residual plot and order of the data plots like that. So, if you click ok what happens is that it it suggests selection process and then what we have selected like that by default. And these are the two for 302 types of cutting tool this is the surface finish is related to the RPN this is the equation and for 416 it will give a different equation. So, categorical variables different levels that we have selected each levels we will have a different equation with the continuous variable. So, this is shown over here and the coefficients are also given over here. So, based on which we have developed the regression equation and we see variation inflation factors is approximated. So, that is not an issue. So, these two variables type of cutting tool and also RPN both are significant over here and about R square adjusted value is 97.5 and cross validation is 97.02. So, the very close. So, this model seems to be very accurate and what we are getting is that no lack of it is observed over here. Only thing is that we have to check whether normality assumptions is violated. So, here what you see is maybe there is a problem with normality over here one point is over here some has gone outside like that. So, we have to test that one and heteroscedasticity does not seem to be a problem because it seems to be random and also we can see that this one may not be auto correlation is not so significant. So, these two checks can be done, but what we want to see normality. So, this C33 column will tell me whether normality assumptions is violated or not. So, what I will do is that basic stat and normality test and what we will do is that normality residual we can check and try to see what happens. Normality test and what we observe over here for the residuals is that P value is less than 0.05 that indicates that there is a problem of normality issue over here of the residuals. So, in this case directly the Y characteristics needs some transformation and we have boxbox transformation and also Johnson's transformation both options are there. So, in this case what we will do is that first we will see whether boxbox transformation works for this for the Y variable. So, what we will do control chart boxbox transformation and for the surface finish we have taken this variable over here subgroup size is 1 selected and in options optimal or rounded value of alpha just mention that one and when you just click that option what happens is that it suggests you that rounded value is 0.5 that means lambda is approximately we can take as 0.5 although the actual value is 0.72, but we can take a rounded value which is lying within a confidence interval over here. So, 0.5 we are selecting over here as rounded value. So, I have taken a square root transformation over here. So, 0.5 means Y to the power 0.5 means square root transformation that is suggested. Which square root transformation then what we have to do is the regress variable speed instead of surface finish what we will do is that we will we will mention that let us go for a square root transformation of Y that values and do all the analysis same analysis over here. So, regression analysis over here and here also we see that R square adjusted 97 and so it has regressed square root of Y with the variable and only one with the variable r p n over here. So, this both the variables are significant and the equations is given lack of it is not prominent. So, in this case one one outlier is recorded over here as it is minus 3 point more than plus or minus 2. So, in this case what we have to see is that again there is a after this transformation there is a residual that is generated over here. Let us try to check whether the correction has happened with box box transformation or this is not adequate. So, in this case what we will do is that we will see the residual over here and try to see. So, let me just check this is stat basic stat normality test over here. So, I will go to the last variable that is recorded and then I see what is the value of the p value. So, p value is again less than 0.05. So, again there is a problem and there the problem is not resolved. So, error is not coming out to be normal. So, in this case again it is not white noise. So, in this case what we have to do is that we have to so then what I have done is that I have gone for Johnson's transformation family of transformation. So, Johnson's transformation. So, what I have done is that basic stat sorry this is quality tools and in that case Johnson's transformation is there and in this case I have reported that place it into single column data are arranged where the data is. So, I will say surface finish is the data and store in which column. So, I have mentioned over here as C 31. So, when you when you click that one C 31 over here. So, in this case what happens is that if I if I click ok and options what we have given is that point 1 is the value to select the best fit. So, in this case ok. So, it will give you the transformation that is required. So, this is the transformed equation transformed function over here that is given l n what you see the last last over here and this is transforming the variable. So, initially p value is less than point the original data is less than 0.05 and after transformation p value is coming out to be 0.906 that means it has done a rightful transformation now we have to only confirm and this is saved over here Johnson's transformation is saved over here in C 31 column. Now, what we will do is that with this column we will regress with RPN and type of over here. So, what we will do is that we will go to regression regression analysis fit regression instead of the square root what we will do is that we will use Johnson's transform variable and other things remain same. So, I will click ok and we will get a residual over here and we will get the equation. So, after Johnson's transform variable with the RPN this is given for 302 and 416 types of tools like that and what we observe is that 91 percent r square adjusted 88. So, very close. So, this is quite acceptable and in this case there is no lack of it and both the variables are significant that is observed over here categorical as well as RPM over here. So, in this case residual we can check whether the correction has happened some positive things have happened over here. So, I am going to normality test over here and what I do is that I go to the last residual values and I click ok. And what I observe is that when I have done this transformation certainly this problem has gone. So, that means, the normality problem of the error residual has gone. So, when I am using Johnson's transformation this has happened when I use Johnson's transformation this is giving results like that. So, which one box cocks or Johnson you have to try out and figure out that whichever gives you error as white noise. So, that has to be adopted like that. So, this is one of the example when categorical variable is also considered in the model and we are able to address that one. So, categorical variable and how to address in case there is a non-normal situation in multiple regression how it is to be addressed that we can see. And how to select the variables in case we are in dilemma which variable. So, we have talked about stepwise regression. So, this regression is an important aspects which which which can suggest that which are the variables or potential variables can be considered in experimentation. So, but we have to understand that regression does not say we cannot extrapolate regression equation. So, whatever region that experiment is happening that means this is y and this is the range of x domain within which we have got the information that means data are collected over here and these are the observations over here. So, I can only restrict to this region and predict. So, for a given value of x at a given value of x what is the predicted value of expected value of y. So, expectation of y for a given x can be only calculated within the domain or within the range of x over here. So, x we can say upper bound and x lower bound within this. So, no extrapolation is generally preferred in regression equation and association does not mean that means y is related with x function of x does not mean that it is a real variable that impacts y. So, certain scenarios can be there is a relationship between two variables which are not physically any way connected, but there can be high correlation. So, many examples can be cited like that. So, it does not mean that there is a causality. So, regression does not prove causality. So, design of experiment is the only way to define causalities like that. So, to understand causalities between the variables we have to intentionally induce variation by changing the factors changing the factors of variables that we are interested into and try to understand what is the functional relationship. So, the best appropriate way to develop the functional relationship is by design of experiments. So, there is no other alternative. So, regression with historical data does not give you proper association or causality cannot be proved based on that. So, then what is the option? Then what is the option for quality improvements? Then the options we have identified few variables based on regression. So, there are variables x matrix over here and I have collected also y variable information. So, how do I connect these two and this connection and develop the function? So, I need a function over here that will explain the variability of y over here. So, what is required is that systematic way of variation is required systematic way and then what comes is statistical experimentation which is the improvement phase until and unless I have a right function I cannot improve I cannot reach to the global optimal point like that or setting points like that ok. So, design of experiment is the stepping stone for improvement phase like that. So, basic things that our understanding over here is that there are types of variability types of variation we have already spoken about it and some basic information I am providing over here which we can think of as recap. So, there are common cause variability in the process there are special cause abnormal variability which are easy to detect based on SPC process control chart and and then what we do is that to understand the functional relationship between y and x we have to do it a systematic change in we have to induce variability we have to induce variability by changing the factor by changing the factor x over here. And if we can do that systematically what happens is that we can generate a function and the function can be optimized that function we know this is a real equation between the in the process this is the empirical relationship that exists and this is based on systematic theoretically strong models that we have developed over here. Then we can use optimization technique just to reach to the global optimal point like that. So, we are interested into and for that design of experiment is most preferred like that ok. So, design of experiment what we are seeing over here is basically reducing the common cause variability. So, this can be applied when the process is in statistical control. So, when the process is in in stable process. So, whenever it is stable process we say and it is in statistical control then only we can go for design of experiment because we want to reduce the common cause variability further because more and more I reduce the variability. So, accuracy and precision both improves over here. So, we are interested in being and variance both aspects like that what we have seen in capability analysis also. So, for that what is required reduce further variability requires design of experiment. So, for that we need to. So, that helps statistical design of experiment helps us to bring the mean to target value and also to reduce the variability near to 0. So, we want minimum variability over here. So, various design of experiment techniques are proposed and huge huge amount of resources available for design of experiment. So, you can see. But preliminary book that that you can see is a Montgomery's book which is which which can be a good resource of learning initial steps like that, but there are other books also you can see. So, there are many other books which you can see like that Amitav Amitri is one of the books where design of experiment is detailed ok. And so, this is one of the techniques and also I want to recap that one of the concepts that if you have variability that means causes and this is impacting my why and causes are interlinked also somewhat they are interlinked with each other and they together can impact the why like this. So, this is the scenario like this is one of the variable x 1 this is one of the variable x 2 x 3 sorry this is x 3 and this is x 4 over here. And they are interrelated with either in this what you are seeing and they are not completely independent over here what is observed over here. So, if they are completely independent in that case we can understand that what is the level of x that that will deliver the optimal y over here. So, this is possible, but this is not possible when we are having a complex relationship between multiple x's over here and together they can impact the why that that can be scenario. So, it can be scenarios which is not possible to identify. So, by simple other analysis like design failure mode effect analysis or process failure mode effect analysis that is not possible by that ok. So, complex relationship and developing the function when we are developing the functions over here only design of experiments can help. So, design of experiments can help and before we go into details of design of experiment this process P information is again I am recapping over here. There will be certain control variables which is in the hand of experimenters experimenters. So, experimenter will change this control variable basically. So, when you go to a process what you observe is that you will see the operator is changing some of the variables not all variables some of the key variables which is possible to control. So, these are the variables x 1 to x p which is known as controllable variables. Design of experiment is all about controllable variable most of the time we deal with controllable variables. There will be some inputs and I also talked about covariates which can influence the outcomes over here and that can also be considered in the model when we are doing experimentation we can also consider the covariates as a variable which will vary we cannot control that one, but influence of that has to be considered when we are developing the mathematical models or something like that or we can deal with that so covariates also we can see. And there will be some uncontrollable factors that means it is uneconomical to control and sometimes we do not have enough information about the some of the variables because you cannot build a perfect model like that why is a function of all x p variables x 1 to x p is like that. There will be some amount of error in the model because of this noise variables over here because of this noise variable. So, we do not get a perfect function because all the variables that is impacting the process is some of the variables are unknown to us. So, in that case these are known as noise variables or and the influence of that is very less because we have identified most of this. So, these are the variables and some of them are controllable some of them are covariates over here. So, we have knowledge about that. So, and there will be some noise variable which we do not have any control or we do not have enough information about these variables like that ok. So, we want to we want to get a setting. So, in design of experiment what we are trying to do is that this is control variable. So, it is in our hand to control. So, we want to get a combination of this combination of this that will optimize my y c t q and y will be the average value of y should be close to the target value and the variability of y that is that we will generate is very close to 0 values like that. That means, there is no as such variability we want to develop. So, our objective of design of experiment is twofold objective. One is which are the variables impacting y and then if these are the variables which is significantly impacting y, what can be the combination or setting conditions of this x p variables that is impacting over here which I can control. So, that I get the y exactly to the target value defined by the designer and also the variability near the target value is near to 0 like that. So, that is the overall objective of design of experiment. So, and another objective is to develop the functional relationship so that we can optimize like that. So, whenever I have a function I can optimize. So, I need a function. So, function to need a function I need what are the x p variables which is controllable which is significantly contributing to the variability in y. That means, this is and there will be some error that can be because of this noise variables which are not considered and which is difficult to control and uneconomical to control. This can also be impacted this errors may also be impacted by these covariates which are influencing the but we do not have any control on that. So, I can only control this x p variables I do not have any control on the inputs over here and the uncontrollable variables over here. So, most of the time we try to get the best combination of x 1 to x p that in presence of this noise and in presence of covariates or inputs variability that we are experiencing in the process like that ok. So, this is all about what we will try to understand in design of experiments and how to do the experiments that we will try to understand over here. There are different ways of doing experimentation, different designs are available and we will see some very few of them. So, that that is our objective in experimentation. So, that we will cover in our next next session ok. Thank you for listening.


Hey everybody, it's Hari Swaminathan from optiontiger.com. I wanted to create this video on the SPX intraday trading using highly customized indicators, proprietary indicators rather. So now what you're looking at here is two charts. One is of the ES on the left hand side. We're looking at a five day, five minute and we're looking at Monday's price action, which is yesterday's price action. Market started here and the market ended here. Pretty much started and ended at around the same level, maybe a point or so higher. And then on the right hand side, what you're looking at is the tick SP. So the tick is a measure of the 500 SMP stocks and whether they are ticking up or down on an each stock basis. And then it tells you, okay, so at this time what happened? It probably started here. It's a red bar, the first bar. So it probably started at a tick of 137 to the positive. And then it ended over here, 60. So it's a red bar. And the next bar starts here and it goes down further. And so it even dips into the negative. So this is the tick and it keeps going like that for every five minutes. But this is not so useful because this is a very noisy chart. What is useful is if we can combine the cumulative ticks. So from the beginning, so we're looking at the market internals at this point. So once the market starts, these ticks start calculating. And so what happens after that is if at this point it is, let's say it ended here, this would be plus 60. And here it ended here, this would be minus 71. So at this point, the cumulative between these two is minus 11. And over here, if it ended here, it will be minus five, but it's actually improved from before. So the cumulative tick will be minus 11, minus five. So which would be plus five, which would be about six, which would still be minus, I know, negative six. So that's what you'll see here on the tick. The tick STX is a customized proprietary indicator. And this one, it calculates the cumulative tick. So as these ticks goes on, it either adds or subtracts that particular bars tick from the cumulative total.  So in this case, as you can see, the first five minutes or 10 minutes here, it was pretty choppy here. As you can see, it goes down and comes up and you can also see the market action is also choppy. Now remember, this is a cumulative total. So even though you might see a green bar here, it may be a red bar, it may be a red dot here, because this dot actually takes into consideration all the previous ticks before it. And so as far as the ticks are concerned, we really don't get a good signal until you come somewhere here. Over here, you start getting a persistent signal of the ticks, cumulative ticks weakening. Now what I have here is a custom RSI indicator also. And so this custom RSI indicator is used as a confirmation indicator. And whenever it goes below the 60 level and it's going persistently down, it is confirming that the price trend is moving down. And here, this is a market internal trend and that is also moving down. So when you see both of these in sync, that's when you want to get into the trade. So somewhere here, you would get into a put trade and you would write this all the way up until here perhaps because you don't want to get shaken out by one green tick in the middle of reds. If there's one, that's fine, but if there's two, then it's time to get out. So here, for example, we would have gotten in on this bar. So this would be approximately 2899. And then we would have rode it all the way until here, which would be about 2897 or you could have even gotten out even before that, especially when you see this guy moving up. So you would have gotten out on this bar. So it should be about 2896, 2895. So that's about three, four points on the S&P. It would be a nice put trade. Then you move to the other side. Now, as you can see now here, we are seeing some bullish action here. You can see the ticks increasing. And so is the RSI. And the RSI is about to go into the bullish, but it never really pierces it there. However, if you took a trade somewhere here, you could probably get out somewhere here. Not a big trade, but still nevertheless, it could be a profitable trade. Then the next one we see is right here. So the second tick is right here. You would get in somewhere here at 2898. And this one goes all the way till here.  That's when you would get out. And then we have one more trading opportunity right there in the end. As you can see over here, the market starts falling. The red ticks are accumulating and the RSI confirms it is weak. So whenever it is in this zone here, you don't want to get out of the trade because this is telling you to stay in the trade. Similarly, on the bullish side, this is telling you to stay in the call position. This is telling you that you stay in the put position until it starts to change direction and go until then you want to stay in this trade. So this is what we are going to see today in the live webinar, day trading webinar. We won't be looking at this because this is not very important. I just wanted to give this picture here. But what we are going to be doing is trading based on these signals here and they are both proprietary indicators. Thank you.


So there's a conspiracy theory out there, you know, one among gazillion others, but one that has had pretty horrific consequences already. And the conspiracy theory basically goes like this. It basically says the Jews in the United States want to bring, you know, illegal aliens, illegal migrants into the country, people from particularly immigrants from non-western countries, non-white countries. And they want to bring them into the United States.  And that they're trying to import all this into America.  If you go on Twitter, you can find many, many, many people who are tweeting this conspiracy theory with quotes from Jews prominent or not so prominent, primarily not so prominent, talking negatively about white people, and so on. I don't know. I have no idea where these quotes come from. But they're there. And you can find them on Twitter, all with the idea that Jews want to get rid of whiteness, and they want to get rid of white people. And the whole illegal immigration into the United States is really just Jews, you know, just Jews going after white people. Some people take this stuff seriously. Indeed, Robert Bowers, you know, took it seriously as he walked into the Tree of Life synagogue in Pittsburgh in 2018 and started shooting people, shooting Jews and murdering 11 people. You know, he wrote a whole social media post. And he was basically basically there to protect us from these Jews who are bringing in invaders over borders over borders in order to replace white people in America. This is part of Tucker Carlson's and others replacement theory, just replace Democrats with Jews, but it's all the same kind of idea. Anyway, this, this was articulated, this idea was articulated in a post on Twitter by some anti semi, who I'm not going to give the URL for. It was viewed by 1.2 million people. And this, this is the post is quoting the post quote, Jewish communities have been pushing the exact kind of dialectical, dialectical hatred against whites that they claim to want people to stop using against them.   And now those black and brown people are turning against the Jews and he shows no sympathy towards that. So this is the anti-semitic explicitly anti-semitic conspiracy theory. It is disgusting on so many fronts in the way it deals with immigrants and the way it treats immigrants.  I mean, that's the Jewish community has been involved in that. I'm pretty stunning stuff. This is anti-semitism 101.  And, and I wouldn't want to give it a platform on my show. And I don't think it's that that big of a deal out there. I don't think there are that many people engaged in it other than the fact that Elon Musk tweeted in response to this, you have said the actual truth. And that's the reason this tweet got 1.2 million views. Indeed, Elon Musk's retweet got 6.4 million views. So now Elon Musk is playing around with anti-semitic slogans, anti-semitic conspiracy theories, anti-semitic ideas. And nobody seems to care. Certainly nobody on the right is going to criticize him. Certainly Scott isn't going to criticize him. He's far too busy, worried about Taleb and whatever, to actually go after explicit anti-Semites because they're on the right. And after all, the right, they're our allies. They're against the left. So this is good. We should support them. These are the kind of people that promote hatred of Jews and at the same time, promote so-called Western values. These people in the long run, in my view, are much more dangerous in terms of what they can do and what impact they would have. And yeah, this is the world in which we live. This is the world in which we live with the richest man in the world. A man admired by pretty much everybody on the right, a man that supposedly stands up for liberty and freedom and American way and free speech, is expressing support for an anti-semitic conspiracy theory and almost nobody says anything. Because who wants to go up against Elon Musk? This is how civilization disappears. This is how, yeah. This is how anti-Semitism and conspiracy theories like this become mainstream. Become just like everything else.


Great. So thanks so much everybody for coming. We're going to be talking about metadata management with attribute option combos and approvals. If you don't know what any of those terms mean yet, we're going to explain a little bit about them as we delve into the presentation. But first, we want to introduce ourselves. So I'm Jim Grace. I'm a DHS II core developer. I've been working in areas such as validation rules, predictors, expression parsing. And one of the areas I've been working in is data approvals based on requirements from PEPFAR and other places. I designed the feature and did all the back-end coding and have stuck with it as it has evolved to where it is now, which we'll tell you some about. And I'm Ben Geraldi. And I'm the technical lead for datum. And datum is the PEPFAR installation of the DHS II. So it's used to record information about the AIDS interventions by the US government across the world. All right. So what are we talking about today? So we're talking about metadata management, how to manage the metadata in your system. So the information about the information in your system. We're going to talk about two DHS II features that are used for metadata management. The first one is attribute option combos. And the second one is approvals. And we'll explain what those each are in the part of the presentation that's about them. Both of these features were designed and developed by Lars and Jim. Jim, who's here. Lars, who's somewhere else. With PEPFAR in mind, around about 2015. The commits in GitHub don't go back that far. So we're not sure whether it was 2015 or 2016 or 2014. But it was around about that. But they were also both designed to be useful in situations that are very different from PEPFAR. They were designed as general tools. And we'll talk a little bit about how PEPFAR uses them and then how they're used in the Sierra Leone demo. But you may be able to imagine ways that they can be used in your instance. All right. So the first one we're going to talk about is dimensions. And this is sort of a roundabout way of getting to attribute option combos. Okay. So what are the basic dimensions of a data value? What it is? Like what is the information that we're recording about? And this is represented in DHIS 2 with a data element and then aggregations of the data element, which are represented by category options. Where is the data value? And this is represented in DHIS 2 generally by the organization unit, somewhere in the organization unit hierarchy. And when is the data value? And that's represented by the period. Right. So these may be sort of generally familiar terms to you. Okay. So disaggregation categories. So we talked a little bit about category options. How do category options work? Or so what are some examples of category options? Oh, and we're going to begin, but we're talking mostly about aggregate data in this presentation. But some of the things that we're going to talk about do apply to event data as well. All right. So the first kind of category that we're going to talk about is the disaggregation category. So some examples of that are age, sex, test result. And these are things that can be specified for a data value. What age is that data value for? What sex is that data value for? Is the test result positive or negative or indeterminate, et cetera? And we're going to show you how this works with two dimensions, one dimension, and zero dimensions in the next couple of slides. You could have more than two dimensions. You could have, I mean, you would have to have a finite number of dimensions, but you could have a very large number of dimensions as long as you have a database that's large enough to handle all the dimensions. And we'll talk about what that means a little bit in a little bit. And as long as you have a system that's sort of performant enough to deal with those dimensions. But we'll talk about two, one, and zero, and hopefully you'll be able to generalize for the two. All right. So here's a disaggregation category with two dimensions. So the two dimensions are age and gender. So this is an example from the Sierra Leone demo. It's ART enrollment stage one. ART enrollment stage one gets disaggregated by the category option comp, the category option that I always want to say option, the category combination HIV, age plus gender. That's the category option. The category combination HIV, age, and gender. And that is made up of the two categories, HIV, age, and gender. Those are each separate categories. Now in each of those categories, there is an option that is selected for this particular data value. So for the data value that we're talking about, you've got the option less than 15 years old and the gender female. So those are the two options that are selected for this data value. Now the category combination has a variety of options as well, if you will, or category option combinations that are combinations of the category options that are selected. So we combine less than 15 years old and female to get the category option combination less than 15 years old female. That whole thing with the comma is one object in the database. And that is the object that the data value is pointing to. That category option combo is, and we're using the ID for that. It's pointed to in the data value table. And we're showing the data value table not because you necessarily need to know about the data value table, but because we think it helps envision and understand how the HIS-2 is dealing with categories. So that's two dimensions. So one dimension is actually pretty similar, and this again is from this here, Leon Daniel. HIV currently on care is attached to the category combination HIV, age, which only has one category, which is named the same thing, HIV, age. But those are two different objects in the system, the HIV age category combination and the HIV age category. And that category combination combines HIV age with nothing. So it's just HIV age, but that combination is restored separately. Then we have an option from the HIV age category. That's less than 15 years old again. And we've got the category option combination less than 15 years, because again, it's not being combined with anything. And that's what gets assigned to the data value table. We happen to be wondering why we're talking about HIV age rather than regular age or just basic age. That's because there can be different age bands in the system. For instance, there could be HIV age and TB age. If those were tracked differently in a Ministry of Health, there could be a general age. There could be an age band that was used prior to 2015 and an age band that was used after 2015. And those would be different categories with different options. Okay. Just looking at my notes. All right. Okay, so what happens if we've got a data element that we don't want to disaggregate at all? An example of this in the Sierra Leone database is facility has access to running water. There's no disaggregations with that. It's either it does or it doesn't. So that attaches to the category combination default. In every DHIS2 system, there's a default category combination. And that is part of the default category. So the default category is combined with nothing to make the default category combination. Well, which option are we taking from the default category? There's only one option, default. So we've got the default category option. And that's combined with nothing to make the default category option combination. And so that ends up in the data value table. Right. And so that's how we do it with zero dimensions. All right. So we've talked a bunch about disaggregation categories, which are categories as applied to data elements. But now we're going to talk about attribute categories, which are attached to data sets. So data elements are sort of atomic data values on a form, right? But a data set can be thought of as an entire form. And those can be disaggregated as well. So just back to where we were talking about, where is the organization unit? When is the period? But I'm going to complicate what we said before. Data elements and disaggregate category options are not just what. They can be who, they can be how, they can be why, and they can even be additional dimensions of where and when. Perhaps a period says what day it happened, but it refers to a calendar year as well. And that can be present as a data element. Similarly, data sets can also represent who, what, where, when, how, and why. And so can attribute category options. All right. So here, so, oh, here are some visual examples of what these look like. So what does this look like in the UI? So here is a disaggregation, disaggregation categories with a data element, right? This is the ART stage one that we're looking at before. And we see that there's male and female. These are the two genders that happen to be present in the Sierra Leone database. And then we can see the age bands, less than 15, 15 through 24, 25 through 49, greater than 49. And we see those age bands reproduced for both male and female. So you can put something in each of these boxes and have this data element fully disaggregated. But what does it look like with the data set in an attribute category? You may have seen this on the data entry screen at the top. You can pick two options, right? And these are attribute category options, the implementing partner, which right now Afea Plus is selected in the project, nothing is selected there yet. Okay, attribute category. So some examples of this project could be a good one that you'd want a whole form of partners is a good one, accounts, etc. And we're going to reverse it a little bit. I think it's clearer to show this way. We're going to show zero dimensions first, then one dimension, then two dimensions. All right. So again, an example from the Sierra Leone database, the child health data element does not have any disaggregation in attribute categories. So similarly to when you didn't have any disaggregation for the disaggregation categories, the default category combination, default category, the default category option combination, and the default category option are all used. And these are the same default category combination, category, category option combination, and category option as was used in the previous slide. So this is the same object in the database. It's going off of the same table. But it applies to a different part of the data value table. So we saw the category option combo ID before. And there's also in every single data value an attribute option combo ID. And that is what is attached here to the category option combination. It's a little confusing, I know, because the category, you would think the category option combination attaches the category option combo ID and it does a different category option combo, but it also attaches to the attribute option combo ID. And that enables DHIS to use the same set of tables to represent two different pieces. All right. So what does it look like with one dimension? So the emergency response data element has one dimension, target versus result, which has two options, target or result. And so the category combination is target versus result. The category is target versus result. It's a different object, one's a category combination, one's a category. So that category is combined with nothing to make the category combination. Which option have we selected? We selected the target option. So that's the category option. And then the category option combination is target combined with nothing. So get target again. And again, that goes to the attribute option combo ID. And then here we see it with two dimensions, partner and project. This is for ART monthly summary. And this is the, we saw this bit in the UI earlier. The partner in the project is the category combination. And then there are two categories, partner and project, that could combined to make the category combination. And then each one of those has an option selected. So UNICEF is the partner and clean water is the project. And those two are combined to make the category option combination UNICEF clean water. And that one object UNICEF clean water is attached to the attribute option combo ID. All right. So when do you use disaggregation categories and when to use attribute categories? Basically when you're using them on a data element or a data set. But here's some other guidelines. You use disaggregation categories, DCs, when you, it's a long thing to write on slide. So I abbreviated it here. So you use DCs when you want different combinations of categories on different data elements within a data set. So we want, in a data set, we want to see data elements with disaggregations. Use disaggregation categories when you want to enter all of the category option combinations on one data entry screen. And so as we saw in the UI earlier, you see the whole cross product of all the possible combinations on one data entry screen. Use ACs basically in the opposite situation. You want to use the same combination of categories for all of the data in the data set. So you want to select it at the very beginning and then use it for all the data. And then when you just want to see one screen for each category option combination that you're selecting. All right. And then just so you know, there are also some constraints that can be applied to category options. You can add a start date to a category option. And these constraints apply to both disaggregation categories and attribute categories. So you can apply a start date. And so then it is not valid before that start date. You can apply an end date. Then it's not valid after that end date. You can create open periods after the end date. This was something that was added recently to Core DHIS2. So it may not be available in older versions, but it certainly is in 2.36. Then you can specify an end date. And then based on the data set, have it stay open after that end date. So not based on the category option. And then you can assign it to organization units, which means that it's available for the organization unit and all of the descendants of that organization unit. All right. So we've got some data. Now we want to analyze it. So let's talk about analyzing dimensions. So and we'll talk mostly about analyzing attribute categories. You could create category option groups and category option group sets. So the PEPFAR use case as an example, one of the reasons that this all got put into DHIS2. So PEPFAR has these things called mechanisms, which is the US government term for a project. It's basically, it's got a number and that number represents a bunch of work that is going to be done. Hang on. I need to get my notes out. Yeah. So it represents work that's going to be done. It represents money that's going to be spent. And it's going to represent results, hopefully, that are achieved. So each mechanism has exactly one partner. And that partner is usually an NGO or sometimes it's an MOH or sometimes it's actually a different part of the US government. And that partner is the group that's going to do the work. And then it has exactly one agency in the US government. And that agency is going to supervise the work. And so the partner also reports to the agency as you see here. So each mechanism has exactly one partner and exactly one agency. A partner might have more than one mechanism though. And an agency almost always has more than one mechanism. And a partner might share a mechanism with more than one agency. So they might have a mechanism with USAID and then they might have a mechanism with Peace Corps or something like that. So we want to limit what the users from the agency can see to only their mechanisms. We want to limit what the users from the partner can see to only their mechanisms, which is a different subset of mechanisms. So it's not exactly hierarchical and it will involve sharing, as you'll see. Okay. So let's start here. How are mechanisms represented in data, which is PEPFAR's installation of DHIS2? The mechanisms are represented by a single, a one-dimension attribute category, right? So the category is mechanism, right? And then in this case, we've got the category option selected, which is mechanism one, two, three, four. And because it's a single dimension that's combined with nothing to give you the category option combination one, two, three, four. And that is peers in the data value table at the attribute option combo ID. You should seem really familiar because we were just talking about a similar case. Then that category option is put in a category option group. So it's a group of category options that belong to this particular partner, which is named FHI 360. And there are several other category options that are in that group. And then that category option group is grouped together with other category option groups to make a category option group set. So that's a group of category option groups. And that is called partner. It's the word partner. And so when you look up that category option group set in the pivot table, you can see all of the different partners that you're allowed to see. All right. So that's how we deal with the partners and put their category options together. But we do the same thing with the agencies. We group the category options together into a category option group, in this case USAID. And then we take those category option groups that represent the agencies and we put them together into a category option group set, which would be agency in this case. All right. So that doesn't get us all the way there. We still need to restrict and allow users to see what they should see. Restrict users from seeing what they shouldn't see and allow users to see what they should see. All right. So this is a different representation of what we were just seeing. So here are these two mechanisms, one, two, three, four, and five, six, seven, eight that belong to FHI 360, the category options. And they're part of the category option group, FHI 360. And that category option group is grouped with the other category option groups that represent partners and put into the category option group set partner. So now we're going to apply some sharing to it. So both the category option group and the category options are shared with a user group, Kenya FHI 360 users, which is assigned to the Kenya organization unit. And also the users are assigned to the Kenya organization unit. And so they're restricted by their organization unit. They can only see data that's within their organization unit or its descendants. And they are also restricted by another thing, the user dimension restriction of mechanism. And that's something that you apply to the user in the user's app or via the API if you're modifying users via the API. And that user dimension restriction of this category of mechanism means that the users can only see the category options that are assigned to them. Agency users are done in a similar way. Here are the category options that represent mechanisms that belong to USAID and the category option group that it means that is USA. And that's again put into an agency category option group set. And those are shared with a different user group, Kenya USAID users. And that user group is assigned to the Kenya organization unit. And these users also have the user dimension restriction of mechanism. And I should say that each user will only appear in one of these user groups. And that indicates what type of user they are in the PEPFAR system. They're either a partner user in which case they appear in a group that looks like that or they're an agency user in which case they appear in a group like that or they're one of the other user types which we'll cover in just a moment. So country users have all of the mechanisms in Kenya, all the category options that appear in Kenya shared with them. They have all the category option groups that are applicable for Kenya shared with them. And they are assigned to the Kenya organization unit. So they have no user dimension restrictions. And so they are only restricted by what is in their organization unit. And so this is another type of user, third type that we're going to cover today. And then the fourth type of user is global users. So the top level organization unit in datum is called global. And there are users assigned to global. And they have all mechanisms, all of them are shared with them. They have all category option groups shared with them. And they're assigned to the global organization unit. So there are no user dimension restrictions. Not that they would have any applicability because everything is shared with them, but there aren't any. And they have no restrictions based on their organization. Whoo, I'm out of breath. So that was a lot. If you have any questions, feel free to put them in the chat and then we'll handle them at the end of Jim's talk. And so Jim is now going to talk about approvals. Thank you, Ben. I'm going to talk about data approvals. And next, there will be two parts of the data approvals that I'll be talking about. First of all, a data approval is based on an organization, a level of data approval is based on an organization unit level. And so in the first part of this talk about data approvals, we'll just talk about how data works with organization units. So you get the basic idea of how approvals work. Then I will talk about how approval, data approval levels can also use the attribute categories that Ben just introduced us to. So this is where things get kind of interesting and not intuitive. But first, let's just review data approval with regards to organization units next. So data approval always follows a hierarchy. The idea is you have one or more levels. You only need to have one level of data approval. But if you have multiple levels, then you approve data starting first at the lowest level, and you approve it at higher levels as you go up. So in this example, C and D or units might be facilities. And once the C and D have approved their data, then B, which might be at a district level, I can approve its data, can approve its data, including all the organization units underneath it. Organization unit A in this scenario is not yet ready to approve because not all of its children organization units have approved. So the hierarchy always flows from the lower levels to the higher levels. Next. So a data approval level is based on an organization unit level, even when we get fancier and add attribute options as you'll see later. Okay. Yet an organization unit level may or may not have a corresponding data approval level. So you could have just a single organization unit level or as we're about to see, you could skip organization unit levels next. So for example, yeah, you could have five organization unit levels in your system. And maybe you only want to approve data at the sub district level, and then approve it at the province level. So it's up to you when you design the metadata for data approvals as to which org unit levels, if any, you want to approve data at. Next. The number of steps in approving data can be configured by a system settings. There are two different options here. And this is another thing that's important to decide before you deploy data approvals, which option you want. Next, the first option is where you just have your approved data at one level. And then once it's approved for all the children of the next level up, it can be approved at the next level. Now, why would we want anything different from this? Well, one of the challenges, if you have a system like this, let's say you have a district with five facilities in it. Let's say that facility one approves its data. So it's ready for review at the district level. Oh, first I should say, one of the design principles we used in data approval is if you do something by accident, as much as possible, you should be able to do it. So let's say that facility one had their data approved, and then they realized, oh, I hit the wrong button. I hit approve or something, or perhaps they just got in some late coming data or a correction, and the people at the district office haven't looked at it yet. So you can actually un-approves, you can undo the approval, fix the data, and then hit approve again. So that's great at the facility level. But what about if the district, let's say facility one submits their data, approves it, facility two does. So meanwhile, at the district level, they look over the the data for facility one, it looks good. They look over the data for facility two, it looks good. But maybe at the time they're looking over the data for facility four, facility one goes back and oops, they made a mistake, they approve the data, they fix it, they re-approve it. So in order to be more sophisticated for that kind of scenario, we offer another option next, which has two steps at each approval level. So the facility can approve the data, and then when the person at the district starts reviewing it, they can accept that approval. The accept means that the approval is locked, and the lower level can't just undo it anymore. Now at the higher level, they are guaranteed that the data will stay the same as they review the data from various facilities or various lower level units. And when the higher level feels like all the data is good, then they can say approve, and the next level up can't accept. So this process goes up. So it's important when you're configuring data approvals, understand these two different scenarios and pick the one that's the right balance for you between the amount of work and how your workflow goes. Next. Speaking of workflows, a data approval workflow is something that we designed for reasons I will say in a moment. It specifies a period type and one or more data approval levels. Next. And you can spec, you can join data sets to data approval workflows. If you're using data approvals, you don't need to use them for every data set. That's your choice. You can have every data set assigned to some workflow or not. So next. For example, you might have oops, you might have a one workflow that you here we go. You might have one workflow that you call HIV. And it approves the data in multiple data sets like HIV care and HIV pediatric pediatric. Another word I can't say monthly summary. You might have other workflows that are assigned for other data sets. And when we first implemented data approvals, we didn't have workflows. You just approved a data set. The reason we developed workflows is that PEPFAR found that they had many, many different data sets that they all wanted to approve pretty much at the same time. And then it just became an administrative hassle to go through well, let's approve this data set and that data set and the next data set. So we created an abstraction in a subsequent release of DHIS2. They have a data approval workflow where you could just approve for one workflow and it would cover a bunch of data sets. In fact, the old data approvals app, some of you may remember that, is organized around data set. You select the data set for approving. We are now in the process of designing a new data approvals app that should be available in 2.37 that will let you select the workflow. It will better match the model that we have underneath. Okay, next. So here's a question. Next. If you have a data approval workflow, you're going to be approving data for some period, some aggregate period, a month, or a quarter, a week, a year, whatever. And you're going to be approving data in one or more data sets. So here's a question. Does the period type for the data approval workflow have to be the same as the period type for the data set that actually contains the data? So think for a moment before Ben shows you the answer and think, what would your answer to this question be? And this was a design question that we were facing and had to figure out, did we want to constrict it? What would we want to do? Okay, so Ben, you can show the answer. The answer is no. The data set may have a period type that's smaller or the same or larger than the workflow period type. And the next slide will show how that works. Next. So for example, if you have a approval workflow that has a quarterly period type and you approve things for the last quarter, that looks like it should be Q4 on that slide. We'll have to fix that. Of 2018 shows you in these slides, the slide deck was first written, or at least this slide. That quarterly workflow approval can approve data in monthly data sets. In this case, it will approve all three months in the quarter. It can approve data in a quarterly data set for that quarter. It can even approve data in a yearly data set. And how does that work? Next. What we did was we decided that if a data set ends during the approved period, that approval will approve it. So if you want, your fourth quarter approval process can also cover all of the annual data. Now you don't have to do it this way, but it's flexible you can if you want. Next. Next. One of the things that's been suggested is that we extend approval and there's even a ticket, I believe it's even marked to do in JIRA, that we extend approvals to handle event data as well. Currently, data approvals only apply to aggregate data, the standard in data sets. Next. But it can be extended in future by allowing a program to be a member of a workflow, as well as allowing a data set to be a member of a workflow. And this is one of the things that we envisioned when we designed workflows and moved away from improving data sets that have provided an additional level of abstraction, so workflow in future could apply not only to data sets, but to programs. Next. So we talk about data approvals as if we have some shared understanding of what it does. Someone says, oh, I approve, I like that data. It looks good. I think it's correct. But what does it actually do in the system? Well, data approvals actually do two things in the system. Next. First, it locks data that you've entered from being new data from being entered or existing data from being changed. You can go ahead and build next. It allows it for a given organization unit. Next. And all the organization units under it, an approval at a district level will lock the data for all of the organization units under that. Next. Next. For a given period. Next. For all data elements belonging to any of the data sets in that approval workflow. Next. And for attribute categories, which we will talk about very soon. You'll see how that factors into data approvals. Next. The second thing that data approvals do in the system is option. You can figure it with a system setting. Next. It can hide unapproved data from higher level users. And the reason for this is that in the case of PEPFAR and maybe some other cases as well, what we were trying to model is a country collects all of its data. And it used to be that a country would physically say physically. A country would upload their collected data in a some form, in a zip file, to the central system. And until the country decides that their data is good and they like it and they approve it. They have it on their own system. The central system doesn't have it. And when they decide it's good, they upload it. So we wanted to do, we wanted to model that because sometimes a lower level system does not want the higher ups to see their data until they believe it's correct. So there is an optional system setting. Again, another thing to consider when you're configuring data approvals that can hide the data entered from higher level users. And then, of course, there's also a special way of privileging a user by giving them a certain authority where if you have users who are auditing or they're consulting with the lower level users, you can let them have a special privilege to get around this restriction. But mostly it restricts higher level users from seeing lower level data. Next. So the data hiding. This is a kind of arcane thing but good to know. If you have users assigned at different levels, let's say users A, B, C, and D are assigned at four org unit hierarchy levels. But let's say you don't approve data at the province level. Where would user B fall in? You might assign them the privilege to approve data. Where would they approve it? The answer is they would approve it at the next level down where there is an approval level. So user B could operate to approve district level data. In fact, user B could approve district level data for any of the districts in the province. And like user C, they could see the data from lower levels. Sorry about the use of the word C in two different contexts there. But they can view the data at different levels because they are authorized to approve it. So this is again something that you might want to consider as you're designing the data approvals for your system. Next. So I've been talking now about how data approvals work in an organization unit hierarchy. Now we'll add the fun stuff. We'll show how data approvals work in an organization unit hierarchy and using attribute categories. Next. Next. Data for each attribute category when you use them are approved independently. So you can have, you can approve data for each project or for each partner or in the case of PEPFAR for each partner and then for each agency that's overseeing the partner. Next. Approval levels, data approval levels can be defined in one of the following ways. Next. You can have, as we've seen before, you can just have one approval level for each organization unit level. Next. You can have, these refer to examples. Next build. These we're about to see in the following slides. You can have two approval levels in the same organization unit level. And I will show you how that works just very shortly or three. You can have more than two. In fact, you can have any number depending on how complex your requirements are of approval levels in an organization unit level. I'm, I do a lousy job at looking at the chat at the same time I speak. So if there's something I should be aware of, someone please read the chat. Yeah, I'm reading it, Jim. Like, there may be some questions that we'll take at the end if we have time. Otherwise, and we might be able to run over, I don't know. And then otherwise we'll take them to the community of practice. Good. So next slide. If you have just one approval level for an organization unit level, you just define the levels. And so for example, you might have approval level one, you might call country, and that's at the organization unit level country. You're in perhaps you have some other levels in your hierarchy at which you don't approve data. Maybe you go all the way down to organization unit level four, before you have your second approval level at the district level. And then your approval level three might be for facility at the Oregon level five. Now you see this extra column called category option group set.  So we'll get to that in a moment. But if you only have one approval level at each organization unit level, and you're not approving by individual category options, then you just leave that leave that blank. Don't fill it in. Next slide. If you have if you want to have two approval levels at the same or unit level, you can do that by assigning a category option group set to one of your approval levels. So in the Kepler example, they want to be able to have a project partner approve data on a project basis or on a mechanism basis. And for example, you could do that at district at the district org unit level. And then at the same org unit level, you could have someone responsible for approving all the partners within that district. And so those users would be operating at approval level two, whereas the partners who were only assigned to a certain number of attribute options, the way Ben described earlier in the session, would be approving data at approval level three. Now we did this because we wanted to have something that was flexible for PEPFAR. And with all the hundreds, thousands, I guess, of users in the system, we didn't want to have them have to configure a separate standard workflow engine off the shelf and re-enter all the users and all the groups. So this workflow, this model of approvals follows all the sharing setup that Ben just showed, where you can have partners that are shared with some attribute options, and you can have above them agencies that are shared with some attribute options. We go to the next slide, okay. Yes, go to the next slide. So what PEPFAR wanted, and we had to model, was three approval levels at the country level. They wanted the country partner user to look through the data and make sure that the user, the data for that partner was okay. They wanted a country agency user such as CDC or USAID, in PEPFAR's case, to look through the data from all the partners that they supervised within that country. And then they wanted a country team, a country interagency user, to approve the data that came from all of the agency approvals that came from all of the partner approvals, all this at the single org unit level of country. Next slide. So this is modeled in DHAS2 by setting up two different category option group sets for different approval levels. If you remember back from Ben's slide, there was no hierarchy in how the sharing was set up between partners and agencies. It's just that an agency happened to see more mechanisms than a partner, but there was no hierarchy in that configuration of the metadata. Here is where the only hierarchy exists within PEPFAR between partners and agencies is in the approval levels. A partner has first crack at approving the partner is configured to be at approval level four because the category option group set is part of approval level four. So this means that a partner, I'm sorry if this is all a little bit confusing. This is how we got it to work for PEPFAR without the need for reconfiguring thousands of users and groups. So an implementing partner will have first crack at the data. They'll implement it for all the mechanisms, for all the category options they're responsible for, then funding up or at the code already, then up at the next level agency can approve what had been done by the implementing partner. So that was a little bit rushed and it is a little bit complicated and the code to implement this is a little bit complicated as well inside the backend. And I'm guessing that you might have some questions. So who would like to ask a question? So I could read a couple of the questions that are in the chat. So is it too cumbersome, Stephen asked, is it too cumbersome in practice to take and click the complete button as a kind of approval at that level? Do you understand that question? Yes. So what we were asked for is to have approvals be a different process from completion. And if you have data completion that goes into certain reporting. But the idea is that we wanted perhaps a different set of people to look at the data quality and not just vouch that it was complete, but that it looks good. And so again, from the user requirements, we developed this as a separate facility. But if there are ways in which data approvals should relate more to data completion, please let us know your use case. Ideally, say something about that in the community of practice because we're always open to how we can make the system work better. And it's all driven from user use cases. All right. So the next question. Antonio and I were talking a little bit about how user groups are managed. And he was asking whether it was possible for user administrators to be restricted in the users that they managed. And I mentioned that we do that through a parallel set of user administrator groups. So I was talking about all the different user groups. If you're a user administrator at these different levels, then you're in the user administrators group. And the user group has the managed by property set to the user administrators. And actually, what Jim said before, there are actually two hierarchies where agencies are above partners in PEPFAR. One is approvals and the other is in the sort of managed of user groups. And that's pretty outside of the scope of this presentation. But I'd be happy to talk about other portions of that in the community of practice. There are a lot of user groups. It's basically the answer to that question. Okay. So Stefan asked, I think this one's for you, Jim. I think I noticed that approvals only make data read-only. Facilities that didn't report can still report. Is that correct in your knowledge? And if so, is there a way to prevent that? It should block new data entry as well as existing data. The thing about approvals is they don't apply to every data value. They only apply to a group of dimensions. So they apply to an org unit for a particular approved period of time for a particular set of data elements that belong to the data sets that you approved. And if you use attribute options, they apply only to those attribute options. So we don't store an approval for each data value in the system. We just store one approval for those, oh, and for the workflow that's approved. We just store one database record for those set of dimensions and we don't allow new data entry or data changing in those dimensions. So theoretically, if a new facility were created, and if it was underneath a different organization unit that had an approval level, then it would not be able to enter data. But if it wasn't, it would not have been approved and would be. Is that how that works? Correct. If a new org unit is under something that's been approved, then it's approved as well. Yeah. But if it's not, then it could be open, even if all the other org units that are sort of at the same level that are. So anyway, we could talk about more of that in the community to practice too. So Maria writes, an important functionality that is missing for us from the approval app is being able to see a list of what has been approved, what is pending approval. We are on an older version of DHIS2. So I don't know if this has already been implemented in newer versions or if there are any plans to implement this in the near future. You've looked at the specs of the new data approval app. Can you talk to whether that's covered in those? The new, I'm sorry, I was just noticing someone had their hand up and I missed part of the question, but the new data approval will allow you to select the workflow and this might not be there in the first release, but the plan is ultimately you can drill down the organization unit tree and you can see where the organization units below have been approved or not. And again, this might not be there for 2.37 that's currently under discussion. That particular feature might wait for a later release as we reconfigure some of our front-end software that we use to display organization units and such. I'm sorry, did that address the question? I think so, yeah. And Stefan mentioned that his organization has an app, but it's not publicly available that does that. And PEPFAR does too, but our app is a very tailored to our own approval process and I don't think it would work on another DHIS-2 system. So if you were interested in attempting to configure it to your system, we could talk about that and see if we can release the source code to you. I'm all looking at CA Hussain. Hussain has his hand raised and I'd like to put that question. Yeah. Yes. Thank you very, very much. The approval process works with organization unit levels. That means it's assuming that every organization hierarchy in a country or an DHIS instance has the same organization level structure. But for instance, if you come to Ethiopia, we have about 10 regions. Most of them have different hierarchy levels. For instance, if you come to Addis Ababa, it has only four levels. We go to Oromia in Aparah, if they have six levels, some regions have five levels. So using the approval process, the approval workflow here is not working. It cannot work. How do you think we can achieve this approval process? Do you suggest any workflow? Or are there any methods that are planned like Orgid Groups or something else? What do you propose for this? Because we really like the approval process, but we cannot make it work here. Thank you. That is a very good question. The traditional advice for creating your organization unit structure from DHIS too is that if at all possible, keep all the like org units at the same level. Keep all your facilities at one level and so on. And in some installations, that just doesn't work for various reasons. In PEPFAR, it is not that way because PEPFAR is in multiple countries and each has their own administrative hierarchy. But that is a good question and it is a real challenge because as you were saying, approval levels follow the organization unit level. That is something that we might be able to look at. It would be it would be great if in the community of practice you could document exactly what your challenge is there. And I am not promising anything, but we will look at it and we will see if we can think of a way to do that as you say with Orgid Groups or maybe some other way. Thank you. Great question. Well, I took a look at the community of practice and there aren't any questions there yet, but feel free to ask questions after the session. Any last questions that folks want to put in the chat or raise your hand to ask? All right. Well, thanks so much for having us. This presentation we are going to put on GitHub at that link right there and it will be on SCED as well. And then I know that there will be a video recording of this that will be posted to the DHS2 YouTube channel. So, yeah, thank you so much for coming. And oh, we forgot to mention something. So hopefully you're still here. If you didn't learn anything in the session and you listened to the whole thing, then the next time that Jim or I sees you in person, we'll buy you a drink. So, let us know if you want to take us up on that. I can see Steven Napaua has a hand up. Do you want to take one last question? Absolutely. Are you able to do that? I don't know how to do that. Okay, great. Yeah. Thank you very much for this wonderful opportunity. My question is, I'm Steven calling from Ghana specifically and I'm a health worker. I'm a physician assistant. And currently I'm managing a health center in Ghana here. My question is, how different is this one, the teams that you are getting the training from, the one we are using in Ghana here? We have the system whereby we enter our data, then the district director of health service approve the data before it goes to the regional, then before it goes to national. And looking at the approval levels, I have an issue because the facility that the data is coming from, the in charge or the head of that facility cannot approve the data before it goes to the district level before it goes to the regional or go to the national. So my question is, looking at the way the approval level has been presented here, I would have been very much happy if our own here to something like that can be implemented so that lower facilities can own the data and then they can approve it before the data is also approved by the district director or maybe the regional. Thank you. Sure, good question. So that is a decision that would be made for the entire installation. Either you have an approval, you add a new approval level at the facility level, and then all facilities would need to approve their data. Or you just have the first approval level as it sounds your system is currently configured at the district level. At the moment, there's no way to do it. Only some facilities, but not others. However, you can also have users, higher level users, maybe at the district levels, who are authorized to approve at a lower level. So for example, if at the district level, you have three facilities or five facilities, and you know that facilities A and B want to approve their own data and they want you to wait until they've approved it. And maybe you know that at facilities C, D, and E, they don't have anyone to approve the data. You can assign an authority to the district level so that they could on behalf of facilities C, D, and E, they could approve their data up to the district level. So there are options for making it work, but it would need to be configured at the national instance in order for it to work. And Phillip writes in the chat, I think referring to this, that it's a critical process that should be done very carefully where every stakeholder will not have a functionality problem. And related to the previous presentation where Bob and Austin were talking about upgrading things, when you make changes like this, you want to do a backup beforehand so that if you realize that it doesn't work, you can abandon it. So yes, that another piece that may be useful is with the PEPFAR instance, there's a period of time where the approval levels are set by the different folks at the different levels, as we've been talking about. But then at the end of a period, all the approval levels get pulled up to accept it by global. So everything gets auto approved. So things don't languish in those sort of low levels of approval. So that is something that your ops staff would have to do or something like that, or you would need an app that could do that or something. But that is a potential approach that is admittedly a little janky. Well, I think there's a poster session going on right now. So we're happy to, I don't know, Martin.  Otherwise, we could take some more questions or we call here.