Hi, I'm Andrew Josie, Vice President of Standards and Certification at the Open Group. I'm joined today by Paul Holman, Vice Chairman of the Open Group's Architecture Forum, and we're going to talk about business architecture and the latest developments in the Togo of Standard, version 9.2. Thanks, and I'm looking forward to the conversation. Well, Paul, how do you think business architecture should be addressing today's challenges with business transformation? So, I think that's a really key point, and one of the things I was reflecting on not that long ago was, I used to say business architecture was about mirroring the business, in other words, understanding what the business did so that you could kind of reflect that in the enterprise architecture. Today, I think it is still that, but I think it's more than that as well. I think there's so much around trying to understand disruption, trying to understand how you make a business agile and how businesses themselves can innovate, that business architecture has actually become even more important than it was before, and I think that's the kind of key challenge for business architects and business architecture. OK, how do you see it supporting this sort of movement to agile in particular? Well, one of the key things in agile is understanding value. So, agile is all about value and value-add, and if you really get down to the principles of agile and things like lean, value is why you do something, and frankly, if it doesn't add value, you shouldn't do it. That's a key agile principle. But that begs the question, what is valuable to my organisation? So, you need to have a way of understanding what the value propositions are, what the value streams are in the business in order to be able to deliver anything that's agile. The two are inextricably linked. OK, and I understand there are some new Togaf series guides that sort of help in that area in particular, especially around value capabilities, business models, information mapping. Could you tell us a little bit about those? Yeah, those are the four main things that exist as series guides now. Some great work has been done by many people looking at exactly those topics. If I pick each one out, I'll start with just my personal favourite. I'm prejudiced slightly. I like capability models. I've always found that whenever I use a capability model, it's been a really good ally and friend. But should I show favouritism? But capability models are one critical area that has been captured. There are some good standards and some good examples about use of those, how to develop them and how to share them with the business. Capability models help identify what an organisation has to do, what abilities it has to have to be able to deliver its value propositions. Against a capability, you can kind of understand what businesses need to do and what, therefore, any projects you might work on, any change initiatives, any gaps you want to close, can address. Allied to that, and we talked about value streams. Value streams are ways of representing how value is added by an organisation in a joined up way. Similar to value chains, but broken out into that each step along the way individually adds value in its own right. And a great way of being able to illustrate what an organisation is trying to achieve. That can be mapped to capabilities, what capabilities do you need to deliver those values. The other really key part is anyone who's ever worked in architecture knows that information is critical. It's the lifeblood that runs throughout the business. Information modelling, whilst information architecture is a domain in its own right, within business architecture there's a special kind of place for key level entities. Things that are absolutely critical to those businesses. Things like plan or part or process that mean something very, very specific to that business. Everybody who's ever worked in any industry will know what those words are for them. Those exist in their own right in the business architecture. We have some serious guides that look at the information modelling for those key entities. The last one, which in some ways sits first, but I've left it to bring them all together, is the business modelling. And some really sage advice around how to do business modelling, but bearing in mind that a lot of business modelling is about communication. So understanding who the audience is, and what message you want to get over to your stakeholders. The number of times I've been asked, can you put a single, my whole business on a single A4, where in fact actually there are so many different stakeholders, you kind of want to have different models for different purposes, depending on what message you're trying to put over. But that series guide to address them comes up with some great hints, some great techniques for being able to address business modelling. And those four really are the fundamental updates, separate series guides, but also in the TOGAF standard specification. So Paul, I understand professionalism is very important to the open group and they've been making some changes in this area, bringing on some new programmes in the area of business architecture. Can you tell us a little bit more about that? Absolutely. I mean, professionalism has always been important for people who want to build their career out. And I think business architecture has been an area where as a business architect, a lot of people who've felt a little bit left out to be quite honest, a little bit left out in terms of what is the career path for a business architect? How do you become a business architect? And how do you demonstrate that you are a good business architect? And with the development now of a separate series of set of series guides and the ability to build a credential that actually allows for an open badge to be awarded for somebody who is a business architect, business architects can now actually demonstrate and say, I have a level of knowledge and a level of practice around business architecture akin to other similar kind of professions and I can display that. And for me that's been shown to be valuable amongst people, especially when they're starting their careers or they're coming through, a lot of people these days are working as say business analysts in agile teams and they'd like to see where that could go and having there for a credential they can aim for that helps build up their CV, helps build up their professional status that is recognised by their peers is hugely valuable for them. So the initial business architecture credential is a great step towards adding to that portfolio. OK, well thank you very much for talking with us today Paul and it's really good to hear about all these new things that are happening around the business architecture space in the open group. It's a pleasure.


David, you ready? All right. So today, I'm going to describe the idea of maximum likelihood, which is the beginning of thinking about what we call generative models, which is the idea that you're going to describe a model which your data was generated, not just fitting your data to some model, but describing the model with which this data was generated, which means that you're going to have to make some assumptions about the probabilistic nature of the data. So this is going to lead us to the notion that maximum likelihood, which is the way to say, what's the most likely scenario in which the data that I collected was generated? So there are some parameters in your model, and these parameters include like the weights, maybe the variances associated with the noise. And what maximum likelihood does is to say, well, all right, given the data that I saw, how do I maximize the likelihood of seeing this data given the parameters that are such and such? So it's more than just describing a model that fits the data, it's describing a model that produces the data. And so that's called the generative model. At the end of the lecture, what I want to show you is the concept of how we can use this to describe mixing of multiple sensors. So for example, I may have one sensor that measures something and another sensor that measures the same thing, and how do I combine those two things using, again, these kinds of concept of maximum likelihood. And then we'll talk about an experiment where they looked at how people combine what they see with how they feel. So in this experiment, what was happening is that people were feeling an object haptically, so they had a sense of their size, they could also see it, and they combined these two pieces of information to form an estimate of the size of the object. And we're going to use maximum likelihood to understand how they combine these two kinds of sensors. So first we build a mathematical framework to describe maximum likelihood. We'll link it to the work that we've been doing so far, minimizing the loss function and so forth, and then we'll link it to biology and how in this one condition, when you have multiple sensors, you combine the information from those two sensors. So we've been talking about a loss function. And for most of the class, we've been describing the loss function as a quadratic function, where we have some sum, something like this. And it's fair to ask, what's good about this loss function and where does it come from? So today we're going to talk about our generative models, which describe how we believe our data is generated. And we'll see that under certain assumptions, this is the maximum likelihood thing that you would estimate, meaning that under conditions where you believe your noise is Gaussian with zero mean, then this loss function that you are minimizing is equivalent to maximizing the likelihood of measurement y. So we'll see there's a relationship between a quadratic loss function and a generative model in which you believe there's Gaussian noise. So let's see how that would work. So in principle, what I mean by generative models is that, for example, in this case, we have two things that we observe. We observe x and we observe y. And so the relationship is as follows. So there's a y that we measure. There's an x that is related to this y. And there's this thing called w that we don't know what it is. And so what I'm going to use in terms of our nomenclature from here on is a description where things that I measure are going to have double circles. Arrows are going to describe conditional probabilities. And one circle means that I can't measure it. It's something that I'm going to try to estimate. And what I'm trying to do in such models is describe the relationship between things that I see and things that are producing it. So for example, in this case, I have y is equal to w transpose x plus some noise, epsilon. And let's assume that this epsilon has the following properties, say that it has mean 0 and variance sigma squared. So that's a generative model. It says that if you were to give me an x, I would give you a y. But you notice that this y that I'm going to measure is not going to be the same for every x. So for the same x, I might get a different y, depending on what the epsilon is, what the noise is on that term. So what maximum likelihood is, is the idea that find a way to maximize the likelihood of seeing all the y's that you saw given that you had this particular model. So you're going to write the probability of y given w, x, and sigma squared. And basically, that probability in this case is going to be a normal distribution with mean w transpose x and variance sigma squared. So if your y that you observed, say you observed y of 1, this is the first data point that you got. And if this particular position is far from this mean, w, t, x, i, then that means that that's a very unlikely data point that you have. On the other hand, the certainty that you have regarding this data point that you observe is related not just to the mean, but it's also related to this variance here. So if your model says that I have a lot of noise in my measurement, this is sigma squared that I can see versus this, then, of course, a small difference between what you predicted, w transpose x1, and what you observed here, y1, can make it so that this is very unlikely on the this scenario, but it's pretty likely on the this scenario, depending on the noise that you have in your model. So the same weights, depending on the noise, is going to give you a different certainty associated with your particular observation. What maximum likelihood does is says that let's find the parameters, w and sigma, in such a way that we maximize the likelihood of actually observing the data that we did. So I'm giving you some data. You have a model that says how this data was generated. Find the parameters, w and sigma, that produce the data that you saw. And if you do that, then you've maximized the likelihood of seeing that data. That's what maximum likelihood is about. So in principle, what we're going to do is maximize the probability of the observations that we had, given that there was some w that generated it, and there were these x's that we saw, and there was some variance sigma squared that was associated with the distribution that generated that data. That's the likelihood. And what maximum likelihood does is find the parameters. This is one of the parameters. This is the other parameter. We're going to call that theta. That produce the data that you saw. In principle, you could have whatever generative model that you have, however you believe that your data was generated, by maximizing the likelihood of the data that you saw, you estimate the parameters. And what we will see is that in the case of this particular cost function, when we were trying to find the parameter w, what we're really doing is maximizing the likelihood under the data assumption. This particular generative model with this particular noise. Any questions about that? So let me give you some examples. So let's start with a particular distribution. Say I have an exponential distribution. I'm going to give you some data points. Say I give you n data points. Let's make a capital N drawn from an exponential distribution. So what that means is that probability of y being equal to with some yi, the particular data point that you saw, given that I have some parameter a associated with my exponential distribution, if 1 over a exponent of minus 1 over a times y. So this says how likely is it that the data that you saw came with this particular probability distribution? And what I want to know is what's the parameter a that would maximize the particular data that I saw? So how many data points that I saw? I saw data points from 1 to n, so I have the probability of y1, yn, given a, that's my likelihood. And because each of one of these observations is an independent observation, what that becomes is the multiplication of this quantity n times. And if I multiply this quantity n times, it becomes 1 over a to the n exponential minus 1 over a sum of. So OK, what I want to do is maximize this likelihood. Well, how do I maximize that function? Typically what we do is that instead of maximizing an exponential, what we do is that we take a logarithm of it so that we have a simpler function to maximize. A logarithm is just fine, because it's a function that's always going to be increasing as this component increases. So basically, if I want to maximize this, I can just as well maximize the logarithm of this function. So we call that a log likelihood, which is basically the log of this probability, which when I find the log of that, I want to find the log of this quantity. Let's see what that would be. It would be the log of the first quantity plus the log of the second quantity, because log of a times b is log of a plus log of b. So log of the first term plus log of the second term, I'm going to get a negative of a sum, which is going to be equal to log of a. This is log of a to the minus n, which is going to become minus n times log of a minus 1 over a sum of y of i. So now what I want to do is find the a that maximizes this likelihood. So what I do is that I find the derivative of the log of this probability with respect to a. So derivative of log of x dx is 1 over x. So this becomes minus n over a minus, this is a to the minus 1, so it becomes plus a squared. Sum of i is equal to 1 to n, yi. Set this equal to 0. I get, so then I have a is equal to 1 over n. Sum of i is equal to 1 to n, yi. So this is my maximum likelihood estimate of a. Does this look all right? OK, let's now do this for another problem. Say that what we have is a data set that's generated. So we have n data points from a Gaussian distribution. And so in this case, the probability of y being equal to yi is dependent on some mean mu and some variance sigma squared. And that's a Gaussian, so that's 1 over this quantity. OK, so we have been given n data points. So what we have is the probability of y1, yn given mu and sigma squared is going to be equal to the multiplication of this quantity, which is equal to 1 over 2 pi n over 2 sigma to the power of n, the exponential of. So next what I do is I find the log of this. So this is my likelihood of seeing those data points. And it's just a multiplication of all those normals. So now what I find is the log of this, the log likelihood, that's going to be equal to the log of the first term, which is, well, let me write it out. So this is log of 2 pi to minus n over 2 plus log of sigma raised to the power minus n plus or minus, which is equal to minus n over 2 log of 2 pi minus n log of sigma. OK, now I find the derivative of this log likelihood divided by the derivative with respect to mu, the first parameter that I want to estimate. So I have two parameters, mean and variance. So I want to find what's the mean of the distribution. I find the derivative with respect to this mean. I get minus 1 over 2 sigma squared sum of minus mu times minus 1, which gives me this as a plus here. And the 2 cancels out. There's a 2 there as well. So when I solve for mu, set this equal to 0, solve for mu, I get this is irrelevant. I just get 1 over n sum of yi. So that's your typical, this is how you find the mean of a normal distribution by finding the sum of the values dividing by the number of elements. And this is the maximum likelihood estimate of the mean of that distribution. Let's find the variance of the estimate of the variance. So we want to find derivative of the log likelihood with respect to sigma. So I want to find the derivative of this with respect to sigma. I get minus n over sigma. Minus sigma to the minus 2. Minus 2 comes in the front. I get a minus 3. So I have plus 1 over sigma cubed. So I set this equal to 0. What I have is minus n. I want to solve for sigma squared. This doesn't matter because there's a 0 over there. So what I end up with is sigma squared is equal to 1 over n sum of yi minus mu squared. So this is my maximum likelihood estimate of the variance of my data. Any questions about this? Yeah? In that last equation, is the ML estimate of mu? Yep. OK. So let's now do the problem for our estimating w in our linear model with a generative model that says how our data was produced. So suppose what we have is a likelihood that looks like this is equal to the sum, sorry, the multiplication of each one of those likelihoods which are given by my normal distribution. Something like this. So if I multiply this out, I get sigma raised to the power of n. You can see that. So this is the likelihood for the model where every data point that you see is generated by your weight that you're trying to estimate and some noise that you have in your measurement assuming that noise is Gaussian with zero mean and variance sigma squared. So every data point I gave you, you're going to assume that it was generated by a model in which there was noise. And that noise was sigma squared with property of zero mean. OK. So if that's the model that generates your data, then this is the likelihood at which every data point that you see can be given a probability. So what we want to do is find these parameters that produce our data. This is what we need to find, w and sigma. This is our unknowns. So we want to find the log likelihood and then maximize it for the parameters w and sigma squared. So the log likelihood, what is that? So I have a multiplication here. So what I have is that I have the first term minus n over 2 log of 2 pi minus n log of sigma minus 1 over 2 sigma squared sum of i is equal to 1 to n. OK. So if you look at this equation, what I want to do is find the w that maximizes this function. Well, look here. What is this? This was our loss function. And of course, what we did is we found the w that minimized this loss function. So what we're doing is exactly the same thing now. So if we had assumed that the data that was generated was through a process by which there was a true w and there was a true sigma squared and the relationship between the data that I saw and the true measurements were through this Gaussian process, then by minimizing a loss function, by finding a w that minimizes this, we also are maximizing this, which is a negative of it. So we're maximizing the likelihood of seeing the data by finding the w, by minimizing the determined side of that. So let's do this. So let's find the derivative of the log likelihood with respect to w. And so I can write this term here as my matrix way of writing it, xw transpose y minus xw. And so all right, that this becomes exactly as what we had before. The derivative of this with respect to w becomes minus 1 over 2 sigma squared, multiply it out. I don't remember the. So the derivative that with respect to w, it's going to be minus x transpose y minus x transpose y plus 2 x transpose xw. Said that equal to 0. And what we get is w is going to be equal to x transpose x minus 1 x transpose y. So this is the maximum likelihood estimate of w. What's the maximum likelihood estimate of sigma? Let's go back to our equation here, find the derivative now with respect to sigma and maximize the log likelihood. So the derivative of the log likelihood with respect to sigma is going to be equal to minus n over sigma. That's the derivative of this term with respect to sigma. And the derivative of the second term is going to be plus 1 over 2, plus 1 over sigma cubed times this sum, which is equal to 0. Sigma squared is going to be equal to 1 over n times the sum of yi minus w transpose xi squared. So this is the maximum likelihood estimate of the variance in the noise associated with the process that generated my data. This is the maximum likelihood estimate of the weights that generated that data. Any questions so far? You see what I did? First describe the likelihood of seeing your data, then find the parameters that maximizes that joint probability distribution. All right, let's try a different example. Let's try an example where we have the following scenario. Oh, I forgot to tell you. So we are now starting down the book. So all the stuff that I'm going to be talking about from the rest of the class will come from your book. And we are now in chapter 4.1. And we're going to cover today 4.1 to 4.5. So that's the material that I'm going to cover today. So here's the problem that I want you to consider.  But you also are worried about this thing being not accurate. So you bought another GPS. So you have two GPS's. Now one of them was purchased in the United States. So it uses US satellites. The other one was purchased in Europe. So it uses European satellites. So they're independent estimates of where you are. So now I want to set up the problem so that you determine where you are based on the readings that you have, based from those two GPS's. And the idea is to show you how to use maximum likelihood to basically give the best estimate that you can. So here's our problem. It's hiking in the woods problem. So you are at some location. Let's call that x. This is a 2 by 1 vector, some position in the world. You have two sensors. Call it A. For one of them, you have B for the other one. And we want to set up this problem so that with these two sensors, after you make your measurements, you make an estimate about where you are. So just give you some intuition about what this problem is going to be like. In some sense, you're going to be averaging these two sensors. Because one of them is going to tell you I'm over here. The other one tells me I'm over here. I'm probably somewhere in the middle. Now, the way you're going to average it depends on the uncertainty you have associated with each sensor, with meaning that the noise that there is in each sensor. So presumably, what you're going to do is that you're going to weigh the information that you get based on this noise. Whichever sensor is the more noisy one is going to get downgraded more. So if you're going to rely more on the sensor, that's going to be less noisy. So in describing our generative model, we're going to have to assume that we know the noises in this system.  And then you're going to make a decision about where am I so that I maximize the likelihood that these sensors are telling me what they are telling us. In a few lectures, I'm going to give you the example of what happens when one sensor tells you, I'm over here. The other one tells me that I'm over here. And if I do the two together, I'm going to end up in the middle of a river. So I know I'm not in the middle of a river. So then something is wrong. So in that case, it doesn't make sense for me to combine these two pieces of information. So I'm not going to believe the two. So those are scenarios that we're going to consider. But today, let's assume that we have the simplest case. We have two sensors that are going to tell us something about the world. Now, this is a biologically relevant thing. Because in your brain, you have the capacity to also rely on multiple sensors. You have things that you can touch, and you have things that you can see. And the two of them are going to tell you things about the world. And presumably, the way you combine them is by taking into account something about the quality of each sensor. And that's one of the papers that I want to show you at the end of the lecture, how they use this idea to demonstrate that when the brain made an estimate based on incorporating multiple sensory modalities, it used something like maximum likelihood. So that's where we're going. All right, so our problem is we're hiking in the woods. We're standing at some location. And we take out our two GPSs, our two GPSsis, I guess. And we're going to try to estimate where we are. So let's set up our problem. So I'm going to write my probability distributions as follows.  It's a place that I'm reading. It depends on these two variables, y a and y b. Each one is a 2 by 1 vector. So this is a 4 by 1 vector. This is one GPS. The other one is the second GPS. I'm going to write my relationship between what I measure and the true location where I'm at as follows, c times x plus epsilon. And what I mean by c is an identity 2 by 2, an identity 2 by 2. So I'm going to believe that what I'm measuring depends on where I'm actually standing. And that relationship is described by this matrix c, which is basically saying that two things are measuring the same. My two sensors are measuring the same position that I'm at. And they're unbiased plus some noise epsilon, which is going to be normally distributed. And it's going to have variance, which I'm going to call r. And my mean is 0. My variance is going to be r. And what this r is, is a matrix made up of two kinds of noises, r, a, and r, b, where these are the covariances associated with each sensor. So for example, one of these sensors is going to have noise a. One of them is going to have noise b. And each one of them is going to be a 2 by 2 matrix. So this is a 2 by 2. Let me stop. Make sure we understand what the problem is. Yeah. Is that mean r, b is a 2 by 2 matrix or r? Yeah, it's a 4 by 4. OK. This is a two-dimensional problem. That's all. All right. So let's write a likelihood. So what's the probability of y by a and yb given x? That's going to be a normal distribution with mean c times x and variance r. So what that looks like is 1 over square root of 2 pi raised to the power of 4 times the determinant of r times an exponential of minus 1 over 2 y minus cx r minus 1. OK. So that's my conditional probability that tells me the relationship that I'm looking for. This is the likelihood. So let's find x that maximizes this likelihood. So what is my position estimate? Where do you think you are? Well, that position is going to be my maximum likelihood estimate, and that's going to be the position that maximizes the two-sensor of measurements that I made. Find the x that maximizes the observation. OK. So before we were using x for something that we know was input, but don't get confused by that. So x is now what I want to find. I'm standing at some location. I have two measurements, y, a, y, b. Tell me the maximum likelihood estimate of your position x. So the way I'm going to find it is by finding x that maximizes this likelihood. Does that make sense? All right. To do that, I will find the log of this function. What is the log likelihood? It's going to be minus 2 log of 2 pi plus the determinant of r minus, OK. So let's find the x that maximizes likelihood. So to help us through, let me multiply these out.  So the derivative of that with respect to x, the derivative of the log likelihood with respect to x, is going to be minus c transpose r minus 1y minus c transpose r minus 1y plus 2 times c transpose r minus 1cx. So x is going to be equal to c transpose r minus 1c minus 1c transpose r minus 1y. So this is my maximum likelihood estimator where I am, which looks very similar to what we've seen with regard to a weighted least squares problem. But I want to give you some intuition about what these things are. So let's write out some of these matrices. Let's see what is c transpose r minus 1. So what is c transpose? So c we had was this identity matrix. So c transpose is identity, identity. r minus 1 is going to be rA minus 1, rB minus 1, 0, 0. So this is a 4 by 4 matrix. It's a symmetric matrix. When I invert it, I get the inversion of individual elements. So c transpose r minus 1 is going to be equal to rA minus 1, rB minus 1. And what's the term c transpose r minus 1c?  So the term on the right, c transpose r minus 1y, is going to be equal to rA minus 1 times yA plus rB minus 1yB. And so that whole term, x, is going to be equal to rA minus 1 plus rB minus 1 minus 1 times. So let me give you intuition about what we just did. So this is saying that your best estimate of where you are is to take your measurement, rA, weigh it by the inverse of the variance of that sensor, take the measurement, rB, weigh it by the inverse of the variance of that sensor, and then normalize it by the sum of the variances. Just like what we thought. So what we're going to do is that we're going to have two sensors. Each one is going to have a noise, rA and rB. When I want to combine the two pieces of information, the way I do it is by weighing each information by the uncertainty that I have in that information. That uncertainty is the noise in that measurement. So if the noise in one of the elements is rA, I weigh it by that. The other one is rB. I weigh it by that. And then you normalize it by the sum of the two things. So basically, we now have an estimate of maximum likelihood. So this is where I believe I'm at. Now, my belief about where I am isn't just the mean of it. So this is my mean. This is the expected value of where I am. What is the variance of my belief? So I have some belief about where I am. And now, because I've done this problem probabilistically, I not only have a mean, but I also have some uncertainty about my mean. Where is this location? And that would be the variance of my equation. So that variance is here. So this is my estimate of it. So I can find the variance of that equation. And the variance of that equation depends on the variance of y, which is going to look like this. C transpose r minus 1, C minus 1, C transpose r minus 1 times variance of y times that term transpose. C transpose r minus 1, C minus 1. So if I am just standing still, then variance of y, given that I'm not moving, if I look at the top equation, y is equal to Cx plus epsilon, if I'm not moving, so I know I'm not moving, then the only variance that I have is the variance in my reading. So this term is going to be r. If I multiply this, r's cancel out. I get C transpose r minus 1, C minus 1, C transpose times the transpose of this, which is going to be r minus 1, C times C transpose r minus 1, C minus 1, quantity transpose. This cancels out with this. I get C transpose r minus 1, C minus 1. And the transpose is irrelevant because this is a symmetric matrix, so that's the variance of my estimate. Now, why is this important? Let's see what this is. What's C transpose r minus 1, C? C transpose r minus 1, C is here, so that I can just write this out. That's rA minus 1 plus rB minus 1, quantity minus 1. So if you look at this, what this is telling you is that if you have one measurement that has variance rA, if you have another measurement that you have variance rB, if you combine the two measurements, your posterior, your end what you end up with is something better than both of those things. So you're going to end up with a variance in your belief that's smaller than either of those two measurements. That number there, rA minus 1 plus rB minus 1, inverted is smaller than either rA or rB. So two sensors are better than one. In what sense? They give you an uncertainty that's smaller than either of those two variances. And this is why having more sensors is good, as long as each one of them is unbiased, and you know the variance is associated with them. All right, so for example, if I had, if you go through this and draw up where you think the way things would look like, so say that you have an xy axis that describes where you think you are. And you know one of the sensors tells you, with this uncertainty, here's where I am, the other sensor tells you, with this uncertainty, here's where I am, where your maximum likelihood estimate is going to be some place in the middle, but not along that line. It's going to be down here. It's going to be here, like this. So not necessarily between the two means, because it depends on the shape of those distribution functions. The most likelihood, most likely location is going to be somewhere in between, not directly in the same line, with a distribution that's going to be tighter than either of those two. And that's going to be that distribution. So that distribution is shown here. Here's its mean, and here's its variance. That's your maximum likelihood estimate of where you are. Any questions? OK, let me give you an example now from biology. So in 2002 it was, there was a paper that appeared in Nature that began to use these concepts to describe how the brain incorporated data for multiple sensors. So I want you guys to understand that, so the last week, sorry, Monday, when we talked about estimating generalization, the paper that I described to you was a science paper that appeared in 1990, and then there was a Nature paper that appeared in the year 2000. They used the concept of generalization. So these things are, you can begin to see how this mathematics of basic statistics and learning are being now used to just try to understand how the brain does it. And so it results in very flashy papers that appeared not that long ago, 10 years ago, 11 years ago or so. So the reason why I think it's useful for you guys to know that is because you can begin to see, I hope, that this kind of thinking is beginning to make major inroads into understanding how the brain works. And that's why we have this class, so you guys can see things. This is not stuff from two centuries ago. This is from really just a decade ago as it's being used. So maximum likelihood has been around a long time. But it really wasn't until about 12 years ago when people began applying it to asking, does the nervous system incorporate decision processes that coincide with maximum likelihood? And the experiment was very simple. So experiment was that you're going to have a device that you're going to use to estimate, basically, to feel the size of something. So they give you a little robot that can determine something of a particular size.  And as you feel it haptically, they give you visual feedback on a monitor that displays what the size is.  And now your brain forms an estimate of the size of that object by using these two pieces of information. I saw it was this wide, this long, and I felt, and it felt this long. So the aperture of my hand is one sensor. Visually, I have another piece of information. Now, if you did this in a maximum likelihood way, what your brain would have done is to say, I have certain uncertainty about my haptic sense. There's some noise there. I have certain uncertainty about my visual sense, what I see. And you would think that your noise associated with your eyes is much less than the noise associated with what you feel. Generally, we're much better in visually estimating things than feeling our way. So I have two kinds of noises. I have noise associated with my visual sensors. I have noise associated with my haptic sensors. The experiment is just like this. I have a true size that I'm trying to estimate. Now, again, the caveat is if what they're showing you is tiny, but what you're feeling is really large, it's sort of like the problem of one sensor is telling me I'm on this side of the river, the other one is here, I'm not in the middle. So I'm just going to ignore one of them. But that's not the case here. These two things are very close to each other. So we're going to believe that the two sensors are really measuring the same thing. That's the important initial assumption. Our generative model is that these two sensors are telling me the same thing. They're measuring the same thing. But one sensor happens to be much more accurate than the other. Now, what they try to do in this experiment is ask, when you are making a decision about what size this thing is, are you combining these two pieces of information in a maximum likelihood way? So how can they do that? Well, first, they have to estimate the noise in each one of your sensors. So you have to know for Reza, when he's doing this experiment, what is the noise in his haptic sense? And then what is the noise in his visual sense? So this has to be done independently first. Then when I give it to you together, do you make an estimate about size based on this? The second problem is, these are distributions. But when I answer a question for you, I don't give you a distribution. I just tell you it's bigger than the other one. Or it's less than this one. So you're going to have to use my result from the way I describe it to you to infer what is the probability distribution that I'm talking about. And the technique for that are dependent on things called psychometric functions and what I want to show you. So mathematically, we have a framework to describe the problem. But we still have the experimental problem of how do we actually go measure somebody's uncertainty about what the size is? How do we measure the noise in a particular sensor? So that's where we're going. All right. So let me show you their experiment. And this experiment is, as I said, the material that I'm talking about from here on is going to be the material in your book. And this material is going to be covered in chapter four, which I suggest you read. So our generative problem is as follows. There is some height that we're trying to estimate through vision and through haptics. And so as before, y is equal to Cx plus epsilon. And in this case, epsilon is normal, 0r. And r now is going to be a 2 by 2 matrix, sigma squared h, 0, 0 sigma squared v. It's a one-dimensional problem. And I have a variance associated with my sensors, sigma squared h, and a variance associated with my vision, sigma squared v. C is the 1, 1 vector, OK? So y, my measurement, is a 2 by 1, my haptics and my vision. So if I do the maximum likelihood estimate, what I will end up with is the following.  So that's my normalized relationship. This is my best estimate of where I am, where I take my measurement from my haptics, I weigh it by the inverse of the noise, I take my measurement in vision, I weigh it by the inverse of the noise, and I normalize it by the sum of those inverses. And my variance, my uncertainty about x is going to be this, 1 over the denominator of that function. So to give you a sense of what this means, if you had haptically estimate that said you're here, so this is the mean of your haptic sensor measurements, and then you had your vision tell you you're here, that's the mean of your visual sense, and say that in this case, we have a person who has really bad vision, so in this case, sigma squared v is equal to sigma squared h. You see these two distributions have the same width. Then your maximum likelihood estimate is going to be right down the middle with a variance that's going to be smaller. So this is x maximum likelihood, and your variance is going to be this, this quantity. On the other hand, most of us have a scenario like this. I have some belief about what I feel. I have a much sharper estimate of what I see, so sigma squared v is much smaller than sigma squared h, in which case then my maximum likelihood estimate is going to look like this. It's going to be much closer to the visual estimate. Does that make sense? You believe one sensor much more, and you notice that my Gaussians are narrower than either of those two estimates, and the probabilities are, of course, higher, because the integral of those Gaussians is always one. All right, so how do they measure the noise in each sensor? Let me introduce to you the concept of a psychometric function. So the way they're going to do it is that they're going to give you a standard size object, say 5 centimeter, and say they want to figure out the noise in your haptic measurement. You're going to feel that 5 centimeter object, then they're going to give you a second object, bigger, and they're going to ask you, is this bigger than the first one or not, and you're going to tell them something, and then they're going to give you one a little bit smaller. Is this bigger than the first one or not, and you're going to tell them yes or no. This is the kind of process they're going to go through. So they're going to give you two objects, one, I'm going to call it Y1. Say that we're only talking about one sensor now, just they're going to try to measure the noise in only your haptic sense. So say that Y1, the first time that you say it, is a normally distributed with size mu1 and variance sigma squared. Let's call it mu, a variance sigma squared. This is going to be your haptic sense. And you're going to give you another object, Y2, is also normally distributed with mu plus delta. It's going to be a little bit of a different size with the same variance. And so when you compare these two, what you're going to estimate is this thing delta, which is going to be just Y2 minus Y1. And that's going to be a random variable, delta hat is going to be a random variable with mean delta and variance two sigma squared. So, and this sigma squared is your haptic noise. And we want to know, well, you know, something about it. How would we be able to estimate it? Well, you're going to tell us whether the second object was bigger than the first object. So what's the probability of Y2 being bigger than Y1? Let's see if we can figure this out. So suppose that delta is two centimeters. So this is one centimeter, this is two centimeters. So the second object is two centimeters bigger. And you have this Gaussian here now that's going to be centered on two centimeters. So you're going to be able to estimate that, you know, the second object was bigger than the first one. But what's going to be the probability that you're going to detect the second object as being bigger than the first one? Well, that's the integral of this probability distribution above zero, because if the integral, that probability distribution is above zero, then you're going to say the second object is bigger. If the integral of it is, if the smaller that integral becomes from here on, bigger than zero, the smaller is going to be the probability of you saying that the second object is bigger. So if my sigma is, so this is, that's called a sigma, sigma in a condition where, you know, I have a pretty good sensor. If I have a bad sensor, here's what it's going to look like. So this is a good sensor, meaning that it has low noise. Here's where it has large noise. So you notice now that this integral is going to be smaller for, I'm going to have more of my function over here to the left when I have large noise than if I have small noise. So the probability of detecting that difference is going to become smaller. So now if I plot the probability of detecting the second object being bigger than the first object, so this is probability of y2 being bigger than y1, as a function of delta, so here's one, here's two, here's minus one, here's minus two. So the second object is bigger than the first object by two centimeters, by one centimeter, it's actually smaller, it's really small. The probability is going to be the integral of that function is going to look like this. So if it turns out that the first and the second object are exactly the same size, then of course with probability one-half, I can tell you that the second object is bigger than one. If the second object is two centimeters, I'm pretty darn sure that the second object is bigger, but the noise in my sensor tells you the steepness of this function. So if my noise is really not very, is a terrible noise, I have a lot of noise, so I have large noise, then my ability to detect the second object is not going to be nearly as good as if I have small noise. And this is called a psychometric function. So you can calculate the noise that there is in each sensor by looking at the probability of detecting something being bigger or smaller than a standard size object. And so again, this is large noise, this is small noise, sorry, I'm using the same color for both of them. Does it make sense? Yeah. Probability. This one here? Also probability. This is the distribution that I'm plotting, yeah. So this number is the integral of zero to this side. So if that function is centered at zero, then half of it is to the right, half of it is to the left, right? If the function is moving to the right, then a larger part is gonna be to the right. But the amount that it's gonna grow depends on if I have small noise or large noise. So if I have large noise, this integral is gonna have a whole bunch of it to the left, right? So that means that it's not gonna be nearly as big as if it's a small noise. Okay, so that's how they did it. They asked people to detect the difference between a small object, a standard object, and another object, the second one. And they did it repeatedly on the same person, so then they basically came up with the probability of the, you know, the probability they could detect a one centimeter difference, probability of detecting a two centimeter difference. And they fitted these sigmoids. And from those sigmoids, they computed the variance associated with the normal distribution that generated that data. So the left side is the integral of the right side. And by doing so, they computed the noises. Then what they did is that, okay, they did the same thing for vision, detecting a small object and a slightly bigger one. How well could they do it? They had the noise in the two sensors. Next what they did is they said, all right, now you can see this object, you can feel this object, this is the same object. How, what's your belief about the size of that object? And what they showed was that their belief was a combination of these two sensors where each sensor was weighted by the variance of that sensor, the confidence that they had. That was the maximum likelihood estimate. Any questions? Okay, see you Monday.


Everybody welcome to another class on non-linear control, yeah SC602. So I believe we have already looked at more or less the entire analysis, entire breadth of whatever is the analysis that we intend to look at. Of course there are more courses on pure analysis like I said analysis itself is a very big area, but I personally do not prefer to go into and spend so much time in the analysis that we are not doing any design. So we have already taken one step in the form of control Lyapunov functions to get into the design. So now what we want to do is we want to continue getting into the design and today we will sort of look at the first formal design method, okay. So CLF was of course a tool which is used for design, yeah, but that is not a formal method for design itself, okay. So starting today, so pretty much as expected you know at the end of the you know midterm and then beginning you know just after the midterm we typically target that we start with the design aspects of things, okay. So that is the idea, all right. So we are today going to start with back stepping and I am going to write a little bit today. The note that I am using is by Daniel Lieber on and it is I believe it is his free version of his book Nonlinear and Adaptive Control, okay. So his book on nonlinear adaptive control, this is what we are using as a reference right now. So there is an interesting history, so I just want to put in a little bit of the history, historical context here. So the idea of back stepping was sort of seen first in Russian literature. Interestingly a lot of controls related work has been in pretty much Russian literature, yeah. So it is not such a big surprise, A.M. Lyapunov himself also Russian. So yeah, so a lot of work in control literature you will see is in Russian language, Russian literature. So this back stepping also is from 1978, a Maillac's Russian literature and then it was sort of extended I mean in the US which is I mean slightly later by these folks and finally came into the book of KKK, yeah. I think this is I have already mentioned this book, yeah.  So this is Kanalakopolos-Kristik-Kokotovish, this is sort of my standard text reference for adaptive control, very difficult to say, so if you can say it really fast. So I usually just call it the KKK book, so this is the one of our standard texts for you know for adaptive control and these are the guys who introduced the terminology, yeah. They are the ones who introduced the terminology back stepping, yeah. So in research sometimes more than the folks who discovered it, folks who name the method are way more popular, okay. So remember this, coming up with good names for the methods is also important, okay. So somehow back stepping as the name stuck and you will see why. The idea of back stepping is primarily a way of constructing these control Lyapunov functions, okay. So basically I would say that back stepping and this I want you to remember a way to design CLFs, okay. So you already know the power of CLFs, right. If you have a CLF you know that there exists a controller, you know I mean CLF and some small control, some nice properties you know that there exists a smooth, almost smooth controller and not just the existence of an almost smooth controller you also have a formula, yeah. You have the Sontag's universal formula, okay which essentially gives you this very nice way of doing control design, okay. So very simple, very standard. So back stepping is a means of constructing a CLF and this sort of helps a lot of people because a lot of folks come by and ask me how to construct Lyapunov functions. So this is one of the simplest methods, okay. This is one of the simplest methods where you will see how step by step you can construct a Lyapunov function. Of course it has its own limitations, there is no general solution to anything, yeah. You will also see that, okay. But where we will start is with the notion of integrator back stepping. Why folks like Kristik, one of the authors of this book picked up this back stepping notions is because it is very powerful as a tool, not just in control, nonlinear control but also in adaptive control, okay. A lot of adaptive control, modern adaptive control results are based on back stepping, okay. So this is actually a very powerful tool. If you can master this well, you will be able to do a lot of nonlinear control design for a lot of different systems, okay. And we will try to see, you know, a couple of examples and then of course, you know, more and more to follow in homeworks and things like that, all right, okay. So what is integrator back stepping? I start with the system, x dot is fx plus gx u where I say x is in Rm and u is in Rm. So basically gx is typically of the form g1x all the way to gmx, right. So this is just a way of writing fx plus summation of giui, yeah. The way we have been writing control affine systems, okay. So we always now go along with control affine systems, yeah. We are not trying to, you know, have some kind of nonlinear dependence on control because it is usual, okay. In most cases, you can always figure out a linear dependence with the control, okay, all right. So now as always we, you know, we assume a few things. One is that there exists a control Lyapunov function v0x for above, yeah. You might say that, oh, okay. You have already made life too easy that you already are assuming a CLF, okay. We will see how, you know, we will have simple solutions there. There exists a smooth control law u equal to k0 of x which stabilizes above, okay. So we are assuming a, obviously it should be evident to you that if I already have a v0 then the k0 is obvious, right. So in fact, if you want to be, you know, more precise you can say you just have a almost smooth control law but I am making my life easy by assuming that I have a smooth control law, okay. So I have not just a v0 which is a CLF but I have some, I also have a smooth controller available to me for this system, okay. And of course we are assuming that all the nice properties that v0, f, g, k0 are smooth, okay. All these things of course give me a lot of nice new properties, okay. So I make my life significantly easy, okay. k00 is equal to 0, k0 satisfies v0 times fx plus del v0 del x times gx is thus then equal to minus wx. So basically w is positive, yeah. So why I am specifying the third point although this is the definition of, seems like the definition of the CLF itself, why I am specifying it is I am sort of seeing that this, with this control this sort of negative definiteness is obtained, okay. Typically if you have a CLF it means that you have Lf v0 plus Lg v0 times control is anyway negative definite, right. But in this case I am seeing with this particular control this is negative definite. So if you want to sort of think of it sequentially you can just use the first step and use the universal controller to obtain this k0. And then this is obvious, right. So with v0 itself using the universal controller I can obtain a k0 and with that this step is this is anyway obvious because this is the definition, yeah. But because I said separately that there exists a control that is why I am actually writing it out, okay. There is nothing special about it. So these three things anyway for safety it is always good to write out all the assumptions. Somebody might always say ah you assume too much it is already obvious that is okay. The problem typically comes up if you do not assume whatever you need, okay. If you under assume that is a problem. If you assume more it is okay. You know you are still giving a result may be a little bit more conservative but that is still fine, okay. But like I said you can always start with a v0 and then you can go on with the universal control formula to get a k0 and from there this is evident. It does not need to be written out separately. So only this is required, okay. Alright, great. Now that we have these assumptions. So we already know that this system that you have is already stabilized, okay. It is in fact asymptotically stable, right. The way we understand asymptotic stability because w is positive definite, sorry because v0 is positive definite, right. That is the requirement of a CLF. So v0 is already positive definite and its partial and its directional derivative is negative definite. So we have asymptotic stability by Lyapunov theorem, alright. So great. Now the question arises what happens if I add an integrator, okay. So what do I mean by adding an integrator? I am looking at this new system now which is x dot is f of x plus g of x psi and psi dot is now the control, okay. It looks like the same dynamics, right. The first system looks very similar but the control is now being replaced by a state and then the derivative of that state is actually the control, okay. This structure is very common for most aeromechanical systems, okay. Because just think double integrator. All double integrators have this sort of a structure, x1 dot is x2, x2 dot is u, okay. So already very close to a mechanical system. So spring mass damper systems already very close to this.  So again you have a similar structure, okay. So most aeromechanical systems have this kind of a structure, okay. And therefore there is great value in studying this structure, okay. So just a second. Yes, tell me what are you saying? No, I mean all I did was you had this system, right. Yeah. So all I said was I do not have the control here anymore but I have the control in the next stage, okay. So that is there is a state here instead of the control and then the derivative of the state is this. Like I said it is not like we get this from anything. This is actually how back stepping works by working over layers of integrators and the motivation is aeromechanical systems. Most aeromechanical systems will have this kind of an integrator structure, okay. We will see some examples. It is not difficult to see, yeah. That you have this kind of a structure. So the idea is if you have some nice result for this top system, you can extend that nice result to this entire system. That is the whole idea. Basically what we will do is we will construct a CLF for this system, okay. We already have a CLF for this and we will use this to construct a CLF for this entire system. That is the simple plan, alright. So what is it? I will also write the aim construct CLF for above using V0, X and K0X, okay, alright. So I have been given a V0 and a K0 for the original system and I want to construct a CLF for this new system using that. Now the simple logic here. I earlier had this if I could ensure that this quantity psi was K0X, right. Then I have my original system which is already asymptotically stable. I am in good shape, okay. So basically the logic is if psi is identically equal to K0X, then done, right. Because this is actually a stable system now. So X actually converges to 0, great, okay. And I already know not just that X converges to 0, I also know that K0 of 0 is 0. So basically if X goes to 0, psi also goes to 0. Because if psi is K0X and X goes to 0, then K0 of X also goes to 0, right. So I have obtained my stabilization like I want, okay. Now the problem is psi is a state of a system. It is not a control that I can specify. I cannot actually ensure that psi is exactly equal to K0X, right. I cannot just say that psi is I will give you a function of the state and my new state will just follow this function of state. That is impossible, okay. So we do the next best thing. What do we do? We create an error, okay. By the way notice that just for us to be able to write this psi has to be the same dimension as the control, okay. I hope this is obvious, right. Because I wrote psi dot is u. So if the psi, the new state has to be same dimension as the control. If not, cannot use backsticking, okay. This is the restriction of backsticking, okay. This dimensionality has to be maintained. Again for aeromechanical systems works out well, yeah. Usually there is position and velocity as states and the velocity derivative is the control. So position and velocity is the same number of dimensions usually, right. Because if you are in three-dimension, three-dimensional position, three-dimensional velocities. Two-dimension, two-dimensional position, two-dimensional velocities. So works very well for aeromechanical systems, yeah. For electrical biological systems may not be so easy, yeah. Because here the physical degrees of freedom restricts how many states you will have, yeah. For electrical systems and biological systems that may not be the case, right. There is no notion of, I mean, there may be an equivalent notion but there is no obvious notion of degrees of freedom, okay, alright. So we cannot make sure that psi is actually equal to k0x. But what do we do? We try to drive this state to 0, okay. So try that this happens and how do I try that? I construct a new v of x and psi which is v0x plus half norm of psi minus k0x squared, okay, alright. Basically see what do I mean? I am basically using the logic that folks in control just want to drive things to 0, yeah. So I am trying to drive something. Right now my original requirement was that I drive x and psi to 0, okay. But I knew from my previous knowledge that if psi is actually equal to k0x then I drive x to 0 and in fact if x goes to 0 psi also goes to 0. Since this is not possible I try to make the difference go to 0, yeah. I try to make this error go to 0, yeah. This is not like tracking. Tracking is very different from this. Do not think of this as tracking. In tracking I would try to find the difference with a function of time, okay. For example when I am looking at robot tracking a path the path is specified as a function of time, okay. Or a drone trying to fly a path or a trajectory. Trajectory is specified as a function of time, okay. This is very different. Here I am trying to make the error with a state and another function of state to 0, okay. This is only done in back stepping and nowhere else, okay. This is an unusual thing actually, yeah. We will get to situations and lot of geometers do not like this idea, yeah. Because if you think about again the first state as position states, second state as velocity states, okay. So back stepping requires a comparison between the velocity state and the position state, right. You are somehow subtracting or adding velocity and some function of position. So to folks in geometry and topology and stuff they freak out because they say how can you add velocity and position? What does it even mean, okay. So it is a pretty valid question, right. If I ask you to add position and velocity say this function was nothing but minus of the position, okay. Then you are essentially adding position and velocity. Then the question is what space are you even working in, right. So it is very difficult to wrap for them to wrap their heads around it, yeah. For us who are basically more like applying nonlinear control for us it is just a tool, okay. This variable especially when you are working in Euclidean space everything is Euclidean, right. Everything is just in RNRM and so on. So I can freely add them, no problem. It is just another variable in Euclidean space. Problems happen when you are working with something that is like a not a linear space, not Euclidean so something like a manifold. Then the position and velocity are not in the same space anymore, okay. Position is typically on the manifold for those who have seen anything in geometry they will know. Position is in manifold and velocity is in tangent manifold, okay. Or the tangent bundle or tangent space, yeah. So the tangent velocities are actually linear space. Positions are in a manifold, yeah. It is very interesting. This is why geometry is in general interesting and you are looking at. So this is all natural when you are looking at rotations and so on. Rotation is on a manifold.  So they are actually sitting on a circle, right. Not unlike what we think they are not on the straight line. Angle is not from minus infinity to infinity, right. Because you just repeat, right. If you think of it as minus infinity infinity you are just cheating yourself because you are actually just counting the repetitions also, okay. So they are actually on a circle. Similarly, if you go to three dimensional rotations they are on the SO3 another manifold, alright. So for geometry is difficult to digest but you can make sense of it. Here in Euclidean spaces for us it is very easy to make sense. You are just adding two variables in Euclidean space. It is just giving us some other variable. We do not care if it is making sense or not, okay. It is just for the purpose of analysis, alright. Eventually after all this monologue the point is that this I claim is a CLF. Claim above V is a CLF for x psi system, okay. So this is the claim, alright. Okay.


So, starting out, this is the, um, okay, working group meeting for November 9th of 2021. Don't forget to put your name in the attendees list. So we know here and, uh, we'll do a quick agenda reviews, everything in the agenda, anything that people want to change your ad. While we've got it up before we get started. Going once. Going twice. All right, and let's jump into it with release updates with the dean. Um, first of all, can you hear me? Is my mic is kind of weird? Yes, we can hear you quite clearly or I can. Um, so this weekend, we were supposed to have a 40 to release, but it's kind of postponed because. Uh, the rebase to 121.5 of Kubernetes and Kubelet and things didn't land just yet. So it's going to land today. Uh, so we kind of skip this release because first of all, we didn't have much to fix and for it. Uh, second issue on the table is, uh, for nine. For nine, at least already, uh, for at least the testing and we would need to switch to for nine stable eventually. Um, I think the safest way would be releasing for eight. This weekend. And releasing for nine is stable the weekend after. That gives us quite enough time. To test for nine at least that also approximately at this time. OCP upgrades would be allowed from for eight to 49. I think they are allowed now, but probably blocked by something at least. Um, we don't see a huge volume of that. When it starts happening, we would have quite a good results of the health of for nine. And, uh, we'll get you safe to move. Uh, next on, uh, Mustafa has fixed the storage test. In, uh, communities, all cherry picks have landed in. Master 122 and 121 that the branches we care about basically. They didn't land in. Micro releases yet, but once they do. Um, open shift with rebates on top of them, we'll get these fixes and we would finally be able to. And then those Linux policy. Uh, package, which has been causing us quite a lot of problems. Rather, it's been way to secure. And, um, another problem has been found in for 10. Rather, it affects. Technically it affects all the installations, but it's most visible on for 10 because, um, AWS M six families using. Uh, Nitro, uh, hypervisor and it's crashing on latest kernel. Five 14. So you won't be able to launch latest door across. Uh, once you launch, uh, for nine or for eight. Using. This it would crash and for M six, but, uh, and five would work. And unfortunately for 10 has moved to M six by default, because these machines are actually cheaper. And so far the mitigation is that we rolled back the kernel to five 14. You'd still have to use an older for the request due to this issue and some more previous ones, but in the end it starts correctly. So that's what we're testing right now. Um, it's, uh, it appears to be an actual either AWS issue or upstream. We're doing most likely upstream kernel issues. So not much we can do until we wait for the fixed land. Um, I believe the, the interested parties and notified identified all of the upstream bug report. That's what we'll do. Um, if this issue won't get resolved. Um, GCV is known to be working fine, but it appears to be affected. At least AWS and VCR. I don't think it would be, um, would be affected. Um, yeah, that's pretty much, that's pretty much it. I don't think there's been a new changes. But again, um, the door across testing develop has moved to 35 and would really be interested to try this out. But the kernel problem is, uh, at this point looking at once we would push it away and would be able to fix the saloon. Policy pin. We would release for 10 to 35 and check out the results. Yeah, I think that's all I've got. Any questions or comments for Vadim and the. Okay, the updates video release updates. All right, seeing nothing we'll move on to the, uh, F cause updates with Timothy. Yes, so hi everyone. Um, so I've got two items today. The first one is that, uh, with federal certified released, we are now moving to federal certified for federal chorus. So I've put the link into the document with a link to the tracker of where we are right now. And so next has been on federal certified for a while now. Uh, testing is either just right now or just going to be in federal certified in the next one. I don't remember. And then we move on to next to stable in two weeks. So testing in is not right now. So it should be in. Yes, right in the next release, which so with the. With the base with the major releases we've reached to a new way of doing the release. This is the way so we kind of accelerate the pace. So the documentation somewhere link in the ticket if I can find again. So that we do the updates a little bit faster during the move to federal certified. But yeah, so essentially it's coming really, really soon to a testing release near you and then two weeks next weeks after to a stable release. There are three that will help. Okay. D a lot because a lot of the hacks you currently have with. You will be able to drop them. So that should have things. Um, yeah. That's on the front of the, on this point. And then we have a second one, which is kind of like an update of what we discussed next previous time. About the previous time was two times ago. Where we're working on creating the way to quickly use for chorus systems. So, like for federal chorus, you could have your base system and then overlay on top much more in a much more easier way all the packages you need. For example, for OKD and the instruments is posted there. I have put the link. The progress. We're progressing quite rapidly with that and support so should should be available shortly, at least for testing it out until it's fully integrated into the MCO and everything.  And yeah, essentially right now I think we've got like the ability to do local. Tools from containers and clean layers and everything so like the basics to work and hopefully this will like really help for OKD. To with with issues and everything avoid the rebuilds and enable. Overriding packages when needed, overriding things, changing config defaults and everything that essentially you just use the base for the first image and then do the tweak some stuff. And yeah, and that's the two main items for me today. For this week. Any questions. The Fedora cordless side of the house questions or comments. Excellent. Moving on to the docs updates with Brian and 3t. Actually, she's on so it might just be you. Okay. And yeah, so the, the new documentation system is now live. So if you go look it looks a little bit different. And it's now based on MK docs or it's pure mockdown. So pretty easy to update if you want to. I've now put the instructions for if you want to go and update the docs and now they do it within the container. And if you've got a chair or already workspaces on a cluster, you can edit it in the cluster or you can install local tooling. So that's all set up. And let me know if you need any help or any issues, but that's all good to go. And there is one issue that I think might be worth talking in this group. And, and it's an issue raised by Sandra, I know Sandra was on so I don't know whether you want to actually pick this up and talk about it. I stumbled upon a CVE report for OpenShift virtualization last week. And I was wondering if there is any anyone tracking CVE that are also affecting OKD from coming from F course or coming from OpenShift or any other layer that we are building upon. So I looked for the website searching for a security contact and couldn't find one. I saw there is a security policy for OKD itself in the GitHub code repo of OKD pointing to Reddit security, but I'm wondering if it's someone is actively looking at that. Don't just raising the topic. Yeah, no, that's a great, great question. We don't currently have anyone volunteering to scan the security related security issues. I think that's a great idea. Is there anyone in current anyone currently in attendance that would be interested in doing that legwork like every two weeks or so basically looking for stuff that would impact OKD so that we can let users know that we are aware of these things. And if it's something upstream, then we can say, hey, we're waiting for this fix upstream. If it's something that we can mitigate in the few cases where we can, we can talk about that. So I don't want to be our security liaison. Maybe something you want to put out on the Google. Yeah, I think we need someone. I think that's a great, it would be a great thing to have someone doing that legwork for us. If nothing else, you know, as far as the open shift code base itself goes, because it's the same, it's the same images, same code base. To CVE really could track from the red hat site, Fedora, Cora OS on the other, that's a different matter. There are likely being reported to Fedora, but we don't have that. We don't have anything that's tying those together. There's one exception to that, Cora. Occasionally, we are not occasionally like, you know, semi regularly, we have CVEs come in that we don't disclose through bugzilla. So they're private bugs that only red headers could see, but they might affect pieces of open shift that those CVEs would not be discoverable by people outside of red hat. And I think even for red headers, there's probably some document we signed that says we won't expose those things. So I think that's probably the one area where this might be difficult. Yeah, that is true. Yeah. So on the point of contact, I'm looking for the exact URLs and things, but essentially for all Cora 3 pose. We have this one security policy on GitHub, which I think we can apply to OKD, which by default says contact red hat security team. And we'll take care of that. And yeah, I'm looking for the exact link where this is stored and all this is set up, but I don't remember and I'll have to find that again. Yeah, that seems like a good default like defaulting to giving the red hat, you know, address this one. That seems like a good place to push people towards. Is that something red hat is okay receiving in that way. So speaking for federal chorus at least we perfectly fine receiving that this way and I think it would should be perfectly fine for OKD too. I mean, yeah, I concur with what Timothy saying pretty much. Considering that OKD and OCP are actually identical. I hope they're okay with it because if they're not, then there's a problem. A bigger problem, right? All right, well, let's what we'll do is as a task will add the docs group to actually come up with a security page. That sort of outlines our security posture. And then we'll provide those 2 links for people to be able to submit and get info in terms of F costs and in terms of OKD. And we'll go from there and I'll send something out over the working group for security liaison. Anything else on the topic of security, Sandra does that does that get us closer to what you're thinking? Yeah, that makes sense. So on the topic too, we have this one. So this is like the row list of all CVs or all adversaries for that OCP. So this one probably also applies to OKD in a similar manner. Considering that you will have to remove all the real specific ones and add all the federal chorus ones in there. But essentially, it might be a good place to start. And this one should be full as far as I know, like fully accurate. Thank you. Anyone else have any thoughts on this in terms of getting some of the volunteer security liaison or any of the documentation, anything else we would we could add to round it out. Okay, then, Brian, you want to continue on with. I guess we'd be a code of conduct and then Twitter and then the survey. Yes, so we're looking at adding the code of conduct. We're taking from the Ansible repo. Michael is adapting that. And we're going to actually use that as a code of conduct. And I think the idea would be that we could refer to it when we start a community event or a meeting. We can actually refer to that and it'll be actually on the dock side as well. And I think as far as Twitter is going, we're looking to set up a Twitter account and really was going to actually set it up. So we didn't get a we didn't come up with a good background image yet. Like I think we're using just something kind of she's been handling all this. She's not here. But yeah, it's it's up. We just need to start deciding on content. And so that we've got the one background image that was sort of pulled from like the website. I think it was like just grabbing the website. But we might be able to come up with something that looks a little bit nicer. It's a little bit fuzzy. Yeah, so she did come up with that. And I guess the docs group or this group, if you have any thoughts on what type of things we should be putting out over this. Like do we want to put every release that comes out, you know, Vadim, does that make sense for every single one? That's an official release that we, you know, announce that through the Twitter, announce the meetings, things like that. Is there anything else that we would want to put through the Twitter and the docs group? Of course, we'll deal with this as well. All right, we'll make sure you follow it. And I think we'll probably have it start following other people to sort of get that, you know, read going. And we will start using it and then folks can forward it. That's the other thing is for social media like this to be successful. So folks need to forward and reshare social media. So in this case, if you could retweet the OKD tweets when they start coming out, that would be super helpful because then that builds up, you know, that social media grid and makes those connections and stuff like that. And survey, Drity hasn't gotten much farther on the survey, but it's, but she's taken that on and we'll know more about it. I'll touch base right now. It's like 11 PM or getting close to 11 PM there, but I'll reach out to her and see where things are. And CRC stuff, Charles, take it away. A couple of free moments. So I built a 4.8 version and stood up. That's about all I know that's happening with CRC. Are you getting any feedback from it? Has anyone else played with it? Somebody tried to use the an earlier version of the Linux one that had a the upstream group made some changes. I had to I had to make a patch and push a new version. But other than that, I don't know how much use it gets guys using or not. I don't. I actually don't use code for any containers. Yeah, I see it mentioned occasionally the OKD variant. But it's like maybe once every couple of weeks, I'll see someone talk about it on a mailing list. That's about it. Do we have any any sort of stats in terms of visitors or downloads? Does anyone have access to the account that could pull that off? It's it's just a URL hanging off of the OKD.io. And we're using some borrowed space on one of the Fedora servers. So that's speak of. Yeah, doesn't seem like it's easily. Neil, do you happen to know who in Fedora gets logs for that infrastructure? Do you even know what infrastructure that is? So dl.fedoraproject.org, which is where it is is the download master server DL IB 01, which is in the ID data center in Virginia. That is managed by Fedora infrastructure specifically by the release engineering team. So the access to that is controlled exclusively by Red Haters. So someone on the Red Hat side will have to talk to the CPE team. For those who don't know, that's community platform engineering to talk to the Fedora infrastructure subgroup of CPE about if we if there are any stats, if not, if we can start collecting any. Justy might know who to talk to. I think he's the one that got me access to it. Yeah, probably. I can do a Fedora project. Can we put that as a to do for either one of you to hunt that down so to see if we can get some stats on on downloads or anything. Even remotely letting us know, I mean, we'll have the survey, obviously. Sorry, we're the two people. So, well, three of you any of the three of you, yes, absolutely. Are any of the three of us Red Haters. Okay, cool. So let's start with you, Charles, since you might be the best at figuring out from the Red Hat side, who has access to that. And if you already have access to it, you may have log access to. Or if you want to decide to delegate that to one of the rest of us, and you can get us access to do that, that's fine too. You know, I got access to this machine before I went to work for Red Hat. Yeah, this was back. Oh, goodness, when did we I guess it was last summer. It was before that when we when we pushed the first. Maybe it was, I guess it was 20 early 2020. Yeah, because the actual okay before release was late summer of last year. No, it's been out longer than that. Nope. You're thinking about the COVID year that everybody lost. I've been doing that all the I've been I've been conflating 2019 and 2020. Yeah, I'm talking about the COVID year. So, does it make sense to spin off the CRC stuff into a separate meeting? I noticed that's been added as an item. Can someone take that on so that maybe the CRC folks can. I had volunteered to organize the meeting last time and I was actually waiting on you, Jamie for instructions on organize said meeting that didn't actually make it into the task list last time. So that's that's my bad. I put it in the task list this time. Okay, great. Yes. Okay. If I if I don't hear from you in like a week. Should I email you slack you? What is the best way you can slack me. You can smack me. You can smack me. You can smoke a machine, whatever, whatever it is to get my to be honest. I was busy with enough other things that I wasn't like itching to do this. So that's why I'm up, but okay, cool. It's on the task list now. So hopefully we'll close that loop. And I apologize at my day job. I'm pushing a project to production. The past 2 weeks and it's just been fun and excitement. So yes, I totally understand the information and then we will go from there. Okay, anything else on CRC that the that the whole group needs to hear about. Um, so I think the main 2 things that we'd want from a from a CRC. Subgroup to start with based on the conversations we've had here are. Um, discussions about automated builds and the info for them. And then the larger existential question of does anyone use this. And if not, is it worth. Should it even exist? Are there other are there other agenda items that people think are important for a CRC subgroup to go off and figure out. And if not, that's fine. I just wanted to ask the larger group before we broke off to make sure that we weren't missing key things. Yeah, Daniel, whether or not people are using it. And whether or not people would use it more with different configurations or sort of 2 separate questions. Sure. Um, yeah, does anyone use this and is it is it worth doing or 2 separate questions. Thank you. Because like, if you if you look at say mini cube or something like that. As a, an alternative, which doesn't have the same usability. And you look at the resources that that takes.  So much more compact. Well, exactly. It was a lot easier to cluster up. It was definitely a lot easier to run mini shift. Okay, so, yeah, so that's that's that's a good question to answer. Besides a metric for anyone using it. Um, the, the, is there a configuration of it that would make it useful for people to kind of figuring out what the direction of it should be. Okay, cool. Um, that's enough of an agenda to get started. I just wanted to make sure that. That the things people wanted that committee or that some committee to do were actually going to be done. Yeah, yeah, and Mike, your point is valid. It really is. It runs better when it's running on it. I was just thinking like, this is like, it would be really cool if you could take a CRC. And like run it in such a way where it actually installed the stuff on the host machine instead of trying to create like, you know, a virtual machine to do it in and then like. And then you could have a situation where you could just take a bare metal host or just a bear host, run some sort of CRC installer and have it actually deploy to fill the resources on that machine. Like, I think that would be really interesting use case kind of going back to the days of OC cluster up kind of behaviors. Yeah, yeah, absolutely. Cool. Okay. So that that's, um, that's a bunch of good stuff that we can that we can cover in that in that meeting and I'm just saying if you're looking for big fish to fry, you know, I don't know. I don't know what I'm looking for. I don't know what I'm getting myself into but that that looks like a nice big fish there. Yeah. Maybe you'll boy get around on one of these. That's what you want. All right, let's move on to any are there any issues in the repo that are something that came to our attention that we need to pay any attention to. There are those kernel crashes. Vadim mentioned that most like all of the other issues are very old. So not a lot of issues. Of course, Kai. Ramell posted something that came up that looks like there is a must gather that he posted about 7 hours ago. That's someone that we may want to court actually for the working group. He's always he's actually running OKD in production and is always posting stuff where he's finding bugs and whatnot. Maybe I'll start a conversation with him and see if he's interested because it'd be cool to have him sort of contributing or at least coming on every once in a while. We definitely want to hit him up with the survey. Anything else in the issues section of the repo that you want to bring to our attention or talk about? Okay, is there anything in the discussion section that folks want to talk about? Someone mentioned the CRC OKD mismatch. Packaging OKD binaries for Fedora. I did not see that one come in. Oh, yeah, I did. Yeah, yeah, yeah. Mike Rocheford is not here and he's yeah, I just noticed that. Yeah, that was Mike's line. All right. Well, we will shelve that one to until the next meeting. Anything else in the discussion items there? Generic certificates for old OpenShift origin. Yeah, I don't see anything. All right, let's move on. A lot of the discussion stuff it seems to be like just a user question to resolve here. Location of the main repo Diane is going to check with legal. I need to actually create a discussion item. So for tasks moving forward, and if folks could remind me or help me with this, the goal is to create discussion items for every task. And then this way we can actually track that people are getting their tasks done and folks can talk about tasks and things like that and follow up on them. So my goal is starting in the next week or so to start creating discussion items for things that don't already have discussion items related to tasks. So Diane never responded to me. Charo, did she respond to your internal? Yeah, in fact, she's not, she's not online right now. Okay. Yeah, so we'll find out about moving the main repo. The other thing I wanted to talk about, which is kind of a loaded question, and it may lead to a long discussion or short discussion. How do folks feel about set weeks for the OKD working group? In other words, the first and third week of the month, or the second and fourth week of the month. This came up and I'm starting to sort of ease into participating in the GitOps working group. And I asked the question there and it apparently was something they were talking about. And I think it's something we could talk about here. It would be a lot easier to know which meeting between the docs meeting and this meeting and be easier to invite people because they'll always know it'd be easier for scheduling. Is there any reason why we wouldn't want to go or anyone have any general thoughts on going to like a first and third week for the main meeting? Are we already effectively on that? It ends every two weeks. Yeah, but you end up eventually it ends up being you get out of sync with that because if there's a fifth week on that Tuesday, like if the month ends, do you see what I'm saying? I don't know. For me, it's easier just being biweekly. If I have to go in and figure out how to schedule it for the first three years. It's kind of a pain to fix that in the calendar. Like I rely pretty heavily on my Google calendar telling me I'm about to miss a meeting to actually join and doing those kinds of things, those kinds of gymnastics are quite hard. I move mine into exchange calendar so that all the other people at B-CIT can see it. And I don't think that that's as flexible as Chrome jobs so that you can do like the first Thursday of whatever. So like every other, like a repeat every other week is something that is easily manageable in all calendars. Jamie, was there a reason giving for moving to a 1st and 3rd rather than just the current biweekly cadence? So some people have scheduling in their day jobs that are the 1st and 2nd week or the, you know, or the 1st and 3rd or the 2nd and 4th week. Some people have voiced that as something of a concern. And then the other one is that in those weeks where you don't fall on the 1st and 3rd or 2nd and 4th on that 5th week, there would essentially be a break. And so people could breathe a little bit. But it's, I mean, is anyone else have any opinions? It sounds like most folks are into the just every other week thing. That's, and that was the other thing. By the way, the other, that was the impetus is the initial impetus was me actually asking the get-ups working group, well, what week is this? Because it says every other week for their meetings. And I was like, well, what does that translate to? When is the next one? So for inviting people, it's problematic or for posting it in such a way that people could just happen upon knowing when the next meeting is. So we would have to, and Mike, I think you just posted something. Yeah, so Mike just posted something, right, putting something on the website and also via the new Twitter and stuff like that. So, either way, I think, yeah, we should do that, like put it on. I'm kind of curious. Couldn't we just have like, okd.io slash meetings where we have like the iCal files like easily downloadable and maybe like just a display that says this week's meetings are like x, y and z or whatever. Yeah, well, so we have the FedoCal entry. Yeah, the FedoCal calendar, you could just like import it. Yeah. If FedoCal can be configured, you know, the FedoCal setup for this event can be configured with whatever way you want. Then it's fine because it doesn't actually matter what restrictions everybody else has, because if they just subscribe to the live iCal feed, they'll just get the events as the calendar system pushes them. But like, if I have to make calendar events for this is going to be very, there's going to be some gymnastics involved, and I don't know if I'm going to get it right. Mike, what were you going to say? I just, I've had a poor user experience with the FedoCal stuff like it doesn't import nicely to my Google calendar for some reason. Yeah, I noticed that as well. I actually have, on my personal Google account, I have like 20 different FedoCal things imported in. You have, and it works fine. You have to tell it when you're pulling in a calendar event from there, you have to grab the iCal link. You know, so you go to that and say copy iCal link, and then you tell Google Calendar to subscribe from a URL. That means that's not the same as importing. Importing from a URL does a one time thing, and then that'll be wrong. But if you tell it to subscribe to it from a URL, then it will actually work correctly. Why these are two different things? Hell, if I know. Yeah, I mean, yeah, maybe my Google food is just weak here. Thanks, thanks, Neil. Alright, so so is the temperature of the room then that folks want to just continue with bi-weekly? Alternating bi-weekly? That's the temperature? Okay. That's what we will do then. But don't be surprised if we do cancel one of the meetings on either of them just so that people do have a break. Because Diana and I do burnout. Okay, we have about 15 minutes, but we've made it through our meeting items. Is there anything else that folks want to talk about before we break? Do we want to just pre-cancel all the December ones? Good. Good. It's like last year we ran up right up to the line and then we decided we were going to cancel all the December ones. Well, so what do we, let's see, we have December, there is... Maybe we could the first two weeks and then we cancel. Sounds like a plan that we come out with a new F-class version, major version, a new OCD major version, and then we cancel all the meetings at the same time, right? That sounds like a great plan. Yes. And shut down the website for maintenance. Right. So, are we saying the 14th? Through the 4th? No meetings? I'm fine with that. Or do we want to do that the 14th and then go 21st? That's kind of late though into December. I mean, if folks are going to be really busy. I mean, my PTO basically starts on the week of the 12th. So, so after that it's just... And my PTO doesn't end until after New Year's. So, I expect at least from my perspective, I'm personally canceling everything that I'm attending in those three last weeks of December. I would say, yeah, the 21st nobody's going to show up and the 4th nobody's going to show up. All right. So, straw poll vote for canceling meetings from December 21st through January 4th. Raise your hand or say aye. Do it. Even if you don't, I'm not going to be here. You're welcome to meet. Okay. Does anyone have an opposing opinion on this or a different perspective? Let me say different opposing frames it has. Different perspective on this than what we've just... Anyone? Okay. Now, where are we in terms of actually a little bit closer? November, are folks okay doing the 23rd? Because that would be our next one for November. I'm fine with it. Yeah. All right. So, I will see you all in two weeks. Expect your task list to be created as discussion items. And of course, if anything comes up that you need to talk about the group, you can use the Slack. I've been successful at making the dev Slack channel mostly for dev questions or dev discussions. So, feel free to pop in there or use the Google Working Group Google group. Awesome. Thanks, folks. Thank you. Bye.


And this question is from Jane and Jane says, how will I learn to recognize which journeys are most important? And I guess this one relates also to our previous question, right, Daniel? Yeah, yeah. Yeah, it does, in a way. But I would say that most important, I mean, important in this, from this perspective, when starting to journey mapping and start to focus more on customers' interests, I would say that most important is kind of a trap. Because this is not about doing the most important, it's doing the ones where you and the committed people internally have most impact on what can be created. So where do you have mandate? It's more about mandate and you could create the impact and do things that go all the way through the organization and meet the customer so you could create a really good case and a best practice internally and that it is connected to the important KPIs within the organization's goals. So you will sooner reach the most important journeys, but not at this step. So it's not about understanding the most important ones, it's understanding where could you create impact? Where could you create change by doing this way? Yeah, that's the most important criteria when you start off bike. Yeah, I mean, I have several times done those mistakes in my previous life as a consultant, that we were focusing on the important ones, the most important ones for this organization and if it was an authority or something. And it really ended up with nothing because they were so complex, so many holy cows internally and so much politics connected to that and there were no mandates. So nothing, we ended up with nothing. But when you say you made the important ones, what were, how did you think about important back then? What did you consider to be important journeys back then? The most important from a business perspective for this organization or if it's a public organization, the most important connected to the purpose of this public organization. And where we have biggest business or purpose connected problems, those should be the most important ones. Or where the organization were really treating the customers badly, so they were important from that perspective. Yeah, so the things where there's a lot at stake. Yes, exactly. Business wise, where there's a lot of stake business wise. Yeah, what I learned was that that was not the important thing here. The important thing was where do we have mandates? Where can we create a really great case? Because sooner or later we will come to those most critical ones. But we will get there with a good case showing that this will show a result and we will have a bigger mandate at that time.