So, this is where we stopped at the end of the last class and we were actually looking at the significance of the equations here for learning or updating the weights based on samples from two different classes and based on the inequalities here whether you are in the positive or negative side of the hyper plane you adjust the weights based on the sign of the direction of the test training samples okay and we also discussed about the significance of the learning rate parameter as it is called and you start with the larger value and keep on reducing them as time progresses and you stop actually when the iteration converges and which basically means that these conditions are not met typically the condition of convergence in a learning algorithm is not based on the these inequalities basically you find out the errors in predicting the classification of samples as given during the training and the error is very very less or does not change over certain amount of time you stop the iteration of the convergence. To be very very specific in the case of a feed forward neural network where a back projection learning algorithm is used the objective is to basically minimize an error term okay. So this objective of the minimization of the error term is also along similar logics where given a certain weight wk at any point of time of an iteration you adjust the weight and get the new weight wk plus 1 the change in the weight wk is basically given by this particular expression and remember as shown in this diagram the change in the weight is along the direction of the vector okay of course it could be the negative side as well depending upon which class you are working on and the least mean square learning algorithm that is the derivation which I am skipping here which says that the change in the weight should be eta which is the learning rate parameter multiplied by the direction vector of the sample multiplied by an error term this is something new which you have not discussed earlier remember the logic is the same that means we change it along the direction of the vector the delta wk is a vector as given here it is parallel to or along the direction of the test sample which is here multiplied by the learning rate parameter and there is another scalar quantity here which is an error term defined by this what is this error term indicating you look at the difference between the desired value of the discriminant function and the actual function of classification and then decide typically you want this you stop when the error this error term is 0 this error term is 0 when the corresponding value of this g which is the discriminant function here is the same as the desired value in this case in our case it is simply positive or negative but if in a multiclass situation this could actually indicate a value. So when this value of the error that means the desired value for the classification required for the discriminant function g and the actual value of g which is x multiplied by w as given in the earlier class if this value is large you move you actually have a larger value of delta wk because this value will then be larger if this value is negligible or small the change will be also small. So you see there are three factor direction given by xk learning rate parameter which reduces over time and the error. So the error is large you move larger in larger steps if the error is small you move with smaller steps and of course as iteration proceeds the iteration learning rate parameter will also go down forcing you to anyway move at smaller steps when do you converge now you have an error term ek you look at the error term and say for all training samples if the error term is less or some of the error term is less or individual error terms are small or they do not change over time then you can stop because you have reached the solution space you have reached the solution space because this term x multiplied by w is actually equal to dk and are very close to that giving result to a null or a very very small. So this is the essence of the learning algorithm in case of a perceptron as well as the feed forward neural network in general and this is applicable with minor modifications for a multi layered feed forward neural network algorithm as well where the learning algorithm is actually called the back propagation learning law BP neural network back propagation neural network very casual it is called but actually the error is actually back propagated the signal is moving from left to right from the signal input towards the output but the errors are propagated back which is the error term this is an example of such an error term which is propagated back to know how much of the weights should I adjust depending upon the error the error is back propagated but the signal flows forward from the input multiplied by the weights to the output to the next layer and so on in a similar case it is done so it is applicable for a perceptron but a perceptron you can actually use the simplistic form which you have discussed in the previous slide without the error term as well but if you have a layer of different perceptrons forming a single layer or multiple layers with different number of neurons or perceptrons as they are called to form a multi layer feed forward neural network a learning law of this type is adopted where the errors are propagated back they are measured multiplied with an learning rate parameter eta then how the weights are modified always along the direction of the training sample this is what we have learned through the end of the last class and also today that it brings you in the weight space in the weight space towards the positive zone of G for of course the correct set of samples we move ahead and get back to our linear discussion of linear decision boundaries remember in the last class we talked about that we will not discriminate between decision boundaries and discriminate functions now we will discriminate it exactly formally earlier we were not discriminating much one discriminant function is discriminating between one class with respect to the other or the rest and that is also acting as a decision boundary now we will actually do is what we will do is take two discriminant functions both of them being linear what are these two discriminant functions for two separate classes i and j 1 and 2 a and b examples again I repeat fruits and flowers cars versus trucks aircrafts versus trains let us say or two different landscapes for in the case of remote sensing applications whatever you are trying to discriminate one class with respect to the other let us say you form discriminant functions for each class and we have seen expressions of that curtsy using the multivariate Gaussian distribution under the Bayes law that gives us a discriminant function the modern distance criteria under that when we took the covariance term equal to an anti-dematrix we it boiled on to a linear discriminant function if we take two such linear discriminant functions as given here you get a decision region boundary by solving this this is a general expression for a DB decision boundary between two regions if individually each of them are linear then you actually get an expression like this okay where these W's are the corresponding weights these are the corresponding bias terms they are for two different classes i and j and each of them and this actually indicates a linear decision boundary precisely now discriminating between these two classes so you are using two discriminant functions for each class or this is actually represent also a group of classes put together if necessary for certain applications or i could be one class j could be a mixture of other classes but it is basically a binary classification problem two linear discriminant functions representing two groups or two sets of classes or even just two classes themselves or even one versus the rest whatever the case may be we are not worried about this at this point of time but we put them under this expression then you get a linear discriminant function and this is a course an expression of a hyperplane a point in 1D line in 2D planar surface in 3D hyperplanes in higher dimensions separating the decision regions or DRs in high dimensional space of course we know that the hyperplane will pass through the origin if the bias term themselves as 0 or they cancel out either they are 0 or they cancel out such cases we get back to the expression of that capital G which gave us that discriminant function so we are looking at the generalized results of the Gaussian case of a discriminant function G remember in we started with these two classes back we linearize it under the assumption that this is an identity matrix now we will not make an identity matrix let this be a general covariance matrix but again we will take some special cases so this expression is not new this is the malnubis term this is the constant term and this is of course there is a class variant because if the sigma i changes from one class to other this could be separate and this malnubis distance in general if this is not an identity matrix it gives you quadratic terms or quadratic expression it spawns a number of different type of quadratic surfaces some of these examples we will take in the next class when you talk of nonlinear decision boundaries because we will still talk of linear discriminant functions under special cases of covariance matrix not equal to an identity matrix also we will talk about that today and still we will be discussing in detail about linear decision boundaries but remember in general this is a malnubis distance criteria it is a vector distance using the inverse of the covariance matrix term and it is denoted by this this is the symbol we might use in certain cases when this remember if this is an identity matrix then this is the simple equilibrium distance norm which we have used and then we are linearized as well by ignoring the class invariant term the quadratic term okay but in general this is the malnubis distance criteria okay so under the Bayes rule Mohsen class assignments we have assumed that the G of x is equal to this what is this term called in the Bayes theorem I repeat we have done this this is called there are 4 terms class prior unconditional conditional and a posterior which one is this the class contour posterior given a sample x what is the class to which it belongs to correct so this is the left hand side of the Bayes expression that is our discriminant function we started with that and we made several assumptions we ignored class prior of course we always ignore the unconditional denominator term okay but we are of course mainly concentrated on the conditional distribution function that is where we brought in the multivariate normal density distribution so we will assume that the class prior is still the same for the time being okay so the in some sense basically this is what we are talking about G of x well I just give a different notation because this is not the same as this so put the the unconditional density function here product of these two is log of this plus this and under this if the class is same this is the one the class conditional function here is the one which is going to dictate and we take this to be our normal density function this one is this is nothing new we have all done this but I am just recollecting all of that so that we can put ourselves in perspective the log prior term is still here and if you break it into parts and simplify this expression will give you this term and this term the log prior is still here this is the expression within the exponent sorry it was within the exponent it has already taken out so the mononibusian's criteria is still here it is still here so we have just ignored the constant look at the final expression here what we get I leave this simple derivation here this what is the constant term we ignored this one this is the constant term we ignored when we went from this step to this step here the log prior is still here the log of the covariance matrix is still here yes and the inverse this is the mononibus distance here okay so this is what we are working on and earlier we have assumed this to be an identity matrix. So now let us take the next simplest case or the simpler case of the covariance matrix instead of taking it to be an identity matrix we will take it to be a diagonal matrix not only diagonal all the terms variance terms are same and that to follow classes physically if you want to interpret this what are you talking about you are distinguish between distinguishing between say fruits and flowers two different classes and I have taken feature samples like color weight smell size and so on and so forth what I have found is across the two different classes across the two different classes it seems I am having the same variance same scatter of all these features the variance of color feature across the fruits which I have taken is the same as the flowers well from my statement itself you can almost imagine how little bit quite a bit unrealistic this is this assumption but for the sake of mathematical argument we will start to relax these constraints one after another in the previous case mind you we took the covariance to be an identity matrix that means there were no variance at all or variance was equal to one now at least we are having some variance but the variance is same across all dimensions across all classes fruits are having the same variance of color as the case of flower the variance of weight is also the same size is also the same and so on and so forth not an idealistic assumption here but just to show that this may not be good but for the sake of mathematical argument yes we will still proceed and go on and we will eliminate the class independent bias which one did we eliminate in the previous expression first of all there were three terms the log parade is there what did we eliminate can you guess and tell me from the previous one we will go back and have a look this term how could we eliminate this because the log of this matrix actually I should put a mod here because the determinant of this so proof determinant term is missing here so please put the determinant determinant of the sigma okay for a diagonal matrix what is the determinant because we have made an assumption that it is strictly diagonal so diagonal matrix the determinant will be the product product of all the diagonal terms correct product of all the diagonal terms so that is the same for across classes that is why the term could be ignored here and inverse of a diagonal matrix inverse of a diagonal matrix will be 1 by those elements okay and the elements are sigma square so 1 by sigma square now it is taken out and the factor of 2 actually is coming out anyway here this is already there so in fact this term remains with a 1 by sigma square here this is ignored and the log parade is there so this is how you get this expression what do you do with this in general this expression indicates constant hypersphere centered around the class mean where is the class mean here mu i I live it for you to write this expression in two dimension write this in two dimension you will get the equation of a what is the geometry you will get go back and look if you write this expression this part forget the constant term here this if you write in two dimension will actually give you the expression of a you should be able to extrapolate the statement which was there at the bottom of the slide will go back then look at the slide if these are constant hyperspheres in 2D this should indicate what is the projection of a sphere in 2D projection of a sphere in 2D louder you take a sphere in three dimension projected on the two dimensional world simple projection take a ball think of a ball what will the figure indicate geometry in 2D it should be a circle okay so that is an equation of a circle which you get in 2D it will be a sphere in 3D and hyperspheres okay but more of this later on because we will still looking at so this in general appears to give an only in addition boundary because the nonlinearity term here the quadratic term here is what this will do okay but more of this later on so this is an example of a diagonal covariance matrix and this is the case when sigma 1 is not equal to sigma 2 then of course you have to write the inverse of the covariance matrix in this form of course if we ignore the subscript that means the individual variance terms are same you can have the same term along all of this in fact you can take this constant term out and say this is an identity matrix multiplied by 1 by sigma square which we have done little bit earlier so considering the instrument function we had ignored this a little bit earlier in general this will yield a weighted distance classifier and depending upon the covariance term larger small scatters spread of this we tend to put more emphasis on some feature vector components of the other this will give in general hyper elliptical surfaces in rd in d dimension space for each look at this expression now with this covariance matrix how is this different from the previous covariance matrix sigma 1 is equal to sigma 2 equal to sigma 3 and sigma d remember the sigma is still same for all classes but the terms are different now in the previous case the diagonal terms were all same so those were giving hyper spheres nonlinear decision boundaries we are approaching the case where we can get nonlinear decision boundaries because of the quadratic term and the decision boundaries will be spheres in the case when you have equivalent equal terms or identical terms as it is called identical terms along the diagonal if they are unequal as given here if they are unequal you will have hyper ellipsoidal surfaces hyper ellipsoidal surfaces if the values are unequal if these are all are same a special case ellipse becomes a sphere 2d so circular ellipse in 3d you have sphere ellipsoids and in higher dimensions you have hyper spheres or hyper ellipsoid we will discuss this in detail in the next class of nonlinear decision boundaries with examples of nonlinear at least in 2d we can show this sort of thing carrying on with the discussion of the decision boundaries let us assume if all the class priors are same well I must warn you little bit that sometimes we are writing dfp of wi but w is weight so let us stick to ci for all k then eliminating the class independent term which is basically if you look at this term so these two are ignored you are just left with the model of business let us concentrate on this term which is actually telling you that this covariance term tells you where it should more give more importance towards a particular dimension or not because this is what will dictate which dimension has more importance okay so this is the model of this term and if you expand that in this particular form you can write this expression in this form and this can be now written in terms of gix as this provided you can switch of this term you can switch of this term if all the corresponding covariance matrix is same for all classes now what you are saying is this is not a constraint which we are putting that the covariance matrix is diagonal not it can be any arbitrary covariance matrix may or may not be diagonal what we are saying is it is same for all classes how unrealistic that can be just to show okay it may be possible in certain situations but what I am saying is features such as height weight color size spatial extent for two different types of fruits and flowers are also okay and they are symmetric of course it is symmetric matrix but they are all same so if it is same you can switch of this i and if you can switch of this i you can switch of this particular term you will be left with only these two terms here and typically gi is gi by 2 so the 2 will go off you would not have a 2 you will have a minus half factor here so we will concentrate on this term it has come back again this is not a new term 1 or 2 classes back couple of classes back we are talking about this term the last class we discussed at length which gave us to the concept of perceptron learning but linear decision bond is the only difference now is earlier this weight was based on only the mean now the covariance term has come and sit here the covariance term is coming inverse of the covariance matrix to be very precise is coming and sitting here these weights I have just changed the notation to short of a omega here small w you can treat this and this is an inverse of the covariance matrix okay so we have discussed this case earlier when this was an identity matrix and when this was the covariance matrix was an identity matrix we just had this mu as the weights and mu transpose mu without this term is what we had for linear decision we are still having linear discriminant functions we can still have linear decision boundaries but the covariance term is coming and sitting here it is may be an arbitrary covariance matrix just its symmetric that is all so this is what we have and we analyze this for the rest of the class and move over to non-linear decision boundaries in the next class so this is the last discussion on last part of the discussion on linear decision boundaries because these type of gix will give us dr's dv's which are hyper planes and they will be linear by exploiting this constraint as we have done earlier at the beginning of the class beyond this if you have whenever diagonal sigma which is class dependent remember this is coming under the constraint that the covariance matrix is class independent we will go back and look at this constraint look this is class independent that means you keep change go from class 1 to 2 i to j is the same covariance matrix so it is class independent it is class independent if it is class dependent or of the off diagonal terms are non-zero you will typically have non-linear decision functions discriminant functions decision regions or decision boundaries non-linear discriminant functions and decision boundaries are the correct way of saying there is no non-linear decision region but whatever it means is that discriminant functions and decision boundaries actually give you dr's so the boundary of those regions will be non-linear if you have class dependent covariance matrix and the off diagonal terms typically have more roles to play but as long as you are having class independent covariance matrix diagonal or not including identity matrix which is a special case of diagonal covariance matrix you will have linear df's linear db's so again to repeat what are the special cases of covariance matrix in which you will have linear db's or linear discriminant functions one is if the covariance matrix is class independent that is all straight away first of all even if it is class dependent one well in some sense you can say a special case of what we consider as an identity matrix that is also class independent so as long as it is class independent you will have linear discriminant functions if it is class dependent and diagonal you will still have hyper ellipsoids or hyper spheres which gives rise to the simplest case of non-linear discriminant functions which we will discuss next let us proceed with the discussion on the decision boundaries and discuss the effect of class priors in a more general case remember this just to revisit the equations this is what is this equation you have seen this many times Bayes theorem under the Bayes theorem this is the posterior remember the c is changed to wi be careful this is not the same as the class prior w sorry this is the same as the class w but not the weight this is the conditional density function which we are talking about here this is the by this is the multivariate Gaussian density which we started few classes back the normal distribution okay and if you take gi to be this plus the class prior that means you are taking the this term which is the class conditional density function as given by this plus the class prior you can expand it using this expression that means what you are doing this log prior is here and we have done this many many times take the log of this expression you have malonov resistance plus these two terms which one of them is a constant anyway the other could be class dependent or class independent since we are talking about linear decision boundaries what assumption did we make about the covariance matrix it is class independent so there is no subscript here you can see although I have put a subscript here but the covariance matrix is same so we can switch it off here carrying on so we can switch off these two terms because they are class independent we are left with the malonovist distance function that is equivalent distance weighted by the covariance matrix it is same for all classes plus the class prior what is this term doing if it is the class prior is not same remember what is class prior you are discriminating between two types of fruits say mangoes versus apple both are let us say seasonal fruits depending upon certain time of the year you may have more apples let us say in winter in summer or rainy season you may have more mangoes so the class priors will change it may not be the same that is what we are saying what does this affect the decision boundary let us look cancelling the class invariant terms we have just left with these two remember this is the main one which is responsible for all my linear and nonlinear decision boundaries but under the case same diagonal sigma that means we are switching off i diagonal elements class independent what do we have for the expression so we are switching off this diagonal this is the expression which we have plus log prior this is the case a we are discussing diagonal we have done this already this is a diagonal matrix we can take out this half is here inverse of the covenants matrix will ill this term and then we have the class prior when we break open the expression this is the expression which we have nonlinear term here the linear term particularly here as a function of x we will analyze this expression in the next class in the next slide analyzing this expression again the nonlinear term here the linear part here why are you switching off this because if you move from one class to the other change i to j you say this will change this will change as well this will change this won't change so this is a class independent term again it is a quadratic term but class independent term okay and why it has come because the class independent covenants matrix the same sigma which is diagonal okay so once you do this this is what you are and this is the same expression which we had just a few slides back sometime back but instead of the writing inverse of the covenants matrix here multiplied by the mean you are able to write now this is minus sigma square sorry divided by the variance divided by the variance here the same thing which comes here remember the 2 and 2 cancels out here so you will be left with this divided by sigma square here mu mu transport divided by c in general you will have it as a inverse of the covenants matrix here basically this term is indicating the inverse of the covenants matrix but the class power is still here earlier we have ignored this the linear distance boundary is now this we have seen at the beginning of the class today that it can be written in this where individually each of these terms are given by this expression so it is wk transpose minus l where knl at the for the 2 different classes they are not identical and the bias term is also written here let us observe this expression in the next slide the linear distance boundary is given by this which is basically this and I leave this is an exercise for you to prove that this difference in the bias term can also be written as a function of this which is like this where the x0 this is interesting is given as this particular term look at this term I am leaving this is an exercise for you to prove it analytically you can prove this with a little bit of juggling maybe about a half a page of derivation okay and hence forth if you can write this term in terms of this and substitute back here this expression can be written this is nothing new we have this two classes back w into x minus d is equal to 0 from where we derive the perceptron the hyper plane this is the plane passing through the origin or place plane passing through a particular point the only thing is what is this w it is the it is the difference of these two main vectors what is the mean vectors if there are two main vectors this is the vector difference between the two main vectors that is the w w will be the remember the w is the normal to the hyper plane the normal to the hyper plane is now the line joining the two means and if it is so the hyper plane will be normal to the line joining the two means I repeat again the w which you see here we had seen this earlier two classes back that if this is an hyper plane this is the w and now what we are saying is this w is the vector joining the two class means so if there is a class mean one here in the class mean two here this is the my w this is the hyper plane we will have it is in a diagram coming up in the next slide but I hope you got the justification of that maybe I will go to the board and just draw that for you because I may not have a slide ready so if this is a mu k and this is mu l then mu k minus mu l this will be the w the vector direction will be here or here although it strictly does not matter what do you think it is given as mu k the expression says it is mu k minus mu l mu k minus mu l it will be pointing towards this toward mu this will be w and we said that the w is orthogonal to the hyper plane so this is the hyper plane h I am drawing in 2D this is my pattern space or feature space not the weight space like we did in case of perceptron so this should be x1 this will be x2 two dimensional feature space you can visualize in third dimension also this is my hyper plane and which I wrote that equation here w x minus x0 hyper plane which is equal to if you select points here in this hyper plane this will be equal to 0 so this is the expression which defines this hyper plane this h w is normal to that okay and you could ask me where is this x0 it is a point here it is a point here on this plane okay and let us look at the expression for x0 now in the slide look at that expression of the x0 x0 is a point on h and if you look at this expression of x0 look at this term can you tell me where is this term it is the average of the two means go back to the board these are the two means mu k and mu l average of the two means will be the central point let us say so for the time being I will let this some x0 tilde some average value of the two means and it seems to be an approximation because it is not the full expression this containing the first term in the slide so let us go back to the slide so what I have marked there on the board is basically the yeah this term here but there is another factor which is based on the class mean we will explore that now we will explore that now what is the effect of class priors on that point x0 and the x0 is a point which is actually lying on the hyper plane okay so this is the x0 so we had this w is very simply this is the vector it is the line join the two means vector from one mean to the other one orthogonal to the hyper plane and the hyper plane is passing through x0 because that will actually decide my decision boundary it should have been in between x0 will be equal to this under what condition tell me what condition the expression of the second term don't tell me sigma equal to 0 what what constant will make this vanish don't tell me mu k should be equal to mu l don't tell me this will be equal to 0 these two probability terms if they are priors are same class priors are same that means I get equal number of mangoes as equal number of apples and if I can do that then if that happens this term vanishes forcing x0 will be equal to the mean of these two classes that means my hyper plane is strictly in the middle of the two class means in this case hyper plane h is the decision boundary earlier it was df discriminant function but now it is a difference of two g's which is a linear decision boundary we will have some examples let's look at the simple example which I have borrowed from the website I will give that reference a little bit later on look these are two Gaussian functions drawn here and if you look at the Bayes theorem ignoring the class priors this point at the center of intersection of these two should have been the actual value of x0 x0 is a point here mind you average of the two means look at this is one mean this is the other mean class 1 w1 class 2 w2 two Gaussian distributions intersection of this at the middle should have been the point x0 corresponding to this expression but in this figure one class prior is more than the other one so this term will not vanish it pushes the decision boundary more towards the other class ideally I would have loved to have the decision boundary exactly in the middle of the two Gaussian functions provided the class priors are the same but if the class priors are more what does it mean in reality there are more apples than mangoes there are more flowers than fruits there are more men than women there are more forests than mountains okay there are more in landscape there is more water than deserts okay there are more people smiling than more people anger I am talking about discrimination of expressions in the human face these are examples only and in certain cases class priors could be more class prior pushes the decision boundary this will be an additive corrective term to this value of x0 the point is pushed there it could go to the other side if this value of the class prior of class 2 would have been more increase the class prior more you can see how far it goes why because this value is 0.9 this is 0.1 this is 0.7 and 0.3 so this value of class is now more than this it pushes the decision boundary from this point here more towards the other side why because this term will be now larger because this numerator is more than denominator making this term more for this figure than the left-hand side this is a case of a DBE with same diagonal sigma same for all classes alright identical diagonal elements that's why you could have taken out that sigma but one class prior is more than the other one the decision boundary is not lying in the middle it's a point in 1D we will show examples now in 2D where it will become a line and you can extend that to a plane in three dimension hyper planes in higher dimension this is a Java applet available as an open source which you can play and manipulate the individual terms of variances and the class priors so there is a slide bar to change the standard standard deviation is equal to 1 for both both equal standard deviation identical covariance matrix 2D but what happens both identical class priors so this is this is now look at the DBE here this is that hyper plane H which is a DB where it is sitting this is that X naught why because it is exactly the midpoint of these two means mu 1 mu 2 well is written as M 1 M 2 so assume this to be mu 1 mu 2 2 classes equal priors 0.5 0.5 each as given in this 2 bars at the bottom class priors are same so ideal condition so class priors are same if you go back this terms cancels out you exactly have the X naught at the bisector point between mu 1 and now DB is actually a perpendicular bisector of the line joining the means DB is a perpendicular bisector as a special case of the class joining the class means if the class priors are same identical covariance matrices that means class independent covariance matrix what I will do now first I will change the class priors and see what happens to this diagram so now what we will do is we will first change the class priors and see what does it cause in what they what is the effect in the location of the DB that is number one and then we will also of course change the variance terms and see if that has an effect as well it should have an effect as well let us look at this diagram on the right hand side what I have done is maintain the same value of the variance which is equal to 1 both here as well as there but I have changed one of the class priors look at the value here which is given and this blue bar indicating that the value is much larger is 0.9 and the other class bar is 0.1 earlier the both were identical what it has done earlier the decision boundary was at this point of the line here as given in this diagram it has pushed it towards which class it had pushed it towards the other class which had a less class prior class one had the less class prior the DB has moved towards that earlier it was at the center because the class past were same the class prior for class one is now 0.8 indicated by the big red bar there the other value is 0.2 which is much less here look what it has done because the class prior of class one is higher it has pushed the decision boundary from this point has given here to more towards the class why the second term the decision boundary will now change its sign and move in the other direction remember it was a log ratio of class priors so depending upon which term is more you will have a positive value or a negative numerical value pushing the the separating hyperplane as the DB towards one particular class which particular class does it move towards the class mean which is having the less class prior you see the less class prior the DB is moving towards that class mean here it is moving towards class 2 mean because that has the less class prior so this is the effect of class prior in Bayes decision rule under the normal distribution all that has been taken all right the class prior is not allowing the decision boundary to be strictly at the perpendicular bisector whether it is a line or a plane or even a point in one day remember the previous slide where it pushed it towards the class mean or even further away let us go back you look at this example here you look at this it has even gone past the class mean of class 2 because it is really very very large the class prior for class I think there is a typo here this should be w1 class prior for w1 is 0.7 this is 0.9 so it has gone past the class prior for the second class itself the mean is here somewhere gone past them so if it is too big it can even go past the class means whether it will go past the class mean or not depends on the individual variance so what I will do now is keeping these unequal class priors we will change the class variance now the scatter we will reduce in the next slide look this is the effect of class priors unequal class priors but a lesser scatter this has moved but not that much that means the variance has a role to play this is standard division let us go back to the expression there look this is multiplying factor so if this is larger this effectively more pronounced if this is less this will be lost pronounced so that is why in these cases the effect is more and if you enlarge it more if the variance is more then there will be a class overlap plus the decision boundary will skip moving more towards the class mean and even further away remember it will always orthogonal the plane will be orthogonal the separating hyper plane or the db in this case will be normal to the line or vector joining the two class means or the vector joining the two class means will be orthogonal to the plane that will always be the case the plane will remain orthogonal to this vector but it may go away if the ratio of this is higher or the variance terms increases here we have reduced the variance the shift is there is not at the perpendicular bisector because its prior is less let us look at this case here this is the case where the variance is really large it has come the separating hyper plane has come more closer to the class mean it may go further away if we increase this or reduce this further and change this increase it more towards the value 1 so this shows an effect of where the decision boundary linear decision boundary will be located depending upon two factors the spread or scatter of the individual scatter matrices or the variance of the features and the class priors this will decide where the decision boundary will be located remember always it will be the separating hyper plane will be orthogonal to the line joining the two class means or the line joining two class means will be orthogonal to the hyper plane but its position dictated by x naught will be dictated by the second term in general it will be in the perpendicular bisector at the center point of the line joining the two class means but it will go up and down more towards the class mean depending upon the ratio of the class priors multiplied by a factor which also depends on the variance plus there is another factor in between let us go back to the expression normalized by see this is the vector this is also a vector term so this this is an unit vector along the line joining the two class means so the numerical value dictated by the log prior ratio and the variance or the scatter or spread dictates where you are we will stop with the discussion today on linearization boundaries we will move towards non-linearization non-linearization boundaries more where we will have we will have class dependent variances or covenants matrices and the effect of off diagonal terms also as long as it is unequal as long as it is unequal we will have non-linearization boundaries thank you we will come back to the next class


We're going to discuss the external features of the spinal cord in this video. So it lies in the vertebral canal of the vertebral column, which provides support and protection. The cord is continuous with the medulla oblongata, the lowest section of the brainstem. The spinal cord is covered by three thin membranous layers known as meninges. That's the pier, arachnoid and dura marta from inside to out. Between the pier and arachnoid marta lies the subarachnoid space, which contains cerebrospinal fluid. Between the arachnoid and dura is the subdural space, but this is just a potential space. There's nothing actually in here. Marta means mother in Latin, pier means tender. The tender mother sticks close to its baby the spinal cord. Dura, on the other hand, means tough and the tough mother is separated to a degree from her baby. Between the dura marta and the surrounding bone is the epidural space, which is filled with epidural fat. And these spaces and their sheaths continue cordially down to the level of the second sacral vertebra. The spinal cord is divided into segments and these segments relate to spinal nerves. There are eight cervical segments known as C1 to C8, 12 thoracic, T1 to T12, 5 lumbar, L1 to L5, 5 sacral, S1 to S5, and then one coxigil. It is roughly cylindrical in shape, but enlarges in two areas. The cervical enlargement consists of the spinal cord segments C5 to T1 and provides innovation to the upper limb via the brachial plexus. It corresponds to vertebra C3 to T1. The lumbar enlargement consists of spinal cord segments L1 to S3 and innovates the lower limb via the lumbo sacral plexus. It corresponds to vertebra T9 to L1 and much of these lower nerve roots make up the quarter equina. Or horse's tail. We'll make our way through these layers again to have a closer look at that. So there it is there, the quarter equina, the constituents of which head out, lumbar vertebrae, down to the sacrum and down to the coccyx as well. The spinal cord proper thins around L1 into the conus medularis. At the very tip of this structure, we can see the phylum terminale, which extends quarterly to attach to the first coxigil vertebra. Let's revise some embryology now. Remember the spinal cord occupies the entire length of the vertebral canal up until the third month of life. Thereafter the vertebral column elongates faster than the spinal cord and at birth the cord terminates at L3.  So why is this important? Because when doctors are performing a lumbar puncture or an epidural they need to access the subarachnoid space or the epidural space. But they do not want to put their needle through the spinal cord. So these procedures are performed below L3 in children and below L1 and L2 in adults. Click the link in the video description to navigate around this model yourself. Hit subscribe if you liked this video. Thanks for watching and we will see you next time.


Modern science, medicine, political freedom, the market economy, all of them were told are the result of a sort of miracle that took place 250 years ago. That miracle is called the Enlightenment. All right. You didn't even make it to 30 seconds. No, I can't. This is how it's going to be, guys. And unfortunately, I think pinker uses the term miracle in and a lot of what we're going to hear today is kind of a response to Pinkers Enlightenment now book. But pinker doesn't take miracle seriously as Hosoni tries to imply in this is kind of coming out of nowhere. He does give the historical context for this. But nobody actually believes that the Enlightenment is just. Yeah, I mean he goes on in this video. The miracle is calling like a moment in history as though it's again just this one split second 30 years ago or a couple of years and that's a straw man who argues for that. That to actual Enlightenment thinkers, people like Dylan bear the encyclopedists, they're very historical they're seeing this is a progression that's been developing bacon with a progenitor of it and people before that and they all see this as something, a light that's been dawning over civilization for hundreds of years and they see themselves as out of kind of cresting time in it, but not, you know, it came out of the blue. He's going to go on in a few seconds to quote Pinker, and the quote is going to be progress is the gift of the ideals of the Enlightenment. And that's not the same thing as calling it a miracle that it's a gift of the ideals that were embraced it doesn't mean those ideals don't have any precursors. But that the Enlightenment's a period when the ideals of reason and political freedom are embraced and it's that those two things above all else that produces the progress that we all enjoy today that is a gift of those thinkers who champion those ideals so it's not putting in the category of what he quotes and there's a lot of that like there's no evidence for a lot of the things that he presents and this is there's no reason to think it's a miracle that that that's what Pinkers argue. No and I read Pinker's book he talks about he talks about the development of secular humanism in the Renaissance and he talks about the importance of scientific revolution to the Enlightenment how it couldn't have come about without scientific revolution. And so, you know, I don't know that Pinker does complete justice to the precursors of the Enlightenment he probably doesn't that's not his job that's not what he's trying to do in the book, but he certainly doesn't view it as coming out of nowhere is this this. Now he does view it and there's a sense in which it's a miracle. I mean not really but there's a sense in which. Wow. I mean all these amazing thinkers all at once doing this amazing work coming up with these amazing ideas and look at the consequences. That sense of wonder, I think is what Pinker would use the word miracle for it is in wonder. Wow isn't this amazing, rather than as unexplained coming out of no way starting point. It's a fantastic kind of synergy of that happens when good ideas catch on right we've been partial good ideas catch on. And that's what we're all trying to make there be more of in the world. And from that perspective it really is. So you can use miracle in that sense that it's wondrous but not inexplicable. And I mean what we're doing now the three of us talking in different geographical locations and watching a fourth video from some. They couldn't have even dreamt that people would be doing this in 200 300 years and in that sense it is miraculous and streaming it on three different platforms getting live comments from people it truly is amazing. All right, let's I'll try to hold off and more than 16 seconds and more than 16 seconds. A moment in history and philosophers suddenly overthrew religious dogma and tradition and replaced it with human reason. Harvard professor Stephen Pinker puts it this way. Progress is a gift of the ideals of the Enlightenment. There's just one problem with this claim. It isn't really true. Consider the US Constitution, which is frequently said to be a product of Enlightenment thought. But you only need to read about English common law, which Alexander Hamilton and James Madison certainly did to see that this isn't so. And this is such a strong man. Right. I mean, English common law was already significantly influenced by Renaissance and Enlightenment thought and maybe we should talk about the a little bit about the origins of the Enlightenment and different views of it and ran view of it. But there are all kinds of influences on the Constitution that are claimed by different people, some of them plausibly from like a certain Iroquois documents that Ben Franklin like to this and France and that and England and especially the British constantly English Constitution. And these things are all real influences, but part of what a philosophy of reason is about is about integrating and finding the best material from different sources critically evaluating it thinking of how to put it together. And not going by it because it's traditional. And that this is the idea of reason is overthrowing tradition. It's not saying nothing from the past is any good. It's overthrowing the idea that we should follow it because it's the past, because it's venerable because it's old, we have to think about it consider it keep what's good get rid of what's bad. And of course there's good stuff in English common law and lots of other places that they're going to keep, or stuff that they think is good some of it wasn't but they thought it was and some of it was really good. And it's, it's way under playing what the achievement was so if you say that the Constitution was an achievement of enlightenment thought, or the creation I think better the creation of the United States of America is the product of the enlightenment. It's the creation of a new country it's not just thinking, you know, it would be better if we had a government that had a little more checks and balances not this absolute power. It's, and this goes back to Pinker is putting it as an ideal, they're thinking of it as this is what we're striving for, and this is what we're going to work in action to create and having a whole vision for that, and then taking all the steps to create the actual government that's the achievement, and you don't get that in any other era and nothing similar to it that we're going to rethink government from the ground up and create a new government based on these principles that we've articulated. And they never claim we've come up with every principle that's embedded in this, but it's rather we've done something that is unprecedented and that they're right. And it's such a breaking with tradition so the idea that what they're doing is looking back, and they really used to do that so that's what we're going to do. It's that is not at all proper view of what the founding fathers of America are doing. Already in the 15th century, the English jurist, John Fortescue, elaborated the theory of checks and balances due process and the role of private property in securing individual freedom and economic prosperity. Similarly, the US Bill of Rights has its sources in English common law of the 1600s, or consider modern science and medicine. Long before the Enlightenment, tradition bound English King sponsored pathbreaking scientific institutions such as the Royal College of Physicians founded in 1518 and the Royal Society of London founded in 1660. The truth is that statesmen and philosophers, especially in England and the Netherlands, articulated the principles of free government centuries before America was founded. I mean, this strikes me as, you know, a historical in a sense of a that they articulated to the extent that a lock and then the founders articulated in that's a kind of integrated way, and be that it was taken seriously. There's a reason why I mean there might have been some scholar a long time ago came up with a good idea, and then a culture ignored it so he's ignoring the culture of the Enlightenment, a culture that was created by these thinking by these thinkers as well. Yeah, I mean I would say it's, I would put it way stronger than it's a historical, it's completely wrong that so and no and there's, I think here you start seeing the slanting you put it if you know his views about nationalism, conservatism, religion, you could predict what's going to be in this video. What's absent, I think it's worth thinking about what's absent there's no mention of greaser Rome. And I think the basic reason is because we can't ascribe that to Christianity or Judeo Christian tradition. So I don't want. But if you ask who did the founding fathers study so they certainly studied the English tradition English common law. They were experts in knowledge of Rome, and Greece and Rome's political history and drawing on that. And that's a far greater achievement than anything in the medieval period if we're thinking politically, and they knew that and that was their view so to. So, and this is the sense the issue that obviously this had precursors the founding fathers were students of history. And they said they took all kinds of lessons from history positive and negative, but it's fascinated now into a new ideal of political freedom and of rights that you get really with lock as articulating it as a full ideal, and that government should be based on this. And that's what they're trying to put into practice. So it has a history, but it also has many novelties. And that's what the enlightenment political achievement is. And also, what are we contrasting the enlightenment with. If we're saying oh there was this stuff in 1600 like the Royal Society is the contrast, which is a clear precursor to it. So the enlightenment is part of a process and it's seen by the enlightenment intellectual, again by the people who are later in this period and reflecting on what it is. And the obvious touch zone here is the encyclopedist, and particularly the there was a dark ages there was a period where people were ignorant they didn't know anything there had been Roman Greece and a flowering of learning and all of that was thrown away by Hassani's ilk these religionist anti intellectual bastards right and then there's a period where this is being rediscovered and it takes hundreds of years for it to filter out and and the the enlightenment figures think about this it was the Renaissance. There's a rebirth of knowledge. There's a period where people are studying up on the classics, but doing so in in two slavish away so there's a rebirth of science but it's a kind of science where we're timidly following Aristotle or and not thinking about, you know, how we might differ from them. And then you get figures like bacon, saying, no, we've got a what the right lesson from the Greeks is to think about their methodology and how do you prove things from the census. And what is the nature and standards of evidence we need a new organ and you get a scientific revolution you get Galileo you get all of these people you get the Royal Society is an organ for the kind of addition of information between these people as part of a process of enlightenment. And although the scientific revolution and bacon are considered Renaissance thinkers, they're seen as kind of patron saints to speak of this developing movement. And now you get after there's a scientific revolution after there's a Newton or being the period when Newton's writing a kind of graph now we're really onto something. And you get in the kind of high enlightenment the 18th century. The idea of this is now catching fire culture wide and Leonard Peacup has a really great quote about what's special about the enlightenment. It's the first time in modern history, modern as opposed to ancient right the first time in modern history, when an authentic respect for reason became the mark of an entire culture. That's what's special here, but it happens as a process, and the Royal Society and some of the things are part of that process. The contrast is before that process was going. When people were prostrating themselves before God, thinking that reason is the harlot of the senses which is a protesting quote but never got relevant here. Because we can talk about the role of the reformation of it.  So why give the enlightenment all the credit apparently because it doesn't look good to admit that the best and most important parts of modernity were given to us by individuals who nearly all held conservative religious and political beliefs. So here's the cash in right this is this is the goal ultimately is to bolster the idea that that all the good stuff ultimately comes from religion and tradition. And religionists people who are going to tinker just a little bit on the margins and would never pledge their life. Fortran and sacred honor to the cause of defeating the most powerful government in the world. And it's a, it's an example of the of slanting and of weasel kind of formulation so it's, it gives the enlightenment, all the credit. Everything hinges on the all that it's it's that's the it's goes back to activating the miracle like it's came out of the blue and it dessert there was no precursor to it no history. But if you take seriously that it's coming out of the Renaissance and I think Rand's view which I agree with that the enlightenment is Aristotelian. So if you think of the great battle and philosophy between Plato and Aristotle Christianity was when Plato is dominant and from the Renaissance on its Aristotle is coming back on the scene and even though the many of the others, because Aristotle was sort of incorporated into the church saw themselves as rebelling against it. It's in the deepest sense in their methodology. And in the non skepticism that so that you can both use your senses in your mind and you have to use both if what you really as after his knowledge, which is what the enlightenment is about that is all Aristotelian. That it just, it's all, as he puts it all the credit belongs to the enlightenment that nobody I think it no real student of the enlightenment thinks that the claim that all good things come from the enlightenment is most closely associated with the late 18th century German philosopher Emanuel Kant for Kant reason is universal infallible and independent of experience. His extraordinarily dogmatic philosophy insisted that there can be only one correct answer to every question in science morality and politics and that to reach the one correct answer mankind had to free itself from the chains of the past. He almost makes Kant sound good. There is one correct answer to every question in those fields. We don't always know it but I mean, there are, you know, provisional answers on the what does it mean to say there's not one correct answer to questions and morality or politics or science. Like that there either are aren't atoms which one is it. It is either either is not right to enslave people which one is it. And that is not a distinctive or radical view of cons that there's one answer it was I mean, did the Christian philosophers not think there was one right answer to things. They thought there was he just had to destroy your mind or subvert it to get it. And there's such package deals going on. I think the looking at the conservative perspective is interesting for what gets packaged together. It's so it's breaking free of the chains of the past history tradition and experience. They don't belong together into one. So it's not true that they were breaking free of history in the sense of ignoring history. As we talked about there. It's incredible study of history to learn positive and negative lessons and not to repeat the negative. And in that sense, there is no worship of tradition. They're interested in history, not traditions. To learn from history what you can learn positive and negative and then experience is if he's going to go on to the view that I mean this is part of what why the enlightenment is incomplete or it's just a partial success. We have this view of reason that is pulling apart that it's either you deal with abstract theory concepts and you don't engage in observation in sensory experience, or you have observation but you can't have any grand theorizing and so on. And his experience is using that and experience like experience of the past, which is why he puts it into that you're right, which is itself a package so you can't think with like if these are your categories that you just throw out as though you know what you're talking about you can't think with these. Now is it even true with regard to content he said all good things came from the enlightenment and that he abandoned he ejected tradition. It's not true that I don't know any time places had all good things come from the enlightenment. I don't know every word that I wrote, but I can't imagine him saying that. I mean, he was a political, kind of pretty moderate political liberal of his time so he was, you know, he was in favor of the American, well, the little ambiguous if you use the American revolution actually but generally pro. So, but he was not a kind of, I mean he was pro the Prussian, you know, state and he wasn't a firebrand burn it all down. But what he does say in the essay what is enlightenment is that what enlightenment is about is kind of growing up intellectually. The culture is growing up intellectually and therefore standing on its own two feet intellectually rather than being tradition bound hide bound accepting things because they're true using your own reason and just, you know, taking out of context understood that way. That's true I think and he can't write about that his view of what it is to reason for yourself and what yourself is and what reasoning is is very different from mine and very different from the majority of the enlightenment figures. And I think he represents a kind of distorting and perverting of it and a carrying on from Rousseau which we can talk about. Yeah. Well, so he doesn't think reasons infallible I don't know where that comes from. So, and it's a major, I mean I think part of the that we're now that contests the end of the enlightenment, it's that it's cementing home that reason is independent of experience hit and that here means sensory experience sense perception. And you get the view of it's he's on the one hand supposedly pro science but science doesn't tell you about true reality. And it's, it's the voice of Plato in a different kind of form that I think is coming back with con and it's the end of the Aristotelian dominance in philosophy. And this thing that that, particularly when I mean pinker does this and as any starts lining up these figures either enlightenment to these are not enlightenment. One thing that's worth keeping in mind is, this isn't a club that people have like cards that they're members of and then it's very clear who's enlightenment and who's counter enlightenment. It's a broad intellectual movement. It's the fact that people are becoming educated and thinking and debating ideas on this implicit premise of the centrality of reason. And there are all sorts of huge disagreements among these people about basically everything, and about which ideas are the important ones where there are conflicts, and then how to resolve them and whether they can be resolved. I want to talk about enlightenment thought. It's not like there's one philosophy that's the philosophy of the enlightenment. There are debates and then trends and things emphasized among these within these debates. What we need today, what I called a new intellectual would be any man or woman who is willing to think, meaning any man or woman who knows that man's life must be guided by reason by the intellect, by feelings, wishes, wins or mystic revelations. Any man or woman who values his life and who does not want to give in to today's cult of despair, cynicism and impotence and does not intend to give up the world to the dark ages and to the role of the collectivist. So using the super chat and I noticed yesterday when I appealed for support for the show, many of you step forward and actually supported the show for the first time so I'll do it again maybe we'll get some more today. If you think what you're hearing, if you appreciate what I'm doing, then I appreciate your support. Those of you who don't yet support the show, please take this opportunity go to your own book show dot com slash support or go to subscribe star dot com your own book show. And, and, and make a kind of a monthly contribution to keep this to keep this going. Thank you.


Would you rather pay $650 or $66 for a smart phone? Simple.  Better service, bigger savings. That's today's Appalachian Wireless. We're going to have different events going on but one special part is going to be Judy's place. We're taking up donations. These children over the last little bit have come very dear to our clinic and we wanted to be able to help them in some way. In this clinic we're all very blessed to have families and we want to reach out and help other families. And we know these children are someone that can benefit from what we have extra and we want to be able to help them in some way. And we see the different needs of the people and we love them as our own families. And the Judy's place is something that like I said has been dear to our heart because we've had the children come into the clinic, we see their needs, we see their fears and we want to somehow make their Christmas better like the Christmases our families will share. And I think that with Judy's place and the other organizations it creates a closer relationship between the hospital. It helps the people trust us more and we are part of the community and people know that we are here to help them and serve them. And that's our whole purpose is to make this a better place for the community. We were contacted by King's Daughters, the specialty clinic here in Prestonburg about collecting items for Judy's place for kids and celebrating us at the open house and the work that we can do together. A lot of families have a difficult time meeting even basic needs and I think you mentioned you saw even items like shampoo, toothpaste, those kinds of things and so any kind of burden that can be relieved like that on families to meet those needs then that just relieves the stress and the pressure and that's prevention of child abuse really when you can reduce the stress on a family. There's a lot of expectation and pressure from society to make Christmas be a certain way when really the spirit is about love and care. And so we accept donations throughout the year and there's always more information at judysplace.org and there's a link right there that gives all the donation information. Everyone can play a part and it warms my heart that other people will step up and it means the world to me to get to work in this field and to know that the work that I get to do and the privilege to do every day is on behalf of children and to improve their lives. We would love to invite anyone down to open house tonight, it's from five to seven at the Prestonburg Clinic. We'll be taking donations for Judy's place.  Also we'll have Santa Claus and we'll have cookies and milk and we'll have a good time and we'd really enjoy seeing you. Thank you. I've seen many of the young people we worked with go on to lead successful lives and it just, it's beautiful.