Let us continue with the discussion on deoptimal designs and other optimal designs looking at the deoptimality value for first order designs when we take the determinant of the moment matrix M given as determinant of X prime X by N power P where P is the total number of parameters we get that as the determinant of the identity matrix of order P which will be equal to 1. So for models which discuss the pure first order terms and the first order plus interaction cases the value of maximum of zeta determinant of M of zeta is equal to 1. Hence the variance optimal designs are also deoptimal in nature. Let us now look at another alphabetical optimal design the A optimal design this criterion addresses the issue of suitable estimation of the experimental designs model coefficients it deals with only the individual variances of the regression coefficients and we also note that to find the variance of the estimated parameters we refer to the variance covariance matrix and we look at the diagonal terms of this matrix and those are the variances of the regression coefficients. So the diagonal elements have to be multiplied by sigma squared the error variance to get the variance of the estimated parameters. Now how do we define the A optimality criterion it is defined as identify the minimum corresponding to the trace of M of zeta inverse. So what we are doing is we are looking at the trace elements of the M of zeta inverse matrix and we want to minimize the values of the diagonal terms okay and here trace represents the sum of variances of the coefficients weighted by n. So there are some experimental design based computer software that use A optimality and previously we have talked in length about the scaled prediction variance and it is a useful measure of performance the variances of the predictions should be kept under control and even if there are certain regions where they become unbounded then we have to relook at the relook the experimental design strategy. And the scaled prediction variance control helps us to make reliable predictions in the experimental design space there are many experimental designs and in the analysis unfortunately the scaled prediction variance is not given as much attention as it should be. So related to the scaled prediction variance is the G optimality criterion and the objective of this is to protect against the worst case variance prediction. So the model should not only predict well at the design points but also at all other points in the design space and we will assume that the factors are all quantitative and the G optimality criterion is given by identifying the scaled prediction variance in the experimental design space and finding the maximum value of that. Then our aim is to find the design zeta such that the maximum value of the scaled prediction variance is minimized okay. So the objective of the G optimality criterion is to minimize the maximum value of the scaled prediction variance so which is the design that is going to meet this criterion. And for the G optimality criterion we have to identify a region usually the region is cubical in nature or spherical in nature. So we can expect the G optimal criterion as n xm prime x prime x inverse xm and try to find within the region of interest R which is the maximum value and then identify the design zeta such that the maximum value of the scaled prediction variance is minimized a small typos there and just correct it. So now the inverse is expressed properly so what we are just doing here is substituting for the scaled prediction variance. So when the assumption that the errors are independent and have equal variances is a reasonable one we can see what is the minimum limit for the maximum value of the scaled prediction variance. So the region R of interest the maximum value of the scaled prediction variance of x will be greater than or equal to p. So the minimum value would be p where p is the number of parameters and the maximum value can be higher than the lower limit higher than this lower limit of p the number of parameters including the intercept beta hat not. Now we can define the G efficiency my target is to have the maximum value of the scaled prediction variance to be as low as possible and it should be p that is the lower limit but the actual design under consideration based on several factors may have a different value and that value would be higher than the value of p and the ratio of p to the maximum value of the scaled prediction variance for this particular design under consideration is termed as the G efficiency. So let us see what is the G optimality criterion for 2 power 2 design considering only the main factors and not even looking at the interactions. The number of parameters is 3 that is also equal to p and the design space is bounded by plus or minus 1 for all the variables x. The scaled prediction variance is given by n into variance of y hat x by sigma squared and we get n xm prime x inverse xm. We have divided by sigma squared because we do not know the value and that does not really depend upon the design it depends upon the actual experiment once it is performed it depends upon the error variance which is assumed to be constant at sigma squared. We do not know the value but for evaluating different designs we do not need it and so we divide by sigma squared and get rid of it. And then you can also have an artificially created small scaled prediction variance by having a design with large number of values. So n would be there and where n is the size of the run and by now you can show that n x prime x inverse for the design under consideration is I3 which is the identity matrix of order 3, 3 rows and 3 columns. So you may want to check what would be the n for the 2 power 2 design that I think you will figure out by now and you can also see what is the terms coming in the diagonal. Well the answer is very straight forward you are going to have 4 run experiment because it is a 2 power 2 design so the size of the run would be 4. So the scaled prediction variance is related to xm prime and that xm prime is the coordinate point expanded to model space so we have a scaled prediction variance comprising of xm prime that is written as 1 x1 x2 1 corresponds to the intercept because we are going to multiply this one with the intercept x1 would be corresponding to the setting of factor A, x2 would be corresponding to the setting for factor B and then we have I3 which is the diagonal matrix identity matrix with diagonal elements of 1 order is 3 and then you also have xm which is converting this matrix into its transpose so 1 x1 x2 along the rows will become 1 x1 x2 along the column. And so you have the scaled prediction variance of x as 1 plus x1 squared plus x2 squared and in the design space of interest our design is a 2 power 2 factorial design where all the factorial points are located at plus 1 or minus 1 so x1 squared will be 1, x2 squared will be again 1 so you will have 1 plus 1 plus 1 which is 3 and that is exactly equal to the number of parameters which are being estimated. So we have P is equal to 3. Now the G efficiency criterion for this particular design would be 3 divided by 3 that is the P which is the number of parameters in the maximum value of the scaled prediction variance in the region of interest for the previous design the maximum value of SPV is equal to 3 and so we have a G efficiency of 1. So the D optimal design is also G optimal for this case and in the same fashion it is easy to show that the first order design in K design variables for the cuboidal regions where the design points are located at plus or minus 1 only that means all the points are at their extremes in the design space at 2 level design of resolution greater than or equal to 3 results in a maximum x belonging to RSPV of x is equal to P okay. So a 2 level design means factorial design with 2 levels and the resolution greater than or equal to 3 you please refer to the lecture on fractional factorial designs where the design resolutions have been discussed and hence all these orthogonal designs 2 level orthogonal designs with resolutions greater than or equal to 3 would result in G optimal 1 for the first order model. Now let us look at another optimality criterion this is called as the V optimality criterion and the basis for this is to consider the prediction variance at a selected set of coordinates representing different points that are of interest to the experiment in the design region. So we identify some coordinates in the design space which are of certain interest this is coordinate 1 x 1 coordinate 2 x 2 so on to R such coordinates in the design space and how to choose those points there is no hard and fast rule here a set of test points from which the design was selected or it could be coordinates that have some specific importance to the experimenter and so any design that minimizes the average prediction variance over the set of m points is said to be a V optimal design. So actually there is a typo here it should be R I just make that correction then we look at the related criterion called as the I optimality criterion and the objective of this is to produce or provide a single measure of the model's prediction performance by means of an averaging process. So the averaging is carried out over a certain region of interest R and the moment we have averaging it also implies integration over a continuous domain and then after carrying out the required integration using the suitable function we divide it by the volume of the region R and we get the average this is a standard mathematical procedure for finding out average. So let us look at the integral over the region of interest that gives the volume of the region and that is equal to k the I integral criterion is identify the design such that it minimizes the average given by 1 by k integral over R scaled the prediction variance of x dx and that is denoted by the design zeta such that it takes a minimum value of I of zeta where I refers to the integral the integral is given here 1 by k integration over R scaled the prediction variance of x dx. So we are substituting the definition for SPV into the integral and we get this. So earlier we had n into x prime x inverse but from the definition for m which is equal to x prime x by n. So x prime x inverse into n will be m of zeta inverse. So this derivation is pretty straightforward and that is why instead of having x prime x inverse into n in addition to the other 2 terms we have m of zeta inverse and that is the I of zeta the integral value for a particular design under consideration and we have to identify the design such that the integral is minimized. So even though these mathematical formulae look a bit formidable the meanings are pretty straightforward. So you can build even more sophistication into this approach by assigning weights to certain points in the domain or coordinates in the domain which perhaps are more valuable than other points. So it is very design specific and after this definition it is up to the individual experimental program where this may be applied. So the I optimality criterion is a reasonable method for deciding upon the suitable experimental design. Designs based on this criteria of a good average SPV are expected to yield a satisfactory results throughout the design space. However the D optimality criterion is more popular and the G optimality as well as the I optimality criterion are not as popular as the D optimality one. So we also have the I efficiency criterion we define a particular optimality criterion and then we also try to find the efficiency. So we have seen that the I optimality criterion was given according to this integral and we can also find the minimum value of such an integral. What design will give you the minimum value and then the actual value I of zeta star. For a particular design I have chosen I will get a certain value of I of zeta star after the averaging is done. Then I compare it with the minimum value of I of zeta. I compare it with the design which is going to give me for the same number of variables a minimum value of I of zeta. The ratio of the 2 is termed as the I efficiency. So we have used simple first order models to illustrate different optimality criteria just to see how to carry out the different matrix operations the inverse of the matrix operations and then how the integration has been done and so on. So the design optimality criterion is very interesting and a very useful concept. It gives you additional insight or information to your experimental design. There is more to every experimental design than simply finding the regression model or the equation describing the designs performance over the design space. So the objective is much deeper than that after all we are developing only a simple regression model straight forward regression one getting some statistical parameters and finding which of them are significant which of the coefficients are significant. So over and above all these things we have to look at how good the model is in the design space and we have to a priori feel or select a model which will give us the desired features. So lot of planning is there in the choice of the experimental design. So upfront we have to ask ourselves what do we want out of this model and based on that we can develop a suitable optimality criterion and see which model or design will fit into that. We will give a desirable value of the optimality criterion and there is no one single optimality criterion there are many optimality criterion and we can choose one which is closest to our expectation. And the design of experiments and response surface methodology are becoming very popular due to many software availability and also it is not that difficult to apply the different optimality criteria. So we have lot of flexibility in choosing a particular experimental model. We know we are not constrained to select one particular model because that is being commonly or popularly used in the literature. So we need good designs that perform satisfactorily over a wide variety of possible circumstances and designs which are optimized over a narrow value of idealized conditions and assumptions are of little value. So the first order models including the interactions between the various factors the standard designs are optimal. However the standard response surface methodology designs like the box benken design or the central composite design are rarely optimal for the second order case even though they have several desirable features. And for the purpose of response surface methodology analysis we more frequently resort to the central composite design or the box benken design strategy and they should be used or considered whenever possible. And there may be many situations where the CCD or the BBD may not be implemented these could be due to constraints or unusual design sizes. Constraints can be you may not be able to perform certain experiments along the axial nodes which is recommended by the central composite design. So when we are not able to do the standard designs we have to go for a computer generated designs that are based on certain optimality criteria we have discussed previously. Okay so there is a table which is given by Myers et al. 2009 which compares typical standard second order designs for a spherical region. I am just giving an illustrative summary for a more complete summary you may refer to the reference book. So we can compare standard designs I have only chosen the central composite and the box benken design. You can see an additional parameter which is the number of center runs NC. K refers to the number of factors in the design 2 factors, 3 factors, 4 factors. Then we have the different designs and you can see that the D efficiency as well as the G efficiency have been listed. For number of center points equal to 1 the D efficiency is 98.62 but the G efficiency is 66.67%. When you increase the number of center points the D efficiency slightly reduces from 98.6 to 96.9 whereas the G efficiency dramatically improves to 87.27 from 66.67. And then for K is equal to 3 you have a experimental size of 15 for a central composite design with only one center point. The D efficiency is quite high but the G efficiency is pretty low. When you increase the number of center points again the D efficiency slightly falls off but the G efficiency improves dramatically. For a box benken design even though you have only one center point the G efficiency is quite decent. And when you actually increase the number of center points both the D efficiency and the G efficiency decrease for the box benken design. For a box benken design with NC is equal to 1 the D efficiency is quite okay at 97 but when you increase the number of center points to 3 it drops slightly to 93.82. Whereas the G efficiency which was quite low to start with one center point has now reduced even further to 66.67. And then with 4 factors the central composite design and box benken design give pretty much the same D efficiency and G efficiency. So it appears that increasing the number of center points in the CCD or BBD is not really good because the optimality criteria may decline. So when we went from 1 center point to 3 center points the box benken designs D efficiency and G efficiency actually declined. And here also for the central composite design increasing the number of center points from 1 to 3 actually did reduce the D efficiency percentage but on the other hand the G efficiency percentage increased. So the moral of this table is both the optimality criteria need not go in the same direction if one increases the other may decrease. But we cannot rule out the center points because the larger the number of center points the more would be the degrees of freedom for pure error. It also helps to estimate the pure error and when there are more center points you can estimate the pure error component more reliably and it also helps to stabilize the scaled prediction variance. So when the CCD had only one center run the G efficiency was quite low. So when only one center run was there the G efficiency was quite low at 66.67. For the spherical design when the center run is absent the X prime X matrix may be singular or near singular and that was not acceptable. So the use of only a single center point leads to a large SPV of X in the design center as the number of center points increase these prediction variance becomes more stable and the G and D criteria react adversely to the presence of many center points due to the nature of their definitions. You can see in the box Benken design even the D efficiency reduced from 97 to 93.82 when you increase the number of center points from 1 to 3. So that is because of their nature of their derivations and we have seen that the standard RSM designs are reasonably close to the best optimal design in the D optimal or the G optimal sense. For example when you look at 4 factors central composite or box Benken design the D efficiency is close to 100% and the G efficiency is also close to 100%. So the CCD and the BBD designs would be used when there are more number of factors that we considered and they are pretty efficient and it does not mean that you should always get all the criteria to be close to 100% whenever you are going with less number of factors okay. Even for smaller number of factors the D efficiency values are pretty high it is only the G efficiency which are quite low for cases involving less number of center points. So there is no one hard and fast rule or magic rule to get the best design and another thing to notice the 2 criteria which are very commonly used do not show us the stability of the scaled prediction of variance in the design space. So just relying on a single value of the D optimality criterion is not recommended and the design is multidimensional in character and we cannot go with only one criterion. We should actually look at the scaled prediction variance in the design region. So in addition to the different optimality criteria we should also pay attention to the prediction capability of the model in the design space. So how to represent the scaled prediction variance for a 2 dimensional problem involving only 2 variables 2 factors A and B or X1 and X2 the scaled prediction variance is quite easy to plot but when you have more number of parameters it is not easy and also instead of sigma squared we use the S squared which is the mean square error. So we use variance of Y hat of X and estimated value of that that is why it is represented as variance of Y hat of X hat and instead of using sigma squared here we use S squared. So now we are talking about the scaled prediction variance estimation or the estimated scaled prediction variance. So in addition to the optimality criterion we also have to pay a closer look at the scaled prediction variance. So from graphical methods we aim to get a bird's eye view of the scaled prediction variance of the entire design space irrespective of the number of factors and the number of center points. And so if we are able to represent even for multidimensional cases the distribution of the scaled prediction variance in the design space we can have overall view or a bird's eye view on how good the design is. How good the design is performing at different locations in the experimental space. So we can see which regions in the design predict poorly or nicely and it also helps us to see how the non-designed points or future points in the design space get predicted. So when you look at CCD and BBD from the scaled prediction variance point of view the box benken design has superior minimum prediction of SPV of X at the cost of inferior maximum prediction at the range of the design space. So what this means is BBD ensures that the scaled prediction variance is quite small at the interior of the design space. But the SPV actually blows up when you go to the extremes or the boundaries of your design space. So at the design edge the scaled prediction variance of X is pretty high but only a small proportion of the design region have these high values. And the G efficiency which considers only the maximum SPV of X may be quite deceptive as it does not consider what is happening in the interior of the design space. So a single number efficiency hence cannot be truly reflective of what is happening in the entire design space. Mostly the maximum scaled prediction variance of X in a second order design occurs at the design boundary or at the design perimeter. So the G efficiency criterion reflects on what is happening at the edge of the design only. So I have just given you a illustration or a brief introduction to the scaled prediction variance distribution in the design space. It is important for us to represent it graphically and there are certain graphs called as the variance dispersion graph and the fraction of design space plots. They are somewhat difficult to plot and they cannot be done manually in most cases and we have to resort to statistical software which can plot these fractional design space plots and variance dispersion graphs. So I have given only a brief overview on some of the advanced features of experimental design but this introduction should form the suitable basis for further reading. So we have come to the end of the introduction to advanced or optimal design concepts. So this may be the starting point for a further advanced course on statistical design of experiments. So I have given a complete overview of the basic statistical principles that are involved in design of experiments, looked at some popular design of experiment techniques and finally given an introduction to the more advanced experimental design strategies. So we have now a good background and knowledge of conventional design strategies and also the capability to read or no further on advanced experimental design concepts. So I will conclude at this point and in the final lecture next I will summarize whatever we have covered in this course. Thank you for your attention.


Hi and welcome to Program Analysis. In this lecture, we will look into call graphs, which are one representation of a program that tells you what calls may happen when you execute a program. We will look into static call graph analysis, so techniques for statically reason about the calls that may happen when you execute a program. And most of the examples and analyses we look here are designed for Java, but they are also generalized to other object-oriented languages. Let's start by looking at what call graph analysis and call graphs actually are. So a call graph is an abstraction of the program that considers all the method calls that may happen when you execute this program. In such a graph, you have nodes that represent methods, and you have edges that represent calls or possible calls between these edges. All of this is flow insensitive, which as we've seen last time essentially means that the analysis does not consider the execution order. So it doesn't really tell you in which order these calls happen. It just tells you that there may be a call from this one method to this other method. All the approaches we talk about here will be static call graph analyses. That means these approaches abstract what calls may happen at runtime without really executing the program. You can also have dynamic call graph analysis, which we do not cover in this course, but it would essentially execute a program and then look at what calls really happen at runtime, which typically gives you an under-approximation of the calls, whereas a static call graph analysis gives you an over-approximation of the calls that may actually happen in the program. Let's start by looking at an example. And that example is an implementation of the observer pattern, where you have some object that may change at some point, and then some other object that is observing the first object in order to get notified whenever such a change happens. So in our example here, we have one class called subject, which is an extension of the observable class. So that's the kind of object that you can observe. And then the main class itself implements the observer interface. So this is the class that will then get notified whenever the subject is changing. And then here in the main method of this example, we are first of all instantiating the main class itself, then create a subject, and then add the main object to the subject as an observer in this ad observer call down here. And then whenever the subject gets changed, for example, because we just call this modify method, what will happen is that the subject is notifying its observers, and this will lead to this update method being called, which then knows that the subject has been changed. So let's look at the call graph that we would get for this simple example. As I said, every method in the program is represented by a node, and the edges represent the calls relationships between these nodes. So for the example, we would have one method for our main method of the main class. So this would be one of the nodes here. Then we would have another node for the constructors of the two classes that we have here. So there would be one for main init, and we are denoting constructors with this init notation similar to what happens in Java. And another one for the subject, where we also have a constructor. And now looking at what the main method is actually doing, we see that it's invoking the constructors of the main class and of the subject class, and therefore we will have edges here that go from the corresponding call node to the call node. Then we also have some more calls in the main method. For example, there's a call to subject.addObserver, and again we will have a corresponding edge for that. And finally, there's a call to subject.modify, and again there will be a corresponding edge. Note that the order of these calls is not specified, so I'm drawing them in some order which happens to be the one that you see in the source code, but this is just coincidental. It's just a graph and the edges that are outgoing from a specific node are not ordered. Now let's look at the modify method itself, and as you can see there are two more calls here. One of them goes to observable.setChanged. So there will be another node for that method here, which is called from subject.modify, and then when the subject has been modified it will notify its observers. So there's another call to observable.notifyObserver, and we're having another edge for that. Note that when the observers are notified because of the implementation of the observable, what will eventually happen is that main.update is called. This piece of code is not shown here because it's in the observable class, but in the complete call graph there will also be a call to main.update that comes from the observable.notifyObservers method possibly through some other methods, so I'm just putting a dashed line here to show this. There's also another call in the update method itself to system.printLine, which I'm also just omitting here for simplicity. So far so good. So this was the easy case. Now the interesting cases occur when the program has polymorphic calls. What is a polymorphic call? Well, a polymorphic call is essentially a call that may go to different methods and you just don't know which method is called based on the statically declared type of the base object. So here's an example of such a polymorphic call where in the main method we are creating some collection using this makeCollection method, which is defined down here. And what this makeCollection method is doing is that it's sometimes creating an array list and sometimes creating a hash set. And then no matter what this collection really is, in the main method we are calling the add method of this collection. How will the call graph for this program look like? So again, we will have a note for the main method. And because it's calling the makeCollection method, which happens to be implemented in the same class, there will be an edge for that. So far so good. So now the interesting call is the one that goes to c.add. And here the problem is that by just looking at the code we do not really know what method exactly gets called here. If you just look at the main method itself, you only know that c is a collection and that means in principle every subclass of a connection that has an add method could be the method that gets called here. So there may be an array list.add, call, there may be a link list.add, call, there may be a hash set.add, call, and many others. Now if you have a slightly more clever static analysis, it could maybe also find out that in this program c can only be an array list or a hash set. So we have at least these two nodes here, one for array list.add and one for hash set.add. But because we statically do not know which of these two gets called, we will have two edges that go to these two methods because we just don't know which one will be called. And again if the static analysis is a little bit more naive, it would also have edges to many more methods, not just these two subclasses of collection. So to better understand the problem of building a good call graph, let's look at a slightly larger example which does not come with concrete source code but just remains an abstract example. So in general in the program you will have many methods and each of them is represented as a node in this graph here. Some of these nodes are so-called entry points, which are basically nodes that you assume can be called from somewhere. For example if you have a library, these may be all the public API methods that the library is exposing. And then in the program itself there are lots of core relationships that we have depicted here and what you see in this version of the graph here is that there are a lot of them. And in many cases methods have more than one outgoing node edge, sorry. So there may be different reasons for that. One may be that the method is just calling different methods, so it may have different call sites and it calls one method in the first call site, another method in the second and then you have two outgoing edges. But it could also be that some of these call sites are polymorphic or at least potentially polymorphic based on what the static analysis knows. And in our example here we have a lot of those, so for example this one is a potentially polymorphic call site simply because it has two outgoing edges here. And then we see many more of those, so this one also has two outgoing edges which could be because it's a polymorphic call site. Same here, same here, same here, same here, same here. Here you have even three outgoing edges and then again two here and two here. Now let's assume that this is the most conservative call graph that our static analysis could produce. Then what you actually want to do is that you want the static analysis to rule out some of these potential calls in order to get a smaller call graph that still captures all the calls that may happen at runtime but does not include too many of those that may actually never happen. And to do this one thing a static analysis can do is to distinguish between methods that are reachable and methods that are not reachable for example because they are certainly not called anywhere in the program. So for our example let's assume that we have some reachable methods for which we know that they can be reached and I'll just use this green color to denote those. And then let's assume we also have some unreachable methods where we statically know that they can actually not be reached. So for this example let's for example say that this one is a reachable method and that means that this one is also reachable method and this one as well. And then some others that are called here are also reachable, let's say also this one and this one and also this and this and this. But then there are also some that are not reachable. For example this may be the case for some of these entry points. So let's say you're analyzing a library but you know for sure that none of the library clients is ever calling this one public API method then this is unreachable and therefore you can discard it from the call graph. And then if one of these methods is unreachable there are also some calls that may never happen and therefore you will find out that some of the edges may also be unreachable. And let me just make the arrows red to denote that fact. So these red edges will recall edges that actually may be eliminated. So for our example here let's for example assume that this edge is one of these edges that may be eliminated and the reason why the analysis may know about this is for example that it knows that the type that is called here can actually never be used in this program. And if we know that some edge can be eliminated it also means that we do not need the method node anymore because then effectively if this method is not called anywhere else this method will also be unreachable. So for example let's say this also happens here and at a couple of other places then we can step by step remove nodes and edges from this call graph. If we have like in this example a node that can be reached by some other method but maybe not by the one above here then it still means that we need to keep this node because there is an edge that can reach this method and therefore it's still a reachable method. So for the example let's assume that we know about a few more edges that are unreachable let's say these and also these and these maybe also this one here. Then based on that also that one based on that we can remove some more methods because they become unreachable and in this example that would be this one also this one and then because that is unreachable we can also get rid of this one and also of that one. And now once the analysis has found out that some of these methods are unreachable and some of the call edges that we had in the conservative call graph are actually not needed it can prune the graph by essentially removing everything that is unreachable which means in our example that we will get rid of all of this and also we can remove that edge and that edge and those parts here and then also this edge which means that the call graph that we now have is much smaller and is much closer to what may actually happen at runtime. So in general what a static call graph analysis wants to do is to prune the call graph as much as possible in order to focus only on feasible behavior so things that may actually happen when you execute the program and in order to do this it typically wants to minimize three things one is the set of reachable methods so that some methods that certainly do not get called are just removed from the call graph entirely also it wants to minimize the call edges so even if a method is included and maybe called somewhere maybe it doesn't have to be called everywhere so we can remove the edges that we know may not be feasible at or are not feasible at runtime and it by doing this also tries to minimize the potentially polymorphic call sites by essentially figuring out that a call is monomorphic and we know that a particular method is called instead of considering all the methods that may be called all right so now you've seen what a call graph is and you know what the goal of aesthetic analysis that builds a call graph is namely to get a precise but compact call graph meaning that it has all the edges that may happen at runtime but ideally not too many in addition to that and what we'll do in the remaining videos of this lecture is to look at different algorithms for achieving these goals and we'll start with a relatively simple and efficient set of algorithms called cha and rta then we will look at slightly more complex algorithms that also consider assignments in the program and then finally we look at the most complex set of algorithms that we consider here in this lecture which will not only create call graphs but at the same time also perform a points to analysis which as we'll see in the fourth video of that lecture is actually pretty useful to get precise call graphs all right and that's all I have in this first video thank you very much for listening and see you next time


My name is Sai Venom, and I'm a developer advocate with AWS. A lot of you might know me from my days at IBM. I'm the guy that does the lightboard videos. Well, I was a big fan of OpenShift back then, and it's still following me. I just can't shake this thing. But all jokes aside, I am a big fan of OpenShift, and today we've got an exciting demo and presentation ready for you. So again, we're going to be talking about app modernization on AWS, the focus on Rosa that's Red Hat OpenShift on Amazon Web Services. But I do have a couple of really cool demos lined up. But first, I'm going to pass it to my co-speaker Javier to introduce himself, and he's going to do the first part of the presentation today. OK, thank you, Sai. My name is Javier Naranjo. I'm doing the business development for AWS in Iberia. And today we are going to talk about what's I said, how do we see modernization based on the experience of our customers, and also explaining a little bit about Rosa, how we position it, which is the added value, and so on. So first of all, let's take a look to what our customers are asking. So when we are facing projects, when we are facing partners, what they say is the main four things that our customers are asking for, first of all, is to build applications, not to take so much care about the infrastructure, because actually they are looking for an infrastructure that is managing infrastructure. Of course, they want to scale quickly, because when we are talking about modernization, it's not only that you are creating application and features in a quick way, in a fast way, you also need to scale it in a very quick way. And of course, everything that they want to do and that we are doing in AWS actually is around security. So everything that we do had to do with security also. When we are building that application, we have to take a look to which are the key features of a modern application, but we call also cloud-native application. And this is a scale to a million of users has a global availability, because actually every single feature that we want to deliver needs to be available for everybody inside a company, or if we are talking about a service that is for our customers, then to our final users, being able to respond even in mild seconds, because actually every time that we are talking about the modern application, we are talking about two facts. First is velocity, how quick you are. And the other is how you are saving costs. So this is key when we are talking about modern application. And of course, also that cloud-native application still is working with a lot of data. So we need to have that into account. The white boxes with the blue letters that you see over there is just the benefit of the modern application. So luckily, we are just answering to the questions to our customers, because actually with each of any of the features that you see over there, we are satisfying the facts that you see around return of investment, cost, TCO, increasing the business agility. Of course, the customer experience, the user experience is key for us also, and increasing the efficiency of the developers, because we are providing global tools in order that they can use it to create their applications. And they are able not only to create in a quick way, but as far as we are working online with DevOps, it's not only creating the application in a quick way, but also being able to operate it in the day too, also in a modern scenario. What kind of workloads are using our customers for containers? Of course, applications, and this is the kind of applications you see over there. We are talking about web interfaces. We are talking about backends. We are talking about EOT data processing. But around, we see also staff regarding operational expertise, and then we are focusing on stuff like platform as a service, things like that, in which we are providing CI CD, infrastructure as a service, solution around management, security, governance of the staff that we are creating. Of course, what we are doing when we are modernizing sometimes is doing that lift and shift, not only modernizing. So we have to take care about moving applications, legacy applications, even like .NET, Linux, stuff that our customers are using on premise, and they want to use it in a modern way in the cloud. And of course, all around machine learning, because actually, I think every modern company that is trying to create a modern solution is focusing in that kind of stuff. What we see in our customers when they are modernizing actually is the steps that you see over there. Sometimes they discover that they are using some stuff that is not useful anymore, because maybe it's just a couple of users are using it, and they are spending a lot of time operating it, creating it, and just trying to provide new features when nobody is using it, so they try to reduce it. Sometimes they find a solution as a service, or software as a service, that it's better for them. It's not expensive, and they can use it to replace the stuff that they were doing, and they were investing a lot of time on that. OK, I'm going to migrate before being modern. So they do the lift and see, but the key point here is when you focus on modernize, you can decide if you go to replatform, and that way you will do it with containers, or you're going to refactor, and that way you will do it in an event-driving architecture using our Lambda function. You see over there that we are having different solutions for containers, even hybrid solutions in which we are providing the capacity to our customers to run containers on premise and also containers on the cloud. And of course, appears also this last piece that you see over there that is the platform as a service. In this case, what we are releasing is the Rosa service, Red Hat Open C for AWS. And this is because when we are talking to our customers, we see that most of the time, our CIOs, CEOs, they say, I'm investing a lot of time in working or creating the platform that I will use to create my applications. And I'm not focusing in my applications. I'm not focusing in the added value that I want to create to my customers and to my service. And this is something that we need to fix, because when you need to create your own Kubernetes platform, if you are doing it yourself, then you are in a trouble. Because sometimes when we are talking about enterprise, we are not doing IKEA furniture. We are not taking different pieces and putting it together. Because after this is happening, maybe we are doing at home. But when you are going to an office and you see the furniture that you have over there, it's office staff, professional staff, professional furniture. So the difference is key. I mean, it's just not something that you need to create. If you have tools that are already doing this for you, you need to be a professional with your business. And you need to do it using a professional tool, in this case, a platform as a service, which all the pieces integrate. You all know Opensiv. So I'm not going to talk in deep here about Opensiv. But the key difference is that we are not doing a building block solution. Just trying to integrate anything. So when we are thinking about this and we are talking about the alias that we have with Red Hat, it's key to understand that having all these pieces together and working integrates, it's really key. What we are doing is a natural way, a natural roadmap from the self-managed solution with Opensiv that you can run it even on premise. Then you can run that self-managed solution also in the cloud, in AWS. Red Hat was selling their own managed service solution with Opensiv dedicated that was transparent to the customer about what's running on top of AWS. But the relevant thing now is we have a new solution. Everything is integrated. It's a joint managed service. It's like Opensiv dedicated with a new flavor because actually we are providing from AWS support to the infrastructure and to the integration with that infrastructure. And Red Hat is providing support to the platform as a service to the solution that is running on top. And that said, I'm going to pass this now to my colleagues side. And we are going to see a little demo. We'll be coming back on, but it's time for the demo and just want to kind of recap a little bit. I think the way that Rosa is operated and managed, we're going to dive a little bit more into detail there. But I think it's a model that works really well. And that's because even looking at the roadmap presentation earlier, it's very clear there's a lot of complexities that are associated with managing Opensift. Yes, of course, with all of the tools and services that make Opensift easier to run, a lot of that is easier than bare vanilla Kubernetes, but there's still a level of complexity. And so right now what I want to do is kick off a demo. I want to show you what the Rosa experience looks like. And it's an experience that starts in the console, in the AWS console. And for better or worse, I'll say that this is about the extent of which it's integrated into the console today. I think Opensift users like Opensift and Opensift-esque consoles and user experiences. So when you start in the console, you really just have to click one button, you enable the service, and then immediately you'll notice it says download CLI. And the first thing it'll do is it'll take you to a Red Hat documentation page. Let's quickly check that the ethernet is connected here. I think we are good. Well, luckily I have it open in another tab. Regardless, you can see here that it's pretty straightforward. It's a one, two, three, four step. You download the CLI, you make sure that you're kind of logged in, and I'm so confident that this demo is gonna work today. And that's partly because Rosa has this amazing command. It's called Rosa init. And what this does is essentially make sure that the system is ready. It's ready to create a cluster. You have the right quotas in place. I ran the command twice, but just I wanted to zoom in a little bit. You can kind of see it right here. What it's doing is it logs in with my username and validates the credentials. Make sure I have the quota, the policies are in place. Now folks, I know that if you're like me working with AWS, one of the things that trips you up probably is making sure all your IAM policies are in place. And so I think this experience just ensures that all the permissions are set correctly. I'm feeling good. I'm feeling ready about this demo right now. So what I'm gonna do is create a cluster using the Rosa CLI. And it's very straightforward. I will quickly point out that the dash dash STS. This is the security token service from AWS that makes sure that the service is created securely. Obviously Rosa needs a lot of permissions. And so the cluster itself is gonna need to call APIs of AWS on my behalf. So it makes sure that all of that is running correctly. So let's say Rosa demo comments. It's too long. There we go. So pick something a little shorter. We picked the OpenShift version. Now what I'm gonna do is walk you through some of the different options that'll kind of give me. So it's using the ARNs that I've already created. External ID, this is for if you have a special account where you need a custom account ID. Operator roles, prefixes are some more permissions, things for Rosa to have access to AWS. Here's a cool one. You can actually deploy Rosa across multiple availability zones. When we talk about DR strategy, this is gonna be the best possible approach for a majority of customers. The chance that when we look at a deployment across multiple availability zones, you get that high availability that a lot of industries and customers need. Keep going here. Let's pick a region that we wanna deploy it in. Don't worry folks, we're all almost done through this set of kind of prompts here. A private link cluster. Say you wanted to have a cluster that's not exposed to the public internet, workloads that can run within the context of their cluster without needing to reach out or have services reach in. And of course, if you do need to access certain services there's a number of AWS services that have private link end points. Essentially make sure traffic doesn't go on public pipes. Kind of going forward here. If you wanna install into an existing VPC, let's just say no, we'll create a new one. Customer managed key. If you wanna use a custom key for encrypting data, you could choose that right there. Choose the compute nodes. A lot of options here. Let's choose M5 X large auto scaling. Now you can actually configure this automatically here. So essentially it'll automatically scale up the nodes in response to load. And there's some default parameters that are set there. Two compute nodes by default. Some CIDR block information. It's a question for if you wanna encrypt at CD data. At CD data is already encrypted. And by the way, this terminal is so cool. If I want some more information I can just hit the question mark and it'll tell me a little bit more about what exactly it's doing. So by default it is already encrypted. It's kind of an additional encryption on top. So we'll go with the default there. Workload monitoring. This is if you don't want Red Hat SREs to see your application logs. And there we go. And it's gonna start creating that cluster. Now we went through a lot of different options there. What if I wanted to do that again in the future and change just one option? It gives me the entire command. So in the future I could just copy that, change the command a little bit and redeploy it. I think that's one of the, I think cool things about this kind of interface. All right, let's jump back to the console. Now something that I'm sure a lot of you are familiar with here is the Red Hat Hybrid Cloud Console allowing you to see clusters across a number of different environments. So here let's click on the clusters itself which is gonna launch into the OpenShift dashboard specifically and so I created a cluster last week that I'm gonna use for the demo today. By the way that one that we just created we already see it in the console and it's installing. But we're not gonna wait for that today. We're gonna switch over to the cluster that I've already created and here we can see a few things. Now first off, I think this is what I was saying about how although this is OpenShift running on AWS it's the same familiar OpenShift console and experience that you know. And so here we can see the cluster, we can do things like updating the version. I won't go into too much detail here because again, OpenShift is OpenShift. But there is one thing that I wanna quickly show here. If we go to the add-ons tab there's some really interesting things here. Today there's only three but I think there's a lot of potential with what Red Hat and AWS can do together here with these add-ons. For example, there's the cluster logging operator. And if we go into the configuration here we can see that I installed it saying I wanted to stream logs to AWS CloudWatch and collect application logs, infrastructure logs. And in fact, if I go to my CloudWatch console of course I'll need to do a little login again because it's invalidated so we'll give that a second. So if we go to CloudWatch we'll see that the logs have actually started streaming into CloudWatch from my cluster and then go to my log insights. Let's make sure I'm in the right region. Love it, the internet's working great. We'll pick all my log groups. So we'll run a quick query and boom we can see the logs coming directly from my cluster. For example, I can open up one of these and we can see that this particular container is running in Rosa demo and it's the IP controller manager. But there you go, that's the first part of the demo here. I'm gonna switch back to the slides here and I wanna quickly talk about a few things and we'll kick into the next one. So when we look at the number of options that are available on AWS for running compute as a whole it really comes down to a matter of speed versus flexibility versus whether or not the platform is opinionated. And so what we can kinda see here is that the most opinionated gonna be kind of a serverless approach. Really the only thing you're worrying about there is application code. And on the other end of the spectrum we have things like EC2 which have ultimate flexibility. In fact, you can run OpenShift yourself on EC2 if you wanted and a number of customers do that. But when we talk about containers specifically when we look at something like ECS, EKS now there's a number of solutions on AWS that support things like automating the delivery with EKS blueprints, things like AppMesh for worrying about service level networking. There's container registry, there's a number of solutions but with the Rosa which you get is a turnkey application platform and that is it's kind of baked into the platform itself and so I wanna quickly talk about what that means for users. So again with EKS we saw that landscape slide tons of solutions in the CNCF. You can use GitHub, Jenkins, you can use Istio, you can plug and play with all of these solutions and you can get exactly what you're looking for. And this is very powerful. But for many customers, and I think this is a very kind of powerful image here, you flip the switch here and you realize Red Hat provides baked in supported blessed solutions for basically kind of all of the things that you see up there and they're based on open source technologies. And I think this is a very powerful message for customers that need that opinionated approach when building a complete Kubernetes application platform. And so I think one thing that I really wanna talk about here, we mentioned it a few times already is what's the support model? Who's responsible for what? Because when you're managing OpenShift by yourself or even on AWS running on something like EC2 you're actually responsible for quite a bit. The control plane, the data plane, as well as the compute itself. Of course, support and billing will come through Red Hat. Now the model, the difference with Rosa is that you have Red Hat SREs that actually manage that compute data plane and control plane. They're in the back end and they're making sure things run smoothly. When you open a support ticket, you can open it against AWS or Red Hat. Of course, billing is gonna be coming through AWS. I think this is kind of critical to understand that Rosa is this kind of fully managed approach to running OpenShift. Now, of course today we were talking about modernization of applications. What we see here is that there's a number of services on AWS. Really, this is the bread and butter of AWS. It's what we do well. We have so many services that allow you to do things from storage to CI CD, for storing containers, a central managed service. Now, of course, a lot of these things customers may choose to run them within cluster, but also when you're looking at a bigger scope, when you're running multiple clusters in production, you may wanna have a managed service centrally that handles some of these things. And so that's what I wanna talk about in my next demo here. I wanna show exactly how easy it is to work directly with AWS from within your clusters. So going back to my cluster here, let's see. I was actually on the right page. I'm gonna open the console. And so what I'm doing is launching into my OpenShift cluster, my Rosa cluster on AWS. I'm gonna go to operators and go to the operator hub. And I'm just gonna search Amazon. And what we see here is a number of operators for AWS controllers for Kubernetes. This allows us to operate AWS from within OpenShift. This is pretty powerful. So let me scroll down here. Let me find one of these. Let's say RDS and we'll hit install. And we'll see that it gives me the option to go through here and install this operator into my cluster. And notably, and I think OpenShift does a great job with this, it shows me the APIs that become exposed. What can I actually do once this operator is installed? So let's go to my installed operators. I've set up the S3 operator and I wanna create a bucket. So we'll say create bucket. And we'll name this bucket commons demo one. And then the name of the bucket itself actually needs to be completely unique. So seems random enough. And we'll hit create, keeping all the other defaults. All right, so let's take a look. We'll click the bucket, make sure there's no errors. Make sure I didn't mess anything up. Resource sync successfully. Jump to my S3 console. I'm in the right region already. There we go. And just like that, I was able to get a bucket created using a Kubernetes-based artifact. That is, I'm able to work with AWS using a CRD within Kubernetes. I think this is very powerful. And of course, if you wanted to take the information from that actual deployed cluster, of course I lost it. Here we go. And say I wanted to pass in something like the location into a config map or secret. So that my containers could access it. You can do that fairly easily.  I really wanted to show this today because it shows that with OpenShift running on AWS, you're able to actually integrate and operate services running within AWS as well. All right, that's my demo for today. I'm gonna pass it back to the slides. Javier's got a couple more that he wants to share with you. Okay, so that was a really good content, really good demo. Finally, we want to talk about which is the consumption-based pricing. Of course, when we are talking about OpenShift self-managed, you will see that the price is much cheaper when you go to a managed service solution. Actually, you have to dimension it in the proper way, but you need to understand which stuff you are going to pay and which part of the price that you are pricing is mapped to any of the stuff that you see over there. First, there's some Rosa services fees, and these fees are compelling or compelling to the worker nodes. Actually, every four CPUs, you are paying a price for the fee in that workload worker node infrastructure. So it depends on the kind of instance that you are using. I mean, if you need to use for your worker, for your workloads in that node's more capacity, then you are paying more. It depends on the dimension that you are doing, and also you have to pay a little fee for the control plane, but actually you will see in the public places that it's a ridiculous price for that, and then of course you have to pay for the infrastructure that you are going to use for the control plane and also for the workers. The good news is that the same way that you are paying for one year, up to three years for the subscriptions with Red Hat, you have a similar model, so if you are quite compromised for Rosa for one to three years, the price will be changing, and of course if you are going to use it for more years, it's going to be cheaper, but the pricing is really simple. If you go to the internet and you just write Rosa pricing, you will see different samples with different control planes, different worker nodes, and so on, and it's really useful. My last slide, can you pass it there because this is not working, please? So I'm here. It's just to talk about the alliance. This is not something that we are just creating now for Rosa. I mean, we have been working with the AWS for many years now. We have a lot of workloads and different solutions on top of AWS. Red Hat is using a lot of stuff on top of us. In fact, you have to understand that the first managed service that they were creating, OpenCiv dedicated, was directly on top of AWS, and what they say, my colleagues and my peers from Red Hat, they say, you know, the first stuff that we are testing for new releases and the first cloud in which we are testing everything is AWS. So I think that is a natural step in the growing of a managed service to go to AWS with Rosa for that integration with OpenCiv in a joint managed service with AWS in order that we can provide a better service and a better solution to our customers. And that's all that we have for today. We hope that was really interesting for you and hopefully see you around during the next days here. Thank you. Thank you.


Dysgr朗, studied D exams, I want a little UK perspective about how we're thinking about this whole area across a lot of committees in the UK. Obviously, as Eric said, it's all being driven by this cost. Here, of course, it's in pounds rather than dollars, but it's the same sort of picture. Full Genome sequencing is approaching the costs of clinical tests, which are already being done inside our health system. Of course, we have an NHS system and comprehensive healthcare in the UK, you can see a point where those lines are going to cross and it's going to be worth thinking about storing individual sequences and processing over those. So, there have been a number of committees looking at the kind of things you need over a number of years. These are a few of the high points. Firstly, linking those kind of research activities, we have the Medical Research Council, the Wellcome Trust funders like that, sorry? A little closer to the microphone. A little closer to the microphone, right. And then we have the NHS which also has a research side and so there was a creation of an office to kind of link those two together. One of the things that came out of that was a board looking at health records and so the need to have a generic ability to research over health records. We have some legal provisions that makes that feasible in terms of ethics. It's a question of actually putting in place the infrastructure. And so we have some representatives from the Scottish program here and also from one of the English programs, John Parkinson. So that's one side. Then, of course, there's the genomic side. So there was a House of Laws report in 2009 saying that something was happening in genomics and the medical system should start taking that into account. And the government's response to that has been the creation of a committee which has been looking at this whole area, this human genome strategy group. So what do you need, coming out of some of those discussions, what do you really need to make this feasible? You know, at the top you've got healthcare professionals. At the bottom you've got the kind of research activities in yellow, the sequencing type things and then the sort of patient record researching over those. So, you know, down here we have the kind of research activities, NHGRI, Welcome Trust, Sanger, EBI involved in the UK sort of side of things is EBI and Sanger. We have the reference genome. So you can start thinking about a healthcare professional producing, asking for a whole genome sequence and comparing that with the reference sequence and ending up with just a variant file and attaching that to the patient record. It's small enough to attach to the patient record. Those patient records, you know, they exist inside GP surgeries. You could attach files for images there, so there's no reason why you can't attach a summary of variants. So, once you've got that, so what's actually missing in terms of being able to analyse that, use it in a clinical setting in a sort of, you know, large scale way as opposed to the sort of individual gene tests we have right now? Well, certainly this sort of annotation, I think that's the crux of this meeting. In our discussions in the UK, if you go and look at the House of Law's report, it was proposing a whole institute saying that this needs some sort of institute to address that kind of delivery at the edge of regulation. You also need this actual infrastructure to do the comparison. We think, well, that could be this institute providing some of those services, but it could also be a whole load of small companies, SMEs, providing different competing services to GPs, to medical professionals. But in order to enable that, actually it's important that that underlying data set, the database, is open data so that lots of different people can get access to that and it can interoperate with other countries, you know, other countries' data sets. Then there's the other side, which is research over this because whatever you put in that database is going to be driven by what you can extract from the information across, you know, maybe the whole countries' records if you start sequencing everybody. And so we have this research capability program, which is being set up to aggregate patient records. If those patient records have genetic data, that could be a point for doing that research as well. Of course, there's other types of data. There's cancer data, pathogen data. It can all kind of be fit into the same sort of framework. But the bottom line is if you've got these systems, you've got the research databases, lots of variants, you've got very few that have been identified as clinical ones. In terms of the funders, in terms of the policy makers within the National Health Service, they want to know, am I going to get benefit if I start sequencing people, at what point am I actually going to get improved healthcare. It's really a cost-benefit analysis. Where do these two lines cross? And I think that's an area where we don't know. It relates directly to how we define clinical variants that are significant and how many we can define. So this is the overall picture. It's clearly there's this feedback loop in the middle. And I think one thing that's going to come out of this is that if you look at the funding that's been going on up to now, in the past, all the genome data that's been feeding into this process has been coming through the kind of research activities, welcome trust funded, NIH funded, but if this connection to a large number of individuals within the health system is made, that's going to become the dominant source of information in the future. That's going to drive this kind of analysis rather than the research-led things coming out of doctor of hospitals. I think the other thing is that this is just the start of digital medicine. So this is genomic medicine, looking at variants, but there's talk within the EU of much broader projects building digital integration of lots of other types of data around the patient images, other types of high throughput data. And there's in fact a project. There's a proposal for flagship projects, large-scale developments funded in the next framework program. And there's a pilot going on at the moment called IT Future of Medicine, which is around producing virtual patients around digital integration. So that's all I'm going to say. I just acknowledge the different groups that have fed into this. There have been lots and lots of discussions over the last five years in the UK around what's necessary to make this happen. I think this is going to be an important meeting in terms of this defining this, how do you capture the variants which are going to feed into the clinical value? Thanks very much.


Hi everyone Joe for jazby's case breaks comm with another four box break of 2023 panini phoenix football pick your team number two just dropped today Big thanks to everyone here for getting in on the action On a Friday. Thanks for spending a bit of your Friday night with me. I appreciate that pick your team, too. I Usually don't see you on a Friday Patrick got double last spot mojo with the Broncos and the commanders people have prism at 91 next year That means you won that the one you won that team in the filler 2019 prison pack Awesome. All right now You can see right over here. We have those stacks of four. We got Thinking the first break you saw me put it on this card. I haven't moved them since so now that we got three stacks right here We'll go one two three four five six on the dice roll so from left to right one two three four five six and It's five one two three four five six this these four right over here and Those next two stacks will be will be for pick your team three and four All right two autographs one memorabilia card per box on average We're looking for and Fuego's genies and more nine number parallels per box on average now Chad Says that I'm not allowed to pull genies in this case. That's what he said so There's no genies. You can blame Chad Gladly repay you for that That's Chad da DAW you can find him at I think what what did I pull before though? Are there other case hits too I pulled a Standby pulling up the checklist Phoenix football car will come the other inserts Now I did not pull any of those what are the genies even look like I guess we'll find out I got Michael Mayer at $199 Just gonna set those number cards aside. I'm gonna sleeve those a little bit later. There's a fired up Nick Bosa AJ Brown to $99 Michael Mayer going to Adam and my Raiders AJ Brown going to the Eagles Eric Blanks we got Trevor Lawrence to $199 that'll be for Matthew and the Jaguars and We got siaka Ika Rookie Auto for the Browns Jerry Cleveland this is for you 77 out of 99 Devonte Adams contours Those contours cars are pretty cool There's Buddha Baker to 125 for Ryan Heinz in Arizona There's a Josh Jacobs green Contours six out of 25 And a CJ Stroud fire forged That'll be for Lauren with the Texans CJ Stroud Ford fire forage refractor to Sean Watson to 125 Cleveland and What is this we've got a A Phoenician Jalen Hertz. I think this might be a short print. I think that's the case it Case it sets include archetype and Fuego metropolis and Phoenician Nice Jalen Hertz Eric Leo Eagles It's on a slightly thicker card stock to them. I'd be out of a 55 55 pointer It's almost like a like a playing card. It's gonna be a cool set to build who else is a Phoenician in here Don't they have Phoenician on the checklist On the checklist is not even up. It's not even released yet All right We got Keanu Benton to 250 and more a Raiders Fire fabrics Josh Jacobs another one for Adam Another Josh Jacobs got the out of 25 card and I got a out of 75 relic. Oh It's on Beckett has it so a cardboard connection a little slow That this is pretty I think it's like a 16 card set maybe a 15 card set. That's pretty cool All right, Michael Wilson at 25 we'll go to Ryan and the Cardinals and we've got a Dalton King Kade Jersey and autograph Like you had a pretty nice season this year Buffalo Matt with the bills 299 there's Raheem Moser To 150 Juju Smith's juicer hot rouse to 199 Patriots. That'll be for Karen and the Raheem Moser will go to bro Wait, had they had they haven't done this and we'll do a recap at the end here. They haven't Release Phoenix since 2020. Has it been that long? I don't think they did one last year, right? 2022 Or did they do a 2020 Sometimes it all blurs together. It's only so much hard drive space up here. So Raheem Morris is the is the new Falcons coach, right? Yeah, Raheem Morris Falcons coach For a split second when I Oh I Don't remember But when I first saw that headline at first I thought The Atlanta Falcons had had hired Raheem Moser I got as far as Raheem and then MO and I was like I was like what? They hired Raheem Moser as the head coach of the Atlanta Falcons The hell's Arthur Blank doing there's Luke Schoonmaker, but I should have kept reading So so that's a lesson for everybody Learn from my mistakes 97 out of 199 Luke schoonmaker Dallas that'll be for David Duffy DeAndre Swift out of 150 That'll be for the Eagles It's gonna be for Eric and the Eagles There's a Bijan Robinson red tonight 199 speaking of Atlanta Rup, and there's Jared Goff to 175. We'll see him in action this Sunday Anyone have any takes on any of the championship games? Nice contours Puka Nukui had a nice season, and we got signature highlights We got Trey Hendrickson Got some of that ice pattern. That's pretty sharp. That's gonna be a bangle going to Kevin since Natty We're on since Natty. I'm going pretty chock. I think I'm going pretty chock. I'm going uh One Ravens Niners 005 out of 250 Zack Charbonnet Matthew Stafford to 199 Must not harm Harry Potter out of 125 Rishi Rice Chiefs Jerry Judy to 199 Broncos and a CJ Stroud rally nice Brupp it's Lauren with the Texans 59 out of 75 Jake Moody to 150 for the Niners. That'll be for B. Cords Any guesses on this redemption? Jameer Gibbs to 75 fired up for the Lions Brupp Jackson Smith and Jigba to 175 Seattle James and the Hawks of the Sea That's a good guess a shame with that. That's a good guess from Nike Phil rookie Otto Jersey Orange No, J a H M Jameer Jameer Gibbs notorious non-signer Jameer Gibbs Notorious But still nice We'll see you in the playoffs this weekend And that's Lauren with the Lions Brupp there you go Brupp now sign your cards Jameer Please thank you Kevin guessing Anthony Richardson Kevin's and Kevin's lagging behind by a little bit Hit that live button if all exams are canceled run through like Fourth and fifties taking owls and newts. How do they know what their grades are they stop to take those right? Let's know what courses are taking the following in Harry Potter This reckless from Dumbledore. He's always been a little reckless box three of four All right, I got Luke Musgrave to 199 That's for Kevin and the Packers His numbered Marquise Brown to 199 Ryan with the Cardinals Miles Sanders Tony Pollard to 99 that'll be for Dallas David Duffy and there's Gervon Dexter senior Rookie Otto for the Bears Christopher with Chicago got the Bears straight up to 125 Breeze Hall Jets Adam of the Jets Jackson Smith and Jigba Kanan's brother to 199 it'll be for James James Stevens and see out Josh Downs 70 out of 99 for the Colts. That's gonna be for Matt when there's a die-cut in there. What is that? There's a base CJ Stroud for the Texans Which is gonna be for brup and the die-cut is a Brock Purdy die-cut and Fuego isn't that another case it? Did we get two in this maybe just a short print, but still pretty nice Yeah, Chad's saying that's just that's another case it I'll take it. Good time to get this one as well be cords. That's nice. All right onwards. We got Devonta Smith 10 out of 50 For the Eagles, it'll be for Eric Leal and we've got a Michael Wilson relic That will be for Ryan Hines and the Cardinals Piece of the Jersey 40 out of 125 We've got Jalen Carter orange to 75 for Eric and the Eagles Behind Cooper Cup is a Zay flowers Two-color relic and autograph. We'll see him in action this weekend Kevin with the Ravens The spot that he wanted in the filler Kevin's hoping guys a monster game this weekend. That'd be good for the hobby. That'd be good for the value of this card Remandry Siemens into 150 for Karen and the Patriots We've got Jared Goff to 175 and Jameer Gibbs contours insert All right I'm more Harry Potter. Hey, you're welcome Kevin. Thank you. What does a flowers do this weekend? you know He gets a What would be good to how what would be good for the hobby just stats might not be good enough maybe like a Game-winning touch that does he return? Did you anything in the special teams? Maybe free to return something Punt return for a touchdown or something like that Or is it Justice Hill that does Mark Andrews off the IR and gonna be in the game if that that should help that team Ravens I don't think they would have had too much trouble without him, but with him I feel like it really balances everything out. All right, just a Ravens win A flowers contributing maybe a touchdown. All right fourth and final. Good luck everybody. Thanks for getting in appreciate you Taja Spears blank Travis Etienne to 125 that'll be for the Jaguars That's gonna go to Matthew Slade and this nice contours CJ Stroud For brup and there's this tank right here tank Bigsby two-color patch and auto Maybe should go to Jaspie's autograph camp Maybe learn a thing or two here Jaguars that'll be for Matthew But a hit nonetheless Matt eight out of 50 You got Julius Brents to 250 Matt L and the Colts and Another CJ Stroud fire forged To 199 What a season he had We want continued health for CJ Stroud Another good draft and another good free agent window by the Texans keep helping him out in a very winnable AFC South It can be you can get right back into the playoffs again win that division Christian Gonzalez 19 out of 25 although Anthony Richardson would probably be back to they were looking pretty good early on Karen with the Patriots Marquise Brown to 175 for Ryan and then we got Quintin Johnson 15 out of 150 fire fabrics and He has a new coach maybe Jim Harbaugh Unlocks the potential of Quintin Johnson We got Miles Murphy to 175 Had a Bryce young a juggier Jameer Gibbs not a Gammier jibs and Dornass and he had a nice season Special world Kirk Cousins was slinging it Jersey and autograph Minnesota That's her Phil There you go Phil That is 71 out of 99 Bryce young will go to Brup and the Panthers Now Devin Witherspoon to 199 Hot routes Evan Ingram Feeling hot hot hot to 175 that'll be for Jacksonville. That'll be for Matthew s We got to a tongue of aloe to 125 That's a flamethrowers for Miami. That'll be for brup Derek Carter 199 for the Saints Marty Adam Thielen 4 out of 99 that'll be for Carolina For brup and then Devin Witherspoon to 199 for James and the Seahawks, and that's that that's your four box break brand new Phoenix football Here's a quick little recap Pretty nice stuff. There's the Jameer Gibbs redemption Nice signature highlights the Dalton Kincaid So on and so forth and of course two case hits in this one here the Phoenician and the Enfuego die-cut Awesome stuff. That's what Phoenix can deliver ladies and gentlemen and a lot of blanks. Thanks for watching. Thanks for breaking with me I'm Joe for jashby's case breaks calm and I'll see you next time for the next break. Bye. Bye


If you understand the basic idea of modeling, kitbashing, UV mapping, texturing, procedurals, rendering, rigging, weight painting, drivers, animation, motion capture, exporting, and the fundamental philosophy behind each, then you are not a beginner.  But they don't. Anyone who has graduated from this channel is a knight of the royal kingdom. So it is my pleasure, and my honor, to witness your graduation from Noob tonight. Congratulations, welcome to the next level. Now that you've gotten a taste of the different 3D fields, the next step is to ask yourself, which field did you enjoy the most? Which field do you never want to do again? Seriously, leave a comment down below, I'd really like to know. Really curious to see which parts you had the most fun with. It's completely possible to make a living off of just texturing. You can make a living off of just rigging. You can make a living off of just animating, or just modeling things in general. It doesn't really matter what you get good at, just get good at something. If you practice anything for 365 days, you are going to be amazing at it. So pick a field, hit it hard, but above all have some fun. Then next year, you'll have a portfolio of all the things that you've done, and you'll be able to confidently say, I have a year's worth of experience in this, and it will be a true statement. And being able to say that with confidence is an awesome feeling that I can't wait for you to experience. If you were a patron, I just want you to know that you are the only reason that we were able to pump out 115 videos of blended tutorials this year. What took me 13 years to figure out and learn the hard way? You can now binge watch in a single day, and these playlists are now forever available for any new aspiring 3D artist to learn from absolutely and completely free.  Originally, I just started the series as a way to make sure that no one else wasted their life like I did, but I never expected to find you guys in the process. I never expected to be a part of this incredible community, with goals and dreams that we would work towards together. You guys have brought a new layer of meaning into my life that I never thought I'd find again. So, thank you so much. In the end, I guess I really didn't waste those years, I just ended up giving them all to you. As for the future of this channel, as some of you may already know, I don't want to beat the jack of all trades for the rest of my life. So next year, in pursuit of specializing in character creation, I'm gonna start my journey master in sculpting with the secondary in character animation. We could honestly spend our entire lives practicing character creation and still never reach the end. But it's my favorite field, and I think we would be doing the entire community a favor by finding the most efficient workflow for sculpting, animation, and exporting to Unreal. But before that, I have two promises left to keep to the community first, Anime Shading and passing on what I know about programming in Unity. Once I have completed those two things, we will then focus the remainder of our time on mastering everything we can about character design and creation. Also, I know a lot of you are waiting for me to open up a Discord, and I do plan to make one. I just want to do it after the Unity and Anime series. The main reason is creating tutorials every day really takes up all the spare time I have, and any extra spare time I have, I usually spend responding to questions, comments, and messages on Patreon. Patreon is usually the last thing in the day I check, and sometimes the day is so busy that I don't even get to check Patreon. So as it stands right now with my schedule, I don't want to start a Discord only to end up ignoring it three to four days a week. So when things calm down, and I've finished sharing everything I know, that's when the Discord will start to roll in. And I've already got a few things I'd like to work together with you guys on Discord planned out. But for now, please hang tight for a bit, I really appreciate it. I will be taking a short three to four week hiatus, and in this time, I'm going to run all my tests and prepare all the material that I'd like to teach you guys next year regarding Anime Shading, programming, and Sculpting, which of course, once I have organized, will be teaching to you guys absolutely free. So please look forward to that next year. And the last thing that I would like to announce is merch. That's right. The long overdue Yadun t-shirts are finally in the house and come in the five fantastic colors of ravaging red, boldly blue, yolo yellow, beyond black, and of course, everyone's personal, classic, favorite, wildly white. For a limited time only, get yours now while supplies last. We also have the traditional Royal Skies t-shirt, but you probably want to get the seasonal one while you can. Hope you all had a wonderful Christmas and a happy new year. Thank you so much for watching. And as always, hope you have a fantastic day and I'll see you.


Automatic reinforcement folks. I can't think of a better example. There's also this one. Oh, my last name is Sane and Sane's pick their noses. It must be a sort of automatically reinforcing thing. It's when the behavior that you're engaging in produces reinforcers as the result of engaging in the behavior that are not like delivered from an outside source that maybe it's sensory, maybe just feels good. Oh, like rubbing my head and it just feels great. It's automatically reinforcing. I could just do that all day. The only reason I stopped doing this is because it's socially inappropriate and they get punished if it's all they want. Oh, stop. Right. Anyway, so automatically reinforcing things is just that when the behavior itself produces the reinforcer, oftentimes nicotine produces an automatic reinforcement sort of scenario. People go, well, just, just you can extinguish cigarette smoking. Really? How? How are you going to extinguish the delivery of nicotine through the? It doesn't make sense. It's automatically reinforcing, right? Same thing with most drugs. They produce automatic reinforcement. So you have to work on different aspects of the behavior. So automatic reinforcement, the behavior produces the reinforcer. Done.