The talk, a real talk after so many months is a little bit impressive, to give a real talk in such a big room that makes it a little bit less sharp, the transition between Zoom and remote. So this is quasi remote talk. Dobro. Zelo. Tak. Zelo. Zelo. Zelo. Zelo. Zelo. Zelo. Zelo. Zelo. Zelo. Zelo. sletnjih dekad, lahko smo videli, da je to vse zelo potravil tega vsega. In vznikam, da smo, neko vkratil, vsega pojav njega konputarna, da vsegačimo, kako bilo vsegači zelo. Zato, kot sem ječe, je vsegača, da ima vsegač, in v tepikovih molekodynamicih simulacijov, vzvečnjih, na zelo, z vsem, in in in in in in in in in in in in in in in in in in in. Takaj, je zelo potrebno, da je zelo potrebno, zelo konformacijov v tem samih sistemem. Zato je imaš nekaj protein, zato nekaj, da je se poznaj, zelo se poznaj, zelo se poznaj, zelo se poznaj. Tako, kako jaz semulajte kristal in defekte, kako je kristal veliko, jaz sem zelo, da se vsega realizacija vsega defekte, vsega vsega kristal. Zato, to je motivato. Zelo, da je vsega, ali sem zelo, vsega vsega vsega vsega vsega realizacija in se iz zanimivaj. OK, zato tudi se izgleda tudi svoje tega vsega v komplečni molekularne simulacije tako, da je zelo tudi zelo, nekaj zelo, nekaj je našem vzelo? ko ime, da ime dobro. Ze invesevalo in vsakam je komputera kaj došlo vsakamšno predstavno sestem in ovo je zelo za vse trajektori in ovo je vse zelo, da hatiramo zelo za trajektori. Dobro, se je, izgleda h regularno, različno, da lahko el ne zelo. Vse, posleda... Majem izgleda ne bo in jejo dobro, ne da bo in ob henjel. včakaj, da nekaj, da pristajem, pa vzvečem, njih nekaj, da se jih početila. Vsak je počet, da je izločnico, nekaj? Vsak ima izločnico projekciju, da mi je dobeče. Če so pri vse našta našta, da je dobeče, da mi je dobeče, da je dobeče, da se pri vse našta našta našta, da je dobeče, da je dobeče, In, da početim projekčenje, tjela, da imaš veči metod, tudi sem vlistešen kaj je zapotratil. Tudi sem v početni, ložitej, ko je začal, insupervizem. Insupervizem, tudi, da, ne zelo, ne zelo, ki se početimo, svolim, početimo variabost. Ok, pa... Tudi se jaz vse, da so početiti, prob Express of this pipeline. J mean that after I perform my projection then I estimate the probability density as a function of the variables which I have selected. And then I basically, somehow end up with something, which is let's say in the pretty general case is kinetic model. Model. Akinetic model means that I am describing a complex molecular system with a few states which interconnect onto each other on a rare on a long time scale. So this is what is called a Markov state model. So this is, let's say, the typical analysis pipeline of molecular simulations. So here this pipeline is, well, on this pipeline I would say that there is a large community of people which is still now working hard and there are new things that get discovered. Well, basically, even now, so it's basically, well, any of this passage is somehow, well, I would say there are good solutions available, but they are not totally solved. So critical point means set of points. So, so here I have, once again, critical, but nearly this refers to the difficulty. So the difficulty, a critical difficulty is actually the one that we were mentioning before, projecting a multidimensional data landscape into a, well, a number of variable which in principle can be visualized or understood with a small a priori knowledge on the nature of your system. Okay, so here, well, here this I'm going to run quickly, like, so if, so let's say the hydrogen atom of unsupervised learning dimensional reduction is the principal component analysis, which basically, well, means computing the covariance matrix of your data, diagonalizing it and projecting it on the main eigenvectors of the covariance matrix. Okay, so this approach, which, which is really a, I would say, a holy grail in anything that we are doing, we are all somehow using it in some of its variants. And, yeah, as well, in principle, well, let's say, practically fails, fails to find, let's say, a minimal description if the manifold containing the data is curved and twisted. So here, this is once again a very, very important and famous problem in many, many for learning, which, like all the famous problem has been solved in many different manner. And there are now several approaches, which allow basically ironing your, your embedding ironing your embedding manifold. So really ironing. So here, if I can find. So here, if, so the thing is the following that clearly, if you have this piece of paper, it doesn't matter how twisted it is, you can always tear it apart and map it into a flat piece of paper, but now if this thing is a tube, now you start having a problem, right? Because, because in order, if this is a tube, in order to map it to a piece of paper, even if it's clearly two dimensional, I have to break it, right? So this is clearly a problem of topology. So here, if the topology of your manifold is, so is the topology of an hyperplane, then, then, well, there are, actually it is possible to map your description into something, as the correct dimension, and which is the correct dimension, and basically you can visualize it on your, on your paper. But if your manifold is not isomorphic to an hyperplane, then, well, then you are in trouble because either you break it or you renounce to perform in, let's say, an optimal dimensional reduction. Like, for example, if this is a tube, of course you can say, OK, I keep it in three dimensions. So in three dimensions, I don't have to break it in any manner. No, but this means that I will have to use one extra dimension to represent my data. OK, unfortunately, the situation is even a little bit worse than tubes. Tubes are optimistic, let's say. So actually, in many real-world cases, the data manifold, the manifold containing the data, looks more like a form. A form means an object, an object in which I have, well, basically, the intrinsic dimension of my manifold is typically locally very small. But then I have, so here in this specific case is one, but then I have several points, which have a null measure, where there are different manifolds that meet each other. No, so there in some way one can say that the intrinsic dimension is larger. I think, actually, hinders the possibility of deriving a meaningful description in terms of a few explicit coordinates. So here, imagine that you really want to map this complicated object into something where the coordinates describing these objects are explicit functions. So this is really, really very, very difficult, no? So this thing is actually even worse than that, because the walls of these form bubbles happen to be typically high-dimensional, no? Well, complex system, molecular system, solid state physics system, well, we would like them to be low-dimensional, but they are typically not. So here, now one of the analysis tools that one can actually use for unsupervised meaningful learning is, so, well, a very important one actually is estimating what we call the intrinsic dimension of the dataset. So the intrinsic dimension of the dataset is basically the dimension of the form walls. So here in this case, we will get an intrinsic dimension of one, right? So now if you take, for example, a protein, which is folding, and then you compute the intrinsic dimension in the space of all its internal dihedrals, you find the number, which is of order 10. So 10 means that the walls of the bubbles are 10-dimensional. In other words, it means that if I am in any configuration, in any configuration, there are 10 linearly independent directions in which my system can move on a, well, significantly large scale. OK, well, clearly this means that even if I was able to get an explicit map of this form, then this map would be useless because I'm not able to visualize more than three coordinates, right? So here this thing means that performing a real, well, let me be clear, a dimension reduction with no information loss, that's what we are talking about. Of course, you can always reduce the dimension of your data representation if you accept to throw away some information. No, if you don't want to do it, then you are typically in trouble, exactly in this kind of troubles, all right? So here this actually means that mapping this object to dimension 2 or 3 in such a way that I can visualize it. Normally, I would say typically, implies an important information loss. OK, still, let's try to do it. So let's, for example, let's try to do a two-dimensional projection of the folding of a small protein. Well, here for this protein, a very large molecular simulation was available, and we applied one of these methods for ironing the embedding surfaces, which is called isomap. And well, if you try to iron the data landscape explored in this extremely long molecular simulation, you get a representation like this. Here, well, blue point and yellow points should be separated in two clusters, because one should represent the folded state and the other should represent the unfolded state. So here clearly there is a very high degree of mixture between states, which have clearly nothing to do with each other. So this thing doesn't give you, and let's call it an insightful two-dimensional representation of the data. OK, so here, let's say that my research, so now we are actually going to move to a description of what we have been trying to do in my group and with other collaborators, also in other groups in the last, yeah, eight years. OK, well, the idea is trying to develop a set of methods, which allow to understand something about the structure and the properties of complicated data meaningful. The idea, like in all the tools of computer modeling, so one should first try simple things. So here my suggestion to everybody, when you have a problem, the first thing that you should always try is a principal component analysis and then try to make an histogram as a function of the first two principal component. If this doesn't work, if you have something that doesn't fit what you should find or the structure is too complicated, then you should move forward and then you can try kernel principal component analysis, which is doing already something better. There are other very powerful methods, and if you are really, really desperate, what do you do? You try to learn from Marshall's sailors. So Marshall's sailors, well, I would actually like to learn something from them, but unfortunately they live really on the opposite side of Earth, so they are incredibly skilled sailor men's who actually sail in South Pacific. In South Pacific, and they used to sail in South Pacific since centuries. And in order to do that, they had in practice to record the relative positions of several islands, and at the same time the main direction of the wind, which you have in between, and the main direction of the currents of the ocean underneath. So let's say that it's a six-dimensional landscape, because you have the coordinates of the islands, the directions of the wind, and the direction of the water streams. So, clearly, they immediately realized that they were not able to make a graph with six coordinates, and so they ended up with a compact representation of this six-dimensional landscape, which is totally unreadable to any person who is not a Marshallist sailor, which looks exactly like that. In this thing, basically, so this is a map for them. It's a map that allows them to safely or quasi safely go from an island to the next one. In this thing, even if you don't notice it, it's six-dimensional. But okay, good. So here, so, the thing is that, at variance with ordinary maps, which actually is something which are, let's say, a graph of how a land looks like, is, so this is, let's say, a map which conveys information about abstract quantities. But these abstract quantities must mean something to the person that are able to handle this map. So you need a translator, typically, for this map. Okay, so here, so now, well, what we, our perspective on complex data landscape is that we do not even attempt to project the data on a low-dimensional manifold, because this thing, for the kind of manifolds we are interested in, this is meaningless. And instead, what we do, we build a topography of the landscape. A topography of the landscape is basically a list of the probability peaks and of the saddle points connecting them. So this topography, even if then it is possible to represent it in two dimensions, well, even if it is not always possible to do it, but it is typically possible. This map, typically, you can think, well, it can be merged in a space of any dimension that you are willing, no? Because it's like you basically map your data landscape into, let's say, 20 probability peaks, which can be connected or not with each other. So you can imagine that you map your probability landscape into a matrix, in a symmetric matrix, which would be 20 by 20. So, like on the diagonal of this matrix, you will have the height of the probability peaks and then the off-diagonal elements of the matrix, you will have the height of the saddle points, of the critical points. OK, so this is our, so in general, our perspective on how we try to represent the data landscape. And this is an illustration, like this is a probability landscape. You harvest points from that, then you assign points to states, and then you build representation, which can look like that. Or, like that, if you only care about, let's say, the highest saddle points between two different probability peaks. So we are going to see examples of these representations in the rest of the presentation. OK, so here in order to do this, well, we had either to learn to use or to develop, well, many, many tools. In particular, well, we had to develop tools to compute the intrinsic dimension of the manifold containing the data. These we are going to see why this number is so incredibly important, also computationally, it's not only conceptually. So then, well, basically, after you have computed the intrinsic dimension, we estimate the probability density. But here this is where, in a way, the importance of estimating the intrinsic dimension comes out clearly the first time. So basically, the data are contained in the intrinsic dimension, in the embedding manifold. Therefore, in order to define what the probability density is, you need a measure. So this measure must be defined on the embedding manifold. It cannot be defined into the space of features, because otherwise you would have basically a lot of empty space. So imagine that you have a sheet of paper and the data can only be here. If my measure, I say that it leaves into the three-dimensional space, basically even defining what the probability density is would be somehow tricky. So the point is that the support of our probability density is the embedding manifold, and we explicitly estimate it on it. Okay, then, well, then we find the probability maxima by density peak clustering. We compute the probability at the boundary between each pair of minima, and then we move to a more standard ground, which is how to represent topographies, which in real-case applications include tens, if not hundreds, of probability peaks in a manner which conveys useful information. So this is somehow nontrivial, but this is where we actually exploit many, many tools which have already been developed by other people. Okay, start from the estimate of the intrinsic dimension. So here, from now on, well, I will have a part of my top which will be partially technical, so I will necessarily have to go pretty quickly, but if you want to have more details, you are more than welcome to interact with. The intrinsic dimension is the minimum number of parameters or coordinates required to describe the data while minimizing the information loss. So this definition is from Wikipedia. So here, let's say, the classical manner of estimating the intrinsic dimension is basically uses this equation here, which actually says that the number of data points within a distance are scales with r to the d, where this is the intrinsic dimension. So clearly this is a very obvious equation. So like if I am on a line, it will scale linearly. If I am on my sheet of paper, it will scale with the square of the distance. So the critical thing is that this scaling law, which has actually been used in many, many different approaches for estimating the intrinsic dimension, in which, like in the 70s and the 80s, was called the fractal dimension, even now, actually. But, OK, well, I call it the intrinsic dimension, is that this equation has a prefactor. This prefactor is rho i, rho i is the density, local density of the data points. So here, this actually is equation tells that the estimate of the intrinsic dimension and of the densities are in principle entangled. So one, using this formula, you basically, you cannot do inference on d and rho i separately. So they are because they actually appear in the same scaling law. So this thing, if you are studying a dynamic system, this is not really a problem, actually, because the density, like on a Lorentz attractor, for example, is typically not so variable in the relevant, so in a large region of your phase space. But if you are studying a molecular system, the density varies orders of magnitude. So imagine, like in a metastable system, by definition, the density varies by orders of magnitude, because in practice, in fact, we don't really even talk about density in molecular simulations, we talk about free energy. The free energy is a logarithm of the density, right? So in fact, we do a logarithmic transformation in order to appreciate the extremely large variations in density that we have in molecular system. So here, the prefactor in such an estimator will actually vary brutally in a molecular simulation, and then you would get artifacts. So this was actually well known. So one of the development that we have actually done five years ago now was developing an approach which allows disentangling the estimate of the intrinsic dimension from the estimate of the local density. How do you do that? So basically, we consider, as a probe variable, instead of the distance, a ratio of distances. So this is actually the key point. So here the ratio of basically, so for each point i, we find the nearest neighbor, and its distance we call it ri1, and if we find the second nearest neighbor, whose distance is ri2. Then the ratio of the two is dimensionless, and you can qualitatively understand that basically its probability distribution is independent on the density, because if the density is large, ri1 is small, but also ri2 is small, so their ratio is qualitatively less dependent on the density. Actually, you can actually prove that it is not only qualitatively dependent on the density, but it is exactly independent on the density, because indeed the probability distribution of this ratio nu given the density rho is actually exactly independent on rho, and it is a Pareto distribution, which is this expression here, which is parametrized only by the intrinsic dimension. So basically, the density has disappeared. So basically what we do to compute the intrinsic dimension is measuring this nu i on all our data points, and then do inference. You can do it with Bayesian inference, you can do it with maximum likelihood, you can do it in many different manners, but basically it is a simple inference problem, because we have to infer the value of a single number, just the intrinsic dimension. OK, questions? So the intrinsic dimension, imagine that here you have a tangent. So the intrinsic dimension is the dimension of the intrinsic dimension of this is 1, even if I twist it around. So the intrinsic dimension of this is 2. And the intrinsic dimension is the dimension of the manifold, which contains my data. And so it's just the dimension of this object. So that's what we are trying to compute. This quantity, even in material science databases, which are objects that are, like that you for sure have to handle within this workshop, because it is highly nontrivial number, typically, because you have many, many different features. These features are, let's say, well, locally correlated. So if you are in a specific data point, which can be a specific material, you cannot, you don't find other materials in all the possible directions. You can find materials only in few directions, which unfortunately happen not to be the same directions everywhere. So this is the reason why you cannot simply use the principal component analysis. Is it more clear? All right. So now, so here a very, very important observation is that the intrinsic dimension, while mathematically speaking, is very well defined. So you have a manifold, and then it's the dimension, well, let's say that if it is a Riemannian manifold, it is totally well defined what the dimension of the manifold is. So in real-world application, well, you have a problem. So the problem is, let's take the example of a configuration in a molecular dynamic run of a biomolec with an atoms. So they say that you have no constraints on the bone. So then you have a thousand atoms. What will be the intrinsic dimension of the manifold containing the data? Well, will be 3000, right? Because in principle, well, you have no constraints, so you can actually go everywhere, right? OK. This estimate is irrelevant. Is irrelevant? Is irrelevant because, well, let's say a useful, where useful, we will see what it means in the following. A useful estimate of the intrinsic dimension should provide the number of independent directions in which the system can move significantly, no? So this is, well, in a way, we are not discovering anything. If you do principle component analysis, you will never, in a real-world data set, you will never find any eigenvalue, which is exactly zero, right? So you will find maybe few eigenvalues, which are very large, and these large eigenvalue will tell you what are the meaningful components, but then all the others will be different from zero, right?  So this is the same thing that we are trying to do here with the intrinsic dimension, no? So for the intrinsic dimension, so the components, in principle component analysis that you throw away are, let's say, short components. Are components, which matter only for a description of your system on a very, very small length scale? Now, if you start looking your system at a larger scale, then you magically end up discovering that few components are actually strictly necessary to describe your system. So, well, here, for example, well, here this is a typical example. Here, if you look at this data landscape on the small scale, the intrinsic dimension is two, but on the length scale of the red line is clearly one, right? So a human eye is always capable of saying this is one dimensional, almost everywhere, except in points here where you can really go in two directions. In points where you would say this system is two dimensional are only this and this. You can even try to do this in a neuroscience experiments. We have tried with 10 subjects and all gave the same answers. Great statistics. OK, so here, well, here, here I'm not going to spend a lot of time here, but basically, when you have a real world system, when you say a database in material science or molecular dynamics trajectory, you always have to estimate the intrinsic dimension as a function of the scale. And like, for example, you say my intrinsic dimension for a variable for scale of 0.1 Armstrong is 100, then the scale of one Armstrong is seven. And then maybe it's seven also at scale of three Armstrong, no. So, well, it's very, very important to date your estimate of intrinsic dimension comes together with the scale analysis, a scale analysis because the scale is exactly something which gives you a, let's say, a handle on the meaning of the number that you are measuring, no. So, to say, I have an intrinsic dimension of four on the scale of five Armstrong, you will know that it means that if I move my system by five Armstrong of RMSD, then I can move in five different direction, no. If I simply say you, so, so, so this thing always must come together. So the reason is because the intrinsic dimension is rigorously independent on the scale, only in realistic test systems. Okay, so here, well, we are, we have our procedure for studying the dependence of the intrinsic dimension of the scale, and we call an intrinsic dimension meaningful if it is, let's say, scaling variant, at least on a range of scales. So, that's in a way our definition. There are systems in which you cannot define an intrinsic dimension. You cannot define it because you don't have such a plateau. So, according to the scale at which you are looking at your system, you will have a different intrinsic dimension. If you are in such a condition, then your tour through the methods developed in my group is over. So, you cannot continue. You simply say, I cannot, well, I cannot continue because, for example, one thing that you cannot do in that case is the second step. And the second step is estimating the probability density at each point. Okay, so here I have finished the part of the intrinsic dimension. I don't know if you have other questions, also remotely, or curiosities. Thanks, Alessandro. Can I just ask this last thing you said? So, for example, kind of fractal, would a fractal be a case where you don't have an intrinsic dimensionality? Yeah, well, the fact that your intrinsic dimension is not integer doesn't imply that the intrinsic dimension is not defined. So, the thing that kills our possibility, for example, of estimating, of estimating the density are so-called multifractals. Multifractals are basically where you have no scaling variance of any sort. So, actually, what makes everything else possible is not the fact of having an integer number there, but having a number which is at least approximately scaling variance. Okay. One more if I may. So, maybe you'll cover this, but are there methods that can calculate intrinsic dimensionality taking into account symmetry equivalent possible configurations? This thing is absolutely a key question. The thing is the following that the features that one has to use in order to describe the system must obey all the symmetries that you have in your system. Because otherwise, you are basically computing a number which doesn't make sense. Like, for example, if you are studying, like, I don't know, a silicon cluster, we are going maybe to show you an example. You need to use atomic symmetry functions, or you need to use SOAP, or you need to do, well, features which automatically encode translational and rotational invariance and encode permutational invariance. Otherwise, well, so this is something that you have to solve in advance. So, in a way, finding a manner of, well, a description which is automatically resilient to all, I managed to use resilient because it's a very fashionable word in Italy. Sorry for that. Okay. Good. Is actually a key prerequisite in all our analysis. But presumably, you can go both ways, right? You can either have features that respect the symmetry, or consider all possibilities, not taking into account symmetry, and then determine the amount of symmetry operators and subtract that from your intrinsic functionality. Well, this is something that we don't do that I think is very meaningful. Okay. Thanks a lot. Okay. Now, let me see how we are doing with times. Yeah. Time is running. Amazing. No. All right. So, how do we estimate the density? So, let's start from, yeah, the K nearest neighbor density estimator. So, the K nearest neighbor density estimator works like that. The point here, and you fix a parameter, which is called K, which here is 13, and then you measure the distance between the central point and the 13th neighbor. And then your density will be the number of data points 13 divided by the volume occupied by your data points, which is pi r 13 to the power two, right? So, here it's clear that if you don't know these two, then you are in troubles, right? This is our intrinsic dimension. No, so you cannot, so if you only measure distances between data points, and you don't know the intrinsic dimensions of the manifold in which they are embedded, here you can put 1000, for example, your number of particles, and then you get a known sense, right? Okay, so here, I don't know if you can see it, but basically, well, K nearest neighbor estimator has a problem in data landscapes in which the density is varying sharply, is varying sharply. For example, here, if I use K equal 20, then I have a large statistical error here. Here the estimator is somehow good, but here if I use K 375, then on this point I have a small statistical error on the estimate, but here I have a systematic error, right? Because I am including in my estimate regions with a totally different density of data points. Okay, we need a different data point, a different K for each point, so this thing is well known. Well, here it's so well known that this has been studied a lot. The classical problem in our supervised data analysis, in particular in density estimate, is called a bias variance trade off. So the idea is finding a compromise between taking your neighborhood large enough that your statistical error on your estimate is small, but it cannot be too large, because if it is too large, then you are basically inducing systematic error in your estimate. So here we have to find the compromise. Here our contribution, which is, let's say, of course almost needless to say strongly connected to things that other people have done, is actually has been developed with a specific goal performing an adaptive density estimate in which, well, two conditions are satisfied. First, the density is implicitly computed on the embedding manifold without defining explicitly its coordinate. So these two things are very, very important. I don't want to define the explicit coordinates of my embedding manifold. Still, I want to compute the density exactly on the embedding manifold. How do we do that? We do that throwing away features and using only distances, because if my data are contained in a manifold, what it doesn't matter how curved it is, as long as this manifold is locally flat, the distance, the local distance on my manifold will be locally Euclidean. So you can actually measure this distance using all your original features, and then this distance, which you measure in the full feature space, will be a meaningful distance also on your manifold. It means that everything that we are doing, we cannot use features, we use only distances. So in this manner we are able to basically compute things on the embedding manifold without defining explicitly the coordinates of this embedding manifold. So this is a subtlety, but it's an important subtlety. So how do we perform this thing? So the idea is basically finding for each point the largest neighborhood in which the density can be considered constant. So this is the idea. So we do this by performing a statistical test. So the statistical test, well, the question that we try to address is, well, we start from a very small k. Let's say, for example, k equal to 1 or 2. And the question that we ask is, should we include the next neighbor in the density estimate? Well, the answer will be yes if the density doesn't vary too much. And how do we assess this thing? We compare two hypotheses. So the hypothesis number one is that point i and these k neighbors have different densities. So the density is different. So the hypothesis number two is instead that the two points have the same density, right? So these two hypotheses. We do, well, let's say, a model comparison. So this is a typical procedure in statistics. You have two hypotheses. And then you want to find out, well, you basically want to find out a criterion, which tells you at this point, I want to reject, for example, hypothesis two. And that's exactly what we are doing by model comparison. So we have, we reject hypothesis two when, well, a likelihood, which now I will tell you what it is, because I cannot derive it due to the fact that I don't have enough time. I reject hypothesis two in the likelihood of hypothesis one is much larger than the likelihood of hypothesis two. Well, this is a meta parameter of our approach. So basically I have to say at which confidence level I have to reject the hypothesis that the density is constant. All right. When this happens, the size of the neighborhood is i is optimal. Well, a small illustration of what happens here. I am increasing more and more the size of my neighborhood and comparing the density of the two neighborhoods. And this is the difference in the likelihood. And here you see that this is our confidence threshold at a certain point. Basically, basically, the likelihood of the two hypotheses is very different. And this happens when my red point reaches this region where the density is significantly higher. So at this point, I have that I stop and then I use this neighborhood, this neighborhood to estimate my density of my central point. Is it qualitatively clear what we are doing? So this procedure has the advantage that can be performed. This test can be performed parallelly for each data point. So it's something that, well, if I want to write a code that performs this test, I can parallelize it very easily because this test is performed independently for each data point. So this is an important difference with respect to other adaptive approaches, which typically require n-square operations. All right, so here this is the outcome of this procedure. 13 for this data point and 270 for these other data points as optimal neighborhood size. So clearly this implies the statistical confidence on the density in different points in my landscape will be different, right? So for example, here I will have a very high statistical confidence here and a low statistical confidence there. So basically here each point in our density landscape will come with an error bar and this error bar is not uniform. So this is another important difference with respect to other approaches for density estimates where the errors are typically uniform. In our case, they are not. OK, a point dependent KI, OK, so this is what we already said. So our model, basically it's a likelihood based model, therefore it entails also an estimate of the confidence on the quantity I am inferring. So this can be done using, well, basically fisher entropy, basically the curvature of your likelihood at your maximum. Very trivial. All right, so here we did benchmark of this procedure on basically artificial data landscapes, where it can be considered challenging for probability density estimate. So basically we started from analytical probability density in dimension between 2 and 7. And then from these landscapes we sampled 10,000 points. And then we transform. OK, so let's say that we are in dimension 7. Then we basically, well, basically we enlarge the feature space by adding, well, in principle as many variables as we want. And then we rotate the landscape and we make it curved. So this we can do it like for example, we can put this thing on a torus, we can put it on a spiral on a sphere. There are many, many manners of making it complex, topologically complex and curved. And then we embed it in a high dimensional space. So basically, so this thing is the correlation plot between the ground truth density on the embedding manifold and the density measured with this procedure. And we have very, very good correlation. And as you can see, also the error bars are correct. Well, are actually rigorously, well rigorously. They say they are correct in the sense that the difference between the true and measured density divided by the estimated error is a normal Gaussian process with variance 1. So this actually means that our model of our estimator is unbiased and the model of the error is correct. Of course, it is, well, how correct it is, it depends on the intrinsic dimension. So it is basically almost perfect for an intrinsic dimension of 2, while in an intrinsic dimension of 7 we start seeing some deviations between a normal distribution and the observed distribution. So, well, this means in practice that we can estimate the density or equivalently the free energy on systems of intrinsic dimension. Well, now we are using this approach up to an intrinsic dimension of 15, let's say, above 15, then the errors become too much, but up to 15, we are still able to keep things under control. 15 means, doesn't mean that we have only 15 coordinates. We can have basically 1000 coordinates, but the intrinsic dimension is 15. Clear? OK. So here, since I'm running out of time, I have a little bit to decide which material I'm going to show. Yes, maybe I'm not going to show you how you analyze entropic basins, but I want to tell you a little bit about finding the probability peaks. So now let's say that we have a data landscape with 100,000 points. Now, so this data landscape is immersed in a space of, I don't know, 100 features, and it is an intrinsic dimension of 7. OK, now for each point, at this point we have an estimate of, we have an estimate of the probability density. So now, since these points are immersed in a very high dimensional space, it is not possible simply to visualize this density, right? So we need a procedure which allows to automatically find the probability peaks. So we do this by density peak clustering. OK. All right. Where are we? Come on. Yes. OK. How does it work? OK. So the idea is of density peak clustering, which is once again a method which requires only distances between data points. This is once again extremely important. We don't want to use features. We want to use distance between data points. Because this allows to work directly on the embedding manifold. So the idea is that the points at the top of the density peak is far from any other point with higher density. OK. So then what we do is, OK, first of all compute the local density around each point. In this, well, we can do it with our advanced method, which I have shown you before, or simply counting the number of points within enablehood. Like, for example, the density around point one will be seven, and the density around point eight will be five. And around point ten will be four, right? Then we have to encode this idea here. And then what we do is that for each point, we compute the distance with all the points with the higher density. So this is actually the key idea. So we only look at points which have a higher density. And then we take the minimum value. So let's look here, for example, data point number ten here is a local density maximum, right? But, of course, its density is relatively low. Its density is only four. So how can we distinguish this data point, which has a density four, from another point, which has also a density four, but is not a local maximum? So the thing is that this point, if I look for another point, which has a higher density, which has, for example, density five, this will be far. It will be, for example, in this region here, right? Instead, for another data point with density four, the other point will be close by. OK. So here, that's exactly what I am depicting here. These are two points, which have basically the same density, but the nearest neighbor with higher density in one case is close, and in the other case is far. OK. So now, so then, if one makes a plot, where here, now, on the x-axis, I have the density, and on the y-axis, I have this distance delta, which is the, I recall, the minimum distance from all the points with the higher density. Basically, the cluster centers pop up as outliers. So this is actually the key observation at the basis of this method, and, well, then the rest is trivial. So the key thing is that this approach is deterministic, so there is no optimization required. If you are a little bit familiar with the clustering methods, for example, you may know that in k-means, basically your results depends on the initialization, no? So you basically, when you do the clustering, you have to do repeated many times, and then make sure that you take the best solution, for example, right? So here in this case is deterministic, and so the cluster centers is deterministic and also unsupervised. Is unsupervised because we don't have to forget that when one estimates the density, one with our procedure, you also know the neighborhood size, the size of the neighborhood in which the density can be considered constant, right? So here the points whose delta, this distance, which I have just defined, is larger than the radius of the neighborhood used to compute its density, is what we call a cluster center. So we don't, we are able basically to find cluster centers without making any specific system dependent choice. So this thing can be pipelined, let's say. So here, so all the procedure is nonparametric in the sense that the density estimate is nonparametric except for the statistical confidence, but we have a manner for correcting its effect, which I haven't described today, and the number of clusters is determined automatically. So here, actually, well, here maybe I can first show you, well, the results, which, okay, I have a delated slide, well, here you would see that actually the clusters come out to be what you would think they are. So here there is, there is a very, very important, a very important detail. And with this detail, yeah, I don't think I'm going to have time to show you many other things, but this I want to tell you. So as I told you, our density estimate comes together with an error bar, right? So this means that if I have two clusters, so imagine that I have two probability peaks. If I have two probability peaks, I will have that the saddle points separating them, so that all the points in this profile will come together with an error bar, right? Error bar at the top of the peaks and error bar at the saddle, right? So in any real world application, these error bars will never be zero, right? So this actually means that when you talk about the clustering partition based on the density like we are doing, this clustering partition always comes together with the statistical confidence, right? So somebody tells you, I have five clusters, so this thing is not a well defined statement, especially if you do it with this approach. No, so you have to say, I have five clusters with the statistical confidence of z equal to, for example, no? So what is, how do you estimate the statistical confidence? Let's say qualitatively is just given by the diff, so, for example, the statistical confidence of a peak would be the difference between the density in the peak, the density in the saddle, this difference divided by the sum of the two errors, right? So basically this ratio, if it is large, then this structure in your probability days, it is statistically robust. If it is of order one, this means that it is not robust. Well, it's not robust. It has a low level of statistical confidence, right? So here in a way, the true final meta parameter of our approach is the statistical confidence. So when we talk about the probability landscape, we always have to set the value of this number, and then, in principle, see if how the results change by varying these parameters. There was a question? No, maybe not. All right, so here I think I'm running really out of time, so I'm not going to tell you how we find the saddle points. One thing, maybe before going really to the conclusion, what I want to, well, I just want to, well, tell you one thing that, so this thing is something that we have actually verified on many, many systems of increasing complexity. The peak, the probability peak, the density peak clusters that we find in this manner correspond one to one, almost, well, I would say one to one with the inherent state of a Markov state model. So here I think many of you have heard about Markov state modeling is a very, very classical approach for characterizing the kinetics of complex system, and it basically gives you out as an output the number of metastable states. So here we are basically finding the same manner, the same thing, but we are finding the same thing without looking at the kinetics. So we have absolutely zero information on the transition probabilities in our approach. And this only requires only, well, basically we only estimate a density, so we have no kinetics, no transition matrices, no diagonalization of anything. We think, well, actually we find, we actually find that our inherent states are actually better than the inherent states that one would find with the classical Markov state modeling analysis, because actually here this shows that the mean first passage time in a pretty complex molecular system. If you use our description, we obtain flat lines as a function of the time lag. So do you think I'm really sorry, it's very technical, only for those of you who know about Markov state modeling, this graph would be understandable. But basically, so here, well, this is even more evident here. So this is even a more complex system using the standard procedure, your mean first passage time looks like that. No, well, let's say you have a plateau only, but not such a good one. If you use our inherent state, we have something which is much more flat, which is this one and this one. All right, so with this, I think I'm going to skip the part on entropic traps. And I would only, well, I'm really moving towards the conclusions of my talk. So first of all, I would like to bring to your attention that, well, tomorrow there will be a tutorial tomorrow morning on a Python package implementing. Well, all these methods which I mentioned and actually several others. And this package was developed by this group of people. Well, in particular, Aldo played a really, really major role in making this complicated project work. And with this, well, here, this is where you can read about this package and you can eventually download the code. And with this, I would like to, well, I'm really going to the conclusion. So here, what I presented today is an approach, which is actually something combined in different approaches, which allows computing the, what we call the topography of a multidimensional probability distributions. So this approach is, well, first of all, a robust algorithm to determine the intrinsic dimension. This is really, really a key thing. This intrinsic dimension is a number which enters in our likelihood model. If you go wrong with this number, the likelihood model basically selects the neighborhood in a totally wrong manner. So you cannot avoid computing it. The probability of density estimator capable also providing an estimate of the error and a procedure of finding automatically. The probability peaks, regardless of their shape and of the dimensionality in which the peaks are embedded. So this procedure also provide a statistical confidence on these peaks. I think I am going to close the presentation, and I am ready to take questions. Thank you very much, Alessandra, for this super elegant talk. And the floor is open for questions. Yes. So if there are two regions in the total data space, which have two different intrinsic densities. So which one would you should we choose? The intrinsic dimensions. Yes. Well, the thing is that, in principle, you should use for each data point the local intrinsic dimension. Because it enters your likelihood function. So that would be the local. Well, the thing is the following that. Of course, in the approach, which I presented, the intrinsic dimension is only one. But, in principle, you can do this thing. So there is an approach which allows detecting regions of your data landscape characterized by the fact that the intrinsic dimension is constant. And you're also able to estimate how many regions you have. No, so this is like performing clustering based on the intrinsic dimension. There you have actually some surprises. For example, in protein folding, the intrinsic dimension is, one would actually say, if now I would make a small exercise, how many of you would vote for higher intrinsic dimension in the folded state? Well, all of you maybe would say that the intrinsic dimension is higher in the unfolded state, right, because it's disordered. Yes. Well, this is not the case. It's actually the opposite. The reason is because the relevant intrinsic dimension in the two states is on a totally different scale. In the folded state, the relevant scale of fluctuations is to Armstrong, for example, and in to Armstrong, you can actually move in many different directions. In the unfolded state, since it's really disordered, basically fluctuations of two Armstrongs are totally relevant in the thing that matters are scales, which are of the order of five Armstrong or ten Armstrong. And on this scale, the intrinsic dimension is lower. So the intrinsic dimension on the scale between five and ten Armstrong is of the order ten, which is the relevant one. On the scale of one Armstrong is of the order 22, if I remember correctly. So it's much higher. But this thing that you said is actually very, very important. Mary, Alessandro, again. Maybe I missed it, but so once you've done all this machinery, do you have access to the data manifold? Can you sample from it? No. So you don't have a model. So you are, so you are, well, this is actually a very, very interesting observation. And this is one of the things that I would really like to do. For example, I would really love to merge this kind of approach with the Boltzmann generators approaches, like, for example, things that have been done by in Francois group and many others. So this thing could be actually something that we would like to do. So here in our case, we simply have a set of observations, which are our data points. And from the set of observation, we have basically a measure of the probability density on embedding manifold with error bars. That's it. No manner of generating anything. Okay, let's talk after them. One thing, I don't know if there are questions on the chat. Maybe you can, if there are questions from the chat, you can tell me, all right. Great. Thank you very much for the talk. So, so given a set of data points, we are able to estimate a density. I would be interested in, which points should I consider so that if I run this method on a subset of my original set, I obtain a density, which is as near as possible as the one that I computed originally. So, so this thing can be assessed very rigorously using the tools of statistics, since we haven't estimated the variance of our density. So, basically, if you have two, let's say, statistically in the, well, a subsample of your sample of a given size, you basically rigorously know how your variance should scale. So, basically, well, trivially, if you have like, so if you sample your data set, so you subsample your data set by a factor of two. Basically, qualitatively, you should expect your error bar to increase by square root of two. And, well, of course, you should have that your estimate is consistent, should be consistent with the previous estimate. So, this thing is actually a very, very powerful procedure, which one can apply to actually test statistical independence. Because in practice, in implicit assumption of everything that we are doing that I haven't commented is the fact that we must assume that our measures are at least approximately statistical independent. And therefore, if this thing is violated, the scaling, which I described, will be only approximate. So, how approximate the scaling is tells you how correlated your data are. Like in block analysis now. Thank you. I think I'll thank you again on the behalf of everyone. Nice catch. The number of coordinates should be greater than the, well, it's not that it should be. It's actually that the maximum value of the intrinsic dimension is equal to the number of coordinates. Because if I want to, so I cannot have that this sheet of paper is immersed in one dimensional space, right? So, I can have that one line here is immersed in this two dimensional sheet of paper, but the reverse is not possible. So, the intrinsic dimension is always smaller than the number of features. Typically, fortunately for us is significantly smaller. Coffee. Let's thank again Alessandro for the coffee.


Welcome to Monet Café. I'm artist Susan Jenkins. Join me in this lesson as we explore beginner products and tips for soft pastel painting. Now, you're going to get a lot, even if you're not a beginner to pastel painting, I'm going to share a lot of products that I think are affordable, great for beginners, but I was amazed at a lot of these products. These are ones I had not used before, so you're going to get the low down. And a lot of practical tips for painting on surfaces that are affordable, DIY, creating your own pastel surfaces, and lots more. So come join the painting fun. And very soon I will have the actual tutorials from some of these paintings. So enjoy, get ready, get your popcorn. Here we go. If you would, go ahead and like this video. Go ahead and subscribe if you haven't already. And if you hit the little bell next to the subscribe button, you'll be notified of any future videos I post. This free video tutorial is brought to you because of the faithful support from my patrons on my Patreon page. For $5 a month, you can support this channel and you get extra content. Hello, artistic friends on YouTube, fans and followers. I'm excited to talk about some new products that I think are going to be so great for the beginner artist. And I love these products, so no matter what your level, I think you're going to be excited about some of my new product discoveries. I'm always looking though for inexpensive ways to paint with soft pastels because it can get really pricey. Well, any painting medium really. But where we spend our most money, where I believe you should spend your most money with soft pastel painting, is in the actual soft pastels themselves. You really can't beat the quality of a good soft pastel and having a lot of pigment versus binder, which is so common with cheap sets. There's a lot of binder, a lot of filler, if you want to think of it that way. So first I'm going to talk about some pastels I've discovered. Then I'm going to talk about some new papers that I've discovered that are all very affordable. And I'm going to be sharing some of my paintings with you and my techniques for protecting them. So you're going to get a whole lot in these next few videos. First, let me talk about these pastels. Now I have yet to do an actual pastel painting with these, but I can already tell that I love them. A lot of my patrons on my Patreon page have said they've tried, well, not a lot, a few, have said they've tried these and they were very impressed. Okay, so I'm going to tell you the price here. This is a 40 set of pastels. I love the size. They're not called half sticks, but they're just kind of nice and nice size. I'll show them to you in just a second. I love the fact that they don't have labels. That's just so awesome. But 40 colors of these pastels is $44 on Amazon. And the company contacted me to see if I wanted to do a review. And I always say yes to a pastel review. And I'm always very honest about them too. If I think it's going to be enough of a negative review on something, I just probably won't share it because I don't want to make the company feel bad. But with these, I can tell I'm already excited. So let's just do kind of a little product reveal here. The packaging is amazing. It's called Paul Rubens, but they do have some Asian lettering here. It's really pretty. I love this gold leaf graphic here. So artist soft pastels. And I love also too they have a nice little brochure in here that has also little instructions for use. I have never seen this in a set of soft pastels. I mean, they may have links or things online. But this has some really neat little pastel instructions for use. Two pages of it actually. And also it has a color chart. Oh my goodness. I have recently been talking to some of my friends and patrons on my Patreon page about how certain sets, they might have their colors listed, but they don't have them in the order. So you can't know where they go. Now this is awesome right here. I love that they have this and also to here's what's even better. I love love love it when a pastel art pastel company will include a color chart meaning a chart that you can fill in yourself a blank one. So what it is is these numbers correspond with exactly where these pastels are. Now why would we need that? Well, it's basically because if you go by the color that's printed on this little printout they don't they never look the same. You know, we're dealing with a printer. We're dealing with, you know, sometimes just lighting things look different when you look at it on paper versus the actual. So if you run out of a pastel and you don't remember which one it was, you may not be able to find it from here. The color may not and you don't have the color anymore. So this is what's so awesome about this chart here is that you literally you don't have to rely on a printout. You take your actual pastels and you go in here and you make a mark on every spot here. It also helps you to rearrange them in the right order if you want to keep them as a set. I've been keeping a lot of my pastels a set. So all right, I've already let's pretend I didn't show it. Voila. It's so awesome. I love the vibrancy of these colors. Now I could pretty much instantly tell just when I opened the box that I was impressed because I could see the color vibrancy and I do something. Whenever I open a box of pastels, I do something I call it the finger test and I literally just take a clean finger and I rub it across one of the pastels to see how much color actually comes off on my finger. So you can already see we've got some gorgeous, luscious colors in here, but this red is literally just singing to me. So let's take a take a look at that. I'm just going to rub my finger across. Are you ready? Oh my goodness. That is such smooth and bright, bold application of color outstanding. Oh my goodness. That means that the manufacturer of this pastel used almost all color with very little binder. It's the ingredient that keeps them holding together as a stick. And the reason soft pastels, the ones that are super, super soft, the ones that are super, super colorful, the reason they fall apart kind of easily sometimes is because they are almost all pure color. Now different products are used for all the different colors. I'd like to research that. Where do they get that red from? Is it from the dirt, the side of a mountain? I mean that's interesting, but all the colors come from different elements I should say. So sometimes they're not as bold or bright as others, but this blue for example is just really good. Oh my goodness gracious. I really like this blue right here. That is such a pretty blue. See that? I already rubbed the red off my finger. So I'm very impressed with their amount of color. Now what I really was curious about was their layout of color here. And I was just kind of examining. I often like to look at the rhyme or the reason behind the layout of a box of pastels. And I can already see they have a lot of nice neutrals. Look at all these beautiful. I love this as a neutral color. I have one that I use very similar to this. It's almost like a dull neutral lavender and gray lavender. And it is great for neutralizing elements in a painting. They have a lot of nice darks. They have a lot of nice earthy colors. Look at all these earthy colors. Orange is another one. Of course I love this red. But look at the reds they have that lean towards a cooler red. Sometimes it's hard to find cooler reds. They lean a little bit towards pink. Look at these reds. Oh my goodness. And almost a little salmony color, especially in this one's kind of peachy color. And this one too. Here's two other cool reds. This one's a really, really nice darker red. And this one, oh my gosh, I just love that color. So those are really nice. They have some nice light values. Love this light blue, like I said. I love this pink that would make a nice sky. I love this light blue that leans a little warm, a little warm blue. And this nice neutral blue here too. Look how gorgeous these are. But again, look at all the neutrals, neutrals, browns. Do you notice something that's kind of missing in this set? Well, it's common for a lot of sets. A lot of sets don't have purples. I don't know why. I love purple. But this set doesn't have any other than this. It does have some nice blues. We've got this nice warm blue. This would be great in a sky. Oh, this pretty bright blue in a sky, upper part of the sky. This gorgeous blue for water, distant mountains. I mean, that's just absolutely beautiful. So again, a lot of interesting colors. Now, I contacted the company. One of my patrons asked me if they have other pastel colors. And they do have a 36 set. Now, I already contacted them. They're sending it to me. I think they were out of stock. And I'm going to hopefully be getting that soon. I did not ask them if it's all different colors than these. I kind of hope it is because I liked some of the colors that I saw on this box top. Look how bright those are. So anyway, that's a little bit about the set. I'm going to be doing a painting demo with just this set. I always find it fun to challenge myself and paint from just one set. Oh, I just spit on my pastel. They are water friendly. So anyway, I am excited. Only $44 on Amazon. Wow. And I'm hoping to really work with this company and see about getting more colors. All right. So that is a very affordable product. You're going to be seeing a painting soon. And as far as papers go, I recently wanted to try out a new paper. Well, it was new to me. I don't know how long this has been out. It's called Canson. Now, Canson makes a paper. They call it a pastel paper. It's called Canson Mettance. And it's a French word. I used to always say it wrong. And it's it's really just a pastel paper. It's not sanded. We often as pastel artists like a grit, like hardware store sandpaper, we can get all these layers. Your colors show up more bold. But the Canson Mettance paper is it's not sanded. It's just textured. And on one side, it's a little more bumpy than the other side. And I've done plenty of videos where I show you can paint with pastels on those surfaces. They also have another paper Canson made that's kind of their version of a sanded surface. It's called Canson Touch. And it's a little bit sanded. And it's nice. It's not as sanded as a lot of the other ones. It's a little more affordable, but Canson Touch. So we've got Canson Mettance, Canson Touch. And now we've got this Canson XL. It's called Sand Grain. There's two colors in it. There's gray and natural. It also comes in two different sizes. I think the other one's more of a wide size. But the price of these, again, yay, the price is awesome. This pad, these two pads, each of them was $21. So 21 for this one, 21 for this one. And it has 40 sheets. Now that is unheard of in the pastel paper world. So I was obviously very curious. Is this a really a sanded surface? Well, I'm going to show you a little bit about what the, I'm going to just show you in this one. And then I'm going to, I'll show you this one. I've actually done more artwork in that one. So this surface is called Sand Grain. I love this nice gray color to this. And it's not grit. Okay. I'm just going to let you know. It's not like a sanded grit like we have with UART, Fisher, what I use a lot, Sennelier, LeCart, Pastel, Card. Gosh, there's so many sanded surfaces. Pastel matte doesn't really have a grit, but it allows for a lot of layers. But it has a sand texture to it. Okay. It's not the actual grit. Many of you know I make my own texture by using a product called Clear Gesso. And I'm going to show you that in the other notebook. So I wanted to experiment first on just the sand grain texture that they have here. So without adding any additional product, other than just a soft pastel painting. So with this, I did this painting actually with just one set of pastels. I'm going to have to go back and look at the video to see which pastels I used for this set. I mean, for this. I can't remember. But anyway, so I completed this painting just directly on the sand grain surface. And as you can see, you can create, you know, a pretty decent pastel painting. One thing I do love about working on surfaces that are not really gritty and sanded is you have to be a bit more strategic and learn to have more efficient strokes, not wasting strokes. I find my work is more fresh when I work on unsanded paper because I'm not relying on the ability to have a lot of layers. And so I actually really like working on surfaces that aren't overly sanded like this one or aren't sanded at all. I should say like this one. Just a sand grain. I do like the sand grain. I would buy this all day long over the Canson-Mittance, the one that's just kind of a textured paper. I like this one much better. So again, the price is right. I also take, I love to just make little pastel journals like this. A lot of people ask me, where do you buy those pads that have the paper protection between them? This is just tracing paper. Well, I don't. I make them myself.  Let me find it. I just buy whatever's cheap. It is acid free. So your work will be protected. And this I got from Dick Blick and I just taped it down and cut it to size to fit that. So then you can have all of your paintings all protected in your notebook. I think that's really fun. So let me talk about this other pad that is the natural grain and let me also talk about the method that I'm using for this book. That book you just saw, I will probably only paint on the pages. I won't add any additional product. But with this one, I decided what my husband and I were going on a trip and I wanted to get some surface ready real quickly. And I thought having just a notebook to take when you're traveling is really great rather than taking all these different pastel papers. So while he was getting in the shower and getting ready to go, I was in here in the studio frantically prepping some of these pages. I thought what I would do, I've painted on a lot of them, so I'll try to show you some of the pages that I just created. That has a painting. And I did videotape a lot of these. Okay, I'm going to show you this one. I decided to just tone because this one's natural color. It's kind of creamy like you are paper or fissure. I decided to just tone it with pastel. I'm going to show you video on this. Okay, you just take a color of pastel, put it on your paper, and you liquefy it with water or alcohol and a big brush. I used a foam brush. Then I went in and added, did I do it to this one yet? I don't think so, no. I went and added, I did to this one. Yes, I went in and added clear gesso. Let me make sure this way. Yeah, I can tell this one has the gesso. Clear gesso is a product that I have talked about for years and how to make your own pastel surface very affordable. DIY homemade pastel surface. Literally, you can get watercolor paper, you can use this paper, use any paper I say that's a little lighter in its background, and you can tone it. You can actually do a little base pastel painting and then wet it strategically. Then when it dries, like I let this yellow dry after I liquefied it with the foam brush, then I put on my clear gesso. Let me grab it. So clear gesso is just like this amazing product for me anyway, making my own surfaces, because it has a little bit of grit, a little bit of sand to it. I don't know why, but it's got a little texture to it.  It goes on a little cloudy at first, and then once it dries, you can see through to what you created. So the reason you get, well, there's two reasons you want to get the clear gesso rather than regular gesso. Many artists have heard of regular gesso. It's what a lot of artists use to prep their canvas surfaces, and it's not clear. So if you use, it's white. If you use this to give some texture on this, it's going to cover up all of the yellow. And this doesn't have any grit. Somebody told me that it did recently. We're going to do a little test right here, right here, right now. Oh, yeah, yeah, yeah. Okay. So let me see. It's white. I don't feel any grit in that. So I don't know. Now I got to go wash my hands. Well, maybe not. All right. So that is the reason that you want to use the clear gesso versus the regular gesso. And just use a big wide brush or a foam brush. Now it's going to curl a little bit. Okay. Even this one after I finished it is it's got a little curl to it. So what I did is I just created a whole lot of different colors. And then I put my, you can hear that. You can hear that grit. The clear gesso is on that. And I just let that inspire me to create all kinds of different paintings. So I created the first one is just a field of flowers. I made a purple background and I created this field of flowers. So real loose and I let that purple of the background show through. And I just think it just vibrates with color. I love that. Again, I used my little tracing paper to cover up the pages. And that's one I didn't do. That's one I didn't do. And this is one that I did while I was out on that trip with my husband I was talking about. This one had a pink background. And I took a reference image and just gave my own interpretation to it. And I was kind of on a wildflower kick like I am so often. And my daughter-in-law said, why don't you try something? We ran across one of these pages that had, I'm not going to show you the painting yet, it had this green that I toned it. And I've done some videos where I showed, yes, you can do a green underpainting even with a green landscape. But she said that looks like it would be something underwater. So I found this reference image of a coral reef with some fish underwater. Now I didn't get time to finish it. I wanted to give the fish a little bit more detail. But you can see how fun it is to create your own services with all these different colors. And I think it inspires you to get creative and to break out of your comfort zones. So I'm going to be doing a lot more of these. But I just wanted to share with you these surfaces before I share the tutorial. So you'll kind of know what I'm talking about. And I, in the video, when I share the actual paintings, I can say, hey, refer back to this video so you can see what I'm talking about. And now also too, I thought I would go ahead and show you some of my, well, one thing I want to show you is back to the Paul Rubin's Pastels. This was another set of soft pastels. Somebody recommended that I try. I actually just bought this one off of Amazon. And I was skeptical. I was actually skeptical of this set because of the price. When a price is too low, I can almost assure you there's going to be a lot of binder and very little color. So I was like, I don't know. I'll check these out again. So pleasantly surprised. Can't wait to paint with them. But this set is a 36 color set made by Gondola Soft Pastels. And nothing against these pastels. I'm just comparing prices here.  Okay, it was very nice. The packaging was great. And when I opened it up, they also have a really nice little brochure. But it is in another language. So that makes it a little difficult. But that's okay. I know God bless them for, you know, shipping things to the United States. That's awesome. It also has a color guide. Okay, a color chart for you to fill out. Love that. Love, love, love that.  Just like with these. These are even smaller. Okay, so they're really small. And there's only 36 of them. They're not in the nice little sponge holders like these. But they were secure. None of them were broken. But this set of 36 is $47. Again, versus this set of 40 that's less money. That's $44 really rich, gorgeous color. Again, nothing against this little set. But I could instantly tell it had less color to binder ratio. Let's just find I'll just do this blue one right here. I'll just take it out because it's little. Okay, so I'm going to do, let me find my pinky. I'm just going to rub across. Do you see that? How there's hardly any color on it? There's a lot of binder. Now, again, nothing against these pastels. These are a little bit more similar to a Rembrandt pastel. They're kind of a medium hardness and they have their purpose. And I actually did do a little quick painting. Oh, I just, I can't say it was a painting. I was just, I got them in the mail and I couldn't wait to try them out. So literally I was cooking dinner and I just laid an apple and a lemon down and I tried to paint with them. So I did a little study and I also like this. I mean, this is just a craft pad. Okay, but if you want some quick, cheap little paper to do studies on, it says recommended for temporary drawings. It's probably not acid free. But if you just want to practice, I was like kind of craft paper anyway. So this was my little practice with these and, you know, look they worked pretty good. But did I do that color there? Yeah, see the color comes off really pretty good. When you work on this kind of paper though, you cannot get a lot of layering. Let me just attempt that right here real quick. We'll put a darker down. You see that color's not bad. Darker down. This is how I usually do like if I'm doing a poppy flower. I get a combination of colors going and let's get a dark center in here. We'll get a little hint of a dark center. I can already tell it's not layering as good. I'll get a little bit of this. It has some nice darks in it too. I like that dark blue. And then I gradually kind of lighten things up or warm them up. Let me lighten it up with a little pink here. Let's say the sun's just catching on some of these areas right here where the petals are kind of reaching out. It's not layering. See how little layering you get with that? So anyway, that's just a quick idea of how these work. And again, very similar to a Rembrandt pastel, which is great. So that's another little set that's affordable and has its own purposes. So anyway, there are so many different ways you can get started with soft pastels and not spend an arm and a leg. I also want to show you real quick how I do the same thing with these black cancel drawing pads. I like working on black surfaces. With this one, I had done some lessons. Let me find one in here. And I adhere my paper differently. I experimented. I like having the tracing paper protection on the opposite side so I don't have to fiddle with lifting it. You see how I taped it back here? Same thing here. Just tape it to the back side. And I use black tape because I just wanted it to have a nice presentation. This is one where I didn't do it that way. I need to tape this side down. So they make nice little sketch journals. I'm not going to take the time to show you too much of these. So it's fun and it's a neat way that you can just have a little pastel journal. And take them on traveling, take them to go and then you can also use them to store, not just to paint on the pages, but to store other pastel paintings. So this is a larger one that I actually take some of my paintings that I just kind of want to store them. I don't want to put them in the plastic bags just yet. And this one was kind of fun. I actually experimented with using a stencil with pastel painting. It was fun. I'm going to make a video on that. So anyway, these pads have a lot of function for you as an artist. I love them. And also too, this was a painting I did where I used a fixative. I still haven't made this video. Everybody ask about fixatives. Watch this. Would you do this with a pastel painting? Let me get the pastel off my hand. Okay, wait. This hand's cleaner. Do you ever do that? Use your shirt. All right, watch this. Okay, you can see my hand other than the pastels I've done here. I'll do my hand part. Nothing. That is, it's called Spectra Fix. Let me see if that's what it's called. It's a product called Spectra Fix. And it's a fixative that I experimented with on this painting and a few other ones. And it did work pretty good, but hold off on experimenting with it. I mean, unless you just want to try until I make the video because I have some suggestions on how to use it. But anyway, so it's a great way to store your artwork and a great way to save money if you want to actually paint on the pages. So I hope you guys learned a lot from this and I am going to be... Oh, this was from a Patreon lesson. I am going to be actually doing, uploading some of the paintings that I've already done and giving you guys my tips and my feedback on some of these products. So anyway, guys, thank you all. Thank you subscribers. If you liked this video give it a thumbs up. Give me a comment. Many of you said you liked this raw footage. I don't have to edit it as much. I leave in my ums and all of my things that don't go right in the videos. So thank you guys for your comments and leave me a comment. I love to hear what you have to say. Become a patron if you'd like to keep this Monet Cafe train going. It's what allows me to keep the free videos coming like this one on Monet Cafe. It's only $5 a month to become a patron and you get extra goodies. You get to submit your work in a homework album. Sometimes we have these critique sessions where I give you suggestions. So it's a lot of fun. Alright guys, and happy painting.


Elmo Bud Zumalt Jr., the Navy's youngest chief of naval operations, was born on November 29, 1920 in California. After leaving California, Bud Zumalt attended the United States Naval Academy, graduating early in 1942 because of World War II. At the end of World War II, his ship had captured a Japanese gunboat and my father and 10 sailors were put as prized crew on this ship with about 200 Japanese prisoners. My father ended up commanding the first ship to come into Shanghai flying the American flag at the end of World War II. Bud served at a variety of commands during the Korean conflict and early Cold War, eventually working under Paul Nitsa, under Secretary of Defense for Foreign Affairs. Promoted to Vice Admiral, Zumalt moved on to command naval forces Vietnam. He was an extraordinary sailor, an extraordinary destroyer man his whole life, and then including obviously his time in a combat zone in Vietnam as a leader there. What he did was he consolidated the various forces into a new thing called Task Force 194, which was the Operation Sea Lords. To protect the riverine forces from being fired on from ashore, Admiral Zumalt ordered the use of a defoliant called Agent Orange that wiped out the trees and so forth along the banks and made it much harder for the enemy to have places of concealment. Got the attention of Richard Nixon among others through his efforts in Vietnam and ultimately wound up being jumped over I think 33 admirals to become the chief of naval operations the youngest one ever at that point. But began his term as CNO by implementing Project 60, a plan to reduce ship numbers across the fleet. Out of Project 60 emerged the concept of the high-low mix and a plan to quickly eliminate obsolete ships to counter the modernizing Soviet fleet. Zumalt also began issuing z-grams, policy changes aimed at deck-plate sailors to improve conditions and quality of life. They covered a vast array of topics, increased opportunities for women in the Navy, increased opportunities for minorities. We were not trying to create a Navy in which any group, any segment was anything other than an integral part. Bear in mind that at that time in the minority community we had the lowest number of minority officers and we had a tradition of segregation, the remnants that were still there all the way around. How could we quickly change it? The changes had to be made and we now have not only African American at all ranks but we also have Filipinos at all ranks and in the enlisted categories the restriction of those ethnic groups to just the housekeeping rates was eliminated and they now are doing wonderfully in all the ratings. After his term as CNO, Bud retired from the Navy. His son, Elmo III, who had served under Bud in Vietnam had come down with cancer. Bud Zumalt believed this to be directly related to the use of Agent Orange during the war. The tragedy of Agent Orange led Bud Zumalt to work tirelessly to get veterans benefits and compensation for those ill from its use in Vietnam. He also became a founder of the National Bone Marrow Directory. For this work, Elmo Zumalt was awarded the Presidential Medal of Freedom by Bill Clinton in 1998. Elmo Bud Zumalt passed away on January 2, 2000. His legacy lives on in the ship that bears his name. I think it's going to be a transitional vessel. I think it's going to sort of be the vessel that takes us from the kind of ships that we build today into ships of the future. I think it's a great ship. It has a great future and I think just like Admiral Zumalt when he was CNO, it will make a huge difference in the United States Navy for the 21st century. When our historians look back on the century we have just left, they may well record that Arleigh Burke was the spirit of the United States Navy. They will certainly recall that Bud Zumalt was its conscience.


Mr. District Attorney, starring David Bryan. Mr. District Attorney, champion of the people, defender of truth, guardian of our fundamental rights to life, liberty and the pursuit of happiness. And it shall be my duty as District Attorney, not only to prosecute to the limit of the law, all persons accused of crimes perpetrated within this county, but to defend with equal vigor the rights and privileges of all its citizens. This is David Bryan. In a moment we'll bring you another case from the files of Mr. District Attorney. The first a word from our sponsor. And now here is our star, David Bryan as Paul Garrett, Mr. District Attorney. In order to prove the guilt of a murderer, a District Attorney must establish a motive. But a motive is sometimes hard to prove, especially when the killer and his victim are total strangers to each other. This case started late one night at the service station on the outskirts of the city. Sorry fellas, but I'm just closing for the night. We need some gas. I've already locked the pumps. There's an old night place, bought two blocks of a big truck stop. You can't miss it. You hear that, Fred? Yeah, Benny, I heard. Man, don't want to wait on us, Fred. I didn't say that. I said I'm closed. He's laughing at us, Benny. He thinks it's funny that he don't want to wait on us. There's no point in getting out of the car. I'm not going to wait on you. You hear that, Fred? This is a real nasty guy. Yeah, no wonder his wife don't like him. What's that supposed to mean? You know, Benny, this guy ain't very bright. Yeah, maybe his wife never told him. Maybe she didn't want to hurt his feelings. What are you guys up to? What's in there? Get inside to your friends. Hey, what the... He said get inside. Gun, huh?  Move! If this is a stick up, you're out of luck. All the money's in the safe and the day man has the keys. He thinks it's a hold up, Fred. I don't know why, Benny. You got a nice, honest face. Could it be me? No, you got an honest face, too.  What is this? Some kind of a gag, Estella's poem? Oh, no, Mr. Wade. It's not a gag. Would you think it was a gag when she took out the insurance policy last year? Why... I don't understand, you guy. Step into the accident room, Mr. Wade. You heard me get in the storeroom. No, no, listen to me, kid. In there, he... Well, now we have got it nice and private. Don't be crazy. You guys, you'll get caught. You'll go to the chair. You know what she was right, Benny? This boy's a very dull talker. Yeah, he grabbed a tie, Ryan, Fred. No, no, let me call Estella. Let me talk to her. Not only Bora.  Give it to him, will you? You can't! Can't we? Oh, I did it. Let's get out of here. Hope his wife don't get nervous collecting that insurance money. Well, why should she? She's got a perfect dollar, boy. You're spending the evening with her mother-in-law. That's mother-in-law you mean. Do you know something, Benny? Oh, what? I'm hungry. So stop when we get to town. Let's eat. Be with you as soon as I finish this paragraph, Mrs. Wade. Thank you. I'm sorry to keep you waiting. That's all right. It's all I've done for a month. Wait. What does Mr. Garrett want to see me about this time? Well, I'm afraid that's something he'll have to tell you. Mr. Garrett. Yes, Miss Miller? Mrs. Wade is here. Thank you. Come in, Mrs. Wade. Go right in and have a seat. Call Hank and Miss Miller. Ask him to come in. Yes, sir. I'm sorry to have to call you in again, Mrs. Wade. I'm sorry too, Mr. Garrett. My husband's been dead for almost five weeks. I want to get away someplace by myself and forget about it. But you won't let me. You want to help us find out who murdered your husband, don't you? Of course I do if I can. But I wasn't with him when it happened. I was spending the evening with my mother-in-law. Yes, I know. As a matter of fact, that's the only time your mother-in-law saw you in more than a year, except for your husband's funeral, of course. Well, what of it? Well, you hadn't been speaking. But you paid her rather a sudden visit at what turns out to be a very strange time. What are you insinuating? I'm not insinuating anything, Mrs. Wade. I'm just stating a fact. I might as well tell you, Mr. Garrett. I don't like your way of stating things. I certainly... Come in, Hargan. Hello, Chief. Afternoon, Mrs. Wade. Sit down. Pardon the interruption, Mrs. Wade. I'd like Hargan to hear this, too. You don't seem to be able to find my husband's murderer. So you seem to have selected me as some sort of a target to hide your inefficiency. Well, I'm not going to stand for any more of it. The insurance company was perfectly satisfied. They paid me despite your investigation. They paid you because I suggested they pay you. Is that supposed to be amusing, Mr. Garrett? I hope it was, because this isn't going to be. Read your report, Hargan. Be glad to. You got a check for $40,000 last Thursday, Mrs. Wade. Double indemnity payment on a policy on your husband's life taken out 11 months ago. What else is new? Don't be impatient, Mrs. Wade. It gets more interesting. Go ahead, Hargan. During the 11 months, the policy was in force. You borrowed the money to pay the premiums from France. Girls you used to work with before you got married. Is that a crime? Your husband's salary didn't warrant that kind of insurance, Mrs. Wade. The agent advised you against it when you bought the policy. Both he and your husband wanted to write it for a smaller amount. Unfortunate for me, I didn't let them. Suppose your husband had lived out a normal lifespan of 70 years, Mrs. Wade. Did you intend to keep borrowing premium payments for 40 years? Go ahead, Hargan. Mrs. Wade deposited the insurance company check in her bank on the day it was issued. But the next day, Mrs. Wade, you made a withdrawal of $10,000 in cash. That's a large sum, Mrs. Wade. Where is it? I'm not obliged to answer that. In other words, you haven't got it anymore. I had some bills to pay. Might I see your receipts? Personal loans from friends. How about a list of their names, then? We can ask them. I won't have you embarrassing me. No, and I'm sure you would be embarrassed if we could find out who got that money and why. I don't have to listen to this. I'm leaving. You do, and I'll have you brought back on a warrant. All right. I'll listen to you for one more minute. But the next time you call me in here, I'm going to bring an attorney. I advise it, because the next time I call you in, you may need one. Do you know what these are? No. Reports on your husband. No criminal record. No dubious associates. No enemies. Well liked by all who knew him. Except you. Do you know what reports like this, in cases like this, always make me think? I'm not a mind reader. They make me think of professional killers. Killers hired by somebody who can profit by the murder. But be shielded by the safety of a perfect alibi. Do I make myself clear? Too clear. What did you do with that $10,000? None of your business. If the lecture is over, I'll leave now. Mrs. Wade? Yes? A word of advice. Don't make any attempt to leave this state. If you do, I'll have you arrested at the border. That's a cold tomato, chief. Ice cold. Where did you post Tennessee? No, he's waiting in a phone booth in the drugstore on the street level. Call him. Tell him she's on the way out. From now on, I want her under 24 hours surveillance. Kraft? Yes, sir. I just left Garrett's office. What did he want this time? He... wait a minute. Can't you find another booth to wait for? I'll be some time. Hello? What was that? It was nothing, Kraft. There's somebody waiting for the booth. What about Garrett? He asked a lot of questions about the $10,000 I gave you two. You know. Let him ask. What if they happen to pick up those boys you hired for the job? They're going to ask them. All right. I waited a long time for this money. I want to enjoy it. Goodbye. Not at Tennessee Fine, not at what's Max Kraft, she called. He was coming out of the drugstore to tell her after we called him. But when she came off the elevator, she had it for a phone booth herself, so he tagged. He pretended to be waiting for the booth, too, to see if he could hear any of what she said. Well, that's a lucky break for us. Let's hope Max Kraft actually has a hand in this. How do you know he'd be at the cameo restaurant, though? It's his hangout. Goes there for dinner every night, about eight. Sticks around until midnight or later, listening to himself talk. He gets quite an audience, too. True. You know, thugs, hoodlums.  Well, here we are. We'll have a good show for his audience tonight when you start asking him questions. Well, there he is. Corner booth with those two guys. Quick, into the alcove before he seizes. Before he seizes? He came down here to talk to him, didn't he? Not now. Not after seeing who is with him. Take a look. Yeah. You recognize them? I've seen them on a hundred wanted bulletins. They've done time and half the pens in the country. Benny Fox and Fred Lesser. Yes, Benny Fox and Fred Lesser. The only two mentors, the Black Sash Gang, the Detroit police didn't put away for good. They were with the Black Sash Gang? That's right, Hankin. The Black Sash Gang. Alias, the murder syndicate. Calling us Paul Garrett. Mr. District Attorney.  We suspected professional killers hired for the job. But the only link between the widow and the killers seemed to be a disbarred attorney Harrington and I had traced to a midtown restaurant. They're getting up to leave, Chief. Yes. Benny and Fred seem to be unhappy about something. Have they ever seen you? Well, they know you by sight. Benny and Fred? No. Max Kraft knows me, though. I don't want the others to see you, not just yet.  Ah, what are you going to do? Say hello to the gentlemen. Here they come.  Can you tell us by phone? You worry too much, Benny. We like to worry, Max. Well, hello, Paul. Hello, Max. Well, I'd like you to meet a couple of friends of mine. They're from California. Mr. Evans and Mr. Johnson. Gentlemen, this is Paul Garrett, our District Attorney. Nice to meet you, Mr. Garrett. Yeah, real nice. Your friends don't happen to be in the stolen car business, do they, Max? Paul, these gentlemen are in the real estate business. Yeah. Yeah, real estate. Thinking of opening offices here. I've been telling them about you. About how nice and clean you keep this county. And tell them I intend to keep it clean of automobile fees. Good evening, gentlemen.  Good night.  Yes, Harrington. I'm afraid he does. What was that stolen car business all about? Just a throw-off. To make him keep on thinking as he does. Why didn't you want them to see me? Because you're going to be part of a little scheme, Harrington. A scheme to draw them out into the open. Oh, how? Oh, to be brief.  Well, how nice for me. It's our only chance, Harrington. If we could catch Benny and Fred red-handed, they might open up on Max. And if Max gets trapped, he'll dump anybody overboard trying to get himself out. That's right. Let's go. You know, there's just one thing about it that bothers me, though. What? How dead do I have to get to make this thing work? Just dead enough to make it convincing. Convincing to who? Them or the undertaker? Harrington, you worry too much. Who's going to hire them to kill me? Miss Miller. And from now on, her name is Mrs. Miller. And you're Mr. Miller, her husband. Any objection? No. Just make sure of one thing. What's that? I'd like my maiden name back when you order the tombstone. Yes? Are you Max Kraft? That's right. I phoned you yesterday, Mrs. Miller. Oh, yes. Come on in. What's so important that you couldn't talk about it over the phone? My husband was around. I didn't want him to know yet. I want a divorcee. I see. I want you to handle it. I can't. Why not?  Uh, disbarred attorney. Let me make a call. I'll get somebody for you. But I want you to handle it. I've read so much about you. I remember so many of your cases. You were always so clever. Well, I didn't know I had such a good-looking fan. Please. You can find some way to do it. Well, maybe I could work behind the scenes. Sit down, won't you? Okay. I can't pay you too much, though. Even if I get you a nice slice of alimony? From my husband? You'll never have a dime. The only way I could get any money out of him would be if he'd drop dead. Dead men don't pay alimony. No, but insurance companies do. That's all you ever think of, insurance. How's your husband's health? Too good. I wouldn't be here. Why? I don't like to see young ladies rush into a divorce without thinking of all the possibilities. I've thought of all of them. Have you? Let me think about it. How about having dinner with me tomorrow night? Then you will help me? Maybe. After we get to know each other better, I can help you a lot.  You will find out tomorrow night? Where? In the cameo, eight o'clock. If I can get away? I'll be waiting. Goodbye. Yeah, Kraft. You get my letter? Yeah, we got it. You burn it? Of course. What do you think we're stupid? The instructions, though. Well, remember, you just tell us when. He goes bowling tomorrow night. Eight o'clock sharp. Yeah?  Mm-hmm. Do it when he's on his way back home from the bowling alley. Okay. Is Fred there? He's across the street eating again. I'll go tell him. Goodbye. They still out there, Chief? Yes. Hennessy said they're parked in the far corner of the parking lot. It's almost eleven o'clock. You can get ready to go. You know what to do. Drive straight up Franklin Avenue. There are unmarked, prowled cars at every intersection. As soon as they gun you, we'll get them. I hope that car was as bulletproof as you think it is. Don't worry, it is. And when they open fire, break, skid, run into a building, make it look good. The inside of the car is covered with crash padding. You won't get hurt. If I don't, it's a cinch. Nobody else will. Well, here goes nothing. All right. Here, you forgot your bowling ball. Thanks. I wish my head was this hard. Just puts it back. He's not gone very fast, Bunny. Just fast enough to make the lights without stopping the time for twenty-five miles an hour. The street up ahead looks good. Yeah, street light in the middle of the block. I'll pass him on the outside just as he gets to it. No, no, no. Not too fast. Look, I know how to drive. Okay, okay.  I'm rolling it down. Happy? Yeah. Then speed it up if you're going to pass at the light. Now, how's this? Perfect. We're coming up just right. Now! It was sweet. Get rollin' fast. There isn't another car in sight. Benny, Benny, Benny, Benny, Benny, up ahead. Two cars blocking intersection. Back up. Turn around. It's no good. Other cars coming from back there. Red lights flashing. It's police cars. Alright, pile up. No, no. The wall over there. They'll chop us to ribbons with that thing, Benny. Alright, drop your gun. Hey, we give up. We drop that gun. That's better. Looks like we got to you a bit too late to say that man you killed back there, though. Who tipped you to this? His wife got cold feet at the last moment. I've seen you two before, haven't I? With Max Kraft. I told you, Benny, we were crazy to trust Max.   He was so sure his stain wouldn't crack up. Oh, no, he tells us. Iron nerves, just like the Wade Dame. Mrs. Wade, the wife of the service station operator who was killed? Who else? Max and his bright idea.   If I get the chair, I'm going to make sure one thing, Max Kraft is going to be sitting there right in my lap when he turned the juice on.  That's right. And he got the pay off from Wade's wife, but still out of the insurance money. You hear that, Harrington? Yeah, Chief. I heard. Hey! This guy ain't dead. Not because you didn't try. You see what your blabber did, Fred? They had nothing on us. Until now. All right, boys. Take them in. Go ahead. All right. Quit your shop. All right, Harrington. Bring your bowl and bowl, and let's roll for the spare. Mrs. Stell Wade and Max Kraft are all set up in the middle alley. Go ahead, Mrs. Wade. Lock it up, Mulligan. Down that way for you, Kraft. What do you want, Max? How about me turning the state's witness, Paul? Want to make a deal? I don't want to. And I don't have to. Don't be too sure, Paul. When it comes to courtrooms, I know a few tricks. You'll need them because I've got a surprise witness who might be too much for you. Who? Mrs. Miller. She knows your whole operation. Don't kid me, Garrett. Why would she testify and incriminate herself? Because she isn't Mrs. Miller. She's Miss Miller, and she happens to be my secretary. You see, Max, I know a few tricks, too. This is David Bryan. I hope you enjoy this case from the files of Mr. District Attorney. I'll be back in just a moment after this message from our sponsor. Now, here is the star of Mr. District Attorney, David Bryan, with a word about the program you have just heard. The law clearly states that the person who engages a professional killer or a homicide is fully as guilty as the harrowing who wields the murder weapon. Estelle Wade, Benny Fox, Fred Lesser, and Max Kraft were all found guilty of murder in the first degree. There were no recommendations for clemency. Now, this is David Bryan inviting you to join us when we present our next case based on the facts of crime from the file of Mr. District Attorney. Mr. District Attorney was originated by Phillips H. Lord.