So thanks to the organizers for organizing such a nice workshop that I'm enjoying a lot. So I will be telling you about analyzing performance of gradient descent based algorithms but before getting to that I have some motivational slide and the main bulk of the work was done by my students, Stefano Sara Omanelli who is one of the two students actually is not here. And our collaborators, Giulio Chiara, Floron and Pierre Francesco and part of the talk will also be mentioning work that has been done with Jean and Nicolas and Leo. So the motivational slide so we all know and that's what we are discussing here that there was some revolutionary progress in machine learning done in the last 20 years or so but there also are some open questions of course so here I take a snapshot from an article that I have read recently that summarizes probably the three questions that I think are kind of the most kind of relevant or interesting for me at least. So let us go through them. So why don't heavily parametrized neural networks overfit the data? So we already discussed that a little bit in the workshop. This kind of a riddle of the overparameterization that even though we have so many parameters and there are many possible ways how to set them to fit the training data that the algorithm somehow magically managed to choose a configuration that generalizes as well as well. So how is this possible, this lack of overfitting in neural networks? The second question is about what is the effective number of parameters, what is kind of the effective dimension related to the data set and to the task that would somehow translate directly for instance to the sample complexity, that is how many samples of a given data set do we need to be able to learn something correctly? So we don't really have a good understanding of that either. And the third one, why doesn't back propagation, so this gradient descent is like very, very straightforward algorithm, why doesn't it head for a local minima? Why does it go to a local minima with a good generalization? So these are the questions that are discussed these days, all deep learning theory meetings. The interesting thing about them is that this article I read it recently but it's actually really old, it's from 95, so 25 years ago, it's written by Leo Breidman, it's a reflection after referring papers for NIPs and it's still very current, right? So this is kind of telling me that although we moved really a lot on the engineering and application size, we didn't move so much on the theory side because we don't understand these questions so much more better than we did 25 years ago. So this being said, kind of I like to put the, how do we go towards theory of deep learning in this kind of a triangle diagram, where I say that in order to build a theory we really need to take into account interplay of these following three ingredients. The architecture, the data structure and the algorithm. Now you see if we don't take for instance into account the algorithm as for instance in yesterday's talk by Helmut, then we have beautiful theory but we don't know whether the configurations that exist are tractably findable, so this is useless for practice. If we don't take into account the data structure or in other words if we consider that the data are the worst possible case, then we have very strong theorems in computer science that are telling us that training even the simplest neural network is NP-complete. So of course the real data are not the worst case data but what we lack completely is some kind of characterization of what's the magical property of the data that makes the task easy. We know that the worst case data would not make it easy. And the architecture of course if you are not considering the right architecture so we don't really know that there is no shallow network that is able to do the tasks we are seeing nowadays done by deep networks. We just haven't found any, like the best thing we have are the deep networks so if we are building theory of that well we better take into account the right architecture. So the kind of trouble of interaction of these three ingredients is that each of them is studied in different of the traditional fields or subfields of engineering and computer science. Right, so it's the computer science and optimization theory that deals with the algorithms and is the signal processing that deals with the data structures and is the approximation theory and the learning theory that deals with the architectures and they are not always talking to each other. Nowadays maybe there are more and more but traditionally not so much. So that's maybe where the trouble kind of of putting these things together comes from. So I also put here a talk by Elhanan Mosul this year in a bootcamp in the Simon's Institute for Theory of Computing who put things slightly differently but is making the same point. Now Elhanan went even farther, I loved what he did. He took some of the influential papers in the theory of deep learning and he rated them according to how well they are taking into account these three ingredients. So I was tempted to do something like that for the talks in this conference and then I shied away from it. Maybe it would be better to do it in some collective effort to kind of remove the personal biases in that. So now going to the statistical physics, kind of the approach from statistical physics from the very beginning was taking into account all these three. Not necessarily as well or not taking the architectures as they should be etc. But that's kind of on which I'm trying to build. So where did it all started? In my point of view the most influential of these early papers are these works by Gardner and Derrida and notably the one where they studied a teacher-student perceptor. So we already heard about this in Mark's talk and also a bit in Andrew's talk. So what is this? This is the single layer neural network with no hidden unit whatsoever. There is only the input and the output and then there is the teacher that takes some ground through vector of weights w and with the input pushing it through some non-linear function phi, imagine a sign here, generates an output. And then the student observes the input data and the output labels. She doesn't know the ground through vector of weights w star and the student is trying to reconstruct this w star or at least a function that implement, some w that implements the same function. So that's the student, teacher-student perceptron in this regime that kind of is not the traditional statistics where the number of samples is much bigger than the number of dimensions. Here the number of dimensions and the number of samples is comparable. The ratio between them is constant, that is called alpha. This is kind of the regime in which the current machine learning is working when these two are comparable. One may be bigger or the other but they are not vastly different. So this problem was also solved back then. But I guess at Georgia just one year after with the replica method that in this community we know very well. And when I say solved, what he computed is summarized in this picture here. So he computed the optimal generalization error that any algorithm, even exponentially costly can reach as a function of this parameter alpha. The alpha again was the number of samples divided by the dimension. So as alpha is bigger, I have more samples, it's easier to learn and when alpha is small, it's harder to learn the error is big. And what he realized is that, for instance, for the case where the teacher weights are binary, this optimal generalization error has a phase transition. There is a sudden drop from some crappy generalization to zero generalization error at the gardener Derrida transition point, which is 1.245, etc. So this kind of sharp drop in this generalization curve is for physicists very interesting of course. And from the point of view of learning theory, this is one of the few cases where we can actually up to all constants compute something like that. This is very difficult to compute in general, for general network, general data, we have no idea. So the point is we would like to get pictures like that for more realistic settings. So now let me just show you this is all 35 years ago or so. So what is the recent progress on this specific model? The teacher student perceptome. So that's something from our paper from two years ago, that will kind of motivate the second part of the talk. So I split this in three points like the kind of contributions that we did in two years ago is that first of all, gardener Derrida considered the activation to be sine and the ways to be the binary or spherical. But we can write a closed formula for any activation function and any class of priors, okay? We don't have to write a new paper for each of them. It's a formula that has a special function that depends on the prior and a special function that depends on the activation. So it's nice to have. The second point is that the work of Gezadorji was purely information theoretic. He didn't speak about whether this generalization error is or not reachable algorithmically. So what we understand now is that by working with an algorithm called approximate message passing that is really closely related to what we call the top equations in physics. We do understand which part of this curve is reachable and which one is not and I will show you in two slides. And the third point is that, speaking in front of mathematicians and computer scientists, we always used to have to apologize that the replica method is non-rigorous. Nevertheless, we believe it is correct, etc. So we also managed to actually prove that this Therida Gardner and George Sompolinsky, whatever, formulas are actually rigorously correct formulas for the best generalization error. And that was largely thanks to efforts of Jean and Nicolas and Leo in this collaboration. And Florian, well, everybody else, but me, whatever. So to show you what we get, let me show you some examples of, say, phase diagrams of this teacher-student perceptron. So here I'm taking spherical weights and sine activation. So that's the most commonly studied case. And the best generalization error as a function of alpha is the red curve. Black points are the performance of the algorithm on size of, on instances of size, something like 10,000, I forgot, something like that. And the blue points, that's just some black box binary classification. In this case, we just took logistic regression from psychic learn. And you see that in this case, first of all, the theory tells us the algorithm should do as the theory predicts. This is the case. And the black box algorithm that is kind of not knowing much about nothing about the problem is very close, so there is not a big gap. So that's kind of a nice case. But it's not always so nice. So for instance, in this case where the teacher weights are binary, the activation is still a sign. The optimal curve is that's the red curve here. That's the one that I showed already in this Georgie paper, is the red curve. But now what the algorithm does is the black points on a single instance, and the green curve on average from the theory. And you see that there is a gap, that the algorithm doesn't do as well as the theory says we could do with an exponentially costly algorithm. And this gap is actually, we believe it's of the very same nature as the one that Afonso was discussing on Monday. That actually, if this approximate message passing doesn't manage to do it, there is no polynomial algorithm that will manage to do it. That's of course conjectured, this is not proven. But there is something fundamentally hard about this region. But maybe what's more interesting here is now the difference with the black box algorithm. So this is still the regression, logistic regression from scikit-learn. But it doesn't know that the weights are binary, it just looks at the problem, it's just a black box classification problem. And it's not picking this phase transition, it just continues decreasing basically as it was before, it's not picking the structure. So there is a gap between the performance of the approximate message passing and this kind of traditional gradient based algorithm. And there are cases where things are even worse. That would be, for instance, again, binary weights. And now the activation is slightly different. So I take absolute value of this color product, I shift it a little bit and I take a sign of that. So we call that the symmetric door activation. The constant is chosen so that half of the outputs are plus one and half of the minus are minus one. Then again, the meaning of the red curve is the information theoretic curve. Green is the performance of the approximate message passing algorithm. And here in the inset, here we tried, we tried pretty much with TensorFlow a bunch of networks and we ended up with some three layer fully connected neural network. It was kind of managing to learn the functions, say, for alpha around 10, 15, and not quite even exactly. Whereas the approximate message passing does it at around one. So there is a gap here. So where does this gap come from? So that will be the subject of the rest of the talk. So to summarize what I said so far, we have in these simple models, good understanding of the information theoretically of the performance of what the message passing algorithms do. But in practice, people don't do message passing algorithms because they are very sensitive to the data being really random and we don't have their versions for the multilayer network. In practice, people train with gradient descent based algorithms. And even in these simple settings, we have gaps between what the gradient descent based algorithms are doing and what the message passing is doing. So we would like to understand where these gaps come from. Can we get rid of them? By changing the algorithms, it's something that we need to understand if we are to progress on what the gradient descent is doing in the more practical settings. And we don't have any such understanding. We have some dynamical mean field theories, but none of them is actually giving us an equation that we could plot and that would give us performance of those algorithms. So this is kind of embarrassingly an open question. And that's the one that we are trying to attack. So deep learning is fueled by gradient descent. So we need to understand gradient descent. We need to progress on the methods to do that in our community. So I just mentioned here that the kind of nice exception where we do understand the gradient descent as was really nicely presented yesterday by undoing out the linear networks. So the dynamics there is really interesting, but the fixed point is not interesting. The fixed point is just linear regression. So there is a lot of features that are missing in there. So we can learn a lot, but not everything. So we need to develop methods that would let us analyze the nonlinear cases. And so this is where I'm going. So what are these gradient descent based algorithms that we will be analyzing? So in physics, we most often talk about the Langevin algorithm. That's the one where we take the Hamiltonian and the gradient of the Hamiltonian with respect to the spins or variables. I will be working in a spherical model. So here is a spherical constraint. In machine learning, this constraint is called the weighted decay. So that's something that people use often in practice in training. And then there is this Langevin noise that if it has zero mean and the variance is proportional to the temperature, if this temperature is one, then this is designed exactly in a way that for finite system size at infinite time, it's sampling the Boltzmann measure of that Hamiltonian. So that's a nice property of the Langevin algorithm. But we are of course not interested what happens for a finite system at infinite time. We are interested what happens for a large system in linear time because there's the practical scaling that we can like hope to achieve in practice. And then we are not guaranteed to sample the posterior and then there's the whole question, what is happening in large constant time if we divide it by the number of spins? And then being able to analyze that at any T, we also give access to being able to analyze what the gradient flow is doing. There's the continuous time gradient descent algorithm that doesn't have the noise. It's just going in the direction of the gradient. So in which model can we analyze it? Can we analyze it in this teacher student? Ideally, what do we want? We want some high dimensional non-convex loss function that will resemble these neural networks. So the random perceptron would be a candidate but we don't want the random perceptron because we also want to have the notion of the generalization error. This is really important. We don't want to be analyzing optimization problem because as we correctly heard yesterday from Joachim, learning is not optimization. We want to understand a performance of a learning problem where we have some loss function that we optimize but that's not the goal. The goal is actually to achieve a good generalization. So we need some other notion of what is good than just being low in the energy landscape. So we need some teacher-student model. We need some ground truth and be true to the ground truth but the teacher-student perceptron that would be a good model but unfortunately we don't know how to analyze the dynamics for the teacher-student perceptron. That's too difficult. There is, I could have put the citation. There is actually a paper by Giulio Biorli, Francesco Boni, Valentina Ross, Frances, Pierre Francesco Boni, Francesco Zamponi. Maybe I forgot somebody. Where they are kind of going towards the mean field theory for the perceptron but they end up with equations that they basically cannot solve and it's, you know, we are not quite there. It's very difficult. So we need a problem where we can solve the dynamics. And this brings us, you know, the only problem in physics where we know how to solve the Langevin dynamics is the spherical P-spin model. So we try to do something like the spherical P-spin model but it also needs to satisfy the other criteria. So we could do the spherical P-spin model but that one actually behaves bizarrely. It's very special. We know in physics that it's very special. So the one that is more generic that has a better chance to be kind of universal in how the landscape behaves is actually the mixed spherical P-spin model. And so this is how we set the problem that kind of satisfies all the shopping list. So this is the model for which we will analyze the dynamics. So we call it the mixed spiked matrix tensor model. How do we, what's the problem? The problem is there is some ground through X star, that's a n dimensional vector, that we observe in two different ways. We take the vector times it's transposed and add a load of noise of variance delta two and that's a matrix. Then you observe the matrix. And we take, you know, P-uple kind of product of tensor product of this vector and add a noise of variance delta P and that's a tensor and we observe the tensor. And then the problem is from knowledge of this matrix Y and the tensor T can we reconstruct the X star. That's the inference problem and the notion of goodness is are we at the end close to the X star or not? But the way we will be solving the problem because that's the algorithm we want to analyze is we write the corresponding Hamiltonian and we will be doing the gradient descent in this Hamiltonian, a temperature one or zero depending whether we want to look at the launch of our gradient descent. And so this Hamiltonian that's really the Hamiltonian of the spherical mixed two plus p spin glass for those of you who know with a little difference that in the usual problem the T and Y are completely random matrix and tensor and here they're not completely random. They have this small perturbation that has a low rank and that kind of makes it possible to discover this ground through X star and that changes slightly them all. So now, okay before going to the dynamics let me first show you kind of what we know about the information theoretical possibility to reconstruct the signal in this problem and what the approximate message passing is doing because this is what we will be comparing to. So I have two slides on that. So first of all, what is the estimator that is kind of telling us about the information theoretic performance? That's the one where we take the posterior and we sample the posterior, we compute the marginals with respect to the posterior and that's the right estimator and that's what the Langev algorithm aims to do. But we can also do it with the message passing and then the maximum apostrophe inference as we heard from Joachim yesterday that's not the right thing to do. This is the right thing to do. Here we know what's the right thing to do because we know them all. This is not the right thing to do. Nevertheless, everybody is doing it so let's analyze it as well. Okay, so these two things we will be analyzing them. So let me show you was the result of the first one. So this is just a list of references. It goes pretty much in the same line as this paper I was telling you about in the perceptron. In the last, say, three, four years we know how to solve such problems for generic priors and generic activation functions and we understand the behavior of the approximate message passing algorithm and we have a rigorous proof of all that and these are the references. If you want to go into details, but the result is that all we need to write for this problem is the so-called replica symmetric free entropy or energy if you want, I just changed the sign and I called it entropy. It looks like this for the present mall. It's very simple. It just has the order parameter which is the overlap with the ground through configuration and that's it. Then it has the two P is the order of the tensor and it has two variances of the noise. There's the only two parameters that we have in the mall and the P if you want, but I will always be taking P equals three up to the very last slide. So it's a very simple formula and we know from the previous theory that the optimal error corresponds to M that is the global maximum of this formula and that the error that the algorithm is reaching is the local maximum that has the worst error. So sometimes they are the same but they don't have to be the same and if I collect this information in a phase diagram that is on the X axis is the variance of the noise on the tensor and on the Y axis is one over the variance of the noise on the matrix. Maybe it is our choice but when you stare at it for two years you will get used to it. So this is how the phase diagram looks. If the variance on the matrix is smaller than one that's the green region then the approximate message passing is matching the information theoretic performance is recovering something correlated with the signal the magnetization is positive. In the red region, the information theoretic performance optimal performance is such that the magnetization is still zero. So this is a region where the noise is too big the signal is completely hidden and no matter what you do algorithmically either exponentially costly you cannot find it back. So it's information theoretically undetectable. And the hard region that's the one where information theoretically is detectable physically this is a ferromagnet but the approximate message passing is not able to do it. So this is this hard phase the disconnection to be hard for all polynomial algorithms. So now the whole question is how does the Langevin Dynamics do in this diagram? Does it do, it could do better than the approximate message passing but I just told you that this is conjectured not to be the case it could do the same, it could be worse. So that's our question. What does it do? So this is back the Langevin Dynamics and now when we go to solving it I said the mix, the spherical P-spin mix model is the only one where we can actually where we actually know how to solve the dynamical equations with the dynamical mean field theory. And that is something that has been done long time ago in these very influential and nice works that basically standard the basis of our understanding of structural glasses today. By Crisanti Horner-Sommel and Kulian Dolo-Kurcha for the mathematicians or mathematically minded colleagues in the audience the correctness of these equations that they derived using non-rigorous techniques was proven by Gerard Benarus and Amir Dembo and Alice Guiones some years later. But that's for the random spherical P-spin model. We are in the planted one so we cannot completely readily use their equations and their proof. We have to redo them to take into account that there is the planted solution. So fortunately this is possible, this is not a big deal and this is how the equations look like. So I of course don't understand unless you already know the Kulian Dolo-Kurcha equations you will probably not be able to understand where this comes from. But just to give you an idea how they look these dynamic equations so I split these three are for temperature one these three are for temperature zero. They are written in terms of three order parameters the autocorrelation between configurations at different times the response function between some perturbation at time t prime and what is happening at time t. So that's the ones that also appear in the Kulian Dolo-Kurcha equations and here because we have the planted configuration we have a third order parameter which is simply the overlap with the planted configuration and that's the one we are really interested in. So they are self consistent equations on these three order parameters but the one that we are most interested in is this one because that's directly the correlation with the ground true our question is does it stay zero during the dynamics or does it correlate positively with the signal? So we solve these equations numerically and what we get so if you are interested in the code is here and what we get is in this picture compared to the corresponding dynamical evolution of the magnetization in the approximate message passing algorithm. So in the approximate message passing algorithm the magnetization as a function of time does something very intuitive as the noise is getting smaller, the magnetization, the correlation is getting bigger and the time it takes is basically independent on the noise. Slightly shorter for smaller noise. So this is intuitive, smaller noise, easier problem. Bigger magnetization, a bit shorter time. What happens with the Langevin dynamics is intuitive in the sense that as the noise is getting smaller the magnetization we reach at the end is bigger it's actually the same as for the AMP algorithm that's consistency check but it's taking longer. So as the noise is getting smaller it should get easier but to converge to the right point takes actually longer time so it's not getting easier, it's getting harder. So this is not intuitive nevertheless this is what is happening as the noise on the tensor is getting smaller the tensor is becoming stronger is making the landscape more complex and it takes to the algorithm longer to actually find where it should go. That's what's going on. But I will, this is intuition I will develop on this intuition. So what we can do now we can collect these times it takes to converge and extrapolate where do they diverge. So there was a picture like with these extrapolations but you can imagine what we are doing here these are like standard methods we fit with the Paolo we are careful about like being doing things properly not over like half decade but several et cetera. And when we collect the points where the Langevin dynamics stops working these are the points on this picture we split this phase diagram into again like by this line into a region where the Langevin algorithm finds a correlation with the signal as good one as the approximate message passing and then this region here where the approximate message passing worked but the Langevin algorithm stays stuck at zero correlation zero magnetization. So this is what's going on. So see this kind of tells you know explains the pictures that we saw before that the problems are harder for the Langevin algorithm than for the message passing even though these two algorithms have the same goal they are both settled to kind of tell us something about the marginals of the posterior. Okay, so we can do the same thing for the gradient flow just doing the same thing but setting the temperature to zero this gives us a curve that is slightly above. So this is expected that it will be worse than the Langevin because it's optimizing the wrong objective as we discussed yesterday. But you know it's not infinitely farther away there is some region where gradient flow is working. Now how do people in computer science in optimization theory explain something like that? A popular explanation is okay, look at the landscape of the problem there are some spurious local minima and you need the signal to noise ratio to be large enough so that only the minima you are interested in survives and all the spurious ones go away. There is a long line of work in computer science where they are trying to prove by the fact the architecture is set in deep learning the landscape is such that there are no spurious local minima. Of course under some assumptions that are mostly not realistic I mean this is if it was true then things would be solved it would be easy but these works it's an interesting line of work and kind of our question here okay is this the only thing there is? Like is it really true that we need to get rid of all the spurious local minima for the algorithm to work? So if it was true then the line that I showed on the previous slide should be exactly the line where all the spurious local minima disappear. Now in general this is something that we don't know how to analyze. In our model because it's so simple you know it's basically like fine-tuned for all the physics methods to work we can analyze that. So how do we do that? We count the number of minima of a given order so given number of negative directions. So no negative direction in the case of minima with the so-called cat's rice formula. And again I will not be going into the details. You know this is a well-known technique that you know just this year and past year it has been used for the non-mixed case of this model in this paper. And also so that would be the annealed case and the quench case was developed in this very nice paper again by co-authors and collaborators. And once again you know that's the formula I am not expecting you to really understand how we obtained it. What it is it gives us the logarithm of the number of minima that have given magnetization, given overlap with the signal and given energy term for the matrix and given energy term for the tensor. Okay so this is what the calculation splits and now when we analyze actually what this gives us we can plot the line where all the spurious minima disappear where only the one that corresponds to the planted configuration remains. So let's do it, there's the purple line so we call it the landscape trivialization threshold if you want. So now the interesting thing is that it's not the one where the gradient flow stops working. So this is interesting right because what it means is that there is a region where there still are spurious local minima but the gradient flow doesn't care, it works. It finds the best possible correlation with the signal. So how is that possible? So what's going on? So looking in more detail into how the dynamics behaves we can understand the following. If we plot the energy that we reach as a function of time at four different points that correspond to points along a line here in the phase diagram we see that either the noise is too, we are in the very hard region where we just are stuck to some plateau the same for the yellow curve or we are stuck to the plateau but eventually we go away or we kind of go away sooner and what is this plateau? Okay here, so this is from solving numerically the differential equations that we derived for the dynamics. These are the same things from simulations from rather big system that we sparsified so that it can be so big but you see I mean this is kind of, this is the same curves. Here you can't see it really but there are many, many curves and this is the upper envelope and this is their average but they all kind of look like that but the plateau has different length. If you want to see the picture I will show you on the monitor then but what are these, what is this plateau? That's the key point here. So this plateau is nothing else than what we call the threshold energy in the spin glass literature for the non-planted system. So what is the algorithm doing? The algorithm, so how to interpret it? The way to interpret it is that at the beginning the algorithm has no idea that the planted solution is there so it just converges, it's just just what it would do if it was not there and if it was not there it would happily converge with some kind of aging and people study that a lot in the physics literature to these threshold states. And so then what matters is what is the nature of these threshold states? Are they actually nice minima that will block the dynamics or do they have a negative direction towards the signal? And if they have a negative direction towards the signal the dynamics will realize it and go to the signal. And so looking at these curves we hypothesize this, we say okay let's assume the dynamics goes to the threshold and then all we need to look is to look whether the threshold states have a negative direction towards the signal. And this is again something that we can do within the equations. It is all there already. Like once you already wrote all these equations it's just a matter of fact of putting them together correctly. So we need to compute the overlap of the threshold states and plug them into the condition that tells us is there or not a negative direction towards the signal and putting these two together we get a formula which is some expression between delta P and delta two but if we plot it, there's the blue curve and that's perfectly explaining the points that we obtain by integrating these equations numerically. So this seems to be the right curve and it has a simple formula like that. So that's the explanation of what the dynamics is doing in this problem. And kind of I like to, so this is the same, this is analogous curve. This was for the gradient descent, so zero temperature and this is this green curve that you actually already seen is the curve for the Langevin dynamic. So again it goes perfectly through the points and stops here at two. There is a multi-critical point that is interesting one. We don't really know what nature it is and okay. Yet, but what we kind of, what I like to do is to correct this wrong explanation. Okay, so to correct this wrong explanation this is the right explanation. So as the signal to noise ratio increases there is eventually the trivialization of the landscape where only the good minima remains all the others became settles but that's not what matters for the dynamics. What matters for the dynamics is these threshold states that happen to be the highest lying ones whether they are settles or minima. And when they are minima they will block the dynamics and when they are settles they will make the dynamics go to the solution. And so is this kind of transition that explains whether the gradient descent is working or not. And that was the line that I plotted on the previous slide. So this slide is just to say all these plots were for p equal three so tensor of order three. We don't quite understand everything because if we go to tensor of order four things are a bit trickier. The phase diagram is more complicated. There are some really interesting things happening close to the critical line but it's close to the critical line. So and our integration of the equations is not so precise to be able to go to so large times to actually properly exploit what is happening here. So we quite don't understand what is happening here. This curve would be just our formula that predicts some kind of a re-entrance here but we don't really know whether that re-entrance is actually real or not. So just for those of you who want to think about something more, you can think about the p equal four case. But the p equal three case seem to be all. Everything is consistent. There is no mystery that we could identify and we try, we always try. That's what we do. So a conclusion about this gradient descent part is that the gradient flow is worse than the Langevin algorithm that's expected because it's optimizing the wrong objective. But nevertheless, in practice it's more robust, faster, blah, blah, blah. So both are worse than the approximate message passing. So that's kind of, two years ago I would have been surprised if you told me that's the case, we didn't know. Now we are, it's clear evidence. The question is how can we tune them? How can we change them? That they actually reach the performance of the approximate message passing. So, if you wait for a couple of months, we will have an answer. I hope we will have an answer for that. Pretty confident we will. We actually do have, we just need to confirm it. The gradient flow works even when spurious local minima are present. And this is something that we can quantify with the use of the cat's rice formula. And the third point is that this is the first model that is nonchival resembles in its nature, learning, or at least it's an inference problem where we have a closed form conjecture for the threshold for the gradient based algorithms that include all the constants. In computer science, people would like have some log power something and so we really care in physics about the constants. I mean, not always in statistic in computer science but we should, I think. And so the question here is, is it applicable beyond the present model? And again, the answer is yes and we are kind of confirming it because the recipe that we followed at the end didn't need the original equations. So that's kind of the nice result here. The recipe is simpler and we can use it. Hopefully we need to confirm that for models for which we are actually not able to write the dynamic and mean field theory. So that would be really cool. But again, wait a few months. And to finish, I kind of want to give back this triangle or diagram and tell you where we are, where we meaning is my group. So everybody's kind of making effort in their way and kind of my way of making effort on this phase diagram, like filling things into each of these directions and making it realistic is, you know, this like analyzing the realistic algorithms or the algorithms that people are actually using, that's this talk. Okay, on the side of the architecture, that's actually where we are not so good yet. So we can deal with the committee machine which has a single hidden layer or few hidden units. Those of you who have seen the posters yesterday, there was a poster on the batch training of the committee machine by Benjamin Antoine and there was a poster by Sebastian on the online committee machine. So we are like understanding things and having progress on that. But so far, it's a technical challenge. We don't know how to derive corresponding formulas for the case where the hidden layer is extensive or where we have several extensive hidden layers. You know, even the non rigorous replica method doesn't work. I mean, it's not straightforward to apply. It doesn't, you know, you don't get to a nice formula. So this is kind of a big open problem. How to go beyond the committee machine. And on the side of the structure of the data, you know, most of you have heard the first talk of this conference by Mark Mazar. So that's our current progress on how to put the relevant structure in the data to make the data lie on this hidden low dimensional manifold. And we have a really nice progress on the analysis of that model, but that again is in a couple of months. But that's kind of where we are going. So the biggest open problem is in this direction. These ones, we are kind of confident that we are getting what we want to be getting. And with this, I just give you the list of references on which this is based, this talk is based. This one is unrelated, but for those who are interested in kind of the general area of applying machine learning to physical sciences, we kind of reviewed what has been done in the past, say 10 years or so here. So kind of happy about that paper. And with this side, let's open this to the discussion and to the questions. Let's thank the speaker first. Thank you.


Hi Ashwant, let's start with a little bit of work yourself, but you would not know. Can you introduce yourself, who you are and what you're doing? Okay, so my name is Ashwant, I work as a principal engineer at NDEX. I'm basically a developer and I like code for work and I have a lot of interest in operations, like how my code actually works in production and what both behind the scenes to make it work. So that's how I entered into this whole operation space before it develops. That's what I do. So I talked about NDEX, so NDEX is a data as a service company. We crawl and collect products that you find on save the cards, and after you'll target whatever, Walmart, and then we sort of normalize them, classify them, and then catalog them, and then give that to work NDEX. So that's somebody who wants to work with product information. NDEX is a data source that you can work with. So that's basically what we do as a business. And where are your clients basically? So predominantly most of our clients are from US. We do have some in India. We're very small, we're just expanding the whole catalog in India space. Great. Can you tell us a little bit about your background and how did you stumble upon your current job role at NDEX? Okay, so I did my data computer science from Shasta University. And then I landed NDEX as my as graduate, just after finishing my college at John's NDEX. I started off as being a software engineer, worked on the whole big data system, working with jobs, working with large amounts of data. And then as I started working with them, I realized that there are a lot of things that goes behind the scenes that we take for granted. And sometimes I get these prudious errors that is an out of memory that happened, or a GCO that didn't happen. And you really don't know why. I mean, at that point in time it was early in my career. And I was fortunate enough to have a very good team of mentors. Actually it was a team of mentors because I was the in-guest full-time employee at NDEX when I joined. So everybody who I worked with was very senior. So I learned a lot of stuff from them. And then when I realized that operation is something that I really found interesting, NDEX was also helpful enough to be an opportunity to work on that space, to contribute to the company. Challenges that come with the show? I mean, the being in... Okay, I'm not a first-class operationals engineer. I don't work in operation space. But as the culture at NDEX is that even if you are a developer, you need to take care of your operations. The one thing that you will learn is generally when I was in college, I was all about writing code. Can I get this outcome? Can I get this output? Can I make this page more prettier? That's basically what you work on. It's what you focus on. And you don't really worry about how does something like Google work? How does a system company like Twitter or Facebook work? They have also HTML and CSS for you. But there are a lot of systems that go behind the scenes. Now, if you're really passionate about learning, you don't just want systems. But how also they operate? Now, it's very easy to say it was a database to solve, to store your blocks. It's very easy for you. What do you... I need your database. I'm not saying you need to write a query and stuff, which is a good thing to have. That's what DB here holds at fault. But what does it mean to maintain a database? What it means to have a backup in place for a database? How do you keep replicas and things like that? What goes to make sure your production... I mean, as somebody, if you're working with, say, Twitter, you're using Twitter or Facebook, and then you suddenly see they don't open up, the first thing he goes is at down.com and then check the Facebook.com or Twitter.com to check if it's really down or not. And there are people who work day and night and put in wrong hours to make sure that doesn't happen for users like us. Now, if you are interested and passionate about stuff like that, now that's how you sort of get into understanding what was being in the scenes and then you get into these whole operations in this place. Got it. So, should you also act there in an order of community, like tech communities, some of the sort of gender, like the local gender group and the gender and tech community and such? Can you tell us a little bit about why people should even get involved?  Despite that, some of us put in, I think, recently a lot of time and effort to promote the events, to speak at events, to organize events and do that. Can you tell us a little bit about why that is important? Personally, for me, I participate in a lot of events right away that I can organize and be just a participant and listen to what people are saying because to me, it opens a perspective. So, I mean, I work at a startup and I'm exposed to a bunch of a problem space that I get to work on and I have enough opportunities to work on that, but there are also other people in different domains who are trying to solve a similar problem probably in a different way, taking inspiration from somewhere else that you might not even know of, they are not of.  To me, I mean, it's like a difference between you writing examinations by reading textbook, this is you doing your research or doing a project on your company, that's to get into that zone, the next zone of doing this if you want to do a project or a research, you need to have more additional inputs than what it takes for a service. So, that's when you come out of your circle, go meet new people, just listen to them, you can still be passive if you are starting off, I would recommend be a passive participant, just go listen to what they want to say and you can see in your community you get a lot of repetition, same people turn up over and over again for different set of things and you know, you can learn from this person and most of the times all people are super helpful, you just have to go talk to them or ask them, you know what, how do you solve this problem? And I learned a lot from going talking to people there and a lot of stuff that I do at work is not technically my own, it's just the inspiration that I got from different other people over time. So, that's one of the reasons why I go attend these meetings or talk at conferences. That's how I would recommend anybody to go to school also. Got it. Can you also, I know today has not been yet but so far what has been your team recovery at this conference? I mean, it's, my first and foremost thing is it's very nice to see there a big operational interest at conference like this, I think it's the first time that's happening and there are quite a bit of crowd, honestly, much larger than what I expected, so that's really good, which means there is a lot of school, there are a lot of people and I've seen a lot of people for the first time, so I've met a lot of people that I can go interact with, learn from, so that's one key takeaway. And then there are individual talks each of them gave me, to me it comes in two parts, one is some of it is a validation that some approaches are ideas that I follow, though I might think that maybe I discovered it, I follow it personally, if you get validated with the idea because a lot of other organizations in your exact same city know the same thing, all of them talk about resource schedulers, that's exactly what a key learning was, all of us, all of people talk about centralized DevOps, there's not a wave of scale, that was our key learning takeaway, so one thing is definitely the validation and hopefully from my talk I was also able to give some part, I mean though I didn't have a lot of time for feedback I'm hoping to interact with a lot of people post this thing, to get some feedback on what they thought about some of our evolution, to me that's, I mean it hasn't, it's not efficient take away, but I'm really looking forward to those discussions, so that's something, I think we are the main thing for today. Thank you so much for spending time with us. Thanks.


We'll call the meeting order for roll call. Coyunas. Here. We'll tree here. The other two are excused. They're working. I have the Pledge of Allegiance. Pledge of Allegiance to the fire of the United States of America and to the republic for which it stands, one nation under God, indivisible with liberty and justice for all. I would need a motion to approve the minutes of June 16th. So moved. We've got a motion and a second. Any discussion? I want to bring that up. Mr. President, the minutes is currently stated. Apparently, our listing candidates in alphabetical order. The commission had chosen candidates from their number one preference to the lowest scoring candidate. These minutes do not reflect that. So I would ask that the minutes be changed to reflect the hiring order recommendations of the commission. Yes. They listed them in alphabetical order and they were given in a numerical order of hiring. I believe they should be changed. That is the intent of my motion. So your motion should probably also include the actual order. Yes. OK. Mr. Chairman, I'd like to amend my motion to reflect the following order. Thank you, Jane. Stephen Shuldes was the number one candidate. Christopher Strozewski was the number two candidate. Joshua Becker was the number three candidate. Armand Meaghan was the fourth candidate. Kyle Casasola was the fifth candidate. Sean Trell Garner was the sixth candidate. And Jackson Haig was the seventh candidate. All right, we've got a motion. Do we have a second? Second. I have a motion and a second. Is there any discussion? Seeing none, all in favor signify Aye. Opposed, motion carried. Moves item three, proceeding for the board of Police and Fire Commission related to a complaint filed by Chad Shelton. OK, so as counsel for the Police and Fire Commission, I have provided for you a pre-hearing brief. And I'll walk through that because I know you probably haven't had time to go through it. I will note that I did provide a copy of the pre-hearing brief a couple of weeks ago to Mr. Shelton, as well as to the police department as sort of the parties thus far involves. And my suggestion would be after I give you my advice and you have any questions you may have for me, I would suggest that you do give Mr. Shelton and probably Chief Delmagalsky on behalf of the PD the opportunity to give any response that they want to give and give their opinions on the issues that I'm going to raise. In essence, my issues, what I want to point out, first of all, is my issues that I'm raising today are not at all to the substance of the complaint. I'm not saying anything about whether Mr. Shelton's complaint is true or false. That would be for you to determine at a later date if you decide that he has met the threshold to obtain a hearing in front of this commission. So I'm only going to be speaking to, did he meet the threshold requirements to get a hearing? So these are procedural issues only. And so while I'll be telling you, I'll just lay it right up front. I'm going to be giving you the advice that I do not believe that he met that threshold to get a hearing. And I'll explain why. But I do want you to understand that in doing so, I'm not making any judgment either on the veracity of Mr. Shelton's statements or the veracity of the findings in what I entitled the Zemple Report, which is the response to that Lieutenant Zemple did in response to Mr. Shelton's complaint. So just sort of to lay out some of the timeline. December 21, actually already of 2020, Mr. Shelton filed what purported to be a complaint with you, with police and fire commissioners. But he did not use the complaint form provided by the PFC, at least for the most part. What he did is he kind of attached his own narrative and then just put the word see attach in each section of the complaint form. And so the substance of his complaint is in that four-page narrative that he attached. And it describes a series of alleged events that begin in 1999. Shelton describes those as, quote, a negative history of double standards and bias against myself by the Sheboygan Police Department. And so as noted, I don't think he's met the standard for consideration and so the complaint should be dismissed. So here are some of the issues. First, in the section of the complaint form, which is attached and you have that as part of your packet, the complaint form is Appendix A. Oh, no, sorry, it's Appendix B. He lists the name of the person to be named in the complaint as Sheboygan Police Department and then describes the rank of this person as chief. The attached narrative then names a number of officers either directly or indirectly. Basically, it suggests that he's blaming multiple officers for this history that he alleges of double standards and bias. Some officers are named one or actually multiple times in the narrative. Others are just sort of referred to generically as officers from the SPD. So Appendix A is actually your rules of procedure that you adopted September 24th, 2015 and those rules govern disciplinary actions that are heard by the Police and Fire Commission. And those policies require that prior to the filing of a charge against a police officer by an aggrieved person, the person shall file a complaint with the Police Department under the citizen complaint procedure. So as I noted, when he first provided that complaint to the Police and Fire Commission, he had not actually provided it to the Police Department. Chair LaTrie had received that document. He consulted with me as counsel for the Police and Fire Commission and he then advised Mr. Shelton that he should then file that complaint with the Sheboygan Police Department and that if he's not satisfied as your procedures provide, then he would have the option of filing a new charge or if he wanted, he could reinstate the complaint that he'd previously filed. So he did go and file that complaint with the Police Department under that citizen complaint procedure and basically what he did is he just took that four-page complaint and he turned it into the Police Department. So it's the same thing that you've received as part of the complaint filed with you. Captain Zemple investigated the complaint and issued a six-page report dated May 13th, 2021 and throughout I've referred to that as quote the Zemple Report just so that you know what is being talked about and the Zemple Report is attached as Exhibit C. And the Zemple Report identifies 16 allegations or complaints made by Shelton in his complaint and identifies either directly or indirectly a large number of officers who in some way are alleged to be connected with the events that Shelton alleges. The Zemple Report though also either exonerates each officer involved in the allegations or at the very least fails to sustain each of the allegations. After receiving a copy of that Zemple Report and Shelton did talk to Chair LaTrie and Chair LaTrie talked to me again and got my advice and he reiterated with Mr. Shelton that again he could file a new set of charges in response to what he received in the Zemple Report that would more closely track the requirements of PFC policies or if he wanted to he could just simply turn in the document as he'd previously done. And that's what Shelton decided to do. He declined to create a new set of charges and just indicated he wanted his initial complaint to serve as the complaint that's now the subject of this matter. So that's kind of the timeline. And so now I'm kind of thinking about the procedure here. How do we proceed with this? And the first thing you look at is Section 14 of your policies, of your PFC complaint policies indicate that the first order of business at any scheduled hearing you can take certain actions and that would be things like hearing and deciding procedural motions, trying to determine if there are any issues that are not contested, trying to simplify the issues, conducting other procedural matters as maybe deemed necessary. In essence, it basically says the first thing you do is you deal with the procedural stuff before you get into the substance. And so that's what we're here to do today. And so what I'm recommending to you today is that you limit your scope initially to just determining whether or not Mr. Shelton has met that procedural threshold in your rules for allowing the complaint to continue. And so what I'm gonna do is just kind of lay out my analysis of why I think he's not met that threshold and then give you some advice on what further proceedings may take place depending on what you decide. So as noted, I don't think his complaint meets the procedural standards and here's why. First, he does fail to identify the person against whom charges are brought as required in Section 5C of your PFC policies. What he does is he just simply names the Shcherboygan Police Department as the respondent. Now, I think there is perhaps an argument to be made that when he says person to be named as the complaint, Shcherboygan Police Department and then as after that rank, Chief, you could take from that that maybe what he's trying to do is name Chief Damogalski, but he doesn't actually name him. And then as you look at the complaint, it really does list actions taken by officers, not by the Chief per se, but by a variety of officers and as noted, also by more generic reference to officers from the SPD. And that's why I think the Zemple report, I think Captain Zemple really did, I call it a valiant attempt to interpret the complaint. He really took very seriously what was said in there, tried to interpret what was in there and then identified 16 events that were complained of by Mr. Shelton and named them, named 16 different officers as being potentially involved. Wait for your turn. You'll have a chance to speak. So what the Zemple report does is it names 16 separate officers as potentially being involved in one or more of those events. None of those officers is actually named in the required complaint form. So we haven't named them as actually being respondents here to the complaint. If at some point we determine that those officers are meant to be respondents, we'll have to do some procedural things to get them the proper notice. So that's my first reason why I think the complaint needs to be dismissed. He hasn't met the requirement of Section 5C. Second requirement that I believe he's not met yet at this point is that he's failed to state the specific code of conduct, rules and regulations, city work rules, and or any state or federal law which the accused is charged with violating. That's a requirement of Section 5E of your policies. So if you read through his complaints, he never cites to any specific violation relative to the various allegations being made. He does, at the very end of his complaint, simply lists two Wisconsin statutes. He also lists the Fourth Amendment to the U.S. Constitution basically saying they're applicable statutes but never ties those provisions specifically to any particular activity that's been complained of in the complaint. He doesn't specifically refer to any particular actions as being violative of either statute or of his constitutional rights. I did provide you a copy of the language of those statutes and the language of the constitutional amendment as Appendix D. But based on his failure to do that, I think he's also failed to meet the requirements of 5E. And so based on his failure to meet the requirements of both 5C and 5E, it's my opinion that the complaint needs to be dismissed. Now, again, this is just as a preliminary issue. There may be other issues that may get raised at a later point if you decide to continue with this. There may be other issues that get raised either by Chief Damogalski on behalf of the PD or by Mr. Shelton. But these are sort of the initial issues that I see that lead to my advice and require you to deal with the issue at this stage of the process. Now, a few other sort of pieces of information that I think you need to have. As noted, he hasn't identified a person. And so the only parties to this process at this point are Mr. Shelton and the Sheboygan Police Department generally. And all the other persons named, I did provide them information, notice of this hearing as potential parties as a matter of transparency so that they're at least aware that there's the possibility that at some point they could become involved in this, but none of them yet have any rights that attach as a party. And so if you decide to move forward, decide to determine that certain people are meant to be parties to this, we will have to do that. You're not gonna be able to hold the substantive part of the hearing today. We'll talk about that if you decide not to take my advice and decide to continue, which you may certainly choose to do. If you decide that in the end you're going to dismiss the complaint, it's a pretty simple process. You just vote to do that and notify the complainant he's here, but we would also send a written notification of that. If you were to decide to proceed with hearing the substantive portion of the complaint, then you would also need to do some things. You would need to determine specifically which potential parties are the persons against whom charges are brought. You would need to determine what specific code of conduct, rules of regulation, city rules and federal or state laws. Each person is charged with violating and then you would also need to set forth a schedule for hearing the matters that takes into account all of the rights of all of the parties. I think that section 16 would probably apply where you'd still have two additional phases to the hearing as provided for under section 16. Now I do think one thing that you could do if you decided not to immediately dismiss, you could potentially also basically hold this matter in advance and instruct Mr. Shelton that what he needs to do is go back and cure the defects in his complaint and that could be done by drafting a new complaint for your consideration or amending the complaint to cure those defects. But you're not required to give him that opportunity. You could also just tell him, sorry, this one, it's done at this point. You can file another complaint if you want to do that. Given that, I'll sort of, I think you should probably open the floor probably first to Mr. Shelton and then second to Chief Darmigalski on behalf of the SPD basically to talk about and what I would suggest is that you limit each of them to talking about just those specific procedural issues that we've talked about and to stay away from sort of the substance, to stay away from the facts because you may end up having to decide but that'll be at a different time and we don't wanna sort of dive into those waters until it's time to do that and until the people whose rights it is to be involved in this have been informed of those rights. All right, Mr. Shelton. Thank you, commission. What some of you may not know but Chief Darmigalski and Mr. Adams are well aware of is that I have stage form metastatic colon cancer. I do not bring this up because I want or desire any sympathy whatsoever. I bring this up merely as a statement of fact that I have as little as six months to live. I have doctors who can verify that. I have paperwork that can verify that. As stated, this complaint was filed in December. It has already been more than I have been given to live that the police department has dragged this out. They received my initial complaint in February and did not issue the simple report until June. They are May, May 21st, I apologize. The chemotherapy affects my memory and so sometimes I get my dates confused but somewhere around May 21st is when the Zemple report was finally completed. They had 30 days. They dragged it out for three months. In my complaint, I did my best to identify the officers in incidents that have been over a decade since they've occurred. My memory is not what it once was and certainly anyone can be forgiven for forgetting the names of officers that they interacted with 10, 15, 20 years ago. I did my absolute best to identify the officers and I told Sergeant Walsh and Captain Zemple all of the relevant information that they needed to be able to find the correct officers and the incidences that I needed the reports for. Despite giving them as much information as I possibly could, they still falsely identified numerous officers and then exonerated them because of course those officers would be exonerated because they were not the officers involved and instead of investigating the officers involved, which they could have easily found because the records clerk was able to find it, they instead went with wrong information and returned a report of exoneration. Now as to Mr. Adams' advice, I would like to note a few things. First and foremost is the language with which he chooses to address my purported complaint. This is a complaint. I have filed it officially and gotten it notarized before a notary public after reading that I could be tried criminally for lying about it. This is not a purported complaint. This is not a piece of fiction. This is me giving my best attempt to explain the circumstances behind why I'm here before you today. I admit that I am not a professional clerk who knows every procedural rule but to the point there is in Mr. Adams' own pamphlet, the procedures which although do spell out some of what Mr. Adams is saying also clearly states that information regarding filing a complaint, if you choose not to use the enclosed form but wish to make the board aware of a complaint you have about the actions of a police officer or a firefighter, you may write a letter to the board detailing your concerns. It says right there that you don't even have to use the official form, that a letter will suffice. That's what I turned in back in December and yet Mr. Adams advised that that would not do, that I needed to go through many more months of delaying this process and in the meantime, I have been in the hospital three times, the latest for five days because I had an intestinal blockage. I don't expect everybody to be medical experts and know all of the things that can happen but what you need to know about colon cancer and intestinal blockage is that I was as close to dying during those five days as I have been since I got cancer. I was scared and knowing that this was still hanging over my head and not resolved, only added to my feelings of despair and hopelessness and absolutely as confirmed by my doctor had a physical effect on my recovery. I am not asking that the commission bend over backwards and allow me special treatment. I am asking that I be allowed to present the evidence of what has been occurring for two decades now. Thank you. Thank you, Mr. Schell. Chief Telekowski, you want time to respond? Yeah, the only comments I have is that for the process, whether it's an investigation by the police department or whether it's a hearing in front of your commission for due process to occur, specific allegations have to be made. No officer or myself can defend myself or the officer can't hire an attorney to defend them in front of the commission if they do not and are not given proper to notice about what the allegation is. And so that's what the process requires. I don't want to sit up here and really make this more of what it is, but Mr. Shelton really challenges me with some of this. And so if he wants to file a complaint with the police department and we're going to investigate that, Captain Zempel or Sergeant Walsh can only investigate the information that he gives. So for him to stand up here and say that they're skewing this because he didn't provide them the proper information is not on them. They don't know what his complaint is, only he does. And so he has to cooperate in that complaint process in order for that to happen. Second, he brought up timelines. The normal complaint process deals with 30 days. If you read our rules and policies, it talks about essentially the fact that not every situation is the same. And so things could take longer. So the normal complaint process would be done in 30 days because it would be one complaint. When somebody files 16 complaints over a 10 year plus time period, that's not gonna happen in 30 days. It's gonna require more time than that. So I think if you really look at the timeline of what we did, the investigators were very diligent in trying to do things as fast as possible. All right, good, Chief. So basically where that leaves you is you need to sort of decide how you're going to handle this initial question of whether Mr. Shelton's complaint meets the threshold to actually be a complaint pursuant to your rules. I've provided my opinion and the other Mr. Shelton has provided his information as has the Chief. I'm happy to answer any questions that you may have about my advice. But in the end, it's really your decision as to the next step to take. And then whatever you decide on that, I'll assist you in taking the next steps from there. Now what is it? Can you repeat the options to the commission? Sure. So your options are first, you could dismiss the complaint. That's my recommendation. And so that's one option that you could take. You could take no action yet. And basically explain to Mr. Shelton that his complaint doesn't meet the requirements but give him an opportunity to either amend or file a new complaint that meets the requirements. Third thing you could do is decide that it does meet the requirements. And then if it does, there's a series of things that you'll have to do in order to provide notice to the proper parties, you'll have to start with, determine who those parties are so that we can provide notice to them so that you can then move on to the substantive portion of that process and so that they can be provided counsel for the officers who are alleged to have violated these rules. I would just, I guess, ask a question. Is an allegation a narrative? What we have here is a narrative. Right. And is an allegation a narrative? Or can a narrative be an allegation? Yes, the fact that he chose to staple four pages to the back of your complaint form, that is not what makes the complaint an inappropriate complaint. But the problem when one does that is that it does, the form is designed to assist a person in meeting the requirements of your rules to name who the officers are and what are the allegations against them. So he has every right to do what he did, unfortunately in doing that, he in some ways may have hamstrung himself in actually providing a proper complaint. Mr. Shelton, are there any area you've acknowledged to us that you perhaps have medical conditions that impact your memory? I don't know if that short term or long term, but just in reference to what attorney Adams has said regarding the criteria that you need to meet, is it clear to you or do you have any questions or need further explanation on what you are required to do? Have to go to the mic. We can't hear you, can you go to the mic, please? Is there any way you could? Microphone be brought to him? Is it possible that the microphone could go to this table, I'm not sure. I don't think it moves. There's one there. I understand that my format may not be entirely correct. But I believe that I have met the requirements. I have named the officers and now that I have gotten the reports, I have all of the officers' names that we need in order to proceed, as well as the fact that I did not make a mistake in naming the Sheboygan Police Department, specifically Chief Dalmugalski as the primary respondents in this case. My complaint is against the entire Sheboygan Police Department for fostering an environment in which now me and my estranged wife will never call the police again. In fact, the last time I did, they refused to even take my report. So I believe you're saying that you understand everything, Brett? I do understand everything and if there needs to be clarification about which officers and which specific charges in the charges page apply to those officers, I can certainly clear that up. I thought it would be obvious based off of the statutes, which officers I was accusing of violating which statutes, as well as the Fourth Amendment, which was violated by officer prey. And they knew that it was officer prey and still decided to go with a different officer because I wasn't able to name him. And so therefore disregarded and exonerated that specific complaint despite them knowing at the time that it was officer prey who attempted to force open my front door with no warrant. That is a violation of the Fourth Amendment plain and simple. Thank you. I don't know. It's 20 years is more than Chief Domogalski was in charge of the police department as well. So that's incorrect. In terms of the police department is most guided by someone else at that time, the beginning of the complaints. It seems as if you want something from the police department and I don't know what it is. I don't know. I don't think, I read everything. I read everything two weeks ago. And Officer Zempel's report is based on records and that would be evidence that we would use. And I feel as if the report is using the sources that you refer to with the police department's records and videos. Mr. Adams. So one thing that may be worth Mr. Shelton indicates that really his complaint is against the entire police department as a whole. I don't know whether he means that in contrast to against specific officers. But you do have to consider your role in that regard. You do have a role in terms of the potential discipline of individual officers who violate the law or who violate policies. And so there's a sense in which if he names specific people with specific violations, you could proceed with that. You don't have authority to just sort of take a complaint against the police department as a whole because there's nothing you can do about that. Basically saying, police department equals bad. What do you do with that? There's nothing you can do with that. You don't have any authority to exercise discipline in that regard. Really Mr. Shelton's appeal in that case is really to the political system. It's basically to vote in people that he thinks will handle things differently and that he can do or appeal to the court of public opinion. But the only thing that he can appeal to you for is to appeal a decision of the police department regarding discipline of individual officers. And that's why you're here today. And so that, and again, it may be that he wishes to pursue that as well, but as far as it relates to his sort of statements about my complaints against the whole department, that was my perception in the first place and that's one of the reasons why my advice is that the complaint is not sufficient. Can we have a couple minutes to deliberate? You can deliberate, yes. Can we leave the chamber or do we have to deliberate? You can't do it outside of the chamber. You can't have a separate meeting. You can do it right here. I see. An open discussion or a private discussion? You can have an open discussion. There's no basis to go into closed session. That's what I thought. Mr. Shelton, my name is Larry Samet. I'm one of the commissioners. I have listened and read all of your material that you presented to us. There are policies, criteria, ordinances, laws, guidelines that all of us in society have to follow, including commissioners. We are being advised by our council to dismiss this and I understand his reasoning and he feels and is recommending to us that you have not met criteria at this point. You may not like that recommendation, but that is recommendation and he is our attorney and we rely on his expertise to advise us accordingly. That said, speaking for myself as one commissioner, it's in your court, in my eyes. You can either say I've done everything and leave the door closed or I am prepared at this time not to take any action and give you another opportunity to meet the criteria as laid out as attorney Adams has advised. If you choose not to do this, in my opinion, I feel I can do nothing further at this point, but I, speaking for myself, I'm prepared as of today to take no action, okay? You can use the mic right there. First of all, thank you, commissioner. I appreciate your candid honesty and I would just like to state that I don't know your name but you had stated that you feel as though I want something from the police department. I do. I want justice. I want the police department to follow the law and to apply the law equally and fairly for the rest of the time that I have here on this earth. That is what I have wanted for a long time now and that's all I want. Mr. Dolmogalski may not have been the chief of the police department, but to say that if there's a corrupt police department for 20 years that there's just nothing to be done, that is not the America that I know and I believe wholeheartedly that there is a wave of change sweeping this nation and that police departments are being held to the standard that they should have been held to from the beginning. That means that they cannot rip an infant child from a parent's arms without a court order to do so. It is a civil matter. That means that they cannot force open mine or any of your front doors without a warrant. That's in the constitution.  That is what I want and I am willing to do whatever it takes to get it. If that means that I have to fill out paperwork more correctly in order to move forward then yes, commissioner, I will do whatever it takes in order to satisfy whatever procedural requirements are laid out before me. And so I thank you for being honest with me about what would be needed in order to proceed and I am 100% willing to accept that and do that. Thank you. I'm only one of three commissioners though. The majority rules here, okay, Jean. I would accept your offer to revise your complaint in terms of the guidelines of the commission. And I would hope that you would seek the help of the attorney if you need it in terms of doing the thing correctly so that you don't do extra effort on this especially in your condition. So I would concur that or I would say that we accept his offer to resubmit the complaint. Complaints I should say. As long as you understand what they're saying is that the second option was to take no action and they give you the opportunity to amend or refile. You understand that, right? Use the microphone, just it's loud enough. Yes, sir. Okay, thank you. Do we need to put that in motion? Yeah, you should put it in a motion. I would suggest that you make it a motion and give him a whatever period of time you feel is appropriate to accomplish that. Okay, I move that we take no action at this time. Well, he wants us to give him time. What are you suggesting, Mr. Sheldon? 10 days would be fine. 10 days? Yes, sir. Give him two weeks, take a motion and give him two weeks. Okay. Yeah, all right. Do you want me to make it or do you want me to? You can make it. It's up to you, see either one of us. Mr. Sheldon, at this time, the red is on. I'm on now, okay. Mr. Sheldon, at this time, I will put forth the motion that we will not take action and give you two weeks from today to file a complaint with the following the appropriate procedures. Is that accurate? That sounds good. Second. All right, we got a motion and a second. Is there any further discussion? All in favor signify with aye. Aye. Opposed? Opposed and carried. So Mr. Sheldon, you know what you need to do. Thank you very much. I'll motion to submit to dismiss, I mean adjourn. We've got the motion. I move that we adjourn this meeting. Second. We have a motion and a second to adjourn. Any discussion? All in favor signify with aye. Aye. Opposed? Motion carried or adjourned.