OK. OK, now it's better. OK, so sorry for those of you who, I think, in the master in Paris have done this exercise already. But OK, if you're bored, you can ask a lot of questions. No, no, no, no, shall we give it to them? Oh, I took one because they're numbered. No, maybe I took it. Here you are. Is this number four? Yeah. OK, OK. So our only handles up to now are the frequencies of the oscillators, which I am free to choose. There will be many. So this is something that I cannot choose because we want a bath, which is, by definition, something that it's infinite, potentially. And so we have the frequencies and the strength of the interactions. I am free to choose them. And the result does and does not depend, you will see. But this is, for the moment, the freedom I have. Then I will take another freedom that you will see. OK? So now it's very easy what we have to do. We have to solve the bath so that we can get rid of it. So let's get rid of the bath. So OK, the equations for the system are Newton's equations. I can even skip. Sorry, sorry, don't copy anything. This time I'm, oh, yeah, yeah. Let's skip one step for brevity. So acceleration equals force. And then we have the equations for the bath. OK, so the trick is the following. This I chose one degree of freedom because, as I told you, conceptually it's the same to do one or more.  But it's not a bath, it's a system. And you, I cannot solve this equation for sure because the potential could be anything. So this is not something I can solve analytically. The trick is that the equation for the bath, you see it's linear, except that it has this thing which plays the role of a field. So you can think of this as a harmonic oscillator. They're all decoupled. And this is a term that depends on time. It comes from the other equation, but forget it for a second. And this plays a role of a field that is entering this equation. So the equation for each oscillator of the bath is simply the equation of a forced oscillator. And this we know how to solve. And the solution is this thing I'm going to write, which is really ugly, but it's an oscillator that has a past in which it has been forced. This is solvable with the initial condition of which, more in a second, the moment at time zero. And this term that comes from precisely, this is just the generic solution which you can look in Wikipedia, of a harmonic oscillator that is being forced. And this is the forcing term. Of course, it is forced by my system. And OK. And now the strategy is very simple. We have this guy now. So now we have solved for the bath. We can put it in here. The equation we didn't know how to solve. And we will end up with a gigantic monstrous equation for q, which could have been. And this is going to be our effective equation, having gotten rid of the bath. OK. So if now I get this thing and I plug it in here, we are done. And the equation we get is the following. Oh, I'm sorry. I should have put a mass here. Forgot it. Sorry. Initial equation. Then there is a term, which I have to tell you what it is. And then there is another term, which I also have to tell you what it is. This is a memory of what's going on. And we have another term. The other definition that we're missing. OK, so now copy this and then we will stop and think for a second. And what I did is completely elementary. You can follow those steps. I'm going to give you a reference. Sorry, what's wrong? Is it q dot inside the integral there, or q? Because upstairs is? It's q dot. There has been an integration by parts. It's just for convenience. We could have integrated the other way around, and this would be. But it's nicer to write it this way. So yes, this is a good question. Here there would be q, because it is q somewhere. But because it is in an integral, we integrate ones by part. Because we prefer the derivative here for reasons that will be clear in a second. No. No, this is the one that belongs to the other sum. So I'm going to make it a bit shorter. So what is qA of 0? And the parenthesis there, maybe it's just q of 0. Thank you. Mahesh, you have to keep up to his standards when he goes away now. If there are doubts, please ask me. We cannot let Matteo do all the asking. He's already graduated. OK, so now let us think five minutes. This is, I think, a lovely exercise. So let's go back in time. You remember Brownian motion? So Mr. Brown discovered that particles of pollen sort of jitter in the water. And then at the beginning, they thought that perhaps they were alive. Then they decided that no. And then there was this jittering motion. And then Einstein understood what was going on.  And it's the following.  And you expect it to have an energy that is compatible, a kinetic energy, that is compatible with the temperature at which the water is. Because at his time, atoms were not so consolidated, as you would think. Even at the beginning of the 20th century, the idea of atoms was still. But thermodynamics, they did know. And they expected this to be in equilibrium with a bath. So have the energy compatible because it's been sitting there for hours. OK, so this is doing a jittery motion. And of course, Einstein already, he believed in atoms and importance of atoms. And so he understood immediately, I don't know if he was the first, that the jittery motion comes from, you are being bombarded. You pollen are being bombarded all the time by molecules of the water, which are kicking you in directions. Because the molecules have a finite size, they are not infinitely small, you are pushed by a discrete number of quantized objects that have a given mass. So even though on average, you are being kicked in all directions, because the molecules are finite, notice that this is already telling you something about the size of the molecules.  And this is why it's visible. And this already tells you this parenthetic remark that the size of the pollen is not so enormously larger than the size of a molecule. Yes, it is 10 to the something. But it is not 10 to the 24, the size of a single molecule. And then you have two things. But because you are being kicked in this way, why don't you simply gain energy? No, because if I kick something all the time, it should heat up, get more energy on average. And the reason why, and this is what Einstein understood, this is the heart of the matter, is that the water does not only give you the kicks, but it also gives you some friction, which stops you. And now the point that he understood, and this is crucial, is that you see that friction and noise, noise is this kicking, have to be in an appropriate balance so that you have the right energy corresponding to the temperature you have. So miracle, miracle, he discovered, or he realized, that friction and noise are not independent. They are tied to you by the necessity of being in thermal equilibrium. This is an extremely deep insight. I mean, it's one of the big things he did. In our modest little measure, we are discovering a bit the same thing. This, which I will call noise for the moment, it is not random. But it is an additive term that is shaking you. And, well, it is a function, of course, of the initial condition of the oscillators, which I will have to this one. On the other hand, this term, you have to interpret it as a term of friction. What is friction? Friction is something that is proportional to the velocity. When you want to move fast, 0. So this is the initial condition of the, again, sorry. And also the other one? Yes, this is very strange. The nodes have a mistake. It's right, you're right, sorry. This is the initial conditions of that. You're still losing towards Matteo, sorry. It's amazing the printed nodes were wrong. OK, yes, of course. Thank you. So why do I call this friction? Friction is, in general, something that depends on velocity. It doesn't want you to move. So if I'm standing still, I don't feel friction. I mean, I don't feel it as a force. And then when I try to move, the friction stops me. So this is a term with friction. But it's a bizarre friction because it's a friction that has memory of the past. So it's a friction with memory. It's a friction that doesn't depend only on the velocity you have now, but it depends on the velocity you had in the past. This is generic about friction. Now we will say a lot about what this function gamma looks like, so don't worry. But OK, so this is noise, and this is memory. Sorry, this is a friction term. So we have the two elements here. One is like, in this case, it's not molecules. It's harmonic oscillators that are bothering you. And the friction is stopping you. And this balance between bothering you and stopping you has to exist and give you something to do with, we will see, the temperature of the bath because this is what we want. But we are not there yet. So now comes the up to now everything is still absolutely Hamilton-Newton's equation. If you give me the initial condition of the oscillator and the initial condition of my particle in the potential, I can follow the thing, no problem. And it's completely deterministic. And it's only that I partially solved it, but that's only the only thing I did. So the memory that you speak of, in the example of Brownian motion, that would be of very short times, case of order of 10 to the minus 13 or minus 14. Absolutely. The memory term is the memory that, of course, if a molecule is stopping me now, so I am touching a molecule, and then comes another molecule which touches me in another direction, there is some interval between them because they occupy space. This is how he estimated the time to replace one molecule by the other. In the case of a liquid, it takes you some time. And this is a characteristic time of your bath. We will have here a complete freedom to choose the time scales you will see. So the principle I am telling you of balance between noise and friction is called the fluctuation dissipation theorem. And we will say a lot about this because it's very important. So now we have to say what the bath is like. Remember that we already chose arbitrarily this and this. And we will play with the choices. This was up to now our choice. But I didn't tell you how hot the bath was. I have to tell you what is the temperature of the bath. So I have to give you the energy of the bath, for example. Yes, OK. So now we make a crucial step is that we are going to assume, and Matteo yesterday wasn't happy with it, but even so we will go on, that we are going to choose that the pi a's and the x a's are in equilibrium, microcanonical equilibrium at temperature T. What Matteo didn't like is that ideally I should do microcanonical because I am now including the canonical information which I ask you to believe. But OK, it's really only simplifies life a bit. So I want to ask them to be some energy compatible with equilibrium at temperature T.  This is very important. But then I'm going to do a bit more. And here comes the enormous step, which is very easy, but it is very big also, which is to say, yes, I'm going to ask this, but I'm going to ask them to be taken at random. So they will be taken at random with probability e to the minus beta h of bath divided by the normalization. But randomly, the energy. So I will throw a problem with a lot of these variables now are random. Random and with the typical energy in equilibrium, it's very easy because it's a harmonic oscillator. So it's just the equipartition of a harmonic oscillator. So what is the energy of a harmonic oscillator at temperature T? A half kT for the potential energy, a half kT for the kinetic and so on. So we are going to replace the initial values, but we are going to consider them a random variable. So this is the handle number 2, 3 we have of the problem. Handle number 1 was what the oscillator's frequencies are. This will survive to the end of our problem. Handle number 2 is the interactions between the oscillators of my system, handle number 2. And now the temperature of the initial conditions of the bath. The bath is so big that it doesn't care very much about you. So these we are going to choose so that on average, they have the right energy. And what is the consequence of this? That we are choosing a temperature. This is where temperature enters in our problem, in saying what is the distribution of the oscillators at the beginning of time. There is another point that is very important that now we have at this moment introduced randomness in the initial condition of the oscillators. The future life of their oscillators has entered in the initial condition and the initial condition is random. So what was a deterministic problem now is becoming a stochastic problem. And this has dramatic consequences because it seems like an innocent step and you will see that it takes one second to do it. But it has completely dramatic consequences because, for example, remember I told you that the volume of a drop of ink, let's say, initial conditions in phase space, preserves its volume in Hamilton's equation. Now the equations we will obtain for the system in contact with this bath do not do this and tend to equilibrium and everything is nice and easy. So we are going through all this detour and at the end of the day, at the end of today, you will see that we will end up in a stochastic equation which, in spite of appearances, is infinitely easier than doing Hamilton. And there is an entire branch of physics that is enormous that is dedicated to this and then they call it stochastic thermodynamics. You will see that although mathematically you have to deal with a random noise, in practice, all issues concerning ergodicity become trivial. Do you have something? Yeah, so no, actually, from what I see, so the dynamics is still deterministic here, in the sense that, at this stage, at least, you are just saying the initial conditions are random. But after that, it's like you are saying that I have my system and butter in a bigger butter up to time t. No, no, no, no, no. And then after that I take out and there is isolated and they go on deterministically. OK, yes. But not the system and bath. I have only the bath. I prepare my bath with a good temperature. God knows how, but I couldn't do it micro-canonically. We will be the same. But the calculation here will be a bit nastier. And then it's deterministic. Only that, as you will see, we will study trajectories on average with respect to the initial conditions. And that's what stochasticity enters. OK? It's an ensemble of possible things. Yes, yes, yes. But this is where irreversibility comes in. OK, so now, we have to compute this. And with this, we really and finally get rid of the bath. Get rid of the oscillators. This is a function. It's OK. I don't have much to say. Now let me say something about the noise. You see that it has signs and cosines. So on average, you believe me, it's zero. So it's something that is zero. And now I'm telling you that it's random via these numbers. OK? So now, what is it that I have to ask? Well, I have to ask for the correlation.  So now, because this is going to be a sum of many things, what happens with a sum of many things, it's going to be a Gaussian, but then correlate it in time. So now, we have to take this expression, multiply two of these, yes, of t and of t prime. And then remember that the average here is an average over initial condition. This is where randomness came in. OK? So I think I'm tempted to leave this as an exercise for you. You take this thing at t. You take this thing at t prime. And then the only thing you have to remember is that every time that pA squared appears, you are considering that this is the only random thing, pA of 0, sorry. And every time xA of 0 appears, this is the average over initial conditions. And when pA times of 0 times xA of 0 appears, this is 0. This is a harmonic oscillator. It's this way. And in a harmonic oscillator in equilibrium, this is a half kT. So you make the product in the product terms that have pA squared or have xA squared have to be replaced by the temperature, taking into account the temperature with the Boltzmann constant, of course. But as I told you, it's 1. If there is a term like this, pA, pB, or pA, sorry, xA, xB, et cetera, all these are 0 on average because the oscillators are completely independent, these are averages, remember, over initial conditions, and so on and so forth. Is it clear? I just don't want to fill the blackboard, but I just have to write this at T, write at T prime, and then collect all the terms that are pairs and replace the expectation values by the initial thing. And the consequence of this is that what I get, miracle, miracle, and it's not a very hard thing to do. This is not factorial, it's just an admiration sign. And of course, the average of eta is 0 at any time. And now we have finally gotten rid completely of the bath, and now we are left with, OK, so we manage. Yes, please. So the temperature is fixed, so the condition is isothermal, so can we make it to other variables like changing T to some pressure or volume? Yes, yes, yes, yes, yes. Yes, so, yes, for example, so what he is asking is can I have a bath that instead of keeping temperature, keeps pressure, OK? So one way to do it and in simulations, you take your system and then you can put a piston that is in contact with the harmonic oscillator and is pushing your system. So it fluctuates, so it's not the energy that fluctuates, it is the depth of the piston, and it has a, you can do it in the computer. In fact, it could be a nice exercise for one of the groups we will make. And this keeps, you can do it at constant energy, so it's conserved, but it's interacting with the piston, and you might make it in such a way that both pressure and temperature are maintained. You can do it with any variable. In practice, one way to do it, which is super nice, is imagine you have a gas of spheres. Let's say hard spheres, no, boing, boing, boing, boing. And you want to fix the pressure. So what you can do is you can make the radius of the sphere a dynamic variable, which is a bit like using a piston because when they inflate, it's as if they were more dense. If you do that and you make a kind of spring that doesn't want the radius to change, you can fix the pressure in your Monte Carlo procedure. So yes, you can do whatever. You can fix magnetization with a little bit of imagination. You can do whatever. Thank you. The idea of being always the same. OK, so here appears the thing that is so important. Remember what the thing I said about my pollen particle and the realization that the intensity of the kicks and the intensity of the friction have to be matched. And the matching, the dictionary matching, is precisely the temperature.  So when you, as a pollen particle, are swimming in this water, you receive a lot of energy from the bath and relatively are taken away by friction and more modest quantity, and you are hotter yourself, which is good because this guarantees that you have the right energy in equilibrium with the bath. So fluctuation dissipation, which is the fluctuation dissipation, this one is called of the first kind. When we do the one of the second kind, you will see that it's of the second kind. Flactuation dissipation is precisely the statement that the properties of the noise, the variance, and the properties of the friction, the memory kernel of the friction, are exactly equal except for a temperature. They are not similar. They are exactly equal. You underlined the same function. Properties of the noise, gamma. What did I underline? Here. Yes. Yes. So OK. So now, forget the oscillators forever, almost forever. But if for some reason, the gamma and the gamma are not the same function, you could imagine that I write this equation and I choose a random Gaussian noise with a gamma and a different gamma here. I can do it mathematically. Then it does not correspond to an equilibrium bath. And you are in trouble. You're in trouble. What does it mean? No, you have a dynamics that is legitimate. But don't expect the system to reach any micro canonical or canonical or anything. You don't know what it will reach. So if the gamma here and the gamma here do not match. They are not proportional. Then you have a system that is inherently an unequilibrium system. Inherently means that if I started, even an equilibrium system, when I started, I put water here in this room, which is at, I don't know, 30 degrees. What was this room? 25 degrees. You put it a glass of water, which is at 5 degrees. It will take some time to equilibrate, after which it will equilibrate. A system that is in contact with a decent bath with fluctuation dissipation, meaning that the gamma here and the gamma here match, will do that. It will take some time, but it will equilibrate. It will become an equilibrium system. And from there onwards, I can apply canonical measure for everything. If instead this gamma and that gamma do not match, they will reach a stationary measure because you have the mechanism of pumping heat in and pumping heat out by noise and friction. But the measure it will reach is not the canonical measure because the bath in itself was not an equilibrium bath. So what measure does it reach? You don't know. You have to solve the entire system. There is no quick rule to discover that. Once the bath is a bad bath, then there is no general rule for anything. You have to solve the dynamics and see what you get. You cannot get rid of time, for example. So strictly speaking, essentially, so this noise here remains a deterministic function, the eta. If somebody gives you the initial positions of yours. The argument is that essentially you compute the average of eta, eta, eta of the d prime. And then you can say, OK, so this given the initial condition of Gaussian, this is a linear combination. So the noise will be Gaussian throughout. Exactly. And then you say, I can substitute this eta with, say, white noise with the same correlation. We will come to the white noise in a second. OK. So this is as far as we get. I'm so sorry. You should also have that this gamma is finite, right? So this imposes when, I mean, so you should have some limit when the number of oscillators goes to infinity to have this type of molecular chaos type of. Sure, your noise is a sum of oscillations. If you want the noise to look like a noise, you need many oscillators. We have many oscillators.  OK, now we're coming to this now. All right, so now that we are here, OK, we're here. And OK, so now let me cancel this and say, OK, there's only one function gamma. And now you have to choose it. Yes? When the body is not good, when you say that you cannot assume the ensemble of the system anymore, it means that you cannot put the canonical or micro canonical here. And how will you introduce? At the beginning, yes. How will you introduce the randomness? You could, for example, because you decided that, for example, the initial condition is non-equilibrium one. For example, the initial condition, you can have half of the bath at temperature T1 and half of the bath at temperature T2. And with different omegas, we will play with this. So the high frequencies of this thing are hotter than the low frequencies. We will do this. And then your system doesn't know what temperature because there is no single temperature. We will do this, because as you will see when you do models of active matter, which is very important subject now, precisely it's as if you have a bath that is not at a temperature, but it's giving you energy, pumping in and then pumping out, but not with this precise, delicate equilibrium, which is called detail balance, by the way. Or while depending on the context, we will come back to it. But fluctuation, dissipation, theorem. And you will see that it bleeds you to detail balance. So you will see how this comes about, yeah? I know we cannot know that as a priori. We have to just try what works. There is no once your bath is not a decent equilibrium bath, you're on your own. You have to solve the dynamics. And I don't know of any problem that is not purely harmonic. Because when it's purely harmonic, then you can do everything. But a problem that is nonlinear and at the same time has a bath that is not a good bath, let's say. There is no like a Boltzmann weight, a bit modified, that helps your life. There's nothing like that. There is no generic rule to calculate what the measure will be. You will get a measure that will replace Boltzmann measure, but there is no nothing. You have to solve your entire problem. When it happens that you can, it's because it can be mapped. It's a hidden equilibrium one. Because you can remap it into equilibrium. But if not, there is nothing. So for example, when you have active matter and you ask what is the probability distribution of velocities, these are motorized particles. You know that, have some little motor that moves them and they lose energy when they crash. So you ask, well, what is the distribution of velocity? Is it the Maxwell distribution? You don't know. Simply you don't know. The only thing you have to do is do the simulation, take the histogram and record it. There is nothing that can be done. OK. Yes? What's the distribution function you are taking here to calculate this averages? Sorry? What's the probability distribution? The probability here is the one that comes from having, so the averages here are the averages over the initial velocities. No, I'm asking what's the kind of distribution you are taking, Gaussian or probability? Oh, the distribution of a task? Yeah. What is it? Yeah. So it's a distribution when you have many, many oscillators. It becomes a Gaussian with correlated in time in this way. With this one, it's a Gaussian, so with that, you completely have specified it. OK. So now we can choose whatever we want from gamma.  Remember that these were the quantities I now forgot. Now all this in blue, what I told you was one of my handles. And handle number one, handle number two, and handle number three was the temperature. OK. So these I can choose as I want. And so these cosines I can choose as I want. But then you see that this is the Fourier transform, more or less, of this. So I can choose the Fourier transform the way I like. So basically, basically, I can choose this and I have a lot of freedom to choose it. But then this gamma, I don't have any freedom anymore if I want my system to correspond to an equilibrium path. But whatever the gamma I choose, with very few conditions of positivity, because you see this is a correlation. So the Fourier transform has to be positive definite. OK. But this thing, I am very free. So because I am very free, let me erase it and tell you what this means. So OK, this one is obvious, so I erase it. So this is what is called a generalized Langevin equation. I think that Langevin himself wrote this one. Usually this is called generalized, because now you're going to see the one that is not generalized. So just to be clear, this is gamma. And this memory kernel affects your past and multiplies your velocity in the immediate past. And you are free to choose, provided, of course, the fluctuation dissipation relation holds. And so now what people do is take the easiest of the easiest thing, which is the following. It's if we take for gamma a very peaked function, a delta. So this would be a delta, delta of t minus t prime, to be proper. And here it's a half of the delta, because you see I'm integrating up to t, so it's only the half of the delta that I'm choosing. So in this case, you end up with this lovely equation. Why lovely? Because now the memory kernel doesn't go to the past and still respects this important thing. So now this equation in that case becomes what is more often called the Langevin equation. Q dot, gamma q dot, because this is a delta, it evaluates, or half a delta in reality, it evaluates q dot just in the time in the immediate past, so proportional to some gamma. Eventually, I didn't use it before, but we could ask, we could add a force that doesn't derive from a potential if you want. The eta i's now are independent, and I am now putting it with many variables so that it's nicer. So when gamma is very peaked in the recent past, you get this equation where the noise is now white. It's only correlated at very short times. And delta ij means that different noises corresponding to different variables are completely independent. So every variable of the q's, remember that here I have only one q, here I made many, but the proof is the same. Every degree of freedom q has a private bath of its own, which is completely independent, has a different independent set of oscillators that talk to him. So this is a bit rare in the literature. You find it, but you find it a bit less. I'm sorry, there's something I don't like here. My god. There has to be a gamma here. If not, fluctuation dissipation is not satisfied. Yes, there has to be a gamma here. Because that relation between the intensity of the noise and the friction has to be respected. Because if not, it's a bad bath. So you see that, again, this t here is this t here. So now you recognize the friction of a true friction. And the gamma that appears here has to appear here, because if the friction is stronger, and I want to keep the temperature fixed, I need to make the noise stronger. So if you're, yeah. Ah, OK. You want something whose Fourier, so you have to, I erased it, but if you go to the expression I did for gamma, you see that it's a sum of cosines with coefficients. You want that to be, to give you a delta. So take Fourier transform right and left. The Fourier transform of the delta is a constant. And then the Fourier transform of the right side is the sum of cosines that you had, because it's a Fourier transform of cosines. The sum of spikes at the values, so you see my point of the cosines. And then check that your density of oscillators gives you this constant. I'm not telling you it's flat, because there is a 1 over omega squared that you have to take into account and so on. But yes, yes. There is a condition. And then there is another condition that is very famous and gives you a noise that is not white, which is called the Debye condition, which corresponds to, you are in a solid, and the oscillators are in fact the sound modes within this sample. And these are distributed in a way that is a Debye way, and this is the one you use for solid state. So there are several famous, the omic, that one is the one that is called omic, low bomb, and so on. So there are some famous things. For the white one, you can. In order to make this generalization to many particles, does we have to assume that none of them has interaction with each other? They have interaction with each other, but the baths are independent. The oscillators of each other, Q1 talks to some oscillators, Q2 talks to other oscillators. But Q1 and Q2 can talk with each other, yes. If not, this wouldn't be very useful. And if not, they wouldn't be in equilibrium. Well, in the end, they wouldn't, yeah. OK, so this is the second, as I said, most well-known Langevin equation. So gamma, if you are in air, gamma is very small. Air at 20 degrees, gamma is small, so you have little friction, but you also have little noise. If you are in water, the gamma is going to be much larger. You have much more friction, but the noise you will get will also be much larger. And if you are in honey at 20 degrees, this is going to be very big. And this is going to be the same thing, very big. And the proportionality is always t, so there are three things that are at the same temperature. So the viscosity of your system where you're moving is decided by this gamma, at least in comparison with the inertia you have. This is the inertia of your problem, the mass. So the idea is that if gamma is large and M is small, maybe you can forget the inertia. Think of, for example, a Brownian particle in water. The inertia of a particle of pollen is nothing, or the inertia of a bacterium in water is nothing. So we have the temptation of throwing this away sometimes. Now, this brings a lot of headaches, but the Langevin equation most of you will find in life is without this term. So this is general. This is medium general with inertia. And the one you find most often is this one without this term. And when you don't have this term, then you can absorb gamma into time, absorb it here in the derivative, and absorb it here in the delta. And you get a Langevin equation that is only like this. Eventually, there could be some forcing that, of course, will drive the system out of equilibrium. And this equation is called Langevin, or sometimes Moluchov. I think the correct name should be Smoluchovsky. So regarding this neglecting the mass, I mean, if I have a forcing term, there can be trouble if I neglect the mass or? Well, if your forces sometimes are very impulsive and you want to see how inertia reacts, yes, you have tried. So here we've got three levels. And so, sorry, and this has to be supplemented with gamma now went away, the same with a two, but without the gamma. So gamma has been eliminated by absorbing it in time. And there is a temperature after so much saying that it was important. OK. So, and what we will do is we will talk about an ensemble of trajectories. OK, finished copying. Does that mean in overdamp Langevin equation there is no memory? There is no memory in the sense that this one is completely Markovian, yes. Markovian means that it depends on the immediate past, this one. This one is a bit Markovian, but it has an acceleration term, which a bit remembers if you want the past because you cannot do immediate changes. And this one is the generic one where you have all the memory. Thank you. So we've gone a long way. And now we have a situation where we have this equation of motion, which would be, sorry, not this one, this equation of motion, which would be Newton or Hamilton. Then we have a random noise. And you have a memory that is deterministic, but it's very much related to the noise by way of its variance. Here we lost this one, so it's not Newton anymore. So we only have the motion due to the potential. Eventually a force that doesn't derive from a potential, which, of course, will make the system an unequilibrium system. I put it here also, and then the noise. And why is the noise random? Well, if you go back and back and back and back, you will see that it's random because the initial conditions were random. And those are of the bath. Those are the ones that decide the randomness. And when we will study the dynamics of this and of this, we will find that we are not going to do a trajectory, or sometimes we will do a trajectory. But a trajectory is for given realization of noise. But many times, we will be interested in the average over all the possible noises that are chosen with the appropriate rule. They are Gaussian noises. And we do the average over trajectory. Even in your computer, when you want to simulate a problem, you run it once with random numbers that are the ietas that are chosen intelligently. And then you make a dynamics, and you record the values your particle took at given times. And then you run it again with another seed for the random numbers. And then again, and then again, and then again. And you make a histogram. So at each time, the location of a particle is a distributed quantity. You consider the ensemble of trajectories given the ensemble of noises. And all this comes back to, if you want to go backwards, in the original problem, to the ensemble of initial conditions of the bath, which is where the randomness was born. OK? Good. I think we are almost done for today. The only thing that I want to say now is that what we will do next time is that if I have an equation like this one, or like, oops, not this one, this one, I have the possibility, for example, of studying a trajectory. This is q with a given eta. Or I can, with another eta, have another trajectory, which will be slightly different. OK? And then if I consider all the possible eta's, I will have a probability distribution for q. It could be a vector. OK? This is one description. One description is to solve the problem for a given realization of eta, even in your computer. And another possibility is to study at each time what the probability cloud of this distribution, how it is evolving. OK? So if I start here in this is a free motion, so let's say this is 0, this is 0, and I only have eta, one particle will do like this, another one will do like this, and so on and so forth. And I will have, if I consider my envelope, all these. And at a longer time, of course, I will have this. And at a shorter time, I will have this. So I can do two things. I can study one trajectory, and there's a lot of developments that you do for a single trajectory. And you can instead study how this potato of probability is expanding. So for studying this thing, you have this. But you want to ask yourself, can I have an equation for p of q directly without doing individual trajectory? And the answer is yes. And this is the Smoluchowski equation. Fokker Planck, Smoluchowski, sorry. Maybe Smoluchowski doesn't apply here. Let's leave it for Langevin. But we will find that there is an equation for p of q and t. There is an equation that tells me, given that my initial things evolve like this one or this one, there are two different equations. How do these probabilities in space evolve? And this is going to be what we're going to do tomorrow. And with this, we complete, if you want, the passage from a world where everything was deterministic to a world where we have introduced randomness. So all this subject of adding randomness is very old. But in the last 25 years, there has been an enormous revival of activity here under the name. It is thermodynamics because you have a big bath at the end of the story. So even if you have one degree of freedom, remember there is a bath involved. And there have been incredibly, for something that is so elementary and so old, a lot of developments in the last 25 years on this enormously many. We will try to see a few of these depending time permitting. But there's a lot, a lot of things one can say. I think that I will stop here. So can we derive entropy of the system by calculating this PQ, this probability of Q? Yes. Yes. Yes. We will do the exercise. So we will do P of Qt log of P of Qt, which is the sensible thing to do. And we will minus. And then this quantity, which we will call entropy by analogy with the usual entropy, this is a kind of dynamical entropy. We will see that this quantity minus the energy to get a free energy is always going down with time. Going up, sorry, the entropy is going up with time. So is this branch interesting because we are getting thermodynamics from single particle dynamics? Yes.  So there is a coarse-graining involved. And I told you that always when you have second principle, there is a coarse-graining. There is a coarse-graining involved in the fact that you're not considering one trajectory, but many, meaning that you are not looking at the details of the thermal bath. You're taking an average. So yes, you recover thermodynamics. Thanks to that. Thank you. One thing that I will insist is that unless I am very much mistaken in the books of thermodynamics, sometimes it is sort of implied that the Boltzmann equation in the old books somehow proves irreversibility. And I remember the books with which I studied and Huang. And the answer is no. I mean, it doesn't. What makes irreversibility for you in a Hamiltonian system is coarse-graining. It so happens that Boltzmann's equation has coarse-graining implicit. This procedure also has coarse-graining where in the fact that I don't give you the details of the bath, I give you only an average. So it is coarse-graining that gives you irreversibility, not a miracle of Boltzmann equation. That's historically people to be faithful to Boltzmann. They over, I think, represent the value of that equation. Maybe they were over-correcting for all the attacks he received. Yes, and that poor man ended his days a few, an hour walk from here. OK. Sorry, other questions? OK, so let's thank. And so I think now there is a coffee break. And Matteo has just asked me to tell you to not take the other coffee break from, I mean, because I understood. I mean, do not act like grasshoppers, you know. Thank you. See you in half an hour. We will meet in the info lab.


So impact is a huge word. Some of you may be familiar with it. Some of you may be less familiar with it. And the idea is simply to ask yourself, what am I changing when I'm doing this? Whatever action you will do in public engagement, it will probably have an impact. Maybe a huge one, maybe a tiny one. It may be only on one person, or it might be on 2 million per people, depending on your action. It may be a deep one, meaning that the person will completely change their whole life, or it can be a very shallow one. Just the people will be aware of that one new word, and that's also a kind of impact. So the question we ask ourselves when we think about impact is, what am I changing in the world, in people, in society, in research right now? And for us to explore a little bit this idea of impact, I will ask you to do one more activity on the mural when we speak about evaluation. So, first of all, what is evaluation? When we talk about evaluation in public engagement, we can refer to two things that are quite different in the goal that we have, and sometime in the methods that we use as well. The first one is formative evaluation. Formative evaluation is typically what you do to improve your action. So you do it very often, even before you start your activity. So it's not something you do afterwards to reflect on it, but before to help you build it, or during the action to help you improve it. So typically, I did a formative evaluation at the very beginning of this training session. Can you identify what it was? You can write this in the chat. Did you go through a formative evaluation action, or at least, let's say, a part of something that could look like formative evaluation action at the very beginning of this session? No? What was the first task I gave you? I asked you when I say evaluation, what words come into your mind? And you wrote in the mural some words, such as improving, reflecting, investigating. And by doing this, I am not only starting to engage you with all the training, I'm also having a glance at, do you have any idea of what evaluation may mean? Maybe you are going to tell me, oh, for me, evaluation means when I'm at school and I'm receiving gradients. And I know that maybe it's not that all the same meaning that I am going to use it for. So I had to clarify this. So basically, I did a kind of formative evaluation very symbolically, it didn't do a full evaluation, but by just asking you before we have a moment to discuss about evaluation, what does it bring as words, as ideas when we say evaluation? So this could be a formative evaluation. Typically, you have an action about climate change. Well, you're going to take some time with some people who are typically going to be part of your audience and you're going to ask them, when we say climate change, which words come into your mind? What does it bring as memory? What does it refer to in your mind so that you know what the links are already present and when frames are already present in the heads of your visitors, of your audiences? And typically, you could build something like a mind map, like the one you are seeing now, where you would map with these people in the audience what they already know or think about climate change and you'll have a first map of what you can use to discuss this and to build your own action in a more relevant way and more tailored way for your audience. Then we have a second part, which is the summative evaluation and summative evaluation is the kind of evaluation we do almost systematically when we do public engagement action and it's very often required by the funders and if you are a funder or a supporter of research, please require it, even a very small one that do require some evaluation, which is to document your action and to assess your activity after it's delivered. Not necessarily assess to say if it was good or bad but at least to be able to reflect upon it to build lessons out of this. And we'll mostly talk here about summative evaluation although a large part of what we'll see can be applied as well to formative evaluation. So should you evaluate? Obviously I'm gonna say yes, you should. You should evaluate your action and the first reason I think you should evaluate it is to simply clarify your aims and objectives even before going to the let's say real reasons of evaluation, engaging in an evaluation framework, building an evaluation plan will force you to clarify a lot of things about your action. Suppose you say, I just want to make a game about my research and have this game played by teenagers, you make this, you start doing it, that's great. If before you do it or while you are doing it, you start building an evaluation plan, you will be forced to ask yourself, okay, what are my objectives here? What do I want to change here? What is the long-term impact I want to contribute to? And that will absolutely change the way you build your game. Very often if you do this, your game will be much more tailored to have the right impact. If you build the evaluation plan after you build your action and just before you're going to implement it, very often you'll see that there are some little discrepancies, little, it will not fit perfectly together. And that means your action may not be tailored perfectly to have been back to evaluating. So I really, really engage you to build the evaluation framework and plans. At the same time as you are with the action you are building and use it as a way to build properly your action. Obviously, the goal of the evaluation is not to help you develop your action, but to help you improve your own practices afterwards, reflect on them to document your action and provide evidence for funders, for supporters, but also from other researchers that you can share it widely. You can demonstrate the value, the benefits and the impact of your activity. And you have, and I would say that one of the most important element, you can share the lesson learned with others. That means it's extremely important whenever you do a public engagement action, sorry, to evaluate it and share what happened with your co-researchers, so as to build a culture of research to be able to reflect with the others and get support from the others or support the others. So one way to define evaluation or at least summative evaluation would just be to say summative evaluation is just asking yourself, what happened? What just happened? And this is a good question to ask yourself after you've done any public engagement action. Personally, when I do an evaluation of an action that I did not lead and I'm meeting, for example, the public for the first time and I didn't see the public before, it's a very nice question to start a discussion about, ask them, well, you've been through this, what just happened? What happened? And they tell you about what they lived, what's their experience, what did they perceive as what happened? And that is sometimes quite different from what you intended. And that doesn't mean it's bad, it just means it's good to be aware of what we are actually creating. To plan your evaluation, you have several steps. So first thing you need to define is your general aim. So for this is the very big picture of what the action you want to have. This general aim is going to translate it to objectives. Objectives are going to be clear things you want to reach in order to say the aim have been somewhat achieved. Obviously you'll have to frame the evaluation as you framed the public engagement action knowing the topic, the audience, the partners that what we have seen in the second session. And you are going to ask yourself evaluation questions to know if the objectives have been reached. Now to answer those questions, you'll need to collect data and you'll need to analyze data through a methodology. And finally, you will have a report which can be just a 200 pages report extremely documented or just one sheet of paper with a few notes depending on the level of details you want to give and depending also on what you will do with this report, who will read it and how you will share it. First of all, the aims and objectives. So the aim is very, at the same time, a very simple thing but not always very easy to frame as it's what do you want to achieve? And it's quite nice when you are building a public engagement action. I'm going to do a talk at the radio. I'm going to build and I'm going to share on social media my research. What do I want to achieve? Why do I want to globally share my research on social media? What will it change? And then translate into objectives. Well, if I share my research on social media and that's because maybe I want to have it to create a culture of research on my field. I'm doing it, let's say a research on molecular biology and I want to create a culture of research on social media on that field. So the objectives are obviously has my research been seen on social media? Has people engaged with my research? Have they commented? Have they posted other things? Have I been quoted by other accounts? Have it created discussions and debates? And we will try to have these objectives as specific as possible, measurable so as to know if they have been reached or not or if they have been halfway reached, 30% reached. Are they achievable? So we ensure that these are not just utopic ideas that cannot be reached, that they are relevant obviously and they are time defined so that we are aware of this. This is not specific to public engagement so I'll go quite quickly here. These are usual project management, objectives, qualities. Now, let's go more specifically into evaluation. For the evaluation questions, you're first going to ask yourself, what do I want to know about the outputs? So what are the outputs? The outputs are simply the elements produced, the games, the events, the talk you've made on the radio. These are what happens. So what is their quality? What was their reach? Did they reach a lot of people? Who did they reach? Did they reach only educated people between 30 and 60 years old? Did they reach mostly young girls between 10 and 15 years old? So what was their reach in not only in terms of numbers but also if possible in terms of groups? And what is the general quality of these outputs? And here on the right, you can see an element coming from the Quest European project that ended recently. It's an H2020 project that built indicators on quality for good science communication. And I will send you the link as well if you'd like to download the whole deliverable, but here you have elements of all its indicators so as to see if your outputs are good in terms of science communication. Are they trustworthy? So on what science and on what knowledge are they based? Are they engaging? Are they clear, coherent? Is there some interaction? Then obviously also, are they connecting with society? Do they have, can society relate to it? Is it targeted? Is it impactful? So we can discuss this more in details if you'd like, but that's the first question you can ask yourself about the outputs. And this is the first of your remember when we did these three boards about we would like our events and science communication to be so as to create, so as to contribute to. So this is the first element. The second one, so as to create would be what we call usually the outcomes, which is quite different from the output. So the outcomes are not the events itself that you have done. It's not the book itself that you have written. The outcomes are the benefits, the effects of changes that were immediately created. So did your book create a new awareness of something? Did you promote, did you create a new understanding? Did you lead to a change of perception of who are researchers or of that element linked to your research or that part of society? Did you link to a change of attitude, to a change of behavior? People are now going to act differently in their daily life. Did you create a new capacity? People know now how to build something just like the young lady you can see here in the picture. She now knows how to build a kind of box and prototype some kind of things and maybe it also created it to her an empowerment in terms of confidence, in terms of the way she perceives herself, in terms of how she trusts herself or how she trusts other parts of research, for example, so that she now feel empowered to do things she did not dare to do before. So this would be the outcomes. And the last aspect of course, the third box that we discussed before is the long-term impact. So these are the long-term changes. So the transformation of society, of research or of the relationships between research and society that you are contributing to. And obviously you are not doing this all by yourself only by your public engagement action. You are contributing to it. For example, you want to foster science literacy or you want to strengthen the public trust in science and you are not going to change this all by yourself but your action will contribute to this long-term wide impact. You are you going to support your sitting becoming carbon neutral? Are you going to develop a scientific culture on social networks? Are you going to make research more relevant and responsive to social needs? Once again, these are wide, wide, wide impacts that you can only contribute to but it's extremely important for you to be able to link your action to outcomes and then these outcomes to impact. Why is it important? It's important first of all for let's say funders and supporters, usually funders if we'll fund your action because they are funding several other actions and if all these actions are contributing to the same wider impact for them, it's very coherent or because it links to the funder's mission. So looking for those links is quite important and let's apply this right away. You have access to the chat right now. Can you write in the chat what was the output of this training session? So this element, the output. So you are participating to right now a researcher and supporters of research training on public engagement in the GRACE project. What is the output here? Can you write it in the chat? The video recording of each session is definitely an output. All right, I'll say the first output is simply this training you are participating in. This is the first output. What we are, the event we are having right now. The second output would be definitely the recording of what has happened. Another output will be the report that might be prepared out of this, absolutely. Another output will be a resource sheet that I will send you after this session where you will have links to all the various topics we have discussed in the past so you can look for more detailed reports and other resources. So all these are outputs, things that are created and in themselves. Now what do you think is the outcome of this training session? I would say that Jessica has already written one of the outcome, Jovita, another one. So one of the outcome is a new knowledge. So I don't think I created new knowledge by myself in those sessions, but I would say that indeed maybe we created a new awareness of what public engagement is, maybe even a new understanding of the various public engagement types of action or maybe a new understanding of what is evaluation of public engagement. We also created maybe new skills, maybe now you have some basic skills about how to evaluate or how to build the public engagement action. We created an awareness about public engagement. There was a knowledge transfer, that's absolutely true. And now what is the wider impact? And we already have one from Maline here in the chat. More public engagement activities. So one of the wider impact I could claim if I was going to do a report on this would be to say, I expect, I hope I designed this action so as to have more public engagement activities done by researchers in Europe. Obviously this is just one actions among many for this but I'm contributing to this. What else do you think this training session is trying to have as an impact, as a wide impact? So we saw already one from Maline, more public engagement activity. Oh, transformation of researcher public relationship. Great, so this is also a wider impact. We would like in Europe to have the relationship between researchers and the public to be transformed. So probably if I were to write it, I would be a little bit more specific in terms of what kind of transformation do I want? Do I want it to become awful? Do I want it to become hateful? Or do I want it to become closer? What kind of transformation? But that's exactly a wide impact that we could be contributing to right now. Transformation of the relationship between researcher and the public. We could say that we are widely, we could say that we are now contributing to a new culture of research that in the past research used to be only focusing on basically publishing articles and from your work and I mean, academic articles. And that now we are trying to transform what the culture of research is and having a new culture of research that is embedding public engagement and research. We can have, yes, to have more effective collaborations between public and researchers. We are contributing to a more transparent research. Absolutely, Mimi. Thank you very much. So I think you've identified very relevant ones here. So you can see the output is very clear. It's the things we are creating. It's the object or the events that are happening or the reports done or the recordings, the outcomes. We created maybe a moment of shared discussion around public engagement. We also created a new awareness and new understanding, several elements. Maybe I'm going to ask you in a few minutes to create plans for public engagement and so we created a bit more than awareness. It's also maybe I'm going to change some of your behaviors maybe because maybe you are going to engage in public engagement in the next few weeks. And last, we are with this contributing to wider impacts, transformation of the relationship between researchers and audiences, contributing to transparency of research and fostering and strengthening public engagement activities from researchers in Europe. Great. So these three elements, I would say the first three things you have to identify when you are planning your public engagement evaluation. What are the outputs? What are the outcomes? What are the impacts? And then you can ask yourself, well, how can I know? How can I know if I did this? Did I produce the outputs? Are there good quality? Did the outcome happen? Did we create this new awareness? Or maybe at the end of this evaluation, you will say, at the end of this training session, sorry, you will say, well, I don't know at all what is public engagement and I don't have a clue anymore. So maybe it was not productive in creating this new awareness. And last, it did contribute and that's the most difficult thing to evaluate, of course. Did it contribute in some ways to this wide long-term impact? Very usual, let's come back to the presentation, a very usual public engagement question you can ask and that we very often ask. So the very first thing we almost always ask ourselves when we evaluate is what is the reach? So what are the numbers? How many people read our book? How many people were coming to the event? Who were the people involved? What were their age groups, for example, or were there specific groups that were present? Were there people? I also wrote People Are Rich and People Involved. This can be quite different, usually by People Are Rich. I mean people who are exposed to the material, so people who are listening to the talk, listening to the radio, seeing my show. People involved mean usually that they have been involved in a stronger way, that they have participated in the action. It's not just a reach. A reach would be people so opposed on social media. It's people who actually got involved into an activity. What is their general experience? We can ask for satisfaction for their emotions and feelings. So just a word about this. Satisfaction is something we very often, if not always ask and to be honest, it's something that is not always very useful. If I ask yourself, are you satisfied with this training? And if you tell me, yes, I'll be very happy, but will I know a lot about what happened not much? If you tell me no, I'll be very unhappy, but will I know a lot not much? So it's something you can ask, but that's usually not very, will not give you much insight. Whereas if I ask you, what emotions did you feel? What feelings did you have? That might give me much more precise information. We want to, we wonder if we created new awareness on some things. If we change the perceptions of researchers or the perception of the research topic or of your daily life. If I created a new understanding or knowledge and just also questions like, what do people remember of it? Does it resonate with other experiences because your public engagement action is never alone. It's always linked to other elements. And does it change people's behavior? Does it change people's behavior individually or collectively? So these are just usual questions that we are going to ask ourselves when we evaluate public engagement. And for data collection, the first thing that I have to say, we always have to say this is the distinction between quantitative and qualitative data. So some of you may be already familiar with this, what we call quantitative data. Thank you. Platforms like SurveyMonkey is definitely also very good. Something that can be very relevant if people have access to these. Quantitative data would be numbers, ratings or responses to factual questions. Did you like the event? How would you rate the event between excellent, very good, so-so, quite not really nice and absolutely awful. And you have people rating the event and you can gather their answers and you can do statistics with them. So quantitative data is great to get a lot of responses. You can send this to every participants and then you just do statistics with them and that tells you quite an important insight about your event. And they can be very representative as you can have a lot of answers. The drawback of this is that you don't know what actually happened in people's mind people tell you what they liked, but if they liked, if they didn't like it, if they felt like this or like that, they give a rating, they give you numbers, but what actually happened in their mind, in their hearts, in what happened to them more in the complexity of it, you usually don't know what happened. So whenever you want to improve with quantitative data, usually you find if you need to improve and sometimes the kind of area you need to improve but not very precisely what happened, what was the issue and what needs improvement in details, you don't get it. So for this, you will turn to qualitative data which are responses to open questions, discussions, sometimes even artworks that can tell you elements of qualitative, some can give you qualitative data. And here you will understand what is happening with a lot more depth, but usually it requires a lot of time to get responses only from a limited number of people. So typically qualitative data would be an interview. You have an interview with someone, you ask them question, they give you answers and discuss with you. You discuss a lot, you spend one hour or half an hour doing the interview, then you spend sometimes analyzing what they said and that takes a lot of time for only a few people interviewed, which is in a way less representative than the quantitative data approach, but then you really understand what happened in much more details. So obviously, ideally, you will try to mix both techniques if you can, meaning so as to have a quite representative, broad overview and then understanding in a few people what actually happened. So that's the ideal. You cannot always do this because obviously evaluation can be constrained in terms of time and resources and in that case, you will just do one of the two, but it's great if you can have both. An element I have to mention when we speak about evaluation, it's always to be ethical, which means first of all, to treat participants with respect, which means if someone says they don't want to answer one question in your interview, please respect it. You never know what people are actually living and what is their experience. Sometimes they have very good reason not to answer to a specific question or to ask for a specific element. So be extremely respectful when you're evaluating. Be fully honest and transparent about what are you evaluating? Why are you evaluating this? So that they do not take part of an action that they would consider contrary to their beliefs or to their opinions and values, for example. The question of ownership is very important. You always ask permission to record or to take data. And usually if you do a recording of an event where you will also record the participants, that's great for evaluation purposes because you can come back to the recording and see what was happening to people. But always obviously ask for permission and use for this. You have very classical forms where people are giving their consent, make sure they are giving their consent, make sure that they are able to not give their consent and still participate to the event. And that did not feel pushed to give their consent. Be honest about the constraint and influencing decision and do not leap to conclusion without evidence. So this is just in terms of integrity. It's when you will typically use the data, be very, very honest about what the data can help you tell and what it cannot help you, what it does not support as a conclusion. And the last one, which is extremely important is the confidentiality. If you are taking pictures, filming, if you are taking any data, if people are having interviews with them, you are obviously recording the interviews, please protect the data, keep the recording on a safe place. There may be some elements of their lives they will be telling you that they do not want these elements to be shared widely or to be accessed by anyone. So always keep it in a non-accessible repository. All right, I will give you a few examples of tools now and just three to begin with. You already mentioned questionnaires. So questionnaires is the most classical usual is the first evaluation tool we always jump on because it's the most obvious one. It's also the most boring one to be honest. It's also the one that people hate filling in. So it's not necessarily the best one. Just a couple of reflection on this. We love questionnaires. We love questionnaires because we can ask directly the questions. And if we want to know if it was fun, we can ask people, was it fun? And they say yes or no. So it seems very straightforward. But this is often an illusion I found because very often when people see this, first of all, many people can't get bothered to answer the questionnaires. They don't want to spend more time just answering a questionnaire, especially if it's long. So they often don't answer a part of them or the ones that found even boring are not gonna now answer a questionnaires saying it is boring and even more bored than before. So usually the only one who will answer will be the ones who found it fun and you will get plenty of answer, but the great answer is telling you're even was fun. So the questionnaire is not always the best one and be aware of this kind of bias. By using a questionnaire, I am only getting the answer of people who are happy to spend 10 more minutes on the event so they are not already bored. On when I give a questionnaire and I say, was the even fun? People might also be tempted to write yes so as to please me and to flatter me because they want to be nice with me and they are nice people, but that's not the answer I want to get. I want to know what's actually happened. So questionnaires are always possible, but they have drawbacks.  The questionnaire of one question is almost never happening, but would be the ideal. Just have a few questions. If you can have less than five questions, I think that's the best. Getting more than five question is, it's not a way, it's becoming a real task to do and that's not, I think that's not very kind to the audience, to ask them more than five questionnaires. It's possible, I've done it several times, we all do it, but I would encourage you not to do more than five questions. If you can design the questionnaires, having the graphical design here, you can see the one that you see on the picture is designed for children and that is already seen that the children see that it's something for them and that it's not, let's say, a much more formal usual questionnaire. And then on the right, you can see a kind of flower and imagine that at the end of your event, rather than giving a questionnaire, you just put some posters and one of the posters is, did you enjoy? This event was super fun, nice, not bad, but they're boring, extremely boring and people have sticky dots, little sticky round of colored dots and they just put their sticky dot on the answer they find appropriate. So just by going outside of the event, they just put a sticky dot on the flower and they are actually answering your questionnaire, but in a much more engaging way, much nicer way. So you'll get much more response and probably not only more data, but also more quality data, some responses are also often more honest. Let's move now to interviews. You've mentioned them. I just want to speak about this. So for semi-structured interviews, you'll often just pick a few participants, maybe two or three, and have a discussion with them, either as a focus group, so the three of them and you, or individually, one to one, one after the other, have interviews. I wanted to stress a few things regarding interviews. In semi-structured interviews, you are looking for qualitative data, which means what you are looking for is people speaking. You want them to speak as much as possible and you will make usually for interviews some interview questions. We often speak about semi-structured interviews, meaning we'll have a few open questions like four or five questions and we'll ask them, so what happened? So what was the event like for you? What feelings did you have? How do you perceive climate change now? And people will speak. And our goal is to have them speak as much as possible, which means the questions have to be open. We don't ask questions that can be answered by yes or no or that can be answered with just one word. We ask questions like what happened or how do you perceive climate change after this conference? And people have to elaborate sentences to answer. We try to make these questions very simple so that people don't have to struggle to understand the question, but they are forced to speak a little bit to us to answer. And usually we want them to speak a bit more. So typically, you ask, how did you experience? How did you live? How was it for you to perceive it? And someone tells you, well, I felt quite awkward. Just repeat a very usual trick we all use is repeat the last sentence of the person we're interviewing with an interrogation mark at the end. Someone tells you, well, I felt quite awkward during the event.  And usually they will start elaborating. And this is something, well, it's very useful. I urge you to use it as well in your personal discussions in your family and in your professional discussions in your teams. But this tool of whenever there is a statement where you feel everything has not been said, just repeating the last words with an interrogation mark is a way to gently invite to say more without whether if I said, oh, why did you feel awkward? This is a little bit more difficult to answer. Or did you feel really awkward? This is much more directive and even can be a bit judgmental. So we try to be as little to not be judgmental at all. And that's why we often just say, repeat the last one and say, oh, yes, just an, oh, yes. Oh, that's interesting. Can you tell me more? Just these things make people speak. And that's how we will gather the data and as much as possible. We, and the questions you have in interviews are questions to start with, what we are looking for is not so much the first answer but the discussion that comes afterwards where people will often tell you a bit more, be a bit more intimate. And the very first words are often words to please you. They would say, oh, yes, the event was great. That was great. And you said, oh, that was great. Yes, yes, that was great. Although there was this and that and this and this and that's when you will gather the interesting bits. And the last thing I wanted to stress is that you can also do evaluation without words. And that can be important for some specific audiences. You can have people speak or answer questionnaires but you can also have people draw or also, sorry, here we are having people write again words. These are just with sticky notes on a board or on an object. If you have an object which is representative of the research, I don't know. For example, if you talk about climate change you can have the planet globe of the earth and you have people write a sticky note and put it on the globe. So that it's a bit more meaningful that putting it on a board. I remember one time I was doing an activity about new technology. So we were in a room with screens and new, not computers, but let's say technologies of virtual reality and things like this and augmented reality. And at the end, I asked participants to write just a question on there posted. I said, after you have discovered this new technology I would like you to ask a question about this new technology about what it is, what we can do with it in the future. So they had just been half an hour exploring this technology. So they knew a lot, but I asked them what question would you ask about it? Write it on a post-it and put the post-it on the technology, on the object. So at the beginning we had a shiny new screen and an object of technology. And at the end, we have the screen and this object full of post-it notes questioning it. And this can be quite engaging as well. And once again, that's a nice way to collect data. And so the last one, yes, if you don't want to use words at all, with the post-it you can also ask for drawings. But one thing I like very much. So it's quite intense in terms of time. So, but it's nice because people don't have to draw and some people are a bit afraid to draw. They feel they are clumsy and they feel they don't know how to draw is collage. So for a collage, you will just give a pile of paper of magazines to people, a lot of magazines of different kinds so that they have different kinds of photos inside. You give some markers if people want to draw or to add some elements, obviously. You give them some glue, you give them some scissors and you ask them to make a collage with what happened or a collage of how they feel or a collage of something related to your evaluation question. And they can even put some legends or with the markers indicate some words if they want. But they can also leave it just like this. And you look at the collage and you ask just people to tell you what their collage is. And that's a nice way to move a bit away from the words. And so that people can just feel, what did I feel? And they look at the magazines and usually they don't know what they feel. They don't know what happened. They look at the magazines and suddenly they see a forest. Suddenly they see an image of the sky. They see an image of someone who is happy or someone who is miserable and they will start adding these on their papers and organizing it and it will make sense for them little by little progressively. And at the end, they will show you their picture. This is part of the data. They may present you also with a few words, their picture and that's part of the data as well you will collect. And that's a very, very nice way. Once again, quite long, it takes some time but it's a very nice way to collect some data and to have an insight about the experience of participants in your event and to get some feedback. There are many, many, many more. And one of the resources you will find in the resource sheet is the Public Engagement Evaluation Toolkit that has been developed by Queen Mary University of London. So for evaluating public engagement you have a lot of toolkits, of guides, of handbooks that have been developed. You have a nice one from the NCCPE, you have several ones from British universities and other universities in Europe. One, this one, which is the one that was developed by Queen Mary University is quite nice, it's in two booklets and the second one is focusing mostly on tools. And you see here a wide variety of tools that you will be able to explore more in depth in this report. But that will give you once again more ideas about how to collect data that is not just sending a question that can be more engaging but also more easy to handle for you and give you another kind of information. Last thing you will do obviously is to analyze the data. So you will do this in a three steps process. You will, first of all, try to notice and collect. Noticing is quite important. Once again, we come back to the idea of observation but also when you have gathered your data and noticing what is happening when you look at it at first glance, does it tell you something already? Do you notice already a pattern or something specific that's happened? Then you will try to sort and think. So you'll try to see, oh, are there some clusters? Lots of people saying it's great. Lots of people saying it's not great. Lots of people saying that it has changed in this way. And obviously you will use your critical thinking to interpret the data and to say, all right. So we can see that it changed people perception in that way but it don't seem to be ready to change their behavior. Or this is right and this is not right. Or that part worked very, very well but that part didn't seem to have any impact. So how can we change that part? And you will, this can be done in a very structured way or it can be done in a quite loose way. Obviously for analyzing the data you will need some time. You will need also to come back all the time to the objectives and to the evaluation questions. What do I want to get answer to? Where do I find in this data some answers to my evaluation questions? Does this data tell me if I have reached or not my objectives? Are there some patterns or is there some data to group? So the idea of coding and this on the side it's quite technical and I don't think you will do any coding at that stage but more simply if you have to gather some posted notes just like we did in the activities here gather them if some of them are similar or if there are some groups and see what are the main clusters emerging. You can, you simply print some text that has been written and highlights the key points that you see. Always find some representative quotes. Nice quotes are great because they are also giving a little bit more of a sensitive content. So they can be great always use some code try to find some quotes that are representative but are also quite nice to put in your report. Be critical in the interpretation obviously and be reflective. The ultimate goal of all evaluation is to ask what happened, what worked well, what didn't work well what would I do differently if I were to restart? And that is even for me for my new action or for other people when I will share with them so that it can support them other researchers doing public engagement. All right, we still have a bit of time and I would like to use this time to ask you so just checking in the, yes to ask you what are your plans for the future? How do you plan to add public engagement in your research? If you're a researcher or if you're a supporter of research how will you plan to add public engagement in your support? Do you plan to do a talk? Do you plan to do a call for public engagement actions or to add a public engagement condition to some support you are giving to researchers? And you want to plan public engagement in your lectures, great. So I would like you to reflect on this and then to reflect we'll do it right away on the very first steps that you have to do for this. So not only the whole plan of everything you have to do to get there but what is the first step? And the other question I was wondering is with whom you need to speak about it? You need to speak about it with a supermarket who will be a partner. You need to speak about it with your supervisor or with a collaborator. You need to speak about it with the possible audience. With whom will you have the first conversations about this?