Ah. Do you have enough left? Are you ready to start again? Okay. For the third lecture, shall we start? Okay. Thank you. Okay, so let me continue my lecture too, even though this is a third lecture. Okay, so in lecture two, so let me first briefly review what we learned in the second lecture. So in the second lecture, we learned how to define entropy production, so which is a logarithmic ratio between time forward and time reverse path probability. And then we learned the fluctuation theorem. Actually, the fluctuation theorem means that this average is equal to one. So because R is equal to total entropy production, it means that total entropy production satisfies this fluctuation theorem. And from this fluctuation theorem, we can derive the thermodynamic second law. And of course, there is a freedom to choose the dual path probability. So by choosing some useful dual path probability, then we can show various fluctuation theorems. So the first one was Dresden's key equality, and second one is Krupp's relation. And then now I will continue to the third one, which is called the Hattano-Sassoff fluctuation theorem. Okay, so what is the Hattano-Sassoff fluctuation theorem? So it is about the separation of the total entropy into two parts. Okay? So this is a total entropy production, which is given in this way. Forward and time reverse path probability. But one natural separation of the total entropy into two parts is that, now we know that, I mean, this is a system entropy change, and this is a reservoir entropy change. So this is a one, I mean, natural separation of the total entropy production into two parts. So then next question will be like this. Now we know that this total entropy production satisfies the fluctuation theorem. Then what about these two separated entropy production? So this system entropy production satisfies the fluctuation theorem, or this heat satisfies the fluctuation theorem. So this is the next question. So let's look at, I mean, step by step. So by definition, this average can be written in this way. So this is an exponential factor, and this is a time forward path to probability. And now here we use this equation. So here we plug this equation to this. Then we can show that this becomes this one. And you know, I mean, you see that this is not the path to probability. So this is not the normalization condition. It means that this summation is not equal to one. So it means that the system entropy production does not satisfy the fluctuation theorem. Any question here? Okay. And then the second one. So by definition, so it can be written in this way. So here this is exponential part, and this is a path to probability. And now same thing. By using this equation, so here we plug this equation into this one, and then we can show that this becomes this one. So this is not the probability. This is not the path to probability. So the summation of this one, this is not the normalization condition. So actually it is not equal to zero. It is not equal to one. So it means that this average also that the heat does not satisfy the fluctuation theorem either. So this separation actually does not, I mean, the separated these two entropy production do not satisfy the fluctuation theorem separately. Then the next question will be then, is there any some clever way such that two separated entropy production, entropy productions satisfy the fluctuation theorem simultaneously. So there is a way. So Hatana also found it. Okay, so this is a total entropy production. For example, I mean, this is a total entropy production for Markov's jumper process. But actually the derivation process is essentially same for long-term dynamics, but here I will only focus on the Markov's jumper process. So this is a total entropy production. And here we add this term. This term is, SS means that this is a probability in the steady state. So this is a probability in the steady state. So we add this term. And here look this. And this is actually the upside down. This is the upside down fleet. So actually this term is minus this term. So it is like something minus something. So actually they are canceled out. So these terms are actually vanishing time. But anyway, we add this term and we subtract this term. Okay, so here we define this first part as a non-idiotic entropy production.  And this non-idiotic entropy production actually is related to the excess heat.  And sometimes this non-idiotic entropy production is called HATANA-SASA entropy production. Okay, so this is the definitions. And now let's look at whether these two entropy productions satisfy the fluctuations here and separately. So let's first look at this non-idiotic entropy production. So we can write this term in this way. Actually, this part are same. And this term becomes this one here. This one, R-degas, R-degas is defined as this quantity. You see that here X and X prime order is reversed, and X prime X. So the order is reversed. So you see that if we plug this equation into this one, and the transition rates are canceled out, and only this ratio, steady-state probability ratio remains. So that's why this term is exactly same as this term. Okay, now this beta transition rate has an important property. So this is it. So this one is actually the transition rate of the original dynamics. So the summation over X, actually, this is escape rate of the original dynamics. And you can also show that the escape rate of this beta dynamics is actually the same as that of the original dynamics. You can show easily, I mean, by summing over X here. But anyway, after the lecture, you can show it. So by using this property, we can say that the same probabilities are same for each transition matrix. So we can now write this non-ideactive entropy production in this logarithmic ratio of these two path probabilities, time for the path of probability and time reverse path probability. And not the time reverse path probability, but this is kind of some dual path probability. And this dual path probability satisfies a normalization condition. Because actually the escape rate are same. So the same probabilities are also same. So the same probabilities are canceled out. So only actually if we calculate this one, then it becomes this one. So in such a way, we can write the non-idearity entropy production. So because this dual path probability satisfies the normalization condition, then the non-ideactive entropy production satisfies the fluctuation theorem. So in such a way, we can show that this non-ideactive entropy production satisfies fluctuation theorem. And this is the same for IWT part. So this is IWT entropy production. Actually, you see that this part is actually canceled out. So this term is actually zero. So if we plug this definition into this one, then you can check that this ratio becomes this one. And this two transition matrix ratio becomes this one. So actually this term and this term are same. And because it has also same properties, same escape rate, so we can also write this as a logarithmic ratio of forward path probability and some kind of a dual path probability. And also this dual path probability satisfies the normalization condition. So this IWT entropy production also satisfies the fluctuation theorem. So the thing is that the total entropy production can be divided into two parts, IWT and non-ideactive entropy production, and each one satisfies the fluctuation theorem. So it tells us that total entropy production has some kind of a hierarchy. Okay, so this is about the hot and the sauce of fluctuation theorem. So here, any questions here? Ah, measure, I mean you mean excessive housekeeping. So actually it is not easy to experiment and measure this quantity, but the housekeeping entropy production is actually when in the steady state, in the steady state actually the IWT entropy production is equal to total entropy production because in excess entropy is actually some entropy necessary for making some system transition. So in the steady state actually this term goes out and only this IWT entropy production term remains. So in the steady state we can easily measure the housekeeping here. But in a general case, I mean it is not easy to measure experimentally. Actually it is, I mean this term is related to non-ideability. This term is related to some process, I mean process. In the steady state process actually it is related to some steady state process entropy production. So I mean that's why I mean they call it IWT entropy production and because this is not so they call it non-ideability entropy production. But that's what I know, but yeah, yeah. Okay, so any other question? Okay, so now I want to show you the expression of entropy production rate. So I mean this is different from the entropy production. Entropy production is a total entropy production during finite time, but this is the entropy production rate. So let's look at the over-adent long-term system first. So this is a long-term equation, given long-term equation. And here this stochastic differential equation can be converted to this partial differential equation and this partial differential equation is called focal Planck equation. And this here P is a probability distribution function of X at time t and this J is a probability current which is defined in this way. Force F and some temperature and another partial derivative. So probably there may be some, if you are unfamiliar with this focal Planck equation, in this lecture due to the time limit, I cannot explain how this converted into this partial differential equation. But if you are interested in, you can read the book of Ritzkin. Okay, so here let's, if you don't know what about this then first let's accept that this can be converted into this partial differential equation now. Then the system entropy is given by this Shannon entropy form, minus kB log this probability density function. So its time derivative can be written in this way. So it can be calculated in this way and because this total time derivative, because this P is a function of X and T, so by using the chain rule, this is a partial derivative of X, partial derivative of T. And as I mentioned when we make a partial, I mean the chain rule for the over-damped electron dynamics, we have to use this Stradenovitz product. And here now by using this equation, we can convert this partial derivative of X of P. We can replace this term by using this formula. Okay, from this one. So, okay let me arrange the terms. So this is the first term and this is a second term and this last term gives this third term. And here now we replace this f function by using this equation of motion in terms of x dot and Krissi. Then we can replace this f function in this way and the same thing, same thing. So and this term, the meaning of this term is actually the heat rate. So now let's move this term, Q dot over T, move this term to the left-hand side. Then the left-hand side becomes this one and right-hand side becomes this one. And the meaning of this term is actually the total entropy production rate. Okay, so it means that the total entropy production rate is expressed in this way. So the average value of this total entropy production rate can be written in this way. And the meaning of this average value is this one, average over this probability density function. So let's calculate this average first. And then from this definition, it can be written in this way.  So the time derivative of this one is just simply zero. So it means that this second term vanishes. And now I will use some general equation relation. So I mean for a general function g, then g multiplied by x tau. And here the product is Strato-Novich. So this average can be, we can show that this average is equal to this integration. So g times this j, the probability current j and integration of our all position. Then it is, they are saying, we can show that. In the next slide, there is a derivation in the next slide. So anyway, at this point, let's use this equation relation. Then we can calculate this average by using this equality. So this is it. So this is a p, a j square over p and integration over a whole range of x. So this is an expression of entropy production rate in the over-dental algebra system. And for under-dental algebra system, the expressions are almost the same. Except instead of this j, we have to replace j into irreversible current. So the same form but a little bit different function. But you don't care about that. Anyway, so almost similar functional form. And then I will use this expression to derive thermodynamic uncertainty relations. So please memorize this form. Okay, and this is a derivation of this relation. But due to the time limit, so if you are interested in, then you can read my lecture notes. So here I will skip this. Okay, then what is the expression for entropy production rate for the Markov-Jumper process? So I mean, in this case, it is easy to find entropy production rate expression. Because in the Markov-Jumper process, entropy is produced only when jump occurs. When it stays at the same state, then actually no entropy is produced. Only when jump occurs, entropy is produced. So let's say that jump from y state to x state, let's say that this jump occurs. So the during time t, so when this jump occurs once, then the probability of serving this jump from y state to x state is given by this way. And the time reverse, time reverse path probability is given in this way. So the logarithmic ratio gives the entropy production induced by this jump during time delta t. But this delta t are same, so actually we can write in this way. And the number of jumps from y to x during delta t is given by this way. So the entropy production, total entropy production during delta t is given in that way. So the entropy production rate divided by delta t and time delta t goes to zero limit, then it becomes the entropy production rate. So it can be written in this way. So this is an entropy production rate expression in the Markov-Jumper process. Okay, so this is a summary of the whole lecture too. So now I just talked about the Hatana-Sasa entropy production. So the entropy can be divided into two parts, and each one satisfies the fluctuation theorem separately. And then this is an entropy production rate expression over the entire dynamics. And this is an expression of entropy production rate for Markov-Jumper process. Okay, so I'll use these two expressions later. So okay, so now let's turn to my lecture, the original lecture three. Okay, so I'll skip this summary. Okay, so let's summarize what we have learned up to this point by using this schematic restochastic system. So there is a stochastic system. And let's say that t0, this is the initial time, and this is the final time tau, and this is the initial state, and this is the final state, x1 prime. And gamma1 is one single stochastic trajectory. And there are many, there are many initial states, there are many final states, so this initial state constitutes this initial distribution, and final states constitute final distribution, and there are many stochastic trajectories. So here we are interested in measuring some observable theta. Of course, this theta is a function of trajectory gamma. So for example, in lecture one, we learned how to define heat and how to define work, or theta can be displacement. So they are actually the observables we are interested in. And one special observable is entropy production. So entropy production is also a function of gamma trajectory. So we learned this entropy production in lecture two. So in the first lecture, I mentioned that the thermodynamics is physics, which is about the relation, which is about to study relations between these observables. So the important question for this stochastic system is that, is there any general relation for entropy production and these measurable observables? So in lecture two, as a one important relation, we learned this fluctuation theorem. Here, from the lecture three, I will use this notation sigma instead of a delta as total for simplicity. So this sigma now means the total entropy production from the lecture three. So in lecture two, I talked about the fluctuation theorem. This is an important relation for entropy production. So by using the Johnson's inequality, we can derive this thermodynamic second law. And so it can be called as a generalized second law. This is what we learned in lecture two. So by choosing the dual dynamics, we can derive many other fluctuation theorems. Johnson's inequality, Kirch's fluctuation theorem, Hatano's as a fluctuation theorem, and information fluctuation theorem. Even though I did not talk about this information fluctuation theorem, then I mean Tadaito will talk about this information fluctuation theorem in his lecture. And then the second relation is about a thermodynamic, this is called a thermodynamic uncertainty relation. And shortly we call this TUR. And TUR is a relation between entropy production and observable. So what is TUR? So TUR looks like this. Fluctuation theorem is equality, is given by equality, but TUR is given by inequality. So it looks like this, and this is a variance of observable. So this term is called relative fluctuation. And this is entropy production average. And this KB is Boltzmann constant. So what it means? So let me give you an example. So here, let's say that observable. Here, let's say that we set theta as a displacement of this molecular motor. Of course, because of the stochasticity, X-variable is a stochastic variable. So it will have some kind of distribution, and it will have some finite variance. Okay, so here. Sometimes we want to reduce this relative fluctuation. For example, let's say that we want to make this relative fluctuation zero. But if we want to decrease this term as zero, then from this inequality, the entropy production should be infinity. So it means that if we want to reduce this relative fluctuation, then entropy production should be increased. And if we want to decrease this entropy production, then relative fluctuation should increase. So it means that this TUR is kind of a trade-off relation between the relative fluctuation and entropy production. And entropy production is, we can understand, it is a thermodynamic cost. So we have to pay more thermodynamic costs for reducing fluctuation. This is the meaning of this thermodynamic uncertainty relation. Okay, and if we rearrange this thermodynamic uncertainty relation, then we can have this relation. Then you see that the thermodynamic second law just tells us that entropy production is larger than zero. But this TUR tells us that entropy production is larger than some positive value. So it means that TUR gives us a stronger bound than the thermodynamic second law. And this is an importance of TUR in theoretical viewpoints. Okay, so I'll turn out clear. Okay, so in lecture three, I'll talk about TUR. So in the first section, I'll talk about how to derive TUR. So for over-the-enter long-term system and for Markov-Jumper process. And in the second section, I'll talk about how to apply this TUR to our some experimental system. Okay, so this is a brief history of TUR discovery. So it was first reported in 2015, so seven years ago, very recent one. And here what I mean by report is that in this paper, actually they discover. I mean, they discovered this TUR, but they couldn't derive. They couldn't prove it. In the one-year layer, this group first derived this TUR by using large-division theory. We learned this large-division theory in the morning by Vipul's lecture. And then in 2018, this Japanese group derived TUR by using generating function method. And the next year, 2019, this group derived TUR by using Kramer-Rau inequality. And this Kramer-Rau inequality, I mean, this is very famous inequality in information science. So it looks like this. And here, I, I is means Fisher information. And this is variance. Okay, so this Kramer-Rau inequality greatly simplifies the TUR derivation. So I will skip all this derivation, but I'll only focus on this Kramer-Rau inequality derivation. Okay, then what is the Kramer-Rau inequality? So here, let's say that there is some general observable theta. And theta is a function of z. And there is some probability distribution, which is also function of z. And this is parameterized by some parameter epsilon. So it means that P also depends on epsilon. And here, so this bracket, epsilon, means that average over this distribution. So if I write in this way that it means that the observable average, indeed by using this probability distribution. Okay, so now let's look at this one. So this is observable average and partial derivative of epsilon. And square. So by using this, we can write in this way. Because theta has no dependence on epsilon, so this partial derivative only applies to this probability distribution. And total square, right? And then here, let's look at this note. So now let's consider this term. And because this part is already integrated our z variable, so actually it has nothing to do with this integration. So it is kind of a constant, so we can take this term out of this integration. Then the remaining term can be written in this way. And because this is simply one, because this is a normalization condition, so this partial derivative is simply zero. So actually this term, a little bit looks complicated, but actually this is just a zero number. So here we just add this zero number here. So these are the same one. And then here we bind these two quantities with this common factor in this way. And here this one can be rewritten in this way. And so by using this equality, now this equation can be written in this way. So now I will use the Cauchy-Schwarz inequality. Cauchy-Schwarz inequality is something that if there is a two vector, and this vector product square is smaller than each vector square product. So Cauchy-Schwarz is essentially the same as this one. So in this quantity, there are two vectors. If let's say that this is a vector and this is a c vector, then this a vector square and this is c vector square. So we can separate in this way. And this product, these two products is larger than this one. This is a Cauchy-Schwarz inequality. If you are not familiar with this Cauchy-Schwarz inequality, you can, after the lecture, you can sit in your chair and follow this line. So this first term is nothing but a variance of theta. And the second term, we can rewrite this integrand in this way. Because this is a square, so we write second time and distribution. And by using this equality, now we change this term into this one. So this can be written in this way. And here we use integration by parts. So we move this partial integration in front of this one and take a minus sign. Then it becomes this one. So this is the integration by parts. And this one is so we can write, this is an average of this integrand. So we can write in this way. So here we define this average value is equal to feature information. So this is the definition of feature information. So this, I mean this equation can be written, simply written in this way. So variance times feature information. And you see that this variance times feature information is larger than this value. So we are arranging the terms. So we now finally have this Kramer-Rau inequality. Okay, this is a derivation. I mean this is a derivation for this Kramer-Rau inequality. So this Kramer-Rau inequality holds for any observable, any probability distribution. This is very general inequality. So now I will use this Kramer-Rau inequality to the stochastic thermodynamic system. So let's consider there is some original dynamics over them to run the system. Here the external force is f here. And now in this original dynamics I will add some perturbation force. So here the perturbation force is epsilon g of x. So g of x can be any arbitrary perturbation force. And here so you can regard this epsilon as a perturbation parameter. So small number, epsilon is a small number. So when epsilon goes to zero limit then this perturb dynamics returns back to the original dynamics. So when you now think of this Kramer-Rau inequality then you can regard this epsilon as some perturbation parameter from now on. Because I mean it always holds for any observable, any probability distribution, any epsilon. And so now from now on I will consider some observable which is a function of gamma instead of z. And I will consider path probability which is a function of gamma also. And because we are now considering this perturb dynamics so this path probability also depends on epsilon. Because this holds for any theta, any probability density function so I mean it also holds for this observable and path probability. So now take epsilon goes to zero limit. It means that so we evaluate the Kramer-Rau inequality at this original dynamics limit then it will becomes like this. Then here let's compare this Kramer-Rau inequality and this T-R. Then you see that there is a similarity right? So if this one, this one and this one so these three relations are satisfied then actually this Kramer-Rau inequality exactly same as a T-R. So and the first relation is actually automatically true. So we don't care about this first relation and actually this second relation, I mean second and third relation. So by choosing some proper perturbation force g of x so by choosing this proper function g x we can show these two relations are satisfied then we can derive this T-R from this Kramer-Rau inequality. We already derived this Kramer-Rau inequality. So if we only show these two relations by choosing some proper g of x then we can show this T-R. So this is the basic strategy to derive the T-R by using Kramer-Rau inequality. Okay then the remaining task is that what kind of perturbation leads to these two relations we have to find such a perturbation force. Okay so there are many, I think there will be many possible trials but one intuitive one is like this. So this is an observable average in the perturbation dynamics and this is an observable average in the original dynamics. So if these two averages have this relation then the first relation is automatically true. Right? This is trivial. So what kind of perturbation leads to this relation? So from now on our first simplicity I will only focus on steady state. Okay so in the steady state we can define the steady state rate. So let's say that this theta dot steady state means that this is an observable rate. Then the total, I mean the accumulated observable in the steady state is simply given by multiplying by total time tau times rate. So here now here we make this time scaling perturbation. It means that time is scaled by this quantity, one plus epsilon quantity, here epsilon is a small number. Then total time is also scaled by the same factor. Because the total time is increased so the total accumulated average observable is scaled by the same factor. Right? So it means that in the steady state if we make this scaling perturbation then this relation holds. Then the first relation can be satisfied. Okay so the remaining task we have to do is that if we make this scaling perturbation then what is the value of this feature information? It is really same as the total entropy production. So if it is so then we can derive the TOR. Okay so let's check it. Okay so this is the original dynamics. And the original dynamics of Focke-Planck equation as I mentioned that we can write this Focke-Planck equation. Actually they are same dynamics. So here let's make this time scaling perturbation. So instead of T we put one plus epsilon T. Here one plus epsilon T. And this one plus epsilon comes from the derivative of this time. Okay now let's consider only the steady state. Then in the steady state actually the probability density function does not no longer depends on time. So actually we can just write the steady state in this way. And in the partial derivative of this steady state probability density function is simply zero because it does not change in time. So the remaining thing is this one. And one important thing is that the steady state solution of this perturbative dynamics is the same as the original dynamics steady state distribution. Okay so this one is a probability density function of the original dynamics. So we can rewrite this term into this way. This is nothing but this first part comes from this one factor. And this second part comes from this epsilon factor by definition of this probability current.  Then we can write function in this way.  So now let's define this function as g function g of x in this way. Then let's look at I mean now this focal plan equation in the steady state. Then the corresponding long-term dynamics of this focal plan equation is this one. So it means that I started from this time-scaling perturbation. But come to a conclusion that actually this time-scaling perturbation is equivalent to this perturbive dynamics in the steady state. So this is a point. Okay so now we have this perturb dynamics and the function of g of x looks like this. So now because now we know what is the perturbation force and we can now calculate the future information. So this is a path probability so we know how to write the path probability for this perturb dynamics. So this is it and here I used the product notation e to product for convenience. So by plugging this equation to here actually this term and this term does not have any epsilon dependence. So because of this second derivative of epsilon so actually these two terms goes away. We don't care about these two terms. So only remaining term, the remaining part is this one. So this is it. Now we expand this square in this way. So this term square and this term square and their cross product becomes this one. And because this is an e to product so actually this is just a normal product. And look at this second partial derivative of epsilon. Then if it applies to this one actually this is vanishes because there is no epsilon dependence. This also vanishes because it only has a linear dependence. So the remaining term is only this one. This has a second and a squared term, epsilon squared term. So only this term remains by using this, applying this second derivative. So the result is this one. And now we know what is g function. G function is given in this way. So if you plug in this equation into this one and then now and evaluate this average value, then it becomes this one. So probably you are familiar with this expression. Previously I mean learn this expression is expression for entropy production rate. So the entropy production rate and integration over from zero time to tau, then actually this term is nothing but total entropy production. So what we show is that this fissure information is equal to the entropy production. So it means that this relation I mean now is satisfied. So by making the time perturbation, by making the time scale perturbation, so it satisfies these two relations. So because these two, because now these three relations are satisfied, so we derive the TOR. Okay, so this is the derivation of how to derive TOR. Okay, so are you now following me? So is there any question up to this point? I mean you can check by yourself by reading my lecture note, but the important thing is that some big picture how to derive a TOR. To derive TOR we use this Cramor-Rau inequality, and by using the time scaling perturbation, then we can show these three relations are satisfied. In such a way we can derive TOR.  Okay, can you microphone? Thank you. Maybe you will call this in the following, but the question arises since the Cramor-Rau and the fish information come from essentially developing to second order flows to a certain probability distribution, one might wonder whether there exists sort of higher order, more linear expressions of this. Yes, right. As I said, and this is actually when epsilon goes to zero, then the fish information is order of epsilon squared. So I mean, so this derivation is we call this linear perturbation, I mean the inequality, but we can also find more higher order inequality. Yes, as I said, it is possible. For example, Haydn, I forget his name. Dunsko-Varadan inequality, so such an inequality is kind of such a higher order inequality. So is this TOR is only valid for molecular equilibrium in steady state or it can be valid for the time dynamic equation also? Yes, very important question. So to derive this TOR, it should be steady state, non-equilibrium steady state. Then the next question will be like this. Then what happens if the system is not in the steady state? So for example, non-steady state situation, then in such a case, this TOR form is changed a little bit. I'll show you later how to change. In the proof, it seems that the TOR is a special case of taking special perturbation function. Then I wonder why we use TOR rather than the left-hand side of that arrow? So your question is that we can also use this inequality. Why do we have to use this one? That's a good question. But in experiments, so let's do some experiments. Then measuring the average of some observables I think are easier than to get this some partial, some different. So in such a way, directly using this inequality is difficult in experiments. So that's why we want to use just this average value, not the derivative value. Because if we get this number, then we have to change the perturbation force a little bit. And then we have to evaluate this one from the experiment, but it is not easy matter, so that's why. Thank you. I'm of course a g of x function. The perturbation function is quite a fictitious function. It is not real function. I mean fictitious because it looks like a very strange way. The form is actually we have to understand why is the steady-state probability density function. So it is not easy to experimentally apply this kind of force to the experiment. It is not easy matter. Is there any other perturbation function g that makes some inequality or entropy production? I cannot say this is a unique way because I have to not try all possible perturbation force. But I think it is almost a unique way to derive. Because we know what is the expression for, I mean we know what is the expression of entropy production rate. So to derive such expression, then I think the possible trial function, I think it should be very limited. So, yeah. Thank you. Okay, so is there any other question? Okay, so this is a derivation of TOR over the influence of dynamics. And let's turn to then how to derive TOR for Markov-Champ process. But strategy is simply same. So we will use this time scaling perturbation. So if we make this time scaling perturbation in the steady-state, then this is automatically holds. So the first relation is automatically true. So the remaining thing is that by using this scaling perturbation, when we calculate this feature information, which is really gives this total entropy production. So the remaining point we have to check is this second relation. Okay, so this is original dynamics. This is original master equation. So here let's make it this time scaling perturbation. So instead of T, we put at 1 plus epsilon T. 1 plus epsilon T here and there. And this 1 plus epsilon originate from this T. So in the steady-state, this simply becomes 0. And this one actually does not depends on time in the steady-state. So we can write in this way. And this 1 plus epsilon terms come into this summation. So we can write in this way. And here let's... Okay, so this is the same one. Same one. But here let's subtract this value and add this value. And you see that the denominators are same. And the numerator are also same. So it means that minus something plus something. And actually this is a 0. So here we define this function as eta xy. Then because this yx and xy is reversed. So if it is eta xy, then it is eta yx. So now add this term and this term. Then this term, addition of this term gives this first part. And addition of this term, this term gives the second part. Okay, so I mean these two master equations are equivalent to one. But it looks like different, but they are equivalent actually. So we can write, in general way, we can write this master equation in this way. So here now transition rate depends on epsilon. So in this case, transition rate is this one. And in this case transition rate is this one. So in general way we can write the transition rate in this way. So here when delta equal to 0, actually it becomes this one. And when delta equal to 1, it becomes this one. Probably you are wondering why they are same, but we have this kind of strange things. But you can see why later. So it means that this time-standing perturbation is equivalent to this perturb dynamics. Here the transition matrix is perturbed by the factor of epsilon. Okay, so with this perturbed transition matrix, now let's evaluate this Fisher information. So this is the definition of Fisher information. And then to evaluate this Fisher information, we have to write the path probability of this Markov-Jumper process. And so the change is nothing but this transition matrix depends on epsilon. And here now let's look at this too. I mean the whole staying probability. And we can write this whole staying probability in this way. So here x of t denotes the state at time t. So this delta function picks the state at time t. So we can write this whole staying probability in this way. And this transition part. So transition part can be written in this way. So actually this term comes from this logarithmic function. And there is some m dot x, y, m x, y dot here. So m x, y dot means that actually here t x, y means that time at which transition from y to x occurs. So this delta function picks the time at which some transition occurs. So it only counts when the transition occurs. So actually these two expressions are same. So anyway we can rewrite this path probability by using this delta function and this m dot function. And the thing is that the average value of this delta function is equal to the probability. And this m dot x, y, this picks the transition time. So actually this average value is equal to this value. Number of transitions per unit time. Okay so I mean anyway this is an expression rewritten of the previously known path probability. Okay so now let's calculate this Fisher information. And because actually this term does not have any epsilon dependence so we can ignore this term because there is a second derivative of epsilon here. So the remaining part is given in that way. So second derivative applies to this first part and the second derivative applies to this second part. And we know that from the definition of this r epsilon this is a linear function in epsilon. So it means that this second derivative of this actually vanishes. So we don't need to care about this. So the only the remaining term is this one, second term. So if we take this second derivative, if we calculate this second derivative, then the result looks like this. And for the same reason this second derivative of r epsilon vanishes and the remaining term is only this term. And we can explicitly calculate this part because this is I mean linear in epsilon. This is the first derivative of epsilon which gives only this part. Okay so this is the result. And this term is cancelled out. So only the remaining part is this one. Okay so we obtain the expression for Fisher information. So when delta equal to zero, then this term is neglected and only this term remains.


Good morning, I guess yep, I can get away with still saying good morning. So thank you everybody for coming This is the fourth session in the Cisco sponsored track room. My name is Gary. I'm the host and emcee for the day We do have one more session after lunch today so I hope you can come back and join us then with Lou Tucker and Steven and some folks from SAP but we're gonna get going right now with what you're all here for and to hear from Jerome and Ian two of our most senior engineers at Cisco on Superfast network performance with Thank with VPP. Thank you. It's been a very long week. So With that I'll turn it over to them. Thank you Oh one quick thing as you all came in this morning or today. You should have gotten a little card We are doing a drawing at the end of the session For a very very nice Phillips Bluetooth Speaker so fill out the cards will grab the cards at the end and do the drawing Okay Okay, so Thank you for coming. We are Thursday and it just before lunch. So I'm not sure this is the best time slot But we'll try to be as quick as possible. But if you have any questions, I will have a long Q&A session afterward So my name is Jerome and I'm managing this networking dash VPP project And I'm working with Yann as well as other engineers including the guy by the name of an aving joy in the room here today Before we start who in the room is already familiar with VPP and FD IO Okay, so that's a good 40% so what I'll do is I'll tell you a few words about what is VPP and why we are trying to integrate that into Open stack and then I'll end over to Yann will explain you the print design principles and the architecture of the software So VPP to make it simple is a is an extremely fast virtual switch It's a data plane that is extremely fast and can process huge amount of packet per second you will get more details afterwards and That can be used to for east-west communications in inside a VM between different VMs For North-South communication when it goes to the physical world So that's based on DP DK and that The way it works is that the reason why it's called VPP It's vector packet processing because instead of working packet per packet It works vectors of packets by vectors of packets and you will get more detailed afterwards And what we are trying to do in the frame of this project called networking VPP is to make sure that this virtual switch is properly integrated into OpenStack and that anyone using VMs VNF can take benefit of that We want it to be simple Robust and production great. So we are not looking at the fancy feature to start We really want to have something production grade to be used in the cloud or for VNF in production So it's not a toy. It's something that is designed to be in to go in real life in production and in order to do that we want to comply and to adhere with all the regular OpenStack rules including blueprints code reviews Mailing list blah blah blah. So we have an open community working on that And if some of you are interested to work with us, of course, you are more than welcome so Yann will go in more details on this topic, but Just Just to give you a rough idea of what we are trying to do We want to do a software which is highly scalable not only in terms of packet per second But also in terms of number of nodes we can deploy We want something really simple to operate and to debug because some of you might have some experience with a virtual switch and That's not always super easy to operate and to debug so we want we want that to be simple to operate and Of course because it has to be deployed in real life environment High availability is not an option. This is something you need to get by design by default and Will explain you how we did that So in terms of efficient so what what we did to reach these goals is all the communications are rest-based and jason-based you'll see that there is a at CD software which is used as a cornerstone of this project and Everything is based on asynchronous communication. So when you create a port you give the order to create a port So this is a desired state and then later on when the port is actually created you receive this information So all the things you do when you want to do Async communications, which is really useful for distributed systems and in terms of High availability we use a neutron journaling system as well as key value store based on at CD At CD currently is not being being used in cluster mode But eventually it will be used in cluster mode to provide, you know, what the best availability that you can think about and last but at least code is small and Easy to understand and debug so today we are speaking about a new project very interesting project But at the end of the day if you have a look at the number of lines of code We are speaking about it's between 2000 to 3000 lines of C code So that's extremely complex compact and the reason for that is because of this architecture as communications and because it's we started from scratch So that's that's another advantage Okay, so for those who are not familiar with with VPP So VPP vector packet processing is one of the project of the product of the FDIO project so Fido FD.io Fido is a project under Linux foundation and It's a multi-project project. So you have VPP, which is this fast data plane Virtual switch virtual water, but then you have other projects such as a CCIT which does Continuous integration you have t-rex, which is a packet generator. So we have many many projects Fido is a is a project being driven by you know Intel Cisco Red Hat, Ericsson, so different companies different level of involvement and But the but the VPP software itself has been designed since 2002 So it's not something new, but it has been open sourced last February So the fact it's open source is really new, but the software is is rock solid now So again, what makes it different from other V-switch is that it's I think there are several things One is that it's extremely fast and it's really important to be extremely fast because we now want to deal with multiple Tengig links and multiple Tengig links means that you want to have Like 150 cycle per clock cycle per packet, which is not a lot Right and you cannot achieve that if for each and every packet you need to fetch states from memory So what we do here is we process vectors of packets per vectors of packets so we get a bunch of packets on which we apply similar processing and That makes us able to warm the cache with the first packet and then to process all the remaining packets Which are building to the same vector, but without the penalty of cache effect And that's a very important way to a very very very significant speed of speed up in terms of of performance that is based on DPDK, but Nothing you know forbids, you know as to use all their interface in the future, but today it's fully user space only DPDK and That gives us very very high performance very very high performance for physical Interface, but also very very high performance for the host user kind of connections between virtual machines and The v-switch so that makes perfect sense both for cloud and for NAV kind of environment Recent results, you know performance results we did you know in based on 1609 Which was released in september as you can guess Typical processing on two threads two hyper threads one core It's like 10 gigabit per second So in this setup as you can see on the slide we inject five Million packet per second And then these five million packet per second are going to a vnf or to a vm Using vios to the interface they are sent back so all in all it's five plus five Which is 10 million packet per second and that is only on one core and that scales really Linearly with the number of cores So this is something you can reproduce if you just you just have to go To download the code and and do this setup. That's really easy to to reproduce And we are increasing that with multicu support for for next releases Anything to add on these topics? Now I think that's good What are the main components? So the design is actually very simple We have developed a mechanical driver which sits on a neutral server And this neutral this mechanical server have to populate States into an hcd key value store right to do that we use This json rest apis and then we have all the compute nodes while watching state changes on this on this hcd key value store So as soon as the new desired state Is available for a given node this node is a wake wakes up and retrieve the states And populate that into vpp So vpp is a data plane only and it's not a control plane. So we have a python api That use a shared memory with vpp and that's that sits on the agent side and the compute node side So we retrieve all these states populate that into Into vpp and once the port is created we give this information back to hcd and that is given back to neutron So that's that's the way it's done And one of the benefits of this architecture is that it's extremely simple to debug If you are familiar with hcd you can do a simple hcd watch Cli command and you will see all the port created all the port deleted Value states that's extremely easy to analyze So this is The architecture and jan perhaps you can Commend that so um what we're trying to do here and and another reason for using etcd is what we're trying to Implement is a model of distributed state not distributed commands So instead of telling each of the agents what it must do next Which means you've got a stream of commands running out to the agent and if the agent gets behind then the backlog increases phenomenally Then etcd stores the state it should be in if it ever loses track or if it gets behind it can always catch up to the current state So our aim here is that neutron When you run a neutron command as you would normally as you're as you're bringing a virtual machine up Or even when nova does it for you as it's starting a virtual machine Then we store that command in the neutron database the same as usual So that you can retrieve it so that neutrons got consistent state And then we um we put it in a journal within the database at the same time Which gives us the ability to do it synchronously within a single commit We're guaranteed that the operation succeeds or fails as a whole Which means that you're never getting into awkward states if there's any kind of system failure Um a background thread then takes that and pushes it into etcd We use a background thread so that the user space calls are always very fast when when you tell neutron to do something You're saying I would like you to create a port neutron is always effectively saying at that point I'll see about that at some point in the future And then the background thread is part of actually getting the job done But it doesn't make the user wait to hear that the operation has succeeded And then from the back end the agents for efficiency sake Watch etcd for state changes and individually the state changes every time they see something new that they've got to do They go and tell vpp to get that sorted If as I say they run out of history and and etcd has a limited record of changes If they do run out of history, they don't get into the backlog state But they instead say well, I'll just catch up with what's current I'll go and find the whole thing and I will resynchronize everything that I'm doing If the agents die or if vpp dies then they go to the state and they say well I know what I'm supposed to be doing right now. I will go and make that happen So the architecture is designed so that failures in any of these systems or intermittent network connectivity problems don't cause you issues And we can all wish that failures don't happen But if you're running a system that's got tens or hundreds of servers And you're running it for weeks and months on end and if in the case of nfe where you're a service provider And your main source of income is absolutely dependent on this system working flawlessly Then you have to tolerate failures and this is what we've designed it to do Yeah, so there's also a little bit to talk about here in terms of the port binding workflow So port binding is when nova and neutron together try and come to an arrangement about where traffic is dropped off From a network so that vp so that the virtual machine can then go and pick it up and there's a whole bunch of Slightly asynchronous calls that go on between nova and neutron to kind of negotiate that settlement between the two of them What we do is again slightly different from from what other mechanism drivers within neutron do we Propagate that information through etcd whenever neutron nova says I would like to bind this port to that host over there So we tell you know We put something in the in the etcd state saying that that host is the place that's now where this this traffic needs to drop And then the agent on that host will go and try and Land the information in a in a common point basically a v-host user interface This is the shared memory fast interface that we use for virtual machines to use a space switches And ultimately when it's done that it will notify back to the neutron agent again through etcd what it's done And neutron will tell nova asynchronously nova gets the virtual machine all prepared and paused But it doesn't actually start the virtual machine until that binding is completed So that when the virtual machine actually starts to run It's got working networking and it can actually get its dhcp addresses Because obviously that's pretty much the first thing that a virtual machine will do Having done it this way we the change is basically that we are completely Certain from top to bottom neutron the server but also neutron the agent that the whole system is set up And that message that's being given to nova says we are absolutely ready and we're totally certain at this point And it's also fault tolerant nova here in this particular instance Will time out if neutron doesn't get its job done in time So the fault tolerance is is the same as you usually find with mechanism drivers So our key value store here we've used etcd. It's fairly common key value store It's a synchronous key value store. That's kind of important So when you've written something then any reader from that point on will get the information you've written And as I say it keeps notifications and it will send notifications If you're watching a subset of the keys within etcd It will tell you when any of those keys have changed so that you can do updates on demand If you want to but if you fail to get the update if you find the connection has dropped Or if you think you've dropped out of sync, then you can always go and get the entire state from etcd As drone was saying debugging is this was a side effect. We didn't expect this We're so happy that it works like this If you've ever tried to work out what ovs is thinking when you're debugging neutron and why it's configured the way it's configured Horrible don't go there. It's a miserable experience There are about four different ways that ovs gets its properties plus all the ip tables not pleasant In this instance etcd contains a breakdown of what you're trying to configure what what what you want the Vpp to be doing and you can simply compare the vpp state has a nice little cli So you can always look at it and see what it's doing with the state that you're seeing in etcd And you can see whether it's sensible or whether you've got a bug which is great And in fact one of the things we're looking at doing there is actually adding a few pieces of Debugging checking code at some point in the future so that we can given that the information is there for the administrator to check We'll just write a script to pull both and compare to see whether or not everything is working properly Or whether there's been some sort of fault And within the etc data structure we basically we've set the the data structure up in a slightly interesting way So there's an outbound set of keys that are always written by the neutron server in the center of this And there's an inbound set of keys that are only ever written by one individual agent So you haven't got agents trampling over each other changing things that each other are You know information being put into a key from six agents at the same time And you haven't got the server in the agent fighting It's always totally clear where any bit of information has come from when you look into that data still This is admittedly a horrendous I should blame Jerome for this particular slide a fairly horrible Example of what comes back from etcd looks pretty nasty But the key bit in the middle there is the highlighted section And that should look fairly familiar if you've ever seen what a port looks like in neutron It's a whole bunch of settings that you find on a port that describe its MAC address and how it's bound And a whole bunch of bits and pieces And that's basically our message of communication. We just trimmed down the port to the bare essentials We stick it into etcd and that allows the distribution to go on And a little bit more there Perhaps slightly nicer formatting, but you can see there there's nothing terribly surprising in this It's just boring old neutron information that describes how the network is working and where the packet needs to be landed So we have a list of supported features and as Jerome was saying the aim here was to Perhaps I keep saying this to people people will always talk about building insecurity from the ground up You must remember security is the first thing that you build into an application I wish they would remember that when we talked about robustness as well If you want something to be robust and highly available and fault tolerant It has to be the first thing you design into the application. That's what we've done here So we set ourselves a sort of timeline to stage Firstly a framework that would do what we wanted and then we brought in features one by one So so far what we've got to is it works with flat networks and with VLAN networks It works with the host user ports for virtual machines But it's also capable of connecting to the namespaces that that neutron already provides for DHCP for metadata and for routing So all of the DHCP metadata and routing features exist today and you get them just as you would expect The db journaling bit that was added a little after we first got it in that was the synchronization to make sure that Etcd is always in sync with the database and never loses a change So we added that just a step or two down the road And it actually was surprisingly simple to do I have to say We've supported the ha scenarios we can restart the agents we can restart the drivers We can restart the server we can restart etcd very carefully because it's redundant So you tend to want to restart it one one node at a time, but we can And we've also managed to get this Deployable obviously the first thing we did for our own benefit honestly was to get a dev stack plugin working You can certainly go and download that Download vpp using dev stack today and it will deploy a dev stack with vpp running and configured in it We've also got it working in the opnfv apex installer Which is a triple o based installer that that sysco and red hack collaborate on And that's working today as well And we've looked at integrating it a little bit with red hats packages Most of what we're doing here is either in red hats package repositories So etcd is easy to get hold of or you can download pre-built packages like rpms for vpp components So again, you know, it's usable today in reasonable systems for test I would suggest not production just yet, but you can certainly get a handle on it and see what you think So the the next bits and pieces that we're planning on dealing with is It might seem odd that we didn't implement security groups up front There's two reasons for that one is as it happens if you're doing nfe Then you often find that security groups are the thing that you first turn off Because they slow everything down and if you're looking for the absolute flat out straight line performance Then then for nfe, um, it's not what you want So so we put that on one side for the time being and said we'll do that Second, but also security groups and anti-spoofing require a little bit more work on the firewalling functionality of vpp So we've talked to the vpp team. They're actually implementing some apis for us right now and we hope to have That implementation Released with our second round of releases. Um, which should come with the next release of vpp in january next year And we're also oddly a strange service to think of but we're looking at the tap as a service functionality Because again, if you're working with nfe applications and networking high performance applications Being able to take a copy of the traffic that's coming in is absolutely critical to being able to debug what's just gone on With your application. It's not the infrastructure is working But the application is not working or the other way round How can you tell the only way to tell is to have decent debugging functionality? And obviously, um, and I would admit I'd be the first to admit that that testing kind of came Slightly late in the day with what we're doing, but we're working on our enhanced test plan More in the way of unit tests for the code which fortunately is very small So the unit test should equally be very small But also more importantly integration and feature tests to demonstrate that in a deployed system It's actually doing its job And then there's a couple of things we've been talking about in the longer term One is telemetry So it's really useful to be able to get out statistics from your virtual switches To see what's happening with your network and where the network load is And sometimes even down from just port load to individual load on certain kinds of flow or certain ports on a virtual machine There is quite a lot of that already in vpp So we're looking at how we can pull that data out and make it consumable by open stat users And also we this was actually a discussion yesterday morning How we can make vxlan support work vxlan is supported within vpp But configuring vxlan is very different to configuring vlan's So that's something we're planning on down the road And oh yes, and I should mention in fact colorado here colorado is the release for opnfee of their integrated system So they have a number of combinations of installer and components that together deliver an nfee platform An nfvi as it's known or a vim In in nfee world And we're actually trying to get this into the well We have this in the colorado automated testing through that apex deployment system now And we're seeing whether we can get this into the next the next release of the colorado version of the opnfee platform So hopefully in the next few days you'll hear how that's going and I'll be able to give you good news on that subject But yes, if you're interested in looking at this then obviously we're now an open stack project So you can find our git repository under the open stack repository You will find obviously it was on review.openstack.org You're welcome to review our patches and tell us that we can't write code Which people do to me a little more often than I would like but such is life And obviously you can submit patches as well and we'll give them a good hearing We're always pleased to see people using this and trying to make it better And there's a launchpad there if you use it and you find that something's not working Then we would be we would be happy that you file as a bug bugs are good. We like bugs We don't like bugs, but we like having bug reports at least And even blueprints as well. So again, please come and join us participate in the project Okay So That's Okay, I'm live. We've got some times for some time for questions Or you guys did oh, okay, here we go. All right Yeah, I just had a question about there's also an effort to build an open daylight based control for vpp Using gbp currently and just wondered if what you know, how do you compare contrast the two? It's a good question. And I've had that question before I I totally understand that it's confusing that we would do this twice over But there's a logic to this This model here is very simple as I said two and a half thousand lines of code and it gets your functionality working This model here will get you Virtual machines connected to each other into the world But there are a lot of things you can do with networking. There are a lot of apis we could potentially create To do things like service chaining connectivity to mpls connectivity to lttpv3 or whatever other tunneling solutions that You find in service providers and again if you focus on the service provider world No two service provider networks use the same technologies and are configured in quite the same way So we know full well. There's a lot of variety out there. This is the simple version This is the one that's intended to be useful to most people Open daylight is the complex version which lets us add more shiny features Now it may be at some point in the future You'll want one of those shiny features and you will go and use open daylight But at least in the meantime you can use this and see how it works for you And it may be that this will serve for all of your needs as well This is actually perfectly fine for for typical enterprise and and similar workloads Okay, well two two for the price of one. Okay Uh, have you guys done any kind of benchmarking with mlto vs versus networking We've done some um, I'm going to look pointedly in that direction But I don't think Alex given me some test results on this yet We we've certainly done tests of ovs alone against vpp alone And and this is a point um, Jerome made earlier We always talk about packets per second when we're talking about this because packets per second are interesting And simple overall bandwidth is kind of boring honestly I can fill a 10 gig link and I can get ovs to forward the traffic of a 10 gig link and that will work just fine And when I do that, I'll be using something like a perf and I perf will be sending 1500 byte packets down the link And the packet rate will be down here. I mean, it'll be well under Well, it's it's a few hundred thousand packets per second that you're looking at Vpp is good for 14.8 million packets per second on that same link Which is a 64 byte packet filling a 10 gig link So the difference here is Not can I fill 10 gigs, but how many packets per second can I get down that link now when we've tested the two things alone And I believe we've published this information again. You probably want to check on the opnfv website We found that we can fill 10 gig link with 64 byte packets You need to be very careful about how you configure things and I'd like to know how close we are to that with vpp With networking vpp's config, but we can fill a link quite easily with with vpp Whereas if you start Shrinking the packet size of ovs you find that you're down to about 1200 bytes and you're already starting to see packet losses When you get to 64 bytes, there's a dribble of traffic going down that link So so there is a distinct performance difference between ovs and vpp A slightly more comparable version with ovs dpdk with vpp and I know we have some results on that as well It's certainly better than stock ovs in the kernel, but it's still not terribly high performance by comparison In addition to that, I'd like to add a couple of things One we we run a test that did by the eantc to compare ovs dpdk Versus vpp. So that's something which is Available and then we are also publishing regular results because Under fidot you have this cc project I was speaking about, you know, and this project does for every commit, you know, you do Performance testing benchmarking blah blah blah, so you can get all these results And last point on this topic we had In Seattle there were during last odl summit There was a presentation about what is the typical throughput you can get at zero packet loss That's very important because you can have different Numbers in terms of throughput If you accept like a dot two percent packet loss, that's very very different from zero packet loss, right? So you have full Benchmark with this level of details. It's not a comparison. It's absolute result in that case. Thanks guys Any We're there. Okay anybody else any other questions? That means you guys did a really good job. Thank you, and you are thanks again to ian and jerome