I am telling you that the we have first representation of reactions species j equals 1 to n let us say i equals 1 to r this nu ij is called stoichiometric matrix we said the significance of nu ij is that dn ij which is change in moles of j due to progress of reaction i as the reaction progress as the change is or that okay we will just keep this as and d lambda j will be the progress variable and d lambda i if you like for reaction i then we have that dn ij by d lambda i no by for each reaction i the change in the number of moles of j is proportional to its stoichiometric number by convention the stoichiometric coefficients are positive these are remember these are positive for products in negative if we look at equilibrium I have a box in which many reactions take place if there is face equilibrium I will consider the homogeneous reaction and say equilibrium of homogeneous reactions actually it does not have to be homogeneous it can be heterogeneous but equilibrium thermodynamic equilibrium means face equilibrium and reaction equilibrium so I can pretend that face equilibrium first occurred and all the reactions occur in one phase so formulation would really be the same then I need dg less than or equal to 0 at equilibrium at constant Tn so at equilibrium actually you have dg is equal to 0 is the criterion of equilibrium but dg is equal to minus stt plus vdp plus sum over mu i dn i now this is 0 this is 0 so you get sum over i mu i dn i is equal to 0 make this j actually it makes no difference but dn j is the total change the number of moles of j this is equal to sum over i dn i j remember we defined dn i j is change in moles of j due to progressive reaction i if I sum over all the reactions I get the total change in moles of j but dn i j is equal to nu i j times d lambda i so if you call this we call this equation 1 this is not a mathematical equation this one is only a representation this is 2 this is 3 and finally 4 substitute this back you get sum over j of mu j nu i j into i is where they get to from sum over i this is our species you made this j oh I am sorry d this is correct dn i j is simply there is no sum over i there is no sum over j wondering what happened dn i j is simply nu i j times d lambda i so nu i j times d lambda i and then there is a sum over i this is over all species this is over all reactions like j equals 1 i equals 1 to r now this quantity this is sum over i d lambda sum over j this is dg this is a criterion of equilibrium now if all the if our reactions are independent then thermodynamics demands that dg should be 0 regardless of the progress of any one reaction then these quantities are linearly independent and you know from linear algebra that a certain number of quantities certain quantities a set of quantities is linearly independent and the sum of a coefficient times this quantity will be equal to 0 only if each of the coefficients is 0 so this implies sum over j equals 1 to n of nu i j mu j should be equal to 0 for all i so this is the final criterion of equilibrium 5 it is important that the reactions be independent then the progress of each reaction is independent this summation sum over j sum over i I am just switching the sums and first summing over j and then summing over i as long as the left hand side is finite mu j nu i j will come there yeah first you have to do this summation then you have to do this that is all when I finish this summation I will have a function of i that multiplied by d lambda i but the argument next step says that d lambda i are independent you these linear you must have done vector algebra you have ci vi is equal to 0 if ci vi is equal to 0 and the set vi are independent are linearly independent this implies ci is equal to 0 for all i this is your usual vector decomposition also you decompose a vector in three dimensions into three components the vector is 0 if and only if every each of the components is 0 it is the same way so you get this result that if the reactions are independent if our reactions are independent each has a progress variable that is independent of others thermodynamics demands that the coefficient should be 0 because if it was not 0 then I choose to progress that reaction for which it is not 0 and dg so if dg changes then the original situation could not have been at equilibrium so if the system is at equilibrium then this is true now remember the representation of the reaction was nu ij a j is equal to 0 this is representation this is not an algebraic equation but if you have this representation then the equilibrium criterion is simply obtained by replacing a j by the chemical potential of j and of course in this case as I said nu ij is also is equal to 0 for all j not participating in reaction i in systems in large systems with many reactions these matrices this nu ij matrix is often called as parse matrix because you may have 22 components each of the reactions may involve only 2 or 3 so most of the nu ij's will be 0 and in large number of computational techniques you deal with parse matrices you need special techniques when computationally sparse matrices are difficult to deal with so this is the criterion equilibrium but this is also equal to remember that this is this is defined as ?g for any reaction i is defined as sum over j equals 1 nu ij nu j this is for reaction i the free energy change is exactly this this is because you write for example let us say you are burning carbon let us say you get half O2 you get CO and write another reaction CO plus half O2 you can use CO2 so in this nomenclature we just rewrite this as CO minus C minus half O2 is equal to 0 and you write CO2 minus CO minus half O2 is equal to 0 so this could be component 1 this could be component 2 this is 3 and this is 4 so the number of species at equilibrium is 4 and you write the coefficient matrix nu ij is the same as for component 1 in the first reaction it is minus 1 for component 2 it is minus half for component 3 it is plus 1 for component 4 it is 0 for the second reaction component 1 is absent component 2 is oxygen which is minus half for component 3 which is CO it is minus 1 for component 4 CO2 so these are linear algebraic equations the number of independent equations is simply the rank of this matrix so in this case the rank of this matrix rank of nu ij is equal to number of independent reactions in this case it is 2 clearly the rank is of course the largest square matrix whose determinant is non-zero you could have written C plus O2 equal to CO2 but that would not have been an independent reaction because you can produce it by combination of the other two so this ?g for example for this reaction ?g would be let me write the change in free energy let us write down the moles let us say at equilibrium I will write these out equal to CO and CO plus half let us say I start with one mole of carbon and one mole of oxygen is initially this is what I will write here initial carbon is carbon suppose I should write it separately species initial in moles at equilibrium you have carbon oxygen carbon monoxide and carbon dioxide initial moles 1 1 0 0 this is arbitrary you can start with any at equilibrium if this is 1 minus x 1 minus x 2 let us say at equilibrium let x 1 x 1 be equal to moles CO produce in reaction 1 and x 2 is equal to moles CO2 produce in reaction 2 so oxygen is consumed in both its mine 1 minus x 1 by 2 minus x 2 by 2 carbon monoxide x 1 is produced minus x 2 then carbon dioxide is x 2 I have chosen a bad example because I have chosen a heterogeneous system carbon started with 1 minus x 1 that is all x 2 sorry thanks only only in the first reaction 1 minus x 1 is the value at equilibrium. So I have criteria of equilibrium I have first reaction which is I call this component 1 this is 1 this is 2 this is 3 and this is 4 okay for the first reaction I have carbon monoxide for carbon monoxide the coefficient is 1 in the first reaction so µ of carbon dioxide minus µ of carbon minus half the chemical potential of oxygen is equal to 0 the second reaction it is µ 4 minus these are numbers sorry µ carbon is 2 is it now carbon is 1 and oxygen is 2 this is µ 4 minus µ 3 minus half µ 2 is equal to 0 where does it gives you an equation go or is it still around you cannot get rid of equations and you do not like them take a guess is it still around or have we lost it it is around right what the gives you an equation still demands is whether there is reaction or not sort of relentless you all µ and µ 2 µ 3 etc have to satisfy the gives you an equation. So basically all of these will have to be functions of composition such that the gives you an equation is satisfied now suppose in this particular case I happen to have a solid and a gas phase so this is carbon here I have CO2 CO and oxygen right so I have two phases so I have to write down the models in this phase I have to write the Gibbs regime equation for components 2 3 and 4 component 1 is pure solid. So I have first of all µ 1 is equal to µ 1 pure I could write an equation for it in terms of µ 1 pure being in equilibrium with its vapor phase µ 1 solid is equal to µ 1 vapor and I could include µ 1 here but I will write it as it is I will tell you why the others µ i is equal to µ i all of the others are in the gas phase so you will get µ i 0 plus RT ln P yi times Vi now i equals to 3 and 4 in the reaction equilibrium context all you do is separate things the things that depend on composition and other things that depend on temperature and pressure then if possible you separate out also the pressure dependence here µ i 0 is a function only of temperature µ 1 pure is a function of temperature and pressure but you know the condensed phase the pressure dependence of the chemical potential is very weak so this is approximately equal to µ 1 pure at T and P equal to 1 means it is independent of pressure that is the only important point that is effectively function of temperature alone I am going to write this I am interested in the composition at I may be burning coal and all I want to make sure is for example there is no carbon monoxide because I want the fellow is burning the coal to survive let us say I want carbon monoxide below a certain level or I will set it equal to 0 at equilibrium so I am asking I want to solve these equilibrium equations for the compositions at equilibrium and I can put constraints on it as a designer so if I am writing these out RT ln P yi contains the composition dependence I will separate that out and get log of P 3 y 3 P 3 µ 1 is effectively independent of this thing I will take the µs to the other side µ 0 the functions of temperature to the other side µ 1 will all automatically come to the side half µ 2 0 minus or I will take the ratio P 2 y 2 P 2 or half this is your first equation all these components are gas phases so I will get log of P 4 y 4 and P 4 divided by P 3 y 3 P 3 or there is no P 4 sorry no P 3 here is just P equal to these things will come to that side will become µ 3 0 plus half µ 2 0 µ 4 0 because this quantity is called delta G we denote this delta G 0 for reaction 1 I put approximately because µ 1 is not µ 1 0 µ 1 is chemical potential of component 1 as a solid at the temperature and pressure of the system it is also the same at pressure equal to 1 but it is not in the gas phase so the superscript 0 does not apply to that but delta G 1 0 by convention is simply the standard free energy change of the reaction with each of the components in its standard state so carbon will be in the solid state the others will be in the gaseous state in this quantity the only thing that matters is that this quantity is not a function of pressure it is a function of temperature low effectively a function of temperature low now for this particular system there is a coupling the mole fractions depend on both the reactions so let me write down the mole fractions as far as gas phase is concerned this is not in the gas phase this is solid all these three are in the gas phase the mole fractions are determined only by these components so total moles in the gas phase at equilibrium is 1 plus x 1 by 2 and counting only gas phase moles incidentally there is a convention is delta G 0 you often put a prime there to indicate that one of the components is not in the gas phase nevertheless delta G is the function of temperature the prime is often left out these days the original convention was to indicate it you do put a prime on this ratio I will tell you in a minute so I can write down the mole fractions here the mole fractions are simply 1 minus x 1 by 2 minus x 2 by 2 divided by 1 plus x 1 by 2 minus these are being added x 2 minus x 2 by 2 is also there okay thanks and so on you write this as number of moles by total number of moles so I have this p y 3 y 3 is actually n 3 by mt n 3 by mt and this is n t is this total moles at equilibrium and each of these is ni so let me take this ratio I have p power half and I will combine these things I will write p power half I have y 3 which is n 3 by n total this is n 2 by n total to the power half I have n 2 to the power half times p 3 I will pull out the n t power half here incidentally that whole thing is actually fugacity of component 3 by fugacity of component 2 power half it is also equal to p power half by n t power half into all this various forms of it so also equal to p power half into y 3 by y 2 power half into p 3 by p 2 power half this is equal to on the right hand side exponential of minus delta g 1 0 by RT the second reaction is f 4 by f 3 f 2 power half this is equal to 1 by p power half number of moles is n t power half will go to the numerator into if you like n 4 by n 3 n 2 power half it is also equal to n t by p power half by y 3 by 2 power half this is equal to exponential of it is a lot of ways of writing the same equation because this ratio is a function of temperature alone so at a given temperature this is a constant this is given a name kf is the equilibrium constant in terms of fugacities ratio of this this is equal to equilibrium constant for reaction to gain return in terms of fugacity notice exactly product over I each of these actually becomes product over j of fj to the power no ij in reaction one I must put a prime here to indicate that this only involves gaseous components so if you take the first reaction c o and o 2 are involved f 3 to the power 1 times f 2 to the power minus half so that comes with the denominator so this is a very compact notation this is your kf prime so there should be a prime here to indicate that I do not include non-gaseous components in calculation this is valid even here so this is kfi the second case you do not have a solid component you directly have which formulation you use actually basically you need to solve for at equilibrium you are solving for x 1 x 2 the variables at equilibrium are x 1 x 2 pressure and temperature and you have two equations to solve for I get a certain value for x 1 – x 2 I could change the temperature so that x 1 – x 2 is 0 for example I may demand it is very difficult to ask in any reaction that x 1 – x 2 be exactly 0 but you can ask that it should be less than 10 to the power – 10 some number predetermined number then you can find the conditions under which this is true for example if I fix the temperature then the right hand side is completely fixed in practice what you do is carry out this reaction may be possible to carry out the reaction at one convenient pressure so you carry it out at some low pressure where the gas phase behaves ideally so all the fees are 0 for example because this ratio it is the form of the ratio this is f3 by f2 to the power half and there should be a power half here right n3 by n2 power half so this is called kn for example and this is called kfi and this is called ky to add to the fun you can take the p inside and call it kp you can define k is simply this symbol k sub i would should simply be k sub i of any donkey would be simply donkey to the power no ij for that that is all it is simply a symbolism the thing is depending on the kind of data you have available you can solve for different quantities fact reaction equilibrium is very neat you could have in fact had instead of k prime you could have used k itself except that you would then add to this reaction set carbon solid this is solid you will write to carbon gas and here you would have saturation pressure this mu 1 solid pure would be equal to mu 1 gas pure is equal to actually mu 1 gas would not be pure because once you have the gaseous phase carbon is going to be in combination with all of those so this will be mu 1 0 plus RT ln p y 1 p 1 but this y 1 would be so small that effectively if the other components do not affect it you would have mu 1 gas would have been mu 1 gas pure which would have been mu 1 0 plus RT ln p 1 saturation times p 1 saturation so this will be almost effectively p 1 saturation times p 1 saturation but the point is the whole treatment of reaction equilibrium depends on the fact that mu 1 0 is a function only of temperature but mu 1 0 plus RT ln p 1 saturation is also a function only of temperature the number does not matter because what you do here you have to get an experimental value for delta g 1 0 somehow you can get that from others experiments or by combining others experiments that is you can add and subtract reaction so that you get your desired reaction but this must come from measurements of these quantities for example if it comes from this reaction you have to measure y 3 y 2 and p at a sufficiently low pressure where p 3 p 2 or 1 if you do that then you get the quantity at one pressure it is not a function of pressure so you can use it at any other pressure the entire treatment of equilibrium again thermodynamics will always duck the issue of calculating mu 1 0 you do not know that number but you know what it is you know it is the chemical potential of pure 1 in the gas phase at temperature Tn pressure equal to 1 so if you can extract a number for that or number for a combination of mu 0's you do not need the individual mu 0's you can get a count get the value of combination if you do that then you can get it from the tables and use the fact that this is a function of temperature alone the whole idea is to separate everything into a function of temperature in fact I would always start with this formulation the nu ij you can never go wrong in the action equilibria if you start with this formulation and then throw in the models for the chemical potential depending on the phase in which they exist anything is you got used to doing this and this is already lot of algebra has been done for you so you can go ahead actually carried out I have done this for a multi component system I could have started off with one but you have already seen reaction equilibria very often you could have for example you could have a catalytic reaction in this case the different issue but you could have a more complicated reaction where you have a catalyst involved and very often the catalyst gets poison because its surface is occupied by coke by the carbon particles therefore it becomes ineffective then you can demand that the carbon deposition should be below a certain limit so you can have a set of reactions in which one of the constraints you throw in additionally is simply a design constraint so the constraints are the equilibrium equation thermodynamics imposes on you the equilibrium criteria for every independent reaction that you write then you may have a temperature constraint simply because the container you may be doing carrying out reaction here in the refineries for example one of the problems in the refineries is catalytic conversion one of the largest economic drivers of the whole thing depending on the market you may want to sell more diesel or you may want to sell more lubricating oil so you may in the distillation column itself you may have a whole separation so what you do is if you want more diesel you take this lubricating oil you will get the heavy components and you crack it in a catalytic cracker that is you heat it and decompose the higher molecular weight compounds into lower molecular weight compounds ultimately the diesel is only a lower molecular weight and you feed this back to the distillation column it comes out in the diesel stream in fact one of the big problems that they have is how do you get more diesel out of the system that is how do you get more naphtha how do you get more gasoline the other problem the present refineries as is the removal of sulphur how do you desulfurize because the oil we get is crude oil which contains a high sulphur component now if you do these cracking operations the catalyst is very effective initially but the carbon deposition makes the catalyst ineffective so you can ask under these conditions you can ask how can I make the carbon 0 equilibrium so I can ask that x1 should be equal to 1 this is a trivial problem it does not make sense here but if I write down a more complex system I can simply ask when does when should the carbon deposition on the catalyst be very small so the whole set of reaction equilibrium problem many of them are from denby that I have given you will go through some of these we will just ask various questions for example why does the you know basically the ores in an organic chemistry all the ores occur as a sulphides or oxides first question you can ask is why do they occur as sulphides or oxides you will find if you do a delta g calculation that is the lowest free energy form then you have to you can the other questions are when you roast zinc sulphide why do you get zinc oxide at what time what temperature do you get zinc oxide or if you use calcium limestone you get calcium oxide but is calcium oxide stable can you store it you will find after a while become calcium carbonate because even at the low 10 to the power minus 2 carbon dioxide in the atmosphere the equilibrium contains a significant amount of calcium carbonate you can calculate the equilibrium quantities of calcium carbonate calcium oxide so I think that is all the real interest the theory part of it is trivial but doing the calculations is has to do with the kind of data that you have.


Sounds great. Great. Welcome everyone. This is the Jenkins platform special interest group. It's June 18 2020. Thanks for joining us. Let's look at the at the agenda and be sure that we agree on topics and then we'll work through the agenda. So what we've got is we'll talk about open action items. Then we are delighted to have Rishabh with us to talk about his progress on the get plug-in performance improvement Google summer of code project. Interesting results, good results, and fascinating progress. So we'll give him as much time as he needs. Ask questions, record notes, etc. Then Alex and I are going to talk briefly about the adopt open JDK for Docker image change and work through what it means. Do some discussions briefly and we'll conclude there. Are there any other topics we should add to the agenda? No, I think that sounds good. Okay, great. All right. So let's do the review of action items. I have the action to switch the meeting URL to use the CDF Zoom account. I've done that. Rishabh, you will notice that you can share the screen without me having to do anything. That is one of the results switching to use F Zoom account. I have still had the open action item to open a Jenkins enhancement proposal for Docker operating system support. I apologize. Yes, I will get that done. We are in progress right now on the Docker build rework PR or a derivative of it. So that's encouraging. Oh, no, I take it back. The one I'm in progress on is the next one. So we still need to make some further progress on the Docker build rework PR because this one gives us a better structure. Alex, this is the Docker manifest and parallel build changes that I haven't done yet. Anything you'd like to report there? I've started to take a look at it. I'm running some local builds and things like that. So far it's looking good. The way I apparently have my Docker authentication set up is I'm getting a 401 at some point when it's trying to publish because my login is timing out. So I need to look into that. Okay. Now, and this is one that since I don't think I have authorization to do anything to the Docker hub account and I hope I don't, I should be able to do the evaluation as well and run it in some sort of local mode, a dry run mode. Yes. I think the dry run is still there. Okay. Great. All right. So this still needs review. There's lots to do in Docker image work. And the next was the Alpine image update PR and that one is PR 956 here. That one I would had started review on it last week, some additional review yesterday and further this morning. So, all right. Anything, Alex, you wanted to report otherwise on action items? Any other topics there? Okay. Rishabh, I think we're ready for you on the Git plug-in project. Okay. Thanks, Mark. Let's see. I need to stop sharing, don't I? Here we go. Yes. All right. All yours. Okay. Git plug-in performance improvement. As Mark mentioned, I'm the GSOC 2020 student. Rishabh is my name. So, the objective of the project, so it's a simple objective to improve the performance of Git plug-in and we've identified two ways to do it. The first is to use JMH, which is a framework, Java micro benchmarking harness, which provides us a safe environment to test benchmarks. So, we want to use benchmarks to identify known or unknown performance issues with the existing Git implementations we have, that is the CLI Git and the Java, pure Java implementation, JGit. So, this is the first objective. We have created, we have already created Git fetch benchmarks and we have some great insights from those benchmarks and we are going to implement those inside the Git plug-in. The second is to fix the existing performance issues we have in the Git plug-in and I'm going to discuss them further in the presentation. So, what have we done and what are we doing? The first thing we have done is to integrate JMH, the module, inside the Git client plug-in. So, basically what I've done is that I've added the benchmarks to the test module of the plug-in and now the benchmarks can run on cijengins.io, which gives us a wider selection of platforms where we can run the benchmarks and have a comprehensive result profile and how it looks. It's basically like this, a pipeline, a sample pipeline. So, we build the repository, then we check it, we run the benchmarks and then we have a JSON, which is a report which we can use to feed into a visualizer where we can see the benchmarks visually how things are going and how the operations are performing. So, this is the first thing we've done. After this, we've tried to fix the redundant fetch issue. So, what is the issue and how was it affecting the Git plug-in performance or is it affecting actually not was? So, the issue is very simple. The reason behind it is not simple, but it's... So, this is a freestyle project. I'm trying to check out a repository. So, as you can see here, I am trying to fetch the upstream changes from the repository once, which is expected, which is normal. But then again, I'm doing the same thing incrementally. So, this second fetch is redundant. And we've actually written test cases to understand and we've worked through some scenarios to understand if we're losing or breaking any use cases, if we're avoiding this second fetch. And we've come to the conclusion that though we have some interactive testing left right now, but still we have come to the conclusion that we can remove the second fetch, but we were not sure that if the removal of this second fetch is going to improve the performance of the Git plug-in, we did not have concrete numbers to show that, to support that hypothesis. If this is a hypothesis that removing the second fetch would actually improve the Git plug-in performance. So, for that, initially, I created a benchmark, four benchmarks actually, with Git fetch. The benchmarks were simple. The first benchmark is a single Git fetch with a narrow ref spec, which basically means that it is trying to fetch one single branch, which is the master branch. The second test was also a baseline single Git fetch test. But with a wider ref spec, that means all the branches. Then the third and fourth tests are double fetches with the same thing, narrow ref spec and a wider ref spec. So, before even fixing the issue, what we could see from the benchmarks was, as you can see, this is an interactive chart, as I progress with test one, test two, test three and test four, for each repository, the execution time increases when I'm moving towards test three and test four. Yes, Mark. Okay. So, what we're seeing here is, as you're iterating through the horizontal dot that moves across, that's telling me repository size. Is that what it is? So, test one is smaller? No, no. Tell me again, whether test one, two, three, four. Okay. Test one, test two, test three and test four are basically benchmarks. Test one and test two are benchmarks which calculate the execution time for a single Git fetch. Test three and test four are benchmarks, which calculate the execution time for double Git fetches. What you're seeing here, the x-axis is the repository size and the y-axis is the average execution time, which is increasing. I have four repositories. The size is for repo one, it's less than one MP. For the second one, it's five MP. For the third, it's 90. And for the fourth, it's 300. While we're iterating dots, it's basically running through the benchmarks I've done and the results of those benchmarks. So, what you can see here is, once I go from test one to test three, four, with repository three and four, you can see a remarkable increase in the execution time. And with repo one and two, there is not much of an increase, which kind of tells us, gives us a hint that with a small repository size, the incremental fetch would not add too much performance overhead. But with a larger repository size, there is a possible chance that the second fetch would be adding considerable performance overhead. So, this was kind of a theoretical experiment before fixing the fetch, which gave us a reason to actually remove the redundant fetch. Now, so how did we fix it? It was a simple fix. We added a Boolean to check once we're using this first fetch. We will not use the second fetch. And so, once we did that, are we seeing any change in performance? And to see that, I used profiling, profiling with the Java Flight Recorder on JDK 11. So, I ran the Jenkins war with the updated Git plugin with my changes, with my fix for the redundant fetch issue on the Jenkins war on JDK 11 with the JFR Java Flight Recorder, which is a profiling tool, comes with the JDK 11. It has very low performance overhead. So, what you can see here is, so I took two repositories. One is the Jenkins IO of 40 MB size. And one, I took the Samba Git repository, which is near about 300 MB. So, with these two repositories, I performed a simple checkout build. And here, what you can see is, without the fix for the smaller repository, there is not much of a difference. It's basically a 10 second difference. Without fix, it's two minutes, 55 seconds. And with the fix, it's two minutes, 46 seconds. So, there's a 10 second decrease in the execution time for the Git fetch calls for the plugin overall. And with the second, with a much larger repository, 300 MB size, there is approximately a two minute difference once we remove the second fetch, which is, I think, a considerable degree improvement in the performance. And if we increase the size of the repository, I'm sure we will have much larger differences. So, I've done this once or I've done this actually twice to confirm if is my performing results, profiling results correct or not. Maybe I need to do this with a wider variety of repository size to confirm that, okay, the redundant fetch is actually, the removal is actually improving the performance. But I think this is, yeah. Yes, Ma. That's gorgeous. So, this was with command line Git as the implementation or with JGit as the implementation? Command line Git. Excellent. Okay. So, then your Java flight recorder could not even, could not have any impact actually on the actual operations performed by CLI Git because it's a separate sub-process. So, that gives me even more confidence in your benchmark numbers and your measurements because, all right, there's not even the profiling from the Java flight recorder component basically pauses or has just sub-process or just process impact not. It can't touch the Git command line process that's a C program running entirety separate. Very nice. Excellent. Okay. So, at that large repository, that's really cool. So, for this issue, this is what we've done. So, the next step forward for us is to implement the performance improvement inside the plugins and to do that, we've figured out two steps right now. The first is to provide a compatibility switch to the users. The switch is going to be basically for users. We were assuming that once we, once we add the improvements, whatever we get from the benchmarks, we add those improvements inside the plugin, there might be cases where the user's functionality might be affected. The performance might be affected in some ways. We do not, we did not anticipate. So, we are providing a switch and it's going to look like this roughly I have it. It's in the configured system in Jenkins. Inside the Git plugin, you will have a checkbox which says, Revert Performance Improvement changes. So, once checked, it will revert to the old version of the Git plugin. So, this is the first step. The second step is to actually selectively switch between the implementation, that is the CLI Git or JGit. And from the benchmarks, we've found out that for Git fetch, we've found out that the size of repository is the biggest parameter which is affecting its performance. For an example, one insight we have is that for a repository less than five MB size, JGit is going to perform better than Git. And for a larger repository, Git is considerably performing way better. It's performing way better than JGit. So, if we want to implement that inside Git plugin, we need to estimate the repository size before creating the client. And so, we have, we've had some discussions on using heuristics like using Git LS remote to calculate the number of branches we have without cloning the repository inside the plugin. And we can use a rough estimate so that we know maybe 50 branches means a large repository or a lesser number means a smaller size could possibly use that. Second could be, another option could be to use existing REST APIs exposed by GitHub or GitLab where they provide the size of the repository. So, we could also do that. We're thinking how to do it currently. It's in process. And any suggestions on how to do this, how to estimate the repository size without cloning the repository first would be greatly appreciated. So, yeah, this is what we have done and what we plan to do. And any questions? Okay, so the first cycle then is the redundant fetch. You've already got a target where you say we know that there's a significant performance improvement to be gained here. And this redundancy affects both command line Git and JGit. So, it's redundant in both cases. It's not just one or the other. They're both doing this redundant fetch. Yes. Okay, so everybody gets benefit if no matter which implementation they chose or no matter which implementation we were to choose, they get benefit potentially large, potentially small, but they get benefit by this choice. Good. Okay.  No, no. Then on the benchmarking, the JMH-based benchmarking, the challenge there was that you've got to figure out before knowing, before having performed the full operation, you have to decide which implementation should actually execute the operation. So, before doing a fetch, you need to make a decision, shall we use CLI Git or shall we use JGit? Yes. Excellent. That's something you would have to figure out. Yes. Thank you very much. I think you've covered it well. Alex, did you have any questions you wanted to ask or let's see, has Oleg joined us? Is there, so with the JSON file that's created, is there any way of visualizing that in Jenkins itself as a way to kind of get a snapshot of these? Yes, Alex. There is an already existing JMH visualizer plugin. I haven't tested it. I haven't integrated it with the Jenkins instance, but I have to test it. But what it does is it consumes the JSON file and maybe I have just one second how it looks. I can show you. It's called JMH report and I cannot see how the visualization is made. I don't have it here, but there's a website that does the same thing. The plugin has basically used this website to do so. And I have a JMH JSON sample, just bear with me for a second. So this is how the visualizer visualizes the JSON. So once we add that plugin with our process of building the benchmarks, then we can see the results like this. That'd be awesome. Well, open an info issue to get that plugin installed on ci.jdc.io. Well, and another variant here is that one of the benefits of ci.jnk.io for Rishabh is that whereas he's got his Mac OS machine that he runs base checks on, ci.jnk.io provides him access to ARM64, to AMD64, to S390X, and to PowerPC. So he can safety check that, hey, are the benchmarks that we're using representative even across the different environments that ci.jnk.io represents? Oh, and Windows. Yes, I forget. There's one other platform there. We regularly have surprises with Windows. So yes. So then the second question I have is the integration of JMH stuff. Is that something that can be easily added to maybe the parent palm and optionally enabled for all plugins? Or is there a plan to do anything like that? So it's, oh, go ahead, Rishabh. I should not answer. This is yours. That's it. It's okay. So yes, there is an option that the Java, the Jenkins test harness already as it has the JMH harness inside the dependency. So once you have, so there's a version for the palm you need to have for Jenkins for that particular dependency. Once you have your grid, I don't remember the version. I think it's 2.5 for the Jenkins test harness. Once you have that, you don't need anything else. Just you can use the JMH benchmarks inside your plugin, in any plugin. That was done before. It was done in a previous GSOC project. Role strategy plugin. Okay. And does it require special tests? Or can, like, do you have to mark tests as part of the JMH data collection? I can actually show you how is a benchmark made. So, yes, like JUnit test, it's a little similar like that. So you annotate it with a benchmark annotation and you need a benchmark runner as well, which will identify the benchmarks and run it with some options you'll need.  So all of those options, they are included in the runner and that class. So how do you run that? So there's a profile in Maven profile, a JMH benchmark. So your benchmarks will run from that command and you'll have a generated JSON report from that command. And if you want to integrate the benchmarks, you want to run them on cijenkins.io, there's already a build step called run benchmarks. You just have to add that in your Jenkins file. And if you have benchmarks, they will be run on the CI Jenkins infrastructure. Cool, thank you. So Rishabh, would you show the Jenkins file? I'm enamored with how elegant the work from the role strategy plugin made it for this, but then we found some interesting issues that made me beg Rishabh to do some extensions. So in the get client plugin, he's going to show us what the Jenkins file looks like. Yeah, so this is the step which enables running the benchmarks. And we provide the name of the JSON we want as an output. So before, so what was happening before was that for every pull request for every branch, Mark was creating for testing, the benchmarks, they have a considerable time duration, they run for maybe an hour, the four hour depends on the kind of benchmarks we have, it takes time. So it's an unnecessary addition for people who do not want to test the performance maybe. So what we've done is a simple pattern matching that we will run the benchmarks only for the master branch or for any branch which is related to GSOC. So it's pretty simple. See, for me, Alex, the treat here was that somebody else had taken the time, the last year role strategy had taken the time to create this plug-in, this pipeline shared library called run benchmarks. And so Rishabh was able to without doing anything more than using run benchmarks, he gets it to execute now on the ci.jankens.io infrastructure. And then for my needs, he also gets the ability to put conditionals in there so we don't have to run it on any branches except specific targets. Very cool. So that's it from my side. Excellent work, Rishabh. Thanks very much. Thank you, thank you, and good luck with your ongoing efforts. I'm excited to be able to announce the release of the get plug-in that includes this. This will be so cool. Thank you, thank you. Thank you so much, Rishabh. Okay, so I should stop sharing my screen. Yeah, and I'm going to switch on screen share and let's take a look at the next topic. It was that we want to do share, look, share, screen, really, there, okay. I think the next topic is around the alpine image. Oh, I didn't take any notes, Rishabh. I'll insert some notes here on your results so that we've got them in the text as well. The recording of the meeting will be posted shortly after the meeting. Okay, should I add the notes while you're... Oh, that would be great if you're willing to do it. That would be wonderful. I'll do that. So adopt open JDK for Docker on alpine, Debian Buster, and Centos. So Alex, this was for me an excuse to just have a conversation with you to be sure that I've understood what we're expecting in PR956 and what we should be reviewing. Sure. So as far as I can tell, it's giving us a nice integrated new structure where we say we start with the JDK target is the first part of the directory name, then the operating system, and then the, what do you call it, the JVM optimization technology, whether a hotspot or open J9? Yes, that's correct. And so then what I was trying to decode was how does that map to tags in the repository? So for instance, I was looking for a Debian squeeze-based version of this and didn't detect one. Is that... Have I misunderstood? There's Buster. So there's Debian 10, Buster in the slim image, but there doesn't appear to be a Buster. Oh no, there is Buster, but somehow or other, I didn't find that label. Okay. So I, oh, this is JDK 11. I was looking for JDK 8 with Buster. So what you've done here is if using Java 11, I get the newest version of Debian, not the old version of Debian. Well, so this is just taking exactly what we had before and migrating the structure. So it doesn't, like a, the only image, new image I added, I believe, is the, is a JDK 11 alpine. Everything else is already existing. I see. Okay. So the directory structure is a refactoring in that sense, not an increment of adding a whole bunch of new distributions. Thanks. I had misunderstood that. Yeah. And so the directory structure was changed and then the base image was changed, such that it uses the adoptable JDK. Right. And so that's this here, which in particular with alpine is a real win for us because it gives us a newer version of the alpine base operating system, right? Instead of 3.9, it's now 3.11. Correct. It's whatever their latest is. Yeah, exactly. And it's, since the alpine project has stopped any work, or I guess it's the wrong way to say it, open JDK as a project is no longer publishing updates to the alpine image, right? It's stuck on Java 8, 212. As well as they are not publishing correct images or other architectures. Ah, okay. All right. That's a big problem too. Okay. So the transition here to adopt open JDK gives us a more current operating system base, a more current JDK, something that the project adopt open JDK is actually actively testing. And we get a better base image for ourselves. Excellent. Okay. Thank you. All right. So one of my pleas to the viewers of the video is this is a great excuse to do additional testing to help us as we evaluate this because it's, so the intent here is not to alter what we're delivering in the sense of base operating system version, but we are altering which operating system we're using, or which base image we're using. So instead of using open JDK, we'll use adopt open JDK. Correct. Got it. Okay. Excellent. Okay. Thank you for that help. That was what I needed. If there are any other topics we should review today. All right. Let's call an end to the meeting. Rishabh, thanks very much. Excellent summary, great results, and looking forward to further. Alex, thanks for your time as well. A recording of the meeting will be posted within about an hour, hour and a half. Thanks, Alex. Bye.


Additional uses of the method of excess and deficit come from the following idea. So again, let's take a look at that problem from their way. And remember, we solved it by setting up our contribution rates and finding the excess or deficit. And as a system of equations, if we multiplied our first equation by 16 and the second by 11, then added, we were able to get the ratio of the cost to the number of persons. And what this means is the following. If we divide the sum of the cross products by the sum of the excess and deficit, we get the cost per person. And the important idea here is that this cost per person is what would make the excess and deficit 0. So let's consider the following problem. A vine grows downward at 7 inches per day from the top of the 90-inch high wall, while a melon grows upward at 10 inches per day, how long before they meet. To solve this as an excess and deficit problem, we have to figure out what the contribution rates are and what the excess and deficit corresponds to. And so one way we can do that is if our contribution rates are the days that the plants have grown and our excess and deficit is based on the distance apart, then we might be able to construct the following table. So let's consider that we have the number of days we've grown, the length of the vine and the melon, and the distance apart. So if they both grow for one day, the vine has length 7 and the melon has height 10, and so they are 73 inches apart. If they grow for two days, that's 14 inches for the length of the vine and 20 inches for the height of the melon, they're 56 inches apart. And while we could solve this problem as a double deficit problem, let's go ahead and find an actual excess and deficit. So if we keep growing for a couple more days, we see that after five days, the two plants are 5 inches apart, while after six days, the two plants have actually grown past each other by 12 inches. And so there's our excess and our deficit. And so we can set up our table. After five days, we have a deficit of 5, and after six days, we have an excess of 12. We have the sum of the cross-products. We have the sum of the excess and deficit. And the cost per person, this ratio between the sum of the cross-products and the sum of the excess and deficit, is going to be 90 divided by 17. That's 5 and 5 17ths. And the thing to remember here is that this is what's going to make the excess and deficit zero. So in the context of this problem, it's how long it takes before the two plants meet. This approach allows us to apply the method of excess and deficit to many other types of problems. Some of these problems look like but aren't quite exponential in nature. So for example, one problem, there is a wall 5 chi thick, two rats tunnel from opposite sides. On the first day, both rats tunnel 1 chi. Each day thereafter, the bigger rat doubles its rate, while the smaller rat halves its rate. And so, how many days until the two rats meet and how far has each tunneled? So again, we can think about our contribution rates as the number of days the rats have tunneled and the excess and deficit can be measured by how far apart the rats are. So on day one, both rats have tunneled at a rate of 1 chi per day. Both of them have tunneled 1 chi. And so, all together, they've tunneled 2 chi, which puts them 3 chi apart. On the second day, the bigger rat doubles its tunneling rate to 2, the smaller rat halves it to 1 half.   That's our deficit. If we let them dig another day, on day three, the bigger rat has tunneled at a rate of 4 and gone a total distance of 7 while the smaller rat has tunneled at a rate of 1 quarter and gone a total distance of 1 and 3 quarters. So the total distance they've tunneled, 8 and 3 quarters, but since the wall is only 5 chi thick, that means they've actually gone past each other by 3 and 3 quarters, which is our excess. And so again, we'll find the sum of the cross products, the sum of the excess and deficit, the cost per day that's 9 divided by 4 and 1 quarter gives us, and this is the amount that makes the excess and deficit zero. It's the time to completion. Liao Hui also used the method of excess and deficit in a way we would recognize today as the method of double false position. We'll illustrate Liao Hui's approach by trying to find the side of a square with an area of 300. And again, we want to think about our excess and deficit and our contribution rates. And in this particular case, the contribution rates are the lengths. So let's consider a square with a side length of, oh, how about 10 has area 10 squared? That's 100. And since we want our area to be 300, that's actually a deficit of 200. And so a contribution rate of 10 gives us a deficit of 200. Now, if we take a bigger square, let's say a square with a side of 20, the area is 20 squared 400. And again, our target is 300. So that gives us an excess of 100. And so we can set up our table. We find the sum of the cross products. We find the sum of the excess and deficit and our cost per length. That quotient, 5,000 divided by 300, is going to be what makes the excess and deficit equal to zero. Well, not quite. We do observe that our result, 16 and 2 3rd squared, is 277 and 7 9th, which is not quite 300. But we can get a better answer by being closer to the actual values. Now, since we know that 16 is a little bit short, we might go to a square with a side length of 17. So a square with a side length of 17 has area 17 squared 289. That's a deficit of 11. If we go up to the next larger square, a square with a side length of 18 has area 18 squared 324. And that's an excess of 24. So again, we have our excess and deficit. We find the sum of the cross products. We find the sum of the excess and deficit. And again, that cost per length is going to make the excess and deficit zero. And to check our answer, we know that 17 and 11 35th squared is 2 99 and 961 12 25ths. And that gives us a very good approximation to the side of the square with area 300.