Welcome to the NPTEL course on remote sensing and GIS for rural development. This is week 11 lecture 3. I am happy to welcome you for the sections where we will be looking at something very novel and unique that we are learning in this course. As I explained in the urban centres and urban regions there is multiple data that we can readily use. On top of that the government, agencies and private companies collect data. Whereas when it comes to rural regions there is much less of data that comes in therefore in this week we had introduced the concept of synergized mapping which is very new and unique because either people use only remote sensing or only observation data and some people do merge them both but then with crowdsourcing data we are developing new skills. This is going to be the future because a lot of people depend on crowdsourcing for updating the data at a very high speed both spatially and temporally. Now think about adding another layer on this when it comes free open source then the potential benefits are a lot and very enticing for everyone to use. On that note we had looked about the concept of synergized mapping in the previous lectures and we found that OSM open street map has been widely used across regions across the countries globally for crowdsourcing of data. So you don't have to maintain a database, you don't have to maintain servers for the data to come in and use by users because all of this has been funded and operational till date from 2009. Slowly it has grown to a very robust system and while it seems that we are promoting this particular app it is not for their benefit but it is for everyone's benefit. Think about an open source system where we have data and updated regularly. The last slide we looked at someone just entering data 11 hours ago so spatially and temporally also it will be one of the best data and if there is some kind of schematic system that encourages people to participate then more and more data will come. In this world of data as everywhere it is very important to have a system and a platform where data can be easily, seamlessly coming in and used for benefits after we merge it with couple of data. What is missing in the OSM part is the remote sensing applications which we will be doing in our current class. We will be using the benefits of multiple systems, remote sensing, GIS, OSM and then merging them all into GIS platform and then checking on how the research can be done further. On this note in week 11 lecture 3 what we will be doing is seeing how we could combine OSM, RS and GIS, OpenStreetMap remote sensing and GIS interfaces for rural infra mapping and development. Please remember that a lot of people do ask more on water aspects for this course. This is not a rural development for water it is also for other resources and while agriculture is the key dominant livelihood I had put more time on agricultural resources land, soil, water, vegetation but we cannot neglect the other infrastructures because the reviewers of this course while they said it is very good they also wanted these to be reflected. So as a whole I am going to give the next three sections for it and some case studies and references on how these case studies have been done will be conducted in week 12. We are coming to the end and you would have seen that I do not have much summary for each topic because you can always go back to the lectures and view it but if I use summary then I am losing a couple of lectures. So I encourage you to go back I can give a hint of summary at the week 12 but at week 11 what I am proposing is please go back to the lessons I have seen couple of students sending uploading their materials on the forum which is very very useful which is very much making us happy because they are putting so much time on this work so we are very happy for that and we would like to say that even though some of your comments I could not readily address because of the need of the course we have made it more robust. There will be some updates in the columns of data that we are going to see now. So let us see how we are going to map OSM plus RS plus GIS for total infra map plus development. So how we will do is I will just go through the steps and then we will follow the same steps in the OSM platform. So we will first extract by name, name of the attribute and then the boundary. So OSM is mostly for extracting shapefiles, vector shapefiles not for raster because you won't expect people to share raster for a particular area let us say a school boundary. We will be discussing more about the rural infrastructures that I have given you introduction in the previous initial slides because when we talked about rural development we talked about water, soil, nutrients, forest cover, agroforestry which all we have covered but we also mentioned about education, access to schools, access to healthcare these kind of attributes. And these are not rasters these are vectors so these are vectors that are stored as attributes in the system and we will be accessing them through these OSM. The point of introducing this software is also for you to put in the data so it's not only one way it's not only that I should be using this data I can also contribute to this data by having a login and then putting in the attributes. We'll show you how this can come in play especially for the regions that we have already looked at. So we can choose what type of attributes we would like we can choose points, nodes both are kind of similar because those are point shapefiles we can choose a point shapefiles for location of schools, hospitals, educational institutions, kindergartens, ungun bodies etc. Rational shops all these are points because it's a location. We can also have other shapefiles which are under the vector columns which is one as points we have already seen then we have lines and polylines we have multiple lines that can be connected. One good example is your waterways in rural areas, streams, springs etc. and more importantly your road networks because we want to connect the livelihoods to the market. We also have the last one which is polygons and the polygons are very important because parcels of land can be mapped you can map the size of schools. We'll show you how the school size will be different both as a point and as a small area polygon and we'll go through these exercises together. So there are three types of shapefiles that can be generated through this OSM very focused on rural development which is points, multi points and then we have lines, polylines and we have polygons that can be shared. An example would be you can also have queries for crop selection. It can be banana or sugarcane or you can do banana and sugarcane. So how will this be helpful? Let's say I'm going to map cash crops in Pune region. When I went recently to Pune I was shocked to see a lot of people converting from sugarcane to banana because they're having a good market in the Middle East. So this banana can be taken as a crop or it can be combined with sugarcane as a cash crop and then searched. So in OSM we will be doing a lot of queries. We will search for data and then plot the data using internet access. Healthcare also we'll be doing in the sections, schools we'll be doing as a polygon and a node and we can also search by village name. Only problem here is make sure that the village name is same as what the OSM software might have. So OSM is you can imagine as a database sitting in a cloud and what you will be doing is while running the QGIS program to connect with OSM you will be building a query to download the data. When you download the data it will talk to the cloud, bring the data, correct? So in the cloud there are a lot of databases and the spelling might be very different. So for example, India spelling is INDIA. Only when the spelling is correct you will bring India's data. Like if you write like West Indies, if instead of West Indies you will put West India then it won't go to West Indies, it will go to West part of India. So now you have a hint how these names came into existence. So naming is very, very important and you will be very cautious in putting the correct spelling otherwise it will be difficult. And to beat this we always use shapefiles. I'll show you how to do both so that you can quickly do queries in the next three lectures on schools, we'll start with schools today then colleges and then we'll also look at healthcare institutions and then finally go to the location of the crops. It is also important to understand that there are limitations because it is free, because open source, there's a lot of quality checks that we need to do because not all would be doing it for accurate quality, it both happens in proprietary and open source. We cannot say that since people get a salary they're going to do it correctly nor we can say that since it's free it's correct. So there could be some issues and errors, we should be able to check it and what we will be doing is we'll be using Google Earth Pro as a means to check on the data and then save a shapefile. All the data searches that we do and populate in QGIS we can save as a shapefile. For all this you need internet, this OSM package runs on internet because the database is so big and it gets updated often, you cannot keep it outside in a server and then connect to it or on your laptop and connect to it. You need a cloud or a server which is connected to high speed internet. So your internet will talk to the server, collect the data and market. It's not like you download all the data. So when you download the plugin, which we did in the last course, you only download the interface to the plugin, not the data sets. So what we'll be doing is we'll start the interface and show how we will be using a quick query tool to assess these different data sets that we want. So quick query looks like this. You have the preset which is already set conditions to search for data and then you have the data that you want to extract. The key variable, the key column is what is the variable about on the OSM database? And what is the value? So here you could look at it as key is the overall theme and the value is the attribute name. So the overall theme is amenity and then you have hospital. The overall theme is hospital health care and then you have value as hospital. So sometimes why this is needed is sometimes there will be different names given. Instead of in India, let's say we will put all hospital data and health care, not in amenity. But maybe in Nepal, you will be putting it in amenities and not in health care. So what happens is when you're doing a mapping of the Ganges health care facilities, the ones in Nepal will be omitted. So that is why you can have one query built and add other queries to it. You can strictly make it as a condition as and or you can make it a condition as are, which means, so for example, if I put and here, so after the query, the results for wherever amenity is equal to hospital and health care is equal to hospital will only come. And if amenity hospital is not there, health care hospital is not there, it won't come. It won't be populated. Whereas if you say are, then both will come. Amenities that are labeled under hospital and health care, which are labeled as hospitals will also be populated in the page. So let's look at one query and then we'll come back to the next slide, which talks about, it will be talking about the quality checks. I'll just introduce this slide, but we'll come back again. We'll be looking at checking the data, how to check the data. Some key notes is the data of the data is not possible to have for every single data because it is updated frequently, which means the temporal resolution is very, very high. It cannot be updated equally. For example, you can see here for the tricky reason that we'll be doing today, someone has input data just hours ago, 16 hours, one day ago, et cetera. So in that note, we cannot keep on updating the temporal cycle. And also we are not sure like every day the data will come correct. So you cannot guarantee it, but you can see that it is the most updated data because people do put in data. Accuracy may not be the same as different users. So as I said, all volunteers may not be trained in the current level, but all volunteers have equal access to this website and they can contribute equally. There is some clearances of data. You cannot put wrong information in it. There is some checks and all, but it won't check for more, more accuracy. For example, if you're picking a building from the top alien view and then say it's a school, it may be a school, it may be a hospital, it may be a shop, no one knows except the user has put it in. So there's no one on the ground who goes and checks it. Maybe a concept that can come is voting. Like people can vote if they're on there and say, yes, that is a school, yes, that is a hospital or a mall. Then that voting system can create confidence. Like for example, you have a hotel. How do you choose a hotel on your app? You will say, okay, how many people have had food? How many orders? And how many good reviews have been there? Five stars, four stars, et cetera. The same like that, we can map it here and then say that how many people have actually verified it and based on that the star ratings will go up. Most of the time in rural regions, this will not be working because you don't have that much people contributing anyways. So we'll have a very less, less amount of data. The terminology is many have different terms for names. So for example, someone will call a school as school, someone will call it as an Anganwadi, someone will call it as a kindergarten. So you will have to be careful enough to negotiate these names and then pick the data and then make your own database through the Google checking, which we'll be doing. So with this, as promised, let's get into the schools checking for which you would need to have your QGIS up, which we already have. And I'll be sharing the window space here so that we will be looking at the QGIS plus the plugin page. So I'll share the entire screen that you can see. And what we'll be doing now is we will be looking at the plugin. So how do you open the plugin? You'll go to vector, go to QSM and then you'll see it here. Sometimes what happens is you will have the data being mapped at regular intervals, okay? And the software gets updated frequently. So you will need to make sure that the data comes in regular intervals and you keep on updating. Also the software gets updated. So I had run into this issue, so I'm putting it up here while recording if this happened, right? So the software stopped to communicate. Why? Because it was being updated. So always if the software, the plugin crashes, go to the plugins and then see if you need to update them, okay? I'll show you quickly. So the quick QSM, you could see that in the last lecture it was 2021, which is 2.1.1 version we used, but just now, so today's date is 21st March 23, just last night or in the morning today because of the European time, they updated it. So once they update, this will stop working until you update it and it will not tell you that you need to update like your other apps because this is not a paid version, right? They will not force you to update it.  So whenever there's an issue, just quickly update it, it will work. So as I said, I have opened the QSM app and when it comes to the plugin, you will have this quick query already loaded. Go to quick query and then you have the preset values. I'll show you an example of preset. We'll compare that soon, but let's say education. And then you have education kindergarten or facilities, education college, okay? Or school, we can have since school is a theme, we'll have education school. Once you say school, what it does automatically, it says amenity schools will come. This is a preset search. So for this thing, the key theme is amenity and under the value it is school. You can add another one because it's just amenity, right? So you can add another one, say R and then it could be, you can go here and say kindergarten, it won't populate because you have to populate the theme. So let us populate education will not be there, amenity. And kindergarten, okay? So kindergarten has come up. So now we have two questions. It is amenity is to be equal to school plus, and or amenity will be kindergarten. Or if you say and then both have to be yes, yes. Both have to be true for the output to come. Like for example, the school in IIT Bombay does not have a kindergarten. It has only one to 12. So only that school will come up, but the IIT Bombay school will not come up in this map because it has no kindergarten. If you take my school that I went to in Chennai, yes, it will come up because it has schooling from LKG UKG to plus two. So you see how one search, you have to be careful about it. The query button is if you know how to do Python coding, you can type the query, but that is what BigQuery does. It freezes interface so that you can populate the query automatically. And these are other OSM file where you want to save it. What type of OSM files do you want? The dietary can be temporary. Keep it temporary for now because you'll be saving it only if you like the data. And then you have parameters, abort, et cetera. Parameters you need to have this overpass access to API, which will be loaded. So now we have the search things selected. And what we'll do now is in, where do you want? Is it within a village? You can give a village name or a district. So I'm going to say through Sira Palli here. Now what happens is if the spelling is different, it will not pick up. So make sure that if you know the area, I have already shared all the data with you in terms of the links, where to get data, where to get district boundaries, block boundaries, use the boundary, use the spatial boundary rather than this. So I'll show you that. What I have done is I've already loaded Trichy, which is a short form of Trichy Palli. And a lot of people still use that in official names when they write. So it's still usable. Let's say layer extent. And the layer extent are these, one of these three spiles, I'm going to say Trichy. And then I've showed you how to extract that from the overall shapefile. Also you can do this. So for example, if you have this selected Tamil Nadu, which we'll do later as an exercise. So then only Tamil Nadu will come up. So because of this, let me just make this sound. So because of this open box, we cannot do the select tool now because it's already selected, but let's show you the other way. This is your history. It gives you the history of what you have searched and collected data and the advanced part is where you will have to tell, do you want the boundaries? Do you want the lines or nodes? This will take too much time if you have everything, right? So let's keep nodes and points. We want the location of the spools. And then we'll say ways and lines. We can keep that, okay? Nodes and points we'll keep. Ways and lines is paths. Way is a path. Do you want the way pathway to the school? We don't need it. Do you want a road line to the school? We don't need it. Do you want a relationship to the school? Multi-strings, we don't want it. Do you want a polygon? It gives the area of the school? Yes, let's keep it. So some data could have come as polygon. Suppose I'm a user, I gave a data for the school. I would have given it as a polygon or a point, right? So this aspect is where you will be careful to negotiate how to bring the data in. So I'll just say both. It will take some more extra time, maybe a couple of seconds for my system because my area is small. If you use a big state, then a lot of data has to be given. A lot of time has to be given, right? So all this is enough. All these keep it default and then go back here and say run query, okay? If you say show query, like if you go here now, nothing is populated. But if you say show the query, now it gets populated. So the code that you gave in quick query is automatically converted to a Python. The selections you've given, the selections you've given, all these selections have converted to a query and it says school or kindergarten for the particular thing. And it says you need to time out for a particular time and then it says for the particular extent. So everything is coded. Now all we have to do is run the query. I'm running the query. Here on the bottom, it says running the query. One layer has been uploaded, okay? Let's see what layer has been uploaded and it has been the locations. Amenities is school, kindergarten, okay? Let me zoom into that layer, right click, zoom, okay? So once you zoom in, you could see that this is the quality issue I was talking about. Some have been labeled as tricky schools but they're outside the boundary. You cannot blame them, okay? Maybe their location, the phone or the GPS unit they were using has that big of an error, which is possible or and or they would have done a wrong spelling and stuff. So as I said, it is our duty to check the data before you use it. Any student, anyone who's using this for research purposes needs to look at the data. So I'm gonna zoom into the tricky boundary and you will see that both these are outside, there are clusters outside but there are good clusters inside also. So let's open this, amenities schools, right? And then we'll say open attribute table and you have this beautiful table coming up with random IDs, no ID name, no school name, but it's just a school and it's stuff. At least you know like it's a school and under that there is no cities, syllabus, only some have it. For example, this has K-P-A-V-M-G-A-C dot in. It is a school name with the opening hours. So only some people like it could be a good business for them to put it on the map. So people understand where the school is and then you have the maybe the principal's name, the email, street, office address and then what are the brands they have? Suburb, state, postal code, Tanjavur. So here's that, it's a Tanjavur district but why is it in Trichy? We don't know. So these are the outliers, okay? So when they typed it, they put Tanjavur but the location, geo-location is somewhere else or when they did the geo-location it's in Trichy but the name is Tanjavur. So all this should be Trichy or Trichyrapalli but it's not there. The school names are pretty good. At least the names they have put and some random is nothing, no data there but the point is there. So some data errors are there. Okay, so it is our duty to filter it and use it. So now beautifully we have done this part and as I always say, we are going to, we need to, this is in temporary file. So we need to save it, export the save feature as amenity schools will go here to see where we want. So let us create a folder for this. This is all our materials that we have for this course. So I say OSM test and we are going to say Trichy. Amenities, we'll just say schools. Schools under amenity, right? So I'm going to save it as a shape file and save. And then add the file to a thing. Yes, we can add. And now this has been added. So you can remove this file, which is temporary. We can remove the layer. So what I've also done is, I've also done behind this another file with a different set of search. So I'll add that too or we could do another quick search. So here we are, the quick query is there. I've said Trichy boundary is fine. Let's do one with the name. So in and around Tiruchira Palli. Tiruchira Palli, right? I'm going to remove the preset. So I'm going to reset the preset. And then I'm going to go to education. Not there, government. Yes, government is there. Let's make government supports. Okay, there it is. So if you click here, you will find all the different keys that are available. So we have click government and under the government, what are there? So I've clicked education. You can also search for it. I'm going to add another layer saying that amenity, R. And then in the amenity, there's all these things, right? So let's say school plus R. Let's say amenity, kindergarten, plus private schools. We don't want colleges, right? So we'll just keep like that. We can say government, kindergarten. We don't have kindergarten. Anganwadi doesn't come, right? So this is where we don't have the names and all. I purposely wanted to show you. I've already tried these. It doesn't work, correct? So we can have tutorial college under amenities, but not under the government. So the government is only schools are there. So let's try another amenity here, which could be only animal boarding and building. So we can also remove the layer if we don't want. So three days is enough. And as I said, I'm going to do Thiru, Thira, Palli, right? You want to save that in the new, okay, you can save it in the new question. Do you want a preset like this? If you want a preset, you can save. Otherwise it's fine. Again, I'll remove the ways, lines, relation, multi-strings, I'll keep the polygons. I'll show you why and then we'll do it. Okay, oops. So this can be default. And then I'm going to go on the query. So it is to keep it into the column. Okay, the layer has been added. Apparently there has been only points. There is no polygons, which is okay. But for Tamil Nadu, you can get polygons. I'll show you in the next clip for schools. So now we have Prichi schools, just by preset condition, by this condition. And then we have the government entities. Let's save this also, export, save feature as, and then I'll go here in the same Prichi. And this is not a preset. This was just a government plus schools, government schools. Yeah, you cannot add symbols, so don't add symbols. Yeah, this is fine. So I'm just gonna say, okay, add the map. And then I'm going to remove this amenity location. Remove it. Okay, yes, sir. And we have all this, right? Always save your work. Okay, so now we have two shapefiles. I'm going to add another shapefile for Tamil Nadu. We'll see why. I'm going to zoom out a bit. So now, Tamil Nadu, I want to take the schools. So what I'm gonna do, I'm gonna do the select button. Sometimes you can make a shapefile. Suppose you don't know, you don't want to make a shapefile, you just want this. So what you do this is, you can click. You can clear the selection. Okay, so for now, let us take these points out. And then even the G out, and then just keep the states. Okay, we can copy the coordinate, but let me go to open table. Oh yeah, it's not allowing me because the name is not there. There's no create an identifier, so it's fine. So let's go to the table, right? If you double click on this, it can also show you if you want to, you can close this because it is disturbing your screen. And then you can select a particular state and then make a copy of it. Let's say like this, you can zoom in. I think the other one can be corrupted. I'll show you why. So now I've selected this and then you can select this tool and then just click on this. So when you click on a particular file, what happens is you can select a particular district. I'm gonna show you by Tamil Nadu region. So for example, here, as I said, we can do state, district. This is the second name, this is the state name. So you can click on top and then all the Tamil Nadu I can select. So here, here, Tamil Nadu. And then if you click from here to here, all Tamil Nadu I am selecting. So you can have the Tamil Nadu map coming up. So both you can do if you want or you can just select one. So now my select tool is opening up. For some reason, the select tool now it's opened up. So the select tool was kind of hiding behind. Now it's good. So what are these two? These are monetary territories. So some part of it is wrongly named by the dataset that is fine because we are going to go look at this part. So now Tamil Nadu is selected and what name I've given is, I'll show you again, just delete this and then I'm selecting Tamil Nadu. So just click this button, click on the state you want. It'll get selected. Then you go to vector QSM, open the QSM box. Okay, multiple QSMs are opening. We don't want that. Okay, so we've already selected this. So let's close this. Now we are here and then we'll say education, right? How many T's? And here I'm going to say that I wanted in my layer extent, India states, right? India full states, but only the selected. So I don't want the entire country. I want only the selected and I am going to do the advanced. I'm taking the ways out, relations out only the polygons and points. So it's going to be big, but quickly to run at least in my system. So I'm going to run this. Now we have three shapefiles, one as a preset condition for Trichy, two as a search for Trichy box using government and immunity data and number three for Tamil Nadu with the polygons also included, right? So now what we have done is we will be able to reload it. Let me reload it here by selecting this, which I've already shown. And then I've saved it here as a shapefile, okay? So TN scores poly open add, okay? So now we have the TN scores poly which has been selected. And you could see that when I say Tamil Nadu it not only selects Tamil Nadu but all the other regions also. I'm just going to clear this and keep this. As you said, there is a lot of data issues. So these are not definitely Tamil Nadu. These are in Kerala and these are outside. So there are a lot of data issues and errors but nevermind you will be able to look at it differently. So what happens here is this immunity this immunity has been created but it also has something inside, okay? I'll show you. So it is also a polygon not only a point. So within the point there are multiple polygons also stored. That is what is happening. So I have extracted the polygons on the map and then kept it here. So now we have Tamil Nadu polygons, Tamil Nadu locations out of through to searches. We're going to open Google Earth throw and then put the data in. So for example, let me just hide this for now and then remove these because I just tested before we show these in the class. For you also it is necessary to know how to remove these because it will just cause a lot of errors if you have both of them opening at the same time, right? So let us remove your history a bit, okay? And then let's say delete contents. Yes, it will just keep on deleting the contents. So that now we are free to use and we can also delete or remove these. Okay, so now what do you do is you go to file, you say open, you want only the shape file in the OSM thing we have these two schools and stuff. So let us open them. Once you open them, it says, do you want to apply the same style? Okay, say no, why? Because when you apply the same style you are limited for putting the same style location. So I'll say no. And then you want to apply same for the second one to say no again. And it is going to go to the Trichy location, okay? So now these are the schools in Trichy. Again, you have to go to view, reset, tilt. So now it has come up. So if you remember in one of the previous classes we made up a point for my village school at dry school has been marked. However, it is not in the OSM database. You can see here from the previous thing we have drawn the boundary. We have said at dry is the village but there is no school there even though I said that yes, there is a school. So this is the school I told my parents went and I visited the school area a lot but it is not marked because the data has not been input by any user, okay? So maybe after this course we will be inputting that. But if you go to other places in Trichy the city places you will see the schools. So you can see the school here. So one of the school is there. You can click on that. It will give the properties and then say that okay, the school is there, okay? The school name may not be there. That's why it's not populating. It depends on where the school is and if they have given the names. We have seen that in the list. Not all schools are having the proper data entries. So these are schools.  So these are three points for the same school. It's not three schools. Can you see it like a school? Okay, beautifully, the data has captured the location of the school without any government record. And this is open source. People have put in. You can see that you can enter here. Maybe there's a parking. You have this ground. How do you know it's a school? Because the construction, the construction is normally long classrooms and the central ground where students play. So you have a big ground. You have this assembly area, assembly main building, admin building, and then the classrooms. So this is how you could see that there is a lot of names. How do you know? You can check. You can also click one of these places and then see if as you zoom in, if some schools are like famous schools are there, they will populate. So for example, this looks like a small school. This is on the road. So maybe it's not a school, but this could be a school. Correct? Opposite, oh yeah. This is a big school. Oh, there it is. So Tirucharapalli multi-purpose school. The point is outside the road, but this is a small area. You can push it inside.  You can, for example, go to properties in this part and then move it. I don't know which one the data is. So we can quickly check here. It's this one. So you can go to properties and then physically move them if needed, but it's going to take a lot of time. But at least you know which ones are there and you can put a point now. For example, you can put a point saying, okay, there's a school that is the name, it has been checked, et cetera. So now think about instead of starting blindly, you have all the data to start and put these locations. So this is about schools. I'll also add the other shape file, which is the polygon. We will be stopping soon because of the time, but when you add the Tamil Nadu polygon, it will say it's too much. Do you want all of it? No, I'm just going to say restrict the view because we cannot plot thousands. Zero features important because there's no features there. So I have to zoom out for this region. Okay, let's zoom like this and then reload it again. There will be thousand features. So thousand attributes, we don't want that. So let's say restrict the view only for this. 12 bar imported, same thing. I don't want the same template. I want this. So now you're going to see area. This is a polygon. I'm just going to zoom out and then let's see where these... So now I'm going to remove the places and borders so that we can see the buildings, right? Okay, there's one. We've got one. Okay, so this has been labeled as a school. You can click open and you can see the school there. So someone has detailed it very nicely, the school location with a polygon. And you can actually add places and borders here to find what is at school end. Bollford, Planned Girls, High School, School. So you could see even now, if I go back to my Eterite, it's not marked because that is a problem in rural India. We do not have much of the data marked. Whereas in an urban center, yes, it has been marked widely. So here I've shown you how to come to a particular area and then mark schools based on points and polygons from an open source system. And then you put it in Google Earth and check. We'll revisit this in the next class while we do other attributes. So here in this today, we will stop here with an attribute on schools. We have checked the data. We have also done some minor edits if needed. And then we'll be able to create a database. I'll see you in the next class. Thank you.


All right, so Rishabh, go ahead. So what I was saying was to give useful metrics in that presentation in the DevOps, what is its name? I'm sorry, I forgot. The DevOps event hosted by CloudBees. So to give meaningful metrics, I would need to profile an instance which would have jobs which would simulate a near real-world use case of the plugin. And then we could use the updated Git plugin and with the updated Git branch source plugin to see, to measure the differences in performance in a freestyle job in a multi-franch project. And in that way, for ourselves, it will be a great exercise to understand what kind of changes we are expecting once we introduce this class inside Git plugin, this change. And for the presentation, I will be able to give some useful results. And that would be great for then the project. So we would have to figure out how to profile the Jenkins instance running on your machines because I have the access to the local host 8080 port directly. I'm not sure if I can start the instances using. I don't have the permission. I'm not sure how would I start the instances with. I know how to start them if I am starting them using the war, the Jenkins war. So I can use the Java command to run it. And then I append the JFR optional arguments, the Java Flight Recorder optional arguments. It comes packaged with the JDK 11. So we won't have to add anything else for it. But we do need to run. I only know how to run it from the Jenkins war. For a Docker image, I'm not sure how I would do it, how I would proceed. But that's something we might need to figure out. Because for me personally, if I am profiling it for some jobs, I can do it for some jobs, but not for as far as I know, you have a lot of jobs in that instance, which would automatically be profiled. And we would, I think, have a huge range of data on how the plugin is performing with the changes. So that's what I. Would you be willing to consider not using Flight Recorder, but just using timestamp reports from the build logs? They're only one second granularity. But would that be enough to give us a fast approximation? Then we would just, we can enable timestamper. And I think we could even, in that case then, rely on some repositories we already have that are of known size, like the Git point plugin repository, which I think is less than 5 megabytes still. And the Git plugin repository, which is, I suspect, greater than 5 meg, it's how big? 19 MB, I think. Oh, good. OK. And then we could take Jenkins.io, which I think is about 60 now. And there we've got three. Jenkins.io is around 400, I think. Oh, well, that's even worse. That's even great. OK. So if Jenkins.io is that big, it's even, we've got those. And by choosing those, those are repositories that already have a Jenkins file. And so we could enable a multi-branch project on that Jenkins instance for each of those. Some of them already have it. And therefore, the cache will already be available. We don't even have to have you implement anything in the branch sources for that on that instance, because the cache will already be there. OK. But if we are going to do this whole exercise, wouldn't we want to do both of the things to run it for a freestyle job if we don't have a multi-branch project for all of the cases? And this would not just be for the results, but for the, because I have interactively tested this feature, but it maintains you, the mentors haven't. So I think it would still be, it would be better if I provide the implemented branch source plugin and then we can test. Yes. Yes, wholehearted agreement. Absolutely. Yeah, I don't want to say that we should limit that you don't do it, but rather if you need, for instance, my DevOps world talk has to be recorded today. I don't know when they told you you have to record yours, but I'm in a session this afternoon recording my talk. So I've got to get ready for it, practice it, prepare it, et cetera. And so I thought if you're on a very limited time scale, there may be things we could do to get you the data when you need even faster. But if they haven't said, oh, you've got to be recording it today, no problem. We can do as much or as little as we'll help. Earlier they did say that we have to do it till Friday. And I was notified about it on Monday. So I thought that I possibly cannot do this till Friday. But then I asked Oleg. So he said that we will, since the GSOC students were notified at a later stage, they would give us later dates. And I have booked the recording session for 28th of August. So I think I have a function. You've got lots of time. Wow, you'll almost be done with a project by then. Okay, great. Yes, we would have time. So I chose that date for that purpose. I think I should start the meeting with the agenda. I was, okay. Okay, I'm sharing the screen now. So first let's start with the PRs. We have the first PR, which we are done with is the pull request on the addition of unsupported command in the Git client plugin, which has been merged by Mac. So reviewed merged. You also added the test for it, which I'm very grateful for. So that has been done, that PR has been merged. The second, all the other three PRs which we have, they are the open right now. The biggest one is the get to choose it. And the get to choose a test. The second one is the unsupported command, the usage of that class within Git plugin. I have some work on that PR. The third one is the extension implementation on GitHub BrandSource plugin. So I'm going to start with the biggest PR, the major concern for us, the get to choose it and its test. So there has been a changing design for that PR and the reason for that changing design is an observation, brilliant observation by Omkar and Fran. They both pointed out a huge, I would say a hole, a gaping hole inside the get to choose as implementation and I will go through what is the problem and how have I solved it. I think I've added the comments in the commit message and I will just show that and then explain what the decision, the changes. So currently, so we are talking about Git tool here. Git tool is basically a class which is encapsulating the details about the Git implementation which is going to be used to create a client in the Git plugin. So currently a Git client, we know that a Git client is initialized with a Git implementation. Now there are two decision factors which decide what implementation will be used. The first is the user, the second is the system. The user has an option to choose decide between get, jget or jget Apache. These are the three options they have and if they do not choose, we provide the default implementation as the Git implementation if it is available. If it is not available, we have the jget implementation shipped within the plugin. We would provide that. The second decision after the user chooses its implementation, its preferred implementation, the second decision is taken by the system. That is that it takes the user's choice and then it validates with the particular agent or wherever the system is working to check if that particular user's choice is available within the system or not. Now what is happening with the introduction of Git tool chooser is that we have added another decision variable to this whole process of providing the implementation from the user to the client, to the process of making the client. We've added another decision variable which is the Git tool chooser but the Git tool chooser was not considering what the system is evaluating and then providing the implementation. What the Git tool chooser was considering was that, okay, I will tell what the best implementation is but before telling that, I will check if that implementation is installed or not but we are not seeing what the system has already provided and shortly we will see how that will break some use cases existing if we do not take into consideration what the system is providing to us. So the change in design is basically what I'm doing is that I'm in the constructor, in the initialization of the class, I'm adding another argument which is the executable path of the implementation which the user has provided. So how this works within the Git plugin before the Git tool chooser is that the user would provide for Git, for Git, the CLI Git, user can provide the name of the implementation and provide the path within the system that is how it is installed. So the Git plugin uses the path provided by the user. That is what it uses to feed the Git client initializer and then it initializes the Git client. That is how it works. So what we need is, we need that executable path and then we can decide if what we have to preserve, what, how our decision is not affecting the existing case and I think the best way to understand that is, let's just look at the test cases I've added so that those are the scenarios where we can understand how this is affected. So I will, okay, this is the test class, yes. So we have two scenarios here. The first scenario is the size of the repository is less than five and the Git tool chooser will recommend JGit. Now, this is from Git tool chooser side. We have some decision variables we have to consider before that. Those are, let's say the user has installed the installation is called MyGit and its path is set has to be user bin Git. Earlier, if user was providing this path, I would not consider it. I would just say, okay, give JGit because the sizes less than five and the Git tool chooser would provide JGit, but let's just say that we, what if there is a scenario where we have to use Git but we are not using the path for it. We are giving the name for it. And maybe that works here because it is linked. So Git would work in a system where Git is installed in user bin for a next system, but it might not work if there is a difference, if it is installed at a different path instead of the user bin, which might be unlikely, but it's still a case I think. Please interrupt me if my assumptions are wrong. So if this is the case that the user is installed, the Git tool with a different name and a different path, what we would do is, so what we should expect is in this case, we don't have JGit in this system, although we will always have JGit in the system, but for this particular use case, I have removed this Jenkins instance all only has the Git tool, which is the MyGit tool, which is configured by the user. So the tool now will recommend user bin Git instead of, and it should recommend JGit because the size is less than 5MV, but we do not have installed JGit in the system. That is why it is not able to provide it. So this is the first use case. Now the second use case is what if we have the same Git installation and we have JGit within the system, then I think the expected behavior is that it should provide JGit as the expected implementation. And one thing I am missing here is that while writing the Git tool chooser, I did not investigate the fact that we should not provide the implementation name, we are providing the path where the implementation is installed for the Git client. The Git client does not take the name of the implementation. It takes the path of that implementation. That only matters, it is limited to the scope. For JGit and for JGit Apache, it doesn't matter. The path is same as the name. That is what I have seen in the JGit tool class, which is implemented on the extended from the Git tool class. For Git, it matters because the name and the path where it is installed can be different. So this awareness, which I did not have earlier to the observation by Omkar and Fran, was something I now realized. And that is why the Git tool chooser is now aware of that fact and it will provide the path instead of the name. For JGit and JGit Apache, it will still be the same. It is equivalent to the name, but for Git, it will provide the path instead of the name. So I have some more use cases where I was not... Yes, Mark. Okay, so I think the test that you've got on screen right now, the top of the screen indicates a potential problem if this test is passing. Oh, no, no, sorry, not the one down. So the next one down, the one that uses JGit Apache. Okay, so excellent that you wrote these tests. This is brilliant. Thank you so much to Fran and to Omkar for detecting the problem. The, in this case, oh, oh, no, okay, I see. So in this case, you've got command line Git, and JGit Apache, and it chooses JGit. Do you have a test which has only command line Git and JGit Apache? No, I don't. I have one with only JGit Apache, but I don't have one with JGit Apache. Okay, so long as, so it looks, if you've got the, well, if you could just safety check for my comfort, that if CLI Git and JGit Apache are both available, but JGit is not, JGit Apache should be chosen because it is a plug-boom compatible replacement for JGit, right? So you've got the case at line 300 where you are correctly choosing JGit Apache. So that's a big win. But yes, I actually, so what the decision I took was that if we have JGit and JGit Apache, I would use JGit, I would not use JGit Apache. And that is the correct decision. That is absolutely correct. If we don't have JGit enabled and only have JGit Apache, we should still choose JGit Apache for small repositories. Yes, sorry, sorry, please. No, no, they are functionally equivalent except for some relatively subtle details in terms of the authentication protocols used talking to a remote HTTP server. And I have missed that case. I just realized that if I would have JGit, if I have to recommend JGit and available installations are Git and JGit Apache, I would recommend Git. I would not recommend JGit. Okay, so then your tests have exposed a case that I think we want handled differently. Good, very good. Yes, I have noted that case and I will implement that as well. So these are the scenarios. Yes, Omkar. JGit is always available for JGit installation or no? No, no. Well, it's not always enabled. So the user must... I can disable it. Right, it is by default disabled. And because it is by default disabled, it is not in the descriptor list as a set installation. So the tests that, now it's actually the code is available, but because it's disabled, they have to choose to enable it in order to get the benefit. And so this was the scenario when the Git 2 chooser has a repository of size less than five MB. Now, if the scenario is that the repository size is greater than five MB, CLI Git should be recommended. And I think the cases, again, they show us that if we have no tool installed in the Jenkins instance, I assume that the Git executable would, can be named, so it would be named Git, not since the user has not provided the path. So then the Git 2 chooser would recommend Git. And if there is a case where JGit is installed and Git is not installed in the system, and it has to recommend Git, but since the system doesn't have Git, it would provide JGit. It's just, I would say an exhaustive list of the possible scenarios we could have. And maybe they do not make sense in a lot of cases, but it's a possible scenario we could have. So on this one, can you help me understand, or I can ask separately, what do you gain by the reference to the credentials which are here? The sample repo in it? Sample repo in it is for the project I am building. So I, for this, so I need two things. So since we have enabled authentication now, authentication, we provide credentials to the plugins who will implement the extension points. We need to send them two things. And the first is the context of the job. And the second is the credentials ID. So to build, to have the context, I need to build a project so that I have a context. And so that is why I am initializing a sample repository and then I'm building a project. The function is given, it's the latest part of the code. And the credentials, I saw this in one of the tests already written in Git plugin. This is a way to, so initially before the test start, I have at the rate before a method which would initialize a local credential store. And it would have a credential called GitHub with the ID GitHub and a username password sample. And what would happen, what I expect is, I actually expect since so we, so our responsibility is to provide the credentials ID in the context, it's the plugins, the other plugins responsibility to use them and to validate in this particular test class. I have test extensions. I have implemented those extensions just so that we can use them. But I haven't included any logic which checks those credentials and validates them. Because that is, that would be the responsibility of the GitHub branch source plugin. And I'm not sure if there's a point of validating them through our logic. I am not sure if we should do that right now. It just checks if the current, if the repository URL contains GitHub or not. That's the, it's not the right logic to check it, but it's just for this test class. So that is why I am building a project so that we can have the context and then we have the credentials we provide to the Git tool chooser. This is the second point, the change in design I wanted to discuss. I discussed the implementation part first, the Git tool implementation part. So you can look at the test cases and we can see how, what cases I've missed. But this is a big change which I missed while creating this class in line. I have added it currently it's pushed. I locally built it in my system and it is the tests are running fine. So I've missed one case which Mark has pointed out. I'll add that shortly. And I think for this change in design, this is what I wanted to say. Any further questions on this particular change? Any concerns? Possible concerns I've missed or maybe if you later look at the PR and review it then I think that's the better way, right? Okay, so the second thing is the authentication is basically providing user credentials to the plugin. Now, with this change, we have certain issues I want to discuss. The first is that there are multiple ways, multiple credentials, multiple ways a user can create credentials and from Git plug-ins perspective, we have I'm not just maybe more than two or three valid ways of creating user credentials. But when we provide those credentials to the GitHub branch source plugin, the GitHub branch source plugin will only consume username, password type of credentials and it would not consume any other type of credential we may provide to it. So that will limit our, I would say it would limit the cases we would cater in terms of optimization. I could not find a way around to, so there was one possibility suggested by a month that maybe the Git branch source plugin maps the URL, URLs with the credentials. So if I provide the GitHub branch source URL, a valid repository, remote repository URL, it could possibly look at the credentials it might store. But again, I think for that, a user in a Jenkins instance, the user would have a GitHub organization project and would have to set up those credentials. That is the assumption we took. But I cannot find any way where only the repository URL is enough to get the credentials. What I'm seeing is we need the context and we need the credentials ID. So that is a concern. I'm not sure if there is a way to expand the type of credentials we are sending. Also, what I saw in the GitHub API, the Java API implemented in the GitHub branch source plugin, just a second, I'm not sure where. So just a point of reference, I think we were putting the credentials in the Git plugin to support freestyle projects, right? That's the primary reason why it's in the Git plugin. So like GitHub branch source would already have its own credentials and a project would have been configured to project credentials for that project, right? A GitHub branch source plugin, a project, yes, it would. But then, okay, so I have a question related to that statement. If someone is configuring a project using the GitHub plugin, branch source plugin, it will not be needing the Git plugin for any, for the branch source, for scanning the branches, or for, because as far as I've seen the code, they have implemented their own SCM APIs. They wouldn't need a credential from the Git plugin. They would not, but then they would not be able to, we will not be able to actually use that, use the performance improvements because the Git plugin will not be used in that case. If we just have a Jenkins instance where a user has created a project using the Git branch source plugin, I am not sure the optimizations we are talking about, they will be used because the GitHub branch source plugin has its own SCM API implemented and it would not need the Git plugin for any case. Or am I wrong here? I think for the cloning activities and stuff like that, it still uses all the Git plugin stuff, right? Or, actually the API stuff, I was primarily for like, hey, do I have new branches? Hey, are there other things going on? They're also doing like the reach out stuff that we're trying to do, like the estimation. That's all the API driven stuff, but I think all the, Mark, Mark can go ahead and he knows better than me, probably. Justin's got it right, that when a branch source needs to ask REST API questions to, as part of a multi-branch pipeline job, for example, it uses the credential that the user provided as part of the multi-branch pipeline job definition. But when it comes time to clone a repository, it goes down to the Git plugin and says clone the repository for me. But then it provides the credential that it has to the Git plugin. So I think that supports, Rishabh, your assertion to Git plugin has to be willing to provide the credential out to others who may need it, like the branch source. But I wouldn't need the credential, right? Cause it would already have one of its own for scanning. Well, I think the flow of information is that the scanning has a credential, it passes that credential down to the Git plugin. The Git plugin can use it for clone purposes, but if the Git plugin, since it has the credential, now needs to ask a question related to, hey, give me the size of this repository, the estimate of the size of this repository, it can hand that credential back to the branch source and say, here's the credential, use it. So it doesn't, I think it's a simple approach as what Rishabh is planning to do, pass in the, pass back the credential that the Git plugin got, however it got it. I see, this is just flow control within the guts of how they're interacting. I think so, yeah. Okay. But if there's a case, what Justin has just said, if there's a case where we have a multi branch project configured by GitHub, so GitHub has the credentials, and then it is passing those credentials to the Git plugin to clone, and then we are again passing those credentials, we are not those credentials, would we be passing the same credentials? I'm actually not for me to understand that, I think I need to understand the credentials API better. So the credentials store, which stores the credentials, would it be common for the Git plugin and the Git branch source plugin in the same Jenkins instance? As far as I know it is, there is the credential store is shared by, as far as I understand it, all the components inside a Jenkins instance. Okay. So there- That's pluggable. You can have multiple credential types. Okay. So there must be a way for us to, if we, so we could maybe add a functionality before sending the credentials, we could check the store and, but how would we know that we have credentials for this particular case? Well, in fact, for GitHub branch source, I believe it also requires that you have an actual username password credential. I think that's a limitation of that plugin. You can't do, like, you need scanning necessarily by username password because you can't do scanning by an SSH key, for example. That's what I saw has been. Yeah. So I mean, you kind of have a guarantee that you have a credential that, unless it's anonymous access, you kind of have a guarantee that you have a credential that can reach out to the API from the GitHub branch source configuration and project. Okay. So then I would need to create a function which would check the credential store for a use. I'm not, what I'm not able to figure out right now, maybe I need some more time to understand it, is that how would I uniquely identify those credentials from the perspective of Git plugin? How would Git plugin have that information? Or maybe the Git plugin would have that information because the Git Transource plugin has passed down that information to me and then I can use it to check the store. Either that or the GitHub branch source plugin provides you an answer based on the results of the query. Either one of those would work, but I don't think Git plugin would know that or have a way and probably shouldn't reach back into GitHub branch source to try and figure out link mappings of things. But the Git plugin can look at the store and if it has the credentials which are passed down by the GitHub branch source plugin, it can use them if they are mapped with the store. Yeah, I guess I'm wondering what the Git plugin would even do with those credentials. Wouldn't the GitHub branch source plugin do the work? So the problem is that if we need to send when the extension we are implementing, it's a static class, we need to initialize a GitHub object, which is basically it encapsulates the APIs we have for GitHub Java APIs. So it needs the credentials. So at that point of time, it needs the credentials and I cannot take those credentials within the GitHub branch source plugin because that would mean that I have to provide a non-static object to a static class. So what I understand, what my understanding is that the extensions, if they are static classes, we're not meant to take information from the individual objects of that particular instantiation. Or maybe there's something I'm missing here. My question is, I thought about using if the GitHub branch source, initially I thought that if the GitHub branch source plugin has the credentials, why not use them for the extension? But how I was limited there was that the extension is when I implement an extension point, it is a static class and the credentials ID for a particular user credential would be non-static. That means for that particular instance, how would I communicate it with the extension within the GitHub branch source plugin is what I could not figure out at that point of time. And the second thing, my confusion was the context. So what makes sense is that the Git plugin should provide the context where the branch source plugin should scan with which the branch source plugin should scan the credentials because the Git plugin is using those functionalities and the job, which it is running, should be the context it is providing to the branch source plugin is what I understand for scanning the credentials correctly. So... I think that's the other way though. I think GitHub branch source is gonna inform Git plugin of context, right? Ignore me. I think I'm wrong. No, I'm just trying to understand because frankly, I don't have a good knowledge of the credentials API. I just learned enough to implement the scanning abilities, but I don't have... I think I need to study more how context and... I think off the top of my head, I think it's fairly normal if you need to use your credentials ID in a plugin that you would potentially get that credentials ID, get it within your context and then use it. So Justin, you are right here, but my issue is that I want to show you the issue through code. So the estimator, so I've created this extension inside the Git branch source plugin. Now, this is a strategy class. How do I access the context of the current branch source plugin and its credential ID and provide it here where I'm trying to connect with GitHub, the GitHub APIs, and then retrieve the size of the repository? How would I transfer that information within the plugin? What information? The context... So if you're talking about using the context provided by the GitHub branch source plugin and the credentials ID of that plugin only, then we need to provide those two parameters to this extension where it is trying to query the size of the repository. How would I... So right now, how this thing is working is that these three parameters are being provided by the Git plugin to the Git branch source plugin and then it uses the context and the credentials ID to scan for the credentials. Once it finds them, it creates a connection and it gets us the size. That is how this is working. But if we are thinking of use, if we know that if there is a project created by the GitHub branch source plugin, then we would have the credentials within the plugin. Then how would I use its credential ID and the context for this extension is something I am not able to understand, couldn't you? Yeah, that one might be a particularity depending on the plugin itself, maybe. I don't know if there's... Are there common patterns? Do you guys know if there's common patterns for some of these on how their credential IDs are attached to their context? I would think you would need to use your item context to retrieve that from the branch source plugin itself for any instance. Yeah, I confess, I don't know. I thought that Rishabh, this was working the way you had hoped and so I'm not seeing the compelling reason to change what this is doing right now. It is working, so we are getting this size for... I have tested it with a private repository and a public repository, so if I configure the Git plugin to use a username password credential, I am able to retrieve the size and I am able to run the Git to a user as expected but the whole discussion started from the issue that we are narrowing our scope of usability once we are only limited to the username password credential, but I think that that's not a choice we can make, it's what the GitHub branch source plugin supports. And then, so there was a possibility discussed by Justin that what if we use the credentials in the context within the branch source plugin that would make us less dependent on what the Git plugin is providing and because we know that the GitHub branch source plugin would definitely have the credentials if it is being used in the Jenkins instance. The Git plugin might have it or might not, even if it does have the credentials, it might have the type of credentials which are not supported by the GitHub branch source plugin, so that is why if we are passing down the credentials from the Git plugin to the Git branch source plugin, that is how the current flow is working, we are limited to the user's choice of using username and password credentials in the Git plugin. So the problem is that the user might, they will not know about this performance improvement, will not know that using the username password credentials would actually enable something which the user doesn't even know. So the point is that if the user is using the GitHub branch source plugin, the user will be forced to use username password credentials and would add that in the Jenkins instance. But if the user is, if we are looking at this problem from the Git plugin's perspective, from a user who is only using the Git plugin and not aware of the GitHub branch source plugin, for that user, he will use whatever way of credentials, whatever type of credentials they use and it may be username password, it might not be. For us, it will only help if it's username password credentials. That's what we're doing. Yeah, I think the primary challenge is going to be usability for getting a branch source. Like, if you think about how you would maybe need to set that, because maybe I'm not understanding this correctly on how you set this up for Git plugin and how this would look. But I think if I'm understanding this correctly, if you're setting up a multi-branch project or something like that, you would select your credential for your multi-branch project and then you'd also have to, like we would have to warn people, select the same credential that you picked for your branch source so that it aligns in terms of username password and token. Right? That's your concern too, right? Rishabh? It is, but then that's an added responsibility to the user of Git plugin who doesn't, who is not aware of the performance improvement. It's supposedly a hidden feature. User is not aware of something. Wouldn't the user think about why is it happening? Why would it? I'm not sure from the perspective of user how it would go, but then if we have a multi-branch project, we wouldn't even need the credentials, we wouldn't need this API, we would have the cache. So I think if we have a multi-branch project, we would not have to worry about querying the size of the repository via a REST API. So I think this needs more thinking and probably to clarify, to have a clearer discussion, maybe I can make a diagram with the current flow of authentication, how the Git plugin is expecting how the size, how is it working so that then maybe the mentors can review the diagram and the current implementation, the way it's working, because the PR has had, I think it's greater than 16 messages, it's a long conversation, it might be difficult for people who haven't reviewed it yet. So I would add a diagram to it, attach a diagram with the current flow of authentication. And in meanwhile, I'll think more about how... So currently it's a non-blocking, this discussion is non-blocking to the Git tool chooser's release, if you're talking about that, because the Git tool chooser expects the size of a repository from the REST APIs, it is able to do that. Conditioned that it needs a username password type of credential, but if we have that, we're able to do it. So we can say that we support that currently, but meanwhile we can look into how we can expand that scope of usability here. Yeah, because I mean, I guess you have a couple of pieces of work and I think to Mark's point, if this works for now, you can refine the extension thing as its own probably, because the Git tool chooser for Git itself seems like its own big chunk of work that we've been working on for a while. So I think if this works for now, like probably refining the extension seems reasonable. Yes, so... Mark's thinking. So currently after this discussion and the changes I've done, the status of Git tool chooser is the interactively, I have the current changes I've done, I've tested the Git tool chooser with multiple implementations and in the case of a cache, cache repository, if we have that, then it should use that to estimate the size and I have also used a modified version of the GitHub brand source plugin, which is local in my instance. And I've used that to interactively test that the Git tool chooser is able to recommend the optimal implementation while we are checking out a repository using the Git plugin. So I have been able to done that the next step after the review we have for the current changes is for me to instantiate to add the Git tool chooser everywhere the client is being created within the Git plugin. So I haven't done that. And the reason was that I wanted Git tool chooser to be a functionally working class before we decide to add it at places. It should work how it's supposed to. And after I think what Justin and Omkar pointed out, I assume that we're, and one case I missed which Marcus pointed out, I will add that. I assume we are code-wise we are done with Git tool chooser. But again, I think the mentors need to review it first. There might be things I've missed. There are more test cases I have to add. And those are related to unsupported extensions when we're using that in the environment. Unit test cases do not cover those cases where we are using unsupported extensions and JKit should not be recommended for those cases. So I have to add those test cases. Test cases-wise that is what I haven't added for the current change which is using the existing, the system-provided path, Git executable path and then deciding to implement, to recommend the implementation. I've added those cases. I've just shown you the cases that the latest commit in our PR number 931. Apart from all of this, I have one more thing I needed to discuss was the extension work. So that is something I am doing parallely. And I have been working with Liam on merging the PR for the GitHub branch source plugin. I was stuck at creating unit test cases for that because I think it was hard for me to mock a GitHub web server and then ask for, so what is happening with my test cases is that it's actually trying to contact the HTTPS GitHub API server. And that is creating a problem because the credentials I'm providing are they're not correct credentials. So I'm stuck at that point and I think Liam has told me that he will help me in writing those cases. And once I'm able to mock that, then I think those test cases are done and we will be able to, I have since I'm talking about the extension, I have one more issue I haven't discussed with the mentors is that, so for the GitHub branch source plugin, Git plugin is added within that plugin via the GitHub plugin. So the Git plugins dependency is not directly exposed in the form of the GitHub branch source plugin. The GitHub branch source plugin contains the Git plugin using the GitHub plugin. It comes with the GitHub plugin dependencies. I checked the dependency graph for the GitHub branch source plugin and that is how I got to do that. So now my problem is that I want to use a class of the Git plugin which is not available, which the GitHub plugin does not have because it is depending on a 4.2.2 version of the plugin. And so my concern is that if I directly add the Git plugin, the latest version, the snapshot version I have to the form, it creates the enforcer issues, upper bound dependency issues. And so I was not able to solve it at that point for, so I did a very bad hack. I just replaced the jar of 4.2.2 with my updated jar in the M2 repository and that is how I'm working with it right now. But I suppose that is also a concern I have with that plugin and yes ma. There is a trick used inside the Git plugin where it has an optional dependency on some other plugin. I forget which plugin it is that's optional matrix plugin maybe. And inside the source code it does a dynamic check to ask if an API is available. That might be an option here at least short term that you can use reflection to ask, is this API even available? It's uglier code, it's less attractive code. It's much better if you can update the palm to say I depend on this newer version but a reflective check, is this API even available? Is used in some other places? Okay, okay. I know the thing you can do is try and get a local GitHub build and bump its dependency and then bump to that GitHub version. Right. If you're just trying to get infrastructure and getting this going. But then ultimately when we are, even if we release the latest version of the Git plugin, wouldn't we would have to go to GitHub plugin repository, the maintainers and then update their version of Git plugin so that we are able to use it in the GitHub Brunswick plugin. Yes. Yeah, what I suggested is probably your next PR. You'll have to change it anyways. So you'll have to send them a PR so that might actually work out well for your local testing. And you'll also be kind of like testing out that PR as well if that works for you. Perhaps you need what Mark's suggesting to you. But you'll have to do that at some point. Okay, so. So for me the task right now, the first is to add the case I've missed which Mark pointed out with the implementations that if we have Git and JGIT Apache, we should recommend JGIT Apache and not just look for JGIT. That is what I have to add. And the second thing is that functionally I'm done with the GitHub Brunswick plugin extension implementation. I think I can, I will wait for Liam for working on the tests but the classes is done. And I will move on to the GitLab Brunswick plugin or should I move to the GTI Brunswick? Is there a preference to which Brunswick plugin we should move on to or? I would guess personally that the GitLab Brunswick plugin is probably more popular but you could check stats.jankins.io or plugins.jankins.io to see here. Actually, I'm just gonna do it. It's easy to ask the question. How many installations are there of the GTI plugin? Okay. There are 1,700. How many are there of the GitLab Brunch Source plugin? There are 4,000 of the GitLab Brunch Source plugin. So just based on the count of installations, GitLab Brunch Source wins. Okay. And actually, if you're already talking to Liam too, I wonder if he would be able to give you an opinion on the credentials thing or GitLab Brunch Source. Yes, actually, I could not, I should ask him, yes. And I'll ask him when I'm having the conversation with a good test, I'll ask him, yes. Yeah. Again, don't need to completely couple this all to, especially GitToolChooser and stuff like that but that seems like a good thing to answer for the extensions, especially as you move to other branch source plugins because you might end up using similar patterns. Yes. Agreed, agreed, Justin. That's a great advice. I'll do that. Because I think it's best if we resolve the authentication discussion before moving on to other Brunch Source plugins because if we do not, and then we have to change the GitToolChooser for some reason, then we would have to change all of those Brunch Source plugin extension implementations as well. So yes, that's a valid point. It's still reversible at some point. So don't worry about if you make a mistake or something, we'll do that. Okay, so I think I've taken a lot of time today. Double of what we usually take. Thank you, everyone. Before you close, I've got one more open question. Any objections to a release of Git Client Plugin 3.4 so that we have a published version with the unsupported command in it? Good. Okay, no objections. I'll go with it. Thanks. So it'll probably happen within the next day or two. Okay, great, that's it. All right. Thank you, everyone. Awesome. Thanks. Bye.


Hello everyone, my name is Nikhil Gadjum working as assistant professor in department of computer science and engineering in Valchand Institute of Technology, Solapur. In today's video, we will see a deletion of node from singly circular linked list. Learning outcome at the end of this session, student will be able to write a C program for deletion of node from singly circular linked list. Now this is the difference between singly linked list and singly circular linked list. In singly linked list, last node's next part will have null to indicate this is the last node or end of the singly linked list. In singly circular linked list, last node's next part, next pointer holds the address of start or holds the address of first node that becomes a circular. So this is a singly circular linked list. Now we will see first algorithm for deletion of node from singly circular linked list. Then we will see a program. Now step one, first task is to check whether a singly linked list is empty or not. If it is an empty, we cannot delete the node from the singly circular linked list. You are simply written from that function. Second step, if singly circular linked list is not empty, then define two pointers. First one is TAMP and second one is PE TAMP. Some programs use three pointers instead of two, but we require two pointers. TAMP will always points to that node which is to be deleted from singly circular linked list. So first initialize TAMP to the start, then traverse TAMP to that node which is to be deleted and set every time PE TAMP equal to TAMP before TAMP is moving forward. So PE TAMP is a previous to TAMP pointer which always points to previous node of TAMP and some programs will have N TAMP also which is next to TAMP which will hold or which will point to the node which is next to TAMP. So after TAMP is pointing to that particular node which is to be deleted, next task now check whether a node to be deleted is first node or not. If it is first node, again you are checking if that singly circular linked list contains only one node, in that case set start equal to null. If singly circular linked list contains only one node and that node is to be deleted, so after deletion of that node singly circular linked list don't contain any node and that will become empty. If singly circular linked list contains more than one node and you want to delete the first node, in that case set PE TAMP at the end of singly circular linked list or point PE TAMP at the end of singly circular linked list. In that case after pointing set PE TAMP arrow next is equal to TAMP arrow next and start equal to TAMP arrow next. So you are deleting the node where TAMP is pointing and last one set if TAMP is not the first node. If node is to be deleted is not the first node, in that case set PE TAMP arrow next is equal to TAMP arrow next and at the last for all these steps write one extra step called as free TAMP. You have to free whatever allocated you have allocated memory for TAMP. So this is algorithm for deleting a node from singly circular linked list. You can pause the video and you can answer this question. In singly circular linked list time complexity for deleting the last node is, you can pause and you can answer for this question is big of n. You cannot point directly to the last node which is to be deleted, you cannot go directly to the last node in singly circular linked list because there is only one pointer which is next pointer. So you have to traverse from first node to the last node for deletion of last node then only you can delete the last node. So complexity time complexity will be obviously big of n because you are traversing n nodes for deletion of last node. Now we will see a program or we will write a function called as delete to delete a particular node from the singly circular linked list. This is online GDB where you can write your code and you can compile and execute. So this is your main function. Now it contains four cases. So case one is for insert at begin, second one is display. These two functions I have already implemented. Now delete function which we are going to write a code right now and exit for exiting from the program. So this is insert at beginning, this is a code, this is a display function. Now delete function we are writing now. First use one variable called as element to take the value from the user which node is to be deleted. So I have taken int element. Next one this is your struct node star temp comma star pre-temp comma star n-temp. n-temp is extra pointer you can take or you can write this code without n-temp also. So first you ask user which node is to be deleted. So this is printf enter data which is to be deleted. This is canf %d comma address of element. Now check if this element is equal to start arrow data means you are checking whether a node is to be deleted is the first node or not. So in that case now first you initialize temp to start and you initialize n-temp is also equal to start. Now traverse n-temp to the last one. Now how to traverse that while this n-temp arrow next is not equal to start n-temp equal to n-temp arrow next. So every time you are moving n-temp till it reach to last node. Now once n-temp is reach to last node in that case n-temp arrow next equal to temp arrow next start equal to start arrow next move start to the next node and next free temp. So you are deleted first node. Now suppose a node which is to be deleted is not the first node right else part means a node which is to be deleted is not the first node in that case you have to traverse n-temp from first node to that node. So this is temp equal to start initialize temp to start now write down while this temp arrow data is not equal to element you have to move temp to the forward direction till temp reaches to that node which is to be deleted. So every time p-temp will point to previous of temp so p-temp equal to temp and temp equal to temp arrow next. So p-temp and temp both will move side by side. So once temp has point to that node which is to be deleted now join the pointer parts how to join p-temp arrow next equal to temp arrow next and last one this is free temp. So what we have done first we have initialized the pointers next we have taken the value which is to be deleted I have stored that inside a variable called as element. Next check whether element is double equal to start arrow data means first node you have to delete then temp equal to start initialize temp and n-temp both to the start then while n-temp arrow next is not equal to start n-temp equal to n-temp arrow next move n-temp to the last node. Once n-temp has reached to last node the next step is n-temp arrow next equal to temp arrow next start equal to start arrow next move start to the next node and free temp which will deallocate the memory which is allocated earlier else part means whatever node which you have to delete which you want to delete that is not the first node in that case. So use while loop to traverse temp to point to that node which is to be deleted. So while temp arrow data is not equal to element p-temp equal to temp and temp equal to temp arrow next then join the pointers p-temp arrow next is equal to temp arrow next and free temp. We will execute this we will check whether this code is working correctly or not. So first I will insert some values insert singly circular linked list. So this is insert at beginning suppose 50 again insert at beginning 40 insert at beginning 30 I will display. So now this singly circular linked list contains 3 nodes 30, 40 and 50. Suppose I want to delete now so I will call delete function which data is to be deleted is suppose I want to delete 40 suppose so I will display call the display function now 40 is deleted from the singly circular linked list now it contains only 2 nodes okay we will move back again to the presentation. So these are some of the references I have used for creating this video. Thank you.


My fellow Americans, I have two subjects to speak about today. First, taxes. I think you know how strongly I want to improve our unfair and complicated tax code. A problem that's grown so monstrous over the years isn't easy to correct. But we're on the edge of a breakthrough that can bring personal income tax rates down to the lowest level in over 50 years. This past week, the House began final consideration of crucial reform legislation. But because of a parliamentary impasse, the House was unable to vote. Unless it's overturned, the result can only be a defeat for all Americans who know taxes are too high and the system is hopelessly unfair. Today, I'd like to make a personal request of the House of Representatives. You have the opportunity to provide the relief millions of Americans demand. If together, Republicans and Democrats would agree to a format for considering this vote, I believe there will be sufficient bipartisan support for tax reform to pass. This is one time politics must be put aside on both sides of the aisle. There are ways to permit this much-needed reform to go forward. There are ample excuses for not going forward. There will be no justification if it does not. Now, I must address recent disturbing events in a country close to our borders, the Communist dictatorship in Nicaragua. Nicaragua today is an imprisoned nation.  They stripped the Nicaraguan people of their rights by a state decree last October 15th. Yet that decree only made official, and by their reckoning permissible, the theft of liberty that took place years ago. No institution more deeply embodies or glorifies or seeks to perfect the moral and spiritual goodness of man than the church in all of its denominations. Yet in Nicaragua, the church is the enemy. Protestant ministers and lay people have been arrested, interrogated and tormented at secret police headquarters. Some forced to stand naked in very cold rooms for long periods. A tiny population of Jews was bullied and driven out. Cardinal Obanda Ibravo, a great hero of truth and courage, is prevented from speaking freely to his flock. The state police have expelled foreign priests and drafted seminarians who are virtual prisoners in the Sandinistas Armed Forces. And the Catholic Church's newspaper has been seized, and Radio Catolica censored, sometimes shut down entirely. The same dictators who insulted Pope John Paul II also stopped Radio Catolica from broadcasting a letter from the Pope and this beatitude. Blessed are those who are persecuted for righteousness' sake, for theirs is the kingdom of heaven. The truth is, these men are nothing but thugs, a gang of hardcore communists to whom the word of God is a declaration of liberation that must be stamped out.  But there's a cause for deeper concern. The specter of Nicaragua transformed into an international aggressor nation, a base for subversion and terror. Some 3,000 Cuban military personnel now lead and advise the Nicaraguan forces down to the smallest combat units. The Cubans fly the Soviet assault helicopters that gun down Nicaraguan freedom fighters. Over 7,000 Cubans, Soviets, East Germans, Bulgarians, Libyans, PLO and other bloc and terror groups are turning Managua into a breeding ground for subversion. A delegation of Nicaraguans is now in Iran. Nicaragua's border violations against Honduras and Costa Rica continue. And Nicaragua's connection with the recent terrorist attack against Columbia's Supreme Court is now clear. What are we to do about such aggressions? What are we to do about Cuba's willful disregard of the 1962 Kennedy-Cruze-Chef understanding of which President Kennedy said, If Cuba is not used for the export of aggressive communist purposes, there will be peace in the Caribbean. Well, the answer is more than we're doing now. If Nicaragua can get material support from communist states and terrorist regimes and prop up a hated communist dictatorship, should not the forces fighting for liberation now numbering over 20,000 be entitled to more effective help in their struggle for freedom. Yes, and to reinforce this message, I sent my new National Security Advisor John Poindexter this week to visit the Central American democracies and make clear our commitment to a democratic outcome in Nicaragua. Those who struggle for freedom look to America. If we fail them in their hour of need, we fail ourselves as the last best hope of liberty. Until next week, thanks for listening. God bless you.