Hello everyone. I would like to welcome you all back to our Water Productivity Masterclass series. My name is Lauren Zalinski and I work at IHE Delft on the Water Pip Project. And I'll be moderating the webinar today. And I'm also joined by Abraham Abeshek from MESA MESA who will be helping with the IT and technical part of the webinar. So I think we have some people who have introduced themselves already, but we're very excited to have you all here. And we would like you to say hello in the chat and just put your name, the institute, or the organization where you work from in the country. And it really helps us understand who we're engaging with and the different regions and different sectors of water from around the world. So thank you. If you're new, this is your first week to the Masterclass series. It is brought to you by the Water Pip Project, which stands for Water Productivity Improvement and Practice. And it's a project that is funded by the Dutch Ministry of Foreign Affairs, bringing together organizations focused on water science and water management to improve water productivity in the agricultural sector. So the main organizations that are leading the activities are IHE Delft Institute for Water Education, Wageningen University in Research, and MESA MESA. So you'll be seeing different people from these organizations on these webinars. And so I'd like to welcome you to week three. So we're actually on part two of monitoring water productivity using WAPOR. Last week in part one, we talked about what is the WAPOR portal? What are the different data that you can find on the portal and how to do different analyses on that portal? But today we're going to be focusing on how to take that data and use it in a more advanced way using GIS and Python to get the specific information that you're looking for for your project. In the remaining three weeks, we'll be talking about water productivity and sugar cane production. So focusing on a specific crop and also looking at different scales, such as the river basin scale. The following week, we will discuss socioeconomic water productivity. So how can we look at water productivity from different viewpoints around social uses and economics? And then our last week, we will talk about monitoring water productivity using Aquacrop, which is another open source software focused on irrigation and crop analyses. And just a reminder, if you would like to rewatch any of these webinars or download the presentation, there's two locations you can go to. First is the Waterpip project website. So that is waterpip.un-ie.org. It's functional now. It's live. We're very excited. So you can go there and look at the different videos, download the presentation. If there were unanswered questions, we will be putting them on the website as well. And in the future, we'll have more resources available to look at different parts of water productivity, more reports, and things like that. So please visit our website frequently and you can stay up to speed on those things. You can also go to the waterchannel.tv and go to their webinars page. And then you can see all the previous webinars from this series, including the videos and the presentation. So our agenda for today, we're going to start with a video from Vic Tran from IHE. And she will talk about how do you download the data using the WAPOR portal. That will be followed by Sadiq Pariz, also from IHE Delft. And he will talk about how to process the data from the WAPOR portal in QGIS. After that, Vic will talk about how to do bulk downloading using Python script. And then Abebe Chakala, also from IHE Delft, will take some time to talk about how to assess water productivity and other irrigation performance indicators using Python. And then after that, we will have a Q&A session where we can answer everyone's questions of the group. So like the previous weeks, we will not be taking breaks in between the presentation. But if you have a question, please put it in the chat box. And Abraham and I will be collecting the questions and we will put it into the Q&A session at the end. So we encourage you to make comments and questions in the chat box during the whole time to stay engaged. And just as a quick note, today the presenters will give a short presentation, but then we'll be live sharing their screen, so using the different programs. We're going to make the screen a little bit bigger so you can see it better. But if you would like to zoom in to see really specific things, you can use the buttons on the top right-hand corner of this part of the screen. And you can zoom in on where the mouse is. And if you click all the way to the right, where the options are, you can choose your zoom level. So depending on which view you would like, you have some options to see things better. So with that, I think we're going to start with the video from Vick. And she's going to go through step-by-step on how to download data using the WAPOR portal. My name is Vick. I'm a member of Water Accounting Group at IG Delve. In this video I will show you how to download data using the WAPOR portal. I will now demonstrate how to download raster data from WAPOR portal. You can either download a whole layer data through the catalog or using the crop raster in the analysis. Let's first look at the catalog. In the catalog you can select which level of WAPOR data you want to download. You can also filter by theme, like water productivity, water, land, climate. For example, if I filter the data layer by climate theme, I will find reference evapotranspiration and precipitation data layers. When clicking at the data layer card, I will find a full description of the data, including unit, special resolution, special reference system, and conversion factor. Here you can filter the data set by year or month. For example, here I want to collect data for October 2009. Then I will just have to click on download button. Then I will be downloading the reference evapotranspiration data for the whole WAPOR region. Since the conversion factor of reference evapotranspiration layer is 0.1, we will need to correct the downloaded raster with this value. I will open the downloaded raster in QTS. Then we will raster calculation to multiply the raster data with 0.1. Then we will have the corrected raster. And here we have the unit in millimeter per month. Back to the catalog. For the level 2 national 100 meter, you can also filter the raster by countries and by water basins. Similarly for the subnational 30 meter level, we will need to select the area with available data. Notice that with the catalog you will download the raster data for the whole WAPOR region or whole country or water basin. So in case you want to just download for example a very small area, you will need to use the analysis tool. Before that we need to select the layers that we want to clip raster. For example here we have gross biomass water productivity. I will select to download net biomass water productivity. And you can select the time also. Or we check the legends if you are selecting the right layers. And now we click on analysis. In the operation you need to select area raster download. Here you click on select area. You can choose one of the saved area in your WAPOR or draw a new one. Or upload a shape file. So let's draw a new one. Here I want to collect data for the Office Dunige in Mali. Let's for example draw a simple shape around this area. Then I will save this for later use. Click run operation. Once it's completed I can click on the green button to download. Now I will open this crop raster in QGS. Now I will zoom in these layers and hide the other layers. Now don't forget that all WAPOR layer will have a conversion factor. So we go back to the catalog and check the net biomass water productivity information. So here we find that the conversion factor is 0.001. So now I will go back to QGS. And similarly I can use the raster calculator to multiply with the conversion factor. And then I can save this as corrected raster with the unit is kilogram per cubic meter. And this is the end of the demonstration. Thank you for watching. Thanks Vic for that video. I think it's quite helpful to see how to do all the downloading procedures on the WAPOR portal. Because we will need them to do the analysis in QGIF. So now Sajifari, he will present more on how to do this in QGIF. Hello everyone. I hope you can all hear me well. So I will continue from where Vic stopped. I will continue to show you how to process WAPOR data which we have downloaded for a particular study area. And how to process that data to compute some of the performance indicators in QGIF. So here is a study area which me I think my following presentations we will all focus on this particular study area. This is the sugarcane scheme at Sinawane in Mozambique. And the season which we are going to consider is 12 months from September 2018 to October 2019. In this particular session I will focus only on one year of one season in one year which is 2018-19, the crop season. I will be mainly showing three WAPOR data, three variables from WAPOR which is EPA, actually WAPOR transpiration, transpiration and reference EPA. And all the data sets which I am going to show you is decadal. So one of the time frame which WAPOR offer is 10 days data sets. They call it as decadal data sets. So for a year there will be 36 maps. So if you download for a particular year for example 2009 you can see that there are 36 maps. And on the right side you will see 2018-19 maps which I will be using here which cover the end date season. So because the season covers one year it has 36 maps as well. But it can start from 2018-October to 2019-September. So what I am going to show you is to compute irrigation performance indicators in QTIS which are available in the following session. We will explain it further how to do that for all the years using Python. Here I will show you one year. I will show you how to compute these indicators. And this is a workflow. We have three variables actually WAPOR transpiration, reference ET and transpiration. First we have to do a seasonal aggregation to get seasonal maps. We have to mask to study area. And then from the masked seasonal maps we will compute those indicators. So without wasting much for the time let us get into QTIS. Can I share the screen? So I am going to share my screen to show you the process in QTIS. I hope you all are seeing my screen now. So as you can see this for me here the QTIS is open. I have a background map and you can see Sinawane which is the sugarcane scheme in boundary of Sinawane also open here. Now I will show you first how to compute seasonal maps. So let us compute seasonal actual WAPOR transpiration from the data downloaded from WAPOR. Just keep in mind that all the data which have downloaded in WAPOR is already clipped to this rectangle. And also the conversion factor is already applied as we just mentioned in the previous video. So to do seasonal map let me just start from the beginning. I always go to processing toolbox where you get all the modules available for raster and vector processing. I will use a module called r.series which can be used to aggregate a list of raster maps based on different functions. So what we need to do is we have to add all the maps all the decadent maps in that particular year to get the seasonal map. So I will go to the folder add files. So as you can see in I have already arranged it. So actually WAPOR transpiration 36 maps. These are the 36 maps covering that particular year. I will add them here and you can see that 36 maps are selected and aggregate operation I will select. There are many operations available but in sort of average what I need is some because we need to sum all those maps. So I will select that one and you can keep all of the defaults. And you can save this file and give a sensible name and give it a name called seasonal each. It looks like maybe Sajji dropped out for a moment but hopefully he can join us again in a few minutes to finish his presentation. But in the meantime, so maybe we see him again. Yeah, can you hear me now? Yeah, welcome back. Yeah, sorry I was lost again. I'm using connector internet but still. Okay, can you see the screen again? Yeah. Okay, so you can see that the r.series is finished. So the map is open in the map view. Now the next step is to clip this particular raster layer based on the shapefile which is open here at Sinawane. So for that I will go back to processing a few toolbox. I will search for clip and I can see there is a module called clip raster by mask layer. I will select the input layer at seasonal ETA mask layer at Sinawane shapefile. I will select the source and target CRS both are same. So even if you don't select it, it should work. And then I will save this file as clip underscore clip and I will run it very quick. And you can see that I have seasonal ETA already prepared here. I will remove the other one. So similarly I have computed clipped seasonal transpiration and reference ETA maps. So you can follow the same steps later on. But I will open them again. I will open them which I have already prepared previously earlier. So I have the seasonal ETA, seasonal transpiration and seasonal RET here. Let's give some color quickly to the transpiration. See I am doing some random color scheme here. So this is how transpiration looks like. Now coming back to my presentation, here you can see the first indicator which we need to prepare is equity. And equity is coefficient of variation of the seasonal ETA over the ender study area. Here it's safe for us because the ender study area is one crop which is sugarcane and it's already clipped to that boundary. So we assume that there is no mixed crops here. And it's true for in the case of Sinavana, it's sugarcane. So let's compute the CD coefficient of variation. And to do that, I will use a module called r.univar. Univar computes the univariate statistics of a particular raster map. And I will open it. I will select the seasonal ETA. And for another indicator which is coming following this particular indicator, we have also have to compute the percentile, 99 percentile. So I'm also computing 99 percentile here. I will use coma as a separator. And then I will save the univariate results into a text file and run it. So as we use the coma as a separator, it can be opened as a coma separator file. And I have already opened it here and you can see that it compute a lot of statistics. And one of the statistics is coefficient of variance which is 13.8. So that is you can consider as equity. And you can also see that it has computed 99 percentile here, which we will use later. The next indicator which we are interested in is beneficial fraction. It is kind of easy compute. It's seasonal transpiration divided by seasonal evapopranspiration. So as we showed earlier, we will go to raster, raster calculator. I will set the output layer first and I will use this name. I will call it bf2009 in clear. And what is the equation is ta divided by eda. And it should give me the beneficial fraction. We can also give it a nice color. And it will give you much more information. So this is the beneficial fraction map for 2018-19 crop year. Now let us do the next indicator which is adequacy. For adequacy, we need potential et. So the equation we use is actual evapopranspiration divided by potential et. And the potential et is calculated by multiplying seasonal reference et by kc. And the kc value which we are using here is 1.045 for sugar cane. So what we need to do is we will go to the calculator again, compute seasonal etp. Then select reference et multiplied by 1.05. That will give you etp, etp is there. Then let us do, so as I said adequacy is eta by etp. Now let us select eda divided by edp which will give you the adequacy map. In this case maybe you are interested in the univariate statistics. So you can quickly do statistics here, faster layer statistics. Select adequacy, just run it to give you minimum, maximum and the mean value. Maybe the mean adequacy value is more meaningful here which is 0.63. Now let us come to the last one which is relative water deficit which is 1 minus eta by etx. So I have already showed, I have already shown you how to compute 99 percentile because in this case the maximum crop et the compute as 99 percentile of the seasonal etx. So one way of doing it. And I have already shown you using r.univar how to compute different percentiles. There was a question in one of the last webinars. So that is already there in our statistics table, 1644.27 millimeter per season. So we will use that value to compute rwd. So we will go back to RASA calculator again. I will use, I will give it a name rwd2090. And I will use the expression 1 minus, open the bracket. Let us verify this tta divided by what is the 99 percentile, 1644.27. 1644.27 plus the bracket et cetera. And I get the rwd. You can also use it, use the same color as before. So this is the relative water deficit. As I shown you before you can use the RASA layer statistics to compute the mean rwd value which is 0.19. And yeah, and that is it from my side on a hands-on demo. Thank you Sajin. That was an explanation on how to calculate in QJS the different performance indicators. So if you missed a few things or you would like to re-watch the videos, remember you can always go to the Waterpip project website or thewaterchannel.tv and you can re-watch these videos again and again to make sure that you catch all the steps that this went through. So now we're going to move on to Bix Tran who will talk about how to bulk download WAPOR data using Python. Lauren, hello everyone. So as you can see from the presentation by Sajin and also from the demonstration using VapoPorto, you see that using the VapoPorto there's no programming needed and it's very ready to use. The portal has very nice visualization. However, sometimes it's very easy to download a few RASA files. But for example, if you want to download and to process a large amount of data, like for example you want to process decado data for 10 years, then with this VapoPorto it might take a lot of clicking and downloading data. That's why we want to use the Vapo API because it's more time efficient to download large amount of data. It can also link with other programming language to automate processing and analyzing data by scripting. So just a brief introduction, what is the API? It stands for Application Programming Interface. So it's just a set of functions to communicate between the Vapo database or the Vapo service with other applications like Web Access or Python or any programming language. In order to connect with Vapo, you need an API token. This is a unique identifier associated with your Vapo account. So in order to get this token you need to go to the website and choose My Vapo and in the profile session you will find the button to generate API token. So this token is private and should not be shared with other. You can save it on your laptop or somewhere. However, if you forget this you can go back to the My Vapo website to create a new one. So in this demonstration, and also in our database demonstration, we will be using the open source Python script that we developed for water productivity and assessment. You can accept by clicking on this link here. You can directly click on this link I guess. Then on the GitHub web page, which is a web service to share all those open source code, not only in Python but also in other programming language. In this web page you will find a button to clone or download. If you want to contribute or as a developer you can clone it or you can also just download the Zify so you can collect all the script in this repository. So in order to run the script that we will demonstrate today, you will need to install Python. We highly recommend that you using the Anaconda Python distribution. When you install Anaconda you will also have Jupyter Notebook which is a web-based application including all the text and picture, also instruction and the life code either in Python or in our programming language. If you are using Python you might have already installed some of these main Python packages like NumPy or Pandas or GDAR or Geospatial Data Analysis. In case you are new to Python, you can install these packages by open the command prompt windows and using the fifth install command with the name of the package and then you can install this Python package. So once you have an active FAPO API token and install all the Python requirements and download the open source script that we provided, you will be able to follow the steps in this bug-dollard-fapo with Python script demonstration. So now I would like to share my screen to show the demonstration. So is everyone seeing my screen yet? So now I hope you can see my screen at the moment. Lauren, can you check if everyone has seen it? Yes, we can see it. So this is the GitHub page as I've showed you before. You can click on this and to download the zip file. Then once you download it and extract the zip file, you will get this folder called vapo-wp. And in this folder you can go directly to the notebooks folder here are all the steps of this demonstration. If you install Python or Anaconda distribution, you can start Jupyter notebook directly from the folder and then here you can start Jupyter notebook from here. So in this folder you will find the six notebooks from step 0 to 5 and here I will demonstrate how to bug-dollard data from step 0. So you click on this notebook and then it will open a new tab. For some of you who have never used Jupyter notebook, there's something you must first to know is this is where you check the status of Python kernel. If it's a circle like this, which means it's now addressed, so there's no code running. In this notebook you will find some cells with the code and some cells with text explanation. So in this notebook, vapo-data needed for whatever CBT assessment will be downloaded in bug and corrected to the right unit. So in this notebook we will download six data sets, actually evapotranspiration and interception, transpiration, net primary production, land cover classification, precipitation, and reference evapotranspiration. So all of this data except for land cover classification will be decadal temporal resolution and the time coverage will be from 2009 to 2019. So as you can see here, it's more than 360 roster per variable, so it's a lot of data and that's why we need to use the Python script. So here you will find the cell with the code inside. In order to run the cell, you will need to click run or you can also press control seat. This is to import some of the models in Python in order to run the script. I will first click run here. And then the vapo module that we will ask you to provide the API token. Here I have my API token. I can copy this and paste it here. And then during that, the Python script will load the catalog information from vapo server. You will only have to provide the API token once because the script will automatically save your token in this folder. If you need to revoke a new API token to create a new one, then you have to delete this file called vapo API token. So just to remind you of the staff to get the API token, you need to go to my vapo and in this session you will see the button to generate API token. And you just have to copy this code. So now you see the number of the cell is one which means the cell has finished running and also the status of the Python kernel is idle. Then next step, we will run this to read the shapefile of the study area. So in this example, I will also provide a shapefile of Cinevan the same with Saty. So this code here to run and to read the extent. So here you will extract the longitude and latitude limit of the shapefile. You can also run this cell to lost the shapefile. And in order to both download vapo data for the study area extent, first we need to define the output directory. Here I put a path to the folder that I want to save data. And then I click run. So here we have provided a script in form of modules so that you can easily just use one function to download the data that you need. Here you only the land cover classification yearly. You only have to provide the arguments for start date and date and the extent of latitude and longitude. You can also select which level you want to download and which vapo version you want to download. So here if I run the script, as you can see here the status of the kernel is busy. So now we start downloading and using this process it will also correct for the unit conversion of the vapo data. So in the end you will have all the data in the right unit. If you want to stop the kernel, you can also click here in the kernel and choose interrupt or restart or shut down. And then the same thing for other data sets you can also just click run and everything will be automatically downloaded in. For example here I save the data in this data folder. So here's all the data has been downloaded and corrected for the unit. For example precipitation in decado data. Yeah. Thanks Nick, is that the end of your tutorial? Very nice. So I think as you can see if you will be downloading a lot of these layers from the Wattport portal it would be really time saving to be able to use Python to do bulk downloading. So again if you missed some details feel free to re-watch the video and follow all the steps that Vic just presented in her presentation. So now we will be joined by Abebe Chakala and he will talk about assessing water productivity and other irrigation performance. So my presentation will be on assessing water productivity and other irrigation performance indicator using Python scripts. In fact this is a follow-up from the previous two webinars. In the first one we introduced the concept and the second one also we showed what kind of indicators we could derive from the Wattport dataset. And in this one we will try to answer how we could calculate those indicators. So this of course will be built up on the download data as Vic already explained. As Vic already explained then we will show the first step which is re-analyzing the data. As some of the layers are coming in larger or coarser resolutions so they need re-sampling. For instance the reference evapotranspiration or precipitation they need to be re-sampled or scaled down to the scale that fits the highest resolution Wattport data we have either level 1, 2, 3 that is under 50 meter, 100 meter or 30 meter depending on what we have on the ground. And then again the other thing we could do as part of the pre-analysis is masking out crop or non-crop areas. And as part of step 3 then once we have those corrected and prepared data we calculate the seasonal water consumption or production. Consumption related to potential actual or just the productive evaporation or transpiration. And as part of step 4 we calculate the seasonal based on the seasonal water consumption then we calculate the irrigation performance indicators that include water consumption, beneficial fraction, equity, adequacy, relative water deficit and re-labelating. And as first step we calculated productivity. So the productivity includes both land and water. And as part of fifth step then we try to identify the productivity targets. So the productivity targets as we defined before they are values compared across all the space in the project area and which are the highest percentile. And in fact we defined when only both water productivity as well as biomass are the highest then that place or spots is identified as bright spots. And then as part of again the fifth step we try to identify where exactly that bright spot is located within or across the project area. In fact we did this systematically so starting from downloading up to the whole analysis and then as already explained we developed a standard protocol for those analysis and those protocols can be downloaded in our GitHub repository which is water accounting slash VapourWP. And it's freely available and I'm going to walk you through that how you could go from step one up to step five. And I will share now my second screen which will show that in the Python screen. So do you see I hope you see my screen right? Thank you. The link is already explained in our previous presentation. It's water accounting slash VapourWP and this link you get all the scripts and also the data. And in fact if you want to use this one there is a nice precise explanation which is in the readme file how to go or what are the definition and also how to go through each steps. And in fact what let's say package to download and install and also even how to install them is already explained here and also by my colleague Vick in her presentation. So now let's say once you have this one then you need to download this and then you can work on that one. So now I'm going to show you how because once I already have the downloaded one here then how will you do the analysis? So if you click then after downloading this is what you are going to write. So you have the data. So these are the data, the ship file are around here in this folder and then the reference actual ET and transpiration and net primary production. Those are the download data which are named or started with one. But the others are with two and three are going to be calculated and I will show you how we will calculate. So the notes or then the scripts are in the notebook. In the notebook then you could open Jupyter notebook either through the command prompt or directly if you type Jupyter notebook then still Jupyter notebook can open for you here. So then we have this Jupyter notebook. This already explained by big. Then I will start from the first step. So the first step is about re-sampling. As I said there are two activities that we will undertake in this step. So the first one is re-sampling and masking out in a proper season. And as it's already explained we need to import either standard libraries or modules that we already wrote. So there is a module that is adopted by our group and then this is from where we are downloading but then the others are standard library and then we need to import them. So I first import that and then and again I import all the data to be re-sampled. So my data to be re-sampled in this case is just to reference evapotranspiration. So I will then import it and then these are the data. And then I create or I indicate the folder where I want to save that re-sample the data.  This one 1L 1L 1RAT then a Decada one. So whether we created or whether we have that folder we can save. For instance in my folder in the directory of C then I will get it in the folder of app for WP data and this one. And then the re-sampling need to start. It may take some time that's why you see here the start and also a dark spots at this point. So when it's finished it become white. So at this stage we can see for instance the size and the shape of the re-sample data. So the size and the shape of the re-sample data is now 151 row and 250 columnars. For the total file name is this much, about 37,000. And the second step is in fact filtering name cropped area. So there is a land classification map in WAPO and this is how the numbers in the class are how they are classified. And the ID indicates for instance 42 is cropped or irrigated land. So in my case I want to do only the analysis for the cropped land. Then I will filter out all the rest and then retain only the pixels which have a land class 42. So this is first importing the data. So the template data I need to use a template data with which I can do the filtering. So for instance I want to filter out from actual evapotranspiration data. And then if I plot just the whole boundary of my project area this is how it look like. But then before removing or filtering out the non-cropped data the actual evapotranspiration look like this one. So it's beyond the project area. Even in the project area there are spots where there is no irrigation which are settlements for instance in this area but they are there. So I want to filter out those and then the step I did in my previous and then here I could show you in the picture also. So this one is in fact after filtering out the non-irrigated area. So you see that now all non-irrigated area are removed. But still there are areas which are part of the analysis. And again I want to remove all of them which are outside my scheme because my scheme is somewhere it look like this one. So anything outside this and even in between should be removed. So in this step then that's what I'm doing. Then this code that that. So this is the first step and the second step once we have a clean data is about analysis. So the analysis will be in our case the first the second step I mean the step to be analyzed in this case is the seasonal water consumption and production. That includes seasonal actual evapotranspiration seasonal transpiration, seasonal reference evapotranspiration and seasonal net primary production. It's already shown by my colleague using QJS by Sajid but now I will do the same not for one season but for multiple season just with a click just using the script we have here. So the first thing is importing the libraries the libraries functions which help me to calculate and the second to analyze and the second step is again importing all the data which I want to do the analysis of. And in this case I need to communicate with the VAPOR because when I let's say when I add the the rasters then the script needs to identify if the rasters fall within that crop season or not. When we download from the VAPOR database there is already around it's by its name. So the name for instance here is level 2 actual evapotranspiration in 2009 and the first ticket which is t5 then for that reason I need to communicate with the VAPOR API and the modules that explained but in this case and always if you use this script I will recommend you only to run these once because when you run it then it saves already automatically in your computer with this name and it is in fact saved here in the data all the names here are the saved one. So once you saved it then the second time you need to just use or read from whatever you saved and in this one I will just read whatever I saved before and then here I will create a folder. So the folder where I want to save the seasonal transpiration actual evapotranspiration and net primary production are named with this folder name. So they are now created already and this function in fact adds all the seasonal from the starting to harvesting and for that in fact it uses planting date and harvesting date and I will explain in a bit where you will edit your own planting and harvesting date. So the planting end or the starting and end of the crop season it's already in the Excel file I saved it with SOS EOS which is starting of season and end of the season and I will show you where I have it in my folder. In the folder in the data at the end you see here the seasonal starting and ending of the crop season. If you open this Excel file of course you can edit the planting well I should just open it okay well until it opens then I can continue. So those planting and harvesting date then if I run it then it reads from the Excel file. So it means I will use those planting seasons that is from 1st of October up to end of September. That is along all the years from 2009 up to 2018-19. Then we start to calculate the seasonal for instance the seasonal transpiration if we click then it save it in a folder because we are giving the folder where it has to be saved plus it's here. And similarly we can calculate the seasonal for transpiration that is just for the whole season. So you might see here it's busy because it's calculating and it adds the graph and then just within a click we have a seasonal actual transpiration of the whole 10 years. And similarly for seasonal transpiration the same can be done and the final seasonal for crop production is the seasonal parameter I want to calculate is the primary production and then again this can be calculated through the same just clicking. And of course you can do the same. I mean that is also the idea of you can use or modify this script to calculate the seasonal water consumption and transpiration. So the third step is calculating the performance indicators. So the performance indicators in this case are water consumption, beneficial fraction, equity, ADKC, relative water deficit and reliability and you could read about those in part they were already explained in our previous two webinars but still you could use some of those references and also that I show you from our Github repository what they mean and also how they are calculated all the equations and again we start to import all the equations, the functions the standard libraries in the item and then of course the first one is calculating the equity. So equity is explained as a coefficient of variation of the actual evapotranspiration across the whole area and then if we want to calculate then it can be also saved these are the values. So for each when I printed so for each season 2009 slash 2010 these are the coefficient of variations for different years and in fact if they are fair or poor those indicators are also mentioned here and again as I said from the reference you could check what mean by fair uniform or not and the third things to calculate among the indicator is the beneficial fraction again for this one we need input of transpiration layer and evapotranspiration layer which are the actual and across the whole season first we read and we import those files and in fact if we want to see whether we import them or not this is a actual evapotranspiration and this is the transpiration if I run it then I can see yes I'm downloading 10 season transpiration and 10 season actual evapotranspiration and then first I specify the output folder where I want to save then in this case I name it it's three beneficial fraction so that is the folder where I want to save it and then if I run it then it means this is the folder and it is created and if I want to know whether it's created or not and where I have it then I just run yeah run also after including this name then it will be in the data with the names the beneficial fraction so then I will calculate again the seasonal and then I can save it also in the folder so if I run this one you see the beneficial fraction maps for the 10 seasons the other parameter is or indicator is actual adequacy it's calculated with this formula which is actual ETA seasonal over seasonal potential evapotranspiration and the seasonal evapotranspiration as explained by my colleague Sejid this average KC that is the weighted average times the reference evapotranspiration from the season so since we are talking about seasonal layer then this has to be averaged over the season as you could see here from the graph the KC varies along the crop stage so initial development and mid-season late-season so we weighted based on the value as well as duration of each crop stage so I think this formula explains so to do that then after calculating for sugarcane we got 1.045 and also we need to import the other parameter which is actual ET and reference evapotranspiration and then again similarly I will specify or create a folder where I want to save the adequacy that is what is being done at this step and finally I can calculate and then I can plot I will show you how the calculation is done so in fact the whole script here is just import and then do the calculation and then save it in the folder and also name it with the name adequacy so if I run this one then we start to generate the map at the same time saving the whole folder the whole layers in the folder and in fact we can also plot this along the time just averaging one value per area and then I can show you here in fact this script is doing that so the adequacy for each season and of course comparing with good performance acceptance range and poor performance are again closer and the other indicators I have in this step is in this step 3 is calculate a relative water deficit which is 1 minus actual evapotranspiration seasonal over the maximum evapotranspiration seasonal again I import the input data which I need to calculate this and then using this script then we can calculate automatically very quick so with this I will go to the here I would calculate the productivity that is another indicator so we have productivity that is biomass productivity and also biomass water productivity and the yield so in this case if I use harvest index then I can talk about crop yield otherwise without harvest next I'm going to talk the biomass so the biomass over the actual evapotranspiration is defined as biomass water productivity so again I start importing libraries which I need to calculate those indicators and also input data or layers to calculate the biomass or the crop yield so that is the net primary production so I collected from this folder and then again I create a folder where to save those layers or those analysis now I have the biome 4 L2 biomass of course you can give your own name it doesn't matter so it means this is the way to calculate it the biomass is we have the net primary production the seasonal one first we need to transform it into the dry biomass by multiplying by 22.22 and then from the dry biomass into the fresh biomass by dividing or correcting by moisture content and then multiplied with a factor for instance if the crop is C3 or C4 crop and in our case sugarcane is C4 crops then we use a value 1.8 which is to transform a light use efficiency which is used in the NAPP in the net primary production calculation in VAPO which is for C3 to the C4 crops and then finally we multiply with above ground to the total biomass ratio so these inputs are needed then if you want to run this script for another crops you need to change those crop parameters for instance for sugarcane we have a value of 0.8 or 80% of the total biomass will be above total biomass and as I said this is 1.8 for C4 crops and for C3 crops it's just 1 because the whole layer NPP in VAPO is calculated assuming it is C3 crops that's why we just use 1 if it is with for instance but for C4 crops then we need to change those parameters and we have a moisture content for instance this is a value which goes from experiment one of our students did as part of his MSC in sugarcane production in Wengie so we have this we just use that value 0.59 and the harvest index we use one if we wanted to talk about biomass but if we want to talk about harvest a marketable yield then we have to change this into the appropriate harvest and maybe in our repository we will add some of those parameters for other crops so we need to import those parameters because I want to calculate for sugarcane and then I can calculate the biomass for sugarcane so now you see it's being calculated and saved in the folder that I created and similarly we can import and calculate the water productivity this script does that one so I can just quickly run because it doesn't take much time once you have the script then it means at the end you have the water productivity map so my final step to explain is how we could identify the bright spot and also how we could calculate the target, productivity target and also identify bright spots and also finally if we want to calculate a productivity gap how we could do that one so again I will start just importing the modules and libraries which I need to calculate and then I also input all the input data which are needed for that it's actually about transpiration, biomass and biomass water productivity and then I will calculate the target productivity the target productivity is defined as 95 percentile of the productivity so it means for biomass 95 percentile of the biomass across the space will be the target and similarly for the biomass water productivity so I could use this function to calculate the target it means now it's already calculated then the calculated the target value can be calculated like this so targets impact biomass and target water productivity of this one and also if I plot just for one season to look like water productivity in the y-axis and then the biomass in the x-axis and then this one will be the target one so this is for one season but then I mean this is also the distribution functions and then when we talk about this point it is around this value but then all points beyond this one can be identified as a bright spot and this one, this function will do that one it will try to find exactly where this bright spot or this spot will be located in our geographical area, project area so it's now for each season where the biomass is greater than 95 percentile are the green areas where the water productivity are greater than 95 percentile are again scattered with the red one and the combination or the intersection of the two are now shown in the spots so those are areas which are which have higher biomass and higher biomass water productivity and in fact we can trace across all the seasons and also we can identify which location is really bright spot and from where we could learn the management the smart management which is going there can be replicated for other areas and then okay if we identify this target we can also calculate the gap that is the productivity gap by comparing what is the target and also what is the actual per each pixel then this the last function is doing that one so first we calculate the first we create the folder where we want to save the productivity gap and then the biomass gap can be calculated while being saved in the folder and then it can be calculated so this map shows for 10 seasons what are the biomass gap and then similarly for water productivity gap this is biomass water productivity so with this one I am close to finish just only showing you now where are those results in the folder so it means now you could see I have already those folders 2 3 if I open them now I have 10 seasons where everything is calculated so feel free to use this free github I mean this this is script and come to us if you have a question thank you very much that was a really nice and thorough overview of how to use that jupiter notebook to go through all the different steps for calculating the performance indicators so it sounds like you need a little bit of familiarity with python and jupiter notebooks but you can still make it work and work it for your own areas which is really nice so just as a side comment if people are new to python or gith there is a course a free course from IHE I will put the link in the chat that goes over the basics of python and qgif so if you are new and you would like to really use this jupiter notebook but this looks very confusing to you that course would be a really nice place to start so I put that link in the chat and I will also put it on the waterpip website that people are interested so I would like to thank all of our presenters really appreciate all of your knowledge and your live demonstrations and now we will move on to the Q&A session so the first question is for a baby it looks like multiple questions so first did you use AETI or ETA for the seasonal actual evapotranspiration and then why is there no download product for ETA and if you use AETI does that mean we have to consider the interception in our calculations maybe you can address those thank you I think this is interesting question in fact when we download the data from WAPOR you might have seen the naming for evapotranspiration for actual evapotranspiration AETI includes evaporation, transpiration and interception and that is just decadent and when I transform it into seasonal I just name it ETA that's actual evapotranspiration still including evaporation from the soil also transpiration which is through stomata and also evaporation from the canopy which is named transpiration so indeed then it includes all those components for Vic can I add any additional data to the layers I choose and save it to the device needs I'm also confused about the question maybe Rania can explain a bit more if it's about the portal sure you can add additional data to the portal to show two or three layers at the same time I hope it answers the questions because I'm not really I don't really understand the question sorry that's okay if she would like to post a clarification please put it in the chat but we'll go on to the next question a nice presentation Seji it will be complex for multi cropping and I think there are a few other comments about multi cropping scenarios using this tool so maybe someone can address yeah thank you yes it will be complex for multi cropping it will be complex from a temporal and spatial point of view so you really need to understand what is the crop calendar and their system accordingly so for example one scheme where we know that it's pretty homogeneous it's sugarcane, it's season is from October to September next year for that scheme we can standardize all these steps so that is what Abebe has done with this Python library so for each scheme you may have to do according to the local cropping calendar process wise yes that is why we showed two aspects one is using QGIS you show that we have to manually choose all the layers and do it year by year at the same time Abebe using the Python script it was able to do all the all the years in one go but of course there is a curve of developing that scripts but that GitHub which is available could be a good starting point to extend it to another area and use this in small scale multi crop areas common in Africa if you meant small areas as in spatially so as you may know Abebe has three levels of data first level is 250 meters which is available for in their Africa and near East countries but I don't think that that will be suitable for small scale multi crop areas and the second level is 100 meters so all the analysis which we showed today was done on 100 meter level 2 data sets so scheme of this size reasonably can be done can be processed using level 2 data but if your area is smaller than what we showed today for selected areas we have level 3 data which is available in 30 meters but it's not available for in their Africa so if you go to the WAPR website you can see which all areas 30 meter data is available I think it is on the way I'm excited for that clarification another question how do you validate your results versus observed data Abebe in the Sinawana case you want to answer this I think we had some time in fact for validation it's very crucial to have observed data on the ground so if we are referring to evapotranspiration or transpiration data then of course we need to have observed one if possible I haven't done or we haven't done that one specifically for this one but of course we have done this in another publications which is a quality analysis and you could reference those documents maybe you could put those links big in the chat box so there are already some analysis and comparison in that case but for this specific Sinawana case we compared for with FAO statistics reported data for CIGETCAM but then the value we got is average over the whole is and it means including FAO and FAO but then it looks like very reasonable in that case and the other thing is we try to compare also conceptually for instance in our previous in our second webinar our colleague Polat Karimi already showed for instance how the KY value yield response developed impact not necessarily for this area but for BECA value for instance fit the theoretical one and similarly we try to also explain for instance if we draw the averaged water productivity the normalized one whether it speak C4 crops also then that is also another conceptual validation we did so yeah then in terms of yield for specific to Sinawana we did a comparison and it's more or less within the observer or reported in FAO statistics statistics that was a really detailed answer and moving on to the next question can we compare or open two layers QGS and in QGS you can open two more even more than two layers however I think there's not yet an option to put two layers beside each other so you can overlay one layer on top of another and then visually you can see the differences thanks Vic Sajid here he's back you took the KC value yeah I think his presentation has already explained how the KCU we came up with the KC value of 1.045 you're right it's over the over the end of the whole growth period and also it's based upon some experimentation study which we one of our students has conducted in the area so you're right I see Marlos already answered that but yeah no what a quality our colleague Marlos she's answering through the chat box indeed yes it's only better quality and only production related things are in VAPO not related to water quality nothing is we don't have data water quality related layers in the VAPO and a question from Benjamin how much time would it take to program a user friendly front end so instead of going to the I think it depends it depends on the skill of the developer but if you are interested and willing to learn then I highly recommend to start with looking at the source codes from our script so with open source you can look at all the script from the models to see how we scripting out all of these Python codes from the VAPO API so based on that if you understand the VAPO API in general you can start scripting your own tool I hope this answers the question if people are interested in collaborating please contact us if that's a project you would like to work on next question are there any light or offline yeah I think yeah the good thing about VAPO is that it has very high resolution compared to other remote sensing products like 100 meter and you see 30 meters is very high for ET products so if we have light version then we have to reduce a lot of the quality of the resolution of the data you can have offline if you download the data of course to get the data you have to have internet access Sachi do you want to add something no I mean you covered the best way is to download the data whenever you have access to good internet and then do all the processing offline so whatever indicators and what productivity assessment which Abebe has shown is being done offline so only the data download part was online you need internet only for that no this question how long will it take to download all VAPO data the question is the answer depends on the internet connection the speed of your internet and normally if I want to download the data for the 100 meter resolution of monthly data for 10 years for 10 years for example in the Niger basin the area is about 2 million square kilometers I guess then it will take a few hours with my script up to 3 hours so it will take a lot of time because of the high resolution and also depends on the area that you want to download so yeah I think you said 3 hours right? it's a very large big river basin maybe I can put it also in the context of Sinawana for instance if I want to download only one layer then it will take not more than one hour in fact and of course if you want to download one layer by one layer time separately it might take a lot but still you can run the script in parallel and you could download the whole at the same time it means in one hour you could have the whole so that is in fact based on the connection I have here it might take a bit longer for other area of course or less or faster depending on so if you want to download raster data it would be fine if you need to download raster data but if you only want to extract some time series I recommend that you use a portal and extract the time series from the rasters because if you use a portal with the analysis tool that I've shown in the previous webinar all the processing would be done on the web portal then you can just download a very light file of CSV file with all the data for the point or the area of your interest it seems like there's different ways to work around the issue of internet speed one more question we know that satellite date that's currently in the African condition so how sure are we in using yeah again it depends well in general in Africa we have small holder farmers then if those sizes the small holder farmer size if they are too small than even smaller than for instance one pixel size it's very difficult because a pixel covers not only the size of one farmer but also even the neighboring farm or maybe the farm boundaries or other crops and so on it's very difficult again it depends for instance maybe if we are talking about the water consumption for instance still reasonably we might speak but if it is crop specific like yield or biomass indeed it's difficult but yeah again we have also in the VAPOR the finest resolution which is certain meter and in some older farmers again it's even less than certain meter by certain meter again it could be difficult but again what I would say is whether it work or not for instance if the small holder farmer is still a bit larger in extent it's possible to calculate and then to compare with the reality on the ground and then to speak otherwise just judging only without doing anything would be difficult and also some of the things which are happening now on the ground are also becoming smaller but again even larger in a sense for instance in Ethiopia I know there is a cluster farming it means some couple of farmers together they crop the same crops for instance weeds and it means you have weeds almost everywhere in one neighborhood so it means you can just do the calculation yeah that's what I could say I think that's our last question for the webinar we only have a few minutes left so I would like to say we'll be moving on to different topics so if you have more questions about WAPOR please feel free to go to the portal and look at the documentation that is there but IHE is also working on an open course so we are developing an online course focusing on water accounting and water productivity using VAPL so the course will have three models the first model will cover how to use VAPL how to develop using QJS also Python programming and the second model will focus on water productivity assessment and the third will be about water accounting so at the moment the course is not ready yet but if you are interested in the course and you want to join please after this webinar you will see the feedback form you can provide us with your email and then once the course is online we can share with you the link so I hope if you are interested you can join with us again the survey really helps us figure out who we are engaging with and how to and you can also sign up for mailing lists for the project and keep in touch and have the latest news on everything that is going on so with that I would like to thank the presenters thank you very much for your presentations and your input and I would like to thank Abraham in the background for making this all work so smoothly and also all of you who participated today