Hello everyone, welcome to NPTEL course on Emote Sensing and GIS for Rural Development. This is week one lecture three. As indicated earlier, this week is mostly for you to get introduced to the topic and to understand why do we need to contribute to rural development, what are the issues in rural development and what are the reasons for the issues. We will also focus on tools that are available to lessen these issues or provide solutions to the problems and more importantly, what are the open source systems? My definition open source includes free software, free data or open to public tools. So let's go ahead with the lecture three today. In the last lectures, we looked at the water security and how it is tied to rural development. Another thing which is very, very important is food security, malnutrition in rural context. Food security doesn't only mean that are we producing enough food, but is it also going to the vulnerable people and in this case, the most vulnerable are the rural population. Sometimes it is sad to understand that those who produce the food cannot afford to consume it. We will go through the reasons on how and why and what can be done in this context. To start with, while we discuss this important topic today, as per the news that came up today, the world has hit 8 billion population and the projections are not that good also. The hope is we will be peaking by 2060, you can see here a peak is still rising. So we are on the rising level of the population. And due to many, many schemes and information, education, training, the population curve is expected to come down after 2060. And then India's population will come down hopefully. We should understand that the population has increased not only because of our past activities in having more children, but also it has been due to advancements in science and technology that has saved many lives. So the death rate has come down, the population has still increased. So there is an ongoing trend and we cannot expect sudden tapering off or sudden peaking and then coming down like the COVID cases we saw last two years. It will be slowly growing and then slowly coming down. So for this, approximately we are one seventh of the world, India's population and 70 percent of the population lives in rural entities. So leave the population that is at the rural entities, for all this 1.4 billion as per this data, 1.4 billion, 1.7 billion people in India, the food mostly comes from within India. We do import food produce, but it's not that much. Still the major staple, as in rice, meat, pulses, cereals, oil, all these come from local entities. Some of the oil like sunflower and palm oil does come from outside, but mostly we do produce our coconut oil and etc., sesame oil, mustard oil. So to cater for the entire population, there is tremendous stress on rural entities. And this stress comes into major resources which is water and the soil fertility, etc., that contribute to the food production. So the natural resources as water has been highly stressed, which we saw in the previous lecture. In today's lecture, all these population increase and the slowing or lessening of the soil fertility and water resources is contributing to lesser food production or yield as we call crop yield from the ground. And this impacts food security. So food security is important for development. At the end of the day, we all eat food. Development happens or not, you are in a high developmentary or in a poor country, we need food. The only common denominator here is food. May not be a luxury food, but it at least should be the staple staple food as rice, millet or something. So we have to produce food and that is why food security is very important. So please understand that rural development should incorporate food security, but also for the nation's development, we do need food security. Suppose we just develop industry, produce more cars, computers, cell phones and the food security is not attained, then we will have to go and ask food from other countries wherein we'll have to be dependent on other countries for the basic needs. So food is always a basic need, food, water, clothing, air quality, etc. It is important to understand that even though our development is speaking, which is healthy, your GDP is rising, we need to make it sustainable, but by also considering the food security and the food security comes from rural entities. So we need to develop rural economies to support the Food Security Act. Another widespread thing is the man nutrition, which means people get food, but is it nutritious? In those days, we had chicken and eggs coming from the local breeds, but now we do have other breeds, right? So quick, fast growing chickens, more eggs produced, etc. So if you compare the nutritious value of both these produce, you will see that the local variety and the native species would have high nutritious content compared to the broiler or poultry industry chicken. However, this takes a long time to produce the desi chicken we call or the native species, whereas the poultry chicken is quick, which is better for the Food Security Act, right? In those days, just compare this to a small comparison for those who eat non-veg. In those days, I'm talking about like 20 years ago or even 30 years ago. The food we eat, the non-veg or because non-veg consumes a lot of effort and water and other resources, that's why I'm picking on these aspects. Suppose we would have only once a week or once during the festival seasons, but now due to the demand and the affordability because we do have a high GDP compared to the previous 30 years, we are buying and demand is high, so we have industry produced. So food security is important, however, is the food given at nutritious level is also important. Otherwise, we'll be spending more money on health issues. So the green revolution was a big part, as you would have known, because the population was increasing. Before the green revolution, we used to import a lot of food. There was a lot of starvation, rationing of food, but nowadays we don't see that much because of thanks to green revolution. The way green revolution expanded, there is a lot of debate, is it sustainable or not? We would only concentrate on how it impacted the food security because once you understand that concept, you will also understand that for rural development, green revolution is important, however, it has to also be sustainable. And for that, there are a lot of indicators that we can attach to green revolution and this increase in food productivity. And that can come from your remote sensing and GIS, this is how we tie it up because there's no data. So for example, I need to go 20 years ago, 30 years ago and look at the soil fertility, the water availability, there's no data, observation data. But if you use remote sensing and GIS, yes, you can go back to at least 40, 50 years. Some level of accuracy is there. Right now, the accuracy level is very high, but if you go 50 years ago and use remote sensing and GIS imagery to understand the water resources for rural development or soil and food security, the accuracy level is low, but still you get the best vision, you get the best data available because observation data was not there. So let's look at green revolution and production. This is for food security. Please put on a lens that looks at this data as what has it done for food security, but also growing these require a lot of natural resources, which may impact rural development. For example, if you take so much water out, then you don't have that good quality water for drinking or for industrial purposes, as for sanitation. This has been a cause in big urban cities also you would have noticed. So let's look at the green revolution production, what has happened, and also you will understand the nutritious value right now. So you could see rice in 1950s, we were 26.29, but then it increased almost four times by your 2010, 2017. This is highly unsustainable. If you can go four times in as much as 60 years, because rice and wheat requires a lot of water and soil fertility. It doesn't put back soil fertility like other crops I'm going to talk about. So the traditional crops as sorghum, perl millet, finger millet, small millets, maize, corn, barley, all these traditional food. It consumed less water, less fertility, and more important, it also helped in fixing certain kind of nutrients in the soil. So when I said go back 50 years, so you would have seen that 50 years ago, using satellites and GIS data, the rice cultivation was less, wheat was less compared to now, and the millets was really, really low compared to the other crops. And those regions would have had good amount of water and good amount of soil fertility, which contributed to rural development, but that is not the case now. So let's look at it. Rice has increased four times, whereas your wheat has increased 10 times in 15 years. Just look at how it's just rising. And if you take the data now, which is four, five years from the last data in this study, you will see that still more crops in these two categories have been growing. On the other hand, you will see sorghum coming down, jawar, and your bajra and ragi, your perl millet and finger millet, which are claimed to be very, very good for diabetics, cholesterol, hypertension, et cetera. So those have actually come down, except bajra. The others have come down. The point is, even though green revolution helped, what did green revolution bring? A lot of technology, fertilizers, pesticides, groundwater pumps to enhance the yield and to increase the intensity of crops. Okay, I'll have another slide to talk about that. But here what you could see that, even though the technology came in in the green revolution time, 1960s, I would say, it did not impact more on the low cost or low impacted crops, which is the millet, sorghum, maize, barley. It was mostly used for the high cash crops, rice, wheat, sugarcane, even cereals and pulses. But mostly rice and wheat. We used to import a lot. Now we are exporting. You would have looked at the news and all. So the point here is, yes, green revolution helped to increase your crops yield production, et cetera. But also it has damaged a lot on the fertility and the water availability. And these may not be the most nutritious food, because nutritious food did not grow as much. We compare the nutrition values. You can see that how it has compared. And when something grows very fast and using different techniques, the nutrition value is different compared to traditional. Always the traditional, that's why organic is that, right? Organic food has a lot of nutrition value. And traditionally it has been grown, as in with vermicompost, opposed to fertilizers, chemical fertilizers, and with good water resources. So these crops have been tremendously increased due to green revolution. And now people consume a lot and have access to food, but are they eating the correct food? So that is a different angle. But since this course is going to be looking at food security, which is, are you getting food the first step, then the nutritious value. So first you need food to live, and then healthy food to be nutritious. And then you go down high level. So let's look at the food security. Yes, we have better food security compared to the last 40, 50 years. Thanks to green revolution. And to keep this sustainable and to develop rural economies, we need to be cautious about it. We should also look at, has these actually impacted rural development? As in has the farmers become more economically stable or socially stable, is a question. So there are a lot of aspects, and that is where I'm going to come into the picture of wastage and how much wastage we do. Then we will look at the issues and concerns. So has this addressed food security in rural areas? Almost yes, because the access to food has increased. Rice and wheat has been the most stable food, and now people have access to it. However, there are some issues and concerns. Let's look at them. The first major issue is crop loss. The crop loss is defined as you are growing a crop. And before it is time to harvest, you lose the crop. And the most important factor that drives these is climate extremes. What do you mean by climate extremes? Is tremendous floods, too much water, because it just washes away the crops, or the flood water stagnates the on top of the soil and suffocates the roots, because the roots need also air to breathe. But then if water is totally inside the roots, it suffocates and dies. You should not let the roots rot in water, but that happens during floods. So that is one climate extreme. The other climate extreme is too less water, which is drought. So if you have a tremendous long spell of summertime, you will see nowadays, like a lot of heat records broken, the summer is expanding, and all the soil water is evaporated or lost in the groundwater pumping. In those times, what happens is the crop is lost. The crop starts to grow healthy, but because of the intense heat, too much temperature, air temperature, and less soil water availability, you lose the crop. So the crop loss is driven by climate extremes, one of which it's two floods and droughts. You could see here in this study that impact of climate change on muster, just one yield, one crop yield, and adaptation gains in 2020 scenario. So what they have done is you could see that if you look at the mustard yield, the standard deviation or the red yield deviation, you could see that most of India is in the red, which means because of climate, the yield is reducing, the crop is lost. Yield reduction can also happen by the crop can grow, but the produce doesn't come as big. So for example, if you are growing a coconut tree, or let's say mango tree, and each mango is 200 grams, a yield reduction can be the total tree is lost, or the mango is only 30 grams, smaller in size, no flesh, and that is a loss in the yield. So coming back, the first image which is A shows that there is tremendous loss in crop yield across the mustard species, only some areas they have positive, because the climate change also has positive side effects in a very small, small places, but most of India is negative impact. You can see all of it is red or orange, which is below zero or zero, and only some parts are getting more water because of climate change impacts, and those are the green. So the net is we are losing mustard due to climate impact. But if you do adaptation, so climate adaptation plans, which is part of the rural development program, which I explained about Mandrega, IWMP in the previous lectures, if you do that, then yes, there is potential to lessen the climate impact due to adaptation, and then the produce is increased, or yield is increased, and you can see in the positives, most of India is in the positive. This will be water sharing, soil and water conservation activities, better crop varieties, hybrids, et cetera, or pest management. Okay, so that is one. The next is pest. Pest is a tremendous reducer of crop yield and crop productivity. If you look at this study, you could see that mostly the potential yield is approximately 60% for wheat, maize, soybean, groundnut, and sorghum. So 60% of the yield, so let's say one acre you are planting, more than 50% of the area is lost. So think about the water conception for the loss, the time labor conception loss, and all the natural resources, the soil fertility, et cetera, et cetera, because the crop will grow, but the produce won't come. So the crop is still taking the soil fertility, not only the seeds, which is the wheat and rice that we consume, but also the shoots, the plant itself consumes a lot of energy and nutrition from the soil. So that is the potential loss due to weeds and pest. Weeds are different crops that grow in between, and most of them may also be less impacted by pest. Pest are the insects that eat on the crops and stuff, so that reduction in yield is there. And then the actual yield is, so the potential is the maximum, the actual yield is around 30 to 40% from this data. The other biggest concern that we have is food waste or crop waste. So now the farmer has sweated 50% loss in the land, but still they grow something, put it in your transportation containers or sack bags, and then they send it to the city or the industries to process. But on the way, transportation is bad, the roads are not well connected, so some crops yield is lost. Let's say you're having eggs and the transportation is very bad, so you do some eggs. Let's think about fruits. On the way, because the roads are not good, the connectivity is not good, you have loss in the fruit produce. So you could see that the fruit that is really squished and not as good as when you harvested, those sell for very less price on the street markets, whereas the fruit which is covered in paper and plastic bags and thermocol, those are in a high level shops. So those do not cater back to the farmer because in between there is a medium, there is a broker that works for this transportation and they take all the benefits. So for the farmer, it's still a loss because all the food is not consumed at the market. There's a big loss. Then there is quality during harvest, food wasted during harvest. So the quality of the food that you take, produce rice and wheat that you harvest is not the same as it ends up because there is a lot of loss in the transportation, the processing of the food. Limited storage, this is a big, big concern. You put all this grid revolution, the techniques and models and you have made this harvest. However, if you do not have a storage, then you lose the harvest. Think about this, you're going to a market.  However, in your fridge, you only have 10 kilograms space. So you stuff everything, but you leave the rest 10 kilograms outside. If you don't consume it fast, the 10 kilograms is lost. So that is what actually happens in a big scale. When you talk about storage, so there are new rules and laws being drafted by the government to help build more storage structures and better energy consuming storage structures to help farmers. Where do you place them? Where is the need? Comes from remote sensing and GIS. We'll look into that in the specific food sector. And we also are losing a lot of food at the household level. So you bring all the food and then you save it. The farmer has sweated, the rural economy has worked very hard to bring the food to your table, but then still there's a lot of food. So through this lecture, I also wanted to put a fact that India is the second highest food-wasted in the world when people are starving for food, when people do not have access to food, it is very, very important to save food. So please do not waste food and think about rural development, how people have worked very hard to bring this food to your table. So let's try to bring this chart down. Okay, so while we discuss this, the point, the driving point is crop yield growth is needed to supply food security for the growing population, right? And for that, we need to have raise the food production by 35%. So global food demand is expected to go 35% up. So you have to match that 35% by increasing the crop production. So what has happened from 2005 from the study till date is that the land is limited. You're not going to build new land and then put crops on it and grow. The land is still the same, right? So you can convert like some barren land, some science and technology interventions, water that comes or aquaponics or use coconut fibers or wastage from fish farms to grow some crops. So that is very limited, only 9% you can increase, okay? So let's say 35%, you have to increase. So let's break it into 100 and then 9% of that can come from increased land cultivation. There are some people who would, some countries if you see have cleared for us and then put crops in it, especially South America. But that we're not talking about. We're talking about using science and technology and increased land cultivation, very minimal. But the majority will still come from yield increase, which means you are having the land, the land was giving 100 kilos. Now you're going to convert it to give 200, 300 kilos of crop. That is called yield increase. What is cropping intensity increase? So for example, there is a land, it was growing only one crop per year. When you say cropping intensity increase, you're going to make it grow two crops, which means use two different types of crops that grow in six, six months in this land. So that is increasing the crop intensity, but still that is only 14%. So if you combine 14% and 9%, almost you have 25%, which is not good for your entire 35% increase. So the rest, 77% has to come from yield increase, which is already the land is there and that land has been producing crops 100 kilograms, but you're going to increase it to 200 or 300 kilograms. And that comes by creating more rural development structures, let's say water, energy and food processing so that they develop and then contribute to more yield increase. If this is not sustainable, if the yield increase is not sustainable, suddenly it will fall. So it's better to do it through the rural development. For example, how do you capture water, excess water running off and capture it, apply it to the field, increase the yield? How do you capture your nutrients that are in the air like nitrogen and then fix it in the soil and create yield increase? Or how do you manage the land well using vermicompose and other things to do yield increase? So this is rural development and all these yield increase can be studied very well using remote sensing and GIS, but more importantly, the negative impacts can also be studied using remote sensing and GIS, which we'll be looking at. So to conclude today's lecture, food security is very important. There are food banks you have seen across the world and in urban cities we have some fridges on the roads like Chennai has in some places where people have food. There are small canteens by the government which sell idlies and dosa for two rupees, five rupees, very, very minimum so that the food security is reached. People should not go hungry. You should bring down the death rate due to hunger. So hunger deaths is not good for a country. It doesn't reflect well as a developing nation, right? So you try to bring it down. Soup kitchens are set up something like canteens that give free food, a lot of NGOs are doing it and then child nutrition is being taken up. If you look at the government's program, there's tremendous increase in the child nutrition by supplying nutritious food like millets in their midday scheme. For example, Odisha and Karnataka are giving millets through energy called Vasan. I work very closely with them but mostly you can also see other states picking up ideas from like, for example, Tamil Nadu on introducing eggs in the midday scheme. So eggs have a lot of protein and nutrients, right? So this is about food security is feeding them but also bringing up the nutrition value. So to cater to that, the rural development should occur and then the food is produced and given to these sectors and then the food security is reached. Let's see how it is. So you have better gardening and farmers markets to connect farmers to the local demand. We know where the farmers are but we don't know where the demand is. So if you have two maps about a map with farmers and a map with demand, then you can match them and see how long it takes and distance, et cetera. Food production farmers, the farmers need high technology and to understand what is happening in their soil and water resources, the most sensing in GIS we'll be using for that. Education, we need to know where to put the schools and then the mentor program where like, for example, IIT, VEGO and teach some schools in the villages. So how do you connect with that? How far it is? All these can be mapped. First you map it and then you put infrastructures to create this education system. So food security combines all this structures for food distribution and nutrition checking, food production on the ground through rural enterprises and most importantly, educating the people. So key to understand food security for rural development is to understand the impact of farmers socially, economically and their health because they have to eat healthy nutritious food to produce nutritious food. So if farmers are weak, they cannot put so much time and energy in the field to bring food to you. So it is also as important to check on their health and nutritious level. Rural regions work harder to protect food security actually the hardest I would say because no one else works that hard to attain the food security. If the farmers stop, if the farmers do not produce food for our food security, then we will have to go to other countries and ask for food. That doesn't look good on the international scale, right? So we are giving a lot of food as ration, as an aid. So the government helps and supports a lot of countries during tremendous times. Like for example, you would have seen in news that Sri Lanka was given a lot of food because there was a food security issue. And that time, India was giving milk powders, food, rice, meat, sugar, all these things to keep the people under food security and then develop. So need to provide data to increase food security. Actual data is needed. Otherwise, if farmers overproduce, there is no storage to keep the food. So that is the other issue. So we do need a lot of food-related data and map it on remote sensing and GIS platforms to understand food security. So data collection may be expensive and time-consuming. Thankfully, we have this course where we will look at remote sensing and GIS tools which are free and open source and it is less time-consuming to understand this data for food security. All this is tied back to rural development. I hope this lecture gave ideas about food security and rural development. I will see you in the next class. Thank you.


Hello, my name is Ariel Mikuras, I am from Klužnapoka. I have a master's degree in cybersecurity and I also am a teaching assistant for X86 assembly language programming. I'm a software engineer at Cisco and today I will talk about PuzzleF as the next generation container file system. So let's get started. First I will present a short introduction, then I will talk about the OCI drawbacks. So this will be the problem statement, moving on to design goals, status and a quick demo. Then the PuzzleFS data format, the results, the Linux kernel file system that I've implemented and at the end I will be happy to answer your questions. So for some context, PuzzleFS was started by Tyco Anderson in 2021. He had a first-time presentation in 2019 about AtomFS. My colleague Scott Moser also presented AtomFS earlier this year. So the idea with AtomFS was to replace the tar layers of the OCI image with SquashFS layers and also add the unverity data for integrity. This has the advantage that you can directly mount the SquashFS layers and also verify the integrity. What is DM Verity? In short, it is an integrity protection at the block device layer. And if someone wants to tamper with any of the blocks, instead of returning the tamper data, the Linux kernel will just return an error. So you also cannot write to those blocks. Why I'm mentioning AtomFS? This is because PuzzleFS aims to be its successor and it is part of the larger project machine that we are working on at Cisco. This is, of course, also open source and this is an OCI-based secure container Linux. For some OCI format basics, OCI comes from open container initiative. It is a standardization of the original Docker format. And this is the directory layout. It all starts from index.json, where you will usually find a tag. For example, if you want the latest Ubuntu image, then the latest is actually the name of the tag. So using this tag, you will get a pointer to a manifest and from the manifest you will have pointers to the image configuration, which stores stuff like environment variables. And also you will have multiple pointers to the layers. These layers actually contain the bits of the image, so the actual data on disk. These layers are usually tar layers, but they are also compressed, so it will be a tar.gz file. And if you look at the file names, these are hashes. This is because they are stored content addressed. So their SHA-256 checksum is also their name on the file system. And when you hear me talk about the data store, what I'm actually referring to is this Blobs SHA-256 directory, where all the data blobs reside. So all the problems stem from this tar file representation in the first version of OCI. There's a really detailed blog post written by Alex Azaray in 2019, where he describes all the issues with this format. And I will try to present a few of them. As I've mentioned, the layers are usually in the tar.gz format. So first of all, tar is not a well-defined format. It's not a standard. It is rather a collection of different formats, and each have their own extensions. It has no index. Archive entries simply consist of header plus content. So files are actually concatenated together, and this is basically the tar archive format. And you can imagine if you have a very large archive, it will take some time to seek and find the file that you are looking for. Another problem is that compression and encryption cannot happen underneath the tar archive. So you cannot store compressed files inside the archive. You need to apply compression on top of the archive. And if you imagine the same large file, you have to decompress everything before you can then search again for the files that you are looking for. So another disadvantage is that there is no application. Any change you make in the tar file will lead to an entirely different checksum, shot to 56 hash. So this means that if you want to download another version of the same image, you have to download the entire tar archive. You cannot download just the parts that have changed between the images. But there is one small mention here. If you have multiple images which share the same layer, then you will get some interlayer sharing. This is fragile because as I mentioned, any small change will lead to an entirely different layer. There is no true machine independent representation. This is because the directory entries and the extended attributes can be presented by the file system in different orders. And you can have the files in the tar archive in any order that you like. It's the same file system that is represented, but there is no canonical representation. You cannot say, this is from this file system, this is how I want my tar archive to look like. And this leads to this lack of reproducibility problem. You cannot reproduce the tar archives. And on top of this, the multiple extensions add to this problem. For example, there are five different extensions for specifying extended attributes. Our design goals for PuzzleFS are basically to solve the most pertinent OCI view on problems. We would like to have reduced application, reproducible image build, direct mounting support, data integrity, and also memory safety guarantees, and ideally the same implementation in user space and kernel. So how do we achieve reduced application? Well, we can use content defined chunking. This solves the boundary shift problem that I will talk about in the next slide. We use the FastCDC algorithm to chunk an entire file system into variable sized chunks. With FastCDC, you can specify a minimum, average, and maximum chunk size that you... So basically a range for the sizes of your chunks. So this is the boundary shift problem. On the top, you can see we have file A, and the file A prime is almost the same as file A, but we have inserted one byte at the beginning, this FF byte. So with the traditional FSC approach, this stands for fixed size chunking, where you just split the file into equally sized blocks. In this case, inserting a byte at the beginning of the file has an unwanted avalanche effect. So every byte shifts to the right, and then you will have no duplicates detected. On the contrary, if we look at the content defined chunking approach, doing the same operation, we will get most of the duplicates detected. The first chunk will be changed because we have inserted an FF byte, and all the other chunks will stay the same. You can see here that every time we find a nine in this file, we declare a cut point. Of course, the algorithm is more complicated than this, but I will show it later. So for real-world application, imagine we have Ubuntu image at version N, which has 80 megabytes in size, and we want to apply a small patch to libssl.so. Maybe this is a security patch. And now we get another version of Ubuntu, which is at version N plus one. It's still around 80 megabytes in size, but the delta size, the size we have to ship to the end user, it will also be 80 megabytes, because we need an entirely new Tor layer. So the solution here is to use CDC, content-defined chunking. If we chunk the entire file system into chunks and then we only do a small patch to libssl.so, only one chunk will be changed in this entire list of chunks. And then the Ubuntu image at end plus one will still be around 80 megabytes in size, but the size that we need to ship this small patch is around 80 kilobytes. This is based on the average chunk size that PuzzleFS uses. So for getting into the details of content-ified chunking, this uses the sliding window technique, and it computes the hash of the window. This is called rolling hash. And then we take the last n bits of the hash, and if these are zero, we generate a cut point. And the interesting thing about this is that the cut points only depend on the last window size bytes, which is usually 48 bytes. So we start from the window, we start with this window, we compute the hash. If the hash has the last n bits zero, then we say we found a cut point. If not, then we just move the window one position to the right and repeat the same process. So you can imagine if you insert multiple bytes in the beginning or in the middle, then you still have a pretty good chance of identifying the same cut points. Our next goal is to have reproducible image builds. We want to have a canonical representation of the file system. So for this, we define the same traversal order of the file system. So for example, you can traverse the file system in breadth-first or depth-first. We want to make sure it's the same order when we are building a puzzle-FS image. Then we make sure the directory entries and the extended attributes are sorted lexicographically. And finally, as an implementation detail, we use B3 maps because they have a defined order, unlike regular hash maps. Another goal we have is to prevent tampering. So for this, we would like direct mounting support. So one other issue with the TAR archive is that you need to extract it first before making use of it. So this leads to a problem because then you cannot make sure that the data was not changed in the meantime. You cannot put your extracted data back into a TAR archive and then compare the two and say, okay, nobody tampered with the data. So what we want is to remove this extraction step and mount the file system directly. So then we can make sure we are using the same thing that we've originally built. And also, we want the format to be simple enough so we can decode it in the kernel. Speaking of data integrity, if you remember I mentioned the FMFS at the beginning of the talk and FMFS was using the n-varity but this doesn't fit our use case. This is because even if PuzzleFS is a read-only file system, we still want to write to the data store. If we want to download a new image, a new PuzzleFS image, then we need to write the new blobs to the data store. So it doesn't fit our use case. But we have optional support for FS-Varity. FS-Varity is very similar to DM-Varity but instead of working on block devices, it works on individual files. And it must be supported by the underlying file system on which PuzzleFS resides. What we do is we compute FS-Varity digest for each file and we store it in the PuzzleFS image manifest. And then the PuzzleFS image manifest itself, we also need to compute the FS-Varity hash and pass that information in an out-of-bound way. So we do this by passing it on the command line of PuzzleFS mount. For PuzzleFS, we want memory safety guarantees and so this led to the decision of implementing it in Rust. Both the Fuse version and the Incarnal file system. Rust has many benefits. It entirely eliminates undefined behavior and entire classes of bugs. For example, dangling pointers use after free buffer overflows. It has a very strong type system and a first class support for writing unit and integration tests. So it makes it very easy for you to actually write the tests. And for all of this reason, in my personal experience, it leads to a very painless iterative development. And finally, we want to share the code both in user and kernel space. Luckily, Rust support for the kernel was merged in Linux 6.1. We don't want to write the same code twice. But of course there are some differences. First of all, the kernel only allows valuable allocations. So everything that you allocate, you must be able to deal with the fact that this allocation might fail. It's not like in user space where you don't care and then the out-of-memory killer jumps in and kills your process. And also we cannot handle the file operations in the same way. You can imagine the user space API is quite different from the kernel abstraction of files. And we also must duplicate the code for practical purposes because the kernel has its own build system. You cannot just download the crate from crates.io. Or for that matter, you cannot even use the cargo build system. You need to use a K build. But apart from these small differences, a large amount of code I was able to share between the two implementations. Moving on to the status of the project, we can build, extract and fuse mount PuzzleFS file systems. We have support for FS variety, but as I mentioned, it requires file system support from the underlying data store. And we also have an optional ZSTD compression for the data blobs. What I've done in the past months, I've also implemented a proof of concept file system driver written in Rust. So for a quick demo, you can see here at the top I have a root file system which contains two directories and two files. If we want to build a PuzzleFS image, we specify the build command and we pass the source of our root file system. Then the destination where the PuzzleFS image will be stored. And then we also need to specify a tag for this image. What we get back from this build command is the FS variety digest of the PuzzleFS image. And we will use this in our subsequent commands. The next step is optional. If we want, we can enable FS variety. We need to specify the path to the PuzzleFS image, the tag and also this hash that I was talking about. If we enable FS variety, it actually sends some eye octals to the kernel and then the files in the data store will be marked read only. So you cannot write to them. And also if tampering was detected, the kernel would just return an error code instead of giving you the tampered data. And lastly, we use the mount command where here we can also optionally specify this FS variety digest. And then we use the path to the PuzzleFS image, the tag and also a mount point. And if all goes well, you can see the output in the journal where it says mounting slash temp slash puzzle. And also the output of mount, in the output of mount you will see we have slash temp slash puzzle, which is of type fuels. If we do specify this digest on the command line of mount, then PuzzleFS will make sure before opening any of the files in the data store, it will make sure first that the FS variety is enabled for those files. And secondly, that the FS variety measurement matches what it has in its database. Now let's move on to the PuzzleFS data format. We have an index.json file and with the help of a tag, we can find the manifest for this tag. The manifest contains a list of metadata layers and these layers contain a list of inodes. And here I wanted to show the different ways in which the data blobs could be produced by the chunking algorithm. On the left, you see that inode1 and inode2, there are two files, but their contents is stored in a single large data blob. On the right hand side, for inode2, you can see that it has a list of chunks that it points to. So probably it's a larger file and it needs more chunks to represent the entire data for this file. And here you can see that these are hashes because these data blobs are content-addressed. So what we do for metadata serialization is we use CaptainProto. This is the serialization protocol. It has many advantages. Basically, the on-disk format is the same as the in-memory format, so you can just memory map the entire file and then you will have to use some accessor methods to get to the fields because it has a custom representation in memory. And as you've seen in the previous slide, we have two levels of indirection. The image manifest, this contains a list of metadata layers and also the FS-Varity data for the blobs. And each metadata layer contains, first of all, the metadata for the files and directories, but also those links that you've just seen to the data blobs. And as I have already mentioned, we store them content-addressed. So here is the CaptainProto schema for the metadata, for the manifest on the left and the metadata on the right. With bold, I have highlighted that this is the root structure for this file. So this is a structure that you will use when you want to decode the CaptainProto file. And as you see, the root file system contains a list of metadata, the FS-Varity data, and also the manifest version. And on the right we have the inode vector, which is just a list of inodes. So one thing that the CaptainProto gives us is a compact inode representation. So this entire file actually represents only two inodes. But CaptainProto splits a structure into the data part and a pointer part. So if you have nested structures, then the outer structure will be completely self-contained. And the nested structures will be stored elsewhere, and you will have a pointer from the outer structure to the nested structure. Here with red, you can see the inode number of the inode. And with white, I have shown the pointer part for both these inodes. So why is this important? Because we want to fit as many inodes in the cache. And this is because we want to have a very fast binary search when you are searching for an inode in the file system. Now let's look at the results. So what I did was I downloaded 10 versions of Jemi from Docker Hub. And I've put these versions into separate directories. So separate OCI repositories for each of these versions. Then what is also important is that all these images only have one layer in tar.gz format. So you will not get any interlayer sharing in this case. And then what I've did, I've converted these images to PuzzleFS images. And I've made some calculations, so I wanted to know how much all these versions take up if they would be stored as simply as tar files without any compression. And this is the baseline that I will use for comparing the space savings. Then what I did is I've taken all these 10 OCI repositories and I have just summed their sizes. This will give us the total size. And I did this for both the OCI and the PuzzleFS images. And then what is really important is what is the space savings if all these 10 versions would be in a single OCI repository or a single PuzzleFS repository. I call this the unified size because here is where we will get all the data sharing with PuzzleFS. And then for computing the saved space, I just took the tar ball total size and I have then subtracted total unified size from it. And in this table, we can see at the top we have the OCI format, but this is the uncompressed case which you will not see very often. Usually compression is also applied. So we have a total size of 766 megabytes. So this is if you just add the size of the tar balls for each version. Then we have an average layer size of 77 megabytes and the unified size. This will also be 766 megabytes. This is because with tar you don't get any sharing. So even if you put them in a single OCI repository, it will still be 10 different tar balls, each with their own unique hash. Then I did the same thing for PuzzleFS in the uncompressed case. We have a similar total size and average layer size, but you can see here that the unified size it is much smaller. It's only 130 megabytes and the saved space is 83%. Then for the compressed case of OCI, we have a much smaller size. It's only 282 megabytes with an average layer size of 28 megabytes. Here you see that the unified size is still the same as the total size because we just applied compression. We didn't do anything special. Here the saved space is 63%. You can see that even in the uncompressed case PuzzleFS still beats the compressed OCI with around 20% in space savings. If we apply compression with PuzzleFS, then we will get an even smaller size. The unified size is only 53 megabytes and we have 93% space savings. This is 30% more than in the OCI case. Now let's move on to the kernel file system driver. We at Cisco want to have PuzzleFS in the upstream kernel. We posted an RFC to the Linux mailing list. This driver is written in Rust. There are actually two versions of this driver. They are both based on Almeida's work. He had originally a set of file system abstractions, but recently he made a set of abstractions only for read-only file systems. Neither of these are yet upstream. I've implemented these versions on top of his abstractions. He's also working on TorFS, so this is why he needs these read-only file system abstractions. What we need to do is add third-party crates to the Linux kernel. This is for the metadata serialization and hacks to deal with the hacks strings that appear in the digests. There are some challenges writing file system drivers in Rust. First of all, we have many missing Rust abstractions because the infrastructure is immature and it is still under development. If you want to integrate third-party crates, then we require no STD support first of all. We also can only use fallible allocation APIs. We have to try new and deal with the potential error instead of new, try push instead of push, and so on. I've actually been to the Rust for Linux workshop this weekend. It happened in Astoria and we had interesting discussions there about file systems. Now for a quick demo of the file system driver. We can see here in slash proc slash file systems that we have a puzzle FS entry. If we want to mount it, we have to specify two mount options. First of all is the OCI root directory. This is the path to the puzzle FS image. Then we also have to specify the image manifest, which is this long digest. We have to do this because we don't want to read JSON files from inside the kernel, so we cannot just simply use a tag and then get from the tag to the image manifest. You can see here that I have shown where I got this digest from, so it's in this index.json file. Now once it is mounted, we can list the directory entries. You can see I have four directories and two files, so we can read the files, we can count the number of words in the files, and so on. Now finally for Captain Proto Rust kernel integration. Captain Proto is an external crate. I wanted to integrate it in the kernel, so I had to do some work to implement full NOALOX support. There was some work, but it was not finalized. What they did, they used strings in error codes, and we don't have string support in the kernel, so I had to replace this with enums. And I also had to implement some other structures, because basically it was easier to implement full NOALOX support than to try to replace the infallible allocation API with the fallible allocation API. So I introduced the NOALOX buffer segments, which is a version of buffer segments suitable for NOALOX environments. Basically the reason for this is that I wanted to avoid parsing the Captain Proto message every time a field is accessed, so I needed a way to store the reader somewhere. And finally, if you have any questions, I'm happy to answer them. You can find the project on GitHub under Project Machine PuzzleFS, and also you can contact me at amikulashet.com. Thank you.