The framework for both fluid and smoke simulations. Which one of you has used the fluid simulation in Blender at least once? Okay, and who did use the smoke simulation at least once? Who thinks it would be cool if fluid and smoke simulation were one and the same in Blender Unified? Okay, I see. Who did use Maya at least once? The simulation tools in Maya? Okay, because if I recall correctly in Maya, fluid and smoke simulation is integrated into one tool, right? Okay, so, good. Let me first tell you a little bit about the history of this project. This mantra flow was developed by a person named Niels, and he's the developer who developed both the fluid simulation in Blender and the smoke simulation. And now he created a framework that can be used standalone and it's mainly for research purposes, but he would really like to see it integrated into Blender. Unfortunately, Niels cannot be here today, and that's why I'm here to fill his position. And me myself, I also want this to be integrated into Blender, and there are a few reasons for this, and I will show you the first reason now. And that is, take a look at the current smoke simulation in Blender, and we might encounter some issues with it. So these are demos I created of the current smoke simulation and stuff. Okay, let me see whether you can see it or you can hardly see anything. Can somebody turn down the light, please? Turn down the light, because I actually wanted to show a little bit about small details in here. It seems like, can somebody please turn down the light, or maybe let me just tell you what you should see here, and that is, especially on the part on the top left corner, there are hardly any details in the smoke. And so there was a thread on Blender artists about this, and it was by somebody who wanted people to rate the simulation on a scale from 1 to 10. And if you look at this, how much points would you give it? Just show me with your hands, like 5 or 7. So unfortunately, I know you cannot see a lot, but I see most people give it at least 5, between 5 and 8. Some people even 9. That's nice, because on Blender artists, the person gave it, great. He gave it 5 points because of the lack of details in the simulation, and I see you cannot see anything. Okay, let me just switch over to the next slide, because here, this was also in the demo, here we get a lot of details on the surface. The question is, why do we have details here, but in this shot? And let me tell you what the reason is. The reason for this was, in this shot, I had to use a very high resolution for the base simulation, because down here, where the glass is, the interaction only with obstacles only works right when you have a very high base resolution. And the Blender smoke simulation also has a tool to crank up the resolution by adding artificial detail. That is what I used here, but it comes with a penalty on the usage of RAM. So for both simulations, I maxed out the RAM on my workstation, but because I couldn't use as much high resolution here, the smoke looks like kind of dull, but I get the interaction with the glass. So this is actually a very common problem in smoke simulation, and very common problem if you use it, because it only can have a low base resolution and you have to add details somehow. This is for example how other companies are adding detail to their simulations. They are basically mapping textures on top of the simulation and the texture is mapped to the flow. Like you can see it here in the middle, all the small details in here, they are not in the actual simulation, they are added as a texture that is mapped to the flow coordinates of their smoke. This is pretty cool, you get it in Fumifix for example, but this is actually not what I'm talking about here, it's just to tell you that this problem is persistent and many people have tried to find different solutions. And the solution by Nils is actually the following and his idea that is in Mantaflow is to add detail only to the surface of the smoke and thus save computation time and save on RAM. And if you don't really get any idea about this by just looking at these four images, don't worry, I also have the entire video demonstration he used for SIGGRAPH. So this is a demonstration about these techniques about the so-called vortex feed. What you see here is basically it's like the smoke simulation, but it has a measure integrated Mantaflow. So this is a demo drive in Mantaflow and it has a measure, so you only see the surface. They did it because you can see the details they will add later on better. Personally, I already think that this would be pretty cool to have a possibility to mesh the surface of a smoke simulation like you do it with a fluid one. Just imagine the, like for example, in the tornado they want to create for Gooseberry. I guess this would look pretty cool to just use the surface of a simulation and not the volume rendering because they have this very water-like tornado they want to create. And the idea is also, hopefully you get the idea that they are adding details just on the surface. This is now a version where details are added everywhere and on the right where the details are only added at the surface where they are meaningful. So it gets a little better, the quality gets higher and also you use less RAM, higher quality and it just looks better and Mantaflow has a few other cool things. Okay, let's just watch the demo again and then let's move on. This is actually a standard scene that's used quite often for papers when papers want to present something like fluid or smoke simulations they usually use this scene. You might have noticed at the beginning where the smoke comes out of the ale that there is like a boundary and that was because the resolution was so low. It was a really low resolution simulation and all the fine details were added by the Water Street method. And yeah, I guess we've seen enough. Okay, so this is one thing and there are a few other things I'd really like to see in Blender for the smoke simulation which we have unfortunately not yet here because Mantaflow isn't integrated yet totally and that is so-called vortex particles. You see it on the left. Is anyone familiar with the concept of these vortex particles? Basically, it's the idea that you have a particle and it will induce this curled emotion into the smoke at the place where the particle is and this is actually a very nice feature because usually the smoke simulation is just not high resolution enough to capture things like flow over the surface and when the smoke reaches the edge for example there should be electricity but due to our low resolution it usually isn't and with these vortex particles you could just at the edge create an emitter for particles and it will create the vortex particles and you will have a curled emotion of the smoke after it has reached the edge, for example. That's something I really need because once I had to cancel a job and tell the blender can do this I would need vortex particles for it and I also told them that they should look for a Houdini guy because Houdini has this and then the Houdini guy got the job, did it well and yeah, that's one of the reasons I would really like to see this in Blender and these warthorns are actually pretty common and they are already in Mantaflow in the middle you see vortex filaments and they are used to create things like smoke rings they are not in there yet they are still in development but I talked with the developers and they are actually trying to add those so there's more in store I guess most of you won't laugh to see the last part the Balas bullet points because like doing things on a graphics card really fasten everything I know there is at least one person here who would love to do two dimensional simulations is it right? yeah, Mantaflow can do simulations in 2D and in 3D, pretty cool and yeah I cannot say much about the fluid simulation part because the integration work started with the smoke part the only thing I can say for sure for fluid simulations that we can have multiple domains if it's integrated like there for smoke now and now let me just do a small life demonstration because Mantaflow is actually a framework that is yeah, it's standalone usually let me try to actually do this on the big screen simple preview basically for Mantaflow you have Python files that define your scene and then you can see the simulation in a very special kind of view so if you take a look at this let me make it bigger and once again you hardly see anything because you have all those small it looks like dots here and those are actually small vectors so you see the vectors and you can skip through it because this is just a plane that's cutting through the simulation and this way you can actually take a look and analyze for example if there's something wrong with your simulation you can look at the vectors and see whether there are any problems in it and this is actually the very same way that Houdini is working in this case so if you're working with Houdini you can also have the option to take a look at your smoke simulation and we'll use like a plane cut through it take a look at all the motion vectors and all the vectors that are defining the curl and everything so this is actually pretty cool and the developer told me that Mantaflow is getting integrated into Blender you might also get this kind of analyzing tools directly in the viewport for our simulation switch I would really love to see and now let me demonstrate the integration of Mantaflow into Blender because this is something actually that it works since yesterday evening that's why I cannot, unfortunately I cannot show you much but it's really actually working without too many big problems on Linux since yesterday evening so the developer worked hard and he worked hard up until the last minute this was totally cool so basically I got a simulation saved here it supports at the moment the idea is to create an integration that is like working exactly like this smoke simulation in Blender is doing now so what has been integrated is Mantaflow and you see below after you select the domain just Mantaflow settings if you turn on this then you will be using Mantaflow and the idea is to get a normal integration first that is integrated like the normal smoke simulation and then add the additional features that means at the moment we don't really have any additional features except for the noise settings down here which will just induce noise into your simulation and I'm not sure whether solver resolution is working actually so I'm not going to demo this and in the end you have some smoke so this is the current state of the integration it supports collision objects and it supports force fields and I will start to do some test renderings and some benchmarks because this is what I wanted to present here and it wouldn't work out but yesterday evening I did a few benchmarks and it's still a little slower than the current implementation in Blender but the idea is to get it at least the same speed up to the same speed and then add additional features we get from Manta and then let's see where this is going because the developers of the original Manta want this as well and I want this and probably a few other users of the simulation tools in Blender who also want Manta and why would this be good both for the developers of Manta and for us well the developers they're from academia they're now working in academia he had been working for scan by visual effects but he's now in Switzerland ETH Susich as a professor and he's using the framework for research like a lot of other people are doing this for research and of course they would like to get a nice possibility to do visualizations with it to visualize the research with easy-editing objects and nice render in the back end so they would be very happy and we as users could be happy as well because yeah there is a lot of cool features in there and this is something that's really used in academia so if they're adding new features like it will always be on bleeding edge cutting edge research so this way Blender would be probably the first package to get all the new crazy new features so we would be at the forefront not lagging behind okay so this is all I have to show you and I wanted to show you are there any questions you say this is being developed in a university I work in a university academic software is notoriously unreliable and shoddy and is designed just to get the next academic paper out how confident are you in the quality of the code that's being produced basically the the fluid simulation we have in Blender right now by Niels is also an academic code it was developed in the university and it integrated into Blender using a Google server of codes the quality of the tool is at least the same quality as the current simulation we have also the smoke simulation the base was also an academic paper by Niels and so he's actually trying to get a nice software engineering so it's easily extendable so people can do actual research with the framework what you showed in Blender was a real-time calculation of the smoke actually it was pre-baked but it's actually pretty fast I can create a new scene if you want to and demo the speed it's right no no it was just a question maybe up there there's a chance to have it really real-time and also it's portable real-time so that it can be used in simulations actually not this kind of simulation also not for the big visual effects houses it's no way real-time for real-time things you would need a different kind of architecture like there was a nice paper by some guy from Intel who would totally fake the appearance of viscosity and vorticity by using just particles and this was really speedy but this quality and this kind of fluid and smoke simulation real-time no how physically correct is the simulation is it only pleasant or is it physically correct like in the FDS simulator for example like the original Nebius-Dokes equations the base simulation is physically correct all the other things you've seen are tricks to add detail because the problem with the physical correct part is that you would need a tremendous resolution that you couldn't ever compute and that's why all the stuff that you see all the eye candy is not physically correct it's meant to be like cheating to get results that are looking like they are physically correct and this is actually where most of the research in regards to fluid and smoke simulation is going on at the moment so this is less a question and more some kind to get some influence into this I guess there are lots of people doing these kinds of simulations like I don't know with different software like OpenFoam and I have no idea whatever exists and it might be nice if you could have some common interfaces to do this kind of stuff if we want to if you're already developing this with a new engine so that actually you can plug in your own pre-calculated mesh because there are standard formats for that in computational fluid dynamics to do this so that one can do results simulation with another software tool and can get it into blender yeah actually I couldn't really hear most of it but I guess so what I really wanted to ask this if we could have some common interfaces to do this kind of simulations because I mean I guess there are lots of people who do computational fluid dynamics and who would like to do their simulations or at least to visualize them in blender and right now well we have the blender internal smoke and no one actually knew how to plug his own smoke engine into it and right now you're porting it to a new engine so it would be nice that we would have some kind of common interfaces to do this kind of stuff yeah that's what I think as well questions so what's happening with OpenVDB and if it has anything to do if it's Mantaflow or the old simulation or it's just container format and also the second question is the mesh remesh is that something we will see integrated to or is it only the smoke algorithm actually the thing that it is porting the meshes is something that would be totally integrated into a blender because this is actually like the fluid simulation at the moment it's working it has a simulation under a hood and then it's doing a meshing and if it integrated Mantaflow we would also get the meshing of the simulation like in the video and in regards to OpenVDB OpenVDB is a general library for volumetric data that also has a few nice features like compositing with volumes and everything and that should also be integrated in blender in my opinion the guys from Dreamworks who developed OpenVDB they told me that they are eager to help if there was a blender developer who would integrate OpenVDB but OpenVDB is so big that we really need to give the good design on how to integrate it because I guess from the way they designed it it's actually meant for a software that is completely node based from the ground up like Koudini and to get it into blender we really need to find a good design for the integration and OpenVDB is actually just a storage format for volumetric data and I guess the simulation might if they incorporate OpenVDB they might get a quicker simulation probably because they get look up cells faster or so but I don't think there is any actual plans for them to directly support OpenVDB I actually know the people who developed at OpenVDB because we worked together at Digital Domain and then he went to Dreamworks but one thing I think with the question before one thing Koudini is doing very well is the interaction imagine I throw a ball against like a cloth and then it interacts with the cloth simulation then it's dropping down into a piece of water then it's interacting with the fluid I think what they did very well in Koudini is like thinking about different kinds of engines and how they can interact and I think that's if you are planning to do this with a blender you should do the same thing that you have like with bullet you have a collision detection system and stuff like that all this stuff should be in the framework which can interact with each other without knowing you know the fluid guy might not know about the collision detection stuff and so on and it's a very tricky business to do that right that's actually one of the targets for Gooseberry to get a unified physics somewhat from what I've heard so I guess the talk is over


In this video I'm going to introduce blood, but I want to start with one of the most amazing facts in all of human anatomy and physiology. If I were to ask you what are you made of, what might you say? Water would be a good answer. It might think muscle, bone, these types of things. But from a cell standpoint, you are somewhere between 70% to 80% red blood cells. That shows you how important these cells are. Now how can you make sense of this? Well, there are trillions of red blood cells somewhere in the neighborhood of 23 to 25 trillion red blood cells in your body. There's only 30 to 35 trillion cells in your body. So from a cell count standpoint, you are almost all red blood cell. Now of course these are very tiny cells and water isn't cellular, so from a volume standpoint that isn't true. But from a cell standpoint that is actually true. So let's talk about blood. Why is it even important? Small, single cell organisms don't need blood. Small organisms don't need blood. Blood is the reason that we can be so large. Blood is also the reason that we can have organs that have a division of labor. Organs that can each do their own job because blood is what carries nutrients to all these different organs and structures and carries all the waste products back. So we would think about that as its key function, right? But there are lots of other functions too. It plays a role in defense and that's because of your white blood cells, right? They play a big role in your immune function. We'll cover that a little bit here this chapter and a lot more when we talk about the immune system in just a few chapters. It plays a huge role in the distribution of heat and that's because, like you'll see in a moment, blood is warmer than the rest of your body. So your body uses blood and your skin like a radiator system to carry excess heat away or to trap heat at your core. It plays countless roles in the maintenance of homeostasis, the things we've talked about throughout the semester, by carrying hormones and all sorts of chemical messengers that keep our body in balance. And then lastly, it plays a big role in the maintenance of our body's pH and we'll see why. There's lots of reasons but the glaring reason is that carbon dioxide is actually an acid. It becomes carbonic acid anyways, which is a weak acid. So how much carbon dioxide we're carrying plays a major role in what the pH of our body fluids are. All right, just some basic facts about blood and then we'll break it down into these three key components here in just a moment. Blood is thicker than water like you've heard. The blood is about five times more viscous than water, which means it's resistant to flow or more syrupy than water. That's what viscosity means. So blood is five times more viscous than water. Like I just mentioned, blood is warmer than the rest of your body. Your normal core temperature is 98.6 degrees Fahrenheit. Blood usually runs at 100.4 degrees. It has to do with the friction and the resistance in your blood vessels. Even when you're young and you have healthy, smooth blood vessels, there's some resistance and friction and that actually heats things up. Like if you were to rub your hands together, they're going to get warmer. So blood is slightly warmer than the rest of your body. pH of blood normally is 7.4, but it runs in a range of 7.35 to 7.45. In a later chapter, we'll talk about things that cause the pH to drop outside of those normal ranges and then how to hopefully deal with them. We talked about the cell number, standpoint 23 trillion red blood cells in your body, give or take. But the amount of blood is usually in the five to six liter range for a man and four to five liters for a woman. It's size dependent, but so five liters is a good safe number. So we see here, we see three different blood samples that have been spun in a centrifuge. The top is the plasma. So let's look at the normal blood. We won't talk about anemia or polysothemia here, but you see that obviously you can get some abnormal results with blood as well, like every other part of your body. But plasma is going to be the fluid portion of blood. It's primarily water and that's very important because water is the universal solvent. It carries things that can be dissolved in it. So about 45 to 65% give or take of your blood is plasma. Then this Buffy coat is going to be in the middle.  And you can see that there's not a lot there compared to red blood cells.  Excuse me, I apologize. I'm probably going to cough a lot today. I don't feel great, but let's start with looking at what plasma is. So we just said that plasma is somewhere in that 45 to 65% of your blood. As you can see from the top here, though, it is 92% water. So the huge majority of your plasma is water. And its job is to carry things that are water soluble. They'll dissolve right in the water. Things that can't be dissolved in your water are going to be carried by the plasma proteins. So as you can see here, about 7% of your blood plasma is plasma proteins. And there are three key types and you have to know them all. You have the albumin, the globulins and fibrinogen. So albumin, the first thing I think of is that it is a transporter. So it is a very important transporter. But also as you read there, it maintains osmotic concentrations. We'll cover this more in just a couple of chapters. But if you don't have enough protein in your blood, then you're going to be losing too much fluid into your body.  So someone has a swollen abdomen full of fluid. It's usually because in the United States it'd be liver failure, liver cancer, these types of things. So they're not producing enough albumin. But if you see it in the developing world, it could be that someone is starving and they're so protein malnourished that they can't make albumin. So yes, albumin is a transporter, but don't forget that piece. It maintains osmotic concentration. What's the most, I mean, there's all sorts of different binding proteins that are going to be albumins there. The globulins, they're a little unique because most of them are made by the liver just like your albumins are, but not all of them. So the alpha and beta globulins, they're primarily going to be transporters. Think about them as transporters. But the gamma globulins are your immunoglobulins, like IgG, IgA, IgM. We'll cover them in a few chapters. They're actually produced by plasma cells. So a plasma cell is one of your lymphocytes, a B cell, becomes a plasma cell to make those antibodies and they're going to be a huge part of your immune system. So that's albumin and the globulins. The third plasma protein is fibrinogen and this plays a huge role in blood clotting. The end result of our blood clotting cascade is fibrinogen becoming fibrin. So just remember, fibrinogen, critical role in blood clotting. The other less than 1% that's left is going to be hormones and other chemicals and gases and nutrients that are dissolved in your blood and being carried there. All right, so that is plasma. The rest of your blood is going to be called your formed elements. So here you see some pictures of them. Now why don't I call them the blood cells? So you may have learned them as your blood cells, but they're not actually cells. Red blood cells technically aren't cells anymore. They used to be, but they lost their organelles. They lost their nucleus, so they're no longer classified as a cell. And platelets aren't cells. They're cell fragments. They come from a huge cell called a mega karyocyte, but that one cell makes 2, 3,000 of these little cell fragments. So red blood cells used to be cells, platelets are cell fragments.  So that's why we call them formed elements. Okay, so that is a quick introduction to blood, why it's significant, what makes it amazing, and what it's made up of. We'll go into detail on all the other structures moving forward. Okay, I hope this helps. Have a wonderful day. Be blessed.


Today's tale is titled, All Is Calm, Yes, All Is Well. Sparks and metal scattering, the battlefield turned to ash. Bolts of arcane flying past, men vanished within a flash. These men they cry out to you, the supplies are falling low. Your enemies approaching, surely with more to follow. Emotions blurred with panic yet a plan begins to form. Your fists sweep in a pattern, energy begins to storm.  Hold your weapon at your side and surge on as magic flies. You holler as you charge in, in tandem machine and spell. The enemy still roaring, all is calm, yes, all is well. Your laboratory sterile, the room shaking from the blast.  Those awful beasts towering, shadows cast upon your sight.  Swiftly carving the symbols as they're banging on your door. Hit the ground, do not cower, show them what you have in store. You pull the trigger justly, bullets turn their bones to ash. In the ammo, magic flows, causing their bodies to mash. Your last attacker fallen, long gone far as you can tell. Rise to your feet with a sigh, all is calm, yes, all is well. Our numbers they keep dwindling, back to them we do the same. Both our men now so fatigued yet our anger still aflame. As tired as we may be, strategy will win this bout. For we refuse to meet an end, I swear we shall remain stout. Sigils carved into the dirt, brave men dare not be replaced. As unholy beasts shall rise, charge straight into danger's face. Terrible forces raging with magic's frightening blast. The battlefield prays for death as your spell begins to cast.  As it is released you grin, all is calm, yes, all is well. Dust awaiting to settle, the sun's light rises once more. Such death, bodies uncounted, I blink my eyes, my bones soar. The chemist groans to his feet, healing magic on his mind. The material he gathers, giving help to all he finds. In such men the foremen, ash covering his steel hands, raising men back to their feet, repairing all where he can. So the general lights the sig, her breath quick and unsteady, rallying us into our place, a speech they all made ready. Their men looked to them for aid, brushing my arm, ashes fell.  Thank you for listening.  Please become a patron or visit our merch store at the link in our bio to support our work. Secure. Contain. Protect.


Hey everybody, this is Marley with Columbia Animal Services. We are running another adoption special because this time of year, September, we have plenty of cats and kittens and a lot of this is due to a lot of our foster babies that have been in homes being raised up to get them ready for adoption or now coming back into the shelter so it's creating a bit of a space issue for us but we really wanna get these little guys into new homes. So for September 24th and 25th and next Thursday and Friday, we are going to offer all cat and kitten adoptions for just $25 and of course that includes the already spayed neutered microchip, vaccinated, dewormed, they're just ready to go. You know, the hardest part is just coming in and trying to pick one out. So we have plenty to choose from. A lot of great cats and kittens and they are waiting for their forever homes. We hope you will come see us on September 24th and 25th. Thanks so much.


Hope everybody had a good break everybody have a good break. How many people had a good break? How many people worked on? Assignments for this class during break, huh, and it was still good. Yeah all right, so today we are gonna keep going with memory management and I hope so how about this you'd be hackathon They have a 421 521 team wins the hackathon Then I will release one of the exam questions early Fair What? There aren't that many One of the multiple choice questions, right exactly one about the M&M. That's what I'm gonna. No No, no, we did this way We had this last year we use some carrots at the end of the year last year was for something different But but yeah, so if a team wins I'll release one of the long answer or medium answer questions early And everybody can can know what the question is and have an answer ready, etc. So how about that for a For incentives you guys should sign up. It's fun. It's a it's a fun chance to you know Get together with a bunch of people and you know program something up and you know If you have a project that's been kind of kicking around the back of your head It's a chance to sit down and just you know devote some good time to it So I'll probably be there and I may have something to work on All right, so on Wednesday We will hand back the midterms midterms are graded at some point figure out how to release their grades maybe on the website before Wednesday, but if not then Wednesday in the class The assignment to Auto grader is working so you guys can submit submit your patches and have them be tested There were some notes released on piazzas about that Because gurus out of town And I'm hosting a faculty candidate this week there's a few tweaks to the office hours for the next two weeks so please check the calendar before you come in and Any questions about this sort of stuff? so basically we guys have you know roughly a month and a half to Finish the rest of the assignments, and there's actually you know, there's like quite a bit to do so You know so I hope you guys rested and and rejuvenated and ready to kick some butt on December 2nd December 3. Yeah, Jim. I do know those things Well, let's see here I Mean I can you know how about this why don't we talk about that on Wednesday when we hand back the exams?  I think they have the average was pretty good. I think it was I Won't say I I don't remember exactly. I don't want to frighten people So we'll hand back the exams on Wednesday and summary statistics will be will be released about the exams of the questions Any other questions? other questions all right, so today we're going to do a little bit of kind of a somewhat longish review of memory management because it's been a little while and then we'll talk a little bit about Haging and hardware support for Efficient address slowness. So does any I mean I find it unlikely that people have questions because how many people remember anything about memory management? So, okay, so as I have any questions popped up over break before we go and do a little bit of review Okay, so We had these four Goals of memory multiplexing. So we're talking about Dividing the memory on the system. There were four things that we were trying to accomplish. Does anyone remember what they are? Well, that's up on the slide. So can anyone describe one of these to me? Yeah, great Yeah, so we want to we need to allow processes to use the memory on the system That's a pretty basic requirement of multiplexing, right? There is a resource. We're in charge of distributing it So we need to allow process to use it. What about enforcing those allocations Frank? Yeah, so we need to to be able to say if I give a process a certain amount of memory that it can use that memory But it cannot use use other pieces of memory What about reclaiming memory? Yeah Yeah, so we're going to talk about over the next week about ways that the operating system might be able to identify Memory and not necessarily memory that's not being used by a process This is memory that I've allocated to a process but maybe the process isn't using it anymore or isn't using it very much and In certain cases if I'm on a heavily loaded system I might need to I might want to be able to take that memory away and it's sort of Transparent sneaky way and give it to another process to use another process is going to take better advantage of that resource, right? And what about revocation? Sean Yeah, so I should be able to take away. I should be able to take away allocations from process, right? So This is pretty much what we talked about, okay? And In order to account, so we talked about some of the problems of direct physical Add, you know direct physical memory Allocation and we came up with this nice abstraction of an address space, right? So an address space is our memory management abstraction It's what we're going to try to implement we started to talk about some of the challenges in implementing it But what was the goal of an address space? What were some of the things that we liked about the address space of abstraction? You just answered a question Yeah Yeah, so we want to well It's I so first of all we get each process an identical view of memory, right? and a view of memory regardless of the machine that they're running on regardless of the Time that they're running regardless of how many copies of that process are running regardless of what other processes are running on the system, right? So this is a nice Thing that makes it easy to write process memory layout and for processes to be able to to use memory at runtime your memory Your address space looks the same Regardless of whether you're running on one machine or another machine, right? And then one of the things about the address space abstraction was we were able to you make memory look uniform What else were we able to do? Yeah, but yeah So one thing we were able to do is first of all make it look like there was a lot of it, right? We can give a process of we had these address spaces We were providing a processes that on your OS 161 system are two gigabytes big. I mean which is huge, right? I mean that's a that's way more memory than the system has, right? We make it look like it's contiguous, right? So it's all laid out next to each other which is very nice despite the fact that that actual address space of abstraction is going to end up Fragmented all over all over real memory, right? We'll talk about how that works That it's uniform, right? That it you know kind of all behaves in roughly the same way And then it's private it's about any pointed out. This is a private abstraction So there are ways to share memory between processes, but they require explicit coordination and approval by the processes involved, right? So we had Our goal here is we're trying to implement this abstraction So we had this nice abstraction of these great features. We want to make it work, right? And clearly what we discovered is that there's no way to implement that without breaking breaking the direct connection between physical Between memory addresses and physical memory, right? So we have to put something in between What was this what was this level of indirection that we that we came up with? Question is I can't give I can't give processes physical addresses anymore I can't allow them to use physical addresses. So what am I going to allow in these virtual addresses, right? So I'm going to introduce the idea of virtual addresses that require translation, right? So virtual address doesn't necessarily point to a specific physical address It's up to the kernel to translate it we talked about some of the nice things that this allows us to do, right? So if I give out references rather than, you know, direct access to an object, I can revoke the references That's one of the things I wanted to do in order to Multiplex memory, right? I can share references and there's some clever things that the operating systems will do to try to identify Cases where sharing can be done safely, right? Remember, we don't want, you know We need to preserve the privacy of the address-based abstraction So we might be able to share pages, but we need to make sure that the doing that doesn't violate any of the guarantees That we're trying to establish, right? I can move things around and that's actually going to consume us for the next couple weeks It's talking about when memory stops, when virtual addresses stop pointing to memory What do they point to and how do I use that ability to move things around behind this reference to allocate physical memory more efficiently And finally I can also alter the object behind them without, without changing the reference, right? What's the interface to memory? It's an interesting question people usually don't think about. Wembley Load and store, right? This is not an interface that's provided by the operating system, right? This is a hardware interface This is an interface that's provided by the CPU, right? These are instructions that the CPU will execute on your behalf, right? But what we do with virtual addresses is we change this interface, we change the semantics, we make it much richer, right? And that operates us and participates us, right? All right, so we talked about virtual addresses So when we talk about physical and virtual addresses, just to review, a physical address points to memory, right? A physical address refers to a, you know, a specific byte of physical memory on the machine, right? That holds value and is, you know, cleared it, you know, is able to be loaded and stored it and loaded and stored it Load and stored, I don't even know how to say that, I'm just going to stop trying We're able to load and store to it and it, you know, powers off at runtime And a virtual address points to something that obeys the memory abstraction, but isn't necessarily actually memory, right? And again, we talked about how using virtual addresses allows us to layer on a bunch of additional semantics to physical addresses, to memory addresses, which we liked, right? All right, so who can tell me how base and bounds mapping work? So now we have this abstraction, right? We have an address-based abstraction We've discovered that the way to implement it is by introducing this level of Indirection into our memory addresses themselves, right? So we've come up with this idea of a virtual address. Now the problem is how do we translate this quickly, right? What happens if every translation, you know, let's say we had an architecture that required every memory address to be explicitly translated by the kernel? How, what would that architecture be like? Really slow, yeah, like unusably slow, right? So that would not be good But so what we want is we remember we have this nice We've talked about in this class one of the design principles that operating systems try to adhere to is the separation between mechanism and policy So we want the mechanism here is the actual process of translation The policy is how the translation should happen and what we want is we want a way for the hardware to allow the operating system to set the policy That's used to translate addresses without having to translate every single address by itself, right? So the hardware is going to provide the mechanism operating system is going to provide the policy, right? So base and bounds address translation, this was the first and one of the simplest approaches address sensation we talked about how Does it work? Heaven need a base address, right? And what else do I need? There's two things I need to tell the hardware memory management unit, right? The base address is one of them, right? What's the other one? Yeah, so I need a base and a bound, right? I need a base address, right? And I need a bound, okay? So I assign each process a base for this address and a bound that this determines essentially the amount of memory that this Process is allowed to use, right? We're talking about a single base and bounds per process. How do I translate that address? AJ, okay, how do I do that? Well, remember here, so there this is similar to segmentation, right? But how in base and bounds addressing how do I check if an address is valid? It's a little bit It's even simpler than you think German check the virtual address against the base Does anyone else have a different theory lovely? Yeah, remember that the idea here is that the the base virtual address for every process is assumed to be zero, right? The base is if the base here is a physical address, right? So the bound all I do is I check if the virtual address is okay by just checking if it's less than the bound How do I translate the virtual address, right? So now I have a virtual address. That's okay. How do I complete the translation? Doesn't I add it to the base physical address? Okay So what's nice about this? It's simple, right What's bad about it? What does this require about my address spaces? Let's say remember I wanted to give each process this really nice huge view of memory So if I give each process a 2 gigabyte address space, how big does the physical Address space that I have to give it how big does that have to be? Yeah, so now now I'm now I you know I'm really not able to to provide one of the things I wanted to provide about my address base abstraction, right? Basin Bounds means that you know the bounds is the same for virtual the virtual addresses and for the physical addresses So the size in order to implement an address space abstraction The size of the address space determines the amount of contiguous physical memory. I have to give every process, right? So either I have to make the addresses spaces very small Which I don't want to do or I have to have a huge amount of memory on the system, right? And remember this has happened in like 1960, okay, so you didn't have a huge amount of memory on this Okay, so this is not this is not good, right? And this and this also leads to a lot of internal fragmentation because the address space abstraction encourages Processes to spread out right to use a lot of the you know Let the heap start here and let the stack start a gigabyte Above the heap in virtual address space right in physical now in the physical address space that whole gigabyte is wasted Right because most of it's never going to be used, okay? So so we extended this idea right so we said okay This isn't a good fit for the address space abstraction because it doesn't address the sparsity of address spaces right most Virtual address spaces in my address space abstraction are not ever going to be allocated right if you look at virtual address spaces There are huge You know look at my address space There's huge Swassif it that are never used right and then little bits of data little bits of code little bits of stack right So how can we extend this idea to better? Better match that abstraction Yeah, so I came with this idea of segmentation right I like basin bounds the comparison and the translation are very easy And very easy to do in hardware right but Instead of one basin bounds, which is bad. I just say I can have an arbitrary number of basin bounds per process Then we call each one of them a second Right and we now what we can do is we can essentially take this address space Which is extreme again remember it's extremely sparse most of it is unused and we can cover it with a number of segments Right and now the total amount of physical address space Physical memory that I give to the process is essentially more of a reflection of how much of its virtual address space It's actually used right don't have to cover the large areas that aren't allocated right I can just take four or five or six segments and use them to cover the parts that are in use right? And each one can can grow and shrink independently each one can be protected differently So now each segment remember I I took the basin bounds idea and I had to add one additional thing right So now I need to know the basin bounds We always assume started at virtual address zero with segments. I don't I can't make that assumption So the segment I need to know where it starts in the virtual address space Where it should be mapped to in the physical address space and the size right? So I have a starting virtual address a base physical address and about okay Then my translation works as follows I'd see if they're if that virtual address falls inside of some Segment that's been defined for the process if it does then I can translate it by taking the offset within that segment and Applying it to the physical address right so I figure out What's the offset inside the segment by subtracting out the segment start in virtual address space and as I apply that to the physical address Right so in case people don't remember how this works Well, we can run through our example again, right? So what happens? When I the process tries to translate address 10,000 Yeah Okay, an exception right why? How Right because the MMU knows nothing right like this is the sum total of the MMUs in You know knowledge about the world right? So it says I don't know how to translate this address right so it asked the kernel right now Let's say this and so what does the kernel need to tell the MMU now in order to allow it to translate this segment? Tim Well, there were three things associated with the segment right the three pieces of data that the MMU is going to need to Translate this address Alyssa Right, so I need I need to know where I need to base address right so I need to know where is this in physical memory? What else do I need to know? Yeah Right so the bound which Alyssa got and the start address in the virtual address space right so I tell it The kernel tells the MMU yes in fact there is a segment right so the MMU at this point doesn't know whether This is a valid address or not what the kernel is going to tell it is yes MMU. This is a valid address There's a segment for this process that starts at virtual address 10,000 It's located in memory at physical address 43,000 and at a size 1,000 so Here's something else. I want to point out that we didn't really talk about last time How many addresses so when we started this example how many addresses did the MMU know how to translate? zero right How many does it know how to translate now a thousand right though every address within this segment can now be translated without Causing an exception right so this is something you want to think about when you think about how this works because we've caused this exception Right, we didn't want to cause exceptions right, but in certain cases there's no other way around it Right the MMU has no idea what's what's happening doesn't know how to do this But now the nice thing is the kernel has just told the MMU with three You know with just three pieces of information how to translate a thousand different virtual addresses right so and you know assuming that this process is going to use other addresses inside that segment then All those addresses can now be translated without without requiring that the MMU asked the kernel for help right So again now I know how to translate this I can find this in physical memory and I can complete the you know the loader store How many people remember how this works now? Okay, let's let's do another one right so what happens here goes to the kernel ask for help the kernel tells it Here's another segment. So how many addresses does the MMU know how to translate now? 1500 right for two exceptions. It's pretty good right And remember that The other thing that's going to happen here is that a lot of times So so may ask you a question when a when a process is running right if you looked at the stream of addresses that a Process is is accessing Do you think those that stream is just completely? Randomly distributed all over the address space What would you say? What do you think? That's in your intuition about how processes work Jeremy Well, I'm not even talking about allocation. I'm just saying let's say you were sitting here in the MMU and do what you were just watching for a particular process right and You know you were just looking at the addresses it was using and you and you just kind of map them into its Virtual address space into its you know into the address space that it's using Do you think those like we would you see sprinkles of addresses all over the place? Yeah, I'm only talking about addresses translated by this process. Yeah, maybe the same ones Okay, so that's a start of an answer. Yeah, why yeah, okay. We're getting warmer here Well, okay, so the process will use the same virtual but I'm saying like more on a temporal basis, right? So first of all what what provides at least one stream of addresses that the MMU needs to translate, right? So loads and stores from memory Provide some indication of where the data is that the program is using but what provides another stream of of virtual addresses that the MMU needs to translate Looked up Yeah, what about fetch right fetches and executes So when you start thinking about the code that the process is executing right first of all it all kind of lives in the same part of it Right, but but what's the usual? Cycle when up when a process is running right? How does how does the how is the program counter normally manipulated at runtime? This is like Computer organization review, right? What's the typical cycle for the one that the processor is going through right? Fetch decode and execute and then in the normal case unless I'm doing a jump. What do I do what? No, no I fetch decode execute and then where's the next instruction I'm going to execute? Manish Yeah, it's the program counter plus four right Unless I do a jump right? I'm just you know linearly moving through the you know the code right now There are jumps in your code right? That's how you do program flow but the idea here is that Processes exhibit a fair amount of locality right address locality So the idea here is that let's say I'm running some loop right and that loop is located in this piece of code Let's say there's code here, right? So now not only have I told the CPU the MMU how to you know translate a thousand addresses I might sit there, you know executing like 20 of those addresses over and over and over and over right? I'm in a loop that you know that the loop count is like a thousand or something I'm gonna sit there spitting in this tiny tiny little bit of code and the nice thing is the MMU never needs to ask the Colonel again how to translate those out right so one of the things that helps us here is locality Now wish I had some statistics about locality to show you guys, but but assuming their processes exhibit good locality These types of address translations could be quite efficient right because you know if it was jumping all over the place Right, I'd have to be constantly telling the MMU about new regions of addresses and we're gonna talk about paging today Which makes this even a little bit worse, right? All right, so yeah, we decided to kill the process because it tried to add just that even though dead beast is a perfectly Okay address okay, so any questions about this before we talk about Before we talk a little bit more about segmentation and introduce a new idea Any questions about segmentation in particular? Yeah How okay, it's a great question. How do processes decide what is the physical address? Who thinks they can answer that question? No, Jeremy. I'm gonna know you yeah So the process never knows a physical address. So what does this mean about the process's decision about what physical addresses to use? Yeah, great it doesn't make a decision about it Right doesn't know right if it doesn't know it can't make a decision process doesn't get to say Hey, I would like that piece of physical memory over there. It never knows it doesn't know anything about physical right all the addresses processes ever use our Virtual addresses right processes never see physical memory Okay, it's a good question any other questions. Oh How the okay good okay fair. How does the kernel decide how to allocate physical memory? We haven't discussed that yet, but we but we will yeah, right Yeah, so so first of all when you start so this is a this is a good review, right? So where do a lot of addresses come from right? Where's what's one time during a program's execution where it tells the operating system about a lot of different parts of the Virtual address space that it's going to use. What does this happen? I hear muttering but no answers Paul No Pick on somebody new yeah on earth Okay, when does that when does that happen? We'd be more specific That's correct Nathan, when does that happen when I call exec? Exact is Tells the kernel here's this blueprint of how I want my address space to be laid out right and here's a file that contains a Fair amount of the contents that I want you to put into my address space for right That file has this l format that you guys are gonna learn a little bit about for assignment to not a huge amount But a little bit right so there's this blueprint and the Program tells the operating system. Here's how I want my address space laid out Okay, so at that point the program is essentially telling the operating system I need a segment of this size. I need a segment of this size. I need a segment of this size, right? So the operating system knows what physical memory is allocated and has to somehow figure out Where to find segments of that size, right? So this is essentially an allocation problem, right? It's quite similar to the types of allocation problems that are solved by things like malloc Right, we're actually gonna so I'm gonna come back to your question Because we're talking about today an approach that makes this allocation problem much easier Right that avoid some of the fragmentation problems but in general the operating system needs to keep track of the physical memory that's been allocated and when it receives a Request for memory it needs to find an area of physical memory that it can use to satisfy that allocation, right? This is a good question Any other questions and you guys will write this for assignment 3 for assignment 3 You guys will be in charge of keeping track of what memory is allocated on this, right? And you will have to figure out how to do this, right? Your system right now Doesn't keep track of it at all, right? That's why it runs out of memory and dies, right? It keeps no track, right? You guys will be in charge of making sure that that you can you can track and reuse allocations, right? But there's a nice idea that we'll talk about today that makes this a little bit easier, okay? Right, so we came up with this idea of segmentation and segmentation seems pretty nice, right? It seems like I have this nice abstraction. I can use it It's it's a nice balance between base and bounds the simplicity of base and bounds But it it meets the features of the address space that I want, right? It gives me a way to allocate only the parts of the address space that are actually used, right? So that's kind of nice, right? The the if you think about the pros of segmentation, right? So I can Segments are pretty still fairly simple. I mean base and bounds are simpler, right? But base and bounds also don't do what do we want, right? So with segments The translation requires one addition So I have to find that you think about it You have to think about what the hardware has to do, right? The hardware has to be able to keep track of some number of segments, right? It's gonna be a fixed number because this is hardware, right? So hardware needs to be able to to keep track of a certain number of segments When it translates each address it needs to find the segment that that address is in and then the translation takes one One addition, right? So essentially, you know The hardware does needs to do a search somehow through all the segments that it knows about and then it needs to Once it finds the right segment assuming the segment exists, and it doesn't need to interrupt the kernel It needs to perform one addition, right? And Again, the nice, you know segments are a nice concept, right? So for example when you guys, you know If you guys have right written C programs before you've seen this error about a segmentation violation, right? a lot of modern operating systems despite the fact that they Utilize a technique called paging which we're going to talk about today to actually translate memory Still organize the address space into segments, right? Segments are kind of a conceptual idea that allows me to say this segment is code This segment is data because usually a segment has permissions that are the same, right? So a data segment may be read write a code segment may be read only, right? and This was also a nice fit for our address basis at least it led to less internal fragmentation, right? So it was based on balance there was this huge amount of memory in the address space that was wasted in the segments I got to wait with this, right? But there are still some problems with segments okay One of them is that if you think about it the segment the whole segment still has to be resident in memory so the operating systems decision is now You know should you know first of all as segments get big I have to find large amounts of contiguous memory, right? So let's say that a process has to heap and I found a nice space for it in between a couple of other Allocations and then it says I want to extend my heap, right?  Well now the kernel has to find it a whole another piece of contiguous physical memory, right? And that can become difficult, right? There I might have this you know in external fragmentation problem, right? Where I have pieces of memory, right that add up to an amount that's large enough to satisfy the allocation But I don't have a contiguous piece now. What could I do if I was the kernel? Let's say I had this problem, right? I had a segment that somebody tried to grow and I couldn't grow it because it was penned in by other segments I need their side of it. What could I do? Somebody asked about this last time What what do I have the power to do, right? Remember I've introduced these virtual addresses. That's fantastic, right? So now I what can I do to all of it to any allocation on the system? Yeah, yeah, so I could say hey, you know I'm gonna like run some sort of compacting algorithm on my you know on my memory, right? So I'm gonna take all the allocations out there. I'm gonna start moving them around, right? But what do I have to do in order to move them? I have to change the virtual to physical mappings But what else do I have to do? This is important. Well, yeah, I mean I need to make sure that they don't try to change it But but again what it I wish it was that simple, right? I wish all you had to do was change the virtual to physical addresses, but what else do I have to do? Yeah No, that's not that's going to be very easy to yeah I have to copy the data, right? I can't just like let's say I had a segment of data over here that somebody had had written some really important data into I can't just move it to you know change the mappings and say hey use that memory over there I have to choose I have to make the contents the same right that would be really Unfortunate for processes if when I moved their allocations around the contents changed right that that would break the guarantee I wanted about memory right virtual addresses are supposed to act like memory so when I do a load After I did a store. It's supposed to be the same thing So if I start moving allocations around I have to do all this copying in physical memory, right? Which is you know not necessarily terrible. I might have to do it sometimes, but it starts to sound kind of terrible I guess I said it wasn't terrible, but it's kind of terrible right You know it's potentially high overhead to start moving segments around and now right and I still again. I still have this problem where I have this external fragmentation I wish I had a graphic for this right, but I could have cases where you know I've got a bunch of segments allocated and I have space in between them But I can't I can't satisfy a new segment allocation Without moving all these segments around which is a potentially pretty high overhead thing to do. Yeah, Jim Yeah, but you can imagine like if I had a badly fragmented I mean this is very similar to how disfragmentation works right if I had a bad a fragmented memory space I could stop all the processes and just carefully rewrite all the segments to the beginning of memory and and give myself like one big contiguous piece that I could use that right But that would also be slow, right? So let's let's take a step back right so Really the reason why we started this is we wanted a way to map virtual addresses, right? And we started talking about Ways to do this that would kind of be a reasonable approach given hardware capabilities, right? So we know that hardware is going to have to do most of this itself, right? I still want to be able to tell hardware how to do it And I don't want to have to tell hardware too often, right? But ideally what I would love is to just be able to flexibly allocate any bite in my virtual address space to any physical point, right? So we know that the operating system can't do this, right? And what we've been talking about are ways that the hardware or can assist us, okay? and you know on some level we've been talking about the MMU up until this point as This kind of cash, right? So the MMU has been caching these address translations, right? The MMU asked the kernel the first time it doesn't know something about how to translate a virtual address And then in the future assuming that it understands how to translate that virtual address It doesn't have to ask the kernel again, right? And this is a common trick that we use in operating systems, right? If there's a big thing that's slow, right? Or a certain type of process that's slow in the general case Well, we try to make it as faster in a specific set of cases, you know And one one way of thinking about operating systems and computer systems in general is computer systems are a series of caches, right? So, you know the disk is a cache for memory, which is a cache for the L3 cat Wait, sorry. I'm going backwards Registers are a cache for the L1 cache, which is a cache for the L2 cache Which may be a cache for the L3 cache, which is a cache for memory, which is a cache for your disk, right? Like that's how it works, right? And the speed, you know that I can access those different caches goes You know get slower and slower and slower as they get bigger, right? Registers are the fastest thing on your system to access. You only have a few of them The disk is the slowest thing to access. You have a huge amount of disk, right? So Specifically the piece of hardware I'm going to use here is something called a translation look-aside buffer How many people have heard of have know about TLB's already? Okay, handful, right? So What TLB's are is a form of what's called content addressable memory And TLB's allow Hardware to look up address translations fairly quickly and we're going to use TLB's to implement it to implement a new idea Which is which is called paging, right? so Hold on a sec. I think I might have gotten ahead of myself here Okay, so Okay Let me go forward and see if Okay, so let's talk about pages and then we'll talk about this is a little bit backwards, right? We'll talk about pages and we'll talk about how we can use the TLB to table, right? so So our problem with segments is that segments Segments allow us to map fairly efficiently, right, but they can get too big Right and what so here's the other problem, right? If I have a let's say I have a data the Program has a portion of its address space that it's using in as a heap, right or for data or for its code or something That's going to all be one second So the operating system is such as to find a piece of physical memory for that segment, right? But what's also true about that segment in memory is is every part of that segment going to be equally used Let's say I have a segment that has all my code in As every piece of code the operating that the program could possibly use What percentage of the code that a program could possibly execute is it probably executing in and any given point in time a large amount Small amount Yeah, I mean there's like features of the programs you guys use that you've probably never activated, right? You've never used and never know how it will never use right, you know features of you know Microsoft Word that like our six are buried in six different sub menus, right? And that that part of the code space you're never going to use, right? So now I've got this big blob of code that's sitting in physical memory and Only a small amount of it is is an active use So what kind of fragmentation is this? Okay It's internal right because I have this big allocation. It's sitting there in physical memory I can move it right I might be able to remove it from physical memory and put it somewhere else But what's true about the segment abstraction? I have to move the whole thing, right? So if I have a big piece of code and Let's say I you know I say well I want to reclaim that memory So I'm going to copy that that piece of memory somewhere else, but as soon as the Program tries to use an address inside the segment I have to get the whole segment back into memory right regardless of whether or not it's using big pieces of it So I have this internal fragmentation, right? So segments are kind of too big right, but if I started mapping individual bytes, what would what would happen? Let's say I let's say I decided that my segments are going to be one byte in size, right? I can do that. You know the bound is one for every segment, right? There's a base virtual address and there's a physical address So so I basically what I'm doing is I'm telling the hardware how to map addresses one at a time What would happen remember we talked about before I interrupt the kernel the kernel tells me something and then I can run for a while, right? What would this mean nothing? Yeah, so remember the hardware can only store a fixed number of these translations, right? So let's say the hardware can store 32 translations and each segment is a thousand bytes is on average a thousand bytes So now I can map 32,000 bytes of the virtual address base without requiring the kernels help now I make the segments one byte, right? So how many bytes can I map now? 32 right if I'm lucky the process is sitting in a teeny weeny little loop, right? Act that axis is like 16 memory addresses, but that's not very common, right? So is the segment, you know is the size of the amount, you know essentially if you think about it what we're doing is we're telling the MMU how to map regions of addresses, right? As those regions get small the number of times the MMU has to ask for help goes up, right? Is those regions get really big the internal fragmentation that occurs inside those regions gets gets back, right? Does this make sense? This is kind of like the big trade-off that's involved in memory management, right? As the regions get small I have to tell the hardware all the time what to do, right? As the regions get big I have huge chunks of physical memory that have to be resident, right? Because any address inside of them could be being used because the hardware knows how to translate it But large portions of it are not going to be used, right? So what we do is we come up with something we try to choose a fixed Granulary, right? What we're going to do is we're going to say we're going to try to choose a middle ground You know a middle ground that's what we want is a unit of Virtual memory that's big enough That the hardware can translate large amounts of virtual address space efficiently But small enough that there's not too much internal fragmentation. It's going to take place, right? The other thing is by choosing a fixed size, right? We're also going to choose a fixed size, right? We're going to say, you know, essentially every segment has a fixed size, right? So we're not allowing segment size to vary anymore, right? What's nice about having fixed size segments? What does that make easy? There was this problem before that we had about What that we're going to be able to solve now. Yeah It makes kernel allocation really easy, right? Because remember before I had this problem where if my segment size changed I Might have to move it and move other things around, right? Now my segment size never changes So what I go what I'm doing is I'm allocating fixed units of memory, right? I break up the memory into fixed units and I can if I have a fixed unit of allocation I can never have External fragmentation. Yeah It does mean that right so now what I'm going to do is I'm going to say the bound is fixed, right? So the hardware doesn't have to know the bound, right? That also makes translation a little bit little bit easier, right? So now let's go back and talk about the TLB. I'm sorry. I got this backwards. I don't know why I ever thought this made sense this way So so right so the nice thing here is is we can use this Content addressable memory in a clever way in order to do this virtual to physical translation, right? so so what we're going to do now is we're going to identify identify Areas of virtual and physical memory By there what we call them. We call them page numbers, right? So I take physical memory. I divide it up into a fixed size, right? It doesn't matter what the fixed size is for this example, right? Could be anything it could be a bite It could be a thousand bites, but it's fixed, okay? Now the nice thing is what I care about is what page of virtual memory is this particular address it, right? And it turns out on modern systems. Usually I use a page size of either 4k or 8k The nice thing about using a power of 2 it is allows me to lop off some bits to get the page, right? But let's say let's say I'm doing this right so I have a virtual address that comes in right and it's in page 800 What content addressable memory allows me to do right? This is what TLBs are very efficient at Well content addressable memory does it essentially searches all of the entries in this buffer in parallel so there's a fixed cost to kind of And again, you can just you can read about TLBs and hardware if you're curious about how this is actually implemented, right? But the idea with the TLB is that there's a fixed cost for looking up this address in the TLB And in some sense it is searching all the addresses in parallel hardware, right? So what's going to happen is it's going to match to this address, right? And then what I'm going to do is I'm going to say virtual page 800 for this process maps to physical page 306 This is how I use this particular feature of hardware to help me translate page numbers, right? I want to slow down a little bit because I feel like I'd be wielded maybe some people so So again because we have this problem with cans, right? We're going to We're going to choose a fixed segment size that's big enough to allow the super so for example I could have done this on bytes, right? Let's pretend that we've decided that we're going to translate byte to byte, right? A byte of virtual address to a byte of physical address, right? So again, here's my virtual to physical mappings. How many bytes of memory does this cam know how to translate? four Right and in modern systems, I think cams are usually limited to maybe like a thousand addresses, right the the the size of this particular piece of hardware scales Exponentially with the number of addresses that it's able to translate, right? So you can't just make one of I mean it would be great if you can make huge Content-addressable memories, right? They'd be really useful for a lot of things, right? But it turns out that to make them fast the the the complexity of the cam scales Exponentially with the number of addresses, so we can't make them arbitrarily large So if let's say my again, let's say my segment my fixed segment size is one byte So there's four bytes that this that this TLB knows how to translate So let's say I wanted to use this TLB to translate 16 bytes, how would I do that? Yeah Yeah, so if this if each one of these addresses identifies four bytes of memory, right? Now my cam can translate 16 bytes of memory, right? Let's say I wanted it to translate 32 kilobytes of memory How big would each segment have to be? This is Fairly simple now 8k right if each one of these virtual segments is 8k now I can translate 8k worth of worth of memory, right? So Okay, so we talked about pages, right? The the nice thing that pages that page sizes also do is Fixing a page size also limits the size of kernel data structures that are associated with managing memory, right? Because if you imagine I so if I have my virtual address space Let's say my virtual address space is 4 gigabytes and I break it into 4k pages How many what's the maximum number of pages that could be present in a given processes address space? like a million Rough and the other thing that help so the thing that we come back to that helps us here, too Right is that execution locality and process locality really makes this work, right? because the the Hardware is never going to be able to translate large portions of the process address, right? It just becomes too inefficient to do it quickly and hard, right? There's a trade-off in hardware between the amount of address space I can translate and the speed at which I can perform each individual translation So what I've done is I've came up with the compromise but again the nice thing here is that most processes You know they do jump around within their address space But you know while they're executing they're basically loading and storing from the same thing and then they might jump And I might have to reload a new translation and then they're going to execute there for a while And then they might jump somewhere else, but there's a fair amount of memory locality, right? So as I said before 4k is a very very common page size, right? Nobody really knows why this is true, right? This page size was picked probably in the 1970s and But you know it seems pretty robust like there are some systems now that are using 8k pages or sometimes even larger pages like 64k pages I think some systems have support for much bigger pages I mean modern systems have a lot more memory than they used to right so it's kind of interesting that we're still using roughly the same page size Right, but but there hasn't been a huge amount of changes So again if you imagine that I have 4k pages and I have 128 entry to B now I can catch translations for a megabyte of virtual memory, right? That's pretty and that's pretty good, okay? And again, we think of pages as fixed-size segments, right? I think I have an example here Okay, so now how do we do page? Translation so let's walk through this and then and then we'll be done for the day. I think so I'm going to break up my address space into Let me just get through this and then I'll let you guys go and we'll come back to this on Wednesday So I'm breaking up my address space into fixed-size pages. I'm breaking up physical memory into fixed-size pages, okay? Every virtual page in my address space that's valid maps to a physical page Or it might be somewhere else But the point is that if I'm using it it has to be in memory somewhere and there's a page to page mapping Right remember essentially I'm thinking of everything now in units of 4k right 4k of memory So If I'm going to translate an address inside a page It turns out that the page number of a 32-bit address is now just the top 20 bits right again These are fixed-size segments the bottom 12 bits identifies the offset into my fixed-size segment or my page Right the top 20 bits identifies the virtual page number, right? So what I do and we're going to talk about address We're going to talk about different data structures to do this right is the kernel if the MMU doesn't know whether or not this virtual pages Ballot it's going to ask the kernel and the kernel has to figure out if there's a physical page translation for this virtual page Right, so we're going to talk about data structures for doing this right and Now what I do is I look at the physical page I take the physical address I take the physical page number and I add on that offset right so this is very much like Segmentation right this is like segmentation with a fixed segment size which is 4k, right? It just turns out that because the segment size is 4k I can do this fancy, you know bit masking to do what I want to do right This is essentially like taking this and subtracting off the offset in the react right, but I can do it using using bitmasks, right? Okay, so on Wednesday, we'll start with this example and we'll get through paging and we'll talk about some kernel data structures that are used to make paging more efficient you


Hopefully I'm all good zapped. Oh! Woo-hoo! Yow! Today we're going to do something super special. I've been hitting up these estate sales a lot lately just to try to get some CRTs and other old tech. And luckily for me, I've been able to find a few of them and I actually won some items. So today we're going to go and we're going to pick up those items, a couple CRTs, some other hardware. And we'll see how all this goes because this is always an adventure. I have to drive about an hour from my shop here to pick up the items. So let's go ahead now and hop in the old Mercury Grand Marquis, baby, and cruise on up to a little town called Markham, Virginia. Just a absolutely beautiful drive here today through the woods. Just wanted to give you a look around this, I mean, middle of nowhere where I'm at.  But that's where we're heading back from. I did get the stuff and it's actually in the trunk because I had to put the bicycle back here with me. But nice drive. I'm going to show you all the things I got here in a second because I think you're going to have quite an enjoyable time checking out the estate sales CRTs. All right, let's see what secrets we scored, not a whole lot back here. First off, check it out. Did find this cool Sanyo tape deck player. Now the problem with it is that it doesn't have the power supply with it so it's not a simple power cable. It's actually got a DC power supply right in there. So we need to figure out what we're going to do with that. We'll take a close look at this inside, especially if we can find a power supply for that thing. M7700K by Sanyo. Now the real deal is this one. The Sony Trinitron, baby. Look at this lovely, cool RF only Trinitron. I think it's RF only. Wow, yeah, look at that. UHF and VHF, my friends. Now this is the KV9400 and this is actually older than I am. It's from 1980. So we finally have scored a CRT older than me. Man, I hope this thing works. Oh, it's heavy. Let's get it down into the bunker and see how it looks. All right, let's take an opportunity to look closer at this amazing KV9000 and we'll even try a test run. Man, it's got some color, hue, and brightness adjustments over here. Auto, okay. So I don't know, that's probably gonna be wild looking if it's not been messed with in a long time. If it even turns on, right? If we look back here, we've got color controls, screen. This is our back of our fly bag, most likely entrance to that. Now it does have UHF and VHF connections. As I said, okay, I've got some things set up here.  I'm thinking VHF is the right one. I can't remember off the top of my head but I believe VHF is the frequency I wanna try to get channel two or four into. I've got this little adapter that lets me go coax into this old school antenna connection. So let's hope that will give us a signal. Now when I turn on the Famicom here, I'll get a red light there. I'll get a red light here saying my box is on. Actually it's channel three or four. So we'll start with channel four. That means it's getting a signal. I do have the CRT powered, well plugged in, not powered on. And let's do it. Let's go for an initial power test here of the KV9000. Oh my goodness, I'm kind of nervous that it's not gonna work. But let's just see what happens. Oh, I think I hear some life to the set. Let's see. Oh yeah, we got some scrolling. Oh, we actually have a screen here. So let's see if we can figure out what the heck to do even with this guy or how to change channels. C seven, two. Oh my gosh, I have no idea what channels these are, right? Let's try channel three on the modulator. I don't know what channels these are giving me, right? So let's go over here. Whoops, so that's my thing there. This is channel five. I don't know. Maybe let's go to this one. What happens when I scroll this around? Says here, turn knob until the desired channel is properly tuned, lower number channels. So I don't know, let's see. Oh, we're getting somewhere. Okay, this is gonna take a minute. Let me see if I can figure out how to use this old tuning machine. Holy moly, this is way crazier than anything I've ever seen before. But thankfully the screen's working. So hopefully it's just a matter of us getting something on there to show up. So let me work a little bit more with this whole setup and we'll see if we can't get a screen or a nice image on there for you. All right, well that only took me about 20 minutes to try to figure out, but you can tell I've got a picture on here now. Just the normal picture. I'm gonna try to turn down the brightness on there so you can get a better look at it. We've got all colors. It did take a while to tune this. It definitely is the VHF. And then I had to go in here and do the old school tuning of a number between two and four. And that's what I've got here. So let's just check out something on here about Super Metroid since it's got a lot of dark backgrounds. Look at this. I mean, there's definitely noise in the image because I mean, it's not been serviced of course, but it looks pretty good right now. I love that glass overlay. Or it's not glass, it's actually plastic. But yeah, what do you think? Let's see. Let's see if we can get some volume. Yeah, we definitely have sound. Wow. So yeah, there you go. Interesting picture, huh? Let's turn it down. Cool. Well, at least this one works. I did only pay $22 for this. I think I paid 20% fees on top of that. So another $5 almost in fees for the auction house. And that's about it. Probably gonna do some more things with this TV. Look at this. This is an auto color setting because you can adjust colors on it too. Look at that. If you like. Can adjust the hue and temperature of colors. And I've got my CRTL torn open here. I thought I'd get in and see a little bit more about what was going on inside of here. First off, here's our tube number. It's a 250AKB22 with an orange Trinitron label that is made in Japan. Here's our lovely yoke. Pretty nice yoke for a nine inch tube. And then if you see here, I don't actually wanna discharge this tube. This is the weird setup that is on actually the 2530 PVM, which I've never seen a breakout like this on a smaller one. That's kind of interesting to me. This little contraption here, probably yes, it is gonna control our horizontal static convergence. If we spin that wheel, and we'll do some of that in a minute when I start it back up. But that means that this has that style of anode cap where there's like a little plastic connector under there. And if you mess that up, it's really easy to snap a little piece of it off. And if you mess that up, you'll never get the picture to look quite right. And you can't find a replacement for these anymore. So just to look down here, this is where all our power and deflection is down in here. The flyback transformer neck board has a lot of color controls on there. Everything's looking pretty good.  But other than that, it doesn't look too bad. Over here's more color controls and really tuning controls for the set over on this side. And if we look over here, this is the top. I did take the speaker out, which is a nice shielded, look at that shielded 1.5 watt eight ohm Sony speaker. And then this is the top. I've taken this all apart just to try to clean it a little more. I'm gonna try to avoid taking out all the channel selectors and things. This is about as far as I wanna take this apart. But I'd also like to get the screen out and try to clean under the front. But I don't wanna take the whole TV apart. We'll see if this is even possible here in a second. I'm gonna try to move some things around so if I can get that tube maybe moved out to the side over here a little bit and not have to worry about tearing down everything because I don't wanna remove the flyback at all. All right, let's see what I can figure out. All right, well, the entire CRT, as you can see, has been broken all the way down. And I had to remove the tube and the chassis and everything in one bundle like this. I could break some of these boards out, make it a little easier. But again, I don't wanna mess up that flyback at all because if I do, it is toast and this TV is pretty worthless after that. There's another look at that tube number down there and our deflection yoke right up in here and then our neck of our tube. And then if we look over the boards a little bit, this is our CRT socket right there on this neck board. And we have some adjustments on here but this is a really old school neck board. You see that's a primitive early design for the CRT socket itself in there. That Sony eventually changed. That's quite interesting. But you have color controls up here, manual color controls by these variable potentiometers, resistors, resistant potentiometers that are gonna have variations. Then you can actually adjust them from the other side. So you have three colors over here and then your drives for, is that just green and blue there? Down here you have a G2 adjustment and then over here is the focus adjustment. Down there right there, that's focus and that can again be adjusted on the backside, thankfully. It's got this nice plastic shield right here so you won't hurt yourself and accidentally shock yourself when you're back here looking at it. That's a pretty nice touch. A lot of this stuff is pretty well isolated and protected so you couldn't, you'd have a harder time shocking yourself with this than you may think. But I'm sure somebody could do it. Anyway, that's a look inside here. This is our main board. Like this is our hot and our flyback over here. I don't know, what do you think? Has that been worked on before? I'm not sure, that's pretty nasty over in there. Just some flux and stuff over in here. I'm not sure, but maybe it was done in factory.  And so that's all our board's broken out and the thing I really wanted to do was clean up this bezel. So that's what I'm gonna do. I'm gonna get in here, clean up the sludge and stuff from the inside of this bezel. And the plastic overlay is actually one piece kind of melted inside this frame. So that's gonna be interesting to clean because I can't, I don't think I can get that out of there. I haven't figured out a way to without busting some plastic like rivets, you see? Like is that a rivet there? And there's some things like that. Anyway, I'm gonna do my best to clean it and then we'll come and hopefully get it set back up. In one piece. Got a little bit of an experiment set up. I'm gonna do some adjustments back here behind the set. And as I do them, I'm gonna slowly show you what happens on the screen with my phone camera up there. So I'm gonna set this down here and I'm gonna spin some of these potentiometers over here. There we go. Get that camera set up. So the first thing I'm gonna do is I'm gonna spin this one for the horizontal static convergence control and watch the front of the screen. I'm turning it. And do you see how on the screen you have complete separation horizontally of red, green, and blue? So the guns are just not aim, or the gun is not aiming all the colors in the right spot. So if you turn this horizontal static convergence to the correct spot, you should have a uniform beam. There you go. If you go too far the other way, same thing happens. Blue, green, red, or red, green, and blue. Do you try to get them all matched up as best as you can? Something like that. And that's how you adjust it. Now let's see the next one I'd like to do is gonna be the focus, which is down here. I'm not sure. Hopefully I'm all good zapped. Ooh! Woo-hoo! Yow! Did you see that? Woo-hoo!  I'm sure you saw that on the screen. I will not edit that out, because I'm not a scaredy cat. I'm just gonna have to do it again. And I don't mean shock myself, so I'm not gonna try that way. Okay. That's too big. Holy moly, that was a zapperino. There we go. I finally got it in there. And I'm gonna spin it safely this time. And it doesn't really have all the effect of the screen. There we go. Just a little bit of movement on the beam. Not really worth messing with too much. That's it for that. Here's the front screen area. And I want you to notice this has a really light gray tube. And partially, to help that, is this darkened overlay that was included on top of the TV. And as I said before, you can definitely remove this. No, I was totally wrong earlier. You can remove this. It actually fell out while I was cleaning. And if I look up here, there's a couple of slots where it can just slide in. And you can do like that. Kind of push it in. But before I take and put it all back together, I wanna show you the inside of the shell because it may be helping or hurting. I mean, my reception a little bit on my tuner and my RF signal. Look in here, there's our antenna, right? It goes actually through here and then there's a cable up in here. And I redid all this and cleaned out all in here. But there's that and it goes around to this point which is actually got a screw in it. And that screw comes in under here and connects in right where my finger is. And that's part of the antenna loop. So maybe that'll help clear up some of my snow. I'm having very light snow issues on my picture still but I wanted to show you that. Here's the inside of this shell when it's all cleaned up. Now there's some interesting pictures here which I'll show you about the inside of this. It's pretty nice. Well, I've been toying around with the signal here a little bit more. And I'm actually not so much seeing a clarity increase from adding that screw there. Now I'm sure it's working better and back to its normal intention but I'm still getting a lot of RF noise on this set. Now not a ton but more than I would off newer sets which I guess makes sense for a 43 year old tuner. Ladies and gentlemen, what you are seeing is the completely fixed restored and ready to go Sony KV9400. It is running in crispy, beautiful RF here. We have the Super Nintendo and Earthworm Gym. Yes, it looks glorious. I wanted to give you one last shot of this amazing CRT. How about that? Looks so good. Okay, just to give you a final run through, we've got our lovely tray up here that I've set to channels and you can come in here and tune to channels that you need to. There's not too many options at this point. Got our picture adjustment and then our volume. Oh yeah. So that's nice. The speaker's right in here. This is our antenna. And then our screen does have that overlay on it so that kinda does help on the softer image due to that RF fuzz which sometimes you get. I mean right now I'm not getting any RF fuzz so it looks really nice. I love the patina on these old sets. Just came out wonderful. Here's our little hue and color adjustment knobs down here and this is our auto color which it's off right now but it's so tuned in that you can barely tell the difference between on and off. We have brightness control over here. On the other side, there's nothing really on the other side there. Down at the bottom you do have a couple of earphones for mono earphone input. There's our input board, right? What a lovely input board. How high tech. How wonderful of an input board. Easy enough, right? Well folks, there you have it.  When you could play video games on this like it's sitting now, it's not a big deal but it's not something I would think you would wanna use as like your daily CRT if you had another option. And the great thing about this one is it's nine inches. It's an iconic Sony. It's from the 1980s. It works perfectly I guess as intended you could say but all around you could get much newer CRTs and get a better picture experience unless this is what you're looking for. You know, so I don't really consider this a high usable CRT. Again, more of a fun, novelty, beautiful art piece that works that you can get a signal into if you go to more lengths. I mean, right now I'm using a RF modulator and I'm going in really with just RF into this set. So anyway, what do you guys think? What do you think of this set? Does it make you wanna buy more old sets? Does it make you wanna look for specifically an old Trinitron maybe? Let me know what you think in the comments below and please leave a like for me if you enjoyed the video. I'll see you guys next time with some more retro content.