All right, let's see you guys can hear me. I'm assuming again And hopefully you can can you see my picture up in the upper corner there? Great all right Well, welcome to the first online version of 162 we'll see how this works and hopefully this won't be a long term Fix but for now it's good to keep everybody safe What we're going to do today is we're going to pick up where we left off on Last Thursday and continue with our discussion of address translation and as you recall We sort of had a multi-segment Mode that we were talking about which had a set of Registers that were base limit pairs and by having more than one segment We at least would be able to have a little more flexibility to share segments between multiple processes and so on And so the idea is that this segment map this green one was in the processor You take a virtual address Or the top how many bits is this going to be this is going to be three bits because we have eight options the top Basically would point at the particular segment we were interested in we would take that base We'd add it to our offset and that would essentially give us a Physical address and then of course the other thing we need to do is check to make sure we didn't Go over the the offset wasn't too big and go over the limit and we also are going to want to check The valid bits to make sure that this is even a valid segment now We'll talk more about this later, but the other thing I mentioned is although we're talking about the segment itself Coming up here As part of the address so we're taking three bits off the top of the address Later we'll also talk how about how certain processors like the x86 the segment is actually in the Is actually in the Instruction itself now I'm kind of interested to see how we're going to make this work in terms of questions One of the things is you all have a group chat you guys can send chats to me if you like The other option. I suppose we could try is having people raise their hand to actually ask a question That may be another option Let's see what works and what doesn't so this is our experiment for today So Continuing on with this. I'm just wondering if there were any questions or are we good on this? All right. Good. We've even got a thumbs up very nice. So I'm moving along here The the next thing that we might want to do is Continue on this to see what we can do about Fixing a pretty serious problem we've got here. So the pretty serious problem is that essentially with segmentation we get Holes that are not easy to get rid of so we have to take variable size chunks You know, these processes are of different sizes We got to fit them into physical memory in some way that's contiguous because if you look back here We're really talking about a base plus a limit which Essentially means that we're talking about a chunk of memory. That's Possibly variable size that limit could be quite large and so You might have to end up moving for instance perhaps Process 11 doesn't fit in either of these chunks We might have to actually copy process 9 and 10 up so that we make room for process 11 And that's that's a lot of overhead. And so this basically gives us pretty limited options for swapping to disk and so You know, we basically have to swap out the whole thing we have to deal with fragments and this fragmentation problem is essentially a wasted space problem because Notice that right now these two chunks of unused memory can't be used by anybody. So that's actually external fragmentation And then we have to move that up. Maybe move process 11 in here And then perhaps just to make sure that we can grow process 11 We have to leave some space and that potentially would be internal fragmentation. So This seems like a nice first step because it's very easy to do but let's see if we can do better All right. And so to do that we're going to Have an essential observation here. The problem is that we have big chunks of variable size and that basically leads us to this situation where we Have to move things around and so instead what we're going to do is we're going to have smaller chunks of fixed size Okay, and so this is going to be our solution. So our fixed size chunks We're going to call pages not to surprise anybody I'm sure and every chunk of physical memory is going to be equivalent so that we don't have to go copying around things and And Let's see as we Since they're all the same size. In fact, we can even have free memories kept as a bit vector So it's sort of a zero means that the memory is free and a one means that it's allocated And we can do this because every chunk is equivalent and so each one of these might mean If there's a chunk of memory that's in use And each of these might be of some size we'll decide what that is but we're going to say 4k for instance for now all right and Leading back to our question about fragmentation We could ask ourselves whether we have Pages as big as the previous ones and the answer that's going to be no because if we allocate really large pages We're going to have a lot of internal fragmentation Okay, and so typically we have small pages in the 1k to 16k range 4k is pretty common 16k more common And so every segment that we were talking about previously is now going to be a number of segments. Okay, and We could selectively have very large pages for things that don't change much like the kernel Maybe we even have gigabyte size pages there and we'll talk more about that later But for now, we're going to keep small ones to avoid internal fragmentation All right, so Any questions on that are we good Okay, and I'm going to assume that people who have questions raise their hand or Because there's a little hand symbol you can do or they send something to the group chat That's probably the simplest thing for now because that way I can Just look over off to the side here. All right, so let's let's do something about this so if we want to implement simple paging we need to somehow come up with our Come up with our translation table and so here's what this might look like We're going to have each of our pages is going to have a separate entry now. Okay one per process and Each of these entries is now going to define a particular physical page in memory and some permission bits Okay, and it's going to give us a virtual address mapping here. All right, so our virtual address mapping is Going to be as follows. We take our virtual page number now and Keeper just copy the offset and our virtual page number is going to help us pick One entry in here, which is going to give us the physical page number and by Concatenating the physical page in the offset we get our final Total mapping so this is a little different from what we did with segments if you remember earlier Where we're actually doing some summation and the reason for that is that all of these chunks are the same size say 4k and of course Here you know if they're 1k for instance that might be 10 bits if they're 4k that's going to be 12 bits If you remember I warned you guys last time that it's going to be time for you to learn your powers of two but So basically this virtual page number is an index into our page and what we get out of that is a real page number or Physical page number that we replace those bits with for the offset So if this offset were 10 bits, so it's a thousand 24 bytes per page Then how much is this left? Well, I have 32 bit address. There would be 22 bits here That's a lot of pages and those 22 bits would select an index Which would then give us a new 22 bits which points to a particular 4k page, okay And then of course the problem is we're not going to have all of these so 22 is basically 22 bits is 4 megabytes So that have four millions That's a lot of pages and so we basically have our page table size Which we're going to compare with the particular page number we asked for and if we're too big so we're down here page six for instance Then we're going to get an access error and we're also going to check our permission bits Okay, just like we did before now. It's different here over what we had before and Potentially a lot more powerful is that now every one of these pages are the same size and we can map them into physical Memory, I'll show you that in a little bit And if we decide that we're no longer using these six pages We can just put them back on the free list and another process can use them And we don't have to do any copying or whatever because just because page six might not be In this contiguous in physical memory It totally doesn't matter because the virtual address space is still contiguous and I have a couple of really good slides I'm going to show you in a second that make that a little more clear Okay, so I know it's very hard to ask questions in a virtual environment like this, but are we Oh, we do have a couple of questions. Okay, so Both offset lengths so one question here was are both offset lengths the same here And the answer is yes because all we're doing is we're literally taking the offset bits from this and copying them over here Okay, so you're just copying them and I see we also have people answering questions. Okay. That's great Sorry, I'll have to watch this a little more carefully But so basically the reason we're copying is we're not adding and we're literally we're picking a 4k page Based on what we want and then once we have that 4k page The offset is sort of saying which of for instance those 4k bytes are in use or in the case of a 1k page Which of those 1k bytes are in use? All right Great we're on the same virtual page So let me show you a very simple example here. So these are four byte pages. All right, and So this is pretty small, but it's trying to get you the idea. So this page table has a Small number of entries in it. Okay, so the first entry points to physical page for the second entry points to physical page three physical page one And if you notice here Let's take a look so if we're interested in the following address so zero zero zero zero zero zero zero What we're going to do is translate that and since our pages are only four bytes In size we know that only two bits are enough to talk about every byte that we might run into so our offset For our four byte pages is just two bits All right, and the the rest of the address here all is all zeros So that's going to point at this first entry and so the way we get our translated address is we literally take the Offset bits copy them here, and then we take our virtual page number And that's all zeros and we put our virtual page number in here to get index zero giving us back for now You guys all see this zero zero zero one zero zero is four We concatenate the two of them together and that actually points us somewhere down here now I say that this is hex one zero and that's because if you divide four bits off you get a one and a zero And so this is down here. So what we've translated is this part in the virtual space points to this part in the physical space all right and For instance another example here here's the and by the way since these two bits are zero That means we're talking about the first bite a is really down here in physical memory here. The two bits are again zero but this is now Page number one because it's zero zero zero zero one. All right, and so e Because that's the beginning of that page gets translated down here to this address zero C And by the way one one zero zero is C. Okay So the question that is on here is why is the virtual page number? Six bits. Don't we just need two bits for it? That's a good. That's a good question And the answer is that in this architecture we have an eight-bit architecture and so we need to fill out all the bits Okay, and so we need all six bits even though we don't have many pages in our page table And the way that would be reflected if you look back at this previous slide is this Virtual page number is compared with the total page table size and if we're up here Then we get a fault. Okay, and so if you look here In this case, we've got we've got six bits Which if you were to very quickly compute would give us 64 total page Possibilities we only have three pages and so if we were to have something in these bits up top here That would cause an access error. Hopefully that answered that question All right now Are we good on that Great, so let's go forward here now. So if you look at eight page eight here Which is I or excuse me. That's bite eight is zero zero zero zero one zero and the two lower bits And so if you look at page two is a one so we put a zero zero zero zero zero one concatenate the two zeros and that has I up here now You know, this is kind of We all always got zeros here. So that's kind of boring. But what about for instance by six, which is G If you look here six is zero zero zero zero zero one one zero So that's the virtual address for bite six if we split that up We notice that we are offset to one zero and page one and so when we translate that we look up page one That gives us a three. We copy the offset and that ends up being right here That's why G is over there and the same with nine Again, you can do that translation and that's excuse me up there. Okay? So basically this offset is literally copied and because there are two bits It's four possible bites in the page which corresponds to four possible bites You know in the virtual address corresponds to four possible bites in the physical address Okay Good now Next What about sharing so what's interesting is one of our Justifications originally for I'm going to the multi-segment model was sort of to give us a sharing model Where we could share pages and if you look here for instance in this case of the page table we can have Process a has a page table and virtual page might look up page number two We copy the offset to find out where we are Process B could have a different page table but by design these two entries the physical page Number in page number two of this page table in page number four of this page table point to the shared page Which means ta-da we have a physical page shared between these two processes All right So that sounds interesting so that means that and notice by the way the permissions here are Valid read and this is valid read and write so this process only gets to read the page Where is this process gets to write the page? But those either thinking out there might wonder a little bit about this Can anybody tell me what's a little weird about the way I set up sharing in this instance? Anybody think all right? Ah Well one one possibility is if one process crash will crash the other one. That's a good. That's a good thought but the answer is no There's something funny about what I did here in terms of where I placed this Pointer see how it's number two here, but it's number four here Can anybody figure out what that would do to the way? Process A and B view this data Yeah, different virtual addresses very good. So What that means here is that yes this page is shared But if I put any object in here like a linked list or something where the address is mattered These two processes are going to see this data at different addresses in a linked list. It's not going to work Because an address that makes sense for process a isn't going to make sense for process B So the way I've set this up the only type of sharing you could do is something that's location independent I mean you could have I don't know Objects that don't have pointers in them. For instance Probably would want to actually map this Second one at page two in process B, and then they could put linked lists that are in there. Okay, so good catch now Where is page sharing useful? Well the kernel region of every process typically has the same page table entries and Although we'll talk about some of the consequences of that in a second the process can't access it but when the kernel is running When we're running in the kernel mode then the kernel can get at stuff easily and potentially What it really does is it kind of allows the Kernel to have access to the users pages. So what happens is when we go into kernel mode the kernel gets all of the users pages, all right Different processes Can be running the same binary. So what's interesting about what I showed you here is if this were location independent code or We had mapped it to the same address then you could actually put a binary in here and suddenly You could execute the same code in A and B With only one physical copy and this is exactly what's done to make sure that If you load multiple versions of the same program at the same time they can all share the same executable Okay, so the question is why won't pointer dereferencing work? Because basically they Let's suppose that there's a pointer that says that the next So this is this question is back to this slide again here So the question is really if I have a linked list and I have a pointer From one link to the next that pointer is going to point to an address that makes sense to say process a Which is going to be in this part of the address space So that's going to be where there's a two in here and it's going to point to the next link But something that's got a two in it is going to point off into private space in process B So those pointers are not Are not going to be common to the two so that makes sense great. So The other thing that this is very useful for as we talked about shared libraries last class and By sharing data you can share libraries and in fact, that's exactly what happens when you have libraries that Might be commonly linked of course across a bunch of processes And then in that instance you actually get to share the binaries. Okay, so And then another thing that's in general kind of useful is shared memory segments Now we haven't talked about how to do this But if you have two processes, you know, we talked about well the big thing about processes is they they are protected from each other Well, in some instances the two processes actually want to be talking to each other And you've already done things like generated pipes and used files and so on sockets to communicate together What you can also do is you can you can ask for a shared memory segment to be created Between the two processes and now suddenly just like we were talking about in this previous slide They can read and write an actual shared page and this is technology that you'll we'll talk more about Maybe next time that it's a system call that you can call to to get a shared page mapped between two processes and now suddenly you can share a linked list and So that's I like to think of that is intentional or careful hole punching between the The boundaries between the two processes we poke a very careful hole and share a page so that now they can communicate But in a way that we're intending to communicate Okay, so I Don't want to Get past our basic page mapping here. We're gonna do some optimizations on it in a moment, but We could always map things in exactly the same way Including the kernel and binaries and so on and that is actually what was done once but Many of you who might have taken 161 or whatever are probably hold of heard of things like buffer overflow attacks where an attacker manages to Cause the program that's running to overflow its stack in a way that lets that Process introduce maybe a new return code return code or whatever so that the Rather than returning to where it was supposed to it actually ends up returning to something that the attacker is selected And so one of the ways that modern systems work against that is they Randomize the placement of where binaries and stacks and heaps are as a way of making it very hard to do these kind of buffer overflow attacks and so To do that typically you need to have code that can run anywhere and so you can build essentially position independent code where you can have just a random start address and As a result the things that an attacker might try to insert into a stack once they've taken control Are much harder to figure out because the code that they're trying to jump to is Different every time they attack The stack and the heat can clearly start anywhere so you can randomize placement of those and by and large These are pretty good counter measures They don't entirely Prevent buffer overflow attacks, but they make them much more difficult and they're easy enough to do once you've got memory mapping Another thing that's interesting here is you don't map the whole kernel. So and I'll show you a map in a second of a standard Linux mapping from I'm going to say a couple of years ago where you basically had user space was the The bottom three quarters of the space and the upper three quarters was kernel and the kernel all of the kernel data Structures were mapped up here in the purple But they were mapped in a way that was supposedly protected from the user Okay, and if you don't map as much kernel space then it's harder to It's harder to attack and so in user mode. There might be just a little bit of kernel That's peeking through and when you when you transition through a system call or whatever You have a fully separate page table that has the user space mapped exactly the same way But the kernel is now mapped in full and so that's basically going for one page table to two almost identical page tables And I'm going to show you you know a little bit why it's very easy to do something like this because we can essentially have the same part of the page table that maps user space map in a different kernel Or excuse me in a different page table for the kernel Okay, and then of course we talked about meltdown and we'll talk more about this, but basically because of the meltdown Attack and call it a bug in processors We noticed that if we had done this where we have user mode kernel mode together in the same page table protected Only by protection bits that it was possible for a user program to essentially get all of the Protected data out of the kernel and that was bad And so this meltdown protection basically encouraged there to be essentially nothing of the kernel mapped at all in The user's page table and we will talk a little bit more about that as we go forward So here we go. Here is an example of a pre meltdown 32-bit address space for Linux and It's more interesting than we've kind of led you to believe up till now so clearly from zero XC zero zero up is kernel space and This is protected With page table entries. It's a kernel only and I'll show you the page table entries Momentarily, but then the stack is not jammed up against the top of the kernel It's randomly placed and the heap is randomly placed okay, and There's an additional memory mapping segment that's kind of in the middle of these so unlike what we talked you before where the stack grows down The heap grows up and there's nothing between There's actually a segment in the middle where they put things like dynamic libraries and anonymous memory mappings for shared pages and so on okay, and then the other things are kind of put down at the bottom and Carefully initialized in some randomization there too where it's required Okay So that's our that's our actual memory map. Are we good for questions? now I Wanted to show you this is exactly the same thing we said with the page table earlier, but it's a different Way of looking at things so on the left we have virtual memory and on the right we have physical memory and The virtual memory is an address space That's used by the processor and the physical memory is the address space Used by the DRAM and if you look what happens is what's in the middle the translation of the page table? So I'm going back to our simplistic view of the memory ignoring what I said here about more interesting complexities Here what you see is the stack has a couple of pages that are mapped through the page table to a Place that we've randomly placed in physical memory So the fact that this stack is jammed up at the top of our address space doesn't say anything About where these pages are in physical memory because they're all the same size. There's no requirement Okay, so for instance this You know 1111 and the top basically goes through the page table and maps down through the page table to something That's not necessarily jammed at the top and Could in fact be down at the bottom here and it would still work And the stack could continue to grow down and the heap up but notice that we've in fact got a bunch of holes in here and so interestingly here if you have a program that Pushes and needs more stack what's happened here is if you notice before we had Only two pages of stack and now we're trying to use more pages and what happens here Well, we get a page fault which we'll talk more about and now the kernel Says oh the program needs more stack. What are we going to do? Well, it finds a couple of new pages puts them into the page table and now though they although we have a Continuous stack in the virtual space. It's split up in physical space and this just There's no problem with this because the page table basically Stitches these together to give the user virtual memory view a nice contiguous view of things. Okay, and this is a much Better from an organizational standpoint than what we showed you earlier with it We had to deal with segments that maybe needed to be copied and moved here We never have to copy and move because every page is the same size All right Now the challenge Unfortunately here is this page table is big Okay, notice all these null entries Okay, even though I have a bunch of fiscal memory that's not in use by this process I still have all of these null entries now these things in gray might be being used by a different process But that doesn't mean that this page table doesn't have to have null entries for everything whether or not it's being used Okay, and if you think through why that is the reason is that we have a very simplistic translation mechanism We take and we split some bits off The top and we use those bits as an index into the page table. Let me back this up So if we didn't have all these nulls in here, then the parts down here That we're trying to look up Parts of the table would go to index and it would index incorrectly because we need to reserve space in this page table for future use And this space is directly related to the size of virtual memory not to the amount of physical memory that we're using Okay So potentially Problematic from that standpoint All right, so how big does this get all right So if you look we have a 32 bit address space for instance, so 2 to the 32 bytes is about 4 gigabytes Note here that B is a bit and cat is a small bees a bit and capital bees a bite keep that in mind This is a very important convention. So we're talking about giga bites here not gigabits and For memory the other thing to note about is that a kilo grand kilo bite or bit whatever kilo is two to the tenth Which is a thousand twenty four Omega is two to the twentieth Which is 1024 squared and a giga is two to the 30th Which is 1024 cubed and the thing to note and I'm gonna say this and the reason I'm saying this is because you need to know This so you're not confused in the future Notice that a kilo bite is not quite 10 to the third bites and a mega bite It's not quite a million bites and a gigabyte is not quite a billion bites Okay, it's a little more Okay, but that is convention and we use that convention when we're talking about memory sizes The way the reason this gets very confusing is when we start talking about bandwidth for the network We're talking about the size even of Discs disc blocks then we go back to the standard kilo mega giga Which are powers of 10 rather than the kilo mega and giga that are powers of two and if in 61 a Or B, I think you talked about kibby maybe and givey bites which is all very cute sounding but not as Often used as you might like So I guess you just have to deal with Ambiguity and if you choose to put a kibby to down Most people will know what you're talking about but not all of them So typical page ties is four kilobytes Ah To clarify they're pronounced The question is are these pronounced the same but capitalized differently? No Unfortunately, they're pronounced They're capitalized the same and pronounced differently if you wanted to say that so kibby bite is K little I be bite Kilobyte When it's dealing with memory it looks exactly like kilobyte when it's dealing with bandwidth And you just got to know the context to know whether we're talking about a power or two or a power of 10 so Welcome to the world of Ambiguity and yes indeed it's context-specific isn't this fun If if everything did was done the way they teach you in 61 a it'd be great But unfortunately not everybody does that so a typical page size is four kilobytes. So that's how big is it? Well One kilobyte is two to the tenth. That's the number you all should know So four kilobytes is two to the 12th, which is 12 bits So for pages four kilobytes are offsets 12 bits. So how big is a page table? So look back here See this how big is this guy? Well, we split off 12 bits. What's left in 32 bits is 20 Which is well, it's a million And each one of those entries is about four bites. You'll see that at the end of the lecture So we're talking about four megabytes just for this page table much of which is null because we've got a sparse address space So that seems pretty wasteful Okay, and back in the day. This was really bad. Okay, because 16 megabytes was a huge amount and So wasting four megabytes on your page table seemed pretty bad So how big is a 64 bit processors page table if we do it this way? Well, two to the 64th over two to the 12th Is going to give us about Times eight bytes each because the entries have to be bigger where that's about 36 exabytes Which is ridiculous amount of memory. So this page table thing. I just showed you by itself is clearly broken Too big too much empty space Okay And so the address space is sparse and what we need is something to deal with a sparse address space So what needs to be switched on a context switch here? Well The page table pointer clearly needs to be switched All right, what provides protection both Per process translation table and the dual mode. We're going to mention that again that prevents the process from changing its own page table So pros of this page table is it's very simple to do out memory allocation because all the pages are the same size It's very easy to share the cons are if the address space is sparse, which it is You waste a lot of space if the table is really big Not all the pages are used at the time. So Even this big page table would be great if we could page out page table So way too big So we need to do something else and what we're going to do is we're going to figure out a better structure than just a linear page table All right, so administer via uh well cs162 went virtual woohoo welcome everybody to uh to kubatow. It's cs162 zoomcast We will have to decide whether this works or not Is there a kobe? Are you asking a question? Are you clapping? Oh, I see you're clapping. Oh very good I have to look at I have to figure out the symbols Uh, so um zoom links for discussion sections and office hours are not all available yet but they're going to start being available um and Because uh, we don't have to go in person anymore. Um, I think we should get back and they have it essentially attending discussion sessions because I'm very worried. We're going to lose you guys um So that uh, I don't want I don't want you to get lost and not have contact with staff. So Make sure that you try to attend the discussion sessions. There is a calendar at the bottom of the course calendar page.   I haven't figured that out yet. But uh, we're Making this all up as we go Were there any questions on administer via we could okay so, um Let me know, uh, whether this is working or not Let's see our section discussion zoom links to be posted on piazza. Probably that's a good That's a good question. And I think we could certainly do that. Um, I There'll be lots of places to put them and uh, however we can get that info to you would be Good the other question was are the design documents? Our design doc review going to be due virtually. Yes Uh, and this is why this is going to be very important that we figure out a good etiquette for talking I think in the design reviews will have everybody's mic can be enabled and and let's just Figure out how to make this work Personally, I want to make sure that you guys all get as much contact with The staff as as you're used to it's just going to be Talking heads on a screen And i'm i'm quite sure that the the the status of uh fake People is not quite good enough for us to uh Head off to the Bahamas. Well, uh, well these AI Teach you um cs 162 that unfortunately isn't going to happen yet and besides the fact that I don't think we'd want to get on an airplane To do that. So you're going to have real people behind behind the mask and Let's see what we can do to make this work for you guys and maybe People will calm down and be everything will be Looking a little less scary for everybody and we can get back to live classes before the end of the term that would be my goal I'm not sure that's going to happen But let's see if it can and I'm certainly Up for talking with anybody individually if you Really want to chat and of course if you're not sick, but Um, let me know. I will try to answer emails and you can post things on piazza and so on All right, so uh, sorry about this It's uh, and if you see your faculty running around looking crazy trying to figure out what's going on It's because none of us know how this is going to work. So just bear with us and we'll figure it out so Alrighty I don't any other questions or should we go on? Okay so Let's fix this problem. Um, and The problem i'm referring to is just this ridiculous size of the page table So if what we're going to do is we're going to build a two-level page table Oh when we talk about sparse address space is that relevant for virtual memory and not the the d-ram This is going to be relevant for virtual memory, but it's mapping to the d-ram. So, uh, let's see if uh If we answer your question or not and you can ask it again Later, how's that sound? So, um, basically we're going to have a pointer to a page table But this page table is going to be small. Okay, and in fact, we're going to have four byte entries, which I will explain Non very shortly and these four byte entries are going to be Pointers to other pages, but this thing here is going to be just a single 4k page and We're going to do what I like to call the magic 10 bit 10 bit 12 bit pattern And what's magic about this is this is a pattern that works perfectly. Okay, so we take 10 bits, which is if you remember 1,000 excuse me 24 And that 1,024 entries 1,024 times 4 is 4k. So ta-da there is this is exactly 4k in size And we're going to pick one of 1,024 entries based on the first 10 bits That's going to give us a page table entry that points to another page Where we're going to take the next 10 bits Okay, which is going to pick a page table entry and that's going to finally point to Uh a page and if you look we'll copy the offset of course From here to here and when we're done we have the physical page number Of the final address and so these guys in the middle are a couple of levels of look up And this is the final page and so when we're accessing this virtual address this page over in the far Um In the far right here is the page we access in physical DRAM But what's interesting about this is that for instance these guys in the middle are also the same size So these are also 4k And what's all what's and what I call this magic again is the 10 bits 10 bits 12 bits Exactly works out and we use 32 bits when we're done And um interestingly enough, uh once we've done it this way Then uh as you probably realize Some of these things can be empty Or sparse or null is what we called it earlier. And so we could actually If we were not using this part of the page at this moment we could mark This page table entry up at the root as null or uh not present as I'll show you in a moment And then we could send this page out to disk and so this page table could be only partially in memory So what's cool about this is that we have a two level page table And we don't even have to have all of it in memory at once and furthermore It deals with sparseness much better So those big tracks if you remember where uh, we left space for the stack to grow and space for the heat to grow They would correspond to null entries here In this top level page table that mapped to nothing and so we don't have to waste that space on the second level page table All right, and so this uh the good thing about this is really that We can now suddenly deal with sparseness pretty well Okay and This is by the way, the you know the 10 10 12 Is exactly what you see in the classic 32 bit address space translation for x86 Intel which loves to name things in their own way call this Here's the two levels of page table. They tell the top they call the top on a page directory The the lower one a page table, but it's exactly what we just talked about this little ps is page size And i'll show you that in a moment. This basically says it's pointing at a page table as opposed to Pointing at a four megabyte page. Okay, and which so if you uh, look at that We could split off 22 bits and just have a top level directory pointed a really large chunk of memory And we might want to do that in the um kernel for instance if we weren't moving things around And the other thing to point out is this thing called register cr3 is one of those Uh control registers that you can only get at from kernel mode and it points to the top of the page directory And so this is what we were calling the page table pointer earlier and now you can see that when we Contact switch to a new process all we have to do is Change cr3 and suddenly we have a brand new page table that we're using and we don't even have to touch these things in memory So they just stay in physical memory We change cr3 to a different set of things in physical memory and suddenly we've just changed the address space And so this is pretty nice from the standpoint of uh, if we change from a thread in one process to a friend another We're just going to change cr3 Now of course everything's more complicated than that Then you hear the first time because we also have to deal with Um segment registers and so on but this basic idea is pretty good. Okay pretty close So now um The uh, what's in each one of these page table entries? Okay, so a page table entry is this thing here or You know these things here. Okay, each of those are page table entries There are four bytes. What do they look like? Well, um, it's got to have a pointer to the next level page table and permission bits And so here's the x86 page table entry It's 32 bits The top 20 are literally pointing to the next page in either the page table or The final page that we're mapping to and why is 20? Why is 20 bits the right thing? Well 20 bits is the right thing because Uh page is 12 it takes out 12 bits and we need to map the physical page frame number is 20 bits And so that's a million possible 4k pages in the world. I'll go back to this picture again This guy when it's mapping to some page in physical space There are 20 bits worth of possible physical pages In a maximally filled, uh 32 bit d-round Okay So That's why these this top page frame is 20 bits. But what about the rest of them? Well, this lower bit is a very important one. That's the present bit Which intel calls present. It's called valid and everywhere else And that basically says whether this page table entry is even valid So if we want to have a null entry we mark this as a zero And then suddenly the rest of the bits all 31 of them are available for the operating system to do something else with We'll talk more about that when we talk about paging next time. The w-bit is whether it's writable or not. Okay, so if it's not writable then You set that to a zero and an attempt to write would give you a page fault U is whether the user can do it or not use it or not if that's a zero then only the kernel can touch it And these other ones are about External caching and so on maybe we'll talk about that a little bit later This a and d bits are information bits. So what's interesting about this is they keep track of Whether a page has been accessed recently or whether it's been written to and those will be part of the clock algorithm when we talk about paging This is the page size bit that I mentioned earlier If it's a zero then this is just pointing to the next level page table But if it's a one we can point it four megabyte pages, etc Okay, so this is our basic page table entry and if you add these all up there four There's four total bytes here because it's 32 bits and that is literally what I meant by These four this is a four byte entry And what can we do with this? This is just a preview but there's some things we talked about in The use of fork and so on that suddenly become valid here. So how do we use this page table entry? Well, there's a lot of bits, right? So one of the things we'll talk about next time is We can use the a and d bits and the p bit and the w bit for a clock algorithm So that's kind of interesting and that's for Making the virtual memory look like a cache. But what else can we do with it? Well, for instance We can and by the way, I Just so you know if we mark something as invalid then that's a part of the address space that We'll cause a page fault and you may have to go to disk Because we've paged things out. But other things that are interesting Along those lines is that means that we can page out things from a process We're not using set their bits in the page table is invalid And if only if the process tries to touch that part of the address space then we get a page fault We pull them in off of the disk. That's called demand paging Copy on right was mentioned with fork Right. So the idea with fork was not that you copy all of the data What you do is you copy the page tables and mark them as read only So that then if one of the two processes tries to do a right what happens there is you'll get a page fault Because that w bit's not set and then you could copy two copies of the page Mark them both is writable and now suddenly you've only copied the parts of the Physical data that are actually going to be different because the two processes decide to change them And then another version like this is zero fill on demand You can mark a page is totally invalid and when you touch it Then what happens is the operating system fills it with zeros and then gives it to you And this can be a very important mechanism So that we don't accidentally give data from one process to another just because we've reused the physical page Okay, so um, these are some interesting uses of the page table. I was wondering if there are any questions on that I'll remind you a bit of this next time as well But so you can start to see for instance this one. I think in the middle of how copy on right works is potentially pretty interesting all right, so um, the next Thing that we can talk about here as well. We're going to share Data now with this multi-level page table. There's no reason we can't right? So here's Uh, this is the value of cr3 for process a and the value of crb for process b And somewhere in the middle here notice that we have a couple of entries that end up mapping to this, uh A whole chunk Of the second level page table So suddenly there's a whole chunk of pages that are shared between the two processes just by mapping A lower level chunk and so we can have many pages and so not just a single page which we could clearly share by Mapping something from the bottom level to the same physical page But we can have a whole bunch of pages just by mapping a middle level chunk and this is how we can Do all sorts of complex sharing potentially Also during fork we can set this up so that we copy Kind of only the top level pages could point at all the second level ones rather than having to copy all the bottom level ones Um, so to summarize Just like before we kind of showed you this is the virtual memory view But now we go through a couple of hops and so there's many less nulls in here You see the word null is used a lot less frequently Even though we still have a sparse space and that's really because Big chunks of null end up only showing up at the top level Giving you a couple of nulls and we don't even bother having a second level Of the page table for those things that are null and so we save a whole bunch of space in the page All right So for instance, here's a an address in the heap We take the first couple of bits off that gives us something in the top level We take a second set of bits that gives us something a second level and eventually that says that this page in the heap Translates to part of the virtual address space translates to this part in the physical space All right Okay, i'm going to pause. Is there anybody here with questions? I Realize it. Um, I don't know am I talking uh too fast or too slow It's kind of a little hard for me to figure thing uh feedback here when I've got live folks on the line here, so How we're doing okay Good, okay I realize that asking questions is a little tricky Um Can I sign your hat? Sure Uh, I'm still a little confused about copy on write. Yeah Okay, we can talk about that. So um, and the question of is this faster than single level paging or just more memory efficient So it certainly isn't faster because we're doing more lookups. So we're going to have to do some work to keep it fast As far as the copy on write, let me just give that again. That's good question. So if you think about it Uh, when we copy The address space for fork, we're not copying the physical pages what we're copying is pointers To the pages we're copying the page tables. In fact, we even could just copy the top level parts of the page table We're marking things as uh read only Okay, and then if one process or the other goes to do a write at that point what happens is Um, that write ends up causing a fault. So if you imagine that the original process parent process had things marked writeable You fork now everything is readable. So now we're not even the parent can write their pages Um, either the parent or the child tries to do a write and at that point We notice there's an attempt to change something and we say, oh, we can no longer share the same underlying Physical pages what we're going to do is we're going to take that page that just got written to Copy it into two physical pages Put one of them into the parent and one into the child And now they both can write because we'll mark them as writeable And neither of them knows the difference because neither of them will see the rights of the other ones So we give it the illusion That we copied everything without The actuality of copying everything So Why wouldn't we want to share the change the physical changes? Uh, so i'm not entirely sure I understand so if oh if um process if the parent process writes Uh, something we want to make sure the child Doesn't see those rights because that's part of what fork means fork gives you a brand new process with completely different Uh address space So the model is that writes in the parent don't show up in the in the child or vice versa that answer that question Right so because remember the the fork is this pretty Um, I know it's a little strange the first time you see it, but the semantics literally are that um, you Are getting a full copy and uh, not seeing any changes that your parent or a child have put in there. Okay so Um, how do you know the difference between whether a page has been sent to disk or simply doesn't exist? Ah So good question. So what you know is um that The bit in physical uh, the physical bit says the page is not in memory What that means is that the uh memory management unit fails when it tries to translate. It's up to the kernel to figure it out Um, whether a zero means it's on disk or a zero means it's invalid or a zero means something more interesting Um, and what I did mention is when the when the present bit is zero The rest of the 32 bits could be used by the operating system to have a flag or give you something about where it is on disk Um, or there could be uh things in the kernel that basically make that indication So Okay, so the key thing here is uh, we now have uh We've now sort of gotten rid of a lot of the nulls But pretty much, um the total size of the page table is still somewhat related to the the uh amount of memory used Um, the amount of excuse me address space used in the virtual memory Not in physical memory and oftentimes especially when things are paged out Um, I guess I guess I was writing or something. That's funny when things are paged out Uh, let's see what did I do here? Other than hold on a sec somehow I Wrote on the screen somebody annotating can somebody unannotate please? Um, when we use I think somebody accidentally drew on the screen because I'm not doing it. Um So let's see so anyway So what I was going to say so the size of this page table is related to the size of virtual memory And we may need to address that because oftentimes there's a lot more virtual memory that's mapped even then there is physical memory Okay, so Um, moving on again Thank you. Uh, so We can this idea of multi-level page table is easy to generalize So for instance, we could have a tree of tables where the top level gives you a segment ID So we copy the offset We do a segment and then that gives us a page table Which we then map to give us a physical page number Um, and we could have access errors and permissions And so now this is two levels top level of which is the segment next level is page table We could do multiple levels of page table, etc. These are all possibilities. There are multiple levels of translation We could use that idea to share segments. So here's multiple levels where we have the page table base here, um, and that Basically maps to the same page table in the shared segment, which then we use The second level of the mapping to talk about a particular page too. So many options here and in fact Many permutations, okay And I don't want to dwell on this too much more But you could come up with it We could divide bits up any way we wanted and have multiple levels of page table The pros of this are we only need to allocate as many entries as we need So sparse address spaces are pretty easy memory allocation is easy because at the bottom if we have pages They're all the same size Sharing's easy because we just share segments or pages whatever cons are The size is still a little big. We still have a pointer for actual page Page tables need to be contiguous Although that magic 10 10 12 configuration the page table pieces are all the same size as the page swaps well And if we have lots of levels then things get slow really quickly so that question that was there earlier is this more faster or Just more efficient. The answer is more efficient at the moment okay, so Why don't we take a uh, do you guys want to take a brief break or should we just finish the last 20 minutes here? I guess we can I guess we can keep going if people want to because they can always step away for a little bit What do you think? plow through Okay, so So if you remember just as a refresher here We talked about dual mode operation early on in the course And I want to reiterate that a process better not be able to modify its page table So that means none of these things here none of these entries that we've got ought to be Changeable by the process because that would defeat the whole point okay, and so We have this idea of dual mode Operation in which the hardware gives us at least two modes one which we call kernel and the other which we call user and the modes are basically set with the bits in the control register only accessible in kernel mode and only if you take an interrupt or A fault of some sort to get you into kernel mode Okay, and so the user can't easily switch into the kernel We talked a lot about that before the kernel can obviously switch back when it's ready to So x86 again, there's Four rings which are priority Most operating systems only use two of them kernel mode is at zero and user modes at three and This the ring that's currently in use is called the current privilege level or cpl newer processors Also have hypervisor modes which are often called ring minus one But for now we're going to talk about the current privilege level which could be zero or three and How do we protect things while certain things are only allowed in kernel mode and in particular modifying cr3 Which is the base of the page table is only available in kernel mode And the segment descriptor tables, which I'll show you in a moment are only available in in kernel mode And all page tables must be mapped in Memory that can be only modified in kernel mode. So we basically have to make sure that we protect Pages And so here just so you can see it all together here is a 16 or 32 bit x86 memory model space and what you see here is Uh, the segment selector rather than coming from an actual address comes from the instruction Okay, and so this instruction Is uh for instance Telling us exactly which segment we want to be in and That might be because we're executing an instruction itself It would be the uh code segment or if we're doing certain operations like stack operations. That's implicit. Otherwise we can put this directly into Take it directly out of the instruction That segment is then used as a look up in a lookup table called the segment descriptor table and there are two of them Um a global one and a local one And then once we're done, we basically take the offset. We add the base to it That's what we talked about before and now that gives us a linear address Which now we put into a two level page table Okay, so the typical x86 Uh mode translation starts by coming up with a segment selector Which gives you a linear address and then that linear address is uh put through a two level page table All right, and so this is exactly What we talked about before except we're not wasting address bits picking the selector that kind of comes out of the instruction So Uh now um what are in the segments? So there is the least segment six segment registers The ss is for stacks cs is for code and then the ds es fs and ts are for data They reference a segment selector which actually references um A table so rather than the segment registers in the processor pointing at the actual base and and length of the Of the segment we actually have an index and and whether it's g or l and that points to a table full of Segment descriptors and uh, we won't go into this in great detail right now, but what you see is uh that This is very complicated and it's really for backward compatibility But what you get in this segment descriptor is not only what it's uh, whether you have to be kernel or user to use it But also where the segment is and the base address and the fact that these are all split up all over the places to try to keep it Compatible with 16 bit modes And so how are segments used I just want to finish that up So one set of global segments for everyone another set of local for every process In legacy applications they get used exactly like we've been talking about so you can have Chunks of memory for code and data and stacks and they're They're protected with a length Just like we've been talking about for segments. The code segments rpl is actually the current process level privilege level And modern 32 bit operating systems like linux actually has all of this functionality Through the x86, but they ignore it. So in most cases, although there are six segments the first four Other than gs and fs are all basically Set where the base is zero and the size is four and that's basically to eliminate kind of This whole level of translation pretty much does nothing if you always have a base of zero and a length of four gigabytes Then whatever comes in as the offset is the full space And that's the way linux and a lot of modern operating systems use those segment registers, which means they don't bother The only exception is gs and fs which are tend to be used like segments And are used for thread local storage and a few other things like that 64 bit in fact long mode Totally eliminates ssc sds and es in the sense that they don't do anything useful They're all like they have a zero base and no length and only fs and gs Still get worked used okay All right, and then of course For we could go further right we could do a four level page table for a 64 bit address and in this case By the way, the most of it's not all 64 bits are used for this Usually only the lower 48 mean anything um Because 64 bits is just a lot of space We'll talk more about that later, but um you get sort of nine bits at a time. These are eight bite chunks And uh when you're done that gives you a physical page number 12 bit offset, which then gives you an address into d-wrap Pretty big Um, and here it is. So uh from the architecture spec You've got a bunch of nine bit things here And of course intel calls the lowest one a table and then a directory and a directory pointer and then a Page Page mode level I believe whatever doesn't really matter too much, but these are all just uh 12 9 9 9 9 And so that's fine, I guess But we can also Cut off things. So if we uh instead of having the table We had this directory point directly to a page then we could have two megabyte pages or one gigabyte pages And so these uh options are often used for things like the kernel where you're not really Paging and there isn't a fragmentation problem. They're just a bunch of memory that you want to pin Okay, now, uh, you might say well, what about the ia64 which is an architecture You've probably never heard of because it didn't get very far. Look at all these levels. No Uh, I mean this gets pretty ridiculous pretty rapidly, right? Because we're doing all these memory lookups to try to translate and we're trying to do that with a single address What else could we do and I wanted to mention the other option which is out there called an inverted page table so an inverted page table is uh The opposite of a forward page table and in the forward page table that we've been talking about Essentially you take chunks of the address and go forward um to look at Each you know levels of the table an inverted page table is exactly like a hash table where you take the virtual page number It goes through a hashing process to give you a physical page number And uh, this hashing is actually done entirely in hardware just like the lookup was done Earlier as well. Um, there's a question here about whether page table look look up is usually pipelines It's very hard to pipeline something that goes to DRAM Because it takes a very long time to get something from DRAM and you can't go to the next link Until you get the previous one So the page table lookup if it has to go to DRAM is not going to be pipelined And so clearly we've got a we've got a performance issue. We've got to address Hopefully that helped with that question the If we can cache parts of it, that's a little bit of a preview then we might start pipelining it pretty gets tricky So what does this do for us? What it does is it says that we look up this page number, which might be uh, you know 52 bits Uh into the hash table to get us the physical Page number and what's good about this is This says that the size of the page table is now more related to the amount of physical memory We're using rather than the amount of virtual address space so back here It doesn't matter the fact that it's multiple levels and we have less nulls than we might otherwise This is still a huge page table And it's very slow and the size of that page table is related to all of this virtual address space Even when we only have a you know a much smaller amount of DRAM mapping this one on the other hand has something that's of order of the size of the DRAM not of the virtual address space and uh There were several processors that use this some of them still in use and places The cons are this is a very complicated thing to do directly in hardware because you build a hash table You have to do uh tracing down the hash links and so on all entirely in hardware Okay So uh total size of the page table here is more like the number of page used in physical memory rather than in virtual space So here's a combination of options here So we could do simple segmentation Which is very fast context switching and it's very simple to imagine But we get a lot of external fragmentation Single level paging we don't get external fragmentation, but we have a large table size Multi-level tables let us start Wasting less and less memory and then an inverted page table Basically has a table size that's closer to the size of physical memory rather than virtual memory Which is where the pages uh the page segmentation and two-level pages come into play so um, I'd like to So um, there's plenty of questions. I'm sure about some of these options here I just wanted to talk to you through some of the more common ones to give you A context in which to look through those slides and sort of get a better idea and I think asking during Discussion sessions a little bit more about some of these translation options certainly makes sense for office hours We have a couple of critical questions now that I think we we need to answer Which is you know, we're talking about processor takes a virtual address and gets you to translate to memory And uh, how do you translate this fast enough? I mean, uh, and you think about it And any of the you know, those of you that remember from 61c We got caching and so loads and stores are really fast if we have a cache that works But now suddenly we're talking about going through you know four to five levels of memory lookup before we even have a physical address to possibly put in the cache And so this just sounds Awful like we went way backwards in time in terms of performance and we got to do something about it There's another question, which is interesting as well as what to do if the translation fails um Get a page fault and we start doing interesting things like copy and write and so on but We'll say that for next time but uh You know, how is this translation accomplished and um, what is the mm you need to do to translate an address Well in the case of a first level page table, you read the page table entry from memory Check the valid bits you merge the address with the uh, with the offset and then you go and look at d ram Um, and then potentially set the access bit in the page table entry in the dirty bit if it's a right Two level page table you read and check first level reading check and update pte look at second level then get to the third level You can start to see that there's many d ram lookups involved here Uh, if it's an n level page table even more And so, you know, the mm you is doing what we would call tree traversal For the page table where the page table think of it as a tree. We're talking forward page tables It's like a tree of lookups and we're looking them up Uh in hardware by doing d ram accesses and how can we possibly make this go fast?  We got to do something because this sounds awful like we just destroyed the performance of our processor And so what are we going to do? Well, if you remember from 61c, we have a memory hierarchy And the trick of this memory hierarchy is we have very fast Things at the left side and very big things at the right side and what we want to do Is through cleverness and caching we'd like to have something That has the speed or appears to have the speed of things on the left But the size of things on the right and what is that that's called caching and in the case of Looking things up a dress translation needs to be at the speed Of registers and faster than the l1 cache because otherwise we get no advantage out of the cache And it's uh page table is down here in memory. And so there's some Mismatch here that we got to figure out. Okay, and Where is this mmu? So the mmu is typically on the same chip As the processor core and it's in front of the cache And here maybe we have a memory bus that gets to our physical DRAM The processor reads virtual addresses And uh, they send sends them to the memory system and it goes through the mmu to the cache And sometime later the memory system figures out what data you're actually supposed to get and brings it back and So what is that mmu doing to make this work? And um, you know on every reference instruction fetches loads stores. We have to read many levels of the page table Maybe have page faults on some of those levels if some of them are up on disk And then we got to go through caches to memory and we want to make this fast enough So that a load that's cached in the cache Actually looks like it's faster than coming out of memory. Okay, and so that seems like it's complicated And so if you remember from 61c, we have this idea of the cache Which is a repository for copies that can be accessed more quickly than the original And we're going to make the frequent case fast and the infrequent case as less dominant as possible And um, you know, what's interesting about Operating systems. I like to joke that operating systems are all about caches Um, so everything's a cache So you can cache memory locations and address translations and pages and file blocks and file names and network routes all of these things We can cache and uh, we will do that in the rest of the term And of course, this is only good So why is this a good caching picture because this person's clearly caching everything on their desk? This kind of looks like my desk And the idea hopefully is that things that you're using frequently are on the desk and things that you're not or somewhere off in a file cabinet And we want the frequent case frequent enough And the infrequent case not very expensive and there's a measure called the average memory access time For you all to remember which is hit rate times hit time Plus miss rate times miss time Is at average access time and we want that to be as low as possible And the hit rate is the time basically If everything were on the cache, that's how fast things are and the miss rate excuse me the hit time is uh Basically the time if everything were on your desk say and the miss time is how long if you had to go fetch it And we'd like it to be the case that most of the time we get something that's close to the hit time rather than close to the miss time And uh, so if you remember in 61c We had a picture probably like this where the processor might have an access time to dram of 100 nanoseconds Uh, but if we could put a cache that would say one nanosecond in the middle Maybe we could cache things we are using frequently in the s-ramp And so how do we compute average memory access time? So this is a blast from the past It's uh, the hit rate times a hit time plus the miss rate times a miss time Where the sum of hit rate and miss rate better be equal to one. Otherwise, we have some really weird probability going on And uh, the hit rate here for instance, if it were 90 percent Of this cached idea then the average memory access time would be well, it's a 0.9 times one so the the Hit rate is one nanosecond. So 90 percent of the time we get that and another 10 percent of the time We have to go all the way out to d-ramp and then we have to um Loaded from the cache and so this miss time is a combination of going to d-ramp and pulling out out of the cache And so on this hit rate of 90 percent we get 11.1 nanoseconds If we are much better and get a hit rate of 99 percent Then what you see here is well, this is pretty good, right? Because 99 percent of the time we get that one nanosecond that only one percent of the time we get a 101 And now our average access time is dropped to two nanoseconds. So look at the difference in that, right? That's a that's a factor of five or better difference in performance just by having a higher hit rate And so this is of course why caching works And by the way, this miss time as I mentioned includes the hit time and the miss penalty When the miss penalty is the time to go down to d-ramp And of course another reason that this is coming up Uh Is if you remember if you look at this, right? We've got look up something in a register and then start looking up many levels of page table and then look up Take the physical page and then look it up to memory and you can't afford to translate on every access So wouldn't it be great if we could cash? Um, and what if we're using caching to make memory faster, but we still have to go to d-ramp to do all these translations All right, so the solution is caching. We'll talk a lot about this next time But um, we're going to have a translation cache called a tlb It's the translation look aside buffer and you might ask why is it called something so strange why isn't it called the uh, the look up cache or the The translation cache and the reason is that um, well it was actually invented before caches But um, the other way you could say this is that ibm Invented it and they can call it anything they want They could have called it fred I suppose and then we would be talking about we're going to make this fast by invoking fred But nonetheless, it's the tlb and why is caching going to help it's going to be about locality and We'll talk about this later Next time but basically the idea is that we have temporal locality, which is if we access something we tend to access it again and spatial locality where if we access something we tend to access something close and that's going to be how we speed this up and so Um, I'm going to end now because we're over our time But next time we'll talk about fred or tlb as we call it but in conclusion. We talked a lot about page table Right now. We talked about how memories is divided into fixed size chunks And uh, this basically gives us the ability to avoid external fragmentation We talked about how the virtual page number from the virtual address goes through the page table However big it is and eventually mapped to a physical page number And then the offset from the virtual address is just copied to the physical address and large page tables Can actually be placed into virtual memory and paged out, which is a kind of a cool thing that you probably didn't realize We talked about multi-level page tables virtual addresses in this case mapped to a series of tables And it gives a much better use of space for sparse Addresses and by the way, all the addresses you're ever going to use are sparse Especially in 64 bit where they're very large And then we also talked about an alternative Used by a few processors called an inverted page table Which is using a hash table for translation entries And now we're starting to talk about principles of locality and caching we'll pick that up next time and Try to remind you a little bit of your 61c on caching and talk about why that works well for TLB so Hopefully this worked out well enough for everybody. This is our First example of a stream and what I will do it again Yes, we'll definitely do this again on thursday because this is how we're going I guess until they Let us go back to reality But we will be posting this as soon as I can and i'm also going to have it labeled With closed captioning for those of you that wanted to do it. So thanks everybody Talk to you soon. See you on thursday and remember You should start re attending discussion sessions virtually and we'll hopefully we'll post all the information for you By friday talk to you later. Everybody's doing virtual office hours by the way too. So ciao and have a good evening. Bye