Let's get started Wednesday morning. You guys have how many hours until assignment one is due? Thanks for coming to class. What's that? Somebody is counting very carefully. I think it's less, more than 36, actually, 39. So how many people are making good progress on assignment one? Raise your hand if you feel good. OK, good. That's good. If you didn't raise your hand, then come start making progress on assignment one. All right, so today we're going to, on Monday, we talked a little bit about some of the problems. So we're still kind of getting into our unit on memory management. We talked a little bit about the problems with doing things in kind of a dumb way. And I may have actually spent too much time belaboring the point that if we do things in a stupid way, then that causes a lot of problems, which maybe everybody didn't need as much convincing of as I offered. So today we're going to kind of move on and decide that we're going to try to be a little bit smarter. And what we're going to do today is we're going to talk about how the operating system extends the memory abstraction. So the idea of memory as something that has an interface and something that we can assign kind of more properties to in a way that's richer and allows us to do some really interesting things. So essentially, this comes down to taking the idea of memory as having an address and an interface and extending that in ways that make this into a very, very powerful abstraction that we can do a lot of neat things. So today, essentially, we're going to be talking about what are called virtual addresses. But what I really want to convince you is that there's a lot of different parts of the system that look like memory. There are kind of memory-like things that aren't actually memory, don't actually have to be memory. And the virtual addressing and this extra level of indirection that we're introducing is what allows to do this. And this is always the thing that I forget to do. Six things to get ready and that is the one I forget. So announcements, not too much. You guys know, assignment one is due tomorrow night at midnight. We are also, it's not up here, but we're very close to finishing the assignment zero grading. So those assignments, you'll have your scores back for that pretty soon. All the code reading questions are graded at this point. We just need to figure out how to grade some of the script output that you guys submitted. So once we're done with that, we'll put that up and we will release all of the correct answers. So my intention is to have it be, you can go back to assignments here and see sort of side by side your answer and the correct answer and then some feedback in terms of the answer you gave and how it was graded. And I pushed some additional stuff onto the assignment one page yesterday, specifically details about testing. So I haven't given you the scripts that we're going to run, but I've definitely given you enough information that you could reproduce those scripts yourself. And particularly when it comes to the synchronization primitives and problems, one of the things that we're going to be doing is writing code that tries to tickle corner cases. So you should think about your code, especially the problems, but also the primitives, you know, reader-writer locks, for example. What will we be looking for? Well, one thing we'll be looking for is that you don't starve readers or writers. Can you guys design a test that allows you to determine whether or not you're doing that? I think you can. And we will. So you need to make sure that you are kind of writing these tests that push your code in the same way that we are. And if you have questions about that, ask. But I think there's a lot of information on the website now. And to some degree, what we're trying to do is to get you to think a little bit more defensively about how to write these tests yourself, because this is a really important part of good programming practice. Any questions on assignment one or the assignment? All right. So again, on Monday we introduced this idea of address spaces as an abstraction. We looked at what would happen if we just allocated physical memory directly. And we thought that that caused a lot of problems. And so we said, OK, what if we could give each process this really nice uniform view of its own memory? And then we kind of decided that that would be great and also hard. So today we're going to talk about the hard part. But does anyone have any questions about the address space, abstraction, problems with physical allocation, any of the stuff that we talked about on Monday? Questions? Doubts. Confusion. General irritation that might be laboring of things that might have seemed obvious. Wow, OK, you guys are really forgiving. OK, so let's review. So let's talk about our requirements. We had four requirements in order to do memory multiplexing. Remember the operating system? Responsible for multiplexing. That's why we have kernel privilege. And there were four things we had to do with memory. So we had to be able to grant memory. What does this mean? Let's start in the back today. What does it mean to be able to grant memory? You're looking around like I'm not calling any of you, but I am. Grant memory. What does that mean? What's that? Don't remember. Let's slide down the row. Can you help them out? What does it sound like? Grant memory. Give it out. I need to be able to distribute it. When an application needs it, when a process needs it, I want to be able to hand it out. So that's important. And there's two cases. We'll talk about those in a little bit more detail today. One case is when an application starts up, when a process starts up, there's some memory that it needs right away to hold code and some of its static variables. And then over time, I also might need to allocate more memory as the process continues and things happen. So two different ways I want to allocate memory, statically and dynamically. So that's one of my goals. OK, let's go this direction. What about enforce? What does it mean to be able to enforce multiplex? So it's ownership. So when I give memory out to a process in general, I'm giving it to that process. So enforcement means what? Yeah, and more importantly, make sure that the wrong person cannot access it. So remember, one of the colonel's jobs is protecting processes from each other. When I give memory to a process, that memory is private. And I want to be able to enforce these memory allocations. And also, I want to be able to do so efficiently. The speed at which I can do this really becomes an issue. Because as we talked about, I could check every memory reference that a process makes. But that would make your machine so slow that you would feel like you were living in 1965. We don't want that. All right, reclaim. This was a little bit different. We hadn't talked about reclamation before. So let me go back to the back here, John. Reclamation. What does it mean to be able to reclaim memory? So it means that when I'm finished with memory, I want to be able to destroy it. That's one thing. That's kind of revocation. But also, when memory is not in use, I should be able to take the memory away from a process seamlessly allocated to another process without destroying the contents. And a lot of what we're going to talk about for next week is essentially mechanisms for doing that. Last thing, revocation, all the way in the back. Yes. Oh, man, I gave you the answer. I haven't got a twitchy finger. Stop a process of using memory. So if I'm taking away a memory from a process, if it's stopped using it, I should be able to make sure that it doesn't touch that memory again. We talked a little bit about fragmentation. So what's our definition of fragmentation? What's a good working definition of fragmentation? What about over here? Definition of fragmentation. Anybody remember? Right. When a request for resource, and in particular memory, because we're talking about memory, when a request for memory fails, despite the fact that there's enough unused memory for me to satisfy that request. Now remember, there are two places that unused memory could be hiding. And this distinguishes between different types of fragmentation. So what is one? So we call this internal and external. So in internal fragmentation, where is the unused memory hiding? Garen? Internal fragmentation? Yeah. Between. OK, so between is external. Internal. Want to help them out? Oh, yeah. Right. So in internal fragmentation, the unused memory is inside of the allocations that I've already handed out. So a process asked for a big chunk of memory, and it turns out it's only using half of it. So this is internal fragmentation. Now, Garen, we'll come back. External fragmentation. So what is external fragmentation? You answered this. Between two allocations. Let's be more careful here. Between existing allocations. So especially with memory, when I need contiguous blocks potentially, I may have to allocate a contiguous 1k block.  And they may add up to like 10k. But I can't satisfy the allocation. All right, so we came up with this idea of an address space. And what is an address space intended to do? Provides every process with a what view of memory? Uniform, what else? Identical, right? I want every process to be able to look at memory the exact same way. What else? And what do I want to make it appear? I think someone said uniform. That's one good thing. What else? Plentiful, I want it to look like there's a lot of memory, maybe even more memory than the system actually has. What else? Contiguous, right? I want to make it look, we talked about, if I just handed out physical memory, I would have all these sprinkling of discontinuous allocations that would make it very difficult for processes to use that memory. So I'd like it to look contiguous, despite the fact it might not be contiguous. What else? I think made it uniform. We got that. Private, right? I want it to, unless a process explicitly chooses to share memory with another process, that memory is by default not shared. And it shouldn't be shared, right? By accident, okay? Let's see. One last little question, because I always think this is cool. This is one of those things, when I learned this in this class, I was like, cool, I learned something, right? Why don't I load the code for a process into the very, very, very bottom, i.e. 0x0 virtual address? Why don't I use that virtual address, Ben? To catch no pointer exceptions, right? This is such a clever thing, right? I mean, I can help programmers out by leaving a big chunk of memory at the bottom of the address space empty, right? And there's no penalty to that, as we'll see, in any way. But it allows, it makes sure that any dereferencing of null will land in an area of virtual memory that the process is not allowed to use, right? And that'll cause a fault, Isaac. What's that? Okay, fair enough, right? But in general, that's what people end up doing, right? So, okay, so, okay, this is actually, this is, okay, this is a really good point, right? It, and we'll come back to this because it's interesting, right? So there's two, there's two dereferencing problems people usually have, right? It's rare that a programmer sets something to null and then dereferences, right? What usually happens, right? Well, I usually forget to initialize the pointer at all, right? So I allocate a pointer to something and then I forget to initialize it. Now, we'll talk about this next week. What happens is when the operating system gives a process a new page, that page is zeroed, right? So the page is essentially full of zeros. The memory contents are quote unquote blank. And so that's what causes this, right? Because if you try to dereference an uninitialized pointer, it should point it to the bottom here, right? There's no requirement that to be null, you could fill it with random data or with a pattern, right? Like I think our kernel at times for testing uses dead beef, right? Because dead beef is something that you can spell with hexadecimal characters, right? Not for any other really good reason, okay? All right, so any other questions about address spaces before we forget into a little bit about virtual addresses? Any other questions, doubts? Okay, you guys did well on this, so I think we're ready to go on. Okay, so we have this nice abstraction, right? And now the challenge is how do we make it work, okay? And clearly, as we saw, right? Direct access to memory is in physical addresses is off the table, right? So what I've done is implementing address spaces requires me to break this direct connection between an address and where it lands, right? Okay? And we call, in operating systems, we frequently refer to this as introducing a level of indirection, right? So a level of indirection means that rather than letting you access something directly, I set up a pointer to something else that maps through that to get to the thing that you want, right? Now, we've already seen this one other place in this class earlier this semester. Does anybody remember where I set up a level of indirection so that I could do something that I wanted to do? Anybody over on this side of the room? File? What's that? Files? Files, right? It's specifically what? File handles. File handles, right? So exactly, we talked before about how I don't give processes direct access to files, I give them these things called file handles, right? Which, or I give them pointers to file handles, actually references to file handles, which then themselves point to files, right? And the reason I did this was partly because I wanted to be able to share these objects between processes after fork to enable IPC, right? But this is another example of this kind of design pattern, right, where I add another level of indirection here, right? And doing this is a really powerful idea, right? Because forcing the kernel to get involved, or allowing the kernel to get involved when processes access objects gives us a lot of control, right, and allows us to do some really, really clever things. So I mean, I've always really just loved this, you know, the virtual address idea, and I don't really know the history of it, so I made up my own pretend history, right? And in my pretend history, right, it's like, I don't know, somewhere I'm in the 1950s or something, and there's like a, you know, a software engineer and a hardware engineer out at a bar together, you know, and they've had a round of drinks, and they're kind of chatting, and they're like, yeah, man, they're talking about all these problems with memory accesses, and this is also terrible. And the software guy, because the software guy doesn't really know much about hardware, right? He's just like a scissor and tacker, of course it's not true, because at that time you would have known everything about hardware if you were a computer programmer. But anyway, let's pretend. And so he's like, you know, it would be great if we could just introduce another level of indirection, right? It would be great if we could just translate all the addresses, right? And he's thinking this is crazy, right? And the hardware guy kind of like starts to giggle, right? Like the way that you would giggle if somebody said something nuts to you, right? But also the way that you would giggle maybe if you realized that this was actually possible, right? And then once you do this, there's just so much cool stuff you can do, right? Once you make this one little change, right? Once you have this one level of indirection, there's so much power here, right? And just the tricks that you can play with memory start to open up, right? So again, I mean, one simple thing I can do that I wanted to do, right? We talked about it as a requirement for multiplexing was that I can revoke access to an object, right? And I do that simply by refusing to continue to translate that reference, right? You had a reference to the object one moment and then the kernel decided that you couldn't use that object anymore and it just, it said, no, I'm not gonna allow you to use that object anymore. So when you try to get to it, I'm just gonna stop that, right?  So that's one thing I can do, okay? I can also allow reference objects to be shared. This is what we did with file handles, right? I allowed file handles to be shared between processes for the purposes of doing IPC, right? And object sharing through references can happen either within the same process or within multiple processes, right? So there's actually functionality on most systems as I described for explicitly sharing memory between processes, right? And this virtual address abstraction we're gonna talk about today allows me to do that, right? That's what allows me to do that, okay? The other thing I can do is I can move stuff, right? I can move things around, right? And we'll talk about why I might want to do this, right?  I can move it here, I can move it there, you know? I can do that. I have that power now, right? Because your reference doesn't change but where the data is might change, right? And that's really cool, okay? And then I can alter the object too. I might change the object in some interesting way while preserving the interface that you're using to access it, right? You might not know that that object is different. The data is different. The data might be stored somewhere else in particular, which is what we're gonna talk a little bit about today, right? So this, I just wanna, you know, give you guys some sense of the beauty of this technique because this is a really, really powerful tool that the kernel is gonna use. All right, so let's talk a little bit about what it means for memory to even have an interface, right? So most of the time, as programmers, we don't think of memory having an interface, right? You know, it doesn't have objects or, you know, it's not implemented in software or something like this, but it has an interface, right?   So what is the memory interface? It's very simple, right? Ben? Ooh, no, no, no, you've gone to the software level. I'm talking about even lower than that, even lower than that, it's a hardware level here. Right, I can load, load, right? Load and store, right? And if you guys look, I meant to get this into the slides and I didn't have a chance today, but if you go look at the MIPS architecture description, there's a bunch of instructions in the MIPS instruction set that do this, right? All these kind of like slightly different versions of load and store, right? Load into a register, load 32 bits into a register, load 60 bits into a register, blah, blah, blah, load into the upper part of register, lower part of register, whatever. And this is all variations on this theme, right? Load and store, okay? And what we're gonna talk about today is how I can, because I start to break this direct connection to physical memory, I can start using this interface to point to things that aren't memory at all, right? And figure out how to do this, right? But this is the memory interface, and it's interesting to think about having an interface in this one, right? Any questions about stuff that we've covered so far? All right, so again, we talked about the address-based abstraction requiring that I break this connection, right? And in order to make sure that we understand how this connection gets severed, we're gonna start using more precise terminology, okay? So rather than talking about a memory address, we're gonna talk about virtual addresses and physical addresses, okay? And those are the terms we'll use in this class so that we can know what we're talking about, right? A physical address is easy, right? When we talk about physical addresses, we're talking about memory, actual memory, you know, like, you know, whatever it is, you know, DDR3 or whatever, like a chip of memory inside your computer somewhere, right? That's what a physical address is, right? A virtual address, as I just pointed out, points to something that acts like memory, right? Meaning that loads and stores to that address that are executed by the process, you know, act as I would expect, right? A load, load some data and a store stores data, right? Now, virtual addresses starts to introduce more semantics about exactly how load and stores behave, right, so that's one of the things we can do. We took an interface that was very, very nicely specified and now we're gonna start introducing some new properties to it, which we'll talk about, right? And these are the semantics of virtual addresses, right? So now, virtual addresses can encapsulate location, right? We'll talk about virtual addresses that point to memory. Other virtual addresses might point directly to a file, right? So now, depending on the virtual address I'm using, that data might end up somewhere else, right? I could even have a virtual address that points to memory on another machine, right? So now, rather than just mapping to physical addresses, I can point those virtual addresses to a wider number of places, right? Permanence, this is another thing, right? How permanent do I expect the data that I write to that virtual address to be, right? Physical memory, we expect to be transient. When the machine shuts down or the process ends, that contents any data that's stored in physical memory is lost or things that look like physical memory, right? Virtual memory that looks like physical memory. But virtual memory that points to other places may have different properties. And finally, protection, right? So virtual memory also allows us to assign protection to addresses, right? None of, again, none of this stuff we can do with just physical addresses, not possible, right? All of this is made possible by this level of indirection and having the kernel be involved, right? So let's talk a little bit about location. So when I access a virtual address, right? I do a loader store. So that address could be in memory, right? This is the simplest case and this is what we'll talk a lot about, right? Translating virtual addresses to physical addresses that are in memory. But because the kernel is involved in this translation, what can I do? Does anybody know? What can I do with the data that the process thinks is in memory? What if I run out of memory and the process hasn't used some of its memory for a while? Where would I want to move it? I can move it to disk, right? So the data that was in this memory might have been moved behind the processes back to disk, right? So this is, again, something that is possible because of this level of indirection, right? Now the data referred to by virtual address might also be on disk, right? And again, what I'm talking about here are the processes' expectations about where the memory is, right? And what it acts like, right? So I can have virtual addresses that the process has set up to point to a file on disk. But what if I want to improve the performance of those virtual addresses? What might I do? I might be caching it in memory, right? So the process might do a right to something that it's set up to point to a disk and I actually might only do that right into a memory location, right? The problem is I need to make sure that that data makes it to the disk eventually, right? Because when the process used that virtual address, it expected the data to get to that file, right? And then finally, again, I mean, I can, you know, I can extend these semantics as much as I want, right? I could have virtual addresses that point to memory on some other machine, right? And doing loads in stores means figuring out, you know, how to access that machine, how to do the right, et cetera, et cetera. I might do caching the same way I do on disk, et cetera, right? So again, I just have these much, much, much more powerful semantics now that I've introduced this one level of indirection, right? Last thing that is worth talking about briefly is that memory can also be used to simplify access to hardware devices. So frequently what the system will do is it will set up hardware devices so they look like memory. What does that mean? It means the ports on the hardware device that the operating system uses to communicate with it are memory addresses. So, you know, to send the value of five to the port on the disk that tells it which block I want to read, I write the value five to a particular memory address. And then the memory management unit, which we'll talk about a little bit later, routes that properly and makes sure that that byte ends up at that port, right? I also can set up hardware so that hardware can write directly to memory, right? So, you know, I'll always hardware to do that means that rather than having to read data out of it byte by byte, I can give it a big chunk of memory that it can write data to. And then after I ask it to do a read, that's where the data ends up, right? So I expect it to be at that part of memory once the data, once the read completes, right? So this is just another example of the flexibility that I get out of this, right? So let's talk about about the permanence sort of properties that get, end up getting mixed up with this, right? Because this is another thing about how virtual addresses are created and the permanence properties I expect, right? So again, when I'm writing to physical memory, I expect this to be transient, right? It should last as long as the process lasts, right? Might be until the machine shuts down or crashes, it might be until the process exits, right? At that point, memory is clear. That's the implicit agreement that the operating system makes with the process. That we're gonna preserve the contents of your memory until you call exit, right? What about if I write stuff to disk? What do I expect there? Permanence, right? I expect it to store values permanently and the operating system better make sure that happens even if it's cached in memory, temporarily. So what about device ports? This is an interesting question. What sort of permanence do I expect with a device? What's that? If it's a removable device, I'll use transient. Well, let's say I'm communicating with the device via port and I write some data to that port, right? Do I expect that data to necessarily be there if I try to read from that port? No, because the device can change its own values, right? I mean, all these other things are storage, right? There's some kind of storage, transient storage of physical memory, disk storage. Devices aren't storage, right? Devices have a mind of their own, right? So I might write the value five and the next time I read it, I might contend, right? Because the device has changed the value on its own, right? So it's just a little bit different, right? Okay, so let's talk about, again, the last thing I can do and another really important thing is that I can do, yeah, yeah. No, no, no, I just mean, when I write a byte of data, what are the expectations about how long that data lives, right? And again, when I write the disk in general on a system like this, I expect it to live until I do something explicit to get rid of it, right? Or till somebody else overwrites the file, right? Whereas with memory, I don't expect the contents of memory to last after I call X, right? So again, another thing I can do with these addresses, I can imbue them with protection, right? So I can say, for example, there are some addresses on the machine that can only be accessed by the kernel or more specifically, will only be allowed to be used while in kernel mode, right? And this is a fairly simple way for the kernel to use a part of memory for itself, right? And to make sure that no process can ever access, right? Another way of doing that is just to make sure that none of the virtual addresses I give out to processes ever translate to a physical address that the kernel is using private, right? But on some systems that they do have this idea of protection, right? And I can also assign any of these protection bits or protection capabilities to an address, right? So for example, read write means a process can load and store from that address, right? And I might take away one or, well, I wouldn't take away both because then it would be difficult to use, right? But I might take away write permissions, right? And have something be read only or in certain cases I might take away repermissions and have something be write them, right? And then execute, right? So execute usually means that a process is allowed to load and execute instructions from this area of memory, from this virtual address. Specifically, on the program counter of my program is allowed to fetch instructions from that area, right? Why would I want to protect or have special permissions for executable parts of the code? Why might I wanna do that? Uh-oh. So I certainly, yeah, I certainly might want, I certainly might want, I'm not sure why that happened. I definitely might want to mark the executable section as read only, right? Now, there used to be this, people used to write what's called self-modifying code. Has anyone ever written any self-modifying code like for a program? Okay, Carl, yeah, see? So I think that this is kind of not in vogue anymore, I think people have decided it's hard enough to write code that doesn't modify itself. But there are ways that you can write these weird things with self-modifying code that actually overwrites itself as it's running, right? But typically we don't, a lot of machines don't support that anymore because they do mark the executable section read only. But what about other parts of the code or data in the process? Why might I not want them to have execute permissions? Yeah? We made good choices. Right, so I mean, there's a portion of the address space that I expect to execute instructions out of, right? And then maybe if I'm trying to launch some sort of buffer overflow attack, I might ask the system to be executing instructions out of an area that was actually set up as stack, right? And the system might want to say, no, you know, like, that's not okay, right? So I might want to explicitly say it's okay to execute out of this area and not out of this area because if somebody tries to execute code out of this area, something is happening that I didn't expect and I don't understand and I don't want to have, right? So that's a good example, right? All right, so now some of this, you know, we've talked about in the past. So this is gonna be a little bit of review about how we set up virtual addresses in the first place. I'm gonna move through this a little bit more quickly because you guys have seen some of this before and we talked about process life cycle, right? So you remember that exec is in charge of establishing a new address space for a process that is sort of reinventing itself, right? That is changing into some other process, right? It does that by looking in the CLF file, the executable and linkable format and that file contains among other things a blueprint for how to set up the address space, right? It says this portion of the file contains code that is supposed to be loaded at virtual address here, right? Essentially, when you guys do assignment two, you're not gonna have to write this, the code is already there, but you will receive calls from the ELF loader that essentially says here's a pointer to this box in a file, here's the length and here's where to put it, right? It's like you're setting up a house and the movers are telling you this box goes in the kitchen, right? The ELF format has very, very specific instructions saying these 10K are loaded at virtual address OX 10,000 or whatever, right? And your job is to set up the address space correctly so that that happens, right? And mainly what exec does is it sets up virtual addresses that point to other physical addresses, right? Virtual addresses that point to memory, right? Most of the things we're gonna talk about in this class, we talk about virtual addresses, our virtual address is the point to memory, right? But I think it's important to recognize that that's not required and that because I do translation, I can do other things, right? All right, and again, so I usually have a code section that's marked read-only and probably it's execute permissions and then data that I would mark as read-write but not executable, right? This is what comes out of the ELF file, right? This is what's there when the program starts. And so the heap is also established at runtime, right? And the heap starts normally from the lowest address that isn't used by code and data, right? That's where the heap starts and then as the process runs, the heap is allowed to grow through a system call that we're gonna talk about in about two slots, right? Meaning that the process is allowed to allocate more virtual memory that points to physical memory, right? And again, stack space for the first slide is set up. So this is what ELF kind of gets you going at runtime. This is what gets started off the ground, right? And let's go back. So this is our PMAP. This was PMAP output. We used this a couple of weeks ago to look at the address space of a process. And what you can see here is, you know, again, I mean, these are virtual addresses. This is a portion that Bash is using for its code, right? And you can see that these get loaded into, you know, pretty low in the address space, not super low, but pretty low. You know what, OX, you know, 80 million, 8 million, right? This is, this area right here, you'll see it's read write. This is going, and this is read execute. So this is code, right? This is read only. This is probably static variables. And then this stuff is read write, right? So this is probably global variables that Bash uses while it's running, right? That gets set up where space is allocated at runtime, right? All right, and then we saw we had some loadable libraries that got loaded much higher and then the stack is way up here, right? So this is on Linux. So the other question is, how large is the virtual address space on Linux? Can you guys tell by looking at this? So it probably starts from OX0, right? Despite the fact that the code is loaded pretty high, right? And the stack, again, we talked about last time is being way up at the top and that's at BF something, right? So where do you think this at, where do you think the virtual address space for process on Linux on this particular system ends? What address? What's that? No, no, B, B, remember hex. So it's probably C, OXC0s, right? So there's the, and I think that's what? That's, oh man, my hex is so bad. That's like a half a gig left at the top, right? That that's actually not part of the virtual address space of the process, right? So I've got a 32-bit address I can access at most four gigs, but there's half a gig that's chopped off here that the process is not potentially not allowed to use, right? As the stack starts at BFFFF, right? And we'll talk more about how different towards the end of class, we're gonna go through the MIPS memory model. And so you'll see a picture of how a particular architecture establishes guidelines about how these addresses get mapped, right? All right, so then we remember fork, right? What does fork do? Fork copies the address space, right? When a process forks, it's supposed to look identical to its parent. And fork accomplishes this by copying the address space of the calling process, right? Now the contents start out the same, but do they stay the same? No, right? Fork is supposed to establish an address space that looks identical, but where those virtual addresses point to new and distinct physical locations, right? So here, for example, if I ran this code, I don't know if this would actually print this, I don't know where this would be, right? But here I have a little program. It has a variable here that's either a global variable or maybe local to the function. It calls fork, right? If my return code is not equal to zero and the parent, right? And the parent is going to print the memory address, all right? So this is printing the address of i, right? Where i is located. And that's gonna print some value, right? Some virtual address that was set up in my address space to hold this particular int. And then if I print the int itself, it prints four, right? So what happens down here at the bottom in the child, right? What will this line print? Anybody? The address of i and what's the address of i? It's a virtual address. What's the virtual address of i after I call fork? Four? The same as what? And what is this parent? Zero x 20, yeah, hex is terrible, right? O x two zero zero one zero, right? So that should print the same virtual address, right? Again, the address spaces are identical, okay? Right after I call fork. But if I do a store to that address and then print the value, what's it gonna print? Well, it should print three, right? So the idea here is the virtual addresses are the same, but they point to memory that is supposed to look private. It is private, it has to be private, right? So when he stores to those addresses, now diverge, right? The addresses themselves, the virtual addresses are the same, right? But again, we've broken this connection with physical memory and so the kernel is required to make sure that those addresses map down to different pieces of actual physical memory, right? All right, we talked also about fork and the fact that copying all of the, because remember, if I really wanted to set up a private address space, I would have to take all the physical memory that was in use by the parent and copy it somewhere else and create new mappings for the child so that it pointed to those new locations, right? And that could potentially be really terrible. And next week when we talk a little bit about some clever memory, I'm out of water. Clever memory management tricks. We'll come back to this because there's a clever way to get around having to do this every time, right? A couple other ways that I set up virtual addresses, right? One of them is dynamically, right? So everything we've talked about up till now is static, right? It's exact, it's fork, right? It's stuff that happens essentially when a process is being created. But at runtime, I may need more memory, right? Now, Ben, back to your earlier answer. What, as a C programmer, what's the interface to memory? Malik and free, right? And what Malik and free do internally, right? Now, Malik is what's called a memory allocator, right? And it's good, Malik is designed to be good at allocating different size pieces of memory, right? But eventually, Malik, no matter how much memory it starts with, may run out of virtual addresses to allocate virtual address space. And at that point, what it will do is it will ask the kernel for more memory, right? And the system call that does this is called S-break, right? And essentially what S-break does is it says, dear kernel, I would like to be able to use a larger heap. So I would like you to move the break point, right? Or actually, another spug in the slide. It's where the process heap ends, right? So could I please have this much more virtual addresses that point to memory that I can use, right? That's what S-break does. And you guys have an implementation of Malik in your OS 161 tree in the user libraries, and you can look at it and low and behold, it calls S-break, right? So behind the scenes, when Malik runs out of memory and can't satisfy an allocation internally, it calls the kernel for help, right? It gets another page and then sets up that page to do smaller allocations, right? All right. The last thing I want to mention just briefly, because I talked about addresses that point it to files, is that this is the unique system call that sets up addresses that point to files. I'm not going to talk much more about it, but essentially what it does is it says, dear kernel, I would like this portion of this file to be mapped into my address space. And then any reads or writes to memory to the virtual addresses that I've established that point to this file will end up at that location in the file, right? And this could be a very clever way of setting up data structures and things in files, right? Rather than having to read, read, write stuff explicitly, I just take a file, I M-map all of it or a big portion of it, and then I could just use it like memory, right? Then if I establish a big array in there and write a bunch of values to it, when the process ends, the file will have that data. And if I load it again, the next process, it'll just be there, right? So this is kind of a way of making memory that looks like memory but has the permanence of a file, which can be a nice thing, right? All right, so let's look at, so every architecture establishes different guidelines for how its own addresses are laid out, right? And part of this is because, as I'm going to mention in a second, I probably should mention it beforehand, figuring out how to do this translation ends up involving both the operating system and hardware, right? So I need hardware support. I need hardware help. And because I need hardware help, unfortunately, hardware gets to set some of the rules, right? Like, the operating system can't just make up all the ground rules, right? Because the hardware has to help me, and hardware is inflexible and stubborn, right? I have to kind of take what hardware is going to give me. So every piece of hardware may define a little bit of a different way of doing this, right? So let's talk about MIPS, OK? And it turns out that this is an architecture you guys may care about because for assignment 3, this is the one that you're going to have to use, right? So the MIPS architecture that you guys are working on defines four address regions, right? So it breaks up 32-bit addresses into four distinct regions, right? So from 0x0 to 0x7, right? That is, these are addresses that are used by processes as process virtual addresses, right? So these addresses are translated by the kernel. And this means that a process can potentially address up to two gigabytes of memory. That's it, right? Only half of these addresses or of the whole 32-bit wide address can be used and mapped to memory by a process, right? So you'll notice when you start doing assignment 2 that the process stacks always start right up near 0x7f, I don't know, right? Now, remember, this architecture was developed at a time where machines had less memory, right? So now, if you took a machine with four gigabytes and you said a process could only address two of them, it would seem like a limitation. If you take a machine with 512 megabytes and say a process can only address two gigabytes of its 32-bit virtual address space, that doesn't seem like such a problem, right? It's four times larger than the amount of memory that the machine is going to have, right? So the next region, I have a slide breaking these down, right, but I just want to talk about the properties first. The next region are what are called kernel direct mapped addresses. And by direct mapped, I mean that the MMU translates these addresses, but it does so in a very, very, very simple way, right? All it does to translate this address is it subtracts 0x80 million, right? So 0x80 million maps to physical address what? Zero, right? And 0x9 accesses the last 512 megabyte bit, right? Now, having memory set up like this is actually really helpful in certain places because the hardware will always be able to translate these addresses, right? The kernel doesn't have to tell the hardware how to translate these addresses, right? They'll always work. And there's certain cases where this is helpful, right? Because I may be trying to set up a translation and I don't want to fail because there's some other translation that's not loaded, right? So this can be a nice feature. And some architectures had this idea. However, this only allows me to access the first 512 megabytes of physical memory, right? After that, the direct mapped addresses don't work, right? But this allows me to see directly the first 512 megabytes of physical memory, right? If there's less physical memory than that, then a portion of this is unused, right? And attempts to translate it will fail, right? The next region are what are called direct map. These are also direct mapped addresses, but these are uncached, right? So this region is cached and this region is uncached. And it turns out that the reason this is uncached, can anybody guess? Remember what we talked about before in terms of what different memory addresses could point to? And we said it might be a good idea, for example, to point to what certain thing to make it easier to use. What certain thing that the kernel might use? Anybody remember? Hardware devices. So this is where hardware devices live, right? This is how I, if I'm gonna use a hardware device by mapping the memory, I'm mapping it to this area. And the reason I don't cache values is because the hardware can change them, right? So here, the system can cache values because I know that the only way they can change is by the system reading and writing into those memory addresses. But here I don't do that because the hardware may change the value behind my back, right? I don't wanna fool myself. All right, finally, the very, very bottom here, these are kernel virtual addresses. So the top gigabyte of the address space are virtual addresses. They're translated the same, through the same mechanisms we're gonna talk about for translating user virtual addresses. The only difference is they are not translatable when you are running in user mode, right? So the other thing is all this part here, all the addresses that have the top bit set cannot be accessed when you are not running in kernel mode. That's just the semantics of the MIPS architecture, right? So two gigabytes of the virtual address space is set aside for the kernel's use only, right? Now for this class, these are really the only things you are going to care about, right? There have been people in the past who have decided, and you guys have a month for assignment three, so maybe some of you will try it, right? And I would certainly encourage you after you get everything else to work, right? But using the bottom section here for kernel virtual addresses has never been required for this class, right? It's a little bit tricky and it introduces some issues. So you guys will only really be playing in these areas, right? You will, assignment three is essentially figure out how to get this process of translating virtual addresses to addresses to point to memory to work, right? And in order to do that, you frequently need direct access to the memory on the machine, and in order to do that, use addresses in this range, right? So those addresses you know exactly where they're gonna map, right? Okay, and then here is a picture, right? These pictures are very helpful, right? So this is how this address space gets broken up. First half, user of virtual addresses, right? Next part, direct mapped cached, right? And I should have a picture of that just pointing right into the first 512 megabytes of physical memory, right? Then I have the kernel virtual addresses, which you may or may not care about, and then this, which you will definitely probably very, very not care about, right? All right, any questions about the MIPS virtual memory layout? This is an example, right? Different machines do this differently, right? As we saw before, Linux clearly has a larger process virtual address space, right? Because the stack's up there at BF whatever, right? On MIPS, that address, you know, that BF address is like in here somewhere, right? So basically, if you tried to run a process that was set up that way on MIPS, it would fail, right? Because we're trying to use these kernel virtual addresses to point to hardware, right? As a stack, not a good idea. All right, so the last thing I wanna point out today is that we're gonna start talking on Friday about how to do translation, right? How do I actually translate a virtual address that points to physical memory, right? What is the, you know, I've showed you all these great things I can do now that I have the ability to translate addresses, but I actually have to get it to work, right? And the thing here is that in a more, I think in a more fleshed out way than when we talked about threads. Although you remember when we talked about threads in the CPU, there were CPU features that were built specifically to give the operating system the ability to do certain things, right? I mean, if you look on the MIPS, for example, when an exception happens, it saves the cause of the exception in a special register so that the kernel can figure out what happened when it starts to run, right? That's an example of sort of hardware, software, co-evolution, right? In this area, this is even more pronounced, right? And the main reason for that is because the kernel cannot, software cannot translate addresses fast enough to get a system that would perform in a non-terrible way, okay? And so, what I do is I use hard, we're gonna talk about specific pieces of hardware that I use, but the memory management unit, you can really think about in our sort of policy and mechanism split, this is mechanism, right? This is how this actually happens, address after address after address, right? And the operating system's job is essentially to tell the MMU how, right? How do I translate addresses, right? The MMU takes care of most of the translation once the operating system has told it what to do, right? So this is another nice example of policy and mechanism split, and in this case, the mechanism and the performance of the mechanism is so important, right? Then what we've done is we've essentially baked it into specialized hardware, right? And we'll talk more about some of that hardware on Friday. So on Friday, and again, the goal here is essentially to allow the operating system to set the policy, but to involve the operating system as little as possible, right? So when the operating system gets involved, what you want it to be doing is to be making policy decisions, right? And telling the hardware something new about policies on the system, right? And as soon as the hardware knows, the hardware shouldn't ask again, right? And this is our goal here. This is the only way to get this to work, okay? So on Friday, we're gonna talk about address translation, where I talk about specific mechanisms. For again, now I've got this virtual address, and I need to figure out what physical address does it point to, right? And there's a lot of different ways to do this. So sort of historically, we'll talk about some of the simple approaches, and then we'll talk about some of the more modern, more realistic approaches, and then we'll talk specifically about more hardware features that allow this to happen really rapidly, okay? I'll see you on Friday. Good luck with assignment one.


Determine the convergence of the series where n equals one to infinity, that was missing there, the natural log of n over n. Now I wanna actually determine this convergence and I'm gonna use the integral test to help me here because what I know is that the integral test, its convergence, the integral test tells us that the convergence of the series will be comparable to the convergence of the integral one to infinity of the natural log of x over x dx, like so. And so this could actually be very helpful for us because we could try to integrate this thing on the right. But in order to use the integral test, we have to make sure our function, which here it's the natural log of x over x, we have to make sure that it is positive decreasing and continuous. Continuity is not such a big deal, right? Is it continuous? And look at the functions involved here. It involves a natural log of x, which is a continuous function. It involves just x itself, which is continuous. And we take the quotient. So as long as we are avoiding division by zero, we'll be continuous. Now we're trying to do the interval one to infinity because that's where this series starts. And so from one to infinity, we never divide by zero. So the answer is yes, this is a continuous function. Is it a positive function? Is it positive? Well, let's consider that for a moment. The number x is always positive. The natural log, well, I should say that x is positive when we're greater than one, right? What about the natural log? Well, the natural log also is positive when we're greater than one. It's negative when between zero and one. So on the interval in question, the natural log is positive, x is positive, the ratio will be positive. So again, the answer here is yes. What about decreasing? It turns out this is where things get a little bit more tricky, right? Because if we just look at the first couple terms of this thing, we could try to plug in like x equals one, x equals two, x equals three, try to get an idea. Middly, we have to start computing natural logs of two and three and such. That might not exactly be obvious. And it turns out that that only gives you the integer values for determining whether it's decreasing or not. We really should be looking at the derivative, right? Because if we take the derivative, what's that real quick? We'll do that calculation. By the quotient rule, we get low d high minus high d low. Square the bottom, here we go. Let's simplify that a little bit. The derivative of the natural log was one over x. That cancels with the x right there. So that's gonna give us a one minus the natural log of x. This sits above x squared. And to determine whether this thing is decreasing or not, we wanna figure out when is, when is the derivative less than zero? Now, the x squared here, since we're greater than zero here, we're actually greater than one. There's no way that thing could ever be zero and x squared's always gonna be positive otherwise. So if you times both sides by x squared, you're just gonna end up with the inequality one minus the natural log of x. When is that thing less than zero? Well, adding the natural log of both sides, we get one is less than the natural log of x. And then exponentiate. If we take the exponential of both sides e and e here, the natural log and exponential cancel out. The exponential function's increasing so we don't have to flip the direction here it's preserved. And we end up that e is less than x. Or if you prefer, we get that x should be greater than e. So the thing is our function is actually only decreasing when x is greater than the number e. But in particular, one is actually smaller than e. So it turns out our function is not always decreasing like the interval test requires. But one thing I wanna mention about the integral test and many other test and convergence we talk about is that, well, yeah, it has some assumptions, right? On the interval one to infinity, we wanna be positive, decreasing and continuous but really these things just have to be eventualities. It just has to be eventually, eventually continuous, eventually positive, eventually decreasing. That is to say the interval one to infinity is somewhat insignificant. We need to pick a starting place, yes, but that starting place can be much later on. Like for example, if you take the number e, the number e right here is approximately 2.7, blah, blah, blah, blah, that's actually enough. Turns out that if we take x to be greater than three, well, three is bigger than e and that will actually take care of it. That when x is greater than three, then, so that is we're on the interval three to infinity. When x is greater than three, the function will have a negative derivative which means the function's decreasing. It was positive when you're bigger than one, so that's still true for three and then it'll be continuous when you're bigger than one which is still true when you're bigger than three. So what we actually wanna do is substitute out here and we wanna use the interval for three to infinity. So that is we're gonna compare the series n equals three to infinity of the natural log of n over n and we wanna compare that, that's comparable to the integral from three to infinity of the natural log of x over x. So the integral test applies here. The integral test is gonna apply only to this improper norm, we go from three to infinity. Now that's not a big deal for us because if it's eventually decreasing, that means that these guys will be convergent but then we can also go back to one here. The starting value of three is somewhat insignificant. It has to n equal, because if the series is convergent when it starts at three it'll be convergent when it starts at one as well. And so even though the integral test doesn't apply when it starts at one, since it applies when it starts to three, then we can determine the convergence or divergence of this series for which then we can then replace that with n equals one and good to go. And that's why I emphasize the eventuality that if our function is eventually positive decreasing and continuous then we can apply the integral test. It doesn't actually have to be continuous decreasing or positive from one to infinity just eventually, right? So our function could be doing something like no, I'm positive, now I'm negative, now I'm positive, now I'm really negative, now I'm positive again, you know, something like this. And then it's like, okay, now I'm gonna get my life together and get something like this. So you might see like this sporadic behavior at the beginning of the function, but it eventually corrects itself into this positive decreasing continuous function that's sufficient to use the integral test. And you're gonna see us being kind of loose about those assumptions in the future because of this, because of this observation. Now let's consider the anti-derivative here of the natural log of x over x. My actually, I would wanna use here u substitution take u to be the natural log of x and hence du equals dx over x for which we can see that's exactly present in the situation. And so the integral then becomes just a u, a u du, and as we also change the bounds, right, you're gonna get the natural log of three on the bottom, you're gonna get infinity still on the top. Like so, find the anti-derivative of this, you're gonna get u squared over two as you go from the natural log of three to infinity and plug in a finite number doesn't have much of a problem. The issue is when you plug this infinite, you're gonna end up with a infinity squared and then everything else will just become infinite as well. So this improper integral will go off towards infinity, which means that it is divergent. So the improper integral is divergent. So then by the integral test, right, good friend, the integral test right here, this implies that the series is likewise divergent. And since this series is divergent when you start at three, we could swap that to n equals one and it'll be divergent when it goes from one to infinity as well. So the final answer is when it comes to this series in question, the original series n equals one to infinity of natural log of n over n, it'll be divergent by the integral test.


I really appreciate all of you taking the time to come here and join me this afternoon, particularly during the lunch hour. My name is Robert Schimpb, Bob. And I run product management for Oracle's infrastructure products. There's a whole variety of things, but most especially Linux and virtualization. And my personal passion right now happens to be around distributed computing infrastructure. I work with a lot of large companies around distributed computing for automotive systems or for blockchain-style financial services systems and different things of that sort. So what I wanted to do today is give you a broad overview on where things are going with distributed computing and then drive down into some Linux specifics. I will be the first to say that rather than commentary, I suspect by the end I'm going to have a lot more questions than answers about where things are going. But hopefully, this will be useful or interesting for you. At the end, if you stay all the way to the end, we are going to have a giveaway, the UBS charger. So feel free if you stay. You'll be able to pick one of those up from Dave Gilpin in the back of the room. So if you think about cloud computing, broadly speaking, I think most people tend to think of really large, complex system of records type of infrastructure moving up into the public cloud. This is things like ERP systems or supply chain systems, sales and marketing on the front office. And that's a big switch from where we were a few years ago where most people use public clouds for very tactical stuff because they weren't sure about the quality or security or whatever. But nowadays, all these large, complex system of record type of applications are moving into the public cloud, which tends to leave the question, OK, what does that mean for me as a Linux or IT ops person? It's a fact that over the next few years, over the coming years, about 50% of all the corporate data centers, the privately held corporate data centers and large companies are going to go away. And for small or mid-sized businesses, the percentage is going to be dramatically higher. So that leaves, I think, the IT ops person with a couple of choices. Either you're going to go to work for some intergalactic infrastructure provider or you're going to find something else interesting for your company to do. And I think the answer is that there's a lot of very interesting things to do. And in fact, I will make the forecast that no more than one third of the business applications out there are going to run in some giant hyperscale data center in Chicago or wherever. Two thirds of all the business applications are, in fact, going to be distributed computing types of applications that do much, much more interesting things than ERP, including peer-to-peer social and mobile networking applications, a lot of business applications for virtual reality, augmented reality, and then, of course, the whole world of real time control, internet of things, and so on. So the picture isn't as bleak as it might appear. In fact, I would argue that for those who really want to embrace it, there's a tremendous new opportunity to do a lot of interesting things. And that's what I want to kind of talk about. OK. So by way of background here, there are, roughly speaking, 20-some-odd billion devices out there on the edge of the network and intelligent devices. That's going to 80 billion by 2025. It's a very dramatic shift. And there is a lot more intelligence in people's smartphone today. That's going to only increase more and more over the coming years. And that's going to create a lot of computing capacity at the edge of the network to do really, really interesting things. But the flip side of that is that it does stress the internet or network connections, broadly speaking. Most of these types of devices are incredibly chatty. And if you allow them to simply connect up to the hyperscale clouds and do whatever they're doing, it's basically going to bring the internet to its knees over time. So rather than go with that approach, which is going to be incredibly expensive, the idea is to move to distributed applications in which we push the computing capacity, all the data, and the applications as close to the edge of the network as we can and primarily reserve those big hyperscale clouds for the classic system of record application that requires a central database to do a whole bunch of transaction processing and analytics and so on. So the way to sort of think of this is a classic three-tier architecture in which all of your big ERP and so on, OLTP-based applications, decision support, remain up in the hyperscale cloud.  You have actual intelligence in the devices, and they're communicating with each other directly. And then in the middle are smaller distributed clouds that are used for systems of engagement. So my personal view of the world is that everything is not going to go up to Amazon or to Azure or anybody else. And just all your stuff is going to sit there in one or two or three data centers. In fact, I think most IT ops people are going to be actually dealing with hundreds, if not thousands of data centers, maybe very small ones, very small clouds, but distributed globally and locally to their particular application. Now, I particularly want to talk about core infrastructure stuff, but there's a whole bunch of initiatives going on in the industry. You're probably familiar with a lot of these, but there are many, many different types of initiatives to reduce the amount of traffic on the networks in order to make this sort of thing possible. The very first and easiest, simplest one is HTTP2. As most of you may know, we've been running on 1.1 forever. That's gotten very, very old, and now in the last year or so, things have been rolling out to HTTP2, and that allows very simple things like much more data compression, a whole lot of push capabilities, and so on. That's great for reducing HTML-based traffic and so on on the networks, and it's had a real effect, and that's going to continue to roll out more and more and reduce traffic. Another example, however, are next generation 5G communications networks. So the existing 4G networks, the things you use today, and you probably have a lot of experience with this, can be extremely slow. The new 5G networks, which are now in very late testing and are gonna roll out in production over the next couple of years from different vendors, dramatically increases the amount of capacity, certainly thousands of times more bandwidth, and so on than what you would get in 4G networks today, as well as offering tremendous interesting new features for slicing up that bandwidth in different ways so that you can have all kinds of multimodal traffic over those same networks. So there's a lot of creativity that you can start to take advantage of in terms of the types of applications you can deploy based on those 5G networks. So those are two very simple examples broadly across the industry that are improving network traffic, but it's also imperative that companies start to think in terms of distributed architectures, what is the most appropriate approach for the type of application they're gonna run? And I can tell you that these are very interesting and subtle architectural questions depending on the specific applications. In some cases, a hierarchical model where you simply are pushing data out and caches further towards the edge of the network's valuable, some time a leaf and spine approach is really good if in fact the devices at the edge of the network are particularly chatty and they need to have connections to various other random devices on the network. And then finally, architectures like peer to peer that enable many, many different devices to talk directly to one another. And I could go on and on. There are multiple other strategies for distributed computing networks that ultimately can run different types of applications really, really well. By the way, I'm probably gonna blow through this pretty fast so I may be able to give you back some time at the end of this or we can take questions if you'd like. So one of the other networking strategies to reduce overhead is something called Cloudlets. This was, I believe, initially proposed at Carnegie Mellon some years ago and has been steadily being fleshed out over time but basically this is the architectural concept of converging cloud computing and mobile computing into a single device or machine appliance could be a software based appliance that sits in the middle of the network between the intelligent devices and these hyperscale clouds and is designed to maintain soft state around basically for microservices and containers. And this will put your data and your application logic as close as possible to the devices that you're operating with. A classic example of this kind of thing would be in a hospital, they would have a small WAN based network with a Cloudlet in place that would talk to lots of intelligent devices that could be in patient care or surgery and so on. What you find for the telecommunications companies are that many of them are pushing cloud infrastructure very far out to the edge of the network even into the cell phone tower because as you get into the future where you have self-driving cars, two cars going down the road at 65 miles an hour are not going to talk to each other through some hyperscale data center in San Francisco. They're gonna talk to each other directly if they have connections into their manufacturer and so on that's gonna be through a cell phone tower and so on. So you need to have much more computing capacity out towards the edge of the network to do that sort of thing. Okay, so there are numerous ways that you can go about doing this. As I said, you could have actual cloud computing capacity out on the communications network even so far out is actually in the cell phone tower and that's happening today. You can have it in branch offices so I would expect retailers if you're a very large coffee retailer you're gonna put a system, a cloud directly inside the retailing store and you think, well, why would I do that? Well, it's all about engagement with the customer. You want to be able to have the customer that has a smartphone as soon as they walk through the door their orders automatically placed to the coffee machine that the barista's managing. So it produces their favorite coffee before they've even gotten to the cash register and they can take that and go. But it also manages inventory, local inventory point of sale, a whole bunch of the facilities maintenance and so on directly in that store or potentially in that neighborhood. And then finally, of course, you could actually just have software based appliances where you're pushing capacity to various Amazon data centers all around the globe and you're just time slicing maybe using those data centers as a hotel for your particular containers for microservices and so on for some application that you need widely distributed. An example of that might be something like Pokemon Go which is a highly distributed application very geocentric. Now, I mentioned Cloud Lits and the whole sort of putting capacity as close to the edge of the network. There is another strategy Mesh Networks which pushes, takes advantage of the capacity that is actually out there in the edge devices. If you look at your smartphone and where the next couple of generations those devices collectively are tremendously powerful computer. If we took every phone in this room and we're able to network that into a single system what you'd find is we have plenty of capacity to run some very large complex applications. And in fact, we are seeing exactly that that mesh networking is continuing to grow very rapidly and as we move into the 5G networks it's gonna be even bigger. So some of the changes here are specifically around proximal discovery and communications between devices at the edge of the network. There are numerous strategies and industry standards that are being used for these devices. You've got LTE direct types of communication and standards around there, com0, com1 and comM. You've got low power, Wi-Fi, low pan and so on. You've got other types of things like 802.15.4 and ZigBee and all these other home wireless types of capabilities. But essentially all of these different standards all of these strategies boil down to having intelligent devices that can communicate directly to one another from 100 or 200 meters away. So if I needed to do a phone call to somebody say in another part of this hotel that phone call doesn't go through the telephone network. It goes from my phone to their phone. So that dramatically reduces the amount of bandwidth required on the infrastructure if we can do that. And there is just an unbelievable range of possible applications that could be done using that.  Imagine that you go to a concert, a music concert somewhere in a big stadium and if you're as cheap as me you end up in the bleachers somewhere. You don't have a particularly good view. Imagine if you can use your phone to take images of what's going on on the stage but also other people whether they're down in the first row or off to the other side of the stadium they can also be streaming video of what they see from the concert. And imagine all those phones could interconnect so that you could flip through whichever view of the video that you want to see whichever stream that you want to connect to. But it would only be directly between devices that are mesh networked within that stadium. It wouldn't be like this was being broadcast to the universe. It's for the paying audience that came to the show and want to be able to see things.  There are hundreds, thousands. I've certainly come up with many, many use cases with some of our corporate customers around the type of things you could do with this distributed technology. So the essence of this is that with LTE direct types of devices that have the chips inside them are capable of handling this sort of thing. It's incredibly low power, which means that they're turned on all the time.  And it constantly broadcasts to the universe who you are, what your interests are, things you want to do, people you know or you want to meet. And it allows you to discover all the other devices around you that are broadcasting similar information or metadata so that it has some sense of what's going on around you. And as you move through your environment, it senses where you are, what your motion or direction is, and what's happening all around you. Because it knows your preferences, it can tell you things like this store over here is having a sale on watches and you're interested in buying a watch. You might want to go over and look at the watches they have to offer. Or if you walk into a store, the clerks know automatically or immediately what your interests are. Now, that's kind of a human-centric view of systems of engagement, but it also very directly applies to any other intelligent device. So for example, today I've worked with customers using robotics on the factory floor and other construction type of environments and so on. And they're beginning to instrument all these, the heavy machinery, the trucks, the different pieces of equipment and the buildings themselves to be able to interact, and in particular to know where they're located and where they're going to so that they can improve overall safety and performance and efficiency in a factory or a job site and so on. The devices themselves have preferences, the devices have interests, they need resources and so on. They can communicate both with people and other devices to orchestrate their work that they're doing. So again, some extremely interesting types of applications are possible in virtually every industry that you can think of. I certainly have touched customers in financial services, automotive, agriculture, manufacturing, high-tech manufacturing, IT data centers and so on doing this kind of thing.  And this creates tremendously interesting things, but also is incredibly complicated to manage. So you've got the continuous discovery applications, personalized interactions, lots and lots of the distributed control applications. Traffic management is one that's coming to Silicon Valley for sure, quite a bit nowadays. And then in the financial services arena, there's automated transactions and so on using distributed blockchain technologies for import, export types of applications and things of that sort. Okay, so now we get into the next level of detail. Exactly how would you build an application to run in that type of infrastructure? Well, we've got very fortunate timing here as microservices are beginning to mature and the availability of containers has a way to package and deploy and manage those microservices. So we're able to create very small, single purpose services that are very easy to deploy. They have a very small footprint so they'll fit on the thinner infrastructure out towards the edge. And the overall model is that you don't try to patch all this stuff that is sitting in maybe 100,000 cars or smartphones or thousands of cloudlets out at the edge, but instead they're immutable. You just replace them entirely with the next version that you're gonna use. So all you have to do is be able to keep track of where they are and when you wanna upgrade, you just dump all the old ones and replace them with new. And of course, these things can be stored at the edge. They don't have to be actively running because they start up incredibly quickly so that as they're required, they can be booted and providing whatever service you need and then shut down and put off to the side again until the next time you need them. So if you try to think of this environment, you've got incredible range of container-based services talking to each other, very small, very light. They're very ephemeral, they come and go as needed and provide the services required. Now microservices themselves are definitely designed for distributed computing. And I don't wanna oversell microservices, they are very complicated. A lot of people think for some odd reason that monolithic applications are kinda equals legacy or something and that's idiotic. Monolithic applications have tremendous advantages in their own way in terms of overall ability to maintain them and to upgrade them and so on. But microservices are far, far better in a distributed computing type of environment. So each man to his trade, if you will, monolithic applications for SaaS in a hyperscale cloud make tons of sense and microservices in a hyperscale cloud can also make sense but they're really very well optimized for a distributed environment. So each microservice because it should be independent of or not dependent on any other microservice has its own data store and that's what causes this need for coordination between the microservices and fundamentally things like REST and HTTP are too slow for microservices to communicate with each other. You basically want an in-memory model if you can do it, something like coherence where you have a memory bus and everything is talking at very, very high speeds in memory. You wanna reduce that overall latency because there's already inherent in the fact that it's a distributed set of services. There's gonna be a lot of latency in those types of applications. So now we get to Eric Brewer's CAP Theorem and this is the fundamental computer science problem that we face and that is that you can have consistency availability and partition tolerance but it's near impossible to have all three at the same time. This is something I'm sure you've heard many times before. I personally don't believe that this is a theorem. It's never been actually mathematically proven to the best of my knowledge and in fact I think there are particular cases where they actually, it can be all three but let's just say for purposes today that all mainstream mundane sort of applications you have to make this trade off. So typically for a big system of record or enterprise IT system, consistency is the most important thing you can do. For distributed application where eventual consistency can be perfectly reasonable, you're basically gonna try to focus on availability more than anything else. And then partition tolerance is a given that you have to have that because in a distributed computing environment you cannot design an architecture that assumes all networks will always be up and available. It just isn't gonna happen.  Okay. So I said containers are the primary deployment model that we're all in this industry looking at as the way to deliver these distributed computing applications. There are tremendous advantages to containers. They do not, in my opinion, replace VMs. VMs have their own special role in the world and those are gonna continue for a very, very long time. But containers definitely provide developers with a much easier, faster way to put microservices together and to integrate and package everything up and then send it out very cheaply, deploy and destroy whatever you don't use and then replace with something new. For administrators, containers mean you know exactly what the microservice is running, what all the environmentals are that it has. You can keep track of where it's going. You can optimize the overall performance and throughput of these systems and so on. So containers look very attractive from both sides and as far as I can see, they're gonna continue to be extremely popular, extremely important as we move into the next generation of applications. One of the tricks to all this, of course, is that when you're talking about building a large, complex microservices-based application with a whole bunch of containers and then deploy that very broadly geographically, let's say even around the globe, you're talking about an awful lot of containers and then you pile that up over time. If you think that there was a crazy overuse of VMs and that they just spiraled out of control, well containers are gonna go an order of magnitude worse in terms of the sheer volume of these things that are gonna be deployed. You need to have a core infrastructure in place that enables the developers to do their thing, dev and test and provide a staging area, delivery model, the whole set of management tools around that that allows you to properly maintain the entire application life cycle because as you push this stuff out, it's out in the wild now, it's gonna be out in the world. In the case of automobiles, containers are very attractive because if you've got a system or a service that must be fail-safe, well you don't really wanna be doing patching on that application out in the field on cars. It's an iffy proposition in the first place and you may not even be able to attach to every car in a timely fashion to do patches and so on. Containers are nice because you're basically gonna flush the entire service or set of services and just replace all new with a new container on that automobile and that's to me a much safer, saner way to go about putting fail-safe types of applications on heavy machinery. But it also applies to every other type of device you can imagine out there and in various cloudlets that may be scattered around the world. Now, that doesn't mean that containers don't have their issues. Containers really still are fairly immature and it's gonna take a couple more years of work to make this all come together, not the least of which are things like how do I build these things in a high volume production environment? How do I do debugging of containers to understand what's going on inside them? There's a lot of questions around security. I think there's a whole presentation or set of maybe a couple sessions on container security going on here this week. And then a lot of new questions about management tools because most customers I know don't wanna take on a whole new set of management tools just for this type of deployment. So over time, all this stuff is gonna get worked out and things will smooth out but for now, there's still a lot of questions about how would you actually use containers embedded in intelligent devices that may be tens of thousands or millions of devices out there or hundreds or thousands of cloudlets that may be scattered everywhere. So for me, this boils down to four big topics that you wanna think about when you think about how would I build out this infrastructure. The first is to provide the actual immutable infrastructure for the containers in the first place.  It's gonna evolve very, very rapidly all the time and what you have six months from now is not gonna necessarily look like what you have today. And for every type of distributed application, there's gonna be variations on how you want the architecture set up and so on as I mentioned earlier. And then you need to manage the whole life cycle. So how are you going to deal with building and publishing and orchestrating and so on all of these different container-based applications built with all these different microservices and then how do you secure that entire infrastructure? Now I'm not gonna say that I can, I'm gonna stand up here in the next 15 minutes and give you all the answers to this because I can tell you we're struggling ourselves as we work with our biggest customers around the world on how are we gonna really build this out to do very sophisticated kinds of things. But let me make a few observations that may be worth noting. The first is around the immutable infrastructure. By this I mean the actual immutable OS and so on that you're gonna have throughout the network. You need to have an OS that is very small footprint for a couple of reasons. You're not gonna have as many resources at the edge of the network to run this stuff. So the tighter, smaller that you can make it, the better you're gonna be, off you're gonna be. And of course the fewer packages that you have in that OS and so on, the more secure it's gonna be. And since you have less direct control, physical control over those systems necessarily, you wanna have the highest security you can get. And then finally of course, that OS has to be small enough, easy enough that you can just throw it away and replace it with a new version very, very quickly and get it up and running. So getting the right host image for those containers is step one in my opinion for understanding how you're gonna build this architecture. Now, I took this from, what is it, InnoVex, Vex is a company I just saw this article online and this is from May of last year. But I think the numbers are probably in the ballpark of being correct from my experience. There are a ton of different micro OS's that you might choose from. And they do vary significantly in size but size is not the only consideration here. So I would quickly note that some of the larger ones actually have more security features built in than the smaller ones. So there's some trade offs here. You need to think, look at these kinds of things a little bit more closely before you decide which one's the most appropriate for your needs. But I will say that on a purely philosophical basis, I really like the Rancher OS because it's container-based and everything it builds is based on containers in the user space and so on. So to my mind, it's a very pure kind of design but I think there's a lot to be said for many of these other, certainly core OS is very popular and so on. And one of the areas that I just wanna quickly mention is, okay, so you pick an OS, how exactly is it upgraded out there at the edge of the network? And it's interesting all these different vendors have different strategies that they're deploying. Atomic uses just a whole new directory structure and you just update that and you can roll back if you need to and so on. Core OS uses two partitions, an active and a passive partition so that you update the passive partition, make it the active partition and then your formerly active partition becomes passive for some reason you need to roll back, you can shut down the active one and go back to the passive one, sort of roll back to where you were. And then Rancher OS, as I mentioned, because it's a pure Docker container construct, if you will, it basically just has a very small system Docker container that can be easily replaced and then you can upgrade all the other containers in the OS. So using Docker commands and so on. So there are numerous different strategies being thought about by different vendors. These aren't all of them, there's other choices. I don't know how this is all gonna evolve over time, again for me personally from a philosophical viewpoint, I like the Rancher OS idea but the others are totally valid ideas too. Now in terms of application portability, broadly speaking containers were designed to try to be portable, that is core to the philosophy of Docker and all the other container options out there. The open container initiative is very broad based, has very strong support from everybody in the industry so I'm personally pretty optimistic that containers will continue to be relatively portable over time. I will say some of the vendors who make containers are doing their best to try to put a mode around their value add or their IP and that always creates some problems but broadly speaking I think we're gonna see very good application portability with containers. And then earlier I alluded to the security challenges for containers, again the philosophy here is you do not patch, you replace software but who is the gatekeeper, who makes those decisions on what will be done, you have to make sure that you set these policies early in your IT environment. Is it going to be the development team or the operations team? Quite often the operations team wants to move faster than the developers on security issues, at least that's what I find in my day to day. And so you've got to have a strategy going in before you start production deployments of all this stuff because it definitely becomes an issue.  Very different for say companies that are embedding a container and intelligent device that's a big piece of machinery, a robot or something like that than if it's just in a smartphone. We also have been going back and forth a lot of discussions around whether hot patching, do you think of K-Patch or K-Splice or some of these other things, is that technology relevant anymore in this new distributed world? And while certainly we do want to have this immutable model where you simply throw away containers and so on, I do think for micro OS is that there's going to continue to be a place for hot patching OSs out on the distributed network for security reasons. Again, this remains to be seen over time but I think that this will end up being something that we need to maintain. Okay, so ultimately the mindset does have to shift for Linux administrators or other IT ops people that doesn't matter what type of failure happens out there in the distributed environment, you do not attempt to patch hundreds or thousands of systems out there where you can't even get at them. If there's a hardware failure, use auto scaling to move on to the other machines. There's network failure similarly. If you've got system software failures, just replace the OS. If you've got an application software failure, replace the container, just replace everything as you go over time. You gotta design or architect that application with that kind of mindset because once you have this stuff out there in the world, there's no way you're gonna be able to fix it on the fly. In terms of management, I do just generally want to endorse OpenStack. I think it was Rich Bo and I was speaking here in the prior session on OpenStack.org. These guys are doing great stuff. In my opinion, it's still very immature. A couple more years, it'll be much better, but it has a lot of interesting projects and potential with Magnum and Triple O and some of these other things where you can, where we'll soon be able to handle a lot more of these distributed environments and I think OpenStack's gonna be a very, very good choice over the long haul for distributed applications. Yeah, so the OpenStack community through the Magnum project has a lot going on to integrate with the various orchestration engines and so on like Kubernetes and Swarm so that you'll be able to have a common infrastructure where you can see and manage both the hardware, the systems, the networks, as well as all of the containers and the application infrastructure, the microservices and so on behind that. I think that's gonna be fantastic. So in closing, I just wanna say a huge proportion, 80% plus of all of the companies out there are moving to a hybrid cloud infrastructure. Fundamentally, that's the first step towards a distributed computing architecture. I believe that there is a tremendous amount of work to be done before companies are really ready for large scale production, distributed applications that take full advantage of all the things that 5G networks and the global cloud infrastructure can provide them. What I would recommend is first, you need to have executive recognition that there is a whole new class of distributed applications out there that harness intelligent devices to do really cool things for engaging customers and consumers and users and just make them understand the business value and that there's a technology shift that needs to be made. The second thing is, and I find this with a lot of customers, they do not put in place a mature DevOps model before they start leaping forward into other stuff. I think the DevOps piece is really hard to do well and is critical to have in place before you attempt much else. And then examine the business processes in your company. What would you like to re-architect? What types of applications you have that would be amenable to a distributed environment that would do something new and interesting for your business and begin that to plan how you're gonna take each business process and implement distributed strategy to that. So those are my recommendations on next steps and that is it. I'm saving you five minutes here. I guess I'm happy to take questions if there is anything of general interest. Okay, well, I'm not gonna take up people's time. I'll be here for a few more minutes and then feel free to ask me or you can email me too after the fact. Thank you.


In anemia associated with a nomocytic picture, the first consideration would be that there isn't an underlying cytoplasmic delay in maturation or there may be an underlying cytoplasmic delay in maturation that is subtle and balanced out by a nuclear maturation delay. Anemia of chronic disease usually presents as a nomocytic anemia. The reason being that the RBC, the red blood cell count is reduced. The total hemoglobin is reduced but in the face of normal stages of maturation of both cytoplasm and nucleus. And so you're going to see a normal distribution curve except you would notice the reduced total hemoglobin and the reduced total number of red blood cells drawing attention to the fact that there is reduced cellular output from the bone marrow. In other words, the factory is not producing the amount that you expect rather than an abnormality in quality. So it's an abnormality in quantity.


و أقولوا في القرآن ما جاءت به آياته فهو الكاريم المنزاله و أقولوا قال الله جل جلاله والمصطفة الهادي و لا أتأوله لكن نحن لا نستطيع أن نتركه أمامه فهو سيكون معنا في نهاية أخي يتوقفني أن أقوم بعملها كيف can I handle this situation in the best way الحمد لله رب العالمين والصلاة والسلام على عبد الله و رسوله نبينا محمد و على أهلكم و صحبكم أجمعي أعتقد أن في حالة مثل هذا حيث أن أخيه أمامه لأنها هي حالة لا يستطيع أن نستطيع أن نعيي أخرى أعتقد أن هناك أشياء that are really really important in this أعتقد أن أول شيء هو أنك كمباركة في العالم ترى هذا as your means to worship Allah و كمباركة أن الله has given you to achieve و to attain jannah by his permission و أنك أردت أن تعمل جيداً و أنك أردت الله و أنك أردت أن أردت أخي و أنك ترى هذا كمباركة و أنك أردت له أن هذا هو أمامه و أن أحاول أن أحاول أن أخذ أمه لأنها أنها في حالة where she doesn't have anybody else to أردتها أو أردتها أعتقد أن أفضل أنك تحاول أن تحاول جيداً و أنك تفعل أفضل و أنك تفعل ذلك و أنك تحاول أن تحاول أن تحاول أحاول أن تحاول الأحاول من الله و أن أعتقد أن الله يحاول أن تحاول أنك تحاول أفضل الثاني أنه يهم جداً أن تسأل أخي أن تسأل أخي في such a way that you do have your own space now there's no doubt that this is different for different people it depends on the house in which you live it depends on your husband's means is he able to get his mum a separate apartment or not is he able to separate a part of the house like the upstairs and downstairs is he able to give her own space her own room or own kitchen everybody has different means but as much as possible you ask your husband to provide you with your own private space so it doesn't have to be the case that you're always together with each other and there's the danger of clashes or she could be difficult and then it might be hard for you to maintain your patience but to have some degree of separation and so part of that I've even in places which are very small even when some husbands don't have the means to do too much there's still a lot of ways that you can maintain that privacy even if it's just in your own bedroom that you ask him to provide a chair next to the bed perhaps a sofa or something like that whereby you can have some time on your own and you can have some space and I think that is something which is really really worthwhile so I would say two things first of all that you see this as an opportunity to earn reward from Allah you try your very best you try to help your husband out by helping his mom but on the other side you ask your husband to give you an appropriate amount of space to be able to just maintain the distance which allows the relationship to be good and stops that relationship from turning sour because you're too close together and there are too many clashes that happen that's what I would suggest initially just after having read through your question and Allah is so generous best if you have any questions you'd like to see answered as part of this series and you can email us at questionsatamau.org