We're having some many technical difficulties on that. We are promised someone to take a look at this specifically, neither the keyboard nor the mouse works on this. So it looks like someone is already monkeying around with it. It looks like it's a brand new keyboard, so I'm not sure what's going on. So I do have a mouse. The problem is I need a keyboard to get to the website. So to pull up. Use the onscreen. I'm sorry? So you probably would use the mouse. Can I actually do that? That would be great. This might. Is this like?  Can you walk me through that? That would be great. So yeah, I do have a mouse. So OK.  So where we don't start? Yep, OK, start. All apps. Let's just talk to him. Oh my god. Anyway, so that things are not a total loss in the meantime. By the way, Vicky, Mike, Chuck, just want to make sure that the level is exactly right. So OK, level is fine. OK, sounds good. OK, a couple comments just in the meantime, though. I can actually get started with some general announcements and administrative review on this. I assume everyone is tanned, relaxed from the white or black sand beaches up to Haiti on this, depending. So it's back to reality. And the upshot is this week, obviously, you have the midterm. I did confirm with Jeff. It will be held as scheduled this upcoming Friday. So more information will be forthcoming about that. I would expect it will be in this room unless otherwise announced on that. You may have heard that Jeff is very strict about the midterm. I would view that as a good thing for those of you who view tests as an individual as opposed to a group effort on that. So in other words, there's going to be some more information. Make sure I can guarantee you this. We're going to be carding you. So make sure that you bring your UBIT or photo ID. So the TAs are going to be checking that while the test is in progress. Also make sure that you arrive in plenty of time. Also, there will be assigned seating. Typically, you'll also have assigned tests on that. So there's going to be like a seating chart that will typically be placed outside the room on that. And make sure that you check to see wherever it is, like what the particular seat that you need to go to and that the test that you're taking, it will typically have your name and your mug shot on the back of it. So make sure that you take your own test on that. And typically also at the end of the test, pay attention to the time on that. Jeff will typically put a timer on there and saying how many minutes and seconds is left. You don't want to turn in your tests late. This is one class where that is not an idle threat. I can guarantee you on that. So in other words, budget your time on that. Just a couple of words about Jeff's test in the past. I've certainly taken them myself as a student and I've seen enough of them over the past several years on them. They're very predictable. He's not trying to trick you. His tests are neither hard nor difficult nor easy. He wants you to think on this. And this is one thing he's very upfront about it. In other words, this is not going to be a regurgitation or memorization test on that. So take a look at the material you have. I also confirmed with Jeff that everything through Wednesday will be on the midterm. So right up through this upcoming Wednesday, anything is fair game for the test itself. The other thing to keep in mind is that in terms of what he's looking for is, again, you're not simply trying to memorize facts, but he's going to want you to actually apply and think with them. In other words, he's going to ask you questions about, let's say, okay, what happens if we came out with a new piece of hardware, let's say a really fast processor or let's say we came out with, let's say, a disk drive that is much bigger. How would this impact, let's say, some of the policies in terms of, let's say, scheduling or paging or what have you? He's going to actually ask you to, let's say, occasionally, let's say, what is the best way of approaching this problem? Or let's say, doing some pros and cons. In other words, this particular approach is going to be better for this type of a situation. This other type of approach is going to be better for another situation and explain your line of reasoning on this. So again, he's going to ask you to take what's talked about in the lectures, but go beyond them. I remember, I got asked a question at one point about, well, let's say, as you know, typically, let's say when you, let's say, call the disk drive to let's say read a bunch of bytes from disk, it's a blocking operation, you know, sis read blocks. What happens if, let's say, we implemented asynchronous IO? What would that impact your program design? Because, again, there's pros and cons to that. Obviously, the asynchronous IO read operation returns immediately, but you don't have the data for that. So how are you going to design your user programs and what are some of the policies that you need to kind of consider in the kernel? That's a good, if you will, type of question that you can expect on that. There's probably going to be, typically, something involving a little bit of mechanical math in terms like, say, if you will, page table like TLB and whatnot. So in the same way, if you've taken, let's say, 341 and you know about, let's say, if you will, a cache eviction and cache hits for that class, it's going to be fairly similar for, let's say, the TLB for this class too. So, again, yeah, it looks like we've got it dead and or, as long as I can get to a website, I would be happy as a clam. Thank you. So, anyway, back to reality on this. So, again, the midterm Friday, again, look over a couple things. Look over all the lecture notes. Also, if you have not done so already, again, attendance is definitely better now than it was just before spring break. But, again, to those of you who are in video land, I please emphasize, do not try to watch 15 to 20 hours worth of lectures in the day or two before the test itself. Your mind is going to melt and you're not going to retain what you need to. So, if you do have lectures to make up, you want to do that starting like now ASAP to get those out of the way. So, look over the lectures. Look over the lecture notes and also old tests. I know as far as like his test follow a very good pattern, he's not like one year's test, if you will, is suddenly different from previous tests. You have access to all of the old tests. They're posted on the website and you can kind of see what the format is like for that. In terms of the general format, they are essentially essay tests. That seems to be kind of the trend for most operating systems classes in most universities on this. There's typically going to be a few multiple-choice questions that are right up at the beginning that are essentially police questions. If you've come to class and you've paid attention to the class, you should be getting close to a perfect score on the multiple-choice questions. But I know my year, and I believe pretty much all the other years, the multiple-choice is typically there's only like 10 of them, one point each. The bulk of the points is going to be on the essays on this. I can also say like, I certainly, myself have served as a grader. I'm probably going to be helping with grading with this. What are you graders looking for? So I can answer that. Again, first and foremost, how I approach this is in terms of we go through, we come up with a grading rubric so that we can be, if all else, consistent with the direction of the questions. What we want you to do is, again, answer the question on this. If you have a question during the test about like this is ambiguous, ask us beforehand. Don't try to make any assumptions on this. But again, we're going to be looking for can you answer it based upon what the question itself asked and what in terms of can you think, again, beyond what is explicitly stated in the test itself. So again, take a few minutes or so, look over the old exams, probably more than a few minutes because again, I would strongly suspect that Friday's exam is going to follow the leader of what previous exams have been like on that. And so again, to recap, midterm quiz on Friday, make sure that you follow, I strongly suspect there's going to be a bunch of emails, follow them to the letter in terms of when you arrive, seating, bringing your ID on that. The exam will be videotaped, so I can tell you that. Like I say, for those of you who are viewing this as an individual effort, this is all good. It's essentially meant to crack down on people who view tests as a group effort on this, which is kind of difficult anyway for a essay exam. But again, for the rest of you, I think it should only be a plus on this. And that should be all she wrote on that. So questions I can answer about the test itself. What's coming up? Procedures, style, what have you. And then again, just to recap, next week will be assignment 3.1 is due. So again, that's coming up right after the midterm. The core map assignment, there's already some post about how to get started. We had a lecture, I'm sorry, a recitation about the mechanics of assignment 3.1 the week before spring break. We're going to try to focus this week. The staff will be working on getting you some midterm review material on this. So we're not gonna be talking about assignment 3.1 this week. Next week essentially is wrap up a few days before when it's actually due. And we're gonna kind of start moving over to assignment 3.2. Again, 3.1 is not a big assignment, but I also don't want you to become falsely confident. Talk to the people who have already started it. You're gonna hit some, if you will, arithmetic issues fairly early on. Part of it is your kernel is not even gonna boot once you configure the bloomer for assignment 3. And so it's gonna take a little bit of time to work through, if you will, getting, well, how can you debug stuff when you can't even have a booting kernel on that? So allow enough time to get through with that. So again, the sooner you can get on to assignment 3.2, because I can tell you assignment 3.2 is going to be a big one. 3.1 is not bad, but 3.2, just a word of warning, it is bigger than assignment 2.2. And you only have two weeks for it. So this is something to kind of just budget in terms of your timing. If you can get started with that, so much the better. All right, now let's see here if I can actually make use of modern technology. And ooh, the mouse works. And how about that? Okay, the keyboard works too. So let's see here, swapping slides. There we go. Oh, this is so cool. All right, let's see. Onto the substance of today's class. Let's see what we can get through about this today. Essentially, what we're going to be talking about today is, if you will, what happens when you run out of memory on this? Up until now, what we've been talking about in terms of virtual memory is, well, what is virtual memory as opposed to physical memory? But we're still talking about essentially referencing, roughly speaking, the RAM chips in the system. Virtual memory, again, let me emphasize that there's a lot of reasons why we have it. We talked about, for example, security. We talked about the example that we can map a whole bunch of disparate fuel physical pages to one big, continuous virtual page on it. Another advantage of virtual memory is that we can make virtual memory map onto a lot more apparent memory than appears on the memory chips itself. You've all experienced this. Windows is kind of grinding away with a swap file or what have you on this. So let's actually talk about the mechanics of how this works. This is actually going to be directly applicable to assignment 3.3, a.k.a. swapping or eviction. It's the toughest part of the course assignments. It's going to be something that you're going to really need to think about and think about a lot. So do a lot of planning before you actually start coding on this here. Well, what happens when we do run out of memory here? Well, a couple options. The first option, kind of the easy one, and essentially, croak. We just can't handle that. So in other words, if we try to, let's say, call fork or what have you, and the operating system determines that it has no memory, we'll just return back to the user and say, I can't help you on that. Same thing goes for exact S-break, what have you. But that's not a very good option, and we want to avoid that if at all possible. So let's take advantage of, if you will, backing storage. Remember, again, back to 341. Essentially, a good mode of computer design is you have at the top few fast but very expensive types of memory, and it kind of boils down a pyramid shape to, at the bottom, is plentiful, cheap, but slow storage at the bottom. And right up at the top, we have, let's say, in CPU registers. They're at the top, and then like L1 cache, L2 cache, then mainboard memory, and then below that, the next level is the hard disk drive. And that's actually what we're going to be talking about today. You can even extend it below hard disk drive and talk about, let's say, tape backups or something like that. But this is all, if you will, of the levels, if you will, of the trade-off between time, money, if you will. So let's see, how can we actually create more space if we run out of mainboard memory on this? Well, a couple things here. We want to make sure that this appears as transparently as possible to the end user on this. Again, that's one of the nice things about virtual memory, that the end user, and indeed the end user program, doesn't have to worry about this. I, as a program, can simply refer to memory and the operating system is going to make all this magic happen behind my back. Well, how does this actually happen here? A couple things here is, essentially, the last time it behaved like memory. In other words, we thought that something was memory. And guess what? The next time I want to access this virtual address, we also want to have it somehow behave like memory too. And in between, OK, we've got to, if you will, preserve memory on this. So in other words, if I referred to, let's say, virtual address 1,000, and I stored a byte, and then let's say I, let's say, a timer fired, and let's say the operating system decided that it needed my memory, OK, behind my back, the operating system transparently, to me, is putting me to sleep. Again, think about Han Solo being stuffed into carbonite on this. So I'm in suspended animation. The operating system takes the value of virtual address 1,000, dumps it on the disk drive. Later on decides to reanimate me, OK, and takes the value that was on the hard drive, puts it back into some place in mainboard memory that now, once again, maps to virtual address 1,000 and reanimates me. As far as I'm concerned, huh, who, what, I'm like, I'm Rip Van Winkle. I think that nothing has happened, but all this stuff has happened behind my back. That's the essence of paging on this. It's great. It's magical. It can be somewhat of a pain to implement because, as you know in general, if you will, the easier things become to the end user or appear to be to the end user, it's probably going to be slightly more complicated to actually make it happen. So let's take a look at how we can actually do this here. Swapping, a.k.a. eviction, if you will. Eviction, OK, I'm using that a very sloppy expression. That's really only part of swapping here. And again, in general, what's going on here is we are using the hard disk as a backing store for what is in mainboard memory on this, all right? So what we're talking about in terms of moving stuff in and out, we are actually swapping pages from mainboard memory into, if you will, the hard drive. When I say eviction, I'm talking about typically pulling stuff. That's one of the stages. It's pulling stuff from mainboard memory and, if you will, evicting out a page so that someone else can use it. OK, and again, ultimate goal here is we want it to appear to the end user as much as possible to be like it has more memory than actually has on this. Now, it's not going to be a complete match. As you know, if you are, let's say, trying to run too many processes of memory intensive stuff on your laptop, you're going to know when it starts hitting the swap file, because guess what? The computer is going to do what? Exactly. Remember, it slows down. Remember the old thing about, if your computer is slow, well, besides getting rid of the viruses on the computer, the big fix for that is put in more memory. Does this all make sense now? Because the more memory you have, the less need the operating system has to go to disk. So swapping is great if we need it, but we want to avoid it if at all possible. Everyone clear on that distinction? We want it as a last resort backup, but we only want it as a last resort backup. We want to avoid it if at all possible on this. And I can actually say right now, one of the tough parts about assignment 3.3. Up until now, essentially, Jeff's tests have been focusing on, if you will, logic, passing the tests, getting the logic right. Hasn't really concerned much about performance. With 3.3, one of the things that you're going to need to deal with is some tough timer hurdles on this. So even if your logic is correct, there's going to be a timer running on test 161. And if you don't complete it in so and so much time, it still is a test fail. Because that's, after all, one of the points of eviction is to make sure that this works efficiently so that the end user is not simply sitting there twiddling thumbs. Because this is actually really the first part of the computer system and the project in this semester anyway, that you will actually see noticeable lag to the end user on this. Most of the time, computers are a lot faster than us. There's page faults going on probably millions of times a second in any given particular computer. But you all know it. That's why we're just asking. You know when the computer is going into swapping mode. Because at this point, there's too much going on. And the hard disk is really, at this point, kind of noticeably slow. And so that's, again, why you're going to need to deal with some timing hurdles on this. OK. So again, this problem we're talking about, we want to be as close as possible to RAM. But again, we do have some problems here. Very often, it's the exact opposite on here. Namely put, we have kind of the worst of both worlds. It feels like it's the disk, but it's as small as RAM if things are not done well. Sometimes this is unavoidable. You are simply trying to run too much on a computer. Sometimes it's because you need a memory upgrade. Sometimes it could be because you have a badly designed operating system on here. So again, speak to the support staff when it comes time to designing 3.3. All right, question so far. Talk in general about what swapping is before we get into some details. This slide actually brings up a point here. I do need to apologize. I have been sloppy in some of my lectures. I have been using the term page faults as essentially a big blob of goo that really I should have been, if you were more specific. It kind of includes, if you will, both TLB and page faults. So let's actually be a little bit more specific on this. Number one here is what happens when we have some sort of a memory related fault here. Essentially, first thing is let's say that your operating or your user program refers to a memory location that is not in the TLB. Remember we talked about this actually the week before spring break? The TLB has a series of virtual to physical address translations. And if there is not a mapping corresponding to the address you want, well, that generates a TLB fault on this. So again, I was, again, I do apologize. I was referring to both of these as page faults on this. Strictly speaking, for the purposes of swapping, we need to be a little bit more specific on this here. So again, it all faults on pages, but this is a type of faulting as opposed to let's say it was not in the TLB, so we have a fault. And then it goes to the operating system. And the operating system has to decide what to do. There was a reference to virtual address 1,000. We need to see, well, where is it in physical memory? Oh, it maps to physical address 2,000. In that case, we simply populate the TLB with a new entry and go on. That's what you're going to be doing for assignment 3.2. Maybe virtual address 1,000 doesn't refer to physical address 2,000. Maybe it refers to something on disk. That is a page fault. So essentially, let me kind of take a step back here and say, this is going to be kind of the types of states you're going to have to deal with, with implementing assignments 3.2 and 3.3. If you have a reference to a particular virtual address, well, scenario number one is there's already an entry in the TLB, and it's handled by hardware, transparently done. Everything's good. Life is good. You go on. But if it's not in the TLB, we have, number one, an option that it is still somewhere in physical memory. And we need to update the TLB. By the way, how do we know whether or not a virtual address is in physical memory? What type of a data structure do we have to check? I'm a user program. I'm referring to virtual address 1,000. We got a TLB fault on this. How do I know whether or not it refers to any physical address? I have to check what? Exactly. Page table and more specifically, the what corresponding to that virtual page. The page table, PTE, page table entry, OK. Which, remember, a page table is essentially a data structure consisting of a whole bunch of page table entries. That's going to tell you what the state is of a particular virtual page. So again, if I refer to virtual address 1,000, what's the page table entry for virtual address 1,000? It's going to tell me, yeah, it's in physical address 2,000. Or maybe, you know what, it's nowhere. It's somewhere on disk, OK. And that means that's a page fault. I have to retrieve it from, if you will, the hard disk. Or if you will, the, I should say this, there is a third option, which I'm kind of gooseing over here, which is it's neither in, if you will, physical memory, nor in hard disk. Now I have to determine whether or not, essentially, I should create it. In other words, is it a valid address? Or is it a totally invalid address? Because if I refer to virtual address 0, I can bet your dollars to donuts that it is not in my page table. Because remember, the operating system typically is going to be expecting, if you will, virtual addresses of 0s to be the result of common, if you will, programming errors. So we want to trap for these null pointers. So again, is a reference to a virtual address? Is it in the TLB? Is it in physical memory? Is it on disk? Is it a potentially valid address? Or is it potentially a completely invalid address? We have to determine that by looking at our data structures. And again, this is kind of the grovemen of assignment 3.2 and 3.3. Make sense? OK. So back to TLB and page faults on here. Essentially, TLB fault is if I refer to a virtual address and there's not an entry in the TLB, well, that's a TLB fault on this. And remember what happens when a TLB fault? Well, it's not in a TLB. So I go to the operating system. And it's the operating system that has the information about, well, what do you do in that case? And now we can see, if you will, page fault or not. So in terms of dependencies, essentially every page fault is preceded by a TLB fault. But not every TLB fault generates a page fault on that. Make sense on this? That was my subwoofer. Yes, excellent question. Did everyone hear that? When a program, let's say, is just launched, if you will. OK. The question, is that a question? Are you just confirming that that's the case? In other words? That's a question. OK, yes. OK. Because essentially, let's say we're in the midst of launching a program on this. And you're correct to assume that we don't have any entries in the TLB, at least corresponding to that. So question, will this impact the performance of the program because we're going to have a whole bunch of page faults? Yes, we will have a whole bunch of page faults. And now I will say this, because we're kind of assuming that we are implementing what we call, if you will, allocation on demand, or on-demand allocation. I talked about this in the recitation, if you will, the week before spring break. So again, take a look at the recitation slides. But this is something you will need to implement, which is allocation on demand. There's another way of handling, which is static allocation. In other words, let's say that I load program foo from disk. And let's say the program foo takes up 100k. During the load phase itself, load elf is would automatically allocate 100k or pages or what have you worth of physical memory. That's static allocation. So that when it runs, it doesn't have to do all that, if you will, TLB faulting on that. That's a pro and con. As a matter of fact, that might be a great exam question on this. Let's say compare, let's say, static allocation versus on-demand allocation on this. We already know what one of the downsides of on-demand allocation is. Namely, we get all these page faults that we have to service at the beginning here. What might be one of, let's say, the upsides of, if you will, let's say, what's one of the upsides of static allocation? We allocate everything right up front. What am I avoiding? Yeah, I'm avoiding all these page faults, right? Once I go through that initial hump, from that point forward, things go a little bit faster on this. But there's some downsides to that too, which we'll get to in just a few minutes. I can tell you static allocation is actually not going to work. It's going to be breaking on some of the first tests that Scott will run against your code on this. So let's do on that. Answer your question? OK. Most modern operating systems do do on-demand allocation. All right. OK. Oh, and everyone again. He'll be page fault. OK, swap out. OK, swapping in and swapping out. Now, I should say this. You swap out essentially, well, when do you swap out on this? Because we know when we need to swap in, when we need something. In other words, I've referenced a memory address. And that memory address, when I check my page table entry, it says that it's on disk. I've got to pull it in. But swapping out, it appears to be a little bit less, if you will, urgent on this. Because, well, why would I want to get rid of something that I am, in effect, not using? Well, I can see why it might be useful in the future. But it actually is kind of urgent, because typically, why are you swapping something out? Because, yes. The TLB is full. Ah, OK. Everyone hear that? OK, we're talking about swapping out here. OK, you're actually, OK, this is two very closely related concepts here, OK? Namely, the TLB can be full, OK? Because remember, the TLB only has 64 entries on this. And that is something that, by the way, will be checked in 3.2 on this. What happens when you fill up all 64 entries in the TLB? So you do need to have to kind of, if you will, think about how you're going to have to, if you will replace entries in the TLB. What I'm referring to here is, if you will, swapping out from memory to disk, OK? So, like, why would, because if I have a TLB that's full, what would I need to do in order to, let's say, I've used up all 64 entries here? I'm going to need to get rid of another TLB entry, OK? But I don't need to get rid of, if you will, let's say, evict a page out to disk. Does that make sense? OK, but that's good. Everyone clear on the distinction between, sometimes this gets confused here, and don't do it on the test, OK? If you will, swapping out TLB entries versus swapping out physical pages. And also, again, TLB faults versus page faults on this. It was a question, yes? Well, OK, what TLB, yeah, if you look at the TLB allocator in dumb VM, it essentially runs out of, it just picks the first 64 and then it panics, OK? Which in a real system is never going to work at all. Because you will definitely have many, many cases where you will be using more than 64 different pages in virtual memory. And you don't, because essentially what are you doing? You're saying is that you are limiting your user program to 64 times 4k, or what is that, 256k, OK? Which that means essentially any program that uses more than 256k of memory is going to panic on, if you will, dumb VM. So we definitely need to have that fixed, OK? It's not a big fix, but it's something that does need to be changed for assignment 3.2 to work. OK? So let's actually kind of talk about this. I was saying, well, what's the urgency contained in swapping out? What I'm trying to get at is the reason we swap out typically is because we are indeed under pressure because we need to swap in, OK? In other words, we're trying to pull something from disk and we need to create a physical table. Well, why did we put something on disk in the first place? Because we were probably out of memory. My guess is that condition has not really changed. So in other words, you're kind of getting a little bit of churn, if you will, of processes kind of evicting each other out here. So that's why this is typically going to be, if you will, something we need to do quickly on here. So this is the process here. OK, first off, number one is remove the translation from the TLB if it exists here. Because remember, I'm swapping out, let's say, a page that probably is being used by another process. Do I ever want to swap out my own pages? No, I want to swap out someone other process, OK? So the thing is, in order to do that, I have to at the very least make sure that the TLB for the other process is updated. And if you're taking notes, make a note of this, this is actually where you take a look at what the mechanism is in dumb VM for doing that. You'll see that it doesn't even support it. It's like this TLB shootdown stuff. But point number one is we need to adjust the TLB for the victim process. Number two is write it out to disk. Number three is update the page table. Guess what, the page table entry of the victim process again. And let's actually take a look at the graphical representation of this. Let's just say here that I have, OK, here is my particular page that I've chosen as my victim. First thing that I want to do is get rid of the TLB entry for this process. And why do I want to do this? Because if I don't do that, and then another process refers to, let's say, the virtual page 10, OK, the TLB is going to happily point it to, if you will, the same physical page. In other words, probably the physical page that I'm now using for my process on this. So this is going to result in essentially a mess. So I got to get rid of that TLB entry. Next thing I want to do is take a look at the page table entry, which currently says that blah, blah is somewhere in physical memory. I want it to be somewhere in disk instead. So what I have to do here is, blah, blah, blah, because this is the slow step. You're probably blocking on IO. And then switch the state that says that this particular virtual page is no longer in main bus memory, it is on disk. So again, general procedure here. Clear the TLB, if you will, write out to disk, update the state, yes. Question, is the TLB per processor or per system per processor? Yes? OK, definitely not, because again, translation is per CPU on this. So in other words, this is actually, this is a good question here. In terms of invalidating the entries here, what you need to do is, number one is, what is the victim process on this? And where is the victim process running on what CPU? It may not even be running at all. It may be sleeping, or maybe it's scheduled, but it's not currently running on this. So that's kind of already hints at a bit of an optimization right there. But let's say, if I'm on processor zero, and my victim process is also on processor zero, I can already take a bit of a shortcut there, because I know it ain't running, I'm running. But let's say the victim process is currently scheduled on, let's say, processor two, and let's say that it's running, well, then I'm actually going to have to give that other processor a poke. In other words, I'm going to have to generate an inter-processor interrupt. This, by the way, up until now, you have been servicing interrupts. This is now the first time in the class where you have to be concerned with directly generating an interrupt, because you are, if you will, affecting another CPU, another TLB, and another process on that. Does that answer your question? And it's running on two CPUs at once. Do you need to reach out to both of those CPUs to validate? It's whatever CPU, technically speaking, if you want to be really optimal about it, you need to pick the CPU on which the victim thread is running. Now, typically, we want to try to avoid evicting threads of our own process, but sometimes we might need to. And if we do need that, then we have to figure out, what is the victim CPU of the victim thread? Does that answer your question? So again, up until now, I should say, up till assignment 3.2, we can talk about, if you will, pages, page tables, address spaces, they are essentially per process. Because the only process that is concerned about a address space is its own process, until you get to eviction. Because now the whole point is we have a case of other processes, or in the case of a multi-threaded operating system, other threads monkeying around with my address space. So we're going to have to deal with some nasty synchronization issues on this. Yes? So the information about which process is using it and everything is stored in the core map? I'm sorry? The information about which process is using the memory and in which CPU is it stored in the core map? And how do we know that? Good point. How do we know? OK, let's say that I decide that, OK, I want to evict out, let's say, who knows this particular area in physical memory? What stores information about physical memory? Exactly. So that's going to be, if you will, my anchor between, if you will, how processes communicate with each other on this. So in some way, I'm going to have to lead breadcrumbs, if you will, from one process and one address space. And I can go through my core map to get to other processes and other address spaces on this. Correct? Hey, also I will say this, in terms of implementation, you're going to have to hold multiple locks during eviction. That is also part of the tricky part. So you have to avoid deadlocks. There was a hand raised. Oh, OK, in other words, what's the policy? In other words, what particular one to pick is my victim? Wednesday's lecture. OK, to make a long story short, that's actually one of these policy thingamabobs.  There are tougher to implement. But more efficient algorithms. It's kind of like scheduling on this. But you can also use the past to predict the future on this. Like you can start with a simple round robin algorithm. It's easy to implement. I'll also tell you, you will not pass the time barriers with that. But it's a place to get started. There was a question over here. Well, yeah, I mean, you're going to have to, well, I guess that, yes and no, you can do a very quick clutch. And what's the easiest way of, if you will, invalidating TLBs that's that way, I don't have to keep track of everything. Let's say that I want to evict a particular page. How can I make absolutely sure, quick and dirty, atom bomb approach, that that victim page is no longer in the TLB? I can do what to what? Turn off almost, OK? But what else here? But in effect, it has almost the same effect here. And clear the TLB for what? But for what? For which CPU? All of them. Nuke them. In other words, now, it is going to be inefficient because I'm probably just evicted my own entries on this. But again, in terms of just getting your code working, this might be a quick step. In other words, if I have to evict, I nuke all the TLBs. I know for sure that there ain't going to be any bad entries in there because there ain't going to be any entries. They're gone. But again, that's going to be, again, inefficient. You want to think about how can you actually make this a little bit more efficient on that. All righty, so swap out. Again, remove copy, slow, update. OK, so page cleaning here. OK, one of the things that Jeff's trying to get at here is, can we try to speed this up a little bit? We know, for example, that we're probably going to have to evict at some point. So remember what I was saying earlier, what's the pressure in swapping out? Well, probably because we need to swap in. Maybe we can anticipate this a little bit of time. And in effect here, you know what? Let's convince ourselves that each page has a dedicated place on disk. Not too difficult because the disk is going to be a lot bigger than memory on this. And here's the key part here. During idle periods of the operating system writes data from an active memory page to the swap disk here. In other words, let's take advantage of the fact. You know, in the same way that sometimes when the CPU is blocking, we can do other stuff so that we're not wasting CPU cycles? Well, sometimes when the system is idle, can we kind of take advantage and start preemptively writing stuff out to disk on this? Now, a little bit of a caveat here. And namely, let's say, do I myself want to write out my own pages to disk? Because almost by definition, I'm doing something here. If I'm in the middle of a mathematical calculation, I don't want to block to write stuff out to disk. And even if, let's say, I'm blocking on keyboard I.O. or something like that, it's going to be tricky if you will coordinating. Let's say I write out to disk, but in the meantime, the user has typed several characters on this. So essentially what I'm trying to drive at here is the operating system as a whole wants to do this, but you're going to want to try to rather than have the, if you will, the user processes do it, have, guess what, a dedicated process to do this. Has Jeff talked about like paging daemons? So nutshell version on this is, this is what real operating systems do, is they have dedicated threads that essentially, on their own in the background, are constantly writing stuff out, writing stuff out. And that way, when the dedicated thread blocks to write stuff out, it's not blocking, if you will, the user threads on this. And we can spin up as many of these dedicated user, if you will, paging daemon threads as we need to on this. So the nice thing about this is, and actually there are people who have done this here. So if you're looking for a little bit of an extra challenge for swapping, but on the other hand, it'll make it run faster, write yourself a paging daemon on it. Because that way, if you will, when you need to evict something out to disk, guess what, it's already evicted. You will need to, in one sense, kind of keep track of which ones have been evicted out to disk and when. That's kind of a little bit of, if you will, a implementation, make it a little bit more complicated. Okay, swap in. Let's talk about the last part of this. Namely, how can we actually get stuff back into memory? We're gonna reverse to this here. Okay, first thing is we need to stop the instruction here, allocate a memory page, locate a memory page on disk, copy it. In other words, this is kind of the reverse of what we were talking about with swapping out. Let's actually see how this actually works in practice. Okay, so here is our graphical input. Let's just say that we have a reference to memory location one, oh, something. It ain't in the TLB. So immediately we have, if you will, let's say here, we need to, we have a TLB fault and the page table entry says that it's somewhere on disk. Well, in order to service this TLB fault, what I need to do is get it from disk and put it into memory here. So guess what, it goes from, excuse me, blah, blah, blah as we're reading it in. And then guess what, update the PTE state to point to someplace in main bus memory and update the TLB and now I can finally restart the instruction. Okay, at that point it will be handled transparently. So again, to recap on this, if I have a page fault, I need to check the TLB entry. It'll tell where it is. I need to grab a new space in physical memory. I need to read it in from disk and I need to update my TLB and my page table entry data structures as appropriate. Questions on this? Okay. On-demand paging. This is what I was talking about earlier. You were actually going to need to do this. Simply put is let's procrastinate as long as possible. Remember you can have static allocation or on-demand allocation. What you're gonna need to do for OS 161 is on-demand allocation here. And essentially what we're doing here is you know what? If the process says load a bunch of memory, let's say the code or what have you, I'm actually going to mark a particular area in my change tables as being reserved but I'm not actually gonna load it from disk until I need to or vice versa on this. Let's say that I expand. Let's say the heap on this. I'm not actually going to allocate the physical memory until I actually page fault on this. Now we said that one of the advantages of static allocations is that it is fast once you get through it. But there's a disadvantage of static allocation and there's a big advantage of this. Namely, we don't have to actually allocate memory unless and until we actually use it. And sometimes the point being is we'll never use it. Questions on this? Talked about if you will static versus on-demand or dynamic allocation on this. I think at this point, yeah. Demanded pages, I'll mention this. Yeah, okay, one last page and I'll let you go here. Okay, page 16 here. Demand paging here is what happens when the process executes instruction from a new code. Okay, we essentially load it in from disk and restart as needed. And the bottom of this page here, this is key here. When a user process refers to a page in memory for the first time, you need to allocate it and really, really important, make sure that you zero the bugger. Okay, because otherwise you will be leaking states about other processes and worse yet the kernel on this. So from a standpoint of both security and stability, always make sure that you zero stuff. Okay, thank you people. Wednesday, Jeff's going to be going over paging policies, midterm quiz Friday, assignment 3.1 next week.


Mindfulness is being present, being in the moment. It is observing our thoughts without becoming them and obsessing over them. You don't need some Zen mountain retreat to practice mindfulness. You can begin to do it right now, today. Mindfulness enables us to break the continual on loop old stuck narratives about our lives and about life and gives us a little space of separation, a degree of separation from that loop that we get stuck in and mindfulness perceives it and creates a chance to step away from it even to a small degree. I'm going to be speaking and teaching and holding a discussion on mindfulness in my next online live mentorship session. We have my mentorship trial from all over the world very soon. Dates are going to be the link in the bio. Come and join me. I love you to join us for this session on mindfulness.  See you there.


So, I'm a soil scientist and agronomist from Seirad. I've been working in Zimbabwe for seven years now. I mainly work in agroecological region two and three, so I don't have much experience in this kind of very arch ecosystem, but I'm very happy to be here to give some insights about what is soil else and why it should be fully considered in the one-else approach. So I'm sorry, I will start with a definition, what is a soil? So soil is defined as a biological active porous medium that has developed in the uppermost layer of the Earth's crust. So it's basically made of five main ingredients. So you have the minerals, sand, silt, clay. You have soil organic matter, living organisms, gas and water. So as you can see here is average composition of soil at the global scale from FAO. We can see the weight composition, most of the mass of a soil of course is made of the minerals, then you have some water and you can see that the organic matter is a very small percentage of the mass of a soil. Here 3%, but actually many soils in sub-Saharan Africa is much lower than that, less than 1%. So 3% is already quite good soil. So how does a soil is formed? So soil formation is called pilogenesis. So everything starts from a parent material rock and then with interaction with climate and biological activity you have some physical and chemical weathering and with time you have the development of different soil horizons. But it's a very, very slow process.  It takes about 100 to 1000 years just to form one centimeter of soil. So you now understand that soil is a non-renewable resource at human scale and the problem is that currently we are losing soil at a much faster rate than it is formed in many regions of the world through erosion or through bad land management practices. I just want also to mention this citation from a very famous soil scientist. I think it's a very good summary of what is a soil and what are the importance of soil. So essentially our life depends upon the soil. There can be no life without soil and no soil without life. They have evolved all together. So now when you are talking about soils actually we are talking about a huge diversity of soils. FAO has defined 30 major soil types but actually there are many, many more, many subgroups but these are the major ones. So you can see different colors, different horizons and of course all these soils have very different properties, very different characteristics. Some are suitable for agriculture and others are not at all. So now at the global scale here is diversity of soils so you can immediately see that for example in sub-Saharan Africa you don't have the same soil that we have in Europe and this is because of initial rock parent material that was different and the interaction with climate that has led to very different soils in different contexts. Here just a brief focus on Zimbabwe. We could think like looking at the landscapes it's all the same kind of soils but actually it's not at all. There are many, many different soil types even in Zimbabwe as you can see here on this map. So it's important to keep in mind this diversity because when we are talking about soil else we are not talking about a unique soil else. It's always context specific and soil else is also defined according to the type of soil that you are dealing with. Okay so now if we look at a larger scale, so this is the volume occupied by the earth. If we were able to squeeze all the atmosphere in a bowl, this is the volume that will occupy the atmosphere. Same for the water, all the water of the world, oceans, lakes, etc. They could be put in this bowl. But now if you look at soils, this is the volume that occupies all the soils of the world. So you can see it's very, very small and actually this small volume is then spread on all the continents, on all the land. So it makes a very, very thin layer from a few centimetres to a few metres depending on the context. So it's a very thin layer almost like a skin, very fragile. But despite of that, it's actually the support of so many ecosystem services. These are the seven main soil functions that are usually mentioned. So of course soils are involved into biomass production, forestry, agriculture. Huge role into storing, filtering and transforming nutrients, water. It's a huge biodiversity pool. It's also the physical and cultural environment for human activities. It's a source of raw material and it's a huge carbon pool as you will see later. And it's also an archive for geological and archaeological heritage. So now if we look at this figure here on the right, you have the soil science in the middle. Then the following cycle are the seven soil functions that I have just described. The following cycle is ecosystem services that are emerging from these soil functions and the outside cycles are the SDGs, so the sustainable development goals. And you can see the ones with the red dots are all the SDGs where soils are involved. So actually soils are involved directly into 13 out of 17 SDGs. So what does that mean? That means basically it's impossible to achieve these SDGs if you don't manage your soil in a sustainable way. Just a quick look on the importance of soil for climate change and on the global carbon cycle, you can see that the carbon in all the soils, it's huge, it's actually three times more carbon in the soils than in all the atmosphere. And much more than in all the vegetation, all the forest, etc. So I will not focus more on the fluxes and on the 4% initiative we can come back after if we have questions, but there was this initiative launched at the COP21 in Paris about soils for food security and climate, and it is directly linked to this importance of soil for climate change mitigation, adaptation, and food security. So now I'm not sure how many of you know that soil is actually the most biodiverse singular habitat on earth. These are the few pictures of animals, microphones that you can find in the soils. And soil is likely home to about 60% of all the species that we have on earth. So it's absolutely huge. And of course it really depends on the soil groups, on the fauna groups, sorry. So like for example the fungi, most of the fungi are found in the soil, plantae also growing on the soil, but you can see also for bacteria directly related to earth, half of the bacteria are found in the soil. And viruses, significant amount also of viruses are found in the soil. So now just to continue on this biodiversity aspect, if you take just one handful of grassland soil, this is what you get, 50 kilometers of mycelium, so 50 kilometers of fungi, 100 billion bacteria from 10,000 species, 500 meters of plants roots, then you have also protozoa, 100,000 of species, nematodes, algae, etc. So it's absolutely a huge biodiversity. So of course all this soil biodiversity, this soil microbiome is very much related to the one earth concept because soils is actually a source but also a reservoir for pathogens but also for beneficial microorganisms. So as you can see on the picture on the right, you have all this pool of microbes in the soil and then part of these microorganisms are found in our gut, humans, animals through the feed and food, and also in the roots of plants. Plants are able to actually recruit their own microorganisms through exudation of molecules in the soil. So all these microbes found in the soil, they are directly related of course to plant pathogens, plant growth promotion, also antibiotic resistance, bacteria, soil-borne pathogens, etc. So yes, soil health is defined as a continued capacity of soil to function as a vital living ecosystem that sustains plants, animals, and humans. So it has actually emerged from a previous concept that was called the soil quality concept but this concept was much more focused on humans. As you can see, the soil health definition includes also plants and animals. So it's much more inclusive for planetary health. And actually very recently in the last year, there is another concept that has emerged. It's called the soil security concept and in this concept, soil is seen as a common good similar to water and to air. And it's very much used in the policy context because it states also that soil, well, access to soil ecosystem services should be on the same level as other human rights. Okay, so now how we do measure and monitor soil health? There are different types of indicators that can be classified into three main categories. So you have the physical, the chemical, and the biological indicators. You have a list here, so it's very small on the right of very commonly used indicators classified in these three categories. And these indicators are designed in a way that they are informative, so it means we should be able to interpret them. They should be sensitive. For example, you should be able to see the effect of land management or land use change. They should be effective or practical and they should be relevant. And after all these multitude of soil health indicators, they are often integrated into a single index that we call the soil health index. And so for example, the soil health index of a crop land is then compared to a reference soil health index. So usually it's a natural ecosystem on the same soil type, same climatic conditions on the region of interest. I mean, so just to mention that there is no universal soil health index. It's always context specific. So here is a review done by colleagues recently. They analyze different soil health assessment schemes so far. And you can see that in these schemes, most of the time they use chemical and physical indicators. So this is due to a historic focus on soil health indices for crops, crop production. And you can see that so far there are very few assessments that include the biological indicators. So if you wanted to have a more comprehensive soil health index, of course we should have a more balanced set of indicators with at least 20% of each of these categories. And sometimes you might need also different types of indices for different ecosystem services that you want to assess. For example, if you want to assess the capacity of a soil to conserve biodiversity, of course you will have more biological indicators. But maybe for crops, you will have more chemical indicators for climate, maybe physical and chemical, etc. So at CIRAD actually we have developed a tool to assess soil health. It's called the biofung tool. So using also these three types of indicators, chemical, physical and biological, and it has been designed in a way that it is a low cost test and it can be done in situ in the field. So basically you have a list of different analyses that are directly performed into the field and then you are able to rank your soil and compare it to another management, for example. And here is an example on your right, for example on the right you have the forest compared to the left the cassava crop. You can see of course the forest has a much higher index compared to the cassava crop land. So now I just want to mention that 95% of our food is produced on soils, of course, but about one third of the world's soils are already very much degraded. So there are different types of degradation. So it can be biodiversity loss, salinization, nutrient imbalance, compaction, stealing, pollution, acidification, erosion, loss of organic carbon. So there are many, many types of degradation. But there are also some solutions. So the first of all of course is to protect the intact land. Then we need to better manage the crop lands and restore the native cover. So for crop land management it can be achieved, for example, through agroecology which has been defined by FAO by 13 principles. Some of these principles are at the agroecosystem scale like recycling, input reduction, etc. And some other principles are more at the food system level. So now what can we do in crop lands to improve soil health? Well here is just a study we performed recently looking at soil organic carbon which is a very common indicator of soil health. And we look at different practices. And you can see that there is quite a number of practices that have a very positive impact on soil carbon. So it can be biochar, intercropping, manure, rotations, mulch, agroforestry, cover crops, integrated soil fertility management, reduced tillage, etc., etc. So there are a lot of options that needs to be tailored to the local context. So now as we are in one gates my last slide we can wonder what is the impact of conservation areas on soil biodiversity and soil health? We know the impact of course on elephants, on lions, but what is the impact on soil health? So there is actually a huge ongoing project which is called the Soil Biodiversity Observation Network involving 80 countries globally with a lot of countries in Africa as well. And then we are comparing the same land use inside the conservation area and outside the conservation area. So for example it can be a forest inside the conservation area and just outside the conservation area. And then we are taking soil samples of course and we analyze a lot of parameters and we hope to have the conclusion that conservation areas are beneficial also for soil biodiversity and health. I don't have any results so far because it's ongoing but I hope we'll have good results. So a few take home messages. So I hope you are not convinced that soil health is a critical part of environmental health. In my sense it should be fully considered in the one health approach as it is also critical to achieve the SDGs. Then there is no universal soil health index, it's always context specific regarding the climate, soil type, etc. And also depending on the ecosystem services that we want to consider. And there is still a lot of research ongoing on this to have better indexes. So also I think there is a potential for win-win opportunities for different challenges that we have at the global scales like climate change, mitigation, biodiversity loss and what else. I think all of this can be linked just by protecting our soils. So thank you very much for your attention.


Lao and I'm a software developer. I really like elixir How many of you like elixir? Yeah, and I have done some open source a lot of elixir Stuff and one of the things I've done is calendar library and also time zone T-set data library and I have a blog where I've written a lot about Time and date and that kind of stuff See it's creative deletion the calm and I'm also on Twitter and github so First of all, what is a type? There are a lot of definitions of it one of them is that it's a set of possible values And why do we want types why even have them? So again, there's a lot of reasons to have types a lot of Opinions one reason is that for instance if you have five divided by two you get something meaningful But if you have five five by Orlando that doesn't make so much sense so in that case we want errors and That's a good reason to have types that if something goes wrong we want to know about it and I'm gonna talk about explicit data Do any of you know what this is? Well, it isn't anymore That was that the Mars climate orbiter and it went along towards Mars on a 286-day journey and Then it failed and Why was that? because there's some people that was supposed to send data about how it was going About some thruster data and they used pound seconds instead of Newton seconds and I'm used to the metric system, but I really like the imperial system because it's very convenient I like the whole thing with like feet and inches based on your body. So Vince my hide I found out just recently that Converted to the imperial system. It's One foot one leg one torso one neck and one head So it's very easy. I like that but It's not really necessary that important What units you use as long as you know which units you're using? And in Physics class in school. I was taught that you always need to carry your units with you when you do calculations So at the end, do you know what you? What you have because if you just have numbers At some point something might go wrong. You don't know what you end up with so imagine if They're probably just sending numbers But imagine if with this example of this spacecraft, they'd send numbers and the unit Maybe as a top like this in elixir or maybe as A structs and then you could do pattern matching and so imagine that You required to have Newton seconds and then you you sent Pound seconds instead you get an error, right? So imagine if you had they've been doing testing with just integers It would all work and they were like this is ready to go to space But if they use this instead, maybe they would have seen the error Because one thing is to have something in documentation. Another thing is to have code So with pattern matching and testing You get errors, which is good. I really like errors It's a bit like, you know, like pain. It's not so nice But if you have your hand on a hot plate, it can help save your hand. So There are a lot of units of time and date We have years we have years and months and we have Times we have data and times together and they know time together in a specific location So how's that handled in? computers often For instance in Unix, there's just one type which is A date a time in a specific time zone, which is always UTC so if you have a date it gets jammed into this concept, which is a Date time in a specific time zone if you have a time it gets jammed into the same concept again and Even if it's a daytime you always get that Time zone attached to it so then you run into a problem of If you have you want to encode just a time and you ask afterwards what years that time in and There might be some random year that was never there in the data originally But somehow it got into your system Because it has to be there because it's kind of forced into this concept of a daytime Even though it was just a time And the same thing for just dates I wrote also blog posts that Everyone knows what a date is except programmers because For most people including the dictionary if they can be Time sorry, it cannot be a time can be see even I'm confused So you have you have if they can be a year a month or A year in a month or a year and a month and a day of the month But it doesn't really have a time So I think in general in computer science and programming a lot of these concepts get confusing because They're made more confusing that they really are So instead of having just one one type We can have multiple ones and so I made this library to the county library where there are four different types and you could have Many different ones you could have for instance a type just for a year But in this case There's no time for just a year for that you would have to just use an integer But the ones that are there are ones that are commonly used and They also fit well with for instance Databases in a database that date has a year a month and a day of the month So let's look at them Yeah, so in 2014 you had encounter You haven't so have these these types A date a time they're pretty simple Like what you would expect them to be and then an a daytime is a combination of a date and a time and then you have a Date time so let's see here date time And they also have a microsecond And here's the combination So the date time is special at date time is the result of a calculation where you also have time zones Into it. So let's say what is the time right now? in New York, which is the the time zone we're in right now here in Florida as well It's like an ephemeral Struct that is based on also the current time zone rules So in addition to what you have in an a daytime you have a time zone To see offset a standard offset and so on abbreviation So the the reason I say is ephemeral is because the rules can change so you ask the library for a daytime and and with the current rules the standard offset are in one way, but if you Do the same calculation in 10 years? Maybe it's different because the rules were changed So something about the the naming in In some languages you have something called local in Java this is the Description of a local day date in Java. So if you have 2007 December the 3rd that is a local date so why not call a Date in elixir the local date. Well, it turns out that This is another case of things being more confusing That they should be because actually local means something that belongs to a particular area or a neighborhood or something that that is has something else then it's not global Right, it's it's local so Then if you ask yourself 2007-1203 Which area does that belong to? Yeah, I don't know either like All right, it's it's it's not a local thing It's just a more like a global date Exist without any tie to a particular place So that is why it is called Just a date. So here's a quiz Does anyone want to try there? No lifelines? Anyone want to get it's a it's a trick question And then the hands went down Right, so a lot of a lot of bees here The answer is that We can't know What the answer is here and what why is that because we do not know What time zone it is so Eunice Unix time is always in UTC so if you have this string of 1970 January 1st at midnight You don't know what the Unix time is the equivalent of that unless you know what time zone it is in Yeah, so so you like you like errors. I like errors too. So The way it works right now in in elixir 1.3 is like this you can do this and you get an error Because it doesn't match if you have this is not naive I'm gonna you're gonna see more of that like it's a sickle for a naive daytime and It doesn't match if you try to Pass that into this date time Function it needs a daytime that has as we saw before it has time zone information So these are the structs. They're built into elixir 1.3 So they're Pretty much they're almost the same But the difference is that there is no Calendar propended in the module names that does one of the main differences and We also have sickles for date for time and for a night daytime and You might notice that there's no sickle for daytime So we have it just for the three of these So one of the the changes from the calendar library to the To the structs that are built into elixir 1.3 is that we have this field called calendar Which is so that if you want to use a different calendar than the ISO calendar, for instance a cop take calendar Something like that. You could you could use the same struct But there's no functionality for that built into elixir. So Some external library would have to To do that kind of calculation, but the good news is that most of the world Except like one country uses the ISO calendar at least for business purposes so for for most of us we don't have to Most of us don't have to Use any other calendar So another thing we have is micro seconds That is and there was that in the calendar library as well But this is a new thing where you have a couple of two integers so the first Element is the number of microseconds and the second one is the Significant fractional seconds so a cool thing about that is for instance if you have Here the first example you have point one two three four and in many other libraries Maybe the result you wouldn't know that you only knew about these first four Decimals so it would be converted to point one two three four zero zero zero Or two zeros and and in the second example you see that when you parse this String again, we only have dot one and we don't get dot one zero zero zero So the way it works is if you if you get the microsecond Field you can see that this is how it works. So in the second example for instance you have two significant Decimals, so that's why it says to There's a hundred thousand microseconds and then two significant digits so in order to With the sickles and the built-in Modules in elixir You can do some things But you can't do much with time zones for that you need You should add this to your dependencies and then you can for instance you can do stuff like Converting and I date time to a daytime so if you remember remember the example from the From our little quiz That We didn't have a date time To pass to the date time to your next function, but in this case we explicitly Transform this late daytime to a daytime in we say it's UTC like if you know that what you have here is UTC you can transform it and then Functions that only accept Date times that are not made date times. They can use it So here's an example of a function that's built in so we see there's no calendar propended to the module and We can pretty much the only thing the only way you can create a date time With the built-in library as of now is from a UNIX integer, but if we If we then have calendar there's a way to shift that That daytime to a different time zone so you can notice that The hour here was 17 hours and when we shift it it's 13 hours and then here's an example of How you can You can combine combine the different Libraries you can like the building library returns a daytime and then you can shift it and then because you get the same struck back you can use a function that again expects daytime and you You see you get a string back and here's another example where you have a string for time and Yeah, again, you can notice that there's combination of the different Different modules and here's a sickle We can ask if it's Friday and here we can compare if today is before tomorrow and another example For more examples and here is You can get the day today when you provide a time zone I think you should notice is When you ask what what date is it today you could have like a function that just says Today and then there's a default, but that's not really a good idea because If you don't if you're not aware of What you have like if you're not aware of the time zone you're not aware what the default is you might get confused so that's why it takes a specific time zone and then Here's yet another example and this is we can add dates to a date from a single and then call it here So what about getting a time zone of the user?  The thing is For instance, if you're in Phoenix You and you have some web app with the JavaScript You might use what you can get is you can get an offset but JavaScript Doesn't really have a way to get the canonical time zone of the user There are some libraries that Can get can can guess Make a qualified guess about it So what you want is for instance something like America slash New York Instead of let's say minus four hours, so there's a bunch of ways you can also You can use those JavaScript libraries and hopefully I hope that in the future That in JavaScript is going to be a standard where you can just say what is the time zone of the current user another thing you can do is you can You can use like geo IP so there are ways that you provide an IP address and you get back Iana times ID again like America slash New York So if you have a web app You could try to do all these guesses, but really the best way is to have a drop-down and the user can choose Can choose the time zone so persistence in Exile so she was if a lean told me that in the plan is that in version 2.1 There's the these structs are going to be built into ecto But for now what you can do is Use the the collector library so you add that and then for the for the types in your schema instead of having for instance ecto dot Date you can have collector dot date and there's a special There's a special thing for if you use postgres You can have a special postgres collector type where there's a Composite type that has This is for a daytime that has the time zone the local time and then an offset And the good thing about that is that if there are any changes to the rules, let's say for this That this Talk was scheduled a year ago, and in the meantime there have been changes to the rules If you had used this type it would still go and ask When you retreat retreat this from the database again after use the rules have changed it will it would use the new rules So that is Whatever it may come in to do So the the main takeaway here is that you want to Select a type that fits the data you have so if you have some Data that it comes in you want to preserve as much information about that as possible But you don't want to make up new things And and one thing about the new types is for instance one thing I would like to see is that let's say you have a log Currently the logging in elixir uses tuples, but it doesn't have any time zone information So you could configure your logging to be local time or to be utc but when you actually get that That information you don't know what it is So this might be passed on to your application a bunch of different places, and then you have a function That received this and you don't know what time zone is and it isn't so that means that you can't do There's a lot of things you can't do you can't compare it to another time and see what happened first right so in This case I would like to have Logger use always use utc like what one thing I think that people that work in in Computer science, especially sys admins and also programmers You should all know what the time is in utc and all set the service to utc like one thing is if you have Finishes your schedule a talk or sketch a schedule of conference You can't expect people to use you to see and I wouldn't want to either. I think time zones are very useful But for for things that happens on computers you You would want to use you to see because of the issues that if you use let's say you have a cron job in Local time you have your server set to something else in utc your cron job might never run it might run twice So come back to the elixir locks I Hope that in the future they will be in the native day time type and then you can do all kinds of things you like you have in your lock You can actually compare it to other day times and you can find out where it actually was When it actually happened so the whole idea of having these structs in elixir is that for instance in ecto you can use the same type as in other In other libraries and it can all work together so the in the speaking about the future of these of calendar types in in Elixir you can see that right now there's The functionality is kind of limited and you need an external library The hope is that in the future in future versions of elixir. There's going to be a Way to have time zone data And time zone data functionality shipping with elixir either you would have to add the time zone data library or would it would ship with elixir but if you right now you could use the the counter library and It should be in the future an easy transition when it gets when it if it hopefully gets built in This kind of functionality Yeah, that's a that's a that's a very good example Yeah, actually at first I did some start doing some code to handle those Structs or sorry those tuples and it was confused because ecto used microseconds and logger used Miliseconds, but that that's a very good example of Why it's these new Types are good because you could use either naive day times or day times and at least that the The micro the whole micro my millisecond thing would be solved right so we have some extra time and I would love to Answer any questions before we move on to the questions. Let's just give now a big round of applause right questions The country that does not use the calendar the rest of the world is that Thailand or the Buddhist calendar? Or is there another one that I don't know about Yeah, I I think it might be Ethiopia That's I think that's the one I was thinking about To correct myself. I think they do use the same calendar as we do but the year is They had 500 something to the year because they count since the year of Buddha's death Okay, so I guess Yeah, so So there are a lot of There are a bunch of places that have Other calendars than the ISO calendar But for business purposes basically the entire world when doing International business They use the ISO calendar So and sometimes you use I mean even Like in for Like for many holidays, they are calculated based on lunar calendars or other things but still in and Daily live for many of the other things we use the ISO calendar Hi First I just want to say thank you for working on this I know this isn't like the most obviously sexy thing, but it's so painful without your work So the question though is you mentioned The that there was an Ecto adapter like the Cal Ecto. Yeah, Postgres. Is does that apply to my sequel too? Or is that yeah, so yeah Collector works for all the databases. It's just that there's this one special Case where if you want to save at daytime Like a daytime that has the like the daytime time you have in an elixir that has all this The time zone information with it you can use this type so if you don't use Postgres what you can do is You're kind of forced to save the save it as a naive daytime And then what you can do is you along with that you can save the the time song So you can use it Yeah, you can use it with any of the the databases Sure, I just want to say real quick Fred Hebert's talk at Erling factory this year history of time was awesome and goes over all that stuff about Lunar calendars, right? Yeah, I have to watch it. It's Yeah, also If you're more interested in this kind of stuff, you can also see I did a talk last year And it's on it's on YouTube and it also talk about stuff that like Tommy clocks and Planets and stuff still up still have nice mess Some things I learned from Lous talk last year. It's well worth looking into Well More questions see a hand going up Hi I've been using time mix in combination with my sequel and run to a number of problems when doing tests, especially with Microseconds, mm-hmm because I take a date save in the database and when getting my record out again It would have lost the microseconds does Caliq to Offering help here or? Yeah, what what you can do is when you If you use collector the way you saw that is there is an option Where you say that you want to save the fractional seconds and then that gets Saved into your database and you get them back again You don't want it So the value that's returned from the database Has no microseconds, but the value that I initially sent to the database had microseconds, and I'm comparing them of course my testing fail Okay so you you you save Something with microseconds, but you don't want them to be saved or well Well, it's very simple. I create I just take the for example the current time. Yes I save it to the database and Later on in my tests I get the record out of the database and verify whether the date is the one that I send But the date that I send and microseconds and the one that I get from my sequel has no microseconds So my test fails Right. Yeah in this case It should if you configure it to save the microseconds you get you have them both Both places so they should match Yeah, but it's the database that's used in also in the real system and they're not allowed to have a microseconds for other reasons So it's just wondering there was a way to get rid of the microseconds. Yeah, you can Ask me after what's and I'll and in general Anyone also that wants to talk to me about this stuff find me after this and Think we'll type for another question. Okay then we have applause


Martin, thank you for coming and joining us today. Welcome to Eden. Your keynote this morning was about learning networks. Now, obviously, that's very vital for the future of education, but just how vital is it? For the future of education? Well, I thought you were going to say how vital is it for you? For me, networks are everything. I don't think you can do anything on your own anymore. So, for me, it's network learning is about creating a social web around you, if you like, sort of have access to people who you can talk to, who you can share issues with, who you can do things together with and throughout the presentation also try to make clear that education, of course, is also transforming much more towards this sort of active learning where students are joining networks and also learn how to reach out to, if you like, professional learning communities, which is already happening a lot, especially in higher education and things like that, which also means that you need to be able to validate the kind of information that you receive or the debates that you're taking part in. But in terms of educational future, I think it's very important to learn and teach those learning and thinking skills in order to participate in the debates and being able to contribute. So, for me, networking or communities or any social structure really is very important part of education. Is there a difference between a network and a community online? Online or not online? Well, you could say that every connection is sort of a network connection, but not every network is a community, so you could buy those kind of rules, if you like, you can make distinctions, of course. To me, networking is far more fluid and open and people maybe come in and out, whereas communities or communities of practice have maybe a longer tradition and a longer focus on a particular domain, as Ichen Wenger would probably define it. And also, the community itself may have developed some rules about how to engage and how to be a member of a community. So, a community you could see maybe, if you like, as a very stable kind of network in that respect. Now, Howard Reingold, I'm sure you've heard of, has written several books on this subject. I'll quote him. He said that being networked in the 21st century, that's the most important skill we need. But there must be other skills as well that align to that, aren't there? You mentioned some this morning. What kind of skills are we talking about? Well, I think you can sum them up probably by these 21st century skills, which are a lot talked about nowadays. Of course, being networked is very important, but also you're an individual in a network as well, so you need to be able to digest information. And in terms of the 21st century skills, you need to be a collaborator, a creator. You need to be able to work things out together. You need to be able to communicate. There are quite a lot of skills around it. But of course, these skills are used in a social setting as well. So, yeah, I think it's about the ability to learn and the ability to communicate about what you're learning are two very important things. Do you think we can teach those skills? Yeah, I think we can practice them and we can improve them. But yeah, I think you need to be very open. You need to have trust amongst each other that you can say and feel that you can say whatever you like. So, these skills reside in a culture if you like. And if the culture is not conducive to these kind of things, then you can teach all you like, but it's not being put into practice. So, I think that's also what I tried to say this morning. Before you can start thinking about networks or communities or stuff like that, you need to develop a culture in which those social structures can flourish. So, that's very important to me. So, talking about social structures, I'd like to tell us about the crowd. Sorry, the crowd. The crowd. As you mentioned this morning. Yeah, the crowd is a very nice initiative. It's an initiative by teachers, for teachers. It's a kind of association, I believe, where teachers who are interested in learning from and with other teachers, they can pay a kind of a membership on a yearly basis. And it gives them access to the crowd.  I don't know really what the difference maybe is in this context between the two concepts. But the idea, of course, is that you join a larger group of people who are interested in working together, interested in hearing about each other's practices. And I think that's a wonderful step towards stimulating teacher professional development from a sort of a more bottom-up approach. Because a lot of professional development is sort of still dictated by organizational future scopes or whatever, you know. And I think it's very important to stay in touch also with the issues that teachers encounter in daily practice and develop a culture of learning around that. And I think the crowd is doing a very nice job of setting that up. One of the issues that they do encounter, and you mentioned this is a wicked problem, I think, was the term you used, was the lack of discoverability of some of these networks. They are invisible until it's somehow flagged up that they exist and then you join them. But how do we get over this barrier of the networks being invisible and not discoverable? It's a tricky question, you know. In my work I tend to make a bit of a distinction between sort of what's happening formally and what is formally accounted for and stuff like that and what do people do spontaneously through self-organization and things like that. And I always make a comparison. Like if you look at an organization from that kind of organizational chart everything is very clearly put into boxes and people have assigned tasks and things like that. So the flowchart of the organization is very well formally established. But then if you look into daily practice what it is that people do and how they organize themselves more in the informal daily routines they don't follow these boxes, they don't follow these charts. You know you phone somebody or you send an email to somebody who you know and trust that they can help you. So this informal reality within organizations and things like that is not really formally recognized and therefore also not very utilized. So what I try to do is to create within the formal structure more space for this informal self-organization. And by that I don't mean to formalize informal activity but I just want to create more scope for it. And then of course the critique you often get is yeah because when you do this you make it more visible. People you know they may not want to engage with it and I think that's fine. The informal can even go further informal if you know. So you can have formal, you can have informal formal and you can have truly, truly informal if you like. So I think there's different levels of scale that you can work with. But in terms of the visibility or the invisibility aspect of it all if you don't know what is happening around you it's hard to engage, it's hard to be a member of it. So that's why I think it's important that these networks that are spontaneously organized they get a voice and they show what they're working on. Not only just for the benefit of themselves but also for the benefit of the organization or the domain or the professional identity. Well thank you for giving us a nice insight into your research and into network learning as it stands. Martin, thank you for joining us today. Yeah, you're welcome. Thank you.


I didn't put Venom officially on this list, but if you want to know where that one stands, I'd put it at the end. That would be the last one. I own The Amazing Spider-Man 2 on Amazon Prime Video. Some may say, Adam, you have kids. You're doing parenting wrong.  Of course, by that logic, people would say, well, what if he enjoys math? You're gonna let him do that. For the rest of his life? And I hear you. I hear, I hear and understand you. It was a bad purchase. It's a tough watch. But I do like some of the action. I like that slow-motion CG work going on. I also like the GoPro-style shots they did while he's web-swinging. It gives a very visceral feel to it all. It really does make you feel like you're swinging around the city as him for a little while, and it's something the MCU Spider-Man doesn't even bother attempting. Up next is The Amazing Spider-Man. Oh, well, you're putting this above Spider-Man 3, Adam, this is crazy. The show's called Hot Takes for a reason. The Amazing Spider-Man gives us a decent Andrew Garfield Spider-Man. It gives us a solid Emma Stone Gwen Stacy. It gives us a very nice Aunt May with a Sally Field. I wasn't feeling the Lizard Man. It felt like a retread of the Green Goblin, and I really wasn't into the tone, which was a lot more Batman-esque, mostly night fights, a slick city. It just felt really uninspired after the Raimi trilogy. The music isn't there at all. There's not a lot of standout scenes outside of him, like on a web in a sewer, which is cool. It's a cool shot. And there it is, Spider-Man 3, everybody's favorite to make fun of. It's a disaster. It's a glorious disaster. Chock full of some amazing moments, some fantastic cinematography, and then we have that 70s show, Eric Foreman, Topher Grace as Venom. For all of maybe eight minutes, it's miserable. It's just terrible. Sandman's cool. The effects are cool, but they forget about him halfway through the script, and then he comes back in, like, the final five minutes, too. It's just like, this movie just is all over the place. Emo Peter Parker, he's made fun of to death. The best thing that's come from Spider-Man 3 are the animated gifs. It's not all bad. You could see a good movie in this. There's probably a fan edit, probably by Topher Grace, that removes all of his own scenes and just focuses on the Sandman and the Green Goblin. Although that pipe scene is cool, where Spider-Man slam it down the pipes and ringing them all and messing with Venom's head. It's a glorious disaster. This is going to seem far down in the list for a lot of people. They might have it at number one, but Spider-Man Far From Home is pretty much in the middle for me. I enjoyed Far From Home. It's really funny. It's got a great cast. Tom Holland is just as lovable and likeable as he was in the first. There is some decent action, although it's not the most exciting thing to watch him fight drones for five minutes in the air. Fighting bad guys with guns is just cooler to me. Coincidentally enough, the scene from the trailer where he fights bad guys with guns is not in the film. Jake Gyllenhaal is always fantastic though and here is no exception. He's great as Mysterio. There's some cool twists and turns. You kind of see it coming pretty much right away. You can guess what's going to happen pretty early on. I do like how the MCU now is just this big layered cake they can pull chunks out of and reuse later and say, hey, you remember that from the earlier film? That's coming back here in a bigger way. My problem with this film and with Homecoming is it's a bit too light for me. It's a personal problem. I know that. You know, if you're looking for a Ferris Bueller's Day off type, this will fit the bill. It's a great comedy. It's a buddy road trip comedy type of affair. I just like Spider-Man a little bit more dramatic, a little bit more bombastic. And yeah, I didn't read the comic. This is just me as a movie goer who liked the Sam Raimi films. Who wasn't ever really a big Spider-Man guy prior to those. And it's almost trolling level in terms of how it goes out of its way to be very different than those Raimi films. Whether it's releasing the identity to an important character or just the way they handle smaller moments. In a Raimi film it would be done with a huge orchestral soundtrack in the background. The camera would be whipping all over the place. And in Homecoming and Far From Home it would just be a single shot. The person's like, yeah, yeah, I know that. I was trying to not spoil things, but it's in the trailer. So if you avoided all trailers and for some reason watching this before seeing the movie, that's on you. But in the trailer they reveal that MJ finds out that he's Spider-Man. It's right there. They care so little about such a dramatic moment that they just put it in the trailer. And it's a cliche, I know, but big reveals are big reveals for a reason. You want to build them up. You keep that identity secret for a movie or two, not just because the character is, you know, protecting those around him, but also so that the audience gets to revel in the astonishing looks on people's faces when they find out the truth. Spider-Man Homecoming isn't the first time we're introduced to Tomahawk and Spider-Man. That was in Civil War. Although this was the first attempt to really make us fall in love with this character, see why he puts on the suit and goes to school and does all the things he needs to. They do an admirable job here. For the most part this is a very fun, lofty affair where he's just attending school, going through boys being boys' situations. Things do get a bit more personal and intense when it comes to the villain of the picture, Michael Keaton as Vulture. I've said this before in multiple videos in the past because YouTube's not anything if not redundant. I also like the Raimi movies and even the amazing Spider-Man movies when it comes to his powers. I know he didn't normally have organic webbing. That was something Raimi brought to the table. I like that. I like the idea of him having more abilities outside of just having super strength and spidey sense, one of which wasn't even really used at all in Homecoming. It's assumed he has spidey sense, but he does a pretty poor job of using it properly, which was another bizarre hang-up I had when it came too far from home. They kind of do the Spider-Man 2 route there also where he's having a hard time using his tingle ability. The problem is they never really showcased it in the other films. I mean, not to the level where it was a cool power. You know, people online and what in passing are like, oh, he had it. Yeah, yeah, it's just not in your face. Well, I want it in my face. It's a cool power. You also have the Tony mentor relationship. You got some great behind-the-scenes footage where he's filming some of the stuff.  So yeah, it's got plenty of good on its side, and I do really genuinely like most of the cast, if not everybody. Still the number one spot on a lot of people's lists for good reason, Sam Raimi's Spider-Man 2. And I think a big reason for that and one I was just talking about was the reveal at the end of that film to Mary Jane that Peter Parker is Spider-Man. Kirsten Dunst no doubt doesn't get a ton of opportunity to show her acting range in these three films. But when she gets that opportunity, whew, she shines. The expression on her face in the final moments of the film when she looks over and sees her boy Toby McGuire holding up that steel is nothing short of brilliant. You heard me. It is an amazing scene, one of the best moments in a Spider-Man film to date. Not only that, but we got Dr Octavius, who steals the spotlight as one of the best villains in the franchise. Insane action sequences, a musical score that is out of this world, and some genuine conflict that for some reason people on the internet I noticed have failed to understand fully. Time can be a fickle, cruel mistress. It's odd how now we look at Spider-Man 2, not we some people, look at Spider-Man 2 and say, How did he lose his powers again? Because he was mopey and depressed? Yeah, that's a dumb way to summarize it. Let's play back the tape, shall we? He previously had just murdered his best friend's dad. He's working a dead-end job while trying to pay his bills and go to college. He's at the most challenging and critical times of a young person's life. He's got girl problems, which all of us can relate to in some way or the other, either by an actual girlfriend or your mother whose basement you're living in. And I feel like I'm forgetting one thing. Oh yeah, his uncle died in his arms. So yeah, he's a bit bummed out and his powers maybe aren't working the best. In short, Spider-Man 2's awesome. Sam Raimi's Spider-Man is the movie that sold me on Spider-Man. As I've stated earlier, I wasn't into the character. I didn't really know much about him at all. So when I saw the movie, I went in pretty low expectations. I'd already got my X-Men film, you know, just a few months before that. So I was set. I was set for life. If I had to go, now would be the time. It was clear that Sony had invested a lot of money into this, a much bigger budget than X-Men and it was on display. The camera work was unreal, watching him zip through the city with ease. The characters were cartoonishly fun like J. Jonah Jameson or Willem Dafoe as Green Goblin. Has it aged? Of course. The actors in this film look like that of Saved by the Bell, meaning 30 year olds playing what is supposed to be 16 or 17 year olds. One thing that is timeless though is story. It has a feel to it, a tone that you just don't see in other movies still. Much like a Ghostbusters or an Indiana Jones or the Matrix or Kill Bill, there's a stylistic vision to it. And those Raimi films have that. Homecoming on the other hand or the amazing Spider-Man, they don't. They feel very manufactured. Although there is one scene that I forgot to mention in Far From Home where Mysterio gives Spider-Man a whole bunch of weird visions, that is cool. That's some Doctor Strange level stuff and I wanted to see like an hour of that alone. To be a broken record, I'm all about that spectacle. I'm all about that big blockbuster. And same Raimi Spider-Man, that provides it. I'm not like sold on this pic for number one. Into the Spider-Verse is an amazing movie. I really like it. I also really like the first two Spider-Man's. So like the top three is kind of a three way interchangeable tie for me. This one like caught me off guard. I did not think it was going to be near as good as it was. Especially when the trailer is, you know, we're so herky jerky because of the animation style being so different. I just didn't think this was going to be that good. And I was completely blown away by it. Miles is a great lead. Plus you have all the callbacks to the other Spider-Man movies, which once again, it kind of makes sense that I put this in like a three way tie. Because without the Raimi films especially, you wouldn't have some of these tips of the hat that make this film so fun. They wouldn't be able to play off of it. Of course, they go further. They go back to early issues of the comics. They even bring in, you know, Spider-Ham and all that stuff. I think they said they're making it into the Spider-Verse too, even though the first one didn't do like as big of numbers as they would have liked. So I am very optimistic about that. So far, Sony, I mean, oddly enough, is doing well for the most part with Spider-Man. Into the Spider-Verse almost kind of makes me mad at Disney too, where Disney has basically abandoned traditional hand-drawn animation. Into the Spider-Verse is like, look, you can do some really cool stuff now way better than you could have 30 years ago, or even 20 years ago, or 10 years ago. And we're doing it. So Disney should be looking at this and going, you know what, maybe we don't need to just keep remaking all of our classics. Instead, we can make new classics with the tried and true formula, but up for the modern times. This is a stunning film full of top-tier animation, a great cast, a heartwarming story, incredible music, fantastic action. You know what I sold myself on it. It is the number one Spider-Man for me. Well, that's my list. Thanks for watching. Hopefully you agreed with it. If not, it is what it is. Feel free to subscribe if you like the content. I do have other things on this channel, including my flagship show, Movie Feuds, where I pit movies head-to-head. The one I have upcoming is Spider-Man 2 versus the amazing Spider-Man 2 versus, you guessed it, Far From Home. It's a sequel feud. Alright, take care.