So, today what we will do is, we will start with something called a cache hierarchy, that is what we used in today's course. So, this what you will get by the observation is that ideally, I want to hold everything So, I completely want to avoid going to memory. But as we know, with increasing size, the access time increases. So, for any structure, if you make it larger, it will take more time to access. Which essentially means, a large cache will slow down every access. So, whenever we access the cache, it will take longer and more time. So, trade-off should be very clear here. If I have a very large cache, I have a large number of hits. So, I will never, probably I will never listen to the cache. You can make it large enough. But every hit will be extremely slow. So, if you look at the total time spent, that is essentially the product of number of hits. And the time to access the cache. Plus, it means how much time it takes to get the data from the memory. Except the total time spent in accessing the data. Whatever I want to get that you have. So, that is often called the average memory access time. That is the number of hits multiplied by time to access cache. Plus, time to sort the misses. Or that this also can be decomposed into number of misses times or miss. So, essentially a large cache nullifies the second term completely. An infinite cache will not have the second term. Well, not exactly true. Initially when you start executing, you have to bring some data from the memory. And once you have all the data in the cache, it will never miss again. So, nonetheless a large cache tries to maximize this particular term, number of hits. But this is going to be also very large. So, it depends on how these are balanced and a large cache normally is not very good. So, what people do is, they put increasingly bigger and slower caches between the processor and the memory. And the rational here is that, again the locality principles that we discussed last time is working here. That is when you access a folder to a data point. If you speculate that, your near future will access the data point once again as a temporal locality. So, what you are doing here essentially is that, you put a small cache close to the processor. You bring a data point, put it in the cache. And the hope is that, you will access it again in future and have a cache. But over time, this data will become old and will probably become useless. And that data will eventually get replaced by some other data. Again, that will not happen. So, this small cache close to your processor is a very fast cache. That is giving you most of the hits on the cache.  Then you put a slightly bigger cache after it. Which will accommodate some more data. And it will be slightly slower and slower and slower. So, overall this is going to be better than this. And this is the punch line that is, keep this most recently used data in the nearest cache. Which is the register file. However, what you put in the register file is not really under the control of the hardware or the architecture. This is decided by the compiler, when you compiler program. What data goes into what register and at what time. The next level of the cache is the L1 cache. The level one cache. It is the same speed or slightly slower than the register file. But normally much bigger than the register file. Then you have the L2 cache, way bigger than L1 and much slower. Then you have the L3 cache, which is even bigger and even slower. And today industry is even contemplating an L4 cache. Beyond which there will be more. So, you put gradually larger and slower caches between the processor and the reward. Bridging this particular latency gap. So, here is an example. A couple of examples. So, Intel Pentium 4, it is an old processor. So, I am talking about Intel Pentium 4 had several incarnations. Gradually with increasing frequency. NetBurst was the first architecture that came out. So, here is a memory hierarchy of NetBurst. It had 128 registers, accessible in two cycles. So, in this class whenever I talk about registers, I really need physical registers. We talked about the distinction between physical and logical registers. So, of course, all Intel processors would have only 8 logical registers. Here I am talking about the physical registers inside the processor, not visible to the compiler, managed by the hardware. So, it has 128 registers, accessible in two cycles. However, again, even though this is under the management of hardware, remember that what data goes into what register is still decided by the compiler. Because there is a strict mapping from logical to physical register all the time. Then you have the L1 data cache, which is 8 kilobyte in size, 64 bytes line size, accessible in two cycles for individual loads. Here is the L2 cache, which is 256 kilobyte A2 associative, 128 byte line size, accessible in seven cycles. So, as you can see, as you go down, the latency increases size also. Intel, so again, Intel has several incarnations. So, this is the Madison processor. It has 128 registers, accessible in a cycle. L1 instruction of data caches, 60 kilobyte each. Four-way set of 64-byte line size, accessible in one cycle. Unified L2 cache, 256 kilobyte pipe cycles. Unified L3 cache, 6 megabyte porting cycles. Couple of things to notice here, that is you might ask, well, both of these processors have the same number of registers. This one is accessible in a cycle. This takes two cycles. The question is, both of these processors have the same number of registers. But here the register file requires two cycle access time. Here it requires one cycle access time. Why is that? In fact, cycle times are different. So, Intel Pentium 4 has a much faster clock frequency. So, the actual time is roughly the same. If you look at the total size of the register file here, how much is it? So, Intel Pentium 4, the one that we are talking about here, is 32-bit machine. So, each register is four bytes. So, how much is it? Amount to 512 bytes, accessible in two cycles. Whereas, L1 data cache is 16 times larger. 8 kilobytes has the same latency. How is that possible? So, at one point in time, we made a statement that in a memory structure, if you have more ports, it slows down. So, can you argue along that line? How many? What besides the number of ports in the register file? It depends how many instructions you want to make. That besides what? Number of required three ports. Who decides the right ports? What is that? What does it depend on? Number of instructions. What about cache? Number of ports in the cache? Who decides that? Number of read and write ports in the cache. Sir, write is only one bit. Like this? One write is required. One write is required? One write is required. Why? Cache plays. Yes, you are right. So, why do you write to the cache? Then, I am doubting my question. So, you write back from cache, you will write back to me eventually. That is a replacement. That is a right to memory. We are talking about right to the cache. Is there a cache miss? You have to bring in the new data. At that time, you write to the cache. When do you read from the cache? As instruction is required, it will read from the cache. What do you think? What would be the requirement compared to the register file ports? What would be the requirement in the caches about the read ports? It would be half of the register file requirement. Not really half. It is not installed both. We will read from the cache. Store it to some extra work. If you first read the data about, do the new modification and write the data back to the cache. So, data cache ports is a subset of the peak issue week of the bus. Because peak issue week will contain all types of instructions. And only a subset of that will go to the data cache. Whereas, the register file will have to be avoided all other instructions. Potentially all other instructions. So, it has a much larger number of ports. For example, Pentium 4. If you assume that the issue will be 6. Then your register file will require at most 18 ports. 12 read, 6 write. Whereas, your data cache will probably require a couple of read ports. Assuming that you can only send a pair of load stores and that was two memory operation services. So, that is what is impacting the latency here. If the register file has the same number of ports, it will have a much, much smaller latency than the cache. Just because of number of ports, the register file latency is smaller. So, that is about the hierarchy. Just notice how the latency increases. Usually, it jumps a lot. 5 to 14 is 3 cache. Of course, the size is also much larger. Today's processors, normally the high and server-grade processors have much larger than 3 caches today. Although in this course, probably you will have time how exactly these very large caches are organized. But just keep in mind that they are very large. They routinely cross 10 megabytes. Often, they are more than 20 megabytes. So, we talked a little bit about states of cache line when giving an example. So, let's try to open that up a little bit more. So, life of a cache line starts off in valid state. That is, the line is not in cache. And access to that line takes the cache list and pitches the line from main memory. If it was a read miss, the line is filled in the shared state. We may discuss it later if there is time. For now, just assume that this is equivalent to a valid state. So, shared state is just a valid state. In case of a store miss, the line is filled in a modified state because you know that you are going to modify the line. Instruction cache lines do not normally enter the m state because there is normally no store to the instruction cache. Instructions are read only. You read instructions and execute them. Eviction of the line in m state must write the line back to main memory. This is called a write back cache. Otherwise, the store will be lost. So, there is a second type of cache which is called the write through cache. Well, whenever you do a store, you not only update your cache, you update the main memory also. In which case, you won't require the m state because the memory is always up to date. You only require two states, valid and invalid. For write through caches, there are two flavors. One is called write allocate. One is called write no allocate. So, write allocate, write through caches, what you do is on a store miss, you allocate the block in your cache. And you do the store in both the places, in the cache and in memory. In write no allocate, you don't even allocate the block in the cache, in the store miss. You only send the data to memory and update the data in the cache. Also, the time will be dealing with write by caches. We'll touch upon a little bit on write through caches. And in your homework, you will actually get to compare a write by the write through cache performance. You'll see why one is built around. There is something called an inclusion policy in a cache hierarchy. So, how it is defined. So, normally the contents of level and cache exclude the register file. It's a subset of the contents of level and plus one cache. If this value is satisfied for all values of n, then you say that the cache hierarchy is inclusive. So, essentially what I am saying is that whatever I have in my L1 cache is guaranteed to be there in the L2 cache with inclusive hierarchy. Now, today's processors usually implement something hybrid. So, usually it is part of the hierarchy is inclusive, part of the hierarchy is non-inclusive. So, for example, in a three level hierarchy in Intel processors, typically L1 and L2 will not be inclusive. They won't actually satisfy this property. But L3 will be inclusive with respect to L1 and L2 contents. So, anything the union of L1 and L2 is guaranteed to be in L3. So, that's what it really means. But if something is in L2, if something is in L1, it's not guaranteed to be in L2. So, that this property is not satisfied for L1 and L2. However, in this discussion, we are going to assume that we are talking about inclusive hierarchy where the contents of L1 are guaranteed to be in L2. So, eviction of a line from L2 must ask L1 caches, both instruction and data to invalidate that line in present. The reason should be obvious, because if you don't, you are going to lose this property. Because the line is being evicted from L2, it must also be evicted from L1 caches at that time. Otherwise, there will be a line in L1 cache which is not in L2 cache. And then client's inclusion. Is this clear? Any questions? Yes. It's okay. A storm is, since the L2 cache line in M state, but the storm really happens in L1 data cache. So, we have L1 instruction cache, we have L1 data cache, we have the L2 cache, and we have our main memory. So, this is what we are looking at here. And this is my pipeline here, the processor pipeline, which would access the instructions from the instruction cache and will do read and write to the L1 data cache. So, what I am saying here is that, let's suppose that there is a storm instruction which will follow this path, right? We will first go to L1 data cache, then to the L2 cache, then to main memory, if you don't find the data. So, here I am assuming that I miss in all the caches. So, a storm is, a store access first go to L1 data cache misses here, then moves to L2 cache misses here, brings the data from memory, it fills the data on its return path in L2, and L1 data cache, right, both to maintain inclusion. Now, the question is, what would be the state of the data, that the data block in these two caches. So, what it says is that, the data block in L2 cache will be in states. Even though, the store actually happens in this cache. So, when you modify the data, you actually don't notify the new data to L2 cache. L2 cache is still having the old data. So, L2 cache does not have the most upgrade copy of the line. The reason is that it's not needed. So, implicitly, I am actually saying that, the L1 data cache is a right-back cache. Only when it is evicted, that particular block, the new data will be written back to the L2 cache. That's what it really means. So, eviction of an L1 line in M state, writes back the line to L2, and this is when the L2 cache gets the new data. Until then, it doesn't have it. Now, because the question is there, you ask, why did I do this then? Why should I fill the data in M state in L2 cache, even though the L2 cache is having the old data? The reason is this. If the line is evicted from L2 cache in M state, then you are in trouble. Because unless you send the correct data back to memory, there is a chance of losing the new data actually. So, whenever you evict an M state L2 cache block, you first ask the L1 data cache to send the most up-to-date copy, if any. Then, it writes the line back to the next higher level, that is, L3 or main level. So, why inclusion is important? It simplifies something called a coherence protocol, which we will not at all discuss in this course. So, there is something called a coherence protocol. We will discuss it a little, we will talk about it in two devices, but not much in detail. So, just keep in mind that there is a reason why this is done. Not that one fine morning somebody decided to do this. Any question on inclusion? Sir, why do we need to have the data block in M to L? Whenever it evicts the block, as it asks the L1 to evict the data block. So, at that time, using the statements of L1 cache, we can modify it. But you have to think about the low level implementation details. So, when L2 cache evits the block, it needs to know whether it requires the data response or not from the low level cache. If it does, it has to allocate a buffer to put the data. So, the purpose is reservation, nothing else. You have reserved some buffer for building the data that comes back from the low level data cache. This is like after every store, we have to modify any statements of L2 cache. Right, we have to tell the L2 cache that I am going to do a store. You have to go to L2 cache. Yeah, well, it is actually needed. That is why in a multiprocessor environment. Today, a multiprocessor is a multiprocessor. A multiprocessor, you can avoid doing that. If you are doing a store, you have to tell others that we are doing a store. Others may have the copy of the whole data. What is the first one? Yes, that is right. Just to clarify what they are actually discussing. Consider a sequence of accesses. The processor first loads the data in the valid state in the L1 data cache. Also in the L2 cache. Subsequently, the processor wants to write to that particular cache. So then what has to happen? What he is saying is that at this point, I must tell the L2 cache to move to M state. Because that is what it actually tells me. That is invariant. If you have a dirty block in L1 data cache, the corresponding block in L2 cache must be in M state. So he is saying that that is a big overhead. The answer is yes, it is an overhead. Which we actually cannot avoid in multiprocessors. Which I won't clarify here. But intuitively, why you need to do this is that you may have a copy of data shared by two processors. If one processor modifies the value of that data, other must know the modified time. So before one processor does a store to that data, you must tell the other guy that this data is no longer the correct time. So there has to be a notification, there has to go. Any other questions? Because otherwise, if the block gets evicted from L2, the memory may not get updated with the correct data. But it is not in M state. But what he has suggested, I will reiterate that. What he is saying is that in any case, an eviction from L2 cache will ask the L1 cache. All the time for maintaining inclusions. Then why have an M state? So it is just a matter of low level implementation. Because if it is in M state, L2 will know that there is a potential data response that can come from L1 data cache. So it may deserve some buffer space to fill that data. So it is just a low level implementation. If you want to ignore that, then yes. You don't need to switch the L2 cache to M state. Any other questions? There are actually many other states which I am not really talking about.  Like the case that we are discussing may actually be handled with some extra states. But I will not go into that detail. Alright, so with that, let's try to trace the sequence of events that happen when the first instruction of a newly started program executes. So you take the starting program counter. You access the instruction TLB. Because the program counter is a virtual address that is translated to physical address. You access the instruction. So you access the instruction TLB with the virtual page number extracted from the PC. And since it is the first instruction, you are going to have an instruction TLB miss. You won't be able to find the process. So you invoke the ITL miss handler. The ITL miss handler has the responsibility of calculating the page table entry address. If the page table entries are cached in L1 data and L2 caches, this one we discussed a little bit last time that you can do this. Look them up with the page table entry address. However, in this case, you are going to miss there also within the first instruction. Then you access the page table in main memory. And the page table entry is going to be valid in this case. Which means you take a page font. So you invoke the page font handler. Allocate the page frame. Read page from this. Update the page table entry. Load the page table entry in the instruction TLB. Restart the instruction page. So now you have that physical address. Because this time, of course, the page will get the translation from ITLB. So you can access the instruction cache. Which is going to be a miss. Because for the instruction, you won't have it in the cache. So you send a refill request to higher levels. You will miss everywhere. So in this case, we follow this path. You miss here. Then you look up L2. You are going to miss here. So you go to memory now. And you send the request to the memory controller. Which used to be called the north bench long back. Now it's not really relevant anymore. Because memory controllers are now inside the processor chain. So what guarantees that this particular instruction page will be found in memory? How do you know? Because let's say the next step is memory controller has is a passive device. It gets a request. It gets an address. It sends the request to the memory. That's it. What guarantees you that this particular page is in memory? How do you know the data that the memory will return to the memory controller will be correct? Because at this point, there is no check. The memory controller gets an address. Follows it to the memory. That's it. The answer is already discussed. In the last slide. So this particular action makes sure that the data will be in the memory. So you read the cache block from main memory. So now the cache block comes back. And while coming back on its way, you will fill the L1 instruction cache. And return the particular instruction word to the processor. So extract the appropriate instruction from the cache line with the block offset. And then the printer can start. So now the processor has the instruction finally. It can decode and execute. So this is the longest possible latency in an instruction data access. Is the sequence clear to agree? Nothing more happens. So a little bit more on the TLB access. The important observation to make here is that for every cache access instruction or data, you need to access the TLB first. So that's what gives you the physical access. So it puts the TLB in the critical path of every instruction. If you have a slow TLB, the instruction execution is very slow. So if you remember, the point is that you take the partial page number, look up the TLB. You get a physical page number. You cabinet that with the main offset. You get a physical access. And then you can look up the cache. So that puts your TLB in the critical path. The cache access cannot be done until the TLB access is completed. So this is the physical address. So I put a plus here. This is just a calculation. So this is the sequential operation. I look up the TLB and the cache in sequence. There is no overlap. So what I ideally want is to start indexing into the cache and read the tags while TLB takes place. So I want to do this lookup and this lookup in parallel. How can I do that? So the only way to do it is to index the cache with the virtual address. Because to begin with, I only have the virtual address and nothing else. So the question is, can I use part of the virtual address to index the cache? And I can have my tag in the cache derived from the physical address. Because by the time I look up the cache and the tag comes out, I would have the physical page from number from the TLB so that I cannot do a tag comparison. So this is called virtually indexed physically tagged cache. That is what is used today in all commercial buses, at least for the L1 cache. So you extract index for the virtual address. Start reading tag while looking up the TLB. Once the physical address is available, you do the tag comparison. It overlaps the TLB reading and cache tag reading. So it relaxes these latency a little bit more.  You can afford to make your cache a little slower but still not use it. Because now these two will be going parallel. So here is the typical latency in the memory hierarchy. Don't take these as the authenticative numbers. These are just examples. Just to show you what the tabs are. So L1 heat latency is about nanosecond today. L2 heat latency is about 5 nanoseconds. L3 is 10 to 15 nanosecond. Main memory is about 17 nanosecond. We have access time plus bus transfer stretcher. So it gives you about 110 to 120 nanosecond. And if you have a more complicated system, things may vary more. So here I am talking about a variance of about 10 nanosecond. Variation of 10 nanosecond. The gap between minimum and maximum may be larger depending on the update. So the point here is that there is a very big jump from L3 to main memory. So L3 cache on the last level of the cache is your last line of defense. If you miss there, you are going to take a very big performance. That's pretty obvious. So your last level cache should be very intelligent. It should be able to identify which data blocks are important. It should retain them and make room for them by flowing away the useless blocks. So it has to do some form of prediction. In some way I think this block is going to be reused in future. So I will keep it. That block doesn't look like to be reused in future. I will show you and make room for some other issues. So your cache controller has to be fairly smart to be able to do these things. It's a very hot research topic because you can see why it is. There are good reasons for that. If your last level cache is a dump cache, your processor is not going to give you good performance. That's pretty obvious. Because this jump is enormous actually. So just to give you the problem a little bit more concretely. If a load misses in all caches, it will eventually come to the head of the ROB. You have the ROB. So this is my ROB. So this is the head. So this is where my retirement is currently running. I am retiring instructions from here. And let's suppose there is a load here which is currently executing. This one is currently looking up the cache. So this particular load instruction looks up the L1 data cache. Missiles in the L1 data cache. It goes and looks up the L2 cache. Remember that in the meantime the ROB is progressing. The head is actually moving gradually. It is retiring instructions. So the point is that if before I get the data for the load. If this head moves down here. I have to stop. Because I cannot retire the load at this point. So now not only I can retire the load. I cannot retire any of these instructions. Because the retirement has to be in order. So that's a big problem. Which means I have only this much of time to get the data for the load. To make sure the processor doesn't stop. Now if a particular load instruction misses in all the caches. And has to go to memory. Can you imagine the large amount of time that it has to. If it is going to take. So we did a calculation last time. I should remind you about the calculation. Suppose your processor runs at 3 DRs. Let's suppose that you retire 4 instructions per cycle. So what does it mean? Your cycle time is one third nanosecond. So if I assume that let's say let's be optimistic. Let's suppose that we have a 100 nanosecond memory time. So we do 100 nanosecond. Let's assume that I'll get the data back. So that's essentially how many cycles? 300 cycles. I retire 4 instructions every cycle. So I need an ROB of size 1200. To be able to hide the latency of this load. Which is impossible. Because ROB size is 200. So this is the problem. Why loads impose a problem? This is the reason. You cannot retire anything beyond a certain point. And if you have an ROB of length 100. It will take only 25 cycles. To get to a stall point. Which is a very small fraction of 300 cycle latency. So which is why today actually if you look at applications. That access big data. A large amount of data. You'll hear numbers like 90% stall time. This is the reason. So one way of resolving it. Is to have a smart last level cache. Make it smart enough. So that it can retain important data. So you won't have to go to memory 2.0. So gradually the pipeline backs up. Processor runs out of resources. And ultimately the feature stalls. Which severely limits IAM. So that's the basic problem. So we'll talk about some of the simple solutions here. That the processor industry has adopted over time. To alleviate this particular problem. So essentially what you need is memory level parallelism. So till now we have been talking about instruction level parallelism. Where you find out instructions which can be executed in parallel. And you let them execute in parallel. You offer resources and all that. So the same question we can ask about memory operations. So can I execute multiple memory operations in parallel? That's for memory level parallelism. So simply speaking. You need to mutually overlap several memory operations. Why is that useful? Because then if you have two memory operations. Executing concurrently. For both of them you'll see a latency of 100 nanosecond. It's not like 200 nanosecond. So that's the approach. You get overlap. So how do you achieve that? The first step is to have a non-blocking cache. What is that? You allow multiple outstanding cache uses. That's the first requirement. That is you cannot now say that. Whenever my cache gets the first miss. It is going to reject all subsequent requests until this miss is resolved. Then of course you're not going to get any memory level parallelism. That's all the question. Because the cache itself is acting as a blocking agent. So this is the first requirement. That the cache must allow multiple outstanding cache uses.  It should still be able to accept more requests. Allow them to go through the cache. Maybe some of them will get. Some of them will miss. That should proceed. So you usually overlap multiple cache misses. Supported by all microprocessors today. For example alpha 21264 supported 16 outstanding cache misses. So today's numbers are roughly around this. How many outstanding cache misses you can support. The reason why there has to be a limit is because you have to remember somewhere that what are the misses that are still outstanding. There has to be a table. And that limits the number that you can actually set. And this table is actually pretty much on the critical path. So that's why it has to be small. It cannot be very large. So is this solution clear to everybody? That's the first step. That you have to rate the cache. So you cannot have a blocking cache. The second one is out of order load issue. That is issue loads out of program order. This one also we discussed earlier. Address is not known at the time of issue. So what you do is you first, first thing of the load issue actually computes the address. Comes back, computes its address with the stores before it. And then only it can go and access the memory. So how do you know the load issue before a store to the same address? So issuing stores must check for this memory order violation. So let's talk about it in more detail. So here's an example. I have a store here.  So let's assume that the load issues before the store. Because R20 gets ready before R6 or R7. For the store to issue I need to make sure that both of these are available. R6 and R7. For load to issue I need to make sure that R20 is available. So let's assume that whoever was generating R20 completed first. And so on a load can issue. So load access to the store buffer. This is essentially the value field in the store key that we talked about. This is used for holding already executed store values before they are committed to the cache and to terminate. If it misses in the store buffer it looks at the caches and say gets the value somewhere. After several cycles the store issues and it turns out that these two addresses are actually same. Or they overlap. So now that means the load must have got a wrong value. So maybe you have forgotten so let me try to remind you. So let's suppose that we have this single issue queue. And this load instruction here is this one let's say. This is the load. And let's suppose this is the store. That store instruction. So previously we said that the condition for a load to issue is that all the stores before it must have computed their addresses. That's what we said. Now what I'm doing is I'm going to relax that. Because that we are going to say that this looks very conservative. Because many of the loads will not depend on any of these stores. Why are you delaying the issue of those loads? So now I'm saying that well I don't care. I'll issue the load as soon as its operant gets ready. This is R20. Which means some of these stores may not have executed yet. Which means some of these stores may not have computed their addresses. So what does a load do when it issues? It first computes its own address. It comes back, compares its address with all the stores before it. And here we are assuming that it doesn't match with anybody. And the reason is that this store hasn't yet executed. Because R6 or R7, one of them may not be ready. So then the load goes happily, accesses the cache, gets the data somewhere. Supplies the data to its dependence that is sitting here. They also start executing. Eventually R6 and R7 get ready. The store issues executes. And you find that these addresses are actually same. So load has not only consumed the wrong data, it has supplied the wrong data to many dependents. How do you recover from this error? Any suggestion? Is the problem clear to you? So the problem arises because we are trying to issue loads very aggressively out of the box. So how do I fix it? So the previous slide, there was a comment here. Issuing stores must check for this memory order violation. Somebody expand around that one. Is the load instruction still inside the processor somewhere? Or has it to do that? Why not? Exactly. So the load hasn't yet escaped. You can still catch it and fix it. How? So we may check if the load has loaded from the same memory. And I can fix it. I can issue some signals. So an issuing store should check all loads after it that have already executed. And do a comparison of addresses. And if there are multiple matches, what should we do? There will not be multiple loads that are executed to the same address. Fix all of them. Fix all of them. So by fixing what do you mean? Can you elaborate? Overwrite the value. Load is loaded into some register. That's okay. So we change the value. That's all? That's all. The instructions we are supposed to read to those registers have already read. And have executed. What about those instructions? So you have to also find out the dependence of these loads. Which have executed. So there are two modes of fixing. So the first step is of course that an issuing store must check all subsequent loads that have already executed. By checking I mean compares the address. Of course an issuing store cannot compare addresses. First it issues, computes its address and then only it can do that. So it checks all the loads. The simple solution is that it picks the oldest load that matches and removes all instructions after it. So it reexecutes everything after that. So of course it will do some extra amount of work. It will reexecute many instructions which do not depend on this opening load. But it's a simpler solution. So you don't have to keep track of dependencies. A slightly more complicated solution would actually maintain the dependence tree of each load. So starting from the load, you can think of it as a root.  Form a tree. You keep track of the tree. So that's what today's inter-process actually do. They keep track of the tree of every load. In the buffer. So that actually minimizes your wasted work. So that's precisely what the solution is. So this is called speculative memory design equations. So essentially we are doing one form of speculation here. Why do we issue the load? Because we are speculating that this load won't have any problem. Won't ever conflict. So computer architects are very optimistic about what they do. So here again we are being optimistic. And we are saying that there won't be any problem in the future. The good news is that it is correct most of the time. That's why it works. Otherwise, of course, you will drown in poor performance. So assumes that there will be no conflicting store. If the speculation is correct, we have issued the load much earlier. And we have allowed the dependence to also execute much earlier. If there is a conflicting store, you have to squash the load and all the dependence that have consumed the load value and re-execute them systematically. Turns out that the speculation is correct most of the time. And often this is called a blind speculation because you are not really using any property of the load. You are oblivious about the history of this load. Because in the past this load might have conflicted with the same store. But here you are not caring about that. You are saying that I will do a blind speculation. I will issue this load whatever may be its history telling me. So you can improve upon that. So that's what today's classes do. They use simple memory dependence predictors. Which predicts if a load is going to conflict with a pending store based on that load's past behavior. So you can actually make association between the pair of loads and stores. You can store the association on a table and figure out in future if this load actually is going to conflict with any of the stores that are waiting here. If it is, then you won't actually issue this load. You will wait until all the stores have or at least the predicted conflicting store has gone. So this is what the classes do today. I won't be able to discuss about the actual predictor implementations. If you want to read about that, I can give you some papers. So they look very much like branch predictors, but they have to be a little smarter. Because here you are talking about establishing an association of a pair. A load and a store essentially. But still the query is a binary query. So yes, no answer will come. Just like a branch predictor. You take a load and ask, tell me if it is safe to issue this load. So the predictor will give you an answer. So today microprocessors try to hide cash leases by initiating early prefixes. That's another solution. Hardware prefixes try to predict next several load addresses and initiate cash line prefige if they are not already in the cash. So I hope you are noticing that there are many flavors of pattern predictors that go into a processor. And various levels actually. So this is again another pattern predictor. That mines the pattern of addresses that are accessed by the cash and tries to predict the future. Tell me what address are going to come in the future. Can I fix them now? So by the time it is needed it can be in the cash. All processors today support prefige instructions also. So you can actually specify in your program when to prefige what. So the compiler can actually insert these instructions at appropriate places. This gives much better control compared to hardware prefixes because here you can actually see the whole program. And also you can maintain the semantic of the program. For example, you can see that it's an array access. So you know that it's going to be very predictable. You'd be accessing sequentially the array locations. The address are extremely predictable actually. Whereas this guy, the hardware prefiche that is a finance state machine, sitting there monitoring addresses and just making a blind prediction. So there is a bigger chance that it may be wrong. But of course the advantage is that it can see all addresses which the compiler cannot. For example, dynamically allocated memory addresses won't be visible to the compiler. Another solution that the researchers have explored is load value prediction. That is, I am loading a particular piece of data like here. Can I predict what this load is going to be done? The value. So that also has been explored. It's a much difficult problem because you're not trying to predict a 32-bit or 64-bit value. So the entropy is expected to be much higher. It's not very easy and accuracy is usually low. Which is why the industry hasn't yet adopted this particular solution. But there is a large body of research on this. How to predict the load values. And the good thing is that the programs often load constants. A lot of constants are loaded up. And the most loaded constant is here. So these values can predict extremely easily. Without much talk. Even after doing all these things, the memory latency remains the biggest part of this bottleneck. And that's called the memory war. It's a very hard research topic. And probably it will remain a hard research topic for a long time. Because there is no easy solution visually in the future. Processors are expected to get faster. Memory is not getting faster at the same date. So the speed gap is going to increase. So your 90% stall time may become 95% stall time in future. Okay, going forward. So I'm going to stop here. So next time we'll delve deeper into caches.


Well, hello friends and welcome back to my channel today. We're gonna be doing an Open box from Erin Condren. I actually placed this order almost a month ago Like and I know you guys are probably like why did it take so long to get an order? Well, let me tell you one of the items unfortunately was a little more difficult to find Seems as though some of the inserts were out of stock when I placed the order So they had to print a ton of inserts. Anyway, I already talked to customer service about it It was totally fine But anyway, I wanted to show you guys Another planner that y'all might be interested in for the new planning season Which is coming up a very very soon if you start planning in July You can also clearly start using this in January if you are just a regular calendar person first of all I want to say thank you to Tori over at Erin Condren for thinking of me and sending this my way She did allow me to pick out what is in the box So let's do this without any further ado. So I tried to cover up my Address, but the sticky notes that I'm using are not very sticky. So let's cut this off camera Okay, I just took everything out and placed it right here so we can all take a look at two together All right, here is all the goodies Okay, let me again get things out of the way so So let me take this stuff out of the plastic because it's just gonna crinkle and be really annoying for the sound All right, so here is all of the goodies I've got So I got myself another binder y'all but let's go into like what's in the little pack with every binder purchase and basically every life planner purchase as well you get a free ruler this snaps in really well to the coil bound as well as The rings which I'll show you in a moment.    There's two of them So there's like one for you and one for a friend and it's a unique coupon code So I'm not gonna show that because you know, I like coupons, too You also get two of these beautiful Labels that are like more for gift wrapping So this one says just a little something from then you can put your name or your family or whatever And this one says enjoy two from so loving that And then this is just a little card that just says like what comes with it So, yeah, it's kind of cute and then and then and then you guys oh my gosh I'm so excited. I really wanted to share so I'm gonna put up in the card wherever it goes my Current planner move in for 2020 slash 2021. I just filmed it. I just uploaded it I think it was last week and That is the planner. I'm gonna be using however the binder that I have is the I think it's called champagne It's like a light tan color and I saw this one and I was like, oh damn This is a little busy for me But I think it's so freakin beautiful and I feel like it would look really beautiful like on my shelf, you know So yeah, it's just gorgeous so actually while I'm doing this, let me go grab my Actual binder that I'm gonna be using for this year. All right So here is what I'm gonna be currently using this is in the champagne color So it's called the champagne life planner binder This one's $55 and this is called book cloth wrapped life planner, but to me It feels like it's like almost coated. It does feel like the outside of a book. It really does it is textured You can hear that where this one is Smooth like this one smooth to the touch. We'll reference this a little bit later I'm just gonna put that off to the side and then this one is called the layers colorful life planner binder Also retails for $55 and these specifications are a paper wrapped life planner So it's like a paper, but it's very very thick. I feel like it's very thick and sturdy I'm not sure how it would like hold up if you want to take it with you every day But again, it is beautiful and I love it and we got the teal on the inside party party party Okay, get rid of this Put that over there. Oh, sorry Doyle like hit my dog great So yeah, and then it has these beautiful rings, which I am loving and on the inside I decided I wanted it to get The neutral layers in the hourly layout I want to show you guys This just because I thought it would be kind of interesting to look at Because I don't know. I like don't have enough space on my desk here. Hang on a second Since like working at home and everything I have these like big-ass monitors which are going to be in the way kind of like This is this is a this is a monitor. So just just FYI it's a monitor over there guys and then Okay, so let's kind of Go through this Because I kind of wanted to share Like any differences or whatever. So it's gonna be kind of tight. So just like FYI. Okay Yeah, and I wish I could get this up higher, but I cannot Anyway, so here are the main differences.  It doesn't take very long guys But on the inside of the binders, there's a little bit of a difference with the Um design so there's like a mid-sensory circle on the inside which kind of makes sense because Like the layers is such a pattern that it's kind of makes sense for this to just be like plain Because the outside of this one is plain and then yeah, so you have your life planner And you've got the beautiful vellum page Which is one of my favorite things and then same thing the life planner hourly layout And the font is slightly different in the colorful versus the neutral So um, it says this is my life and more of a script and then this one it has it more in like a block letter So Let's just Go page by page see again The script is just slightly different the message is still the same And then again, like I said, I've already like messed with this one So um, this is all like moved in for me. So again, the script is just slightly different clearly Um, let's see here This is a little bit different. Oh, this is like very different. So with this one Well, I can't remember what was up above here now. I'm sorry Anyway, but it has the same 12 boxes And then if we get into July Which um Is the same quote, but again different script. So just depending to to be completely honest I am I'm a colorful flam flam. I'm a colorful Fan for the inside of my planner, but I love this script so much better Um, I don't know just a personal preference So as for July, it's just more simplistic Um, but again, I love this like script I think it's so pretty and then going in to the next page We've got some more information So we've got like your plain page, which I'm going to be using for my monthly budget, which I haven't set up yet But it's it's going to be there. I'm going to be doing budgets again guys. I know I know crazy, right? um, and then this page is The same except for it has a little bit of design difference like right right here This has just like a little plain thing and then the font is slightly different And then um, it's just more neutral. It doesn't have this blue in here It's kind of this one's more of like a little bit of a like purple haze kind of a thing And then when you get into this again a little bit different with the um script and then also with your uh days so up above here, we've got this colorful Area it is now white up here, which I know lots of people love because it just makes it a little bit more versatile And then also down here, you don't have the color so the color is right down there And to be honest now saying this I might swap out these pages um Like my daily pages maybe I haven't fully decided I haven't decided what I'm going to do with all these inserts But I just I just think they're gorgeous um But anyway, so yeah, and then you know moving on to the next week you got this up here So again, it's all it's all exactly the same at the very end of the month. Hold on. Let me do this one as well It is the same except for um, this has like the little bit of waves pattern up here, but um, this one is Totally totally plain. So it's kind of you know, whatever you want to do and then going into the next month Again same the other one was like a little bit purple This one to me is a little bit more like like a dusty robin's egg blue um, is this color theme, but man, I just The simplicity of this is so incredibly nice um, but again, I love color in my planner. So um, yeah, I don't know I don't know what I want to do with these inserts. I think they're so freaking cute And um, I think I might do like a little hybrid situation. I'm not really sure yet. So anyways But yeah, but getting all the way to the back. So let's do Let's show you what's in zbeck here Oh, you know what? I don't have these pages in here. Um, because I only put 12 months in this planner. Um, did I put this one in? Oh, no, I did. I did put this in. I'm sorry Here we go Um, I did put this one in. So this is the contacts tab. So like on this side it says contacts and then 2022 so I already put in like my Haul it or not holidays. I'm sorry, but like my paydays. I already put those in Oh, and then look at these neutral Stickers. Oh, it's so exciting To be honest, I do not keep stickers inside of my planner. I don't use them if I see if I have them back there um This is something that really bothers me. I know that they're doing it across the board So it's not a flaw, but um This like to have these punched like this and not in This way, you know where you can see all the you know, like the two the two and the four hole I would really love it if it was like that because it would it would make it easier to Flip these in and out Um, because clearly these holes are a lot smaller than those If you're somebody that wants to keep the stickers in the back of your planner So anyway, again, like I said, I am not that person. I am just you know Stating the obvious. So yeah, but I love these those to-do stickers are some of my favorites I love these happy birthday scripts So pretty. Oh gosh. I'm gonna have to I'm gonna have to Put these in my little erin conran sticker area and then also there is The coordinating little pocket in the back. Um, and then now it has the big pocket in the back too, which is totally my jam Um, but to be honest when I'm setting this up I typically don't keep this in there because whatever I put in the back of my planner just goes there to die I don't know about you guys, but it does. So I usually end up putting that in a d-stache And um gifting that stuff to some of my local Minnesota people that love planners just as much as I do So let me get this bad boy out of the way here. All right, so that is a look at the layers colorful life planner binder with the layers neutral hourly layout inserts and um Also, I got one of these and I got this in my like launch day haul as well Which I'll link above for you guys if you're interested. This is a free gift with purchase This is the erin conran. Um, I think it's just like the transparent clutch. Was that what I was called? Oh layers of organized organization clutch. Um These are really nice I know a lot of people have been talking about, you know, what they're going to use this for if it's going to be Planning items or whatever to be honest. Um, I actually have a couple of these so far So I am going to keep this but what I'm going to do is um put it in my little travel area I have a big bin in my office closet that's dedicated just to travel things And I have a bunch of travel cases So I think this would be really nice for either on the plane or to put in a suitcase or something like that Whenever we are able to travel again. So, um, yeah, but I just really like this. It's super neutral The design is really subtle. I love gold. So that's like up my alley and then um, You know, I love this color too. It's not super bold. I feel like it's really really nice So, um, yeah, but that's uh, that's another little thing you get as of right now But uh, who knows I don't I don't know if this is like with every single life planner purchase or um, if it's just like While supplies last. So anyway, that is it for the video. I hope you guys enjoyed Please let me know what you are going to be using for your planner for this upcoming year Um, also, when are you starting to use your planner? I am chomping at the bit to start using mine in july But um, yeah, just tell me what you're planning on using and when you're planning on using it If it's an erin kondran if it's a totally different planner if it's a notebook if it's your phone Doesn't matter. I love to know how all the other people out there plan their world So anyway, thank you guys so much for watching. I really appreciate it If you like the video give it a thumbs up feel free to subscribe And I will be back with more fun content very very soon for you guys Bye Well folks it's time to kick it old school


I see friends and colleagues and clients sometimes posting negative news on social media about you know Donald Trump or some other politician or about some national or global issue. And I really don't think it's that helpful. And I just in this video I want to encourage all of us to be more aware about what we're posting on social media and really more what we're spending our time doing in terms of making the world a better place. So I want to encourage you to think in terms of spheres of influence, spheres of influence and using your time and energy proportionate to the sphere of influence that you have. So for example, what is your amount of influence on what Donald Trump does? Extremely little, almost none. What is your amount of influence on some national policy or you know, yeah, some national policy? Very very little. But then again, what is your influence on local policy? Something in your city or even something in your neighborhood, relatively more, right? And so it only make I think it makes sense for us to be spending more of our time and energy on local things and just a little bit. I mean, again, proportionate to the amount of influence that you have, right?  And so that's where we should spend the proportion of energy and time. So that's my encouragement in this short video. Be especially careful about posting negative stuff on social media on issues that you have relatively little control over. Because number one, it disempowers you with the negativity and disempowers other people as well. Actually, you might say that the negativity angers you and rises you up into action. But let me ask you this question is, are you taking action? If you are in a way that is where you have actual influence and wonderful. But if you are just posting negativity because somehow it's easy to do, then be especially wary of that. The last thing I'll say is that you have enormous power, the most power over your own attention, right? And that's where you have the greatest influence. So let's lift each other up. Let's pay attention to more empowering things that we can do, especially thinking about the spheres of influence, influencing your own life and instead of studying national policy, unless we're going to become an academic in that area, let's study the lives of the people we can touch and how we can help them even more. So focus on your calling, become more effectual in fulfilling your calling. And if you're going to get involved in social justice or environmental well-being, it's probably more beneficial to do it with local organizations where you can develop your leadership capacities and your leadership skills. That is much more effective than just posting negative news on social media. So I hope that this video is encouraging and helpful. And until the next one, my name is George Cao, encouraging you to pay attention to your spheres of influence and where your attention and energy are going. Be well.


Good morning. I'd like to thank the organizers for this opportunity to speak with you and to Convey to you our vision for integration and application of in-code data and differentiating blood cells the the vision is actually a project the project name comes from Validated systematic integration of epigenomics data in hematopoiesis our motivation is that there are large numbers of data sets from in-code other consortia Lots of individual labs that are available to researchers, but it's difficult for them to Effectively incorporate those that information into their research Our vision project aims to integrate these heterogeneous data sets systematically and produce Useful resources for the broader community. I'll be focusing in mostly on these candidate regulatory elements The work I'll be describing is led by Cheryl Keller and Guan Jie shang in my group and we had a Part of this work was published earlier this year in genome research so this Project which is funded by NIDDK it consists of multiple laboratories around the world and we're Working to acquire epigenetic information either through our own experiments or by Mining the work from other laboratories or consortia including in code integrate that information To predict regulatory elements and to give you a painting of the regulatory landscape and turn that integrative information into predictive models for gene regulation to Predict how each regulatory element is impacting target genes then test those Predictions by genome editing evaluate the results and try to improve our modeling We're doing this both in human and in mouse and here's An example of the the systems we're working with this is a mouse hematopoiesis The Hematopoiesis is the production of all the blood cells that you have circulating In your blood and these are all very different from each other very different Abundances very different functions. They all come from a Common hematopoietic stem cell that during differentiation progresses Progressively decreases this potentiality through various progenitors and and then along each lineage You have an expansion and maturation going on so we're looking both at the mature cells and these Progenitors using data from our own laboratories data from Edo Amit's lab from encode and others and Trying to fill in this matrix of histone modifications nuclear accessibility a Structural protein CTCF as well as RNA seek we Don't have a full matrix, but the missing data can be handled reasonably well By our integrative system which comes from dr. Yeh Chung who Developed this when he was on our statistics faculty at Penn State This is a two-dimensional segmentation That's done by a system called integrative and discriminative epigenome annotation system and The object is to assign each Genomic interval to an epigenetic state that itself is a unique and commonly occurring combination of Epigenetic signals such as histone modifications nuclear accessibility and so forth what is powerful about ideas is it does The segmentation jointly Learning both along the genome and across cell types and this preserves position specific information And gives you greater precision in finding cell type specific events. It is a Treats that the data is quantitative variable rather than a binary one And can handle missing data in an elegant way this Explored in this paper from Shang and Mahoney and this is our most recent Package that you can access via that bio archive article now When you apply to take all of that epigenetic data that I showed him that the data matrix and put it through ideas Ideas learns that there are about 27 states and those states are characterized by particular combinations of Features again nuclear accessibility the various Histone modifications not only the combination but also that the amount of contribution that that each of these makes to the To the output and you can see several flavors of promoters flavors of enhancers different Repressive regions different transcribed regions and when you then assign those states to each DNA interval across all of these blood cell types the progenitors the erythroid cells the Lymphoid cells you get a very informative painting of the epigenetic landscape. You can easily see Broadly expressed genes with their common epigenetic profiles across cell types. You can see more lineage specific genes with patterns of Active regions of which we associate with promoters or enhancers these orange colors for enhancers Restricted to a particular cell types and in fact you can see that not these Epigenetic state assignments are supported by orthogonal data such as a co-activate or binding which was not part of our training but you see it binding around these predicted enhancers and Actually independently That these regions have been examined for activity and transgenic mice and they actually are active. So that's actually looking pretty Pretty good and and we're utilizing that that these this epigenetic painting the segmentation in many many ways The thing I want to focus on now is not just the entire genome not not the entire genome But rather let's make some discrete calls for likely regulatory elements candid it's this regulatory elements and with this Integrative modeling. It's a fairly straightforward a procedure. We find all of the Peaks called by chromatin accessibility attack secret DNA seek and then we just ensure that those peaks are not solely in quiescent regions Of the genome and then we got our CCRE calls and we also know what the Epigenetic state of each of these elements is across the cell types Excuse me and Both in mouse and in human blood cell types we get slightly over 200,000 of these Elements you can see kind of similar patterns in mouse and in human for the same locus that we were just looking at a substantial Accumulation of these elements both within and between the genes Looking more broadly we ask what How does our collection of vision CCREs? How does it compare with earlier catalogs of Blood cell enhancers and how does it compare with the larger collection of CCREs from from the Encodes of the phase 3 encode which of course as many more elements because it's looking at meaning more cell types and The the overlaps that you can see in this upset plot are actually a pretty supportive. I think That there are about 60,000 CCREs that are found in almost all of these Collections so you should feel that these are very strongly supported of the Elements that The other elements in vision a whole lot of them are also match up with Data that that are in the larger encode Collections so I said almost 60,000 of them there Apologize for my my voice here and And as I mentioned before that you expect there to be a lot of them Elements that that are it found by encode that Because they're examining way beyond of blood cell types and what they're looking at One of the several things that we've been doing with these epigenetic states is to use them as an input to try to get at a an estimate of the regulatory potential the regulatory potential based on the epigenetic signatures where that potential is what's the impact of each element on a target gene and Our first approach to this was a classic multivariate regression analysis where What we wanted to do is to explain expression levels of the different expression levels of genes across cell types There's a gene A and gene B. We know or we've already made calls for the where the Regulatory elements are and we know what the profile of epigenetic states are in each of these cell types So we try to ask them how well do these? States explain the Expression and we look separately at proximal and distal we convert this categorical variable into a continuous one just by Letting our X be the proportion of the pooled CCREs that are covered by each of the states of fairly simple methodology there and then the Multivariate regression then well learn Estimated values for these betas the the coefficients. So now instead of just having a Color assignment for each state we can assign it a number That's based on whether it's having a negative or positive impact on the Expression of particular genes and that's what we call our ERP score, which is actually the weighted sum of them coefficients for each a state for each of these CCREs and then this is calculated for each gene and in each cell type and This is available at our website and We're starting to try to dig into this and let me just show you one example one of the genes that that we like to focus on ALAS2 is a clinically imported erythroid-induced gene it encodes the enzyme that catalyzes the rate limiting step in a heme biosynthesis Erythroid cells need to make lots of heme to bind up with lots of globin to make lots of heme and globin and be a nice red blood cell You can see from the RNA seek results here across this maturation series is Differentiation and maturation series you go from almost no RNA seek signal to a really really high signal and you can see within these introns very strong Chromatin accessibility which has been noticed before Our lab has studied them many other labs have studied these and showing for instance that reporter gene expression is greatly enhanced But by these intervals and importantly There are some families with X-linked citroblastic anemia or show mutations in this Element so it's clinically important What if we look a little more broadly though we were just focusing in on ALAS2 before in our previous studies well Looking within the TAD this is that these are the the TAD boundaries for the region including ALAS2 You see a lot of places where there are epigenetic signatures states that seem to be Associated with that with the gene activation. So maybe there are some of these are artistal regulators We were particularly interest particularly interested in the ones that had binding by known erythroid transcription factors like God a one tau one and so forth so In and here's that the ERP scores that we were getting across this you see there's a lot of positives These are the the the known ones are sevens and some of these are much larger ERP scores. So we wanted to test these out and we focused in well, we utilized this R2 and R3 the intronic ones as positive controls and we I want to look at this R4, which is a more distal element and April Coburn in my lab has been conducting Directive mutations of these elements using CRISPR-Cas9 and trying to target the guide RNAs to to hit around these Conserved God a one motifs and sure enough when the deletion goes into those motifs You can see a substantial reduction. So that's a good positive controlling and actually confirms that this has an element has an effect on ALAS2 similarly this R3 which has not has been studied as much but her mutagenesis is Hitting these goddess sites up pretty frequently and and we see reductions. So this is all looking good the more distal element We have more limited data on but and this is very recent and we're still working on this But you can see mutations in this element Generating substantial losses and activity and we're following up on that, but it's very very promising so I hope I've convinced you that integration of these epigenetic Signals to get a regulatory landscape is useful. You can combine that with accessibility to come up with a good set of regulatory elements converting states to numbers by these regression models is actually Has a lot of potential and it's very promising. We're very happy with our initial investigations of These experimental evaluations our resources are at this website I want to thank the members of my own laboratory other labs at Penn State and other members of the Division Consortium for working with us and of course I'm very grateful for our funding and I'm grateful for your attention. I'll be happy to take any questions


USIP's Youth Advisory Council is launching a series of events called the Youth Talks. The Youth Talks facilitates conversations among peace builders from conflict-affected regions on key issues on peace and security at the community, national, and international levels. The series aims to further the UN Security Council Resolution 2250 on youth, peace, and security by highlighting the experience and expertise of youth leaders of USIP's Generation Change Fellows Program. Having the perspectives of young peace builders with different backgrounds from gender-based violence to migration, environment, preventing countering extremism, among many others, is an excellent opportunity to hear into testimonies, to strategies, to new approaches to addressing the most pressing challenges to peace today. We invite you to follow the conversation on social media by using the hashtag USIP Youth Talks.