today's lecture on in quest of higher ILP. In the last lecture, we have started our discussion on this topic and we have seen how we can achieve higher instruction level parallelism and we can have machine organization to facilitate that. So, in this lecture, we shall continue on that topic and before I proceed, here are some important parameters I should tell you. Number one is instruction pipeline cycle. Instruction pipeline cycle essentially tells you the cycle time that means, we have seen that the machine is controlled by clock and whenever you do pipelining, then the clock frequency increases and cycle time reduces. So, this is the instruction pipeline cycle. On the other hand, it may take let us assume a machine cycle may take four such clock cycles. So, in other words, if it requires four stages of pipeline, then 1, 2, 3, 4. So, this will be your machine cycle rather the instruction cycle, but pipeline cycle time clock cycle time will be this one. So, this parameter you should remember. Second is instruction issue latency. Instruction issue latency is you are issuing instructions one after the other. So, the time required to issue an adjacent instruction after a particular after a previous instruction that tells you the instruction issue latency. That means, you cannot issue all the instructions simultaneously. So, there will be a difference in time or latency in issuing instructions one after the other. So, that is defined as the instruction issue latency or abbreviated form IL. Then instruction issue parallelism IP. IP stands for the number of instructions that can be issued simultaneously concurrently that is actually known as the instruction issue parallelism. And then the fourth parameter is simple operation latency that is the processor will be performing operations instruction execution. And that operation latency is the essentially the time required to perform the operation of a particular stage that is the operation latency OL. And then machine parallelism will tell you the number of instructions in flight simultaneously at a particular instant of time. For example, if you have got K stage pipeline, then you will find that when the pipeline is full there will be four. If the number of stages is K, then K instructions will be in flight simultaneously as the instructions get executed. So, this is the machine parallelism for a simple pipeline processor it is equal to K. So, we can start with whenever we go for discussing higher and higher ILP you will see you will require a reference. So, initially when we discussed the pipeline processor our reference was non pipeline processor. And with respect to non pipeline processor we have discussed various parameters like speed up throughput and so on. Now, since we shall be concerned with increasing the instruction level parallelism greater than 1, our reference point will be the baseline pipeline processor which I have discussed in my previous lectures processor or machine whatever you call it. So, this baseline pipeline processor obviously uses temporal parallelism we have already seen. And by doing that it achieves instruction level parallelism and we have seen that for the baseline pipeline processor which we also call scalar pipeline because it issues only one instruction at a time. That is why since issue parallelism IP is equal to 1 the for the baseline scalar risk processor another thing we have added that is risk we are our discussion will be centered around only this processor that is why this is being mentioned for the baseline scalar pipeline risk processor the issue parallelism IP is equal to 1 and output latency is also 1. Output latency is also 1 because you can see here it is generating output at the interval of 1 clock cycle. So, the output operational latency that is equal to 1 and the machine parallelism as I have already told which is equal to k. K means as you can see here at any particular instant of time since it has got 4 stages this particular pipeline processor maximum of 4 instructions will be simultaneously in flight during execution. Of course, they may be in different stages of execution, but you know the maximum number that can be present in the in the in the processor I mean when they are in during execution is restricted to k where k is the number of stages in the pipeline processor. And we have already seen that peak IPC is instruction per cycle it is just the opposite of CPI cycles per instruction and IPC peak IPC is equal to 1. So, this we shall be used as our reference. So, there was a paper published by Joe P he classified various ILP machine and this is the first classification where he classified the baseline scalar processor and these are the various parameters for that processor. Now, we have discussed that scalar pipelines have got many restrictions and first one maximum throughput is bounded by one instruction per cycle. So, IPC is less than 1 or CPI is greater than 1. So, what is the solution to this? How we can achieve better performance that means, how we can achieve IPC greater than 1 or CPI less than 1 obvious solution is make the pipeline wider. So, make the pipeline wider which is also known as super scalar. So, by doing that you can increase your I mean IPC can be instruction per cycle can be greater than 1 and your CPI can be less than 1. And we have seen that there are two paths in this direction. I have already mentioned it in the in my last lecture one is your VLIW very large instruction word and in the last lecture we have discussed in detail how it really works and we have seen that the compiler has complete responsibility for selecting a set of instructions to be executed concurrently. So, the compiler identifies instructions which can be executed in parallel then several instructions are concurrently issued because this of course will require multiple functional units present in the processor. And for this VLIW processor we have seen that the hardware requirement is simple because it does not require complex issue hardware. However, it requires a smart compiler or it will require what is known as optimizing compiler. That means the compiler has to analyze the instructions after fetching from the memory it has to analyze they have to be kept in a buffer then they will be analyzed to identify which instructions can be issued in parallel. And then the compiler will bundle several instructions together and we have already discussed this in detail in the last lecture. And another alternative is super scalar processor that we shall discuss and we have already mentioned about VLSI. So, based on the classification done by JOP this VLIW has got issue parallelism that is M instructions per cycle. It can issue M instructions per cycle obviously it will depend on the machine parallelism available in the program. Although maximum possible is M as you can see in this you cannot really fill off all the fields and as a result the actual parallelism that is possible will be less than M. So, here also the operation latency is equal to one cycle because as you can see here the after one cycle here the outputs are generated after one cycle here also. So, the issue parallelism is M instructions per cycle of course, M is the maximum value and operation latency is equal to 1 and machine parallelism is equal to M into K because M is the number of instructions that can be issued in parallel. And you can see if all the fields are filled up then you can see simultaneously you can have M into K means you can see here the pipeline depth is 4. So, 4 into 3 in this particular case you can have 1 2 3 4 4 into 4 16 there is a possibility of 16 instructions getting executed in parallel that means instructions may be in flight. So, that is why the machine parallelism is equal to M into K and peak IPC instruction level parallelism that is possible is M instructions per cycle or one VLIW instruction per cycle. So, one VLIW instruction will comprise M instructions here M is the number of fields present in the machine. And we have seen that this VLIW processors have got some drawbacks number one is it has got large number of registers needed in order to keep the functional units active and these functional units are needed to store operands to store results and whenever you feel I mean say we pick up 4 instructions to be executed in parallel. Obviously, you will require large number of registers present in the processor. So, this is one drawback unless the processor has got large number of registers it is not feasible then large data transport capacity is needed between functional units and the register file and between the register file and the memory. So, you can see your instruction is quite long that has to be fetched from the memory then those particular instructions will be loaded in the instruction register. And then instruction register will give I mean will apply the input to the execution unit which has to fetch the operands from the registers. So, you have to fetch the operands from large number of registers because you have to feed them to several functional units simultaneously. So, and so between functional units and the register file and between register files and memory. So, this is the large data transport capacity or bandwidth is required and third is high bandwidth between instruction cache and fetch cache. So, you know you have your program stored in the as you know nowadays all processors use cache memory to store the program. So, instruction programs are stored in the instruction cache and data is stored in the data cache. Later on we shall discuss in detail about the cache memory and obviously you will require high bandwidth between instruction cache and fetch cache. And why it is required because one instruction with 5 operations in a particular instruction and each of 32 bits will require 160 bits per instruction. So, this example shows why you require high bandwidth between instruction cache and the fetch unit. So, there are other drawbacks large code size partially because of unused operations and wasted bits in the instruction words. So, we have seen although we have used a large instruction word, but a large number of fields are unused. So, primarily because of this you know you have got large code size partially because of unused operations and wasted bits in instruction words. So, although it is there, but a significant portion is wasted. Another very important aspect is incompatibility of binary code. You know whenever a next generation processor is introduced, it is expected that it will be backward compatible with the next processor, new generation of processor. That means, we have seen that you can execute 80386 program in a 80486 machine or a 80486 program can be executed in Pentium, but in this particular case there is a problem. Problem is arising because for example, if for a new version of the processor additional functional units are introduced, then the number of operations possible to execute in parallel is increased. For example, earlier you are having say four functional units and obviously, your processor that VLIW instruction was having four fields, this was your instruction. Now, a new generation this is the old machine and suppose you have introduced a new machine has been introduced where you have got five functional units. So, the instruction will change, so it will be like this and as a consequence the program for this machine cannot be executed in this new machine. So, backward compatibility is lost. So, this is referred to as incompatibility of binary code. So, because of these limitations VLIW processors, although I have discussed in detail for the sake of completeness and commercial processors have been implemented, commercial VLIW processors have been implemented. For example, transmitters Crusoe processors because of low power and but they are not very commercially successful. So, commercially successful processors are superscalar, but before I discuss superscalar and variations of superscalar, let me touch upon one important aspect that is your limits on ILP. You know all our processors that we shall be discussing will heavily rely on the instruction level parallelism that is available. Now, if the instruction level parallelism that is present is small, then obviously there is no gain in implementing a processor with large instruction field or processor which can issue large instruction simultaneously. So, lot of simulation studies have been carried out by many researchers to identify the limits on instruction level parallelism and there are two extreme observations number one which was by Flyance that is known as Flyance bottleneck and here it says that instruction level parallelism available in the basic block is always less than 2, it is 1.8, 1.87, 1.96. So, it is less than 2 and that means if you do not do specialized operations like loop unrolling, software 5 lining which I have discussed, then if you simply restrict to the basic block as it is present in a program, then the instruction level parallelism that is available is only 2. However, contradictory results were published by other researchers. For example, Fisher along with his colleagues published a paper back in 1984 where he said that the instruction level parallelism available is 90. So, there is a big gap between 2 and 90 and what he did actually he identified some programs where you know essentially numeric processing is involved and there he found that the instruction level parallelism available is 90 and in fact, later on this was referred to as Fisher's optimism. So, you can see there is pessimistic view about the instruction level parallelism available that means it is 2 of 2, on the other hand there is a optimistic view which says that instruction level parallelism is much larger. Subsequent researchers have confirmed that this 90 is really too big, but definitely more than 2 instruction level parallelism is possible. So, 3, 5.8, 6, 7, so these are the different type, different instruction level parallelism which are available and obviously people will be interested in exploiting the instruction level parallelism that is available in programs and for that if necessary special techniques like software loop unrolling, software pipelining those techniques are to be incorporated to increase the instruction level parallelism. So, with this background let us discussed the motivation for superscalar processor, why superscalar processor is proposed or required. You can see here, here the vectorizability of a program that means a particular program which can be vectorized vectorizability means parallelism. So, you can see vectorizability it varies from say may be 0.8 to 0.4 to 0.8 and this is the typical range. Now, suppose you have got two processors, one processor is where n is equal to 1 I mean n is the number of stages in this particular case it has been n is the number of stages. Now, n is the number of stages and you have got a simple pipeline processors obviously speed up will vary depending on the vectorizability or parallelism available in the program. So, you can see depending on the parallelism available the speed up will vary if n is equal to 4, if you have got 4 stages maximum value is 4 and this will drop sharply as the vectorizability parameter decreases. Similarly, for say for n is equal to 6 that is the n is the number of stages the maximum possible speed up is 6 as we know and this also drops rapidly as the vectorizability or that it may be called as instruction level parallelism that is present that decreases. So, but if we go for say superscalar processor by superscalar processor mean the processor can issue more than one instruction simultaneously. So, this is the corresponding curve. So, in this curve as we can see speed up even is the minimum speed up is 2 that is possible and maximum speed up can be 12 with number of stages in the pipeline is equal to 6. So, for the same number of stages if we increase the number of issue from 1 to 2 as you can see here this point corresponding to number of for say ILP or vectorizability is equal to 0.8 we get 3 speed up we get 3 for conventional pipeline without any parallelism I mean more than one instruction issued. Now, you can see here as the as you increase the if you increase the number of issue to 2 the speed up jumps from 3 to 4.3 for n is equal to 6 n is the number of stages and f is equal to 0.8 that is parallelism, but m is equal to 2 that means number of instructions that is been issued is equal to 2 instead of m is equal to 1, m is equal to 1 corresponds to that scalar pipeline processor. So, this simple observation tells you that it is essential to go for superscalar processor to achieve higher speed up and higher speed up means higher performance. So, superscalar processors you can have statically schedule superscalar processor that is one version and later on we shall discuss about the dynamically scheduled instruction processor. So, in case of statically schedule superscalar processor you can have multiple issue in order execution and later on we shall discuss about dynamically scheduled processor where you will see out of order execution is possible. So, this is the superscalar processor proposal where you will try to go beyond simple instruction pipeline to achieve instructions per cycle greater than 1 dispatch multiple instructions per cycle that means the processor whenever it is executed I mean executing a program it will issue more than one instructions and it may be 2, it may be 3, it may be 4. So, it will dispatch multiple instructions per cycle and provide more generally applicable form of concurrency. You see vector processing vectorization or vector processing is a kind of a parallelism where you know whenever you are executing a loop different iterations are independent. So, since different iterations are independent in a vector processor or different iterations can be executed simultaneously that is the that is one kind of parallelism. But that kind of parallelism is present only when a program involves vector processing, but conventional programs may not be may not always contain vectorizability or processing of vectors. So, in such a case how to provide in a concurrency that is what is being done in superscalar processor. So, it is geared for sequential code that is hard to parallelize otherwise and obviously it will exploit fine gain on instruction level parallelism that is your superscalar processor and how is it done is shown with the help of this diagram. Here you have got we are utilizing spatial parallelism you know in case of conventional pipelining we have seen we tried to use temporal parallelism. So, instructions were issued one after the other in time at the interval of one clock cycle, but here what is being done several instructions are simultaneously issued. So, issue parallelism is M instructions per cycle. So, this is the parallelism that is present in a superscalar processor. So, three instructions are issued here in the next cycle. So, another three instructions are issued in the next cycle another three instructions are issued and operation latency of course, it remains one cycle as you can see the first result is coming out after k cycles k is the depth of the pipeline and then after one cycle you get another result. So, operation latency is one cycle in this case also and machine parallelism is equal to M into k. M into k as I have already told you can have the total number of instructions in flight simultaneously is M into k and the peak IPC, peak number of instructions per cycle that you can achieve is M instructions per cycle. So, that speed up you know this is the ideal case M instructions per cycle, but obviously the speed up factor will be less than one because there will be some stalls. So, here when there is no stall when there is no dependency the instructions can be issued in parallel depending on the parallelism observable in the processor then we get an ideal speed up that is equal to M, M into k. However, we cannot really do that in real life practical programs where there will be dependencies and as a consequence there will be stalls. So, in this particular diagram does not show in stall. So, that speed up factor will be less than one. So, M into point something so that will be the value that we will get. So, peak IPC will be in practice will be less than M. Now, based on this supercalor processors have been introduced commercially. So, commercial desktop processors now do four or more issues per clock and even in the embedded processors market dual issue super scalar pipelines are becoming common. So, we have already seen that minimum instruction level parallelism that is present is two even without doing you know loop unrolling and other sophisticated things in the basic block of the program that instruction level parallelism available is two. So, super scalar processor with issue rate of two is quite common and then of course, you can go for go beyond two four or more. Here is some example. So, here two processors are shown the first one A this corresponds to the five stage I 486 that is your Intel 486 processor where no parallelism I mean super scalar technique was introduced but it is a simple scalar pipeline with five stages instruction phase decode stage I decode stage II decoding was complex the reason for that was you know these processors use complex instructions. So, since they use complex instructions decoding is little complicated that is why decoder stage was divided into two stages decoder stage I and decoder stage II then there is an execution stage and write back or operator row that was the 486 processor pipeline. So, it is a scalar pipeline now when Pentium was introduced Pentium was having a parallel pipeline with two. So, it is a super scalar processor with of degree II. So, as you can see here it will fetch two instructions decode two instructions then issue two instructions because it has got two pipes U pipe and V pipe two separate pipelines through which two instructions can be issued and we will get executed in parallel. So, the D2 you have got two separate stages execution and write back. So, this is the super scalar processor first introduced in Pentium. Now, let us focus on the performance that you can achieve whenever you go for super scalar execution. We have seen that k stage baseline pipeline processor will require n tasks and it will require k plus n minus one clock cycles. Now, let us see what is the time required to execute the same program ideally. Obviously, we shall go for the ideal one k stage m issue super scalar processor. So, here you have got say let us assume there are four stages in the pipeline and three instructions are issued simultaneously. So, three instructions are issued simultaneously then in the next cycle another three instructions are issued in the next cycle another three instructions are issued in this way it continues. So, this is the super scalar processor of degree III. So, in this case what is the time required to execute n instructions. So, for the sake of generality we shall consider it that it has a degree of m, but in this example it is a degree shown is 3. Now, as you can see here the after k clock cycles k is the number of stages it will result for three instructions will be available. Then in the subsequent clock cycles as you can see each time you will get result from three instructions. So, here the first three instructions will require k clock cycles then you are left with n minus n minus m instructions. So, n minus m instructions will be results will be produced in n minus m by a three instructions are three instructions results of three instructions are produced per cycle. So, this by three. So, this is the sorry let me put m in general. So, this is the time required k plus n minus m by m. So, this is the number of you can say t m 1 for the super scalar processor. Now, what is the speed up? What is the speed up of this processor with respect to our baseline processor? For our baseline processor we have seen the baseline that is your baseline pipeline processor the time required was k plus n minus 1 that was the time required. On this case in this for the super scalar processor of degree gain time required is k plus n minus m by m. So, speed up is equal to t 1 comma 1 by t m comma 1. So, that means, you have to divide this by this and you will get is equal to m into k plus n minus 1 by this factor n minus n plus m into k k minus 1. So, this is the speed up they will get in case of your super scalar processor. So, this is m 1 this is shown here. Now, whenever you are executing large number of instructions then n is infinity or n is equal to infinity this is speed up limit s m comma 1 that becomes is equal to m. We have seen in case of your normal processor normal processor means pipeline processor we are considering pipeline processor the speed up was k with respect to non pipeline. Now, in this particular case speed up is m with respect to the baseline pipeline processor. So, if we consider the speed up with respect to the original non pipeline processor the speed up will be equal to maximum speed up with respect to the non pipeline processor will be m into k or machine parallelism is m into k as expected with respect to the baseline pipeline processor the speed up is m. Now, here is some comparison between the VLIW and superscalar processor. We have seen that VLIW in case of VLIW compiler finds the parallelism in superscalar hardware finds the parallelism. So, VLIW simple hardware superscalar will require more complex hardware and but one important point that you should notice here is that VLIW has less parallelism because it can exploit less parallelism because you know that parallelism is extracted with the help of a programmer compiler and compiler cannot identify all the parallelism that is present in the program at compile time. On the other hand, since it is done by hardware by the superscalar processor at run time it can identify more parallelism instruction level parallelism as a result superscalar gives you better performance. So, ideally you know we have seen for both the cases speed up is m but in case of VLIW lesser number of fields of the VLIW instruction will be filled up. On the other hand in case of superscalar processor more number of instructions will be executed in parallel because parallelism is extracted with the help of a hardware. So, superscalar will give you better performance. Now, let us look at the superscalar organization machine cycle time is shorter than the baseline processor the cycle time is another variation that is super pipeline. We have discussed about simple pipeline or scalar pipeline we have discussed about VLIW we have discussed about superscalar. Now we are considering another extension that is known as super pipeline what is the basic idea behind the super pipeline processors. It has been observed that we have seen that the different stages of a pipeline processor do not take are not uniform because you know we are trying to combine different types of instructions which are not same. So, as a consequence there is a possibility to further divide a particular stage of a pipeline. For example, suppose originally a processor was having say four stages. Now each stage this is the basic number of stages now this is further divided each stage is further divided into two sub stages. So, you can say you have got a major cycle as defined by the pipeline now you have got minor cycle. So, in each cycle some minor cycles are introduced what is the advantage that is that you get advantage is now as if the number of stages is increased. So, you have introduced n minor cycles as you introduce m minor cycles the clock frequency is the cycle time is now becomes 1 by n of the baseline processor. So, in this particular case I have shown n is equal to 2. So, the clock frequency doubles or the cycle times is 1 by n cycle time is reduced. So, this is 1 by 2 of this now what benefit do you get out of it? Benefit is now another instruction is issued after a minor cycle. So, instead of waiting for the major cycle instructions are issued after a minor cycle. So, as a consequence although the initial delay is same as the pipeline processor that means it will again take k clock cycles or k into n you know that clock cycles in terms of this super pipeline processor. But subsequently you will get result at the interval of 1 by n of the cycle time of the basic pipeline processor. So, your throughput will increase and it has been found that your m I mean whenever you have got super scalar processor of degree m and super pipeline processor of degree n when m and n are equal you get more or less same performance. But what is the difference between the two? Difference between the two is in this particular case you can see the number of I mean the clock frequency is increasing cycle time is getting reduced and the processor has to run at a faster with a faster clock. So, here it has to run at a faster clock and however the number of you know that number of instructions that will be if you consider the throughput it will although it is same, but the clock frequency is increasing. So, it is characterized by output latency of 1 cycle that is n minus cycles and I l is equal to 1 minus cycle and I p is equal to 1 instruction 1 instruction per minus cycles or you can say n instructions per cycle. So, here the machine parallelism is again is equal to n into k and in case of super scalar processor we have seen it is equal to m into k when m and m and n are same the parallelism that is available for both the machines is identical. So, it is not different, but you are achieving the same performance using a different approach altogether different approach. And it may be considered as a deeply pipeline processor with n into k stages. So, if you want to say in simple terms you may say that again it is nothing but a pipeline processor only thing that the number of stage has been increased earlier it was having k stage now it is now it is having k into m stages, but the although this statement is correct, but in practice there is some difference what is the difference? Difference is we have seen whenever we go for forwarding in k whenever we go for forwarding results from the pipe the buffers are applied to the functional units. That means, your from this from output of this output from here is fed is taken output from here is taken output from here is taken output from I mean from these intermediate stages are taken, but not you cannot take from the minus cycles. That means, the output from the at in the output available that is available from the minus cycles are not accessible are not available. So, that means, whenever you go for implementing forwarding which is necessary to overcome data hazards as we have seen by hardware means there you will feel the difference there you cannot consider it as a pipeline with m into k stages there you will consider it as if it is a pipeline with k stages. So, there lies the difference. So, outputs of some stages cannot be accessed for forwarding which is available for minus cycles and some super pipeline processors have been designed one is cdc6600 that was designed k1 was also super pipeline and MIPS R 4000 is also a super pipeline processor which has got 8 stage pipeline with 2 minus cycles. For example, 2 minus cycles per basic cycle. So, it has got instruction fetch that is the first stage and instruction fetch second I a stands for instruction fetch second. So, you can see instruction fetch is now divided into 2 stages. Then the second stage that you know that instruction decode and register read or the execution stage is again divided into 2 stages register fetch and execution then that then data fetch and data second that is this stage is also divided into 2 stages. Similarly, the right back stage is also divided the tc stands for tag check you know that you are reading from the cache memory. So, name has been tag check. So, you can see the it has got 4 you can consider it as a 8 stage pipeline. However, here it has got 2 minus cycles in it stage. So, this is an example of super pipeline organization. So, according to the classification of JOP this is the super pipeline processor where cycle time is 1 by n of the baseline processor issue parallelism is 1 instruction per minus cycle. So, you can see you are issuing 1 instruction for minus cycle output less and less latency is m minus cycle. So, you can see after m minus you are introducing output generated after m minus cycles and peak IPC is n instructions per major cycle. So, n into speed up. So, here you can achieve as I have said machine parallelism you can achieve is n into k. So, in a similar way you can find out the performance of the super pipeline processor and time required is you can say t 1 n is equal to k plus n minus 1 per n. So, you can consider it as a pipeline processor and in that way you can find out and speed up is equal to t 1 1 that is the super pipeline processor by t 1 n that is super pipeline processor we get k plus n minus 1 by k plus n minus 1 by n. So, this is the speed up that we get n into k plus n minus 1 by n k plus n minus 1 and whenever n is very large then you get s 1 n is equal to n as expected. That means, with respect to the base pipeline processor we get a speed up of n. However, whenever we consider the speed up with respect to non pipeline processor we shall get a speed up of n into k. So, here always we are considering with respect to the pipeline processor that is why the speed up limit is n, but that is again the ideal situation. Now we can extend the idea further and we can have super pipeline super scalar organization. That means, we can combine super scalar along with super pipeline and processors commercial processors are available for that and here the processor executes m instructions every cycle with a pipeline cycle 1 by n of the base line processor and it is characterized by l is equal to 1 cycles is equal to m minus cycles or i l is equal to 1 minus cycle and i p is equal to 1 instructions per minus cycle or n instructions per cycle and here machine parallelism is equal to n into k. Now it will be n into m into k I believe. So, here it is wrong machine parallelism will be n into m into k because it is you can see here the number of instructions which are number of instructions in flight during execution will be equal to n into m into k. So, machine parallelism will be m into n into k. So, this you should modify. So, the execution of instructions for a super pipeline super scalar processor of degree 3 comma 3 is shown here. So, 3 instructions are issued and after each minus cycle another 3 instruction is issued. So, after 3 minus cycle another 3 instruction is issued. So, this is how instruction issue is taking place and execution is taking place this is the ideal situation. So, according to the classification by Jopi again i p is equal to m instructions for minus cycle. So, p is equal to n minus cycles and i p is equal to m into n instructions for major cycle. So, this is the performance of super pipeline super scalar with degree n here t m n time required to execute n instructions will be equal to k plus n minus m by m n. So, you see you are dividing it by a factor of the first n instruction first k instructions will I mean first m instructions will take k cycles and after that the remaining n minus m instructions will require n by m by m n cycles. So, based on that you get a speed up which is equal to which will be this is this will be equal to s m n s m n is equal to n m into k plus n minus 1 by n m k plus n minus m. So, speed up limit in this case is m n as expected. Now, we can we have seen that inefficient unification of instructions lead to some problem because we try to execute to combine different types of pipelines which are not identical. So, ALU operation memory operation. So, instead of that we can go for diversified specialized pipelines as it is shown here ALU through one pipeline memory operation through another pipeline floating point through another pipeline. Instead of trying to combine different types of instruction in a single pipeline we can have separate diversified pipeline for different stages and along with that we can combine super scalar execution. And another limitation that we have seen that is because of rigid nature of in order pipeline. So, this problem can be overcome by having out of order execution and distributed execution pipeline as you can see here. However, this will require dispatch buffer that means you will require some buffer where the instructions will be temporarily stored. So, multiple entry buffer will be required for example, in this case this dispatch buffer through which you will be storing several instructions multiple instructions and then you will the order in which the instruction cell will be that will be issued from dispatch buffer may not be same as the program order. That means that this from the dispatch buffer instructions will be issued out of order. So, whenever the instructions are issued out of order there is a possibility that the instruction the instruction will be executed also out of order. So, out of order execution will take place and results will be stored in another buffer known as reorder buffer. So, the reorder buffer the order in which the results are stored in the reorder buffer you know ultimately you have to convert them in order the way it is available in your program order. So, from this reorder buffer the outputs are generated as it comes out from the program order. So, then the writing in the processors is required. So, the processor design superscalar processor design will involve different functional units fetch, decode, dispatch, execute, complete, retire along with that you will require different types of buffers instruction buffer dispatch buffer issuing buffer completion buffer store buffer. And these designs will be we shall in my subsequent lectures I shall discuss about the design of superscalar processor involving this type of buffers and functional units diversified functional units in my next two lectures. Thank you.


Better varieties means better yield, more say maybe sometimes better form or better quality of proteins or better quality of other products like eggs or milk. The biologists actually study the life cycles and the data about races of different organisms like cows, like buffaloes, like goats or other organisms and through improved breeding techniques we can use better varieties which can yield more proteins, which can yield more milk or which can yield more eggs for us. Very importantly in farming, in poultry and dairy farming diseases are a major factor which can sometimes destroy the whole form, the whole form of the animals. In disease control, biology helps us because biologists, they are the scientists of the living organisms, they study the mechanisms of disease, they study the treatments of their diseases and they can tell different the causes of the disease that this disease is caused by this particular maybe organism, maybe chemical, maybe food or something else, they also can help us in making their treatments. So in dairy and poultry, we can also make better feed for these organisms, we can control diseases, we can make better varieties to get more yield and better products. Now the apiculture, honey bee keeping, honey bees are also kept by people from centuries. People grow them for getting honey, large honey bees, small honey bees, both are cultured. Honey bee, we know that biology tells us that honey bees are social insect and it makes colonies. There is a queen bee which lays eggs, there are drones which are males and which fertilize these eggs and there are workers, the worker bees, which actually collect the nectar from different flowers and convert it into honey. Biology helps us in improving the varieties of honey bees that we can improve better variety, we can provide them with some specific flowers which have more nectar, for example, which is a better nectar for them and also we can help, biology can help us in the disease control. The biologists also tells us, does the farmers that how to deal with these honey bees, that is at which time they can collect honey and how can we deal with the honey bees, so that they do not harm them and they can collect the honey. So biology is very useful, in biological knowledge is very useful in honey bee farming which is called technically apiculture, here we can see a farmer collecting, separating honey bees from its home, house, then comes the biological pest control, we can control the pests which are attacking our crops or which are attacking our animals, dairy or poultry farms by biological means, what is a pest? A large number of organisms damage our crops or damage our animals by acting as parasites, attacking upon them, these are called pests, for example rats, rats are the pests of different types of crops, ticks and mites, they are we can call them pests or we can we actually call them parasites upon buffaloes and cows.  We control these insect pests by various means, initial old method was a crop rotation that is we change our crop in different seasons, that is once we grow another crop, second time we grow a different crop, we use chemical controls, we put different types of chemicals on our crops to control the insects, but insects become resistant and also these chemicals are dangerous for our crops, biological control is a much much better method. We can actually use the natural predators of the pests to control these organisms, that is we introduce the predator of the pest in that field and these predators can eat upon those pests, the result is our crops are safe and the benefit of this biological control is this, that this is a permanent control, that is those predators which are introduced, they will stay there and they will control the crops, control the pests in maybe next year or for long term, secondly these are safe, they are not chemicals, they are organisms, they do not harm our environment or harm our crops, but this is extremely important in the selection of a biological control mean any of the organism that it should be safe for our environment, that it should not happen that they eat upon the pest and they also eat upon our crop, so selection of a biological control method is also extremely important. Now applications of biology in growing crops and plants, as we know that plants are producer part of the ecosystem, they are very important, we grow them as food crops, we grow them as other cash crops like cotton and others, we get different products like vegetables, we get wheat, we get perfumes, we get medicines and furniture from the plants, they are duty of the nature, they are ornamental plants, how biology help us in growing crops, number one we can provide them with more suitable quantities of water, we can grow them in a better soil in which we know that they grow better according to their life cycles, we can add better means of their cross pollination or something else and we can control diseases, we can produce biogenetic engineering or some other method to make more resistant plants to the environmental stresses or different types of environmental changes, we can make better breeds of plants which give us better yield and which grows comparatively easily in comparison to the normal plants, the other plants, so the knowledge of biology helps us in different fields of life, starting from the pharmaceutical industry or health or medicine to the agriculture, to the animal culturing and to all other fields of life, there are lot many fields of life.


Welcome back folks to Dow.  That's a downdraft inside the Nasdaq composite folks of two...one second. Why is that not giving me? Anyway, approximately 2.9% S&Ps, 1.9% as the Dow Industries. Gold. Gold contract down 90 cents, trading at $16.69 an ounce. We had silver down 8 cents, $18.80 an ounce, light sweet crude off 60 cents, $81.55 a barrel, notes and bonds. A 10-year note down 18 ticks, trading $112.09, the 30-year off 27 ticks, at $126.27 in Kingdala. Kingdala was down 439 ticks, $112.165, Euro is at 97. Yen is out here at a price point of 144, the British pound, is at a price point of $110 to one US dollar. We get over to the S&Ps, bottom line, the S&Ps didn't have the volume for an ABC structure down, closed pretty much to the lows. You had 104 million, you needed 120 million, 108 million to do an ABC down. You still might have it then, you still might get it. The B point on there is at 360.87 here, 362. NQs, a different story, bottom line, you get the volume on the NQs, let's see if it's closed below it, it did both. Okay, the volume, you needed the volume of 74 million or at 77 right now. Close, the B point is 272.02, you're 271.87. That also took out the low that was established out here on the 16th, and it looks like we'll have the volume there, so the Qs will do another 5 or 10 million. And right now, you're at 77, that low of June was 81, you're coming into a Friday, yeah, we'll see while this baby's shaking out. Gold, gold contract caught a bid yesterday, pulled back today with light volume, rejected lower price, so gold looks like it wants to go higher, and that's going to be all about the US dollar. So we did with the gold contract yesterday, we got down to the 1622 area, rallied to 1671, had over 200,000 contracts, 280,000 contracts to be exact, so good contract volume. Have a great night folks, have a safe night, you get option expiration tomorrow, we get action happening, come back, visit Tommy tomorrow morning, kicks us off 9 a.m., thanks for all the prayers, the white light, we missed the bullet, it's a beautiful thing. Look at him folks.


Here we go then, he number nine at the Scandinavian FAM, Speedway Grand Prix, Max Frick, Jakub Torschel, he goes off gate two, then it's Leon Madsen on a warning, he's off gate three and he leads the world championship by a point over Bartosz Marzlik. Yeah, it is Doyle off the outside, Doyle is looking for points, came through the second place but Frick on the inside, but Madsen favourite here, what can he produce from gate three this time? Yep, so he's nine it is then, here we go! Max Frick has got there, watch out for Madsen, oh Madsen gets so much drive in white and round the outside, Jason Doyle, what a move from the 2017 world champion. Madsen now comes back on the inside, great move from Leon Madsen, Max Frick is third and the man in the back is Jakub Torschel, but what a race here! Brilliant step from Doyle to repay the compliment, Madsen once again stunning round the inside, got so much speed, look at him climb his push from Leon Madsen, but Doyle has just pulled off a fantastic ride, second place Madsen, Max Frick third, that was another great race and well done Doyle and Madsen putting on a super show up front. Well he worked overtime there, Jason Doyle, he had to have all his strength there, his upper body strength there because a couple of times on the outside turned to the bike, lifting quite violently. Madsen's bike is so rapid, once again running hard in second place, but Doyle needed that big win for him!


Okay, let's continue. I'm super excited to have Holden here again this year. And she's gonna talk about accelerating big data beyond the JVM. Awesome, thank you, thank you for having me back. I don't know if last year's talk was good or there just weren't too many people. Regardless, I hope you're interested in going beyond the JVM. So I'm Holden, my preferred pronouns are she or her. It's tattooed on my wrist in case you forget. I am a developer advocate at Google. They pay me money and therefore are excellent people if you consider buying things from them, please do. I am on the Hashi Spark PMC. This colors my views of the world, but not that much because it also makes me hate Spark because I work on it. Previously I worked at IBM with Nick. He still has not merged my pull requests. I'm very upset with him, but I am planning to track him down before his flight and a bunch of other companies as well. I'm a co-author of two Spark books. I think you can buy them from the low early booth upstairs if you want. One Spark book is about a quarter of a cup of coffee in San Francisco, so you can help keep me from having to do enterprise support contracts, although I already worked at IBM. You can follow me on Twitter. There's not a lot going on. It's a lot of pictures of Boo who, while adorable, does not do a lot of programming. She's great. In addition to who I am professionally, I'm trans, queer, Canadian, on a work visa in America, which is just really fun right now and part of the leather community. And this maybe doesn't seem super important, but I think it's just nice to remember that we all come from different places and we should work together. There's no secret Canadian garbage collector or anything that we're hiding from you. We all have out-of-memory exceptions. Our lives are all terrible, but if we work together, our lives can be terrible together. And I think I just saw someone having the Spark book with the bird on it. Yay! Okay, so thank you for the quarter of a cup of coffee. I really appreciate it. So I'm hoping you're nice people. If you don't like pictures of cats, this is not the talk for you. There are two exits, but you can also just work on your computer. It's fine. It's possible that you... How many people are Spark users? Oh, that's a lot of people. That's awesome. Okay, oh, and Nick, merge my PR. How many people are Flink users? There is some overlap, but not as much as I was hoping. Now for the real long shot, are there any Apache Beam users in the house? Wow, we should talk afterwards. I'm really excited to see if there's some Apache Beam users here. That's great. Is there anyone who doesn't use any of those systems? Cool. What do you do for big data? Or is this more just like whatever there's a talk sort of situation? Okay, it's the whatever there's a talk situation. That's fine. That's all right. I will try and convert you to one of the big data systems which indirectly pays me money. It's pretty much all of them at this point. I'm very much hedging my bets. So how many people know some Python? Oh, thank God, my people. How many people are our programmers? I am really sorry.  There are only like six people. That is weird. Okay, the slide has clearly gone on for too long. Okay, well you're in the big data room, so whatever. This is for big data, for those of you that don't have a big data system, please pick one, preferably one that pays me money. Most big data systems are written in the JVM, not all of them. There's Dask and other exciting things like this which have decided that the path to happiness does not involve Oracle. And I respect that choice as well.  So there is no escaping the wonders of the JVM. We're gonna look at how PySpark works. We're gonna look at how other similar systems work as well. We're gonna talk about why this kind of sucks, and we're gonna also talk about why things are finally changing, and how the experience for Python users is finally actually getting better. And I'll try and get you to buy my book many times over. The TLDR is we've made some bad choices. But we're finally fixing them because investors noticed. And yeah, moving along. Okay, so what is the state of non-JVM big data? It's a wonderful mixture of Pickling, Strings, JSON, and XML, often all inside of the same project. Yay! I love three different serialization libraries involved in calling a single function. No one is excited. Okay, the other one is sometimes people look at that and they go, you know what? That doesn't sound slow enough. I'm gonna use Unix pipes. Or I alternatively, this sounds too portable. I'm gonna use Unix pipes. And that's often used, not always. Some other people look at this and say, maybe I'll use Sockets to connect Python and Java. And alternatively, other people look at this and they go, you are all just out of your minds. I'm just gonna rewrite this in Python. How hard could it be to rewrite all of Spark in Python? I'm sure there wasn't a lot of development work that went into that. I'll just like read a paper over the weekend. And that's how we got Dask. And then they did a lot of work. And it got a lot better. But it's that classic problem of, well, what does your field do? I bet I could do that with a Perl script. But Perl is no longer cool. So we use Python. So PySpark, are there PySpark users in the house? Yay, and I'm sorry. So PySpark has the same general technique for how most of the other non-GVM languages work with Spark. It turns out most of the other big data systems looked at Spark and went, that was interesting. And took different directions to integrate languages. But fundamentally made pretty much the same trade-offs. And it has some serious performance issues that we're finally gonna fix, maybe. Okay, and because this is a big data talk, I had to put in a word count example. It's pretty much, it's a licensing requirement. I'm from California and we're very serious about not getting our licenses revoked out there. So this is the word count example.  They're our friends. We can tokenize things. I can add numbers together. It's very exciting. Anyways, we'll move along. So how does this monstrosity work? We take pickles. I guess we couldn't figure out an image to represent JSON, so we just wrote JSON. We could have probably just had a programmer crying. I don't know. Okay, yeah. So how do we take, how does Spark do this? Spark uses Py4j on the driver side. And the driver is sort of like it's the thing which is sort of coordinating a lot of the work. And it's where your Python program, the part of it that you see runs. And it's the part which throws exceptions which uses JSON and magic. And then we use pipes because that's a great idea. We also don't use the standard pickling library because that was far too, uh-oh. It worked like that. And then we decided that we were just going to write our own pickling library. But we stole it from another project, copied it in and called it cloudpickle.py. Someone copied it out and has been fixing bugs ever since that we have been back porting about a year after they fixed them. And yeah, okay. And there's a way around this. We can use Spark SQL, but then we lose our Lambda functions and it makes us sad. So this architecture diagram, in addition to showing that I do not work with designers, shows Python, talks to Java, and then Java is the one who does a lot of coordinating. And then this, you know, sadness happens. In Flink, the same thing happens except for place all of the things with other things. And instead of sockets, use mmap. I'm pretty sure this is how it works. I looked at the code this morning. And so how does this impact PySpark? I mean, I'm pretty sure it's an mmap file. There were a lot of bugs about an mmap file behaving poorly. And I was like, that is probably a core piece of your tech. So we have this really expensive double serialization that's happening. Python Worker Startup takes a little bit of extra time. And the error messages don't make any sense. Excitingly, we're not gonna solve this one. We're gonna make this one worse, but we're gonna try and solve this one. So that's okay. So Spark DataFrames, they're nice, but they're not perfect. We're just gonna sort of skim over them. But we can see that when we take out the serialization cost, the blue is like good PySpark code. And then the yellow, the hideous yellow is the PySpark DataFrame code where we don't actually do any of our work in Python. And so we can see that this is a pretty big difference. This Python overhead is huge. And so if I can reduce the amount of overhead, I'm gonna see some pretty big improvements. And so we're gonna take a look at this problem. And we found out we were spending most of our time copying data from Java into Python and back. And then it turned out we were actually spending a lot of time serializing and deserializing the data. And when I looked inside of all of the other open source projects, everyone had their own custom serializers because apparently, that is a great way to add value in open source is adding a new serializer. And so yeah, so by the future, actually in theory, we're voting on the release right now. So this should probably be one month's to eternity. But I'm optimistic that we'll actually get a release out in February, probably, but it's okay. I live in California. It'll be hard for you to track me down. And so this code exists. If you wanna try it, you just need to check out this branch two, three from Spark.  How many people have heard about Apache Arrow? Five people? That's low. Okay, so Apache Arrow is what happens when your cat decides they like cadnip and serialization frameworks. Right, so Apache Arrow gives us this really nice performance graph. Big numbers are bad unless I'm selling you compute time, in which case continue to use slow serializers. And small numbers are better. But Apache Arrow is really awesome and it gives us a common format that both the JVM and Python can understand so I don't have to write custom serializers and I can start to use shared memory buffers intelligently as opposed to putting strings in shared memory buffers and being sad about my life choices.  But from the big data point of view, I mostly care about that it can talk to Java. It can also directly read in Parquet files and I can put Apache Arrow records in TensorFlow, or Python and even feed it into CUDA if I decide that I wanna give Nvidia all of my money. Which is fine, Nvidia makes great products too. And so this is really nice because this interchange has historically been really, really expensive and you can see that this, I mean when we're adding the number one, we're doing pretty much no work and we go from three seconds to 0.9 seconds and so that's really awesome. Actually this might be ours, I'm not super sure. Oh no, it says seconds, yeah, okay. So adding the number one pretty fast. And this is not obviously on one record. If it took us three seconds to add the number one to one record, we'd be very sad people. Yeah, okay, but so this is awesome and it means now that I can start to use the cool tools that the Python kids have been using because I live in San Francisco and if I don't put TensorFlow into my product, I am not raising another round of venture capital and then I cannot afford to go to BlueBottle anymore. And so this is pretty much a mission critical thing for people like me. So how do we access this? In Python, and here we're gonna add not the number one, but we're gonna add two arbitrary numbers together.  Now we say Panda's UDF rather than arrow UDF because this is the way that we expose arrow into Python is as a Panda's data frame because Panda's data frames are nice and happy. And so yeah, that's great. Okay, what do the other systems do?  Hadoop streaming is not streaming despite its name. It uses units pipes and sadness. Kafka re-implements everything, so they take sort of the Dask approach to life. And that's great unless you wanna use new features. So features from two years ago work great. Dask re-implements everything, has multiple backends, it's pretty awesome. The only problem has been historically it's difficult to get data into Dask but thankfully we can now use arrow to get data into Dask from Parquet data. And so that's pretty awesome. Beam, yeah, we don't do things beyond the JVM very well right now, but we have a bunch of JIRA issues which you are more than welcome to join me on. Yay, does anyone wanna do my job for free? Alternatively, wait three to six months and someone else will fix it for you. There's a bunch of references. I really don't have a lot of time to go into the C++ code that is scary and powers arrow but you can go check it out. There's some interesting blog posts if you want to verify the benchmark slides that I showed you you can go get them and run them on your own data. And that's pretty much it. The most important slide is this one. You can buy high performance Spark. It covers none of the things I talked about today but it is an excellent book. And if you buy it online you can receive it in a nice box that your cat will love. And so that's pretty much it. I might do office hours at six but honestly I think I'm just gonna go get chocolate. So let's pretend the slide didn't happen. I don't know if I have time for one question or if I use that up with the VGA adapter. You haven't? Okay, cool. Sorry, I saw the five minute slide and I wanted to make sure there were time for people to heckle me just in case anyone had questions about how arrow works or Spark. We have plenty of time for questions. Or if anyone wants to like buy several copies of my book I can go back to the website, you know. Oh wait, I didn't put the website name on it. Well just high performance Spark. This is the keyword to type in. I noticed very few laptops open right now. Very few laptops, that's unfortunate. Okay, hope. There's a question. Yay, questions. I'm curious a bit about the, a bit about Dask. Just, I've heard it from our users. I'm not super familiar with it, but you said maybe in the ecosystem it's not as up to date is that what you would say? Right, so I think Dask is a really interesting system and its main challenge is that it eschews most of the existing ecosystem. So it's very Python focused and it doesn't necessarily do a great job of integrating with other data sources. So if you've got like existing big data you might find it painful to integrate with your other tools. On the other hand, if you're sort of like starting from like a small data shop which has then realized now you have customers and you have big data then Dask can be pretty cool. It also doesn't have a great machine learning story right now, although some people might be working on that hypothetically speaking. And honestly it does a really nice thing. Dask data frames actually look and behave like Panda's data frames to a much larger degree than any of the other distributed systems. All of the other distributed systems approach to data frames is really great. Yeah, really delightful and unique, right? There's a camera and I like my job. Yay, a second question. Yeah, and there's a coworker in the back row. That's a good thing I didn't say what I was gonna say about data frames. So if I understood correctly, Arrow wants to provide the serialization format across Java, Python, R. Yeah. So why not something like Protobufs or Thrift? What's the difference? I mean this is a great question. And so if you look at Arrow, it actually does use like plot buffers in some places and stuff like this. Arrow is more than just a serialization format. The part of it that I care about is primarily the serialization format, but there's also operations that are defined on the serialized data as well, which is really nice and convenient. Lots of fancy numerical stuff. I mean honestly so we use Protobufs a lot in Beam and there's nothing wrong with those. Arrow is just much more focused on making things that look like data frames for serialization between these different systems and providing the tools you need to build nice data frame like interfaces on top of it. So we can have some of this shared code between R and Python. Most of the times we actually end up calling FORTRAN code that was written in the 70s, but now we can share the FORTRAN code that we've hopefully rewritten into C++ to switch how much I wanna gouge my eye out with a rusty spoon to be less so. Yeah, if you were to go and make your own, you could go ahead and use Protobufs. But everyone else is using this one so maybe we should all be friends together because if we all use the same thing, then I can just steal your users or whatever it is. We can collaborate. Yeah, that's why I'm asking these. Well, I mean, I think fundamentally a large part of it is probably social in that the person who started the Apache Arrow project is also the same person who started the Pandas project. And so it's like, oh yeah, this person knows what they're doing. And so it's a lot easier to gain adoption when you have a good reputation for making good design decisions. So apart from a serialization format is also focused on data frames. It's focused on data frame like data. So yeah, if you wanna serialize a random number generator, Arrow is probably not the format for you. So you wouldn't use Arrow to serialize your Lambda functions either, right? This is very much just serializing your data, your functions. And we pretend there's a difference because otherwise I have to go and relearn scheme. Okay, cool, I think that's it. If anyone wants to come find me or boo, I'll be around until I find coffee.