In this lecture, we will talk about microwave transistor. So, let us start the lecture. So, transistor is a basic building block in electronic devices. It has replaced many of the vacuum tubes in electronic circuits. The output current or power or voltage of a transistor is controlled by either the input current or the input voltage of the transistor. Now, depending upon that the transistors are divided into two categories bipolar junction transistor and field effect transistor. In case of bipolar junction transistor, the output current of the transistor is controlled by the input current of the transistor. In these transistors, the current conduction is due to two type of charge carriers electrons and holes. Now, depending upon the type of charge carriers, they are divided into two categories PNP and NPN type. In case of PNP transistor, holes are the majority carriers and electrons are the minority carriers. However, in case of NPN transistors, electrons are the majority carriers and holes are the minority carriers. Now, the most commonly used configuration of the transistor is the common emitter bipolar junction transistor configuration. Now, these transistors do not provide desirable characteristics at higher frequencies due to their high base resistance and it limits the transition frequency of the transistor. So, there is a new type of bipolar junction transistor that is known as hetero junction bipolar transistor. It utilizes the different semiconductor materials to form emitter and base junction and provides low base resistance. So, they can operate up to very high frequency range. Next type of transistor is field effect transistor. It is a unipolar device. It means that the current conduction in these transistors is due to only one type of charge carriers. It could be either electron or hole. The output current in these transistors is controlled by the input voltage of the transistor. Now, these transistors are further subdivided into various categories depending upon how the channel is isolated with the gate. In case of junction field effect transistors, the gate is isolated from the channel using a reverse bias p-n junction. In case of metal oxide semiconductor field effect transistors, the gate is isolated from the channel using an insulating oxide layer. So, that is why they are known as metal oxide semiconductor. The next type of semiconductor is metal semiconductor field effect transistor. In this the gate is on the top of the semiconductor transistor. So, in this case the reverse bias p-n junction is replaced by the metal semiconductor short key region and that is why they are known as metal semiconductor field effect transistor. The next type of transistor we will talk about is high electron mobility transistor. Now, all these transistors do not provide desirable characteristics at higher frequencies due to their internal capacitance. So, the high electron mobility transistors are made, they are the hetero structures and they can operate up to very high frequencies, they provide better performance over these transistors. Then after discussing these transistors, we will take an example of common emitter amplifier design and we will see how the various parameters affects the performance of the amplifier. Then we will talk about the application of microwave transistors. So, a bipolar junction transistor is a 3 terminal device, it is of 2 type PNP type and NPN type. In case of PNP transistors N type of layer is sandwiched between 2 P type of layers. However, in case of NPN transistor a P type layer is sandwiched between 2 N type of layer. Now, these transistors are divided into 3 regions, emitter, base and the collector. The emitter is an outermost region situated on one side of the transistor. The function of the emitter is to inject charge carriers into the collector. So, as the emitter has to inject carriers they should be highly doped. The next type of region is the base region. The function of the base region is to pass these charge carriers into the collector. So, they should be lightly doped and they are relatively very thin. And the function of the collector region is to collect the charge carriers. So, they should have relatively more space, they are situated on other side of the transistor as shown in this geometry. Now, if these transistors is to be used in circuit they are represented by these symbols. Here the arrow represents the direction of current flow. So, in general the current flows from the P type region to the N type region. So, in case of PNP transistors current flows from emitter to the base. However, in case of NPN transistor current flows from base to the emitter. Now, these regions form two type of junctions. One junction is formed between emitter and base region and another junction is formed between collector and base region. This junction is called as the emitter base junction and this junction is called as the collector base junction. Now, if this transistor is biased using DC then this is known as the biasing of the transistor. Since we know that there are two type of junctions emitter base junction and collector base junction. Now, if these junctions are biased they can be biased either in forward bias or in reverse bias. Now, depending upon that they can be biased into four regions they are known as active region saturation region cutoff region and inverted region. In active region emitter base junction is forward bias and the collector base junction is reverse bias. In this region the output current depends upon the input current and it is controlled by the input current. In this region the transistor is used as an amplifier. The next reason is the saturation region. In this region emitter base junction is forward bias and collector base junction is also forward bias. So, in this region output current becomes independent of the input current. In this region transistor is used as a closed switch. The next reason is the cutoff region. In this region emitter base junction and collector base junction both are reverse bias. Since the emitter base junction is reverse bias it does not inject carriers. So, there is no current. So, in this region this transistor acts like a open switch. The next region is the inverted region. In this region emitter base junction is reverse bias and collector base junction is forward bias. So, it does not inject any charged carriers. In this region also current is 0, but this region is not of any use to the designers. Now, to understand the working principle of transistor let us connect this transistor in active region. That means, the emitter base junction should be forward bias and the collector base junction should be reverse biased. When the emitter base junction is forward bias emitter injects the electrons into the base this constitutes the emitter current. A few holes will also pass from the base to the emitter. Since the majority carriers are electrons there will be only few percent may be around 0.5 percent current will be due to the holes passing from base to emitter. When these electron reaches to the base region they will try to combine with the holes of the base region. Since base is relatively thin only few electron will combine with the holes and that will constitute the base current. The rest of the electron will pass to the collector. Now, these electrons will be collected by the collector and this will constitute the collector current. There will be one more component of collector current which will be due to the holes passing from collector to the base. This is known as the reverse saturation current. So, in this case the emitter current is the summation of the base current and the collector current. Now, if you see here the transistor is a 3 terminal device. Now, if you make one terminal as grounded or you make it as common then this can be realized as a 2 terminal device or a 2 port device. So, depending upon the type that which type of terminal is made common or grounded they are divided into 3 configuration. If the base terminal is made common then this is called as common base configuration. In that case the collector current will be the output current and the input current will be the emitter current. In case of emitter terminal when it is grounded or made common then the input will be given to base and the output will be measured at collector terminal. This is called as common emitter configuration and the third type of configuration will be where the common collector will be made and the input will be given to the base and the output will be measured at the emitter terminal. So, this is known as the common collector configuration. Now, among these configuration common emitter configuration is the most widely used configuration due to its desirable characteristics like it provides highest voltage gain and highest power gain. Now, the collector current in case of emitter configuration is given by this expression I c is equals to beta I b plus I c e o where I c o is the reverse saturation current. Here if you see this beta I b and I c o they are temperature dependent. So, they vary when you increase the temperature. So, there are chances that it may increase the collector current and there could be a case that because of the increase in collector current the transistor may break down. So, there is a need of stabilization. So, that is why various stabilization circuits have been suggested you can look into the literature for these. Among the biasing circuits the voltage divider biasing circuit is the most common biasing circuit. So, here is the circuit corresponding to voltage divider bias circuit. This configuration makes the I c independent of the parameters which varies with the temperature. So, in that case if the reverse saturation current increases with increase in temperature there will be a decrement in base current. So, the I c will be constant. In this case if you see I c will be given by V c c minus V c e upon R n. So, it will be constant. Now if you try to draw the output characteristics of this particular configuration that is the variation of I c with respect to V c e for the constant base current then you will see the characteristic curves like this. Here this region represents the cutoff region and this region represents the saturation region. Now if you select a point in such a way that V c e is equals to 0 if you put the V c e value equals to 0 over here you will get the I c is equals to V c c upon R n. This will be the maximum collector current allowable in the transistor. Now if you choose the another point by making I c equals to 0 if you put I c equals to 0 you will get V c e equals to V c c. So, that is represented here. Now if you try to draw a line using these points you will get a line like this and this line is known as the DC load line. This line decides which point one should choose to operate in the amplifier region or in active region. Now if you see one can choose any point corresponding to various IVs in this particular region, but there are few drawbacks like if you select the operating point over here there are chances that the output signal upper cycle may get clipped. Similarly, if you select the second point somewhere here and if you try to draw the amplified output signal there are chances that the lower cycle of this output signal may get clipped. So, this will be distorted. So, one should choose a point in such a way that the amplified output signal should not be distorted. So, it is suggested that you should select the center point of this DC load line to ensure the maximum output signal without distortion. So, the center point for this DC load line will be V c c by 2 R L and 1 by 2 V c c. So, this is how one should choose the operating point. Now as I mentioned these transistors do not work properly at higher frequencies due to their internal limitations like these transistors provide higher base resistance at the higher frequency. So, the transition frequency of these transistors will be relatively less. Now what is transition frequency? So, transition frequency denotes the frequency at which the gain of the transistor drops down to 1. Now one should decrease the base resistance to increase the transition frequency. So, there are configuration which was suggested this is known as the hetero junction bipolar junction transistor. In this transistor the emitter and base region are doped by different semiconductor materials and by using heavily doped base region the base resistance of these transistors can be reduced. So, they can operate up to very high frequency range. So, here you can see that the emitter and base regions are made using different type of materials. Now these transistors provide better performance over other bipolar junction transistors. They provide low transition time due to the type of material used in these transistors that are like gallium arsenide and they relatively have high mobility. Similarly, they provide low base to emitter capacitance through to the lower doping of the emitter region. Similarly, they provide high trans inductance and output resistance. They also provide higher gain and they have high power handling capability and the breakdown voltage is also high for these type of transistors. So, there is a progress in these type of hetero junction bipolar transistors due to the development of various type of semiconductor materials and due to the improvement in the manufacturing process. Recently, hetero junction transistor is made using the silicon germanium technology and they provide the similar DC RF efficiency as the transistors provide with the help of these group 3 and group 5 materials, but they are made at relatively low cost and with relatively low complexity in the manufacturing process. Till now, we just discussed about the operation of the bipolar transistor. So, here is the example of the practically available RF transistor its name is BFP 520. It provides higher gain and low noise and the transition frequency of this transistor is 45 gigahertz. So, it is fairly good. Now, as I mentioned that the performance of these transistors degrade with temperature. So, here I have shown the variation of the power with respect to temperature. It provides the 125 milli watt power up to 100 degree centigrade and the transition frequency for this transistor is 45 gigahertz for VCE is equals to 2 volt. Here I have also listed down few other specifications like reverse voltage and the forward voltage and the output current of these transistors. So, it provides a a decent gain of around 20 dB. Till now, we discussed about the bipolar junction transistors. So, these transistors suffers from the minority carrier effect. The next type of transistor is a field effect transistor. They are the unipolar device that means the current conduction in the field effect transistor is due to only one type of charge carriers. It could be either electron or hole depending upon the type of the channel. So, in case of n type channel the charge carriers will be electron and in case of p type channel charge carriers will be hole. Now, one of the commonly used field effect transistor at low frequency is junction field effect transistor. So, in case of junction field effect transistor the structure is shown here. To p type of lightly doped regions are diffused into n type of channel and then a metallic terminal is deposited in at these terminals. This is known as gate and there are two other metallic terminals using the ohmic contacts. They are known as source and the drain. Now, if you see in this particular configuration if you do not apply any bias voltage between the gate and the source terminal which is known as the input terminal, then there will be a path for electrons to flow from source to drain when you apply even a very small drain to source voltage. So, the maximum current will flow when VDS is very small and it will again increase if you increase the value of VDS. Now, if you apply a reverse bias voltage at these junction VJS that means, the negative voltage is applied at the gate terminal the depletion region will try to increase. Now, if you increase this reverse bias voltage further there will be a situation that these two regions will touch each other. So, there will not be any passage to flow of electrons. So, there will not be any current and the current will reduce to 0. So, the current in these transistor is given by this expression ID is equals to IDSS 1 minus VJS by VP square. Here VP is the pinch of voltage. So, the situation when these depletion region touches each other this situation is known as the pinch of situation and the voltage VJS at that particular moment is known as the pinch of voltage and VJS is the reverse bias voltage along the gate to source junction and IDSS is the maximum current when no reverse bias voltage is applied. There is one more term transconductance. So, transconductance is a measure of change in output current ID with respect to change in the input voltage VJS for a constant value of VDS. Now, if you try to draw the output characteristics of this transistor that means, the variation of ID with respect to variation in drain to source voltage you will see the curves like this. Here this represents a pinch of region and this region represents the ohmic region. In this region there will not be any flow of current. However, in this region the field effect transistor will act like a voltage controlled resistor and this region is the saturation region. In this region the drain current will be independent of the variation in the output voltage and the amplifier should operate in this region if this FET is to be used for the amplification purpose. The next type of transistor is the metal oxide field effect transistor. In this transistor the gate is separated from the channel by an insulating oxide layer. Generally it is of silicon oxide. So, due to the insertion of silicon oxide it provides very high gate capacitance and the impedance of these transistors will be very high may be up to the order of mega ohm. Now, if you see here these MOSFETs are divided into two categories one is known as the depletion MOSFET another one is enhancement type MOSFET. So, in depletion type MOSFET there is a channel between the source and the drain terminal. So, there will be a flow of current when you do not apply any bias voltage between the gate and the source. So, there will be flow of current. Now, if you apply a negative bias voltage it will try to ripple the electrons away from this channel. So, the current will reduce or if you apply a positive bias voltage it will attract the electrons. So, the current will increase. So, the depletion type MOSFET is similar to the junction field effect transistor. The next type of transistor is the enhancement type MOSFET in this the channel doping is either very light or there is no doping. That means, the channel is either undoped or it is lightly doped. So, in this case there is no flow of current when you use this enhancement type transistors. So, if you apply a positive bias in that case it will try to attract the electrons and the electrons will get this in this region and they will try to form a channel. So, when the gate to source voltage is greater than the threshold voltage it will form a channel between the drain and the source terminal and the current conduction will take place from drain to source. Now, if you increase the voltage further the current will increase. Similarly, if you take negative gate voltage in that case it will ripple the electron and the current will be 0. So, now, output characteristics of these transistor are drawn. Here this region represents the cut off region this region represents the linear region or triode region and this is the saturation region it is similar to the JFET region. So, in this region the drain to source voltage should be less than the difference of VGS and VT. However, in this region the drain to source voltage should be greater than the difference of VGS and VT and the transistor should operate in this region for amplification purpose. Now, if this transistor is to be used in the circuit then they are represented by this symbol. Here this broken line represents that there is no flow of current in these type of device. Now, these transistors do not provide desirable performance at higher frequencies due to the internal capacitance of the transistors. So, these capacitance should be reduced the major component is due to the gate capacitance and which is due to the oxide layer in the transistor. Now, if you replace this oxide layer by simply gate semiconductor junction then that type of semiconductor is known as the metal semiconductor field effect transistor. So, here the gate is placed on the top of the semiconductor junction. It behaves in the similar way as the MOSFET and JFET does by applying the negative gate to source voltage it forms the depletion region and it restricts the flow of current. So, the current will reduce. Now, due to the metal semiconductor region it provides relatively fast recovery time. So, they are relatively faster and they can operate up to relatively high frequency range. So, here is the structure of gallium arsenide based MOSFET and it provides better performance over other transistor due to the higher mobility of these materials. So, they provide the various better characteristics over other transistors. They are high electron mobility, low capacitance level, high input impedance and they provide the negative temperature coefficients and there is lack of oxide strap in these transistors. Similarly, there is a high level of control in these type of geometries. Now, one of the critical parameter in these transistor is the gate length. By reducing the gate length it is maximum operating frequency can be varied. So, if the gate length is less the maximum frequency will be more and if the gate width is reduced in that case the noise performance will improve and if the gate width is high this will be a better transistor for the high power application. Now, these transistors are also not very suitable for very high frequency range. So, there is a improvement on transistor by making the changes in the channel. So, they are known as high electron mobility transistors. They are made of heterostructures. In these transistors the channel is made by using different type of materials. So, here is the structure of high electron mobility transistor. They provides very high power dissipation capability and maximum operating frequency and the noise performance over the mesh fit and this better performance is due to the higher mobility of electron in these transistors. Now, in this same way as it was the case of a mesh fit here also the gate length is the critical parameter and this decides the maximum operating frequency and in these transistors the trans conductance is directly proportional to the gate width and inversely proportional to the gate length. There is one more critical parameter in this that is the undoped gallium arsenide layer width and the n-type dope aluminum gallium arsenide width. So, therefore, microwave region this width should be between 0.02 micron to 0.3 micron. However, in this region the width should be of around 5 nanometer. Now, we will discuss about the design example of common emitter amplifier. Now, before going into these let us compare the field effect transistors with the bipolar junction transistors. So, field effect transistors occupy relatively less area. So, they are more suitable to be integrated in the IC form and they do not suffer with the minority carrier effects. Similarly, they have the negative temperature coefficient. So, they do not suffer with the temperature variations. Hence, these devices do not go into the thermal runway situation and the input impedance of these transistors are also high of the order of mega ohm. However, in case of bipolar junction transistors the impedance is of the order of kilo ohm. Now, we will take an example of common emitter amplifier. This is the example of common emitter amplifier using voltage divider bias configuration. The supply voltage is 12 volt. Now, we should choose the operating point in order to act this configuration as an amplifier. So, one should choose the voltage over here in such a way so that its value should be one third and it will depend upon the value of these transistors. So, let us take Vb is Vcc by 3 that will be 4 volt and if you see here the this will corresponds to voltage drop. So, this will be 4 minus 0.7 that is 3.3. Now, if you divide this voltage by emitter resistance Re this will give us the emitter current and that will come out to be 1 milli ampere. Now, we know that the emitter current is given by beta plus 1 times of Ib from there you can calculate the base current and the base current will be 10 micro ampere. Now, if you try to calculate the collector to emitter voltage Vce. So, that will be Vcc minus Icrc minus Ierre. So, Ic and Ib will be approximately same if you put the value of Rc and Re in this expression you will get the value of 4.7 volt. Now, if you see here this is the Re and Re is given by Vt upon Ic here Vt is the thermal voltage at room temperature its value is 25 millivolt. Now, if you put the value of Ic and Vt you will get the impedance of 25 ohm. Now, let us calculate the output and input impedance and other parameters of this particular configuration. To calculate the output impedance just short circuit all the voltage sources and open circuit all the current sources. Now, if you short circuit and you see the impedance at output terminal you will see that the Rout will be Ic and that is equal to 4 kilo ohm. And now to calculate the input impedance this will be the impedance seen at this end. So, it will be a parallel combination of R1 and R2 and the impedance seen by this transistor. So, here I have drawn the Re model of this transistor. So, the input impedance will be the parallel combination of R1, R2 and beta Re. So, beta Re is because of the transistor. So, if you see here and put the value of beta and Re you will get this particular expression. There is one thing to be noted in this configuration it is assumed that the current going in this particular direction that is the base current is very less as compared to the current going in this particular loop. So, the current going in this loop will be given by I equals to Vcc upon R1 plus R2. Now, if you calculate the voltage gain using this model we know that the voltage gain is represented by V0 by Vs and if you bifurcate this in these two expressions like V0 upon Vi into Vi by Vs. So, we know that V0 will be given by this. So, here V0 will be minus beta Ib times of Rc in parallel with Rl. You can neglect Rl not because the value of this R dot is very high. So, it will be beta times of Ib into Rc parallel with Rl and Vi will be this. So, that will be beta into I into Ib. Now if you see the Ib and beta they will cancel out and for this particular expression Vi will be if you equate this equivalent impedance by R in and the source voltage is connected here then the voltage along the Vi is given by R in upon R in plus R s using the simple voltage divider rule. And now if you try to simplify this expression you will get the voltage gain like this and further if you put the value of Rc and Rl and Re you will get the expression this. Now, here the source resistance for the microwave region is 50 ohm because all the source have the 50 ohm impedance. Now, the other parameter that decides this gain is the Rn and that depends upon the value of R1 and R2. Now, what should be the value of R1 and R2? So, we know that this will be less than 1. So, our purpose should be that this should be as close to 1 and for that we should choose R1 and R2 as high as possible. But as I mentioned here this R1 and R2 also limits the current. So, one should choose the point accordingly. So, let us take here three cases when R1 and R2 are 1 kilo ohm and 2 kilo ohm and for the second case when they are 10 kilo ohm and 20 kilo ohm and for the third case they are 100 kilo ohm and 200 kilo ohm. For the first case if you calculate the Rn it will come out to be 0.53 kilo ohm and if you put this value in this expression the value of the voltage gain will be 104. So, it has reduced the voltage gain significantly. Now if you see the value of the current going in the R1 and R2 is 4 milliampere. So, it is significantly greater than the base current because base current is 10 micro ampere. For the second case the voltage gain will be minus 110. So, it is better than this particular case and the current in this case is 0.4 milliampere. So, this is also relatively much higher as compared to base current and in the third case if you see Rn is 2.41 kilo ohm and the voltage gain for this configuration is even close to 114, but the current if you see here is 40 micro ampere which is not much higher than the base current. So, this case cannot be considered. So, the most appropriate choice for this case is R1 is equals to 10 kilo ohm and R2 equals to 20 kilo ohm. So, one should choose the value of these parameters by keeping in mind different considerations of the common emitter amplifier configurations. Now we will talk about the transistor applications. So, these transistors can be used in attenuators, RF circuits, amplifiers. In amplifiers with the help of transistors you can make the low power amplifier, medium gain amplifiers, high power amplifier. Similarly, you can also make low noise amplifier and again with the help of transistors you can make oscillators and mixers. So, in the previous lecture you have studied the use of transistors in variable attenuators and in RF switches. And in the coming lectures you will see the use of these transistors in amplifier, oscillators and other circuits. Now to conclude this we started with microwave transistors and these are of two types. We started with bipolar junction transistor then we saw the limitations of bipolar junction transistors to overcome the limitation of low frequency bipolar junction transistors. The hetero junction bipolar transistors were introduced they can operate up to very high frequency range may be up to of the order of 100 gigahertz. Then we saw the another type of transistor that is known as the field effect transistor. After that we saw the low frequency transistors that are junction field effect transistor and the metal oxide field effect transistors. And we saw that the internal parameters of these transistors limits the frequency of these transistor. So, to improve the frequency we studied the next type of transistor that is metal semiconductor field effect transistor. Here the reverse bias P N junction or the oxide layer of the transistor is replaced by the metal semiconductor junction. So, they can operate up to relatively very high frequency range and they provides relatively high gain. These misfit cannot operate up to very high frequency range. So, the next type of transistors were introduced that are known as high electron mobility transistors. And in these transistors the channel is made using different type of semiconductor and these transistors provides the better performance over the other transistors. Now among all these transistors the hetero junction bipolar transistors are the best option. If you want to operate at high frequency range. So, in the next lecture you will see the application of these transistors in amplifiers. Thank you very much.


Anybody? No? Cool. So, basically, in this talk, I'm going to talk through what we used to do, why we decided to move to Docker, why we moved to Kubernetes, the problems we faced, and sort of the system that we have now, which works across multiple different cloud providers, and we can install it on on-premises as well. But first, a little bit about me. So, I'm Patrick McGuigan. I work at Algorithmia. We run a mixed hardware cluster, primarily on AWS with CPUs and GPUs.  I don't know the exact number. And I also ran our migration to Kubernetes in late of 2016. What we do is we're basically a serverless AI cloud. The idea is that we have a common API for algorithms, functions, machine learning models to run as microservices. Users can create algorithms in seven or eight different languages. Whenever they push to a Git repo, it automatically creates a version API endpoint, which then anybody can call. It's set up as a marketplace, so users can publish publicly if they desire to. You can set a royalty fee, or you can make it private. You can share it within an organization. There's a whole lot of different options that happen there. The key is that we will scale our cloud, both the number of replicas of a particular algorithm that exists and the number of servers that we have running at any given point in time based on demand as it happens. So for example, we have this one algorithm for doing filtering, like you might see in Snapchat or anything like that. It has a whole bunch of different possible filters.  Thousands of algorithms out there, there's 50,000 different users. Anyone can create anything. Some things are utilities or sentiment analysis, NLP. It kind of goes all over the place. But we do support all the major deep learning frameworks, TensorFlow, MXNet, Cafe, Torch, I'm sure I'm forgetting some, but the point is we're trying to do anything. So with that in mind, I'd like to first start by saying what we used to do, and then why we moved to Docker and Kubernetes, getting into the problems, and then how we generalize for multiple cloud providers. And then I'll finish up with the things I'm sort of excited about with Kubernetes. I'm sure you've heard a lot so far this week on where things are going, and I'll talk a little bit about what I'm personally excited about. So this may look familiar to you. It's a pretty traditional web application diagram. We have DNS that has entries for the website and for the API servers. Those will point to load balancers. We're using AWS, so that's ELB.  For example, a web server may be on port 9,000 and an API server on port 9,100. When an API request comes in, it goes to the load balancer, round robins to one of the virtual machines that does some authentication, and we have our own scheduler to figure out where we actually want to run that work. It's pretty standard, but just in case some of you haven't experienced this before, here's how a deployment might work. So first of all, we have to use right a whole bunch of our own small bash or Python scripts to do each step of the deployment, which was managed by a third party called distelly. And the way that would typically work is you have our three servers. We pick one of them to deregister from the load balancer. The load balancer will stop sending new connections to that server. We wait for work to drain off of that machine. Whenever it's done servicing all active requests, it gets registered fully. You can then update the application binary and then reattach to the load balancer and start responding to health checks. After 30 seconds or whatever a configured interval is of healthy things, then the load balancer will start sending work to it again. This is pretty standard. This is what people have been doing. There's some variation of this for 10, 15 years and any sort of web serving architecture. But it has a whole bunch of limitations to it. In particular, you can really only run one copy of a service per machine. And the big reason for that is if you have a server running on port 9000, there's only one port 9000. And if the load balancer is configured to ping port 9000, well, you're kind of out of luck if you want to run multiple copies. You can do things like run HA proxy to balance between ports or something along those lines. But it starts to get a little bit more messy and it's usually just easier to create more machines running one copy of the service. Additionally, it can be really difficult to add services. You have to install libraries on the host. You may need to configure particular directories. We have to write the deployment pipeline. We have to figure out what's going to monitor it. If the process crashes, how does it come back up? How do we know that it crashed? There's a whole host of problems there. On top of that, like I said, we have a public marketplace. But the customers came to us who a public marketplace doesn't always work for them. They may have data privacy requirements such as a HIPAA requirement or maybe they're in EU and there's data privacy laws where you can't send data across or outside of the EU. There's also bandwidth or latency requirements. Users who are sort of in the big data space, they have all of their data in a certain data center. Why should they send all of that from, say, Europe to our data center in US East 1 just to do some processing? Really, they'd like it to be within their own data center. So with those motivations, we felt like we can't just handle or can't only have a public marketplace, but wanted to have an enterprise offering where we would install our system in whatever cloud, whatever region on premise that users wanted. But before we can do that, there's a whole bunch of barriers. We have to figure out how you're actually going to deliver your applications. You have to figure out how to work on different operating systems based on the customer's requirements. Like I said, we have third party software called Distelli for doing our deployment management. We would also use some things like Algolia to power search on our website. There's a high proof of concept and the list kind of goes on. But what I'd like to talk about first is why Docker. So Docker gives us the first two of these barriers where we want to talk about the operating system and how to actually bundle the application and deliver it to people. So the benefits of containerizing is that in general, you're no longer tied to the host libraries or operating system that the small asterisk will come back to. It gives you an easy way to distribute and upgrade the application images. You can simply take the Docker image, you can push it to a registry, you can dump it to a tar file, you could send it via email, whatever you'd like to do, there's multiple ways to distribute it.  So for example, if you're running HA proxy and some shell scripts and a web server using Apache and a different web server in Python or whatever else, they might all write logs to random spots on the operating system. With Docker, single Docker logs, you can view application logs for anything. It also makes it somewhat easier to run multiple copies on the same machine, but not fully. So I dockerized our system or completely dockerized our system in August of 2016, roughly. Prior to that, we had used Docker for a certain components, but it wasn't a fully fledged entire ecosystem. There's a couple caveats. As I said, you aren't really completely free from knowing what operating system you're running on fully. For us, when we're using GPU devices and other drivers, those actually need to be installed on the host operating system itself. It's not something that you can only have existing in the container, because you have kernel modules and other things that you're dependent on. And the kernel is not a part of the container, it exists outside of the container. Additionally, and this is a little more convoluted, but if you have a container and you want to spawn more containers, you would give that container access to the Docker Unix socket, which is from the Docker daemon. It's var run docker.soc. Since this is a file on the file system, it has a user ID and a group ID. And the user who's running your application inside of the container has to either match the user ID or the group ID for this file or won't have permissions to write to it. Well, it turns out that in Ubuntu or CentOS or a couple other different operating systems, when you install Docker, they get a different group ID. And that means that your container, your application is running with the group ID of say 999, that means that it can only actually access the Docker socket on other hosts that have Docker group 999. If you are fully using Kubernetes, you don't really have to worry about that, because usually you're not interacting with Docker directly.  And when you're using the Kubernetes API server, there's like role-based access controls and tokens that will handle all the access that you need. But if there's some sort of middle ground where you are using Docker but not fully using Kubernetes or some other orchestrator, this might be a problem if you need to target different operating systems. We worked around that by sort of customizing the base image that we're going to use, even if we're using Ubuntu or CentOS or anything. We installed Docker and have a specific group that we use for all of our AMIs that we share with customers. There's a couple other issues and it's something where Docker people like to think that it's very easy, but in some ways it can be complicated and have sort of unknown performance issues. When you go to figure out what storage driver you want to use with Docker, you kind of get this whole little matrix here. If you're using Ubuntu or Debutin or whatever, there's just this whole sort of list and you're like, okay, well, which one do I choose? Turns out the recommended driver has changed multiple times over the last three years that we've used Docker from device mapper to AUFS and now it's at overlay two and that's something that the community sort of realized this one has performance issues or that one has performance issues and it kind of keeps evolving. For us, we ended up using AUFS. We might be changing that, but there are some tricky bits with a particular storage driver that they may not explain to you very clearly in some of the documentation. For example, with AUFS, it does require an extra kernel package to be installed and this is documented, but we were automatically upgrading our kernel versions for security reasons or whatever else and then when we rebooted a host, well, our process, we actually forgot to update the Linux extras package. So you reboot the host, the kernel's new, we're missing this extra kernel package and then the Docker demo to set their crash looping. You can easily be fixed by being a little more intelligent on how you're upgrading your packages and making sure you upgrade everything you really need to, but it's something that sort of caught us off guard when a month or so after we're using Docker in our system suddenly had these weird edge cases we were hitting. Secondly, with AWS specifically, EBS volume performance is really bad until the volume is fully warmed up. Depending on how big your volumes are, this can take several hours to read all the data. You don't actually have to write anything, you just have to read and scan the entire volume. In our case, it was taking 45 minutes to an hour and a half to actually fully read the volume and when you're trying to be an auto-scaling system bringing up and down lots of hosts, an hour and a half, you can't wait. That's not reactive in the way we want to be. Because of that issue, we found that with very right heavy workloads, like when we're spawning lots of containers and we allow to turn in our system, the Docker daemon would end up taking 45 seconds to a minute to create a container or do anything. And that's something that typically takes 100 milliseconds or less, something like that. And all of that time was just file system sync times were way too high. So one way to work around that is if you mount empty EBS volumes, they don't need to actually be fetched or pre-warmed or anything like that. So you can mount empty EBS volumes and do a bunch of work on those and they'll function way better than if you have something that needs to be pre-warmed. This only really matters for situations where you are doing lots of auto-scaling. If you just bring up a host and you expect it to be around for a long time, you might not actually need to worry about this very much. But we'll have questions at the end. Sorry, thank you. But overall, we're able to move everything to Docker, works great, but it doesn't do everything for us. So the next question is kind of why do we move on to Kubernetes? So Docker, we kind of abstracted around our operating system and we have a nice way to bundle our applications. But there's still a whole bunch of other problems. So Kubernetes is going to help us get rid of more of our third party software by being an orchestrator, becomes very easy to plug in new applications that we might need into the platform. And also, we want to get better server utilization. Before we had Kubernetes, we needed about 12 machines in order to run our base system and that's for highly high availability and the fact that you can really run one copy of a service per machine and there's this whole orchestrating problem there. So how does Kubernetes get us that? Since this is the end of the week of Kubernetes talk, I don't think I'm going to run through all of this. But the most important things I found are services which give you ability to do load balancing, service discovery, DNS entries, deployment slash replica sets. They allow you to have end copies of a thing running. You can do rolling updates. It's all native. You can configure how the update's going to go, say, do one at a time, two at a time, wait a minute in between instances. It's highly configurable, lots of things you can do there. And thirdly, the CNI plugin, which is the networking plugin, which allows you to get the actual giving IP addresses to every container in the infrastructure. And it also establishes routing between all the containers and allows you to create network policy so you can say, I want my API servers to talk to Redis, but I don't want the web servers to talk to Redis or whatever situations you may have. You can lock down the way this works. And then on top of that, there's also service mesh, which you can do even more fine grain firewalling. But we'll come back to that in a little bit. An interesting question, I think, is what don't we use Kubernetes for? It's a highly powerful tool. We could probably do everything we wanted to with sort of native Kubernetes ideas and operators and stuff like that. But we kind of chose not to, in part because we're a small team, it's a huge project, there's a lot of things going on. And we didn't want to throw a ton of variability into everything else that we were already doing. But on top of that, we had spent a lot of time optimizing our system for the way that we were scheduling algorithm containers. We would look at things like whether or not images were there, which Kubernetes would also do. But we have higher level data about what will this algorithm need in terms of the data files that it's using? And what are the likely files it might be using based on characteristics we've seen in the past? Things like machine utilization is something Kubernetes is also very like it will natively track for you. But pending requests and queue sizes is something a little bit more abstract it might not know about unless we added that on top. We also have some special security concerns that go as part of our scheduling. We will proxy requests for users, maybe put them on an isolated network if we don't trust it or the user wants it to be on an isolated network. And all of this we were still getting within single digit millisecond performance. It's one of those things where it's like, well, we spent all this time optimizing and investing in this code base. We didn't feel there was a need to move all of that inside of to be a little bit more like Kubernetes native instead of running in our application. Something we may change in the future, especially as we want to extend those capabilities further. But for now, it's something you should consider. Do we need to move every single aspect of our system to be Kubernetes native or can we leave some of it within the application? Also, GPU and device management, Kubernetes will allow you to expose the devices to a particular container or particular pod. You can say I need one, two, four, whatever it may be. But there's a lot of metrics that you can't get out of a GPU. So in particular, GPUs will track memory usage, et cetera, by the process ID. So when you're looking at a GPU, if you're just exporting a process ID, one, two, three is using this much memory, that doesn't tell you anything. Well, you need to know what was that process? Who was it? What's the meta information about it, which would be the user or the algorithm, that kind of information or application knows? So we had to find our own way to expose that information that wasn't sort of natural with CAdvisor and other metrics that are already generated for you. Secondly, with ingress controllers, ingress controllers are great, but they have some certain limitations and different ingress controllers have different limitations in terms of the amounts that you can do with SSL termination and things like that. And for a lot of our enterprise customers, they would have particular certificate authorities, certificates, different, they might need to validate X509 certificates from the clients who are coming in, and all of these configuration options didn't exist in any current ingress format. So rather than go through and figure out how to get all these annotations and other things, we found a way to template our own sort of hop proxy config and just kind of rolled with that. It's very close, it's almost like the ingress controller spec, and so I'd like to get it back in there, but it's something that we kind of have to work with the other projects, and it's hard to figure out whether or not they would accept our changes. But overall, we completed a migration in production in January of this year. Everything was running on Kubernetes fully. We were able to drop the minimum number of servers we needed from that 12 to six, and the six is because there's three Kubernetes masters for a highly available setup, and we'd have three Kubernetes nodes for highly available Redis, API servers, all those other things that we run in our system. We also only need a single load balancer now, and that's part of that ingress mindset where you have a single Nginx, hop proxy, whatever sitting in front of all of your services, and then it routes based off of what the incoming address was. Now we're able to replace our third party search with using elastic search. It's very easy to get elastic search configured and deployed within Kubernetes, and Kubernetes, again, we have native deployments and rolling upgrades, so we could get rid of distally entirely. Overall, it's very easy to add services to the stack as well, such as Prometheus and Grafana, if you hadn't heard about them the rest of this week. There are amazing tools for graphing, monitoring, etc., and it only took about a day for us to integrate Prometheus with our stack because there's so many deployment templates already out there. It was just easy to take this, plug in the little bits that we needed specific. After that, it's probably spent about a week or two weeks adding metrics everywhere I could and creating graphs and templates because it's amazing how much detail you can get about your cluster from these two tools. So what went wrong in this process? There's a lot that can go wrong if you have an application that currently exists not in the context of Kubernetes and you want to migrate it piecewise. It's always possible to duplicate your entire stack inside of Kubernetes and sort of just point your DNS to the Kubernetes one, but that may or may not work depending on your situation. And you might not even be able to migrate everything to Kubernetes if, say, there's some other team you depend on and they aren't willing to do so, something like that. So the projects are moving fast, like really fast. You might have seen that from this whole week, all the features people have talked about in the last year, the things that are upcoming. But they aren't always compatible with each other. So for example, Docker 1.13 came out earlier in this year. And at that time, Kubernetes 1.5 was out. However, it was not compatible. And also Docker 1.13 wasn't targeted for Kubernetes 1.6. And it wasn't until 1.7 that it was targeted and things were fixed. And it resulted in Docker was making some changes to the way that they did IP tables rules, so they added a default deny or default drop policy or something to the top of your chain. And that broke everything. So people try and upgrade and oof, didn't work. Secondly, there's a ton of different versions that are in development and used at the same time. Just in November, I think there was 1.612, 1.710, 1.83, and 1.90, Alpha all released in Kubernetes, in addition to other versions. So if you're going to be taking time to migrate your service, what kind of version are you going to target? It's really a project management decision, but it does become difficult when the docs are continually going to be getting updated and you can't find the information you need.  It's just something worth considering and probably looking at what the chart is and estimate how much time you might need and then try and target something appropriately. There's also a lot of new concepts which can break sort of the way your deployments may work. So readiness checks with a deployment are very similar to a health check from a load balancer, but there's lots of things in the application and pod lifecycle that are different. That you might have to change the way that your application gets stopped. You have to change a rolling update strategy. You may be used to doing certain things at initialization, which now you have to create another container to be in a knit container. And none of this is particularly difficult, but it is something where every service you migrate, you will have to look at all of these things and figure out what do I need, what don't I need, possibly creating five or six containers if you have a lot of initialization that needs to get done. But the real interesting thing is the technical challenges. So as we said, the CNI plug-in will generate IP addresses for every single pod in your cluster. But those are not routable from things not within the Kubernetes cluster. And this can become really awkward if your services are trying to talk to each other via IP address. For example, we have API servers and we have Redis. And Redis needs to talk to itself to replicate data. And if the master goes down, one of the replicas needs to take over. So if you migrate Redis before migrating to API servers, well, your API servers won't be able to talk to Redis unless you put some sort of proxy or something in front of it that they can address. And then that would be able to address the things inside of Redis. Something you just need to understand your own application, what's talking to what before you can really migrate anything safely. Secondly, classic load balancers are not really meant to address containers. Again, the IP address isn't real. So what may happen is, is that you're exposing your containers or your pods on, say, port 9000 and the traffic is getting proxied to that container from the host. But it's just as likely that the traffic is getting forwarded to a container on a different host because that's the way that Kube proxy works. It will forward to something that is active across the fleet. And then what happens if host B goes down or deployment comes along and updates that pod or whatever? Well, if that connection that was going to host B gets reused for some reason, it's possible that ELB will mistakenly think host A is down. Because from ELB's point of view, it's talking to host A. But in reality, the track is getting forwarded. This can happen with any sort of HTTP client and any other application you're using as well. The main reason this comes about is because of connection pooling. So the connection from ELB that was going to host B, if ELB were to create a new connection, it would Kube proxy would only forward it to an active pod. You don't have to worry about that. The problem is that these connections may or may not get reused based on your application config. And if you're not careful about setting headers such as their connection close, you may end up stuck with this problem. Additionally, ELB allows you to do a TCP check, which would not use any pooling ever. So it depends on what sort of situation you have. Secondly, when you have a CNI plugin, you have a whole bunch of hosts and they get networked together somehow. For example, in this diagram, we use the WeaveNet plugin which works great. It has lots of configuration options, but there was one we weren't aware of until it sort of bit us. Since we downscale our cluster, it's possible you can end up with disconnected nodes. So say like we have this diagram here and host B and E we turn off because we don't have traffic or something like that. Well now you can see your cluster is divided into two different groups and they can't talk to each other across that divide. So that's something where you can actually configure the number of connections a given host is going to have. And you can easily prevent, if I expected downscale and nodes, make at least n plus one connections. Simple heuristic, but it works, it gets the job done. But you do need to be aware of your sort of topology there. For us, we highly monitor the number of connections that each host has, how many are failed, how many are whatever, and set alerts on that. All very easy using Prometheus and Grafana. And you could consider your network topology while scaling. These nodes will not refresh their node period list. So if a node goes down, it may not actually try to find a new node. You could make all the nodes aware of that. We thought that was a little too complicated and haven't had a need based on our heuristics and monitoring yet. So how do we go from here to multiple clouds? There's still a lot of moving pieces of infrastructure that you need to create, VPCs, networks, virtual machines, et cetera. And we also have to get it to work on a whole bunch of different cloud providers. So the first step is that, oh, sorry.  With your application, you might want to think of first. Vendors should like to try and lock you into certain to using them. For example, AWS has services like DynamoDB, Kinesis, ECS, all great services. However, they don't all necessarily have an analog in every cloud provider. And if you want to target an on-premises environment, certainly there's no DynamoDB on-premise. Additionally, you can use Kubernetes to replace a lot of things you might otherwise use infrastructure for. For example, you can use a service to replace having physical load balancers, as we've discussed before. But you still have to figure out how to provision. And this is where, I believe this was mentioned in the last talk as well, where Hashicorp's Terraform is great. It allows you to create .tf files. You can commit infrastructure as code. You can version it. You can plug in a whole bunch of different variables for, say, region and AWS account or whatever else. And then we can bring up a cluster with those parameters. But once you have a whole bunch of .tf files for AWS, you can't actually just change those to the Azure equivalents. It's not just to find and replace virtual machine with the virtual machine from Azure. And there's a number of reasons for that. I'll try and go quickly through these. So AWS has an internal DNS to your VPC, but other cloud providers do not. So you might have to reconsider your architectures a little bit there. Again, I would rely on Kubernetes to do services to use DNS and not actually have to use a DNS within the cluster. But you can use a Kubernetes-level DNS. And that solves the problem for you right there. In Rackspace, if you have a public and a private IP address, those come as two different network cards. And there's this issue with Weave and the Kubla where it would end up sending traffic to the wrong network card if it's basically if you're trying to send traffic within the cluster, but a route didn't exist, then it would actually send it through the public network interface instead of the private network interface. And there are ways to work around that. It's on that issue there, how to work around it. Last, API versions are never simple. Rackspace is not exactly the same as OpenStack. Within a different OpenStack clusters, you can have multiple versions of authentication even. And the types of APIs that are available may or may not change between different customers who have an OpenStack cluster. So it's one of those things like you can discover all the services available within OpenStack for a particular environment. But you sort of end up with this giant matrix of what's there and what's not there. And you really just want this to be simple as possible. And it will be a little complicated for OpenStack. Load balancers, if you're doing a lot of copy and paste between Terraform files, you might see AWS, HTTP, load balancer, cool copy, that's OpenStack. But it turns out that with AWS, if you have HTTPS, that means terminate SSL. But in OpenStack, HTTPS means do not terminate SSL. So there's just these really small things that can get lost in this mix of so much going on that you end up not noticing very fine details like that. Additionally, there are differences where Azure, you can only bind to a single internal load balancer for a virtual machine. And again, it's because it's meant to be a classical one application per virtual machine. AWS, Azure, and Google Compute, I believe they all have concepts of like an API gateway or something along those lines, which is an application load balancer, I think is what it's called in AWS, which can route a little more natively to containers instead of virtual machines. It's something that, again, doesn't exist in every environment. Certainly won't exist in an on-premise environment, but if you only want to target a particular cloud provider, you could use those types of load balancers instead. But with Azure, traffic, if it goes from, say like you have two containers, a web and an API server on the same machine, and you want to talk to, the web wants to talk to the API server through a load balancer from Azure, it actually can't do that. Traffic can't go from a machine to a load balancer back to the same machine in Azure, which is just a part of their design because it's not meant for this sort of environment. Again, you can use Kubernetes services and you can just avoid load balancers entirely, avoid DNS entirely, and that's sort of the lesson is that Kubernetes will allow you to become more cloud agnostic just by having all these primitives that exist everywhere. So at the end of all of that, we have a whole bunch of different Terraform files with some parameters, and then a whole bunch of Kubernetes and files for deployments and stuff, and we can go from absolutely nothing to bringing up our entire system, running GPU algorithms and all of that in less than an hour, which I think is pretty good when it takes 25 minutes to provision an RDS instance in AWS. So not much we can do about that one. So looking forward, there's a lot of performance improvements coming up. Just last year, the GPU, exposing GPUs to a container was just added to Kubernetes, I believe, in one four, I think, and since then it's become easier to expose multiple devices, but then you have AWS coming out with FPGA instances, and I do think that FPGAs might be useful for some machine learning algorithms, maybe they could be faster, but I have no idea how to get Kubernetes to run and expose an FPGA, I don't know, but it'd be really interesting to see what can happen in the future there. Next there's sort of, when we're building a whole bunch of images and shipping them around our cluster a lot, we found that the biggest bottleneck ends up being just how big some of the images are and being able to compress them better would be an ideal situation, though I'm not sure this is likely to change. Multi-Cloud is sort of becoming a little more of a native thing within Kubernetes, so what we have is a system we can deploy inside of a particular data center, but you can imagine it would be ideal to be able to run across multiple data centers, which has problems of security concerns and data traveling over the internet, you can also get a lot of latency and it gets confusing and hard to track all of that, and that's where things like with a service mesh would enable you to track better how your services are talking to each other across all of these regions. Now there's the concept of a federated ubernetis, where if you're trying to run, it's sort of the difference between running Kubernetes that has masters in one data center and all the nodes are part of one cluster versus do you have Kubernetes running in five different clusters in some way to federate access to each one of those in a common format. So I think that's about all I have right now, so you can contact me for any other questions at atrickadalgorithmia.com. If you're interested in infrastructure, how artificial intelligence, machine learning, serverless computing, Kubernetes, how all that sort of matches together, we're hiring for pretty much everything and we have a coupon code. We give people free credits every month, but there's also a free coupon code, so. Thanks, so questions and thank you, Ed. It is kind of scary, so there are a couple ways to expose it and that's something where when you're using Kubernetes it doesn't matter anymore anyway. But it's one of those things like as a temporary step, you can also expose it via TCP and then you want to enable TLS and then you have self-signed certificates and it's a little bit annoying to deal with. One of those things where when we are controlling the stuff that's running, you've mounted the Docker socket to only a single thing and again it only has limited permissions and you can't do anything. We don't run any privileged containers or anything like that. But again, it's something you want to get away from and use Kubernetes and then you can have role-based access control and allow certain users to create certain API or only access, certain things, only be able to create, destroy certain resources. It's an AWS thing, it's in there documentation. It's basically if you have an EBS volume that has data on it, the way that the volumes are stored is that they're backed up in S3. So when you create a new volume and you attach it to an instance from a snapshot that volume gets attached immediately but it doesn't actually fetch all the data. So if you have a whole bunch of blocks, basically every time you access a block for the first time it has to get downloaded from S3 which can be slow. And if you have say a 250 gigabyte drive then it's gotta download all of that data from S3 even if there's nothing there. If it's an empty block, it doesn't know it's an empty block until it fetches it. Yeah.