There is time for questions. The question is about fragmentation of data. So in the relational database literature, there is something called horizontal fragmentation versus vertical fragmentation. What does this mean? Supposing I have a big table like this, a lot of rows and then there are columns like this, maybe lots of columns also. What is horizontal fragmentation? In horizontal fragmentation, the table is broken up into pieces, let us say there are 5 machines. This part goes to machine 1, machine 2, machine 3, machine 4 and machine 5. So this is called horizontal fragmentation. Is this notation clear? This is what Hadoop runs on. It is based extensively on horizontal fragmentation because it breaks up data into files. The files are broken up into lines or records or whatever. So that is horizontal fragmentation. In the relational database literature, there is another concept called vertical fragmentation which can say these two columns are processed at machine 1 and these two at machine 2. So I am breaking up the relation by columns not by rows. In fact, you can have a combination. I can say that this part is in machine 1, this part is in machine 2, this part is in machine 3 and so on. You can combine horizontal and vertical fragmentation, both. But Hadoop focuses on horizontal, vertical is, in fact Hadoop has no idea of a schema. All it knows is records, it does not know what is inside a record. That part is up to you when you write the map and reduce function to deal with. So Hadoop is very low level. It does not know anything about schemas and that is also the reason it is a pain to program in Hadoop. We just saw a lot of code for doing something very simple. So it is very nice for many things but it is a lot of work to do simple things. That motivates why SQL is making its way back in this community. So that was a period when people said no SQL. SQL is old, it is, see we can write programs in Hadoop which can run at thousands, tens of thousands of nodes. Two database people are struggling with parallel databases running on 200, 300 nodes. So that is where it started. No SQL, forget SQL. But eventually people realize that sure, you can do it but it is a lot of work to write programs and after all that is very inefficient. So they said we can run programs that runs on 1000 machines. Today the database people are coming back and saying why are you using 1000 machines? We can write new programs better on 100 machines. We don't need 1000 machines. We write better programs, our code is better, it is faster. So that is what is happening in the world today. Of course it is not the original databases. There are other companies which are coming and doing these things and databases in SQL are actually making a comeback. There are a couple more small topics in Hadoop before I move on to some other topics here. The first is what is called local pre-aggregation. So I said that each machine outputs all these values, each mapper outputs all these values which are then sent to a reduced function. Supposing that I need to send all this data over a network, that takes a lot of time. If you see the way technology has gone, CPU speeds kept increasing for a long time. Now CPU speeds are not increasing that much or not at all. But the number of cores per CPU has kept going up. The number of CPUs per machine has gone up. So today it is routine to have 32 cores on a single machine with 32 cores. And internally the bandwidth is very nice. What about this? This bandwidth has been going up steadily. It used to be maybe 10 megabytes, 20 megabytes. These days you can get 300 megabytes even from this. It has gone up a lot. What about network? Network speeds reached 1 gigabit long ago. All your laptops have gigabit Ethernet cards. The servers running at Google probably also have 1 gigabit Ethernet cards. That has become a big bottleneck. So 10 gigabit Ethernet is available, but it's more expensive. I'm sure it is used. But it's still fairly expensive. I don't know why. Networking people have been lazy. Maybe. I don't know. But the bottom line is the network is a bottleneck today for many things. So what local pre-aggregation does, it says, wait, why am I sending all this? On my one machine I have processed, say, 100 documents. Can I group together and do a local group-by on those 100 documents and output the word count so that each word will go out only once with a count across these 100 documents? That makes sense. Why ship all this stuff across the network and make a bottleneck in the network? So what the combiner does is you can run it within a machine. You can even run it in another level. So the way these things are organized, how do you organize a system with a thousand machines? The way you'll do it is you have a rack, there is one rack, not very good at 3D drawings. Here's another rack. What does each rack contain? Each rack has a number of machines. This is one machine. So a typical server these days is called 1U format. This is a standard unit which is like about maybe 3 centimeters or so high, I don't know the exact number, but that's 3, 4 centimeters high. So one rack like this about, say, 6 feet high can take about 40 machines plus some switches on top and a few other things. So this is a typical rack. So within this rack there is a network switch and this network switch in turn is connected to another network switch. So each rack has its own network switch. So all the machines here are connected here and so let's say this switch has 40 connections to the 40 machines and then one or more connections to the main switch. So typically this is 1 gigabit, this is 10 gigabit ports. This is typical. So first of all you can do pre-aggregation within each machine to avoid overloading the 1 gigabit links. Then at the rack level you can send all these to one of the machines which can do another or even that can be parallelized. What it does is across the 40 machines in the rack it can do a sum again. So you don't send out the same word 40 times but only once with a combined count. So that is the rack level combiner. So within machine and at rack level. So the Hadoop framework lets you define a combiner function and you can say use the combiner. That is an option you can give it and then it will run it at each machine and if it knows about the machine layout on the racks you can also guide it to run it at each rack level. And Hadoop the reduce function which you have defined is itself used as a combiner by default if you turn on combiners or you can specify your own combiner. My suggestion is don't do this it gets into trouble if you don't know what you are doing. Initially in your programming don't bother about combiners afterwards if you have time you can play around with it. So what are the implementation as I said Google is not open source we can't use it. Hadoop is what everybody uses available there and there are other asset data and so on. And then there are some others which are used in other places but the Hadoop is today the D thing that is why we are using it. So now I have been mentioning this throughout the talk map reduce versus parallel databases. So there is a lot of buzz about map reduce then the database people said but we have been doing this for a long time the map reduce people said that hey you never ran at such scale fault tolerance where does fault tolerance come in? So back to this picture a lot of map tasks a lot of reduce tasks what if the machine running one of the map tasks fails what to do? The good thing is no it is not replicated ahead of time but the master can detect that this mapper is not generating data or it is dead totally and the same thing which it gave which it told this guy to do it will run it on another machine exact same map tasks will be run on another machine. What about the output of this guy? So this guy's output will not be consumed until it is fully generated. So if it fails in the middle before its output is consumed it can be thrown away and the reduced tasks here will wait for a newly created copy of the task to finish before the reduced tasks start. So the way the thing works is first all the map tasks are run if a map task does not finish does not generate data it rerun effectively and only when all the map tasks are done and have sent the data to the reducers is the map phase finished. Even when the reducers can collect the data but they do not run the reduce function yet. They wait until all the map tasks are finished they have got all the data that they need then they do the final sorting and run the reduced tasks. What if a reducer fails? No problem you can get the data again from the map tasks here and rerun the reduced tasks. So again all of this is done by the master. So even if there are failures it can recover. In fact even if a map task does not fail sometimes what happens is it runs slowly why? Because that machine may be doing some other task it becomes slow somebody is running some very intensive job on it it become very slow. Even in that case the master will rerun the map task so laggards like slow machines can also be it is not pure failures but even slowing down can be dealt with. So this is the kind of fault tolerance that it gives plus it uses the fault tolerance of the distributed file system itself. So failures are nicely handled and then this was the major thing procedural code in map and reduce functions for example page rank and so on can be done in a relational database but you first have to pre-process the data that is a lot of work and all that can be done procedurally. On the other hand map reduce is very cumbersome and slow in fact execution is slow for very simple tasks for which SQL is actually a much better fit. And the second part is that programming in map reduce is very painful so two things happen in parallel. The first was to provide a better interface to map reduce for tasks which are more naturally database tasks and there were three things systems one is a system called pig also from Yahoo this one they didn't public didn't open source it it's internal and no it's sorry pig is open source there's another one I'm mixing it up pig is open source developed at Yahoo so what pig does is it has algebra essentially and lets you specify these operations and run these on the data so you it's predefined a number of operations you can use those. You don't have to write map jobs from scratch so it's declarative. Then there's Hive which came from Facebook which actually lets you run it directly write SQL a dialect of SQL and in fact Hive also specifies a schema for each input source and so although the data may be in files Hive knows what is the format of the data and can read the data properly and then the third one is scope from Microsoft this is again not public they are available but you mean you can purchase it so these are three things which allow declarative query SQL or variants and then there have been many extensions of map reduce to allow joints by planning of data in fact they have come to the point where they are now like a parallel database implementation with map reduce extra on it so there are several such systems that are available so the combination of these now there's huge impact now there are a lot of companies which are building these some open source some not but there are a lot of products out there which some people call this new SQL which are really massively parallel systems which can run SQL queries directly so this is for so far what is map reduce for is for analyzing data what if you want to store data what if you want to do transaction processing at really large scale what can you do map reduce is not the answer to that map reduce is only for querying but databases are not only querying if you know database have two aspects there is the online transaction processing and then there is the decision support so far all we have been talking of in big data decision support about well in fact there are systems which do that and these are called big data storage systems again they started from ground up meaning the initial systems do very simple jobs they don't do a lot in fact all that they do is they are called key value stores looking at the history for it you need to store huge amounts of data you need scalability distributed file systems which are and under like I do have been around but the problem is that they assume each file is very big GFS HDFS will die if you throw one billion files at them they just cannot handle that kind of scale they can handle hundreds of thousands of very large files maybe millions of very large files but not billions but a lot of data which comes from data processing application very small each piece of data is small then you have billions of such data so take an example Facebook what kind of data does Facebook generate each time somebody posts a comment is data each time someone posts a photo it's data each time somebody clicks on like it's a piece of data each data is really small it identifies who clicked it whose page it was on time few other things and this number of such things that are generated are huge absolutely huge what about the friends thing each person has friends on Facebook how many friends does average person have I don't know maybe a few hundred that data is also pretty small it's not very big but there are billions of users on Facebook so if Facebook tried to store all its data you know per user file is created on GFS or HDFS it would die cannot have these systems cannot handle so many files so what happened again the history is again Google took the lead in this it built a system called big table which is built on top of Google file system so underneath it is creating a few a number of large file each file is like few hundred megabytes but within those files it stores much smaller records and the primary interfaces you can put a record with a key you can retrieve a record for a given key and then you can scan all records in a given key range just three operations there are a few minor ones but these are the three main ones put a record with a key get a record with the key range scan now what do you do with these three operations actually you can do a lot with just these three primitives so you lot of applications are built on exactly this kind of infrastructure it's very powerful actually it is not a full-fledged relational database why there's no notion of integrity constraints there's no notion of well there is a equivalent of relation but full-fledged integrity constraints are not there you if you want SQL you need one more layer they don't provide SQL they don't provide transactions they don't provide concurrency control they don't provide many things what they do provide a scale which the traditional databases don't at that level so these have been used extensively first big table from Yahoo and Sunil of Yahoo sorry big table from Google and Sunil of Yahoo came up with its own version called peanuts this one they did not open source it's internal but more recently the Hadoop project has also built something called H base which is a clone of big table so that's also available we are not doing anything with it in this course but if you have students who want to do something new with the latest tools H base is a nice thing to try out so we've had several student projects in IIT this year using H base so what is the key value store this is that you have two main operations put key with an associated value get key and then maybe range scans and the system may store potentially multiple versions of data with a time stamp very simple interface how is it implemented this is a schematic come which comes from the peanuts system from Yahoo so what you have are what are called tablets white tablets it's a nothing to do with medicines it's a piece of a table okay so a piece of a cigar is a cigarette a piece of a table is a tablet so you break up tables horizontally into a lot of pieces and store them across many nodes and now note that these boxes here are machines each machine contains many tablets it's not one tablet per machine it's potentially hundreds of tablets per machine why because it gives flexibility supposing this machine is overloaded there's a lot of demand on tablets in this machine what do you do you can move one tablet from here to here or two tablets or some small number of tablets and rebalance the load or you can you know purchase a new machine and put it in and then take one tablet each from each of these machines and stick it into the new machine that way you have grown the system by one machine without seriously affecting the work on these machines each is copying out a small amount of data each tablet is typically few hundred megabytes okay so that's how the data is partitioned and now what about queries for request comes in you have to know where which tablet contains data and where that tablet resides so there is a index kept here at the master copy of the index is here at the tablet controller and then there are machines for routers which have a copy of that information so given a key value this can look up that index and say this by the way the index is on ranges so any key value between one and twenty thousand is in this thing or hash of key value equal to something is on this machine so that's how that goes so this partition and tablet mapping this thing is actually fairly small it fits in memory although the data is huge the mapping is small so given a key value I know which tablet it is in and I know where that tablet resides quickly I can find that and then the query is sent to the appropriate tablet now what about replication well the master also keeps track of tablet replicas updates have to be sent to all the replicas I've not shown that in this figure but that's how this system works and big table is also essentially the same architect although some details differ and here how is each tablet managed in peanuts these tablets are stored as relations in a MySQL database or in a Berkeley DB store in big table they built their own infrastructure for storing tablets which is highly optimized for writes they have some interesting techniques there so that's the underlying system so again you can't get peanuts or big table but H base is a clone of big table it also uses a similar architecture so I would encourage you to get your students to do some projects with these so many many companies are taking to this so coming back why are companies bothered about it now who who cares right 30 years back it was only Walmart or 20 years back it was only Walmart and a few other companies AT&T then the web came and many companies Yahoo Amazon maybe Flipkart read if many companies have websites but if only these companies were interested in these technologies again it's a small market what is happening is every company today is interested in you know when people are searching for their products on the net they want to know about it I want to know who is searching and I want to advertise that is another side but the search companies will happily sell information about who searched for what and companies will buy that data it's a lot of data which is available to be bought and they want to use that data analyze it to make decisions so even companies which don't even operate a major website have a huge amount of data and they're analyzing it so there's a lot of opportunities in the job market in this area so it's good to have students look at it at least the more advanced students so time for a few questions if a size of the tablet coordinator is last then how we can manage the access time for a tablet coordinator which one this one tablet coordinator suppose size of the tablet coordinator the master copy or the router yeah master copy sir yeah so the the there is a so supposing you have a tablet which is 100 megabytes let's say you have a petabyte of data how many tablets are there right so 100 megabytes a gigabyte is 10 tablets a terabyte is 10,000 tablets a petabyte is 10 million tablets so now you have to store information for 10 million tablets but what is it that you need to store you need some hash function and boundaries and so on 10 million can easily be stored in memory that's nothing okay so it's very easy to fit this in the memory of one machine does that answer your question if size increases day by day yeah so even you okay I think your point is eventually this is going to become very big then what that's a good question so this is an engineering decision that people have taken today today with the memory sizes that are available scaring to a petabyte is easy and the biggest things that are out there today are a few petabytes three four petabytes that's nothing three petabytes is 30 million entries 30 million entries supposing we put 20 bytes for each entry or even 50 bytes for each entry 1.5 gigabytes nothing it's small okay so as an engineer you'd say that for now this is good enough maybe five years from now when it has gone to five times this maybe we need to rethink this in fact this has already happened not for big table but for GFS GFS has been re-engineered GFS also did something like this it stored there's a master which tracks which blocks of which files are on which machine that runs into trouble when you have more than a hundred million files so like I said that 50 million tablets 100 million files there is some comparison here 50 million tablets is fine 100 million files is fine but if you go to a billion files they die now you don't need to go to a billion tablets because tablets are under your control even if the records are small you are grouping many records into a tablet for GFS it's not in their control if an application comes and creates a billion files GFS will die so GFS has been re-architected they have a new system called Colossus I think which is different that this master information here is spread across multiple machines and it can handle much larger sizes okay so the question is what is this request to tablet mapping okay I think I skipped something here I didn't tell you what is a request a request is a get or put call which we just saw it's not a web request not at all it's put a key and a value pair or get key or maybe range scan so the request is specifying a key value or a range of key values and this mapping the router says that this particular key value is at this tablet so that is done in some range partitioning or hash partitioning of some kind and that needs to have essentially one range per tablet and so the that's why I said about say 50 bytes per tablet is enough okay good so now we'll have a coffee break here


Hey, what's good y'all should boy down from the down showing friends that we back for another k-pop reaction now today We're actually gonna be doing late. Okay, so we're gonna check out late official music video view I hope I'm saying that correctly if I'm not my apologies for real We're gonna check out the music video today. And if there's a dance practice to this I'm gonna check it out But I've seen it in the comments and I woke up to a comment saying it and I was like, all right I gotta get this done. Don't worry y'all more videos is on the way win a new season So let's keep going. So well for to do, please make sure you hit the like button share subscribe Bring that bell and more than anything seriously off. Thank y'all so much This is it. This is a dope journey and I'm having a great time Reacting review for y'all. All right, so let's get into it. All right. Here we go and three two one The cinematic feel, you know cinematic Real quick man, I wish I could get away with like, you know Change the color of my hair like no, I'm you might be lapping, but I'm low key serious like I'll put some red I Try different things just create a leak. I love being artist. All right, let's get back to it. Oh Let's get into it. Okay robotics Ah Okay already already first off because I know I don't really watch a lot of music videos But I'm gonna tell y'all seriously genuinely what I see so far. Okay. So already I love how they're playing with the different filters Okay, I'm not a videographer. So I might be saying the wrong But just the way the different colors going like black and white it went from this movie feel Then it went to this black and white feel I like that already because I like I like to see Music artists like challenge like what is supposed to be the normal and music video So all this respect to that and that's why I'm a big fan of kid Cuddy.  Let's go back. Let's go back. I appreciate someone. I'm just seeing right now I like this whole sequence right here. I want to see this again Dot dot dot dot Okay, yeah, I gotta see the dance practice. I know they killed this. I know him and his crew killed this. Oh Yeah, listen, I have a story for y'all, but I tell y'all another day, but like Yeah He's dangerous He's dangerous. He can really get down for real for real and Some of the crumpers that he rolls with that knows like that knows of him They roll with the same people. So it's like really crazy. Shout out to my little homie yank out there You know the same overseas right now doing this thing Yeah, I just wanted to tell you real quick ladies dangerous. I want y'all to see his footwork He's very comfortable when he's moving like you could tell when somebody's not comfortable with the steps He's very comfortable with the steps He's just doing him. I got that. Yeah, he ain't that let's go Okay, okay Yeah, his crump is strong his crump is really strong. You are very strong crump. Look at that again Mmm. Come in here. Bang. Yeah Whoo, okay with the chest. Yeah, okay with the chest here Mmm. Oh man the way they shot this is so dope It's like different scenes is like just a different version of him Now I have to do more homework because one thing I love about y'all especially in the comments They'll always give me the story that I might be missing and ever since the beginning I start doing his reviews I'll always share with me with the story and the meaning behind that I Appreciate that that gives me the texture. So if you do know the story behind this I would love to hear from your perspective shout out to you But I want to get back to like how this is shot like look at this way This is side it felt like it was like a purple like a night like a pinkish not purple. I'm sorry y'all like pinkish I love that. Hey Yeah Yeah, I love this shot. It's different now for me I want to play with this color. I want to play with different oranges and yellow for my work one day I'm gonna see what I can do with that But they sold this very well. This is tight Yeah, I need to see the dance practice Let's go. Ah, you can tell that the dance practice was probably definitely crazy. Hey my homie Carter Y'all know he do reaction videos with me He will go crazy off this video because the way they shot this He will love this This reminds me of the early 2000s y'all Because people that was trying to crump they used to do this That's so funny. That's why I laugh not at him. But the fact that I remember when that first came out Let's go. Whoo. Oh my god. I would love to see The full choreography of this part right here.  I want to see what they really made Like all of it. I love I love what they're doing with this video Let's go get all some your ball when I find that oh Man hey his crump This basis is nice. It got it down. I see I'm working that you would tell him he knows what he's doing Hmm. Yeah, everybody's probably getting down in that cypher though. Mass off So what did you think about it though? Of course, I'm gonna tell you how I thought about it But what y'all thoughts about it? Sorry. I mean it's nice right now. What were your thoughts about it? Like artistically, what did you think story line? Please share with me and Are you interested to see? All the parts of the dancing To see like what parts was cut out and I now this is why I understand why like They make so many from the dance practice Performance videos and more I understand why the k-pop culture truly does that because Sometimes you got to look over and be like oh my god. We didn't put that in the video Because you just remember that part and how hard you went and what if sometimes it's not about just putting that choreography on tour You just want the people to see So I love the fact that there's a chance out there that there's a performance video Or there's a dance practice because I I truly want to see what all did the film crew take out of this This music video because the music video of course is is dope because the last music video that I watched a Lay was crazy. So I'm really not in that place of like oh my god. I'm shocked I'm just I'm just highly highly impressed How they can change it up and every music video feels different even though this is only my second one But I did have a chance to get a glimpse of others other work that I saw like in the past And I and I'm sorry I didn't get a chance to react to them But what I will do is give you my review on it. Got you But just looking it just like look how they shot it it went from a moment of like a movie and Then it went to like these other places of like reality and that's why I appreciate that was that was my whole tape You gave me this look and then right here it start to begin to go into a whole another place So, yeah, shout out shout out to the whole team that put this together just different ways to look at these Music videos and these performances. It's it's just really dope. I'm actually really thankful to see people continue to Go beyond the envelope of creativity. So I'm really happy shout out to all the artists out there Shout out to the kpop shout out to the hip-hop shout out to everyone all genres of music shout out to everybody But listen y'all I got more videos for y'all But first I have to go I gotta go teach my class if you don't know you new to this this channel I am a professor at the Ohio State University And I teach on the street dance culture that I grew up in since I was a baby Technically five and I've been doing it for years, you know technically Also basically my whole life. So I gotta go teach and I'm blogging that I got a mega blog on the way on my other channel So do me a favor, please You know go ahead and subscribe to the other channel that I have on here because you get a chance to check out my vlog That's coming out is this basically a blog that's gonna be let you know what I've been up to this whole time All right, but much love to y'all. I hope you enjoy this reaction review more videos on the way But in that y'all take care yourself take care you mental take care here, too. It's important. All right y'all Peace and drink your water drink your water


In health and physical education we've got 12 focus areas. In the health strand we look at alcohol and other drugs, food and nutrition, health benefits of physical activity, mental health and wellbeing, relationships and sexuality and safety. When we go over to the movement strand, we look at active player minor games and fundamental movement skills in the lower years, challenge and adventure activities, games and sports, rhythmic and expressive activities and lifelong physical activities. There's been big changes in how young people are physically active now, so we've had to look at how we can adapt the curriculum to take that into account and try and develop the skills that students need to be able to participate in physical activity for their wellbeing. We think about it as giving students a toolbox of strategies that they can use so that they're able to lead healthy and safe and active lives. Being able to support them to develop those decision-making skills, problem-solving skills, also those movement skills that they need to be physically active. All of those are tools that we focus on and the refinements in the curriculum are putting a greater emphasis on those tools as well. Personal and social skills are embedded throughout our curriculum both when they're learning about relationships but we also teach them those skills through our movement contexts as well so they learn how to be good team players, they learn how to be leaders, they learn how to work well in groups to solve problems. We'd like to hear your feedback and you can do that very simply by having a look at the curriculum and providing feedback online through the survey.


Okay, can you hear me? I've got some noise here. Okay, cool. All right, so we're going to get started now. And I'd like to thank everyone who's joining us today. So welcome to today's CNCIP webinar, SMP Experience Report, the multi-cloud serverless on Knative. So my name is Daniel. I'm working for Red Hat as a technical marketing major, but also I'm responsible CNCIP investor. So luckily, I will be modeling today's webinar. So we'd like to welcome our awesome presenter, Yvonne Anderson, a software engineer and VMMer and Mark Wang, head of cloud engineering and SMP global ratings. There are a few couple of things of housekeeping items before we get started. So during the webinar, you are not able to talk as an attendee. So but there's a Q&A box at the bottom of your screen. So please feel free to drop your question in there and we will get to as many as we can at the end. This is an official webinar of CNCIP. So please, and this is a subject to CNCIP code of conduct. So please do not add any anything of the chat or question box would be violation of the code of conduct. So please be respectful for all of your fellow participants and presenters. Please also note that the recording and slides that will be posted later at the CNCIP webinar page www.cncip.io slash webinar. And with that, I will hand it over to Yvonne and Mark. Take it away. Hi. So as I was introduced, I'm Evan Anderson from VMware and I'm one of the TOC members on the K-Native project and I've been working for the last couple of years. And we recently had an opportunity to work with Mark at S&P Global on actually rolling out an implementation of K-Native. And so I'm going to let Mark introduce himself a little bit. Thank you, Daniel and Evan. Good afternoon. Good evening, everyone. So my name is Mark Wong. I have the privilege to run cloud engineering at S&P Global Ratings. So like Evan introduced, so we have a current existing relationship with VMware. And once we selected the technology of K-Native for a multi-cloud serverless implementation, then we started working with Evan and the team. It's been a great collaboration and our goal today is to share our experience on this journey. And the talk to me, it sounds like it's almost like 100 level. So I'll go through some 100 level and then 200 level developer experience and Evan's going to get down to the nitty-gritty details of 300 level content. So Evan, anything before we start? Oh, I was just going to say that, you know, when we started this a couple of years ago, it was, you know, kind of a dream that, hey, all the pieces were coming into place, that you could actually run a serverless solution on your own infrastructure without having to build all of the pieces yourself so you didn't need to build your own schedule or Kubernetes had one of those. And so it's exciting to see companies like S&P actually taking that last step of not just having the software, but actually having a platform that they can build and customize to their own needs. And it's been great working with them to be able to get that to happen. So yeah, I'm going to let Mark talk a little bit about how things actually, you know, how S&P actually decided to do a serverless platform. Right, so when you think of serverless technology, right, some of the basic things come to mind, one is you don't have to manage infrastructure. It's the speed to market, right. And cost is another, you know, because of compute density, right, so you're not using machines, you're using everything packed into containers, and even functions, right. So why do we go to a function of the service. To me, the biggest benefit is we can ship software faster, right. So I'll go through more of our background in terms of our technology portfolio. But if we look at the overall goal right so why do we want to adopt that as function of the serverless function at the service or serverless technology right is to ship our products faster. So we want to move from monthly quarterly releases to weekly or daily releases, right, so release frequently so that's one of the benefits from serverless technology, which is that teams from teams can focus on the business logic and not worry about infrastructure worry about coding standards worry about vulnerabilities. A lot of those will be taken care of I'll show a little bit more about that. And then the other thing is that they can own more of their staff right so they don't have to depend on this team or that team. So the automation and the pipeline will take care of a lot of that. So they have to see ICD experience right they can release code, as soon as they're ready to any environment they would like. And the other benefit is that after we break down the monolith and go more and more functions. We want to take out more key capabilities, then we end up retiring our legacy monolithic applications. So less technology that worry about. And one of the things in our journey that we found is that as we moved along. You know, Knative is a new technology even Kubernetes and new technology for us. So for us to have this global fads initiative we really needed to transform our culture. One of the things we noticed as we moved along is that we partner with the application developers, because the subject matter expertise for Python for dot net for angular. And for other technologies resides with application teams right. So these we have an open kind of development open collaboration model, where the engineers can contribute to the platform. I'll show you more detail about that. And then when they contribute they kind of unblock themselves as well. It's almost like a kind of win win situation they're building shared features on the platform, meanwhile they're unblocking themselves, and they're enabling their peers to do more on the platform. So if we go to the next slide. Okay. So this is a background. So 2020 one of our five strategic initiatives is to be certified on function of the service. So if you look at our portfolio two years ago, where majority is on prem. And then last year over a period of nine month or a month we migrated 100% to the cloud that migration was kind of a learning experience and the culture change in itself. If you think about our company right SNP is a financial trusted financial institution in the world, right globally. So we've been around for 160 years. And today we're moving at the speed of fintech. So last year, 2019, by mid 2019, over a period of a month, we have migrated 100% to the cloud. And today we're benefiting from that right with COVID with remote working, we enjoy stability and freedom in the cloud right. So what is our next leap so the team gained a lot of confidence from this experience to move to the cloud, and just a little bit more background on that right While we're doing the move migration mass migration to the cloud, we went through in sourcing as well as agile transformation. So there's a lot going on culturally, but what clear leadership and dedication of our teams, as well as ingenuity from our engineers. So a lot of people don't know the cloud at that time right when we started the journey on the project there was just one and half cloud engineers. And I myself as half of the engineer. So we slowly built up the team internally, we didn't really get much help from outside, other than we partner with VMware to use a VMC solution for some of the VM workloads. So we took. So after that experience we get a lot of confidence. And then we go into 2020 we said, let's take a leap and go all the way to function as a service. So if we go to the next slide. Let's take a little bit of that journey. In terms of the roadmap right. So, we did a market research on what are our options to go to function the service lambda is the first one comes to mind right, and then Azure functions. So we evaluated those as well as other open source solutions besides K-native. And then we came up, came to the conclusion that K-native really gives us a cloud independence and the multi cloud capability that we want. And there's a lot of smart engineers like Evan that, you know, backup the active community behind K-native. Towards the beginning of this year we started experimenting K-native. So if you look at the first swim lane. So we experimented with K-native. And then by and the Q1 in partnership with Evan and team. We were able to release our first version of function as a service platform. And then we were able to release a handful of applications to the cloud to production. And the other, the third swim lane is really around adoption, right. So, regarding that benefit we talked about earlier that goal of shipping product faster, right. So for us to be able to ship product faster we need mass adoption. So we came up with a concept of certification, right. So the certification really is to measure the maturity of application teams. So bronze is a goal for 2020. It's really to make sure teams know how to do function as a service using K-native. And then they understand how to scale up and scale down. So we have certification meetings we make sure the teams are certified. And they understand what is their roadmap, how can they break capabilities out of the monolith to put into serverless. So if you think about it, silver really is to break out more high value capabilities out of the monolith and put it on to K-native, right. And then when we get to gold, we will have a consolidated portfolio of capabilities. And we will also retire bunch of duplications and monolithic applications at that time when we reach gold will be completely on function of the service using K-native. And let's go on to the next slide. So this is kind of a release view of the platform itself. So we majority of the responsibility of my team is to enable the application teams and strong teams to be able to onboard their use cases. Like I talked about earlier we really opened it up we had nine work streams, engineers from different areas or we're helping collaborating and moving this thing along. So in a month we'll have a platform release. So MVP one was that we have a base platform and so we have Kubernetes, we have K-native, we have Istio, we have Azure DevOps for CI CD. And then we have a set of pilot applications. So that was able to go live. And then we also define this model to collaborate with engineers from different groups. So 2.0 was released in Q2. At that time we have 24 by 7 support, and then we have introduced OIDC as a standard for authentication. We have logging, monitoring, onboarding automation as well. I'll show you some of that in the demo. And we have CI CD pipeline automation. So at this time we open it up by 2.0 Q2 we open it up for mass adoption. And we introduce the reference implementations. I'll show you a little bit of that in the demo as well. And in Q3 we are going through more maturity. So now we have K-native function as a service running in DMZ. So we have external phasing application that's using this capability now. And then we have distributed tracing and we're using spot instance for all of our clusters. And we also introduce containers, right? Container as a service because not everything can go to K-native, right? We have commercial off the shelf products that we don't manage the code for. And we also introduce container security. For us to go to DMZ we want to make sure our images, our runtime is secured. We can go to the next slide. Alright, so some highlights on the features. So we support currently these languages for our developers, Java.net, Python and Angular. And then for eventing we support Kafka as well as active MQ. And for security we have Twistlock. We have open policy agent and fortifies for our static code analysis. And then on observability side we have logging. So one of the things is the developers for productivity, right? They don't have to worry about logging into anything. There's obviously no machines. They don't have to care about where the container runs. They have full visibility with logging and monitoring. And we added distributed tracing as well with open telemetry. And like I said, we're using spot instances. I think one thing I want to highlight here, maybe in a later slide as we do blue-green cluster automation. So we'll go to the next slide and we'll talk about that. Yeah, so some of the major components, right, is we're using EKS currently Amazon. And this technology is completely portable on other cloud providers as well, right? We're currently using Alibaba and as well as we're going to start using Azure. So because it's built on Kubernetes and because Knative is built on Kubernetes, all these will be portable as well. And we're using a lot of automation, right? Like I talked about blue-green. So every month when we release, we will build a whole new color cluster, right, from dev all the way onward. The reason why we do that is we want to make sure everything is automated. And we want to also make sure we respond to the changes in the base underlying Knative and Istio features very quickly, right? Because the technology is changing very quickly. We want to be up to date with the underlying technology. Yeah, we can go to the next slide. Yeah, so this is a kind of a quick view for our developer experience. So we have local environment capabilities. So if developer wants to set up local developer experience with building the image locally and running the image locally, they can do that. And we also have pipeline with standards built built in and automation built in as well. So once they go through the pipeline, let's say they deploy the code to dev, then immediately that is the image is shipped to our Artifactory image repository. And then it's put into Kubernetes through Knative and accesses to Istio. So why don't we jump to the demo? So I'll give you a look at this. Let me share my screen. Can you guys see my screen? Yes. All right. So we want this to be a self-service as possible, right? So this is the view of our documentation. So we have a getting started guide that Scrum teams can come in and use it to get onboarded, help themselves with CICD, secret manager, and then we set up a meeting with them in 30 minutes they can get going. And we cut down our onboarding time from when we started in the MVP one that took like a week for a team to onboard a function. So now it takes about two hours. So I'll show you a little bit of that experience. So part of it is not all two hours. What was that? Not all two hours. No, no, no, no, no. I'll make it quick. It's going to be much less than two hours. So it starts with documentation and the onboarding automation I talked about so in Jenkins, right? So based on the different types of reference implementation. So for example, if you're a Java application, we'll create this template. So fill in all the details that you need. And then once you run it, we're going to build the reference implementation. So the reference implementation has the best practices and the standards built in. So once we run through this pipeline, it's going to create that reference implementation into a template for you. And I executed this for our demo here. So for example, I have a demo repository called demo mark rest. So it's a RESTful function and it's a Java function, right? So we have code reviewers, we have other team members and we have secret manager defined here. So after we run this, what it will do is it's going to create a reference implementation here. So if you look at this, this comes sort of out of the box, right? So you have a reference implementation, you know, all the monitoring the log and the documentation comes out of the box as well. So all you have to do then is to cut and paste your code into it. I'll go through a couple of the components of this, right? So since this is a Java function, we have a POM file. So POM file defines your dependencies for Java, right? And then we have a Docker file. These all come out of the box. And then you see this Docker file has my function name built into it. And then we use a base image for Java, for example, and then we use other base images for different reference implementations. And we then have a pipeline file. So this pipeline file is important because we use ADO, Azure DevOps, and this defines what goes into the pipeline. So for this pipeline, for example, I turned off the fortify, right? Because we don't want to run this for like 10 minutes because fortify is going to scan the code. And I turned off a couple of other things in here as well. For example, you can all these things are configurable. So you can turn these on and off. So for example, I turned off some of the QA automation. So you can have CI, CD and CT built into this, right? And as part of the certification, we make sure that you have code coverage, you have continuous testing as well as well as, you know, sanity test, smoke test, regression testing. So these are all configurable.  For this demo, I turned on the observability. So I want to kind of show you that. And the other important file is service YAML. So this is the K-native file. So in here, developers have full control over what is their minimal number of instances they want to run, maximal number. As part of the certification, the teams have to demonstrate that they understand all these underlying technology. So each team is required to test their performance. So one team came in to the certification and they low tested, they performance tested their current usage up to 150%, 150 times the current usage. So for example, they had like 20 concurrent users. So they tested up to 300 or 3000 users, right? So their pot scaled up to 10. And then when they realize that they really don't need 10 pause maximum.  But they have full control over this. And then the other thing I want to show is that going back to the pipeline. So they have full control over here, I just stopped at QA. They have control over deployment all the way to production DR, DMZ and so on. And for the demo purposes, I'm just going to make, oh yeah. So these are the application config files that we can manage. For demo purposes, I have this sample function is a ping pong method. And then there's also a method to pull out some sample ratings. And what I'll do is I'm going to change the code here to, let's see, I will change the code here to, let's say CNCF ping pong. So imagine this is what the developers will go through to change the code or even put the code in the first time. And as part of the pipeline, it's gated release, right? We want to make sure that your release ties to a story, right? So we are continuously delivering value, right? So then they will commit the code and then they'll do a pull request. So even within the pull request, so I will check into the October release. In here, we are sure that you must have code reviewers, right? So this is another gate that you must go through. And I'm just going to, you know, for, I don't want Evan, no, Evan can't do it. I don't need anybody else to approve the code. So I'm going to approve it myself. And then we'll set the automation to build the pipeline here. So that's going to kick off the pipelines there. So we can go in and take a look at this. So the pipeline, what it does is it's going to start with a snapshot build. So you can see all the outputs of these. So we do a snapshot build first is to ensure that the container image will actually build, right? The Java code will build, because if that fails, there's no point to pulling anything into it to canate it. So we'll just fail it right away. So here we're doing a snapshot build. So it's going to do a Maven release. It's going to do a Docker build as well. And your developers don't need to figure out how to set this up, right? This is just from a template that's already published. And they can go in and adjust it later if they need to. But they don't, but just to get started, you've published templates. That's right. If you see this parameter in the template, so all of these are part of that initial automation that would create the reference implementation. This is for Java. And then all of that comes out of the box. So you see this is all based on a template. So it's GitOps, right? So they use template out of the box, and they can control all these different parameters within it for their pipeline. Good point, Evan. So in this case, the snapshot build is successful. And after that, it's going to run a build to deploy to dev environment. And you have certain controls, right? This is one I did earlier. So for example, you can, we don't need to deploy to QA, right? So we can just reject it. So QA has a manual step where somebody has to approve it. You know, they have to make sure that you have enough code coverage. QA will accept that build. So in that case, for my demo, I just stopped that. So for here, we're doing a build and deploy to our dev environment. So remember, we have this configuration here that for dev, it's going to do a very simple, right? For a time sake, we're just going to do a build. And we're going to take that image and deploy it into QA that into the dev environment. Oh yeah. And then here, here's that dev environment. So I'm going to watch that. So it is currently running. And this thing is going to show us that it will deploy the new one. Right. So it's, this is running currently in our dev environment. All this is running. Do you want to show any of the instrumentation or logging? Oh yes. You get out of the box. That's a good point. So for the instrumentation, every function that we release out of the box is a monitoring, right? So you can have health monitoring of your function. So remember I talked about blue green. So this dashboard shows you what color we are, right? Because we don't want developers to be confused going to the wrong cluster. And typically we once we deploy one cluster, we shut down the other cluster so they won't get confused. So you have the developers have full visibility into the health of these, the clusters and their functions, right? You can have, you can look at different namespaces, different functions. So full visibility out of the box. You don't have to, you know, care or, I mean, depending on how much you want to care, right? You just worry about the code. You don't have to care about how any of this stuff works. So we simplify it for you. But if you want to dig into the log, right? For example, this is that function. You have full visibility into the log. You can drill into it to see the log. And then the other thing is that out of the box, we have alerts enabled, right? You don't have to do anything. So this is the container health alert. This is log monitoring. So if you have errors, beta errors in the log, you will receive alert. Because in the onboarding, remember, you have to pull in your contact information. And so that we use that and I set up an alert for you. And then this is more of a native related alerts. And if we go back to this, so the build is done. What happens is after the Docker build, we will publish the artifact into Artifactory. And that is used for the deployment into the Dev environment. You use the same artifact for deploying to both Dev and prod. That's right. That's right. So remember we did a snapshot. The snapshot is making sure that your build is clean. And once we have that, if we remember this pipeline file, you will use the same artifact. You won't have to build again. You use the same artifact that deployed all the way through. That that will be your gold copy, essentially. Yeah, and then so this is, let's see how this is doing. We are terminating it and then put in the new new code. And then the other thing I want to show you is more around time series data so Prometheus gives us all the time series. So this is kind of a high level cluster level view. And then you can also view your functions and drill down to the health of each of your functions as well. So yeah, I won't, I won't give you too much information here. Let's go back to the deployment. So yeah, it's, let's see. So this usually takes about three minutes. So it's deploying the service now. I think in here I put in the destroy command. Yeah. I put in the delete. So I'll delete the previous deployment. If you don't want to delete it, then it's going to just use out of the box rolling deployment. And we're also looking at blue green deployment for the functions as well. So that the functions have full control over different versions being deployed. When we look at the pipeline, so that's done. The pipeline is done and you see all the, I think this is a little delayed the message here. That's the, that's the old containers shutting down now. That's right. The old ones. New ones up and running. So you should be able to show it off. Yeah. And the new one is running. So if I go to here. This method here. Yeah, C and CF ping pong. That was a code change. So I'm going to hit it a couple of times. So then we'll go to open tracing so we can see these things. I hit this a couple of times. So if we look at the open telemetry, right, this is also out of the box. So you can see the, if you want to debug, right, if you can go trace these things. So for example, if we look at, yeah, you see how this is. Let's see if we look at the traces, we're at 133. Yes. So this is that ratings, right, we're putting pulling back some sample ratings. You see this, there's a bit of a cold start right. So the first one took one second and then the subsequent one took milliseconds. So let's drill into it right. So the first one you can see majority of the time is spent on Java right. So this is your, this is your select statement database, which is pretty quick. But then if we look at this one here. This is your credit method pulling back all the data so that took majority of time for one second. But then if we look at the other one. This is after the code start right so you look at this one this total was nine milliseconds. And your select went faster too. Yeah, select went faster. Everything went faster. The second time around it took a little bit of warm up. I didn't actually notice that. Well, I did notice that but I didn't see until we had this instrumentation in place right so developers also get this experience out of the box as well. So that's all I want to demo for today so I will turn over to Evan. Okay, you'll need to stop sharing. Okay. Oh, just one more slide. Okay, so this technology is very new and exciting right I think couple takeaways one is that with that open contribution model right teams are able to come in and contribute and really get excited and get on board it with this right. We have 90% of the application in scope for this year meaning 90% of our application portfolio have either started or completed fast function. So if we look at a team view right 50% of our teams, all of our teams have done fast functions already. So we're kind of sort of following this, this adoption curve we're towards a ladder stages of adoption now so this year nine months from having had something to show them at all. That's right. Yeah. It really helped us with on our journey because it simplifies the developer experience as well as the infrastructure experience with with using Kubernetes. Okay.  But what does that actually mean. Knative is a system for building serverless HTTP applications. So we've talked a little bit about serverless. I'm going to talk a little bit here about why we do HTTP and then we're going to dig into more about how that serverless stuff works. So when we were starting the project, it seemed pretty clear to us, building an open serverless system that we want to build on some pretty robust standards and HTTP kind of struck us as the obvious way to get requests in and out because it's well understood. And it keeps evolving and improving so it's not like we picked a standard and you know it's going to be the same three years or five years from now. It'll keep getting better. Support for things like W3C's tracing will just automatically get added in and so all your tracing tools should work from front end to back end. In the next year or two would be my guess as the you know as that ecosystem evolves. Similarly we bet on Kubernetes for container scheduling. We knew that was going to keep improving and we wanted to be on the ocean where the tide was rising. And that would lift our ship as well. But we also wanted to specialize Kubernetes more than Kubernetes itself is sort of here, here's a tool you can do anything with it and you're like, I can do anything. But most of the time you don't want to do anything. You have a specific thing you want to do and so we wanted to make it a sharp specific tool for cases where you were building something that was basically a 12 factor application. That you were willing to speak HTTP that you didn't need to keep local state and that we could make things a lot simpler. And so the first place this shows up is, I don't know how many of you define Kubernetes deployments on a regular basis. This is kind of the smallest simplest deployment you could have over on the left hand side. You've got a deployment but if you want to talk to it you actually also need a service, and you need to have a bunch of selector labels, and you need to match some ports and stuff like that. And so you have at least two objects you've got to keep in mind and you have to think about labels. There's actually a lot for developer to dig into here because we haven't really talked about like what's a Docker image. Over on the right hand side, you can see a K native service. You basically say, you know, hey, this is a K native service that's the API version and kind. Here's its name. It's container. And it speaks HTTP.  And you get some services that you don't get from a standard Kubernetes deployment. Like an autoscaler, you could go and figure out horizontal pod autoscaler. Oh, look, we just added a little bit more lamble YAML to the left hand side. So, we mentioned autoscaling. And then you get this to some extent with ingress for management of HTTP host names. Again, we just added another object you have to think about to the Kubernetes space. So, all those things are useful when you need to do something strange, you know, when you say, hey, I'm running a bunch of game servers each game server is kind of independent, but they're kind of together. Kubernetes is a great fit for that. But if you're building HTTP applications, K native simplifies that. We also build in some tracking of previous states. Every time you do an update, it creates a new revision. So that's a little bit like what deployment does, but it's a little easier to find them and go back to the earlier ones. And they're the garbage collection policy can be date based and and over time without having to think about, you know, how many replicas sets am I going to keep around.  And we'll get you up to the number you need. And since we're serverless, and we sort of start from zero. It's easy to just start a new pool and scale that up. And then the old ones go away when they need to. So you may burst a little bit more if you want to switch things over quickly, but you can also do it much more slowly or get more control over HTTP. Also, since we knew do new we're doing HTTP and we knew that it's 2020. So we built in integration with Start Manager and less encrypt so that you can automatically also get all your SSL handled without developers having to get involved. And you can have just a single wildcard cert that covers all of your functions in all of your domains. And let's talk a little bit now about how S&P actually ended up using this. So they talked about having blue and green environment. So you can see in the picture down at the bottom, you know, in their dev environment, they'd have cluster one and cluster two. And one would be blue and one would be green. And so they have a specific DNS zone. They also have a top level DNS zone for the environment. So dev has a different domain than UAT or production. And so if you don't want to have to think about, you know, are we in blue or are we in green, you can just hit that top level thing. You can see the details. It's there. And they integrated with AWS certificate manager to do the provisioning out of the box. Knative ships with integration with let's encrypt using Jetstack Start Manager, which is great if you have an internet connected cluster and you don't need too many certs. For a company like S&P, they can afford to send a few dollars AWS's way to get higher rate limits and certain guarantees that let's encrypt just isn't set up to give. Their goal is to encrypt the internet, but not necessarily to run a financial business. So, you know, find the right tool and one of the goals with building Knative was that you should be able to customize this stuff. So I think that was a success. And now we're going to talk a little bit about what the data path looks like for Knative serving because we've talked a whole bunch about it being serverless. And let's see what that actually means. So the first goal for handling requests. I call this life of a query. I got my start at Google and one of the first talks that you get is here's what it looks like when you actually do a search query. So I always call it life of a query. But the goal for steady state is that things should look pretty close to the same cost as if you were just using raw VMs or raw Kubernetes. So a load balancer splits stuff across your HTTP routing layer and stuff gets sent to a user container. And that's all lovely and good. And then so the next question is, you know, okay, lots of traffic comes in, you know, that goal of 3000 concurrent users, for example, that Mark was talking about. How do we actually count those users and then make sure that we've got the right number of containers and Knative does this by injecting this little proxy in front of the user container and being able to count all the requests and feed that back into a request based autoscaler. So if you're familiar with the Kubernetes horizontal pod autoscaler by default that will queue off of CPU or possibly off of a custom metric.  The default behavior of Knative is request based scaling. So you can take out the Knative autoscaler and put in horizontal pod autoscaler. It's possible to do we find that most people actually like the request based scaling. So this is, you know, building a particular focus tool, because we understand HTTP and we can get into the protocol. Okay, so another thing that AWS Lambda gives you is, you can write. Well, in Lambda, you have to write your code with the assumption that there's only going to be one request going on in a process at a time, because that's how Lambda works. But lots of people actually like that because it means that if you want to have globals for stuff or you just want to know that you aren't going to get interfered with anyone else. You have, you know, a container that wants to run a request at a time. We wanted to support that in Knative. So Qproxy also lets us enforce, hey, you know, you may have 50 HTTP routers, but you're only going to get one request per container at a time. You can also crank that up if you want to, you can say, hey, 50 or 100, you know, look, I've written this in Java, it's all re-entrant, you know, let it go to 1000 or keep it at one. And the default is to assume it's re-entrant, but it's easy to just say, you know, one request per container. So now, now, you know, okay, that's all nice kind of nice ergonomics. How does that scale from zero thing work? Because there's nothing actually running. How can we count how many requests there are when a request comes in? So this is where one of the clever tricks of Knative is that we run a single activator for your entire Kubernetes cluster or possibly a replicated set of them, but a small number, when you add a new function, it doesn't add a new activator. The activator is shared and it will pause the request and it'll say, hey, there's no instances of this. Kubernetes, please go talk to Kubelet and actually get a pod ready. And once that pods ready, then the activator will forward the request along. So you'll see a longer response time for those requests, but they won't get dropped on the floor. And so we've talked about, okay, we had zero, we want many, how do we get there? How do we do the opposite? No requests have come in and we've shut down. It turns out that actually takes a little bit of a clever dance as well. But first you need to add the activator in. After the activator is hooked in and the HTTP routers all know about it, then you can scale things down to zero the rest of the way. And Knative tests and handles all of that so you don't have to think about it and you don't have to build anything complicated and you get scaled to and from zero. The last part is Mark talked a little bit about blue green deployments and progressive rollouts and we wanted to support being able to say, hey, this is a new canary thing, we're only going to give it 10% of traffic. Even in that case that Mark was talking about where they have only 20 users, maybe you only want to do 10% of traffic anyway so that you tell people reload a few times and it should work. But you only have four pods. So with standard Kubernetes services, they're all around Robin. And if you replace one pod, it's a 25% rollout. Since we control that HTTP router we can actually and it's something on void based so far, we can actually replace things in there and say do 10% and 90%. I just kind of hinted at this one, we support multiple HTTP routers so contour, Istio ambassador glue and red hat wrote a really lightweight one called courier that's specifically focused for being used by Knative. So if you want to integrate with your HTTP router.  Sorry, throw it all away. So we have adapter layers for all of these and you can choose which one you want to install. I'm going to blaze through this because I have one minute left maybe. And then we'll have time for questions at the end. So the mental model for Knative is, I showed you that service a service is made up of a route which is the networking part and a configuration, which tracks basically how you want things to run and the configuration creates additional revisions. Every time you update it. The route lets you pick which thing receives traffic it can either be the latest revision or a specific revision. We call running with the latest revision at 100% Yolo mode, you only live once. But it's also really handy for for development and you know stuff that's not really critical. And that's the end. We're available for questions and anything else that, you know, feel free to use the Q&A or chat or stuff like that to get in touch with us. We've got contact info afterwards. VMware is hiring. I should probably add, we are looking for people to add to the Knative community and to hire them so feel free to reach out to me. Twitter DM or email. And I don't know if S&P is hiring Mark. We are I'm actually currently looking for two blockchain developers. And by the way, blockchains running on Kubernetes too. Cool. Yeah. Okay. Yeah, thanks to Evan and Mark for great presentation and demo. So we have now some time for question if you have some question around this topic. Yeah, feel free to drop in your Q&A tab. Okay, we got a one question just came up. Who will take this one? I can take that one. Okay. I'm going to answer it live. The question is, is there a significant difference in responsiveness between auto-scaling with Knative versus the horizontal pod auto-scaler? As always, the answer is going to be it depends. I will say that the Kubernetes horizontal pod auto-scaler may be, I don't know what knobs there are to tune how frequently it's checking to decide if it should rescale an application. The Knative auto-scaler team looks at that and has tests that are basically, hey, we're at zero. What happens if we dump a thousand requests per second on a Knative cluster with container concurrency set to 100 or something like that? So how fast do we get to the 10 or 12 instances that we should have for handling that much load? So they have a pretty fast cycle on the metrics collection and they've recently migrated that to GRPC from HDP to get additional efficiency benefits because they were finding that for large clusters, it was too slow. I haven't seen HPA measurements, so I can't tell you HPA is going to be slower, but my suspicion is that it's probably not tuned to check as fast and you may be able to tell it, you know, check every second or check every two seconds. Thanks for the answer. Okay, I think. I'm assuming that Mark, that S&P is using the default auto-scaler, the Knative one and not horizontal auto-scaler. We have a demo of it where we just scaled it up to 100 parts in like seconds, it was very efficient. The goal when we started Knative was to be able to match Lambda's performance eventually and that's going to take more effort and getting down into Kubernetes as well. Some of the limitations we see today are around things like how long does it take to schedule a pod and pull the image. And so I think that there are one or two caps that are percolating about how do we make that faster. For example, readiness probes. If you use the Kubernetes readiness probe, you've got a minimum of a second for your service to become ready and we'd like that to be below a second, you know, every 100 milliseconds or so. And currently the activator goes and does that check even on unready members of the service to see if it can race and beat Kubernetes propagation time on services. I think one of the things with Knative is that with Lambda's and Azure Function in the world, you're very limited. With Knative, we can do so much. We have full control over what we put in there. We're even talking about, with Pivotal, about trying to put a power builder in there. Because we have legacy apps, right? We want to modernize and there are different ways of modernizing. It gives us a lot more flexibility basically because it's in our full control and we can do whatever we want. And it's multi-cloud. Another great example of that is Doug Davis has a prototype that he's trying to figure out how to how to fit into the community for being able to just have a shell script that gets adapted to an HTTP server. So your container just has a shell command in it, you know, as a shell container and then inject an HTTP server binary through a volume mount, rewrite the entry point so that it calls that HTTP server and the HTTP server invokes your normal Docker entry point once for every HTTP request. And it's a pretty cool demo and we're trying to figure out, okay, you know, how do we make it that next level of accessible to developers? Yeah, I mean you guys are moving fast because we had a feature where we wanted to support S3 for IEM roles and that was, you know, we put in the request through GitHub and then that was released to us very quickly. So quick turnaround on that. Definitely been an interesting, oh, there's another question so I'll stop just convincing. What percent of compute or app estate could go to Knative or serverless ultimately? I'm going to pass that over to Mark first to talk for S&P and then I can have some unfounded opinions about the state of the industry. From our experience, they are other than the commercial office shell, right, if you think about breaking capabilities, so we are more mapping capabilities, right?  But then when we look at it, right, what percentage really depends on what percentage of your apps, you know, what is the current state of the target state. The target state for us is 100%, right? Other than the commercial office shell, we can move into, you know, because it's container based, so we have a lot of flexibility in terms of what we put into it. I've also moved some commercial off the shelf stuff to a serverless environment if it can be containerized and is, you know, stateless containers. We are actually, so we move Pega and Documentum, we are moving Documentum into containers. It's a bad idea. I ran Jira once like that, but it turns out that Jira keeps a bunch of local caches. So that was an unsuccessful final experiment because your issues wouldn't show up until somebody told it to re-index. So I would say if you have off the shelf software, try it. You may discover that it's not successful, but some of it actually does work pretty well. In terms of overall, I would say that Knative, specifically the current Knative serving, it might be about 30% of sort of all workloads where all workloads could also include stuff like, you know, databases and storage and stuff. I think there's other places where things could get more serverless than they are today. Looking at stuff, you know, a system like Pulsar or Kafka or ActiveMQ that scaled out just automatically from one node to many without having to think a lot about how many partitions do I have would be a really cool serverless, you know, distributed log system. And I'd love to see one of those. The follow-up question is, is state an issue here? How does Knative deal with it? So Knative and a lot of the serverless platforms today are stateless, like 12-factor. And the assumption is you can start two or five or 10 instances, and when you don't need them, you can just shut them down. So if you need to have state or shared state, using something like Memcache or Redis, if you need lightweight stuff, or just using, you know, your database to share state or an object store to share state, keeping state in your process or on disk, not such a good idea. So don't do that. And then there's a question about plans to support fast and Knative on ESXi through Project Pacific. Well, so two answers. You can run Knative on TKGS clusters today. That works. We use it for some of our internal testing. In terms of product plans, I'm going to have to point you back to a VMware product manager. Valentina Alaria would be probably the best, but feel free to send me email and I can connect you with the right VMware product managers because I mostly focus on the open source software and I don't keep track of VMware's product plans. And so we have, other than HTTP, we also have Chrome jobs. We have messaging, like eventing, I showed in one of the slides, and we also have static web UIs, kind of like you would with hosting user interface on S3. So we have all those running in Knative as well. I think we had some TCP use cases after I think that up. Oh, you mentioned TCP. One of the fun things with Knative, if you're a network, if you were a networking geek, is we actually support HTTP, HTTP2 and WebSockets, all at the same time, which is kind of a fun combination of spaces to talk about because there's lots of different ways to stream things back and forth even though you are just speaking about HTTP. And that's one of the great things about having such a broad ecosystem to pick from. You can auto scale streaming responses, which is, I've seen good use cases for it. I've also seen that start to bleed into the stateful space a little bit more than I'm comfortable with. So with great power comes great responsibility. We are at time. I promise that things would go early. And apparently I've failed at that, but yeah, no worries. Feel free to follow up with questions on either Twitter or by emailing either of us. And thanks for attending. Yeah. So, okay, so any last word, Mark? Thank you for having us. Perfect. So, all right. And that's all the question we have time for today. And thanks for joining us once again. The webinar recording and slides that will be online later today. And we are looking forward to seeing you at the future CNC webinar. But also we have a CubeCon and CloudNativeCon North America next month and November 17. And we are looking forward to seeing you once again, have a good rest of the day. Thank you. And there's no capacity limits on that because it's virtual. Yeah, thanks for having us.  And have a good rest of the day, everybody. Bye-bye.


The narcissist will never change. They will never be different. They will never develop a sense of morality or fairness. They will never develop principles concerning the distinction between right and wrong or good and bad behavior. Because they don't care. It doesn't concern them. They are not worried about what you want or how you feel. Narcissists are self-absorbed. They only care about themselves. They only care about their needs. And they lack empathy. They lack the ability to share your feelings or experiences by imagining what it would be like to be in your situation. They do not have your best interests in mind. Many of them are sadistic. Many of them derive pleasure from inflicting pain, suffering or humiliation on you. So they are not going to try to help or support you. They are not going to try to do what is right or what is best for you. Narcissists are very miserable, dissatisfied people. They are also very envious and jealous. They don't want to see other people having the happiness or satisfaction that they cannot experience or partake in. So if they can't be happy, if they can't be satisfied, they don't want you to be either. And that is why they will do whatever it takes to sabotage your progression or success.  Because they can't get the same fulfillment. And every time that they have to witness your happiness and satisfaction, it becomes a reminder to them that they can't be happy. They can't be satisfied. They are very ungrateful. They don't appreciate anything that they already have. They are too busy comparing themselves to other people, which then makes them envious and jealous. It makes them want to take something away from you. It makes them want to sabotage what you got going on, because how they see you reflects on how they see themselves. They have to manipulate you to regulate their own emotions, to regulate their self-worth, self-esteem and self-importance. They cannot be grateful for what they already have. They are not self-sufficient or self-validated. So they are always looking to someone else to provide some form of fulfillment to them. But no person or external thing can make us feel adequate. No person or external thing can validate us. We have to make ourselves feel as though we are enough. We have to become self-validated. But the narcissist is not capable of doing this. Because at some point they abandoned who they really are. They created this false self. And because it isn't real, it has no foundation. It has no underlying basis or principles. So it requires a constant supply of external validation or energy. But no amount is ever enough for them. They are like a bottomless bucket that can never be filled. They have an empty void within them, because they abandoned who they really are and developed this false self. Which has no foundation, no underlying basis or principles. No bottom surface. Which means that anything that you pour into them will only leak out. There is no amount that you can pour into them to make them happy or satisfied. The narcissist is stuck in an endless loop of repeating the same things again and again, expecting a different result. They are constantly seeking supply from external sources, hoping that at some point they will find someone who can make them happy. Someone who can satisfy them. But it's just an impossible expectation to expect that from someone. No one can make you happy or satisfied. No one can constantly validate you or provide you with an endless supply of energy. But the narcissist is stuck in this endless cycle where that is exactly what they are desiring and expecting from someone. They are expecting someone or something outside of them to complete them. They are expecting someone to fill that void within them. The narcissist will never change if you could look back at their past. You would see that they have always treated people the same or a similar way as they are treating you. You are always expecting people to complete them. Always expecting people to make them happy or satisfied. No different to what you have experienced with them. And if you could look into their future or return to them 5, 10, 20 years down the line, you would find that they are still doing the exact same things with someone else. They are still treating someone else the exact same way as they were treating you 5, 10, 20 years ago. They are still miserable and dissatisfied. Still expecting someone to provide to them. Still depending on someone else for their happiness and satisfaction. Still doing the exact same things that they were doing many years before. Nothing ever changes. They are doing the same things just with a different person or in a different place. They never self reflect or look within. They never take responsibility for their actions. They never hold themselves accountable for their faults or mistakes. Which means that they can never learn from their errors. They can never learn from what they have done wrong. They never change or do anything differently. Because in their minds they never did anything wrong. So they continue to get the same results. They continue to leech off other people. They continue to depend on other people for their happiness and satisfaction. And since no one can make us happy or satisfied, they remain miserable. They remain in a hopeless state when nothing ever gets better. Things only get progressively worse and they are never happy.  And everyone from their past. As they always depended on someone provided something to them. They always depended on someone to make them happy. Which is an impossible expectation. No one can make you happy. Yet they continue to do the same things again and again. Expecting a different result. Because they never look at themselves and realise that maybe they made a fault or mistake. They never take responsibility for their actions. They never hold themselves accountable for anything that they have done wrong. And that is why they continue to repeat the same things again and again. With the same results. Never finding real happiness or satisfaction. Stuck in an endless loop of misery and suffering. That they can never get out of. Thank you for watching. I hope this video resonated with you. Please like, comment, share and subscribe. Click the bell icon to receive notifications for my future videos. If you would like to donate my PayPal links in the video description. Coaching inquiries you can email me at nazfavicoaching at gmail.com Thank you for watching and I'll talk to you soon.


TEFL New York City It is no surprise that a city the size of New York has plenty of accommodation options to suit all budgets and preferences. You will find many options in the immediate vicinity of the school or you can stay further away and use the city's extensive public transport system to travel to and from your classes. You will have the choice of several backpacker style hostels with both dorm rooms and private rooms available at budget friendly prices. For those who would prefer a bit more luxury then there are also plenty of good quality hotels to choose from within easy reach of the training center. We can also arrange a home stay type option where you can stay in a private room within the home of a local family in the city. In short, we can arrange accommodation for every budget and taste. Please be assured that any special requests or requirements regarding accommodation will be taken into consideration and we will do all we can to ensure your individual needs are catered to.