an see the capacity is like so large It's like 800 CPUs there And then you can also have like GPUs and other resources there So like I said you can think of it as like a node But to the back end we'll go somewhere else In any of the providers So for example this is how you would schedule a GPU In Azure container instances You add an annotation for the virtual cubelet that I owe GPU type And then this is going to be the GPU SKU you want In this case it's KAT but there are other SKUs also And then for the node selector This is how you would schedule Just like you're scheduling a specific Like a pod to a specific node This is how you would schedule in a virtual cubelet So if you say type virtual cubelet Hey I want this one And then you can have a node selector as like a Linux or a Windows node So in this case it's Linux And then this is how you how we'll get scheduled to virtual cubelet Which will then which will create a Azure container instances Instance basically All right, so so search Talked a lot about Azure container instance So what is it right? So basically it's a service that allows you to run your containers So you don't have to worry about the underlying infrastructure All the nodes that you have to manage yourself All you need to work all you need to do is bring your container right And all you need to do is tell it how much resource you need And where's your image And if you decide to use like files You can tell it where the network share is So all in all it's a provider That allows you to run your container What without having to worry about the actual infrastructure And the other nice thing is it's It's created as you request it So only it only runs and charges you per second Only when you actually request the resource Right, so if your container's not running You're not getting you you don't have to maintain it And you don't have to pay for it So why is this important in our in this scenario? Well, we worked with a lot of data scientists that were trying different experiments Now if you have ever had to create like your own cluster for your data scientists Obviously, you know, a lot of times what they the experiments that they need May require many GPUs Now if you have to create a cluster and maintain it over time With many many GPUs, that's very expensive Right, so with this what you can actually do Is you can allow your data scientists to schedule their experiments as they need And they don't have to wait for other data scientists to finish their experiments first So that they can run theirs, right So this actually gives them a way to schedule the experiment as they need And scale up and scale out as they need to perform the experiment So let's take a look at what VK and ACI looks like Can you guys hear me? So as as a searcher mentioned earlier here As you can see because of I pre-installed The Virtua-Cubelet Helm chart, I basically now have a virtual node, right So with this, as you can see this because the Virtua-Cubelet node in this case Is actually presenting itself as a virtual node But it's actually running on Azure container instance So now to me the cluster operator It looks like I have 100 GPUs on my disposal But I don't have to pay for it, right Because I'm not running anything So now let's see how a developer or a data scientist can actually schedule A container or a job or a pod in this cluster But really it's running off of an Azure container instance Here as you can see I have the often-demoed NGINX pod And all I'm doing is hey I'm telling NGINX I need this much, I need to run this container in a node called Virtua-Cubelet And I want to be able to schedule this pod there, right And so once this particular pod is scheduled And as you can see it is actually scheduled on this It's actually, yeah maybe this looks better So it will show that it's actually running on the Virtua-Cubelet node Not one of my nodes in the cluster as you can see here And then it gives me a public IP that I can use To actually reach my NGINX pod And there you go So all you have to do is create, schedule this pod like any other pod But the actual resource that is running is off of Azure container instance And as soon as I delete this pod it's no longer running And I'm not no longer paying for it Okay so I might just like maybe skip this Because as you guys see we've seen a lot of diagram But basically let me just quickly walk through this In a typical machine learning workflow right You have some data set that you want to train on And you want to be able to run training Whether that's sequential or maybe you have some hyper parameters That you want to experiment with And maybe you want to distribute that training But those are the parts where it's a little bit difficult Especially if you only have one machine Or maybe you're in an organization You don't have that much GPU right And so the other problem that a lot of data scientists faces Now I have a bunch of models How do I serve it right And how do I make sure the serving application is scalable And so these are the questions that often come to us Coming from the data scientists So for our particular demo We're going to use the same workflow But basically we're going to find a celebrity So the problem is a celebrity that we have And we have a bunch of people who are not that celebrity So basically an image classification problem And what we are actually doing is We're doing transfer learning using Inception V3 Which was previously trained with ImageNet So taking that, what we're doing is we create our own data set Of celebrity and not celebrity And we use transfer learning to train the top layer So that we can actually classify for our particular celebrity And in order to make our model more accurate What we are doing is we can run our experiments repeatedly With containers And because we're going to use Helm and VirtuaCubula and ACI We can actually parametize our experiments And run them up in parallel And we can visualize the training results As they're running with TensorBoard Because all of that comes with Kubeflow And so last but not least We have a pipeline with Kubeflow Pipeline Or Argo of your choosing So you can actually create that pipeline And once your model is done You can use a pipeline to trigger the serving stage Of your pipeline And then the serving stage will basically use The model that you want to use So let's look at the demo All right So here I have just a simple Docker run command And all it's doing is running my code Right So I have a Python code That basically takes Inception v3 base model And I tell in my code I go retrieve the images that I want to use for training Then I create some distorted images for my training And I create the last layer before the actual training starts And as you can see here I'm running this on a single VM And it's using K80, Tesla K80 for my GPU So all this is showing you is It's some container running the training portion of the experiment So next what we have is the serving part Right So once you have a model trained You obviously want to serve it And there are many ways to do it Right You can create your own class application And just serve your model that way Or you can use TensorFlow serving Or sold in however you want So in this demo We are actually using TensorFlow serving Because they just worked out of the box Right All of that is in this demo All right So what I'm doing here And again we're not even on Kubernetes yet We're just using Docker right now So here what I'm doing is I'm using the TensorFlow serving base image And I am basically using it to create my own image Using the model that Using the transfer learning model that I've already trained And then here as you can see I'm creating the serving server And then once that server starts It's running off of 4.8500 And then next you'll see I'm running some Python code That basically grabs an image And going against that TensorFlow serving endpoint And here as you can see is a code that is doing the prediction Right And then here as you can see Running against the serving server And then here I'm getting some results back Which basically tells me how good this prediction is And if it's classified as the celebrity I want So anyway, so that's the serving demo So in the next step we are going to use TensorFlow Training the Kubeflow to show you this We have a Kubernetes cluster with GPUs So you can see an immediate outcome GPU So that's pretty much it And then in the next one We are going to install Kubeflow So this is like an older installation Like older version And it uses the KS And then now there's the KKF CTL To be able to make this more streamlined So we are installing all the components here So at the end you'll be able to see the Kubeflow components Like I said these are the older version of the one that was recorded So there's more stuff now But this is basically a Kubeflow installation And then one of the things that Kubeflow really helps is Like compared to like a Kubernetes batch for example It allows you to just auto-discovery for like things like GPUs So before you had to specifically specify where the volume mounts is It's something that a data scientist or even an engineer might not know How it's configured Especially if it's on a like a managed Kubernetes installation So with the TFJob it kind of abstracts away all these components So you don't have to worry about like Hey, where's my driver pass for example So on the left you'll see that the Cut like a very simple batch implementation for Kubernetes For running the same container and then on the right is the TFJob On this one so we are basically creating that TFJob on Kubernetes So we do the TFJob CRD So we see the TFJob retrain is Running and then we have a persistent volume That's bound now So this is the container that's running And it will start running just like Rita showed earlier So it's very similar to that one But in this case it's running on Kubernetes using Kubeflow In this case So we are doing the same thing as you can see It's reserving GPUs for this container And then once So it will also create a tensorboard instance So once we reserve an external IP And then once we go to the To the external IP for the tensorboard You will be able to see the The tensorboard instance for our training On the next one we are going to deploy this to ACI All right so it's a holy grail right So now we got our training into Kubernetes And we're training with TFJob Now because of Kubeflow We have all this stuff running in our cluster But how do I actually scale it out right How do I have one centralized tensorboard That's looking at all my experiments happening at the same time And how do I only pay for the container The resortings that I need when they're actually running Right and so this is where With Virtua kubelet and ACI This is where all of them actually work together So we're gonna first look at the node And here I'm just showing you like that's the Virtua kubelet And next we're actually using Helm To install the hyperparameter experiment And again an application is an application right So you can basically create your application And package it with Helm like any other application So as you can see here This actually creates nine different jobs Based on the hyperparameters that I've created That I've parameterized And so because there are nine different permutations So it creates nine experiments with nine TF jobs So as you can see here all of the jobs Actually creates a corresponding pod And then the pod is that they're all running on Virtua kubelet As you can see And so what I'm doing right now Is I'm doing az container list So just to show you behind those things Is actually creating a bunch of container groups On azure container instance And as you can see it uses the right skew That I tell it to It's using k80 But you can specify whatever skew you want Right And as you can see here This is basically all the information That you give to azure container instance So it knows how to train And how do you run your container And as you can see It's actually creating these containers right now Okay so now the containers are running And just to show you what it looks like Behind those things right So each pod corresponds to a container group In azure container instance And as you can see all of them are running And here as you can see They're utilizing the resources And then you can actually look at the containers That are actually running And here's the lifecycle of the container And you can look at all the logs Very much like the logs that you saw earlier But now we're just inside of azure That's all And then what this is actually doing Is it's training all of your experiments Based on the parameters that you specify So each experiment will have its own test parameters And then in my Python code I've also told it to persist The experiment summaries and the models To add your files Which is like a network share That you can just write all your stuff to So when we actually load a cancer board That's where it's going to retrieve And looking at all the logs and all the summaries And this is why you're actually seeing All of the test results at the same time Inside a cancer board So you can actually see how each experiment's doing As they're going, as they're running And then all the code and the examples we talked about Are in the first repo From Rita ZH and Kate's dash ml And then also under kitab.com azure We have the kuflo labs That kind of takes you through Like a workshop kind of concept From how to do it with Docker To Kubernetes, to kuflo And then going through all the components And how you can do this one example at a time And then this is how you can create IGP-enabled clusters in azure Using azure community service Thank you But also, so this has all the code and all the Yeah, all that you need All the charts that you need to get all the This entire work demo to work And it also has a lot of different videos I know someone was asking for videos Of all of this end to end So check it out We have stickers How long stickers When would you choose to use the azure container interface Versus azure kubernetes service It seems like you'd have different types of flexibility With those two approaches So it's azure container instance So when you want to run a cluster Say for your organization Where you want to maintain the lifecycle of your cluster For a long time That's when you want to actually have the entire cluster Right And that's when you want to have AKS So azure kubernetes service Whereas azure container instance is more Like dealing with the container itself So it's more for people who just want to run their containers But they might not necessarily want to use anything fancy So I know someone mentioned like Istio Or you write your own CRD and you want to display there Say if I want to display Kubeflow I wouldn't be able to do it on azure container instance Because it's that itself Just runs that workflow on its own Without knowing anything else around it Whereas with Kubernetes If you can only run the CRDs and the services And like the pvcs If you are actually in a cluster itself Right So the question is with Virtua Kubella You can actually leverage both Right And exactly That's exactly why Virtua Kubella created Is so that you can leverage these container Runtime providers Meanwhile you can still like leverage all the goodies of kubernetes cluster itself That way you can manage it Manage all your applications Deploy it and manage the lifecycle security with kubernetes But you actually only like pay for the resources you need Like when you need it with Virtua Kubella Yeah so I'm going to say the same thing basically With Virtua Kubella you can burst out to Istio whenever you need to You don't have to have the necessary resources in your cluster already So it's like on demand basically You can like add more nodes to your kubernetes cluster with like more GPU nodes But then you'll have to scale down those nodes which takes more time and like So actually we forgot to talk about cluster autoscaler Right So cluster autoscaler allows you to scale out your actual nodes in the cluster Right But don't forget you still have to manage those nodes once they're created You have to patch them you have to like you know love them right But with ACI you don't have to You just have to schedule them and run them as the container is alive And then after that you don't have to ever worry about them Right But one thing that I also want to mention is persisted volume right So if you have applications where you might not want to like persist to a network share Like ACI wouldn't make sense right Because it doesn't have this concept of local volume right But if you have an application you do care about like local volume Local persist if you see then that's where kubernetes cluster makes more of it So it depends So but with vk you can have that best of both worlds Along the same lines as opposed to comparing nodes with a magic container What about magic container and magic function like k-native Well, how would you kind of could you kind of get some of the benefit with Like not having to run your own node with the function Isn't that kind of like a more granular way of doing this Comments so much about like k-native But I can talk about like what I do know Like just because I think that there the big difference is like running like running your code As the code itself versus like the life cycle of your container I think they are kind of different right And in this case you're actually treating your application like a container Not not just like I'm running some python code or I'm running some I don't know Like go code or something right So I think it's it's different in that it's more like I guess it's like kind of like Haroku world where you're kind of like a web package Like your your bill package right whereas now your your knowledge and your life cycle is in the container itself I'm not sure if I'm answering but it's a different way of doing Same thing I think it's just like how much freedom you have with your application because there's a lot of stuff you can do With functions and k-native so so I guess it just depends what you need Thanks for your answer tech And next up we have Pete McKinnon from Red Hat who's going to talk about How skill or how flexible coup flow is for deploying and scaling in multiple environments and specifically in this instance He's going to talk about I'm taking a local deploy environment and scaling it up to open ship Hello Welcome to this Slightly delayed presentation. I'd like to thank canonical for sabotaging red hat Thank you, Josh very clever I would like to thank canonical they were the drivers behind putting together coup flow data. So it's a great event. Thanks, Josh Let's see. Let me put this down Since this presentation is starting so late It's going to move pretty quickly And we're going to skip over some stuff. So by now you um and basically the presentation's talking about Specific feature of coup flow, which is its capability Or ability to be deployed into multiple environments and we'll look at what that means with respect to different environments. So You've seen lots of slides up to this point starting with this morning It should give you this sort of sense that It's actually pretty tricky to do To put together a machine learning platform when you consider all these different parts the model training You know the management of the data data provenance Building the model rolling that model out to production a lot of parts involved So when our heroes at coup flow like Jeremy started the project, they set out some sort of high level sort of goals for the project And that is portability scalability and composability Composability is really covered, you know in the sense of microservice architecture But this talk will focus on those first two things portability and scalability So the notion that you can go from the bare metal to the cloud with this machine learning platform open source platform And that it scales easily starting with a developer and their Their machine which is maybe a laptop or something like that. They can scale out the hundred Guess what surprise surprise kubernetes is a terrific enabler for this type of These types of goals, right? And today with kubernetes we have Sort of different dimensions of kubernetes in respect of there's on prem in terms of how you deploy it There's hybrid cloud, which is my company is very much Involved in and then there's the public cloud providers. So gpe, gaf, akf, azure ec2 all that stuff Then so that's one dimension of the problem and then the other part is the kubernetes distribution themselves So there's OpenShift, which my company Red Hat makes Rancher Docker has An implementation of kubernetes that you can install on your desktop So There's a couple of dimensions to this challenge And when it comes to the cloud providers, you can think of the different ways there is to authenticate to these environments And then as kubeflow sort of matures, there's also this notion of how do we sort of take that authentication? Ideally a one-time sso type authentication and apply that and manage it within the cluster itself In this case i'm talking about a kubeflow cluster Then there's considerations with respect to storage Google storage versus s3 local storage or block storage like sef nfs And then there's considerations around the hardware hardware resources, which are very important in machine learning platforms So particularly with GPUs There's different types of ways to install GPU capabilities for the different types of kubernetes providers Certainly my company with respect to rel Is very invested in se linux So doing that in that type of environment provides some or imposes some additional considerations as it were But nonetheless the community project the open source project who strives to kind of Put put all that to rest for the users and the operators So to make it easy as a user and an operator to have a consistent experience so By now you have a sense of the components involved So we sort of have this elastic notion of the core components what's filled out of kf cuddle If you do an install of say zero dot four There is the ambassador component for ingress control jupiter hub. We have an asterisk there Because that is undergoing an evolution. So in zero dot five There will be some significant changes happening in the notebook launch space There's catib which has been discussed for hyper parameter tuning kubeflow pipelines the training controller all that stuff And then you have these various other components So the span of components also provides various different considerations For how you deploy this into different environments starting with just a local dev environment and then scaling up into these cloud environments So uh when our heroes at google started they were looking around in terms of What could provide a capability for config management for this new project? And they settled upon a project called case in that So that's a cli tool that's still used today within kubeflow And it's used to basically specify and generate these kubernetes deployment objects. It's based on the jasonet language And it's more expressive than just using you know directly Json or yaml for defining your kubernetes components. So you have variables, conditionals, mixins You have the notion of prototypes where you can install various Different components from think of them as libraries that are reposed that could be hosted in github Fairly powerful constructs and When case in that generates those resources as they're defined Which you get out of that is entirely compatible with kubernetes Okay, so those objects that are services pods deployments all these things And you can directly edit those you can modify those The idea is that you would have sort of a if you will a source of truth for your application And that would be expressed by this config management. That's Uh managed or defined using case in that The other part of that was um And again sort of thinking of this multi environment sort of problem is that In the enterprise you would have sort of a stage rollout of a machine learning type offering for implementation So you might start out in a dev environment with certain specifications or capabilities test environment on the staging fraud the idea with case in that is that by a minimal specification of a variable or two That could greatly sort of define and impact how the application is actually Generated that is deployed into that specific kubernetes environment It also has the ability for continuous deployment and you could delete these resources by deleting The components themselves But Wah wah sad trombone um What happened at the end of last year is that dmware acquired heptio and heptio is a A very influential or was a very own influential company in the kubernetes space and um They made announcements early in february in a blog post that sort of made some declarations about What they were going to do with some of these open source projects that they started One of those was case in that and unfortunately the decision was made that case and that would be shuttered in terms of new development so What that has done is pushed our happy coup flow community to Kind of take a step back and think about decisions to make about How do we want to come back to this problem of config management for this application? so The key point here is that even with this announcement about case in that there's still Um, they're not underlying goal of the project and that hasn't changed is that it should still be very easy To deploy coup flow into these different types of environments That's still a key goal And also case and that provide an on ramp for the various integrations that were really key to the beginning of the project Going back a year ago. We had Selden Integrated into the project packaderm And the roadmap for that in terms of the integration was provided Via case and that so it's still important to have that type of capability So today the community is evaluating options. There's a lot of interest. I don't know if cams here, uh, jeremy But there's some interest in customized which is an open source project for yaml based type of configuration similar to what case and that provided There's capitan. There's helm two The folks at microsoft are just talking about helm and then there's it's antecedent Helm three so a couple options. We're actively looking at that and making decisions as a community about that So still sticking with case and that it's not dead yet It's still used in our community for the time being But when With the creation of this shell script that basically spawned a coop flow deployment for you You could basically specify these different types of platform setting And what would happen is that from that it would define a set of parameters that would be relevant for what you were doing So in the case of tf serving if you're storing your models in s3 There'd be parameters for zone region end point whether you want to verify ssl things like that the access Docker for desktop and mini cube to sort of developer type Implementations of kubernetes for that it'd be a definition for the local storage Alibaba has been very involved in coop flow and They're you know the chinese company they Won the early issues with alibaba as they realized that they couldn't get access to some of the Public images that were being curated and produced by the coop flow community So that's due to the great firewall. So there's Parameter changes in there so that it provides a different url To a mirror inside the great firewall for pulling those images And then gcp. There's the api version for gke zone email project the endpoint Google application credentials secret all this information So these are the types of parameters that's important to capture and Have sort of defined to make it easier for an operator to deploy coop flow into different environments so I don't know if we've made a decision about customize. It's certainly a frontrunner, but Customize interesting as an alternative and it's still sort of This notion of that we would have a common or a base set of definition and yaml and then those be processed against specific sort of overlays or overrides of Perhaps those base definitions or there would be enhancements to those definitions for that environment So mini shift kind of like mini cube. That's the open shift version of mini cube I'm going to talk about open shift in a second and then you could imagine that there's an overlay of gke type Parameters that would be processed and then used for deployment into google cloud Okay, so that's kind of where the community is at with basically this notion of Uh adopting a new customization or configuration tool So just a word about open shift. That's my company's uh kubernetes distribution. It's uh enterprise kubernetes distribution The design of open shift is really around enhancing the developer experience for developers using kubernetes and also Um for the operators Who are tasked with deployment into the enterprise environment? there's um higher levels of security around things like container uid not being able to run A a container that you know lacks a uid Or a ui uh container that's trying to bind to 480 or something like that And it's still you know, it's not a well known uid. So things like that there's controls within open shift for for managing that Um, it's very much self-service. There's developer tools for basically quickly generating applications And um underneath that it's still all the same kubernetes construct that you know and love So, you know pod services it all integrates seamlessly with kubernetes So There's a developer tool with open shift That is the open shift client as part of that tool it has A ability to fire up a developer kubernetes instance. And basically what it does is It pulls down an image and fires up a container which in itself is a self-contained kubernetes environment whereby you can quickly Deploy an application in this case the application we're interested in deploying is kubernetes itself This runs in developer environments. Um, I know it's heretical to say this a linux conference. This is actually running on my macbook pro or that's where the The video was stolen from so That's a 16 gig environment It's limited, but it can be useful for doing quick sort of evaluations of various kubernetes type applications Notably coop low and then there's the deployment of coop low itself So on our website with the getting started information It's uh, there's a fairly well laid out sort of recipe that involves the kf cuttle Shell script for deploying Into basically a bare metal or on-premise environment So that's what you're looking at here. Again. This is in a local developer environment It's running in this oc cluster up environment And it's the same instructions that you would follow basically anywhere to deploy coop low generally so to run coop low on open shift, um, this enterprise kubernetes distribution There's some considerations because of the extended resource concepts And the rback stuff There's these enhancements around things like namespace sub projects and there's permissions and constraints that are applied to the security Contacts around those things so We had read how I have had a lot of success success with open shift. That's been a popular kubernetes distribution the coop low community does most of its developments Using a general form of kubernetes and a lot of the development is done in gcp and gke so The other thing that's uh come on recently not just in coop low but in the kubernetes space is development of operators So coop low has developed its own set of operators and the ecosystem that we're integrating with is also making a strong use of these operators And operators typically consist of an implementation maybe in go lang of some type of capability that is managing a very specific application resource for you Then in the kubernetes In terms of the resource sense, there's custom resource definitions cluster roles and cluster role binding so, um These all have to a crd as you know if I have to be installed as a cluster admin But these can prevent or you know impose challenges in terms of Deploying coop low on the open shift So there's sort of a I create an issue up on the community and capture the friction log there And there's various sets of um sort of cluster admin type Capabilities that you have to apply to some of the components to get it running smoothly on open shift But having done all that Let's look at a little demo and it's not a live demo it is So that same application of coop low that you saw earlier in the other video there This is the exact same deployment. That's really the point. I'm trying to make here with coop low that This one has been deployed into a much more robust environment That environment is running rel 7.6 It has 64 giga ram 8 cpu, but more importantly it has an nvidia v100 GPU So this playing in a loop, but what it's showing you is that um using the same Um components that are specified from kf cuddle I can basically launch a jupyter hub instance specify one of the curated gpu TensorFlow images from the coop flow project and have it run successfully In this environment I'll say it again. It's all the same code whether I installed it in My macbook or installed it in this this larger environment that just so happens to have a very expensive gpu card in it So that's the beauty of the coop flow. It's something that the community really strives for in terms of A project goal that this is a very portable type of environment Especially with people who are just getting started with machine learning platforms that they can work comfortably And easily sort of in a local environment and then take that out to their to be their bigger game In these larger environments with more capability so back to this guy and It brings us to And you've heard this before From other people in the community. We love getting feedback and in Contacts of this particular presentation What we love hearing about is any issues or challenges that you encounter trying to use coop flow or deploy Into different types of environments. We'd like to try and have All-encompassing experience for coop flow in many different types of environments Whether it's on-prem, bare metal certain cloud environments So what helps us is providing us With um your support and the way of testing GitHub issues friction logs Full requests that you've found something that needs to be fixed Love to hear about it When you generate a GitHub issue it'd be great to have specific details So for example, you had a notebook pod that didn't launch and you discovered it was because of this and Then you need to do something to get it running in a particular cloud environment. Tell us about that Capture the information the logs to config and just redact it if you have to and Throw it over to us and let the community have at it in terms of Making it a better smoother coop flow experience for everyone And that's the abbreviated version of my talk I think Any questions? Good Thank you Thanks, pete. Um up next. Uh, we have my co-worker actually patina our who is New to coop flow. Um, but is going to talk about it in the context of a i use cases He's seeing in the field as a customer engineer at cuckold. Good afternoon Good afternoon So I have some stickers each unique From the friends from japan's to jeepley if you're familiar with your word Each unique and you will not see them ever again other than your laptop, but these are like five of them In order to get it you have to answer question. So I need some interaction here first All right as my friend introduced i'm new to the cube flow However, I am not new to the kubernetes nor did mlai So before I joined google I was part of verizon wireless as a distinguished member Before that I was in canonical ubuntu I did my serving to the sae whatever you call them now self-appointed dictator for life And then before that it was ericsson So through my life, I have worked in different disciplines of technology including voice over ip ip tv media backends infrastructure as a service Platform as a service software as a service now in google helping You developers and customers and enterprises to bring the workloads to the cloud It's easy to say but it's kind of hard to do it, isn't it? Especially if you're if you're in love with the technology But your counterpart on the business side is not seeing the light at the end of the tunnel as a return of investment It's kind of a challenging situation So let me stop there. What's the picture about is this is my personal car I got a big accident last weekend in delos I got t-boned literally And it was unfortunately unfortunately my mistake And I wasn't On my phone or anything was just one second. I guess destruction. I got t-boned And the funny thing is before I left verizon We were working on an edge computing cases and one of them was literally the red light detection and stop light detection And we did it We did it and we deployed some 5g sites and our deployment and still running on top of cubanities And that cubanities deployment The basic quality is redeployable without creating snowflakes iteratively So we can scale and bring workloads on top of it as much as we can without changing the workload But at the time it wasn't possible now with the qplo especially for mlai it's possible So other than this case what i'm going to present you today Go over, you know, you saw this chart a little bit different perspective is I like to talk about more about the use cases that you can go and advocate inside your organizations to get some funding Because at the end of the day Your job Is tied with the business This is the reality If you can bring a good use case for mlai, which is really hard Especially if you're doing unsupervised learning, you don't know what you're gonna get out of the unsupervised training Especially clustering which I will give you a couple of good examples from the field You may not get good sponsors All right, has anybody heard Two words in the same sentence walmart and pop tart anybody Walmart and pop tart Okay Don't wish we didn't see the news about it two years ago So walmart has has crunched big data a lot of big data in the last 15 years of Purchasing And they they did run the came in clustering And the result came interesting that at the particular times of the year where there's extreme weather conditions People are buying a lot of flashlights And also pop tart at first it didn't make sense Then they realized the human behavior to survive is have a light blanket and some comfort food So that drives the different shelving strategy in walmart If the weather predicted weather is pointing extreme conditions So when you go in a rainy or stormy or or hurricane season in particular walmart area, you will notice when you walk in the store blankets flashlights and pop tarts So this didn't come as as a predicted outlet They crunched big data with the terraform and terra data guys and and it took a while They spent a lot of money to learn that pop tart sells good in bad weather So they could do better if they had the cupelow at the time and also transferable Pipelines that in the sense of they can run anytime anywhere without restructuring it Distribute go ahead Okay, is this better? All right. Sorry. Is this better? Better? All right, sorry All right, the microphone's always the problem thing in the consensus All right One key challenge as everybody mentioned is if you're a data scientist or an LAI engineer Knowing all these has been the prerequisite, right? You need to know how to package your software how to distribute your software how to Build your infrastructure such as Kubernetes Collector and maintain it and scale it And and also deploy the stuff on top of it at the same time. It's a lot of work That's true. Yeah, and besides if you're looking into more Accelerated workload cases where your levered GPUs or TPUs it needs more expertise as well So what we did in my previous lives at the time OpenSec was popular before the Kubernetes stole title So the accelerated network applications required better throughput by means of latency and bandwidth And then the industry invented SRIOV That required different talent on the on the hardware specification side designing and drivers and also workload migration It has been a lot of challenge So what it did is is mainly It killed the sense of moving workload on the fly Even though if your infrastructure are failing or you're scaling your workload So what does kubernetes do for you much more than open stack not on not only giving you a place to run your application but also taking care of your your Large cycle of your application by means of replica sets or you can use demons or or jobs As well as as well as monitoring the health as a socket or the Whatever port you like to scan or you want to run as a job to see if my application is healthy On top of these abilities. We are putting as many of the folks mentioned kube flow to give you better Better a tool set to be more productive so edge use cases Everybody's talking about the 5g everybody's talking about the mobile is computing low latency computing The low latency is only possible in the sense of law of physics If you're close to the if to that place where the event is happening Right that is only possible if you're if you're Execution environment for your application is close to that event place So in in the sense of telecommunication everybody put saying that okay, we'll put some edge computing nodes Where the e-note these which is the radio stations are located? That's what verizon and AT&T has been doing with partnering with other telco vendors but we're what they are not thinking yet is mainly okay the The investment i'm doing on those edge i'm putting a couple of servers But they are not thinking a step further how i'm gonna Put an you know infrastructure that can be reused by others Without requiring additional a lot of administration So they first thought about using open stack then realized open stack is not really a way to go for portable workloads And then they start looking at more on the kubernetes and then open stack Changes name to be more agile and close to the kubernetes So at the end of the day what you are hearing from other organizations under the linux foundation Is mainly what you are seeing here is the base infrastructures is depending on kubernetes So you saw on the on the on the other presentations that you can do your ml ai app packaging on your on your developer laptop or a workstation And then push it to a cloud cloud for heavy lifting for training the algorithm because What makes sense is train your algorithm with a bigger data set merge data sets so one reality we saw in the field is Is Do you have a better insights That is valuable for your business or your partners You have to crunch a lot of data a lot of different data So if you think i have a one petabyte data and i'll have a great insights coming out of it May not be happening You have to think differently combine different data sets You have a data set for better forecast Combine with the sales from nearby retail stores if you're not in the sales Only predicting which areas in the neighborhood will going to be more popular for house sales Combine with the geographic topology information And you may say okay, I don't have those data right that's the reality and josh is sleeping right there realities You don't have any excuse Right now the all the political situations in the world is driving data to be opened by governments As well as organizations if you need a certain set of type of data There are already data brokers out there. They are selling data If your organization doesn't know what to do with the data you have You can consider selling your data through broker and be a partner Whoever using it as a kickback for a business So there are different businesses right now running in in the world for big data ml and ai You don't have to necessarily know that This data will be used for this particular business Maybe it may be or if you combine with other data you can purchase from a data broker Getter more different insights. So one of the use cases we saw in the field your partner is one of the retailers Which is very famous one of the seniors in jcp. You picked the name they they put a carpet that pretty much Senses the the footsteps the size of the footsteps time and recording and they saw that In the particular timeline of the week and particular time all the day There are different shopping behaviors After 6 p.m. They noticed that the family with the little kids running fast through the store to the Obviously the post-section And in the other time of the day special lunchtime The woman Behavioral is different than men by means of going which department and purchase something So they cross the data and they come up with that. Okay. How can we restructure the our store dynamically and fast? That we can get more business out of Driven by the data So today one of the key key conversations about all right Good to talk about, you know, running the workflows at the edge, but edge is not ready it's it's through and not through And the one example was given was the raspberry pi obviously raspberry pi doesn't have the enough muscle power to run the inference or prediction at the edge, but now google has, you know has announced the Edge tpu last year now it's available for purchase and this is the purchase set that you can go buy around 149 dollars And you can see the you know test benchmarks the the left most column show showing that basically the There are a lot more forward with the edge tpu versus versus The desktop cpu with the eight core.  Dude, you're going so fast and there's a red light right there It's not a feature. It's right today. It's doable And this hardware is already available for your purchase and consumption to deliver such use cases all right So one of the things I have learned through my life is community is better than the companies This is through companies has their own focus for business has their own agenda Despite they are leveraging the leveraging the same technology, but the community has higher purposes the reason opens that grow a lot because of the Contribution from the larger larger community and hopefully q-flow will grow as fast as big as possible to deliver ready to use pipelines Even more than that ready to use as a base Pipelines that you can put an additional component on top to deliver different experiences All right another question Another question today when you deploy A Kubernetes. I know this is kind of a DevOps question sort of for that We have masters and workers right master nodes and worker nodes Before we call them workers. We used to call them something else No Minions Sticker for you here, right if you like to get it It was kind of a historical question All right another question Another question another question When you deploy A new additional feature on top of Kubernetes such as Such as Istio It's not necessarily a regular pod But it's something else we call What do we call for example envoy or Istio on a on a worker node? Yes, sidecar For you if you want to get it to So the reason I'm asking these silly questions is mainly Kubernetes is open to open to different way of delivering different feature sets It could be demons. It could be sidecars. It could be pods And Q flow is is leveraging that heritage So your business may need You to deliver something brand new Or your business or your partner In the crime needs Something that's already out there to use as an api for example google quad has a vision api You can use vision api to detect objects and label them You can use speech api To mainly from speech to text text to speech conversion Or if you need something a little bit different you can train your algorithm on the ml engine and export it if you want to do something Definitely brand new on the edge by means of either training or just running interference or or a prediction Where you're going to see the benefit of q-flow It's not necessarily these are all separate worlds. It's maybe you can combine them in in a sense that Get best price performance for your spending and what you're getting out of it crunch the data train your algorithm on ml engine Leverage the tpu's leverage the printable vm's with the minimum bucks Export it and run your edge with the run the verify with the htp accelerator Much much less than what you will spend buying server with gpu's maintaining them electricity calling Even securing those servers all right This is just in a self-advertisement just skipping it It's many folks talk about these already In pipelines what I'm going to talk a little bit fast forward is there was a question today. I think it was a user right about the examples right Yeah, let's go them there All right a little bit Here we go The demo that I'm going to talk about is mainly from the q-flow github under examples So we took I took a little bit one step further to give you more easy to grasp and deploy by yourself I know the projector resolution is not good, but let me tell you what we will be you will be saying shortly First we're going to package our application on our laptop or inside the machine that you are using as a jump host Mainly what we are doing is building a docker container. Okay inside the docker container. We have a machine learning algorithm Then we're going to push that container to the google cloud container repository gcr And then at the same time we will deploy a keep small q-flow Cluster which you you will see very easy to do through even ui or cli And then we're going to train the algorithm on the google google Kubernetes engine gkey based q-flow cluster Once the training done we're going to export the model into our object storage google cloud storage And then Imagine this is running at the edge this cluster q-flow cluster running at the edge We will simply push that model from gcs into the edge Infrastructure where the q-flow is running All right, and then we're going to run the prediction in here As a sample of but we will deploy one web app front end to kind of a run sample inference Test cases So the first step is mainly deploying your q q-flow if you go to deploy that q q-flow that cloud You will see a ui like this and on the on the left of it You will see put your project id and deployment name Project id is mainly is the project that you created on google cloud platform So the first precondition is Go with your gmail address log into the google cloud console If this is your first time logging in you'll see you are given a 300 dollars of credit Which is good enough if you ask me and I experienced that through when I wasn't part of the google Good enough for a sample ml ai or even big data analysis workload So what I did before I go on google? I collected the weather forecast from the open data initiative in dallas as well as the ground humidity for agriculture I pushed it pushed it into the into bcury From bcury. I did run Mainly ml ai and that cost me around 100 dollars believe it or not and that data set was around 100 terabyte So once you created your project Put the name in there. What it will do is mainly let me take Let's create another q-flow clusters as a sample in here My project name project name is you can see it in the home button in here Yeah, the project ID is q-flow demo 2019 Oh la Let's go back And look at our deployment Status so what it will do is It will go create a deployment set first And show your deployment in here as you see q-flow la is getting deployed So there are different types of tools that you will see out there from aws as you're in us and also on top of there here On top of there are other umbrella solutions such as terraform and so on and so forth. We have our own Automated way of deploying things we call it cloud deployment model So what is that the ui triggered is automated way of deploying a cluster? What right now is running It will trigger first creation of the virtual machines and on top of the virtual machines will deploy the kubernetes And on top of the kubernetes will install automatically the q-flow. That's the first step and it's only one click through a ui So you may say I will not remember that I'll go to All the references in here. So if I go and open you this github repo quick Which is this page This page is literally I put down all the steps that we are running through with all the details Even though the code is literally under the q-flow example For optical character recognition through mlai all the steps described in here with the with the cli command and everything If you see an error feel free to drop me a message so I can convert it And if if there once I can add this read me in the second read me under the example if you want So after after the cluster is deployed obviously you can go and do the you know get the details of the of the All the cluster for the q-flow namespace the cube cut all get pods namespace equals Here we go. So these are the pods that are deployed automatically under the q-flow namespace that that are delivering the q-flow components in here, right? So the following step, okay Our our q-flow is up and running the package our docker through the steps I have provided next step is running the running the running the ml training and then exporting into exporting the gcs bucket who has Here never touched docker before Everybody knows how to build a container out of a manifest. I assume right it's pretty straightforward who has recently pushed a container into docker repo or gcr All right, so the question here is Imagine your partner in crime says that hey, this is our idea. We cannot put on a public repo, right Man, you cannot push your container in a public repo. They will steal it. They will download it. So what would you do? So the answer could be either Hey, I'll get a managed service from docker or gcr for a private repo, right? or or Set up your own exactly My rocket size But the problem is setting your own is coming with your responsibility What if that machine crashes? But if you have a hard disk failure, you lost the container, right? So that's that's where the managed service comes with the beauty of hey They provide you sla by means of service level agreement whatever container you push out there even it's a private repo. It will be there Since you're paying for it Even the back end crashes. There's a replication coming to a picture that will that will help you continue your business Plus delivery So when you push a container into gcr You can pull it from the edge computing sites with a much better throughput and latency than those edge sites connecting to your headquarters God forbid if you're using a vpm for remote connectivity and so on and so forth. That's another another bottleneck All right So in the previous redhead presentation our friend mentioned and in this demo We are also using casonet as well for mainly for configuration management to to leverage casonet for different Contextual switches So what do you see here is mainly is where we build the first container and then as you'll see pushing Into the path we are given above which is the gcr path And then training based on on the gq cluster that we just deployed as a q-flow Cluster and casonet is mainly giving where to download the Container and running on table on table mainly the Kubernetes cluster And the last step is is is step number eight Where q-flow running as the edge at the edge side where predictions is happening again the same Same path used for gcr.io for downloading the container and applying the config model through casonet And then running your running running the running the inference Obviously when you run a predictions at the edge easy way to see a demo purpose is to build a web front end Which we did in here. Let me show you get to This is the web ui that's running as a docker container on the same edge q-flow kubernetes cluster this is the This is the ui ip address If i go down there Let's see This side is connecting to the Prediction or inference Serving right now on top of q-flow and giving you an image that will translate into it into a character This one says it's most likely in Number one, I guess let's try this again It says six and so on and so forth. So in this demo we leveraged the existing data set from mless and use ML algorithm inside the docker container push to the gcr download it for machine learning training purpose on q-flow Exported the trained algorithm into google storage bucket and then pull it into Again q-flow for for inference. So that's the long story Short that I will give you today Any questions so far Did I make you sleep worse than before? Sure, which step? Oh, okay, sure For this example, let me open the code. Yeah, sure So first, let me take that question so I can show the code quickly. So the good thing the beautiful thing about the About the gcp you can directly use your source repo On there the source supporters in here as well as your container registry And code is and I just switch sync with my github in here, right? If I go and let's take sample in here This is the model in here I'm not sure if you can see it This is the model that we package inside the docker and then we run the training on top of it Can you see it? Okay So this repo so google cloud repository is per project base You can sync with your github two ways and also do the code changes in here automatically trigger the rcrcd Part 1 building the container on the fly and also deploying through through spinnaker So back to the other question going back to which step you said i'm sorry Oh, you're all followed up. Oh, okay I hope this made a little bit of sense By means of not only using the tools for the sake of using it But generating something valuable for an idea becoming a reality Idea becoming reality and reality becoming a business income for you As I said at the end of the day technology comes and goes the main thing is what you're getting out of it as an output Today as you saw kasa that is you know No longer will be the obvious choice for us because of the acquisition of tap to you Something else will be replacing it and later. We'll see other tools and come in and go and mainly If you have an idea If these tools are making your life easier use it if you need to support Leverage the community. That's the all only message I can give you as an experience Community is a two-way road by the way You will not only getting stuck in knowledge and data out of it You may you need to think that hey, I'm part of a family I have to contribute back because I see in my experience that some organizations out there I don't want to name those service providers Getting to open source projects bundling as a business proposal But not delivering anything anything back to community. This is this is not the way. This is so selfish Contribute back, please This is how we how we grow as a knowledge as a as a human organization All right, that's it Stickers are here for the people who likes to Torah Come on and take it if you want. I'll give you one. How about that? All right next up we have a workshop on using model driven architectures To deploy who blow at scale um from Kenneth Koski who works at canonical Kenneth are you here? Yes, excellent as an implementation of kubernetes persistent This is the end-to-end story so This is what we do planet scale data management We can snapshot and version and package and distribute and appear to clear network your data So you can be working locally You can be producing versions of your data You put things in a file system You snapshot it in a notebook and then you start machine learning pipelines from this snapshot This is what we do today and then you distribute these versions of so This is a pipeline. You've heard lots about pipelines today, right? Pipelines are big in the machine learning space and couple give your standard way of running your pipeline Instead of having these pipelines access a data lake directly You have them access Us as your data management player You have them access A local disk a local directory slash data that we manage And that we can then Snapshot and clone for your steps So each step of your pipeline works on a different clone of the data And you can have full reproducibility of how your pipeline runs For each step of the pipeline You can essentially go in and say what despite did the steps start from and what did the steps produce And then you can mount all of this in a notebook and start over This is the idea that it is notebooks to experiment You store your data in persistent volumes You create pipelines from these persistent volumes, and then you can continue experimenting and iterating So this is too much talk, but Let's see what This has been doing so They grant up Okay, it came up Inserted a new interface to Welcome to me in Qtflow run they grant SSH to get started. Okay. I'll do that So at this point This is a full Qtflow deployment on kubernetes So this is kubernetes and you can use those kubernetes commands To a manager to interact with it Okay, welcome to me in Qtflow that me in Qtflow to ensure everything is up and running.  So mini Qtflow is mini Qtflow A small deployment of kubernetes Kupflow and rock our software for data management Okay So at this point the script essentially Declare what you need to be running And then sure to create it by the pipeline into a new notebook So be easy on notebook to explore what happens. So again, I found the notebook But this time it's gonna be seeded with data Again, I can go back into any Qtflow And see that My path has been created Okay, I can actually watch And this was just short cut for this right? Okay, it's running It became running like 29 seconds Jupiter hub is natural flavoring detecting when the path is up. So I can just just try instead of giving it more time So my notebook is up My snapshot is here and it already has file space It contains the state of the pipeline step exactly as it was So step two was expected Let me see the phone so you can actually see So step two was expected to run At this state I have all three files I have the gzip file that step one created And they have the unzip file that step two created So I can actually see What happens So I've got I can have full introspection using notebooks in the result of my pipeline This is it And then why is this important? Because I can split pipelines and run them in a hybrid fashion A cross-portation or I can recreate pipelines But we're interested and produce exciting results And why do we want to do that? So you can have development training and deployment happening in distinct rotations being driven by distinct people using distinct platforms So you're running mini-cube flow on your laptop exactly as it is now You're deploying over your local You're developing and deploying over your local disk Using the exact same API You use pipelines, you use notebooks You can just copy locally You don't have to go to another location to do that And then when we want to train at scale We can move to the google file for example and get two or three gpu enabled in Google And when we're done we can snapshot the results of our training the model And distribute to our fleet of self-guarding cars Which will be running in front at scale And all of this happened with kubernetes and kubflow running on each individual location And we don't have to change our code But we haven't visited it like 20 times today so this has become more or less common knowledge This is it And the way we work is We appear as a CSI to again under kubernetes And we make kubflow data aware by contributing code To have all kubflow components access persistent volume Okay, I'll skip the technical part If you want we can go over them but Let's have an idea of these two technicals Thank you We have a mini kubflow channel In kubflow slack Please download mini kubflow, try it out Let us know how it goes, let us know The description, if there's anything that you don't If you could be improved, if you have suggestions We're really looking forward to your suggestions and contributions Thank you Any questions we may have? I've got these very nice Microphone here Thank you Thank you evangelist, that's super awesome That's very exciting to have launched something today So thanks all of you For coming and thank you For the people who have stayed until like the bitter end here We really appreciate your enthusiasm And so I guess the takeaway here We've seen so many talk about Different use cases for kubflow But the key is We would like you to try it We would like you to send us your friction logs And your bugs, we would like you to try All the different integrations that exist in the community To work with your cloud provider Or Your data Your streaming learning workflow Sorry, it's getting to be that time And one last thing That would be super helpful for us To figure out what kinds of content Would be most useful for us to bring To community conferences like this in the future Would be to respond to an attendee survey That Josh just put together on the fly For you all So if you could visit the link I think that Ilan will also distribute it By email to all the people who have registered For this particular summit But yeah Let us know what you think For all years In case you were wondering We finished up early today I think one of the workshops Became much shorter than expected But I don't think we've Had many breaks today So I for one am not Sorry If you guys are expecting the session at five We did it at 4.30 And it's already done You're more than happy to welcome this day But Yeah, the guys are here We have questions just in the second row here Otherwise that demo has already been done Can you hear me? Okay So thanks for your interest In watching this demo But I have to undo The results of the previous demo So it will give me two seconds So it's all real because Some of the steps fail, right? That's cool Okay Destroyed everything So this is where we start from This is a presentation On Mini Kubeflow Mini Kubeflow is A product that we announced today Essentially a package Version of Kubeflow So that you can run it On your local machine And get onboarded and be able to use Kubeflow As fast as you can So are you familiar With what Kubeflow is? Yes, you're on Kubeflow day Kubeflow is a way to containerize And melt components for Kubernetes I'll be talking more about this Later on because this can take 5 or 10 minutes Which is very short So the fastest way to run Kubeflow Okay What's the story? The story is run These two commands Wait for a few minutes and be done So this is what I'll demo today So You're running You go into a command prompt You create a directory You switch into this directory You use Vagrant Which is a tool to manage virtual machines To initialize it Based on a box that we produce We call it Mini Kubeflow Okay And then you ask Vagrant To bring up the machine This is it So at this point The about 8 gigabytes That this box requires Because it's Kubeflow Is a lot of code Vagrant is extracting this code And then we'll run A short provisioning script So you can try it out So are you familiar With Mini Kube for example Mini Kube is a very easy way To run Kubernetes on your local machine And start experimenting with Kubernetes APIs Why is this important? Because you can have a full Kubernetes deployment And you can use the exact same APIs You can deploy your machine Your workloads In the exact same way you would On the cloud So Mini Kubeflow is Mini Kube Plus Kubeflow on this Mini Kube Plus ROK Arectos Around data management software So you have a full deployment Of Kubeflow On Kubernetes With the data management software That you can use locally in a few minutes And this is now Uploaded to the internet And you can go and try it out right now I mean don't do it right now Because A I'd like to be paying attention to me And B the network you may not be as fast To download 8 gigabytes of data So this is progressing And it's copying lots of stuff on my disk So I'll switch to the presentation And talk a bit more about Kubeflow So what is Kubeflow? This is a Figure from Google's TFX paper TFX is Google's internal Machine learning infrastructure So Google is essentially Open sourcing All of these components To build a machine learning pipeline You need lots of components that have to do With how you manage Your Different steps Of building ML models Okay All of these components have been containerized And have been open sourced And Kubeflow is containerizing These components So they run as components on Kubernetes So Kubeflow is a project to run Machine learning workflows On Kubernetes So this part Of the effects Is essentially the focus of Kubeflow Okay And then you need An illustration and configuration framework And Kubeflow uses Kubernetes For this So Kubeflow orchestrates These components As pods On Kubernetes Okay And then Kubeflow provides An integrated frontend A user interface So you can manage these components And this is what we are contributing to Kubeflow A user interface to manage Kubeflow's notebooks Most data scientists Begin with notebooks So we are contributing Kubeflow To manage notebooks easily and efficiently And then There's these Two big white boxes At the bottom That are garbage collection Data access controls And pipeline storage So where do you store your data And this is what our software brings For the ML pipelines They omit the most important part How do you handle your data And you can always Go to an external data lake To be fetching your data And uploading it back to the data lake But this is not efficient This doesn't work if you want reproducibility If you want performance If you want to work in a hybrid Fashion So we combine Kubeflow Our improvements Are those user interface Which we contribute as open source to Kubeflow We are a core of Kubeflow contributors And our software For data management Into a whole And this is what Mini Kubeflow is For a local deployment Okay So what can you do then Our business is Planet scale data management We allow you to take your data A local file system Or your pods that you are Manipulating from within your notebook And we allow you to snapshot it Take multiple points In time snapshots One every five minutes for example We allow you to version these snapshots And pack up them This is similar to what Git does for your code We are essentially a Git for your data And then we can distribute These snapshots And you can share these snapshots In a completely decentralized way Over a peer to peer network If ten developers are sitting here And producing data snapshots They exchange these snapshots In a secure way Over local links You don't have to upload something to the cloud To bring it back Makes sense So before I move on Let's see what Mini Kubeflow is doing It's done Okay and we started like three minutes ago Okay run vagrant SSH Get started Great Let me move to the top of the screen Okay And then there was a welcome Message that I cleared that said Run the mini kf command Okay I'll run it And this is essentially our provisioning script Which will make sure that everything is up and running So I can start using Kubeflow within minutes Okay I'll run the provisioning script I'll accept the end user license agreement I'm a very Fast reader actually And now the provisioning script Runs And ensures everything is up and running So the provisioning script Will deploy Mini Kube Will deploy Docker Will deploy all of the different Kubeflow components Will make sure they are up and running And eventually It will make sure our software is up and running And then We can demo Using the full set of Kubeflow APIs Over this local deployment So let's give it some more time It taxes the network To make sure there are no new versions And the network queue is not super good So there might be a bit of a latency This happened In the first demo as well This is a live, right? It's happening right now So what you're seeing is actually what's going on right now And because we don't like Silent Periods I'll switch here And show you how I can log into Mini Kubeflow And essentially use it As a local Kubeflow Kubernetes deployment It's still running Okay The network working Yes, it is So let's give it some more time What is it doing? It's updating stuff Okay, why is it updating stuff? Because it's the foreign daughter Our update seems to be Not progressing And I'll blame the network for this But anyway See, I can always blame the network and it works I didn't do anything and just give it some more time And then we provision in Mini Kube And then we're making sure Kubernetes is up and running And I can actually use this As a Kubernetes deployment So I can go and say Tell me, what pods are up and running right now These are the pods that are up and running And the software is initializing itself As Kubernetes components But you don't need to know this To run this, you can just be sitting here And watching the provisioning script run And what the provisioning script does is It makes sure that All of the distinct Components are in place before it says Okay, you can use this So it makes sure to pre-download And notebook emas That you can use to spin up your notebooks quickly It makes sure to ensure that the components Are up and running And when they're up and running And it's been like what, 5 minutes After the machine came up So within 10 minutes total You can have your own Kubeflow running locally So let's give it some more time And talk about What will be demoing after that This is a machine learning pipeline It's three steps for example At each step You want to be Fetching some data, manipulating it And then producing data for the next step So if you use A data lake A big S3 deployment somewhere What most people do is they Download from this lake Because they need the data to be local Otherwise performance is super bad Super bad, sorry And then they upload the data Back to the data lake So your performance Depends on where your data is And this brings your computation Close to your data You cannot be running somewhere And having your data somewhere else This will be super slow Using GAL approach allows you to Essentially Use us as the local Data management layer At each step Uses a local disk To manipulate data And we snapshot The disks after each step We give you APIs in the pipeline To manipulate this So you can have a snapshot Of whatever each step Produced And this allows you the pipeline To be traceable And reproducible There is no problem in running A machine learning pipeline Or any automated pipeline in general If something doesn't do what we Expect it to do There is no way to actually Go inside the step and see What happened, what it did So what we allow you to do is Oh, this step failed Okay, this is where it Started from This is the output it produced I already know its code So let me trace Each step exactly This is what you can do If you run our software That's our end goal So what is our end goal with Kupflow? It's having an end-to-end story That starts From multi-user notebooks Multiple users Working on the same Kupflow deployment Each spinning up their own notebooks Sharing their data Spinning up pipelines From within their notebooks And then creating new notebooks To explore pipeline steps So what could this final result Of the pipeline be? A model, a trained model So then you can have a serving pipeline That starts from this model And serves So why is this important? Because you can have A training pipeline here Producing the trained model here And then you can have a serving pipeline Running somewhere else That starts From The trained model as an input To serve So you can split A full pipeline In individual steps That run in different locations Because you can very efficiently get Output from one step running As input for another step And you can also recreate the pipeline Because if you can recreate its input And it's code You can recreate it fully So let's get back To this, it's up and running How much time passed? 10, 15 minutes at most Okay So I'll go here I'll go to the URL And this is Kupflow You deploy it Kupflow and you have it up and running locally So you can go explore Kupflow APIs So I'll do just that I'll create a notebook Create a pipeline Then create a notebook again To explore one step in the pipeline So I'll create a notebook This is the This is the pipeline's component of Kupflow This is the notebook components This is a release 0.4 We have contributed a new user interface For a release 0.5 Okay So I log in I'm creating a new notebook From this image This notebook will have this much CPU and this much memory And It will have A new volume As my workspace This is where I'll be storing my Python libraries for example And I'll create a data volume That I will mount Under pipeline data Right in size And this is where I'll leave my data As I work in the notebook So then I can start a pipeline From this data So I spawn the notebook At this point Kupflow contacts Kubernetes And asks it to create a new pod That will be used as my notebook server So I can Go back To mini Kupflow and query the pods And see that My container is being created 19 seconds into being created 25 seconds Running Within 27 seconds So it took me 30 seconds To get my container, my notebook up and running I go back here Jupyter Hub hasn't actually realized The pod is up and running So I'll just refresh The new user interface pulls more frequently So it comes up even faster My notebook is up and running You deployed Kupflow within minutes You deployed a notebook On Kupflow and you can start using it So I have The place where I'll leave my data It's empty, pd2 I'll spin up a terminal And I'll create a very complicated data set Which will consist of three files But This is a proof of concept for Any kind of data processing, right? At this point I could be going to my data lake And fetching in a few gigabytes of data This is a local volume So it doesn't cost To move 100 gigabytes of data From one step to another It's just there Another step can come and access the data Instead of having to upload 100 gigabytes of data to history And then bring it back And paying the associated cost So this is where I'll be Leaving my data, pd2 I'll create Let's say Time is 0132 UTC I'll create a few files here Okay I'll create three files Because this is the kind of pipeline That I'll run later on So I have created three files Okay This is the current state of my notebook Three files inside This directory This is my data set And now what I'll do is I'll access our software On mini-cubeflow We call it rock And I'll take a snapshot of this notebook I'll essentially Create a starting point For a pipeline from this notebook So I log in This is an empty bucket We organize Data sets in buckets Okay I'll bring in A full Jupyter lab My notebook server Okay It auto completes That this is the one that I'm currently running And at this point I'm essentially creating a data commit So this is why This looks like it I'll be committing My seed commit For a pipeline For a cubeflow pipeline Okay The message will be Create three files Seen the pipeline Okay And at this point This creates an asynchronous task My notebook is still running That snapshots my notebook So rock can be creating Multiple point in time snapshots Of my notebook as I work I can then Use these snapshots As starting points For pipelines So this is the task It's almost done Everything happens thinly So if I take another snapshot After a few seconds It will take 10 seconds It's done Okay And now My snapshot is done This is my snapshot I snapshot the Jupyter user I snapshot My data volume And my workspace volume So I'll keep The URL To the snapshot of my data volume Because this is where I'll spin A cubeflow pipeline from I copied the URL here Okay So now let's See this pipeline This is another cubeflow API Pipelines So this is This is the definition Of my pipeline In Python So cubeflow pipelines defines A Python based domain Specific language To define your machine learning pipeline So my pipeline Is a Python function It starts from a rock URL The one I just copied And it says Create a new volume As a clone Snapshot this parameter First step Is to mount This volume as SlashData And concatenate all files Into a gzip file So first step is gzip everything Into pull.gz Then Snapshot the results Of step one So I can retrace what step one did Seenlessly Is clone this snapshot Into volume two This happens thinly If this was a 100 gigabyte volume I don't have to copy 100 gigabytes Step two is Unzip this data Pull.gz And create data slash pull That's what gnzip will do Okay, and use volume two As your data SlashData And take another snapshot Step three is start From a clone of this snapshot And just show me What's in data slash pull What do you expect to be SlashData slash pull The concatenation of all three files Okay So I'll compile This pipeline using the Kubeflow Pipelines compiler And compiling the pipeline Produces An Argo workflow And then submit to Kubeflow Pipelines So this is the compiled result A big YAML file That you don't really have to see You are seeing nice idiomatic Python But this is the end result A big YAML file Okay And then I can log into Kubeflow Pipelines Which is here And I'll upload A new pipeline The one I just compiled Example five Okay And I'll Do I have to create a new experiment? I don't remember Let me create a new run Yes, I do have to create a new experiment So I'll create a new experiment first Call it Experiment one And now I'll create a new run Of this pipeline Okay Run name is gonna be run Five And the pipeline needs a Roke URL to start from It needs the seed Snapshot So I'll go into Roke Get this link And give it two pipelines Why is this important? Because I can run multiple pipelines In parallel with different parameters From the same snapshot Why do that? Because I'm an ML I can run the same model From the same input data With ten different values For a hyperparameter So this is how I do it, programmatically I give it the same input data I give it a different value For each parameter And they run the same pipeline ten times And Kubeflow has components That do it automatically And we are extending these components To work over persistent volume So creating the pipeline run And what this will do is Will essentially show me The pipeline graph being created So initially We create a volume This kind of pink-red color And the volume has been created So this is a persistent volume On Kubernetes Managed by our software And now step one is running Okay, let me refresh Step one is still running Step one run Snapshot one is being created Again as a Kubernetes resource And because this is all Kubernetes I can go into Minikup and ask Give me a list of persistent volumes Okay This one, this volume Is volume one It's this run of this pipeline It's bound And managed by rock Okay, give me a list Of snapshots This is the first snapshot It's being created Okay Give me another list of volumes Now there's a new volume Why? Because the pipeline is running as we speak So if I switch to the pipeline's UI And refresh Volume two was created Step two run Snapshot two is being created All automatically Okay, give me a list of snapshots Snapshot two is there 17 seconds ago Okay, let me refresh The UI however does not refresh that well So it will refresh eventually Snapshot two is done Volume three has been created Step three is running Same thing, another volume would have been created So we orchestrate The creation of Kubernetes resources From pipelines This is the easiest way To run full Kubeflow pipeline And finally, step three is done I can go into each step And look at input and output I can look Into logs And the logs of step three Is the concatenation of the three files Because the definition of step three was To cap The slash data Slash full file So this proves that all of the steps run But let's say that Step two had a problem Okay, step two Is behaved Okay, I'd like to see Exactly what its output was Snapshot two Maybe I also want to see What its input was, snap one So what I'll do is I'll destroy my notebook I'll stop the server Jupyter Hub Allows us to run a single notebook Here running multiple notebooks is not very easy But the new user interface Of 0.5 will show multiple Notebooks at once So I wouldn't have to actually destroy my notebook here Okay So what I'll do is I'll spawn a new notebook And I'll attach snapshot one And snapshot two Into this So I can see what step two did So I'll be creating a new notebook Okay This notebook Will Be empty But I'll attach An existing Snapshot which will be step one So I go here I see that rock has picked up The two snapshots So I'll attach Snapshot one Us A mount point And I'll also attach Snapshot two As another This is the old Jupyter Lab And it's dead of course But you close it And I'll attach snapshot two Did I copy it correctly? I don't remember, let me try it again I'll attach snapshot two Here Okay So I'm creating a notebook To explore the execution Of the pipeline This is full insight into how the pipeline runs Okay Creating the notebook Again underneath I can see that my Notebook is being created As a Kubernetes pod Okay Will give it 20, 30 seconds And within 30 seconds I'll be able to explore The results of the pipeline That are immutable and reproducible Because they are snapshots Okay, it's running Within 28 seconds This is fast This is really fast I'll switch to the user interface Jupyter Hub is not that fast On the other hand Okay And what do we have here? We have a notebook, a new notebook That already has Snapshot one and snapshot two Mounted Snapshot one has contents File one, file two, file three Full.gz No.ganzipped file Because step two hadn't run yet This is the input of step two Snapshot two has File one, file two, file three Full.gz and the unzipped file Why? Because this is after the execution of step two So I can have full insight Into my pipeline I can go and see Exactly what has changed Before and after The execution of step two This is super important When you're debugging the pipeline So this is snapshot one And this is snapshot two My own local clones This is it So what we did is We got Kubeflow running Locally within 10 minutes We used Kubeflow to spin up Our notebook within 30 seconds We compiled A pipeline that uses Local volumes for data exchange We ran this pipeline Starting from a snapshot Of a notebook And then when this pipeline Was behaved or broke We were able to attach a new notebook Into clones Of snapshots that happened For full introspection That's it Any questions you may have? Yes, please Let me bring you the mic Thank you for the demo It was very impressive actually Is there a place where We could find documentation About MIDI KF? There is not much documentation On mini Kubeflow But there is lots of nicely written documentation For Kubeflow itself So mini Kubeflow is just A packaging of Kubeflow That's meant to run locally So we have tuned Kubeflow So it runs easily And fast on your local laptop But if you go to What about the vagrant command That you shared on your first slide Are you going to share that somewhere? Yeah, we have a Yes, I can show all of this So There is a blog A blog entry on the Kubeflow blog That describes mini Kubeflow And then it points to our website Which is this And you can find The network is Well You can find Both where to get support We have a channel On Kubeflow Slack And this is the installation guide Essentially you need quite some RAM And then And then It's running vagrant Virtual box and running this to command This is it So if you go To Kubeflow's blog This is the blog You should find this It's up and running And this should point to Whatever you need Got it, thank you But I should have a final Yeah Presentation slide for this as well So let me Oh, I missed it, sorry This is it Yeah, we're at Channel Hash Mini KF On Kubeflow And if you go to the Kubeflow site We'll find the Kubeflow blog and everything Any other questions? Thank you