I know it's hard to pay attention in the evening. So say it again. Good evening. Good evening. OK. That's better. Before I start, let me ask you, how many of your data scientists here? Raise your hands. One, two, three, four. That's like one fourth of you. How many of your developers, software architects, DevOps engineers testing? So this talk is about building tools for DevOps for data science.  How many of your developers built some kind of a tool or a system for data scientists to use? Raise your hands. A few of them. So this talk comes from my experience of building a platform like that and also seeing other building platforms like that. Some of them are good. Some of them are things which could have not went so well. So there are things about how to do things, how not to do things, et cetera. So I'm going to share my experiences in that. My name is Anand. I'm co-founder of a startup called Orade data. It's a past platform for data scientists. Last two years, I've been building this platform for building and deploying machine learning applications to enable data scientists to build and deploy machine learning applications completely on their own without having to depend on other folks. So I also run advanced programming courses. People at Academy also do courses on machine learning. So I get to kind of speak to people who are trying to use machine learning and observe what kind of things are working, what kind of things are not working, et cetera. So let's look at the landscape of DevOps for data science. So if you look at the left, you have these DevOps tools. On the right, you have data science tools.  If you look, it's a fairly complicated landscape there. And if you look at the right, we have these Jupyter Notebooks, NumPy, Pandas for data analysis, and machine learning algorithms, and a bunch of deep learning libraries, et cetera. So if each one of them is kind of fairly complicated and airflow, et cetera, for managing your data pipelines. So when you want to do data science and production, you have to kind of manage both of them together. It's kind of a pretty complicated task. So managing data science and production is really hard, because it's a very angry discipline. So the practices are still evolving. They're not really mature yet. So tools and practices are keep changing, and people don't have, I mean, it's not really proven for everyone kind of thing, right? Whereas if you want to build a web application, everyone kind of follows the same thing. But if you take something in data science, everyone kind of invents their own way of doing things. The problem with that is building own solutions is not that easy. It requires very careful system architecture and very complex DevOps. Very often, the people who are developing it will only have a very short-sighted view of the problem at hand. So they don't really think for a very long term, which kind of makes things not so much what the requirements that they have. And also, sometimes the organization may not even have the capabilities to kind of build those kind of systems. Before we kind of jump in there, let's kind of see what's our goal, what we're trying to achieve. The data science team should be self-sufficient to build their own machine learning applications. What it means is if there is a communication between two different teams, there'll be a lot of latency. There's a lot of lead time. You have to ask someone to do something. I want to work on a bunch of GPUs. Can you please allocate the GPUs for me? I mean, it kind of takes a lot of time. You want to work on a GPU or some kind of compute thing. You should be able to allocate it yourself and then close it when it's done. So having to depend on someone else, some other team, to do some of those tasks kind of slows things down. Also, the other important thing is this learning curve should not be very steep. Why? Let's say a new person joins your team. How long does it take to get him on speed? Couple of hours a day. If it's more than that, then there's something that's seriously wrong with the setup that you have. Can you get an intern for a month and then make him start working from day one? These are the kinds of goals that we're looking at. So how do you build systems to make data science teams more productive? That's what we're trying to do. So how do you do that? That's a problem that we're trying to figure out. Here we jump in. Let's take some inspiration from origami. So this is called a flapping bird. Has anyone seen this before? So this is the flapping bird. So take it and then you pull its wings. The tail flaps its wings.  You can understand that. The kids really enjoy this. They can just pull this and then the flap moves. But behind the scenes, there's a lot of folds going in. So this is made of a square sheet of paper. And there are all the folds that actually goes in. So what you have to do is there are two different kinds of folds. You can fold paper as like a mountain. You can fold it, put a crease that kind of forms like a mountain or a valley. And the two kinds of folds are shown with solid lines and dotted lines. So if you actually see that, it takes about some 15 to 20 different folds to achieve there.  So the key thing is to focus on the experience. So when you give it to a kid, you should be able to do that. And the important thing is the right level of abstraction. So we're talking about the level of abstraction that we're giving to the end user. So if you're talking to a kid, you can understand pulling a tail. That's the abstraction that he really understands. But if you talk about mountain folds and valley folds, that's something that's not his language. So he doesn't really understand that. So when you're talking about building tools for data scientists, we should focus really on the experience. So let's talk about the level of experience a bit. If you look at the UA world, people talk about user experience a lot. What does it mean is if you're on a website, how many clicks does it take to make a purchase? So this is a language that user UI designates often talk about. It takes four clicks to actually make a purchase. It should be able to do it with just two clicks. So they create to optimize that experience. And there is a similar thing that nobody has talked about in the developer world. It's called a developer experience. What it means is, so how many steps does it take to start something? How many steps does it take to start a notebook? So how many tasks that you need to kind of remember and then follow to start a notebook, for example? So what's the cognitive overload? How many things seem to remember? How many times you have to go for a sack of overflow before doing that? So we should try to optimize those things. The other thing is also, when we're designing these kind of systems, often people forget about the end user. When people designing systems, people, as engineers, we kind of love complexity. You want to work on the modern buzzwords. Want to work on Docker, Kubernetes. I want to do lambda, et cetera, right? So what happens a lot of times people want to work on this technologies badly that they can pick those technologies for solving the problem, even though that may not be the most optimal thing for the solution at hand. So often what happens is this leads to produces not very efficient solutions. What I call is resume driven development, right? People can try to optimize for their resume, not really for the organization. And that's actually a very common problem I usually find when looking at solutions built to solve DevOps. When you look at conference proposals and the people talking about the kind of things that are built, it's fairly complicated. It doesn't have to be that complex to be built with and they don't really focus on the end developer experience and what happens after a year or two they move on and who's going to manage that system, right? So this is how people can try to visualize and build complex systems to do things, but what's really doing is push button, right? Start a notebook and notebook should be there. So focusing on the end user experience is the crucial part of this. Let me take an example of Hiroku. How many of you use Hiroku here? Not too many, okay. So Hiroku is a platform of service for deploying web applications. It started almost a decade ago I guess. Back then deploying building web application was really hard. You had to get a server and then set up things, database, et cetera. So what Hiroku did was it simplified the whole process. There's only one way of doing things. There's just a push button. Just get push and your application gets deployed. You don't have to worry about anything else. So that's one of the pioneers of developer experience you would call, right? So that's the best developer experience you can find for deploying web applications. Now how do you kind of take that ideas from there and then apply it to data science? So what I'm going to do is take couple of case studies from what I worked on, okay. Building systems for launching notebooks. Let's take data scientists. You want to start working on something. He needs a notebook. You may need a notebook that you want to run on, to run any kind of software. You may want to run scikit-learn, you want to run TensorFlow or Keras or PyTorch. You want to run it on CPU or a GPU, et cetera. And then deploying machine learning models as an API. So once people kind of work on the notebooks, they kind of end up with some kind of machine learning model and they want to kind of deploy as an API so that they can start using it in production. So I'm going to look at these two case studies and see how I have tried to approach that, keeping the developer experience in mind. Hope that gives some insights. So let's look at launching the notebooks. So if you look at the challenges, one of the very important thing is switching between different compute needs. What I mean by that is when people start working on a problem, they usually start with a smaller subset of the data and then try different algorithms. And once they find that something is working, they want to try on the bigger dataset. So you may have to move to bigger instance with more resources. And it's very important to kind of take whatever you have done so far and then seamlessly move to different instance. And the other thing is installing required software. So one of the usual problems is people try to do things completely on their own systems. What happens is that it's very difficult to replicate on someone else's machine. So the things, what goes in setting up is usually in people's head, it becomes difficult for someone else to reproduce the same thing. So one of the important things is to make sure that software depends on handled automatically by the system. The second thing is data storage. You want to store the data. Typically what people do is download some data from web or database, keep it somewhere. It's an iterative process, right? When you're working on notebooks, you get some data, do some processing and then store the data somewhere else and then run something else and that goes on. So you need to have somewhere you can store the data. And even when you switch between your compute needs or stopping the notebook and coming back again, the data should be there. And if you're working on deep learning, you may need to have some kind of a GPU support. So those are the kind of challenges that we need to address in building such a system. So what we have done is we've created these abstractions to kind of solve this problem. So I'll explain what each means. It's a project runtime and instance size. When it's a project, a project is the component of work that you keep all your code, all your data, et cetera. So all your notebooks' data and software dependencies, et cetera, are kind of part of one project. So you can start multiple projects and work on them and then come back to project or switch between projects, et cetera, in the runtime. So this is one of the key abstractions that we've done. A lot of times people want to work on TensorFlow Keras, PyTorch, et cetera. And these versions keep changing. And let's say, if a TensorFlow new version gets released, to make sure that the compatible version of the GPU or GPU drivers are kind of loaded and all that. So it's kind of very complicated setup. Has anyone kind of set up those things for the GPU? So I have spent sleepless nights whenever an upgrade happens and suddenly system stops working because some library somewhere would have changed and it's kind of hard to kind of go and debug. So it's very important to kind of capture all of them as part of system so that you don't really expose those things to the end user. So what the runtime captures is the base software setup. The TensorFlow includes TensorFlow and all the ML libraries. Keras includes TensorFlow and Keras and PyTorch. So one of the things about runtime is it also manages the different versions. What we do is we follow versions by the date. So we have every three month releases of these runtimes. So 2018.03, 06 are our versions. So when you start a project, you can pin to a version of the runtime so that you always use the same version. And the next thing is the instance size. So we can decide some of these instance sizes. There's one S2, M1. So we run this setup. So this runs in the cloud. So if you want to start working, we start with an S2 which is the 3.5 GB RAM. But someone want to have more memory, they can use an M1 or M2 which is 16 GB, et cetera. But if you really want to do some quick analysis with a terabyte of memory, you can use an instance called X1 that has 64 cores and one terabyte memory. Run it for an hour and you pay some $10, $20 for that. So it's kind of very easy to kind of switch between compute needs without having to worry about spending a lot of time. Because since we have the cloud, you can really exploit that. You can start an instance, use it as long as you want and then stop it. A lot of times when people want to process large amounts of data for a very short amount of times, they put a lot of engineering efforts to set up something Spark or something. It makes sense if a large organization and then you want to do that again and again. But if it's something one-time job or you do it once a month to kind of analyze all the monthly data and then generate some kind of report, you don't really have to spend so much of engineering effort on Hadoop or Spark. But it makes sense when there is a need, but you may be able to buy a couple of years of time before we kind of get those kind of expertise by using some system like that. Right, you can just start an instance which has terabyte of RAM or two terabytes of RAM and you just pay some $20, $30 extra for that and you're done. And then if you want to use it, so these are the things that we have defined. So data scientists can, when you start a project, he picks a project, this starts a project, picks a runtime and then starts a notebook with one of these things. So that's the language that we are speaking now. So data scientists doesn't really worry about where is the machine coming, what's infrastructure, how to install software and all those kind of things. Sure, yeah, so I'll come back to that. I'll show you behind the scenes what happens, then we'll come back to that question. Please wait for that. So now how do you specify additional dependencies? So this kind of, a lot of times when people build systems that solves 80% of the problem and for remaining 20% of the problem, you just have to find workarounds and that kind of takes 90% of your time. That happens if a lot of systems. It's very, very important to make sure that the systems that you're designing are open, that they can actually take the additional. So one of the philosophies is common things should be simple and difficult things should be possible. Right, so how do you do additional dependencies, for example? So the way we do it is we write a runtime.ext that specifies the runtime. And you can write an environment YML that's the, if you're using Python, there's a conda environment specification, you can specify all the dependencies that picks it from there. Or you can write requirements.txt file that just lists Python dependencies. Most of the time people just write requirements.txt, 99% of the time, no one really want to do anything more than that. Actually 50% of people don't even actually come there, just write runtime and then start working on that. If you really want to install some app packages, which is if you're using system packages, I need to have graph-wise installed so that I can visualize my decision to model, okay. So there's an app.txt, you can list down all your app packages that you want to install. That's mostly good enough for almost all people, but some people may, power users want something more. There's a post-built script that you can write that will get executed at the end of it so that anything that doesn't fit in any of them, these things you can write a script and then keep it there. So that takes care of everything else that doesn't work. Now, if you look at this, we've been very careful, there are very careful design decisions that are made here. Data scientists never see this complexity, but if someone wants to do something more than what the system provides, there's a way to come out and then do all those things. Typically we have to do these things by writing a Docker file. People write, use containers, and then ask data scientists to write Docker files so that it is generic, but it's really too low level for data scientists to work on. And you don't want to do that for every time you want to work on it, right? Because there may be a case where you want to do some customizations, like 1% towards 0.5% of the times, and it's a sin to kind of force everyone to kind of write this complexity to anticipate for that. So let's see what happens behind the scenes. Now every time a project, it's a 45 minute session, yeah. So it builds two Docker images. So Docker is a container system where you can take build a Docker image and then run it on any machine. So behind the scenes, it actually builds two Docker images, one for CPU and one for GPU. Randoms are actually also built in the same way. So there's one runtime image for CPU, one for GPU. It manes compute instances automatically. It also pulls the computer sources to optimize the source utilization. For example, if 20 people are working, you can actually put all of them on one machine with half the capacity, so that when your people are working on a notebook, a lot of times they're kind of looking at the code and then deciding what to do next. Computing, I mean, the utilization is very low when people are working with notebooks most of the time. So you can actually, so this has an option to pull resources. So the instance types are dedicated. S1 and S2 are pooled resources. So you don't really get a dedicated instance for you, but you can put a lot of common usage kind of goes there. It uses a network file system to purchase data and notebooks so that you stop in a notebook and then you want to start on a different instance. You actually start from exactly where you left. And it automatically manes URL endpoints and HTTPS. So when you start running a notebook, it gives you an HTTPS URL that you can start working with. Now coming to, so the question that you've asked, you're saying it's not safe, right? So this system runs inside the customer's VPC. So data is still there in the customer's cloud, doesn't really come out. So if you're an organization using a system like that, it stays in your own system. So we are building it for others to use, but if you are designing systems for yourself, you can still, even though it's in the cloud, it's inside your private network, so data will not go out. So that's not a problem at all. I think the important thing is to make sure like who are developing it for, and then are we speaking the same right level of abstraction for that people can use. So the three abstracts that we have created, one is a project, other is run times. The third thing is the instance size. So data scientists just have to focus on those things and doesn't have to worry about anything else. Everything else is managed by the system automatically. Let me show you how it works. They create a project, and they create a project, they pick the runtime, and they select the machine instance size when they're starting instance, and say start a Jupyter lab. That takes a while, and then launch a notebook, and then you get a Jupyter lab that can work on. I think I stop it, and then the system takes care of stopping the resources that you have used and all that. So everything else is kind of managed by the system, except what project you want, what's a runtime, and what kind of instance that you want to start with. And you can stop here, and then come back tomorrow, and then start your work. So, yeah, sure, no, it's already, so when you set it up once, it creates a Docker image behind the scenes, and then keeps it, and when you start running it, it starts from there itself. So that's, so if you set up a machine on a GCP, you tried something, and then you found that an algorithm working better for you. Now, you want to work on the entire data set, so it doesn't work on the same machine. You want to work on a larger instance, but you have to still run it and do it. You still have to run the whole thing, okay? You have to run all the installation sets up again on the new machine, okay, okay? Okay, so, okay, so the thing is, Docker is a design choice that I have taken that's not exposed to the end user, okay? So I think what's important is to keep what experience that we're giving to the end user, okay? So behind the scenes, what technology choice are made are ease of managing for the platform, okay? So Docker is used because, for example, you want to run it on a GPU. Will you be able to still do that? No, because it doesn't work exactly the same. I've never used GCP for GPUs, but the thing is what happens is you need to have completely different software setup for GPUs to work. You need to install TensorFlow CPU for CPU and GPU for that, okay? And that requires a lot of CUDA libraries and then there, whenever the version changes, you have to make sure that appropriate versions are installed, okay? So, believe me, I spent so much time, I kind of cancelled myself as an expert in infrastructure and Docker and all those kind of things and I've spent a countless number of sleepless nights whenever I upgraded from TensorFlow 1.6 to 1.9 or something, okay? Because the CUDA version, let me tell the story, okay? So, it was before one of the deep learning workshops that one of my friend was doing and I was providing the support, the people are using GPU standard platform. So, I upgraded the TensorFlow version to 1.9. It really stopped working, so TensorFlow wasn't working. I realized that it says the CUDA version is mismatched, just 10105 or something, okay? So, I went to Docker Hub and see that it has 1010. I couldn't find the exact version of the CUDA library because that's a minor version change, so they didn't actually maintain, NVIDIA didn't maintain that Docker library for that version, but TensorFlow insisting that it needs only that version. Now, what I had to do was actually figure out, go to the GitLab of NVIDIA, look at their versions and then clone that repository, go back to the date where they actually switched from 1005 to that version, get that code, build it myself and then now I saved it to my Docker repository and then pinned it and then started using it, okay? And this took me like whole, the next day morning was the training, okay? And I had to spend all of it, okay? Given that I had so much of experience in doing this, I could do it, okay? I mean, if you start managing these things on your own, when initially this comes in, will you be able to manage it yourself? Maybe you have experienced Docker, okay? What are the rest of your data science team? Will they be able to do the same thing? Let's say if an intern joins your team, if he gets with this trouble, okay? Will you stop all your work and go and help him? Or, right? So it's very important to build systems and platforms so that it speaks right level of abstraction. Otherwise, you can get this kind of issues. Now, what we do is, so this is part of runtime. So we change runtimes once every three months. So if the TensorFlow version is changed, then we change it and the next three months, we get the new version of it. Then we kind of do a thorough testing and make sure that that is addressed. But if you are trying to do it for your project basis, then you have to repeat the same thing again for every project that you're working on. Does it make sense? Sure. So Google Cloud, so right now this runs on AWS, okay? Could run on, what we have built is on AWS. It could run on Google Cloud as well, but if you look at AWS or Google Cloud, the two low level for data scientists use. Fair, I mean, I'm not saying that you should use this. The thing is about, I'm sure like, what Google is providing fits is that right level abstraction for your team to work. I mean, I think that's a fair choice, okay? What I'm saying is, I think the important message here is that it should keep the end user experience in mind. And I mean, frankly, that I have never found any of those tools that are developed by Google and AWS solving that abstraction. For example, I've looked at even SageMaker, it doesn't allow you to kind of switch between compute needs. For example, you have to decide the machine and then it starts the machine and you can't really switch to a different instance type, even with SageMaker. Though people are doing it, I mean, frankly, I didn't find it. What I'm saying is, I mean, you don't have to kind of buy my idea, but the important thing is to kind of understand the key ideas that are behind the design choices. Why I made the design choices, that I think there is an important element there. Made the design choice because the target audience are the data scientists and can find optimize it for that audience, right? Any other questions? Sure. Sorry? Okay, I think it depends on the platform. So, I mean, in our case, but using, no, no, no, it depends on much lower level how the architect is designed. For example, we use a network file system, okay? So if you create a file and other person can pick it, okay? But if you, people are reading the same notebook at the same time, that might create some trouble, okay? So usually, whenever you have a common state, concurrent rights are a problem, okay? So usually when you are doing concurrent rights, you can acquire a lock and make sure that only one person doing it and all that, okay? But if one person writing and multiple people reading it, usually not a problem. No, no, I think that's probably a limitation. I mean, the way it's kind of architected, okay? I don't really know how that system is designed, okay? Sure, no, we don't.  No, I'll come to that. So I'm gonna talk about deploying later, okay? Sure, no, no, I'm sorry. I never worked on Databricks. It runs on Spark, right? Okay, yeah. Okay, yeah, I think it runs on ML, I believe, yeah. So, see, the thing is, frankly, so this thing kind of came out of our requirements, okay? I'm sure it could have, see, anyone who kind of keep focus on that end user developer experience would design something in similar lines, right? You wanted to, if you kind of study a bit in detail and then keep a long term plan, probably come up with something similar, okay? So that's about the first case study, okay? So that's how we build it, okay? So the important thing is, the focus always on the developer experience. The end user is a data scientist, and then he should be able to work comfortably without having anyone else need. If one started GPU, yeah, sorry. So everything else is behind the scenes. Yeah, so the end user, sorry, yeah, sorry. No, we're just using a network file system. So we provide something that you can also store use anything else, you can save the database or some, or an F3 bucket, et cetera. So this stores inside the network file system. We use EFS for that, but so anything that quick reads and write that you want to do, you kind of keep it there. So it's not very high-performance, but it's very, very, very convenient that you write some file, et cetera, but for hitting your performance, then you probably have to switch to F3 or something or database, but for the most common piece, common needs, it kind of solves it, okay? The second thing that K said that he wanted to talk about is deploying machine learning models. This is often the case, and if I kind of Google for deploying machine learning models, I actually find very complex picture like how people do deploying machine learning models and all that, okay? So therefore, this is more challenging than actually launching notebooks because you have to design and document your APIs. How do you actually build an API? How many of you actually designed some kind of an API before? Okay. So if it involves designing and documenting the API, running the service and config your endpoints, you have to do a lot of devops for that, scale to meet the needs, and then once you kind of build the API and it's running, you probably have to write a client library so that you can actually use it easily. And then how do you authenticate that, tracking the user in performance? So the common task that you have to do it, every time you deploy an API. So you have to kind of, so it kind of become repetitive unless you kind of build some kind of system to handle that. So also, I keep things to keep in mind, it's like once you've done this, okay? How long does it take to deploy another API exactly like that? Would you have to kind of repeat like n different steps or can it be done with one single step? How long does it take to explain to someone new? These are the concerns that you actually keep in mind when you're designing these things. The traditional approach is people write, pick a web framework and then write an API and then deploy using GUNE coordinates and X or something, and then put in some machine deployed, Docker, AWS, I mean these are the kind of, both words that kind of goes in when someone deploys machine learning model. And if you go for how to deploy machine learning model, you'll find some diagram showing like all these things interconnected and all that. But what we have observed is that, when I try to see how to deploy machine learning model, we found that running a Python function in API is really hard. Take any, write a simple Python function and deploy as an API. There's so many things that you have to do. It doesn't have to be that way. So we thought what's right level of abstraction for doing this? So one, you have a function to start with, write your Python function, this is square function. I don't deploy as an API. I don't have to do anything more, right? I won't run this function as an API. That's the best abstraction that you can think of, right? So we've built a tool called Firefly to do that. There's an open source library. So it really built just to solve this problem of deploying machine learning models as an API. All you have to do is Firefly, module name, dot function name, you get an API to use it, okay? Right? So you have a Python function, you already written that takes some parameters. You can build, so the tool what it does is, looks at the function that you have, what arguments it takes, and then creates an API out of the box. And how do you use it? You can create a client. So Firefly comes with a client out of the box. You can create a client and give the URL of the client, and then call that and pass the parameters and it gives you the result. Behind the scenes, it actually calls REST API and gives you the result, okay? So when we started looking at how to deploy machine learning models, we realized that there wasn't any tool that speaks the right level of abstraction. So we built this tool called Firefly, and then this is how it works. What are machine learning models? We're just talking about toy square problem, right? It won't be any different. So you write a phase detection dot pi that take import job live, you load it model, and then there's a pretty function that takes image or image URL, and then runs dot predict on the model and then gives back the results, whatever you're expecting. Make sense? Now that's fine, but you don't really want to do those things yourself. You want to kind of have an integration into the bigger platform. So what we've done is we wrote a config file. We specified a config file. You write a config file in the project, specifies what API is running. It's a simple ML based config file. It says these are services. It's the API I want to run. The function to call is phase detection dot predict and use instance of size S2 and you deploy and it starts running that API. It gives you our endpoint to use that API there. So if you look at the previous thing, we have instance size S1, S2. That's instance size that you can specify here. Now, when we started doing it, we realized that we wanted to, so we kind of built this and then thought, let us build a showcase to kind of demo the APIs that were built and realize that from JavaScript I'm unable to call it because you need to enable cross origin resource sharing.  So what we did is we added that support to the API. So you just add cross allow origins and specify what origins that you allow. So if you ever worked with JavaScript, Browse has a security model that will not allow you to send requests to any other domains other than the domain that you're on. That's a security risk if it allows, okay. To be able to allow that the website, what you're calling should allow it to be called. So you have to specify a special header called cross origin allow, sorry, cross allow origins. So you add that additional flag here in the config file in the deploy again. Now it's ready to be used with JavaScript. I want scale up. Say scale, I want to run four of these instances. That'll run four of them. And then you actually get your land point with GPS and everything. The API is up and running. So if you take a data scientist, you want to run, deploy an API. All you have to do now is write a paid function that's still in this territory, right? Write a paid function, takes these arguments and then give something back. And then you have to write this small animal-based config file and then specify what all things that you need. And they could multiple of these things and this is a push button. Just say a row, row, deploy. That's a command line tool that we use. The deploy, that goes in, deploys it. So sends all the code to the platform and then deploys those applications. See behind the scenes, okay. It's based on the same, part of the same platform that we talked about for the notebooks. It builds the Docker image. Again, it starts with the runtime. You can write your requirements at TXT or app.txt for installing other things. Starts all the specific services, provides your land points with HTTPS automatically. So when you talk about deploying machine-line models, what the data scientist, the end user that want to do is like, I have this function and I want this, I have a model file, I want to deploy that. A lot of times you end up doing some kind of preprocessing, so you make it right by a valid function and then the data scientist just have to specify what function to be called as an API. And he has all the levers to make what all things that he wants, okay. And also after the box it generates the API documentation, picks it from the Python function. So it looks at the doxing of the Python function that you are exposing and looks at the parameters that you are using and then generates the API docs automatically. So if you have an API, you don't have to worry about writing API docs. You don't have to worry about writing a client library because Firefly gives the client library by default. Now we are working on adding support for type checking. Python 3 has this beautiful type checking function that you can write function adaptations. You can actually say that this takes list of strings as argument and gives one string back, et cetera, okay. Like you do in other language like Java. You can do that in Python 3. So we're trying to integrate that and actually make that part of the library and we're also working on adding GRPC support. So what would eventually happen is you kind of say GRPC mode equal to GRPC then it would start the GRPC service instead of the REST service. So GRPC is known to be more efficient than typical REST APIs. So again, that's something that next extension of the kind of planning, but the important thing is having architecting a system like that, keeping then developer experience in mind, allows you to expand it that way and the data scientist will still be able to kind of do those things without having to understand what goes behind the scenes, et cetera. Any questions? So we have built on our own, but right now, so we can build with Docker and all that. Now we're kind of putting that to Kubernetes so that we don't, right now, we kind of manage starting servers and all that. We're kind of putting Kubernetes now so that it automatically takes care of starting and gives the scaling ability out of the box. So that's about the second case study. I think one, let me stand with the quote, the good design is invisible. What it means is you don't really feel the presence of a good design, you only feel the absence of it. When you architect a system with the right abstractions, you don't really feel that it doesn't come on your way at all, you'll never notice that there's something like that system is present. You will not even see that is actually doing all these main scenes because you don't really have to worry about it. But if it's not really the right, then it comes and obstructs you and all your flow and that's when you kind of see that system. Let me summarize. So the making data science team self-sufficient is key to their productivity and optimize, always optimize for developer experience and right level of developer experience, right level of abstraction is the key for kind of making that happen. And thank you and I'm open for questions. Sure, so see when you're deploying machine learning model, the way typically web scales is you can start many of this, the many instance of the same process. So it scales automatically, you're really not doing any parallel computation there. So no, so there are two things, whether you're trying to optimize the throughput or latency, okay. So starting multiple instance, multiple servers will surely improve the throughput. If you want to improve the latency, then you have to make sure that CML government can run on multiple instances, multiple codes. So then the algorithm has to support that. So the other, again the important thing is now, once you have a system like that with the right interface, you can throw the implementation everywhere and then build something else. And still, the data science don't even have to worry about what's changing, okay. For example, there are some libraries to deploy the machine learning models built in Python and run them on JVM. So that the start time and they can actually run faster. For example, I was speaking to someone at Uber who built Michelangelo. They say that they actually try to get five millisecond latency for each API call. How do you achieve that?  So now, if that's the need for you, maybe you can actually add some feature to the platform where once you put the machine learning algorithm, it translates that to something that Java can read from. Then the deployment, the process is the same. Data scientists would see have the same experience, but behind the scenes actually gets deployed. In Java or something, okay.  The other thing is when you have the tool language problem, the way people are solving it now is using microservices. Because you run it as an API and then the end users who want to consume the API, free to choose whatever language that they want to use. So you don't really, so I don't think we're kind of addressing that at this point, there's a prediction, okay. Prediction, so usually inference is very fast. Training is what that takes time, okay. So there are other parts of system where we're working on is trying to give one of the common patterns that people follow, okay. Data science, we're trying to make these patterns available as part of the platform. One of the things is offline queue. So you want to able to submit a job and then give it, get ID and then keep on polling it, okay. So setting up is actually very complicated. Usually you have to set up something like it is and then offline queue and number of workouts and all that. So what we're working on is kind of have a core platform and then have these kind of plugins. Then say I want to, for this project, I want to have an offline queue. I want to have an airflow as a plugin. So all you have to do is write the airflow files and then put it in an airflow directory. It picks it from there and automatically scales, automatically configures how to running them and the end user can say that what scale that you want to operate on, how many workouts you want to run, et cetera. Everything else would be part of the platform. So I think anything that you're doing, what you should keep in mind is what is the level of experience that you want to give? What's the level of abstraction that you want to provide to the end user? And then back what, build those kind of systems. Sure, yeah, yeah, thanks.


Hi, Myself Dr. Ramakrishna. In this video, we will learn about the symptoms of breast cancer and the investigations and the staging of it. How are breast cancer generally presented to patients? Most of the patients present in this breast are lumped out. Some of the patients are enlarged in the lymph nodes of the axilla. There are many patients who have been discharged from the breast. There are more than 100 patients present in the symptoms I am telling now. There are many patients who have been discharged from the breast. Generally, the patients who are present in the lumped out symptoms are diagnosed with a mammogram. There are many patients who are present in the left breast cancer. There are many patients who have been diagnosed with a mammogram. There are many patients who are present in the lumped out symptoms. There are many patients who are present in the lumped out symptoms. There are many patients who are present in the lumped out symptoms. Why should we know? There should be no need of lumped out in the breast. There are many patients who have been discharged from the breast. There are many patients who are present in the lumped out symptoms.ään ఽిచిల్లువాల్లులులు człowieతింపత్వావారంటినిల్లు బర్లు పత్ట్ల్లులోల్లుసిడింనితందాంపల్ఱయ్లులులులు క్మిమార్ల 허팝ంసంత్వార్ల్లుంచ వాటరం boot మానోలూ మాటినోలులూ టానేలువాన్ల్లులులు మాటి వానోలులులులులుమాయాసి పౌమపయకమదంవన్యొసివానృదం సపమమనేనంసంవను వకమనివాన�


Hi everyone, Joe for Jaspy's casebreaks.com coming at you with 2019 Panini Prison Football Hobby Pack number nine. Three randomizers happening here. First dice roll will be given away extra spots in this break, six of them. Then we'll do the pack break itself and then we'll do a third and final dice roll to give away those final three flawless random number block spots. And this knocks out Pick Your Team 2 coming up in the next video. Great thanks to this group right here for making it happen. Now let's see who's going to win those extra spots. Let's roll it and randomize it five out of five, ten times. Top six after ten extra spots for you. One, two, three, four, five, six, seven, eight, nine and tenth and final time. After ten we got Matthew, Matthew, Bradley, Russell, Matthew and Brad again. Little buy one, get one scenario for you, that's nice. Giving yourself a little extra odds to maybe win an extra spot. Hopefully that comes, could come in handy. Let's gather everybody's names here and now let's do the second dice roll which will be the pack break itself. Let's roll it, randomize it five out of six, eleven times, one, two, eleven and final time, five out of six, eleven times for the teams, one, two, three, four and eleven and final time. Alright so here's the first half of the list right here and here's the second half right over there, no trade windows in this quick little filler pack, let's just get on with it and let's print this out, rip open the pack and then we'll re-randomize that name and see who gets those number block spots. Now I don't think anyone's taken me up on this offer yet but there is an outside chance we can get another flawless break done if those three random number blocks randomizers, those three sell out. Ooh we got an autograph, yeah there you go, Darnell Savage Jr. Not sure what the value of this is these days but rookie, silver, autograph for Green Bay, that's for Andrew and a Hunter Renfro rookie card, he just recently got a nice little contract extension for my Raiders, might have a pretty big season this here if he's healthy, Michael Robinson with the Raiders, there you go Michael, nice Hunter Renfro rookie card, alright let's flip back over here, now let's see who's going to get the final three random number block spots for Picker Team 2, third and final dice roll, everyone's involved this time, again top three, we'll get into that number block right there, that's a $577 value, so top three after ten, five and a five, ten the hard way, one, two, three, four, five, six, seven, eight, nine and ten and final time, it's going to be a lot of sad faces because it's only the top three, so from four on down, Kevin, sorry, we'll get him next time, but from four down to 32, appreciate you, thank you for getting in, we'll get him next time, top three, you've got some big random number block spots going your way, you move on to the next round, Sean, Andrew, Matthew, that extra spot coming in handy, so congrats to the top three, everyone else, thank you very much, appreciate you giving this shot, I'm Joe for jaspyscasebrakes.com, I'll see you for the random number block randomizer in the next video, then the video after that, the break itself, jaspyscasebrakes.com.


Hi, I'm Jeff Watts and welcome to another episode of Jeff's top 10 tips This episode is about distributed agile teams and is in response to the request by Dorothy B. Craft Who left a comment after the last top 10 tips video? I think I might try that as a new Tradition going forward. So the first person to like and comment on this video gets to choose what the topic is next night Let's give that one a go In this video, we're going to talk about what the benefits of distributed teams are the Downsides of distributed agile teams and 10 things that you can do to improve things with your distributed teams Well, what is a distributed team first of all? Well for me a distributed team is a team where not everybody's in the same physical location in the early days of distributed teams In the early days of agile in fact and they've got a bad rap simply because a lot of organizations Took the opportunity to try and Offshore a lot of their work to the cheapest possible area. They could find it's where a lot of the stigma came from so There are some common misconceptions one is that in order to be agile you have to have a co-located team Well, that's not the case Personally, I've been involved in lots of teams that have spread across different countries and been able to use scrum To be very agile some people point to the agile manifesto and say well one of the principles is face-to-face communication And it is But it just is the best form of communication the most efficient and effective form of communication is face-to-face Now while I would imagine a lot of the signatories of the agile manifesto would probably agree that co-located teams Are generally more effective as would I doesn't say that you have to be physically co-located Distributed teams generally get a bad rap But they're not all bad and there are some clear benefits to having people in different parts of the world Firstly we get greater coverage so we can use the time zone difference to our advantage as one Part of the team or perhaps one team is finishing up their day Another team is starting and perhaps we can hand over that work for more continual development In terms of coverage, we've also got access to more people more quality people We can fill the team with the best people regardless of where they are rather than just the people who are physically in our geographical area And that will also give us greater diversity So we can have people from different cultures different perspectives different backgrounds Which will allow us to make more rounded holistic decisions and avoid things like groupthink So there are a few benefits to having distributed teams So what are the main downsides to distributed teams? Well, the main one really is in the is in the words that we chose that distributed teams Not a lot of the distributed teams that I've seen or been part of actually feel like a team They're not as close together. They don't get to look each other in the eye. They don't get to Feel connected to one another. They don't have the rapport of a team So given the choice most people would probably like to work in the team where they could see the people they're working with Not always but mostly What is this Happy holidays the way but do not open it till Christmas. So generally morale is lower Issues take longer to resolve things drop between the cracks more often and People leaving it to one another and not really knowing who's doing what and where the responsibilities are So there's more dependencies and less collaboration As well as all that there are the time zone issues Generally, there's going to be little overlap the more distributed you are so that limits the the opportunity for actually working together on things and and Separate us into smaller groups And we tend to focus the work then into areas or patches or batches that individuals can do rather than the team can do It's not impossible But it does make things more difficult So here are a few tips for what you can do to help make things a little bit better But 10 my first tip is to not use it as an excuse What I mean by that well pretty much every team these days is distributed to some degree even even teams where everybody is in the same building quite often are at different parts of the building or different floors and They'll have flexible working hours People working from home quite a lot. So every team has some degree of distribution and disbursement So there's no real opportunity to say it. Well, we can't be agile because our teams all over the place In an ideal work, we probably would have a co-located team But we're probably not in an ideal world so given That this is the case that we find ourselves in would I rather Use an agile approach or a more traditional approach I would always go for the more agile approach because it kind of forces collaboration It reduces the amount of time that things can go wrong and people can not talk to each other Because at least at the end of every iteration We're going to have to collaborate and see where we are and get some feedback and talk to one another So an agile approach allows us to make the best of our situation. It's an easy excuse to put in there But it doesn't really help and this idea I've said this in a few of my videos before whether you believe you can or believe you can't You're probably right and this idea of Getting into our heads that we can't do this because so so is in a different country Means we're generally going to be looking for evidence that backs up that view that confirmation bias So you can't use that as an excuse anymore It's about making the best of the situation and working out ways to make the best use of your time and the people in your team At nine my suggestion is to co-locate as much as you can so perhaps rather than having Two teams where they're both split over locations Have two feature teams that one is in each location If you can't do it by team then then co-locate by triads or pairs or something like that so we there is some element of Co-location at some point some some aspect of direct collaboration all the time That's probably going to be better than having two feature teams split across both those locations And if possible make use of that somewhat provocative term that's coming to the lexicon of promiscuous pairing So rotate people around for a period of time that two weeks working with this person two weeks working with that person That will help spread the knowledge spread the awareness Spread the understanding and the empathy build those connections that rapport and build up that sense of team identity and culture That's going to be a little bit harder to do in a distributed team Tip eight is assume positive intent. What do you mean by that? I hear you ask in general? I find the best teams Give each other the benefit of the doubt They they they work on the assumption that everybody is trying to do a good job They would like to do a good job. I don't know many people that come to work wanting to sabotage the team the organization their colleagues, however The less frequently you see people the further apart you are from them the harder It is to think favorably of them. We generally interpret their actions slightly less Positively if we haven't seen them for a while And so it could be that we need to just kind of force ourselves With a prompt noun again of thinking right this didn't go quite as I would like if I were to give them the benefit of the doubt How would I interpret that? I think that's quite a helpful thing to do for a team So assume positive intent and consciously thinking from a positive interpretation will make those interactions and The sort of bumps in the road that are distributed teams Inevitably going to come across and live a bit easier to tolerate Tip seven is get together when you can this is about getting everybody together and as often as you can Most important times for getting people together with a high band with Ceremonies so things like team kickoff sprint kickoff sprint reviews sprint retrospectives You're going to have such Benefits from that in terms of team dynamics in terms of rapport in terms of understanding it's it's a great way to Build that sense of of team and common ownership and peer accountability Obviously budgets are going to be a challenge find out what you can get as much as you can and then use it in my experience It's never a wasted investment Tip six is to promiscuously pair and particularly around reviews Make sure we're not reviewing our own work, but also make sure we're not reviewing on our own Whatever level we're talking about make sure we're not doing it on our own if we can review together that's much much better feedback and Increases that sense of understanding and empathy and shared perspective and create a common culture within the team over time so Rather than I'll do my bit pass it over to somebody else and they review it pairing as much as you can and then Promiscuously pairing that term of just sharing the pairings around making sure we all get to work with each other And we don't develop little cliques or sub teams Tip five is to agree collaboration times Now I actually I recommend this for teams that are co-located as well as I've already said every team is distributed To some degree agile teams are often quite noisy and that's a good thing Okay, we swarm around a problem. We see what's going on. We hear each other's opinions We're collaborating directly But they can be quite distracting when you really just want to get your head down and do something even the best agile team Even the most agile teams will have quiet time and so the conversation is then around Well, when is the quiet time and when is the collaboration time now in a distributed team That might be less opportunity around that because there may only be a certain amount of time when there is overlap and the possibility For communication and collaboration and perhaps we can flex our working schedules a little bit in in Every every direction to increase that overlap as much as possible And the rest of the time can be our own time But we'll agree sort of interfaces for what we need to hand off to one another as our time zones then split So tip four is invest in tools It might be higher if it didn't make me feel so old Why does that make me feel old because I start I find myself Starting to say things like when I was first doing scrum. We didn't have Slack and we didn't have Skype there was no Facebook. There was no iPhone. I know there was a world before all this stuff kids We had to make it work with rudimentary tools And that's where people start looking at me thinking Jeff. You're you're an old man But it's true and we did make it work then even with with Terrible tools. I mean terrible conference call hardware and software. So teams these days I probably don't appreciate the technology and tools that they have even free or very cheap webcams and iPhone apps and tools and websites and Services out there these days that tools can use Freudian slip there But teams can use to increase their collaboration and actually reduce that feeling of being spread apart Investing in tools to enable the team and that's important enable the team. We don't want to Burden the team with a tool that's going to drag them down bureaucracy and admin But actually enable the teams to collaborate as well as possible Then that's you're going to see the returns on that in terms of collaboration quality morale That sense of team that we were talking about very rarely do I see investment in tools being wasted money? Tip three is to over communicate Basically be prepared to run the risk of sounding repetitively annoying The temptation is to just do the bare minimum because communication is a bit harder when we Distributed in general the tendency in distributed teams is to under communicate not consciously and we kind of make assumptions The other people have got the same interpretation. We've all Misinterpreted an email. Okay, and the more distributed the team is the more we rely on things like email So over communicating things spelling things out at the risk of sounding patronizing repeating things at the risk of sounding annoying Is generally a good thing to do is much better than under communicating when we've made a decision as a team Let's over communicate that let's reinforce that. Let's make sure we're all aware of it and we haven't forgotten it If we've got some standards as a team That's another good thing to over communicate because we need to get into disciplined habits Because we don't have the opportunity to look each other in the eye and and review things a lot more Informally and unorganically as we would if we were sitting next to each other And the third thing which is kind of tied to standards that I think it's important to over communicate is the definition of done Whatever that means at a feature level Perhaps even at a task level perhaps at an individual level what our interfaces are between people if we have any hand-offs within the team It's really important to over emphasize those rather than under emphasize those Tip two is to learn about each other now I'm a big fan of this regardless of how distributed the team is all the best teams that I've seen Know each other as people as well as colleagues that that builds rapport in a sense of accountability and says I care about this person Yeah, but it's even more important than a distributed team because it's very easy to be just a name Rather than an actual person Just a resource rather than actual human being So knowing a little bit about who we are. I'm gonna know what annoys you as well I can use that to help build a relationship and build a successful team that I want to be part of and Building in a little bit of just personal time perhaps into our days and our perhaps our conversations So it's not just all about work again. That that can be seen as a cost, but for me. It's an investment And tip one my number one tip for distributed agile teams is to create some working agreements Coming together and agreeing what what kind of team do we want to be and what does that mean for us? How are our individual values lining up here? So working out what we're prepared to do as individuals to to make that team that we would like to be part of What we're prepared to commit to each other to make that happen what we need from each other to make that happen And of course we'll make mistakes We'll forget things but we can Have positive intent and remember that nobody's deliberately trying to screw up we go again And we'll get better So a quick summary of my top 10 tips then Don't use it as an excuse Almost every team is distributed to a degree Co-locate as much as you can Assume positive intent get everyone together when it's possible promiscuously pair around reviews agree specific collaboration times invest in tools over communicate decisions and standards learn about each other and draft working agreements So that's the end of my top 10 tips video on distributed agile teams Like I said the first person to like and comment on this video can pick the topic for the next top 10 tips video I hope you found it useful and until next time Take care


To even the careful student of the life and ministry of Jesus Christ, the Savior's admonition to be even as I am is daunting and seemingly unattainable. Perhaps you are like me, all too aware of your faults and failings, so you may find it mentally more comfortable to walk a path with no upward incline and little growth. Shortly, this teaching is unrealistic and hyperbole, we rationalize, as we comfortably choose the course of least resistance, thereby burning fewer calories of needed change. But what if becoming even as he is is not figurative, even in our mortal condition? What if it is, to some degree, attainable in this life and indeed a prerequisite to being with him again? What if, even as I am, is exactly and precisely what is meant by the Savior? Then what? What level of effort would we be willing to give to invite His miraculous power into our lives so that we can change our very nature? Elder Neil A. Maxwell taught, quote, as we ponder, having been commanded by Jesus to become like him, we see that our present circumstance is one in which we are not necessarily wicked, but rather is one in which we are so half-hearted and so lacking in enthusiasm for his cause, which is our cause too. We extol but seldom emulate him, end quote. A young minister, Charles M. Sheldon, expressed similar sentiments this way, quote, our Christianity loves its ease and comfort too well to take up anything so rough and heavy as a cross, end quote. In fact, all are under the directive to become like him, just as Jesus Christ became like the Father. As we progress, we become more complete, finished, and fully developed. Such teaching is not based on anyone's sex doctrines, but come directly from the master himself. It is through this lens that life should be lived, communications considered, and relationships fostered. Truly, there is no other way to heal the wounds of broken relationships or of a fractured society than for each of us to more fully emulate the Prince of Peace. Let's consider how to begin a thoughtful, deliberate, and intentional pursuit of becoming as he is by gaining the very attributes of Jesus Christ. A few years ago, my wife and I stood at the trailhead of Japan's highest mountain, Mount Fuji. As we began our ascent, we looked up to this far-distant summit and wondered if we could get there. As we progressed, fatigue, sore muscles, and the effects of altitude set in, mentally, it became important for us to focus on just the next step. We would say, I may not soon make it to the top, but I can do this next step right now. Over time, the daunting task ultimately became achievable step by step. The first step on this path to becoming like Jesus Christ is to have the desire to do so. Understanding the admonition to be like him is good, but that understanding needs to be coupled with a yearning to transform ourselves one step at a time beyond the natural man. To develop the desire, we must know who he is. We must know something of his character, and we must look for his attributes in scripture, worship services, and other holy places. As we begin to know more of him, we will see his attributes reflected in others. This will encourage us on our own quest for if others can attain in some measure his attributes, so can we. If we are honest with ourselves, the light of Christ within us whispers that there is distance between where we are in comparison with the desired character of the Savior. Such honesty is vital if we are to progress in becoming like him. Indeed, honesty is one of his attributes. Now those of us who are brave might consider asking a trusted family member, spouse, friend, or spiritual leader what attribute of Jesus Christ we are in need of, and we may need to brace ourselves for the response. Sometimes we see ourselves with distorted funhouse mirrors that show us either much more round or much more lean than we really are. Trusted friends and family can help us see ourselves more accurately, but even they, as loving and helpful as they would like to be, can see things imperfectly. As a result, it is vital that we ask our loving Heavenly Father what we are need of and where we should focus our efforts. He has a perfect view of us and will lovingly show us our weakness. Perhaps you will learn that you need greater patience, humility, charity, love, hope, diligence, or obedience to name a few. Not long ago I had a soul-stretching experience when a loving church leader made a very direct suggestion that I could use a greater measure of a certain attribute. He lovingly cut through any distortion. That night I shared this experience with my wife. She was mercifully charitable even as she agreed with his suggestion. The Holy Ghost confirmed to me that their counsel was from a loving Heavenly Father. It may also be helpful to honestly complete the Christ-like attribute activity in chapter six of Preach My Gospel. Once you have made an assessment, an honest assessment, and resolved to start the hike up the mountain, you will need to repent. President Russell M. Nelson lovingly taught, quote, when we choose to repent, we choose to change. We allow the Savior to transform us into the best version of ourselves. We choose to grow spiritually and receive joy, the joy of redemption in him. When we choose to repent, we choose to become more like Jesus Christ, end quote. Becoming as Jesus Christ is will require changing our hearts and minds, indeed our very character and doing so is possible only through the saving grace of Jesus Christ. Now that you have resolved to change and repent and have sought guidance through prayer, pondering honestly, and possibly counseling with others, you will need to select an attribute that will keenly become your focus. You will need to commit to exerting meaningful effort. These attributes won't come cheaply and suddenly, but through his grace they will come incrementally while endeavoring. Christ-like attributes are gifts from a loving Heavenly Father to bless us and those around us. Accordingly, all efforts to obtain these attributes will require heartfelt pleas for his divine assistance. If we seek these gifts to better serve others, he will bless us in our efforts. Selfishly pursuing a gift from God will end in disappointment and frustration. By focusing deeply on one needed attribute, as you progress in obtaining that attribute, other attributes begin to accrue to you. Cannot someone who is focusing deeply on charity not increase in love and humility? Cannot someone who is focusing on obedience not gain greater diligence and hope? Your significant efforts to gain one attribute become the tide that raises all boats in the harbor. It is important for me as I strive to become like him to record my experiences and what I'm learning. As I study with one of his attributes deep in my mind, his scriptures become new. As I see examples of this attribute in his teachings, his ministry and his disciples, my eye becomes more focused on recognizing the attribute in others. I have observed wonderful individuals both within and without the church who have attributes that emulate him. They are powerful examples of how those attributes can be manifest in mere mortals through his loving grace. In order to see real progress, you will need to put in sustained effort. Much like climbing a mountain requires preparation before and endurance and perseverance during ascent, so too will this journey require real effort and sacrifice. True Christianity in which we strive to become like our master has always required our best efforts. Now a brief word of caution. The commandment to be like him is not intended to make you feel guilty, unworthy or unloved. Our entire mortal experience is about progression, trying, failing and succeeding. As much as my wife and I may have wished that we could close our eyes and magically transport ourselves to the summit, that is not what life is about. You are good enough, you are loved, but that does not mean that you are yet complete. There is work to be done in this life and the next. Only with his divine help can we progress toward becoming like him. In these times when all things appear to be in commotion and fear is seemingly upon all people, the only antidote, the only remedy is to strive to be like the savior, the redeemer of all mankind, the light of the world, and to seek after he who declared I am the way. I know that becoming like him through his divine help and strength is achievable step by step. If not so, he would not have given us this commandment. I know this in part because I see attributes of him in so many of you, of these things I testify in the name of Jesus Christ, amen. Amen.


Hello, everyone. My name is Katie Gomanje and currently I am a senior field engineer at Apple. I have joined Apple two years ago and as part of this role I am aiming to bring Kubernetes and cloud native expertise to different teams and products within Apple as well I am one of the TOC or Technical Oversight Committee member for CNCF. As part of this role I am joining ten of the champions within the industry and we try to provision a technical vision for the CNCF landscape I have many other roles in the community one of them being the creator of the cloud native fundamental scores Which you will be able to find on Udacity Also, I am one of the co-lead for the KCNA exam or Kubernetes and cloud native associate certification In addition to that I am an industry awarded professional and this year I have been awarded with the next generation leader by the Women in IT Awards and also I have won the Tech Woman 100 within the UK Today, however, I would like to talk about sustainability chronicles and more specifically how we can innovate for green technology by using Kepler and Kedah And to begin with I would like to introduce the notion of cloud sustainability and more importantly Why the tech sector should address their carbon emissions here? I'm going to talk about a strategy as well on how we can introduce sustainability in your day-to-day making decision-making process Next I would like to apply sustainability to the cloud native context So pretty much how can you address emissions within your Kubernetes cluster? First I'm going to talk about Kepler, which will allow us to create this inventory of our emissions and visualize our carbon footprint and Next I'm going to talk about Kedah carbon aware operator that will allow us to scale our applications based on carbon intensity Now I would like to start by giving an introduction to the cloud native landscape Currently it is composed of 175 projects and growing and all of them extend Kubernetes functionalities These landscapes allows the flexibility to create a platform that enables our product with minimal compromises This is the landscape that we the TUC are working with and liaise with every single project to ensure that they reach the maximum potential in maturity Now currently all of these projects are dispersed for three different levels of maturity. We have sandbox incubation and graduation Now sandbox projects are a green-filled ideas that provide a solution for a niche problem space Next move towards incubation and these are the projects that already have usages in production But at the same time we see diversifications in contributions from different organizations And lastly we have graduated projects and these are the projects we refer to as the projects who cross the chasm So moved from the early majority to early adopter stage These projects provide an industry standards for the solution in that problem space Now currently within the CNCF we have a hundred and nine projects within sandbox 36 and incubation and 24 in graduation I would also like to draw your attention towards the faint gray line at the top of the graph Now this gray line represents the archive projects, and I don't think we talk sufficiently about archivals Sometimes we have projects that do not reach the momentum in terms of adoption and contribution And it's only natural for the maintainers to take the lessons learned and move towards a new initiative within the landscape Or perhaps create a new project that they would open source Now considering the ongoing success for Kubernetes and the growing cloud native landscape You might ask yourself why sustainability matters? More importantly how it can be applied to the cloud native context The short answer to this question is that currently we have multiple incentives to intertwine the economic growth With cloud and digital sustainability, and this is because of the two factors that happened The first one is scopes 21 which took place in Paris in 2015 and this led to an international agreement to keep the global warming between 1.5 and 2 degrees Celsius The second factor is the UN SDGs or sustainable development goals SDG 13 focuses on climate action and the fact that as a humanity we actually need to think about sustainability in a proactive manner Now this resulted with regulations at the national levels that in turn resulted with different organizations reporting their greenhouse emissions And we can see this in numbers as well Now currently the tech sector is responsible for 1.4 percent of total greenhouse emissions Now if we move to use renewable energy for running our infrastructure and products, these emissions would drop by 80 percent However, if we don't take any action, we would be responsible for 10 percent of global emissions within a decade Now as you can see we are at a crucial point within the industry We need to drive remediation strategies for our sustainability Build internal expertise and ensure that we think about sustainability on a daily basis Now as part of these missions, we already have the big cloud providers setting net zero targets for themselves Now when we're looking into the renewable energy, we have AWS and Azure aiming to do so by 2025 GCP already runs on renewable energy since 2022 Now when we're looking into offsetting the carbon footprint, we currently have the big cloud providers setting 2030 as their goal AWS will aim to achieve that by 2040 In addition to that we have Azure which is setting further ambitious goals for themselves And they would like to reduce deforestation, be water positive and achieve zero waste certification However, again, we do not or we should actually not only rely on the cloud providers to address sustainability We need to integrate that as part of every single org and this led to the creation of a new school of thought Which is known as green ops which derived from the phenops foundation Now green ops encapsulates all of the tooling processes culture and behavioral changes required and are related to digital and cloud sustainability Now why it actually derived from the phenops foundation? This is because we can see a direct correlation between running your infrastructure efficiently and having a positive sustainability impact So if you're running phenops principles or you're trying to go for them You would perhaps run on spot instances or you would like to move from a SAS solution to serverless So pretty much running an instance of your application only when required Perhaps you're changing the programming language for your application to use something that is more optimal for your containers Now all of these phenops principles will result into a positive sustainability impact because the less computer use The more well actually the less emissions you're going to have and we can link actually sustainability to direct profit for your organization And in addition to that the environmental sustainability working group from the phenops foundation came up with a strategy on how to Integrate sustainability in your day-to-day operations The first phase is going to be awareness Here is where you'd like to introduce sustainability to all of the stakeholders internally You want to talk about your carbon footprint and energy consumption Next we're going to have the discovery value stage Here's where we do the POCs now if you're using a cloud provider You'll be able to use the sustainability calculators or carbon footprint calculators So you already will be able to create that baseline as well at this stage Am I looking to tooling such as Kepler and Keta that I'm going to talk on in next in the next section The next stage we're going to have is the roadmap and here is where we'd like to implement some of these operations and implement them in our day-to-day work and here as well what we want to do is to Identify those anomalies and to address them. So we want a qualitative assessment of our emissions And finally we have repetition and execution now as part of this phase. It is very important for us to have Sustainability goals set up internally and more importantly want to iterate for all of these processes and revise all of our tooling and Automations to ensure that we reach the sustainability goals in an ambitious matter now Measuring the carbon emissions and our greenhouse emissions is just the first step towards mitigation and management We need to build this muscle of carbon accounting and to create this inventory of emissions for our entire work different teams products and services Now, how can you do so in a cloud native context if running Kubernetes? How can we measure our carbon footprint well for that? We have a new tool which emerged within our ecosystem, which is called Kepler Kepler is a Kubernetes efficient power level exporter that uses ebpf to trace Power-related stats for your containers and nodes and explore those as primitive metrics Now Kepler was created by Red Hat and IBM in 2022 and it was admitted within the CNCF as a sandbox project in May 2023 Now I'm going to put my TOC hat on here You can see that Kepler is a sandbox project as such we need more diversifications and maintainership and contributions If you have any usage for Kepler, if you have any POCs over the running internally I definitely encourage you to reach out to the maintainers and become an active contributor Now let's look into how Kepler works. Currently it is going to be deployed as a demon set which means we're going to have a replica of the Kepler exporter and every single node and this will allow us to collect two levels of metrics Container level metrics and node level metrics and pretty much we're going to look into power consumption and resource Utilization This is going to be done to sell for ebpf by tracing the cp performance counters and Linux kernel trace points All of these are going to be exported as Prometheus metrics And this is where we aggregate all of our data and we'll be able to visualize them in a tool such as Grafana Now how exactly does Kepler calculate the carbon footprint? This is going to be done for free emission gases. We're going to look at coal petroleum and natural gas The first thing we're going to need is our input value This is going to be the amount of energy that our application consumes and this is going to be measured in kilowatts hour Next we multiply this amount of energy we consume by an emission factor and this is a constant Now this particular coefficients I have taken from the US energy information US energy information administration Now this emission factor represents the amount of carbon dioxide that is emitted per unit of kilowatt hour consumed and The end result is going to be the amount of CO2 we have per emission gas And how it's actually going to look like if we use the Grafana dashboard for Kepler The first thing we're going to be faced with are free gauges for every single emission gas So here we have coal petroleum and natural gas If you look at natural gas we're doing pretty well in terms of emissions But if you look at coal and petroleum this is where perhaps we would like to assess address and iterate If you look at the top of the screenshot you'll be able to see these coefficients or emission factors that are hard-coded Now very important here if you're running an AWS or GCP in a particular region You'll be able to override these coefficients with the data provision by these co-providers Allowing you to have a more granular and realistic data about your emissions In addition to that you'll be able to see the data aggregated per container, namespace and per day Again, these are very good dashboards for you to assess the spikiness in your emissions and ideally you'd like to address them over the time Now with Kepler we are able to create this inventory of our emissions and we'll be able to divide that for team products and services What happens if you'd like to be more proactive in terms of how we use and schedule our workloads? Having sustainability in mind Well for that we're going to use Keta carbon-aware operator Before I talk about the operator I would like to give a very quick introduction to Keta, which is an event-driven auto-scaler and Pretty much the name says what it does. It scales our application based on an event that is triggered outside of the cluster Keta currently has been developed by Red Hat and Microsoft in 2019 It moved to CNCF sandbox in 2020 and it moved incubation a year later And this year we are very welcome to we're actually very happy to welcome Keta as a fully graduated project Again with my TSE hat on here Keta is a great example to showcase the maturity growth of a project And I definitely encourage you to go through the due diligence docs to actually understand how the project reached this level of maturity Now how Keta works to scale any application with Keta You will need a scaled object CRD or customer service definition Now the scaled object is going to have two level of configurations The first one is going to be the application we'd like to scale this can be a deployment a stateful set or an average CRD The second level of configuration is going to be the actual trigger So what event do we need to monitor and assess in order to scale our application? With Keta will be able to scale our application all the way to zero and to one But more importantly it works in conjunction with HPA or horizontal pod auto-scaler So we'll be able to increase the amount of replicas to a desired number defined by the user It is worth to mention as well that Keta has a very rich portfolio of scalars and currently there are more than 64 scalars that we can actively introduce and Use for our scaling of applications Now let's go back to sustainability How can we scale our application based on carbon awareness all for that We have a carbon aware operator that aims to optimize our carbon emissions and environmental impact by scaling our workloads based on carbon intensity Carbon intensity is pretty much the grams of CO2 or carbon dioxide equivalent emitted per unit of kilowatt hour consumed So for example, if you're plugging your infrastructure completely into a wind farm or a solar farm pretty much using renewable energy 100% your carbon intensity is going to be zero or close to zero because you do not emit any carbon while consuming this energy In a more realistic example, however, you are plugging your infrastructure to a grid that has multiple sources of energy Some of the energy can be produced by burning the coal or fuel some of it can be renewable energy So you're going to have a different level of coefficient for your carbon intensity So how exactly we can scale our application based on carbon emissions? Well for that we're going to use a carbon aware keta scaler CRD or customer service definition If you look into the specs section, the first thing we're going to do is reference our scaled object Which is managed by Keta. Here is pretty much where we say that we want this particular application to be scaled up and down Next we're going to have the carbon intensity forecast So with Keta carbon aware operator You are able to plug it into directly to third cloud providers and to extract this data of your emissions in real time So some of these providers are what time and electricity map However for this particular example, I have used mock data. So my carbon forecast Mock data variable is actually going to be set to true And lastly towards the end section is this is where we describe the behavior of scaling our application The main idea here is that if the carbon intensity is high We want the lowest amount of replicas if we have a high carbon intensity That means we have more emissions. We don't want that. So we reduce amount of replicas and vice versa when the carbon intensity is low This is when we want to increase our amount of replicas So for this particular example, we want to have 15 replicas if our carbon intensity is 543 or less and we want to reduce our amount of replicas to one if our carbon intensity reaches 579 or higher And after deploying this particular CRD will be able to visualize the behavior of our application scaling in this graph The top line represents the carbon intensity and the yellow line at the bottom represents the amount of replica Now currently we start with a carbon intensity of 540 and 15 amount of replicas Towards the middle of the graph you'll be able to see that our carbon intensity is going higher to 580 and the amount of replica Is going to one and this kind of behavior is going to be replicated further through based on the data of carbon intensity We're pulling from the grid and this brings me to the conclusion today Those far we have talked about cloud sustainability and why it is important for the tech sector to introduce sustainability as part of their day-to-day decision-making Next we looked into how we can build this inventor of our emissions by using a tool such as Kepler But more importantly we can actually take action direct action within our clusters on how we scale our applications based on carbon intensity And using Keta carbon-aware operator And all of this has been possible because the cloud native landscape is growing And it is a point where embraces this new tooling that allows us to innovate for green technology and being aware of our carbon emissions and environmental impact and Without saying we are hiring if you would be interested to work for Apple anywhere in the world I definitely encourage you to go to jobs at apple.com or we'll be able to find the latest openings I'm more than happy to answer any questions in regards to the equipment process or the experience of working at Apple If you have any questions in regards to today's talk I'm more than happy to answer thereafter or you can find me on social media such as Twitter and LinkedIn This is Katie Gamangi, and I look forward to seeing how you can shape the cloud native ecosystem Thank you, and enjoy the rest of the conference