Now, here to tell us about the future of OpenShift, Red Hat's own Clayton Coleman. Hey everybody, we'll give him just a minute for people to filter in. Apparently, tonight's going to be a good night. I'm looking for that hello world. All right, so for those of you who don't know me, my name is Clayton Coleman, architect for OpenShift. I've worked on OpenShift for about four years now. I've worked on Kubernetes since it was started. And to me, it's been really important to position why OpenShift exists. Like, why does OpenShift exist? We have Kubernetes, we have Docker. We have all of these tools out there in the ecosystem. What do we create OpenShift for and what's it trying to accomplish? I think of the left side of this chart as it's about Kubernetes. And the right side of the chart is about everything else that isn't Kubernetes. And so OpenShift is about bringing these two together. It's about making infrastructure available for use. Chris talked about this this morning with containers and what advantages containers give to us. Portability is really important. The ability to run something on your laptop and run it in production was the original promise of Docker. And I think one of the things that we've done with Kubernetes in OpenShift is really trying to bring that there. So OC cluster up for anybody who's familiar with it is a command you can run on your local laptop, starts up a container that runs a containerized OpenShift. You can test out your applications and then when you need to scale, you're just running the same application everywhere. That's really important. But on the application side, it's not just about running applications. It's about how those applications come to be. And it's not just about the what, it's the process that gets you to that point. So it's packaging and deployment. It's a config driven approach so that you can define your applications. And there's lots of ways to define config, right? Everybody has their own approach. And instead of trying to pick a particular solution, one of the things we focused on is trying to be as consumable as possible because OpenShift sees use all the way from the very far simplest possible 12-factor app where there's literally one input, your source code, and everything works from there, all the way to the far end, which is you use it to build very, very complex applications with many moving parts. And any solution that has to span that is pretty difficult to represent with one single tool. And we understand that and we try to keep that in mind every time we design, which is everything that the really simple app uses, we want to be able to move all the way to the right end of the spectrum. So OpenShift exists because OpenShift exists as a separate thing from Kubernetes, partially because when Kubernetes was started, we made a very conscious decision that we wanted Kubernetes to be an effective core for running applications. And we wanted to deliver it as fast as possible. This is one of the key goals of the Kubernetes open source project from the very beginning, which was actually put something real in people's hands. And so we looked at what Kubernetes would be an application runtime. We said, well, there's a ton of things that don't fit in that policy. It's how administrators deal with a platform. It's how you keep this running. It's how you install it. It's how you deal with all the existing systems that you have in your enterprise already, authentication, policy management, LDAP. These are all things that are required to be able to run not just a platform for applications, but a tentative platform for applications. And so we worked with the Kubernetes team. We kind of split the responsibility. So the OpenShift engineers work on both Kubernetes and OpenShift. And OpenShift is really everything else that's necessary. It's the asterisk on the end of application platform. It's the stuff that actually makes it possible to run applications in a shared environment. And so that focus from the beginning is we want Kubernetes to be usable in a tentative fashion. We want to optimize for team software development. So we talked about the 12-factor app. All I do is push code and applications tick through. And that is one end of the spectrum. And for a lot of people, that's maybe all they need. But on the other end of the spectrum is I want to deploy an existing application, and Chris alluded to this earlier, is I want to deploy an existing application onto my platform. And I don't want to have to rewrite it in order to do that. So how do we build enough flexibility into a process that can go from one end to the other? So on the very far end in OpenShift, we have the really simple build pipeline concept. And I'll talk about some of the things we've added. But we want to be able to fit into flows that already exist for end users. So if you've already got a continuous deployment pipeline, if you already have a build system, if you already have an extremely complicated process-driven lifecycle, at any of those points, we want to have tools that are valuable to use and consume and fit within the needs of that organization. And build is a great example of this. A lot of people run external build systems that are working perfectly fine for them. They don't need to change them. We want to adapt and be able to run the applications that come out the other end of that pipeline without necessarily forcing people to change everything. We wanted to be able to create and deploy applications in seconds. So the idea of this is everybody, the first thing that you see when you run Docker is you do Docker Run and you give it an image. And something starts in seconds. We want to be able to do the same thing on top of Kubernetes. And so, again, when we started Kubernetes, there's been a lot of development in the ecosystem since then. There's a lot of new ways to build and run applications. Everybody's got their own package manager today. And our focus has always been on, we just want to make it easy for someone to see a list of things that they can consume, that someone's made available for them, that actually just work. And so in OpenShift, we have the concept of templates. Over time, you're likely to see other things in the ecosystem brought in. But the core idea is I push a button and I see value. And I see that right away. And it builds on top of the things in Kubernetes. But there's aspects of it that need to be taken account like security and how this fits into the rest of the application. Does it need to be exposed to the outside world or not? We want to be able to run all the stateful parts of the application. That's part of what I'm going to talk about today. And we want to do this easily. Ease of use is really important. I'm sure if there's some people here who've used OpenShift to say it's very complex. There's a lot of moving pieces. And there's this balance between the real world is about big complex applications at all ends of the spectrum that some of these are really easy to use. Some of these take entire teams weeks to get set up to do a single deployment. And instead of trying to make everything be the Fisher price equivalent of application deployment, what we want to do is meet people where they are and find the specific points of ease of use on the spectrum. So I can push a button and see an application deployed in seconds. All the way to, if I build a big config setup process, I have an existing setup process in my organization that can define and manage all this. I'm using Ansible or I'm using Chef or Puppet. At the end of the day, we have these tools that you can still get the ability to deploy something easily even at the other end. And so our goals with OpenShift is to keep driving in the community, make it the best place to run applications, provide well integrated tools, ensure that applications remain portable to any environment, and to any environment means Kubernetes. But it may also mean if you choose not to use Kubernetes or you want to take your applications outside, we're not going to try and lock everything into Kubernetes. We think Kubernetes has value, and we want to make it easy to build images and run images on a containerized platform. But we always understand there will always be edge cases in all of this, and we try not to be too opinionated about the environment that you run in. We think that Kubernetes is a great place to run applications, and we think OpenShift is a better, more secure place to run applications today. But the end goal is that asterisk for environment is something that's containerized that runs images and it does it well so that you don't have to think about it. And then finally, and this is kind of a, this is a new one, and again Matt alluded to this early on, was we have more and more infrastructure, and more and more clouds. We have machines just sitting under our desks that are, you know, 10 times more powerful than your computer from 20 years ago. All of that capacity is just going to waste. And some of it's not going to waste if it's powered down, but the idea of being able to take advantage of this to make things work at scale, to be able to test big complex applications easily, which was very difficult before. The more you can buy into this model, the more you're able to take advantage of, if you've got a cluster and it's got a thousand nodes on it, maybe I can run a thousand nodes in my application and see what happens. But down the road, I might. So this is just standard architectural diagram of OpenShift. I don't even think it's really that important for what we're going to talk about because what we're talking about is OpenShift 3.3. I talked about a little bit of the principles of where we're going. The highlights were on the infrastructure level, you know, when you talk about the system and down of running applications, containers, and the runtime environment. The next level up, which is making sure that the system is easy to administer, is secure, has enough policy points of control for administrators to be able to integrate to a wide range of processes, and then the developer in ease of use on top of that platform. So I'm going to talk a little bit about most of these. And again, this is OpenShift 3.3, which was released in September. OpenShift 3.4 is coming out very soon now. OpenShift 3.4 has an even longer list of things. OpenShift 3.5 will come after that. And this is really driven by both the Docker, the Kubernetes, and the OpenShift communities. There's a lot of exciting development happening, which is, it seems like every three or four months you just get a whole new bucket full of really useful features that solve real problems. And to me, that's the most exciting thing about working in these ecosystems is there's an incredible amount of people working together. Diane alluded to all of the folks working together to build containers and in the containerized ecosystem. There's just a ton of new stuff that every day we find a new way to bring value to applications and developing applications. So the first and I think most significant from a developer perspective is we looked around and we said Jenkins is the de facto developer continuous integration tool. Not everybody uses it, right? There's lots of alternatives. It's a rich ecosystem. But if you pulled a random set of people, maybe I'll just ask in this room, how many people here use the CI CD system as part of their pipelines? Okay. How many people use Jenkins? And so we said, we don't think that Jenkins is the be all end all. But Jenkins has some of the same value that some of the tools in the real ecosystem do, which is it's used just enough places and there's just enough people working on it that there's a strong community that has a solution for almost everything. There's a ton of great documentation out there and Jenkins can be a beast sometimes. But at the end of the day, we understand that for the most people Jenkins provides a really exciting tool. As part of some of the work that's coming in Jenkins, it's a pretty old project. Jenkins is really, the Jenkins community has embraced the idea of pipelines and building concepts in Jenkins that really try to fit the idea of taking software in a process from development through testing through rollout to staging to production. And again, this is a point on the spectrum. Not everybody runs like that. But we felt that there was a really great opportunity to integrate just enough of Jenkins into OpenShift to make it easy to stand up and deploy a Jenkins cluster and to be able to use it on top of this infrastructure. So as part of OpenShift 3.3 in tech preview, Jenkins, when you go into OpenShift and you launch a build, one of the options for a build is that you can say, well, this build uses a Jenkins file and this is a concept that's been added to Jenkins. A Jenkins file is really a lot like a Docker file. It describes a process. Docker file describes a process for creating image. A Jenkins file describes a development pipeline. There's a lot of work going on in, this is still very early in the Jenkins evolution, of course. But that pipeline is really the idea of, I want to say, here's the steps I go through and I want the infrastructure to make it happen. And with the integration with OpenShift, the Jenkins that runs on top of the cluster integrates the Kubernetes plug-in from Jenkins. It's integrated from an authorization perspective in OpenShift 3.4. So as a user on OpenShift, if I need to get access to a Jenkins instance, I can press a button. Jenkins comes up. I can integrate into a build pipeline. That Jenkins instance can use the platform securely under the same policies and controls that I normally have in that project to build and run apps. And from the UI, we've integrated this into the user interface. We haven't tried to replace the Jenkins user interface, although I'm sure there's some people out there who would be really excited about that. We've tried to integrate it in a way that brings to a developer who's thinking about how I go from beginning to end, a snapshot view of showing what the most important process is. So the UI is showing a set of builds. I can easily take an OpenShift application and switch it over to Jenkins to use Jenkins just by putting a Jenkins file in the source code repository. And then I can define an arbitrary complex pipeline if I wanted. That pipeline, each of those stages is actually using the OpenShift environment. So it's starting containers. It's starting images. This might be building images. It might be building war files. It might be running a completely custom process. It might be running Ansible to publish to a production environment. And so we see there's a lot of... there's a lot of flexibility in this space. And one of the things that we'll be looking at over the next few releases is how to make this easier to consume in a way that doesn't necessarily lock you into any particular platform. You could take this pipeline and run it on another Kubernetes system. You could potentially move it off Jenkins. And we think of this as like a really powerful way to instead of us reinventing a workflow engine, finding workflow engines that people are already using, integrating those communities, supporting and making those secure and easy to use, and then making the entire platform really just represent bringing together the most successful communities out there. There's been a ton of other new user experience changes in OpenShift 1.3. The focus, the OpenShift web console is a developer and application owner focused web console, which is a little bit different than some of the other consoles that exist for Kubernetes, I would say. The Kubernetes dashboard in 1.4 is really great. It's still mostly focused on the operational experience of running Kubernetes objects. In the OpenShift console, we've tried to focus on what's the flow that someone takes going from an idea to production. It's a pretty familiar concept. And the console is intended to make it easy for you to start ideas and also to maintain ideas. So in the OpenShift console, before I showed showing build pipelines, you can get an added glance of what your team or your extended team is doing, but it could also just be you iterating and you just want to see what's going on, run it to production. And depending on whether your production environment is your dev environment, it may actually be that this console is just showing you a personal dev tools, but if you're the kind of team that splits responsibilities or has combined responsibilities across both dev and ops, the OpenShift console is really focused on giving you just enough information to help you decide what's going on in production. So integration of metrics with the user interface, a ton of new, I talked about push button applications. We've really tried to flesh out the experience of I can define a fairly complex application and offer it to my developers. Those developers in a single click can provision and deploy it and extending those capabilities. Also taking images from the Docker Hub, as is running those, making it easier to bring the things that you're already using together in one spot. I talked a little bit in the beginning about stateful applications. And so you can read this title, Pet Sets for Stateful Services. So the Pet Set was a concept in Kubernetes that we shipped as Tech Preview in 1.3. It's now been renamed, so I would call this Stateful Sets for Pet Services. But the idea is... The idea here is that in the real world, there's two types of applications. There's the ones that don't care about their state and you can replicate an infinite number of them. And there's the ones that do where all the money is. And the ones where all the money is is that might be your database, that might be your session store, that could be a message queue. And most of these things are pretty finicky when it comes to you get it wrong and everything goes dark. And so one of the goals is to move the application ecosystem to have a flexible infrastructure, you need concepts that make running services with state easier. And so you can use the existing concepts in Kubernetes 1.0 and 1.1 and 1.2 to accomplish this. But it was obvious that we wanted to build a concept that fit the model of there might be one of me and there never should be more than one of me. And I need access to a database and I have a network identity. I'll talk a lot more about this. I have a talk on Wednesday, Stateful Sets at KubeCon. And I will go into much more detail of this. But at the basic level, we want to make it easy to run databases, message queues, caches, high availability applications, leader elected services on Kubernetes to do it naturally. And there is no magic here. So I want to make that really clear. We all pretend that every solution that comes out is this magic solution that's just going to, oh my gosh, I'm running a HA Stateful Database in production and I've got three levels of failover and if anything happens on my infrastructure, it just automatically works. That's not reality. The reality is there's still a lot of work that operations teams need to work around these kinds of concepts. But we want to make sure that it's possible for an administrator to be able to see what's going on and let the infrastructure do what it does best, which is monitoring health, bringing components in and connecting Stateful applications to attached storage or to security policies. And then on top, allow a very powerful and flexible means for operational teams to manage large clusters. So if you have 10,000 databases that you want to run for developers, most of those databases are probably going to be in a class of dev policy. They're not very important. They don't need to be tuned to a high level. But operationally, you want to make sure that they all keep ticking over. So there's the large scale, broad swath of simple Stateful surfaces. But then there's also, I wouldn't recommend doing this, but if you want to run SAP HANA or something like that on top of OpenShift, we're trying to build the primitives that make it easier and possible to run those safely and then help people move up those levels. And it'll take time. So this is the first step in OpenShift 3.3 and in Kubernetes 1.3, and there's a lot of exciting work coming in the future. This gets a little bit more into the guts. I apologize for digging too down in the weeds, but an important part of Kubernetes is making it easy to run applications portably.  And a really important part of Kubernetes that we don't talk about a lot is making it easy to take applications and run them in multiple spots by putting just enough glue that an application running on development works the same way as an application running on production. And so a couple of the concepts that came into Kubernetes in 1.3 that are part of OpenShift 3.3 config maps, this is really just the idea that just like secrets, you can store those, you can manage configuration as part of the platform or define it as part of your application, which gives anybody who comes and looks at that config a better idea of where that config actually comes from. You don't have to use this. It's a tool, it's a building block pattern that can be composed into more complex forms. That's tied with a little bit with what we call init containers. The init container is the idea that, and this is kind of the big difference between you and maybe the Kubernetes mindset and where we all started with containers, which is when we all started with containers, we're like, great, I'm going to run my container in production. But with Kubernetes, one of the really important concepts was the pod, which is it's not usually, it might just be one container, but sometimes you need two containers together. You've got a container that's your web server and a container that's your Node.js application. Or you've got a cache service co-located with all of your web services. And so this idea of putting containers together is something that's at the bedrock of Kubernetes. It means that you can ship parts of an application independently. So you can ship the web serving differently from the log processing. And init containers are really a way of adding a third point of variation there, which is before all those containers start up, you may want to do something. You might want to download the dependencies for your application on demand from an object store. Or you might want to check to see whether your database service is actually running before you start your Java legacy enterprise application because it doesn't do so well when the database isn't running when it starts up. You might want to generate a config file. And so init containers is really just an expression of we want to make it easier for system administrators, application authors to pull together the pieces of real applications. Chris talked about object-oriented programming. Well, it's not just about services and microservices. It's about different teams working together. So an operations team that's building, it's building monitoring tools or building log processing tools from the data team that sit alongside the web application team and trying to bring all those together in a way that can split responsibilities down the middle is really important to ensuring that you can run all these different classes of applications on OpenShift and Kubernetes. And then finally, the downward API is just another way of saying, I've got all these containers that are running legacy applications that expect to find certain files in certain locations. So it might be an Apache, it would be the configuration file for Apache. It might be a database configuration file. The downward API is really the mechanism whereby you can take some of the dynamic nature of where you're running stuff and put it into a container that doesn't have to care. So this idea of separating out the application from knowing about its environment. It looks at the file system, it sees some files. It works the same on a development environment if you have those files in place as well as in a production environment. And it keeps the application somewhat decoupled from its environment. Again, it doesn't stop you from doing more complex solutions using tools like ZooKeeper or XED to share a config across a large fleet of applications. But it does offer another point in the spectrum of flexibility for running applications. A big feature that came up in OpenShift 33 was AB routing. So there's lots of different ways that people do rollouts.  That's something that's been in OpenShift since the beginning. There's a slightly more sophisticated version which is I want to do a rolling update. Rolling updates work for a large percentage of software, but there's always some wrinkles in that process. You don't really get a chance to test the new stuff in an actual production environment. And that's actually really important. As much as we try to make development, they're the same, they don't. You can guarantee you're probably not running Oracle on your laptop. And so those sorts of setups lead into some more sophisticated stories. So blue-green services or blue-green deployments is something that, again, is possible in Kubernetes. It's gotten better doc and better examples, better setup over the last couple of releases to make it easier. In OpenShift 13, we've added another concept which is the idea of AB routing. So you can use AB routing for a lot of that, two back-in services, one of which might be the official supposed to be running in production. And I'm going to move a percentage of traffic to another service, like a test service, or it could be something like I'm testing a new version of my software for end-users before they actually deploy. And so AB routing is just another tool in the toolbox of real operations teams have to deal with real problems around big, complex applications. And this reuses, again, all of the Kubernetes concepts, and hopefully over time this kind of stuff will make it into Kubernetes. We've been working a lot with the Kubernetes community to take these features and make sure they're consumable. But for the foreseeable future, our goal is to make it as easy as possible to split traffic, to deal with real applications in place, and to work them through the system. So there's been some work as well on the image registry, and I'm starting to run a little bit out of time to talk a lot. We've made a bunch of integration points available in builds. I alluded to that before. We have Jenkins, but we also recognize that people bring builds and tools from outside the ecosystem and want to make them easy to use with the platform. So with OpenShift 3.3, we've really focused on just a lot of little improvements that make OpenShift a good target when external build systems are targeting it.  So, a ton of exciting stuff in OpenShift 3.3. There's a huge number of exciting things coming in Kubernetes 1.4, 1.5, OpenShift 1.3.4, 3.5. Our goals make Kubernetes more extensible and multi-tenant aware. So taking all the things you can get with OpenShift today, making it available to an even broader community, making Cubic extensible is a really big goal for us because when Kubernetes is extensible, anything that somebody in the community can build and use, we'd like to be able to take that and offer that as part of OpenShift and make it available in a supportive way to customers. Continuing to evolve the idea of how you deploy on the platform, Jenkins Pipelines becoming even more integrated in terms of security and reliability and control, and then just a general focus on ensuring that you can run OpenShift from the simplest possible deployment easily to the very largest scales and to do that with a complex set of administrative and networking integrations as well as, you know, meeting individual deployments where they are. And then in the future, I've probably spent like three days talking about all this stuff. So as a sneak peek for this, you can come and find me and there's a number of developers and engineers from the OpenShift team. We'll be happy to talk everyone's ear off about all the great stuff that's kind of becoming in the future. And then please come to the Stateful Sets talk on Wednesday if you're interested in running productionized enterprise applications on OpenShift and Kubernetes. With that, I'd like to invite Mike Barrett and Joe Fernandez to come up and have some Q&A with Clayton. So if there's questions, raise your hands and we'll give you the microphone or you can come stand up. Hi. I'm following OpenShift since almost the beginning and I saw when the switch from V2 to V3 you started to build this container D. Is that correct? Yeah, can you make a parallel between container D and what is going on with Kubernetes? Are you referring to container D from Docker? Are you talking about GearD? So GearD was in a lot of respects an evolutionary step working with Docker and the container ecosystem to get a feel for what we wanted to do on the host. Most of the ideas from GearD actually ended up being part of Kubernetes. So we took a lot of the lessons learned from the node level stuff, worked with a lot of the teams from Google and so GearD eventually was replaced by what we call the node agent or the kubelet in Kubernetes and I think at this point most of the ideas that were generated through that process are part of Kubernetes now. I did forget to put that on there so the question was TCP routing so in OpenShift 3.0 we launched with the router which is the idea of HTTP or HTTPS load balancing and it also supported what's called SNI, server name indication which just means if you have an SSL connection you can load balance it but an additional part of that is there were some lower level primitives in Kubernetes starting in 1.1 and 1.2 and what we've tried to do is there's kind of the easy stuff which is the web load balancing and then there's all the insane software out there that makes very specific assumptions about how it can be load balanced and so in Kubernetes in OpenShift 1.1 there was the node ports concept in OpenShift 3.3 we enabled the cloud load balancer which basically means you can use the same Kubernetes service load balancer in a cloud environment and say I have a service that needs to be load balanced that's TCP only you can enable that in cloud environments and then in I guess it was 3.3 in 3.3 we added the initial work for if metal and you want to load balance a service you can set up your networking so that you can do transparent, highly available TCP based load balancing we also have integration with F5 the F5 integration only does HTTP and HTTPS today but the goal would be at some point to turn that into TCP load balancing for it as well when Clayton refers to the router we ship with a default HA proxy router but the router is one of those pluggable components because we know working with a number of enterprise customers that not everybody is going to make the same choices that we made so F5 is one of our plugins and now the work we're doing with Ingress upstream is making that even more pluggable but that goes for our onboard networking which is we ship with an SDN network based on OpenVswitch but we work with all the SDN partners to make that pluggable Clayton talked about our build system our CI stuff so you can do builds on OpenShift you can do CI, CD on OpenShift but we also make sure that we work with what you have and if you don't want to do builds at all or you want to do a hybrid scenario where maybe your builds are doing application binaries and then our build service is doing just the container containerization that's what we're trying to enable to make sure you can fit it in your environment so in 3.3 there's three or four different ways that you can do TCP load balancing they're not done through routes though because of the slightly different needs of TCP load balancing and some of the documentation is still in the process of being improved for that Is it coming throughout or something? It's tough because we actually want to do that through ingress and Kubernetes so I think as a timing thing it's possible today and you can use all the same Kubernetes primitives that you want and we also have added support that makes it work on bare metal if you're willing to set up the network configuration for it and then we will see a convergence of that probably in another few releases Any update on the policies and network overlay technologies you guys built in with Kubernetes? Sure, so in 3.3 we added egress policy so egress policy is something that had been discussed in Kubernetes but we actually have a large number of customers who need this requirement now so egress policy with the out of the box SDN will firewall you can define firewalls per tenant basically for a project and that will block outgoing traffic from those containers and you can set up you know it's a whitelisting blacklisting setup so you can do whatever you want for that the goal in 3.4 is to enable no sorry 3.5 is to enable ingress policy so this is the work that's been done in Kubernetes around the ingress policy object the network policy objects the goal for that for OpenShift SDN would be 3.5 and they've already started working so the pieces of that and then there's a story coming kind of over the next couple releases is to make it easier to use vendor SDN solutions that may already support network policy so there's no one answer there should be possible in 3.3 to define exit traffic and then ingress traffic you could probably script it on your own and we're gonna get better on the I think egress policy is also a good example of how do you take a cloud native platform and make it work in a traditional enterprise environment right so you spin up all these services on Kubernetes and containers and they need to talk to existing services in your data center but you have all these IT rules about who can talk to a particular service and what IP address or IP range they can come from and on Kubernetes the IP address it's unknown right so this whole notion of taking services making sure that they talk at fixed IP addresses or a fixed IP range so that when they try to connect to your Oracle database or whatever it's actually allowed that's a lot of what Red Hat works on right these are problems maybe that Google wouldn't encounter in their data centers but we encounter it every day in the data centers of enterprise customers that we deal with whether it's on the commercial side or on the public sector side it's kind of part of the bringing Kubernetes to the enterprise if you will that Red Hat specializes in yeah so I noticed having followed OpenShift of course from the beginning there's been OpenShift specific features like secrets and security context that ended up getting migrated into upstream Kubernetes and I was wondering if you could talk a little bit about what the process is on the Red Hat side for targeting what might be a good fit for Kubernetes and then how you work with the community to find the Kubernetes community to figure out what so I think this is the million dollar question and it's part of the longer term evolution of Kubernetes which is how do we define what Kubernetes is that everybody has the same and how do people bring their own special sauce and collaborate to find solutions that benefit multiple people so in the early days it was easy the core Kubernetes 1.0 mission was run containerized applications at scale and we accomplished that and the core OpenShift 3.0 goal at the same time was add a policy multi-tenant layer so we could run multi-tenant containers and then we also knew that we had a whole bunch of requirements like I want to be able to roll updates to applications is a crazy concept I know like I have to update my applications that was something that we added as part of OpenShift in 1.0 because it was not time to do it in cube there was no bandwidth and so now what we've done is we kind of worked through this process is if it's a feature that'll benefit everybody in the entire world we try to put it into core Kubernetes and so we think of multi-tenancy is a good example of that Kubernetes has to be multi-tenant if the core isn't multi-tenant aware then we really end up having security and that's partially why OpenShift is different is a different set of source code or is built you know it's downstream from Kubernetes is because the only way to apply all of that security policy is to rebuild cube to build in the pieces of cube and so our goal is to actually move all of that into Kubernetes over you know several leases egress policy is actually a good example of experimentation so what we like to do is one of the things we're good at Red Hat is supporting customers that we invest in if we put something out there ensuring customers have a clean migration path over multiple years so anything that you run on OpenShift today is going to keep working and we'll give everyone a migration path so that you do have zero down time migration as these features make it into cube it's kind of insulating people from that that cost and on some of the other features in the middle it's actually really a tough choice so a good example is the service catalog work this is something that a bunch of companies Google and IBM and Fujitsu and I believe some pivotal folks involved as well on a large number of companies coming together to say Kubernetes needs a really great service catalog concept that allows you to pick services so you don't have to go stand up everything yourself but you can consume something an IT organization has provided for you that's something where from the very beginning we believe the best place to do that is in the Kubernetes community and it may not actually be part of Kubernetes core it'll be a key extension that we expect most people to run with that was kind of in the middle kind of choice and this is part of the dynamic between Mike and I on the product management side and Clay and his colleagues on the engineering side is you know we we're kind of competing with trying to get some stuff to market quickly and respond to customer demand but always trying to push as much as we can as far enough upstream as we can so that it can be maintained by a broader community so we did come out early in OpenShift with concepts like like deployment configuration like you know authorization that didn't actually yet exist in Kubernetes 1.0, 1.1, 1.2 but then we worked to bring those upstream and in bringing that upstream we're now committed to making sure that our initial implementations are sort of subsumed or integrated with the upstream and again our commitment is to support this over multiple years in multiple instances. I think there's a little attention to this which is sorry Dan. All right these guys are going to be around for the rest of the day and for lunch but I do want to get to the next speaker so thank you very much and hit them up at lunchtime.


Howdy how's it going? My name's Davy Chappy, and it seems that my predictions were correct in that a whole new slew of subclasses are getting shurned out of the Unearthed Arcana warehouse, this article hosting subclasses for the Bard, the Cleric, and the Sorcerer, and like with all the Unearthed Arcana releases, it has become my daft given duty to talk about them. So, as always, keep in mind that a lot of this is just my opinion, so if you feel like I'm way off base with easy way videos, feel free to play your games however you want. And real quick, I'd like to give a brief shoutout to my new patrons this month. Midor, James, Nick, Darla Ramirez, Michael Cruz, Geron Howes. It is because of you that when I saw a moldy sandwich laying on the side of the road, I didn't stop to eat it. But, without other way, let's begin. So, starting off with the Bard, we have The College of Creation, an outer-planar scholar that has become aware of the clearest, purest form of magic, The Song of Creation. With this song, Creation Bards can affect the world around them on a much more drastic level, an ability that manifests in the outrageously cartoony way of having floating music notes around you. These music notes work in tandem with your bardic inspiration, and are given out to your allies at the same time, and they can be used to either explode in a wave of sound around the inspired ally, give them temporary hit points, or completely re-roll that use of bardic inspiration. Pushing the boundaries of The Song of Creation even further, you can spin life into mundane objects to get them to fight for you, and beyond that, A College of Creation Bard can even use the power of The Song of Creation to will certain items into existence for as long as they maintain the song. So long as the item costs less than 20 gold per bard level, and is only up to large in size. And if you maintain the song for one whole minute, then the object becomes semi-permanent for an amount of hours equal to your bard level. As a concept, I'm very much in love with The College of Creation. It's so cool to think of this as a mage that was able to reach so far back into the very idea of magic that they ended up hitting the end of the thread, as in the very point where it all began. My only gripe is that every other bardic subclass gets two features at third level, but this subclass only has one, and while the nodes of potential are cool, they're not so powerful that they make up for the lack of features.   Pulling the power of God, such as Athena, Heimdall, Eldath, and Rao. The Unity Domain embodies the platonic attachment that people have for each other, from spouses to siblings to friends.  As a Cleric of the Unity Domain, you can bond two creatures together so that they perform better while they're nearby each other, and they can later use this bond to shield the other from harm. You can use your channel divinity to absorb the damage from one ally and spread it across all your friends so that everybody can feel the pain. You gain the potent spellcasting feature, and your final ability strengthens your bond power so much that the range becomes near infinite. And if one of the two bondmates goes down, the other gets a massive power boost for either one minute or until the fallen partner wakes up so that they can avenge their teammate's defeat and restore their honor, including getting advantage on everything, resistance to everything, and the ability to shake their partner awake by gifting them a box of heart-shaped hit dice. Now I really like most of what I'm reading here, but the one big gripe that I have is that I think the duration of the bond should be reduced to one minute. Adding 1d4 to something once per turn isn't a super big deal, but once you hit 6 level, you're basically giving free resistance to all damage to two of your allies for an exceptionally long time. Yeah, I know that you have to give up your reaction for it, but let's be honest, there aren't many things that reactions are good for, and none of them beat free resistance to everything. Like make it a bonus action, at least, or perhaps reduce the effectiveness from all damage. The barototum is already broken, we don't need two of them. Even at one minute, the power is still a bit obnoxious, since one minute boils down to 10 rounds of combat, but it at least means that the cleric will have to be a little conservative about using the ability instead of just throwing it up before entering a dungeon and letting that be that. Especially considering that they can use the feature again if they burn a spell slot, it just means that they won't be caught completely with their pants down even if they've used the power already. Hopefully, after that, this domain will once again find unity. And rounding out this batch of archetypes comes the strangest addition to D&D in a while, Steampunk, or more specifically, the Clockwork Soul, a brand of sorcerer that went into the lawful land of Mechanus and decided that a fantastical land full of Modren's gears, Clockwork and Brass, being completely canon to a D&D setting, was a really cool idea. As a Clockwork Soul, you hail from this land, and so your entire being is now caught up in the Clockwork, giving you strange, otherworldly features that help distinguish you as a primal force of the law. And looks aren't the only thing that Primus has gifted you, because as you look out at the world through a pair of brass goggles, you gain abilities, such as being able to balance out rolls by cancelling out any advantage or disadvantage that they had, you can spend sorcery points to ward creatures against damage, you can channel the Speared Animal of the Clock to somehow make it impossible to roll lower than a 10, and your final ability lets you summon the pure forces of law in order for a split second to massively heal everybody, completely repair everything, and end all spells of 6th level or lower in a 30 foot area around you. Overall, this last subclass confuses me, and not because of the thematic idea of it being a steampunk character in a largely fantasy setting. There are guns and dinosaurs in this world too, you can get over it. But it confuses me because the mechanics sort of teeter-totter between the theme of balance and mediation that it's trying to convey. The negation of advantage and disadvantage is cool, but the warding ability is similar to the abjuration wizards, and it's honestly a little better, which shouldn't happen when you are an abjuration wizard. The ability to turn rolls into a base of 10 is super good, because being able to make random parts of the game almost entirely reliable is crazy, but even then I'm not bothered by it. And then you have the ability to essentially reset damage things around you to their base form. All these abilities are totally cool, but I'm ironically having a hard time seeing how they link together. Also, like the Bardic subclass, this one is missing a first level feature that all the other ones get, although I am happy to see that they're continuing the trend of giving the sorcerer bonus spells. The point that I'm trying to make with this rambling is that of all the subclasses presented in this Unearthed Arcana, the Clockwork Soul Sorcerer looks like it might have a few gears loose, but that'll about do it. I hope you enjoyed this video, be sure to leave a like, comment, subscribe, bring the bell to summon the Geometry Goblins, and maybe support me on Patreon so that I can afford to find some unity of my own. But yeah, Davy out.


Beltrami County Deputy Sheriff Jeff Roberts and his wife Linda Roberts were recently honored and presented awards from the Wyoming Highway Patrol Association at a ceremony recently held at the Beltrami County Law Enforcement Center. The awards were presented by Minnesota State Patrol Captain Mike Wedden and Sheriff Phil Hodape on behalf of the Wyoming Highway Patrol Association for the efforts of the Roberts to save the life of a Wyoming Highway Patrol involved in a motor vehicle collision on September 22nd of last year. The Roberts were vacationing at the time with their family and were traveling near Sheridan, Wyoming on Interstate 90 when they witnessed Wyoming State Trooper David Motsick collide head-on with a runway driver. Trooper Motsick suffered multiple and significant life-threatening injuries and Deputy Roberts and his wife Linda who is a North Memorial Aircare Flight Paramedic stopped and rendered immediate aid to the trooper and the driver of the other vehicle. Despite their efforts the driver of the other vehicle died but Trooper Motsick survived and continues to rehabilitate his injuries. For the critical and urgent care rendered to Trooper Motsick, Flight Medic Linda Roberts and paramedics from the Rapid City Fire Department were awarded the Luke Shawland Medal of Life and Deputy Jeff Roberts the Merit Conduct Award. If you've enjoyed this segment of Lakeland news please consider making a tax-deductible contribution to Lakeland Public Television.


let's make sure we are coming in live and we are very nice very nice hi everyone this is chichou welcome to my channel and welcome to another live stream today today is july 22nd and it's been about two weeks if not longer that we did live stream and it couldn't be helped i was sort of recouping from a little flu that's been going around and uh had to get my energy back so that's the reason why we haven't been streaming and i'll give you guys a low down of what took place in the last couple weeks and whatnot and how i dealt with it and what not finished dude how are you doing welcome welcome to another live stream glad to be back wow wow was itching itching itching itching to start streaming but i just didn't have the energy to do it my voice still a little raunchy as you can tell a little bit right i'm not sure if you can i can for some reason it echoes with me more maybe that's the way i hear it anyway but it's good to be back we're going to go into this nice and slow sleepy waves how's it going what's up chichou long time no see in a stream indeed i think two weeks i think it's been two weeks uh yeah yeah probably two weeks i can't remember really two weeks probably longer two and a half weeks or something and it's sort of okay i've been recouping uh getting my energy back uh catching up a lot of news catching up on uploading previous streams content on there so um on our video sharing platform shirtless kylo how are you doing long time brother long time hey gang been a long time since i've got a stream live formally night night how are you doing i actually saw uh you went live a few days ago i got a little notification slip mc 99 great to be back here it's been way too long since i've seen you i hope you're doing well brother hi to sharyl dingbobber all the classics and new people to indeed salutations everyone briefcase wanker what are you doing welcome to our live stream gaming goods hello hello figgy boy 96 what's up everyone it's been a minute since i've joined only got 30 minutes then i have to dip out for a meeting okay figgy okay figgy we'll get you know what let me do my little intro sharyl thank you very much for the sub and hello hello salutations i hope you're munching on almonds three vass those conan books are indeed conan the barbarian one the greatest characters ever created right edwardo salutations salutations and bone and bone my favorite children might my favorite one of my favorite adult books it's amazing and berry wind is fantastic as well zot is pretty good zot is pretty good shirtless kind of off i've i'm playing with the idea of streaming some video games low-key chill stuff not quite sure yet i'm a little busy yeah yeah i've man i was meaning to start streaming playing video games just haven't got around to it that's my four years four five years on twitch 19 days since last stream older god says 19 days so coming up to three weeks now so it took me a little bit of time to try to recoup from this flu that's been going around yeah three vass yeah bone is a children of all ages indeed i recommend to i recommend it to everyone i've read it three times phenomenal phenomenal slick make crush your enemies see them driven before you and to hear the lamentations of their women horrors crumb and that is that is conan the barbarian right haven't read zot haven't even heard of it maybe i'll buy it zot is not bad zot is actually i forget who the person is it's scott mcdonald and he put the quintessential book out there on how to write comic books how to create comic books so it's it it's epic to a certain degree it's science fiction it's nicely told the characters aren't as endearing as bone without a doubt because bone is like wow magnificent right but it was a good read it was it was well presented so it's not bad felix howdy geecho i was laying in pretty good i've read both three times well nice nice nice three vases felix wearing a pretty similar shirt to yours right now nice summertime shirts summertime shirts slick make i would fully watch a 24-hour stream of you playing pa pong one of the best games of all time indeed and i played when i was a little kid when paul first hit the scene atari we played it first time i ever saw pong i must have been like six seven eight years old in iran it was family gathering lots of families getting together having dinners and the kids were sitting in front of tv watching people play pong it was incredible it was incredible never forget that moment grown-ups were coming by going it's like tennis or something and all the kids are like oh my god listening what is this it was friends that had it we never had it when i was in iran no video games back then uh gang let me do my little intro because we're gonna get into the nitty-gritty and we've got to let the people know that are watching this little intro on censor 2 this live stream cannot be loaded on censor 2 because we're gonna talk about this little nasty flu that's been going around and talk about some other things so the technocrats on censor 2 will not allow you to consume information content which they cannot control right so if you are watching this live stream this little short introduction on censor 2 you want to be on biture rumble or rc that's where you want to follow our work straight up gang the amount of censorship taking place on censor tube and technocratic site it's mind boggling it's mind boggling we can't even we can't even discuss what they're not allowing people that are only consuming content on censor 2 to consume and it is some of the most benign some of the most ridiculous things that they are centering and some of the most important things that you need to know about life politics economics health everything everything everything so biture rumble and odyssey and if you want to catch these live streams lives that they're happening we're live streaming on twitch so gang thank you very much for being here we're live streaming on twitch at some point we're going to start live streaming on rumble as well okay so thank you for being here thank you for supporting on twitch gang and all the other platforms as well and indeed thank you for the people that are supporting this work on patreon if you want to follow this work patreon is a great way to do so if you want to support this work patreon is a great way to do so as well we're also have a presence on subscribe star and sub stack i'm going to start writing more stuff on sub stack okay the odds are we're going to build our math content on sub stack because we won't get censored patreon might censor us as well depending on what we talk about so it's we're walking a fine line there okay we're walking a fine line there okay so gang thank you very much for the sport on patreon on twitch and all these platforms it is in large part because of your support that we're able to do what it is that we are doing we do not see these live streams 30 minutes before we go live on mines we can get a part of the getter and bitcloud and we'll see which other platforms we go to and twitter won't be one of them i doubt it i doubt it we will be uploading the audio of this live stream to soundcloud.com forward slash chico chychos a podcast and this podcast should be available your favorite podcasting platform including Spotify iTunes and Google Play and we do have a gilded server you want to join the conversation participate in discussions free speech forum okay join us and gild it agree with us disagree with us challenge us challenge yourself to consume information without having to go through the technocratic sensors okay aside from that gang thank you for thank you for being here let's get down to the nitty-gritty i'm just going to get caught up with chat take these guys down and again if you are on sensor tube links in the description of this video to bit you rumble in odyssey that's where you want to be


Okay, boys and girls, today we're going to be learning about counting by twos starting anywhere on a hundred chart And then I'm gonna let you guys try on your own watch. Mrs. Sims first. I'm gonna start it two four six eight ten twelve fourteen sixteen eighteen Twenty I can also start anywhere else on this hundred chart and count by twos I started a number and I jump over the next and land on the following one just like this if I start at 42 I'm gonna jump over 43 and land on 44. I'm gonna continue to do that and count by twos. Watch me 46 48 50 I'm gonna jump over 51 and land on 52 54 56 58 60, okay first graders what we're gonna do for our second part today is called we do we're gonna do it together and count by twos I'm gonna point to a number on our hundred chart and you're going to say them aloud as I point to them Okay, first number. We're gonna start out is 72. Okay, we're gonna count by twos ready count with me 74 76 78 80 82 84 86 88 90 race give yourself a pat on the back nice job Next time we're going to counting by twos the same way But we're gonna start at 22 count with me ready 22 24 26 28 30 32 34 36 38 40 nice job you rock today I have these little sticky notes and I'm gonna ask that you put a sticky note on the next number when you count by twos Chloe, why don't you come up first? Okay, so let's start at two right up here I'm gonna put my number on my sticky note on two and I like for you Your note on the next number counting by twos Good job say it out loud Nice job Good job Chloe. All right.  Okay, this time. We're just gonna keep going. What's the next number after number 20? 22 nice job 24 Oops, I like how you changed your mistake 26 28 Thank you Maddie you rock


Hello, geometricians and logicians. Today is your introduction to flow proofs. The reason we work with proofs is because in geometry that is a way to be absolutely certain that you can say something is true using deductive reasoning, which you've learned about during this section on logic. There is no note sheet for this section, so get out your notebook and we need you to copy these notes about the five essential parts to a good proof. Pause the video and copy down these five points of important parts to a good proof. Now that you have those copied down, I need you to be aware or remember, these are things you learned back in middle school, but remember some of the properties of algebra. You don't need to copy these down. These are on page 111 in your textbook, but these properties are the things we're going to use when we design our proof. A proof will have a statement, a mathematical statement, and then a reason for each statement. And these properties are the reasons why we can say something or move from one step to the next step in a proof. So it's important for you to kind of remember these properties and have them in mind. Again, if you choose to copy them down, that's great. If you don't want to copy them down, you don't have to. We will be using them in the subsequent proofs.


But there's more. First of all, we improve the relationships in Archimedes. And one of them is just a renaming of the used by relationship, serving. Used by was the only relationship that had a sort of a backwards name, a passive name, and that had to do with the direction of the relationship. It's very important that Archimedes' relationships point in a certain direction, and especially for used by, there was sometimes some confusion around that. The idea is that this relationship points towards the user of a certain service, for example. That's different from languages like UML, which use the direction of the relationship, also to indicate who is initiating the communication. But for enterprise architecture, it's much more relevant to the mode who is actually the user of a certain service, regardless whether that service is offered proactively or reactively. And that's the reason that the relationship points in that direction, but the name used by was confusing. So it's now called serving, a service, or an application component or anything else, serves, say, a business process or an actor or what have you. Then we slightly changed the notation of influence. We changed the arrowhead because it's now more consistent with the access and serving relationships, and that also has to do with the structure of the set of relationships. Those three are kind of dependency relationships, and we wanted to reflect that in the notation. So it used to look much like the flow relationship, and that created some confusion, so we changed the arrowhead to make that a little clearer. And then, like you just saw before in one of the pictures, we changed the notation for assignment. Instead of having these two black circles at both ends, we now show that it actually has a direction, and if you know the standard, it has this big table at the back showing all the relationships. So you can see that it has a direction there, but the notation didn't show that. So we added this new notation. The old one will stay, well, we deprecated that in the standard, but it will stay correct in argument for a while, so in argument three, you can still use the old notation. It's not becoming incorrect, but I would advise you to use the new notation because it's much clearer to see what is assigned to what. So you typically assign actor structure to behavior, and actor structure to passive structure, or actors to roles. So there is a direction that was already there, but it's not shown in the notation. And then we have changed some other aspects of the relationships. We have relaxed the constraints between the different layers. You can now have serving relationships, for example, between any layers, so the business layer, say people in the business layer, can provide services to the application layer. It's no longer just a bottom-up layering, it's much more flexible. And another one that might be important is that we allowed some relationships to other relationships. So you can associate objects with flows, for example. You can actually model that something flows from one, say, application component to another. You can actually show that a certain data object is flowing there. And another important example is aggregation, so that you can aggregate relationships within plateaus or within a group, and I'll get to grouping later on as well. And then we improved the derivation of relationships, and that's a rather complicated subject. You know from the standard perhaps that there's this story about how you can derive relationships. I don't want to go into too much detail there. I advise you to read the standard there, but we improved these properties as well. These are the two examples of relationships to relationships I want to show you. So on the left you see that there's this insurance policy now related, just with an association, to this flow relationship between policy creation and policy management. So you can actually model what is flowing there. And on the right you see that you can model which relations are part of a set of plateau. We couldn't do that, and that was quite a problem, because if you want to model that between two plateaus, there is, say, an additional relationship implemented between, in this case, two application components, there's an additional flow. You couldn't model that. So we fixed that now, and you can have certain relationships with other relationships. So please read the standard to find out what the exact specification of that is. You can't do this in all places, of course, that would create a huge mess. You wouldn't want to have, say, a relationship to a relationship between an object and another relationship that would become pretty complicated, but you can do several things with this. Then we have added another kind of junction, the OR junction. We have the regular junction, the black dots. Now we have an OR junction as well to denote a fork in the path of a process. So here we have in this example that you can either accept or reject a request. This provides you with a little more flexibility in modeling business processes. We do not want to duplicate BP&MN in any way, so if you really have a need for more detailed gateways or what have you, then please use BP&MN, but this provides you with a little more detail on process modeling. We also added the option to use junctions on more than just triggering and flow relationships to denote explicitly that in this example on the right, sales and finance together realize invoicing. There was a confusion always about having a relationship and having a realization relationship from two different, say, in this case, functions to a service. What does that mean? Are they alternative realizations or do they jointly realize it? Using junction in this way helps you express that explicitly. In this case, these two jointly implement realize invoicing. Then we have improved the grouping concept. Grouping used to be just a graphical thing, just a border around a few parts of your architecture, but it didn't have any relationship with what's inside of that. So it didn't mean anything. It was just a graphical thing. Now in argument three, it actually aggregates the stuff inside the group. What you see on the right is equivalent to what you see on the left. Next to aggregating stuff inside the group, you can also relate things to a group. So you can draw relationships to and from the group, and there's an actual meaning to that. That's explained in the standard as well. There's an actual meaning to this realization relationship. These things inside the group together realize this service. Then there are some smaller changes. It used to be in the business layer. That was a bit strange. The location was used across the entire structure of our community, across all the three layers. So we've moved it to a different position in the set of concepts. For the user, that doesn't make much of a difference, but for the people implementing the standard, they're looking at the structure that might make some difference. And we've also changed the relationship between location and what's on that location. It used to be a sign that aggregation is a much more natural way of expressing this. So location now aggregates what's on that location. Assignment is still valid in many places for this because it's the right relationship that I won't go into the details there. The regular way of doing that is like this, aggregating stuff at a location.


Okay, I think that that looks right. So I'm T. J. Alenbaugh. My co-presenter today is Juanchu, who I think should be on Zoom and maybe will just be a disembodied voice through the speakers. So, yeah, but we'll see if his audio is working here in a minute. But yeah, I'm really grateful to have a co-presenter here. Hopefully what we want to show today is looking at a single problem that occurs in two different contexts and hopefully moving in a path forward that allows us both to solve our problem. So it's nice to be able to see the two different contexts here. So Juanchu, are you there? Hello. Oh, okay, great. So yeah, you want to why don't you just take it away and then I'll be the button pusher for the slides. See this title looks slightly different. Yeah, so I'm going to and I'll be talking about memory overcommit in containerized environments with TJ. Next slide, please. Oh, discover playlists. Okay, there we go. So for both clients and servers, workloads can be containerized with VMs, Kubernetes containers, MCGs, and the workloads differ between clients and servers. The client's jobs are more bursty and unpredictable since they react to user interaction and the systems need to respond quickly to interesting events and be aware of energy usage. And the server jobs have a more predictable memory footprint and they're concerned with stability and performance. A common technique is proactive reclaim, where it reclaims memory ahead of memory pressure and it makes apparent the amount of actually free memory on the machine. So some client use cases can include a virtualized OS on desktop and tablets or isolated execution environments for security. And the servers are more concerned about SLOs for different availability tiers, proactive reclaim, like I said before, and the motion promotion between tiers. Next slide, please. So to view this problem of overcommitted memory, we want to introduce the concept of working set as a histogram. So this working set idea is a binning of pages by time or coldness, something like that, and each bin contains a page cap. The diagram on the right, you can see that the bottom two arrows there, they say idle age interval, so all the pages in the particular bin are considered idle for some time between two adjacent interval values. And this could be the active and active LRU split, the empty LRU generations, or access frequency for daemon, or even estimated through user space. So in the system we're presenting today, we collect the working set in the guest or the memcg hierarchy for a better estimation of memory utilization inside containers. This is generated on demand from reclaim activity or queried by the controller, and we use the balloon device to send the working set to the host, which enables the host to make balloon size decisions for each guest. Next slide, please. Right. So for containerized workloads, the data center and client use cases are quite similar. In both cases, we kind of have a controller listening for events, maybe once in a while probing the containers and VMs, and based on the information they receive, they implement policy decisions. So on the left hand side, we show the data center use case. Say we have three jobs, and the one on the right is a best effort job that may use some free capacity. But if a latency sensitive job goes into reclaim, the management daemon receives a working set report, it can shrink the latency tolerant job based on the report. We can also try to unkill the best effort job if it's working so it can no longer be kept resident. So the core piece of the puzzle here is the working set report for setting the precise values of the limits. And a very similar story happens with the overcommitted clients, where multiple VMs need to share fairly the system's limited resources. A fair sharing policy can be employed based on the oldest pages in each VM's working set report, but TJ will talk more about that. Yeah, so the use case, just to highlight on the right hand side, being the client side use case, might have a slightly different or additional complexity, which is that oftentimes there's, you know, a significant host application happening too that the user cares about doing some kind of native application in addition to the virtualized applications. And the user might be going back and forth between these different contexts to do useful activity. Again, maybe, you know, running an important application that's only available on a Windows environment that you want to run virtualized, for example. I think the thing that is the same about these two contexts is that there is the limited memory resource that the utilization of that resource is necessarily isolated. The information about how that resource is being used is isolated by the constraints of the system. And so we seek to find some way to get the global picture of the utilization of the memory. And so the right hand side, we would propose to do that through extending the balloon device, which I'll just give some more detail about. So this slide really shows two different components to accomplish the goal of getting the data from, you know, from the guest to the host. The first component is the notification system of some kind that you want to mentioned where we get notified from within the kernel that a new working set is, a working set report is available. You can think of this as analogous to like a shrinker, the shrinker interface where a component of the kernel, in our case the balloon driver is a client of this interface. And we kind of show it as a pub sub sort of interface where you subscribe to these notifications and then in our implementation and during background reclaim activities, you can get notified of that a new report is available. And so the balloon driver will receive that notification. And then the second component is that we extend the capability of the balloon driver to report this out, this information out back to the device. And then after that, there is typically in a VMM there's a specific functionality specific to that VMM that allows a host controller program to, you know, get this information. So in QEMU, this is known as QMP or the QEMU machine protocol that would communicate this information over a socket to some listening program. And other VMMs have similar functionality across VM, for example, has a similar capability. So then just to mention the host controllers responsibilities here that in whether you're in the upper case or the lower case. Sorry, I just lost control here a little bit. There we go. So these controllers then receive the signals and then they give control inputs to the system. And those control inputs, you know, in the in the upper case would be setting memcg limits or driving reclaim for a particular memcg. For the bottom case, of course, we, you know, our control input is changing the balloon size. And then getting this information or also the ability to query for this information. It gives you the information and then you can implement a policy for these adjustments. There's obviously going to be some notion of fairness, even if it's just implied, because there is a limited resource. For the client side use case, we have to be very adaptable to what the user might be doing at any moment. But of course, for more of the data center use case, we might be able to use historical data to inform that policy. So maybe now is a good time to see if there's any questions before I move on to the next part. So just a question on the report part. Brutal balloon already has like a mechanism to report statistics to the hypervisor. Couldn't you simply like extend that that you like just always include whatever broken set size and you just have to hypervisor pull that regular interval? Yeah, I think it would be possible to extend the stats feature potentially. Let me just go back to that slide and talk a little bit about it. You know, the main thing about the stats feature right is the first thing you do is set your polling interval. And then the system begins to report. And so the benefit here, especially as you expand to multiple VMs is a signal based signal based system during reclaim activities. And so that you could imagine that as a kernel feature and then that as the driver of your stats reporting as opposed to a polling. And so maybe extending that way. There's another component of this that could fit inside stats maybe. And that is this thresholding rate limiting that we do so that if there's a lot of reclaim we're not blasting huge amounts of messages through the system. So that's known as a reporting threshold in the system. And the other thing that happens is again in these multi VM systems. If you've just generated maybe some background reclaim activity happened and you just generated a new report and then immediately afterwards you get queried. You want to have some notion of the staleness of that report that's allowed. So again you're not generating additional work. So those two features are a piece of the system that is part of how you configure this what we made here. And you configure it on the host through the CIS FS and then it's part of this expansion of the balloon that we would propose. So I think that you could expand stats to include this. But the way that we did it right now as an RFC proposal is to talk about it as a separate thing and see what the community thinks and proceed in the conversation that way. I don't think it's very crazy to have that. I mean of course the other question is if you could then just have like some other QMU guest agent in the guest push that on demand to your hypervisor. I mean there are various ways to communicate that just to question what the right interfaces. In terms of like a VSOC or something like that. Yeah for example. Yes that that would be a possible solution. It seems there's benefits in terms of a globally useful thing to have something that just works out of the box for everyone. So that is how we've gone so far. But yeah I guess the main thing to say is that you know we I'm actually not sure if I haven't had a chance to ask you on to if he's posted the patches yet. But we have these patches that we would like to have people give some comments on. So just a word about that. Yeah so there's there's two patches that describe this or show what we have so far. And the kernel patches actual changes to have this feature that we call working set reporting. And then the second patch is for the actual driver changes itself. And then we also have the QMU implementation RFC that is actually not out yet but send in the next couple days. And then the cross VM balloon actually already implements a form of this which is available right now. And then for people who are on the vertio mailing list we also describe the spec edition which is essentially this configuration you know sending the configuration information. And then getting the reports back out specifying the bins and how bin sizes and changing those and all of the many details that are necessary for that kind of stuff. Yeah then we just you know you want to kind of refer to this that you can have various kinds of balloon policies will be releasing a basic script that shows a basic policy that how you can implement a basic policy with these unified working set information. And how you can drive basically an auto balloon kind of feature through this to to not have to to make it very simple to control balloons in a in a multi VM scenario. So you want to I don't know did you have any additional comments or. I'm also just doing some final touch ups on the kernel patches as well and then we'll be posting those in the next day or two. Yeah one final note about you know the different because the balloon features there are a number of balloon features there's a free page reporting is another option that you could imagine to help solve this problem. One of the things that we have at least on the client side application and I haven't spoken when you want to detail about this but we have a lot of situations where there's significant page cash in the VM itself. So the number of free pages reported is actually not that large. And so another solution that people proceed down this path is sometimes people call it a balloon pulsing solution where they inflate the balloon to drive reclaim and then they pulse it back down. So hopefully that you now get to report in fact that you have a lot of free pages. And again I think for the data center case that might yield some I think people do yield some wins with that. For our case with the user we're very sensitive to user perceived latency. And so we again prefer like a signal based situation where we can respond very quickly to what a user is doing. And so we proceeded down the path of trying to do a signal based system. So that's really I think that's everything I was planning to go through. Does anyone have any additional questions or I can just add something. So I'm not a friend of auto ballooning if you ask me. I maintain virtual balloon. I don't like balloon inflation deflation. It's worse. I can understand why people do it because it's always been done like that. But what I think would be better in the long term is if you would be using free page reporting to report free pages. But in addition some way to to tell your guests VM to limit the size of its working set meaning that you said some kind of like you would say well your Linux you have like four gigabytes of memory but please only try to use two gigabytes of it something like that. And if the VM then like runs into some out of memory situations you can say yeah well but if you need more instead of just randomly crashing. Like we've seen for auto ballooning just like exceed that and report report me an update that you need more memory desperately. Because like with auto ballooning you sometimes you can really run into these situations where you suddenly have a workload that consumes a lot of memory in your VM. Whatever like policies you have to slow so you have your guest applications crashing your customer will not be happy which is why auto ballooning is usually not used nowadays anymore. So just like to give you like an idea what to do. And I mean it would still fit into the whole picture of the working set size just that you do use a different mechanism to actually put some memory pressure on the under guest without causing too much harm. So the idea being that yeah I recall your comment from the from the mailing list. And essentially one part of this is in you'll see in the spec change is what we would call a notification of virtue where in this case there's really only two notifications one so far. One is a configuration notification the device is now has a message for you of how you would like to how we would like to configure the working set reporting. And then the second notification right now is requesting a working set report. So one of the things that we'd speculated on is is an additional notification of some kind of of this nature of saying this is we would like you to stay within this allocation or something like that. That hasn't been a part of what we've done yet but I see your point for sure. Hi I'm just wondering how this could scale to like a pooled memory type use case. Like do you do you see the working set as something that could be passed externally and be like a per node instead of per process or per VM being a per node metric that's provided up to like an orchestrator that's going to decide to migrate a VM or move a workload or something like that. Well one of the things in that's part of the patch the RFC that you want to is focused on is basically implementing this for hierarchically for you know MCG V2 and so you have to find some way of hierarchically allocating these bins. So if you have two sub MCG's that have a working set and then the parent MCG needs to understand that you have to aggregate right. And so I could imagine a similar you would do a similar kind of operation I would think to sort of aggregate the working sets. Especially let's say people have chosen different binnings and how do you properly you know aggregate up the bins that that kind of thing. So it seems to me I haven't thought very much about that that was you want to know you want to do you have a comment about that. Well you're talking about nodes are you referring to machines or human nodes. Right machine like one CPU or one honestly one root port when it comes to when you're talking about CXL pooling CXL based pooling it's a root port. It's not even necessarily like the CXL plane doesn't have a good concept for which reports are part of a multi socket system or or whatever. But basically having a layer above just sort of process management within one one kernel instance basically. Well I mean the working set would certainly tell you you know there's not very much hot memory on you know this however you want to talk about whatever your system you have. And there's a lot of hot memory over here and not very much over here. And then you could presumably make some kind of decision. You know about that. Yeah and I remember in the previous talk about this idea of like oh there could be many VMs with a lot of hot memory and you know we sold everyone these VMs and this much memory but we know that they're not going to use it but we actually this user or this set of users. Is right now using a lot of memory right now and could exceed our model of how we've set things up. Yes right right so so this kind of is a kind of system that would allow you to detect that right and then maybe make some decision so that you can maintain your you know your SLA. Yeah and it's not unlike your comment about the server use case where you need to juggle these VMs but the VMs might not be the most important thing on those. There are other things going on so you need this broader sense of context and it's actually a thought that occurred when there was discussion about handing that information over to the user space. Well you could have an application say in absolute terms this is the kind of memory I want but what else is going on. Maybe that's important to the application but it's not important to that particular use case or deployment right. It's all it needs to be done in relative terms so that you know there needs to be one single entity that understands the actual priority list. Right yeah I mean this is again you know this driver of trying to construct global knowledge when you have necessarily isolated it because of the constraints of your system. So that is that that's probably the most succinct way of saying what it is where we're trying to do. Thanks. Okay well thanks very much and also thank you you want to.


Hey so I'm hanging out with Warren here on the Houghton Hotel stand here at the Sandton Convention Center. The Houghton is a residential complex with the hotel being in the middle of the development combining both sides together. Wow. The hotel acts as a residence club for the development itself where you have access to pool, a spa, gymnasium, restaurants, deli, scar lounges and pretty much as I said it acts as a residence club for owners in the development. People are moving away from big freestanding homes. There's a lot of maintenance on those homes and they look into downsides and get to secure living. Private property, a home for everyone.


Are you okay? No, I'm not feeling really good. You don't look really good. What are your symptoms? I've been having this weird mood changes and horrible headaches and back pain. Are you feeling tired? Yeah. Nausea sometimes? Yes. What about sensitivity to certain smells?  Have you had cravings? Yeah, some cravings. Shortness of breath? Mm-hmm. Is it any way possible that you are? Oh my god.