I remember when I was training, when I was doing my medical training, I remember we were doing our cardio, I went to the University of Ottawa and we did our cardiology rotation there. And we were at the Heart Institute and I remember a cardiologist telling me about rheumatic fever and like, oh, you'll never see this. And I remember when I finished residency and when I started working up in Sulu account, I remember the first time I had heard of a diagnosis of rheumatic fever. And it was of a patient who died and we got back the report from the coroner's office and it said, you know, findings consistent with acute rheumatic fever. And I remember thinking to myself, wow, this was something that during my med school matriculation I was taught that was a non-issue anymore, right? At least in Canada, right? And here we are seeing it in Northern Ontario. All right, so basics, what is acute rheumatic fever? We believe it originates from group A streptococcal infection. It's sort of typically it can affect, it's typically can affect the skin, the joints, the brain. The heart can of course be affected or so and lead to permanent valvular disease. We think it's mostly from the pharynx shittic strains of group A streptococcus. Now that's been challenged by some recent data out of Australia that may suggest that some skin strains may be implicated as well. Very can be common in children. Now again, very rare in Canada, very, very rare in Canada, but rheumatic heart disease globally is one of the most common forms of acquired heart disease globally, right? So something that we see globally, not infrequently. All right, it's still with us and we still see it in the North, right? I remember back in about almost 10 years ago now we started to see cases of rheumatic fever when I was working in, when I was working in Sulekout. So I worked in Sulekout for approximately 16 years and I worked in Moose Factory for about another two and a half years. And I remember in about 2013, 2014 starting to see cases of rheumatic fever. In fact, we had seen eight cases of rheumatic fever over about a year, year and a half period of time. And there were two children that died because of rheumatic fever. And when we looked at the rates and when we tried to look at the rates, what is the actual incidence of this? We found that the rate was about 75 times what it is in other regions of the country, right? So the question is, why are we seeing it? And why are we seeing such a high number of cases of rheumatic fever? Now, when we have rheumatic fever, just to go through a little bit about what happens, we know there's an antecedent, usually streptococcal infection. We know there's some degree of immune system priming that you encounter. And then with repeated streptococcal infections, you get repeated sort of immune response. And that leads to, that can lead to permanent valvealopathy, right? And rheumatic heart disease, right? Okay, just a little video here. I wanna show, I'm not sure if it's gonna play. So I apologize. One of the things that we started to see was you might remember from medical school, Sydenham's Korea, right? So we used to have children or be referred to us because their teachers were noticing that they were getting ticks, right? And maybe there was a concern that they were having an interaction of their ADHD medication or something else going on or something. This is just a quick little video here. This is a young patient that came to the clinic for ticks. And this child eventually had, we diagnosed with Sydenham's Korea. So this is an example of Sydenham's Korea. We see this not infrequently, not infrequently in loose factories, as well as throughout the North and stuff. So we might remember that from our med school, med school, med school time. So the big question is the why? Why are we seeing so many cases of rheumatic fever? I didn't want this talk to be focused on pathophysiology and heart valves and echoes and that type of thing because the big question that I ask is the why? Why were we seeing so many cases? What is driving some of these cases? Why? Why are we seeing 75 times the number of cases that we see in other places in Southern Ontario? And what are some of the reasons for that? And we looked at the patients and we realized very quickly that rheumatic fever is not reportable. So it was very difficult to even track cases of rheumatic fever because it's not reportable. So the public health infrastructure that you need to track cases and do adequate surveillance, it doesn't apply for rheumatic fever. So that was an incredible challenge. And when we extended it even further into the public health care system that existed in indigenous communities at the time, it's not the same as the provincial system. You see, you can have what we call jurisdictional ambiguity instead of jurisdictional responsibility. Who is ultimately responsible? You know, in most places in the province of Ontario, we have provincial public health laws and there's a provincial medical officer of health and provincial public health and they have that provincial authority over a specified region. And, you know, if there are outbreaks, they have provincial legislation that they can enact that can ensure that there's adequate surveillance. Within the federal system up North, that system, it's unclear who's responsible for that. Is it the provincial system or is it the federal system? And plus two, the federal public health system doesn't have the same type of infrastructure. It doesn't have the same type of legislative teeth as the provincial system. So again, you have a lot of challenges in terms of even keeping track of the number of cases. A lot of the cases that we're seeing was just sort of word of mouth. I remember when you'd be on in the emergency, you'd hear from a physician that was working the night before and says, yeah, we have this kid and I think they might have rheumatic fever and you were going and you'd go back and check the chart and okay, we'll add that child to the list and make sure that they get adequate follow up. But there was no provincial system that was in place to be able to adequately track and adequately follow these patients and there was no adequate surveillance system. And again, because of this idea of jurisdictional ambiguity. And a lot of things that what is driving these things are the fact that we have systemic inequities. There's the effects of systemic racism and the effects of colonization within the healthcare system that produces differential results based on who you are. So, we're getting these cases and going through and again, it was a lot of just word of mouth, right? Who did you see in the emergency? Who did you see in the emergency? Yeah, I saw this one patient that I got on phones. Yeah, I had this other person that passed through the emergency and they had some unexplained tachycardia. We did an echo, we found a huge level of mitral regurgitation and a clinical picture consistent with rheumatic fever. That's how a lot of these cases were recognized. And we were able to sort of assemble a team of people to at least on an informal way try to track the number of patients, the number of children. And what did we learn? We published a little an article in the Canadian Family Physician Journal about acute rheumatic fever in First Nations communities in Northwestern Ontario, social determinants of health a bite to the heart. Because you realize very quickly that medicine is very interesting. A very little bit of inequity can counteract a whole lot of medical ingenuity. That we were sending these patients to tertiary care centers and they were having access to some of the greatest minds in the medical world. And to think that people didn't have access to running water and people didn't have access to adequate housing or had access to overcrowded in adequate housing. That was what was driving some of these outcomes. So a little bit of inequity can counterbalance a hell of a lot of medical ingenuity. As an example of the Sioux Lookout zone, vast, vast zone, right? This is an area the size of France just to put things in a little bit of scale, right? And there's a number of communities in this region. Right now I work in the Moose Factory zone and this is along the James Bay Colts and they face a lot of similar issues. But you can see we're talking about a vast, vast geographic area, right? So what do we find that we're driving some of these things? Housing, a huge, huge issue is housing. If you're thinking about group-based strep, it's a bacteria. And if you have inadequate access to clean running water and you have a lot of overcrowding and housing, you're gonna have a problem with group-based strep. It's gonna go from person to person very, very easily. And if you have a house that's 600 square feet and 17 people living inside, that's gonna create huge, huge opportunities for this bacteria to spread. These bacteria don't grow in agar exclusively, they grow in inequity and injustice and unfairness. But when you have that level of overcrowding, it's going to lead to significant, significant challenges with group-based strep and with rheumatic fever. The housing was inadequate. Building standards in the North are not the same as building standards in the South. There's poor ventilation. There's no such thing as consistent fire codes. Building materials are inadequate compared to what building materials are in the South. So you can see how a situation of overcrowding and the lack of access to adequate housing can create a perfect storm, water. And this has been well publicized that there have been significant challenges in Northern communities for accessing clean running water. Some communities have been under a boil water advisory for 25 years. That's a whole generation. Imagine for 25 years if a community has not known what it's like to drink clean drinking water. And these factors can contribute significantly to the proliferation of group-based strep. But not only group-based strep, the proliferation of other bugs like MRSA and staph aureus. We published on that that our rates of MRSA and just are very, very high and our rates of invasive MRSA were many fold higher than comparable regions in Southern Ontario. So I would say that we don't have a group-based strep problem and we don't have a rheumatic fever problem. We have an injustice problem and we have an inequity problem and we have an unfairness problem. And it's just manifesting with rheumatic fever. And one of the key things that must happen is that we must address what we call the primordial causes of rheumatic fever. Looking at things like overcrowding, looking at things like adequate housing, looking at things like ensuring communities have access to clean running water. If you address those things, you're going to see rheumatic fever incidents goes down. Remember, rheumatic fever 100 years ago was a very common condition. Very, very common condition. And when you look at rheumatic fever incidents, its incidents started to drop significantly not with the introduction of penicillin in the 1940s and 1950s. But 50 years before when we have better sanitation, when public health units were initially being created, when we were recognizing that we couldn't have 50 people living in one space anymore and we started introducing better sanitation laws and better building codes, that's when we saw rheumatic fever incidents start to plummet well before the introduction of antibiotics. So it's those issues that are going to be crucial in keeping the rates low. You know, it's interesting. There are high levels of rheumatic fever in Australia and New Zealand and in their indigenous communities in those countries. But it's interesting because they have much better statewide and provincial wide surveillance systems, much more robust public health in regards to surveillance. They have an echo program that can actually go and make sure that people are getting their echoes with rheumatic fever so that we can prevent some of the sequelae. They have a much better system that is going to do those things than what we have. Where there's no provincial level surveillance. We have jurisdictional ambiguity with the federal public health versus provincial public health. There's no provincial echo strategies for follow-up. There's nothing to guarantee that I have a child and they have rheumatic fever that's seven or eight years from now they're going to get an echo at some particular point. There's nothing, there's no system, there's no safety net that's going to ensure that that happens. So it's not uncommon that you'll have children that in 2014 and 2014-15, does that make sense? That were identified with rheumatic fever and got maybe a couple echoes but they haven't gotten any in the last several years. One of the big challenges as well too is that it's not reportable. We're still dealing with something that is not reportable. And maybe you could make an argument that for Southern Ontario or Toronto or Ottawa or Kitchener, maybe it doesn't need to be instances so low. But what about in our Northern communities where this is still affecting people's lives? You know, our last paragraph of our article that we submitted, I just put it up here because this is not new. We said our case series serves to demonstrate as detailed by the Auditor General of Canada. In 2015, the Auditor General of Canada did an audit on how our health services in the North, in Northern Indigenous communities. And what did the Auditor General find? The Auditor General said that Health Canada has not adequately managed its support of access to health services for remote First Nations. And we remember the Truth and Reconciliation Commission. We also said this community-based research also highlights areas identified by the Recent Truth and Reconciliation Commission. This was in 2015, in which we must close the gaps in health outcomes between Aboriginal and non-Aboriginal communities. And action is urgently, urgently needed. So when I look at rheumatic fever, I don't look at rheumatic fever just on a pathophysiological sense. I look at it in terms of there are justice, fairness, and equity issues that need to be urgently addressed. Because we will, unless we address them, we will always have complex cases of rheumatic fever. And we will always have children, unfortunately, dying of rheumatic fever. Because the solution is not exclusively medicine it's also equity. We have to make sure that the tenants of justice, fairness, and equity are being applied to everyone. And as long as we don't have the proper provision of those, we will always have rheumatic fever. We will always have rheumatic fever. You know, one of the things that we need to watch out that we don't do, and I'm guilty of this, is that sometimes we can normalize dysfunction, right? Like I remember at a time, you were seeing so much rheumatic fever that it almost became normal. I remember when the residents would come up and we would present and they'd be like, I think this kid has rheumatic fever, I've never seen this before.  We knew the penicillin goes off by heart. We knew when they would get their echo. We knew the criteria. In fact, we knew the criteria off by heart for both the high risk environments, right? So we just knew those things. So we were just splurging out all this knowledge about rheumatic fever. But what I found I was doing is I was normalizing dysfunction. And the problem is is that when you normalize dysfunction, you don't see it anymore. It's just, it's another case of rheumatic fever. It's another case of rheumatic fever. It's another case of rheumatic fever. And what happens is that when you do that, you lose the ability or your ability to ask the why becomes impaired. So I want to encourage all of us that when we're hearing about these things, we ask ourselves the why. Because sometimes when you're in an environment where you see this all the time, you can normalize the dysfunction. You can normalize the fact that there's rheumatic fever in high incidence still in Canada. And then you just don't see it anymore. You know, sometimes it's not dysfunction that's problematic, but when you normalize dysfunction because when you normalize dysfunction, you can't see it, right? I want to encourage everyone as well is that you know what I want? I want change. I'll be quite honest with you. I want change. Like it is, it bothers me to know that here in 2022, almost 2023, we still have issues with rheumatic fever. We still have children dying of rheumatic fever. We still have mitral valves in children that are not good with rheumatic fever. And I always ask myself like, you know, what we're gonna have, how are we gonna need to get some change? What is going to need to happen to get some change? And you know what? Change doesn't happen exclusively by education, but change comes by challenging belief systems and practices. That's how change is gonna come. We have to challenge a system, a healthcare system that unfortunately has in some respects systemic racism built into it against indigenous people. We have a system in this country right now that is very effective at triaging people based on race to vastly differential levels of care. And we need to challenge those systems. We need to take a look in a mirror. I remember an elder once said, we've all been dipped in a little bit of colonial tea, whether we like it or not. We all have received an education in a colonial system and that can potentially affect the way on how we look at other individuals. So I challenge everybody today that, you know what, what are we gonna do to get a bit more justice, a bit more equity, a bit more fairness in our healthcare system right now? So maybe the next time this perioperative conference comes up, that we can say that the rates of rheumatic fever are down, not because we not only have excellent access, but we also are addressing some of the primordial issues, housing, overcrowding, lack of clean running water. All right, thank you so much. I think that's about my time. I hope I didn't go over, I apologize for that. Yeah.


Hi, everyone. Welcome to this SIG multi-cluster intro and deep dive session at KubeCon. I am Stephen Kitt. I'm a software engineer at Red Hat. And with me is Jeremy Olmsett-Thompson. I am a software engineer, and I work on Google Kubernetes Engine. Welcome to our maintainer track session at KubeCon North America 2023. In this session, we'll be covering what this SIG is about generally, the problems we're working on, what we're thinking about next. And we'll spend some time bringing those of you who are new up to speed on current activity. There's a bunch of projects that we're very excited about happening right now. We're seeing a lot more engagement from folks from different groups, from different backgrounds. It's an exciting time. And so we'll get through what are we covering right now in the SIG that we've got a new website. We'll talk about a cluster set and what that is. We've got an API, I call it the About API, for metadata, our multi-cluster services project, and exciting new project right now around building blocks for orchestration in the multi-cluster space. And then most importantly, we'll cover how you can contribute. So what is this SIG about? In traditional Kubernetes, you have single clusters. And we want to think about a bigger picture than that, where you have multiple clusters and you want to do interesting things with them all. So that involves, of course, initially being able to expose workloads from multiple clusters to each other using tools that are different from the traditional ingress-egress models in Kubernetes. You also need to share cluster metadata so that clusters know about each other and how they all individually fit into the bigger picture that you're trying to build. And just in general, break down all the walls that exist between clusters in a controlled manner. This involves lots of different functional areas, lots of different usage scenarios. But what we want to do as a SIG is identify what the core primitives to describe all this are, what the semantics are that are universally applicable across all the scenarios that we want to be able to describe, and what the best way of describing all this is in the way that enables interesting use cases without imposing specific implementations. And we really need what we want and we need your input from people who are trying to use multiple clusters together, what your use cases are, what your stories are, the difficulties you run into, or not, what you find helpful when you're trying to think about all this. And we're really interested in finding out what you're working on right now or what you plan on working on in the future. So I'll take a minute to talk about our approach as a SIG. And I think this has evolved over the years. The most important thing we've learned is to avoid premature standardization. The Kubernetes community has gotten really, really big. This is consumers. This is people managing their own clusters. This is cloud providers. This is tool builders. Flexibility is definitely needed. That's one thing that's become clearer and clearer over the years. And we just see it more and more evident. So the idea is let's focus on APIs. Let's focus on building a common language that we can use to connect clusters, to connect concepts that enable future tools to be built, that enable new patterns and practices, not the implementation so much. As different clouds, different environments have different networking constraints. As an example, it can be very hard to standardize there. There are so many different patterns that evolve, whether you're on-prem, whether you're cloud-based, whether you're doing something interesting that we haven't thought about running Kubernetes on Raspberry Pis. You name it, there are so many ways that Kubernetes can be deployed. Focusing on implementation, especially when stitching clusters together can make the problem space too constrained. So we really focus on those APIs and the core problems. So this is another piece. Avoid solving optional problems. This is super important. It can be compelling, especially for engineers, especially for me. I can speak from experience on this to try to solve all the problems. But getting to something useful that we can all agree on with these diverse backgrounds is much, much easier if we just start with the core problems, the problems that everybody has, no matter what. And figure out how to adjust the optional problems in the future as they become less optional, as the core solutions are consumed, and we see what's next. We also have a goal of keeping multi-cluster as consistent with single cluster as possible. I think this is important for just expanding existing tools, not coming up with new concepts that are different names to different ideas, but roughly overlap with things in single cluster. So wherever possible, we reuse concepts. And then the last piece is to try to work backwards from those specific problems into something bigger. And today, we'll talk about how some of the work we've started with, one problem, has kind of evolved into bigger problems involving more SIGs and more people. And you'll find a lot of this information on our nice SIG website, which is multi-cluster.sigs.kates.io. And this has higher level documentation for end users and also product status updates. And the goal is really to describe the different APIs in the SIG cares for, what they provide, how they can be used, and also to help connect implementers who are trying to implement, well, provide real-world services that provide these APIs to help them understand the goal of the APIs, provide also descriptions of our tooling. And once we, for all those implementations that do exist, catalog them so that end users can find them. And we're really interested in feedback on this website. What are we missing? What are the patterns that we should be describing? Because currently, it's mostly about the, well, like I said, what the APIs are, but not so much about how they all fit together and how they are actually used. So we're very interested in feedback on that. And open questions, of course, problem spaces that fit in with the SIG's concerns and that we're not addressing and ways to use all the existing APIs that we haven't described. So now I'm going to talk about the cluster set. The cluster set, this is a concept that we came up with a few years ago. It's basically under the realization that we needed some basic consistency in multi-cluster to start figuring out what we could build next. So the cluster set is the concept we came up with. And this is basically the core group of clusters that the concepts and projects that we work on in SIG multi-cluster rely on. So the cluster set represents a pattern of use from the field. And this is learning from talking to operators, from our own experiences, talking to customers, users, everyone who's been around Kubernetes trying to deploy multiple clusters, learning from what worked, what doesn't work, the pain points we've seen. The key here is that a cluster set represents a group of clusters that are governed by a single authority. Now, the scope of this authority varies in deployments. It might be in an organization that has their own cluster set. It might be a whole company, or it might be the company's platform admins. The important thing is that there is an authority who can make strong statements about the cluster set. Within a cluster set, there's an assumption of a high degree of trust. This is not external consumers. Kubernetes supports that well with Ingress already. If you don't care that the consumer is a Kubernetes cluster and you don't care about that metadata, that you can get from work clothes deployed within Kubernetes and you want to treat the consumer as external. That's great. That's been well supported for a long time. We're trying to solve the problem for within that organization when there is some degree of trust, when you do want to know more details about each consumer, when you might care about identities and access control consistency within clusters. And the core concept that makes this work is namespace sameness. The idea of namespace sameness is that a namespace basically means the same thing in every cluster in which it exists within a cluster set. Permissions and characteristics are consistent across clusters. This is really important. So if I can access namespace foo in cluster A and namespace foo is also in cluster B, I should be able to access both. If a namespace is used for a web service front end, it probably shouldn't use the same namespace for payment processing in another cluster. So same use case, same characteristic, same permissions. Namespaces don't have to exist in every cluster. That's completely fine. But in the clusters in which they exist, they should behave the same. And even you may have some namespaces that are like the default namespace that just have all kinds of things in them or system namespaces that our cube system is inherently local. That's OK as well. So long as if those namespaces are local in one cluster and don't have fleet awareness, they are local in every cluster and don't have fleet awareness. So the important thing here is consistency. The idea is that cluster set and the namespace sameness building block gives us one step closer to cluster fungibility, the idea that you don't have to care about the specific cluster that you're working in, that clusters may be replaceable. By relying on namespace sameness, you know that if you can access a namespace in the cluster, it's always going to be the same. And that takes a huge operational load off is what we've found. Now, given that, we just need to figure out how to solve cluster identification. And this is covered by the Abite API, which allows you to attach metadata to individual clusters. The cluster inside Kubernetes is such a fundamental concept. And it's assumed at such a basic level that Kubernetes itself doesn't actually have a way of talking about clusters at all. So in traditional Kubernetes, a cluster doesn't even have an identifier. And this is addressed by the Abite API. And you can see here, we have all the individual clusters within our cluster set. And they all have a way of attaching identification information and associated metadata to each individual cluster. So you can see all the four clusters are part of the same cluster set. But they all have different names and then additional metadata, which is added on top. The Abite API is described in the Kubernetes Enhancement Proposal Number 2149. It's available at the Abite API site. And it's currently in beta. It is a cluster-scoped cluster property CRD with a very, very simple schema. It's just name and value. And you can store cluster identification and any other properties that you're interested in that are cluster-scoped or rather that are attached to the cluster as a concept. And they replace annotations that you could typically find on objects that are similar or at least feel similarly scoped. So there are some well-known keys that are used as names for the cluster ID, for example, or the organizational ownership which cluster sets a cluster belongs to. So that's clusterset.case.io. And where it is in the overall world of your cluster sets, where it is on the network or networks, which environments it's used for and what the cluster's purpose is. This is really a core building block for higher-level orchestration. And we'll get back to that in just a little while. But before that, I'll talk about the multi-cluster services API. So earlier, I mentioned how we had built some tools to connect clusters together and try to break those walls down between clusters. And the multi-cluster services API was kind of the first big push of the SIG into this space. It was developed under the realization that if you want to connect clusters together and deploy applications across clusters with dependencies across clusters, services are kind of the core multi-cluster building block. You can't do a lot if you can't talk to other applications. So we decided to start by asking ourselves, what are the core problems in multi-cluster service discovery? And how might we extend the service concept, the existing service concept across clusters, really targeting that kind of high-trust east-west case? And what we came up with was a very simple API, actually. It is called a service export. And it's the responsibility of an implementation to figure out the actuation. But the idea is that by creating a service export resource, which is really just a resource name map to a service, we allow a single Kubernetes service to span or be consumed by multiple clusters. When we did this, we focused only on the API. And this is because a lot of platforms have very different traits. And there are a few implementations out there now on different platforms that have various constraints. The core of all of them, though, is that you can create a service export resource. And then that service becomes consumable across all of the clusters in the cluster set. And this is very powerful. One of the constraints we came up with with the API is that consumers should only ever rely on local data. This means that regardless of the implementation, whether it's Submariner or GKE, which I happen to work on, or any of the others, you can consume it just by seeing what's in your cluster. And we have a corresponding service import resource that's created in a cluster as a service is actuated in that cluster that makes it easy to consume. This is a very important resource I'll talk about in a minute. But the result is that cluster IP services and headless services basically just work as expected across clusters, which means that if you've built a multi-tiered application with dependencies on other services, you can start moving those services around across different topologies. And everything continues to work. So it makes it very easy to move from that single cluster model to a multi-cluster world. And that service import resource ended up being even more useful. So we worked with SIG network and the folks there on the Gateway API. We actually figured out that we could get the Gateway API to point at the service import and take the Gateway concept, the new Ingress API, and allow it to point to multiple clusters. So this becomes a really powerful tool with multi-cluster services. We sell that east-west cluster to cluster communication. And then with the Gateway API added on top, we make it possible to consume these services externally without having to care about the topologies and the clusters underneath. So we've talked about so far having services, having clusters that are described. And that works really well if you have an existing set of clusters that have existing services on them and you want to be able to build workloads that use services across different clusters. Using the MCSC API, as Jeremy just said, you can describe which services are available where. And thanks to the main space seamless principle, you know who can access what. And your workloads can have certain expectations about what they're going to find, even though they don't necessarily know or even care about it where things are, and they shouldn't have to care about it. But in practice, that's all well and good when you already have things in your clusters. But when you're trying to build bigger systems across multiple clusters, you quickly end up with a desire to be able to drive changes across those clusters. And this is where orchestration comes in. And this is often talked about as federation. And the goal there is to be able to have some central component that says I want to see this state, let's say, or these features available in these different clusters. So here, for example, to be able to say I want these services in green to be available on these two clusters. And so you're pushing configuration into all the clusters in a cluster set. There have been multiple attempts to address this. The first one was cluster registry, but it didn't really have a well-defined use case. And so it faltered. Another attempt, which was more based on an actual concrete implementation, was KubeFed. And this did have some use, but it ended up trying to address too many concerns. And again, the development stalled because it was too difficult to determine what the core features of this were supposed to be and what a well-defined set of use cases that it was trying to address. And so obviously, since there have been two attempts already, and all the implementers of multicluster solutions have their own private way of dealing with this, there is clear demand for multicluster orchestration. But it's also obviously very hard to define that in a way that makes sense to everybody and where you end up with APIs that work across different types of use cases, across different implementations. And so we're really excited to see a new proposal to try to address this, which is called the Cluster Inventory API. This is backed by a number of different groups and projects being discussed in this SIG right now. And it tries to provide a way, well, to see here are certain workloads that I want to have on different clusters. And it tries to address the problems that come up with that in a fashion that's consistent with the SIG principles where we, as we've described already, try to come up with an API that addresses the core concerns without enforcing implementations and so on. And there is a real opportunity to help influence this proposal to help give it meaning. And we are really interested in feedback on it. So please come work with us. So what's next? And this is really kind of where we look to you. We've been thinking about a few things that come up as recurring themes. But a big one is canonical patterns. What should we be encouraging the community to do with the tools that we've built now that we have them? What are the recommendations we can make around patterns or workflows based on our experience, based on folks using our tools and familiar with the space? What has worked? What should we be sharing more broadly? Another area that's come up many times now is leader election. I think once you have deployments across clusters, especially once you start having this orchestration, there's going to need to be some coordination required, some form of consensus. How do we manage, how do we coordinate in multi-cluster deployments? And then the big open question, what else do we need? We want to shift operations above the cluster and across clusters. What are the other tools that are missing to make that a reality? This is, as we've already said several times, where you come in. How can you be a part of the SIG? We have some well-established APIs, things that we are also working on. But we want to know what you are working on. You've got real scenarios, real use cases that you're trying to address. And if you're making progress towards actually implementing them, or if you already have something, then you've got tools that you've built or that you are building. You've also identified tools that would be really useful but that don't exist. And we are very interested to find out what those are. What problems have you solved? But what problems are also left to address? What problems do your customers have? What are the big missing pieces that we haven't even talked about that you've noticed? And well, is there anything that the SIG can do to help you? Or is there anything that you can do to help the SIG? And we've talked about multiple different areas already in this presentation where we are looking for feedback or looking for help to give direction to future proposals because we are well aware that the current APIs that exist and that the SIG has either defined or contributed to don't provide a complete solution for multi cluster use cases so far. So we are very keen to get more input from more people. And we have a number of places where you can come and meet us. So there's the home page that we already talked about where you can find out more about the SIG and what the SIG has done and what the SIG is working on. And if you want to actually come and talk to us, we have a Slack channel and you can see the address there. We also have a mailing list that you'll find on Google groups that you can join. And it might not seem all that active but there are people reading it. So if you post messages there, you won't just be shouting into the void. And we have bi-weekly meetings on Tuesdays at 12.30 Eastern, 9.30 Pacific, 4.30 PM, UTC. And they are usually well attended. They do get canceled if there is no agenda. So if you want to talk about things, be sure to add your information to the agenda. And you'll find the information about where that lives on the home page. So as usual with Kubernetes SIG activities, the way to find out about all this is to go to the website and join the Google group. And that will automatically push invitations to you for the bi-weekly meetings. And the meeting invites have links to the agenda that you can fill in ahead of time. Or just look out to see what we're going to talk about. And you'll also find there are notes from previous meetings so you can see what's been discussed in the past. And you'll also get a link to the Zoom calls so that you can actually join and practice. Thank you. Thanks, everyone, for the time, for listening. Please post your questions. We'll be standing by for the next few minutes to answer them. And enjoy the rest of KubeCon. Yeah, thank you.


Hello and welcome to the session. In this session we will discuss two variables using frequency and relative frequency. Now suppose Jenny conducts a survey in her class to know how many students have a computer or a video at home. Now there are five possibilities of getting the response when the first possibility is a student has both video and a computer. Second possibility is the student has a computer but not a video. Then third possibility is the student has a video but not a computer and the first possibility is the student neither has a video nor a computer. Now if the total number of students surveyed are fourteen then the information gathered by Jenny can written in the form of a table. Now to read the information easily we construct a two way table. It means both rows and columns together are giving the information. Now here let us take computer response in row and video response in column. Now suppose she gets the following table. Now let us see the method of reading this table. Here the number of students for each response is written in the cells. Now in cell 1 the number is 22. Now here see the corresponding row and column. Number of students having a computer and a video is 22. Now from this table we can find how many students have video and not a computer. For this first of all we will see the second row which represents the number of students who do not have a computer. But here we have to find the number of students who have a video and not a computer. For this we will check the column which represents the number of students who have a video. So here this cell tells us about the number of students having video and not a computer. So the answer for this question is 8. This means there are 8 students who have video but not a computer. Similarly we can find other number of students. Now here we can see that 6 students have computer but they do not have a video and 4 students neither have video nor computer. Now here you should know one thing that these numbers are the frequencies of the collected data. Also here the sum of all the numbers should be equal to total number of students. Now let us see the sum of all these numbers which is equal to 22 plus 6 plus 8 plus 4 which is equal to 40. And we know that here the total number of students are 40. So sum of these frequencies is equal to total number of students. Now from this table we can also find the number of students who have computer irrespective of having a video or not. Now let us see this table again. Here if we add the frequencies given in the first row it is 22 plus 6 which is equal to 28. It means 28 students have computer. Similarly adding the entries in second row we get 8 plus 4 which is equal to 12. So here 12 students do not have computer. Similarly if we add the frequencies in first column then it will be 22 plus 8 which is 30. And if we add the frequencies in the second column then it will be 6 plus 4 which is equal to 10. It means 30 students have video and 30 students do not have video. Now here you can see that 28 plus 12 is 40 and 30 plus 10 is again 40. So here we can see that total number of students are 40 and out of these 40 students 28 students have computer. 12 students do not have computer and out of these 40 students 30 students have video and 10 students do not have a video. Now let us see related frequencies or percentages using two way table. Now suppose we have to find what percent of students have computer and a video. Now from the table we know that 22 students out of 40 have computer and a video. So related frequency is equal to number of students having a computer and a video which is 22 upon total number of students which is 40. And this is equal to 0.55. Now let us see where and column related frequency or percentage. Now suppose we have to find what percent of students having computer do not have a video. Now this statement is not same as percent of students having computer and not video. Here first of all we have to find the number of students who do not have a video. Out of those students who have computer. Now here we know that the total number of students who have computer is 38. Now in the first row see the number of students who do not have a video and they are 6. So the required percentage is 6 upon 28 into 100 which is equal to 21 approximately. So here 0.21 is a row related frequency. Now let us find what percent of students not having video have a computer. For this we see only those students who do not have a video and they are 10 in number. Now in this column see the number of students having computer and they are. So the required percentage is equal to 6 upon 10 into 100 and this is equal to 60%. So here the column related frequency is 0.6. Relative frequency or percentage of each cell then we get this relative frequency table. Now we can find the relationship between two variables whenever we are given. Now for categorical data we mean the variables can be divided into categories like colour of girls, wet, green, blue. Now we can find the relationship by making the percentage or relative frequency table. For example let us study this two-way table. Now here for each age group what percent of students in the survey write the bus to the school and what percent of students do not write the bus to the school. Organize the results in a two-way table. Double show any relationship between the age group and the students writing bus or not. Explain. Now let us start with solution. Now where we have given the age group and we have to find percentage of students writing or not writing. It means here we have to find the column relative frequency and hence percentage for each cell. Now we have age group in columns and write by bus or not in rows. Now number of students writing bus in the age group of 12 to 13 is 24. Now in cell 1 the frequency is 24 and the total number of students in this age group are 40. So the required percentage will be equal to 24 upon 40 into 100 which is equal to 60%. Similarly 16 upon 40 into 100 is equal to 40%. So the percentage of students writing bus in the age group of 12 to 13 is 60% and the percentage of not writing the bus in the age group of 12 to 13 is 40%. Now in the second column the total number of students are 25 in the age group of 14 to 15. So percentage for this column is 12 upon 25 into 100 which is equal to 48% and 13 upon 25 into 100 which is equal to 52%. We can find the percentage for the third column. So we have the following table that as the age group increases the number of students travelling by bus decreases. Now here see the percentage that is 16, 48 and 40% they are decreasing for each age group. So the number of students not writing the bus increases with increase in age group. So as the age group increases students are less likely to write by bus. So in this session we have learnt about two-way tables and this completes our session. Hope you all have enjoyed the session.


Okay, so we'll not waste any more time and get Timothy Terry Berry start with his talk on video products All right, so thank you. I'm gonna talk about the Dala video codec project, which is a joint project between the ZIFT.org Foundation and Mozilla And so if you're all at the keynote yesterday you heard Evan tell us that patents are no longer a problem for free software So great. I'm done like I can go home Yeah, so The situation has gotten a lot better I'm not gonna call Evan a liar but There are still places where these things are real problems and codecs are one of them So he mentioned that the the open invention network is a a fantastic tool for free software projects to defend themselves against patent aggressors and while it does a lot of very good things if you go read the agreement somewhere buried deep in all of the legal ease it Has this nice phrase where it points out to the extent that any of the Linux environment components identified in this edition Contained audio still video and or motion video codecs such codecs shall be deemed not to constitute a Linux environment component Which is a good example of why you should not let lawyers near English, but I Also serves illustrate that Even all of these tools that we have managed to build up don't help us in this space So why is it worth solving this problem anyways? And maybe this yeah, just this one codec thing is is we can just ignore that and it won't be a problem Right. We've got the rest of the space is fine Well, the problem is that encumbered codecs are a billion dollar toll tax on communications so Every cost that gets added to a codec, even if it's only a few cents gets repeated a million fold in all multimedia software You go look at the modern cell phone every single component of that cell phone is getting cheaper every every year With one exception and that's the cost of licensing patents and licensing codecs All of this licensing is anti competitive Right, so you hear people Talk about fair reasonable and non discriminatory licensing. I've had very well respected lawyers in this field Tell me they don't know what that means Because it doesn't actually mean anything like the the specific point of a good deal of codec patents is to Create these discriminatory regimes in commodity hardware businesses And so the idea is that you have to pay less to license the patents than your competitors because you happen to own Someone and can get these cross licensing deals and now new people can't come into your space Because the margins on these things are so razor thin that that few extra senses all it takes to make them unprofitable and you profitable They're also an excuse for adding proprietary software to your system, so how many people here have flash on their Linux system Yeah, yeah, there are a few hands and When you start relying on proprietary software for communications You lose the assurances that you get for using free software on the privacy of those communications You no longer have a system that you can look at and validate and guarantee that it isn't actually respecting your freedoms So it speaks to that second problem that ebb and has has turned his focus to as well and So you can say well, but we also have free software implementations of all these proprietary codecs, too, right and We can just use those and just not pay the patents and that won't that nobody nobody's gotten sued over that yet, right? The the problem is is that ignoring these licensing costs Creates risks that can show up at any time and so that works great until you actually build your business around this and you You know sell a few million units and then somebody shows up with their hands out and you get this tax on success right and You'll see that that companies like Skype for example famously used VP 8 and VP 7 before that To build up their their business and it wasn't until they got It wasn't until they got large enough to be able to absorb these licensing costs that they actually switched over to codecs like H.264 But if they had not had the opportunity to use VP 7 and VP 8 they had never have gotten started so Creating good codecs is a challenging problem However, we don't need many of them So ideally we only really need one And the free software Ecosystem is up to the challenge of making them if you go look at the proprietary patenting codecs All the best implementations of them are already free software You may not be able to use it without going and getting a patent license from someone else But the the ability for the open-source community in the free software community to build these things is already there The basic thing that we have to overcome are these network effects, right? So the the primary costs to deploying codecs are Not necessarily the licensing costs and all of that, but the compatibility problems with everybody else So if we want to be able to to build Use free codecs in all of our products. We have to make sure that everybody else uses them also Where royalty free codecs are established Non-free codecs have never displaced them and so some some good examples are JPEG and PNG and FLAC There have been many different competitors that have tried to displace JPEG and None of them have even tried to charge licensing costs All of them have claimed to be technically better than JPEG and they still haven't succeeded just because JPEG is so widespread This is not necessarily true for royalty bearing codecs there have been good examples of those being displaced by royalty free versions But being royalty free is not enough right, so everyone in this room hopefully cares about about Avoiding other people's IPR, but other people out there care about different things and say yeah I can afford to buy a codec in my business, right? What I want to get is the best quality per bit or the simplest integration into my application, right? And so you have to be good on all of these fronts. It's not enough to say Well, we're a little bit worse on quality per bit, but we're free right to get the kind of a network effects You need so that everybody is using these things. You really have to be better in every in every way So we've already done this for audio We made a nice codec called opus it was published in the IETF about two years ago and It can single-handedly replace ten different codecs across all sorts of applications and be better than all of them Despite the fact that when we started and we said our goal was to make a royalty-free codec and You know we wanted to cover some broad uses of applications But you know we were never expecting it to be quite to outperform everything because what we had originally targeted was We're gonna focus on low-latency communications They're gonna make a bunch of trade-offs that make that possible and that means stuff like high-latency Music streaming and that sort of thing. We're not gonna ever can be able to compete on Well, it turned out we were able to compete on that and It wasn't until we got the quality levels that we started to beat everything that people got really interested in this codec So that sort of illustrates the effect of you really do have to be better on a whole bunch of different fronts So opus is now the standard codec for WebRTC It's seeing lots of inroads in other places And so we really hope in the next few years that that this will start to displace a lot of the proprietary audio codecs Which by the way are generally even more expensive than video so Hopefully since we've succeeded in audio we can now turn our attention to video and so the latest generation is Is being fought between HEVC and VP9? Our goal is to be better than both of those without infringing HEVC's IPR So in order to do that We need to not only make a codec. That's really good We need to convince people that it is safe to use and to do that. We need a better strategy then Well, we're gonna go read a lot of patents The problems with that is people don't believe you when you said I read a lot of patents It's okay You know, we've been doing this for I don't know 13 14 years for me personally some some of us in the Zift Auto foundation for almost 20 years now and You would think after that point you would have earned some amount of credibility that yes We know how to design a codec that doesn't read on other people's patents But when you ask somebody to actually ship your codec, they stop and think really hard for a bit And This is you know with not without good reason because doing the kind of analysis is to say that you know There are thousands of patents in this space and we don't infringe on any of them is really error-prone, right? We try to stay fairly far away from the line of what's permissible and what's impermissible But the single mistake can ruin years of development And so a good example of that is actually the H.264 baseline profile Which when they started standardizing this in 2003? They said we would like to make this royalty-free and they got a whole bunch of companies to sign agreements and saying yes This is going to be royalty-free. We will put our patents in if everybody else does Only they didn't get the people who gave who submitted the initial proposal For 264 all they got were all the people who added additions on top of that and the people who has submitted the initial proposal said No, we don't like that idea. And so that's now not royalty-free And There have been various attempts to make it royalty-free you know to try to convince those other parties over the years and we are now you know 12 years later and it's still been unsuccessful and so all the work that people did when they thought they were making this nice free royalty-free codec You know has been locked up behind these patents so our strategy is to try to look for some Some elements that are common to broad classes of patents And so I'm not going to go into a great detailed explanation of how you work around patents If you were in Wellington five years ago Andrew Trigel gave a fantastic talk on how to do just that And I all recommend you go watch the stream if you have not But to To give a short summary, you know Every patent claim is going to have some long list of elements in it And we only need to avoid one of those elements to be able to say we don't do that Which is the best defense you can have to avoid a patent and so What we're going to do is try to identify some elements that show up again and again and again in many many different patents and Replace those with fundamentally different techniques and the idea is that we don't necessarily have to have read every patent And we will still try to do that But if something shows up out of the woodwork some number of years from now There's a good chance that we won't read on it simply because our design was so far out of what they were considering when they filed the patent That it does not apply This is a generally higher risk higher reward strategy Then then the sort of incremental change approach that a lot of codec development has followed for the past 20 years And so people have taken these things and you show up at a big meeting with a hundred different people proposing slightly incremental changes that each get you one in two percent improvements and each one of those was of course patented by your employer and You combine all these things together and at the end of the day you say well look our codec is fifty percent better than the previous one And you can go to this well a few times But eventually you're going to start to hit diminishing returns and it isn't getting harder and harder to find these incremental changes that improve things So we are trying to take sort of this higher risk approach of we're going to go in some fundamental different directions and That will potentially get us out of these local minima At the same time it'll help us avoid wide swaths of these this IPR that other people have have already filed and It's sort of its own reward in the sense of it creates new challenges that other people haven't solved You know if I want to implement technique a and technique a has you know some fundamental engineering challenges associated with it And I go patent all the solutions to those challenges and Somebody else comes along and says I'm going to implement technique B and technique B is not like technique a and it doesn't have the same problems associated with it You know this is sort of the The there is a whole class of inventions where people say you know to to state the problem is to state the solution, right? and a Good patent lawyer has always always trying to patent the problem rather than the solution And by using these different techniques, we are actually trying to solve different problems And so we avoid lots of IPR in that way as well Now all that said we still have to read a lot of patents Because you can never be certain about any of this stuff until you actually do your homework So I said we want to we want to do some things that are fundamentally different So we started out by identifying about four key areas that we thought there were good credible technological alternatives to Then I'm going to talk about each of these and in more or less detail depending on which one So the first one is this idea of a displaced frame difference, which is used in motion compensation If you don't know what that is we're going to talk about it in a couple slides The second one is the this idea of doing adaptive loop filtering which is used to remove blocking artifacts So if you've all seen low-quality video you've seen block edges show up at low bit rates And this this is one of the techniques that's used to avoid that so the next one is spatial prediction or inter prediction and The idea there is that if sometimes you have to code frames without referencing other frames And so you want to predict you want to predict one area of the frame from something that's nearby And finally binary arithmetic coding specifically context modeling and this is this is the way that the final bits getting coded into the bit stream and We will have a different way of doing that as well So let's talk about the displays frame difference so The idea behind motion compensation as traditionally done in most codecs is that you have some Some reference frame that you've already decoded and you want to predict your input frame And so what you do is you copy blocks from the already encoded frame and they're offset by some motion vectors You can count for things moving around the frame and when you've copied all those blocks You subtract them from the input and you're left with this residual and the residual is now much closer To zero and has lots less information in it You know there's still some errors where you weren't able to predict things very well But the idea is that this is where a good percent of your compression comes from The displays frame difference is the term of art for that residual and You know it in in of itself. It's not patentable. I mean people have been using this for 30 years So I'm sure somebody tried to patent it 30 years ago, but at this point you could say it's fairly well clear, but a lot of the other details of Well, how do I code those motion vectors that I'm using to copy the blocks from and if the If the motion vector doesn't point to an integer offset, how do I do the interpolation around that? to to copy those blocks and You know all of these details around the idea of motion compensation. There are many many patents around them But every single one of them to to a larger they look greater or lesser degree says at some point we do a subtraction and compute this displays frame difference and code the residual and we actually filed our own patent on what we're doing and The technical writer that had been assigned by the law firm to write up the details of our patent added the step of subtracting the predictor and Coding the residual to our diagrams and to our claims and we had to go explain to him No, we're not doing that and he he honestly could not understand how you could have a codec that didn't do that So I'm gonna help you try to understand So what we do instead Starts with this idea called perceptual vector quantization and This is based on work that we we did in the opus audio codec though with the idea behind it being to preserve energy and so when audio preserving energy, you know specifically refers to the kind of audio textures that you normally hear and and You know different tones and different Impulse responses and all that stuff In in video it means stuff like film grain and fine details, right? So we don't want to smooth everything out now The way this is done is that we try to separate The energy or the gain of the information we're trying to encode from the shape Which is you know this how it is distributed in frequency space in the spectrum And so what that's all really boils down to is that we have a bunch of numbers We'd like to encode and we put them in a vector and we take the magnitude of the vector the length of the vector And we code that separately from the direction the vector is pointing in which is just some point on a large n dimensional hypersphere So this has a bunch of potential advantages We can give each of those two different components a different number of bits So we have the potential to say there's some energy here But I'm not really going to tell you where it is and this is very hard to do in traditional approaches But that lets us You know preserve some of the contrast of the image that we've been fed Without low-passing everything even if we're throwing away some information in the process We also get this idea of activity masking for free And so what activity masking tries to do is it says we have these regions of high contrast We can throw away more information than we would be able to otherwise before you notice and because the relative error that was of the You know the inaccuracies that we're introducing by doing that is smaller because the numbers are starting off with our bigger right and So the thing you need to know to know how much you can throw away is Exactly this gain this energy which tells you how much contrast there is and So other codecs when they try to make these kinds of adaptive decisions of how much stuff to throw away They have to signal that separately in some other bits, and we don't have to signal anything separately. We get it for free It also winds up being just overall a better representation of these coefficients And so if you think of things like a fade to black You can see how having a single number that represents the magnitude of the information you have You know fits that kind of of change much more easily, right? I can just make that number smaller and the whole thing fades Whereas if I were coding each of those numbers independently, I'd have to change everything every frame So what does all that have to do with splice frame differences? The problem is is that if I subtract my predictor and code a residual I've lost this idea of the energy, you know the energy of my residual was completely unrelated to the energy of my original signal So all of the advantages I got disappear But we really want to do prediction right as I said this is where most of the Compression benefit comes from because they do a great job of reducing the amount of information we need to code So what are we going to do? Well, you step back and look at say what is this prediction actually doing? What it's really doing is changing the probability of Points near the predictor Right, so I'm not going to go into Shannon's entropy theorem and all that stuff with you If you haven't seen it before you can go read Wikipedia but the the gist of it is highly probable things are cheap to code and If you're computing a displaced frame difference what highly probable means is near zero After I've done the subtraction most of my stuff is pretty close to zero And so I can do a very simple model if it's if it's near zero I'll use very few bits and if it's far away from zero. I'll use lots of bits and that's what gives me my compression So we can do the same thing already with with these gains, right? They're just single numbers and so I can subtract I can compute the gain of my predictor and I can compute the gain of the the pixels I'm trying to encode and subtract those and now I have a number near zero And that's that's very easy to fit into the same framework and this is okay because we're not subtracting pixels anymore We're subtracting the lengths of these vectors, which is something very different in terms of patent law, right? These details matter but Then I have the shape to deal with and it turns out that enumerating points on a sphere near some other arbitrary point is Really a hard problem because now I you know How do I how do I represent in the codec that the points over here are more probable? We actually tried to do this with opus and we spent I don't know almost five years toying around with different approaches to it and never really worked very well and Eventually after we finished opus sadly We came up with the idea that we could just transform the space to single out points near the predictor and you're all thinking well Of course, that's exactly what you do and It winds up looking a lot simpler than it sounds so let's go through a simple example So imagine I have this 2d production of my n-dimensional hypersphere and I have this input vector, right? And I've normalized this out so that I'm we're not talking about the gains here We're just talking about the shapes, which just means the direction of the vector is the only thing that's important and so I also have this predictor and The decoder knows what the predictor is because it's going to compute it the same way that the encoder knows it So I don't have to code anything extra to to tell the decoder what that is and Now I'm going to compute what's called a householder reflection that maps the predictor on to one of these axes And if you don't know what a householder reflection is that's something else you can go look up on Wikipedia But the basic idea is it's just a reflection plane that takes an arbitrary vector and maps it to some axis And so all the numbers in the vector are now zero except for one of them Which is one and this makes it very simple to deal with and so in order to deal with this what we're going to do is compute and code an angle between the input and the prediction and That angle is now what represents how close you are to the predictor and so we can model The probabilities of that instead of trying to model the probabilities of all these individual points on the sphere And then we code the other dimensions Somehow and I'm not going to go into details of that somehow But the point is is that you've taken this n-dimensional hypersphere and you've picked out this one direction and now I've I've coded this angle to say how close to this predictor I am and I'm left with this n-minus one dimensional hypersphere Which is kind of seeing 3d product in there and so I've just reduced my problem by one dimension and so this New parameter theta this angle is actually somewhat intuitive as it says how much like the predictor are we You know we started out We had n different numbers that we know we're all unrelated that we you couldn't really point to any one of them say What does this number mean right but now we've got two numbers that are actually relatively intuitive parameters One of them tells me how much contrast I have the other one tells me how close to my predictor I am and you can start to do intuitive things with this you know With when theta equals zero that means use this predictor exactly, right? And so things near zero are highly probable I expect my predictor to be good that should be cheap to code and that's where all of this this probability modeling of points near the predictor should be cheap to code is grouped into this one parameter theta the remaining m-minus one dimensions we code using vector quantization and so if you attended my my talk in 2009 on Kelt how many people were in Hobart? Yeah, so I'll refer you to that presentation on how all that works But the basic idea is that we know the magnitude of that vector that we're coding is is the gain times the sign of theta And so we can use that extra piece of information To remove one degree of freedom and save some save some bits by using vector quantization to code it But at the end of the day what we're doing now is instead of subtracting my predictor from my input pixels Which is essentially if you think of these as two vectors doing a translation What we're really doing is scaling the vectors and then applying a reflection and Whatever else you can say about anything that I just explained. This is nothing like computing a displaced frame difference And the good part is that this works Which is you know was surprising to us, too So the graph on the left there is showing the peak signal-to-noise ratio of using Perceptual vector quantization versus scalar quantization, which is what basically you quantize each inch number individually And that's what most traditional codecs do In this case, we're not actually turning on activity masking and the reason for that is because activity masking makes PSNR worse Because PSNR is a terrible terrible metric, but everybody uses it But we also look at various different perceptual metrics one of them which is called fast SSIM and This is sort of less accepted in the video coding industry But we find it does a good job of telling you for example how much texture you preserve in your original image Which is what activity masking is supposed to do and so we turn activity masking on we get that second curve over there Which shows that we are in fact doing much better And also we look at images and videos that we've encoded and they look better, too Which is a nice thing to double-check sometimes So that's the displaced frame difference There are as I said several other differences So one is this idea of loop filters You know the loop filters are a filter that you run across the edges of two different blocks to try to remove these blocking artifacts and They are adaptive right so the idea is that the filter the strength of that filter Depends upon the amount of difference across those edges So if you have some large difference then that's probably caused by something that was actually different in the image You were trying to code while as if you have a relatively small difference That's probably just noise that you added because you threw away a bunch of information when you coded it And so you want to apply a stronger filter there to get things smoother They're also not invertible in the sense of after you've applied this filter. There's no way to undo it And So these filters have been used for a long time including all the way back to h.263 and in our own codec fiora But they were very primitive and didn't actually work very well because they were trying to be extremely cheap on CPU Because when these codecs were designed, you know 15 20 years ago CPUs were not nearly as good as they are now so as CPUs got better people Designed more complex filters and there's been an explosion of these filters which work much better than the previous ones do but also have lots more patents on it and Since those patents are all less than 20 years old. We said we would like to avoid that whole mess and So what we do instead are use these things called lap transforms The idea of lap transforms is that we have one of these filters that runs between these block edges Just like the the adaptive filters But it's non adaptive and it's invertible Which means I can run forwards. I can run it backwards and get the thing back that I started with and So what being invertible means is that I can run the inverse in the encoder as a pre-filter and now I don't need it to be adaptive Because I don't have to worry about well There was really you know some difference there that that I shouldn't destroy with this filter Because I inverted it already and so it's in the encoder side and so I'm there's nothing to destroy anymore Right. This is not losing information the way that these the loop filter loses information The techniques date back to the mid 90s, which is just about perfect for us There are some patents in this space, but they're vastly fewer of them and vastly they're all much older Than the loop filter patents and so it's a lot easier for us to go through and analyze these things and figure out whether or not this is okay and We've had people come up to us and say oh, but there are patents on those things. You can't use that And we said go go look and they come back to us later and say oh, yeah, okay. That's fine And that makes me feel a lot better But the idea of this pre-filter is That what it's really doing is taking your image and making it blocky Right, and so this is This actually winds up helping compression in the sense of these these discontinuities across these block boundaries are Essentially free to code right when you're coding things a block at a time It doesn't matter what that there's a difference there and So we are taking advantage of that that extra freedom by making things more blocking in the encoder so that on the decoder side we can undo all of that and Get a blocking free image that has taken advantage of the fact that we could code blocks in the interior So one of the problems is that once we start using lap transforms This problem becomes harder and so this tool is called and said spatial or inter prediction And the idea is that you want to predict some block from the neighbors nearby that you have already coded and So the way this normally works is you explicitly code a direction that says I'm going to copy some my neighboring pixels along this direction and Then just extend that boundary Into the current block along that direction just by copying whatever pixels are on our inner near your neighbors, right? Well, that doesn't work with lapping because we can't pop copy the pixels until we undo the lapping But we can't undo the lapping until we've copied the pixels So what we do instead is we don't copy pixels. We copy transform coefficients and We've spent a whole lot of time trying to figure out how to do this and Still copy things in these arbitrary diagonal directions and there's a lot of fun math you can do that to try to to solve that problem and Ultimately, we couldn't make any of it work very well We can make it work fine if you had infinite CPU, but few people have that So currently what we're doing is something that is much simpler. We just copy things in the horizontal and vertical direction and For the chroma planes, which is the color information. We copy that from the the black and white lumen information Which basically means we don't have to care about coding a direction for it because we're trying to say is that you know the boundaries of different colored objects is are In the same place as the boundaries in black and white objects that you know that they're representing when you split up the signal that way So all of this prediction is not as good as copying pixels around But we make up for it in other places in particular the lap transforms give us an improved coding performance You know that we give some back when we do this inter prediction in a slightly less optimal way Finally there's binary arithmetic coding So arithmetic coding is the thing that you use to actually generate a number of bits based on the probability of some Simplier in coding. So this is what gives you the things that things that are higher probability or cheaper to code is because you run them through this process So What traditional video codex do is they only code binary decisions. So it's very cheap to code one symbol Because you only have to consider like am I coding a zero or am I coding a one and you wanted using different numbers of bits to Signal zero or one depending on the probabilities, but it's very easy to make those decisions However to code an entire video frame you need to code lots of symbols and this is an inherently serializable process So you can't paralyze any of this and this actually winds up being one of the bottlenecks in Hardware implementations of modern codex is you need a clock cycle high enough to run the thing through your entropy coder through your arithmetic coder To code all of these symbols So the idea of arithmetic coding and particularly binary arithmetic coding is not in itself Patentable any more the original IBM patents from 1979 are all expired But the things of doing probability modeling so when you have binary arithmetic code, you can do this with a one-byte lookup table the the ideas of taking these non-binary values that you want to encode and Chopping them up until they're until they're individual binary decisions that is also highly patented and so that's all of the stuff That we would like to avoid and while it turns out the original definition of arithmetic coding was not restricted to coding binary decisions So we'll just remove that restriction So we code values with up to 16 different possibilities Which if you think about it means it's equivalent to coding up to four different binary decisions all simultaneously It's slightly more expensive than doing things the binary way, but it's not four times more expensive Because a lot of the computational overheads of running this arithmetic coder are Per simple you encode and not per the number of bits you're running through the number of bits that come out the other end So what that effectively does it gives us four-way parallelism Which is good when you're trying to make these kinds of hardware implementations that need a clock cycle that can now be potentially four times smaller Which means less power Now we can't do probability model in the same way because we can't model 16 different probabilities with one byte We maybe we could but it wouldn't be very informative so Instead we use things like I'm going to assume that the distribution of my 16 points are roughly a Laplacian distribution or an exponential Distribution and then I will use one value the expected value or the average value of those of those points To model the whole distribution And so that keeps the amount of state we need to store relatively small We have to do a little bit of extra computation there to compute the individual probabilities Given that shape of that distribution, but this is stuff is not too expensive The other nice thing is that we're not doing this Binarization we're not converting things into individual binary decisions. We're converting them into hex and so all the code Patents around converting things to individual converting large numbers into individual binary decisions is Basically doesn't apply because we're doing something different We also often will take a Bunch of binary decisions that would have been coded, you know one at a time with a binary arithmetic coder We're all grouping them into one symbol Right, and so this is that is where we get some of our parallelism But people who are doing binary arithmetic coding don't do that at all so There are many more differences between the between our codecs and what traditional codecs do I don't have time to go through all of them, but I do want to give you a brief summary of how we're doing so far So this project is not done. We still have at least a year of development left but currently Using one of our perceptual metrics PSNR HVSM, which is HVS stands for human visual systems So this is something that tries to be a little bit smarter than PSNR Run over 19 different sequences. The red curve there is us The other two curves are x264 and x265 And so you can see that above bit rates of about a half a bit per pixel. We're actually already winning According to this metric, right? So this particular metric we find generally tells you how good you do on coding clean edges We have another metric that I mentioned earlier called fast SSIM Which tells you how good a job you do preserving textures and because we have these features like activity masking built-in We do a fantastic job there and you can see we're actually winning at almost all rates The curious thing here is that if you look at the blue and green curves You can actually see the relative position of x264 and x265 flip between these two metrics One of the reasons for that is because the the x264 developers spent a lot of effort trying to preserve things like film noise and preserve energy, you know given the tools that their codec had and So they're actually currently doing a better job of that than the x265 people That I expect will change as as 265 becomes more mature, but it's an interesting observation So we have a website Called are we compressed yet where we will actually run all these metrics on any git commit If you want to go hack on a repository, we will add your personal git repository and you can run them on these things This is all run off of of a node.js server running on on AWS So you get results back in a few minutes We got some nice details on that. So this was all put together by one of our interns this summer and If you would like more information about how dollar works, we have lots and lots of demo pages giving technical details So some of this is a little bit outdated for example that That part two there on frequency domain it inter prediction as I said, we eventually decided all that stuff really didn't work But the certainly the ones near the end are actually relevant to the current design And since I work for a browser company I can actually get we actually have the entire codec implemented in JavaScript That runs in you know using in script in So this is decoding in JavaScript We haven't done any special optimization for this and you can see it doesn't quite get 24 frames per second on this laptop but we have a lot of work that we can do to make this thing go faster and we also have New developments in the browser coming like SIMD and JS which will make let us use SIMD operations to accelerate this kind of stuff But you know, this is something that you can go to visit that URL at the top there and Run this yourself in your own browser so That's the end of my talk. Are there any questions? Yes Yes, they'll go around with microphones that saves me from having to repeat the question, which I forget to do all the time So what I do work at Google, I don't know actually much about what we're doing with codecs except for webm and VP9 So I was curious. Obviously your codec supposed to be better than the existing one. So that's one reason why you're doing this on the Partwards free and patent-wise is VP9 webm an issue as far as you're concerned or it's just that you wanted to do something new Well, and also it has a Google offer to help you guys if you are better because after all I think Google just wants a free codec, too Right. So yes, so Google does want free codecs So VP9 for VP9 their development strategy has been very much the traditional incremental one Like they did a bunch of incremental improvements over VP8 So we at Mozilla don't have an issue with believing that that stuff is royalty-free But the problem is that the rest of the world is not so sure Given that there were lawsuits against VP8 in Germany all of which, you know, as far as I could tell we're completely bunk But that's the sort of thing that gives people pause and this is what I mean by You know, you're not you not only have to do all the work to make sure that you're royalty-free You have to convince other you explain that. Yeah. Yeah, okay So I get that part and then you also hoping to be better while right hopefully being more trustworthy for people Yeah, so yeah, so my understanding is that Google has started work on VP10 We have been talking about them on trying to collaborate in the ITF to come up with some kind of joint standard And I'm hopeful that that will that will you know come into, you know As get started as a project sometime in the next month or so So certainly there's a lot of interesting stuff that they do in VP9 and You know that they have on the table for VP10 that we would like to be able to use So I hope look forward to seeing contributions from them for that. Okay, and it's something They sound interesting in putting into Chrome Android and all that all those platforms or you haven't had time to tell So they're for that yet Sorry, what would they interested in adding your codec in Android the Chrome browser and all that stuff or well Yeah, so we'll be interested that when it's done. Well, you're not there yet Yeah, there isn't a lot of point of getting it shipped in all these places until people can actually use it You know in the sense of we change it, you know every day So something deployed in JavaScript like this makes some amount of sense because you know at least you're the one Shipping the codec and you can make sure that it will always work But you know until we do that until we have a stable version It doesn't make sense to put it natively into the browser. Okay. Thank you You mentioned something about hardware implementations So do you have already some sort of beta version hardware implementation for this? And do you have something like like power envelope estimates on how it compares with the existing ones? Right, so we have not built our own hardware implementation We have paid a lot of attention to the hardware designed to make sure that doing such a thing is feasible and you know We'll fit into a reasonable amount of complexity So You know we've we've prototyped some things We had someone show up and basically you know Implement the transform stages on an FPGA and start to do some of that stuff for us We do not have a complete codec implementation and we're probably not likely to produce one ourselves just for reasons of resourcing Someone wants to show up and make one that'd be great This work is all really interesting who pays for it So the bulk of our the core developers are funded by Mozilla So we have a team of about six people right now in one manager and We saw we also have you know a various group of volunteers who contribute things as well Like the hardware implementation night, you know the hardware implementation of the transforms I mentioned was done by a volunteer we since hired him right and You mentioned the opus audio codec. Yeah, which I haven't much experience with but it looked quite impressive Except at the very low bit rate End but I was just wondering whether the work you're doing on Dala is going to feed back into a new improved audio codec in the future, right so Honestly at this point, I think opus is done You know it took a lot of effort to go through and get a standard through the ITF And so I don't think we're gonna go back and revise that there is still Opportunity to write better opus encoders to help bring up that performance at the low end But if you actually look at the graph there, I can find my mouse You know that's at around eight kilobits per second You're talking on The internet with RTP on top of UDP on top of TCP on top of TCP Sorry RTP on top of UDP on top of IP You're talking about 40 bytes of headers on 50 packets a second which is another 16 kilobits per second so at that point you're spending twice as many bits on Network headers as you are in the actual audio So we weren't really worried about that small dip at that end of the curve Yes, but GSM is dominated by people who are very very interested in selling patented codecs And so I think suspect we will not If we get into GSM it will because be because of the network effects of everybody else using opus not because they want to use opus Just about that. I think a switch Bp is coming with a new high bands codec because I don't want to use opus so I Think that kind of answers the questions and Even switch people don't really care about low bit rights anymore, and they are not gonna use opus Probably what they're gonna have is gonna be worse, but they want to have patterns Okay, we don't have any other questions. We do have a bit more time for some Okay, then I can Yes You to get a gift from Linux Australia. Thank you


Many people applying for assistance through FEMA find the process of filling out the application difficult and confusing. With FEMA centers available in nearly every flood affected county, assistance through the application process is not hard to find. It is a government program and it does require completing an application. It can be done online at disasterassistance.gov or with a telephone number that I've got right here. It's 1-800-621-3362. If people have trouble with making the application or after they've had it they have questions or don't understand or there's something that doesn't seem to be the way it ought to be, one of the things that they can do is to come into the Disaster Recovery Center that we have right here in the Knot County Sportsplex. With recent claims made that FEMA is denying too many requests for assistance, many may feel discouraged with beginning the application process. However, many services are provided by the FEMA Recovery Centers that can greatly assist in the confusing process. We can help them make a registration right up front if that's what they need to do. If they've already made a registration and they have questions, maybe they've got a letter they don't understand or something like that, we can look up their file in our computer and explain to them what the letter really means. It's all in our hands, this life of time that's given to us all. It is all in our hands, it is all in our hands. Experienced heart specialists and sub-specialists collaborating for the benefit of your care. Top 10% national hospital rankings for patient safety and heart attack and heart failure treatment. The technology is skilled to perform extraordinary complex, minimally invasive cardiac, electrophysiology, interventional and structural heart surgeries. This is a heart and vascular institute at Pikeville Medical Center. It is all in our hands, it is all in our hands. When it comes to your heart, place your care in our hands. It is all in our hands, it is all in our hands. Even two weeks after the flooding disaster, the FEMA Recovery Center in Knot County is receiving a study floor of individuals coming to seek help in the application process. Since we've been in this center, this particular center, Disaster Recovery Center, it's been open about a week and actually it's been very consistent. We're having between 50 and 60 people a day coming in. Some of them have already applied and are coming in because they're doing follow-up on their application, just what I explained, maybe bringing in paperwork or something. Other people are actually applying, but there's still a lot of applications coming in. It's not going down. The numbers are not going down. There was, of course, always in any disaster, there's an immediate number of people who step up to get their application in, but there's still a lot of people, and there are a lot of people out there who do need to apply. Those in need of aid are heavily encouraged to visit a FEMA Recovery Center for any help they need in the application process. The FEMA Recovery Center located in the Knot County Sportsplex is open 7 a.m. to 7 p.m. Monday through Saturday and 1 p.m. to 7 p.m. on Sundays. For Mountain Top News, I'm Kelsey Dean.