Good morning, everybody, and welcome to our December webinar. I am Donna Crosser, Chief Clinical Officer here at the Patient Safety Movement Foundation. And today, we're going to talk about sustaining change, something that health care professionals across the world are concerned about being able to do a little bit better, and how to prevent your hard-won efforts from failing. So we have a fabulous program today. We're going to be examining the current challenges and sustaining improvements that everybody shares. We're going to compare and contrast opportunities for education versus for communication and how you can leverage both of those to sustain your improvements. And then also, we'll discuss how to set up the frontline workers for success so that they can help to sustain improvement initiatives. As always, we're going to be providing continuing education credit from MedStar Health to nurses, pharmacists, and physicians. Respiratory therapists can also, in most cases, claim this credit. And so if you are looking for this type of credit, please do be on the lookout for an email from MedStar Health that you should receive this email within the next week. Sometimes it does take five to seven days, but usually a little bit sooner than that. And as soon as you complete the evaluation, you'll receive that credit from MedStar. Healthcare executives are also eligible for this credit through the American College of Healthcare Executives. You can just log that into your portal on the ACHE website. Certified Professionals in Patient Safety will receive a certificate from the Patient Safety Movement Foundation. And Board Certified Patient Advocates will also receive a certificate from us after this event. Again, please give us about at least a week to get that done. And here you can see that none of the speakers and none of the planning committee has any conflicts to disclose. So as always, we welcome your comments and your questions. Please, if you have comments that you want to make, use the chat function. If you have specific questions that you'd like to ask, use the Q&A. We may or may not answer them as we're going through the program today, but we will save 15 minutes at the end for questions and answers to discuss anything that we didn't get to. Sometimes we don't get to everything. So anything that we don't get to today will make sure that we answer electronically. And we will post that along with this recording on YouTube. We're going to be using a product called Slido today because we want your interaction. We want to ask some questions and know what's going on in your world. So please do help us with this. Join Slido. You can do this on your phone. Just take your phone. Take a picture of that QR code. It'll take you right to Slido so that you can answer the question that's on the screen. Or you can go to slido.com and enter the number 464125. We will repeat those instructions again when we get to the first question. But for now, I am very excited to be able to introduce our moderator today. Chrissy Blackburn is our moderator today. She's the principal advisor for patient and family engagement at University Hospital's Health System in Cleveland, Ohio. She's also the mother of a child with complex care needs. So she has very much able to speak to both sides of the health care coin here. So Chrissy, welcome. Thank you so much for moderating this panel. And I'd love for you to tell us a little bit about yourself. Thank you so much, Donna. And thank you, everyone, who's joining us today. It's always an honor to partner and work with Patient Safety Movement Foundation and all of their efforts to improve patient safety. As Donna said, I'm principal advisor for patient and family engagement at University Hospital's Health System. I've been in my role for about almost eight years. But I've been in the role of patient advocacy work for 14 years as well. Mother to my daughter, Lily, who successfully had her one-year transplant anniversary last Thursday. So that's very exciting. So in my role, I really, this work found me. I did not find it. Like most mothers that have children with medical complexities, we dive in. And are really some of the best patient advocates that want to partner and work with health care teams. My position does sit within our quality institute. So I'm savvy around quality improvement work, what needs to be done, and the difficulties of sustaining that work. But without further ado, I would like to introduce our esteemed panelists today. Kristen Miller, who is the senior scientific director of MedStar Health National Center for Human Factors and Health Care. Also the associate professor of emergency medicine, Georgetown University School of Medicine and affiliate faculty, Georgetown Medical Center Innovation Center with Biomedical Informatics. Wow, Kristen, that is a mouthful. And I would like to pass it on to her to tell a little bit about herself and then I'll introduce the rest of the panelists. Great, thank you, Chrissy. Good morning, everyone. Yeah, so my role as scientific director at the Human Factor Center at MedStar primarily focused on research. My own research portfolio, which is health IT, decision-making, public health, health IT policy, but then responsible for our overarching research portfolio as a center. And so lots of exciting work happening across the spectrum of human factors, things like safety surveillance, built environment, data science, informatics, lots of exciting projects that I can speak to. And then lots of teaching as well. So I teach in the Georgetown Executive Masters in clinical quality safety and leadership. And also as part of ICBI, there's a masters of informatics and data science. And so really making sure the next level leaders or current leaders understand this human factors engineering approach. And I think really relevant to this talk today about sustainability, which can be quite challenging. So excited to be here and look forward to the panel. Thank you. Thank you, Kristen. That's a true love of human experience, human factors and math all rolled into one. Next, I'd like to introduce Joyce Alamno, who is the President and CEO of Health Corps, also President of Health Retirement and Tourism, also known as Heart Alliance of the Philippines. Joyce, please tell us a little bit more about yourself. Hi, good morning to all of you and good evening from the Philippines. It's past midnight for us here in the Far East as they call it. And it's really an honor for me to be with this really dynamic ladies from the US. Thank you so much for the invitation. What I do here, Kristen and Christine, is that I am currently heading this organization called Health Corps Academy. And what we do is that we try to educate our healthcare providers in order for them to comply with the international standards of accreditation like JCI, the Joint Commission, Accreditation Canada, NABH and so on and so forth. And for the last, I'd say what, 15 years of my life, I've been trying to spearhead the globalization of healthcare in this part of the world. I've headed the development of medical tourism program in the Philippines, also called Health and Wellness Tourism and also the requirement tourism. Right now I'm also teaching in foreign service and also strategic marketing. So those are some of the things I do here in the Philippines and happy to be here with all of you. Thank you so much, Joyce. We're so glad to have you here. And last but definitely not least, like to introduce Christine Luzernas, who is the Regional Director of Infection Prevention and Control at the HEROES program at Kaiser Permanente. So next slide please. We're going to jump into our first Slido question and just a reminder on how to do that is you can join at Slido.com using the 464125 or you can use the QR code with your phone's camera to join today. So our first question to everyone is what types of initiatives are you currently prioritizing in your organization? You may select all that apply and then please just take a couple minutes to run through and answer those questions. Thank you. So it looks like our top priorities are workplace safety and wellness related initiatives which actually comes at no surprise during the COVID pandemic right now. Also in second place, we see specific clinical initiatives such as sexist falls, et cetera. Also not a big surprise. I know that falls just on an international rate is a very high priority due to some of those staffing related initiatives which came in force. And I honestly am very happy to see that there's some prioritization there with patient and family engagement related activities and initiatives as well. Thank you so much for sharing your answers. And I think what we're going to do now is we're going to dive into the first panel question. So we'll go to Kristen first and really answering why do our current approaches to sustainment fall short? Sure, yeah, I think this is such a great topic and one that maybe we're hesitant to talk about because everyone's so excited about the actual work, right, all of these new interventions and new approaches. But I think we can all be honest here that this stuff is really hard. Implementation to start is really hard, whether it's some innovative app that we think is going to improve decision making or simple changes in workflow or getting someone to use a checklist, whatever it is that implementation is hard, scaling it can be really challenging. But I think sustaining any of these interventions is probably the hardest piece of this. I think that happens for a number of reasons. We maybe you're working on something that's a funded project. And so your focus is to get something deployed and evaluate it.  And then you find out none of them were ever implemented. That might be a different webinar, not about sustainability, but about actual deployment of this stuff. So I think there are lots of competing priorities. Everyone would agree. There's lots of excitement when you first deploy something. And then over time, if it's not integrated into workflow, if you don't have buy-in from stakeholders, if you haven't thought about the technical components for sustainability, then the stuff just isn't gonna last. And so excited today for us to talk about some of the opportunities here, whether it's a robust evaluation that continues over time, whether it's discussions with the right stakeholders at the table to begin with, to make sure that what you're developing is gonna have some longevity and not run into any sort of technical or socio-technical challenges. And I think we'll talk more a little later about the actual design of something coming from this human factors lens. To me, that's the most important part is, there's energy and there's excitement, but what's the actual thing that you're deploying and what's the likelihood that that's gonna be sustained? So I think lots of challenges here that lead to this lack of sustainment and sort of across the board, lots of different things that we could talk about technically, culturally, socio-technically, lots of different contributing factors. Kristen, thank you so much. It looks like we have Christine back who is next on the question. Christine, were you able to join us via audio? Can you hear me? Yes, wonderful. So, Christine, we were just talking about why do our current approaches to sustainment fail? And I know you were gonna start talking about the overburdenness of staff and health systems. Right, well, first of all, I'm sorry for the technology issues I'm having this morning. And I wanna thank you first of all for having me today and for hosting this conversation, which is very important, always been important and had taken on a new challenge because of what's going on in the world. So, yeah, so I think the number one feedback we get from frontline managers and frontline staff when we introduce another improvement idea or another initiative is that, well, here comes another one. And I think part of the reason why we get that response is because we bring in the frontline manager and the frontline staff quite late in the game. So, I think I'm sure later on we will talk about how we will inspire and promote sustainability. And one of the very important elements is that bringing in those who are doing the work as part of the improvement project from the very beginning, from the design of it all the way to the implementation and of course, sustaining the work. In that way, they were brought into the whole idea from the very beginning and we are listening to the actual problems that they are experiencing because it's their work that we are improving. And that will help with decreasing the feeling of being overburdened or just being put upon. Thank you, Christine. And just in my own personal efforts around patient family engagement and how we try to involve patients and families in our quality initiatives and process improvement sort of on the back end just like you're talking about with, oh, oops, we forgot to include the nurse manager or the associate nurse manager or even the staff and providers. And a lot of what we try to do is look at how we can piggyback or jump on to the end of other initiatives that are going on. So when we talk about fall risks, falls is very tricky because there's so many different pathways and processes that we try to look at and how to help people prevent themselves from falling but self-pride and ego happens with patients. They think they can do it and they get up right to the bathroom. Unfortunately, that happened. So when we talk about that overburden and thinking about how we include the other stakeholders, are there different ways? And, Christine, please feel free to jump in here too that we can really hardwire piggybacking on to other initiatives that we're working on a specific project so that we're not doing the overburdening of the staff. Right, Chrissy, I'll start and I'm sure Christina will jump in. Well, I can only think of our own efforts and my organization where we have really tried hard to engage the frontline from the very beginning. And what we know from the very start is to create kind of the conditions to allow that. And one of the things we've done is including the frontline leaders, both formal and informal into this structure which in our organization, we're calling heroes. And they are part of the conversation, they're part of that group and they are the ones that are identifying the problems in their environment and escalating the issues up to their managers and then thinking of solutions. So, and the role of that bigger group is to not only listen to the problems that are being brought to them but also making sure that they are coordinating and prioritizing so that not three groups are working on the same problem but bringing the three groups together and working on that one problem and then spreading it out. And I think that's what you're speaking to when you talked about coordinating the work of the frontline. So that's one key strategy that we've done is we've created this clinical work groups that consist of frontline managers, nurses, physicians and subject matter experts and then they bring it up to the bigger group so that they can talk about the work that they're doing and also ask for resources and perhaps connect with others doing the same work. Oh, Kristine, I think we lost your audio. I was on mute. Was there anything else to add? Yeah, I mean, I think Kristine hit the main actionable item right away, which is you're designing this for someone and they need to be at the table and I think as engineers, we're really guilty of that of coming to the table and saying, here I built this thing for you and that person saying, well, but that's not what I wanted. There's nothing to do with what the actual problem is. So I think having those stakeholders there and then as we think about actionable solutions, I would say the second piece is making sure it's not just that stakeholder, but it's everyone else and who's gonna be impacted down the line. So you're rolling out some new initiative for, I don't know, screening for palliative care or financial hardship or suicide risk or whatever it is. There's the people that are doing the screening, you get them on board, you get that part implemented and sustained, but who else is going to be impacted, right? Do you have enough resources to have that financial hardship conversation or to do the palliative care consult and then who else is impacted? So the IT folks that are building the screening tool, what do we need to know from their perspective? Maybe there's EHR changes that happen all the time that are gonna impact this process down the road or a number of other parties' leadership and patience. And I think really this thoughtful approach at the very beginning to think about how you're gonna design something so that it has that longevity. And I think we're gonna talk about what sustainability is to because that's a really important part of this conversation. Am I doing something because we're at the peak of COVID and I need to improve screening and then we can sort of roll back the initiative a little bit or am I doing something that I need to sustain forever? And I think those considerations are really important too. What's sort of the end goal and the timeline of the implementation as we think about how robust we need to build the architecture around it to make sure that it's sustained. Yeah, and I really like how you said that moving from the doing for to doing with. And I know when we talk about patient family engagement we say that all the time, we're not doing two, we're not doing four, we're not doing with, but we need to include our providers and our staff into that same statement as well so that we are meeting the needs and we really are truly understanding the problem and not seeing just what the data is but what are the actual operational steps and the barriers that they face too. Thank you so much. We're gonna move on to the next panelist question which is Kristen, thank you. Christine, what does effective sustainment look like? Yes, so we had been thinking about sustainability for quite a long time, formally for the past three years. And what we found is that we talk about sustainability a lot but it turns out that we think about it differently. So I think from the very beginning we've started quality improvement. We've always expected sustainability to be part of the conversation but we haven't really formalized what the definition means. We've shared words, we think we have shared vision but the truth is we have not explicitly defined what sustainability is. And then what we found is that in literature that is exactly the problem. There is no standard definition of what sustainability is especially in the context of quality or healthcare improvement. And that's part of the problem. Now, some people think that, okay, that's fine because that allows for flexibility but in the longer term it makes it difficult to create kind of a evidence-based or an operational paradigm because you don't have a standard definition to look to and it's hard to benchmark what you're doing against others because the other problem is it's not measurable. So given the lack of like formal definition there are many definitions but there's no one definition. It's very important from the very beginning when you're designing a project that you define what sustainability means. So at least your own group or your own organization has a very defined vision of what sustainability means for the project. So for us, for instance, I mean it's a very simple definition which means that we would like to maintain the improvement that we've made the processes that made the improvement possible and then we've set the criteria of what that's gonna look like, you know that we have set perhaps a group perhaps we've defined like an accountability structure and there are definitive roles for every single person in that structure. And then we have identified how we would measure the improvement and we would monitor it and how can problems be escalated and what we would do if there's a drift which we're gonna talk about later or there's like a degradation of the improvement. And yeah, so we've defined what sustainability is we've defined what the structures are that would support it and also the workflow around that is what we think sustainment would look like. And another thing too is that and this one is still a work in progress is that sustainability for us does not mean that you're consistently meeting a target, for instance whether it's SEDF or CLABSI what it means is that you have a structure behind that that could address, right? When your performance in CAUTI or SEDF or what have your falls had degraded, what do you do? That to us is the definition of sustainment. Christine, thank you so much and really hitting it right on the head that definition of sustainability really can throw us as far as finding what that structure is and what that process is. And looking at at the end of the day we are all in healthcare because we want to help we want to help people and patients and families as I like to say and many of my other national colleagues are one of the most underutilized resources in healthcare and it's not that not every patient wants to be involved and when we look at things like patient activation measures and point of care and how a patient or family caregiver is owning the health for themselves or a loved one it's really just tapping into those patients and really getting some of that allotative and quantitative data on what it is that they need as well and really when it comes down to the patient experience it's doing what you said that you were going to do in providing that input and that structure and many on this call as far as the panels and our attendees know CMS have developed that three tiered level of patient family engagement at point of care, policy and protocol and in governance levels because of patients and families we have an expertise whether it's our experiences that we carry with us through the healthcare system or our professional backgrounds as well that can offer input and that co-design in the sustainment process and really when we look at it for those outcomes and what the patients want it's the consistency in care, it's the delivery of care and at their own meeting them where they are as far as being engaged and involved in that process. So we're going to move on to another slide that will talk to patient and family engagement and what we really want to know is how is your organization involving patients and family members? So one, our organization utilizes our patient and family advisory councils and quality initiatives. Our organization invites patient and family advisors to participate on specific QI and patient experience project teams and committees. Our organization needs further education on how to build and utilize PFACs or inviting patients and families and quality improvement work or your organization does not have PFACs or does not involve patient or family members in projects. So it looks like about 46% are involving their patient and family advisory councils and quality initiatives and I just think off the top of my head where are they in the process when you begin? Are they involved in the very beginning? I can tell you just from some of my own experiences that the organization will roll out a project then they'll take it to the PFAC for feedback and input and they find that they're starting at square one again and redoing an entire brochure or something else within a process around QI. So really involving them in the very, very beginning is critically important. It also looks like about 31% do invite patient and family advisors to be part of their quality improvement teams and committees. Very happy to see that. And that about a quarter of you would like more information on how to build and utilize the PFACs. This is something that I have done for many, many years with many of my national colleagues and for those that do not involve PFACs or patient and family advisors maybe rolling in where the barriers are to that. You know, why is that, is it cultural? Is it training? Is it onboarding? Is it resources, et cetera? So thank you so much for answering those questions and we are going to head into our last question for the panel. So Kristen, we'll start with you. What are actionable interventions for effective sustainment? Sure, real quick, I wanna address the comment you just made, because I think about engaging patients and families because I think where people miss the mark is meaningful engagement. And I'm sure, Chrissy, you can talk about this or put a patient on that thing or make sure there's a patient involved. But how are we actually engaging the patient? What's the expectation of them? And this is something I'm obviously passionate about because I think it's the same with human factors, it's the same with health equity, it's the same with patient engagement where there's this like pixie dust sprinkle of, oh no, I had a patient at the table, but what were you asking them? And how did that actually improve the initiative? And it's not an afterthought. Like you said, it's designing with, right? It's at the very beginning, it's not bringing a brochure and saying, does this look good? Like I want your sign off, but meaningful collaboration, I think, from the beginning. But that's not the question you asked me, sorry. So what effective sustainment, again, I'm gonna focus on like going back to the beginning of the design because I think that's the critical part that's gonna lead to sustainability or not. Donna, if you wouldn't mind pulling up that hierarchy of control slide, because I think as we were talking about this idea of designing for sustainment, I immediately thought about all of the work in the engineering field where we think about strength of action and this idea of hierarchy of controls. And when we do physical things, a new construction build or I remove the hazard, there's some sort of foreseen function. We know that these are the most effective controls, but also that they're the most sustainable. So Donna, if you go to the next slide, this is sort of engineering world, but this has been moved into a healthcare framework. And many of you, I'm sure are familiar with this, lots of places where you can find this, but thinking about what we call weak actions, which really are just foundational, you need training and education, you need new policies, but those things alone are not going to be effective and they're not going to be sustained either. Asking people to remember to do something that human vigilance is always going to fail. And then you think about intermediate actions and stronger actions. Are we making software modifications? Is it not just a policy, but some deep simulation training that people are going to remember for a long time, introducing a new cognitive aid or a checklist, and then these really strong things. So again, the foreseen functions, simplifying and completely changing processes, architectural changes, that these are really the most effective and the most sustainable, but I also recognize there's huge trade-offs. We can't just physically redesign everything with new construction. It's very expensive, it takes a long time, you need lots of buy-in, there's competing priorities, but to really think about what your intervention is and where it falls on this. And then, sorry, one more slide, Donna, there's some work that our center has done led by Zac Keddinger and Terry Fairbanks a few years back where they actually mapped these different actions and then looked at sustainability over time and thinking about adverse events that had happened and solutions that were put in place and saw a lot of the same things that we had seen in the literature, but actually in practice. So that these built environment improved processes, the sustainability was higher there, IT structure, the stuff that's really hard, I think institutional change, cultural change. Are we thinking about the well-being of providers and not about production pressure?  So all that to say, I think it starts in the beginning, the design of the intervention, the effectiveness and complexity of it is directly tied to the sustainability that you're gonna see in actual practice. Thank you, Kristin. Joy, we'll move to you as far as commenting as well. So the question was, what are actionable interventions for effective sustainment? All right, sorry about that. Okay, Alba, I can of course give a full lecture on sustainability, but for the sake of, for the interest in the interest of time, I can probably just share with you one slide for us to easily remember as a takeaway regarding sustainability and we can easily remember it as a four-letter word, which is step. And that is to always monitor the implementation of whatever programs and plans that we have set in the beginning, because the question of sustainability is always how to make sure that we deliver with consistency in the future, looking at always at the long-term, how to make sure or ensure that the entire team will really deliver from beginning until the end. And so what for me is most important is to have some tools or monitoring tools that we can easily remember every time. And I would call it as a step. First is to always remember or monitor the status of our patient and always patient being at the center of the core of our services. And considering always the status of the patient with all the plan of care, the medications and the vital signs. And second is to make sure that we are always taking care of our team members, so that's letter T, to always make sure that our workers are not overburdened and that they have the right skills in delivering proper service. They're not stressed out, especially nowadays, we are in the pandemic. Of course, we have to always consider their mental health and their safety. The other thing that we should consider in order for us to make sure that we can sustain the programs and the plans that we had set is that we should always monitor and look at the status of our environment. So when we talk about environment, this involves the facilities, the administrative information. Of course, our staffing, our triaging, our equipment, our inventory and so on and so forth. And the last one will be the letter P which stands for progress toward goal. We have to make sure that we monitor how we are progressing in terms of the achievement of our goal, the status of our team's patients, the goals that we have established as a team, what are the tasks or the actions that we had set and if we are really able to fulfill them and always to check whether the plan or the program that we had set in the beginning is still appropriate. So I think these are some of the tools that I can impart to everyone when they do or try to sustain a program or a project for that matter. Thank you so much, Joyce. And to Christine, how do you identify when an initiative begins to flip and why does flipping happen and how do you correct it and prevent it? Thank you. Yeah, well, first of all, I just wanna say too, just listening to Joyce, I think she gave us kind of the sustainability and action at the frontline level with her staff. And then with Kristen too, she just gave us the rundown of sustainability elements like in the cliff notes. I don't know if people still know what cliff notes are, but I felt like that's what she had given us. So yeah, so drift or degradation. So obviously we're obsessed with data, right? I mean, pretty much after the project has been implemented, what we do is we look at data constantly, we watch that dot every day and every time it goes down or goes in the direction we don't want, we literally throw everything in and start doing action planning, outreaching. I mean, that is our trigger. We are triggered by the dot. So I said earlier that the most important piece of a sustainability approach or framework is that being able to monitor the data. But the other piece to that is knowing what to do when the data shows you something. And the number one thing is not to panic, right? Not to throw everything and all let's start a project because, you know, our CDEF had gone up like last month. So being very clear about what the triggers would be to trigger an action and what the actions would be. And I have said in the beginning that sustainability does not mean perfection. That doesn't mean that you will achieve your target all the time that it will always be zero because we're dealing with humans, right? We're dealing with change. This is the pandemic and it's changed everywhere and for everybody. So there would be the ups and downs in the data and the performance. So the important thing to do is to be able to detect that number one and to know what to do after. And if we are truly sustained, there would be support structures in the medical centers and in our system to be able to respond to that, to bring in the right people to the table and have a conversation. What have you found out? What did you do? Did you do a drill down, which everybody does? And what is it that you've done? And maybe it is just like, you know, we have new people and we have not trained them well enough or there is a new test or perhaps we're getting a different kind of patients. But the important thing is knowing that what could be the factors that led to degradation. So in our organization, I mean, just like everyone else, you know, we watch data from the regional level and the local folks are doing the same thing. And when we have met our trigger, then we would do an outreach or we would reach out to the leaders and ask them what's going on. It's very rare that we would start over and which we've done by the way with CDEF because it had become very critical. But for the most part, it's just conversation. This is what we found. We just need to do a reset and have training and education usually which is a very easy intervention or there's nothing to be found and that is fine too. And we'll watch next month again. So for me, the most important piece is having a detection system and having the support structure to respond to what you're detecting and including those people who are doing the work when you're doing the respond day. Thank you so much. And I think that, you know, this sort of rolls into the next couple of questions that we talked about and I'll open it to all three of you to answer is, you know, Christy, I think you really just described what that evaluation looks like and going back to that definition of sustainability but also looking at the accountability and what role do leaders have in sustaining initiatives on the front line. And in addition to that, Christy, you also mentioned education and what are the ways to optimize education and when it's being delivered and that communication as well, you know, as far as that interrupting staff, providing multiple learning modalities. So I will open that to any one of you to answer. We have a couple of minutes before our Q and A. Maybe I can take that question, Christy, about education and communication because there was a question earlier as to whether which one should be considered in terms of sustainability. But I believe that both education and communication should always come together and believe all of you will agree with me on that. But the question is which one comes first? And I believe in the case of healthcare, in most cases, it really requires technical skills and come with it. And it is important that in healthcare need to be learned. It is not something that you can just acquire through experience. When we are dealing with lives, there is no trial and error. Everything has to be exact, precise, science, and evidence-based. And as what we have always talked about and as what we have been advocating here in patient services, there should be zero harm. And so I believe this is one industry in which we can never afford to make any mistake or defect, just like in manufacturing or in other industries. And since we are all living organisms and viruses and bacteria are very much alive too, such as what we are going through right now in this COVID-19 pandemic, I believe we need to keep learning and studying in order for our health. And I believe that and studying in order for us to be able to adapt and prevent any harm from happening. And now we also hear many of our healthcare professionals being overburdened, they're tired, they're burned out. And so nobody can really complain that they are not doing their best. But the question is, is it enough that we are doing our best? I really remember this quality guru, Edwards Deming, saying that it is not enough that we do our best. First, you must learn and know the right thing to do. And then we do our best. And so once we have educated or we have been educated on what is the right thing to do, and then we start communicating, echoing the right thing to be done, and then work at it towards achieving whatever goal or plans that we have. So I think education should come first, knowing the right thing to do, and then we communicate to our team members or within our network and the community. Thank you so much. It really is, is it the chicken or the egg as far as that education and communication?  Thank you. Yeah, and I love that. What did you say, Joyce? There's no trial and error when it comes to patient lives. A good takeaway. I wanted to add a little bit to what Christine said, which I think was so spot on about the evaluation and not panicking and how much we love data. I think where I see people fail, two things. One, we don't look at anything. And then we come back a year later and say, why aren't you doing this thing that I thought you were still doing? Or we look at the wrong data. So just wanna talk about the evaluation and thinking about this mixed methods, really robust evaluation. Because when people, I think what people tend to do is focus on that main patient outcome. So there's all these things that you put in place and then we're looking at mortality or infection rates. And that's so far removed from the actual intervention that people are missing everything that's happening in between. So we like to look at preference and performance, look at clinical outcomes, but also structure and process outcomes. So can you do a quick survey or an interview of the people that are involved in the intervention and say, how is this going for you? We started this thing six months ago. Is it still what you wanted? Can we think about this iterative design? Can we make any improvements for you? What are all the other data points that you wanna capture? Is it adoption or usage of something? If you had, I do a lot of clinical decision support tools, you rolled something out, are people following the recommendations that were given? So not just using it, but using it in the way that you expected them to. Are there cost issues? Should you be talking to IT to see if there's an economic burden or if this thing that you built, you didn't realize now requires lots of maintenance and that might impact the sustainability. So I would say as much as you can throw a lot of evaluation components around something, qualitative pieces, quantitative pieces, process, structure, everything to really understand if something is slipping, but then even on the positive sides, you've sustained something for so long. Why? What is it that people like about it? And then you can use that, I think to inform other initiatives as well. But to not focus on just that one infection rate or whatever that final endpoint is, even if that's the goal of what you're doing, but what's happening along that process. Kristen, thank you so much. And to all of our panelists, thank you. I mean, we really just demonstrated that healthcare is complex. And when it goes into quality improvement and performance improvement initiatives, it gets even more complex. And really having the appropriate stakeholders, the appropriate evaluations, the appropriate support in place to sustain is really critical. And even jumping on to some of those other initiatives that we may have going on in our healthcare organizations. We do have a few questions that came in through our Q&A box. So the first one we have is, how relevant is assessing organizational readiness for change? Yeah, I can start with that, Chrissy, because that is exactly the first step that we've done when we started working on our sustainability in our organization. And we found that we didn't know how to tell if the organization or the facility, in our case, is ready to take on the improvement that had been put in place as part of their standard work. So we looked far and wide to see if there was any tool out there or any framework that we could use to help us understand whether or not the facilities and the leaders and the people in it are ready to take on this project, right? That is no longer a project, because it's done, we've implemented it, we're now having success. Now, it's all yours to maintain. And it turns out, and we've known this, that that's not enough. So we have developed a tool that we have based actually from the frameworks that IHI had developed on sustaining improvement and put together a facility and leadership readiness assessment tool. And in it, we've added, we've included probes and elements that we expect based on evidence and literature, what a successful leadership should look like. You mentioned, what is the role of leadership and sustainability? Well, a lot, they have a lot and very important role in sustainability and what should be present in the units? What is the role of the nurses, the physicians, the frontline managers, the directors? We've included all of that in the stool and we have set it out to specific leadership in our facilities and asked them, granted it was self-reporting, self-reporting and assessing themselves on these elements and giving it as a demonstration. So yes, you have to develop some sort of assessment methodology to assess the readiness of the medical centers or the facilities or what have you. Otherwise, what would usually happen is that the improvement that you have built up and worked so hard for would just go to waste, which is the theme of this webinar. So it's important to understand where they're at and what kind of support they need. And it's important to have something above them to be able to support and detect when they need that help. So yes, readiness tool is very important. Thank you so much, Christine. Yeah, you have to see where your culture is at and where everyone is in order to really start implementing and moving things forward. As I asked the next question, I do encourage our attendees to put any other questions they might have into the Q&A box. But the next question I have is change episodic or continuous or vis-a-vis sustaining change? Kristen or Joyce? Yeah, I mean, I think it depends on the initiative, right, like there's always change and some that we expect and some that we don't, like a new pandemic that happens. I mean, I think it sort of goes back to the original idea of sustainability and is this something, is this a moment in time sort of change or is this something that you're gonna change for forever? I don't know if I'm answering the question, but all of these considerations are really important. And so again, when you're thinking about the design of something to really think is this something, I'm gonna be continuous improvement. I'm gonna check on this new intervention every month for the next few years or is this something where we need to really focus for sort of three months and then we can move on to the next thing? Folks are so excited about deploying this new thing or making change and not really thinking about some of these more foundational theoretical kind of concepts. Chrissy, can I just add something to that? That is my favorite question. To me, sustainability, change is always present. And sustainability doesn't mean no change. It's not static. Sustainability to me is not an outcome. It is a constant process that starts from the very beginning and there's no end to it. As Joy said, you're dealing with people, right? They always percent change you. Things are changing right now. So change may be episodic in terms of how you respond to it, but there's always change constantly happening. So what's important is to create a culture that is able to continually improve, adapt and learn, as Joy said, about the learning. So sustainability means you're able to detect the changes and you know what to do with it and knowing that that's always going to be present and nothing is static. So being flexible, not being perfect, but being flexible and being able to learn so that you can adapt. So that to me is what sustainability is. Change is always going to be there. Yes, and I think this question has come through very loud and clear as far as open leaders support that sustainment. So for some of our leaders that are removed from units or clinical areas, what can they do to ensure that those offers and sustainment is happening? Joy, would you like to take that question? Yeah, Chrissy, I think there's one word that really, I think hit me and reminded me in all this discussions today and in order for us to sustain any project or program, I believe being able to adapt, adaptability is very important and tenacity at the same time. And in all of this leadership takes on the biggest chunk of the job, I would say, because it is the leaders that will set the tone. They will say what are the correct things to do, the right things to do, but at the same time, they need managers. We all know that leaders managers are not identical, but they have to come together in order for us to, in order for our team to have a high performance. Leaders will say the correct things to do, but the managers will be the ones who will help us in leading the past and in making the operational plans and the strategic plans. So I think this is really the time when, especially during this pandemic, that leaders should come down from their boardroom, but really come down and meet the people and talk to people and make sure that everything is in order. You know, this is the time that leaders will need to develop the trust and the transparency also for the people to understand what's going on and what's happening. So I think at this time during the pandemic, it's difficult to think about sustainability right now. It's really survival. I would say for many of the organizations, but then after survival, we must learn to cope and we must make sure that we will be able to stand all these challenges in the future. For us now, there's a problem of sustainability. Why is that so? Because a lot of our health workers are migrating and leaving their country. So really that's a big problem for us on sustainability if we have a lack of human resources to really sustain and keep the operations together. So right now, what we're really doing is to make sure that there's constant education and training because we don't know when this health workers will leave us. It can be next month, it can be next week. So it really has to keep going, training, really empowering them, equipping them and for the leaders to be really on the ground to see what's happening. Thank you so much, Joy. So we have just one final comment before we have to wrap up today and I just wanna thank everyone. And that was from one of our attendees. Someone famously said that with reference to performance improvement that change is the only constant and that really is true. So thank you again to everyone in our attendees for being here today. And I will turn over the closure of the webinar to Donna. Thank you all so much.  Somebody commented that this was a very inspirational talk and it absolutely was. Especially these days, as Joyce mentioned, in these days of the pandemic. I do just want to reinforce again for anybody who might have joined late that we are providing continuing education credit for anybody who joined the live webinar. So nurses, physicians and pharmacists will get an email from MedStar Health if you registered that way. If you are looking for ACHE credit, you can just log that right into your account on the ACHE website for certified professionals in patient safety and for board certified patient advocates. If you registered indicating that you are looking for that kind of credit, we will be sending you a certificate from the Patient Safety Movement Foundation. So thank you everybody for joining. Every month we provide these webinars free of charge and we'd love for you to support us in that endeavor and be able to continue to offer these for free. So if you're interested in supporting us, please do visit our website at patient.sm.webinar-donations. We will be sharing this presentation with you and all of the resources that everybody spoke about today. So you can always find the link directly here afterwards. So again, thank you to our panelists, to Chrissy for doing a fabulous moderating session today and to everybody out there who joined us. Have a wonderful day and a wonderful weekend, everybody.


Hey guys, I'm Tom Olberg. We've got eight minutes, but we can do this. So, Samantha's a little girl that's upstairs that hopefully everybody got a chance to see this morning. She's a three-year-old girl who, about the time she was one, her parents noted that her left eye was not fully elevating and it was sort of intermittent but became progressively more restricted. They went to see an ophthalmologist who then referred them down here to primary. And then at that time she had no pastematical history, surgical history. She was full-term, uncomplicated delivery. She crawled at seven, she walked at 15, which is a little late, but that's also when her older sister walked. So in September, she was seen, this is the extraculum was of her left eye. So you note that she had a difficult time with elevation, more market when the eye was adducted. And she was diagnosed with likely Brown syndrome and continued to do very well. And then three months later, mom and dad noted that her pupils didn't seem to be acting quite regular. And when we really pushed them, they said, I think the left is usually smaller than the right and that's pretty consistent. And then she also started to develop some left upper lidtosis. Again, that was intermittent in the beginning and became more progressive and pronounced. So they represented back to primaries in August. Again, she had the difficulty with extracular movements, but now she also had a problem with adduction and depression. And then you can see at her pupil exam, the right eye was four millimeters in light and the left was two millimeters in light. And then the dark, the right dilated to six and the left barely budged to three. She was orthotropic at that time. And she did have this ptosis that was seen. So the thought was that this was some sort of weird now coroner syndrome with a third nerve palsy and MRI was ordered and obtained and read as normal. And then she was referred to her neurophthalmology for further workup. We saw her in September and her vision was excellent. Her color was great. You can see her stereo was pretty abnormal. She was only four out of nine on the circles. And then the parents also said that she was starting to act a little clumsy. She was walking into things and they were getting worried. So there's a question of whether or not there was another neurologic component or this is just from the decreased stereopsis that was making her clumsy. And we checked her pupils, same sort of picture. The right eye and the light was six. The left eye was three and a half. In the dark, the right eye dilated to seven. The left eye stayed at three and a half but occasionally dilated up to 5.5 sort of mimicking a dilation light. But that wasn't always present. And her near reflexes were excellent. She did not have an APD and she was a 10 prison bathroom of exotopia. There was no vertical component. And on our extracurricular motility exam, she had difficulty with elevation, depression and her ADduction. She was either full and could bring in very nicely or she could be very limited and was unable to bring in at all. And then her left pupil was also occasionally constricting with ADduction and sometimes dilating with ABduction. But again, this was very variable and it was really hard to pinpoint because the exam was changing and if you met her this morning, she's a very highly energetic young lady. So here is the extracurricular motility pictures. The center is primary gaze. You can see that left upper lidtosis. You can see that meiosis. And then you see she has a difficult time elevating, depressing. Her ADduction looks pretty good there. It's hard to appreciate if the pupil has changed at all with ADduction but then when she ABducks, you can sort of appreciate that her pupil on the left side seems to have dilated a little bit. And here's her pupil photos. So this top left corner is a room light and far which should be very similar to the center picture here which is basically also a room light and far. And you can see how different that is just between these two sets of pictures. So the center picture there and the top left picture there are the same gaze and the same lighting conditions. And then you see she does have a nice consensual response to light in the top right picture. And then at the bottom right picture is a room light and dark. And you see she's a little more tautic and that pupil seemed to have constricted a little bit more. So we were thinking this is maybe an aberrant regeneration of the third versus a cyclical third nerve palsy. More likely than a hornet or a brown or anything else like a dewayne. Does anybody have any other input that saw her today? So that's a really good question. So typically with a cyclical third nerve palsy you would expect there to be the background of a third nerve palsy which would be a dilated pupil. But in the case reports you get all kinds of weird variations and the whole mechanism and theory based on it isn't really well understood but one of the prominent theories is that there's efferent destruction and you get retrograde destruction up to the nuclei and then you get aberrant regeneration plus weird connections so that you can have maybe constant input to the pupil that's trying to make the lid raise and trying to move the muscles so that their pupil function might be better spared. Yes, but hers was not. Right. Yeah. I mean her pupil changes a lot. I mean we didn't do cocaine tests. We did repeat the MRI with contrast and that was again read as normal but again with her pupil and there's one more thing that really wasn't very obvious when we first examined her but then today when she first came in she was bouncing off the walls and running around and her lid was smacked way up high and it was occasionally twitching. So I think that we really didn't think there is much role for a whore so we didn't do any testing. Yeah, so it's very old and more defined. She sees neurology in January. Oh, she didn't. She didn't. That's where she was on.  Thanks, everybody. Okay.


I'm Lucina, and I'm here with my colleague Akash. We work at Vault Cooperative. It's a software cooperative that helps CNCF with strategic initiatives to help cloud native, to help the cloud native computing foundation make cloud native ubiquitous, including with their telco initiatives like the CNF certification program. Has anyone here heard of the CNF certification program by CNCF? About half of you. Great. We're excited to share the latest and greatest of the program. There was a press release out today ahead of telco days, so be sure to check that out. I'll give a history of how we got here, what we're working on now, and what's next. Akash will join us for a deep dive on the CNF certification process, how that works, and the underlying framework, the tooling underneath of the tests. And we do have time for questions at the end, so please hold your time for Q&A. If you'd like to follow along, these slides are all on schedule, so you can check that out. So firstly, CNCF is a community-driven place for projects and initiatives, so we wanted to thank the community, first of all, for helping us with the contributions, the thought leadership, and your support. So thanks everyone who's helped us along the way, and thanks in advance if you're interested in helping us and contributing later. The CNF certification program provides confidence to CSPs to make sure that their vendors are using cloud-native best practices in their networking applications. So we know that CSPs are migrating to cloud-native, and CNCF created a micro-survey earlier this year to better understand why. And as Phillip mentioned earlier, they're looking for eliminating vendor lock-in and dependencies, decoupling hardware and software, interoperability, flexibility, agility. The CNF definition, CNF contains one or more micro-services that follows cloud-native best practices, like declarative API, immutable infrastructure, and a repeatable deployment process. The initial definition of CNF was defined in 2018 by Dan Kahn, Ed Warnacky, and Frederick Kotz at ONS in Los Angeles. And there, a challenge was given to us. How do we apply cloud-native network functions to telco applications, and how would we measure the benefits of going from VNFs to CNFs? And with that, we created a prototype, the CNF testbed, to validate CNF proof-of-concepts with FDIO, Intel, and Cisco. At KubeConnEU, the telco user group kicked off to facilitate discussions between service providers and vendors and also connect like-minded folks who are ready to move from VNFs to CNFs from monoliths to micro-services. We looked into each part of the CNF definition to highlight the attributes of what cloud-native means. In 2020, white papers were published, the cloud-native network principles, and the cloud-native thinking for telecommunications. And the CNF test suite started, all with help with service providers like Bell, sorry, Bell and Vodafone, and vendors like Nokia, Samsung, Intel, Volk and others. To complement the test suite, the CNF working group was created to document best practices, cloud-native best practices for telco use cases. And at the end of 2021, the CNF test suite had over 60 tests running from contributions from CNF projects like litmus chaos to test self-healing, open metrics to test observability, caverno and Cubescape for security. And earlier this year, the CNF certification version 1.0 beta was announced at Valencia. So as of today, we have five certified products from vendors F5, Juniper, Matrix and Pantheon Tech, and more on the way, more in process. The CNF certification program is different because it's open. It uses open source, open standards, open implementations, and it's a community-driven bottom-up approach so that it uses the cloud-native ecosystem experts to create the standards rather than a top-down standards body approach. And there are no kingmakers. The CNF prefers to allow a healthy competition in its ecosystem, so it doesn't like to say this is the best solution for all. It allows competition and it allows choice. So this means that telcos can choose to adopt the cloud-native as practices in addition to standards that also meet their business needs. And it means that tests in the implementation come from multiple sources. So we have security tests from multiple sources. While there are several certifications for telcos that test interoperabilities of CNFs on a vendor-specific Kubernetes platform, the CNF certification tests on any certified Kubernetes environment and checks cloud-native principles like interoperability, resilience, workload self-healing, just to name a few. To earn a certified CNF badge, you can get started on the landing page, cncf.io slash cnf. Participants will fill out a participation form with eligibility requirements. After they run their CNF through the test suite, they'll submit their pull request with their test results to us, and then they'll earn the certified CNF mark. And be listed on the landing page as well as the landscape for CNCF. Some frequently asked questions. The CNF certification is per product, and there's no limit to the number of products a company can certify as long as each product has its own participation form. And there's no charge for CNCF members or nonprofit organizations. So what's next? We'll continue to collaborate with providers and vendors and cloud-native ecosystem to document those best practices, including the operator framework, a CNCF incubating project, the tag app delivery, and LF projects like NFIO for automation. And also we hope to have a new release ahead of KubeCon EU. Speaking of, we wanted to mention some upcoming events. So tomorrow there is a chaos day, and the team will be talking about our litmus chaos test in the CNF test suite and the CNF certification. And next month we'll be at One Summit Seattle for Lightning Talk, and we'll also be doing a hands-on tutorial on how to run the test suite or run your CNF through the certification program so we can walk you through that process. And KubeCon EU will be held in Amsterdam in April. The CFPs are open until November 18th for that, and stay tuned for the CFPs for the next telco day and start thinking about what you might want to see in this program or what you might want to present yourself. All right, please welcome Akash to the stage to talk a bit about the CNF test suite. Hi, I'm Akash and I have a fast heartbeat right now. Okay, so I'm going to walk you through the CNF test suite. It's Breeze. So the CNF test suite is designed to help developers and ops teams by codifying best practices that are part of this CNF certification program. It helps you stay aligned with the cloud-native ecosystem, various concepts, and projects. It also has a fast execution time, so if you were to use this in your CI pipeline, that is supported, and you can have faster feedback loops for your development and for your pre-production checks. Now, the test suite that we have leverages upstream tools wherever possible. We use Prometheus, OpenMetrics, Jager, and Floundy to test for observability. We use Litmus Chaos to test for resilience. And we use Kivarno by Nirmata and Cubescape by Armo to test for security-best practices. We also use Helm to test for configurability and installability of CNFs. Now, these are just some of the tools that we integrated. There are a lot more. Now, the test suite requires a multi-note Kubernetes cluster along with Helm, KubeCuttle, and CURL. The test suite is compatible with any certified Kubernetes platform, including distributions from AWS, Azure, Google, Red Hat, and kind. You could use any of the other distributions also. Let us know if you're using any other distribution or environment and we'll try to support it for you. Now, you can run the test suite in 5G steps, download the test suite, run the setup command, create a configuration file for your CNF, and install the CNF using the CNF setup command, and then run the cert command in order to be able to run all the tests. Now, when you run the setup command, you should see something like this. The setup command checks for prerequisites and it also installs certain tools that are required by the test suite to run certain tests. And the test suite requires a YAML configuration file to test any CNF. At a bare minimum, this configuration file needs to include the release name, anything that you choose for your CNF, the Helm chart to install, and the Helm repository to pick the Helm chart from. To install the CNF, run the CNF setup command with the CNF option with the path to the YAML configuration file that we just created. And the file that we just created is the code ENS and we're going to test that with the test suite. So let's run the cert command in order to run all the tests. Now, you see some output there that's in red, yellow, and green. Any lines in green, it means that your tests have passed. You have lines in red, that means you have failures for your tests. And if you see lines in yellow, those are tests that are skipped. That means that there are certain tests that are being skipped because they don't meet the criteria on your platform. Now, all the tests that we just run as a part of the test suite or the cert command that we just saw, those are the seven categories of tests that are best practices that Lucina just showed off in the previous slides. Now, after the cert command is finished running, it outputs a summary that looks something like this. It tells you the number of tests that have passed. It also displays a file name where the results have been saved. Now, this is what the results file looks like if you open the YAML file that was saved after the cert command. It contains the same information that was output to the console, but it's in machine readable format. So you could include this as a part of your CI pipelines if you were to use it for daily development. You could put this in your GitHub Actions, GenKens, or any other CI system that you work with. And we already have end users like Dish Wireless, Matrix, and Anuket that already integrated the test suite as a part of their development process. Now, the results file is essential in order to get certified. In order to get certified, you would create a pull request to the CNCF slash CNF certification repository with some information that includes the product name, the product version, the vendor of the product, and the results file itself. The repository has a read me with detailed information. The link is on the slides, which should be available to you after the talk or if you have downloaded the slides via the QR code. I would like to bring Lucina back to tell us about how we can work together. Live long and prosper. All right. Thanks, Akash. Yeah, there are a lot of ways that you can collaborate with us. We have meetings on Mondays for the CNF working group to document best practices. We meet on Tuesdays for the CNF certification and CNF test suite office hours. You can also find us on the CNCF public Slack. As well, you can send us an email or forward our email to any of your colleagues who might be interested. If you'd like to schedule a demo or working session of the CNF test suite, this is our calendar. We're available here all week at KubeCon and we can also meet with you on Zoom anytime remotely. And if you're ready to get started and join your CNF in a growing list of certified CNF products on the landing page, feel free to get started at cncfio slash cnf. So we do have some time for Q&A. I'd like to introduce and bring up the contributors and some maintainers of the CNF test suite who can also help answer your questions. You can also put the name to the face and know who to reach out to. Hi, Will. It's about Will Harris. You're welcome to just hang out on the front, too, if you want. Avoid the steps. Taylor Carpenter. Hi, Denver. Watson and Drew. Do we have any questions? Got a first question. Hi. Delore, you actually tried to explain for me how the scoring works, but I do not really get it. Can you explain it a bit more like what cases are mandatory, what cases are not, and how the final score is calculated? All right. The question was, can you explain how scoring works and what is mandatory? And it was specifically to Taylor. So there is an essential test, normal and bonus test. That's kind of how we have it split right now. And there's the set that are included in the certification, and then there's kind of extended beyond that. The essentials test are, and what we're looking at is more of, what do we have now and what are we going for in the future? So there may end up being levels of certification. So certified, maybe we'll have like silver, gold, platinum in the future as we're going forward. The essential are those baseline tests. Here's what we think every application and workload should have. And to get certified, you have to have at this point 10 of 15 of those essential tests. Looking towards those others are saying we're adopting more of those best practices. And as far as scoring goes, that's what it is right now for getting the certified level. It's the number of tests out of the total. Does that answer your question? Thanks. Any other questions? There's a lot of different pieces that make up a stack and can alter the behavior of a given CNF. And so how do you see this interacting at like validating a matrix of different components from the hardware to the Kubernetes layer, to the CNF itself? Or is this really trying to be focused strictly on the software that's powering the network? So currently the CNF certification looks at workloads that are running on Kubernetes. And would anyone like to expand on that? Nina said it's true. There's also, as far as the platform, there's an Anakit. So we could talk to Durga about that. So I think part of the idea is to look at also the applications and you have, in a workload, you may say there's many different applications and some are at a higher level and they look the same as any other Kubernetes applications. In that case, you just say, it's simple, you should be adopting any of the best practices that are applicable to any Kubernetes application. And where it starts looking different, I think, is maybe in applications that are going to be related to like what was the F5 talk earlier. So what are you going to do on these different networking pieces? But even there, you can talk about best practices on different aspects of the application. Before you get to a point where you say, well, we're not quite sure because we don't have a standard yet that the community has adopted. But that doesn't mean all the other parts. And then if you're looking at the something like 5G applications, you may say, well, all of these components in the core network can follow these specific set and focus on getting those going forward. I think with like Facebook's magma, you're seeing some of that where you see some of the interfaces are not going with the standards that were in the past because you can maybe use something newer that's a good fit within a Kubernetes environment. But then they'll have other pieces around the edges that may use a 3GPB standard. So obviously I believe in this because F5 has certified some of our CNFs. But there are other CNF certifications and as much as I love my friends at Red Hat like they have their own and a bunch of their stuff is super specific to them like you have to use their base image, you have to be in their marketplace, this kind of stuff. Is there some thought of maybe using like the CNCF certification as a base? Like having some of these others start leveraging it as like a first step in certification? Has there been any talk there? My initial is yes, absolutely. It is a base layer. We know that it's not going to solve all problems for all telcos. We do hope that it solves, eliminate the ambiguity of what we're saying when we say CNF and what properties we're looking for when we say CNF or cloud native network function. So we do hope that we can gain more end users from those telcos, other conformance that might find value in some of the tests that we're checking that they're not yet. That aren't specific to one vendor's platform but can be applied broadly. We've reached out to several of the projects and orgs too including Red Hat. So there's some of the tests look like they're very similar or a complementary. I don't know if I haven't tried to run the Red Hat test suite which is on GitHub in other environments. I'm not sure if it would work outside of OpenShift. But we've tried to make sure that the CNF test suite works in any environment where we're seeing end users they're running theirs. So running an OpenShift would be a good baseline. So we're saying yes, if you pass the CNF certification, then you should be good on some portion of whatever you're going to have on a maybe a distribution specific testing. And so it would add on there. So I think it is a good place for that. Anacut has been focused on the platform side mainly for the last 18 months or so. But they have some workload application specific that would get into and what I would think of is like a Kubernetes distribution and having opinionated ways of doing stuff. But you're still going to start with base Kubernetes and then build out what are the components on there. Well, you're going to have Kubernetes native pieces that you may want to adopt and you're going to have cloud native pieces that maybe go beyond that. So yes, it's a good place and we're definitely open to those conversations and we're advocating within CNCF to do that. That's part of what they want is to do that. But to work with this specific one is like what is LFN coming out with? What is Red Hat? What is Google coming out with? Definitely open to that. And anyone that has conversation would like to have conversation. Please reach out to us. How much of these is specific to telcon CNFs and how much of these can be used to other types of cloud native workloads? Who wants to take it? Yes, so at the moment the way we've got the platform is it's there's a lot of tests that are you know kind of focused on CNFs but we've got a few there where we're running against core DNS and other applications. So I mean arguably we can test any application containerized application that's doing operating on network layers. So if it's free up if it's operating on an network stack layer then we can do tests against it and check whether it's following cloud native principles. Yeah and I guess that's another thing Taylor just mentioned which is in line with the chaos talk we're doing is we're testing resiliency and availability of the application and that's something that is really applicable to any cloud native application whether it's telco or enterprise. More questions? Just can you go back to the first slide with your opinion? Sure. Okay we got time probably for one more question. Anyone else? Okay right well big applaud, warm applaud to all of you on stage. Thank you very much. Thank you. Okay so we're gonna have a coffee break now out in the foyer.


This is one that can shake you out of your seat in explanatory fiction All right, so explanatory fictions are things that often occur in the field of psychology And many other fields even lay people's fields are non scientific fields anyway I digress so an explanatory fiction is when you're trying to explain a phenomenon using oh, I don't know a hypothetical construct see my other video on hypothetical constructs or any sort of extraneous variable that you have no direct evidence of or something like that to Explain a particular phenomenon my favorite one is intelligence It doesn't explain squat. Why do I know how to use this? I've called it a phone before but it's not why do I know? He's intelligent. How does he know behavior analysis because he's intelligent? Oh, hooey. I am no more intelligent than these lights around me I only do what I've been trained to do so in other words adding the layer of intelligence of this as an explanatory feature for my behavior Means nothing it adds no level of explanation It sounds great on the surface, but it adds no additional information for an explanation about why I'm engaging in the behaviors I am it adds nothing to the relationship as an explanation between my behavior and the environment. That's it