Ladies and gentlemen, I'm delighted to welcome you to today's professionalism seminar that will be given by a dear friend of all of us, but a dear friend of mine, Dr. Wendy Levinson. Some of you remember Dr. Levinson, who is the Chief of General Medicine here at the University of Chicago. When Wendy left about 10 or 11 years ago, she went to take over as Chair of Medicine at the University of Toronto. Some of you may not know that the Department of Medicine at the University of Toronto with over 700 full-time faculty is probably the largest Department of Medicine, at least in North America, if not, if not beyond North America. So Dr. Levinson is the Chair, and I just love her title. You're not going to stop me, but I get to say the title. Dr. Levinson is the Sir John and Lady Eaton Professor of Medicine at the University of Toronto. Dr. Levinson is a former Chair of the American Board of Internal Medicine's Board of Directors. She continues to work closely with Holly Humphrey on activities of the American Board. Wendy is also the incoming Chair of the Association of Professors of Medicine. That is the Chairman's Club of U.S. and Canadian Chairs of Medicine, and Wendy will be taking over in that role next year. Wendy has for a long time been a national leader in internal, general internal medicine, has been an expert on doctor-patient communication, the doctor-patient relationship. I think of that preliminary study you and I did in Oregon listening to tapes of primary care encounters with patients to see if we could hear elements of conflict or disagreement in the give and take on those tapes. But since then, Dr. Levinson has written spectacular stuff, the large movement on disclosing medical errors probably originated with Dr. Levinson's work over the last five or seven years. She's worked also on how surgeons negotiate informed consent with patients, an area of particular interest to me. Today, Wendy is going to speak on a behavioral and systems view of professionalism, a practical approach for physicians and organizations, and Wendy, I can't tell you how happy we are to have you back. Thank you. I'm really delighted to, I have to say a few things about Mark. Well, first I have to tell you that my title that he talked about is the First Chair in the British Empire. But I have affectionately tried to reverse the title to the lady and Sir John, Chair, because it really was the lady who put it in place and I know that. And I also have to say that we haven't, we've been friends for a long time. In fact, we were, I used to live, I used to say one firewall south of Mark and Anna because we actually lived in the attached house. That's right. So we have been friends and neighbors for a long time. So anyhow, it's really fun to be here and I'm really informal in conversation. So I really would like it if you interrupted me. I know that many of you have experience in this area, have ideas about it. I'm sure that you're doing really innovative things that I haven't heard about and I'd love to hear about. So I would be very happy if people just stop me and we chat it along the way. So I don't have any disclosures. I'm one of the few physicians who's never worked with pharma. They never had any interest in what I do. So I can always put this up. And I want to just share this framework. And it's actually something I've worked with. Holly, I just saw Holly Humphrey walk in through the ABIM. But I can tell you that I've done a lot of writing about communication. But this is actually something that I have found really useful to me, really exciting to work on. Because I think I never really understood how to take the concepts of professionalism and make them practical. And so this framework that we worked on through the ABIM Foundation really helped me shift my thinking. And what I want to do is share this framework with you and illustrate it using two examples. I have a third example if we have more time. But I'll use the example of patient-centered care as one. And the second stewardship of finite resources. Really to illustrate how I think this different model and way of thinking about professionalism I think is potentially helpful to both individuals working in the healthcare system. Not just physicians, but all healthcare professionals. And also for the systems in which we work, which are not just the passive contexts, but shape and influence our ability to be professional. So I'll share with you my thinking about it. So I'm sure all of you have seen the charter that came out now 10 years ago, which is pretty remarkable, by the American Board of Internal Medicine, the American College of Physician, and the European Society of Internal Medicine. And as you all know with this charter, I think it was a step forward. It laid out a clear concept of what professionalism in this millennium was about. And it had these three principles that were well-known, patient welfare first, patient autonomy, and put on the radar I think social justice as a third core principle. One I think that's had still less attention up until now. And it also laid out a series of commitments. They were not surprising professional competence, honesty, confidentiality. But at this time 10 years ago, I think it also put on the radar screen things that weren't talked about a lot at that time. Improving the quality of care, which physicians had not overall maybe thought about as their responsibility, improving access to care, also not necessarily on their radar screen, and the just distribution of finite resources. So it is interesting to reflect. This was sort of a foundational document. It wasn't a how-to. It didn't provide practical approaches. But it did articulate what the present commitments to professionalism should be. And it put these, I think, in shone the light on some concepts that we weren't talking about a lot. But it is interesting to reflect how much progress have we made? How much are we enacting this? What do we really need to do to take this from theory to practice? And I think that's challenging. And I myself have often felt not sure how to do that. Is it just a concept? Or can we make it more practical? And that's where I want to share this. Some of the old assumptions of professionalism and I think what has shifted. And I'm drawing heavily on Catherine Lucy's work who wrote an article a number of years ago that I referenced in a minute that I really would commend to people. I think it was very helpful. But I think we've had the attitude about professionalism that is an admittional competence. It's based on character. You're kind of born to be professional. You have the right stuff. It's kind of inherent in you. That physicians or other health professionals who lapse are unprofessional. Kind of a bad apple.  And then we need to figure out ways to punish them, teach them that they should be different. And we've had this sort of responses to elapse or punitive as a kind of underlying theme. We've also sort of thought, I think, in an old way that these challenges are infrequent. That we would go through our careers and be really good. And then occasionally we would encounter a challenge to a professionalism. And I'll share with you that I think a different framework is much more helpful. The healthcare systems, so a university and academic medical center like here, they're the places that healthcare professionals work. And they're the setting in which these lapses occur. But they're not part of the problem or the solution. They're passive places where the work goes on. And I think most important to people like you and like me is that medical schools and residencies are responsible for selecting and training professionalism. I think there's been an underlying kind of view that admissions committees should be able to do an approach which will pick people who are inherently, based on their character, professional. And then they should train them some in professionalism. But really, the medical school should shape us, but that you should get the right protoplasm in place, shape them a little bit during residency, and then they will be set to practice in a highly professional way. And even if not explicitly stated, I think, Holly, these have kind of been a bit the underlying assumptions that we've approached professionalism. And what I think is the different world view that I think really, I would say this has really helped me and my thinking is to shift my thinking in these ways. That it's really a multi-dimensional competency. That it's a competency like other competencies that we are training over time. And that it's a more complicated competency actually than some. That lapses occur in physicians who are good professionals. That it's not bad apples. That we all make mistakes, just as we would make mistakes in medical diagnosis or care. That we can kind of have an error in judgment in the professionalism realm. And that these challenges are not just once in a while, but we confront them actually pretty much on a daily basis. That we talk about teamwork, and then we see disrespectful interaction on the word, but we're not in a position of power, so we kind of ignore it. That really, if you step back and think there's small lapses and professionalism and things we see negative ones, there are also lots of positive ones every day in our workplace that we kind of take for granted. That responses to professionalism as opposed to the let's punish the person and teach them how to do it right should be based on a coaching and a root cause analysis. That we could figure out just like an error disclosure, what really went wrong in the system that led to this challenge in professionalism resulting in something? I mean, I was in a residency and not long ago and a cardiology resident told me that they all knew the cardiology trainees that one of the cardiologists did a lot more invasive procedures that they felt put patients at risk than the other cardiologists. And all the fellows knew it, but they didn't know what to do about it. Because really they didn't have the construct about how they could do a root cause analysis on that. That it wasn't just, there were many factors that allowed that to happen to the cardiologists. And another concept of this is that the sanctions are really reserved for those who fail to respond. That if giving feedback and coaching doesn't work, then sanctions might be required, but that they're not the way we handle professionalism as a routine basis. And that the healthcare system, and I'm going to talk quite a bit about this, really isn't the passive place in which these happen, but by the way we shape our policies and philosophies and culture, and I'll talk about that in the healthcare system that we very much either enhance the opportunities for us all to work in a highly professional way, or we inhibit the professionalism of our colleagues by creating incentives in the healthcare system that are not aligned with us doing what we believe. Like access to care issues, which a healthcare system can clearly inhibit us being able to live up to what we think is professionally responsible. And then healthcare leaders should support ongoing professional development of physicians throughout their career. That we're not selected at the beginning of medical school, baked during our residency, and then set for life in terms of the skill set, but that we're always working on developing them. So it's a really different approach of thinking about the concept of professionalism. And what Catherine Lucy says further in the article, and I'm not going to go through it in it, but this is the reference that was enacted in medicine in 2010, is that you can teach the people to have professional resiliency, that when you develop, when you encounter these challenges that what we need to have is skills and self-awareness in understanding the values that are at play in the situation, just like a differential diagnosis and therapeutic options, thinking about what there are, what are the options for managing the situation, how should that cardiology fellow, what are the options he might have for how we might approach this problem with the faculty member, a perceived problem with the faculty member overusing tests, communication skills that would allow us to think about why not a reasonable person-actist way using skills of active listening and empathy instead of judgment and peer coaching skills in terms of what's the best way to prevent a colleague from having a lapse and how can I handle this. So it's a really different framework, I think, than we might have thought about, in terms of how an individual might develop the skills, the multi-dimensional competency that you develop over your career in encountering and thinking through how to handle these situations. So what we said in this article that I thought really helped shift my views and goes to what David talked about policy is that professionalism happens in multiple settings. The behaviors which demonstrate professionalism happen in the physician-patient encounter, the interactions with the team, the practice settings and training environments in which these occur and those are set in the broader environmental external environment that has payment, the policy regulations, and the socioeconomic factors, etc. that determine health and that this environment, these environments, as I alluded to before, they're not passive. They influence how physicians express their professionalism. If the institution which we work has a very high bar for accepting patients that have good insurance, this affects this interaction very directly. But that also physicians and health care workers, they have a responsibility to exert influence on these environments to encourage them, influence them to make the policies ones which support our being professional. So if we're working in an environment that has undue pressures on us, for example, to see a lot of patients in a short period of time for productivity measures and we think it's not, it's surpassing a threshold which is in the best interest of the patient, we have a responsibility to influence the practice settings, the training environment and the external environment that is putting that undue pressure that we think is influencing our doctor-patient relationship. So it's a system that interacts with one another. So I want to use the example of patient-centered communication to illustrate this and talk about what are the challenges to individual physicians and teams really delivering patient-centered care and then what are the institutional challenges and I'll show you how I think this framework can help us think about what are the ways we can influence the system to make it function to optimal professionalism. So, you know, this is the area I've done my research in. I really think physicians and teams are, their challenges are challenges are that active listening and educating patients takes time.  So there is, you know, we can do things to make visits efficient but ultimately listening, caring and compassion and informing patients takes time that we have in our system in many settings perverse incentives to be efficient, to be, that drive efficiency sometimes beyond where I think we can do this task well. That there's certainly multiple transitions in care, you know, the challenges in, as you know increasingly in handing off care between settings between individuals and in that handoff understanding and passing on patients values and wishes because the next team may be there when the patient has a dilemma or a crisis and the team needs to understand how they understand, how they view their, their care and their wishes. We've all for many years talked about the hidden curriculum in medicine, the curriculum, I think everyone knows that term right, the hidden curriculum of model saying we do this and then in on the words or in practice showing something else especially around compassionate care and respect where I just earlier mentioned the example that you, we teach about team-based care and then on the words we see someone interacting with a staff member in a way that doesn't demonstrate that respect and I think we worship technology. You know, it is just true that, you know, trainees get, we work in an environment where sophisticated medical care and technology is often viewed as the real stuff of medicine and that communicating and sitting down and talking to patients is sort of the soft stuff. I mean, I, I have long experienced in my career the view that I do research on, you know, this soft stuff that doesn't matter as much as the real work of medicine and I think, you know, we, we really do look to technology as the solution for things, thinking that sort of the time with patients is nice but not as important. So, I mean, I think that these have been challenges and the institutions have had challenges, long-standing cultures that don't change easily. The hidden curriculum itself, I mean, a faculty member said to me recently, I said, we talk about respect and caring that she had this dilemma because she was on a search committee being chaired by a very senior member of the faculty but that it was clear that the junior people were not supposed to speak their mind, that they were on the committee but they were encouraged to be silent and she said, you know, what should she do about this? And of course, this is also the hidden curriculum but it's the institutional hidden curriculum around communication and respect.  We often have a lack of champions in teaching doctor-patient communication. Our measurement of teamwork and communication is pretty fledgling, you know, where we really talk about teamwork but it's hard to actually have really good metrics of how to measure that or measure trainees competencies in it although we're, I think, working on developing it. And teaching, you know, patient-centered communication skills requires time and resources. It needs simulated patients, direct observation. It's not something you teach in a lecture and then trainees have the skills. It does require feedback. So these are challenges. So this is the framework that's been, I think, very helpful to me and I'll show you using compassionate care how this has helped me change my thinking. So if we think about these values of professionalism that kind of are categories of the values, compassionate care, integrity, accountability, the pursuit of excellence, and fair and ethical use of resources, which you kind of can map onto the charter if you look at the competencies that I talked about earlier and then say, okay, well, for compassionate care, what is needed to express compassionate care at each level of the system? How could we build a system that really drove us to deliver this kind of care by all of the players playing a role instead of just, you know, because I have to just pause for a minute. I mean, the problem with the old model of professionalism that depends on the healthcare professional alone is I've often said to myself, it's like a hero fighting to be professional against all odds. Or Hallease has said, sometimes residents feel bludgeoned by professionalism. I remember your words. By which I think you meant all of the responsibilities on the professional to do this right. Thou shalt do this, and thou shalt do this, and thou shalt not do this. Instead of this more systems approach to professionalism. So what's nice about this is it helps reframe this as a competency at all levels of the system. So for the individual, and this is just a brief kind of way of thinking about it, to deliver compassionate care, we need to develop our communication skills of patient-centered care. We need to know how to actively listen, to express empathy, to hold silence, to talk about tough things like end of life care, breaking bad news, disclosing medical errors. Those are skills of informed decision making. We need to develop those sets of competencies to express compassionate care. But the team also needs to develop collaborative skills of communication, demonstrate respect amongst team members. Because otherwise there's the hidden curriculum, that we might do it here, but not here. And have skills of, like I alluded to before, handoffs. So that we can share information about what a patient's values and wishes are as we transfer care between us. But the practice setting has to support that. So examples of how the practice setting could support that is ongoing communication skills training. In some, I'd say, settings like Kaiser. Kaiser is a good example. I'm, Park Nicolette in Minnesota. There are some healthcare systems that have for a long time felt that physicians need, you know, can benefit by training and communication. They run workshops on it. Kaiser embeds communication training in all their other education CMEs. So if they do things on low back pain, they talk about communication about low back pain. And so we're engaging families and patients in management. So in some settings, like you know about, I don't know how it is here now, but Dana Farber, for example, after that, the Betsy Layman case has patients and families on every committee in the hospital. Every single committee in the hospital. So it is a way of this practice setting, engaging families and patients in how can we deliver compassionate care. And I'm going to, in a minute, talk about, I'm going to the next slide, I'm going to turn to academic medical centers and the ways that setting can influence this competency. And then there are professional organizations. So, you know, professional organizations can advocate for payment reform to support time in working with patients. If time is so eroded that this is not a fashion, this is partly a payment issue. And it can promote competencies and teamwork. I mean, professional organizations like American College of Physicians and the other medical societies can help teach this in their continuing medical education programs. So this framework, I'm going to turn to academic medical centers, but the point I'm trying to make is this set of values around compassionate care can be supported, nurtured by all levels of the system. And I think that's needed to develop the environment that allows us to express our professionalism. So I'm going to share a little bit more about the practice settings of academic medicine on this theme. And then I'll pause and I'll talk about another one. So I think academic medical centers are critical to developing this competency in all of the people that work there. The leader signaled what's valued. I mean, like I said, that search committee told this young woman, we tell you to be respectful and listen to people's views, but she didn't see it in action. So it was the hidden curriculum. Curricular changes in education, I think we've done pretty well in medical schools over the last number of years introducing communication skills training. But we are absolutely not there in residencies. They vary a lot. Some of the medicine residencies, primary care residencies in particular, family medicine residencies, geriatrics, palliative medicine, I'd say those are the exceptions. But I can tell you in our place, the surgical residents do not get much training in communication skills. And I think that we have a long way to go actually in residency training. I'll be interested in hearing your perspective and faculty development.  We ourselves have to have skills to analyze communication and to give that feedback effectively. So it's an area that needs faculty development. I think faculty feedback on their performance and 360s to faculty. I don't know whether you routinely, David, get 360s. But this drives change. A lot of the feedback from patients. You know, I'll tell you, for example, on the at the ABIM in the practice improvement modules that are part of the maintenance of certification process, the one of the most popular modules has been peer and patient feedback. And faculty and people in MOC tell us that the they required to submit a quality improvement plan. And they tell us that the most frequent thing they submit back to the board is on issues that have been raised by patients. So for example, if patients say, well, the office staff made them were rude, or I didn't get enough information about the instructions, these are the most frequent things that physicians often say, oh, I didn't know that was happening. And so feedback from patients often drives quality improvement. Transparency of patient ratings. I think, you know, there are mixed feelings about the impact of transparency, but in competitive health care systems, if patient satisfaction ratings, of course, you know, this is mainly happening on the internet with all the other forms of feedback about docs. And we talked about the hidden curriculum. I want to say that large medical groups, it's not just academic medical centers, you know, they large medical groups can assess their culture. This is one thing from the the AMA, a communication climate assessment tool, where people can, you know, a group could assess what their culture is about this. There's large scale trainings, like I mentioned, it's a required part of MOC and a lot of large groups. So we do require in all the maintenance of certification for all of the boards, not just the American Board of Internal Medicine, that patient satisfaction. This has not been fully implemented yet. It has to be implemented, I think, in the next year, that there's some feedback from patients as part of the maintenance of certification process. And, you know, we can reframe things and really, I'm very interested in whether we could use techniques like root cause analysis to understand communication breakdowns in groups and try to drive quality improvement from the things that we see that might not be working. And just quickly, you know, there are other things that other people in that last box on the external environment can also influence compassionate care. You know, CME providers could incorporate it into their learning. For example, I've never understood what, you know, in Chicago, the Orthopedic Surgeon Association, the AOS, has a big learning center here that people come to to learn how to put some new joint in, you know, and new things in. Why don't they have a standardized patient there and have the surgeon need to tell a standardized patient about this procedure? It would be a way of driving this. Accreditation bodies clearly play a role, and we talked about maintenance of certification, but, you know, MOL is another place, and hospital standards that are getting involved in communication, like Jaco and disclosing medical errors, and payers we've talked about, and malpractice companies, too, in the external environment. I can tell you that the way I, my work got going, that my very beginning research was on the relationship of communication and medical malpractice, and two very large insurers, the insurance company in Colorado and Portland and the Oregon one where I was living, worked with me and we did some research together, but in the end they started workshops on communication and offered all the physicians who took them premium reductions in their malpractice premium, because they realized that it was breakdowns in communication that led to malpractice. So what I'm trying to say is, if we want to deliver compassionate care as one of the planks of professionalism, there are a whole lot of different places that we could build a system if we thought system-wide of how do you foster this competency across the entire continuum. So I just, I'm not going to take much longer, I want to make sure we have time for questions, but I want to demonstrate it quickly just with one other, and the reason I want to is because I think this is the area we've paid the least attention to, and you know, we all know we're like really in deep trouble as a country if we don't get our act together, and you know, you have to hear Glenn Hackbarth who's the head of the ABIM Foundation and is the head of the head of MedPAC stand up in many venues, and he basically says that when his kids who are now late teenagers want to start a family, every federal dollar that the government collects in taxes will be going to the entitlement programs and paying down the deficit, and that does not leave much room for other things, and that the only place that really can be significantly cut is healthcare.  So we have to think this line through together, and you know, what let me just, what I want to do is talk about stewardship for a minute, and I want to tell you about this choosing wisely campaign, which is down in that box at the far side in what systems can do. So we all know this, that the healthcare expenditures are just not sustainable. We know there's waste in the system. This is Jack Wenberg saying that it's estimated about 30% that's waste. Waste meaning if we did away with it, no harm would come to patients, quality would not suffer. It's just taken up cash. One-third of all physicians say that they acquiesce to patient requests even when they know they're not necessary. This is a article by Campbell. That's the MRI for headache, sure. I'll order it for you. And physician decisions account for a very large part of the healthcare expenditure. You know, David Melser is sitting here, so, you know, I'm on thin ice, but it's okay, he can correct me and add. And Dom Burwick recently published this slide in JAMA talking about the billions of dollars in waste and the different wedges he called them of where they come from over treatment, excess administrative costs, failures to coordinate, and that we could look at all these places that there is theoretical waste in our system and ask what we can do about it. So, obviously, physicians are challenged. As we talked before, we care for the patient in front of us. We have patient expectations who want that MRI. We really don't know how much tests cost. In fact, frankly, it's quite impossible to know how much tests cost, because it costs a different amount. It's like sitting on the airplane. How much did you pay for your ticket? How much did you pay for your ticket? No one knows how much it costs. And it's really difficult to discuss these cost issues with patients. We are not skilled in having conversations about what we're not ordering. It's just not part of what we do. And we also have perverse incentives in some settings. And, of course, the system has many challenges. Expensive tests and procedures generate revenue. We have competition in a medical place, so if we're not going to do it, someone else is going to. We really have very limited incentives to limit certain kinds of expensive procedures. And there are all the political issues that as soon as you start to talk about this, everybody goes rationing and, you know, killing granny. And, you know, it's very hard in our society to talk about cost constraints. So, I just want to tell you about choosing wisely because I think it's really exciting. And I want to actually ask you to think about what are you going to do with this at your institution? Maybe some things are underway, but I think this, the ABIM Foundation started a major project to help physicians choose, physicians and patients to choose wisely. And we did a lot of research about what language was right for this because we didn't want to bump into the same issues of rationing and, as I mentioned earlier. And so, the ABIM Foundation launched a campaign with medical professional societies and nine societies participated in the first wave. This was just public about two weeks ago. So, ASCO, the American Society of Oncology, the ACP, American College of Cardiology, et cetera. And they each picked, here are the societies, they each picked five things that patients and physicians should question. Five procedures for which there was good evidence that it was probably waste, like routine EKGs before surgery or MRIs for low back pain, et cetera. Some very ambulatory, some very inpatient, where we live and breathe in academic settings. They partnered with a variety of consumer groups, particularly consumer reports, who agreed to do partner patient side education at the same time so that patients would be informed about these five things. And so, just to go back, so that they announced these five things. And probably, did everybody, most people see the press about this? That really got a lot of attention. And, you know, if you think about it then, and what they really wanted was the dialogue. You know, our hope is that we could start talking about these things more as physicians. And the key to this campaign is that it's physician-led. It is not the government or an insurance company saying we shouldn't be ordering MRIs for low back pain. It's physicians themselves, the spine surgeons are participating now, or the cardiologists saying we shouldn't be doing this. This is leading to waste. And so, this is an example of this part of the, this square that we have professional organizations advocating for effective care and looking at how we can choose wisely. And if you then think about it, each part of the system similarly can develop competencies to drive this aspect of professionalism. We can do no harm, not provide unnecessary and unwarranted care in our relationships. And I'd add to this, develop communication skills to talk to patients about these choices related to cost. The team could work to improve efficiency in the delivery of care and to look at where they can optimize care or where, you know, think about it on the wards where we check off an order set where, you know, everybody gets everything because, and in fact, when I talked to our faculty about this recently, they told me that some of these order sets were so old that people had, you know, been doing research back then. And so the order set in rheumatology clinic just got to be like this. And then people added to the order set, but no one ever went back and said, oh, you know, that's on there because Dr. So-and-so used to do research on that. I think we can take it off. And so, you know, we can look at, and in the practice environments, of course, we have all kinds of ways that we could influence it, including providing information about the costs to providers in our system and looking at population-based ways in the practice setting. So I don't want to belabor it because I want to leave time for questions, but that was a second example and just briefly of how we could look at finite resources again in the same model. And how do we have the, how can we as physicians in the case of choosing wisely interact with our patients, but in this case have our professional organizations start to stimulate a dialogue about how we choose wisely, which in turn can influence us and change the way we live up to that set of professional values. And, you know, what I did in our department is brought the choosing wisely campaign to our executive leadership. And, you know, we're now engaging in a discussion. We're not finished it. How can we integrate this into our training program? And how can we start to do this in each of the divisions? And so I think that just that beginning that dialogue can help us to create an environment where we can all live up to that value of finite resources in a way that we maybe haven't been in the past. So really, I've tried to talk about refining profession, redefining it from a character trait to a lifelong learning process that lapses and challenges are common. And there are complex situations like Daniel brought up earlier. This is not, these are not simple solutions, but that maybe we could use some of the techniques we've learned in the medical errors and patient safety movement applied to this and help to have the systems in which we work support our being able to live up to our professionalism, both as individuals, as physicians with patients, teams, and in the settings in which we work. So I'll pause there. I'm really interested in your reactions. What's going on here that you think speaks to this, what you're trying. Thank you very much. One comment that comes to mind, particularly with this recapitulation that you offered near the end, is that at the moment, the moment you want to do professionalism, you can do it through the medical student dean's office. You can work with your faculty to gain control of parts of the curriculum and to develop modules in professionalism. But the model that you're proposing, the much more extensive model, requires extensive cooperation, I mean more than buy-in, but extensive cooperation from a whole range of powerful interests, including the institution, which may have its own questions about what is best in its terms of efficiency and its margins and stuff. It's obviously a more embracing model than the limited one, but I think a very hard one to implement. Maybe, but let's just look back at quality and safety. In the last 10 years, I'd say 10 years ago, no one talked about medical errors, like truly no one talked about medical errors. I gave a workshop at SGIM, like I think 10 years ago, on disclosing medical errors. I don't think anyone had done something like that with a colleague, Tom Gallagher, who's done really a lot of work since then. And we've changed the system. I mean, we really have changed the system. There have been external influences. The IOM clearly played a big role in that, like by the errors human, which was an external, then there were these high profile institutions, maybe in that practice environment, like the Dana-Farbers, like University of Illinois at Chicago. And I think it has led to changes in each part of that system. So I would say 10 years ago, we would have said the same. Nice. I mean, it's a good example and model that you're pointing to, that you can actually change it from multiple dimensions at once. And you know what, finite resources yet for the next 10 years. I mean, we are going to have to figure out finite resources. So like if we don't, we really have serious problems. And I really think so we're going to have the external pressure. We have it now, right? The government is telling us that. And what is the role we as physician leaders are going to play in this dialogue? How can we bring this, you know, it is a difficult and it's a culture change and, but we all know how much room we have to move on the finite resources. Every one of us every day knows stuff that's being wasted. But if we sat down at the end of the day and wrote down, where did I see waste today? It would not be a blank piece of paper. I mean, don't you agree with that? Yeah. I am. I have the luxury of being able to see a patient for one hour and enjoy it and a patient enjoys and everything. I really proud of myself. But then I saw a colleague of mine had to see 20 patient during the time I saw four. And all of a sudden I realized if he had to do like the way I do, he has to retire and get out of the system. So my question is, is there any need for professionalism for establishment, for those insurance companies, for the major hospital, for people who I think he meant to say that these are the people are extremely important? Your answer was it's very easy. But at least in U.S. so far it's not easy because this person that I'm talking about had been three times sued for simple things that it was in his problem. So he has to order MRI, he said. Otherwise, he said, I couldn't live. I couldn't stay with it. 10% of cost of the medical practice in U.S. is because of the lawyers. Now they say it is 2%. 10% of them or 20% is because of the unnecessary orders of back pain, cardiac arrest. But all these things are the consequence because 70% of the doctor says, I just can't trust the system. I have to order it. Well, I mean, it's a complicated issue. I mean, it is true that some tests are ordered because people are afraid of malpractice. But if we as a community work like on this five things and there's a standard that this is not really appropriate, then that would certainly help decrease litigation. But I can tell you it's so interesting because people say and I always hear litigation is the reason. So in Canada doctors are sued very little. I mean, it's like shocking. It's too little actually. But in research I've done with Tom Gallagher on the medical errors and attitudes to this, physicians say the same thing. They're just as worried about malpractice in Canada as here and that they ordered these tests for malpractice, but they're never sued. So I think it's more, there are many factors, but one is our attitude toward it. That we think that we have increased in our mind the psychological presence of malpractice and the need to order. And I think that we could work at many levels. We could have standards that show that it's not appropriate care, but we could also work on our culture which supports that belief that we have to order it. And we could work on communication because my earlier research showed that I looked at surgeons who had been sued and primary care doctors who had been sued multiple times and those had been never sued. We audio taped the encounters and we could predict which to some degree, which surgeons had been sued and which had not been sued based on both content and tone of voice. Yes?  I know that as a part of medical school and it's becoming more common as a part of residency training, this changing of culture is an active process, whether it's this discussion of appropriate standards of care, kind of like the things you were talking about that have recently been published and recently being popularized. But I think it's much harder to change the culture because as an active process, because the individuals who are the ultimate decision makers are certainly not the medical students, they're certainly not the residents. It's the people who are ingrained in the system and who are ingrained in the current culture. How do you recommend or how do you foresee the culture being changed before the next generation of medical practitioners, which could be 10, 15 years from now, the ones who are really making the culture? Yeah. Well, it's a great question, but I am remembering an example that Holly told me a few years ago where a group of your medical students analyzed the mammography. You can tell the whole story, but my memory is that by analyzing the mammography rates at the hospital that they were able to influence the center to open more capacity for mammography for underserved patients. Is that not true? That's correct. So when you said that, my head is full of, here is an example right here that Holly told me a while ago where the students saw a problem, analyzed the problem. That was a professionalism problem, right? In the access to care value and they drove the system to change. I think at a micro system, we all can have an impact even if it's small. So I wouldn't wait for the administrator to say we're opening a new mammography machine. I think that's part of what we need to learn. That's the evolution that we have to have in this field, I think. That's my view of it. That was right, right? That's true. Early in your talk, you talked about the importance of looking at this competence professionalism as a multi-dimensional attribute where there's this idea you can be really, really good, whereas we've instead focused on just the really bad apples.  And so you lower the threshold at which you punish to the point that no one could argue. It's like you don't fail a student unless there's a really clear reason. So in other words, some of the problem that arose may have arisen because this was not an aspirational goal that was brought forward, but that it came from a regulatory body. And then to turn to the second part of your story about costs, I think there are lots of people and I might even be one of them who could argue that doctors are at least as much the cause of this problem as the solution to it. And will continue to be because there are fundamental conflicts of interests between what's good for doctors often and what's good for patients or society. And the only reason we're talking about this now is that we've pretty much killed the goose. Okay? And so I guess what I want to ask you and what I wonder about is should we really be looking seriously to professional societies to be doing this? Or should we really be looking elsewhere? Can the profession be professional? Well, I mean, so you're right. I'll give you. So in cardiology in the ACC, if you read the five things, they didn't start with echoes. Like the first thing on their list was we should cut down on all the echoes, right? No, I don't think so. They started with low-hanging fruit that wasn't as threatening to their finances. You know, would it be better in terms of cost if they started with the high ticket items? Yes. But I guess I really do think that starting somewhere and having the dialogue led by physicians, it's such a different dialogue if physicians are coming to the table. And I think that the professional societies can start there. It isn't the echoes. So I agree with you and I think I'm a little more optimistic. I don't have an opinion. I guess I'm just... Yeah, well, that's what I mean. And just about the medical regulators and, you know, if we can get out of the bad apple kind of philosophy, I think maintenance of certification, which we could talk about more. It's a perfect example. It's a more continuous quality improvement type of regulation rather than a bad apple approach. I'd like your comment on this. If you look at a lot of the academic medical centers, there have been significant changes in terms of productivity measures where faculty are just looking for RVU everywhere. And when you bring up the issue of self-interest, which David talked about a few minutes ago, I think it grossly undermines the effort for people to be professional using the metrics that you use in terms of the finite use of resources if you want to promote a time for effective communication. I think that's a major, major factor from all institutions that actually undermines the ability of people to be professional using that in a broad sense. I want to comment on that. The thing is this, these are not easy to change. I agree. But can we in a system in which we work agree on what our values are? I mean, if we had agreement that we wanted to develop compassionate care to the patients we took care of, and then said, well, let's look at all levels of our system at where we facilitate or where we create barriers to that. And what could we modify to, you know, if physicians sat down and said, this is the dilemma, and you know, but I think we have to have agreement on what we value. Like, I don't think we can drive change effectively unless we have some shared common bottom line that we are going to drive compassionate care. If we don't mean it, then it's hard to drive change. This exchange, these last, you know, your comments and David and Scholler's make me wonder if you don't have to broaden your model a little bit and get, perhaps, these two to contribute to it. What's desperately needed according to these questioners is an economic model to encourage professionalism. That is, in addition to the hordatory model of, you know, building on people's good intentions and laudable goals, if you can't actually think of economic incentives that would further the professional model. David? In some ways, we actually already are seeing that across the country as RVUs have increasingly shifted very recently into the bundled payments. So beyond our walls out in the community around us, private physician practices are joining, i.e., being bought by hospitals because of the way the payment is now lining up. So the RVU has become significantly devalued so that in order for private physician groups to capture those dollars, it is to their advantage to be employed by the hospital. So if the hospital and the physicians got together on the same page and said, here are our goals for patient care, and because I have a chance to see this in action, because my husband is now a cardiologist previously in private practice, now employed by a hospital, and that hospital administration and that cardiology group set the goals for patient care, and the payment from Medicare goes straight to the hospital and then to the physicians because of bundled payments. So I think that economic model again isn't perfect, but I think it's actually already happening. So I'm mindful of your time. I'm happy to stay and answer questions, but I also really want people to feel free to go back to what you need to do so that you're not feeling like you can't walk out. So I'm happy to stay. We're going to let you go.


Please raise your hand. Yes, completely, 100%. And how much does the entire QA department of yours trust your automation? No hands? One hand. And how much does your management trust your automation? Yeah? OK, that's nice. Without trust, we don't really collaborate. We merely coordinate or at best cooperate. It is trust that transforms a group of people into a team. I like this quote, because it points out the importance of not just going to work. You spend nine to five at work, and you want to add some meaning to the work that you do. Along with the people you spend most of the time with. You just don't want to go there and listen to someone and do something just cooperate without you liking it. You want to have that team building experience with everyone and enjoy your time at work. Because you go to work because you're passionate about the work, and you want to let everyone know the kind of work that you're doing. So I welcome you all to building the blocks of trust in automation. I am Sneha Vishwalingam, and a little bit about myself. I'm a test automation engineer, and I have worked on setting up automation frameworks in two companies so far, and moving on to the third one when I get back from here. This is my, thank you. This is my LinkedIn profile. I don't have a Twitter handle yet. I need to work on it. But this is how I connect with people. So feel free to connect with me. And if you have any questions, just send me. And I will definitely respond to you. So I'm happy to see so many of you here, and I'm very humbled. What are your expectations out of this talk? Like what do you have in mind when you came in? What do you want to take away with you from this talk? No expectations at all? How do I build up the expectation with not me? You would have read the proposal. You would have read the summary of the talk. What came to your mind when you read that? And what made you come to this room to attend this talk? I'm sure you're getting something out of it. I'm sure you'll take that something out of it. Moving on. So having followed some best practices from the previous Selenium content staff, my team was able to shift from flaky tests to stable, reliable automation tests. And during that time, I learned the importance of actually trusting in your test suite to unite the team as a whole. And once you start trusting your test suite, it becomes crucial for the software development life cycle as well. So this talk is going to be about our journey in building that and how we involve the entire organization in the process. So we're going to talk about building trust among the automation engineers, among the QA team, and having the management actually believe in us and trust us. So a little bit of background of what a company wanted. This company that I got into was looking to embrace automation from a fully manual QA environment so they did not have any automation in place. So the goal was to provide an easily maintainable and extensible framework that would enable the team to contribute to automation. So we wanted a framework that is easy to adapt by even non-technical people in the team so that we could involve all of them in the automation. So the journey began. We went ahead with the BDD approach using specflow and Selenium WebDriver with C-sharp. If you haven't heard specflow before, it's Cucumber for .NET. So the concepts are very similar to Cucumber. And it's an open-source solution trusted by the .NET devs around the world. That's what the website says. And ours was a Microsoft shop, so we used Team Foundation server for continuous integration and we used Gemini for the project management. And if you're not aware of Gemini, it's like the younger brother or Jira, it works. And then we had Nike's scheduled build and then we had CraneShot, we had reports and we had logs and we had data management. We had everything that was needed for a decent automation framework to be good. And we trained two of the manual testers who had a good programming background to help us with automation. So we were a team of the three of us working on automation. Everything was going good. We developed some tests and everything was fancy. Everyone was happy with how the automation was going on until we started getting tested. We started getting some Christmas lights, not these lights. I would have been very happy if I got these, but then we started getting these. So what helped? We knew that we had to do something about it and come up with some strategies to deal with it. So we started with attacking the Fakie test. Fakie tests are the fastest way to lose trust. All it takes is just one Fakie test to spoil the trust. So what we did was we separated the failing tests from the successful tests. We kept them under observation so that we knew that all the tests that were in the build that was running in the continuous integration were going to succeed. And if it failed, it's because there is a genuine defect in the app and not because of Fakie tests, or it's not because of us, it's because of them. So we kept the Fakie test under observation and we kept working on them and we started adding some table identifiers and then we added some intelligent weights. So basically what we did was we worked on them within our team so that the others could trust the automation because we're not giving them bad tests, we're giving them good tests. And then the main aim was to maintain the build always green until there was a change in the app that way everyone trusts the feedback. And while working on the Fakie test, if we had some locator issues and we weren't able to add the right elements and locators, we would just talk to the developers and ask them to help us out because it's a team effort, they should help us help them as well. So we didn't shy away from talking to the developers, they probably hate us now, but it just helps us trust the automation even more. And then what we did was don't use Selenium unnecessarily just because you can use it. We were using Selenium to set up pre-req data as well. It was the data that is supposed to be in the app, but just because we could use Selenium, we just wiped away everything and used Selenium to build the data from scratch and all the tests were dependent on those. So there was a lot of dependency that we were adding. So what we did was we added the data to the database and just restore the database whenever we wanted to run those tests against it. So what you can do is this is something that worked for us. Maybe you could add the REST API calls and then use what works for you in your organization. For us, having our software and conservation, this is what worked best for us. But the idea is to not use Selenium just because you can use it, use it where you really have to use it and use it where it really helps you the most. And then direct URLs. So we had a lot of places where we had to navigate to a lot of buttons to reach to a page. That was also giving a lot of fleekiness because of timeout issues and it was just leading to unnecessary complications. So we just added direct URLs. We obviously parametrized them and all that, but we added direct URLs in places where we had to navigate a lot unnecessarily because that's definitely not something that we are very interested in testing. We're interested in going to the page and testing what's there in the page. Then keep the test short. You don't want to test, yeah, keep the test short. You don't want to test, you don't want to test in which each step depends on the other test, right? So you break your flow down into small, manageable, and independent cases so that there's less dependency. And that way, if one test fails, it won't make the whole test read come to a halt. You can still proceed and you still get feedback and you get shorter feedback. And that way you can effectively increase your test coverage at each execution of your test. So yeah, the aim is to maintain the build screen always unless there's a change in the app. That was the goal that we had in mind and that really helped us. And this is essentially what we did. We just disabled them, not completely. We put them in a different test suite where we are working on them continuously. The idea was to work on it until it succeeds completely for like 10 to 20 run. And then when we are very confident about those tests, we will move those tests that way. It's not because of the flakiness. I mean, I know flakiness happens, but if it runs successfully for 20 runs, I'm sure the flakiness is solved. And we can trust those skills after that. And this is a tip that I got from a presentation from one of the Selenium contents. So there are so many things that we included from the talks and the Selenium contents that really helped us. It might be just small, small tips, but it makes a huge difference when you implement it at work. Code reviews. So we set some standards for code reviews. Initially, we weren't very strict about code reviews. We would just see if everything was fine and just merge them, which was a big fault. And later when we had all these issues, we set some standards for code reviews and we followed the footsteps of the devs. And they had some templates that, so we adopted that from them. And then we had our own rule set for all each and code reviews as well. It started with TINIST, having all the common values in the configuration file. So we used a conflict file to sort common needs, values across the project. And then we extracted the common logic and needed separate functions, which others can invoke. And then we made sure that we avoided repeating the same steps in multiple test cases, which definitely helped avoid redundancy. These are small things to take care of for these values necessary when the test case starts getting bigger and more maintained in a very efficient way. And then we, no documentation, a big no. This is something we really stressed on because people are constantly moving and then someone new comes and if there's no documentation, especially with test cases and cleaning tests, it's very difficult to figure out what works. So we made sure that we have, all of us put in documentation, even if, I mean, not so much, but even if a small child wants to eat it, she can understand what it is. And then relative path always that is something that also are following and fighting for. And if you're not able to get relative path again, go and bug your developers to help test your test. That issue. And then reading files. I didn't understand the importance of reading files before, but one of my colleagues, you need to pay us a lot and make it a beautiful reading file. And ever since, it has been so good for us. So each project should have reading files. And this file should include any external dependencies, URLs, setup, instructions, mode configurations, and values that need to be changed for a local machine and all that. And decorate your code with lost comments and make your code more readable and maintainable in the long run. And basically think of coding as an art. You just decorate it and just make it as legible and beautiful as you can. And this book is, well, basically our code Bible. If you haven't said this before, please add if you are reading this. It would really help my team to be able to achieve the code. And it was very useful. I cannot stress enough on the fact that please add if you are reading this. If you want to. Now let's move on to building first among the two teams. For communication. So we noticed that our manual testers did not have a pure picture of what was automated. Sometimes they would see them regressing tests that are already automated. And that is useless. This is wasting time and wasting your effort and this is not trusting automated. And that's not what we want. And that is partly because we didn't communicate well enough. And if you're not trusting, then it's something that we did as well, I'm sorry. So we addressed the communication gap issues. And as communication is key, we scheduled sharing sessions in our weekly routine. Both automation and manual testing team should be active participants and jointly plan the testing tasks of each project. And we would run all the automation and they'll take a look at it and then they'll tell us if you're missing anything, if you're missing some test cases.  So it was necessary even for us to run it through them and make sure they felt like all the tests were covered correctly. And then the tools for the job. So having an integrated test management environment was the best way to keep the test coverages in check. We've used the test trail and we've even used Excel sheets. Just make sure you mark all of them as automated. And the way they have a clear idea of what are the tests that are automated and what are the tests that may need to request. And yeah, and test trail has some, and all of these tools have some cool way to find metrics that you could use to before the release. So collaboration with manual testers. As I said, they are the ones who know the ins and outs of the system and they know the business centric, tip centrics. They know what kind of test data would be needed for what test, et cetera. So we wanted to leverage test fields into automation. And so we had all the automation tests, manual testers write feature files and then attach them to the tickets. Feature files, if you're not aware of is a part of the spec flow, like the BDD scenario. So the training, how do we train our manual testers to write feature files? We basically explain how feature files are evident to our automation structure. We had workshops where we had them train in our automation framework. We would show them the entire flow of how we create a test and how we generate step files and how we write pages for those step files. And then when they write feature files and they add them as attachments to the ticket, we would take the feature files and generate steps and write pages. But we trained them, we had workshops to let them know just for them to understand how everything fits in place so that they have a detailed knowledge on what they're doing. So tools like spec flow are created to enable non-technical people to participate in development directly. But then expecting the same people to use Visual Studio or Eclipse to write feature files is not a fair expectation in my opinion. Because sometimes they know more about the domain and they're not necessarily very interested in learning about IDEs and tools. So instead of asking them to learn the IDEs, it seemed like a better option to look for ways to serve the same purpose of enabling the team in writing feature files. So they found this app called TidyGurkin. It's a Chrome app. It's very easy to install and you don't require an IDE for that. It is free and it is very, very lightweight. So it was very easy for manual testers to use this to write feature files. And honestly, not all the manual testers are open to learning automation. They are excellent in their field and they're happy with that. They don't want to move into automation, but there are some people who are very excited and who really want to move into automation. So if you want to get something out of someone, you need to give them easier ways to listen to you or help you out with what you want. So that was the reason why we wanted TidyGurkin, why we implemented, we used TidyGurkin while training the manual testers. I will show you how TidyGurkin looks after the doc and we'll just go to a small demo of TidyGurkin. Documentation. Technical documentation is basically an invaluable resource. Everyone knows about it. No one really uses it unless they really are interested, but it is still an invaluable resource. You can use anything for documentation, but it's important to have easy access to the documentation. For example, you have options like Confluence or you can make your own website and maintain documentation there or you can even have Word documents and put them in a common folder where everyone accesses it. Whatever works for you, but make sure you keep it up to date and we tried to do that. I wouldn't say we did it every week or anything, but we tried our best to keep our documents up to date. And I've used Confluence before and it is so much better to collaborate online and work on documentation with set templates. Then you have the collaboration going on when something makes it so easy for you to collaborate. I think it encourages the team to go off in and update all the documentation. So that is how we kind of collaborated with all the manual testers and help the manual testers help us and then we used to have stand-ups in the morning where we would tell the reports, where we would tell the report updates and all that. So it was a good collaboration for us. Now, how do you get your management to trust us? Yeah, marketing automation is kind of a big deal because if you don't market it, no one knows about it, all your effort goes to waste. Benefits of formation are basically faster testing cycles, improve test coverage and ability to catch bugs earlier. But you know about this, but how do you get this across to the management? Because all they need is proof. Without proof, they won't believe you. So what sells to management is basically money. Metrics and visibility. The money part of it, I'm not sure how we can help. Maybe faster, maybe, then you figure it out, I really don't know how we can help. But the metrics and visibility part, we can surely help. Collect metrics. What kind of metrics do you collect? So you collect a test metrics, test basically test coverage, test progress curve. That's the measure of the breadth of the functional coverage that you have so far. And then determine by comparing the number of successful tests to the total number of tests that need to be completed for a given project. And if your coverage is poor, you have to address this issue immediately. And they wanna know that. They wanna make strategies depending on how the situation is. So metrics really help them. Defect metrics. Defect in the final product versus defects reported. Defect turnaround time. So that's basically the ratio of the number of pre-delivery defects versus the number of post-delivery defects. That is a metric that you really need to know to understand how well they're performing in the team. Next, we talk about visibility. How to increase the visibility of formation in the organization. So how many of you already use wallboards in your company? That's a good number. So for those of you who don't know or don't use wallboards, it's a visual aid that displays a real-time metric that we are interested in. So we could configure whatever widgets that we are interested in seeing on a day-to-day basis and have them at the wallboard. It's a large display of information and it's continuously updated. And it's usually located in a place where the whole team can see, right? So why do we have wallboards? You could just walk into the office and see the state of the build. No more digging into email reports. No more trying to figure out from the junk in your email of what is the actual report that you're looking for. If you know what I mean, like there are so many emails that you get from CFS and you have to figure out which one you're interested in and all that. You just need to look over your monitor to see the metrics, right? And it is kind of a double-edged sword if your automation is not set in place very well because that's like openly saying, oh, so many tests failed. And everyone's gonna be behind your back for the entire day about why the test failed. But instead, if you just had it in your email, it's just your headache to figure out why it failed and you can figure it out. But there's so much pressure coming from the entire company because the whole world knows that it has failed and something's wrong. So be careful when you do that. And if you don't want the entire organization to come behind you, maybe you could initially start with having all both only for your team, then for your QA team. And then if you're very confident, go and keep it in your CEO's office or do whatever you want. But be very careful when you go ahead to that step. Then round back sessions. And how many of you have round back sessions regularly in your company? So what are the topics that you cover in your round back sessions? I'm very curious to know what are the topics that you could cover in round back sessions. Hands somewhere over there. Okay, that's nice. So for the ones who don't know what round back sessions are or don't have round back sessions in your company, it's just another form of communication that's used within the group for visibility purposes. And just for discussions and presentations and workshops and so on. And the presenter at a round back session will be aware that the participants will be eating their lunches. If you know the history of round back, that's why it's called round back sessions because they have their lunches in round back sandwiches and Pepsi and all that. And then they come into the session, they'll be eating their lunches and the presenter just presents something. It's a very informal session. So the topics typically in round back sessions are something that you, it'll be nice to know, but it's not something that you really, really need to know. Right? So it's an informal meeting and all that. And why do we have round back sessions? Again, for visibility. The topic itself doesn't matter. It's about your personal preference and obviously the entire organization doesn't want a technical deep dive on Selenium WebDriver. They're not gonna gain anything out of it, but they would be open to hearing some latest developments in the field of automation, right? And on how you plan on implementing them at work. And if you give too much details on the technical aspect, like Selenium Grid or the unit test that you used and how you're gonna implement, how you're gonna implement them is fine, but just general topics on technical deep dives they're not gonna like it and they're gonna get bored very easily.  So be careful when you choose your topics, choose some nice, gentle topics that they'd be interested in listening to. And that also relates to your automation. Then attending networking events. Attend meetups, conferences, workshops, et cetera. It is good for the industry because you will no doubt return from an event workshop or conference with an abundance of fresh knowledge and likely a burning desire to implement all of what you have learned in the conference.  Who doesn't like positive energy? Who doesn't like people who brings in fresh ideas of what they've learned? So I understand sometimes attending conferences, workshops would be expensive, but attending meetups is free. So you'd be very interested to know what are the kind of topics that you can actually learn from the meetups. And you can have wonderful brainstorming sessions out of those and inject inspiration into the work culture. So there's so many advantages out of attending networking events. And I'm sure your work would also encourage such activities. So putting all these blocks together. Oh, and just in case if you haven't noticed we had color schemes in the presentation. So this is like putting all those blocks together. So putting these blocks or strategies together has helped my team gain confidence not only with the tests we wrote but also within each other because there was a lot of collaboration and there was a lot of working together. And there was a lot of transparency about all. When you're transparent with each other, you actually trust each other. So I hope that you could relate to at least a few strategies from this talk that you could actually implement within your team to help the trust. Even if not exactly what we did, you could map what we did into something that fits into your company and something that would benefit you. Thank you so much. Do you have any questions? What about the demo? Remember that everything testing in production always fails. So this is how Kydigokin looks. It's very easy to download. Let me show you, just go to this and install it. I already installed it, so just launch app and it opens up the app. This is how it looks and you could easily insert the template and you could write your feature here.  Don't worry about the template output because you see it tidies it down. That's the beauty of tidigokin and then you just keep adding how many of the scenarios that you want and write your test cases over here. And then you just, well, you save this. It saves as a dot feature file. That way you could just attach this feature file to your ticket and write whatever you want in the ticket and then it comes to the automation engineer. We just have to use the feature file as it is or sometimes make changes according to automation standards and then work with it. See how much time it saves because then you have to see all the specifications in the ticket without tidigokin and then try to figure out what they mean and then write our own test cases out of it but instead you have a living documentation over here that you could just use and use it right away. Can I? What was that? Can you make it a little bit bigger? Probably change the resolution down. So do you have any questions for me? I want scenario outline. Scenario outline. You got scenario but how to get scenario outline? Over here. With examples. Examples. Do we need to mention it? Yes. Scenario means for single-time execution, scenario outline for data driven. Oh yeah.  Examples colon not required, right? Examples colon not required, right? No, you could just remove it. It depends on you. If you want to give examples, you can add examples. Actually we are using Eclipse with cucumber plugin. So definitely we need to mention examples colon before giving data. Okay. Thank you. I was wanting to know how you handle code reviews when you're producing these tests. Feature file is written by manual testers? Yes. Okay, that basically just comes to us and we change it according to our automation. We don't have code reviews for that because it is attaching it to the tickets and we take that and make the necessary changes for us to implement them in the automation. Are there changes to the step definitions that need to undergo some sort of review? Step definitions are made by us. Okay. It's just the feature files that they send to us. I have another question. So building trust in automation maybe in a starting exercise when you start doing and it starts following up but then did you feel that it's like continuous process because by the time you have the trust in automation for one release the next release is going to be get ready and if it involves a lots of UI changes and it's going to change everything and again tests are going to become flaky. So what are the things or tips you would say you would suggest to actually communicate with developers in initial phase itself so that we don't like having rework everything again. There's a lot of changes. We would come to know already that this release is going to have a lot of changes. So be prepared for that. So we try to figure out what are the places that we can expect the changes in and we keep ourselves alert in those spaces and we try to work on them. That starts when the development starts but then it's gonna be changing till it actually gets released or how you keep. That is the thing with automation, right? You automate something and then it changes and then you change. So what we do is we automate it when we know that it's completely done and it's ready for testing and manual testers have already taken a look at it once and then it comes to automation and that's one more thing why it was nice for manual testers to write feature files because they've already tested it once while they've written test cases for us. So when it comes to us, we know that these things are working and we are automating something that's already working and we'll be able to catch it if it stops working. Yeah, thanks. Any more questions? We can have probably one more last question. All right, Sneha, thanks a lot. And brown bag is something I'm going to take away and I'm going to implement in my workplace as well. So thanks for that too. Thank you.


The Cube at OpenStack Summit at Lata 2014 is brought to you by Brocade. Say goodbye to the status quo and hello to Brocade. And Red Hat. Here are your hosts, John Furrier and Stu Miniman. Okay, welcome back everyone. You're live in Atlanta for the OpenStack Summit. This is The Cube, our flagship program. We go out to the events and extract the signal from the noise. I'm John Furrier, the founder of SiliconANGLE. I'm Joe Michoos, Stu Miniman, analyst at Wikibon.org. Our next guest, Cube alumni CEO of SolidFire, Dave Wright. Welcome back. Thank you very much. Good to see you. I did a keynote this morning. How do you feel? I feel great. You guys had a lot of customer testimonies up there. Really kind of introducing a lot of stuff. Quickly just give us an update on what, a review and an update on what you presented this morning. Yeah, so this morning we were really talking about how OpenStack has really emerged as almost this operating system for the next generation data center, is the way that people are able to orchestrate all of their pools of compute networking and storage through a software layer, move away from the siloed infrastructure of the old data center. And as people are trying to do this, they're looking for ways to get started. They're looking for ways to accelerate that process. And so we announced SolidFire AI. It's a SolidFire Agile infrastructure. It's a reference architecture-based converged infrastructure for OpenStack-based clouds that uses Dell hardware, SolidFire, SolidFire storage. So that's SSDs, right? Yeah, all flash storage with it. And then Red Hat OpenStack. So on the quote you had, you said you're going to break down the data silos and move to scale-out databases. Were you referring to transactional data there, primarily? Well, I think we're clearly seeing on the database side of things, the shift from, in many cases, relational databases to big-data data processing workloads to no-SQL databases. But the bottom line is, even if you're still in the relational world, people are having an explosion of these relational databases to manage. And the old approach of, I've got a new database. I need to get a new storage system is clearly not going to fly in the software-defined data center. Yeah, so Dave, we can talk a little bit about active infrastructure. So when your keynote you laid out, that converged infrastructure is all about simplicity. And one of the challenges of OpenStack today is I don't think simple has come up, when people have been describing it. Can you talk a little bit about how those mesh together, your reference converged architecture in OpenStack? Yeah, I mean, OpenStack has definitely gotten simpler as it's matured. The installation, the setup process, the guides that are out there make it a lot easier. But what's really been missing is something that puts all of the pieces together, the hardware and the software pieces, the best practices around deployment, the best practices around scaling to allow particularly enterprises who are trying to get started with OpenStack to have a solution to start with, something that they know is going to work out of the box. Yeah, so as far as I know, I think you have the first reference architecture that takes care of all the infrastructure with the distribution. Do you think that's true? I think we certainly have the first kind of community-based solution, one that takes a best-of-breed multi-vendor approach that doesn't try to deliver all of the hardware and software through one kind of vendor's viewpoint. And I think that's one of the strengths of the solution, is that it is bringing in the best of breed from the community. Okay, so I know from your hardware standpoint you guys have a long relationship with Dell, so I understand if you're looking for kind of a brand-name server that makes a lot of sense. Can you talk to us a little about the decision process? How did Red Hat end up as the distribution of choice? Well, first on the Dell side of things, obviously, Dell's made a huge investment in OpenStack. They're very, very committed to it. They make great servers, they make great networking, but they are really committed to being part of that ecosystem, and that's something we really support. You know, Red Hat, obviously, is a long-time leader in the open-source community and the Linux community, and they've really shown that leadership on the OpenStack side as well. They were a little bit later to the party than some, but they have come at it with tremendous resources on both the engineering side and they've already entered into a strong alliance with Dell, so it's a natural union for our first agile infrastructure. Dave, what's your use cases for your customers you guys are seeing in your traction right now in the product market fit? Is it transactional? Because I'm just thinking about big data as a huge, 80% of the landfill of big data is not necessarily IOP-sensitive, if you will. So you guys have more IOPs focused with SSD.  You know, we do a mix of workloads, and that's actually really the strength and where we see SolidFire being deployed most is places where you have infrastructure as a service, whether it's in a public cloud or a private cloud type environment, you want to run a range of workloads, right? You want to run Hadoop, but you also want to run Cassandra, but you also want to run MongoDB, and by the way, you still have a lot of MySQL and maybe some Oracle and SQL Server in there as well, and you can take the approach of trying to build a storage environment for each of those and maybe you end up with one thing for Hadoop, another thing for MongoDB and another thing but that's not manageable at scale and it doesn't give you the agility that you need as your developers, as your administrators, as your businesses are trying to move faster and the promise of SolidFire is to give a platform that has a tremendous amount of performance but is also very, very cost effective on a CAPEX basis, very, very cost effective from an operational basis but has the scale out capabilities, the quality of service abilities to give each application exactly what it needs. If it needs a lot of capacity and a little bit of performance, we can give it that if it needs a lot of performance and a little bit of capacity, you can give it that and you can run all of those workloads in a single infrastructure as opposed to having to deploy islands of storage for each application. So you guys have an advantage on throughput and cost per gigabyte? We do, I mean it has, you know, just from a raw numbers perspective, our systems have tremendous amount of IOPS, tremendous amount of throughput, very, very capacity efficient with our inline deduplication, compression, thin provisioning but it's really the software on top that delivers guaranteed quality of service. The ability to provision performance separate from capacity, the ability to give each application the performance profile that it needs and then adjust that over time if needed that really allows us to run all of these workloads and not just be tied to a single use case. Where do you guys see the success for you guys? Who's lunch are you starting to nibble away in terms of attraction in the market? Is it EMC? Is it you compete with Pure and these guys? What do you guys fit in there? Yeah, so most of the business that we do is in the infrastructure space. So we work with service providers and large enterprises that are building out their next generation data center infrastructures and it could be open stack based. We do a lot of business with VMware, we do a lot of business with cloud stack as well. So all the infrastructure platforms with solid fire underneath and in many cases we are displacing existing legacy storage systems, EMC or NetApp or something like that. The best of breed approach to the next generation data center they're looking for storage that really is a more modern architecture that is designed for multi-tenancy, designed for scale out, designed for quality of service. So Dave, solid fire started really heavily focused on the service provider market since then you've moved into the enterprise space. Give us what you're hearing in the marketplace for open stack, service providers versus enterprise. Where do you see that going and I guess the follow up for that is this move to open stack, was that a pull for the customers or are you moving ahead of the market? We invested very heavily in the service provider space initially because it was obvious that's where a lot of these next generation data center technologies were being deployed first. The competition factor of the public cloud of Amazon in the marketplace pushing the service providers and the pull that they were having from their customers to deploy infrastructures, service platforms, service type offerings was going to push them to move faster into the next generation data center. But now we're seeing the enterprise as a fast follower on to that because they're now trying to get that agility, that efficiency, that speed that everybody's been talking about in this conference, they're trying to get that in their own data centers and for us open stack was something where we were pretty far ahead of the game. We've been investing in open stack for about two and a half years now really before just about anybody had production deployments of it but it was very clear that something like open stack was going to be necessary. We were going to need a vendor neutral community based open source platform for next generation data centers. It's not that there wouldn't be other options and there wouldn't be proprietary vendors that were coming in there as well but the pull of open source was clearly going to create a vacuum that was going to create an opportunity like this. We didn't know at the time if open stack was going to be successful or not but we saw it as a great starting point a great community, great organization, something we wanted to contribute to and clearly it has become wildly successful. I think it's a great start for the folks at home watching that might not be in the inside baseball as it's evolved. Where is it today? Give them a quick update from your personal perspective. You've been close to it. You've worked at Rackspace so you know what they had to do to kind of get into the cloud business. You helped build that out and now you're participating in the ecosystem as a CEO and founder of the company but people are trying to figure out how real it is. Obviously it's happening. People are certainly interested back to house but what's the state it's very clear open stack is now in its ninth release. It has reached a level of maturity that a lot of commercial software frankly never gets to and it is absolutely production ready and we've heard example after example of customers that are running it at large scale in production today. At the same time the breadth of the project has grown tremendously. It's gone far beyond basic compute networking and storage into a number of other platform technologies that have been rolled into it as well and so with that it comes to complexity. It's easier than ever to get started but now there's this very rich ecosystem of open source projects, of commercial vendors, of other pieces that people need to consider and so it can be fairly intimidating to get started and that's one of the reasons why we wanted to start with agile infrastructure with something that was just a very kind of simple clean standards based approach to open stack but one that people could build on. They could then integrate technologies, other vendors into that solution over time. SolidFire was heavily involved in the creation of Cinder. Can you give us the update what are you seeing in the community who's really helping build that where is there kind of the innovation that's going to happen going forward because it seems some of the projects are a little bit more unstable. Cinder seems pretty solid and we're hearing a lot of announcements from different vendors so give us the Cinder update if you could. Yeah so we early on realized that block storage was something that was going to be important enough that it needed to be broken out and managed its own project. We helped orchestrate the kind of breakout of Cinder from Nova and have contributed very heavily to that project, maturing it over time into something that is a very robust software defined storage management platform and one that is vendor neutral that other vendors can plug into and there are dozens of plugins now for Cinder today and many of those vendors contribute to the core project and we're also finding that many of the customers who are using SolidFire with Cinder have actually started contributing to the project so customers like iWeb and eBay are actually contributing to the development of Cinder and there's really great capabilities in there in each release integrated backup restore new snapshot capabilities remote replication a number of other things that it is enabling and that the vendors can completely tie into and leverage their own kind of storage architectures to optimize around it. So Dave I want to ask you about the update of the company, how many employees you have now what's the level of funding, you're going for another round acquisition talks, I mean obviously peer storage has certainly had a huge funding around that kind of an all flash market EMC world had a huge event around flash what's the update on SolidFire? Yeah so we're growing really fast we grew 700% last year our kind of employees we're now over 230 employees in the company we've raised 68 million in funding and probably we'll raise more in the future but we're also taking a very pragmatic approach to the market, again we are not a flash band-aid approach we're not out just to solve performance problems there's a lot of people that have performance problems and they're looking to flash as a solution for that and flash is a good solution for that but there's bigger problems that people are dealing with and these bigger problems around agility around efficiency, around scaling their infrastructures those are really the things that SolidFire is going to and that we're really uniquely positioned to deliver on among both the flash vendors as well as the traditional display systems. What's the final question, share it in your own words what's going on in the show here what's so important about this year more than any in this industry? Yeah you know I really think this is this year is a huge turning point for OpenStack, it has reached a kind of breakaway velocity where it's clear that nobody's going to put this you know back in a can this is something that is going to be part of our industry for a long, long time to come and everybody's going to have to deal with it service providers are going to have to deal with it Amazon's going to have to deal with it enterprises are going to have to figure it out every vendor has to figure out their role to play in the ecosystem in this summit the developers coming together the ecosystem partners coming together the customers coming together to figure out what that world is going to look like has been exciting but it's now something and it's clearly one of the most talked about technologies in the IT space for a very good reason Dave thanks for coming on the Cube really appreciate it congratulations because that's been following you guys and get to know your team fun to see you guys again here at OpenStack and we'll see you around This is the Cube, we'll be right back with our next guest after this short break


The National Petroleum Investment Management Services says it has contained oil spillage in the Bordeaux community area of River State as the 40 sections of the trans-Niger pipeline has been completely repaired. The Group General Manager, Bala Wunti, disclosed these over the weekend. He stated that with the successful completion of repairs, environmental remediation will commence immediately, noting that the Nigerian National Petroleum Company Limited has an unwavering commitment to ensuring the safety and safeguarding the well-being of their host communities. We should honor our commitment, we should come here and rebuild trust, and today mark a further demonstration of our commitment to ensure that there is trust, trust on the industry and trust on the community. That way we will operate and operate safely for the benefit of Nigerian people and in particular the Bordeaux community and all other oil producing areas. We will continue to pay attention to all our producing areas and we will be closer to them because we believe we cannot succeed without them supporting us and therefore it is very much intentional for us. Hello, hope you enjoyed the news. Please do subscribe to our YouTube channel and don't forget to hit the notification button so you get notified about fresh news updates.


Thanks for joining us. We just want to do a quick demo here of Red Hat Universal Base Image to show you how it works. So on the left, you'll see we have a Fedora test system where we are going to build a new container image from the Red Hat UBI base image. And then on the right, we have a rel8 test system that we're actually going to run it. We are going to share it using Quade.io, which is a publicly available registry server that you can push your own images to and share them with other people. And so we're going to show you these three things all working together. So let's take a quick look at a Docker file I created. In this Docker file, you'll see we are pulling this Red Hat Universal Base Image. Again, this is publicly available on Red Hat's registry. And you see we just embedded maintainer. Then we install a package. You'll see that UBI comes preconfigured with a set of yum repositories. So it actually has extra packages you can pull and install. One of them being a web server.  So let's go ahead and build this.  Wants this build happen. You'll see it's pulling packages from the publicly available yum repositories. And now it's installing software. And you'll see these packages are all rel8 packages, but they are released under a end user license agreement that allows for redistribution. So these are essentially publicly available. You'll see we've built the image now. We have the image locally. You'll see we have the Quay.io, Father Linux, UBI demo. Let's run it real quick just to test it before we push it. We are going to fire it up, expose port to 8080. Let's fire it up. There we go. OK, let's do a curl. OK, there's hello world. So we have a running web server. Let's push this image up to Quay.io here. You'll see it's pushing the image layers nice and quickly. It's right in the manifest. OK, image is pushed. Let's go over to our rel8 test box where UBI 8 was really designed to run in production. And then let's do apartment images. Let's do a pull. Pull this thing down. See it's pulling all the image layers. OK, we have our image now. See UBI demo. Again, let's run it like we did on the other system. Man run. OK, so we fired it up. Now let's curl up. There's a hello world. So what I've shown you here is that you can freely distribute UBI-based container images that you build yourself. So applications, you can build your own application, share them on UBI. And it really gives you an on-ramp to a supported environment if you want one, but at the same time provides a great base image no matter where you want to use it. So thanks for joining.


Freedom! I don't have time to do effects on this video. In fact, I don't even really have time to change clothes. Because this is huge! YouTube now does 60 frames per second for everybody! That means that you can watch this video with all the hands waving and whatnot in a smooth 60 frames per second! Yeah! That's kind of awesome if you ask me. Let us know in the comment section down below if you'll be using this feature, how soon you'll be using this feature, and if you see any evaluating. Sorry about the low production quality, but I had to get this one out, and also, no time for an outro. So, till next time, you've been watching... me!