On behalf of the McLean Center, I'm delighted to welcome you to today's lecture in the series on ethical issues in health care reform. Our speaker today is Dr. Ramona Hasnain-Winnia. Dr. Hasnain-Winnia is the Director of the Addressing Health Disparities Program, one of PCORI's five national research priorities. You see the word PCORI up on the, excuse me, on the title of the talk. PCORI stands for Patient-Centered Outcomes Research Institute, and they have these five national research priorities, and Disparities is one of the five. Prior to joining PCORI, Dr. Hasnain-Winnia was the Director of the Center for Healthcare Equity and Associate Professor at Northwestern University's School of Medicine. She also held an appointment as an Associate Professor at Northwestern's School of Education and Social Policy. Dr. Hasnain-Winnia has served as the principal investigator for national studies on advancing equity for underserved populations, on performance incentive programs, and the healthcare safety net. She was the lead author of the HRET Disparities Toolkit for Collecting Race Ethnicity and Primary Language Data. Dr. Hasnain-Winnia has served committees for the Joint Commission, the Brookings Institution, the Institute of Medicine, and other major organizations. She serves also on the editorial boards of health affairs and health services research. Today, Dr. Hasnain-Winnia will be speaking on the topic that you see above PCORI's role in creating a research agenda to reduce disparities and to improve healthcare outcomes. Please join me in welcoming Dr. Hasnain-Winnia. Thank you. Good afternoon, everyone. I'm hoping you can hear me. I learned when I got here that I'd be mic'd two times, and I'm not really great with even one mic, so we'll see how we do it, too. I'm really, really pleased to be here to speak to you today.  So I never made it from Evanston to University of Chicago to give that talk. So I'm really, really happy to be here, but I'm here in a very different role that I would have been had I come two years ago. I was here about six or six and a half years ago giving, I think, the medicine grand rounds. In that capacity, I was speaking to the audience in my role as a disparities researcher, and at that time I wasn't Northwestern University, and I had just actually left my position at the Health Research and Educational Trust, which is a research and educational arm of the American Hospital Association, to take the position at Northwestern University. But I think it's important for you to understand where I'm coming from because of where I am currently today in terms of my role at PCORI. So I was on the other side of the funding equation, meaning that I was out there looking for research dollars, was funded by a number of organizations from the private foundations to NIH, ARC, and the CDC. And I've been doing disparities research for probably 15 or so years prior to that. The work that I did focused mainly on improving access to health care for uninsured and underinsured populations. And I started my work back in Oregon where, if many of you remember, Oregon instituted a health care reform agenda, which was very, very controversial. And I worked with the Health Services Commission and the Governors' Commission in that capacity and really became quite passionate about the issue of providing high-quality care, not just care, but high-quality care to underserved populations. And I left Oregon and pursued my doctoral degree in Boston. And then subsequently, you know, did the various positions, landed at HRET, led the research group there for about 10 years, which was a wonderful, I always say it's where I grew up professionally. The reason I say that is because though I was leading the research agenda and the research team there and also had to develop my own research portfolio, it was very clear to me that I couldn't do research just for the sake of doing research. I couldn't do research that only interested me because I had these end users known as hospitals because HRET is part of the American Hospital Association. So I had these end users in hospitals, everyone from the C-suite, so the CEOs and the CMOs and the CIOs, the quality managers, nursing staff, admin staff saying, you know, what you're studying, how does that make a difference for the work that we're doing? So when I say I grew up professionally at HRET, I learned how to translate research very quickly. I was doing research, I was funded by kind of the usual suspects, I was, you know, publishing or trying to publish at some, at points and wasn't getting published in peer review, but they really didn't care about the peer review. They really wanted to know, so what does this mean? Give us the three bullet points. What can we do with this information? So when I left HRET and went to Northwestern University in a very traditional academic medical center environment, that model shifted a little bit, but I didn't lose sense of that model. That model was really important to me. So I was at Northwestern for about five years, and then I got a call from PCORI, the Patient-Centered Outcomes Research Institute, which is based in Washington, D.C., and I'm going to tell you a little bit about PCORI during my talk today. So for those of you who don't know a lot about it, hopefully you'll know a lot more. But when I got the call from Joe Selby and from Ann Beale, at that time, Ann Beale was the Chief Operating Officer and Joe Selby is the Executive Director of PCORI, asking if I might be interested in joining PCORI to lead one of their five national priorities, specifically the Addressing Disparities Priority. I was really, really interested, partially because I found that as a researcher, I was beginning to get very frustrated. So I was doing a body of work. I had met amazing people, many of them in the room here today, and that amazing colleagues had learned so much from my colleagues about the challenges of really addressing disparities at a national, regional, state, and local level. And what I have learned was that this is a really entrenched problem, and it really requires a multi-dimensional, multi-level approach. There's not a one-size-fits-all approach that's going to get us to better health outcomes for populations at risk for disparities. So when I got the call from PCORI, I thought, wow, this is an amazing opportunity. I couldn't move to D.C. We have three boys, one of them is a junior in high school. So PCORI very generously agreed to let me stay in Chicago, and I tell a commute to Washington, D.C. So it's wonderful to be able to come here in my hometown to come and speak with you all. But it really, for me, it was an opportunity to hopefully shape the national landscape in terms of what we fund and how we fund it and what we do with the evidence to really move that needle, because that needle has not moved very much. So those of you who do disparities research, for me, it was always something that I believe needed to be addressed as a mainstream issue. I really adhere to the IOM crossing the quality chasm report, which defines equity as one of the key pillars of high quality care. So I really wanted it to be a mainstream issue, but I also wanted it to be called out specifically, because I knew that if it was integrated, it would get subsumed. But I was worried that if it got called out, then it would have this place in the world where there are all these good people over here that are taking care of that problem. And that's something that has been pervasive. So I was hoping that through my role at PCORI, I could begin to integrate those two. So I'm going to tell you about PCORI. So I'm going to ask first in the audience, how many of you have heard of PCORI? A large number, but a significant number have not. So I'm going to tell you about PCORI, and then I'm going to tell you about the Dressing Disparities Program within PCORI, and tell you where we've been to date. And I'll also tell you that I've only been with PCORI for just over a year, and where we're planning on going. And I would love your input, whether it's through questions, challenges. And even after this talk, please reach out to me if you have any further questions. We're very open to dialogue and improving what we're doing. We're new. We keep saying we're a startup, but we can't keep saying that. We've been around for a little while. We were created by the Affordable Care Act, but let me move into my presentation. So I start by saying that PCORI has a really broad and complex mandate. It was established, the organization, the institute was established through the Affordable Care Act. And the purpose of the institute is to assist patients, clinicians, a variety of stakeholders, in making informed healthcare decisions. And the goal is to really do this through evidence, but it's evidence that's garnered through doing primarily comparative effectiveness research. Now the interesting thing about PCORI, let me first say, because a lot of people believe PCORI is a federal entity, that it's a federal organization, it is not. We are non-federal, though we were established through the Affordable Care Act.  So the unique thing about PCORI is that we do not go up for appropriation every year. So our budget is set, and our budget is set through 2019. And you'll see a slide where I show you what our research commitments are from 2012 through 2019. In 2019 we go before Congress. And depending on whether we show an impact or not, PCORI will either live on, because we will be reinstituted for another five years, or PCORI will sunset. So you can imagine that we have a complex mandate that has to be carried out in a relatively short time period. And part of that mandate is that we develop the research evidence, but that in fact, not only do we develop the evidence, but that we show that the evidence that we have developed through our funding mechanisms has in fact been disseminated and has also reduced practice variation and has reduced disparities. So this is a tall, tall order, so we've been really busy. And this is a challenging mandate. So we are governed by a 21-member board representing a variety of stakeholders. We have our expert researchers. We have patient advocates. We have industry. Frances Collins, the head of the NIH, is on the PCORI board representing NIH. Rick Kronik, the head of ARC, is on the NIH board representing, of course, ARC. We collaborate extensively with our federal partners. And our board is very engaged in guiding our work. And again, it's a multi-stakeholder board, so you get a lot of diverse perspectives at the table. PCORI's three strategic goals, which were approved by the Board of Governors in 2013, focus on substantially increasing the quantity, quality, and timeliness of useful, trustworthy evidence that's available to help patients, family members, caregivers, clinicians, and others make healthcare decisions. Our goal is also to speed the implementation and use of the patient-centered outcomes, research evidence that PCORI develops, and then to influence clinical and healthcare research that's funded by others. So we're beginning to see this in the work that we're doing, because our work is always focused on patient-centered outcomes. We focus very extensively in including patients and other stakeholders throughout our process, and I will show you this. And we're beginning to see that this is also beginning to seep into how other funding agencies are thinking about how they fund research. And it is one of our goals, so we're very happy to see that. Why PCORI? Why create this research institute? Well, that's a very lengthy discussion. It's a little bit political, but I'm going to focus on kind of where the rubber meets the roll. Why is PCORI important? Why PCORI? Well, research has traditionally, and again, the reason I spent a little bit of time telling you my story and the fact that I am a researcher, I am trained as a researcher, but now when I reflect back on the research that I did and the research that I contributed to the field, I think some of it was important in some ways. Some less so, other pieces of work that I did more so, but I don't know that my work was necessarily patient-centered. In some ways it was interesting, it was an interesting question, it was an important policy question, but I honestly do not remember thinking have I thought about the research that I'm doing in terms of how it's really, I mean really, not just kind of symbolically, but really going to have an impact on patients, their families, caregivers, etc. People want to know which treatment is right for them. There are a lot of choices out there. We do not have a lack of information, but we do have a lack of how we actually get to the right information. And we also don't know how to present the right information in a way that people can understand to be able to make informed healthcare decisions. And then patients ultimately need information that they can understand and use. So our focus at PCORI is to fund comparative clinical effectiveness research, which is one patient-centered, answers questions that matter to patients and to other clinical decision makers, comparisons. Comparisons is going to be something that is a theme throughout this whole talk because of the comparative effectiveness research piece. So if we get proposals that do not have a comparative component in them, and I will talk about this, they usually get triaged. Comparisons of outcomes that matter to patients, and then attention to possible heterogeneity of treatment effects. So how does this treatment work for different populations, for different subgroups? Which is something that we usually don't get to when we do efficacy trials, for example, because we're so focused on a more or less homogenous set of patients or we don't power the studies to be able to do those subgroup analysis. So this last bullet is actually quite important because of the patient-centered piece of the work that we fund. The establishing legislation also appointed a standing methodology committee. And I will say that the standing methodology committee is appointed through the legislation. Our board of directors, which I mentioned earlier, is appointed through the GAO. Okay? So the institute was also required to have a standing methodology committee to develop and improve the science and methods of comparative clinical effectiveness research. Dr. David Meltzer serves on the methodology committee from the University of Chicago. The methodology committee has developed a set of 47 standards. And the research that PCORI supports must adhere to these methodology standards. They're now embedded into our funding announcements. There are 47 standards that are grouped into 11 categories. Now, not every research project that we support has to comply with all 47 standards. It has to comply based on the context of the research. So depending on the science, the design, et cetera, then the methodology standards come into play. So just to, because when I say this, sometimes people look at us and go, oh my goodness, 47 standards and we have to comply with all of those standards. Based on the methodology, the intent of the work that a research team may be proposing, please look at the methodology standards and see that your work is relevant and complies with those standards. So our work answers questions such as these. What should I expect will happen to me? What are my options and what are the potential benefits and harms of those options? What can I do to improve the outcomes that are most important to me? And how can clinicians and the care delivery systems they work in help me make the best decisions about my healthcare? So I said that we engage stakeholders at every step. And we do, in fact, do this. So we go through a pretty robust process of topic prioritization. There are a number of topics that all deserve attention. So we need to go through a process of prioritization. And we do this through many different vehicles. We hold round tables across the country. We actually have a whole engagement team. So PCORI is, you know, we have our kind of operational folks that support all of our work, our finance, HR, communications, et cetera. But then we have the science team and we have an engagement team. And the engagement team has a director of patient engagement, which is led by Sue Sheridan, and a director of stakeholder engagement. And then both of those positions also have deputy directors. And the goal of these groups is really to get stakeholders and patients at the table with us. So we have regional meetings. We have round tables across the country. And then the engagement team also works with the science team. When we start to think about research topics and we start to think about what the priorities are, we actually hold what we call work groups where we bring together multi-stakeholder groups, which comprise both experts, so clinicians and researchers, as well as patients, patient advocates, payers, insurers, employers, a broad stakeholder to kind of really drill down on what we think those gaps are and what some of those core questions can potentially be that PCORI can support through its funding mechanism. And then we go back on the programmatic side and do the heavy lifting of gap analysis and really targeting what our funding announcements will look like. We also engage patients and other stakeholders in our merit review process. So we have scientific reviewers and then we have patient reviewers, as well as other stakeholders who review all of the proposals that come in. We do train them and the patient and the stakeholder reviewers review on two of the criteria, which are patient centeredness and patient engagement. So we integrate them into the merit review process. Our projects, the projects that we fund are actually reviewed for that engagement component. So if we get a really, really terrific project and scientifically it is amazing, but there is no patient engagement in it, there is no stakeholder engagement, it will not get funded. So we take this very, very seriously. So when I say this is throughout kind of, you know, everything that we do, it is embedded throughout everything that we do at PCORI. So here are our five national priorities. The first one is assessment of prevention diagnosis and treatment options, which is basically our comparative clinical, comparative effectiveness research program. It's the head-to-head trial program and it's led by Dr. David Hickam. Our improving health systems program is focused on exactly what it sounds like, really looking at comparative systems of care to improve outcomes, to improve patient-centered outcomes, to improve health outcomes. And that's led by Dr. Steve Clouser, who just joined PCORI about a month or so ago. Dr. Chad Bolt was the director and Dr. Bolt left, but is still working with Dr. Clouser in leading the improving health systems program. And then communication and dissemination. So we've got all this great evidence. How do we get it out there? What's the most effective means of disseminating and implementing this work? PCORI just hired Gene Slutsky, who was at ARC, to head this program, addressing disparities, which I'm going to be talking about today for the program that I lead. And then our fifth program, which is accelerating PCOR, patient-centered outcomes research and methodological research, is a little bit different. It's kind of like that Sesame Street song from years and years ago. One of these things is not like the other. So it's a little bit different in that it really is a program that focuses on enhancing and accelerating the methods, the scientific methods, the research methods for conducting comparative effectiveness research. It is also the program that houses our infrastructure work. So we've got this charge to do efficient, effective, comparative effectiveness clinical research. We don't necessarily have the data infrastructure nationally to do that. So we also are supporting the development of the national infrastructure to be able to do clinical CER. And this program is led by Dr. Rachel Florence. So here's PCORI's planned funding pattern from 2014 to 2019. I was trying to do this, but I couldn't figure out how. So I gave up, but I was going to put little question marks on 2020, 2021. We don't know what will happen after 2019. We may keep going, or we may sunset. But you can see in 2012, we funded $31 million. That was our first year. And we funded $31 million in pilot projects, 50 pilot projects that really looked at how to enhance these weren't comparative effectiveness studies. They were studies that really looked at how you enhance that collaboration of science with the community and so forth. 2013, you can see what we committed in research spending. 2014, this year, we're slated to commit up to $528 million. And then you can see that the dollars start to decrease as you go further down the line. Because if we sunset, we don't want to have all these projects that are going to be moving into 2021, 22, 23, 24. So hopefully, we expect we will be here. But again, that is a decision that is left to Congress. So let me tell you about our three kinds of funding announcements on the science side. So we have what we call our broad funding announcements. And I'm going to ask if anybody in the room here has applied to PCORI? So just a few people, not very many. OK, so we have our broad funding announcements, which come through each of the programs. So the Improving Health Systems Program has a broad funding announcement addressing disparities, has one, and so forth.   They're like their investigator initiated. We kind of want the field to send us your best work. And the projects are usually for three years. And they cap that $1.5 million, $500,000 a year, per year for three years. We have this tight timeline because of the issue of being able to show impact on a quick time frame. We also have targeted funding announcements, which are much more targeted. So they could be targeted based on a condition. And they can also be targeted in terms of the methodology that we expect the research teams to employ. So what I mean by that is we can say we are looking for comparative observational studies. We are looking for comparative randomized control trials. We are looking for pragmatic trials. And then we can set very clearly what our expectations are. And they vary in terms of the level of specificity within the announcement, but they tend to be targeted. And then we have our pragmatic clinical trials. And I'll tell you a little bit about each of these. So the broad pathway, the investigator initiated pathway is, as I said, kind of the traditional R01 model. Researchers partner, we expect partnership with stakeholders to generate questions. They apply to PCORI. Everything is on our website in terms of the application template, deadlines, et cetera. And we go through a merit review process. We have merit review panels. We have created standing panels for each of our programs. And we have, as I said, we have our scientific, technical merit reviewers, as well as our patient and stakeholder reviewers. And then we fund a diverse portfolio. And I'll be telling you about what the disparities portfolio looks like today. Then we have our patient stakeholder driven targeted funding announcements. And as I said, these are much more targeted. And they are not investigator initiated. We target the topic. We put out the funding announcement. And then we expect a response from the right teams who apply to these targeted calls. So here you can see, these were a set of initial targeted funding announcements that PCORI released on these topics. The topic on asthma, which was actually when the funding announcement went out, and we awarded eight, this was through the disparities program, and ended up focusing on treatment for uncontrolled asthma for African American and Hispanic Latino populations. And then the last one, prevention and management of obesity, that was just released as a targeted funding announcement on February 5th. And we specified that it was obesity treatment set in primary care. And we further specified what we wanted the comparators to be, and we wanted this to be a pragmatic clinical trial, again set in primary care. So these two announcements were released as targeted announcements through the disparities program. We also, in February, just February 5th, we released for the first time what we call our pragmatic clinical studies and large simple trials. And what we have done here is we kind of done a little bit of a hybrid. We have listed 15 priority topics that were generated through input from stakeholders, our advisory panels, et cetera. So we have highlighted 15 topics within this funding announcement and they're pretty diverse. And what we're looking to do is fund pragmatic clinical trials that are up to $10 million in total direct costs. We write, for this first call, we're gonna be reissuing this again. We have allocated $90 million in total for funding. And we want these projects to be completed within five years because we recognize the challenge of doing pragmatic trials, doing studies in the real world setting. So these are some of the essential characteristics that we wanna see in these pragmatic clinical trials. We wanna see broadly representative patient populations. We want to see very strong endorsement of the study of the patient groups and patient organizations that are relevant for whatever topic the applicants are applying for. We wanna see partnerships with professional organizations, payers and purchaser organizations. That piece is really, really important. And the reason for that is if we engage very early with stakeholders that are potential end users, it has an impact on facilitating dissemination. So we don't want the model of let's start the study and then we'll work with, and once we get the results, then we'll start to disseminate, oops, sorry, we'll start to disseminate. We will, we're gonna start to do that early on in the process. So that partnership is set up, let me just get this back on, sorry. No, I've got it, here we go. Okay, so there's three of the programs, three of the national priorities are represented within the pragmatic trial funding announcement. And again, we want these to take place in real world clinical settings. So the sources for the topics, so these are hierarchical, Corey, we have listed 15 topics that are of interest to us, but we're also willing to entertain the IOM top 100 that were released in terms of what the needs are for comparative effectiveness research, AHRQ's future research need projects, and we're even willing to entertain top notch, investigators really have to make a strong case, investigator initiated. Again, as I said, these are somewhat hierarchical, we've stated what our priorities are, but we also don't want to eliminate the potential for really innovative work to come forward that we might not have thought of in a given area. We get this question a lot, so I just want to hit on this a little bit. So we're interested in pragmatic clinical trials, a trial that takes place in the real world clinical population and setting and aims to collect information to help make treatment decisions versus an explanatory trial, which is a trial that is designed to increase the probability of obtaining maximal effect from an intervention of interest in an experimental population. So the results in this case for an efficacy trial can show that an intervention can work, not necessarily that the intervention will work in a real world setting. So we get questions about this quite a bit in terms of, well, I'm interested in this and it's very much designed as an efficacy trial versus a pragmatic trial in the real world setting. So just to remember that distinction. I think that you're gonna have access to this PowerPoint, but I just went ahead and put the link to this PFA within the PowerPoint slides. The two topics that we, we being the addressing disparities program have put into the pragmatic trial, PFA focuses on reduction of cardiovascular disease risk in underserved populations and then also integration of mental and behavioral health services into the primary care of persons at risk for disparities. So these are the two topics that are currently within the pragmatic trial that the addressing disparities program would be interested in seeing potential applications. If you look, if you go to the link, these are described in a little bit more detailed in terms of what it is that we're looking for, but it gives you a sense of what we're interested in. So now let me move into telling you about the program, about the addressing disparities program. So what you see here is our mission statement for the program, it's pretty straightforward. It's to reduce disparities in healthcare outcomes and advance equity in health and healthcare. Our guiding principle is also pretty straightforward and it's basically that we don't want to fund descriptive studies. And the reason that I say this is that when I first joined PCORI, we were still getting a lot of applications that were basically studies that were really interesting studies, but they describe disparities. And there is a great deal of evidence that disparities exist. If you send us an application and tell us there are disparities, we'll believe you because that evidence base is very, very strong. It's stronger in some populations versus others. So we know about racial and ethnic disparities. We know about language-based disparities. I will also say that we know about racial and ethnic disparities for some populations more than others. So there are a lot of subgroups that we don't know a lot about and we fully understand that. But our program also targets individuals with disabilities because we know that individuals with disabilities are at risk for experiencing disparities. It also targets rural populations, LGBT populations and low socioeconomic populations. So it's a broader net. And I do get this question, especially I've gotten it a couple of times from individuals from the disability community and from the LGBTQ community where individuals have said, well, if you don't want descriptive studies there really isn't anything out there right now. So we don't have a lot of, we can't bring forward the evidence in our applications to make the case that this would have an impact on this population because there really aren't a lot of empirical studies in the literature that we can point to. So what do we do in that case given what your guiding principle is? And usually what I say, and we have funded two studies that have done this and done it successfully, where I say, you know what, I understand that. So do a small pilot. Make the case with a small pilot. We understand the limitations that this puts on you for certain populations. But ultimately we wanna see your actual proposal focus on an intervention to improve care. So just to keep that in mind. So the addressing disparities program goals feed right into Cory's goals. We wanna identify high priority research questions. We wanna fund research that will have an impact and we wanna be able to disseminate the best of the promising practices. In terms of impact though, this is the middle column is the one where we are challenged because we need to be able to show that we are making a difference and through the work that we're supporting. And those of you who do disparities research you know this that we can see for example if we're looking at certain measures of quality we can start to see that quality is improving for two different populations. A disparities population and a comparator even. And you can see that the rising tides lifts all boats but the closing of the gap takes a much longer period of time. Or the closing of the gap frankly never happens unless you target the disparity or even worse the gap gets bigger. So you have to target that disparity but the timeline for being able to see an impact usually is not three years. So we really have to be mindful about how we're gonna tell a comprehensive story especially because we're addressing so many different conditions, so many different populations and our goal is to see a reduction in national disparities. So we are very focused on how we're gonna start to tell that story and I'm gonna tell you a little bit about that. So to date we have awarded through our four broad cycles the investigator initiated we've awarded 31 projects totaling about 53 million and then we funded through our targeted announcement on asthma we funded eight projects but totaling 23.2 million. So we have funded in about a year 76 million dollars of comparative effectiveness research focusing on addressing disparities. What's in the pipeline right now I mentioned our targeted funding announcement for obesity that was just released February 5th the letter of intent is due March 7th. What we have actively when I say actively in the pipeline because it's not the only thing these are the only two things in the pipeline there are other things they're further down but we are actively working on management of hypertension. So developing a funding announcement for that we've been working with the million hearts initiative on that and then also reducing disparities in perinatal and maternal outcomes which is also one of our focus areas and for this funding announcement the perinatal outcomes announcement we are partnering with the improving health systems program. So it's the first time that two programs at PCORI are coming together to fund one topic area. So and we're also collaborating with NIH with NICHD. Oh that's my typo it should be 2014. It was already released thank you for that. You know what I did I put in February 5th and I don't know why it became 2015 but that's out there it's already on our website so you can see that funding announcement. So remember when I mentioned earlier in my talk how I was concerned about disparities being a separate priority because I thought it was important that it be integrated as well. So it has to be called out but it also has to be integrated. It was a conversation, it was a very long conversation actually that I had with Joe Selby when I first joined PCORI about my concerns about this and how we could really actively work to make sure that all of the applications that we're getting at PCORI that focus on disparities not only come to the addressing disparities program we should see disparities projects and addressing improving health systems and clinical CER and communication and dissemination. So what you see here are four of the priorities and you can see the total number of projects that we have funded in each of these areas and the reason I think this is so important is because this bubbled up this bubbled up from all of you this bubbled up from the research community. So for addressing disparities of course 100% of our portfolio focuses on addressing disparities for improving health systems for the broad funding announcements they have funded a total of 41 projects 30 of those 73% of the portfolio has targeted a disparities population. For assessing options 48% for communication and dissemination 48% so PCORI's overall portfolio 68% well over half focuses on disparities so this was music to my ears because it's showing me that even though we've been able to call it out as a national priority to have a true focus on it we've also seen that it is beginning to play out within the other programs. So these are the disparities populations these are not mutually exclusive. So you can see though where our funding is going in terms of the populations this is for the disparities portfolio specifically. So the majority of our projects do focus on racial and ethnic minorities but we do have a cross section of other populations that we are also targeting. This basically shows you again this is for our broad the types of studies that we are funding in different areas. The majority so about 36% of our portfolio focuses on chronic conditions and you can see the breakdown of those chronic conditions and about 23% focuses on psychiatric mental health and then you can see the breakdown of the other topic areas where we have funded. So in terms of telling our story we really started to think about having a conceptual framework to think about where we were and where we were heading in terms of what we were funding. And we started to really scour the literature. There's some really good work that's been done by many people in terms of developing and putting out their conceptual frameworks for advancing equity, for reducing disparities. And we were thinking that we would try to develop some sort of a hybrid that could focus on what PCORI's priorities are. But the more we looked, the conceptual model that really made sense for us was Lisa Cooper's, which was published in JGEM in 2002. The reason that we think that this really works for us is because it cuts across so many different dimensions, the multi-dimensional approach towards addressing disparities but the outcomes also have a focus on outcomes that matter, well-being, functioning, patient's view of care. So the framework employs that. So we are basically using this as our guiding principle and then we're looking at what we are funding and what we will need to fund in order to kind of overlay where we're having an impact within this framework. Now this next slide, this is messy, ignore the arrows. The other thing that we did is we developed a driver model. So a model to really look at if our program goal is to reduce or eliminate disparities in healthcare outcomes, what are the drivers that will get us there? What are our primary drivers? What are our secondary? What are our tertiary? This is very much a work in progress. What I want to point out here is that when we looked at our portfolio, we recognized that there are policy level interventions or policies that can have an impact on improving access, on improving quality. We can look at national pay for performance programs, for example, and the jury is very much out on the impact of those types of programs on disparities. But we can also look at what organizations and systems do. What are state level factors? What we learn from doing this, the little numbers, the little number, the boxes with the numbers in them show the number of projects that focus on these areas. What we learned very quickly is that our entire broad portfolio, all 31 projects, focus on point of care or communication interventions. We do not have interventions that focus at the systems or the organizational level, and we don't have projects currently that focus on the policy level. Now the policy level is really an important one because we're in the middle of implementation of the ACA. So we've got some natural experiments. We can look at some comparators across states at some of the exchanges. Commonwealth Fund, Robert Wood Johnson, other foundations are doing some important work in this area, but PCORI could potentially partner with them to really focus on their evaluations and add a patient-centered outcomes component to see what the impact is on patient-centered outcomes. With the organizational piece, it's an area that we would like to move into in terms of our funding portfolio, but we were also heartened to see that the improving health systems program has a high number of projects that focus on addressing disparities. The point is that we recognize, again, getting back to this multi-dimensional approach to reducing disparities, it's not a one-size-fits-all. We're going to have to see kind of a mosaic of all of these factors coming together to really have an impact, but we're trying to really get a sense of what our program looks like right now because that's going to be really important in terms of our guiding it and where it needs to go to be able to achieve its overall goals. We do have a advisory panel. The advisory panels were mandated within the legislation. Again, our advisory panel for the Addressing Disparities Program is a multi-stakeholder advisory panel. Dr. Dorian Miller from the University of Chicago is the chair of our advisory panel. Grant Jones from Denver is the co-chair of our advisory panel. This is a very engaged panel. They help us on research prioritization. They are helping us in terms of really thinking through what this conceptual framework in our driver model looks like. They will be helping us in both the dissemination, implementation of the work that we're supporting. So they are a very, very active and engaged advisory panel. This is just a picture of our advisory panel members. The members are listed with their bios on our website. When we prioritize with our advisory panel, when we prioritize topics, what we do is we present them with topic briefs. We commission topic briefs. We work with ARC on this, which are usually two-page briefs that scour the literature, highlight where the gaps are, and then we have usually two-day meetings where we go through a process. We usually go through about 12 to 15 specific topics. And then based on that, we use a software program for then prioritizing. The criteria for prioritizing, the five criteria that they use are listed here. You don't have to take note of any specific element of this slide at all. The only reason I put it up here is that the process for who gets appointed to our advisory panel is a really rigorous one. The advisory panel members have to be approved by our board of directors. We received over just about 1,300 applications for a total of 84 slots. So we had 21 slots on our advisory panel, so we went through a very rigorous process because people ask us, how do you get appointed to the advisory panels, any one of the advisory panels at PCORI? And so my point here was that it's a rigorous process, but the advisory panel members have terms. They have one, two, and three-year terms. So the people who are on the one-year terms will be rotating out and will be opening up the advisory panel seats again. And anybody is open to apply. So I said that we rank topics. These were our first set of topics that were ranked by the advisory panel in our April 2013 meeting, which was our inaugural meeting. You can see that the first topic is a pretty cross-cutting topic. It can hit across any condition. What we did here is we commissioned a landscape review that was conducted by America's Essentials Hospitals, which was formerly known as the National Association of Public Hospitals. And they did a landscape review working with safety net settings, including community health centers. I know they did some work here at the University of Chicago as well, really helping us to understand what barriers both patients and clinicians face, particularly in safety net settings, around managing chronic conditions. And again, the information that we've derived from that landscape review, which is posted to our website, it's available to the public, and gives us some really good insights in terms of some overarching issues that we really need to be thinking about in terms of when we develop funding announcements and what it is that we're looking for relating to outcomes that matter, specifically for patients who tend to get their care in organizations such as community health centers, federally qualified health centers, public hospitals and other safety net settings. You can see what the other prioritized topics are. Many of them we've moved forward with. The last topic here, we are actively currently doing our gap analysis to see where we can potentially target a funding announcement. We had a second meeting in September, which was a webcast, and then we had our third meeting of the advisory panel in January, and then we re-prioritized, and these are the five topics that were high priorities. So we're rolling up our sleeves and getting ready to work on these top five priorities. One of them, which was already embedded into our pragmatic trial funding announcement. So PCORI's research portfolio continues to grow. This is across all of PCORI. There are a number of states where we have yet to award, and we would actively like to do that. Many of these states don't feel like they're not ready. They're looking for research partners. They're looking for partners across state lines, but you can see we're beginning to have quite a bit of breach, but there are pockets where we have not yet awarded any PCORI awards. We did just recently fund the National Patient-Centered Clinical Research Network, which is a network of networks. This is part of the infrastructure work. So we have funded clinical data research networks, and we have funded patient-powered research networks and a coordinating center. And the goal here is really to develop that national infrastructure to be able to do comparative effectiveness research, to be able to do trials, to be able to recruit patients quickly, to be able to have a patient voice in the research studies that are designed. So it's really beginning to develop that infrastructure. So we funded 11 clinical research data networks. Five of these have a disparities focus. One of the networks is based in Chicago, Capicorn. We're really very, very excited about it. And then we have funded 18 patient-powered research networks. So hopefully that gives you a flavor for PCORI. As I said, we've been quite busy. There's a lot of information on our website. And in some ways, I hit the tip of the iceberg in this presentation. It was pretty high level, but I'm happy to take any questions or comments that you might have. That was really fascinating, and I'm just amazed at the work you're doing. It sounds great. I'm wondering if you've thought about this at all, how your work in this part of PCORI and also how your ultimate agenda of reducing disparities would be facilitated if we were working under a single payer healthcare system? Oh, I think that it would take out one big aspect of variation that we have right now. It would, you know, I think it would, I do think that it would still be a really important part of the agenda, even with a single payer system, because we know that context matters even when you have a single payer system. So I still think the work would be important, but there would be one element of variability that would be eliminated. Yeah. So you were involved in the Oregon Health Plan experiment, which was a fascinating time. Must have been very exciting. So what lessons did you learn personally in health policy working with Governor Kitzhaber that could be translated to the work you're doing now? Nobody has ever asked me that. You know, I mean, I think that what was really informative to me, and that was early in my career, was the focus on input. So, you know, Governor Kitzhaber, Senate Bill 27, basically came out with this notion of rationing healthcare based on, you know, based on health conditions and drawing a line. And one of the things that the legislation specifically said that this had to happen in an open environment, that this had to happen with complete transparency. So part of that transparency was getting input from citizens across Oregon. So one of my jobs was driving to every single county in Oregon and holding community meetings.  You know, this is a really important exercise to people who said, how could you possibly prioritize anything for anyone? And so I think the lessons that I learned there and how in health policy and how they apply to my work at PCORI is really the value of input. It does make a difference. It is very important. And you know, there was a part of my career where that wasn't a huge focal point. And so with PCORI, you know, I've come back to it. Whereas with an Oregon, I was really working, you know, at a very policy level, but it was very grassroots at the same time. With PCORI, I'm beginning to be able to see how we can integrate this aspect of health policy engagement input on the science side. Thank you, Ramana. That was really excellent. And I was one of the people who raised my hand regarding having applied. So I've studied a lot of these materials, but I still realized there was probably 80% that we didn't see until you presented it here. Actually, it was involved a little bit in that Oregon Health Study too. My job was to train the interviewers to collect a blood spot from the fingertip that could then be used to assess for DiB, HBA1C. And I mention that because the people who recruited as interviewers for the Oregon Health Study were lay people. People who were likely to be your neighbor, your friend, you're gonna let in your house to conduct the survey that was needed to evaluate the Oregon Health Insurance experiment. And so the training for those individuals, these were lay people, smart people, they cared about health, they cared about Oregon, but they had no experience, most of them, working as part of a research team. So as we developed our proposal for PCORI and our work more generally to engage patients. Yeah, so we have instituted training for the merit review component, and our engagement team is very actively working now. So we have areas that I didn't even cover because of time. We have a whole set of awards called engagement awards, and there are different types of awards. There's three different types. I encourage you to go on our website to look at these. But they are really focused on that component, that capacity building, the training, training lay people, bringing organizations together that are not traditional research organizations or organizations that have never really been part of a research team, but really have something valuable to contribute. So we have that, so from a structural point of view, we've instituted the engagement awards. And this is the first year. So 2014 is the first year for the engagement awards, and there's three different types. But also on the engagement side, there is a lot of work being done right now on what we're calling kind of an engagement rubric, what it means to really engage. And our engagement team is now working with the science team. So PCORI gives contracts, not grants. We don't award grants. So because we award contracts, we monitor kind of the work that's being done by our awardees. So we have milestones that are identified, and those milestones have to kind of be in place before we sign off on the contract. So one of the things that we have started to now institute is engagement officers, which these individuals are going to be talking to the patient partners and the stakeholder partners on the research teams. And saying, you know, this is not, let me do this behind you, we're gonna be working with the PIs and the technical teams as well, but really to work with the patient partners to say, what is it that you need in terms of training? What is it that you need in terms of capacity building? So we're trying to institute this at multiple levels because what you point out, absolutely, I think that we're trying to do it structurally and we're trying to do it at multiple different levels. Thank you. At one point in your talk, you commented on the short timeline and the difficulty of showing changes in a short timeline. And I saw that one of your priorities was reducing disparities in cardiovascular disease. I mean, a three or five year timeline just won't give you that information. Right. A generation or half a generation might be needed. Now, how are we gonna go back to the Congress in 2019 and show them? Yeah, so we have a team at PCORI who's developing actually, I mean, there are a lot of people who are evaluating PCORI, so we have a lot of external evaluations, but we recognize that we are also responsible for our own internal evaluation. So we have an evaluation group that is developing an evaluation framework. And so this question has come up and it's not just for cardiovascular, there are a number of areas where really being able to show impact, maybe a 10, 15, as you said, a generational issue. So part of what we're asking for are really kind of the foundational work, the proximal measures versus the distal measures. So recognizing that if we can hit these proximal measures that we're on the right path. And if we can show that, I mean, when we go before Congress in 2019, when we go before the GAO and say, this is what we've been able to do, there are clear things that, and we've kind of been the circles that we've been talking to, there are expectations that there are certain things that are so long term that we recognize. Holding you accountable for that and showing actual numbers is not going to be possible, but you need to show us the foundation that you've laid to get us there. Interesting, way up in the back. So you have a lot of questions in there, so I'm gonna start with the one that really kind of, the one that really stood out for me, which is why doesn't PCORI care about costs? So I don't believe that PCORI doesn't care about costs. We do care about costs, but I will tell you that PCORI was mandated not to touch comparative, to do any kind of cost analysis, cost effectiveness analysis. So that is within the legislation and there is a very rich, not that long ago history around that. So in terms of even the formation of PCORI. So there was a real sense that, that if PCORI was going to be doing any kind of cost effectiveness analysis that we would be equivalent to what is taking place in the UK through NICE. And there was just a huge political outcry. So when PCORI was established, it was specifically stated within the legislation that PCORI would not be conducting cost effectiveness analysis, which is different than PCORI doesn't care about costs. We know that cost is very important. We cannot support research that undertakes cost effectiveness analysis, which means any kind of cost effectiveness analysis tied to quality adjusted life years, quality of life. So any kind of, anything that reeks of cost effectiveness analysis PCORI cannot support. That said, what we are interested in is burden. So if there is out of pocket expenses burden of cost on the individual patient population, that's a patient centered outcome. So we would be willing to entertain proposals that actually focus on that. The last thing I want to say about this is it's an important question. A lot of people ask it. It's incredibly relevant. It's a piece of the equation that's missing from PCORI. But that's part of the reason why it is so important for us to partner with stakeholders such as the payers and the purchasers and so forth. So even though we can't support cost effectiveness analysis, we certainly hope that the work that we do support can lead to others supporting the cost effectiveness analysis piece, even though PCORI can't support it. It's a great question. Thank you for asking it. That's for your presentation. I really appreciate it. I just wanted to ask about what your thoughts are in the gaps that you're seeing where technology providers could possibly partner with these programs where to actually be able to get those programs out to the populations, deliver the communication, deliver the tools that are needed, and how we can possibly do that as technology providers. Can I just ask for a point of... So when you say technology, do you mean use of mobile apps and other... Publications, web application, real-time communication. Yeah. So we're funding a number of projects that are actually integrating mobile apps, lots of work focusing on kind of the broader use of HIT, health information technology, and we are very, very interested in projects that actually partner with organizations that can really push that information out. It's a very critical area for us. We're actually... One of the things that we have noticed is that there are areas where there are true gaps, right? There's just... If you look at the bookshelf, the bookshelf is empty. There's really not a lot of information out there, but then there are whole areas that we have seen where there's a lot of information. The bookshelf is packed. In fact, I don't know if you all still have books or if you've all gone electronic, but I still have a lot of books. So my books are... They're not nicely organized. They're sitting on their sides. It's kind of a mess, but that's the way I like it. But sometimes it's hard for me to get the information that I don't know what's on my bookshelf. I have to stand around looking for a long time. So we see this even in the context of the query such as decision aids. There are so many decision aids out there. So what we would really like to see is a synthesis of what decision aids work for who and how do we push that information out there so it's usable. So there's a glut of information, but it's not getting used. And I think your question about the use of technology, particularly mobile apps, that can integrate that information and get it out to the masses is something that PCORI is very, very interested in. Thank you for a very interesting presentation. So in one of your slides, I think you counted up an apportion the percent of grants that were going to disparities research at about 66%. I mean, some people might say that's a lot. And so I wonder if you could just unpack that and let us know sort of what percent of the 66% is going to close in the gap and what percent is going to rise in the tide. That's a, yeah, I don't know that I can specifically answer that question. It's a very good question. So, but, you know, let me answer the question in another way.  So that gets to the point of kind of maybe showing a gap between two different groups. You know, when we talk about disparities, the definition itself requires, you know, comparisons. I served on an ARC expert advisory panel to see whether quality improvement programs actually played a role in reducing disparities. And we did an evidence synthesis and we, you know, defined the exclusion, inclusion criteria for the papers that we would look at. And we went back maybe 20 or 30 years and we came up with 17. Because one of the criteria we established is kind of the comparisons. Like, you have to have two population groups if you're talking about closing the gap. And pre-post studies alone will not do it.  I'm simplifying this. We don't wanna see those studies. If the focus is on improving care, then you don't have to have a comparison group within the population. We want the comparator to be the intervention. Does that make, am I making sense? It is complicated. But the vast majority of our studies, I don't know where you're defining it, kind of the rising tides versus closing the gaps. The vast number of our studies tend to have a focus on the disparities projects in particular, tend to have a focus on specific population groups. And then the interventions themselves are the comparators. Could you give us an example of improving communication to reduce disparities? I saw that that was a topic. Yeah, so one of the topics that was prioritized through our advisory panels, the number one topic was on enhancing communication specifically for populations at risk for disparities. And that's also a very, you know, communications is a broad term. I am not a communications expert. I have many colleagues who are. So when I start to use terms about communication that are not specifically how they define communication, they always jump all over me. So I'm not going to try to do that. I promise you. Yeah, so what we ended up doing in that regard was to really try to understand kind of what are some of the barriers around people with chronic conditions, particularly in the safety net. So one of the examples that came up was, you know, this focus on performance measures. So a lot of work has focused on developing performance measures, paying on performance measures. So performance measures, their success is based on adherence, right? Adherence to specific measures. And so one of the things that we heard from both the clinicians and from patients in the safety net settings through this landscape review around communication was, you know what, people don't understand that this focus on kind of meeting your performance metrics and adherence and reaching all these goals puts a huge burden on patient populations, on individual patients. So just kind of meeting those goals and all those aspects. So if you're an individual with diabetes and you have to do all of these things, adhering to all of this is a huge burden. And that never, ever comes into the conversation. And it doesn't come into the conversation for clinicians in these settings. So how do we enhance communication around that particular topic? Cause this is around care, but we have whittled it down to the minutiae of performance measurement and lost kind of the holistic picture of what it means for the overall patient. Thanks. Thank you. Thanks a million for coming today. It's been so helpful to listen to your talk. My specific question is if we're doing a CER, there's been some talk among our patient stakeholders and our scientists who have wanted to know more about hypothesized mechanisms. And should we integrate that as a specific aim or perhaps put it as an exploratory aim, saying we believe clock genes would be useful to know more about. It would help us to identify subgroups and so forth. What might be your thoughts about that?  What I will say is that I do think that, you know, in terms of placing kind of the hypothesis within your specific aims, I think what you can do first and foremost is, you know, one of our criteria is the impact or burden of the condition or the disease. So make that case, make that case upfront. And then I would, as a secondary piece, add in kind of your hypothesis that supports that or could get to a specific gap within that kind of how you're motivating your proposal to address the specific area. So the primary could be what are rates of recovery, relapse risk and so forth, which is why I'm speaking to a primary interest. But we could take it one step further in the science and understand to help at least push that, nudge that along. Yes, and I mean, your question actually kind of makes me want to highlight one thing that though we focus on patient-centered outcomes and outcomes that are meaningful to patients, we also know that the clinical outcomes and other outcomes are important. So not to leave those out of the equation. Would everybody join me, please, in thanking Dr. Huston-Lynette. Thank you. Thank you. Thank you. Thank you.