Okay, I think we're gonna go ahead and get started so folks can take their seats. That'd be great Hopefully everybody had a chance to have a pleasant dinner last night and get people kind to relax We had really great sessions yesterday, and I'm sure the same will be true today So we're gonna start off. I'm gonna turn it over to Terry for a brief Review of what we heard yesterday Great. Thanks Rex and and good morning everyone Glad to see you. We do have a couple of people who either got ill during the night or not anything that we fed them Or or their children did so so we may be a little bit sparser So so want to be sure everybody who's who is still here actively participates You remember yesterday. We started off with sort of a groundwork laying session talking about principles of basic principles of testing for disease and risk which is something that's a little bit new for Genomicists looking at the risk aspects. There seemed to be general agreement that addressing tier one conditions Is is probably a good consensus starting point But then the question is should we add something else in terms of population screening and how do we make those decisions? I'm not sure we decided that but that was one of the questions that would raise was raised There was a Suggesting that maybe we need a Richards criteria Option for selecting screening tests as much as we we had for a variant curation And the importance of studying more of the biology of penetrance which our Colleagues my colleagues on the on the basic science side of the house and yours in basic science would be well set up to do We also talked a lot about genomic screening technologies the importance of setting a high bar for Implementing technologic advances when we're talking about population screening as opposed to research Bob emphasized the need to assess compound headers eye goats and something that we don't do as well or is as Process actively as we might And the the need to kind of lay out the handoff of positive tests to clinical providers Which in newborn screening setting is handled somewhat by the state labs There doesn't really seem to be anyone who's doing that In the adult setting and a lot of talk also about how people interpret negative tests both patients and providers And and the importance of studying that And talking about the logistics of population screening there was somewhat of a little debate about you know Is primary care better set up is our specialists more interested? How do we thread that needle presumably both can be involved, but how we set that up is an important question a lot of emphasis on the importance of meaningful engagement from the outside of a program and Incorporating social determinants of health and in that And then when in the community engagement session The importance of learning from the community what their values and aspirations are rather than those of the researchers Hopefully you can meet both There the point was made that you know modeling on on what is done in tribal communities that there are most other marginalized communities don't have those kinds of formalized structures and how then do we figure out who the community is and how we reach them and then Wouldn't it be nice if we could identify tests that are sort of almost ready for primetime that we could then Do whatever research is needed to sort of fill those gaps So so that was our our day yesterday, which was very active and very productive Our day today is yes, you've probably seen we'll be looking at evidence needed to support screening obstacles to screening And then we'll have three talks sort of summarizing the research directions that people have heard So far in meeting and we will Refine these send send summaries out to y'all to comment on and kind of go from there with that I think we turn this over to our session chairs who are a path and Dan take it. Good morning everyone This is session five the evidence needed to support screening Dan Rodin and I are going to be facilitating this It will be the same format as yesterday. We'll have our speakers and After the presentations, we'll have ample time for questions So our first speaker this morning is Mark Williams from Geisinger who and he's also a member of the genomic work group That organized this meeting and Mark's going to talk about the value proposition for population genomic screening Great. Thank you very much. Thanks for the opportunity to talk about one of my favorite things so what I'm going to do today is to provide at least A definition of value in the health care context Then I want to explore the concept of value from different stakeholder perspectives and then propose some areas Where we could be doing some research into value in population genomic screening So what is value? It's some sort of a relationship between outcomes and cost of care Michael Porter who's one of the leading proponents of value measuring in health care Defines it as value in health care is a measured improvement in a person's health outcomes for the cost of achieving that improvement Not being a health economist I try to think thing think about things in a more simplistic way and so I came up with this model of three by three grid where we have medical service or some other outcomes on the left-hand side that can be improved can be Unchanged or can worsen and then the cost of care across the top that can decrease be unchanged or increase and Obviously, we would like to be up in the upper left-hand corner in the green And want to avoid the lower and just some examples Many if not most immunizations would fall into the improved outcomes with decreased cost of care That's been one of our major success stories in medicine If we look at molecularly targeted cancer therapies These tend to fall into the in improved outcomes, but also increased cost of care These are new medications or expensive medications, but in many cases they're providing treatments for Previously untreatable diseases and at least by the proxy of reimbursement That our society in the United States at least has made the decision that there's sufficient value In the outcome improvement that we will pay for that therapy one of the most egregious examples of worsening outcomes with increased cost of care was the short-lived use of bone marrow transplantation for advanced breast cancer Which is an interesting case study that I don't have time to go into but unfortunately that one of the reasons that we have The economic situation in health care in this country that we have is that we spend a lot of time Down in the lower right-hand corner of this grid So how about perspectives on outcomes? There's lots of different kinds of outcomes that we can look at there are medical outcomes things like morbidity and mortality disease-free survival treatment effectiveness and safety and Relevant to a screening program preventive services There are patient-centered outcomes many of these are around service satisfaction timeliness access to care Health behavior changes empowerment and engagement knowledge and personal utility Healthcare systems themselves have outcomes that they measure that are not necessarily aligned with either of these although there is overlap They're interested in the cost that they incur and avoid the utilization of their services There are aspects to visibility and reputation and patient experience and then of course there are cost outcomes And we want to use standardized costs associated with the interventions in the health states Experienced by the patient so for an example would be a cost per outcome like a cost per quality The problem is is that measuring costs in the United States at least is extremely difficult We know a lot about charges. We don't know a lot about costs Now I want to break down medical outcomes just a bit because within medical outcomes There are really three categories that we can look at as we assess this The easiest to measure a process outcomes did somebody do something did you get a lipid panel to do the colonoscopy? Did you get a mammogram? But these are pretty distal to the outcome of interest So sometimes we rely on intermediate outcomes Which are a bow marker or some other sort of a marker that can be associated either positively or negatively with a particular health outcome so LDL cholesterol and Arterosclerotic cardiovascular disease would be an example there But at the end of the day, we're really interested in health outcomes Which is the change of the health of an individual or group of people or population That's attributable to a given intervention The challenge with health outcomes is that many times the actual health outcome may occur years or decades After the intervention and that's why we tend to rely on some of these intermediate and process outcomes So we've heard a lot over the last few years about the idea of moving towards value-based health care So what are we really talking about here? And so value-based care is meant to transform the health care system to create more value for patients However, we define value But I think there's some conflations here that are not accurate a focus on cost reduction without improvement in outcomes is incomplete although a reduction of cost for the same outcomes would in fact be a Value add because we're achieving the same outcome for lower cost Value-based health care is not the same as quality Although quality is a component of value-based health care because in most cases if you improve the quality of a process You improve outcomes and many times reduce cost Patient satisfaction is not equivalent equivalent to value-based care but Michael Porter would strongly argue that if we don't understand Outcomes from the perspective of the patient then we are likely to miss the boat and then improving patient experience associated with the value-based outcome does enhance value from the patient perspective and I this is taken from one of The articles by Ticeberg who's a colleague of Michael Porter's Where they really I thought laid out a very nice way to think about value-based health care that is centered around Understanding the shared health needs of patients so this engagement piece to define the outcomes of interest to a patient using that to design solutions to improve the outcomes and then Basically implement that while measuring both outcomes and cost and then ultimately if you find things that are working then Disseminate that and expand partnerships So let's drill down a little bit into more of the genomic population screening I'm going to come back to one of our favorite topics that we've heard about Frequently during this meeting, which is the CDC tier one conditions and I'm really going to give you a very quick snapshot of two Examples from the CDC tier one conditions outcomes from the my code community health initiative and then a very brief View of the cost effectiveness study from the rational integration of clinical sequencing study so The my code project we published a tier one outcomes paper in 2020 looking at the clinical utility of genomic screening for the CDC tier one conditions 350 patients With variants that are associated with hereditary breast and ovarian cancer lymph syndrome or familial hypercholesterolemia Did double-coded chart review and had a median follow-up of nearly two years As we heard most people were not aware that they had a tier one variant The vast majority of them were also eligible to perform risk management and we were Very gratified to see that 68% of these individuals actually performed some management Post-disclosure, so this information seemed to carry some value to them We also were able to make a specific diagnosis in that follow-up period in about 13% of Individuals a clinical diagnosis in addition to the molecular diagnosis and that supports the effectiveness of genomic screening programs in individuals with Previously undetected individuals So we did demonstrate some out some aspects of clinical utility the outcomes are mostly process outcomes imaging procedure analyte analysis with a sprinkling of intermediate outcomes like LDL lowering and polyp removal no health outcomes are measured within that short of Period of time and we did not measure costs. So we did not assess Value because we did not have the cost piece Now I'm not going to talk much about this because the next talk is going to be solely focused on this but Using some of the myco data plus data in literature. We in this project did cost effectiveness studies And when we looked at individual conditions, we saw that a population screening approach for hereditary breast and ovarian cancer syndrome was cost-effective by the threshold of a hundred and thousand dollars per quality adjusted life here in this country and Then we combined the three We found that there was cost effectiveness for screening before the age of 40 Again, I do this only to say if we were to use that information that is a value measurement cost effectiveness measures outcomes versus cost and That would plop us up into the upper right-hand corner there where we've improved outcomes at an acceptable cost I Think outcomes definition standardization is really critical here if we're going to do research We all have to be measuring the same thing Outcomes are critical to determining the utility and value We're really just beginning to collect outcome measures for genomic medicine a lot of NHGRI and NIH funded programs are collecting Outcomes and then there are other institutions which are also doing this as well I just want to give three quick examples about how we can begin to think about harmonizing outcomes In emerge three We worked with the clean gen action ability working group to compare health outcomes who were being captured in emerge three with the clean gen outcome intervention pairs and we identified concordance and discordance as a starting point for harmonization That was published in 2018 Ignite Created a number of outcome measures which are currently represented in the phoenix toolkit They have an extensive list of outcome measures from different sources with validated tools however a relative minority of them actually are focused around medical outcomes and that was published in genetics and medicine and then Caesar to Developed a number of survey measures and outcomes Including clinical outcomes health care utilization and health economics and they engage not only with Clinicians but also with payers and other policy makers to make sure that we also paid attention to outcomes of revinterest to these groups and they came up with this graphic that Illustrates the idea that the perspective that you take is really important And you can see on the left-hand side here that we have to take into a consideration populations Timing the various conditions and data sources So outcomes definitions and standardization I think is going to be essential as an underpinning to research in genomics we need to Foster engagement with broader stakeholder groups to expand outcomes that would contribute to a more holistic consideration of value and That the definition of health outcomes is of most important From the patient perspective that we need to understand what the patients are looking for as health outcomes and we use that to Orient ourselves. We have to make sure we're using standardized definitions of cost and capture those costs And in conclusion, I'd just like to again return to a quote from Michael Porter Value in health care is determined in addressing the patient's particular medical condition Over the full cycle of care from monitoring and prevention to treatment to ongoing disease management So here I think we're talking about that monitoring and prevention Stage as part of a screening program. And so I thank you very much for your kind of attention. Look forward to the discussion Well done mark under time Our next speaker is Dave Veenster from the University of Washington He's going to talk in more detail about that cost-effectiveness study that Mark showed Great, thanks a lot Pat and thanks to organizers for inviting me to speak today David Veenster University of Washington And I'm a health economist and decision modeler and we've applied some of these tools To try to get at the issue of cost-effectiveness and I'll talk a little bit about that example But I want to talk about kind of the broader landscape of population screening using genomics I think Mark kind of covered some of these key points here You know just about costs in health care and value and why people are increasingly interested in it there are formal methods cost-effectiveness analysis that have been used developed and used over the past 30 to 40 years and Slowly over time. You're seeing increased use of these formal methods Across the world and increasingly also in the United States So that's kind of the framework that we've used now mark had a nice chart that is a different version of this This is the classic cost-effectiveness person will show you this they call it the cost-effectiveness plane And it's the same thing. It's just showing look your costs go up or down and your outcomes improve or they don't Most things in the US upper right quadrant means their costs more and they improve outcomes And that's why you where you want to say hey, what am I getting for my money spent? So there's this study that's been cited a few times This is a project. We actually worked on for about five years a nice group of collaborators from University of Washington Vanderbilt and Geisinger And Greg Gizoskas who's actually has a PhD in public health genomics from University of Washington Was the lead modeler on this project? I don't you guys have heard about these CDC tier one conditions, so I won't go into that This is just a schematic of this model As less said it's a bunch of it's all probabilities and that's what these models are they're really they're not about cost These are just a bunch of probabilities multiplied together and Jonathan Showed you like the next step and going way on just probabilities starting to multiply those together getting number needed to treat and those types of things And these models just do that. They say what's the probability of this happening? What's the probability of that happening and all those things we've talked about the Reverend bays is in there somewhere Number needed to treat but we say at the end of the day if people have this information a certain proportion of them Will act on it that will help prevent future disease that will save a life that will save life years that will improve quality of life and We measure all these things call it using a standard metric in health economics called the quality adjusted life here It's basically a year of life with good quality of life. That's all it is if it sounds confusing Just think of it as a life year What else do I want to say this is a simplified schematic Of the overall model and we just basically used it's a decision model up front You just separate people into different types of people people with variants where you know about it people with variants You don't know about it And once you kind of have them in those different buckets Then we use this longer-term simulation model over in the bottom right hand corner there and we just we Track what happens to people over time At some point they might get a cancer for example and what would happen to them might they die from that What are the costs for that? What's the impact on their quality of life, and we just track them over time? That's that's what these models are doing Getting some of the inputs for the model was challenging even though these are fairly well known conditions Kind of at the last minute during revisions in response to reviewers We were fortunately able to get some nice estimates of prevalence Across different ancestries from the healthy Nevada project so that that was that was quite nice and You can see the estimates there and We need more data on this I think we'll be seeing this coming out of the all of us study for example But maybe not quite as much variability across all three as we expected, but within the different syndromes There is some variability. I just want to mention that you know the cost is pretty inexpensive Pretty relatively speaking the cost that we estimated there are some commercial tests out there in this ballpark of around $250 So we felt that was reasonable and I just wanted to mention that We did assume that People who tested positive got confirmatory testing now. We say here Sanger. We didn't really know exactly what to assume Whether you're gonna run a test that's better than next-gen sequencing You're gonna get a new sample what's gonna happen there, but we did account for that and for there's a really good discussion about false positives I think it's pretty clear especially for conditions where there's intense medical interventions like hereditary breast cancer you You would want to do confirmatory testing I sensed a Maybe concerned about the hassle of doing out or the cost of doing that But I would say from a cost effectiveness standpoint. It doesn't matter if that test cost 200 bucks to do a confirmatory test and only 1% of your population Has a positive finding you're adding like $2 to the cost of your upfront test It's it's not a huge issue in terms of overall value So that is something that we included Sorry, there's a lot of little small numbers here Actually, I think I meant to take this slide out But it reminds me to tell you that we did not assume that people just did what the recommendation was okay, that's not what happens in the real world and You know what we looked at Especially for HBO see, you know women make decisions about mastectomy and oophorectomy and they implement those interventions over time at different ages and These are important things to consider some previous work in the field is just assume that everybody goes and gets that intervention Which is ridiculous and we've talked a lot about, you know, how do people behave and that's You know this these models are all about probabilities But one of the most important things in them is how do people behave? What's the uptake of the intervention? So we included that we also built a whole Kind of sub model look at within the family cascade testing and what impacts that had we have pretty Modest assumptions about how many people would pursue that This slide is just to say and I think Mark just said is it If you look at each screening for each contention condition individually, it's generally Not cost-effective Maybe for HBO see and a younger patient population and these numbers dollar sign per quality Just to orient you about a hundred thousand dollars is how much seems reasonable to spend to get one next year of life with good quality of life, so that's the that's kind of a rough metric for you and It's also influenced by age generally more cost-effective when it's when people are younger for obvious reasons You're able to intervene they have that information on time, etc So we did three separate studies each of these one at a time Some promise but really not coming together as a whole But the key really is when you combine all these conditions and this is just showing On the x-axis it's showing the results at Different ages of screening so this isn't over time. These are different cohorts of people of different ages and On the y-axis is really the clinical benefit measured by a quality and you can see the contribution from the different syndromes, which I think is kind of neat You can see cascade testing helps out It doesn't it's not an overwhelming driver of clinical benefit and that all these things together really are kind of needed To provide sufficient clinical benefit to justify cost-effectiveness as I'll show in the next slide And again, you can see that more cost-effective in a younger patient population There's another chart again. Sorry. These are kind of information dense and just in some ways Here again, we have on the x-axis age at time of screening and on the y-axis We have actually, you know, what's the value the cost per quality and that hundred thousand Dollar per quality is the horizontal line across there and these these error bars just represent uncertainty with probabilistic simulations And it's just saying hey, let's take all three of these together at the same time What are the results and the probability that it's cost-effective and you can see under about 40 Pretty likely cost-effective based on the inputs and the assumptions that we modeled and we do we vary all of our estimates and probabilities We don't assume anything's fixed. So we try to account for uncertainty as best we can Um What else do I want to say here? I guess I guess one thing I want to say is These kinds of cost-effectiveness studies health economic studies. We don't calculate p-values These are more Bayesian in nature that it's not very often when you see interventions where even when you look at the uncertainty You're still below a hundred thousand per Quality so it's a pretty fairly robust findings. I would say and these Stood up to various different scenario analyses, but I think there are still some remaining questions that have been discussed here Yesterday and today Again, sorry for the dense chart. There's just showing these are all these different. What if scenarios we ran? We said what if the test cost more? What if there's a 50% relative decrease in it in follow-up adherence to follow-up? You lose your cost effectiveness Okay, things like that and there was also been some discussion about false reassurance Which I actually think is is one of the more important or most important issues that we should be looking at We made some rough estimates of you know if a certain percent of people Say 10% of women decided to not pursue routine Mimography screening for example and lost a little bit of clinical benefit there all the sudden all the benefit you got from finding Those one the the one one-and-a-half percent of people is gone So there's so many people that don't have a positive test result. You need to be very Pretty confident that they're they're not going to change their behavior and there's There's been there's been some studies on this. There's been tidbits. There's been intention to pursue different activities But I think that's the area that's ripe for research I think Caitlin mentioned this too. It's a good area for research. The nice thing is You've got power. You've got sample. It's not 1% of the people that you can study you can study the 90 899% of people Okay, so you've got the sample size the hard part is that you have to design the study well You want to have a control group? You want to look at you can look at their behavior before and after they were the test and ideally you'd have a control group and you look at the difference in The control group and the difference before and after in the intervention group. So there's there's kind of Nice observational study designs you can use to do this So I expect now that a lot of these programs are being rolled out We'll be seeing more information on this and it's also important to consider what happens to the quote-unquote negatives Were they just not told anything about the test result? Like maybe they nothing came across to them So that's people are going to behave one way based on that. Maybe they're it pops up in their medical record Unlikely but possible they would talk to a genetic counselor now those people might behave differently depending on how that quote-unquote Negative or non-positive result was returned. And so those are all issues that I think are extremely ripe for study I just want to kind of switch gears here a little bit and switch away from the tier ones and talk a little bit about Polygenic risk scores. We've just started some work in this area Getting a sense of it and it's it's very different Okay, you're the proportion of people that you're telling hey your little higher your higher risk You maybe want to do something a little different. It's not it's not 1% right? It's it's 10 20 times higher than that It's 10 or 20% of the population, but their lifetime risks are you know, these are not hereditary conditions The lifetime risks are much lower and you potentially have many many conditions, which Is is potentially somewhat analogous to the tier one conditions as you add things on So just kind of covering these and you know these are just like kind of a toy example back of the envelope calculation But you know my sense of it is the prevalence is higher, but your overall clinical benefit is is lower per patient and so and and more it's more so and So I think that cost effectiveness in this realm of polygenic risk scores is not going to be the same as CDC tier one I think it's going to be more challenging And I came up with these toy examples here very simple model To generate this graph and this is just showing I don't know what X and Z is here, but kind of on the left bottom side there is your incremental clinical benefit It's going up as you go back on the right side is the prevalence of the variant and that goes down as you go back And then on the y-axis is really value. So red's bad blues good And this is about where the tier ones look like So as the benefit per person that you find with a variant Changes the value changes and as the prevalence of the variant changes the value changes but the big picture here is that For a reasonable combinations of numbers that you're below that hundred thousand you're in the blue or the yellow Okay, so you're in the right space. You kind of can play around there. You've got some opportunity When I run this stuff for rough very rough estimates for polygenic risk course, it does not look the same Okay, you've got this tiny little corner down there where oh, yeah, I might be cost effective And so I think I think there's going to be some very interesting work here. I think Is that my sound okay good combining conditions is going to be critical I've got a damn one minute or two. Okay, so I'm gonna skip newborn screening. It's a very different world I think the main implications. I think we've heard this from other speakers You know the the prevalence drives economic value more than like don't focus on cost It's really about the prevalence. Okay of the variants of the underlying condition, etc We really have to have clinical action With a with an intervention that provides clinical benefit to show traditional economic value There's some work that could be done looking at value to families and things like that And I think that this issue around clinical actions is super important as a very nice paper I think it just came out this week from Jody Linder who's here in the audience from a merge three Looking at kind of what what did people do after they receive monogenic findings a nice example where there was a control group I am a co-author just full disclosure. So I'm hoping to see more studies similar to this one coming out And then again look it needs to be efficient and relatively inexpensive And I think a lot of those issues were we're touched on I think one speaker might have been Mike talked about Well, where's this actually gonna end up happening in the public or private sector? You know, it's not clear to me it's happening in both right now, but I think things will begin to move very quickly So just in summary I want to say Three things number one the CDC tier one is a great model for population screening Don't mess it up The trick here you got to hold two ideas in your head at the same time You have to combine conditions to get good value But if you start throwing lots of other conditions in there without careful evaluation, I like the dripping Waterfaucet analogy, maybe we can get to a pressure washer, but not a fire hose You're gonna don't ruin the value here. Don't confuse people So being very careful about how to add conditions And I think coming up with methods and approaches to figure out how to do that in a way that resonates with the evidence-based Medicine folks and clinicians and patients is going to be the trick And then last point is that genomic screening you've seen one you've seen one Okay, each different application is totally different in terms of Economic value and with that I'll wrap it up. Just think my collaborators Josh Peterson who's here in Jing Hao also Kopi eyes and Greg is Oscis who did the vast majority of that work that I showed you. Thanks my I'm slow. So the next two speakers are Our remote so I hope they're on Bruce No, no, I mean Bruce. Are you on? Yes. Yes. I Hope you're not excessively Viral-loaded so Bruce Korf from the University of Alabama talking about uptake and follow-up Evidence of uptake in action in response to screening programs. Thank you. Yeah clever as this virus is I don't think it's gonna transmit through the internet Okay, so I was asked to talk about Evidence of uptake in action and gonna focus on something you heard about yesterday from Kelly East namely the Alabama genomic health initiative Which is a state-funded project that is a collaboration between UAB and Hudson Alpha which Kelly mentioned yesterday and I'm not gonna repeat everything she said I'm gonna try to focus mostly on the motivations of people to participate and Also as you'll see the motivations of health providers it began in 2017 with two cohorts the rare disease cohort were basically people getting whole genome sequencing to establish a diagnosis not relevant for today's discussion The initial population cohort was any adult in Alabama and they would get genotyping using What was originally a global screening array and then evolve to the global diversity array? And a return of actionable variants with genetic counseling and introduction to supportive care the initial demographics We're not ones that we were particularly proud of the proportion particularly of black or African-American participants was slower than the the demographics in the state and it was more heavily weighted towards Female-than-male participation We did however cover all the counties in the state sometimes by working out with local providers to create pop-up enrollment sites and You know that did it help us to increase the diversity but not to the extent that we ultimately did accomplish with the kind of a second version of this and Our return of results were based on the ACMG secondary findings fully aware that that wasn't what they were designed for It was part of the group actually that that created that but it seemed at the time to be a list that could make sense for This kind of activity and it was about one and a half percent realizing that by doing this through a genotyping array we were missing a significant proportion of Variants and also we learned from this that the array has lots and lots of false positives We did a clear Sanger validation of anything that we returned though So we have high confidence in what we return this was the the list It was a moving target of course because it gradually changed over the years and we did try to keep up with it Things in red or things where we had at least one return of results usually many and arranged from people Who already knew that they had something even sometimes the thing that we found to folks that were completely unaware and Some who I think one of the more interesting cases early on was somebody who had had a heart transplant and had a family history of cardiomyopathy But had no idea why and that may be a kind of sad statement more on the way care was provided than on population screening But pretty wide diversity of of findings But it was actually difficult to track outcomes because these were people whose care could be anywhere in the state and Generally took the information well through a genetic counselor who reported it but it wasn't so easy to figure out after the fact where they were getting care and and what was found We did survey them to try to get a sense of what their motivations for participation were two thirds claimed to be interested in contributing to research Almost as many concern about a future health problem. Some were just curious Many about a third had more an interest in what this could mean to members of their family and then there were a few who were adopted and didn't know much about family history So this is a way of maybe filling a gap. The one that worried us the most was close to the bottom That this provided access to testing that insurance wouldn't pay for. We really emphasized in any of the publicity about this in the informed consent process And in the return of results letter, which usually was we didn't find anything that was actionable that this did not substitute for routine clinical testing if you had family history, let's say Breast cancer, for example, and I'm not totally sure that that message got through in spite of it having been hammered home multiple times and you know we we would miss some pathogenic variance given the way we were doing this and we weren't looking at all the genes you would look at if you actually had a clinical indication So, aside from false reassurance just in general there was particular false reassurance and circumstances where you actually had reason not to be reassured. And if you look across the entire group who were tested this is not focusing just on those with positive results. Not many made insurance changes. Lots said they were paying more attention to health and wellness though it's a little hard to know what that means. And the proportion who had actual follow up tests and exams and so forth matches pretty well with the ones that got positive results though, again, we don't have a lot of data on what exact tests were done. So, we stopped enrollment during COVID as you might imagine during the peak of the pandemic and we use that time to think about different ways to do this. Partly for what I mentioned it was a little difficult to track longer term outcomes and secondly, we're also in all of us enrollment site and at the time when we started all of us wasn't doing genomics but then they were beginning to do genomics and it seemed pointless to us to have the money they should enroll in the Alabama initiative when they could enroll in all of us and have their genome sequenced. So we worked with family medicine in three clinics one in Birmingham one in Hoover which is a suburb, and one in Selma which is in the heart of the poorest part of the state and where there was a family medicine clinic that was interested in working with us. Initially, we did not return pharmacogenetic data that we did a background analysis of pharmacogenetic variants but we didn't feel comfortable returning it because we didn't have contact with the primary care provider so we weren't sure what people were going to do with that but now, as we integrated into family medicine, we have a team of pharmacists led by needle empty, who review the, the medication lists for all the participants, and then the pharmacogenetic data that comes back and then they would issue a patient facing report which is pretty general and a physician facing report which is much more particular in terms of pharmacogenetic outcomes that might be actionable. And then that goes unfortunately right now as a PDF into the electronic health record. Well this changed our demographics a lot. Now, you can see the majority are black or African American, and that reflects the demographics of the clinics that we're working with. And so in that sense we were happy still there's a female predominance in terms of participation. But we wanted to get a sense of, you know, what, how do we work this so that it responds to needs of the community, something we hadn't done in the initial iteration realized that we should have. And so we set up a community advisory board led by the people you see here including Kelly who is, I think still in the room, and they then convene this board of 14 individuals, you can see the breakdown in terms of gender and race and education kind of on average a pretty well educated group of individuals, and they had a number of meetings, and these were designed around particular issues that you can see listed here. And the purpose was not to be a lecture on these things but to get some background and then feedback from the group in terms of ways that we could be approaching this that would be sensitive to community needs. So the survey was done before those sessions had begun. It's called here pre test the test was the sessions it's I think it may be a bit of a misnomer but anyway, that they did a survey before that set of sessions and then 10 of the 14 did a post test, if you will, survey and a little bit of data, based on what came from that. So the first thing is that optimism towards genomic medicine in general, actually slightly decreased after they participated in these sessions. At first I thought well that doesn't make any sense you know we were getting feedback from them and, and trying to get an understanding of what we were doing and we've actually convinced them that this was less than they thought it might be and actually when you think about the survey I showed you a while ago it probably does make sense which is that I think people are going into this and at least in the demographics that we're looking at with inflated expectations as to what it might accomplish and maybe have thought that they could learn more than they probably actually will learn from participation and so a more realistic understanding of what can be accomplished is perhaps what was actually happening here. If you look, pre test is in blue and post test is in orange test remember means participation in the sessions. There was not that much change in the assumption of how likely unnecessary medical tests would be it actually went down in their mind after participation in the sessions and this seemed to be somewhat of an increase in the degree to which individuals would have control over their health. They concluded it wouldn't have been as well received by doctors as they originally thought it would be didn't change too much in terms of how well received it would be by patients and these changes, the numbers are too small to do meaningful statistics. So they're really more trends than anything else, just a few quotes from this barrier testing one person said they wouldn't take the test because they worry themselves to death because of it. And then the expected concerns about life insurance wondering whether this would affect eligibility. Well, then the question was what about the providers and Larry Harold from the implementation science group at UAB did a set of interviews and surveys with the primary care physicians to get a sense of how they view this and looking at the climate of the clinic how stressful whether they're adequate resources and so forth. The most important thing turned out to be leadership engagement and sure enough that really is how this all started. The reason we went to this group was somewhere along the way the family medicine chair had come to me and said, if you ever have a genomic study that you think could be done in family medicine, please let me know and well we did. So he was really strongly supportive of this and we identified within the clinics number of champions also who just we're interested in participating and we're really hard with the clinic managers to set up the workflow, so that it wouldn't be disruptive we would have people in the clinic to obtain important consent and explain the study and genetic counselors continue to do the return of results. But they do notify the physician before the patient so that the physician isn't blindsided, and oftentimes would give us information about the patient that would help in terms of making sure that the information was received in a constructive way. So attitudes towards genomic testing and I think these are technical terms used in implementation science which I'll do man I'm not really an expert on. And, you know, basically what they were pointing out here is that none of these were sort of home runs this is a five point scale. So you can see that the attitudes are all sort of slightly positive but not as strongly positive as you might have hoped. The invention I think means the willingness to accept completely new ways of doing business, for example, look at outcomes, acceptability appropriateness and so forth those are all, again, modestly positive, not overwhelmingly so. And just a couple of quotes. In terms of helping patients take a more proactive approach. Think the tools underutilized valuable to give insights to help patients take ownership of their health and be more preventative. An interesting quote, a huge benefit is explaining to them how unique they are and the benefit of knowing. And we came into this figuring they wouldn't really have the time to do very much and we tried to take as much of that off their plate as we could. Interesting to me to figure out the indicate implications of changes and dosing and avoid certain medicines. We would have that we would be able to do on the front end and get that information into a patient's chart I think is what's missing there. It's been patient reactions, it's been positive I haven't had any negative reactions when I do get results it's like, Okay, this is super cool to know but then there are people who are interested in and say what other things can we add to my health record. But then the other hand patients that struggled with anxiety I'm just about to the end here. So the conclusions from this from the public point of view. We found that there was wide public interest, but it's not so clear that it applies equally to all the demographics that in the region, and expectations were not always realistic and from the provider point of view, it's critical to have leadership engagement it took a lot of support and question of whether that's a sustainable model is certainly an important one, and it has to be integrated into the normal clinical workflow that's going to get acceptance and I will end at that thank you. So we'll just go on to the last talk and then we'll have a long discussion last talk is Carol Horowitz who I don't see on screen yet Carol talk about April one screening opportunities lessons learned evidence to support screening. Hi everybody, I dearly wish I was there to be learning and speaking with you all. I miss especially the conversations we have between talks so please reach out to me for any conversations you'd like to continue. So it takes a village to do this kind of work we're specifically now focusing on April one screening and genomic equity. I'd like to call out to specific people, did you show how to get who helped me put this together. And I view as one of the futures of what we do, and elder mims your Robinson, who is one of my most important mentors. So I'm going to talk a little bit about on behalf of us and a genomic stakeholder board that we've been working closely together for quite a few years now about screening with an equity lens, the who's in house. So firstly in equity, what we're talking about here when it comes to April one is specifically looking at kidney disease and equity, and I'm hoping that in another 10 years we won't continue to see these trends, but one data point speaks volumes which is that while 13% of the US population is black, 35% of the US population on dialysis is black. So reasons for these focus on. At this point we are able to think about multiple determinants of health. Early on, and for I assume some of you also had this unfortunate experience of early on, people tended to think about disparities in a somewhat eugenic way with racial inferiority. When people started thinking about patients, I would say in general a lot of the early discussions around patients to me seemed a little bit disparaging that were blaming that people had, you know, people from certain groups had more disease because they were undereducated fatalistic not motivated and then we started turning the lens in at ourselves and saying wait a minute, we provide care to folks. What, what bias is baked in to the access and quality of care we deliver. And then as you as we know we're here particularly now to talk about April one. We started looking at some genomic determinants, and also social determinants. What's fascinating and important I think about looking at April one specifically as, as high risk variance impacting chronic kidney disease is that it gets to this intersectionality that people really starting to think very constructively about, which is that well races is social construct, ancestry has some biology, and together they can make an impact on on on an influence equity. So I'm going to share a little example of that below, which is, and this is our genomic stakeholder board actually conceived of this study, the community partners conceived of it. As we were starting to understand that people with African ancestry, particularly from West Africa had were exposed to sleeping sickness developed April one gene variants to protect themselves, but unfortunately same gene variants seem to be kidney harming for many people. On the other hand, because of structural racism, black people are disproportionately exposed to air pollution, which is nephrotoxic. And what they asked us to do is see if there's any intersection between these. And here you can see in the blue that people have April one low risk, their risk of kidney failure chronic kidney disease goes up as the amount of air pollution goes up. But in the maroon bar you can see that if you're able on high risk, there's clearly a gene environmental interaction and these two interact for like a to hit and like to influence kidney disease. So, getting back to April one. This hits, you know this sent a article from Terry Manolio and colleagues shows that it does fit that red box of something that is common and influences common disease, and that is really really important. So the lifetime risk event stage kidney disease is increased by up to 10 fold if you have April and high risk variants depending on what condition you come in with. But what's important here is it really does have a powerful relation to African ancestry, because of the reason that people develop these able one gene variants. So it's found these high risk variants are found in about one in seven people self report black race or have African ancestry, but it's much less common in other groups. And I think it's important at this point to acknowledge how our field is growing terms of acknowledging these perspectives on what happens when you're looking at this intersection between social determinants and genetics on disparities. When we first started this work, a genetic ethicist came up to me and said, do not touch this, you will set the disparities movement back 30 years. I decided that if I wanted to do work, if I was going to do research in this area, I would ask people in the community who are the ones who are disproportionately and unjustly impacted by inequities and chronic kidney disease. So my mentor on the right, Elder Minzi at a Caesar meeting, I think maybe even at the hotel you're at right now. And when he first heard about April one said, now maybe white doctors won't judge black people on dialysis as not caring enough, or not being compliant. They'll recognize that there's more to kidney disease than bad behavior. So, who do we screen for April one and what conditions what what do we think is ready for prime time now, and what kind of research do we need to do using April one is an emblematic thing but really we can think about this for other kinds of genomic screening as well. Well, first of all, somebody yesterday talked about pretest probability and well this isn't exactly pretest probability. We do need to think about that while April one is not a race gene, we need to figure who we would screen for April one. Not only which health conditions they have but here, what people's backgrounds are. So this map by Garrison at Carney and colleagues shows that in some parts of the world like in Europe. There's not a lot of April one, right, and you can see in the center in purple in West Africa where we think the gene variants began, where it's very common. And, you know, we need to soberingly look at the map to the right and show you know the slave trade routes you can see that April one positivity follows them so going to Salvador in Brazil and up the Caribbean and to North American. And, and this reminds us that you wouldn't want to do mass screening of certain groups of people, but even in places like Africa there are places where there's very low April one, and there are places that there are higher rates of April one. So we use this in our work, when we start engaging to see people want to get able one screening to say we think if you have any African ancestry if you think you have African ancestry because they're self black or African American. And we also do use things like this on the map to say there are people in certain parts of the world that look like they have higher risk of having high risk gene variants. Now the what what target conditions should we be looking at. So, if you look at existing recommendations and the chart below you can see in green, there's certain kidney diseases where if you have high risk variance increases your odds of having been having kidney problems greatly like HIV associated in Africa the odds ratio of 89 times. And there are experts who are now coming out nephrologists who are coming out and saying that you should be screening folks with these conditions to look at prognosis and for adherence. There's a lot of work in kidney transplant in the right circled in red you can see there are certain places where it just doesn't make sense at this point to recommend screening for folks with in this. One is people with diabetes when it looks like there's not really an increased odds of causing creating problems and the others folks that are older we operationalize this by over 70 some of the thought is if you've made it to that age without April and causing problems it may never cause your problems. The ones in orange are the ones I think we should spend a few minutes on because I think that's where a lot of the literature that where we need to do further research. And that's looking at people with non diabetic kidney disease, particularly people with hypertension. And people with early kidney disease for other causes. So, this this figure shows some areas where we think that we need more research. So, starting out in terms of controlling high blood pressure, who should be screened among people who have high blood pressure, does it make sense to screen everybody with African ancestry or consider themselves African American word or some Latin American folks. And because people more than 50% of people with African ancestry and black people have hypertension. This is a really big group. I'm going to show you a little bit that there is some evidence that if you screen people and return high risk results to them. Their blood pressure does come down. Short term, we need more research on long term, and there is some research that if you find people who have high risk April invariance and aggressively control their blood pressure, it reduces mortality. And predicting CVD risk cardiovascular disease risk, we know that population wise it is really, we can see that, you know, that that people have these high risk variants and they're associated with kidney disease. What people are doing now and this is a really important avenue for research is how do we refine this for greater utility. So, not everybody who has high risk gene variants is going to have kidney problems. We're looking at gene gene, gene clinical and gene and gene environmental interactions to try to refine this utility. And finally, we need a little bit of more work in terms of preventing chronic kidney disease and chronic kidney disease progression. Can we find certain ways we should treat hypertension differently and treat early chronic kidney disease differently, because someone has these high risk variants, and also what new therapies do we have to make this even more actionable. So as I said I was going to share a little bit about some of the work that we've done. And our group did a primary care pragmatic trial where we randomized people to receive immediately or delayed receipt of their able on risk variance. These are people who self report African ancestry who are adults in New York City. We had full community engagement. In fact, I would not have done this study unless elder mimsy and colleagues in the neighborhoods that we are studying said, we support you we think you should do this also. And they helped with every single thing that we did. We recruited over 2000 patients at 15 primary care family medicine academic community safety net sites in New York City. What we did is we offered testing to folks. We had genetic counselors trained staff that are from the community, those staff recruited the folks and they return the results. And then providers got best practice alerts in the electronic health records. What you can see here is when you take the people who are told that high risk able on variance versus people who are told they didn't that people have high risk variance had significantly greater decrease in systolic blood pressure than the low risk or people who are controls and there was also more chronic kidney disease screening and chronic kidney disease screening is very low and inappropriately low. What's mentioned is that there is a study showing that intensive blood pressure control in people with able on high risk genotypes does confer survival benefit. You can see your strict blood pressure control and yellow and able on positives versus usual blood pressure how to survival and manage, and even strict blood pressure control and able on positives versus strict blood pressure control in low risk folks need a difference. So how best to screen. What do we do. Translation is a team venture. So what we what we have developed here in New York City is what you call an accelerator model where we say all these folks in in the middle tend to be siloed siloed ideas, questions, strategies. We bring them together as we did for the able one studies and this genomic board I talked about is now been involved in 15 other grants millions of dollars funding and really some amazing breakthroughs. And we bring together patients and advocates clinicians researchers funding public health folks industry, whoever we think will impact this problem. So together we can come up with new designs and processes. And the important part about this is bringing people in from the inception and calling people in, not saying clinicians won't do this or patients won't do this or industry is bad, but how do we all get together and say what do we have in common and answer for each group. Why is this important for me. What do I do with the results. When we go out to clinicians with April one they say, So people ones positive what do you want me to do. You want you to do your micro album and testing. Okay. And if they're positive we might want you to we're going to recommend you change medications that I buy. Well, if they're already on those medications I already screen them what do you want me to do nothing. Okay, then don't take up hours of my time. Tell me what I need to do with the results be very practical. Same for patients. That's the first thing. The second thing is pilot every step of the way.  And really look at the assets look at every one of these people and recognize what they bring to the table. And take the long view. So here's some possible research questions when we get to the steps. What are we doing. First of all, how do we select appropriate patients. As we as you saw on that map. Who are the ones who are most likely to have high risk gene variants. How do we select them. The next thing and I hope the group can think about this. Maybe it's a non starter I don't really know. But are we going to consent every single patient for genetic testing forever, or is there some point when it comes to April one or pharmacogenomics or something where we're going to say we're going to treat it like another lab test. I don't know how that gets decided or who decides it but I think that's important to think about. In terms of results for clinicians. Best practice alerts work for some doesn't work for others how do we continue to refine the ways we bring clinicians in. So it doesn't take too much of their time one minute. Thank you. So we have to think about that for patients. What is the role of genetic counselors do they need to be involved. And as I said before we need to think about further treatments and how to benefit folks. There are two therapies. It's really important. Please remember there's a reason some things get funded more than others. You might want to look at this article in the journal about cystic fibrosis affecting a third few Americans primarily white and sickle cell but receiving about 10 times the funding per patient. Finally, the last two things I want to say are. Remember that we need to improve CKD that we need to improve CKD for everyone. It's not all genetic. And the last thing is be careful not being too paternalistic. We've heard they won't act. They'll be harmed. They don't believe us. You can see here when we ask clinicians of the patients who they thought would change behaviors. They said about a third most patients change behaviors. We asked clinicians if they thought patients would be concerned about insurance. Half of them said they would be our patients even after receiving positive tests were not that concerned. We said who'd be worried more than half clinicians said their patients be worried they weren't. So please remember to listen and learn from patients and the primary care providers. It's okay that people don't think genomics is as important as we all do. And my final thing I want to say is that I've heard a lot today about people saying that patients don't trust us. Clinicians don't believe us. Elder Mims you would ask you all who mistrust who and I want to challenge us all including me to recognize that the people with the patients and clinicians we're working with have a lot of genius and a lot to offer us. Thank you. Okay. So thank you Carol. And now is the time for discussion. I'm going to take the prerogative of the chair and ask my friend Carol to help me and Dan potentially see anyone down here that has their card up. So the same mechanism please turn your card on its side. And we'll be happy to call on you. This is a question for Carol Carol. That was a really nice summary of the situation with April one. So if I guess if you were to think about adding April one to a general population screening panel. What would be the barriers in your view of just having that be part of the panel regardless of the reported ancestry or self reporter race of the individual. I think as I said consent is one of the things remember you know we as clinicians when we order test we order test we don't get consents to do a creatinine. Or for a lot of the other tests we do. So I think it's the entire process I think it's the first thing is how do we order the test. I think it can be pretty simply done able and is easier than a lot of the other tests of folks do. It's getting the consent. It's getting easy to understand information with patients which I think we've done a lot of work qualitative and quantitative and I think we've gotten the patient the patient thing down to very very very brief and I think it could be graphic. And the final thing is continuing to make sure that the way we return the results to providers uses the best practice alerts like the ones that we've been using in the guard us study so that things are very easily actionable. And of course payers Rex and then Jeff and Jessica. Okay. I'm just a little confused. Rex is next. Oh, sorry. So thinking about this as a evidence needed to support screening. I think the presentations all gave pretty good evidence that there's evidence to support screening. So I think that makes the question then what screening and when and I thought, you know, Dave's presentation made a pretty compelling case that earlier the better. But I think some of the other presentations made the case that don't over screen. But yet I'm thinking about yesterday several people during the day yesterday kept and I've thought about this a lot kept pushing us back in the direction of well when do we start and I think Heidi was one of them when do we start thinking about doing this at births. So that's the earliest you can do where you can actually intervene on the widest range of things. And the other thing that I wanted to just factor into this and then I want to hear people's thoughts about this is, you know, the discussion that Dave had about sort of the value of PRS screening versus the value of single gene screening. And of course the ultimate PRS meeting is a genome sequence. So I'm just interested in the context of all of those factors that we've heard about just in the last hour. What about people, you know, what evidence do we need to think about going to screening at birth or is that still just a crazy idea? Well, I'll take a stab at it. There's a couple of factors to weigh in or to consider one is what there's a pragmatic aspect that was alluded to yesterday in terms of screening at birth, which is it's the one time where we sort of capture everybody. So if we're going to generate a sequence, that's probably the time to do it. I don't think we should conflate getting a sequence at birth and screening at birth because there's lots of different things that you can do with the sequence if you were to generate that. There's a lot of things you shouldn't be doing at birth with the sequence, but there would certainly be a defined number of conditions that would be appropriate to screen for at birth that, you know, coincides at least in some way shape or form with our current newborn screening panel. But I don't think we should also assume that it's going to replace analyte screening as I frequently say, no one's going to sit in an office looking at variants and phenylalanine hydroxylase and trying to infer what the phenylalanine level in the blood is. You're going to just measure it. So I think that that, how we do that remains to be seen. But I think having that sequence as a reference that follows the patient then allows for the opportunity to use that for indication based testing as people develop problems. So if they have developmental delay, if they develop seizures or things of that nature, you can use that sequence and reuse that sequence for a variety of different purposes over a person's lifetime. And then at certain points in that lifetime it would be appropriate to apply a screening paradigm. So whether it would be carrier screening when somebody reaches reproductive age or whether we're talking about population screening for disease risk as we've been talking about today. So it's the dynamic and multifaceted use of sequence that would be captured birth that at least is the way I think about it. And that is, you know, yes, that a genome can be used in that way. I completely agree that's a possibility. And probably at some point our genome sequencing technology will get to the point where the incremental improvements you're over a year are over, right? And then you have a complete genome that is stable that you will reanalyze for 20 years. I don't know who's reanalyzing a 10-year-old genome at this point. There's no one. You're going to get a new one, right? You're going to reanalyze it as an opportunistic thing. But if you need to go back and do another diagnostic on somebody, you're probably going to upgrade to a new technology that's going to catch all the variants you need. So I guess my central argument about the sort of should we do it all at birth or should we think about other opportunities for screening is that there's lots of conditions that you will want to screen for at different time points. And I think in terms of operationalizing that the barriers to getting the whole genome at birth that gets reanalyzed and follows people over lifespan I think is an enormous amount of barrier to achieve. Whereas building out really well-designed targeted panels right now for screening at different age and time points is fully feasible and can be done. And probably be pretty cost effectively so that an addition to the newborn screen with a 10, 20, 30, 40 conditions that you check for by next generation sequencing could clearly augment what we're currently getting. And then we can think about offering that type of screening to people throughout their lifespan for conditions that are relevant. I think APOL-1 is a really good target for childhood screening, right? You need to know about it in childhood so that you can prevent the glomerular disease from developing if there's an intervention for it. So I do think we should be thinking about that but also kind of when is the most appropriate time to be looking for things. I just had one thing and it would be an interesting debate between Jemeth and I since we have fairly disparate ideas about how to do this. But I think the other thing that we're missing our opportunity on is that we are doing millions of sequences and we throw them away. Where we do have an indication to do a sequence and we do it and we throw it away.  And so we should also be thinking about the idea that if there's a reason to generate a sequence for whatever reason we've retained that data and we use it. Because while I can't argue against the idea that at some point we'll have better sequences, the reality is that we've got a ton of information in those sequences that can be used today and tomorrow and the next day and probably for the next 5 to 10 years. And yet we just toss it away. Folks, this is Chris. I just want to make a point. First of all, we're part of a study of doing newborn sequencing. This is the baby seat to project. Some of you are familiar with the baby seat original baby seat project which tried to consent parents with newborns and I think they had challenges in getting them to agree to participate in the study because healthy newborn parents of healthy newborns had their mind on other things than enrolling their new child in the study but now we're recruiting in the first year of life. And I'd say there are challenges there too, but it is moving forward and there's a long list of things that will be returned and a lot of outcome measures that are built into this. So it'll be a while before it has reached the point where there's actual data to show but there is an ongoing study. You know, that all said, I think the notion of reusing the genome and now I take Jonathan's point that there's, you know, technology keeps changing maybe it'll someday be cheaper to redo it than to bother storing it anywhere but even if it were the case that you would want to reanalyze the genome. We have to grapple with the fact that individuals are unlikely to live out their entire life connected to the original health system that sequence that genome if that's where it was done. And so, you know, if you had your sequencing done as a newborn, let's say, where is it going to live that it's going to be accessible to the clinician years and years later assuming it's still readable and it's state of the art at that time. You know, in a kind of a integrated health system, maybe that works in our not integrated health system if you can call it that. It could be a real challenge and it may open up opportunities for ways of putting information in places that will be accessible people are willing to trust banks with what may be among the most sensitive information they have which is their financial situation and are there differences where, you know, banks of some sort could be put into place that would store the economic information for future years. So we have lots of people who want to say something so and I'm not true. Sorry. Oh, I think Kate was. Peter, Kate, and Carol is pointing to two people on this side. And I had my thing up before Dave, but he can go before me. It's okay. So, Jessica has been up for quite a while. Jessica, Jessica, Kate, Peter, Heidi, and then we'll go from there. Jessica, you're on. Well, first I have a comment and I have a question just to bounce off of what Bruce just said coming from an integrated health care system is still hard to track people. So even in that environment, keeping track of someone, they move out of that health care system and they're gone, you're not going to find them again. And so that's that's my concern about that approach of kind of following someone over time as where does it live and and how do you actually use it to help them. My question, though, is for Dave to come back to your cost effective. Is there a factor that you see in this entire process that's the heavy hitter of improving cost effectiveness? Is it the uptake? Is it the follow up care? Is it the cascade testing that we could focus on to really improve the cost effectiveness? I'm not worried about improving it. I guess like I'm putting my evidence based medicine hat on that Ned forced down over my head many years ago thinking about the US preventative services task force and that incredibly high bar, etc. There's clinical benefit. The costs are not a huge issue from my perspective. It's potential harm. I think Ned mentioned that yesterday. I think that's the thing where there's the uncertainty. What's going to prevent a large organization or recommendation body from saying, hey, why, why would we not do this? And I think it's concerned about potential harms. And I think it's the false reassurance is my short answer to your question. It's reducing the uncertainty about potential harm. To me, that's like the last major piece that really is a potential concern. All those other things that you could do that you talked about uptake access to care. Those are all very important. But at this point, I feel like it's maybe a bit more of a risk benefit assessment safety concern. And I don't know, Ned, if you have any. Okay. No. So Kate. Yeah. So I was going to make a few comments. So one is I completely agree with Jonathan and Bruce about the like our health care. Like, first of all, the genome keeps changing. You know, the technology keeps changing. There's no way it's going to be reanalyzed and move on. I was a little mystified by this discussion about consent. I want to point out that we don't really consent people for clinical testing as it is. We sort of talked to them about it, but there's no actual consent process. And I also want to like bring this back to sort of the real world of what's actually happening out there. So like, I think we sit and I have this in my own institution, like the genetics people have this whole sort of cognitive discussion about this. But I can tell you my nephrologists in the clinic next year are ordering 450 gene panels, willy nilly with no consent process, nothing about genetic testing and nothing what's happened. I was doing chart reviews for a project and like I literally people were like sending messages through the portal saying I would like to be tested for this. Someone is saying, oh, I ordered this genetic test for you. And here's your genetic test. Go to your local lab or in quest. So I just want to like, you know, like as much as we want to cogitate about like how genetic testing is happening and how we're doing it in the genetics clinic and we're doing the right thing about consenting this that and the other. I can tell you that's not what's happening there. And I think out in the real world and I think we need to think about what are the minimal standards that we think people should be imparting to people about genetic testing and how do we get the word out there and how we educate people to say you've got to tell them something other than just ordering their test when they ask through it through the patient portal. And so I just like, I agree that we have to do something but I think sort of having a lot of discussion about consenting and not consenting is not consistent with what's actually happening when you look around in the real world. And I just want to say that. I'm going to just stick in a comment and that I want to ask you a question or follow up question is that is what happens when the results of the genetic tests are in fact, if I find a pathogenic variant is those people referred back to the person who ordered the test, or, or is there a fall off mechanism. What we find in our clinic is that lots of cardiologists order lots of testing and then they have no idea what to do with it. Patients have not been warned that they might have a VOS and then they all go nuts.  Well, Dan, good question. That's exactly what I'm actually have this new center for Penn Center for Genome. That's exactly the question that we're sort of working on because I agree with you. Like that information is not being, being disseminated and worse than what you just said, I had another conversation with one of our neurologists who said, well, the neurology residents are all ordering all these tests and they're getting all these VOS and he's doing muscle biopsies on all these people with VOS to sort out the VOS problem because now they've had, I'm just saying, like, you're like, oh, like, that's worse. Like, you know, at least you're not getting muscle biopsy out of the situation. So I'm just saying, like, I think we need to think about, like, what we talk about versus what's actually happening when we go and look around our institutions and how we should be thinking about those types of issues, Dan, and it sounds like you and I have had exactly the sort of the same conversations. I was just going to say in response to Kate, if I could just add that there, you know, there is a patchwork of laws, state laws that influence this. So your experience in Philadelphia is, I think, different than New York and Massachusetts where I've been. Maybe there's a multi-center study. I would also say that no one, the physicians who are ordering the testing don't know what the state laws are. So that you're assuming that they have an understanding of those state laws. And I would suggest to you they have no idea what the laws are. So in New Jersey, you're not supposed to do this. I can tell you, I don't think they know what the laws are. It might depend on how strict the law is. Genetic testing is so broad. I think part of it is it might be up to this community to determine if there are tiered approaches to it. So it might be that something like pharmacogenomic testing has a lower bar and you don't need to have as much of a consent discussion. And maybe that's like assessing an allergy or doing a creatinine. But when you have other things that could lead to needs for biopsies or life-altering surgeries. So I think we have to start thinking about the nuances of this and how we divide up and it might not be a one-size-fits-all. And I do really support really more work to try to understand this better. So just a couple of comments when back to Rex's question around timing. I agree right now that the genome and the trackability of patients is in a state that I just don't think it's feasible to run once and think you're going to really make use of that data. So I think you have to really focus on the point in time you screen and what you can do right then in terms of the sort of cost and utility models. The problem with newborn screening is there's an assumption that you will then look for newborn things at the same time you might be looking for adult. And that means you have to do it rapidly because the intervention for dietary change and things like that is going to be critical. And the cost goes way up when you have to do a rapid interpretation analysis. And so right now they're charging over $14,000 for a rapid NICU genomic analysis. So I think another consideration is the prenatal period because now you have nine months of a framework to think about your results and what they mean. So it's another consideration of a time point to think about screening which although also has some urgency to it, which is why I don't think carrier screening should be effectively done in the prenatal period where two-thirds of it is being done today. At the same time it's probably in some ways better for the newborn aspect. So just one thing to think about in terms of the timing question. The other thing around the consent that Kate brought up, I do agree with Mike. It varies by state and Massachusetts is an extremely strict laws around that consent process that led us to come up with a very focused consent form and implementation of a policy and ease of electronic consent in Mass Gen Obrigham system so that every genetic test order would go through the same process and be supportive. So I do think that that is something that would be a good thing to focus on is coming up with that minimum standard so that this process isn't a 10-page consent form that takes forever. And we can really hone in on the critical elements of that consent so it can be universal and sort of standardized and include things like data sharing which are critical for our field. Hi. Hi. Thank you. I have a question for Bruce. I think you mentioned that your program that you presented on was state funded by your state of Alabama. I wonder how you built the momentum to approach the state and get them to support this effort. So this came about, I guess, at a point in time when Hudson Alpha was fairly new and I think there was a lot of interest in the state supporting genomics in general and basically the state legislative liaisons between UAB and Hudson Alpha I think worked together to show this as an example of how the two institutions could collaborate and do something that would help the citizens of the state and these two institutions which the state was, I think, really supportive of and interested in bringing visibility to. So I was asked to write a kind of a short, this was the kind of shortest proposal I've written for the largest amount of money maybe over time. But, you know, this went through a legislative sort of process. I don't think there was a lot of written or anything but it was just an allocation that I think reflected the trust and the kind of esteem that the two institutions were held in in the state. So it turned out, although I don't know what really went on behind the scenes, but from my point of view it was something that they've embraced and now we're into seven years I think since this started and we get money each year so probably the most I can tell you. Congratulations. I often thought about how to approach the state that I'm living in which is Texas because I think it's an area with a lot of diversity, big population and I think that this would be something very important for the state. At one other point I wanted to make it not to do with the previous question but the time to screen, I wonder if the thoughts about screening at around 18 years of age, I think there's a lot of concern about screening of newborns without their consent obviously and then using this information as they are into adulthood. So perhaps at 18 at the time of driver's license or registering to vote that this could be something that could be used then for the adults with their consent and as we've mentioned many times screening earlier seems to have better outcomes. It's an interesting concept in the sense that it makes a lot of sense for the reasons that you put forward but it's also the time as a pediatrician I can reflect that it's about the lowest interaction with the healthcare system that you can have but the idea of using a model that would be outside a public health program that would be outside of the healthcare system model I think is really an intriguing thing to explore. Peter has been waiting longer than anybody else. Thanks. I have a couple of comments based on Dave's presentation and the subsequent conversation and then I had a question for Carol. So first Dave as you mentioned I think if you've seen one genetic screening program you've seen one genetic screening program and I think that goes a little bit to this question of over diagnosis. So in the model that you ran if I remember correctly you looked at prophylactic surgery for HPOC didn't include sort of increased screening starting treating in earlier ages. So if the intervention was start getting screened in your 20s you might start to see the offsets of over diagnosis in earlier ages so that might sort of offset things a little bit so getting to the when to start screening earlier is not necessarily always better depending on what the intervention is. The second point is it's a nice framework for exploring areas of uncertainty so again if there is some uncertainty around what the penetrance is you can always run simulations like what if we're off by a factor of X or Y and it may actually still even if you have that uncertainty it may still make sense in a reasonable range. And then finally I don't want to necessarily underscore your general comments on polygenic risk score but I do want to emphasize that it's still worth exploring. So the SysNet group did a cost effectiveness analysis of polygenic screening for breast cancer found that it was cost effective but under a scenario where there was a very detailed sort of schedule of what the recommendations are so they're like five tranches and if you're the lowest tranche you can start screening later and only come back every three years. So in principle you could find something that sort of hit the sweet spot that was cost effective but going back to my comments yesterday is a neck it's really complicated to sort of implement so there might be a trade off there. So I do have a question for Carol if you want to. I just said I didn't mean to be super negative on PRS. I'm just saying it's a very different world but I think you can combine and cross conditions. That's you'll get some value there but you can't be too complicated to your point. And then so Carol yesterday April talked about carrier screening and how there were some missed opportunities if you sort of recommended carrier screening only for people from certain groups. But you were suggesting that the value here is focusing on people of particular ancestry. So I guess I'm wondering how you see that being operationalized. Is it based on people's self reported ancestry or race ethnicity or are you looking sort of at genetic ancestry that might be derived from sequencing or GWAS data? I mean you know it seems my understanding of the literature is that you pick up somewhat similar people, some different but I don't think that we're going to characterize people's genetic ancestry and then offer testing. That seems while it's careful it's not very practical. So I think the only thing we can go by is how people self report their race and their ancestry. And I would say that we do need to be careful about it. That's why we need to have people from diverse communities at the table, representing advocate communities to make sure that we're doing the right thing. And I would say in New York City when we proposed this to people and we went out and said, how do you feel about us offering this just to this population? There are some clinicians that were worried. Our patient advocate communities weren't as worried. And one thing we heard again and again is this is one of the first times we have seen that we will be among the first, not the last to benefit from advances in science. So I think it's who we have at the table on how we frame the conversation that will make us successful. Folks, I'm going to need to sign off now. I actually have to give a talk at another meeting and apparently I'm being introduced right now. But if there are questions for me, just you can put them in the chat or let me know by email and I'll try to answer them. Sorry, I have to duck out. Thanks. Thanks for participating. Bob, I think is next. Yeah. One comment about Heidi's suggestion of prenatal screening, which is in some ways really interesting because of the timing issue, but it adds the huge complication of what some people in the prenatal screening business cause called the intolerant, the variants that are incompatible with life and where the fetus is going to be miscarried. So it opens a significant warmth. My question was for Bruce, I think. Could you talk about the, not this. Could you talk about the evidence gaps or data gaps necessary to generate qualities for some of these possible diseases? Yeah, for Dave, sorry. It's my impression is that qualities have a lot of data inputs and some of them have to be very crudely estimated. And could you say something about that? Yeah, I don't want to go down too much of a route. We have entire conferences on this quality metric. Again, it's the quality just of life here.  That's the main thing the model does. And then the quality is really just saying, are you also affecting people's quality of life? You have to come up with an estimate for, say, what's your quality of life once you have metastatic breast cancer? It's not as good as if you're totally healthy. So you do that usually through some type of survey or research type approaches with patients or their general population. So that's a general approach. There's kind of uncertainty in measuring them. That stuff doesn't, frankly, worry me a ton because the important point is to capture it to some extent. There's other issues with quality metric that if you extend someone's life who doesn't have great quality of life, that can actually be discriminatory against them. So there's all kinds of more detailed layers of issues with it. I think the field is generally uses it because it captures the two things that we manufacture in health care. More life and better life in terms of health. But it's not the be all and end all. And there's other metrics that you can use and there is uncertainty with it. Actually, my thing was up before yours. So I have a question for Carol and maybe I just misunderstood, but the presentation made a very compelling story that ApoL-1 is an important determinant of chronic kidney disease in African Americans. So why not screen all African Americans for ApoL-1 variants before they get kidney disease?  Why not just screen everybody? It's a really good question. I think the challenge is that there are certain places where the odds of having kidney disease, if you're ApoL-1 positive, go way up and certain ones there doesn't seem to be an association. So right now, as far as we know, if you have diabetes, it doesn't seem like having high risk ApoL-1 variants increases your risk of kidney disease, where it does for other things. So it gets to the idea of what does it mean to somebody if you're telling them they have a high risk variant when they have no underlying condition that actually means that they could end up getting kidney disease. And it also gets the idea of inflicted insight, right? Kind of an ethical thing. Are you just telling somebody something bad might happen, even if it might not happen? So I think we do need more research in there. But as far as I know, there's not a lot of ApoL-1 experts, and I'm not an ApoL-1 genetic expert. I'm an equity researcher who are saying that we should be doing ApoL-1 testing for everyone. However, there are communities, patient communities, University of Washington, who have done a lot of work bringing folks and communities together, where patients are saying, we would like to know, and why can't we know? And then it's going to get to who's going to pay for it and what is the action ability. So if somebody has hypertension screening from ApoL-1, make a lot of sense that they have diabetes and normal kidney function, people might be curious about it, but do we think that's a medically necessary thing where people are going to do something that's going to help them? We don't have that data yet. We can debate that forever, I guess. Jillian. I wanted to go back to Jessica's question for Dave, and a little bit of where I think Peter was going to was in thinking about sort of the post-testing interventions for folks who are identified to be high-risk and how much that impacts the models that we have and the degree to which we should be investing in research to find the right follow-up models that will both improve the cost of effectiveness, be better for patients, be potentially more affordable, because mammograms are not great tests. And so tests that tell you to get mammograms are just adding uncertainty in some ways. And there are other tests we have that may not be great, and so it seems like that may be a real opportunity. And then a highly specific question that may or may not be related to that is the difference in qualities in your data between BRCA 1-2 and Lynch syndrome. What was really driving that difference and that dollar amount between those two and the 30-year-old age group? Okay. I don't know that I actually directly answered Jessica's question as well as I could have. So thanks for bringing it back up. Yeah, I think making sure that people and providers and patients hear what the recommendations are and making sure there's good uptake. And what's the mechanism? Is this genetic counselor? So I think that's an area that's ripe for research, is helping people understand the recommendations and making sure there's good uptake. So I think that that's a huge area for improvement. Does that answer that component you're kind of squinting? I think I'm less worried about uptake, but more that the guidelines we may be giving aren't that great right now. Or do we know that, you know, the screening recommendations that we have for high-risk folks? Are these interventions in these specific patient populations proven? And I think you could have some debate about it. There's okay data that more frequent colonoscopies and people with lymph syndrome reduces the risk. It's not great data. But I also think there's, at a certain point, are you going to invest a bunch of money in a big randomized control trial like that?  Let me try to get at your other question, which was, it looked like HPOC provided more, as much or more, or was more cost-effective than, like, Lynch. And the short answer is that the surgical interventions for HPOC are so effective. That's probably the main driver. Some other things come into play, like prevalence and uptake and adherence. But those are particularly ovarian, very high-risk, very bad outcomes, very effective intervention. And even though not all women pursue it, it ends up having big effects. Josh might want to say something since he might remember our analysis better. I was just going to mention that the other thing to think about is what is the standard screening approach. And so with colorectal cancer screening, we have a very effective screening modality with colonoscopy. And so it's the incremental change that will sometimes drive the model as opposed to, you know, exactly what you're doing. I think the other piece that, Jillian, you were getting at was the idea that, you know, right now we're comparing it against the standard population-based screening recommendations. But what Peter was introducing is the idea that we could potentially do better at risk stratifying. Maybe there are people that we could, you know, put into a less intense screening approach, which would have cost implications that could potentially improve the cost effectiveness of the model because you're avoiding that. But the challenge, of course, with that is that we don't do this in a vacuum. And as we've seen with, you know, the debate over do we initiate routine mammography at age 50 or age 40, the battle of, well, we're missing people between the ages of 40 or 50 that have breast cancer. And that drives more interventions than the idea that, well, okay, there's more harms associated with doing it at a younger age as well. So we don't treat those cognitively the same. And so I think there's a really interesting question to be addressed that if we do put people into lower risk categories, will they, in fact, prefer to adhere to population-based recommendations or will they decrease their frequency? For colonoscopy, I can virtually guarantee they'll decrease their frequency colonoscopy. But for other things, maybe not so much. Can I make a quick follow-up on that? I think we often forget that given reasonable assurances of sort of health and safety, most people would rather not be in the healthcare system, right? Great, thanks. David, I wanted to ask you about your modeling of polygenic risk scores. Obviously, there's a continuum of risk when you estimate polygenic risk. And one way to think about screening using PGS is, or PRS, it would be to just pick the very tippy top of risk for a given condition, but do many, many conditions, in which case you probably would get a bunch of people. It seems to me that's not all that different from screening for a monogenic condition and should be cost-effective. But I was wondering what threshold you used when you did your modeling. The short answer is yes. I think that's, I think, I think that's how you're going to get there with PRS is being careful with those thresholds, picking them out, kind of one by one depending on the condition and combining. What I showed you was more of a cartoon. So our intensive modeling work, we're just getting started on it. But that's one of our goals. One of our key goals is using these methods to identify the impact of using different thresholds because I think that's really where you get your efficiency, right? That's what you're pointing out. And so I think there's really a lot of promise in that regard.  And that's why with Emerge, we tried to stick with the 2%. Then people kind of stretched it because the budget was there, which is not a good reason to do that. But at any rate, no, that's great. It'd be great to see that modeling. So my question, my question goes off on a different direction and Peters might be, yours might be more related to what this current track is. Yeah, just really quick to underscore Julien's point and some of the subsequent points is the evaluation of these will change depending on what the intervention is. And the directions are changing. So something that may not make sense now, like population bracket one screening for pancreatic cancer, there's not a great screening test. It's invasive. It's not very sensitive. But touch wood in 10 years if there is something that is cheap and not invasive and very sensitive and is really down staging pancreatic cancer, then maybe screening makes sense. Yeah. And improved clinical treatment for breast cancer makes the clinical benefit of screening lower. There's the cost issue, which pushes in an opposite direction lower. Yeah.  So it gets complicated. But yeah, we stumbled into those issues a little bit in all of our modeling, kind of what's the standard of care and the evidence is coming from an era of different standards. So it gets pretty complicated. I might just know we have four minutes left in this session. Yep. I was going to say that. Carol. I'll try to not take up all four minutes. So my question was actually for Carol, I was really interested in one of your last slides where you compared what kind of physicians expected versus what the patients actually did, the percentage differences. And I wonder if you think that's due to the way the patients were engaged in your particular model. And if that, if there's been comparisons of different engagement models and the responses that patients give or the actions that they might take following genetic testing. It's a good question. We used an engagement model to build the study itself. That engagement model informed how the patients were approached for the study and became part of the study. We had a less than an 8% refusal rate. So I don't think it's a cherry pick patient population. So I think the patients are probably representative of the general population in the New York City area anyhow. But I think the, you know, this was a very simple one. When clinicians came into the study, we asked each of them, how do you think your patients will respond to this? And this is what they said. Patients had a brief encounter, maybe a half hour or so with a clinical research coordinator. Then they got the test results back in a somewhat brief conversation. They were not involved in the bigger engagement model. And as part of the follow-up survey, we just asked them how they felt about the tests that they had. So I don't, I'm not sure the engagement model really influenced them that much. They might have been more comfortable with it because the people returning results to them were from their community. And we're similar to them, that's possible. But it still is striking to me. And I know for me, look, when I came in this as a researcher, I came with a null hypothesis. I had no idea if patients were going to be traumatized for this or not care, if they would be excited about it, if they'd be interested in the study. So this to me was really interesting data that the patients were saying in general, they weren't as upset as we thought they were. And that they liked the way that the participation went. So I don't think it's, I don't think it's, it's what you were saying, although it's interesting, but I do think more research needs to be done because New York is New York. Right. I could ask Ned and David to debate about adding things to Tier 1. You made the comment yesterday that Tier 1 was there 15 years ago, then you left the field and you came back and it's still three conditions. And I said some frustration in that. And Dave's argument was lead Tier 1 alone because it's a great model and we should sort of, there aren't other models. Sir, what's the way forward to, to expand this in a rational way without sort of, I mean, is the next step the ECMG 59 or is the next step to add TTR in people of African American origin or is there another rational next step? Yeah, I was trying to think about this yesterday and again compelled by the two first talks that the, it's possible for us to understand the evidence gaps that keeps, that has prevented more conditions from being added to Tier 1. And so I would reverse engineer it and say, okay, where are the gaps? You know, what's close and what I call it almost ready for prime time and how can we fill those gaps in? That would be one approach to saying, okay, we need the criteria, I mean the criteria for getting to Tier 1 are the same. So one way is to get more tests to meet that criteria. The other one is to think more in the provisional space or the study space. So if you're not ready for prime time, let's not mess with Tier 1. It would be good since we have Tier 1 to implement it. It's one of the things that makes me seize is the number of, you know, A and B recommendations from the Esperanto Services Task Force that we just don't implement and we don't implement it well and implement it for everyone. So you have three tests. Let's at least get those implemented because we believe there's both public health and individual health benefit in doing so. So that's one issue. But thinking about adding to it is, could you put in provisional studies? Could NHRGI think about, you know, who could do this in a closed system or in a system where there are enough controls in the evidence space to get things over the hump or into Tier 1? So that would be my approach. The last one. Then we're going to break. Yeah, no, I agree with Ned. The only other thoughts I've had is, I'm not saying don't add stuff, just saying don't mess it up. And it's really thinking of more of a, okay, we're there clinically. We're maybe good there economically. We're maybe good there in terms of implementation. I'd like to see this. It's being done now. We saw in all of us. More of a risk-based approach is that if you're going to add something to it, what's the risk of modifying the clinical benefit or the economic value, et cetera. So just kind of more of a risk-based approach to adding things rather than, oh, it has to meet this bar. And I think the answer is somewhere in between those two. And I think that based on prevalence being a driver and that if we were going to look for an evidentiary gap, C282-Y and hemochromatosis is the next obvious one in my mind. Well, the other thing is hypertrophic cardiomyopathy. Mark had to get the last word. I mean, I said Dave was going to have the last word. Can I have the last word? Hypertrophic cardiomyopathy is now on the CDC site. I think that's next up. The heart failure person who gets transplanted at Bruce said and had no idea that that was all genetic. That's a tragedy. We're going to break. Thanks to all of our speakers for a great job. We're back at 10.45. So thanks, everyone.