ory, the age of onset and the penetrance for these conditions in the population, not just in our ascertained affected cohorts. And we need these quantitative actionability estimates to know how effective our interventions are gonna be. And then based on that, I think we're gonna do what Les was suggesting is tune the thresholds for what variants get disclosed based on what the condition is and what we're gonna be recommending for the people that have it or that at least screen positive for it. And think about incorporating some of this into the cost effectiveness models to really give us a better sense of where in that process are the key features that we have to really tune to get the best performance out of screening. Well, I'll end there. And I think we have- Thank you, Jonathan. Thank you, Christine, Bob and Jonathan. There was a really neat follow up to what the first talks were this morning. Christine sort of leading us through what happens, has to happen in the lab. And Bob does what happens in the state lab and some of the concerns about that. And then Jonathan was a nice model for how to do it at a population level. I guess one of the things that was sort of missing a little bit taking off on my previous questions about the health system and how well it functions is also how well screening at a population level gets followed up. And so it hurts that we don't only fund the advisory committee in Newborns, which of course, Ned chairs and how the things get onto the rust. But we also fund states for doing Newborn screening follow up. And so that whole system, we didn't really say much about that sort of. So we start off Bob, I wonder if you could say a little bit about your experience in California about what it takes when you do population level screening to make sure that you follow up on all those kids. That's a key part of any large screening program. Yeah, thanks. So all of the Newborns that are identified with a positive screening result are referred to the appropriate specialist for diagnostic evaluation. And that, I think that happens essentially immediately and is really quite universal. I mean, again, it's part of the Newborn screening program. All of that communication takes a lot of keeping crack and monitoring. From after the diagnosis, things start to fall apart depending on how intensive the clinical management becomes and where the family lives. I mean, in California, there are families that live four hours from the nearest geneticist. And so they don't get the kind of care that people in other parts of the state get. And that's just geography. There's also language and, well, just general social economic status and then insurance coverage starts kicking in. So the gaps in long-term follow-up are real. I think I'll stop there. Thanks, Bob. I see Ned and Kailin and then Terry and then Heidi, which is for due. So I think this raises an important issue for thinking about the research agenda that we don't stop our process diagrams at we have a diagnosis and the clinician knows there's a whole lot of things that happen downstream from what we've learned from the white screening is if we don't calculate that into the number needed to treat, we're missing out on something important in our calculations. Ned. Yeah, I appreciate that. And Bob, I appreciated your discussion and I agree with you. I didn't want to come back to something less said that all those false positives, we just diagnosed them. It's way more complex than that. And we're doing research into the harms associated with reporting a false positive result back to a parent and the Odyssey that they have to go on to.  I think the other thing you talked about is the immediacy of treatment. And so crab A is difficult because the bone marrow transplant has to happen so rapidly to be effective that there's worry about the chances to make a mistake in that diagnosis. So we're not supposed to talk about new screening. Let's think about how these apply to adults because it's not that much different. If you have a false positive result, I would argue that wrong information is always a harm and treating a false positive or an over diagnosis is also always a harm. And so it's always that balance. So Jonathan, the only thing I would add to your wonderful talk is calculating the number needed to harm. All the therapies we're talking about, all the interventions to follow up on a positive genetic test also carry harms. None of our therapies or approaches are harm free. And so always thinking about, yes, I only have to screen three to benefit somebody. How many people do I screen before I end up with a harm? So I just always like to come back to harms. Yeah, I just quickly respond. So like, yeah, I agree with you, right? The number of false positives will, I think if we just trust the math, right? That these results are not 100% specific. If we're talking about like the pathogenic variants, then you could get astronomical numbers of false positives and that would just become overwhelming. And so I think that's part of the issue is tuning the types of results you get back to keep it so that you're at least in a manageable number of false positives and then try to get strategies to re-classify those and sort of that. Yeah, I think your name is Katelyn, I can't read sideways. So I think my question is pretty directed for Christine, but of course others chime in. I'm curious around the sample collection element here. And if you have thoughts or recommendations for criteria that organizations who are trying to implement population screening might use to help make decisions about what sample collection method to implement and thinking about from a practical perspective, we've at MUSC have started with a lot of sample collection and are doing that in clinical settings as well as at home, have not gone into blood yet, but because of the logistical barriers, but just would love to hear your thoughts around the decision tree or decision-making there. Yeah, thank you. You know, I think this needs a lot of consideration, you know, first about what your sample volume is going to be and then what your outreach is going to be. So if they're going to be seen at a medical center and it's going to be somehow combined with a visit, then your options are much greater than if this is going to be mainly by outreach through social media or other types of approaches. I think from studies that have taken place so far, it's been mainly non-invasive testing. And from the side I showed, and again, I was surprised at the level of complexity of the blood draw versus these other non-invasive. So you really have to think about the resources that are needed and whether the incremental improvement you get in sample viability and lower failure rates and long-term storage is that meaningful. Also, the mislabeling and there is no full-proof method for this yet and that has to be developed. As I mentioned, the dried blood spot card where you actually put your name on the card and then collect your sample, I think is maybe the closest, but anything where you have to put a label on a tube is prone to error. And then marrying up the patient information downstream to the sample, that's critical as well. But no, we don't, as far as I know, we don't have guidelines, but I think that is definitely needed depending on the scenario. And just one follow-up, I think it's been, maybe it's post-COVID, maybe it's just getting used to fit tests and at-home sample collection. It's just been really amazing to see how people are responsive to the at-home collection. And so I think that's been a lesson learned and really a surprise at home and then high return of those. So. Thank you. I forgot to mention that, which we might talk about. Yes, I think the at-home COVID tests have really sort of made people more knowledgeable and not so anxious about handling their own samples here. I wonder if we're on the technical stuff, Christine. If you could say a word about reanalyzing for variants and later on how the practical aspects of that, have you started thinking about those kinds of things? Yes, so reanalysis is even today in a diagnostic setting is a challenge. So making sure that patients are aware that whatever result they've gotten is in a particular window of time and then making sure that your reanalysis protocol is very well understood and communicated and that should be and is in our like exome and genome consent forms, how often reanalysis is performed. But in the laboratory, it's also a challenge. You have to have teams that are dedicated to your database and making sure that whatever changes you make to your previous interpretations are well vetted because you're gonna have to reach out to those patients again. So it's definitely not an easy process but you have to have the resources dedicated in your laboratory, the protocols and follow them. Heidi, you probably have some thoughts on this as well. Could I just make a quick comment about dried blood spots that that's what newborn screening is. Those spots if stored at minus 40 with desiccant can be analyzed after 20 years and give good DNA for an exome or genome. Yeah, I wonder if there are any differences between the dried blood spots that are collected by a healthcare professional. So newborn screening I think is usually in the hospital versus those that are collected at home. So I was not able to find any data about that. But do you know what the failure rate is typically for newborn screening programs statewide? Less than 2% have Terry and then Heidi and then Josh. We could talk about reanalysis all day long but I think we'll hold that. I have a technical question but you go ahead. Great, yeah, I wonder, Jonathan if you could expand a little bit on your comment on how do we get at prevalence of monogenic diseases and presumably yes, monogenic but breast cancer isn't monogenic. And yet we're looking for a particular gene variant. So both, how do you expand maybe to complex diseases as well as what would be the approach to assessing that prevalence? Yeah, so I mean I've seen sort of the extrapolated numbers based on, you know, if you say that of breast cancer some percentages are related to this gene or that gene and then you sort of extrapolate from the population prevalence of breast cancer to get the prevalence of that monogenic forms, right? So that's one way to go about it. I think that there's actually, my guess is that polygenic risk is probably gonna be more directly sort of applicable straight into the kind of the risk that a given individual has at a level of polygenic score that they have as opposed to with, you know, with our penetrance issues somebody might have a BRCA variant but then we know what the risk for them is some percent by some age is that modified by their other polygenic factors is it modified by environmental factors sort of how does all of that work together which I think is a lot harder whereas I think with at least the polygenic risk you've sort of evened out across all of the other factors that might be involved in the risk. So I think you go directly from the projected risk based on the polygenic score perhaps combined with other environmental things if you're doing a genomic sort of risk prediction that might be more direct to the risk of the individual. So this one's a little more, oh Heidi Rehm, Mr. Nohassan Badal and Brad Institute. So this is a little more probably directed at Christine's talk. Today we have, and you gave a great example of a carrier screening sort of tier four, lots of genes but you also have to supplement that with certain assays because of more difficult to detect variation and certain genes, fragile X, SMA, et cetera. In the secondary findings world we accept that those tests aren't comprehensive. In fact, certain genes have been left off the list knowing that they're technically challenging. And so we sort of view it as opportunistic if you happen to come across it, report it but we're not assuming it's comprehensive. And I'm wondering how we all think about the middle of the road here where we're all starting to think about run a genome at birth and use it throughout the lifetime kind of cost effectiveness approach. But that means that we'll be taking tests like carrier screening and sticking them on a genome where they're not optimized for the comprehensiveness of when we design a test for a specific task. And how do we think about using labeling those tests that are not perfect but we also, the cost to make them perfect would make them less useful at a population level. And how do we make it clear what this test is compared to the gold standard ordered intended for carrier screening, for example, versus we're trying to do this but it's not perfect. Do you have thoughts about how to label offer that kind of scenario? Yeah, very challenging question. I mean, I think we do run up against this in the diagnostic world already where we have our disclaimers and many labs do this gene by gene and the list becomes overwhelming. Exxon five in this gene, Exxon 52 in this gene are not well covered. And so understanding that and then being able to take that information and translate that to the patient to tell them exactly how good this test is for you is very difficult. I think for, let's say our carrier screening test where patients are voluntarily or to some extent voluntarily taking this test, you want to give them as good a test as possible. And so that's why these orthogonal methods are being introduced in order to get the sensitivity where patients expected to be. But I think looking at a population level, obviously it's going to be different and you're going to have to balance cost and time. Thank you, I think Josh is next.  This is a question. It seems like the sensitivity specificity framework works pretty well for identifying the genetic risk and false positives and true positive sort of thing. But I worry a little bit about using that framework for identifying essentially the connection between the risk and the disease because of time. And every, there's a distribution of disease incidents over time and when you go to apply, let's say the idea of penetrance to an individual person that you're trying to counsel or treat, then you need of course account for how old they are, but also essentially when they got that information. So if we're gonna be screening 18 year olds, how do you counsel them? And what do we need to know? What's the right metric to communicate essentially that risk that connects the genetic risk to the actual time related risk of disease? Yeah, I mean, it's a great question. I mean, that's the big problem, right? Is the goal is to find the people before they have symptoms so that we can do something to prevent them from having a disease so they never become penetrified. And so I think it'll be as we sort of roll out screening in the context of the interventions that we want to do with people, it's gonna be really hard to tell which of these people who have that pathogenic variants are gonna benefit from whatever intervention we're offering them, right? And so that trick of sort of communicating the population benefit that may not actually be an individual benefit is gonna be something that differs from when we sort of think about individualized medicine, right? We're doing this for individual benefit. We're actually not, we're doing it for population benefit. And that's part of the communication, I think of what those results mean for that person. Yeah, maybe this is part of the research agenda. I was just struck by some recent articles that showed that you could show stick figures in a diagram and you get twice the sort of actionability based on what patients like to do compared to, let's say a single probability number. So I mean, it's not only the metric itself but the way that you communicated. And it seems like we really need to know more about that in the context of genetics. And maybe to add to that, it's also the consent process upfront, right? That when someone's signing up, we probably see this in clinical medicine all the time right now that this happens to me as a primary care person where one of the specialists is ordered to test because someone had an eye finding and then they find a bunch of things they didn't know what to do with and they get sent to me. And I don't know how much counseling was done upfront. So it's both once we have results, but even before, I think understanding that better probably makes sense too. I have Erin and then Carol and Mark. Thanks, so we heard someone mention all of us and we know other databases like Nomad are increasing their representation from individuals from diverse genetic ancestries. But what else sort of are we at the point are those will those be sufficient? You know, we heard Jonathan you be saying sort of we have to tune our decisions to maximize true positives and minimize false positives. Do we, can we do that yet? Do we have the data from diverse ancestries to make sure those choices are gonna benefit all? I mean, the answer is probably not yet and certainly not for more of the rare diseases. Again, and I think it's, yes, you have to, if the well-established pathogenic variants are concentrated within people of European ancestry because we've seen them the most and studied them the most, then that's gonna be a problem and we'll have to figure out how to address that me. Make sure that the catalog can benefit most people. And maybe a quick follow-up to that Jonathan, how do we, and maybe Christine can help with this as well and Bob, how do we get these new data streams? So as we get more data, how do we make sure it goes someplace like ClinVar so we can use it going forward? You guys thought about that much. Sorry, I was not paying attention. Yes, so I mean, we're like, for instance, we're right now working on the submission from the All of Us research program which obviously has a larger diversity. We're also working through the Global Alliance globally to try to get every country to submit and support their submissions because you can't submit variants, you don't test from patients. So I think it's just a widespread model to sort of support data sharing at various levels including interpretation ClinVar submission but also the raw data and how we all use that. Just launched an OMAD-V4 last week and the contribution of new variants despite the fact that we dumped huge amounts of European individuals into this database. There's a very small increase in the number of variants from Europeans that were above a certain frequency that lets you exclude it. The contribution of the smaller number, 130K of non-European was massive and it just really demonstrates just how important diversity of data is not only for interpreting the population it's in but interpreting other populations. Thank you. Some question was kind of similar in that, Jonathan, you mentioned Maeve, sort of the multiplex assay variant effect and how data from that sort of project could feed into understanding the effect of all of these diverse variants that are reported in. I mean, what's your vision for how to integrate those types of data into this assay? So I would see the need for a good level of communication between the ACMG committee that's starting to say these are the things that we think are worth the population screening for. ClinGen and expert panels to get really high quality specifications for classifying those variants and the Maeve groups to tackle those genes, right? And between all of that, you should be able to have a really good catalog of clearly pathogenic variants and hopefully not as many that are likely path or kind of in that wishy-washy range so that those things could be rolled out with confidence. That's the goal. And that'd be a really good project. Mark Williams, Geisinger. So this is a comment rather than a question but I'm happy to have others comment on my comment. And this comes back to the idea that when you do screening, the balance of sensitivity and specificity results in residual risk which also needs to be communicated and falls for assurance. And we have plenty of examples from newborn screening and also from even direct to consumer testing like 23andMe where somebody has an obvious family history for coronatory breast and ovarian cancer and they say, well, I don't need testing because I had 23andMe. Or a child that presents with chronic rhinocytositis and pneumonitis and poor growth and floaty stools as well I had newborn screening for cystic fibrosis so we don't need to do a sweat test. So I think one of the other aspects of this that we need to consider is downstream how do we communicate the idea that this is screening we are intentionally going to be missing people some for technical reasons that these are just genes or regions that we just can't get at and others for rare diseases that don't meet the thresholds that we would define for screening and how do we facilitate recognition and testing? So at ASHG last week there was some discussion and some of the sessions about whether genetic counseling is actually needed for negative results. So we're all very familiar with genetic counseling for these types of programs for positive results but what about the negatives? And is that actually just as important or perhaps even more important that participants aren't left with the impression that they're no longer they have to be concerned about this particular issue. Yeah, to add on to our discussion from the first thing if we are arguing as some of us are perhaps all of us are that we don't have sufficient genetic counseling resources through the positives then I think we can fairly well assume that we definitely don't have enough resources for the negatives. So then it raises the question of how do we develop resources that can achieve some, I won't say equivalence but at least some acceptable level of communication for those particular issues since we know we won't have the human resources to be able to do it nor can we afford the cost associated with that. So I'm gonna say that the part of what Les said earlier was kind of these automated and computational ways of doing this and I think that the genomic learning healthcare system is going to need to know somebody's gotten screening what the results were but also all of the other phenotypic stuff that you just mentioned and the probability that that represents a disease, right? So that you can calculate what is the actual Bayesian kind of relationship between the fact that they had a negative screening test with its performance versus all of the phenotypic stuff and does another test need to be done and how can we rely on EHR to pull that information together and fly it for somebody to act on which I think will be a really interesting challenge for it. Yeah, it's well outside the realm of this but I think some of the work that's been done at Vanderbilt and other places to say we've got data and EHR that can make it so that we're not reliant on between the years which we know is going to not be successful to really flag folks and say this is an individual that definitely needs to be tested for CF or whatever based on a very high confidence phenotype that raises that prior probability. Sounds an awful lot like a genome informer is consent from the merge. I'm gonna call it quick time because if you guys purpose in trying to confuse me if your card is up and you don't have a question you can put it back down so I can figure out who's next. All right, excellent. I think it's, I think actually Dan is next and then Kayla and then Terry. So three random comments. One is I appreciate the shout out for the phenotype risk score part but we actually looked at CF and Lisa Basterash really couldn't find any undiagosed CFs using the phenotype risk score. So that's an example of the phenotype risk score not finding extra cases but I appreciate the shout out anyway. The business of counseling negatives I think that comes back to what Les was saying. If somebody has a very clear phenotype so I'm an arrhythmia guy so if somebody has a QT interval of 550 milliseconds and their genetic testing is negative they still need to be followed by somebody who knows something about QT intervals of 550 milliseconds. On the other hand, if it's population screening and they were screened for because they don't have an indication I can't see that we need to counsel those people. And then I just have a comment about the maves. We're sort of, I would say we've put more we put several toes in the water around maves and one of the things I think I'm learning is that what maves do is assign pathogenicity or not depending on a particular protein function that's being interrogated. So you can never be sure if you have a particular variant that looks benign on one assay whether it's gonna be benign on all the other assays that you might want. And obviously the field is moving very, very quickly and maves have nothing to do with penetrants. Leases as near as I can tell. So there's still this penetrance problem where you don't be left with. And the other problem that somebody's gonna have to solve with maves is that there are a couple of hundred KCNQ1 variants. For example, perhaps even less than that that have been annotated by ClinVar and ClinGen. And when we do a KCNQ1 mave map there are 13 and a half thousand variants. And so there's a problem scale and how we're gonna sort of be able to accommodate levels of evidence and just be able to present data in a chewable fashion to the wider community. A separate discussion, but worth, I have to say something. Any of our folks wanna respond to those comments? All right. Caitlin, if you put yours down, we know. Oh, I thought I was allowed to. So I was going to follow on to Mark's comments. I think there are, as far as research agenda and research questions, a lot of opportunities to think about from a behavioral perspective what's happening with people who have negative results. Are they, how are they interpreting that? What are they doing with regular screening behaviors? Are they stopping screening because of the way that they've interpreted their results? So I would just maybe emphasize that as a potential research direction. And then I think too, highlighting some of the work that Kim Kappingst and Guirame Adelphiol from Utah have done in the bridge trial and in their ITCR work with returning negative results using a chatbot for genes associated with HBOC and Lynch. And they've seen a non-inferiority to returning those negative results with chatbot through standard care. And I think that it's not population screening, but is potentially a really good example of a mechanism to be returning and educating folks about the negative results. So quick comment on your first point, which I think is a really good one. If I see a teenager who's obese or overweight and I check the cholesterol and it's normal, does that mean that they say, oh, now I can do anything I want, right? And if someone does a whole genome sequence and it's nothing shows up, does that mean I can drink and smoke into where I want because I don't have any risk factors? So I think research in that area is probably critical as you're pointing out. Comments from our panel. And you guys are quiet. All right, I think of Terry and then at the end of the table, I can't see your name, sorry, Kate. Yeah, so I just was curious, Kate, when what is ITCR? That wasn't my question. You asked me too fast. It's a funding mechanism through the NCI. Oh. And so it's focused on developing algorithms for helping to identify cancer risk and then tools and resources. Great. My question actually was for Bob Currier. When you said that when a diagnosis is made in a child, and again, we're not talking about in birth screen, but we can get lessons from it, they're referred to an appropriate specialist, which leaves, you know, there's lots of arrows along the way to that that you commented on. But I wonder, you would think that that would be sort of the upper limit of who would respond, who would actually follow up, et cetera. And that in adults who have fluid choice, et cetera, it would be much lower. And I wondered, is there any estimate of who actually, you know, what proportion of screen positive actually get into care and get appropriate care? You know, if that's... Well, we do know that I would say over 95% of positives get to a diagnosis. Well, after that, I really don't have, unfortunately the way the newborn screening system in California is set up, after a diagnosis is made, the positive case management is handed off to mostly CCS. CCS. Uh, it's the pair of children with special needs. Oh, really? It's another part of the state health system. And newborn screening actually doesn't get data back about long-term care, long-term follow-up. So it's very hard to really have a sense of that. So I will say it hurts so that we are now funding both propel grants and copropel grants for states to be able to do a longer-term follow-up. And you're right, Bob, that there doesn't generally happen. I will give you just one example that we know of what may happen in Ohio where they're screening for Crab A and there's not a secondary test for sight visiting. They have a high number of false positives. And more than half of the kids get completely lost to follow-up. So identify newborn screening as being positive for, but no follow-up. And so it's really concerning what can happen. Yeah, and just to follow on that, I think there's this connection for each state to have who's gonna follow up these results with the state lab is making sure that those are getting acted on by someone. Will there be something similar in an adult population screening program? Is it the responsibility of a health system to identify those pathways? Is it the state that doesn't? I mean, how are we gonna figure that out? It's patchwork, it'll be a problem. And I think we're gonna have to rely on our primary care providers and educate them on what to do when they get a positive. Because that's gonna be the first person that often sees these people, what to do next. And I think it's an important distinction, right? In the newborn screening, since it is a fairly much state mandated test, you could argue that if the state is doing this, there's some obligation to make sure there's a follow-up. What we're talking about some foreign adults though, it's not a state mandated. So it does probably fall on the healthcare system and likely the DCV. I think Kate was done in there. One of the last questions. Okay, online, okay. So after Kate online and then- Right, just a quick. The ITCR is actually a general technology mechanism for the NCI, I've been on that study section several times now. So all sorts of technologies screening is just one of the many technologies. So the other thing I was just gonna comment about the negative results is that there's clearly a trend, I was just on an ESAP for clinical programs to start returning negative results through portal and through low risk BOS. That more and more institutions, especially because of sort of the workload of genetic counselors have really started moving towards returning those results that way. I realized it might be slightly different for population screening, but I think you're gonna see a real clinical trend towards results being returned. And obviously they, people have a mechanism to ask questions to the portal if they have a question. But my impression is that that is really increasingly being adopted in many, many clinical settings just to put that out there. I agree, it's not possible to know if people particularly interpret negative results. I mean, I think people always say, you need to continue your screening, you need to continue your risk. I do this all the time for melanoma genetic testing in particular an area where you wanna make sure people realize that genetic testing does not change their need to have dermatological screening. And I sort of am very upfront about that. But I think that this is a cat is out of the bag phenomenon that negative results are really gonna start to be generally returned clinically via portal. Any of your panelists like to respond? Okay, there's a card up about happy down. I can't see who that is, I apologize. Introduce yourself, please. Sure, Kelly said something. And I don't wanna, I'm gonna talk more about education and training in a little bit. So I won't steal my own thunder, but one thing I just wanted to mention that was talking about these negative results and the people who might get false reassurance or overinterpret that negative. And you're talking with new board screening at State mandated, you have very broad participation in that. But I think at least our experience, I think others today, a lot of our population screening programs are they're opt in. And there's a inherent ascertainment bias in those populations and people who perceive benefit of that program. And you're gonna have higher rates of people with that personal and family history in there. And so that risk is even kind of exacerbated. And I think we just need to acknowledge that be prepared for that. And when we think about the calculations of those risks that, whether we should really be using population prevalence or should we assume a kind of increased ascertainment bias there? Actually, my comment was somewhat related to your comment. As these population-based tests, let's say for familial hypercholesterolemia or other conditions become more available, we have to make sure that they're used in the population setting. There are some examples from our carrier screening where providers are actually using it as a diagnostic test for a suspected, let's say child with CAH or other condition. And it's just, it's not appropriate because of the reporting structure and so forth. So we have to make sure that providers, even though these tests may be more available, may be less expensive and collected at home, et cetera, should not be used for indications other than what they were intended for. We are at the end of our time. Do we have a chair, Erin? Do we have chance for one more question online that you want to go to our summary? Okay, is there someone that are reading?  Thank you so much.  You should stop smoking just because you don't have diabetes doesn't mean you shouldn't eat healthier. So I'm struggling to understand both why we, it almost feels like the concern we have here is more elevated than some of these other times that we screen and have negative tests. And I also am a little bit concerned that the idea coming out is that we want genetic counselors to do everything but we just don't have enough money to do that. And it might actually be that weaving these things into primary care and handling the way we do other things will be viewed positively and we shouldn't view that as a loss. So as I turn it back over to Erin, I wanna echo that and say that PCPs give this kind of counseling all the time. And if there was some tool that let us know, you know, what the data showed, we could probably do it. The research agenda though really should tell us more about whether negatives change behavior in a negative way. Erin. So I'll just do a 30 second wrap up.  So Christine covered important concepts regarding the validating and stress testing, the pipeline, especially when thinking about high throughput screening. It's critical to have robust systems in place to monitor performance, both of the tests and the interpretation. Have we really thoughtful when deciding when new advances plan to be introduced into the clinical sequencing workflow? We didn't talk about that much during the discussion but that same goes for sample collection and choice of platform. We hear from Bob, the new board screening considerations are around serious, urgent and treatable disorders are applicable in the adult context. We really need to figure out how to hand off positive test results to clinical providers in the context of the US healthcare system. Disparities exist. We need to do better to include ancestors from underrepresented populations. We're seeing the value of that particularly like Heidi described with Nomad and the contributions to increase the number of fairings that we can classify as benign. It's imperative to have better estimates of prevalence of conditions, natural history of disease and age-based penetrance. We need to calculate what is required into following up on positive findings, better understand harms of reporting false positives and I'll stop there. Thanks. Thank you both very much. I would note the lunch is up in the far corner there. The hotel people may come out and pull the table out so you can do that without spelling anything. So, yeah, that's right. And we'll be back at 1.20 please, 1.20 to start the next session. Thank you all. Okay, if you could all take your seats. We're about to get started. Okay, welcome to the session three of the meeting. The session is gonna focus on the logistics of population screening. I'm Carol Bolt from the Jackson Laboratory and a member of the Genomic Medicine Working Group. And we're gonna start off this session with Melinda Massart from University of Pittsburgh Medical Center and Melinda, turning it over to you. Okay, great. So it's hard to be the first speaker right after lunch because now everybody is sleepy, but it's really an honor to be here today and to talk about opening the flood gate of results and are we ready and how will we handle this in healthcare? As you can see, I am a family medicine physician. So that probably gives away the punchline of what I'm going to talk about today. But I do want to just sort of acknowledge that I think, I may be posing more questions than answering them in my session today because I think there's a lot we still really need to think about. So I think we all recognize that there is a tipping point and that's why we're here today to really be thinking about the difference between population screening versus risk-based screening. And in our primary care precision medicine clinic, we have the great opportunity to be able to do very extensive primary care-based pedigrees with our patients. And although patients are coming to see us for an indication, we often find one to four additional genetic indications once we go through that pedigree. And even with this level of detail, we know we are missing folks. And so in our office over the last year, we have started offering population-based screening to our patients, even if they don't meet specific criteria for indication-driven testing. And I think that's really because the cost of testing has come down so much that we are even able to consider this. However, this is just on a very low, small scale. So how do we really think about when is the tipping point in the larger scale and going nationally with population screening? So a couple of things I wanna talk about first before talking about how do we handle the results is one, this concept of democratizing genomic testing and using genetic testing as a tool. And someone brought this up earlier even I think using some of these exact same examples, but really genetic testing is a tool now. And we really need to think about how to scale that and put that in the hands of all clinicians just like radiology is a tool, cardiology is a tool, when I order an MRI or a CT scan or an X-ray, I don't refer my patients to go see a radiologist, right? I order those tests myself. And same with many cardiology tests, if I want an echocardiogram or a stress test to restratify someone, I order that myself. And then I take that test result and based on the finding refer to the appropriate specialist to help manage that particular situation. I think we need to start thinking about genetics and genomic testing in that same concept. And many of us have acknowledged the challenge in scaling up access to genetic counselors, which I think also is probably an impossibility when we start talking about population level screening. And so again, how do we restratify patients and then get the right patients to the right level of care after that testing? The next concept I wanted to just point out, and again, this has also come up is really thinking about a single test future. One of the challenges right now around democratizing genomics is that testing is so highly nuanced. And this is really a barrier to most clinicians being able to utilize genetic testing or screening. But if we move towards a single test model in the future, then we can also think about how does the single test be applied across the lifespan? And when is it appropriate and relevant to unmask certain results? Even if we have them all, initially we don't have to interpret and analyze them all initially. And we can think about appropriate times across the lifespan when they are relevant and appropriate or when there's an initial clinical indication. And in the future, this should be able to happen really in seconds, right? When there's a clinical question at the point of care. So what do we need to achieve population scale genomic screening? And again, lots of folks are talking about these different elements today. And this is certainly not exhaustive, but I think we have some critical ingredients that are needed in this recipe. We need national buy-in. We need community and foreign processes. We need integrated clinical decision support for management, informatics, infrastructure and educated workforce, simplified testing, data sharing mechanisms, patient empowerment, enhanced protections through GINA for privacy and security and funding. And if we are able to achieve this, who and how would we handle all of these results? So would testing be centralized like newborn screening within state labs would health departments be responsible for notifying positive results? What would actually trigger the interpretation at what stage of life? And what would we consider actionable and when? Would those results then go to the relevant specialty care providers? I think that this would be very challenging because this would require a chaotic network of referrals and often great delays in being seen as we already know is happening nationally in genetics clinics. And if it doesn't go to the specialty care, does it go to primary care? So I obviously am going to advocate that, yes, the answer is it should go to primary care. And I know many of you are thinking about this as well. I think the primary care workforce makes the most sense. We truly are the orchestra conductors of health. Primary care includes pediatrics, family medicine, internal medicine, obstetrics and gynecology. We are the first line of medical care and have the lowest access barrier. We are available across geography. We provide care across the continuum of life, the age span, it's multi-generational with broad scope of practice. And we are the home of preventative medicine. We have multidisciplinary care models that already exist, including pharmacists, nutritionists, therapists, social workers, and now in our clinic we're adding genetic counselors to explore what this looks like. And patients have honestly already expressed and it's been documented that they have a preference for keeping their genetic concerns within the primary care space. Primary care also the scope of practice aligns with genomic screening, right? So in primary care we do preventative care, which are risk testing panels. We do prescribing management, which is pharmacogenomics. We do routine cancer screening, which is genetic cancer risk assessment and multi-cancer early detection technologies. We do prenatal care, which is prenatal carrier risk and NIPT. We do newborn care, which is following up on those newborn screening results. And we do chronic disease management, which will be in the future polygenic risk scores. Also screening already lives in primary care, right? This is all the national guidelines around screening. And this is already done all in the primary care space. So is the primary care workforce ready for this? No, they are not. I mean, I wanna be positive, it's not an F. I did not give them an F. You know, there's a lot needed to make this happen. And I think the next question though, is primary care able to do this? And the question is emphatically, yes. I absolutely believe that primary care can do this and should do this. So what do we need to do to prepare the primary care workforce? Some of the critical needs that we have are time-saving efficiency measures, right? Time, time, time. There is never enough time in primary care. Knowledge, we already know that's a major barrier. Confidence to manage all of this, both on the side of the providers and the patients. And really, I'm going to lean heavily on robust, informatic infrastructure to support data integration, reanalysis, curated updates, and portability of results. So possible solutions, we need clear and concise, just-in-time clinical decision support to manage results across the lifespan. Clinicians in primary care are never going to have all the knowledge needed. They just won't. They cannot add it to their already very full plates. So they have to be able to lean on clinical decision support and trust that it's up to date and have the confidence that it's going to support them in the algorithms needed to manage all of these screen conditions so that we're actually doing something with those results. We need minimal viable product for supportive management and counseling. Primary care providers should not become genetic counselors. That informed consent and guidance is really the secret sauce of genetic counselors. We should not ask primary care providers to do that, and they don't have the time to do that. So what is that minimum viable product that they need to do informed consent to be able to integrate results into the healthcare records and into the care and management of their patients? And then when to refer those patients up to that next level of care. And of course we need enhanced referral systems for management beyond primary care and better electronic health records ready for the needs of genomic medicine. I think this has already been acknowledged today, but our current electronic health records don't even help us manage diabetes at this point or routine cancer screening guidelines at this point. How are we going to ask it to add all of these additional components? I don't think that's a defeatist kind of thought. I really just think it's a challenge that we have to elevate these EMRs to do what they truly can do. And finally, community informed models are necessary to prepare the public to get to this level of advancing genomic screening. We need to think about who are interested parties, and there are many interested parties that belong on this list, but the two I really want to highlight are the community or public themselves and the clinicians who will be out there doing this work. So how do the key interested parties want this to happen? What should the models look like? Are they federal, state, local, regional? What models will be acceptable to the public and readily adopted? Are they going to be universal or population specific? And how do we ensure diverse and equitable uptake of the models across the population within the US? We talked about these classic screening criteria earlier today. And I guess the question for me every time I read these is who actually decides these answers, right? They kind of pose the questions, but who decides the answers? And I really think that needs to be discussed and engaged with the community, both the population at large, as well as the primary care clinicians who will implement this. And then finally, I'm gonna strongly advocate that we not do a fire hydrant model that we don't prep everything and then just release this massive onslaught of results and information, but instead really think about proposing a trickling faucet model. We pick one high value, high evidence screen. We A-B test this out in different mechanisms in different places with the community, both the patients and the clinicians. And then we layer on additional screening tests when pilot phase is deemed successful. And with that, I'm gonna pass on to the next person. Thank you. So next we have Peter Kraft. Peter makes his way. Who's gonna talk? Whom to screen, when and how? Well, I'm doing this over here for a second. So thanks, it's really great to be able to join this meeting. It's caught the last five minutes of the last session. It was already an exciting discussion. And first the disclaimer, I'm a statistician and a genetic epidemiologist. So I'm primarily interested in gene discovery and gene characterization. So estimating penetrance largely in the general population. So I think about clinical testing and genetic testing both in the clinic and public health, but I, most of the clever things that I have to say about it, I've learned from reading articles or talking to experts like many of the folks in this room. So if I say manage to say something clever, impact yourselves on the back. If I say something foolish, that's entirely my responsibility. So I wanted to start with just reviewing a successful non-genetic screening program. So the widely accepted mammography screening for breast cancer. So the thing to note about this is it does vary across time and across different contexts, different countries, different public health systems, but there's a general agreement that for the general population for average risk women, they should start screening between their late 40s or early 50s. And this is based on a balance of sort of risks and benefits both to the individual and to the health system and society as a whole.  So it's an important health problem. There isn't accepted intervention treatment. We know something about the natural history of the disease. And I think importantly for this context, the case finding is definitely not a once and for all project. The guidelines call for getting repeated mammography. It's not just you show up. We don't find any evidence of cancer in your breast today. Congratulations and have a nice life.  But because a lot of the guidelines are aimed at the general population, there's still a recognition that there will be some people who are at higher risk than average risk who we might be missing. And the question is how do we identify those folks and clearly genetic screening, genetic testing is one way of doing that. And the US Preventive Services Task Force has already made this an area of important research, something they're looking into. So I wanna talk a little bit about genetic screening and how it might help in this context. But before I get to that, I just wanna flag the sort of coverage or the uptake of mammography screening. So it's not enough just to have a set of guidelines that everybody thinks are good guidelines. You actually have to implement them. And even for something that is pretty uniform at age 50, start screening every three years, there are still coverage gaps. And these are influenced by a number of factors. It could be social and economic status. It could be distance to the nearest screening center, the middle panel there is highlighting immigration status or time since moving to the United States as a potential barrier. So all of these things should be kept in mind. We've already heard about a couple of those in the previous talk and I'll come back to this again. So when thinking about the potential utility in a population level genetic screening, we should start with the baseline, which is sort of the current guidelines or the current practice. So this is a schematic that's sort of describing genetic testing, clinical genetic testing for three tier one CDC conditions, hereditary breast and ovarian cancer, Lynch syndrome and familial hypercholesterolemia. So and basically for testing, based on a personal family history of disease. So those bottom three arms there, no pointer. So the bottom three arms are for folks who we've identified should be tested. And of course, if you're tested, you might end up having a pathogenic variant. You may have informative negative test. So we sort of know what the genetic variant that's segregating in your family is and you did not test positive for that. So we're fairly reassured that you don't have the high risk variant. But a lot of folks end up in this middle category where either you test positive for a moderate risk variant. So think check two in the case of breast cancer or it's uninformative. Like there's something going on in your family but we haven't been able to pin down what the genetic variant is. And this is the setting where I think there's been some, there's actually some clinical invitation now. Some companies are offering, some clinics are offering polygenic risk scores to help differentiate those moderate risk folks. So for example, taking that check two example, if you didn't know anything about the polygenic risk score or other risk factors, you're sort of right on the bubble of guidelines in terms of MRI screening. But if you had additional information about the polygenic risk score or other risk factors, you may end up being fairly comfortable that you're actually well below that or well above that thresholds and you could take action accordingly or be, so that's schematic that I was showing. Oh, and the other key thing on this schematic is of course the folks who we didn't test because they don't have a positive family history or haven't been diagnosed with the disease yet or for other reasons, we haven't been able to get them to the testing clinic. So there are folks who are walking around with a variant who haven't been identified. So there's a missed opportunity. So, you know, that little schematic was already somewhat complicated. And when you go to the actual guidelines, they're even more complicated as I don't need to tell many of you. So, you know, before we even get to population screening, there may be an intermediate step which is to automate some of the tests to really identify people like actively go out and look for folks using EHR or other records to flag people who should be screened. And my colleague at the NCI, Kertina Goddard, has implemented a pilot study, Charm, which is the Cancer Health Assessment, and I can't read my handwriting, so RM, which basically takes sort of the typical process where people are sort of opportunistically identified and makes it a little more systematic. And they were able to show that this was able to get more people in for screening who should be for testing, who should be according to guidelines, especially among folks who are underrepresented. So moving to the other scenario where we did undertake population screening for these three conditions. So now everybody gets tested and you can end up in sort of three bins. The middle bin there is folks who would have been discovered using sort of current care. And you can compare these two arms. What would happen if we kept things the way they are versus what if we implemented genomic screening? And in this one particular case, this one particular paper where they did a simulation model, it turned out that the population screening was able for these three diseases together was part of the key argument they were making, was effective, you were able to identify and prevent more cases of cancer or deaths in cardiovascular disease. You had a better quality adjusted life years. And what's particularly interesting or relevant for this session or the title of my talk anyway, is that when you did the screening or when you, maybe you did the testing early but you've unmasked these results at different ages, 30, 40, 50, affected the utility of the screening program. So in this case, starting screening at 30 years old was the most effective. They did look at what if you started it earlier, 20 years or older or so in this particular setting, the gains were marginal and were not necessarily offset by other factors. So starting at 30 made sense. So I'm just gonna come back to this picture and mention some of the coverage gaps that are, I should have changed the title of the slide and that's the, the titles aside but the point is I'm thinking about the gaps of what might exist for population genetic screening. So there's going to be costs. There's gonna be, how do we cover this through insurance or not? There'll be barriers in terms of transportation getting to the testing center, the time to undergo the testing. And there's gonna be a burden on the healthcare system in terms of returning these results and the subsequent follow-up. So all those things will have to be considered. How am I doing on time? Good. And then I just, when I was reading the Wilson and Younger article or the retrospective of the article in preparation for this meeting, it really made me think of the Jeffrey Rose 1985 in our article, Sick Individuals and Sick Populations where he talks about sort of two strategies to lowering the burden of disease. There's the high risk strategy and the population strategy. So the high risk strategy, I think sort of what we're talking about, let's identify the people who are at higher risk and let's intervene on them in the case of cancer screening early if we catch the disease early, versus the population which is about shifting the underlying risk if there's an environmental exposure that's driving a lot of the population burden. Let's change that. So the high risk, the individual gets a big benefit from that intervention but most people will not benefit because we're focused on a small proportion of the population for any particular disease, I should note. Whereas on the other hand, the population, you have actually a big population impact but the difference for any one individual in the population might be small. So this is another slide I borrowed from Katrina sort of making that point contrasting individual level interventions on tobacco control in this case versus sort of population level and you're sort of getting, there's more of an impact for the sort of broader population approaches. But I guess the one thing that I wanted, the point I want to make here is we shouldn't forget about those other approaches especially when we're thinking about complex diseases. So again, breast cancer, there's lots of, there's it's multifactorial, people get breast cancer for all kinds of different reasons.  So there's a potential for intervening in other ways but we should be clear that it's not an either or, right? It's a both and that both of these strategies can be in play at the same time. So thank you very much. Very good. I believe our next speaker, April Adams from Baylor is online. So can we bring up April? April, it's all yours. All right. Can everyone hear me? Yes. Okay. All right. Well, thank you for having me and accommodating my need to be virtual. I'm April Adams. I am, this is professor at Baylor College of Medicine and I'm primary clinical and I work as a reproductive geneticist. So today I'm going to talk to a little bit about addressing the challenges of genomics, screening and populations underrepresented and genomic databases. I'm sure that I don't have to prove to the audience here that the US is a diverse population. This is just a snapshot of the race and ethnicity prevalence by state from 2020. And you can see it kind of goes from the most highly represented group over to the second, third and then kind of a diffusion score of the populations, the lower prevalence. And interestingly, so when you look at this data and you compare it to 2010, you can see that our reporting at least or how people identify and the admixture of people that live in the United States has definitely changed since 2010. So the largest group represented being white identifying is not Hispanic comprised 57.8 and 20% in 2020, but that was down from 63.7% in 2010. And then you can look at this in another way looking at over the different age demographics in the United States. And you can see that also that diversity is reflected in younger populations as well. But the thing to consider with self-reported race and ethnicity is that it can often be pretty unreliable and it may not capture the true diversity of a population. So when you're thinking about what is our actual genetic diversity look like, you find is when you look at people across different populations in different locations with different cultural backgrounds, you see that those processes impact genetic diversity significantly. So even a population next door to each other with the different cultural practice may have some different rate of changes in their genetic diversity. And this can't be reflected in our race groups that are categorized pretty broadly as white, non-Hispanic, black, non-Hispanic, Hispanic, and then we've categorized everybody else into this other group. So just to kind of set that stage up, we are diverse and we're probably not even really capturing how diverse we are. So our identity is multidimensional. So our identity is our ancestry, our genetic lineage, our genealogy, all of those things that we have no control over and then it's still also impacted by population movement and cultural practices and all of that. But then we also have these other categories that are placed on an individual such as race, which is the construct that we create based on physical attributes or things like that, that can also change over time and depending on where you are and where you live. And then ethnicity as well is an area in which it's a construct that really looks at your language and religion and nationality and it can be self-reported, but it is also something that can change. And also people may not know enough about their prior history to really determine their ethnicity and there's really no consensus on what race and ethnicity should look like. And so I say this all to say that we have to be very thoughtful about how we look at race and ethnicity when we are looking at clinical implementation and designing research studies because it's information that we may be building upon the bias that already exists there. And I say that looking at, okay, how are we doing in representing people from diverse backgrounds? So we're probably under-representing our diversity and then we're also not even capturing that diversity that exists. And so when we look at some large studies and large databases, we see that we've made some improvement, but overall it's still heavily weighted as has been previously mentioned towards individuals who have a European ancestry. And that also is going to still limit the amount of diversity that we're seeing in those studies. And so this lack of representation in addition to our inability to really give people the appropriate categories when we're looking at ancestry, race, ethnicity leads to this perpetuation of these health disparities that already exist, which is really opposite of what we want to do with genomic screening and sequencing. And so what are the drivers of these disparities? And a lot of the drivers of these disparities have to do with the social and cultural context in which we live. And so looking at things like social determinants of health and how do these experiences lead to our health and well-being or to decrease life expectancy, higher healthcare costs and things of that nature. And so what we are missing is we are lacking on that diversity of people included in studies. And when we do that, what we find is that you're going to also miss out on the genetic factors that are interacting with our environmental, behavioral and social determinants of health to lead to that disease. And so we know that the parts of genetic diversity that are gonna flow through with our population and cultural processes without including those people, we are gonna miss out on figuring out where we can make that actual impact in health. And I'm sure that I don't also have to explain to the audience that there are clear disparities in our health outcomes. And just a couple of examples of when we look at race and ethnicity, obviously with the caveat of we're mixing race and ethnicity together here, which may not be the same thing. We see that there are big disparities in groups, specifically in non-Hispanic black individuals, the rates of death, secondary to diabetes and heart disease far exceeds the individuals that identify as non-Hispanic white. And so how do we make that gap smaller if we are including people in studies? And not only does this disparity impact adults, this disparity actually starts in utero, right? So if you look at fetal mortality rates, the rate of birth in non-Hispanic black fetuses is twice that of our non-Hispanic white population. And then this still translates to the same when we look at infant mortality as well. So with that, I just wanted to take a case example of looking at reproductive carrier screening. And the reason why is because it's an interesting area of screening. It's an area in which a lot of people are captured, but it's not really done in a population health sort of manner, but pregnancy may be the first time they actually encounter a genetic test. And so with carrier screening, it's really looking at, can you decrease the morbidity and mortality of fetuses and infants by screening asymptomatic individuals for autoresomal recess of an X-linked conditions? And ideally you wanna do this in the preconception period and give people the opportunity to make reproductive decisions based on that information. And traditionally this was done based on, but that was limited because it only captures a small amount of diseases that may impact a fetus or infant. And it also, as we mentioned, people don't know their history or their ancestry or very well. And so you're excluding a lot of people who may actually be at risk. So looking at that and looking at, as we talked about, there's different admixtures, patients can't identify their ethnicity. When you look at something for example, like sickle cell disease, people really targeted African-American patients, which makes sense, but then you're missing lots of newborns who don't identify that way, who also will end up having this condition. And so this is what led to a more expanded carrier screening approach. And when we look at expanded carrier screening, we're gonna be able to look at more diseases, do a pan-ethnic survey, and really try to capture an entire population. And then what we've found is the more diseases you screen for, the more people you'll find are carriers, which is great. And so then we get these guidelines that say, okay, these are the conditions we should be screening for. So frequency, carrier frequency of greater than or equal to one in 200. And thoughtfully, let's make it equitable. And all pregnant patients or those planning of pregnancy should be offered at least this tier three or a carrier screening frequency greater than or equal to one in 200. However, that is a little bit more difficult in actual practice. So when we look at some of the challenges in expanded carrier screening, I really wanna focus on the fact that one, we are counseling people who may not be represented in a lot of these screening panels. And then we really don't have the right access to do this at a population level. So just an example of a study that a systematic review that looked at carrier screening research studies. And in this study, what they really found was that it's been curious, we needed a great job. However, these were small studies and even in these small studies, they only had a very small percentage of patients who identified as not having non-European ancestry. So there again, you can see people are just not being represented and not recruited into these studies. Additionally, when we talk about access, we have these huge barriers and access when you see that patients who identify as non-white are going to have a harder time accessing care due to costs related to care and also have higher rates of being uninsured. And so that makes things that are like carrier screening which can run anywhere from $200 to $2,000, very cost prohibitive. And then another piece that is also gonna be a barrier to care and barrier to access for patients is going to be provider bias and discrimination. And what you see is that not only are patients having barriers with access to cost and insurance and all of those things, they are also seeing barriers in getting the same care offered to them that would be offered to somebody who is a part of the majority. So they may be less likely to get a referral to the genetics clinic. They may be less likely to get a genetic evaluation or to even be offered a screening test. And additionally, they may not access care because of prior experiences with perceived or discrimination and poor treatment in those facilities. So when looking at our criteria for the population-based screening, I think some things definitely are already there but some especially are, can we offer this to anyone who wants it? No, a lot of people still have a lot of lack of healthcare access. So there's a big lift to figuring out how you can offer this in an equitable way. And can you, are we actually offering it to individuals in a routine and an equitable manner because many times patients may not be offered a test just by based on their providers perceived bias about their decision-making, their race, ethnicity, all of those factors? And then obviously as we've discussed, lack of representation in genomic databases really does lead to difficulty in counseling and less downstream research into how do you mitigate the outcomes of positive results. But all of this is not theoretical. There are many, many carrier screening tests out there. You can also do preconception, genome sequencing, exome sequencing. And so the cat's definitely out of the bag and people are being sequenced. We just are a little behind in figuring out how do we implement this in an equitable way? And so right now we're in this phase of kind of having this increased gap in who can get it, who can't and who's benefiting from this process. So in looking at that, we took kind of a step back and said, what are the things we need to be thinking about when you're going to provide equitable genetic services? And kind of looking at the NIMHD framework for equity and looking at the different domains and how do you impact the lack of knowledge about genetic variation and all those different levels of engaging individuals, their families, their communities, educating healthcare providers, building trust in healthcare systems, as well as making sure that people have access to culturally sensitive care, to affordable care and that their wants and desires are incorporated into how we disseminate this kind of care. And so just moving on from that, looking at what are some principles of equity when we have patients who may have been, who may be underrepresented, who may have a marginalized population and really focusing as we develop clinical implementation strategies and research protocols, incorporating person-centered models to help empower marginalized individuals and communities. And I know we're going to talk more about that in later sessions, acknowledging historical harms and using that knowledge about those things to build better systems, right? So when we talk about things like AI and telehealth and all of that, making sure that we are not building that same bias, that structural racism into these systems and having that thought before, instead of an afterthought of, oh, now we realize that this is a disparity, let's try to fix it later. Really having respect for individuals' choices and not creating shame around their choices and their decision-making and being creative about meeting people outside of the healthcare system because sometimes that first step is getting people to trust the system and actually enter it. And then a big piece is also looking at how do you better support health literacy, language, cultural context because uncertainty for one person based on their language, cultural background experiences may be very different than the next individual and understanding that there are definitely first-boat burdens in being able to do all of that. So the gaps that I think need to really be addressed are one, increasing diversity and inclusion in the workforce because a lot of these principles of equity can be done when you really engage what patients need and patients tend to do better and feel less marginalized when they see that there's not only representation in the patient population or the research population, but in the stakeholders and people sitting at the table and making decisions and making policies. Also identifying and limiting barriers to participation as I mentioned, so maybe just getting out into the community being engaged and understanding that there are gonna be multi-modal ways that patients need to be engaged in care and not immediately thinking, oh, they don't want to do it because they didn't show up because maybe they just needed a ride there. And incorporating the principles of equity in all levels of implementation, it really should start with the hypothesis, right? It should be from the very beginning, the first question that you have, how do we create this in an equitable manner? And then the big thing, which is a really heavy lift is how can you expand the on-race? How can you incorporate social determinants of health with ancestry into these research questions and so that people can be moved outside of those boxes to really understand what are the drivers of their health outcomes? Thank you. Thank you, April. And so our final speaker in this session before the discussion is Kelly East from Hudson Alpha. All right, good afternoon. I'm Kelly East. I'm the Vice President for Education at Hudson Alpha as well as a genetic counselor. So I'll be kind of wearing both of those hats in this talk and I really appreciate the opportunity to be here and be a part of the dialogue and share some of our stories and data from our experiences engaging both providers and patients in population screening initiatives. And so one of the things kind of, as I think about population screening, there's some wins in the education space in terms of barriers that, by deploying things on a population level, there's a few less things that a provider necessarily has to know and make decisions around to get patients access to genomic care. But at the same time, population screening requires more education and knowledge and skills and confidence for the patients who are getting that testing to have the maximum amount of benefit and the least amount of harms involved. And so I'm not here to, I mean, everybody in this room, I think would agree that more education and training is needed. But what I'm hoping to do is to share some of our stories and some of the themes that have come out and emerged as places where we can do more and themes that should be a part of the interventions that we need to deploy to provide better education and training around genomic screening for these audiences. And so just as a bit of a context setting, some of the studies that we've been engaged with that I'll be sharing some data from, there is a cancer risk population test that we've been doing for a number of years. So Hudson Alpha is in the northern part of Alabama. So from a context standpoint, the patients and providers we're engaging with it is the Southeast part of the United States where that issue of access to genetics care is kind of at an increasing pinch point. But this is a population consumer directed test that we've been implementing where consumers can go and self-select to have the testing and then we go and engage their providers. But it's the cancer risk gene panel that is definitely meant as a screen not as a diagnostic test. Another study that we've been doing is the Alabama Genomic Health Initiative and I think we'll hear some more about this tomorrow from Dr. Korf. But this is an array-based test that we've been doing again for a number of years for individuals in the state of Alabama for adults. Initially it was actionable disease risk and then more recently we've added some pharmacogenomics to the study as well. Of note from 2017 until COVID in 2020 we were operating under a at what we call our population cohort where we were embedding recruitment sites out in the community, trying to engage Alabamians in the study. But then after COVID we relaunched as a clinical cohort where we started recruiting patients out of specific primary care offices. So it was much more integrated in the patient's clinical care. And then finally, this is another study that we've been engaged with called SouthSeek which is not a population screening test but the reason I included it here is we've got some really interesting data that I think is useful as we think about educational needs and misconceptions in something like population screening. SouthSeek was a whole genome sequencing study for affected infants and NICUs. Although it was a very broad set of inclusion criteria, it was we focused more on which babies shouldn't be enrolled rather than which babies should. So we had a really diverse set of programs that were engaged in that. And also specifically with this study we were interested not only in getting diagnoses for the probands but to test a result delivery model through non-genetic providers and doing a clinical trial around that. For all of these studies, as we thought about scaling up education and scaling up genetic counseling, we focused on how can we best use our precious resources of clinical genetics professionals, largely in all of these studies, the front end, the informed consent conversations and decision making was not done through a individual interaction with the genetic counselor. It was done in different ways for different studies and I don't have time to go into a whole lot of detail. But we focused a lot of our education interventions around the return of results and thinking about on the back end, once you've got results, how do we manage them and make sure that the patients are getting the right downstream care. So the way I wanna frame this is just talk through some of our lessons learned and things that have come up that have framed how we educate our providers and our patients and things that I think we need to care take even more maybe. This is one that we've been talking about already quite a bit today and that population screening is going to identify a lot of people with an unmet need for diagnostic testing. I think we all know that there's so many people out there that should be getting diagnostic testing and they aren't for a whole variety of implementation reasons. And so in a couple of our studies in particular and as I pulled this, I checked myself over and over that these numbers were indeed the same in these two studies and they were. But for the Alabama Genomic Health Initiative and that information is power, cancer screening, cancer risk screening test, both of those we included family history as part of the intake process. So when patients were enrolling into the study or were signing it for the test, they provided some information about their family history. And one of the roles of our genetic counselors was to go through and review this, although we've gotten a little better with automating that as well. But going through and using this as an identity to identify those patients that kind of regardless of the outcome of their test result, there's some additional actions that should be taken based on their family history. And it was almost half of our participants had some kind of what we considered a flag. And generally speaking, those flags were things that came up that would have made them a good candidate for genetics evaluation or additional genetic testing. Another thing that continues to come up as we've been looking at results that have come out of these studies is that oftentimes the people who get positive genetic testing results, it does not corroborate with the family history and personal history that they have told us about. Which not terribly long ago where we went through and looked at all of our positive test results in our in our actionable disease risk in that population cohort of the Alabama Genomic Health Initiative and looked back and said, well, how many of them would they have been flagged? Was there something there that indicated that they were at a heightened risk of getting a positive result back? And the answer was not many. And it also, the answer was it varies wildly by the gene that you're talking about where there's some of the genes that we reported that the corroborated family history or it was 0%. There were others that were 100 and kind of everywhere in between. But what this leads to are really surprising results that we can't necessarily predict or counsel or educate well around at the beginning before you have these results. And in working with our providers, we did a fair amount of education and talking with our providers at the front end. And then as they started getting these results, a lot of them got really uncomfortable about thinking about having to talk to a patient about a genetic test result that seemingly is completely out of the blue. And it also brings up that issue that we've been talking about some today about what do you do with management in a unselected population and someone who doesn't have that personal or family history, you know, what is their actual risk for disease and should they be following the management guidelines that have been published that have come from those high-penetrant or more high-penetrant families. Another theme that we've seen come out kind of over and over from providers and patients alike is this overinterpretation of a negative result. And the risk that those people that have that personal or family history that getting a negative test result doesn't necessarily decrease their risk, especially just depending on what the testing was and how good it is at picking up the known risk factors. And so this is where I wanted to talk a little bit about that South Seek study where we've got some of the, we've got some really robust data around provider errors, provider misconceptions, which are really big opportunities for identifying places where we can do better in terms of education and the tools that our providers need. And so in South Seek, we took all of our, the pro bands, the infants that were having whole genome sequencing and we randomized them to either get their results from a genetic counselor at their site or from a non-genetics provider, most of whom were NICU physicians that we had done some level of training with. And what that looked like was a half-day live training. Also our genetic counselors had a role where we were writing the result reports for the study and going through and kind of creating a bit of a roadmap in those result reports that providers could use that has had a lot of the talking points in it that went beyond just this is what the result means, that included some more things about what this means for the family and contextualizing that result for them a bit more than I think is typical. But so the patients would get their results from one of these two entities and we audio recorded all of it, which has created a really robust data set that we're just now kind of scratching the surface of and dreaming of what else we can do with this data. But the reason that we were recording it was we wanted to be able to look for errors. It started as a safety requirement for the study and then it turned into a really interesting research question. And this is a kind of busy slide, but the take home point is that non-genetic providers they were significantly more likely to make what we consider to be a major error. When we were listening to them, we categorized them as major or minor depending on whether we thought it would have a profound impact on decision-making. So there was a higher percentage of major errors in our non-genetic providers, but 92% of those non-genetic provider disclosures did not have any major errors in them. Of note, we went through and did some thematic analysis of the errors that we identified and the one that came up the most kind of over and over and over was this overinterpretation of negative results. And there's some quotes up here from our non-genetic providers about, that them wanting to kind of think genetic testing is better than it is. Many of us in the genetic space know the nuances and know how much genetic testing can't find. And these were in affected infants that had a suspicion of a genetic disorder and that we had providers that would tell them that this ruled out genetics or something along that theme or that the future children are not at risk. And so that's something that we, and certainly was part of our education. It was on that report that we sent back, but this is still what was happening in those conversations. And you've got providers and patients that are really over-interpreting these negative results. And I don't think this is unique to diagnostic testing and it's something that's gonna be a major part or needs to be a part, a major part of education for providers and for patients when you're giving back negative or non-informative results. We also have also data here from our inherited cancer risk testing where this was a survey, it's probably hard to read, but this was a survey that we sent out to patients after they had gotten their results and they had them for a while. And this was a knowledge question of asking them what does it mean if someone has a genetic risk factor for cancer versus what does it mean for a person to not have a genetic, the reason I point this out is that there's a more accurate for positive results than negative. So almost 100% of those patients are saying that they appropriately picked the right answer for a positive result, but only 72% of people selected the right answer for a negative result and a notable 27% of people who filled out the survey who most of them did indeed have a negative result said that a negative result decreases your risk compared to the general population. And that's problematic that even for people that don't have a family history, getting a negative result is not gonna take you down below the general population risk. But the fact of the matter is that the correct conception for a negative genetic test result really is not the same for everybody and it largely depends on that personal and family history and it needs to be contextualized which makes broad education messaging challenging because you're going to potentially over alarm or under alarm people with that messaging. We talked earlier, a couple of people mentioned cascade testing and I think that's something that from an education perspective, genetic testing is a little bit unique and when we're talking to providers, making sure that we're calling out the fact that getting these test results go, the impact of that goes beyond the patient in front of them which is kind of out of scope and it's a little, providers are not necessarily thinking about or equipped to handle that kind of downstream testing or talking about those risks. When we look at what patients are doing after they get genetic testing results back from a population screen and I think this corroborates with other studies as well, patients talk to their families a lot more than they talk to their doctors about it and we have an opportunity, I think with working with patients and providing them the tools and resources they need to really facilitate those conversations with family members in a way that I think we can do more to provide those tools rather than expecting providers to do that. And finally, I think integration with clinical care is really important to both increase the access to testing as well as the potential, maximizing the benefits of follow-up. So this is just looking at those Alabama Genomic Health Initiative enrollment data and the blue of these donut charts are Caucasian or European individuals and this is the difference in, both of these have thousands of people in these data sets of in the population cohort of the study. We had, and we had people from all 67 counties in Alabama in the study but we were out in the community recruiting patients into this array-based population screening test and despite our best efforts, we still ended up with a fairly non-diverse data set. Meanwhile, in the clinical cohort, these are patients that are recruited through primary care offices, so it's embedded in care and we've been able to strategically pick and partner with diverse populations in the state but we've had a much bigger uptake of diversity and a much more diverse data set after integrating this testing into clinical care but I'd be remiss not to mention that there's probably a whole bunch of people that don't have primary care doctors and so if you only go that route that you're potentially also, you're creating some barriers to access as well. But so getting the test, getting people in the door but what you do with those results, having that integrated in clinical care is also, it makes it much more likely for people to get that benefit either the follow on testing, further evaluation and then the care based on their results but this is where for those things to happen, providers have to have the knowledge and they have to have the tools to be able to do that. And this has got to go beyond providing CMEs for providers and thinking about the things that have already been mentioned today with EMR integration and AI and telegenetic counseling resources and all of that and thinking about from an infrastructure standpoint of how to make this as easy as possible and as leak proof as possible for patients to get the information and the care that they need. So as I think about some opportunities based on the things that we've learned and things that we need to be paying attention to as we go forward, certainly interpreting and communicating negative results needs to be emphasized. The vast majority of patients that do genetic testing are going to get negative results and we need to make sure that we are caretaking those and making sure that we're not treating them all the same and finding ways to contextualize them for the patients that are getting those results back. Thinking about program infrastructure that can help support that family communication and cascade testing that we're thinking about that downstream benefit and there's a lot of education needs in that not only for people to be aware of their risk but where to go to manage that risk especially when family members may live in wildly different places. Infrastructure as part of our screening programs to use that as an opportunity to catch people that should be on a diagnostic genetic pathway and figure out how to make those referrals easy and more likely to be done whether that's a referral or whether that's providers that are able to then add on those additional levels of testing. And then finally, scalable processes for clinical genetic professionals to provide support. I'm very much a proponent of non-genetic provider engagement in population genetic testing. I'm really interested in thinking about how we as genetic providers can become a safety net for those providers and become a resource for them in helping on an individual patient level providing contextualization and guidance but not being the one that is necessarily trying to use our knowledge and our resources as judiciously as possible but thinking about who are the patients that would most benefit from seeing a genetic counselor and how do we focus those resources together? Kelly will have to call it there we're a little bit over time. Okay, sure. I'll leave this slide up for a second but these are when I think about the places I wanna be doing more research and I think we should be coming together to do more research if you're right here. Thanks so much. So Pat, can you help me by, if I miss somebody who has a question on this side I can't really see them and that would be very helpful. So we're open for discussion and questions. So I'd be interested in sort of Kelly following up on your last slide. I'd be interested in hearing from the other presenters about sort of research agendas that can address some of the, we've had a lot of discussion of gaps, right, and barriers. So I'd be interested in hearing some of your thoughts on research agendas that could address those barriers and gaps and maybe we can go in order. Melinda, maybe you could start us off. Yeah, so this is something I think about a lot and I think a lot has been published on the barriers and gaps and really would like to start moving the conversation towards solutions. So we actually just held a day long session using Human Center Design to prioritize solutions and possible interventions for integrating genomics in primary care. And I think that someone mentioned earlier putting out a white paper about these potential solutions and really starting to have teams that are collaborating across the country to test these various different solutions to really find out what works best in different types of care settings. There's no one care setting in our country that is one of the biggest challenges that we face. And so we're probably going to need several different models to find solutions that work successfully for various different care models. But that's one place I would really like to see a lot of research moving forward. Great, thank you. Peter, any comments on? Sure, I mean, two things that I think came up in the previous session were around understanding penetrance in the general population. And I think April mentioned sort of being able to interpret particular variants that may be very rare in the existing databases. But we don't want to say that those data is an artist or a representative. So we made the flag something inappropriately as being pathogenic because it's rare, but in database, but it's actually not that rare in these samples. So I think those are still important areas to research. I do think we're filling in some of those gaps. So certainly for the more common conditions that we might be screening for, I think we are getting robust estimates of population penetrances. And those were, you know, as biobanks and cohorts get bigger, we're going to have better estimates of those. And I mean, Heidi can maybe speak more to the reference databases. So I know Matt just had a big update and there may be some interest in sort of better assigning folks using genetic ancestry as opposed to population levels or self-reported race and ethnicity, though that might help with the interpretation. And that goes to my other thoughts on research areas would be sort of on the informatics and both sort of how do we get the informatics in place so people can interpret these things? And then when we're going to be probably there and maybe Kelly can speak part of this is we will be relying more on some sort of automated reports as a way of sort of getting that conversation going. And then maybe the clinician or the specialist would be sort of more answering questions that folks might have, but there might still be some informatics. And I just want to make a shout out to the leaky tab. So that's, you know, part of the research will be the implementation as well. How do we do this most effectively? April, would you like to make a comment on research agendas that you think would be essential to addressing some of the logistical issues you identified? Yeah, I think that everybody's made really good points. So I think when we talk about how do we, you know, it can engage more people and utilize the workforce appropriately, all of that. I think really being thinking more out of the box about how we're getting out to other communities. Because if you look right now where all the geneticists and genetic counselors are, they're in major urban areas. So, you know, I think definitely focusing on how do we utilize technology better to be that consult or that safety net person for somebody's primary care home? Obviously that implies that people have a primary care home, which is a bigger issue, but utilizing those structures that work and maybe that's an example to grow that type of infrastructure or more. And then I think that also just, yeah, to reiterate, I think that looking better at how we more thoughtfully design studies to look at diversity and how we are categorizing people. Because I do think that we are, you know, the genetics community is definitely aligned in a space to change that perception and help people understand better why ancestry matters and how we utilize it in a, you know, equitable and thoughtful way. Thank you. Kelly, any comments from there? Sure, I agree with everything that everybody just said. And I think when I think of the things that I'm most interested in and think would be really impactful is we're just talking about thinking about how to pilot and test different models for interpreting and managing genetic test results. Thinking about how the combination of the testing labs, the primary care provider and maybe a clinical genetic specialist that how those could come together to create pipelines that they get the providers what they want most, which is an answer for what does this mean for this patient, not what does this result mean or what does it mean for this patient and what are my next steps and thinking about whether that the combination of tools but also where the human brain can supplement those tools in a scalable way. So one of the things, oh, I'll go ahead, Terry. And then Mark. Sure, so Pete, I was delighted to see somebody quote Jeffrey Rose. It's wonderful to see his name again. And I was just curious, you gave some good examples of population wide efforts in smoking tobacco control. What would you think would be population wide efforts in genomic screening or identification of genomic risk? Yeah, so I guess I think of the population screening for genetic risk is falling more in Jeffrey Rose's high risk approach. So it's identifying those folks who are at high risk. So which we're sort of doing now but there's a lot of people we're missing as galley source and data where there's folks who aren't being recommended to testing who would benefit from it. So yeah, I don't know if that, so I guess sort of exactly what this meeting was talking about is what I would think of as genomic screening. Epidemiologists sticking together.  So the question I would have for each of the speakers is, I think you've raised some really interesting research questions that are worth exploring but I'd be interested if you would be willing to propose some research methods or some potential projects that might get at the specific aspects that you were talking about. So a little bit more of a translation of the questions into a project or a methodology. Yeah, for having thought for 20 seconds. I'm gonna go back to what I really emphasized in my presentation, which is clinical decision support. And I think that after 20 years of trying to encourage primary care doctors to adopt genetics into their scope of practice, I realized that that is just not feasible without these tools. And so I think I would highly propose an implementation study around two to three different models of clinical decision support that both provide just in time information to the clinician and education to the clinician around how to interpret and integrate one specific screening, which could be something like pharmacogenomics, could be something like hereditary cancer testing and really look at the success of that implementation and comparing those different products as well as strongly exploring the user experience from the clinician's standpoint. In addition to that, I think I really would encourage us to engage the community of patients and we're not hearing from them today specifically, although I think all of us are also community and patients, but I think we really need to find out what this means to patients and how they would like to see this impact their care going forward. So short answer. Carol, can I hop in here for a second? We've got a fair amount of experience with providing clinical decision support to primary care providers. We've been doing that for quite a few years, both in the area of pharmacogenomics and the other area of genetics, genomics at large. And the experience that we have is that they don't want it. So how do we get over that barrier? So I don't know if you mean the experience in your specific institution. Our specific institution, but I would say more broadly, I can report that at least I would say across the emergency network, which has got eight or 10 clinical sites, that's a fairly consistent result. I mean, people, pharmacogenomics is a good example. We provide pharmacogenomics clinical decision support and they don't want to pay attention to it. They want to just click it off and ignore it. So how do we get past that? Yeah, so I think that's where that end user experience is critical because I think we, a lot of us designed that clinical decision support with interruptive alerts and primary care providers are overwhelmed by interruptive alerts and don't like them. And so I think that, again, with implementation science rolling out an intervention that won't be adopted or received by the end users is gonna always fail. I think in addition to that, historically primary care providers have not felt that genetics has yet reached the level of value for their patients. And until we show the value of integrating this, that it's worth their time, then they're gonna continue to think that this is more of a nuisance than a benefit. Yeah, I really want to echo your second point there because the first point, we all understand alert fatigue and what the impact of that is. But our experience is the more fundamental problem is people don't even believe the evidence.  So what do we need to do to fix that? Yeah, I would weigh in and say that, while I agree with what you were recommending, I think we've got one step before that that we have to do. And this came out of GM 13. So I'm referencing a prior. And that happened to be on Informatics research agenda. And one of the things that clearly emerged there was the idea that people really don't like an impositional model, which is, hi, I'm from Genomics and I'm here to help you. I'm gonna solve all your problems. And they said, well, we don't have any problems for you to solve, thank you very much. Whereas we actually sit down and say, what are the things, what are your pain points and what are the issues? And so I'll just use one example from our institution where there's an institution-wide initiative to improve colorectal cancer screening. And so there's everybody's bought in, there's a big quality initiative. And then we say, let's build a piece in this for those people that are at the highest risk. And then let's sit down with our clinicians to say, okay, you tell us how you want us to do it. And then when we do that, we get actually much higher uptake because they're part of the design process. And then of course we also follow up with the user testing afterwards because as we say every time when we roll something out, we guarantee it's wrong or your money back. The second piece is we'll fix it. But I think we really need to think more about that early engagement in defining the problem and defining the solutions. Carol? Yeah, I've been trying to say, I'm actually, we've had a lot of discussions with our primary care providers about this and they're really not interested. And I have to say, and they're scared, but our specialty providers are extraordinarily interested. We've really intentionally moved this and the things that we've done to mostly to specialty care providers. And I just wanna sort of contrast that because we've really found that offering, our specialty providers are more and more themselves ordering genetic testing and offering them support around what they really wanna do which is order genetic testing has been much more successful than our primary care providers who do not feel comfortable even with placing consults for genetics. And so I just, I'm pointing that out as some alternative models of thinking about how we do it. It's possible that a research topic could be contrasting doing this in primary care versus doing it in specialty care and seeing what the uptake is because my impression would be the uptake and the interest is quite different. Thank you. Yeah, I just, I wanted to touch base with you, Kelly. You brought up cascade testing a couple of times in your study. And at least in our studies, there is a lot of family communication and people are talking about their results positive, negative that their family members, but that doesn't translate to cascade testing. And so I wanted to get your thoughts on how cascade testing fits into this idea of at some point we will be scaling up to large-scale population screening. But in the meantime, as only some people are getting testing, that is a great way to identify families at risk. But there are huge barriers there. And so is that part of what you think should be studied more in depth as we're getting to this larger scale population screening? Like, is that gonna be kind of an interim approach? Yeah, I think one of the big barriers there is the fact that when you think about a family, they're widely spaced out. And even if you can communicate a risk to another family member, the roadmap for that family member to go get the follow-up care and testing that they need, there are hoops upon hoops that they potentially have to jump through to do that. And some of that I think is thinking about how can we as in the disclosure, in the communication to the province and people that we're engaging with, are there better ways to pave that path for those relatives and passing off resources that are not just for the patient, but are actually for those other relatives to make that easier, making it more clear and making sure that no matter where that relative lives, that there is a way for them to get that care that they need and building those pathways and pipelines and communicating that through the patients themselves. I do think that's an area of right for research that if we can improve that, we can exponentially improve the impacts of population screening, that by we can fix that pipeline and make that easier, you can impact many more people by every person that we're identifying through population screening. And I know there are some efforts out there through AI chat, those types of things to help facilitate that. And it's been a really interesting research. And I think that's a place where we need to do more and that we can do more testing of different models to figure out what works best. Carol, Jillian has a question. Yeah, go ahead. And then Ned, did you withdraw? You're good, okay. All right, there we go. And then after this, then we'll go to George. So this question I think is primarily from Melinda and Callie or maybe others in the room who know about this. Do you think of what you're doing in primary care under the umbrella of collaborative care models that are being done in psychiatry, mental health with primary care? And do you think that there are learnings from those models that we should be adopting? Because it does seem like they're solving some pretty important problems in the space, including like even CBT codes for how those models can be sustainable. So I'd be curious for your thoughts on that and also where those models may not apply in genetics. I mean, I think the short answer is yes. I think that there's a lot to be learned there and thinking about what are all the other issues that come along with that in terms of liability and this and that and how those things play together and how we can build models that are scalable, that you also have to figure out how to have those things being reimbursed. But at some point that has to get paid for the effort of these providers in that more support role. Yeah, so I agree. I think that the collaborative care models we have a lot to learn from. And I think just like you were saying is that the big takeaway from collaborative care is giving the primary care provider that confidence, that safety net. I think someone used that term, maybe you used that term earlier. And I think that's exactly what I'm advocating for and replicating now in scalable tools that are creating that same degree of safety net and confidence in the tools that can allow us to scale even beyond. But I don't think that I don't think it's wrong at all for us to be starting in this intermediary space with this collaborative care model and just like we have a pharmacist serving perhaps 20 different clinicians that we could have a genetic counselor serving 20 different clinicians as that resource. So I think it truly is that concept of having that expertise and that confidence building to be able to move forward, that we need to really be learning how to integrate and to replicate the scale. So George and then April and then Karen. Thanks George Manson from NHLBI but maybe to paraphrase Mark, I should say George Manson from the federal government and I'm getting help. But you know, the challenge you described extends beyond primary care. And I mean, I see the same thing in family medicine and the comment I want to make is I hope we would capture that as one of the critical challenges that's really right for research. So for example, the National Academy of Medicine's conceptual model for meaningful community engagement rather than working separately and then bringing the answer to here I have a model to help you, it's not gonna work but it definitely would work if we re-approach it by using a similar conceptual approach of meaningful engagement with whether it's primary care or family medicine and really developing the solutions that they've created, that's more likely to work. I think community engagement and just a teaser for the next session coming up and implementation research can really be very helpful in addressing it. Thank you. April. Yeah, I just wanted to mention when we were kind of talking about engaging primary care and how it's easier with sub-specialists and things like that. And I think that, you know, I think we have to really be careful about focusing a lot on sub-specialists because that's where a lot of people have bigger barriers, right? People have bigger barriers getting into sub-specialty clinics. And so really thinking about, what are we thinking about the long-term thing? So what are we gonna be doing in our medical schools in our residency programs and all of those things to make sure that people who are coming through understand how genetics is incorporated into whatever specialty they're a part of and having that being a more robust, ongoing education so that we are not kind of losing that, you know, interval time period to say, okay, even though you're gonna be a family practice physician, you should still know these are the top-down levels of where genetics is gonna be a part of your patient's care. So it's not an ad for them, it's just a part of their regular flow the same way we order a check-sex rate to evaluate for pneumonia or whatever. Thank you. Carol Horowitz. Yeah, thank you. And thank you for what you said, George. I completely agree with you. I, you know, we do our work in many academic community and safety net, family medicine and primary care practices. We're, and I'm interested in learning from you what the problem is. We do what George said, which is we bring those frontline clinicians in to develop everything with us every step of the way they're saying, this will work, this won't work for me. The message they give us is we care about this, we just don't care about as much as you all do, because we have 50 other things to care about at the same time and we balance it out. So I'm interested from you, are you finding that people are not interested or that they're balanced? Carol, you cut out there just briefly at the very end. Could you restate that? Oh, I was asking if you're finding that the primary care folks, clinicians are not interested or are they just balancing with everything else on their plates? Would anyone like to respond to that? Rex, maybe? Yeah, I can at least tell you what our experience is. And, you know, I think it's both of those things. You know, they're very busy. They have, what is it, seven minutes per patient that they need to get them through. So that's a problem. And then the increased demands I think of, you know, messaging through the EHR and dealing with all of that has put additional stress on people in the primary care community. So I want to acknowledge up front that they've got a lot on their plates already. But what I'm struggling with is the disconnect between people saying the primary care community should be doing more of this with the experience that we've had, which is they don't know how to fit it in. And I think several people have said this. They're maybe a little nervous about making a mistake because they're not well enough prepared for this. But then I think all of it, the second piece of that though, which does surprise me a lot more is, and maybe it's unusual at our place because we're an academic health center, right? We're not a community-based organization. And what we've experienced is they take initiative. They go out and they read the papers and they're unpersuaded by the literature that persuades the rest of us. And that's a separate problem that I don't know how to overcome. And, you know, is it just that we're true believers and since we're true believers, we're willing to accept it? Or is it that we still need it to the idea of a research agenda? I think this was off the table though, but that we should, we need to generate better and more convincing and compelling evidence that what we're proposing is of value. Is it ignorance or apathy? I don't know and I don't care. So I was just gonna add a few comments there. So I think there's a couple of things. One, I think there's a difference between thinking about overall integrating genetic testing beyond genetic specialists. And I totally agree that the subspecialists are highly engaged and highly motivated to do that because there's so much that's impacting their ability to actually do diagnosis now and genetic guided treatment for those patients versus screening, right? And population based screening and really the population based screening being in that primary care wheelhouse. But one of the things that, you know, we need to do to make, I agree that the providers are interested but busy and overwhelmed but we're not helping them prioritize this because there are no guidelines, right? Their societies don't have any guidelines, you know, until it becomes a checkbox that the USPSTF requires primary care doctors to think about, that's really hard for them to think about and then there's also no incentive. And so we are paid to check other boxes. And unfortunately that's the reality of how medicine is practiced. And so we really need to think about not only proving again, getting back to that value for their patients which I think primary care providers really care about but then value to themselves and guidelines for which to follow to do all of this. So, Jessica, did you have something else? And then at the end of the table here, I think. And Jonathan. And then Jonathan next to you. Oh, Jonathan, how do I miss you? And then Dan. Yeah, so I just wanted to say we did a study within Kaiser Permanente of who was helping patients at high risk due to genetic variants with their follow-up care. And everyone pointed to the other primary care point here. The specialists, the specialists, but they're not coming back to us. They're getting their diagnosis and they're leaving and doing chart reviews. A lot of times the guidelines in their record are outdated because there's no contact to help them update it. And so I think it's a problem on both ends. And who do we expect to be taking care of them and the follow-up and is that who we can guide to get the testing in the first place? Can it all be grouped together as part of care? And so anyway, I just wanted to emphasize that's even in a well-resourced system of everyone was pointing at the other. And at the end of the day, the patient was saying, I'm taking care of myself because no one's helping me do it. And I don't think it was for lack of interest. I think they generally thought someone else was doing it. At the end of the table there. Hi, okay. This is obviously me, Alana at Geisinger. And as I wanna echo also what Jessica just said, you know, I did this in the collaborative model with psychology co-located in primary care. When I was at Kaiser, we actually experienced sort of the same thing and our co-location collaborative model ended up with a visible in primary care. How I have no idea, but it was what I wanted to get. So that is one issue. And I wanted to get also at what George and Carol were saying too though is, you know, we've heard this co-creation and co-, the engagement in the co-creation and how important that is. And it sounds like there is some movement in it working. And I wanted to hear what has been created then. And so, you know, we keep talking about what we're doing so far isn't working. So we need to co-create things that could work. So what have we found so far that may be working or maybe those first steps on the pathway? And maybe you haven't gotten that far yet with that. You co-created it, you haven't actually tested it out yet and that's okay. But I'm wondering if there is anything what is working so far when you do that co-creation? Comments from any of our speakers to that point or anybody else in the room? So when we were setting things up for our screening program, the question was, does it sit in a clinical situation and go through clinical care or does it become a research protocol? And through the co-creation and discussions with primary care specialists and others, they wanted it as a research protocol. And so we pulled it out of what would be clinical care and made it into an IRB protocol. It's a whole separate team. It's ordered by one physician that is the PI of NRD and ASC. And so that's not maybe answering the exact question, Alana, about like primary care, but that was sort of a community engaged approach where ultimately the decision was that it's not gonna go into the clinical pathway, it's gonna go in the research world. And did you, in that co-creation process, do you have the why they wanted it as a research? I think a lot of it is what we've been talking about, just being concerned about feeling comfortable ordering, the volume, our physicians are happy to talk about the program, but then to take it to the next step of actually being responsible for some of the ordering and follow-up, they wanted that to be really outside of what they're doing on the day-to-day. So we've exported that to our research protocol. So it wasn't about that we need more research, it was about that they was practical. Practical, needed somebody else to take on the burden. Yep, exactly not. So we're getting pretty close to the end of time. So, Kelly, if you wanna make a comment and then we'll do three quick questions and then I think we'll be pretty close to the end. Sure, I was just gonna echo that we've had very similar feedback. And it comes down to logistics of that, just in talking with our providers from an implementation standpoint with, I was working with primary care clinics in Alabama that the providers see a lot of value and benefit in having people there that are managing this project that are, they do that, you know, nurses and people that are part of the research team doing that navigation that can do this all day long and get really good at this talking points and talking to patients about it and kind of having that as a unique role that's kind of sitting in that and it's not falling to the providers that is paid for by a research study though. And so at some point you gotta figure out how that can become part of clinical care but thinking about who's doing that work and whether there's a role for somebody doing some of that effort specifically. John? So my comment is on something a little bit different it's been a great discussion but April there was something that you said about drawing lessons from ancestry based carrier screening and I think we could probably all agree that we want to avoid going down the road of ancestry specific screening programs but I wonder if you would comment on whether you think that including, we certainly want to include conditions that are of importance to particular groups. So thinking about April one for example is that something that we can envision as a vehicle for community engagement around a condition that is of importance to that community but also then broaden the opportunity for screening for the other conditions that are thought to be important for population screening. Thanks. So your question is really like if we take this condition that's known to impact this population and then use that to start the conversation for what's next, right? And I do think that that is a really a good point. So like for example, if you take sickle cell, right? So you take sickle cell disease which impacts African-American communities and there's a big disconnect between I have diagnosis and I get treatment and what those barriers are. And so talking about what we know we can screen for this and identify you and these are the strides we're making to improve your health via this and showing you're building that trust in the system that maybe we forgot about this here but we're coming back and we're including this population now. And so building that trust that's definitely gives you that opportunity to take the next step. So I definitely agree that's a very good thought process and how can you incorporate some of those things. But I think the other piece of that though is you definitely have to have somebody who's in the community who's also already has that trust, right? So you have to kind of have multiple layers of building trust with people who maybe don't trust the system that exists. Yeah. Despite you can be changing the subject or sort of opening up a can of words but I'd like comments from any of the speakers about the mechanics of cascade screening. So the issue is you have to rely on the family member to get the other family members to do that. There's a real communications problem there both in terms of what they tell their family members and whether they tell their family members and whether the family members understand or respond. And I think that they can sort of a gap in the way we deliver genomic care. And I'd be interested in hearing how you think it, how you deal with it and how you think it ought to be dealt with. Yeah, that's a great point. I think that evidence from a number of studies is that families, I mean, there are absolute exceptions to this rule and lots of reasons. Some families don't communicate. But largely when we ask people, who did you talk to about your results or that you had this testing? The overwhelming majority of people are talking to at least some of their relatives. Whether those are the same relatives as the ones that would be at risk, I don't know. But I think we have an option. The spouses are unrelated to them usually at least in, except in Tennessee and Alabama, maybe by. Yeah, right, right. But I think that there is, it's not a perfect solution, that there's an opportunity to help improve that normal communication that is happening. But thinking about how not to rely on a patient describing those results correctly or what they should do with them and additional tools and resources that can be built and handed off where that all you're relying on the patient to do is to pass this to this other person and that there's enough information there for them to not only understand their risk, but to know what on earth to do with that risk if they live in Alabama or California or wherever else. But I think it's also important to note there's been two or three studies now where direct contact has been used where they haven't gone through a familial intermediary other than make initial permission. The cascade testing rate is exactly the same as familial communication. So there's something beyond just communicating within families that is depressing the uptake of cascade testing. Contact to family members that permitted with permission from the family. Yes, yeah, so that's allowable with permission from the program. In every state. But also those family members are going to have the same barriers as well, right? So, I mean, you have to consider, like, do those patients have access to a lab? Do they have access to pay for this testing? So every barrier you bring down for that initial contact has to be done for all those family members also. And then also you have to consider that some people there may be stigma associated with having an abnormal test result, all of those things. And so even if they've told their family, maybe it stopped there because they're like, we don't wanna talk about this.  It's, this is one of the value ads of population screening though, right? We actually take that burden away from families. You know, if we think of newborn screening, we don't do, we don't really do cascade testing on newborn screening results because we have confidence in the system and that everyone is accessing their screening. Yeah, it becomes a transient problem in a future. And one more comment? No. So I think we're up to time here. So just quickly on this top of the logistics, this was a far ranging discussion, but earlier we had a comment on logistics by Christine about sample collection tracking and all that stuff. That is an important part of the logistics that I think we're talking about here. And in this session covered a wide range of topics. Who to test, when to test, who does the testing, how to report, who reports, who takes primary responsibility for the care, is it primary care, subspecialties, preparing the workforce, the timeliness, how it's delivered, when it's delivered, the need still for the informatics infrastructure to align to make that information exchange seamless. I think we could talk for days on that. And I would love to. Sort of informing the public, conforming positions, informing families, rolling this out in an iterative fashion versus trying to do it all at once. We talked about risk estimates and the idea of maybe systematic or automated processes for implementing guidelines on who to test rather than having very complex guidelines that may be difficult to actually manage. And the idea that a combination of approaches population wide and those focused on high risk groups will be needed to do this effectively. We talked a lo