t about sort of the challenges of underestimating the diversity in patient populations and not capturing that diversity adequately. So the multi-dimensional nature of identity that crosses race, ethnicity, and ancestry, we don't represent that very well. And it is an important part of the logistics of actually delivering genomic medicine at the population scale. And there were a couple other points in there. The barriers to access the cost and everything is not equally shared across the population. And there's logistics around circumventing provider bias and discrimination. And then finally, understanding provider training and patient education and communicating, especially I thought from that, an accurate picture of risk based on screening so that people don't underestimate their risk or overestimate their risk. And I think that balance and that how to communicate that is definitely one of the logistical things that we need to tackle. And there's probably great research project around that. So that's my quick summary of the logistics. Super, thank you, Carol. So at this point, we have a break. We'll break until 325 Eastern and then the back for our final session today, which is on community engagement, which has come up quite a lot. We have several speakers and then a panel as well. So see you at 325. And thanks to all of our speakers. Okay, ladies and gentlemen, the speaker is at the podium. So if you could get to your seat, we've had a really wonderful session already today, series of three sessions. The good news is this is the final session and we're gonna do our best to get you out of here. There are gonna be three presentations and we're followed by a 30 minute panel discussion and Rex and I are gonna do our best and we promise to get you out on time. The first presentation would be given by Vanessa Huretsuka. She's from the Center for Human Development at the University of Alaska Anchorage. She has extensive experience in the ethical, social and legal implications of genomics research and precision medicine, especially in indigenous population. So we're really delighted when she agreed to get this first talk, which is on American Indian, Alaska Native community engagement preferences and tribal code requirements. Vanessa, over to you. Thank you so much. Thank you for joining us here in the room in Bethesda as well as to those of you that are online in Zoomland and on what was it? Genome TV, I think is what I heard. Sounds so cute that name. I'd like to say thank you so much for having me here and allowing me to amplify the voices of the Alaska Native American Indian peoples that I've been working with and what they've been saying about genomic medicine, about precision medicine, about genetic testing. I come to you from Anchorage, Alaska where I am now with the University of Alaska Anchorage have been for the past three years. Before that I was community based in place. I was a senior researcher at South Central Foundation, a tribally owned and managed healthcare system. And what I'll be talking with you about today is largely from that time, as well as some of the work that I've been doing along with others at the Center for the Ethics of Indigenous Genomic Research. My pronouns are she, her, hers, and just a visual description for those of you that may not be able to see as well as others. I'm an American Indian woman with dark brown skin, brown hair that now has some gray in it here and there and more there and more here than I would like. I'm plump, I'm wearing a flower shirt that is somewhat colorful on a black background.  I would also like to do a moment to acknowledge the lands of the Indigenous peoples that we're on. I will massacre the names of the peoples because I'm not from these peoples. And I don't think I've met anybody from these groups either, which is a shame and also part of the historic trauma that Indigenous peoples have as part of our history. But I would like to acknowledge the Anacostin and the Piscataway people whose lands we're on here now. I am a grateful guest to be here as are we all, I'm sure. One of the things that I had wished to do was to just quickly describe that a woman, when I'm speaking of Indigenous research, I'm speaking of the American Indian, Alaska Native peoples. As a Navajo and Dine, so Dine is our tribal name and when I'm a Mwentu person, when I fill out my demographics for a research study, I will go ahead and mark down American Indian or Indigenous or Native. And what that means is that I belong to a particular tribe. I am from the federally recognized tribe of the Navajo Nation. And there are 574 federally recognized tribes. When I mark that down, it's often in this race ethnicity category, but it's also for American Indian, Alaska Native, Native Hawaiian people, a political category. And I really wanna state that outright because much of what I'll be talking about and much of what our participants in our empiric research have alluded to is coming from these 574 tribal nations as sovereign nations that have the ability to and are enacting research policy as it pertains to genetic research but also to other forms of data sovereignty. So I think that's an important context point and I do wish to belabor it because so much of the research and so much of the community engaged work that is occurring amongst Native peoples, both here in the United States where we do have that political designation as well as Indigenous peoples worldwide is with an idea that we are at one with other human beings but also with the spirit, the land, the cosmos, et cetera. And that brings us to a different set of values. Here in this slide, I am showing a slide from the Alaska Native Knowledge Network that's describing some universal values of Alaska Native people. In our state, we have over 200 tribes and like any group of 200 groups of people, it's hard to find universal stuff but this group did and I think it's important to contextualize when we're talking about peoples that we talk about the values for which they hold and some of what I'll be covering to use some of this language that I realize many of you are using other forms of language in the way of our technical terminology as it pertains to genetics, genomics and precision medicine and you may be a little bit less inclined to your humanities. And so let me just give you a reminder about a few of the things that you might have forgotten or just put aside since freshman year of undergrad. So I'll be speaking today about terms that are reflective of ontology, the nature of reality and of what really exists. Epistemology, relationship between the knower and what is known. Axiology, what we value. And of course methodology, those strategies that we use to determine to justify the construct of a specific type of knowledge. And I'll just say it outright, the methodologies, those methods that I use in my research are both post-positive types of methods, the scientific method, but I also utilize an employee within my research because it's of and for indigenous people, indigenous research methods. And all of this is to say that these methods, these paradigms are grounded in the principles of relation, respect, reciprocity, responsibility, resistance, resilience, resurgence, restoration and reparation, all are words that you can use when you're doing these categories. But also very important principles, particularly to indigenous peoples. And so I realized with 15 minutes to, summer, a body of research is a little bit hard, but I did want to leave you with a few of those ideas that what I intend to speak about is how the people that I have come to serve in Alaska, the Alaska native leadership of those tribes have sought to create our research codes and research teams and the questions that we ask in Precision Medicine and Ingenomics Research in a way that where researchers and those being researched and those that are caring for those being researched are working respectfully, ethically, sympathetically and benevolently. And I say that because the way that those terms and what's elevated ethically is determined by the culture there. And that's why I wanted to show terminology of universal values there in English, even though they come to us in other languages. So again, for those of you that are not living in the world of social engagement and community engagement, I had wished to put forward a broad definition of community engagement, the process of working collaboratively to proactively seek out community values, concerns and aspirations. And I deliberately use those words. Proactive is what we've been asked from our tribal leaders in the Anchorage area to work on. So when there as a concept of a research project that that ought to be brought forward and under discussion. And under discussion with whom, I think is probably the question that many of us would ask next. In the case of the Alaska native peoples and many American Indian tribes, we've entered into contracts and compacts with the federal government to engage in healthcare services on with and behalf of our peoples. And so tribal leaders are hiring and administering and managing healthcare systems now. The healthcare system that I had worked for is one of those healthcare systems. So the stakeholders of interest are indeed those administrators that are tribal leaders, those providers within the healthcare system and the community members. And many people that are those community members are both those enrolled tribal members from that particular tribe in that area or people like myself that are guests or moved there. And so I am not from the area that I'm currently residing. But I've been grateful to be able to be accepted there. Some of the terms that we'll be discussing today are listed here. And I just wanted to say that these terms are influenced by community members. And again, when I say community members, I'm thinking of not just the everyday person walking around, but also the people that are wearing special badges when you go into the clinical health system as different forms of providers and support staff, as well as those leaders that have hired those providers and support staff in the health system that I worked in, we were called customer owners to further acknowledge that power relationship, not just a patient, but a customer owner, a person that owns the healthcare system and is also receiving services from it. So our community context is influenced by the hopes and expectations, but also by financial resources, by job and family expectations and by the broader social context of which we will hear more on. And when we talk about participation, I like to reflect on a little thing that's from 1969, but it holds true now. And this is the latter of participation that Sherry Arniston had presented in a journal article back in the, back in 69 in the Journal of the American Planning Association. And I don't know about y'all that are in the room. I think those of you that are on Zoom and in Genome TV can see it a lot better. But here on the ladder, the bottom rung of the ladder is, there's 0.1, manipulation, 0.2 therapy, and those two rungs are considered by Arniston, non-participation, areas where it's not exactly participation in the engagement process. And then moving towards number three informing, number four consultation and number five placation. All three of these are described as tokenism. And then area six, partnership, seven delegated power and eight citizen control. And those are referencing citizen power. And I ask you to consider thinking in these different areas as you're thinking about community engagement. If you define that community engagement as engagement of primary care providers or of another stakeholder group, I tend to think of this across stakeholder groups and triangulate between those various groups that I had mentioned. But this is just something I put forward to you. If you haven't seen it and haven't engaged with it or haven't engaged with it in a while, it's a handy way of thinking about participation and in aspects of engagement. So in brief, how do we do engagement? And I think again, this is about something that's been mentioned several times. It's that forming of trust. And for the work that I've been encouraged to do and required to do by our tribal leadership via tribal codes, it's been to have active engagement of various stakeholder types in all phases of the project. I know during lunch, I was just talking and we were talking about, who it is that I work with. And now I'm at the university and I don't work with students. I work with community members. Now the work that I do is with individuals in the disabilities community broadly. Prior to that, I would do work with various members of the indigenous community as co-researchers, developing those research skills and training. So when we talk about active participation in our tribal codes, we speak of conceptualization of the project as an area for active participation, the conduct of the project, both in the way of how we manage the process of research, how we manage resources themselves. And by that, I mean money and how we manage conflict. So who decides what goals? Community engagement can look like a whole lot of things. We've mentioned several of those things here. Depends on the project. And I've listed here in engagement methods, some of those more post-positivistic types of engagement methods. And these are several of the things that we've done through Seager. Some of the work that I've been participating in in this century has been around bio-banking that's owned and managed by Alaska Native people through the Alaska Area Specimen Bank. And then that led to additional work that was through an R01, looking at community-engaged research methods around precision medicine, where we had conducted multiple public deliberations in a variety of different tribal settings. And then some survey work, as well as focus group work, and interviews, and all with indigenous researchers, led by, with the data owned by, the various tribes in tribal consortia. I've provided here a few different participatory research practices in empiric form, and some discussion for you to look at in the future. And just to note that I wanted to end on in the way of a scoping review. So a literature review. And this is something that my team had conducted looking broadly at participatory research in American Indian, Alaska Native communities. And what we had found after this thorough combing of the literature was there was, and these again are papers that were published, things people wanted to talk about publicly, right? And in this, we still see that there is a need for community engagement in the community. A need for community engagement in the early stages of the research processes that there's an importance of guidelines for American Indian, Alaska Native communities, more tribes, and putting forward tribal code that is very specific to various issues. Vanessa, could you wrap up on this please? Yes, I can. Thank you. And then the last thing that I wanted to leave you with was just some things that we each could do in our own work regardless of who we are. When you're working with tribes to respect sovereignty, when you're working with any humans to respect self-determination, to follow the lead of the community, practice transparency, humility, to acknowledge the harms that maybe you have not caused but that exist for the people that you're working with and those that may come yet to build local capacity, make long-term commitments, and be flexible and creative. Oh, thank you very much. Great. Thank you very much, Vanessa. The next presentation will be given by Crystal Totsi and Crystal is an Indigenous Genesist and Bioethicist from the School of Life Sciences at Arizona State University and she's gonna talk about the opportunities for meaningful Indigenous community engagement for population genomic screening. Crystal, go ahead, please. Yeah, thank you so much for the kind invitation and welcome. Yet, thank you all, hello to my people, my genetics people. I am going to do this rapid fire and I have a timer to keep me at 15 minutes. But I do wanna start that even with the recently expanded ACMG panel, only a handful of medically actionable genes have very information specific to those of Indigenous backgrounds. Well, how does this affect clinical utility? What does this do and what does this mean for equity? Now, we've seen variations of this diagram through the wonderful GWAS Diversity Monitor and this other, which I've circled here, has remained stagnant over the last couple of decades as it pertains to the inclusion of Indigenous peoples in GWAS studies or really any type at large of genomics research. However, this is not a matter of engagement via recruitment nor is it selling Indigenous peoples on the benefits of precision medicine and health. It's a matter of overcoming mistrust thinking more approximately about health and empowering data decision equity. This tells you about the structural inequities in terms of how healthcare is funded. Not only, this is a pre-pandemic statistic in that before in 2018, the rate of funding funds to an Indigenous patient was 2.4 times less compared to the dominant population. So before we sell the next innovations and promises of precision medicine and health, we also have to reckon that there is a severe barrier in the amount of promises that the federal government has long promised Indigenous peoples in exchange for resources. And also, we have to, this is a subject of a paper, that when we keep telling Indigenous peoples that they're gonna miss out on precision health benefits if they don't engage, without changing the power and balances that created research harms to begin with, what we're doing is effectively engaging in a cycle of victim blaming and coercion. I also want to say that simply dropping clinical genetic tests into our communities is not gonna solve the health equity problem. This is just a general clinical pathway of care for most rural travel patients that want to seek genetic testing. In this case, an Indigenous patient has to refer it out by their IHS provider, often located 100 plus miles away, which means that they're taking time off of work. They are driving, if they can find a car, pay for gas, get to a hotel, pay for several nights in that hotel just to see a provider, sometimes appointments several months down the road. A question, of course, is whether or not there's a genetic counselor that's available to contextualize clinical genetic test results. At the end of the slide deck, I have a very sobering map that shows you how very few genetic counselors are available in rural areas, in rural states, in states that have Indigenous communities. Are these results interpreted against that relative lack of information on Indigenous-specific gene variation? What is the potential impact in terms of the validity of the information that we're delivering? What training is available to deliver cultural-specific care to Indigenous genetic test users? Are patients being fully informed about default data sharing? And then here's an emerging question that we should all be considering. What are the risks for Indigenous patients who are not covered by federal privacy laws through GINA, IHS patients, and also many of our veterans, a huge portion of whom are Indigenous, are not covered? This is a catch-22, actually, that it's a decision tree that many Indigenous patients face on asking the question of whether or not to contribute DNA for clinical genetic testing. So on the one hand, some patients might decline because they can't afford access testing or they're concerned about data usage. On the other hand, those that do agree implicitly broadly consent to any secondary data usage and data ownership by testing companies. In any scenario, this is why it's a catch-22, they derive little to no clinical utility due to lack of informative relative variants specific to their peoples. This is outside the consideration of many clinical care providers who, of course, are focused on quality of care. But by simply using a clinical genetic test that means that commercial gene testing companies can co-opt and clean ownership of Indigenous peoples' genomic data. This is something we don't talk about enough, that this data will be deposited into public data sets like ClinVar, even if patients know to ask to opt out of dating sharing due to tribal data sovereignty rules, they don't have this option. Urban tertiary care centers that see Indigenous cancer patients may also be biobanking samples and data for research under broad consent without tribal data nation approval. What does this mean in terms of taking advantage of vulnerable populations that don't have any other alternatives? What is this also should raise ethical concerns about the conflation of research consent versus the consent to care for minoritized communities? There have been many studies that show that when you insert a form that consents to having your data being used for other genomic studies amidst your other forms on intake and at a point of care that many patients implicitly trust that white coat. Do they know what they're signing? On that topic of equity, this is something that commercial companies have told me as a co-founder of an Indigenous-led biological and data repository called the Native Bio Data Consortium. They asked us when we founded this nonprofit research organization, how many Indigenous patients have crossed what phenotypes? And we pushed back. Why don't you use our people's data and our DNA to study conditions that specifically impact us? And of this, we had a list of childhood conditions and metabolic conditions and what you all as a dominant population would call rare variants. And we were told this by every single company that it's not profit generative to use our people's DNA to create therapeutics that specifically impact us. So recruiting more people into data sets is not gonna solve the health equity issue. Dropping genetic tests in their communities is not gonna solve this issue. Using the word democratize is a false synonym for equity. When it's a system that's gonna benefit most, it's gonna continue to disenfranchise us as Indigenous peoples. So this is something to consider when you're writing that next grant call. There are issues in terms of using population descriptors. Like all processes of gene flow and drift, Indigenous peoples have had longstanding systems of kinship and relationality that are not mediated by blood, such as marrying into neighboring tribes and also adoption. And our clanship systems acknowledged this heterogeneous background under unified identity. Didn't have people to navigate people before contact had over 400 clans. Many of these acknowledged that we acquired people from Pueblo individuals, Acoma peoples, Zuni peoples, Mexican peoples, Mexican peoples, nation peoples. Yeah, it wasn't until 1934 that the Wheeler-Howard Act was stamped upon us and posed these racially defined population groups meant to define us and dilute our rights to these resources. But these ended up getting conflated under these eugenic notions of race. And it's a dialogue that we are constantly in consideration and I'll bring that up in a moment. But it's very interesting because of a few data sets that are out there. These are from Indigenous groups with completely distinct genetic and cultural histories from our own. From those disempowered groups that probably are not even recognized by their own colonial governments and in Mexico and further south and Central and South America. And yet these are the available data sets with sometimes as few as 30 Indigenous individuals meant to impute something, make some inferential statement about US Indigenous peoples. I don't have to tell you about some of the issues with using PCAs for principal components analyses for ad mixed populations. For Native Americans, we are often left out because our analysis on our small data sets necessitates a different type of small statistical sampling procedures that are incongruent with standard QAQC pipelines. Or worse for people that don't understand how unique our 574 plus barely recognized tribes in the US are, we get lumped into the same category which doesn't benefit anyone. It also reifies assumptions of biological purity. When we are looking for to recruit least ad mixed Indigenous peoples, we are also ignoring the real life the lived experience of other Indigenous peoples in regions like the Southeast and East of individuals that have had more contact and more recent colonial settler history. It also ignores real contributions of inequities due to social and structural determinants of health. We need to rethink one person, one tribe because we often are categorized as one tribe affiliation. Again, our populations are not stagnant and not even since 1934. We do a disservice when we default to colonial definitions of dignity. Do not acknowledge that peoples can belong to multiple tribes. I'm so grateful for and thank you for one thing that I am loving is this move away from genetic ancestry to genetic similarity. This type of type of logical move is something that I think should make transparent some of those statistical inferences that are irrelevant sometimes for our Indigenous communities. But in terms of like how members feel, the job thing that was a topic of my own research recently because we often default to tribal leader to leader our conversations. Forgetting of course that leaders are elected officials. And we don't have the longevity of office or the transmission of knowledge from those individuals from one administration to the next often time. But those realities are consistent with community members. And I simply ask them, will my genetic data be used in other studies without knowledge or consent? That was actually the top concern. Even more than will the research benefit my tribe? That community members rated job and education opportunities created by health research higher than benefits of researching disease or condition. And this is because we think in terms of seven generations. We take seven generations out. We're not thinking about our pathway of innovation. We also need to think about all the other dimensions of equity to include data decision equity. And this is an awesome article report on which I was a committee member. Crystal, could you please finish on this, please? Yes, thank you. So we are now thinking about using AI machine learning approaches, their blockchain federated learning to advance data consenting and data sharing, creating our own portals that advance on group consent and broad consent through dynamic consent, thinking about metadata labeling for the labels on repositories that we don't own. But we're also thinking about building bio economies that are structured at all these different labels, levels of policy and governance. We also have to ask what happens to indigenous peoples outside of federally recognized territories. And I just asked that when we're thinking about equity that we're also thinking about benefit equity as well as decision-making equity on top of engagement equity. Akiahat, thank you so much. Great. Thank you very much, Crystal. The third presentation will be given by Minkid Lee. Minkid is the deputy chief engagement officer for the NIH All of Us Research Program. And she's gonna be speaking about advancing genomic research through community engagement on the All of Us Research Program. Minkid. Thank you. Good afternoon, everyone. Thank you for the opportunity to participate today. I've been hearing a lot of different perspectives and I love the discussion, the active discussion going on. I'll try to address some of the points where all of us was mentioned today and where I think all of us could provide some insight into best practices in community engagement. So the All of Us Research Program launched in 2018 nationally with the mission to accelerate health research and medical breakthroughs, enabling individualized prevention, treatment, and care for all of us. And we aim to do this by nurturing partnerships with at least one million or more participants who reflect the true diversity of the United States, which we've been hearing about a lot today. Ultimately, with the data that comes from those participants that they've so generously donated, we hope to deliver one of the largest, richest biomedical data sets that is available to researchers. So how are we doing? As of last week, we have reached over 500,000, so half a million participants who have consented to the program, consented to sharing their EHR, and also completed initial steps of the program, which includes a basic survey and also providing their physical measurements and biospecimen, either blood or saliva. Over 728,000 participants have created an account, so enrolled and consented to the program, and we have over 400,000 EHR connected to the database in our platform. And then, diversity-wise, we are also doing pretty well while the All of Us Research Program is not solely a genomic study, with the numbers that we have currently, it does make us one of the most diverse genomic studies so far. We have over 80% of participants self-identifying as coming from underrepresented communities in biomedical research. Our program defines those by both age, race and ethnicity, geography, if you're rural, or access to healthcare, disability status, sexual orientation and gender identity, and educational and social income as well. And then by race and ethnicity, we are almost at 50% by the information provided by participants. The main contributor to how we reach this diversity really goes back to the program's intentionality from the very beginning. I was lucky enough to have been a part of the program as part of the Division of Engagement before national launch. And I can contest that. We had funded and we had selected and funded inaugural community engagement partners even before we enrolled our very first participant into the program. So we began with the Asian Health Coalition, PrideNet at Stanford University, the National Alliance for Hispanic Health, 50 forward in Nashville, Tennessee, and Delta Research and Educational Foundation. Each had a specific community that they worked with. And since then, we have not only continued those five partnerships, but we've also expanded our engagement ecosystem to now include 18 nationally funded community partners. If you count all of the sub-awardees through the different partners, we have over 150 partners across the country. And we know that the messenger matters, especially when you are engaging these diverse communities. So we have a grassroots local organizations as well as national and regional organizations, again, conducting various outreach and engagement activities across the country. And as with the expansion, we were very mindful of certain gaps and needs and priorities of the program. When these partners go out and do their work, we work with them within this framework that we've developed over the years. And in our program, we define engagement as bi-directional relationships, development of those relationships, and also maintaining those trusted relationships. Often their work and our work begins with outreach and awareness, again, focusing on fostering trust with those participants, potential participants, and community members. We build upon those activities and go into education and increasing access to the program, which I'll speak a little bit to in a few slides. We focus on engagement activities that can ultimately lead to enrollment and retention in the program, but also again, focusing on maintaining those trusted relationships with the community members and participants. For participants who have enrolled into the program, we really approach them as true partners. We have several community members who have identified participant representatives. Into the program, we have a panel of participant ambassadors and also participant partners who are serving in various program governance committees. So I know earlier we had talked about co-creation prior to the inception of the program. There were 17 community engagement studios across the country. One of them was with Urban American Indian Alaskan Native communities and those community engagement studio sessions really fed into the development of the program protocol. And we've had participant ambassadors serve on the very first steering committee meetings as well. So we are very big on co-creation and co-development. As an ultimate outcome of all of this engagement, we do focus on knowledge mobilization, where either it's the data or the research outcomes that come from participants. We want this to go back to either the participants directly or the communities that they have come from so that this benefits their communities in the end. So from here, I wanted to share some best practices that I think have worked great for our program in hope that other studies can benefit as well. An example that worked from the beginning is really having those partners at the table. So these partners not only go out into the communities and conduct outreach and engagement as our trusted messengers and liaisons in those communities, but they are also at the table at the program level where they are co-creating and co-developing with us. PrideNet is a great example where even again, before the launch of the program in developing those basics survey questions, the questions that collected information about gender identity or sexual orientation, sex assigned at birth, PrideNet worked with us as a co-developer and helped us revise those questions and really perfect them for the participant that we were hoping to engage. Our more recent partnership with American Association on Health and Disability, we worked with them to develop disability related questions in the basic survey last year, which went out and those have been helping us collect disability UBR metrics. And then also we worked with AHD on a communications toolkit. For example of how fast our program works to really implement feedback from these community partners a couple of weeks ago at our programs face to face, one of the participants noted that we were missing closed captioning and they needed it for the meeting. So within a few hours, we were able to get closed captioning added. And when they presented on their disability community toolkit, they noted several best practices not just being 508 compliant and being ADA compliant when you develop assets that person was disabled that could be inclusive of people with disabilities. And they can often tell if the person who is in the picture is really disabled or if you're just putting a person in a wheelchair. So they said that true, truly being inclusive really matters a lot at all different levels. And those are the kind of values that we have been trying to really incorporate into our program at all levels, not just participant facing, but again at the co-development and program level as well. And then selecting these partners, we're also mindful of being culturally congruent in our engagement strategies. Delta Research and Educational Foundation is one of our partners who was doing that kind of engagement. They partner with HBCUs across the country and one of their programs Delta 2020 plus researchers provides educational activities for students and network members within the sorority. So they present a speaker who comes and discusses their perspective research which is aimed at improving healthcare that is relevant to the people who are attending those sessions. Our other partner Asian Health Coalition is piloting a language working group which right now the program is available fully in English and Spanish and they have been advocating to add additional languages. So they're working on a pilot where they will provide additional assets in the program in these five Asian languages and that's under works right now. And they also partner with their community based organizations. So again, partnering at different levels throughout the community is really important and it has been working really well. One of the biggest barriers that I know I've heard today and I don't think we need to mention again is transportation when it comes to participation is a big barrier in times of time, cost, disability, various issues might just be inconvenient. So one of the models that our program noted that works well is to have this mobile engagement asset. We started with one, we realized that having physical measurement and biospecimen capability onboard would help enroll participants from rural and other areas. Those who didn't really have access to an enrollment site, it worked really well. So now the program has two of those vehicles. It's called the all of us journey. If you go on the website, you can see where it's traveling through the country and that journey partners with community partners again and they host them in these big parking lots. The picture here is the one at Stanford University with PrideNet. One event that was really successful was recently earlier this year in Brownsville, Texas and four different partners, the Alliance and AHD and others partnered to host this vehicle. And within a few days, they were able to have 150 accounts created and also almost a hundred physical measurements and biospecimens collected at that one visit. The Alliance calls it the one stop shop model, which works well for a lot of our participants from our UBR communities. So we've been trying to implement that and expand access to people through that model. And I'm gonna dwell a little bit longer on this slide than I had planned. So I hope I'm doing okay on time because I think this is the most relevant to the different population screening and return of results that I've heard throughout the day today. So the all of us research program began returning health-related DNA results to our participants about a year ago now in December of 2022, prior to that we did launch return of genetic ancestry and traits results in 2020. So as of last month, we now have offered genetic ancestry and traits, hereditary disease risk results, which tests the 59 genes of the ACMG panel and then also pharmacogenomics results, which tests the results of seven genes. We've offered each of these results to over 180,000 participants. And you can see how many have viewed their results here. And as of now, we are offering health-related DNA results to about 5,000 participants per week. And while we still have more work to do in expansion and also providing equitable results experience to all eligible participants, I do wanna note that we estimate based on the results that we are returning in terms of the hereditary disease risk reports that about 3%. So I think 2.9% of participants may receive pathogenic or likely pathogenic results. And that when we reach a million, that translates into almost 30,000 participants receiving actionable results. The program does provide clinical validation testing as a follow-up to those participants who do receive pathogenic or likely pathogenic results. And that is actually an outcome of a series of listening sessions that we held with our participant ambassadors and community partners very early on in the planning stages of genetic-related return of results where we heard from our community partners and participants and consortium members that people without healthcare or lack of resources, the results can be very burdensome. So clinical validation testing was added on in response to that feedback. So that's something that we really are, from the division of engagement, we were really proud of that we were able to achieve that outcome. And then all of us participants have access to our genetic counseling resource center. So they can go into the participant portal and schedule a phone conversation with a genetic counselor. You can also request a interpreter into over 200 different languages. And you don't need to have a specific result to request those sessions. So that's also available as a resource. Thank you. Could you sum up in about a minute, please? Yep, I'm almost done. Thank you. Thank you. And we also recognize the importance of diversity on both ends of research. So when the data platform for the All of Us research program launched in 2020, we recognized that we also wanted to have a community of diverse researchers using the data. So we've put forth efforts to help that happen. You can see the number of researchers currently registered as users. And we currently have 97 minority serving institutions registered with a data use agreement. Our division also supports a lot of research engagement work to support that, recognizing that there is capacity building efforts and things like that that are required for researchers to use our data platform. PrideNet held a researcher boot base camp earlier this year. We have a partner at Baylor College of Medicine who hosts annual faculty summits. It's a three day workshop, essentially training people to use the workbench, but also providing mentoring and support for like grant application writing, things like that. And they follow through for the whole year with these research teams to ensure that they are getting what they need in terms of support to use the workbench. And then we have one other partner, a University of Utah who hosts is a high school student, high school teachers every summer to come and use, learn how to use the workbench and learn about the program. And these are just highlights. I'm happy to talk to you more about other partners as well. And that's it. Thank you. Thank you very much, Binky. We're now gonna move directly to our panel session and I'll call on my twin brother Rex to take it away. Thanks, twin brother. So we've heard three really interesting presentations and I wanna make sure everybody writes down the questions that they have relative to these three presentations because we will be including them in the discussion after we have the panel. And I did wanna recognize actually that the title is fairly uninspired for this panel. And so I wanna thank Caitlin for suggesting a potential alternative title too late to get into the booklet of course, but beyond actionability, discussing the who, what, when and why of expanding population screening to new conditions. So maybe that's a better title for us to think about going forward. So I would invite our four panelists who are Caitlin Allen, Ned Kalange, Jessica Hunter, and Bob McNallis to go up to the front. We're gonna actually use the panel chairs that have been there all day long. And what we're gonna do is we've asked each of the panelists to in two to three minutes and we're gonna try to keep them to that to just make an opening comment about how they think about expanding population screening to new conditions and I should say also to new communities. And we'll start with Caitlin and just go in alphabetical order. And then once we've had the initial comments, then we'll have an opportunity for some discussion about the points that emerge from that. And then we'll open it up to the floor and the folks online. So Caitlin. Thank you. So we at Medical University of South Carolina have recently as of 2021, November, 2021 implemented a population wide genomic screening program. And as I mentioned earlier, this was a decision that we ultimately made was that we would do this as a research protocol as opposed to clinical care. And our goal is to enroll 100,000 individuals into the program and we're at 35,000 participants so far over the course of about two years. We do whole exome sequencing and we are working with Helix as our laboratory partner. So they're CLIA certified and all of the results for this program are returned into the EHR. And we also provide free genetic counseling for all positive individuals. So I'm saying that to just kind of lay the groundwork for where we are and how we, I'll let you know kind of how we got there and to Rex's point about future expansion. So currently we're looking at or we're screening all adults in South Carolina. We are doing this, we're returning tier one conditions and the hereditary breast and ovarian cancer, Lynch-Genderman familial hypoclustralemia. And we did this, made this decision as an institution because these were well vetted CDC tier one conditions. They're considered actionable. All the reasons we've sort of talked about today, but that was something that leadership felt comfortable with moving forward with the CDC tier one conditions as our package. And then considerations that we had early on are a lot of what we have been talking about today. So ensuring that participants are understanding their findings, both positive and negative results. We're making, really trying to make sure that we're be able to connect individuals to clinical care and being able to then ultimately track those downstream clinical outcomes. So future expansion that we have in mind but have not done are actually thinking about pediatric populations and potentially expanding to provide this type of free screening service for our pediatric population, prioritizing more rare conditions. We're also thinking about and actively expanding in the pharmacogenomics space. And so returning results for genes that are associated with our priorities as far as pharmacogenomics goes, as well as returning secondary findings in particular APOL1, PKD1, PKD2. And part of the reasons for focusing on those particular genes is because of our state's sociodemographic and sort of health concerns with kidney disease morbidity as well as disparities in that space. And then I think other considerations as we're thinking about expanding are that we have this under a research protocol, which is great in a lot of ways that allows for us to re-contact individuals who've participated, but we ultimately would need to re-consent them to provide them with additional findings. And then we're also thinking about just what the right combination of considerations are for expanding and looking at ClinGen guidance to help us think through that. Thanks, Caitlin. Next, we can move to Ned.  Hi. So I have kind of three things to talk about. EBM, old, EBM new, research and equity. So there are evidence-based methods, evidence-based medicine methods that have resulted in recommendations for genetic screening on a population-based level. So recognize that the USPSTF does that. Their recommendations are population-based recommendations to be offered to everyone in the groups that are identified for benefit versus harms. And they have one recommendation around genetic screening, which I participated in years ago and they're re-looking at it even as we talk today, which was BRCA one and two. So I think hereditary breast and ovarian cancer screening has a footprint with evidence-based behind it. And there was a method that got the task force there. Then the EGAP working group did its work and from those, I was there as well, we came up with the other tier one tests, Lynch syndrome and familial hypercholesterolemia. Then I retired from genetics for 12 years. 12 years later, we have three conditions. The tier one, I'm just a little amazed. And that's so that tells me it's time to rethink about the methodology to kind of look at what Mike wants to do and look at what Les was talking about and saying, we're not talking about lowering the bar, we're talking about criteria that would help us get the evidence necessary to add additional tier one tests when we start to think about population-based screening. So what I heard from Mike and I heard from Les, we're kind of where the uncertainties are and can we identify those uncertainties and get a set of tests that are almost ready for prime time, that is we can specifically identify what gaps would have to be filled in to get us closer to meeting those bars of either the EGAP working group or the USPSDF and therefore provide additional tests that have that kind of, sorry, we used to say that at the clinical guide, that kind of bulletproof recommendation that you know if you do this, people are going to have better health. So those are those issues. How does that relate to research? I'm glad North Carolina is doing that. I would make a slightly different recommendation. South Carolina, oh, sorry. Remember I'm from Colorado and I know the Mississippi is between us and that's about it. I think the other approach is to think about funding integrated healthcare systems to pilot these. So we do this, well, we don't pay for it, but we do it in newborn screening in order to be considered to be added to the routine uniform screening panel. There has to be a pilot study that's identified at least one case. And CDC may be paced for that every now and then and NIH may be paced for it every now and then. But thinking about taking on a guy singer or a Henry Ford or a Kaiser Permanente and I don't mean to leave anybody out. Inner, inner mountain, there's three or four or others. Well, that would be a great setting for saying we're going to have a pilot of a population-based test for genetics. So Terry and Eric's gone. But if you're thinking about things that you could actually screen, I'm sorry, fund that might end in moving the bar a little forward towards new conditions being on the tier one. That's just a strategy I think about. Then finally, we were reminded so well by the articulate and amazing speakers in this last session that there is this incredible opportunity to make recommendations that increase inequities through genetic testing. And so all of the work we do needs to have the component of community engagement and the lens of how can we do this in a way that does not increase health inequities. So those are my comments. Thanks very much. Yes, the hunter is up next. Yeah, so I've been coming at population screening from the context of clinging. So I lead the actionability work where we're doing evidence-based assessments of actionability generating reports that summarize actionability of gene-conditioned pairs. We score four factors associated with actionability, how severe the outcome is, how effective the intervention is, the nature of the intervention to the patient and how likely the outcome is to occur, AKA penetrance. And all of the scores, these semi quantitative scores and our final actionability assertion are available on the website. We've developed this framework and the context and secondary findings, which is why most of the domains are based on the individual. But we're recently shifting to more population-wide and we've been adapting this framework for the context of polygenic risk scores and now shifting as well to population screening and thinking about as part of that effort what factors need to be added to this framework in order to account for a population-wide impact. Severity, at that point, it's not just severity in the individual, but how impactful is this conditioned at the population level and those kinds of things. And so, yeah, we're beginning to think of what factors we need to include in that framework, a lot of which we've learned through the polygenic risk score effort. But for community engagement, I will say that we have to have this effort to be quite, sorry, that's a little distracting. Somebody logged in. Good afternoon, everybody. And forgive me, I'm just recovering from a cold. So my voice is a little smokers cough. I'm from the Office of Disease Prevention at the National Institutes of Health. And unlike many of you, I don't speak genomics as well as you do, but I do speak prevention. And I was actually quite inspired by Les's talk this morning about that genomics is not an exceptionalism. There's no exceptionalism there with regard to prevention. The Bayesian Theorem still apply to all of the work that you do. At ODP, what we're interested in is sort of assessing and facilitating and stimulating prevention research across the 27 institutes and centers at NIH. We do that partially through developing and coordinating and then implementing some of those prevention programs. We work closely with the US Preventive Services Task Force and the Community Preventive Services Task Force and healthy people to really identify key evidence gaps in prevention. And I think there's some other opportunities here to work with all of you to identify some of the key evidence gaps that might exist as you think about population screening. For us, I know as we look across our prevention portfolio across NIH, of all the prevention research, about a third of it actually looks at leading risk factors. About a ninth of that prevention research actually tests a randomized intervention for that leading risk factor. And about a 20th of that actually tests that randomized intervention for a leading risk factor in a health disparate population. So really, very small representation of health disparate populations, even in traditional, or should I say, just run of the mill prevention research. I think that provides us some opportunities. We've added a whole area of our strategic work that involves advancing research in health disparate populations. And really, I think the important thing that we can offer is ways to understand and address research gaps. And I heard a lot of research gaps today. And I think I can offer a couple of reflections on that a little bit later. But just to let you know, one other sort of interest that I have is implementation of this work. And we heard a lot today about how it's gonna fall on primary care clinicians. We've got a little bit of experience with that. I'm reflecting back, not quite 12 years ago, but 2000, 2005, the American Academy of Family Physicians held its educational focus on genomics to try to help family physicians embrace the era of genomics. I think here we are 18 years later and I'm not quite sure AAFP has, our family physicians in general have embraced it at quite the level that they might have hoped. So I think we do have support there, but I think there's lots of opportunities too. So with that, I'll stop and look forward to the discussion. Okay, so before we open it up to general questions, I wonder if each of you could just take a minute or so and comment about what aspects of community engagement you've used in starting up your projects and how you do that in an ongoing way to make sure that your work is continuing to be relevant to the communities that you're doing your research in. I shouldn't say partnership with. So for our work at MUSC, we developed our own community advisory board. So there were existing patient family advisory councils that the hospital has in place and we felt strongly that we needed to have our own community advisory board specifically for this project. When trying to identify people that were going to be part of the community advisory board, we were very intentional about representation from across the state of South Carolina. So MUSC is in Charleston and mostly as a footprint there but our screening program is designed for the entire state. So we really wanted representation from across the state. We also wanted to find folks that would have differing perspectives about genomics and screening and not all of our community advisory board members agree or that they have varying opinions and that was intentional. And so that's been important for our work as well as the engagement of our clinicians early on and some of the co-creation and decision making we had about how to just set up the program. I guess I'll talk from my role with the state health department and say that as we put together programs now that are designed to improve delivery of care in historically marginalized communities, we make sure first of all that our advisory committee includes a diverse group of individuals with lived experiences for those that are members of the White that still makes up our state. So I think that outreach and inclusion is important. And then the next step is as you design a program even with that representation to take it into communities to a broader discussion with more people who have lived experiences differed from the ones on your communities so that you make sure you have enough input to tailor the delivery of a program that's gonna be most successful for a given community. Well, and I touched on this a little bit already and getting community engagement on kind of validation of our metric. And I think that will be particularly important as we move forward towards population screening. But for our individual reports that's been a bit more ad hoc on when a little red flag pops up maybe and this is an important report to get feedback on but I think perhaps going forward we need to be a bit more systematic about that. And I think that would be a really important addition to our efforts. Thanks, and I'm not speaking for ODP because I'm not sure we engaged public quite the same way but coming off a couple of experiences that I've had recently with us in some research areas. I know Colorado actually has University of Colorado because you're thinking of the boot camp. Yep, really good results. There's a team at University of Colorado that engages patients in development of educational materials. And they found better uptake of whether ever the intervention was whether it was blood pressure screening, cholesterol, whatever it was getting patients directly engaged in developing the educational materials are used is an effective way to help engage patients and then help improve the educational process. The second one is from the organizational standpoint and I'm not quite sure what ACMG's approach is but there are other groups that actually at professional meetings bring together patient clinician or patient researcher pairs, diads to help make sure that the patient perspective is represented in every aspect of the work.  And that also seems to be a very effective way to keep people in line, not in line but engaged in every, along every step of the way. Thank you very much. So we heard a really nice description from Minky about how all of us has reached out and engaged communities. I'd be very interested to hear from Vanessa and Crystal how that might look if we thought about engaging your communities. I was really struck by the fact, Crystal, that you said that only five ACMG medical action, medical actionable genes were even relevant. How do we do a better job? There's two questions there, first of all. I wanna highlight something really important and I'm actually on the Indigenous Research Working Group for the all of us research team. That's where I stepped out about an hour ago and I had this exact conversation that there is a distinct difference between engagement versus research capacity and training. Now, they can be part of the same spectrum but they are two polar opposite sides of the same of that spectrum and you cannot use one versus the other. So when we're going into talking with community members and tribal leaders, we cannot say, oh, we're training students, checkbox, engagement. That is a completely separate activity than what we're calling for, which is the more difficult activities related to conversations that are difficult, building trust, good quality of research and not just surveys and focus groups at democratic deliberations. And Vanessa has given us a couple of great examples, several great examples, the Seeger that we have trialed in our communities as well. So I just wanna have those two distinctions. The other part about the medically actionable genes yes, so of those, those are mostly centralized on colorectal cancer, gene variants are especially related and there's a recent report that stated that that is now the number one cancer in our communities. And so there's obviously a point of intervention here but there's also a lot of concerns. So in terms of when we refer an IHS patient out, what tests are we giving them? Is it relevant? How are those tests results being returned back to the community? Are they validated in a research setting versus a clear certified setting? I know it's not usually a concern for most institutions but it is a concern for institutions that are smaller and that our community held. So there's a thing of scale here but then also what does that mean for meaning for one community versus another, right? And there's just a lot of concerns about Gina, a lot of concerns about harmonization of data. The other definition that I wanted to just state is that we're talking about population screening in like research domains and then public health domains but those two spheres are separate in terms of their ethics and in terms of their regulations and in terms of how those are sanctioned. So, and especially at the tribal IRB, IRB realm, I mean, that there are rules and like past practices for engaging with tribal IRBs when they're recognized by the NIH because under the common rule and single IRB mandate they're not always recognized. There's one like 33 of them. IRBs are not recognized, other forms of tribal research governance are not recognized but we're talking about public health now. That's an entirely different animal. We're talking about different federal agencies. We're talking about different legal proceedings should something go wrong. And then how the IHS and pathways of care especially even for, you know we're just talking about rural tribal settings there's also separate considerations for urban individuals of which according to the last census now 80% of indigenous peoples of AIAN communities now reside in urban areas. There's a lot of issues here. So sorry, I just want to unpack that. Crystal, I'm gonna put you on the spot. Crystal's a great follow on X by the way formerly known as Twitter but you said you had a post earlier today not from this meeting but from another meeting about indigenous leadership. And I think that was, you didn't quite go there. I thought you might but I'd be really interested in your thoughts about leadership of research as opposed to partnership and research. I'm gonna point it back to you. Can you clarify a little bit more? So you basically said that for a particular proposal that it was discussing indigenous leadership within the proposal and the proposals reflected engagement but not leadership. And so that seems a different level of engagement and you were starting to get there but you didn't quite get there. And I think you have some thoughts that would be important as we think about the research agenda. I have a lot of thoughts. Okay, well, right now we're presuming research from a very, our common pathway of these are research proposals initiated by individuals with PhDs working in academic centers and then they're reaching out to communities. And even the work partnership is either thrown around really loosely and it's not really respected in the same way in terms of resource development. The post that I mentioned earlier or wrote earlier was something to the effect of we can no longer call things indigenous led when institutions have 57% indirects or sorry Hopkins near 100% indirects and those go to PIs when most of the grassroots activities are actually from community members themselves. And they're the ones that are generating the data, the protocols going through all the approval processes and they get a tiny fraction of that. And this is something that we need to consider for the NIH models. Now, NIH has been really great in the last two years. You've started to really consider that you do not need to have a terminal PhD status, which is important when you consider that the Seattle Indian Health Board Director who just stepped down recently is a master's level investigator. So this meant that for individuals like her, myself, Joseph Earshada, who's the executive director of the Native Biodata Consortium who were at me recently master's level, we couldn't even be PIs of our own research. So, and also what that meant is in terms of contracting with universities, we were always having to have our lawyers stuff in which we have lawyers, thank God, but that took a long time. And that's the first time and not the last time I'll say thank God for lawyers. But we have a single IRD mandate which has, you know, and common rule, trans language about respecting tribal sovereignty. That only means for tribes that have an IRV that has an FWA, which is like a small, small fraction of the 574 recognized tribes. Those then, you know, you could still work with a large-name institutions that just happen to have institutional standards of refusing to see review to external IRVs to include tribal IRVs. So like how do we win? Even if we wanted to partner with universities, we are constantly at the wrong end of that decision-making equity and power and authority. And there's something to be said of the distinctions between respecting versus operationalizing indigenous data sovereignty. Okay, thank you. I wanted to make sure, Vanessa, you had a chance. Just to ask about the question of what do we need to do a better job? How do we need to think about different ways of thinking? We've heard some great ideas already about helping address the inequity that we have inherently with indigenous people. Well, first off, I don't believe that the inequity that the American government has with indigenous people is alone to indigenous people. I think we were talking about historic inequities with many, many groups of people. And so there are some concepts that ought to be considered, and which is exactly why I put together that I highlighted that ladder of collaboration of engagement. At the very least, we should start attempting to climb such a ladder, recognize that that ladder exists. And then when we move forward on our work, be it as an individual or as a system, that we move up it. I think one of the things that I was hearing from Dr. Lee was, in the example of an individual with disability that is being highlighted on something as simple as a flyer, that indeed the community will recognize if a person is an able-bodied person, that is a model that's sitting in a wheelchair versus an individual that has had that full-lived experience as a, for instance, which is a tokenism, right? Straight up. And also the material. Recording in progress. To the other underrepresented minoritized groups and also the ones not fairly recognized. Can I just confirm? We gotta note that the folks online are not getting sound. Is that still correct? Sorry, Gertrude, either of you, I think what you just addressed relates to that and you could briefly summarize for us. So, oh, you want me to rephrase the question? Okay, it's really interesting. We talk about biology in terms of distinctiveness across populations, like between and within, but then we don't even talk about relationality in terms of how, like in terms of respect of our kids. And I think this is really important because, you know, we have federally recognized tribal nations, but we don't acknowledge their cousins, their neighbors, their, and which have distinct and different types of historic harms, but we try to legislate across genomes and genomes don't care. So this is where I think in terms of the principles that Vanessa brought up in terms of thinking, in terms of respect and humility, but also extending that to how we are thinking and globally and ethically. So I would just like to add that one of the things that I hold to with the work that I've been able to do is the role of sovereignty and how our tribal nations are able to force the issue of the research codes, of the meeting behind those research codes and other marginalized groups don't have that opportunity. And so it doesn't mean that they don't have similar values, that they don't have similar desires to be included in research, to drive the research, to have power sharing, to be all of, to do so many things, all of those Rs that I rattled off earlier. Tribal nations instead have that political authority to engage in tribal consultation, right? One of the things that I had the blessing of being part of was developing the tribal consultation policy for the NIH. That's not something that other marginalized groups get to have because there is no authority to do that. It doesn't mean that there's not the moral imperative for all of us to follow through on such things. And so that's where my mind goes is that we have a responsibility to our various marginalized communities to inquire in the same manner, be it required or not, when we are engaging and seek to engage in whatever type of work. Even in the absence of a legal, no, that's great, thank you. So I think we'd like to open it up for a broader discussion. So go ahead. So I think one of the kind of underpinnings of dissemination and implementation research is that we wish to make the fruits of research as broadly disseminated as possible. And that's a little bit different than thinking along the lines of basic science research, right? But it's engaging with communities and it's sort of finding ways to make something work, right? And so do you see a difference in the way that engagement should happen, say let's say in my state of North Carolina, we do have Native American populations? And if we wanted to say let's have a statewide effort to do this genomic screening, because we think it's useful. So how do we go about doing that with the patchwork of constituencies that we would want to engage with? That's the best way to do that. Yeah, if I could go ahead and start the response. One of the slides that I glossed over quite quickly as I frantically pushed the forward button was a slide that I had to deal with a newborn screening. And in Alaska, one of the secondary additions to newborn screening was a, in what was it, 2003, 2006, somewhere around there was a CPT-1A Arctic variant. And the addition of that variant as well as materials that were developed for the public, those that had screen positive, those were developed between the state of Alaska as well as the Alaska Tribal Health System. And they were directed to providers of care for Alaska Native people that were within the Alaska Tribal Health System. Alaska Tribal Health System is, again, managed by Alaska Native people and is caring for about 40% of the state population when you look at managed care and clinical care there. Kind of sticky thing there is that the materials are being mailed out when a person has a positive. It's a CD, like a DVD, hasn't changed in all these years. And it's not something that was, those materials as patient education materials hadn't been tested with everyday people that were receiving them, but instead were to inform the providers of care for those individuals. It was until a couple of years ago that the health system had put together a research project that was NEH funded to do some community-based inquiry, both with the current providers as well as with the patient population, particularly those that had received such things in the mail and had received information that their child had this particular variant in a double whammy sort of way, right? And these focus groups had indicated that indeed the people had, and the providers still had a lot of questions and the materials weren't quite as helpful as we had all in public health had hoped they would be. I see that little story here because there are procedures that we can all think of now after the fact of what engaging with individuals, with parents that are receiving this information ahead of time and developing it. I mean, goodness, even the title of the materials is something in regards to the metabolism, personally, in the way of the materials. But anyway, I use that as a little case study in the way of what could be done different in the four community engagement on something that is highly specific to an indigenous population and Inuit population, not just here in the United States, but other Inuit populations where that is there. So we've got four hands up and there's one hand on the line. Yeah, sell that one. So Mark, Pat, George and Bruce, and if anybody else wants to see a line, make sure you get your card up, but Mark, you're up. Thank you. This is to the panel, although the handicap here is that my question's about somebody that's not represented at the meeting, and that is we're talking about engagement and there's one entity that has been specifically mandated to do patient-centered outcomes research, that is PCORI. They also have a mandate related to rare disease, which they've had a real challenge meeting their expenditure in the rare disease space. And it seems like there's some opportunities there, particularly since it's for engagement research. And so for any of you on the panel or Alana, who I know has done a lot of work with PCORI that can comment on whether there could be some opportunities for engagement with PCORI in this space, I'd be interested to hear your thoughts. Yes. Yeah, no, I see. Hi, yeah. Could you elaborate? Yeah. All right, I cough when I laugh. PCORI is great. I mean, and they have now years of experience doing this kind of engagement work. I think they're a great target for us to work with. We've worked with them on a couple of different projects and they bring, I think, a unique perspective to it. They build a patient advisory group with each project as it comes along. I think there's some real models they might be able to tap into and use. So I think that's a terrific idea and you're right. I wish I thought of that earlier too, to mention it. Yeah, I just wanted to add to that too. I think we have seen some, I mean, the fact that we're talking about this here and as a grant reviewer, both for PCORI and for NIH over the years, I've seen some of, a lot of the PCORI thinking move its way into the NIH grant review process. We're like, hey, they didn't include, there is no engagement at all in here in an NIH grant. So I think that, yes, there's definitely we can do more, but I think there is a little bit, and I think just what we've seen here and that we're talking about it is the right steps and I'll let the questions continue. So Pat, I think you're next. Okay, yes, my question's for Ninky. I had a question about the all of us research program. So I thought you did a great job of giving us a really good overview of the diverse engagement efforts that all of us pursue. And then I was personally just interested in the last slide that you showed that said for both hereditary cancer and pharmacogenomic variants, then only about half of the participants view the results. So my question is, was that in line with your expectations and do you have any under, have you explored the reasons that only half have chosen to view the results? No, it wasn't, especially since it was a long time coming. So a lot of our participants had voice that they were waiting for it and it was taking too long. So we definitely expected a higher percentage of participants to view their results, especially with all the support that came with it in terms of genetic problems and the reports themselves also look very nice. They're visually appealing. We had a lot of community partners review the language on them. It comes with information on the gene and then just ways to understand the information better. The team is looking into how to increase that. But yeah, right now that's where it seems to be. And for some of the participants, if we anticipate a pathogenic or likely pathogenic, sometimes we do, those are the cases where we do require them to set up a meeting with a counselor in order to receive their results. So that might be an additional threshold in viewing their results. But overall, once they do have a consultation with the counselor, the satisfaction score is very high. It's around 60%. So we don't think it's something to do with the experience itself, but something else must be in action there. And we are having to look into it. It hasn't been less than a year. So maybe it'll just take more time. I'll just make a quick comment that that percentage is actually higher than people see with Biobank return of results. We were just discussing, like for both Penn and Geisinger, that's 50% is actually much higher than we see with trying to return Biobank results and whether people respond to that. We do acknowledge it is high. Bruce. Yes, thank you. This is actually a somewhat technical question, I guess, for Crystal and that is you mentioned the use of blockchain and the very, very, very little bit I understand about blockchain has always made me wonder if it's a way for people to control their genomic information so that they can share it when they want and with whom they want. And I'm wondering, first of all, is that accurate? Is that how you're using it? And then if so, is it scalable and could it help to get us out of the quandary where we have genomic data frequently kind of locked in place, like within an academic medical center, for example, where it isn't easy to port it to other settings? Yeah, so blockchain is more at the infrastructure level and there's smarter people that work within data systems around the table. But basically, first I would say that dynamic consenting is something that provides an individual level of sort of decision as to whom somebody might consent to have their data shared with types of data, they are willing to share rules in terms of data dissolution, are there cultural rules that they would prefer to enact that they wanna remove their data or draw down their data and you can employ dynamic consenting models actually at the earlier design. So I think that UK BioBank employs this type of approach. This is also an approach that we develop at the Native BioData Consortium as part of our infrastructure so that we can license it to other travel nations worldwide for free. The thing about blockchaining is, and it's very important here, is you can sort of define different types of user access, read a rules, but those type of levers are usually done by whomever is designing it, right? But, and then again, I'm speaking more from a community centered approach, you can actually have community members act as the authority nodes for deciding who gets access to certain types of data, I know what conditions, what study types, et cetera within partnership with tribal nations. So sometimes this is like positioned as an individual versus group consent, speaking to an author of a great paper here that interviewed tribal leaders and they say, and one of the leaders and elders or was older and older, both, okay. That stated, it's not individual or group, it's both. And it's actually up to tribal nations to decide how they're gonna make those decisions not outsiders. But in terms of how tribal nations can govern, you can have both an individual level of control and as well as at tribal nation control. Blockchaining is one way, another way is federated approaches where you can actually like read and maybe to execute without having like full access to all of the data within independent nodes that are governed and housed by communities. And this is something that many other systems are employing as well. I can't tell, is it Alana or Kate that has their card up? I made my comment earlier that okay, sorry. Return of results was just like 50% pretty high. Chris? Yeah, I had a follow up question for Mickey Young. So I think you mentioned from your presentation that genetic counseling was offered to all of us participants even if they did not have the results returned. So wondering what kind of uptake you got on that and what were their questions if you know that? Yes, I'm thinking the only information I took away from that in preparation for this talk was that the CSAT scores were high, the satisfaction scores were about 60%. The specific questions I would have to look into that in terms of what they're asking. Yeah, I don't have the feedback off-hand right now, sorry. Is that a second question or no? Great, so I wanted to ask Vanessa, you mentioned in the list of things early on in your talk that should be considered, you mentioned thinking proximally about health and I didn't quite understand what that was. So could you explain what you mean by that? Honestly, I don't know what I meant by that right now. I'm being honest, I'm not reflecting, I'm not recalling where I had that particular point. So I apologize. So did it, I did it, it made, you know, because to me, I might interpret it as the health needs of the community that are primary to the community, but I could put any, you know, interpretation, but maybe you know, or you have some thoughts on that? I have that sentence on a couple of my slides. No, often and unfortunately, the Human Genome Diversity Project, the thousand genomes, genographic projects are great, great unfortunate examples of this in which researchers would enter into communities, promising things that therapeutics will discover the thing that is causing, for instance, the NIH NADDK made a similar promise to the often peoples like in 1965, we're going to treat diabetes and then we not telling disclosing that maybe it'll take a long time for these interventions to actually be translated back to their health and meanwhile, community members are wondering what happened, where'd you go with our data? So that's what I meant by thinking more proximally in terms of the promises, messages, engagement, et cetera. George? Thanks, this has really been a terrific session and as I reflected on the values and the principles that you mentioned, I was reminded of a major principle that we learned as part of the NIH-wide Community Engagement Alliance, particularly during the COVID era, was the issue of trust and we think we know when trust is enriched or when trust is eroded, but from a research perspective and in your experiences, do you have to worry less? I mean, we definitely want to be trustworthy and we want to do everything possible to get there but how do you measure that and what have you found useful from a research perspective in answering questions about trust? If I might go first. Within, let's say, the organization that I had worked for, South Central Foundation, there was a requirement, a research requirement that researchers were to put forward their concept and a concept proposal of one pager. So they could submit, let's say, their specific aims page as well as a cover page, saying this is what my intention is and why we think that we'd like to partner with this agency and cure the outcomes of interest and the procedures that might be used. Again, very broadly, I would say presence or absence, even of the request is an indicator of trust. We had many researchers that once they discovered that that was a requirement for approval, they were out, not interested in participating, not interested in partnering. So just to get presence or absence to engage in a process that's community-led would be a very low bar, one might think, but it's an indicator. There's an emerging field of equity metrics. We keep talking about equity, how do we measure it? That's actually a very interesting question. Some of this is rooted in the health economics. I have a concern that health economics is really short-term and also may substantiate systems of that are incongruent with actual community health, but that is a starting point. In terms of tracking whether or not these researchers are delivering on the promises that we have made, one way to do this is tracking research dollars into communities. So for instance, with Indigigata, which is an indigenous data science education workshop that I started, we actually shifted all of our research compassion training outside of universities into tribal lands. So we're funneling outside dollars from NSF and also philanthropic orgs. And other than paying for or not paying for airlines, everything else goes to tribal nations at a rate of 67%. That is something that we is like above the standard of other types of training opportunities that have federal sponsorship in some form. We can do this with similar other types of grant mechanisms when we start tracking where are those indirects going, where are those directs going, who are we actually training, et cetera. And again, also aligning that education is not the same as a direct, thank you. Perhaps could I have a quick follow up from Vanessa, please, because something she said actually just triggered a reflection, very often when we put out a notice of funding opportunity, we don't always do it, but sometimes we ask for a letter of interest. So did what you say apply also to letters of interest when before we even ask you to come in for the application? Yeah, so let me just step back. The process that I was engaged in with the tribal leadership of South Central Foundation, which gets their tribal authority from the Cook Inlet region, began with a concept proposal phase. And then if approved, the researcher and their team would, and this was also true of those of us that were embedded within this health system. So it wasn't, they do that and we do something else. No, every single researcher, student as well, would complete the concept proposal phase that goes all the way to the board of directors. If approved, then the person would move forward in partnership to develop a full proposal. And then after that had been written and approved, that full proposal would go back through a full process. Now once everything was in hand in the way of this is what the engagement would look like, this is whatever recruitment flyers and materials would look like, these are our research team members on Nazium, on and on, these are the actual majors. Then at that point, then that's when there would be a suggestion of moving forward with funding application. So again, this is a really detailed manner in which engagement would occur where there would be a hand and this required is how that group had chosen to have a hand in the development of the research question, the processes, the et cetera, et cetera and embedding that information within the healthcare system. All right, well, thank you everybody. This has been a terrific session. Appreciate both the, all of the panelists and the presenters as well as the questions. And I feel like I need to summarize but at great peril here because there's so much to be talking about. But first and foremost, I think one of the most important things I heard is the importance of building trust with the communities that you work with. Second, I think I heard, we use the word engagement a lot but that engagement's not enough. We really need building on trust but we need to really think about true co-creation from the beginning of the project. I also heard that I think we need to recognize the need to have a different view of what community leadership means because the community may not always have the same educational opportunities and the same socioeconomic opportunities that the traditional academic community does. So I think that's something that was important. I also heard the importance of recognizing marginalized communities that are not only marginalized but don't even have a voice at the table which some are fortunate enough to have. And so I think we've heard the need in the context today that we really do have a lot of work to do in terms of thinking about making sure our databases include all participants, all humans and make sure that we have a commitment to doing that. And I think I will stop there but I'm sure George, do you have anything else you'd like to add to that? Not a whole lot Rex, I think you did a terrific job only to highlight the importance of the principles and the values and the terms that Vanessa mentioned I think trust is in there. The second thing was also many of the challenges that traditionally in our NIH funding we haven't paid attention to in working with community partners or what can I think it's a little different for sovereign entities but really rethink how we fund and how we support particularly small community-based organizations and small partners. I think the final thing I wrote was paying attention to the power imbalances. We may not think of them that way but we have our structures and we go through with our structures but they may really reflect serious power imbalances that go against meaningful community engagement but this is really been a true of rich dialogues so thank you all. Thank you all and I will just say I think all of us that work with participants in general regardless of where they come from we should put a copy of that ladder in front of our desks. Great, so that thank you very much to everyone. Let's give them a round of applause and thank you very much for sticking it out to the bitter end here of today. We have tomorrow and we'll start at nine o'clock with the session 8.30 with breakfast. No, I'm so sorry. So this is why I have my friends here. Yes, so 8 for breakfast, 8.30 for the session. Thank you very much, goodbye.