All right. Well welcome back everyone. This is the last hour of extension conference. It's just about a wrap. So thank you all for staying here and staying engaged. So our final panel, we are so lucky to have three individuals back here. You know all these folks. Kelly Armbruster, information specialist with Egg Communication. Director Hibbert and Director Larry are back. The topic for today's panel is looking into the future of extension. So we'll start off our panel with some comments from each panelist and then we'll turn it back to you. And I would ask you to be ready with some questions, some thought provoking issues, some things perhaps that you've picked up in the last three days in regards to the future of our extension. So with that, Kelly, or do we flip a coin? Well, good morning, everyone. As Jody said, Greg Lardi, I prefer he and his. And my ancestors came to America from Germany and Luxembourg. And so they came in the 1850s just as a little bit of background. Let's think about extension and let's think about the future. When I think about the future of extension, it's right here in this audience. It's right here in this room. You are the future of this organization. The things that you put up on the wall in terms of the new programming ideas and what you want to do in terms of innovation around those are part of what's the future of extension. The things that we talked about at this conference in terms of relationships and being the unbiased source of scientific information, relevant information is the future of extension. The fact that you're here and engaging in this and participating in this is the future of extension. So when I look into the lens in terms of what's the vision for NDSU extension, what do I see on the horizon? A couple of thoughts. We need to continue to work very hard at engaging and building those relationships, whether you're a specialist in building relationships with stakeholder and commodity groups in the state, whether you're in a county office and engaging your local stakeholders with your programming, but building those relationships, getting out of the office as Chuck talked about the other day and engaging with your audiences, building those interpersonal relationships and then using that digital media to build on that and taking that out and communicating with it, that's the future of extension. So to me, we have to be ready to innovate, we have to be ready to change because the world around us is changing. The environment is around us is changing. At dinner last evening, we were talking about all the changes that have happened in the Bakken region in North Dakota and so you think about all the opportunities that brings for us as an organization. What does that mean? We're able to reach out to new audiences that have never heard of NDSU extension, probably never heard of extension period coming to us from other states. What are their needs? What are the things that we want to do with them in terms of how do we find out what they would most need from our agency? We've got to talk to them, right? We've got to engage in a dialogue with them. So that to me is a future of extension and building those relationships and then continuing to be that unbiased source of scientific information and coming alongside people as Chuck indicated yesterday as co-learners. So what does our audience already know from their experiential learning, from their indigenous knowledge and what do we know based on the research data that's going to help us solve some of these complex problems that are facing us as a state? Great. Great lead in, Greg. Thank you. First of all, it's such an honor and privilege to be invited here to be part of this conversation. So thank you all of you. I just really, really appreciate it. I wanted to start off by talking about value and how do we provide value to learners, to stakeholders, to elected officials? And what does that value look like? And I think the thing, as I think about value, the thing that's really happening is that perception of value is changing and evolving, right? I mean, there were, you know, back in the day when the questions were simple and straightforward, it was pretty easy to deliver on that value proposition that we want to be able to do. Today it's getting much more complex. Again, at dinner last night, we were talking about that really complex system that's evolving in Williston in that area and all the moving parts and the moving parts that we can have some part of and be involved in. And so to me part of that and a big part of that is how do we make sure that the work we're doing is relevant to the stakeholders that care so much about who we are and what we do? And how do we figure out what's really relevant? And so every state does this a little bit differently. What we did about four and a half years ago, we did a statewide survey. We invited certainly our closest friends and stakeholders, but we also invited a bunch of people that weren't necessarily in that category. We did the survey in Spanish as well to try and invite more participation there. We did through that survey process and about 2,000 Nebraskans responded, so it was a good number. But we did have a small group that went through all the information and identified 18 issues that Nebraskans told us were important. Now see, there's a bit of a shift there, right? Because one of the things that people in Extension talk a lot about is we're spread across our states, we're network, we know what the issues are, right? Well, I think in general we know what the issues are, but this survey demonstrated that there were some issues out there that maybe we were not as familiar with or we were not thinking about as much in terms of emerging issues. So we formed 18 issue teams. We used to work in teams that were focused around subject matter, so we had a beef team and a crops team and a family team and a youth team. So this changed the way we thought about collaboration, right? As how we thought about how we worked together to take advantage of interdisciplinary knowledge and expertise to address now these issues. So for example, how do we create an environment that's beneficial to pollinators is one that was really important. How do we create a food system that people appreciate and value in terms of the kind of food they want to be able to access? So you can go to our website and look at those if you're interested. But the other thing that was interesting about that is in the last four and a half years, everyone that I've talked to and it doesn't matter if it's a local stakeholder or an elected official or a university leader or whomever, they like this idea of demand driven extension based on input from stakeholders. So that's one thing I wanted to mention to you around value is how do we make sure we're relevant and we have to be relevant on things we can do well, right? This is not a suggestion that we go, you know, go off the tracks, but how do we be relevant on things that we can do well? The second thing I wanted to mention and I touched on this a little bit yesterday, but we've put a big focus on learner engagement and you remember the stair steps and the idea that we plug in based on the issue or the opportunity. It may be something that we do a lot of media work and we get a lot of information out. It may be something towards the pilot and the mastery where we're doing a deeper dive, a deeper engagement with a smaller group. We need to better understand how people learn, how people engage, how people derive value from the work that we do and so thinking about that and learning more about that, I think is a really important opportunity for us because certainly we can use digital media in really productive ways. We need to think about how we do that well. There are other instances where, you know, it's kind of one-on-one. We've got to be there and we've got to connect and we've got to provide value one-on-one. I think what that challenges us to do then is to really work on all four levels of those stair steps I shared with you yesterday. So think about your learners, think about how they learn. One of the things that was always, I learned a couple of years ago, is a misconception. We need to try and teach to the learning style of the individual, right? Have you heard that? Have you thought about that? Some people are, you know, they're very hands-on learners, others are more auditory, others are more visual. The fallacy in my thinking around that was that, you know, we've got to plan programs that target people who have different learning styles. Well, the reality is, in any group of people, the whole diversity of learning styles is there, right? So how do I vary a specific program to engage and address people that have different learning styles? So those are some of the things that we think about, again, in terms of value, how do you add value being relevant, and then thinking about learner engagement. Thank you, Chuck. So at my very first extension conference four years ago, there was a similar panel on the last day, and I think it was about work-life balance. I think Greg said on it, and Crystal Shahneman started off by saying she didn't know how she got up on that stage, and I'm feeling a little bit of the same way today. I, Crystal, that resonated with me. But I'm going to take it a step even. These guys kind of talked on a macro level. I'm going to talk a little bit on a micro level about once we've established some programs that are innovative and we've identified those needs, how do we then market those programs to our people? If you Google trends in marketing, the number one thing that pops up is purpose-driven marketing, and it so parallels with what we're doing in extension, creating programs with purpose, right? And so as I, you know, kind of survey extension and look at some of the marketing that we're doing, so many times we get stuck in a rut. And last year, let's say last year, you put on a babysitting workshop, for example. It's easy to put the title of that workshop as babysitting workshop and the date at the fairgrounds and go on with it. But what if instead we thought about what's the true purpose of this workshop? Well, and how does, what is the intrinsic motivation that might get someone to come to that program? Well, for a 15-year-old, it might be that they want to earn money on the side that they can buy the latest video game or whatever it is. That, you know, so how can we be more purposeful and think about the true purpose of the programs that we do and do that, you know, and mark it to that? My second point is about influencers. We see on Instagram, Facebook, whatever social media, medium that you are engaged with, we see influencers are taking over. So-and-so likes this product, right? Beauty gurus for the women in the room, how many of us have watched a maybe a makeup tutorial on YouTube from an influencer? Or maybe if you're into crafts, there are people on YouTube that, you know, they have millions of views to make necklaces or quilts or whatever it is. These people are influencers in the industry. And we talk a lot about telling our story, right? We tell the story. But really the future is other people telling our story. And that's what we, you know, we talk about external stakeholders all the time. And, you know, I think about, well, how do we do that? As an introvert, I try to put myself in a county person's shoes. And I think, man, if I was moving to a county and I was asked to go out and find those stakeholders and encourage them to help extension tell their story, I think, how would I do that? I don't, I don't even know anybody here. What, you know, what does that look like for me as a new person? I know we've got a lot of new employees. And so I really kind of thought about that. And I think the key to that is making a connection with just one person, right? Do you have a 4-H parent volunteer that has been part of the program for 30 years? If you can make that one connection and ask them to help you connect with other people, that is so powerful.  Just go to the sale barn. Just walk in, you know, the grocery store and talk to someone that, you know, for a new employee or someone who's not comfortable with that, that is not reality. Getting a list of talking points to hand to stakeholders and saying, would you talk about extension, you know, to benefit us? That's hard to do. How do we really do that? Well, we do that by making connections and by asking people to help us make connections, to introduce us to the movers and shakers in the county, the bank president, the fair board member, whoever that is. So I think that's truly important that influencers, people telling our story for us is truly the future. I'm going to ask for these guys to, we're going to do a little, I want to share a little bit about the power of influence and we're going to have a little fun this morning. This feels very serious so we're going to laugh a little bit. They don't know that this is happening. I didn't preface this, I know. So I've got these two gentlemen here with us and I'm going to ask Greg, now Greg's a very humble, sincere person, but I'm going to ask Greg to take the microphone and to repeat after me. I am the greatest extension administrator in all the land. I am the greatest extension administrator in all the land. And now I'm going to hand this microphone to Chuck and I'm going to ask him to repeat after me. Greg Lardi is the greatest extension administrator in all the land. Greg Lardi is the greatest extension administrator in all the land. Do you believe Chuck when he says that? That's the power of someone else saying it, right? So it's important for us to tell our story but it's just as important for people to help us. So I encourage you to connect with those people and create those relationships so that we can do that. Finally, I've had this mic for a lot longer than I planned. I was told this week that someone referred to me as the queen of branding in one of the sessions which was surprising because I've been called the warden a lot more than the queen. I'll take it. So on the first day of conference, I sat at the back of the room and I got teary-eyed because if you don't know it yet, on Monday morning at 8.30, we flipped the switch and there's a new extension homepage and there's all new topic sites. It's got eight months in the making and it's not a team did this and they are to be credited but so I had my laptop up and the new extension homepage was up, all new branding, all new signs were up, all new banners and Miranda Meehan was speaking using the all new PowerPoint templates and I thought holy cow we look like an awesome modern organization and I got chills because of it and so I think that it is truly, even though I harp on it all the time, it is truly important for us to continue to use those tools to make sure that we consistently brand ourselves, that we ride for the brand every single day so that people in Fargo, Williston, Sargent County, Pemona County across the state know that we are connected to the mothership in DSU and I was listening and that they connect the programs and the things that we do with extension so that's my final thoughts. Thank you. So does anyone have a follow-up question to that powerful final comment here? Questions? Comments? Okay so with the new website redesign and how incredible it looks, do you have any suggestions on for our own county pages, how we can make it look sleek and modern and up to date with today's you know just how life is? So that is the next phase of the web redesign but I can say that that's a major step and one that we're asking the legislature for some I believe that's part of our request is for some additional human capital to be able to do that. In the session that Bob and I did yesterday we it was called county website framework session of some sort and so we ask people to think about their current website and think about the things on that website, on your current website that Build Trust are a local resource, Build Relationships and I just blanked out. I can't remember the fourth one I'm terrible I'm sorry about that but so you know those are the things that that you do anyway so how do we make your digital presence reflect that as well and so you know the simple things like stuff that's old and outdated you know that needs to go away think of your county website as a classroom not a library right and so some quick simple cleanup things can do that but I also want you know that we are working we took the feedback from that group yesterday and that will help us inform your county web strategy as we move forward. I don't know if anybody has gone to food.unl.edu not that you necessarily would but if you have and I'd encourage you to go there and it's for two reasons one is it's a very robust research driven platform that engages all issues around food but here's the other reason I'd like you to go there maybe you do this already and so I'm being a little presumptive but the people that manage that website use social media to drive traffic to the website okay and so we think about social media sometimes as a separate and independent mode of communication but in this case social media is used in very deliberate ways so Facebook Pinterest Twitter all that stuff and and what's happening with that website is that they average 45 to 60 thousand page views a week second only two Husker athletics any other other comments or questions can you repeat that website again food food dot unl.edu so I have a question is we were preparing for the panel we were discussing about how the face of our organization has changed and how different this room looks than it did even three years ago and so we have a great deal of institutional knowledge yet we have some new and exciting ideas in the room as those who participated in the session yesterday on extras and millennials and getting those how do we reach those that audience so I'm wondering if someone has some suggestions for our our newer staff members and we're looking at the future of extension what what are some examples of how you have reached those audiences we can't continue to do the same thing again and again and wonder why things aren't changing so I know some of you have done some really innovative amazing things so how give us an example of of what you've done to reach a new audience don't make me call on you well I don't know about you but being the old guy in the room I still remember when mine John was director of extension okay um our younger crowd is not much of a meeting meeting going crowd so what I have done is is um and I have been somewhat successful with it is do demonstrations do tours do things where we get them actually on farms and not in a formal setting and then after they do that then then they start making the phone calls and then contacting you so sometimes you just have to take it take it to a different place and once they are comfortable with with that different place then they start to contact you and in maybe a more formal fashion although I do still get stuff on my phone and in other things I don't tweet I don't twitter I don't do the bird thing but my young people do find me now that they put a face to me you want else I'll take that other spectrum I am definitely not an old person in extension I've only been with your organization like two and a half years everybody thinks I've been here longer um I engage the youth and the youth help me engage their parents um so if you get involved with your youth and get them active they learn who you are then their parents seek you out my kid came home and told me you know how to do this yes would you like to learn more you know come to this workshop do this we're doing this or you know so-and-so in my county is doing this you know I go to a lot of other organizations I invited to their meetings people see me being visible is important so that's my piece is Lindy Berg still around today I'm gonna put you on the spot Lindy will you talk a little bit about what we talked at lunch today about new farmers and and their need maybe for extension it's a safe place it's a safe place I'll stand next to you do you have any specific questions about it I could piggyback just tell me about like you talked about the study you did and what and what you thought new farmers were needing possibly from extension of terms of certification that might be an opportunity for extension to do that can you um yeah I guess so what I'm working on um as part of my master's um project is um dealing with small farmers and how it kind of got started was um I would get people coming in that wanted they moved out to the country about a few acres or so and they want to kind of start a little farm and I didn't really have a whole lot of information to give them and I'm like how many you know we have some resources and there's a lot of other states that have great resources but I'm like I wonder like especially and now that I've gotten um to do a lot of research on this I know now like a lot of the small farmer organizations and so I know that but I bet a lot of agents do not and so where does that information come in and so my project um basically surveyed small farmers um and there's a few objectives that I covered such as who are the small farmers in North Dakota um what are their challenges and limitations that they're facing what where are they going to get their resources if they're not coming to us and are they using extension do they know who we are and then um oh what was the last one oh and then how are they how are they connecting with extension so um in the survey it kind of answers a lot of the questions and I think I had um 76 usable participants I think at 110 originally but so 76 small farmers um so it's pretty good data and hopefully um that will data will show quite a bit because it does show where those challenges lie and so I'm kind of I'm hoping that um working with some of the things on food safety we have a lot of good resources in food safety and what my survey is showing um that food safety didn't even rank in the top 10 for small farmers and so that's one of the things that it's like okay well how can we get this to be a priority and get them to these different workshops and things like that so yeah hopefully that'll be coming down the pipeline I would encourage you maybe you've already thought about this and done this but share the survey results back with the people who participated in the survey right I mean to me this is one of the this is a great example I think of of a way to engage people in a way where there's a co-discovery a co-learning opportunity you're setting a new baseline you're building the relationships I think they would enjoy hearing back from you what you learned from the study maybe you've done that is that the small farmers um we asked them how they want their information and they wanted one-on-one hands-on workshops and so social media was pretty low on the list and so it's like if we are using Facebook to try and tap small farmers that's not how they want to receive their information so if they want one-on-one well let's get those workshops and let's do those hands-on and let's visit these small farmers and learn how to help them adapt new resources on their farms so it sounds like we need a combination one-two punch um something that was discussed a case I'm going to put you on the spot here if we talk about relationships that's been discussed a lot this morning and all week in a case she is a rock star building relationships she's fearless I saw her walk into the cafe in Napoleon at a table of 10 gentlemen we were there for a tri-county tourism meeting and Acacia just sat down and struck up a conversation and invited them all to the meeting because they were all obviously Germans from Russia and so they needed to attend the meeting and so as Kelly and I talked that that doesn't come so easy for for us or for some of you in the room that's not so easy to do but I'm wondering if perhaps our panel or all of you could give us some exact examples some specific examples of how you have built those relationships because I hear from from newer agents who struggle to build audiences and I offer good programs and people don't show up it's about relationships so could could someone share some examples or specific examples how how do we how do we do that one-on-one Lindy how do we meet our one-on-one producers that want to engage with us in our programs I would say that one way to build those relationships is knowing that you want people to show up to your stuff and knowing that our programs are valuable and we think that it's really great for program or a community or a person or a school but showing up that the things that they value I think that that's important and then it shows that's where the trust is built that's where that relationship is built and some of those conversations can happen when you're showing your vested interest in something else that doesn't have anything to do with your program or project Ellen are you trying not to make eye contact with me that's it I guess the first thing that came to mind is and I think that's where she was going Jan was going is show up at their events be part of the community let them see you as being approachable and let them get to know you as well as you get to know them I mean that's that's critical so join a club go bowling I don't care whatever it is you're interested in but be in the community could I tell you a story that changed my life please so in 2002 when I was working in Scotts Bluff Nebraska we were having a workshop and one of the people that showed up at the workshop was a woman by the name of Jackie Cervantes Guzman and I'd known Jackie from the community she was a school teacher an elementary school teacher and I said you know Jackie it would be awesome if you came to work for extension and she said you know I've been doing this for 25 years I've been thinking about doing something differently and I might consider doing that and so we were successful in hiring Jackie the first Hispanic woman working in my district and I was excited about that opportunity but I as Jackie started I said Jackie I don't want you to do the Hispanic audience engagement for us I want you to do the Hispanic audience engagement with us and she said sure that makes a lot of sense let's bring other resources to this conversation other resources to this audience and I said okay where do we start and she said we started the soup kitchen at the Guadalupe center and I said okay what's that mean I've never done that she said we'll bring a crew in from from the district every month we'll do this once a month and we'll serve soup and we'll clean up and we'll do all that stuff we will be in service to the people that we would like to build a relationship with and so we did that and I remember the very first first event that we did we we serve soup and and you know we go around and ask people if they wanted seconds and so on and I noticed a lot of people that were there and certainly in that setting these are very needy people and and what was interesting was when we went for the second month a lot of the same people came back again and I asked Jackie tell me about some of these people and she said well this family over here is Hanson poor bear the local tribal leader for the Lakota community in Scots Bluff and she said I'd love for you to connect with Hanson and so she went over and we introduced and I I said hello and we chatted for just a few minutes that's all it was and the third month we serve soup and sure enough Hanson and his family were there again and the fourth month Hanson and his family were there again except what happened the fourth month when I went around to serve seconds Hanson said would you like to sit down and I thought oh my gosh here's an opportunity for me to get to know someone that I never would have had an opportunity to get to know under any other circumstances and I sat down and that started a partnership with the Lakota community where we did educational programs we did powwows we did an amazing number of things to bring and engage bring the Lakota people to the Panhandle Research and Extension Center and to do the kinds of things that they valued and they wanted from us so I share that experience with you because yeah maybe it's kind of the coffee shop experience but the point is you go to them and you find out a way you figure out a way to be in service to them and that opens doors and gives you opportunities to connect with people that you may never have ever thought you could communicate in any or connect with in any other way it changed my life. One other thing I'd add to that when when I was getting started as a specialist in Extension and in 97 and 98 when when I got opportunities occasionally people would invite me to their ranches and their farms and say yeah would you like to come out I want to show you you know a cattle handling system or I want you to take a look at this ration I've been feeding take advantage of those invitations that's that's an invitation to develop that relationship and really get one on one because you're you're with them in their environment you've removed a barrier in terms of it's not a meeting setting anymore it's not a classroom setting you're with them in their environment similar to what Chuck was talking about you take that opportunity you take advantage of that invitation so when you get those invitations go do it go go to their farms go to the ranches go their places of work go to their business go to their homes because that's where you can truly start to develop those relationships and pull the trust that was talked about because that's going to lead then to figuring out what are the additional programming opportunities how do we develop a program in this arena how do we develop a program for this audience so we focused a lot about external relationships and the future of extension and something was brought up to me earlier in the conference about building some of those relationships with our newer extension staff and we've used I'm going to use your term Dr. Hubbard co-creating knowledge I love that I'm going to go old school and get out the tape and tape that to my computer when I get back to the office so I'm curious how do we help build relationships within our own organization we have a lot of new faces here today what what can we do to make you feel welcome those of you that are newer what would that look like how do we welcome you how do we make you feel valued because you are I learned a great deal from the session on millennials yesterday you don't want us to leave a phone number for to be called so so what do we what do we do to keep you engaged in our system and feeling valued so as a new agent I I felt really empowered and just great by how much everybody reached out to me people from across the state were you know calling me emailing me just saying welcome to extension here's the advice I have for you please always call me if you ever need help and that just meant so much to me because as a new extension agent it was a little overwhelming and then just it was also my first I don't know how it's called but I call my big girl job so it just it meant so much to me just to have you know the support of just not even my neighboring community and agents but just people from you know across the state it meant so much to me so like I really just appreciated that I feel like fcw agents are really just they're great so if you not to be biased but I mean if you help knowing how to welcome new employees I would reach out to them because they just they made me feel so wonderful and just just so proud to be part of this organization that's a wonderful testimony anyone else yeah great well this isn't my first big girl job so I'll take that one I think it was the first or second day and I don't know where Marsha is but Marsha when I was a little tyke was actually our fcw agent down in Dunn County and she gave me a call and I hadn't heard her voice in what we'll call it many years not to not to date the conversation but she said do you remember me and as soon as I heard her voice I'm like yeah of course I do and it's just that connection of I grew up 4h I had those relationships did ffa did all that and what we're finding in our counties back home or the organizations that we have back home is the people that are active in our communities are the ones that come from those types of organizations so that was what and I had the same experience people were calling in the last seven eight months it's being engaged in all the different things I've you know met all these people that have brought me into different programs showed me what to do and that's the relationships that help bring us in is keep us involved keep us relevant because that first year is really difficult I mean it is hard to get into it kind of a thing because it's different this is completely different than any other job that I've had and I've had some really scary ones and I do some really scary stuff anyway I'm on the fire department and I was in the army so I mean that's something it's keep us involved so maybe so let me let me ask a question so for for those of you that have been in this organization 10 years or more uh raise your hand if you had a mentor somebody in the organization that mentored you so keep your hands up so keep your hands up if you have taken the opportunity now to formally or informally mentor somebody else yeah so we we need to continue to pay this forward when when I came into the organization I had a number of specialists and experienced agents that that taught me the ropes you know what what programs do you want to focus on what advice do you have those are things that are really valuable for our younger people to to hear for somebody to come alongside them and co-learn with them not to tell them what to do but to give them the assurance that they're welcome in this organization that they have a place that that there are things that they can feel where they fit in as you start an extension job and you're doing well what happens to you you get busy right you get busy in fact you can get real busy real fast and I thought about this a lot you know how do we mentor how do we coach not only new employees but everybody in the organization to think about how you're going to use your time what are the things that you want to be able to do in your job that you really value I love seeing the hands of the mentors that's one of the most important things that you can do but if you don't set aside time for that what you're going to add it to your already busy schedule we talked a little bit yesterday about how do you carve out time for professional development and I suggested two to four hours a week how are you going to find time for that see that's one of the challenges that we have in this job right because there's no one there that's that's going to tell us that we've done enough and so the tendency is to do more and pretty soon as Greg talked about yesterday it eats into your evenings and your weekends and your family time and so on so I'd encourage all of you to really think deeply about how you want to invest your time and how you want to invest the best of you in that time in ways that add value to what you're trying to accomplish and the work that you do in service to others so busyness is one of the biggest challenges that we face an extension I think across this country figure out how to manage it so I'm not old and maybe I am some of you I kind of remember near 4-H but I'm also not young but I also want to give a story about taking a chance so there's an old guy up in McLean County named Pat Carpenter he's kind of a rebel rouser I didn't know that when I started but I could call Pat up and come up with the craziest idea ever in fact Greg and Charlie probably were like that's so freaking crazy we can't even believe we hired her and I'd say hey Pat what do you think about this idea I think that's the best damn idea I've ever heard let's try it now Pat at the time was probably 65 or 70 years old okay I could say that to a lot of people that were in my age group and they would have said we tried that already it didn't work they tried that in Morton County it didn't work Pat would never tell me we tried that once and it worked he might say to me you know Lisa we tried that but I think with this change it'll work so one of the things I would tell all of you in this room is that if somebody calls you up with some crazy old hair brain idea don't tell them we tried that and it didn't work don't tell them I don't think that'll work take a chance the worst you're gonna do is fail forward well minus maybe ended up in jail but I hope it's not that crazy of an idea but take that chance the best programs I've ever done were on taking those chances and Scott and Bruce and I went to Kissed yesterday and they were like how do you know all these people in Kissed it's because that somebody took a chance on my crazy ideas or I took a chance on theirs to meet the people who'd be hanging out at Kissed at some point in time and so take that chance you'll meet people that you never ever believed you'd be and you'll make impacts you'd never thought you could and I don't know if that fit into all that but I think taking a chance and moving out of the box a little bit really will get you a long ways and now Nicole Wardner's my yeah I think we can do that I do believe someone talked about being bold the other day thank you Lisa all right so I think our time is about up does he have any final comments or a panel okay start of the day for anybody who doesn't know it was my birthday yesterday so and and we already sang to me so don't feel the need to I do want to add to my comment earlier also I want to give a little shout out to my district director he's been so great and has made me feel so welcome and honestly whenever he talks to me I feel like I'm his only agent and I forget that he's got like just such a huge district and so I saw him say thank you that's a great high point to end on so how about a thank you for our panel today thank you all right thanks Jody and thank you everybody for coming and I think you've heard the word engaged a lot of times the past couple days here thanks again to Elizabeth Cronin I don't think we could have organized this thing without her so and everybody that spoke great job I think we had a nice time to engage with each other we had some nice break time to network and talk and I think it's a good conference so drive safe everybody I know so you got a long ways to go I guess you want to say any final words so just as you leave as Ryan said travel safely home this group is a future eviction so as you leave here and you think about this conference and you think about the innovations you want to put in place those are the things that we're going to be talking about 10 years from now hey do you remember when we came up with this idea do you remember when we did this at the 2018 conference so as you leave here be thinking about the innovations you're going to put in place to drive the future of this organization thanks and have a great weekend


Great now we get to talk about economics and When I gave a lecture in our public health genetics course the constant comment My favorite comment was it was nice to hear from the dark side so here we go and they often save economics for last and In some ways, I think it's because at the end of the day ultimately We all need to make decisions about whether things are a good value or not So I want to tell you a little bit about what's been happening in Caesar, but try to focus a little bit more on what the incredible opportunities are in front of us for future study First I just want to talk a little bit about some terms here We talked about healthcare resource utilization in some ways. That's codename for cost So how much healthcare are people using and we can put a price on that later and figure out what the costs are But some of the main points I think are that just because something is expensive doesn't mean It's not good value just because it's cheap doesn't mean it's cost effective And what we're really looking for in this field is to get the most health benefit that we can for the amount of money that We spend In the field the technical field of health economics. We tend to measure outcomes Using this metric called a quality quality adjusted life here You can think of it as as a year of perfect health and it's a good thing you want more qualities and We calculate something called an incremental cost effectiveness ratio So you might hear me throw around throw around some of those terms But of course payers they care a lot about costs and that's something that we need to pay attention to and that's why We're measuring this healthcare resource utilization a lot of the studies within Caesar Ultimately what we want to do is improve patient outcomes and we want to do that in a cost effective manner Which means the amount of money we're spending is Reasonable compared to other things that we buy in health care or that were cost savings It doesn't necessarily mean that it's cost savings. So To sum it all up when we talk about health economics, we're not just talking about cost and in fact We're more of more often focusing on effectiveness And so I'm really glad that the the session on clinical utility proceeded this one because that really lays the foundation for cost utility Now these slides here just give you Kind of just a sense of what's going on in Caesar in some of the sites and the type of health care Utilization data that that's being collected and you can see that there's you know, there's a lot of different things you can look at and there's a lot of different ways you can categorize these outcomes and It ranges everything from just number of visits what medications people are taking To questions about health insurance and maybe even life insurance And you can see that there's a lot of diversity across the different sites And also there's a lot of different ways Methods for measuring this information Probably in the Caesar consortium leaning a little bit toward patient surveys to find out what kind of resources are being used But also some use of electronic medical records, which can be quite powerful So ultimately what is our goal here? It is to help health care systems and pairs make more informed coverage and reimbursement decisions for clinical sequencing and In order to do that I think we've heard from some of the questions and some of the speakers about challenges they've run into with reimbursement in the Northwest That some decisions that affected us in Seattle were things such as any Gene panel is investigational and therefore will not be covered So I think sometimes pairs are reacting to a lot of change and a lot of uncertainty Sometimes strong marketing push from companies and are coming out with policies Like that and our goal here is really to help them make better decisions And in order to do that we're going to need to collect some data So first I want to talk about just a few of the challenges and then the opportunities that I see for Caesar too The first one and this is really a tough one You know people are always asking what evidence do decision-makers need and then they'll invite the pairs and Pairs will say well, we do like randomized controlled trials and then everyone says oh, they only want randomized controlled trials Well, you know none of that's true. It's a very very hard question It's very difficult to bring someone in and I think Naomi can attest this and say what evidence do we need for a whole Exome sequencing or whole genome sequencing Well, you know, I don't know that's that's a very broad question You can start to ask more specific questions, but then you're asking that person to become an expert in that area So it isn't an important challenge and it's something that I'll come back to a bit and I think Pat will respond to you also I think the other issue really is that there's so many different things you can do with clinical sequencing That you know, there isn't just some generic well is is whole genome sequencing cost-effective well It depends it's like our drugs cost-effective well It depends so you have to be very thoughtful about well What exactly are you talking about doing and what are you going to use it for and then who are you going to use it? But that also presents the challenge that then there's lots of different things that you could be looking at So that's that that's something we'll have to come to terms with And then also obviously I think we need information about all aspects of the delivery of the tests not just the cost of the test and I know that in particular two sites Harvard and Kaiser are collecting very carefully measured quantitative data about How much time is required by their genetics providers and then you know a lot of the Cancer sites are looking at the impact on treatment decisions. So these are the things that I'm that will need to put into the mix I think another issue that really comes up and has been mentioned a couple of times is comparative data or a comparative effectiveness And it's something that I think is really it's quite important It doesn't mean we always have to have it doesn't mean we always have to have randomized control trials But I think that To talk to payers you really need to talk about comparative data. What's your next best? Alternative and what's your opportunity cost? The last lastly, I think we're all where you know sample size is really a key issue Just in general looking at health economics costs health care resource utilization for anything It's very challenging to collect enough data in that area And you know potentially you could quickly find yourself needing studies in the tens of thousands even a hundred thousand people To get definitive health economics answers and you know it for the most part That's not what we do in the field most the time in the field We take data from a randomized controlled trial that's powered for a clinical outcome Maybe a surrogate outcome and then we do our best to estimate the economic outcomes, but this is an important challenge So opportunities for a season are moving forward, and I really think they're tremendous the first one really I think is Doing our best and figuring out ways to have conversations with payers and to get some input from people out there making decisions And and and enable them to tell us Something about what it is we're doing Around the time that we're trying to design and implement these studies and identify our outcomes It is very challenging I think we need to be somewhat indication specific and It's important to talk about types of evidence rather than getting obsessed with what exact level of evidence is needed In order for you to say yes to this technology. That's just it's very hard to do and it doesn't happen in other areas in health care The other is that You know, I think Katrina showed you that the heterogeneity of clinical indications and approaches across Caesar Which I I agree has been a great strength because we've been able to learn a lot from that But there's also kind of pools of similar indications or uses of clinical sequencing And I think we need to figure out how to how to best align Projects in those sub areas so that we can pool data within those areas Obviously thinking I think you know the three major categories would be diagnosis treatment and screening and then particularly within Screening there's this subset of incidental findings Which could be viewed as opportunistic screening and that's something that could apply across all sites and we've already Started efforts to do that. I think maybe tomorrow Jonathan Berg is going to be presenting Some information about the incidental findings across the entire Caesar Consortium and then lastly I really enjoyed The discussion about the families and getting involved with them I think actually from a health economics perspective I think what happens within families will be one of the single most important drivers for the value of precision medicine And I think it's something we really need to give a lot of thought to and then this other discussion about engagement with patients Was great because think about engaging families. It's not easy either right got to figure out approaches to do that In terms of data collection, I think that just briefly mentioning study design again Trying to be comparative. We're feasible trying to have studies that are actually powered on some kind of outcome Okay, we're not always going to power on survival But we should have some kind of clinical outcome that these studies are powered to as feasible I think we've seen a very nice, you know laid out Robert laid out five trials that have been done and are ongoing right now and those are all small kind of phase one I think it's time to think about moving to phase two in terms of in terms of the kinds of studies that we're doing and moving You know finding our initial signals and trying to confirm those and using a lot of the tools that we're developing here in this study In this consortium and some of those tools Really, I think relate to data collection. I think again this it's patient and family-centered cost issue the cost of You know the diet the the odyssey of trying to find a diagnosis for a patient I don't think has been well docked enough what Documentation has gone into that. I think that there also needs to be some communication with payers about what does that really mean and You know, what's their level of engagement and willing to pay? To end the diagnostic odyssey we could probably make a more efficient use of EMRs within Caesar I think that's something we need to think about and then I think this last point Not to get too technical, but as I said, we're never gonna have studies I think that are powered for economic outcomes I think we need to start thinking about developing patient reported health economic outcomes where we can ask patients What actions did you take because of your test result? Now there's some obvious limitations with that, but I think that potentially we could develop some tools that would really Enable us to find the signal and the noise there with the health care resource utilization And then lastly, I'll just mention this general area of policy models. These are quantitative policy models cost-effectiveness models These are calm. These are used throughout health care to try to get a better understanding of the boundaries of the benefits and harms costs and ultimately the economic value of interventions and and even when you don't have a lot of data that can be quite useful and You know one study by Carlos Gallego was mentioned looking at inherited colorectal cancer risk These model, you know, there's been some work within the Caesar one to develop some of these models But they're just sitting there kind of waiting for more data And I think that not only further development of these models But generating data and Caesar to inform those models and then having that interaction and conversation with payers will be quite fruitful So just in summary, I think there's for the for the consortium. I think there's some great opportunities Just to go over the go over that my key points again pair and decision-maker input. I think will be Really important but challenging Developing validating and implementing some common economic measures across the consortium and not just Caesar But also emerge and ignite for instance the the working groups are starting to have conversations across those consortia So I think there's great opportunity there and then again the data pooling within indications and More broadly as feasible and then lastly some of these kind of policy frameworks to in a way fill the gaps Well, we're not going to be able to have these huge randomized trials for for every indication so that's it and If we want to take questions or allow Pat to respond If there's one or two specific questions, and then we'll have the more general discussion So Dave that was really nice. I was just wondering Do you have any thoughts about the prior discussion with regard to diagnosis one obvious economic brooch would be you know Money saved by ending the diagnostic odyssey, but is that really the only way to address that? I think most of us as practitioners take it as almost axiomatic that making a diagnosis as a benefit You know insurers don't pay much attention to what we've considered to be axiomatic Yeah, yeah, I think there's that there's the oh we'll save money because we're not searching anymore But that inherent value that it has to individuals that's real And there are methods that people have used and are actually are using within Caesar to put a monetary value on On that for patients the challenge is kind of as a society as a health care systems Is that something we pay for or not? And I think there's some really nice work that could be done there that you know Is a broader social issue even more than economic Build on that comment for a second one of the Corresponding struggles that has gone on for a long time as many people in the room probably come from academic medical centers One of the challenges academic medical centers have is in any way demonstrating that they have better quality than other centers Because if you look at many of the kind of standard things delivering mammograms to everyone actually it doesn't necessarily look better And so there's a corresponding effort going on now to try to ask the question. Can we actually look at the? Quality that may come from a better diagnostic Center than a non Than a center that has less let's say tools or ability to do that So it'd be interesting to see if that can end up informing this discussion also because it's kind of a corresponding Effort which really relates to why do people come and look for that diagnosis at a tertiary center Okay, great. So then Patricia is going to give her comments and then we'll have a general discussion


Reg Weaver is an accomplished labor leader who served as president of the National Education Association from 2002 to 2008 and was vice president of Education International. He has served on many boards, including the NEA Foundation, and spent 30 years in the classroom as a science and health teacher. His trademark laugh keeps him light on his feet, even when the work is heavy.  You got to find something to laugh about. And so, when I'm speaking with our folks, I usually try to, you know, say something that allows them to have a good laugh. Reg is semi-retired now, but continues to speak and advocate for children and public education. His insights and experience from a long career remind us there are always lessons to be learned and that there's always room to change. You know, we as educators, we all do good things and we are remembered. And you remember those good things, but there are also some things that you do that are not so good. And sometimes you remember those, more so you remember the good things. There was a teacher and she was working with the students and she was giving them a math problem.  And then she went to the students and the students were just laughing and she said, why are you laughing? And they said, because of the mistake that you made when you said 9 times 2 is 56. And the teacher said in essence that was done on purpose because it was done to show that everything else that I said was correct. But you only remember the one thing that I did that was incorrect. And she said that's what you might face in life. So don't give up. And I said wow. And so I thought about again something that I had thought about for years. I was an eighth grade teacher along with another female colleague. And as eighth grade teachers we were the eighth grade sponsors responsible for the graduation, class play, whatever. And so the two of us, mostly the ladies, we decided that the boys, if they came to graduation in these shiny suits, well, we would not allow them to walk across the stage. We would not allow them to participate in graduation. Now, I agree to that. And I didn't give it much thought because I knew that no kid would come to graduation in one of those shiny suits because they were expensive. So lo and behold, graduation came. And this one kid showed up in the shiny suit. They were known as air-descent suits. So the female teacher, she came over edge, edge. You know what we said? You the man. You the man. Now, back then you the man had a different context. You the man back then was, you are the man that is a sponsor and not the main man. I said no. She said yep, yep, you know what we said. So I had to go tell this kid and his parents that he could not walk across that stage. And the father came to me. He said Mr. Weaver, what's wrong? Why is it that my kid cannot graduate? Did he do something wrong? I said no. Did he fail? No. Was he disrespectful? I said no. He said well why is it that my son cannot graduate? You see, his father was a member of a very famous singing group. And on stage this singing group wore these air-descent suits to the pleasure of the crowd. And so he was saying that if it's good for the crowd that I perform in front of, then why is it that it's not good enough for my son to wear on graduation? I don't know whether this kid ever graduated from college or high school, but I do know that he met all of the qualifications to graduate from grade school.  And it just bothered me and bothered me. And fast forward to about 25 or 35 years later, another member of the class, they had a large family and I had six or seven of their kids. You kept up with a lot of these kids, didn't you? Yeah. And they still today. Many of these kids are 60. So these kids invited me to a birthday party, their mother's 95th birthday. So it just so happened that I was in town during the time and when I walked in, the kids, they just bum-rushed me, you know. Mr. Weaver, Mr. Weaver, blah, blah, blah. And one of the kids who was sponsoring the birthday, Mr. Weaver, you know, there's so-and-so over there. And I looked and it was the kid I told couldn't walk across that stage. And he was sitting behind a very sophisticated looking turntable and he was more or less the DJ. And I went over to him. He said, Mr. Weaver. I said, man, is this you? He's, yep, this is what I'm doing now, man. One of the things that I'm doing.  And then I said, look, remember graduation night when I told you that you couldn't walk across the stage because of the expensive suit? He said, yeah. He said, the shoes were expensive too. He said, yeah, I remember. I said, look, man, I said, that has bothered me all of this time and I never knew that I would get a chance to see you. But now that I do, man, I am so sorry. I am so sorry. He said, that's okay, Mr. Weaver. Don't worry about it. Did he tell you anything else about that experience from his perspective? No. No. But it taught me some things. And it taught me that sometimes rules that you make, if you find that they're not working, change them. You know, sometimes people make rules and think that they can't be changed. But if, in fact, you find that there are some rules that you have established that are negatively impacting the lives of students, then you've got to figure out what to do. And if changing that rule is something that needs to be done, then that's something that you should do.


Now, the chief of defense staff, Lieutenant General Locky Erebo, has advised military retirees against being used by politicians to truncate a country's democracy, since such move could lead to a face-up between the military and the citizens. Addressing retirees who graduated from the Nigerian Armed Forces Resettlement Center Oshady having completed the mandatory vocational training, the CDS warned against complacency. Transition to civil life after several years of service is a global practice in the military and the Nigerian Armed Forces is no exception. And to ensure a seamless move back to civil life, the Nigerian Armed Forces Resettlement Center Oshady Lagas was established. The chief of defense staff, General Locky Erebo, enjoined the retiring graduates of the center to take advantage of the skills acquired to better their lives. He advised them to be prudent with their retirement benefits and invest wisely. He further warned them to resist the temptation of meddling in politics as such move could destabilize the country's democracy. I urge you to enter training for peaceful and productive economic life as you reintegrate into the Nigerian civil society. I encourage you to justify the anonymity of the President, Commander-in-Chief of the Armed Forces of the Federal Republic of Nigeria, President Mahmado Buhari, GCFR, who has continued to support the Armed Forces, most especially the retirement deterrent members of the Armed Forces and sleep in the enhanced packages by maintaining your loyalty and dedication to the Federal Republic of Nigeria. 316 graduates drawn from the Mandran Army, Air Force and the Navy, were trained in various vocations for easy reintegration into civil life. You have successfully completed these calls amidst tight schedules and this shows your commitment and discipline which are all attributes you have acquired and the cost of your artist's service to a dear nation. Let me emphasize that the journey's free retirement course is to reorientate and position you for a successful reintegration into civil life after a military service to a dear country. It is our expectation that your graduation, the service and the centre will continue to get positive feedback as you currently soar through the post-suggest life. Be bold and courageous, remain disciplined and focus, the sky certainly is your beginning. It is expected that with the acquisition of needed skills, the retirees will lead productive lives in their post-service years. Destiny Momo Plus TV News