Good morning, and welcome to this week's edition of Encompass Live. I am your host, Krista Porter, here at the Nebraska Library Commission. Encompass Live is the Commission's weekly online event.  We are here live every Wednesday morning at 10 a.m. Central Time. However, if you're unable to join us on Wednesday mornings, that's fine. We do record the show every week and post it to our website, so you will be able to watch any of our recordings later. And I will show you at the end of today's show where the website is and where all those recordings are available for you. Both the live show and our recordings are free and open to anyone to watch. So please do share with your colleagues, friends, neighbors, family, anyone who may have an interest in any of the topics we have on the show. Tell them they can sign up for our live shows or watch any of our recordings out there. Encompass Live premiered in January of 2009, and we have all of our recordings are on our website that you can link to and watch them via our the Nebraska Library Commission's YouTube account. We do a mixture of things here on Encompass Live, book reviews, training sessions, interviews, tours, demos of products and services. Basically our only criteria is that it is something library related. And any kind of libraries, school, public, academic, special, prisons, institutional, whatever, that's really our only thing. Is it something to do with libraries, something libraries are doing themselves, something that may be of interest to them? That's what we have on the show. We do sometimes have Nebraska Library Commission staff do presentations, but we also bring in guest speakers as we have this morning. On the line with us from East Coast, as some of you heard in the beginning, West Virginia, Beth Anderson, director at the Burnsville Public Library in Burnsville, West Virginia. Good morning, Beth. Good morning, everybody. And also along with us is Samantha Lopez, who I think you were up in Chicago, correct? That is correct. I can remember who she is actually from the Public Library Association from PLA. And they're going to talk to us about this new initiative from PLA, Project Outcome, which I had at my radar last year, some time when we first talked about reading up on it, and I finally was able to get Samantha and Beth, who's used this, to come on the show and tell us all about it. So I will just hand over to you guys to take it away and tell us about Project Outcome. All right, sounds good. Thank you, Krista. Welcome, everyone. Thank you for joining us for today's Encompass Live presentation on Project Outcome. Today's presentation is brought to you by the Public Library Association, a division of the American Library Association here in Chicago. PLA provides our members with a vast array of products and services to help public librarians succeed in their profession. Today I will be talking about Project Outcome, which is a free product PLA provides. You do not need to be a PLA member to use Project Outcome. It is available for free for all U.S. and Canadian public libraries. As Krista already said, my name is Samantha Lopez. I am the Project Coordinator for Project Outcome here at PLA, and I'm joined today by Beth Anderson, the Director of Burnsville Public Library in West Virginia. Beth presented with PLA at the West Virginia Library Association conference last fall, and I'm glad to have her back with us to talk about the benefits of Project Outcome for small libraries. I also want to thank you, Krista, again, for inviting us on this presentation. Here's a quick overview of today's agenda and what we'll be covering. Please feel free, as Krista said, to ask questions along the way. In the chat box, we'll have a Q&A session at the end, so please ask any and all questions you might have. So first, what is Project Outcome? You might have heard about us, but you still might be unsure of what we are or what we provide. Project Outcome is a free online toolkit designed to help public libraries understand and share the true impact of their services and programs by providing simple surveys and an easy-to-use process for measuring and analyzing outcomes. Project Outcome also provides libraries with the resources and training support needed to apply their results and confidently advocate for their library's future. You can find us at www.projectoutcome.org. But before I dive into the details of the Project Outcome toolkit and all that it provides, I want to take a few minutes to clarify what we mean by outcomes and why we are helping libraries measure them. For years, libraries have gaged success through their patron stories, and the core metric has been how often they came back for more. But times have changed and so have libraries. Your valuable services can't live on intuition alone because insights, not anecdotes, will improve the way libraries do business. To show their impact, libraries need more data and evidence. You may already be measuring your impact in a variety of ways, like collecting outputs or asking your patrons if they're satisfied with your services. Needs assessments help the library answer what does our community need. Patron satisfaction surveys help answer what should we do better, and outputs help answer how much did we do. Measuring outcomes doesn't mean you stop doing these other things. It simply means you're gathering even more data to create a more compelling story for your library. Measuring outcomes helps the library answer what good did we do. So what exactly do we mean by outcomes? Outcomes are simply what your library patrons feel they learned or gained from attending a library program or service. Outcomes can be qualitative or quantitative, and again, they answer that question, what good did we do? This was an image used recently by one of our guest presenters on our last webinar, which happened last week, and since it made me laugh, I included it here for any visual learners out there. Inputs being what the library puts into a program or service or even circulation. So how much is it costing, how many staff, how many hours, how many checkouts, etc. And outputs are what comes out of the program or service. So how many people attended, how many times was the book checked out, and so on. The outcomes are then what the patron actually gets from the inputs. Did they learn something new? Did they plan on using what they learned when they get home? Are they more confident as a result of the library's program or service? So now you might be thinking, well, that's all well and good, but who has time to do this? I'm busy doing all the other things I need to do to run my library, and if you're someone like Beth and you're the director and you're kind of running the show because you have 1.03 employees at your library, then you might be thinking this exact thing right now. And that's what we're here for. You've been told you should measure outcomes, you understand why they're important, you need to better tell your library's story, and you need a consistent way of doing so. Because you're so busy, PLA has already done a majority of the work for you. Instead of reinventing the wheel, we've created Project Outcome to make measuring your patron's outcomes as quick and easy as possible. Now I'm going to go into the actual surveys we provide and the tools that are available on our website. So Project Outcome, as I said, is a free online toolkit that provides public libraries with access to quick and simple patron surveys, a survey portal to collect and enter your survey data, ready-made data reports so you don't have to do anything once the data is collected. The system generates reports for you. And all of the resources and training libraries need to help them be successful throughout the outcome measurement process. We have heard from our users that the combination of the ready-to-go surveys and the easy-to-use tools really help library staff save time and energy in planning their data collection, leaving more time for decision-making and advocacy once the results are in. To reiterate, all the surveys, tools, and resources that I go through today and are available on Project Outcome are free and you do not need to be a PLA member to use them. You would be surprised how much I have to say that it's all free and I still get the question, but how much does it cost? So I'm going to say that several times. It is free. It is available to you for free. Sounds very, very familiar, yes. Yes. My guide got that along. I handle the... So I'm the project coordinator. I do a lot of things, but on top of that, I do all the help desk things for the Project Outcome website. All your questions come to me, essentially, and so many times it's, oh, I just went to this program. I went to this conference. I heard about Project Outcome. They said it's free, but really, how many... Tell me the truth. Yeah, tell me the truth. So I'm giving it to you. It is free. It was provided initially, grant funded by the Bill and Melinda Gates Foundation, but PLA is dedicated to keeping and maintaining what we have developed so far free for public libraries beyond the grant, which ends in 2018. Nice. Okay. I just want to get that out of the way. So the task force that developed Project Outcome, the task force was developed because in 2013, our then PLA president, Carolyn Anthony, saw a gap in need in the field. While many libraries were measuring outcomes, they weren't doing so consistently or in a standardized way. She wanted to provide a way for libraries to easily and consistently measure the outcomes of their programs and services so that outcome measurement could become a field-wide practice. The surveys and outcomes themselves are the foundation of the Project Outcome Toolkit, and they were created by our performance measurement task force, which is made up of a diverse group of public and state librarians and researchers. And they have gone through the testing and vetting process of the outcome questions so that you don't have to. So if you're wondering if PLA just made all of this up, we did not. It was developed by a task force. It seems very simple and easy, and it's because the task force worked for a long time to make it appear so. So you can kind of rest assured that PLA did not make this up on a whim. It was made and created by your peers. In its early work, the task force identified seven key survey areas that cover the broad range of programs and services libraries provide. Outcome measurement surveys were developed for each of these seven areas. So as you can see here, we have surveys that cover civic community engagement, age-outed literacy, digital learning, education and lifelong learning, job skills, economic development, and of course, summer rating. Not surprisingly, the highest number of surveys we have in a topic is summer rating. Summer rating technically has three surveys underneath it. It's the only one that's like that. We started off with one survey for summer rating, but as we launched in 2015, we heard back from our users that the questions weren't really asking what they needed from their summer rating programs. So the task force developed three different summer rating surveys that can either be administered to the teen or child themselves, to the caregiver of the child, or even for the participants of any adult summer rating programs you might have. So we have three options available there. There are highest number of surveys because that's probably the most likely the highest attended programs that you have throughout the year, but the most popular survey topics that we see come in in terms of scheduling new surveys but might have less responses. That would be education and lifelong learning because it's the broadest survey, so it covers a variety of programs that might not fit under the other survey topics and of course early childhood literacy, that's also one of our more popular surveys. The task force designed all of the project outcome surveys to capture four key outcomes, knowledge, confidence, behavior change, and awareness. They felt these four outcomes when measured together provide the best insight into impact and whether or not patrons will actually use what they learned once they leave the library. All of the surveys include two open-ended questions, so libraries can get the most out of their patron's feedback. What did they like most about the program and what can the library do to improve? These open-ended questions have been invaluable to libraries. Here is where they gain the most insight into how to help their patrons and gather anecdotal evidence to support their impact stories. So those open-ended questions are where libraries can kind of pull those really great quotes to complement the scores from their outcome likert scale questions. There are three outcome measurement tools developed by the task force that help libraries measure different patron-reported benefits based on what the library wants to learn. The immediate surveys are for patron-reported learning. The follow-up surveys are for patron-reported adoption or application. And the outcome measurement guidelines are so you can design your own surveys and data gathering methods to capture long-term impact. You can use all of these tools or any combination or just one. Not surprisingly, our immediate surveys, that's what we initially launched. With Project Outcome, that's all we had in place, and those still are our most popular surveys because they're the easiest to do. The immediate surveys are designed to capture that patron-reported learning and the immediate impact of the program. The immediate surveys take less staff time than the other methods, and they only require a little bit of planning. So you'll simply decide which program to survey, schedule the survey in the portal, give out the surveys at the end of the program, and enter the data. All of our surveys are also available in paper and online formats and are translated into Spanish. The immediate survey data is good for a quick snapshot assessment of how you're doing and what your patrons feel they've gained right after a program or service. They're an excellent way to collect data to understand how the library is doing at any moment in time. The follow-up surveys are designed to capture a patron-reported adoption and the longer impact of the program. The follow-up survey option takes more staff time than the immediate surveys, and it requires more planning and collection of patron's contact information. Some libraries might have limited staff capacity and will choose to just stick with the immediate surveys because they're quick and easy to do, and that's okay too. So when a library needs to answer the question, but what are they actually doing after they leave the library, then the follow-up surveys are the way to go. Follow-up survey data is good for assessing longer-term impact and answering the question, are your patrons actually using what they learned once they leave the library? It also helps provide strong evidence for advocacy and planning and helps measure progress towards strategic goals. Here's a snapshot of the type of outcome data collected from both surveys. Like I said, libraries can pick and choose which surveys they want to use based on their needs and capacity. Combining surveys will give libraries a broader picture of their impact. For example, a library using both would be able to say our programs help patrons feel more confident about searching for a job and that they actually applied for jobs using what they learned. But as a reminder, you don't need to combine the surveys to get useful data. You can still use one or the other. The outcome measurement guidelines are not yet public and will be released sometime this year. They are designed to help libraries measure outcomes with a community impact goal in mind. This type of measurement will likely require libraries to develop their own outcomes based on long-term partnership goals, and the guidelines will help libraries work through that process. Everything for Project Outcome can be found on the website, projectoutcome.org. Registration is free and only takes a few simple steps to begin. The website provides helpful resources for libraries to get started using the tools and how to use the results for advocacy or decision-making. The Survey Portal tool is where you'll schedule, access, and input the data results from all of your surveys. This is a snapshot of the Survey Portal. Right now we use University of Washington's Impact Survey Platform to host our Survey Portal. We are currently developing and designing a brand new portal, which will launch this May, and the new Survey Portal will allow for more flexibility and customization of the surveys. So this is what it looks like right now, but it is changing very, very soon, and I'm very excited about these changes, and I think all of our libraries will be, too. In the Survey Portal, you'll get reports automatically generated for you once you close a survey. They are designed for you to quickly and easily analyze and talk about your results, and even to share with your staff or library board. So you get a CSV Excel download of all of your raw data, and that's where you can kind of, you know, sift through your data. If you'd like to make your own charts, you know, that's where you can do that. That's where you can see all your open-ended comments and the locations where they came from. But if you don't want to do any of that work, you also can look to this summary report that we have available that provides talking points about that specific survey topic and why it's important to the community. It gives a breakdown of the programs and the sessions and the attendance that were measured under that survey. Also provides a quick snapshot of the outcome results of the four, you know, key questions of that survey and what you saw come in. And we know that libraries really like this automated report aspect of the Survey Portal. They can take it, they could share it with their staff, they could share it with their board, and it's really easy to use. You don't have to do anything, and it kind of does all of the work for you. The data dashboard tool is where you will analyze, interact with, and print your survey results. So once you actually have data in the system, it automatically populates in the data dashboards. Inside the data dashboard tool, you'll find five different dashboards. This is what the overview dashboard looks like. This provides a quick snapshot of your aggregate scores by survey topic and outcome type. As you hover over each icon, a window will appear with your total surveys in that category, as well as provide the national and state comparison averages in that category. The matrix dashboard provides a table for libraries to sort and view their scores by survey topic or program name. By sorting by score, you can see quickly where your strengths and weaknesses are in either survey topic or outcome type. You can hover over each box and see that quick breakdown of agreed and disagreed percentages as well. The chord diagram at the top allows you to hover over each segment or chord and see that breakdown again. The chords indicate strengths and weaknesses by color and thickness, so the darker purple color means a stronger score. These tools are really meant to help libraries internally interpret their data. They might not necessarily use these exact visuals to put into a presentation or to present to their board because they take some time to interpret, but this is really for you to kind of play around with your results and see where your strengths and weaknesses are. The detailed dashboard provides a more conventional view for libraries. Here they can see all of their results by each question in each survey. It's not by aggregate. You can see a breakdown of each question. They can filter down to the program level as well. The graph uses a median line to give that quick view of agreed and disagreed scores, and the more purple coloring and heavier responses that fall on the right side of that median line means stronger scores for that particular outcome question. Here's where you can also export any open-ended comments. You can sort and filter through them by library, date, program, or survey. So if you have, say, summer reading and you have over 200 responses and you just want to quickly pull a couple good quotes, you can go into the dashboard and use this open response viewer and export those responses. We also provide a map dashboard that displays demographic census data over your outcome scores. This will show you which branches or outlets are participating and their outcome scores, as well as provide the coinciding demographics for that particular area. This is just a start of how you can blend the many data sets available to you once you start measuring outcomes. And this is just an example of how we want people to think about outcome measurement. So if you're talking about your impact, then you can look at the demographics, which your community made out of, and you can tell a larger and better story. So quick overview. The benefits of project outcome is that we provide the short and simple surveys. And by providing these surveys, it means that you're likely to get a higher response rate. They are short and simple. The task force purposely designed them to be that way. They are not meant to be complicated, so that patrons are not turned off by them. You'll hopefully get higher response rates, so then you can talk about your impact. We aim at capturing snapshot data. We are not. Project outcome does not require rigorous data collection, and we don't recommend talking about it that way. It's snapshot convenience sampling. As I said, the open-ended comments for each survey are a gold mine for libraries. That's where they're learning the most about how they can improve and what their patrons want. And project outcome is at your own pace. You can pick and choose surveys based on the program, the learning objective, the capacity that you have at your library. It's really up to you. You can dip in and out, kind of see if you want to measure story times, and you get repeat customers, so you don't want to be surveying the same people all the time. Then maybe it's once every three months you run surveys. You can kind of set your own schedule with project outcome. The ready-made reports and data dashboards that I just showed you do all of that heavy lifting for you, so you don't have to interpret your results. You don't have to create graphs. We've done all of that for you. And, of course, the standardization of the outcome measures so that you can aggregate them to a larger level and then have those national and state score comparisons. As I've said before, project outcome has everything you need to get started, and our resources and support help guide you along the entire outcome measurement process, from planning for surveys to how to use the results to take action. And speaking of, I'm going to now talk through some of the different ways that we've heard our participating libraries have been able to use the results to take action. Since libraries see the results immediately, they're able to analyze their data and make quick and effective improvements to their programs and services. For example, one library used immediate surveys to measure the effectiveness of their science kit service for one month. The library put the Education and Lifelong Learning paper survey in each of the science kits that they loaned out. They received about 30 responses, and what they learned is that most parents and kids were unaware of the additional science programming that the library provided. One response actually said the libraries should provide kids programs so kids will want to come to the library more often. And this patron unawareness kind of shocked the library and resulted in them inserting their program brochures so that all of the science kit users could learn more about the library's hundreds of kids programs and when they were offered. 90% of the brochures did not come back with the kids. So this is an example of a small quick and inexpensive change that the library can make to promote awareness of their programs. Many of our participating libraries are using project.com to help inform their strategic planning. You should always align outcome measurement goals to strategic goals and community initiatives. How will outcome measurement help your library reach strategic goals in the long run? You should focus outcome measurement on key areas of service. So what programs and services help you achieve your strategic goals? And remember that you are measuring progress towards strategic goals, so think about how outcome measurement data will help support your successes and identify areas for improvement. Many project outcome libraries have been able to use the positive results to reinforce their value in the community. They have been able to use the summary reports and the data dashboard views to give a clear and concise message about their patron impact. One project outcome library said they were able to change the conversation with their library board and internally with staff from numbers being down to what are we accomplishing by being open and what is happening in the lives of our patrons. And we've heard this a lot from many users. So it's changing the communication with their board and increasing that transparency in what we're doing and how we're doing. It's aligning what you're doing with the community and what they want. It's changing conversation with staff. You might have to try to convince your staff that outcome measurement is important, but once they see the results, they're typically on board. And it's also increasing communication with patrons.  This is telling us how we're doing and we want you to be open and honest because your voice matters. We're going to make changes based on your feedback. So please don't provide. We love the library. That's great to hear, but it doesn't help inform anything for the library. So we've also heard that it's nice for the library to have that additional communication with their patrons. And to have the patrons feel that they're actually heard and that they make a difference. Libraries are also able to use their outcome data to start partnership discussions or help reinforce preexisting partnerships. Here are some examples of how libraries have been able to do this. One library is measuring outcomes for the first time as part of a regional initiative to reduce poverty. Many libraries are using key data points from the summer reading surveys, including the outcome that the child maintained or increased their reading skills as a jumping off point to start school partnership discussions. Libraries are also using their data to improve programming, which may include external partners. For example, one library received negative feedback about an instructor. So they looked to the local community college to partner on their business development classes to get better instructors and improve their patrons learning. Libraries have also reported being able to use the results in grant applications. In our December webinar, we heard from an Ohio library that was able to apply for a dollar general grant using project outcome as its evaluation method to measure its impact on early readers of their summer reading program. We've also heard of friends groups being able to get STEAM funding for the library by using the survey results, as well as just hearing that using project outcome, libraries are saving time creating their LSTA reports. So it's saving time for funding you might already have, and it's also providing opportunities to gain new funding. And now I'm going to take a break and hand things off to Beth, who's going to talk about Burnsville's experience using project.com. Thank you, Samantha. As you can see, we're very small. We do a lot of stuff with the grade school, which is right across the street from us. Each class comes in once a week, and all of the students have library cards, and they're able to check out books. It's a nice way to introduce the kids to the library. We do keep their library card with us, but they get their little key tag cards so that they can come in with their parents and get books out. It's really a good program, and it really helps the kids because our library at the school is strictly an accelerated reader library, and it's only open on a volunteer basis. And so it's sometimes hard to get volunteers to commit to an ongoing program. The constraints we have are, we really have 1.03 staff. I am here Monday through Thursday, and then I have an assistant who is here Friday and Saturday. We are almost never here together, so that makes planning programs and doing projects kind of difficult. But project outcome is so simple. If we can take the time to do it, it really shouldn't be a problem for anybody to do it. When I first started as director in July of 2015, coming from a background of nonprofit community organizations, outcomes are huge and have been huge in that area for years. I worked four and a half years for the mental health association in the greater canal valley in Charleston, West Virginia, and our funding was contingent on showing the outcomes of what we were doing. And then I worked for two years as the staff person, but I was a volunteer with the organization. The name of the organization was the canal valley collective, and it was the HUD continuum of care coalition that serves homeless and chronically homeless individuals. And again, funding was contingent on showing your outcomes. So when I saw project outcome in an email, I already knew I wanted to start implementing surveys to try to figure out how we were doing. So it was actually wonderful. It came right at the right time for me. I just wanted to get a snapshot of what we were doing and how the public thought we were doing it, and that it would help us do our program development in the future. And it has done that. Sorry. The slides I had printed have the wrong notes on it, so I'm winging it. No problem. You're doing fine. Since outcome measures are new to our library and therefore new to our patrons, we've had some strange looks from people. When I asked them to fill out a survey, most of them have been extremely happy to help. And once it's explained to them why we're doing it and that the surveys are anonymous and they're going to help us improve and implement new programs, most people are very happy to help. So that's the bonus. And our basic computer skills class was six people who needed help with the basics of computers. The program was a six-week program, and we've had interest in holding another session of basic and then a session of more advanced skills. Unfortunately, with schedules, I haven't been able to make that happen yet. Our On Your Mark Get Set Read summer programming program was very exciting for us. Usually, in the past, our programs had been three to four weeks only, and all ages were together. After I took over as director, that first initial 2015 program was the same because we had one of our board members took it on because we didn't have a director at the time. So she just kept it the same, but in 2016 I knew I wanted to expand it, and we did. We did separate programs for children in early literacy because I have been the parent that has the early literacy children in the regular program, and it's just very hard to keep those kids up with the bigger kids. So we wanted to make it fair for everybody and give them their own program. We also expanded the program to eight weeks, and we did a pizza party at the end. We had 15 children registered for the program. Out of those 15, we had 96 that attended programs, each multiple, so that was a pretty good outcome for us. With the basic computer skills, I was very happy to see that so many people felt that they had learned something. I was terrified when we first got this idea because I was the one that taught the computer class, and I was very skeptical that I was going to be able to convey my knowledge to someone else. So it was a confidence boost for myself. It was also interesting to see that they were interested in more classes. The summer reading and caregiver results, it's always good to see that kids are reading more and wanting to come to the library more. I love it. In the last month, after school, I'm pretty much packed between middle school and high school students, elementary students in here. We have parents that come in when they pick up their kids from school, and it's just very reassuring and such a confidence boost. Parents are often an untapped resource when it comes to planning programs. The comments section of the surveys was great to be able to see what they wanted. For the teen and child surveys, we surveyed some of our younger participants. We actually had two outgoing kindergartners incoming first graders and an outgoing second grader incoming third grader that took our surveys. The only reason there was such a small number is because the last day of the program, I had the surveys printed and I just forgot to hand them out. But I read the survey questions to them and explained the scale and let them fill out their own answers. I in no way led them to anything. I helped them with their open-ended questions. Some of them asked me how to spell Ronald McDonald. Some of them asked me how to spell pizza. But that was the only help that was provided. They also asked why they were doing it. And you should have seen their faces light up when it was explained that it would help me make the program better for the next year. Everyone, even the youngest participants and youngest patrons, likes to know that they have a voice and their voice is being heard. The struggle we have here is whether or not people will attend our programs. In the year and a half that I've been the director, we have offered several programs. Interest is growing and that's wonderful to see. But what I found really encouraging was that the attendees of our basic computer skills class were asking for more. And it's always nice to hear that people enjoyed your programs, but to have a way for kids to participate in the survey was a wonderful way to get their feedback. Because if they're not liking the program, they're not telling their friends about it and their friends aren't telling their parents and we won't get them in. Without outcome measures, we would not have any way to know that parents are wanting homework help or peer-to-peer reading groups. Plus, these measures will be utilized when trying to obtain funding from donors and grant sources. It's always good to be able to put that information in a funding request. Even though this is a new idea for the library, I have been very involved with outcome measures in other areas with other jobs. I knew I wanted to implement it and this came along right at the exact right time. We, myself and the assistant, are going to be working harder to make sure that we have a good system to obtain the survey. I love that you can print paper surveys, but that there is also an option to do it on a computer. Although I have not used that option yet, I will be implementing it. The best thing I can think of, other than the increased ability to make a program productive or viable, is the funding aspect. If you can show funders outcome measures associated with your programs, the chances that equates to funding is increased. It shows them that you care about providing and improving your programs. We are planning on using Project Outcome for each of our upcoming programs. This is such an important idea. I don't want to let any grass grow under my feet. The data and information that we collect is already being used in grant funding requests. If we can demonstrate that our patrons are learning and experiencing it, it will only increase and strengthen our ability to receive funding. And I believe that's my portion. Thank you, Beth. That was great. I just want to go through quick updates and announcements from Project Outcome, and then we will take questions. Since launching last, no, not last June, June of 2015, Project Outcome has seen a lot of participation in a variety of programs and services measured. We have libraries in all 50 states registered, including libraries from DC and Canada. And here's a quick snapshot of our current numbers and the libraries represented. Over 48,000 patron surveys are currently aggregated within the Project Outcome system. The more libraries participate, the more outcome data will be aggregated and the field will move toward outcome measurement as common practice, which has been the goal of our project all along. We also sell a regional training workshop that will bring an expert trainer to your cohort or region for a deep dive training into Project Outcome. So if you receive state emails, we have about 15 trainings booked. So there might be one coming to your region, so look out for that. If you're not sure, you can also contact us and we can let you know where the upcoming regional trainings will be. We also host free monthly webinars. This month we will be focusing on financial literacy programs and how to measure their impact, kind of getting people ready for Money Smart Week in April. This webinar will be on March 16th and registration is open now. You can go to the PLA website under online learning to find all of our archived webinars as well as our upcoming webinars. We will also be hosting a free pre-conference workshop at the upcoming ALA annual conference here in Chicago. So if anyone is planning to go to this conference, the registration form can also be found on the PLA website under conferences, I believe. PLA and Project Outcome will also be presenting at the Association for Rural and Small Libraries Conference. This will be our third year attending ARSL and it's a great opportunity for PLA to connect with libraries that might be outside our membership. So if you're planning on attending ARSL, look out for us. As I briefly mentioned before, Project Outcome is currently building its own custom survey portal. This new system is launching May 1st and will allow more flexibility and customization for survey design and management, including the ability to add up to three additional questions to our pre-set surveys. This will also include custom report building. So stay tuned for these exciting updates if you do decide to register and use Project Outcome. Speaking of, hopefully what's next is that you go on to projectoutcome.org, you register your library for free, you get started viewing our training resources, you decide which program you want to measure, you schedule a survey in the portal. Once you have data, you're analyzing your data and using it in the data dashboards and your ready-made reports. For communication, you can join our Facebook group, follow us on Twitter and also engage in our community of practice, which you can find at the bottom of our resources page. This is a new feature added to the Project Outcome website and you can access it, like I said, from the bottom of the resources page. This is available only to Project Outcome users. And it's a place for you to ask questions and help others out by responding to their questions in an easy to follow discussion form. So this is where that kind of peer-to-peer sharing and learning can happen. And now I'll open things up and see if Krista has any questions for us. Yeah, great. Thank you, Samantha and Beth. Beth, I'm glad you were able to hold on through the whole time so far. Earlier today, there was some bad weather coming through the West Virginia area this morning, so we weren't sure if Beth would be able to stick around for the whole thing. But it looks like we're going to be okay so far. Fingers crossed. If anybody does have any questions, please do type them into the questions section of the GoToWebinar interface and I'll grab them for our presenters. Yes, I was actually wondering about, we have a couple of libraries who registered today from Canada. And I was wondering, and there's a school library too. Yes, a middle school library, that's what I was looking at here. I was interested to see that it is available for Canadian libraries. Is it available for anybody or is PLA specific to where in the world people could be coming from to actually visit? Great question. So the tools are available, and by tools I mean the survey portal and the data dashboard. The tools are available for U.S. and Canadian public libraries only. I mean technically they're available for state libraries. We have state access, so if you sign up your state library staff, you actually will gain access to all the participating libraries in your state, which the states have really liked. But school libraries, you know, academic libraries, research libraries, we don't have the capacity to build out the tool to accommodate those libraries at this time. I think they might have some specific things they want to look at too that maybe, that are different from what public libraries are needing to ask them. The whole system kind of revolves around having a specific library key that ties all of your data together. Everything works and talks to each other between the dashboards and the survey portal and your IMLS data is imported. So right now just the capacity that we have, our focus is really on public libraries since we are PLA. But anyone can register on the website. And by registering it on the website, you get access to all of our resources, so you can kind of take what we've done and adapt it to what you need. The survey is just publicly out there that you can do. Yeah, it is publicly out there. We want anyone to measure outcomes. Part of our initial grant with the Gates Foundation is to just have outcome measurement become that common practice. So whether or not you can use our actual tools, you can adapt our questions and adapt them for what you need and measure your own outcomes. It just wouldn't be in our specific survey portal. But the surveys are available on the website itself under the resources, so you can kind of take those and create your own surveys. Right, you just wouldn't have the built-in tools that are all there yet. Right, yeah. Makes sense, yes. We have a question. The Library says we have been participating in project outcomes since the beginning, but have not used it as systematically as we would like. Don't. She says when I clicked on the data dashboard, I was recently redirected to the Global Gates Foundation website and I could not find the old data dashboard. And do you know what's going on with that? Any suggestions for them? Yeah. Well, email me if you're still having that problem. So we work with a company called Community Attributes and they build and host our data dashboard. And Community Attributes also built the Gates Foundation Global Library Atlas. So we kind of took what they built for the Global Libraries Atlas with the visuals that they were already using and the mapping function and we kind of incorporated it into project outcome. So if you were redirected, that was probably by mistake or if you don't have any data entered, the data dashboards don't display anything for you. But email me if you are still seeing that error message. That should not be happening. So let me know. You can email info at project.com.org and it can help you with that. There you go. Info at project. That's what I was going to ask. I know for this was a few weeks ago, if it was during this timeframe that was happening, people were seeing the Global Libraries Atlas when they were signing in and they were like, what's this? This looks cool. But technically, no. It's just our host website. It's like a sister website of ours. Yeah. She said thanks and that's exactly what was happening. Maybe just check and see if it's still happening now. Right. Hopefully something would have been changed. Yes. We were having a bug that day, I think. Okay. Another question, Samantha. You mentioned peer data. Can you tell us where the data comes from? The peer data. How's that? How do you decide who's the peer? So right now we don't have a breakdown of peer to peer as in like by library size. The only comparison pieces we have in the dashboards is by state and then just nationally. So if you're looking at your summer reading results and you kind of hover over those icons in the dashboard, you'll see the comparison of anyone who ran summer reading in your state and then nationally. We're working on kind of broadening those comparisons so people can see more of who their peers are based on maybe LSA. So that's kind of coming in the future, but right now we only have the state and national comparisons. But if you want to talk to peers, you can go on the community of practice board and kind of see what other people are doing or kind of start your own threads if you have a group that you want to specifically work with. It's still new. We're trying to point people to that so that it's not always coming to PLA for advice or questions. But yeah, that's the only thing we have right now because we're trying to kind of walk that fine line between sharing data and keeping privacy. Right, yes. So when you go into the map dashboard, you're only seeing your library system. You're not seeing everyone. So we're trying to work on how we can better display at least participation without revealing results. So that's coming with the new updates in May. Okay, great. We have another question that just came in. This is LeBron says, I sit on a regional library system board. How can project outcome fit with a regional system? That's something they could use.  So what we would do is work with you, kind of create an account for you to sign up and have libraries be assigned to your group so that you could see all of their survey data and work with them on... So you're kind of helping them do their own surveys and then you can kind of act as administrator. So we do have that feature available and it will be... Can you compare with their peers and compare with each other in that group? Yes, so by group it would roll up to all of those member libraries, then you would see that comparison. Sorry, I interrupted you. No, that's okay. Yes, exactly. So if you're part of a group, then you kind of act as almost how the state view works. So you kind of are administrator and can see that high level comparison piece, as I said, across those member libraries. But you do have to contact PLA to get that kind of account set up because the system isn't naturally set up like that. That's very cool. So you don't have to just... It doesn't have to be independent libraries if you do have someone who you want to help out. Yeah, we have groups. We also have people who come to me. We have public libraries in our membership, but we also have other libraries in our membership. Can we do something for just the public libraries? So we'll kind of work to create a special group for them. And then in the new survey portal, it'll be even easier to create groups, and you'll have more flexibility in creating your own surveys to send out to your member libraries. It'll be really kind of easy and seamless. Yeah, and this would be different. You said that there is also state library access where the state library is not setting up some giant state group. It's just anybody who happens to be in your state using the state library and see their intro. Yes, so the states are already built into the system for groups we have to build them. So, yeah. Sounds good. All right, we're a little after 11 o'clock, and we did start a little after to begin with. Anybody have any last minute, desperate questions they want to ask Samantha or Beth? Type them into your GoToWebinar interface so we can get them answered for you right now. Otherwise, you can reach out to them elsewhere. You said the email address for anything, Project Outcomes, Samantha, again? Yes, it's info at projectoutcome.org. That's the help desk, a.k.a. me. So, I will receive all of your questions and can help you. And Beth's information is also on our website because anyone who presents with us is on our speaker wall of fame. And we have links to their slides or their presentations as well as their contact information. So, if someone kind of really speaks to you and you want to learn more about their experiences, you can go there and reach out to them as well. So, someone who you do think of as a peer, huh? Right, trying to contact each other, yeah. Sounds good. All right. It doesn't look like anybody's typed in anything right now, so I think we will wrap it up this morning then we're perfectly into our hour time. Thanks so much, guys, Samantha and Beth. I'm glad we're as early as John as I said. I had this project outcome on my radar since sometime last year. I don't remember if I saw it online in some journal I read, but I was keeping an eye out for this. And actually, for those of you who may know, I think it was Beth, yeah. Last week, last Friday, we did our big talk from Small Libraries online conference, also co-sponsored by ARSL, who we love here in Nebraska too, had submitted to do that, and we had a lot of proposals submitted to the conference, way too many that I could put in for one day, but luckily I was able to have you guys come on to our weekly show here, so glad to have the info. So I think we will wrap it up this morning. Any last words? Beth, you're still here with us, right? Yeah. My thing is, just jump in and do it. It really is very easy, and there really isn't a whole lot of time involved. I think that's a good thing. So much has been done on the website for you. That it's all pre-made, which that is a lot of the, we do surveys and evaluations of some of our sessions and workshops and programs we do here, and the setting up ahead of time is really the longest amount of time. Mm-hmm. Yeah, jump in as Beth says. All right, all right, thank you. I'm going to pull presenter control back to my screen now, Samantha's here, and I have here, once it pops up, there we go. This is the project outcome website that they're mentioning here with all the different resources on it. In our, when I do our recordings afterwards for Encompass Lab, I do capture all of these, and I have been saving it to our delicious account here, and so they will be the links to this, PLA's online learning library, the session that was for going to be at ALI on here, so all of this you will have afterwards. Here they have their upcoming events, other things you can attend, ooh, and there's our Encompass Live, awesome. So definitely take a look there on their site. All right, so that will wrap it up for today's show. Thank you very much, everyone, for attending, and as I said, here is our Encompass Live website. You can find our site at nlc.nebraska.gov forward slash Encompass Live, or if you just Google Encompass Live, we apparently are the only thing calling themselves that so far, yay. So we come up in the first search results to come up anywhere, but here is our archives. Underneath their upcoming shows is a list of all of our archives, and this is where today's recording will be posted. We'll have the link to the YouTube video, handouts, or not handouts, and we'll be on there as well. Later today, this should be available. I'll send you guys all an email and announce when it is ready for you to take a look there. So also, I hope you'll join us for our other upcoming shows we have. Next week, we have Planning for Successful Internships. Here at the Nebraska Library Commission, we do a Nebraska Library internship grant program, so we'll be talking about that next week about that coming up, about resources and information about internships in general, how you can do one at your library, how you can make that work, and get people into your library to learn more about what it's like to work in a library. So definitely sign up for that and any of our other upcoming shows. Also, Encompass Live is on Facebook, so if you are a big Facebook user, give us a like over there. I do post on here Minders About Upcoming Shows. I post on the fly for people who hadn't pre-registered. When our recordings were available, anything new that comes up about the show, I post here as well. So if you are on Facebook as we are, give us a like over there. Other than that, that wraps up for this morning's show. Thank you very much for attending and we'll see you next time on Encompass Live. Bye-bye.


Good morning, John. It's the final day of Pizzamas. This is not a complaint, but Pizzamas is a challenge. We have more demands on our time than we did in 2007, and even though we mostly clear out our schedules during Pizzamas, stuff creeps in. It's hard to do, especially because I always try to do like one or two really ambitious Pizzamas videos, almost as if I enjoy a challenge as long as it's achievable. But Pizzamas has been extra weird this year for me because I already haven't been working at full capacity because I've been transitioning out of cancer treatment into recovery, which is not insubstantial. Like I should definitely be clear, I don't always feel the way that I look like I feel when I'm on a video. Like this is a challenge. And recovery is just time consuming, whether that's like walking for a half a mile and then for a mile and then for two miles. It just takes time. But the biggest weirdness is that like I'm not going to go back to my jobs the way I had them before I got sick. For more than 10 years, I have been the CEO of two different companies during part of that time. I was the CEO of three different companies. And there's maybe a couple of things that you would know given that I was the CEO of three very different companies at the same time. What are those things? One, I'm a mess. I'm a mess. I'm not making great decisions. But second and more importantly, like I'm not doing this by myself. You might have noticed that also during that period of time, I wrote two novels and made a billion TikToks and spent way, way, way, way too much time on Twitter. I have never in my life as a CEO been a full-time CEO. And there's disadvantages there. Like I have had many times when I have felt bad that I couldn't do all the things I felt like I needed to be able to do. What makes it possible is a bunch of people who understand and believe in the missions of the organizations and great leadership from people who are not me. Two of my best friends, Dave, who has run DFTBA with me and Julie who has run Complexly with me. Like where those companies are now and where they were when those people started working at them is wild. So before I got sick, it was already becoming clear that as these, both of these companies continue to grow, it wasn't going to be sustainable in the long term. And it wasn't an immediate need, but like eventually it would be and you don't want to do a big change when the need is immediate. So we started looking for a CEO. We got some great candidates. I noticed a lymph node in my armpit. We talked to some of those candidates. I got a biopsy. We interviewed more of those candidates. I was diagnosed with cancer and by then we actually had a pretty good idea of who we wanted to hire. But it was like the day, like May 4th was the day I got diagnosed with cancer. I remember that because it's the day before my birthday. And the next day, my birthday, I went to work and had a long meeting with the people we were working with to recruit a CEO. So before we had this big meeting where we're trying to figure it out, I had to send an email to be like, Hey, just so you know, this is why I'm acting weird. But it was great. We handled it fine. It's all part of life. So the person we ended up hiring, LJ Dukowski, is amazing. I am so happy. Like while LJ had family members who were certainly aware of John and I and our work, she's had a bit of a crash course in Pizza John over the last couple of months. But watching her absolutely embrace the weird has been amazing. It's weird hiring a CEO because like different like high level leaders are going to be very motivated by different things. I think it's safe to say, and I don't think LJ would mind me saying it, that she could make a lot more money at a different company. Like DFTBA is not set up to grow super fast, to maximize revenue or profit and like get to the point where you sell and everybody makes a bunch of money when you sell. So we know that we're going to have to find somebody who is like more interested in interesting, you know, like more interested in weird, like I like professionally don't tend to really know what I'm doing. Like I don't have business training. I don't have goals that I'm chasing. I have like, that looks interesting. Let's go look at it. But obviously I have a lot of my identity tied up in these companies. I care a lot about them. I care a lot about the people at them. I worry about them all the time and I like that worry. So letting go of leadership has been weird. Even though it's not just that I thought it was the right thing to do, I also wanted to do it. Like at least a really big part of me wanted to do it. And the thing that was like holding me back, I mean fear, fear that it'd be like the wrong person, that it wouldn't work out, that, you know, that could be really contentious and hard. But also like I know that these companies are weird and special. And I worry that no one else could understand their weird and specialness like I do. And really the root of it is like, what if they make a different decision than I would make? Which like, yeah, yeah, that's how this is going to work. And that's good because LJ knows way more than I do about a lot of things. But also, and like our first conversations with LJ, something she said comforted me a lot. She said, I don't really want to take this job if you're just going to like check out and go do other stuff while I do this. Because part of the appeal is getting to work with you. And I mean, LJ flattery will get you everywhere. That feels really good. And I'm totally aware that like a lot of this is about feelings and it's about like how I imagine myself and how I want to be imagined. And that also means of course that I'm still very involved in DFTBA. We're probably going to get together some kind of actual job title for me at the company eventually. But there's a ton of stuff that DFTBA is still going to be able to do and I want to help it do it. And like, I don't know if you know, but like DFTBA does lots of cool weird things that it's very, it's like surprisingly big. So meanwhile, as all of that was going on, I was getting cancer treatment and you took over as CEO at DFTBA and complexly before LJ started. And I was like, I will start back up as CEO after I finished treatment because I didn't understand how cancer treatment works. I thought I would finish and I'd be like, all right, put me back in coach. And I would do that if I felt like I needed to. But the reality is that the recovery from cancer treatment takes time and I didn't realize that. I don't know, this is all very new to me. The reality is that Julie's been running the day to day complexly for ages anyway. So we have promoted her to acting CEO with the acting being for like, if at some point Hank comes back, then Hank can come back. And obviously I'm also not leaving complexly. Like there's so much still to do there. I think maybe we're going to put me on some special projects and I can do smaller things.  The teams at both of these companies are great and I love working with them. And I'm very glad that I get to continue working with them. And if there's anything I know about me, it's that I will fill the time up. Like I'm not worried about not having stuff to do. I had like two bizarre but not terrible business ideas during chemotherapy. And then before I finished radiation, I was on stage doing cancer standup comedy. I have received a wildly outsized amount of support and opportunity over the years. And there are certainly ways that I have helped and enabled both of these companies. There are also ways that I have been a limitation for sure. And despite my part-time leadership and also I hope a little bit because of my part-time leadership, both of those companies have really thrived and are doing really amazing, cool things. In large part honestly because of the support of this community. Like neither of those companies or VidCon could ever have existed without this foundation to build upon. I don't know. I'm just like really lucky, which is a weird thing to say a few months after you get cancer. But I am. Also you should have seen Catherine's eyes when I told her that I was going to be stepping down from executive leadership at both of these companies. Best anniversary present ever. Oh also I told her that today was my last Pizzamas video so I could shave my mustache and she said you don't have to. What? Anyway, a thing about getting a lot of support over the years is that you understand how much support matters and you want to turn that back and support others when you can. Which is why for over 10 years now half of the advertising revenue from the Vlogbrothers channel has gone to grants to small educational internet content creators. It's just a lot harder to get into this game than it used to be so we want to make it slightly easier in a tiny way. These grants over the years have gone to creators like Alexis Nicole, Answer in Progress, The Financial Diet, Wendover Productions, Real Engineering, a little channel called Kurt's Gazette. Gotta grant the first year. If you're working on educational content on the internet or you know somebody who is, you can apply. The link is in the description to the application. It's for people who are currently making content mostly focused on long former short form video and the grants aren't huge but they can be helpful. They're usually between like one and three thousand dollars. They sometimes go higher than that. And since you made it to the end of the video, do what you will with this. We ordered too many of a few things. This actually happens every year but now we have a CEO who's actively trying to solve problems and LJ is like we don't want to have some of these things in the warehouse for a whole year. Like for example, a hundred of these don't take up like a small amount of space. So we have steeply discounted a few items. They include the rug, which is now a mere fifty dollars. Cheats of John, the earrings and the oven mitt, which I thought was gonna sell great because they're wonderful. All those are steeply discounted right now and since you made it to the end of the video, if you use the code Hank sent me, that will give you an extra five dollars off on any or all of those items. So if you would like to commit to the bit, now is the time. John, this pizza miss has been wonderful. Thank you so much for being my brother and for being a temporary CEO and for helping and supporting me. Thank you to Drawfee for making a Pizza John shirt. I love this is my favorite personally. Teamwork though is the top seller, which is not what I predicted. I thought I was gonna be this one. I thought this was gonna be number one. This is number two. But who knows, it could still switch. There's still time but not very much. This year's Pizza John shirts are only available for the next couple of days and then never again. That's how it works. So you have to act now, John. I'll see you when I see you.