Good afternoon, everyone. Buenos tardes. So welcome to Creating a Culture of Empowerment. My name is Todd Neampirk. I'm a digital strategist and partner at Four Kitchens. If you have any follow-up questions or if you want to touch base after this session, that's my email address. I'm on Twitter at Todd Ross. I warn you in advance, though, it's mostly bad puns. I don't know how puns work in Spanish, but just trust me, they're brilliant English puns. So before we get started, I want to say a few words about the nature of this presentation because this is unusual for a Drupalcon talk. We're not going to talk about Drupal. We're not going to talk about the web. We're not going to talk about technology or even process. Instead, we're going to talk about how we apply an open source philosophy to management, to teams, and to people. So you're also going to hear me talk a lot about the way we do things at Four Kitchens. I want to assure you that this is not intended to be bragging. The presentation I'm about to give is the result of nine years of trial and error and many, many, many failures. We don't have all the answers. We're also trying to find them. But I think we're on to a few things, and I want to share them with you. And also, I want to be attuned to the different customs and cultures in the room as well. So a lot of the opinions and a lot of the experiences that I will discuss are based on things that we've done in the United States. So I recognize that the culture of work varies from country to country, sometimes even from city to city. So I do acknowledge that not everything I say will apply to everybody in this room, but I hope that everybody can take something away from this. So why am I here today? I am here to help people run better companies by making teams and leaders happier. So how many of you are business owners or leaders within your organization in some way? Excellent. That's about right. I want to tell you a story that was the inspiration, really, for this whole talk. So this is the two-snacks-a-day story. In 2000, I worked at a startup just before I started college. And this startup was like any other startup during the dot-com bubble back in 2000. They had a lot of money, and they had no idea what to do with it. So they thought that the best way to spend their money was on benefits for their employees. They did all kinds of unusual things. One of the many, many, many things that they did that cost a lot of money was they had a lot of snacks and drinks and things for everybody to eat and be comfortable throughout the day. A lot of companies do this. But at some point during the six months or so that I worked there, somebody sent an email. And the email said, everybody can only have two snacks per day. We used to have unlimited snacks and cokes and whatever it is we wanted to drink. But now suddenly we were limited to two snacks. So what do you think that did to the team? Everybody started wondering, are we going out of business? Why can't they afford for me to eat three snacks a day or four snacks a day? How much does a snack cost? Is it $0.50? Is it $1? How many people work at this company? 40.  That's $200 a week. That's $800 a month. Are we going to go out of business? Because of all the snacks we eat? Is $800 a month really going to help a company survive if they have to cut back by that much? Who's in charge of policing the snacks that we eat? Is my coworker going to tell on me when I eat three snacks? Do I get to carry over snacks if I eat only one on a Monday?  Is that OK? This stops making sense pretty quickly, right? So it changed the entire culture. People thought we were going out of business. People started telling on other people because, oh, Todd had too many snacks yesterday. Cost $0.50 for a snack, right? That's an example of a bad culture. That was a big mistake that was made at that organization. And towards the end of my time there, they were bought and they eventually went out of business. So let's start by defining culture. What do I mean by culture? Culture is defined as the behaviors and beliefs characteristic of a social, ethnic, or age group. So your company, where you work, is a social group. You spend about a third of your life there. That's about as much time as you spend with your family. But culture also affects how you think and feel about a group. When you think about a culture, you have some positive feelings and some negative feelings. So I want to run a little bit of an experiment right now. And I've never done this before. So bear with me. The experiment is, what is the culture of the United States? Let's just think about that for a moment. I'm going to show you some images. And I want you to think about what culture means and how these images make you feel. How do those images make you feel? Like America? American. What else?  I put some kind of disturbing images in there. I kept it pretty light. But what are some other thoughts? What do you think of when you think of the culture of the United States? Very what? Very familiar. Where are you from? You're from Columbia. Anything else? Focused on ego, egocentric. The class struggle is very much a part of the culture. One last one. Yes, sir. Dichotomy. A dissonance between the image that's portrayed and what's actually happening in society. These are the feelings that we have around the culture of the United States. But this culture has many layers. So we focused here on national culture. But there are many different layers to what we define of this culture. There's human culture, for example. Psychologists know that we have certain universal emotions, facial expressions, ways that we express ourselves, body language. Some of it is divided into nations or other smaller groups. But for the most part, a smile means you're happy. And a frown means you're sad. Psychologists have tested this all over the world, even among some tribes that have barely been contacted. So there's some universal human truths within a universal human culture. We also have a culture around our language. So there are many countries that speak Spanish, many countries that speak English. It spans multiple nations. Neurologists lately have found that there's an assumption we have about how language works in the brain that is wrong. Common sense says that our language exists to express our thoughts. We have thoughts, language provides them with structure, and that's how we communicate them to one another. It turns out that's actually not true. It's the opposite. If you don't have language, you do not have thought. They have found people who were raised in terrible situations where they did not have exposure to language. And it turns out that they did not really possess thought as we know it. Language physically shapes and alters our brain. So it stands to reason that the way your language works changes your brain. That the grammar and structure of Spanish creates a different brain than the grammar and language and structure of English or German or Chinese or Japanese. And that's a kind of culture that spans borders. Obviously, there's national culture. When we say culture, this is usually what we mean. We talk about nations. You also have local culture. Bogota is different than Medellin. New York City is different than Los Angeles. They're very different places. Some cities walk fast, and there's a lot of stress, or there's good public transit, or there's not. You look at a city like Los Angeles. You have to drive to get anywhere. And then there's the culture of the self. We are all individual. We all have different ancestry and family that influences what we, as individuals, behave like. So we each have our own individual culture. So here we are, the levels of culture. This is what I'm talking about when I say culture. I mean all of these things. Second part of this is empowerment. So what do I mean by empowerment? First, I mean the autonomy to decide what you will do and how you will do it. Second, the freedom to challenge without fear of reprisal. Sharing your thoughts and feelings, not being afraid to say something that somebody disagrees with, speaking your mind. The power to effect change. The ability to actually do something. To make something different. And finally, excuse me, ownership of outcomes. You cannot be empowered unless you are responsible for what happens. If you're not responsible for what happens, how can you actually be empowered to do anything? Empowerment comes from autonomy, freedom, power, and ownership. And ultimately it means controlling your destiny. What you are going to do professionally, personally. So why does it matter? Why is it important? Well empowerment makes teams happy. Teams that are empowered are more productive. They enjoy their work more. Quality of work goes up. They pay more attention to details. People want to stick around. You have higher retention. It's just a better place to work. Empowerment will make your clients happy. The positive feelings of your team and the quality of work that they deliver will feed into the work that they do for your clients. They're happy, they do better work, they deliver better results to the clients. And the clients pick up on this energy. They pick up on whether people are happy or sad in an organization. How people respond, the tone in their voice, the word choice they use in email. Additionally, if your team is empowered, everybody has the ability to do what it takes to solve the client's problems. So everybody can do something for the client. And finally, empowerment enables leadership. So for the leaders in the room, the company owners or the managers or supervisors, this means less work for you so that you can focus on bigger problems, long-term issues, planning for the future. If there are fewer things that you have to decide and micromanage, you can redirect all of that attention to bigger problems or more interesting things. So in a moment, I'm going to talk about very specific tactical things that we've done to empower our team. But first I want to talk about the underlying principles that guided those decisions. So very quickly, the guiding principles of empowerment. Build a company you would want to work for, not a company you want to control. So for the business owners in the room, why did you start a business? Why did you start your business? Anybody? Change the world? All right. Somebody made the money sign, so that's fair. Let's acknowledge that. To do what you want. To what? To work the way you want, to be empowered. To do what you want and do it the way you wanna do it. Be responsible for the outcomes. Whether you succeed changing the world, making money, whatever it is, right? You did it because you wanted to be empowered. So make that company that you want to work for. You have to empower other people. Give people control over their destiny. Treat your coworkers as responsible professionals. They are adults, they know what they're doing. They can collaborate with you. They're not children that you should manage. This leads to mutual respect. In the knowledge industry, in what we do, design, technology, development, people are very self-motivated. They had to teach themselves how to do this. There are very few schools that teach what we do. These are generally people who want the freedom of freelancing, of being contractors. And they often only get a job when they feel like they can't make it as a freelancer. But they still want that freedom. They want to do what they want, how they wanna do it. And most importantly, let them succeed or fail. You have to let people fail. It's the best way to learn and it's authentic. And finally, make trust the center of your culture. Trust facilitates empowerment. So you can't actually empower someone until you trust them. And they won't act on that empowerment. They won't believe you until they trust you to let them succeed or fail. This means that you, as the manager or business owner, have to be the first to volunteer your mistakes. You have to be vulnerable. You have to explain that you made a mistake, explain why, explain how you're going to fix it, and expect that from everybody else. So for the rest of the presentation, I want to talk about very specific things that we've done in order to try and achieve this. Now these are just a few. There are many, many, many other things that can be done. And I also wanna turn this into a bit of a conversation. So I want to leave plenty of time at the end for comments, questions, arguments, whatever. So I've already talked about language, how important language is, because language shapes the brain. It physically alters your brain. So the words you use are very important. It's a bit of a corporate cliche to say there is no I in team. But it's a cliche, and we all laugh at it, because most people who say it don't mean it. But it's still true. And if you mean it authentically, it's very powerful. Language is thought. It doesn't express thought, it is thought. So when you say we instead of I, you're demonstrating how you actually think, not just the words that you're saying. We are a team. I don't like the E word, employees. I don't like that word. It immediately creates a structure, and certainly never say my employees. You don't own them, they aren't yours. They are there because they want to be there. And if you keep treating them like that, they won't. So we, not I, team, not employees. Ask for input every day in big ways and small ways. People want to be heard more than they want to be agreed with. I'll say that one more time, because it's really important. People want to be heard more than they want to be agreed with. It doesn't matter if you, it does matter a little bit, but it doesn't matter that much if you agree with them. What matters is that you heard their problems. You really heard them. You listened to them and you thought about them. A lot of people also have an open door policy. That's great, you should have that. But having an open door is very different from inviting somebody into the room. You have to actively get people, bring them into the room and talk to them. You have to set aside time well in advance with an agenda to say this is what we're going to talk about. I want honest feedback about this, this, and this. So they have time to think about it and they're not put on the spot. Ask the right questions. Don't ask, how are things going? Well, what does everybody say? You're fine. How are you? I'm fine. What'd you do today? Well, not much. Ask specific questions that elicit important answers. What's the most proud you've been in the last month? If you could change one thing about this company right now, what would it be? What's the most difficult challenge you've overcome this year? Suddenly those answers go from everything's fine to oh, well actually there's all these things. Now that I'm thinking about it, now that you've activated that part of my brain, I have all kinds of stuff to say. And even when I was creating this presentation, I took a poll of our team to ask, are we actually doing these things? What are some things that we're doing that I didn't mention? What are some things that I'm saying we're doing and we're not? So even in this presentation, I got input from our team. Provide feedback channels that are accessible, non-argumentative, and genuine. So half of that is corporate speak and half of that are just really big words. So I'll break it down a little bit. Feedback channels, ways to communicate. They should be accessible meaning if you want to get feedback through email, you should be checking your email and responding to it. It should be easy for people to get ahold of you. There should be a system. There should be expectations. Non-argumentative meaning you're not gonna fight about it. When somebody gives you feedback, they should not expect a fight. They should expect it to be heard, not necessarily agreed with, but heard and understood. And in some way, acted on, even if that action is to do nothing. And finally, genuine, you have to really mean it. You can't just say, here's a survey, please fill it out and then not look at it. You have to actually look at it and think about it and analyze the results. What does it mean that these four people said this and these 10 people said that? It's really important. How many of you have heard about Toyotas and Don Cord? So Toyotas, the car company has a manufacturing line, right? And they have this cord that runs all along the line, the entire stretch of the line. Anybody can reach up and pull it at any time and it stops the line. The entire production line stops. Now, if this were, say, an American car company, like Ford, something like that, you cannot stop the line. The line keeps going. It doesn't matter if you saw something that's broken. You just say, hey, that's broken. I'm gonna put a note on it. Keep going. How many times have we done that on a project? Oh, that code doesn't look good. That CSS doesn't make a lot of sense, but gotta ship it, keep going. Somebody else did it, but I don't have time. More stuff is coming down the production line. Gotta keep going. So Toyota has this cord, the Andon cord. It's a Japanese word. What makes the cord powerful is that pulling it does not mean you get in trouble. When you pull the cord, supervisors do not come running over and say, why did you do that? What are you doing? You realize how much money we're losing? We're losing a million dollars an hour because you pulled that cord. They don't do that. If they did that, nobody would pull the cord, right? One person would do it once and they'd react like that and that'd be the end of it. Nobody would ever pull it ever again. That's what I mean by accessible, non-argumentative, and genuine. When somebody pulls the cord, a supervisor walks over and says, what can we do to fix this? What did you see that was wrong? How can we improve it? And they will keep that line stopped as long as possible to fix the problem. They lose a lot of money doing it, but ultimately the team is happier and Toyota cars are very reliable. Hire the right people. This is kind of obvious. So I'll dig into it a little bit. You want to make sure that the people you hire share your values. And those values should be defined.  So think about your company and what you stand for. What do you want your company to be? What are the things that define your culture? Write them down. Make them meaningful and genuine. Publish them. Put them online. Be held accountable for them. And now you have rules that help you decide should I hire this person? Should we spend money on this event? Should we accept work from this client? If there's ever a question, go back to your values. They should help you answer those questions. So in hiring the right people, there's a little bit of a test I like to do. I call it the airport test. And credit where credit's due, that name was invented by a guy named Greg Nattison who is still very well known in the Drupal community. The airport test is the idea that you are stuck in an airport for three hours, for five hours, for 10 hours, with this person. What does that feel like? You're waiting for your flight and it's late and you wanna go home and you've been working all week. You're tired and you're irritable. What is it like to be with this person?  Think through that. This is one we did a couple years ago. We went virtual. So this means a lot of different things. Some people call this distributed, distributed team or I guess those are the two big ones. Some people don't like the word virtual because it makes it seem like your company isn't real. It's virtual as opposed to real. What it means is you don't necessarily have an office and you don't all necessarily come to the same place every day. How many of you work for distributed companies or virtual companies, right? And of, and keep your hands up if you don't mind. And how many of you really like it? All right, did anybody's hands go down? I didn't see any. It works, right? When done well, it works. It's great. Why? Because you're empowered. You can live where you want. You can set your own schedule to some degree. If you want to move to a new city or if you have to move to a new city for family reasons or whatever, you have the freedom to do so. You don't have to give up your job to do that. That's really important. So it allows them a lot of flexibility. If you work with people who like to travel in unusual ways, like get on a boat and go up and down the coast for six months, but as long as they have an internet connection and a certain number of office hours where they're working and they're doing good work, what does it matter if they're on a boat or in a house or in an office? The fact that you can grant that person that freedom is incredible and very empowering. Very closely related. Let your team set their own schedules. Allow flexible work schedules. Where possible, consider something like an open vacation policy. Some people call this an unlimited paid leave policy. The idea is take as much vacation as you need when you need it. That's it. Nobody's tracking it. That seems a little crazy to some people, I know. But here's the thing, if they abuse that policy, they're not the right fit because they're letting their teammates down. So work with your team to schedule flexible time, to figure out when you can take time off and take the time off you need. That's very empowering. Again, many people in our industry want to be freelancers. They want to be contractors. Why do people want to be freelancers? Because of this. They can work only in the morning or only in the evening or four days a week, not five.  You should invest time and money in your team's success. Time and money. This means giving people a budget for professional development. Allowing them to travel to events like this, if you can. And the time off to have these conversations. How many people in the last two days, because we're almost at the end of the second day here, how many people have learned something here at DrupalCon in the last two days, right? Everybody, even if you didn't raise your hand, I know you did. That was worth it. That was worth it. You're going to take back the inspiration and the energy and the ideas and everything else that you learned to your organization. Whether you teach them what you learned or if you just go back with a better attitude, it was worth it. So allow them the time and give them some kind of a budget, if you can, to invest in their own development. This is a really important one for us. And I think this one actually scares people more than anything else. Encourage personal brands. See, he didn't like it. A personal brand is the idea that you are really good at what you do and you are known for what you do if you want to be known. And people recognize you and they want to talk to you. And if you like to write blog posts about what you do, you can do that. Some employers, some company owners get really afraid when people they work with start to become famous because they're afraid they're going to leave. And if you're afraid that the people you work with are going to leave because they're going to get too good at their job and too well-known, then there's something wrong with your organization. Because then what are you doing? You're deliberately holding them back. You're not allowing them the time or the budget to get good at what they want to do. You're stifling them. And that's a hard truth. People don't want to hear that. But if you're not allowing people to grow and allow them to leave, if they really do get too good at their job and you can't sustain them at your organization, absolutely you should help them find another place, another career because they're not going to be happy with you and their work will suffer. This is, we have an expression in English, I have no idea if it translates. If you love something set it free, right? So if you really want people to succeed, you have to let them go if they want. But the benefit is if you're the kind of organization that allows people to do this and celebrates it, they will almost certainly stay because they're happy there. This helps eliminate the I am not my job divide. In what we do, knowledge, work, web, technology, we are our jobs. We actually are. We do take it home with us because we're passionate about it. And for some people that's not the case and that's fine. But for some organizations it really is a passion. And for people who are passionate about it, they want to build a personal brand. And one final thought, there's a lot of talk about work-life balance. So I want to work some amount and I want to have a life outside of work. And that's very healthy and everybody should have that. That's very, very important. But at the same time, I don't necessarily believe in work-life separation. I think that there are two things that should be in balance but they're not necessarily entirely separate. If what you want to do is innovate and do really cool things at your organization and change the world, you're doing it 24 hours a day even in your sleep. If that's the kind of organization you want to be, you need to be that. Own it. And if you don't want to do that, don't do that. That's okay. Adopt agile methodologies. Scrum, Kanban, extreme programming, parapro, you know, anything that allows you to work on a project in a more agile way and work with your clients in an agile way. Built into the core of agile development is the idea that teams self-organize around projects and problems. They self-organize. The team decides what it's going to do. Who's going to work on it? When it's done, that's a lot of empowerment, right? You're giving the team the control over what's going to be done in the project, to some degree. Also, very important, built into empowerment is the idea of getting feedback, right? Always ask for feedback every day. Constantly understand what's working and not working. Agile methodologies has this idea of retrospectives or retros, so you do a retro at the end of a period of work, like a sprint. What went well? What did not go well? How can we improve? You're asking those every week or every two weeks, always getting feedback. Don't tell people how to do things. Give people goals. Set goals. Let them figure out how to do it, right? Do what you want, how you want to do it. The technology and the tools that you use at your organization should be decided by your team, not a manager. The team is the one using them. So if they don't like your project management tool, if they don't like JIRA or name a million others, if they really don't like it, work with them to find another and switch, because it's something they have to do every hour and every time they go into that thing, they hate it.  Let them pick it. And if you try a new thing and it doesn't work, throw it away, do something else. Yeah, it's painful. But the team sees how much you care about their input because you genuinely do. You've invested in a new tool or a new technology. You spent all the time migrating to it. You're working in it. It didn't work. Well, okay, on to the next thing. Set boundaries beyond what the person currently feels. This is a little complicated. Here's what I mean. Some people take power. Some people need to have their expectations set beyond what they currently believe in order to find that power. So some people will say, you know what, I'm gonna learn this thing over here because I wanna get better at my job. I'm gonna do this thing I've never done before. I'm gonna take over this story because it's not getting done and somebody needs to do it. I'm gonna do it. Some people are like that. Some people are not. So if you really want to build a culture of empowerment and demonstrate that everybody can do this and it's not just up to the people who go out and take it and then they get praised for it, create opportunities for people who don't take power to receive praise. So tell them, hey, there's this story in the backlog, this user story in the backlog that nobody's touching. Why don't you give it a try? You can do it. Oh, but I don't really, I don't know much about that. I bet you could learn. Give it a week. I bet you could figure it out. I know you can figure it out because you figured out a whole bunch of other things in the past. They'll figure it out. And they will volunteer again in the future. They will take that power in the future because you have empowered them. A lot of business owners and managers take a long time stepping out of the work because they feel like, oh, but I know all this stuff and I've been working with this client for two years and but I don't know, this person's not really up to speed on SaaS or on whatever technology you're using, except the fact that they actually know more than you do because you hired them and you should be hiring people better than you all the time. These people who know more about your work than you do because you're a manager now, you're not actually doing the work anymore. You're doing a different kind of work. You're doing this work. They know more about it. They're in the weeds. They're working every day with the client. They should have a lot of say in what the company does because they see what clients need, what projects need, all the new technologies that are out there. They're trying new things. They're failing and succeeding. Ask them about their vision for where the web is going or technology. They know more about it than you do and finally, you have to constantly reaffirm this. Constantly exemplify, pursue and praise empowerment. When somebody is empowered, you say, hey, they did this really cool thing. Point it out, celebrate it. If people don't feel like they're empowered or they say they're not empowered, you have to figure out how to solve that and constantly work with them. Reinforce, reinforce, reinforce. And one final thought about this. Asking for help is not a sign of weakness. It's a sign of strength. It's a sign of knowing what you don't know and being comfortable with that. So people should never feel like, I don't wanna ask for help because then it's gonna seem like I don't know what I'm doing or everybody else is really busy or because what's gonna happen is that thing that you need help with, it's not gonna fix itself and you're not gonna be able to fix it because you need help doing it. So it's going to fail and then everybody's gonna wanna know why. If you ask for help, you're actually showing that you are strong, not weak. So this idea of empowerment changes everything. When empowered, a person benefits the company, not the other way around. Here's what I mean by this. Traditionally, the company is this big thing with lots of little people, right? The company is big and powerful and there are lots of little people there. And the people, they all rely on the company for everything. The company gives them a salary. The company gives them a job. The company gives them professional development and career growth. But when they are empowered, the company becomes smaller in a way. In this case, the people are allowing the company to succeed, the people are giving to the company and that is in fact what's happening. We spend a third of our lives at work, a third of our lives at work. We are giving a third of our lives to our jobs. They give us something in return, but the company, the job, should be more grateful than we are, always. That is empowerment. And empowerment makes everyone happier and better at their jobs. The teams are happier, they like their work more. Quality goes up, people stick around, they stay, they don't quit. Clients are happier because quality is up. People aren't leaving. Clients never like it when a client leaves, or when a person leaves a project, right? And you have to explain like, oh, our lead architect just left, we're gonna have to replace them and it's gonna take them a month to ramp up. And finally, leadership. It's less work for business managers. It actually takes a lot of the work out of their hands so they can focus on the future and facilitating and empowering their team. So this change is worth it. Briguntas? I don't know if there's, Mike, I'll repeat the question. So, Todd, I completely agree with you. What I would say is some of the challenges that we have also talking from your perspective about culture. The culture in Latin America is a little bit different, of course, than from the US. So some of the challenges to apply some of the concepts of like free time or open door schedule or take any vacations when you want it. It's hard to imply when the culture has been very different, especially in a culture like the US that is very eccentric and individual versus the culture of Latin America that is a bit more like group-wise. Do you have any suggestions or any experience on how to handle that based on your distributed team that has to do with it? Very good question. I don't have a lot of suggestions, actually, on how to implement this in areas or cultures where a lot of these things are difficult. I'll give you an example. So the idea of, I'll put it this way, these are challenges in the United States. This, the stuff that I've talked about now is really unusual. In the United States, this is not a typical thing at all. This is very forward thinking. When you think of something like the open vacation policy, people can take off however much time they want. There's some really unusual legal problems with that in the United States. So everybody's experiencing this, specifically the state of California. The state of California is a problem in lots of ways. I love California. The state of California. The state of California has a policy that says if you leave your job or if you're fired, your employer has to pay all accrued time off. So if you have saved up three weeks of vacation, they have to pay you three extra weeks of salary when you leave or are dismissed. When you have an open vacation policy, is that infinite time? This is a real legal question in California. And I don't think it's been tested in court yet. But when people have, Netflix has this policy, for example, they're kind of famous for it. When somebody leaves Netflix and they're mad, are they going to sue Netflix and say, you owe me for the rest of my working life? I don't know, we haven't tested that yet, right? This is a new policy, this is brand new. Also in Finland, there is, I've heard this sort of second hand, so I may be ruining the story, but there's some kind of policy that says there is a minimum amount of time off that is enforced by criminal law in Finland. And I think it's five weeks.  So guess what everybody does, they lie. All the business owners who have to work because this is the job, they have to lie and say, yeah, I took five weeks off, or I took two weeks here and three weeks there, and they were working secretly on their own company, right? It's kind of silly. So these are big ideas that are new for everybody, really. In places where it's especially difficult, which may be Columbia, I would suggest that you should see this as maybe some inspiration or a way to start thinking a little bit differently about work, or find the things that you feel really touch you emotionally, like what you like about this, and see if there are ways that you can maybe work with the system you have, or even in some cases work against it to change it. Because there are things even in the United States that we have to work against. Here's another example. The airport test that I talked about earlier. Can you sit in an airport with this person for five hours and not hate them? I wrote about this. I wrote about this, I forget where, on some business magazine, and I was all excited. So I got published in this business magazine. It's really cool. Like, oh, time to read the comments. I made the mistake of reading internet comments about my own article. And luckily, this being a business website, it wasn't like YouTube. They were actually thoughtful, but they kind of ripped me to shreds on the policy, because they said that the airport test sounded like a way to enforce discriminatory hiring practices.   And then all of these other biases come into play. And you know what? They're kind of right. They're kind of right about that. So I've headed shift the way I think about the airport policy, because it's more about values now than it is getting along with somebody. It's do you share the same values? Do you have the same goals? That's what you wanna think about. Not necessarily can we tell each other the same jokes and we'll both think they're funny, you know? Any other questions? Just one moment, we're gonna pass a mic to you. In Argentina in 2001, there were social upheavals. Many factory owners tried to close their factories. And they would, what the workers did in many cases is they took over the factory. And they made the factory work on their own and became successful within certain limitations. This has happened in Chicago with a glass company and in France also I believe with a watch company. In that sense, that's a kind of sense where I wanna tell you exactly what I feel. I think that it's a very courageous presentation, very fresh ideas. And if your company is implementing half of this, you know, where's the job application for? But on the other hand, given the fact, I think what we're heading for in this presentation is to optimize a difficult situation. And I have to say in true honesty, my impression that I wanna share with you is that there's a dialogue going on in all of the language statements. It's a dialogue with the manager, which is fine. I mean, nothing wrong with that. But what we're doing here is trying to design a benevolent dictator. And the root problem will not be solved until there's no dictator. So it has to be, I don't know, there are experiments in cooperatives, there are experiments in different kind of designs of companies.  But since there is a fundamental difference in that the manager or the owner owns the means of production in the knowledge factory, it's an insurmountable, I mean, there's a problem there. So in any case, I just wanted to share that with you. I do think I've just given a presentation on lean approaches and evidently I agree with you. I mean, these techniques have to be put into operation because that's the state of affairs right now. So, but I did get that feeling that the dialogue is with the manager only. And even though I'm owner-director of a small group of people, I didn't, I mean, why would that be, why isn't the dialogue also with the people who work at the company? So, but in any case, it's the rules again. So I just wanted to share with you my feelings about listening to you and thank you very much. Well, thank you. Well, a couple of thoughts. There are a lot of interesting things that we could talk about as it relates to power structures and hegemony and all of those things. And that's the kind of stuff that I love to talk about. But I'm gonna focus it very tightly on just this presentation and just some of the ideas we talked about here. Because I will say that I'm, there's a lot of political ideology that goes into that as well. And I think it's all well-founded. And I would describe myself personally as sort of a weird mix of like a capitalist and an anarchist. So I know that seems maybe like they're at odds, but that's just what I'm like. So the audience for this presentation, it's interesting and I think it's very valid feedback, very constructive, critical feedback to hear that you feel that it's more directed, it's almost exclusively directed at managers.  But at the same time, I want to have enough information in here for people who are not in a management position or an ownership position to make them think, hey, I'm not being treated fairly. This doesn't, you know what? There's some suggestions I could make in my organization to do this. So I think maybe casting that, like there is definitely some intent, but it may not be clear. So I take that to heart and I hope to expand on it in the future. We're about one minute over time, but I'd like to take one more question and then we'll wrap up and then I'll be here and I can talk for a few more minutes. So yes, sir. So I have two questions. The second one I will just do it there, but I guess this one is, you have all these philosophy and this, you know, in a way really a manifesto, right? For people how to be a colleague with their people, they might not show supervised or vice versa. But when you see the examples of really bad management, I mean, kind of like even like dictatorship style, like Steve Jobs, right? And these companies are so successful or, you know, keep naming these people like Bill Gates and all of them have this kind of like dictatorship style and like my way or the highway, right? And they are so successful.  Yeah. I think about that a lot because I think a lot about Steve Jobs in particular and it's interesting that you mentioned him as an example because he's the opposite of all of this. And yet Apple is the wealthiest company in the world. It has more cash on hand than any company has ever had and they continue to innovate and do all kinds of impressive things. It's, I don't know yet, I don't know enough yet about Tim Cook to know what his style is like. I think they're still kind of feeling that out over there. I imagine he's probably a lot like Steve Jobs though because why else would he hand off the company to anybody other than someone like himself? I think there are basically two kinds of companies in the world. There's the Steve Jobs kind of company and there's what I think we're aspiring to be here. And I think that looking at the, and this goes towards your question from earlier, looking at the political questions around power and structure. In our world we live in a very capitalist world. So companies like Apple through a dictator like model and I'm using that word very loosely. They succeed really well within the system that has been built. At the same time I think that some people are actually really motivated. I don't understand it myself because I'm not motivated in that way but I think some people are genuinely motivated by fear, by being constantly pushed to go farther and farther and farther until you feel like you, look at how athletes train. Athletes train because they are constantly pushed beyond their limits and then they heal and they get better and they keep going. If athletes were trained because their coach said, hey, that's good enough, I'm your buddy, let's go get beers. Would they ever make it to the Olympics? Maybe not. Now that may not be a fair comparison. But some people are motivated by being pushed really hard and really far. And those people really like working at Apple. I know people who work at Apple and they love it and they love that kind of stuff. I would not do well there at all. What's interesting though is that companies like what we're aspiring to be aren't usually very big. We're pretty small. And we get into some really interesting political discussions now about, well is that because the system that we've built that is even outside of business, that is social and cultural, is it because companies like this can only succeed so far? Because other companies are eating up the rest of the pie, right? Like the Apples and the rest. It's almost a philosophical question. And it's something that I do think about a lot because there are things about Apple and Steve Jobs that I find very inspiring. Their design, how forward thinking they are, how most people think, let's figure out what the client wants, what the consumer wants and give them that. Or let's take something that they have and make it better. But what Apple has done is they've said, there's this thing that you don't even want yet. And we're gonna make it so good that it's gonna be the thing you have to have. That's kind of incredible. Because it goes against all other logic. They can't look at surveys or reports or sales numbers or anything when they try to do something completely different. Anyway, lots of things, I could start talking forever about this because I find it fascinating. So I'll wrap up now because we're a few minutes over. Thank you so much for your time. I really appreciate it.


on behalf of TechSoup, we would like to welcome you to this webinar today. As many of you know, TechSoup is a nonprofit 501C Free, just like your organization, but our mission includes supporting nonprofits working with technology to make this a more equitable planet. And we do this by hosting a catalog of technology products priced specifically for nonprofits from major tech brands. And one of those brands here with us today is Questimpro. We're fortunate to have Christine Welcher, she's the US director for nonprofits at academic institutions at Quest Pro, and she brought some friends along. They're going to tell us how nonprofits are using surveys and data collections to make smarter, more informed decisions, which they turn into equitable goals. My name is Aretha Simon, I'm the webinar producer here. I'm just gonna show you how you can engage in the chat room. We are recording this. You'll get this replay within four to eight hours. If you would like to ask a question, feel free to type in the Q&A. I know a lot of you still type in the chat. We have plenty of people that will be here to answer your questions. If you learned something cool today, why don't you share it with us on social media at TechSoup. And as always, when we have our webinars, we would like you to be courteous to everybody. I'm gonna go ahead and turn this over to Christine.  Emma, Nancy, Christine and Joe. I'm gonna turn it over to you. Thanks so much for being here. Thank you, Aretha. Thank you for having us. And thank you to TechSoup for being such an awesome partner with QuestionPro. As Aretha said, I work for QuestionPro and my job is to work with nonprofits and educational institutions in the United States. And so today I am joined by three wonderful organizations that are doing amazing work. So first we have Emma Nolan, who is from the Ark of Pennsylvania. And I'll let her introduce herself in just a second. Then we have Nancy Shirley. She works with Dream Partnerships. And Joe Crawford, who works with Arizona Adaptive Sports, Water Sports, sorry. So, Emma, do you wanna go first? I'll let you take over. Yep, thanks so much, Christine. I'm gonna go ahead and share my screen and just introduce the organization. All right, so as Christine said, my name is Emma Nolan. I work for the Ark of Pennsylvania. I am the manager of health initiatives. And just to give us a pretty broad overview of what the Ark of Pennsylvania does, we promote human rights of people with intellectual and developmental disabilities and actively support their full inclusion and participation in the community throughout their lifetimes. But what I do personally is work on healthcare initiatives and public health for the intellectual and developmental disability community. So currently we're working on statewide efforts, which include investigating, educating and reporting on challenges and solutions that Pennsylvania's disability community and its diverse populations, such as the intersection of rural communities and racial and ethnic minority groups have in accessing appropriate information and help during the COVID-19 pandemic and how all of this intertwines with healthcare access and the ability to receive quality healthcare services. So that's all I have for my little quick introduction into the Ark of Pennsylvania, but I am excited to be able to answer any of your questions later on after introductions about QuestionPro and about data collection. Wonderful, thank you so much. Nancy, would you like to go next? Here we go. Sorry about that. A little delay and get myself on mute and video here. So my name is Nancy Shirley. I am the Executive Administrator for Dream Partnership. Dream started out like many other disability organizations from a parent, our founder, Donna Parton, had a daughter, has a daughter, Demi, who was graduating from high school and Demi saw all of her other, the students around her applying for college and she approached her mom and she's like, where am I going to college, mom? And Donna was like, well, I don't know, Demi, let's find out. So investigating Pennsylvania, there was one program throughout the state for students with intellectual disabilities. So that's how Dream came to be. So Dream Partnership, basically the mission of Dream Partnership is to develop scholarship and post-secondary education opportunities which lead to independent living and employment for students with intellectual disabilities throughout Pennsylvania. Our mission is that Dream Partnership's leadership and post-secondary education programs will empower individuals of all abilities to work, live and contribute to a better society. So you'll see on the picture there, that's Donna on the left-hand side and Demi in the blue shirt. So Dream Partnership was established in October of 2012. We're a nonprofit organization, which means that we write around grants, we do not get any federal or state funding at this point. We provide technical assistance to colleges to start programs for students with intellectual disabilities. In addition to that, we fund those programs to get started. We also provide scholarship opportunities to the students who attend those programs. Those students that are attending our programs do not usually qualify for other scholarship opportunities. And so far we've granted over 234 scholarships totaling over $967,000. Outcomes for students, basically is employment and independent living. You'll see on the picture here, we've got Bruno and Nigel, the three gentlemen there are graduates from Arcadia University program, and they've actually started their own DJ business. And that's one of our fundraisers. They come every year and they provide the music for us. So 10 years ago, like I said, there was only one program, Temple University, throughout the last 10 years, we have expanded and developed 11 programs throughout the state. There's a nice list for you. I won't go through that. We also provide scholarships to the students that attend those programs. And last year, we were able to expand our network to include other programs that we haven't funded, but actually exist throughout Pennsylvania. So we were able to expand our program to include Gwen and Mercy University and St. Joseph's University as well and grant scholarships to those students. So there you see lots of pictures of our scholarship awardees. And I also wanna say that having students with intellectual disabilities on campus is life-changing for those students, but it's also life-changing for the students around them, their peer mentors, the teachers. It's enabling the students that don't have disabilities to see the capabilities of those individuals.  They're gonna be the advocates for those individuals in the workplace. So that's really the point is to have all of the students, regardless of disability, to learn together and hopefully be employed. So there's internship programs available at these programs. There's job shadowing in addition to that. There's resume building, soft skills building, the students audit classes and they can attend any classes that are interested, interests to them. So that's my little spoiler about dream partnership and we're in Camp Hill, Pennsylvania. Thanks, Christine. Thank you so much, Nancy. And now from Pennsylvania, we're gonna go all the way down to Arizona. Joe, take it away. Joe, you on mute if you're speaking. There we go. How's that? Give me just one second. Perfect. Okay. Let me go straight to sharing my screen and share. So we are an adaptive water sports program and we serve children and adults with disabilities ages three and up and all different disabilities. Our base is of course in Arizona and we provide adaptive water sports for people with spinal cord injury, brain injury, stroke, Parkinson's. We have an ALS day, a kids day, amputee day. So it's really come a long way and many of our lakes, specifically one is very wheelchair accessible. So we have for your list to help people get in and out of the water because as you know, access to recreation is a huge challenge. So what we do is we break barriers across the board and what another part of our ministry is to bring families back together. Sometimes after a person sustains an injury or is born with one, they go to therapy and the rest of the family goes to the gym and our program brings everyone together. So here's a mom with her kids all going tubing and she has a spinal cord injury. So that's another part of the fun things that we get to do. This is adaptive water skiing. We take people with significant types of mobility issues where they can't lift their arms up or then they can't stand, but they can sit and ski and it's very exciting and it's also breaking barriers and attacking some of those sphere components to have after you have an injury and you don't wanna get hurt again. So we're super excited about doing some of those types of things. This year we've served over 350 athletes around the state of Arizona, at least plus their family members. This wake surfing is huge. I don't know if you guys are involved in water sports in your area, but like I said, we do an amputee day and wake surfing is huge. So there's some wake surfing and then the big thing now is seated wake surfing. So again, breaking barriers, getting that adaptive equipment out to these athletes and so they can have some fun. There's some wakeboarding that we do. And for folks who aren't able to walk long distances, it's amazing how they can continue to wakeboard. And so we also do that for the blind and visually impaired as well. We do kayaking around the state of Arizona. Again, for all disabilities in some of those rural areas where folks wouldn't have the opportunity to come down to our home lake. So we received a trailer last year. Now we go to Lake Havasu, Lake Powell, Swarov Lake, so we're hitting more people with disabilities around the state of Arizona. And that's ultimately our goal is to get more and more people out and active. What we know is that 70% of the people who come out to our programs initiate doing something new with their life. And that could be driving, asking somebody out on a date, taking a college class, looking at a new career. And then of course we do boating and fishing. So that in a nutshell is what we do. And then we have the most amazing volunteers that are out there. And we've had volunteers that have been with us for almost 25 years. So when people come, they like to stick around. And then of course training the new generation of young volunteers who have a lot more energy. So we're super grateful for all of that. That was awesome, I love the pictures. Thank you. So yes, thank you ladies. We just wanted to kind of give you all a background on these organizations. So you kind of understand when we're talking about collecting data, how they might be using it and things like that. So without any further ado, we're gonna just kind of jump right in. Just a couple of things. One, yes, this is about data collection and how nonprofits are using data and insights to plan for the future. And these organizations are kind of in all different experience levels of collecting data. Some are just starting. Some have been doing it for a while. So please feel free to share your questions in the chat box or the Q&A. We'll be monitoring both of them. And I just have some basic questions to kind of get the ball rolling here. So I'll be the moderator facilitator and I'll ask some questions. And then I will just leave it up to Emma, Nancy, and Joe to answer as they will. So the first one is pretty basic. Why did you start collecting surveys? Why did you feel the need that you needed to have this kind of data to track and how are you using it basically? So who would like to start? I can start us out. So at the Arc of Pennsylvania, we're currently working on an initiative where we're gathering information and data from statewide participants. So we're doing some focus groups and some individual interviews, but we wanted to also have a format online with that online survey to reach as many people as possible and really ask those questions that are important to our data collection surrounding healthcare disparities for people with disabilities and have it as accessible as possible for a wide range of people. I'll hop in there. I also work with Emma at the Arc of Pennsylvania as a consultant as well. So we started in addition to the data collection piece of it, the Arc was also using question pro for surveys that actually started out after we had a conference. So we had the advocacy and policy day conference and what would normally happen is they would eat or they would snail mail an actual paper survey to find out how the attendees enjoyed the different sessions that we had, things like that. So that's kind of where it started for me. I just went out basically and looked, what kind of survey software is out there. And that's how I stumbled upon question pro, which was so easy to use, the free version is amazing. It gives you so much information. I mean, we had started to utilize our website for that type of survey, which was great to collect data. However, it didn't give us the analytics. So we had to manually calculate everything and question pro does that for you. So we've used it in many, many different ways, strategic planning, surveys of the chapters to find out what their demographics is like, racial demographics. We've used it for training, post-training evaluations. I use it for dream partnership. Every year we do a survey of the college programs, which asks what type of, how many people do you have? What type of programs are you running? Do you have residential programs? How many students graduated? How many students have graduated and have work? How many internships? Things like that. So it's great to be able to collect that type of data and go from year to year to see the difference between the programs and to get the counts too. So when I apply for a grant, that information is invaluable that I can go back and I can say, we've graduated this many people this year and this percentage is working. Excellent. Cho, you actually just started with us recently. Do you have anything you wanna share? You know what? We are very excited about it. We're as much as we've been doing the program for years, it's been under different umbrellas. So for us, we had just hit our fourth year mark and we really need to start collecting the data of pre and post, you know, with the many of our athletes, it's their very first time and quality of life is a huge issue as I'm sure we're all sitting here recognizing that. And so for us to be able to really measure not just your quality of life, but your health and all those disparities that we've talked about and access and what keeps you from being active, you know, and how can we help you overcome those barriers? Then that helps us with the grant writing and that's the goal there and also us from a service line too much in one day. Is it just enough? How are we doing serving our consumers and their families? You know, and is this a blessing or are there times where it's more of a hardship to get there or was just kind of understanding what we can do to better serve our amazing athletes who come with all ages. The other piece that we do is the educational component.  Our oldest athlete is 88. So we really, you know, when you look at grants, it's for the aging, you know? And Sidenote, I'm 54. I'm just about ready to get in that world, you know? So anyway, that's kind of where we're at to hit those bigger areas that sometimes are overlooked because there isn't a quantitative, yes, when you're over 55, when you're over 65, you still play and you should. That's what we're doing. Excellent, excellent. And so kind of along the lines, we have a couple of questions coming in from the Q&A. Someone has asked, what avenues have you found to get responses from your community? Have there been more effective ways that you've been able to get people to answer your questions? I know some of you mentioned something about, you know, like pre and post programs that usually works, but are there other ways that you've been able to get responses? For us, I think survey response rate is always a big issue. I think anyone who's collecting data has to think of that going into it. But having more like individualized emails sent out to people with the link to your survey, posting it on as many social media platforms as your organization has, I know we utilize Facebook, we have an Instagram, we have a Twitter. So just really reaching out to as many people as possible and make it as accessible as possible as well. We have our survey available in English as well as Spanish. So really just taking that extra step to make it as accessible for as many people as possible will really increase your survey response and it will be appreciated by the communities you're trying to reach. Yeah, great points, Emma. Thank you. And I know that we've worked with you, especially with the Arc of Pennsylvania and trying to make it as accessible as possible for everyone to take. So I can speak a little bit to that in the work that Emma and I are doing with at the Arc of Pennsylvania with the Department of Health. One of our initiatives is to track disparities and to get input. And we've been able to utilize question pros specifically throughout our Zoom meetings. So we schedule Zoom meetings with individuals, we do listening sessions with them to get information and feedback and we utilize it during the meeting. So we actually use the format, the question pro format to go through the questions that we're asking them and ask participants to fill it out while we're zooming. And at least if they're not able to do it while we're in the meeting, then they're familiar with it so they can go back in a later time and complete it then. And we've had a pretty good success rate, response success rate doing that as well. And some of the features that we've used when we do breakout sessions in some of these larger meetings is video. So Emma will go through and video herself speaking to a specific question, giving some background information that you can embed right into the survey and we play the video and then go right into the answer. So that's a really nice feature that we've been able to utilize, especially when you're depending on who your audience is, video sometimes gets across a little bit better than a long document does. Great. And then kind of along those lines as well, we've had a couple of questions about, how do you develop a survey so that you're not putting too much stress on the participants? So how do you make it more engaging? How do you make it maybe more fun? Obviously the video is one way. Do you all have any other examples? I think starting out with maybe an easier question, not going right into it, kind of having an introduction paragraph or a couple of bullets of what you're looking for in terms of responses for the survey and then making it a little personalized, ask them their name if that's the sort of information you're looking to collect, what region they're from in your state, again, if that's relevant to your data collection. And then we use a lot of qualitative methods. So we'll ask a broad question and then we insert that paragraph answer format. So they're able to type in what exactly they wanna say and it allows the data collection to be a lot more personalized. And I think it helps people really elaborate on what they wanna say in a survey. And that also makes it a little bit less intimidating because they can say as little or as much as they want to. Great, Joe, do you have anything you wanna add? No, I'm just really kind of getting you to all these different opportunities but I am learning a ton from Emma and Nancy. So, excellent, excellent. Well, that's what this is for. So another, go ahead. I can jump in real quick that QuestionPro has a great rating system too that does the smiley face to the frowny face and that makes it really easy for individuals just to kind of choose their mood versus dislike and agree, totally agree. That type of thing, the smiley face is nice. They do up thumbs, down thumbs. Those types of options within the program are really nice to use. It's different than most just yes, no kind of, questions. Christine, you're on mute. Hi, I'm on mute, there we go. Rick actually asked a question earlier about CMRs or CRMs. This isn't exactly about that but are you using any specific software to keep track of your participants and your clients? Really, we're just using QuestionPro. I mean, you can export the information into Excel and each survey is unique. It can be unique depending on what type of information you're getting and the program makes it really easy to export. So you can export the information into a PDF, a Word document, a PowerPoint presentation. They give you the graphs that they do it. The program does so much. The analytics is amazing. So right now in neither Dream Partnership nor the Arc of Pennsylvania, do we have it tied to a CRM system? We pretty much just utilize QuestionPro as the database. Great, that's good to hear. It's working well for you. Kind of along the same lines with the secreting surveys. We have a question. Has anyone on here used multiple languages in a survey? Go ahead. They're just asking how easy is it? Was it able to, like, I suppose then once you get the answers back, how were you able to interpret all of that? We had ours available in English and Spanish and it was pretty simple on that part. We do have a translator that we work with for larger documents, but this was pretty easy to translate and understand what was going on in those data collections. But I know Nancy, you had a little bit more information about the translation services that we used. Yes, so I will say the multiple language is really nice because you literally say which languages do you want this to be interpreted into? And it does the automatic interpretation and then when the person brings it up, the very first question is what language do you want to use? So they get to choose Spanish or English in our option. Now I will say that when they answer in Spanish, the answers are in Spanish. So we were able to use other translation services. We utilize Microsoft 365 and on the online version of Word, you can have there's translation available. So we were able just to copy, I believe Excel does as well, has that translation for you. So you're able to either copy it or just export it into Excel and then do the translation there. And it translated it well enough that we were able to get the idea of what that person is saying. And then we have a couple of questions just about the platform in general. So I'll just kind of quickly answer those. There are templates.  So you can go into any of those templates and kind of look around to get ideas per questions. You can edit any of those questions and answers to fit more with your organization. So yes, that's available. Is the website mobile? Yes, we do have mobile keys. So with every user, even with the free one, we have one mobile device key that comes with it. So you can sync it to a tablet or your phone or something like that. And then you can go out and collect data without Wi-Fi and then come home or back to the office and sync it at the end when you do have Wi-Fi again. So that works out really well for people, especially if they're doing any kind of community functions, booths at fairs, things like that. I don't know, has anyone used the mobile? Not yet? No, it's there and we do have people using it but it's not, I think more people would like to use it. They just haven't quite figured out how to use it yet. And then I guess the next question is, so survey design, you all are asking questions, you've been doing surveys. How did you kind of come up with how to design a good survey? Are there resources that you've been able to find online or in other places? I think just going into it with the background of what you're looking to get out of this survey will help you a lot in the design and the question formats that you wanna use. As I said before, we really collect more qualitative data and more like life experiences type of answers we're looking for from our participants. So it's really helpful to like before you even begin the survey, knowing what type of data you're looking to collect. And then the question pro website really is helpful in the design of your actual survey itself. And it can show you like what each question type will give you in terms of like data analysis. So that was really helpful when trying to figure out the format that best worked for us. Yeah, I think the first stage of any survey is to write it down, just brainstorm, okay, what information do we wanna get and then go from there. Question pro also has the option for the logic. So if like say I have a, you know, I like or dislike something is the question and the person indicates dislike, well, you can have the logic question to say, okay, if this person answers this way then give them the option to do this. So it would be like an open-ended question like, okay, why did you answer the question that way? Which I think is really helpful, but I agree with Emma 100% it's, you really have to think about what type of information you wanna get. Open-ended questions are great if you're asking somebody to relay the story but if you're asking them, you know, how do you feel about something or a yes or no question? Sometimes those are easier. So you wanna try to design the survey to give you the information that you need versus an open-ended question. Cause if you do an open-ended question, your Emma can attest to this, you're sifting through all of that information, having to code it where if you have a yes or no question, it's really easy or if you have a ranking question or dislike like or I agree with or I disagree. Those type of questions are a lot easier to see the value of, okay, so, you know, out of 180% of the people chose this or said this or agreed with this statement. So I think it's important to format it in a way that the system is doing all the work that you're not manually going through and sifting through paragraphs of information trying to rank or trying to code things and group things together manually. Let the system do what you need it to do. And also when you design it, you wanna make sure that you're gathering the information that you could use maybe later. So if you're putting it out to a group, it may not be necessary in that context to know what county they're from or what state they're from or what zip code they're from, but you may be able to utilize that information later for a different purpose. So getting that information upfront is much easier than going back and trying to ask somebody to do another survey which includes that information. So to kind of think ahead of, okay, how am I gonna use this information? Can I use it in a different way maybe for something else? And then, you know, having them answer those questions upfront is much easier to get the data upfront than it is to go back and get it. Great, great answers. Okay, I'm just gonna keep going with the Q and A because we actually have a lot of questions coming through. This is kind of a, I'm gonna put these two together as sort of a two-parter. One, the first one is what information have you found is most useful when you're writing grants? And then kind of in that same vein, how are you using the data that you are collecting to enhance your strategy, to work on donor relations, probably for grants as well. Kind of a two-parter, but I'll just throw it out there for whoever wants to take it. I'm sorry, can you repeat the first question that you said? Hold on. So the first one is what kind of information are you finding is most useful when you're applying for grants, if you have applied for grants? And then in that, when you're getting this data back, how are you using it then? Are you using it in a strategic way to make decisions about your organization, where you're gonna go? Are you using it to help in donor relations? Are you using it for grants? Things like that. Got it, thank you. So it really helps us in terms of like thinking about your methodology before you're even thinking about a grant or if you're working on some deliverables or strategies, really looking at your methodology and what you're going to be doing with your data analysis and how that informs your decisions. I know for us, we're working right now on some solutions work pertaining to healthcare barriers for people with disabilities and that will inform the Pennsylvania Department of Health as well as other stakeholders across the state. So it's really helpful to be able to keep all of our information in one place and to be able to analyze it as much as possible to really see what themes and what subjects pertaining to solutions are rising to the surface and really gaining traction. So it definitely helps us in terms of strategies moving forward with our grant work. Right. Yeah, we've been able to utilize a lot of that, the information to support the work that we're doing for grants. Emma's been able to analyze a lot of the data because a lot of the information is quantitative having to go through and manually report through people's stories and to code that so that we can create a report for barriers and work in solutions, things like that. For surveys, we do surveys for all different types of reasons. So you can utilize like we used it for our strategic plan. So we're getting ready to go back to, okay, we're a year later, let's see where we are with that plan. So you can use them as progress reports basically, pre, post-test type of thing. I utilize it for dream partnership with our employment numbers and then also how many students were serving each year, the programs that we started serve each year. I mean, that's really helpful to be able to go back to that information and justify what we're doing as far as grant work goes. I think the pre and post type surveys, like the demographics of employees within the chapters. Okay, well, we took that last year, let's see where we are again this year and to utilize the information that way. And I know Emma touched on having the information all in one place. Question Pro makes it really easy to just copy a survey and modify it and to be able to import information as well. Emma did some of that when we have individuals that we called and went through the paper survey and then we were able to upload that information into the database so that it's all in one spot. That was really, really helpful. You can utilize that as well, just walking through somebody on the phone using your paper copy and literally doing it on Question Pro and filling it out for them. Like that's another way to complete surveys as well. So it all depends on what type of information you're looking for, what the situation is. It is really easy to do multiple surveys without reinventing the wheel. Thank you. And Joe, I know you mentioned some funding and working with some grants. How are you using or how have you used surveys in the past, even maybe if it hasn't been with Question Pro? So what we have done in the past is like I had mentioned earlier about their quality of life and how that changes.  So that's another area we'd like to cover just for our own, not necessarily for grant opportunities but because I think most people recognize that but the other piece too from our standpoint is to hit more people around our state and really kind of go towards those guys and just sometimes you think that there's a need there and sometimes there isn't. And that's what we need to find out is there really a need in these different parts of the state or are there two people? Is it worth it for two? Well, to us it is worth it for two. And then how you write that in a grant to let people know that those two count just as much as the others, the other 300 that we serve. So that's I think gonna be the fun part about the survey is really kind of reaching those guys out there to get them active. I think that'll be the blessing of the survey in general. And you mentioned sometimes you ask questions that are just for your internal knowledge. Are you then using those answers to make decisions about like how you structure programming in the future or I think you kind of touched on where are you gonna have expanding your services? But... Well, and even to that point too, Christine, like what types of services when we go there do you want? And then are we reaching all those people in that area? How can we better reach them? And then the other part too is what do they want just because we have these seven water sports programs doesn't mean they wanna do all those. Sometimes it's amazing.  You know what I mean? So trying to find out what they want and what their barriers are and making sure that we have the right of equipment and then making sure that all of our staff, our volunteers are trained for whatever their needs are because we pull into other lakes that are just not accessible at all. And we know how to overcome those and get people in wheelchairs on the sand. We have portable sidewalks. I mean, just knowing the equipment that you need to get and that's important too because every lake is so different and every adaptation is different too. So to make sure we have that wherever we pull in the driveway. Yeah, I'm sure it takes a lot of organization and planning upfront for some of these expeditions that you go on. And I guess kind of to the point that when I asked the question, if you've used other surveys, software besides question pro, I can close my ears or whatever. But I mean, feel free to tell people how they compared or what you liked better, that kind of thing. Go ahead. So I've utilized, sorry, Christine, is there something else? I'm just gonna ask if anyone actually had. Go ahead. I've utilized SurveyMonkey, Google Forms and QuestionPro and then more quantitatively, I've done like our studio, SPSS, if anyone's familiar with like biostatistic software and each I think offers different things depending on again what your organization is looking to collect and what type of data analytics is necessary for whatever grant or project your organization is aiming to work on. I will say QuestionPro and SurveyMonkey are kind of similar in their setup, like the format of the questions and the type of questions you can input, but the analytics is a little bit more advanced on QuestionPro and it just allows you to really dive deeper into the data and separate it out and really focus on some individual responses. I just like not to really sell QuestionPro, but I definitely think those analytics are a little bit more advanced on this platform. Yeah, I agree. I know the question came in about the Google Forms versus QuestionPro, the analytics is the answer to that. I mean, it's amazing that you can just, get the information and they give you the charts and you can do a pie chart, you can see a line chart, you can see a bar graph, just the options of what the analytics look like and Emma in the project that we do based on the health disparities was able to, we're collecting data from different, from across the states, but then also specifically to counties. So Emma was able to have all the information for the entire county, but then also break it into separate groups.  We love the word clouds. That's an amazing feature and you don't have to work at it, it does it itself, it's phenomenal, but to be able to have the entire dataset, but then also be able to break it up into different regions was very helpful in the project that we are working on so that we can actually see the difference between the regions because as Joe said, you may have, you may think that you have the answers, but for one region it might work fine, but then another region, they may need something totally different. So it's interesting to be able to separate out the different segments and see the data based on region, which does make a difference, especially here in Pennsylvania, urban and a lot of rural counties. So the rural needs are a lot different than the urban needs are. Great, thank you. And kind of along those lines, you are working with some sensitive information, I'm sure all of your organizations deal with people, different levels of medical needs and things like that. So how do you address confidentiality? How have you kind of worked that in and how you use the program or if you've thought about that in any way and had to deal with it? So I know something that we do is always state that their name will never be attached to their answer. And that's pretty just set in stone across most data collections. You always wanna make sure that your participants know that even if you do use the information that they provide, it's never gonna go back to their name. None of their identifying information is going to be released with their data that they are inputting into the system. That's a really important step in confidentiality and maintaining that relationship with your respondents. But other than that, I think it's pretty set in stone like the confidentiality agreement and it's pretty secure in the data collection and just all those confidentiality. Great. And also all of you are working with differently-abled people and so we've had a few questions come through on the chat and Q&A about accessibility. So we do have accessibility. We are 508 compliant, but have you actually used it yourselves yet? I personally haven't used it myself. I know we work really hard and to make sure everything's in plain language. So in easily understood by as many people as possible. And then a lot of our respondents do utilize screen readers. So I'm sure it, I think it has been tested for our organization. I think Nancy can speak a little bit further on to that. Yeah. So the assistive technology that most people would use would be the screen reader or voice to command, you know, voice to word. It is functional. We love the video option, which would give, you know, video and then also sound. So rather than reading it, you hear somebody saying it. And I know that question pro, at least in the research, the research level does have the capability to record answers as well. I know we were talking about that, you know, initially when we looked at the research model, which is pretty interesting and exciting for our demographic. Unfortunately, the limit, the time limit per recording was a little bit too short for our demographic. So, but there are definitely, it's definitely accessible on different levels. Yeah, there's actually, so when you're setting up your survey, when you go into designs, there's actually a theme that's called accessibility. And when you click on that, it turns everything into black and white so that you don't have to worry about like color blindness, for example. And then anytime you add a graphic or a logo, anything like that, you'll see a little person pop up that reminds you that you need to put in alt text for screen readers, things like that. So, yeah, some of that stuff is already built in and we'll help you with it. But those, that theme especially is what our accessibility team has been working with. And we actually have like a whole department that's been working with that to try to make surveys more equitable and for everyone, for people to design them and use them and as well as participants when they're taking them. Let's see, I think. So, I think we have like two more questions on here. If I'm missing any questions, please just put them back in because I've obviously missed them. Demos are demos available. Yes, I will put my information in the chat and you feel free to reach out to us. And then I will also put in the link to TechSoup because they have a great offer with us. You can get two licenses, I think for $97 for a year. So that works out really well. And then let's see, what is, what was that other question I just went past it? Okay, panel, could you please give us any advice regarding like general education that you went through to create surveys, developing methodology for your survey projects? I don't know, did you have some training before you took this job or are you just kind of learning it on the fly? So I definitely utilized that demo service that Christine was just talking about. It was very helpful in giving you like a really good introduction to the features that QuestionPro offers and the analytics you can perform within that. But besides that, I mean, training, I didn't come into this role knowing QuestionPro, the format at all. So it was really new to me, but there's so many features on their website that guide you like step by step of what you should be doing for your survey. And it really helps to, again, just know what type of questions you are looking to get, whether that be quantitative or qualitative data and what type of answers your organization is looking for before trying to set up your survey. So that's my word of advice for survey formats. I concur totally with Emma. I did not have any kind of background for surveying. I, again, I was looking for something other than our website to use for a survey link. I came across QuestionPro, so easy to use. You just really just choose the type of question that you want. It's easy for the analytics on the background. I love, or on the back end, I love the idea that you can just export to Word or PDF or PowerPoint presentation. And it gives you all that analytics. It gives you the graphs. Like in my case, the respondents are all, they all receive their own individual number, their own identification number. So for the programs, the colleges, in a Word document, I'm able to export all of that. And then the answers come in as respondent number, blah, blah, blah, respondent number, blah, blah, blah. And in a Word, I can just say, OK, find this and replace it with Temple University. And it makes it really easy to utilize, to copy and paste it. It's super, super user-friendly. It does not take a lot to get started. You know, I'd like to add to that since this is the new part for us as we're continuing to build and grow. I was on the phone yesterday with support, you know, on the live chat and the support system has just been amazing. So it's really nice to know that you can just go directly and get assistance right now, you know, while you're working on some stuff. So that I was really appreciated about that. That's good to hear. Our team works hard at that. And we do have people all over the world. Our offices are all over the world. So even when you email like one of our general emails, someone is always answering, someone's always live on the chat. So it might be a bot like the very first question just so they get you to the right team. But there's always somebody behind the scenes checking on that and monitoring it. So even at two in the morning, when you're trying to do something for the next day, we have someone there that can help you. So we're pretty much done with, we're coming up to the end of the hour. So I just really quickly, the last thing I just kind of want to throw out to each of you if you could just take a couple of seconds. You know, what is the best advice that you have for organizations who are just starting some sort of a data collection program? I think my biggest advice is to not be intimidated. A lot of times the words like data collection and data analytics can have a lot of anxiety behind them and it can be a daunting task to really try and reach a large number of people and make sure that your questions are relevant to your study and are asking this right thing in the right way. So I think just doing as much research on your own about the format that you're looking to do and the methodology that you're looking to do for your project or your grant is really important. And talking to as many people as possible as well attending webinars like this is the first step in really learning the application that you're looking to purchase or utilize and getting as much information as possible before making your survey. I have to agree with Emma again. And really just formatting it, the design is important as far as formatting it into questions that are easy to answer and that you'll get information on the backend and the analytics, so yes or no question. Something that is very specific. Try to stay away from the open-ended questions if possible but if you also need that information, certainly utilize it and there are ways to work that into the analytics as well. But really the design upfront, brainstorm with people, find out exactly what type of information you really wanna get and then just go from there. The program itself is so easy to use and I love the feature to be able to edit the link as well. So your survey link can be changed to whatever you need it to be. So you can include staff survey in the link or the Arc of PA listening session in the link. You can make it unique to what your survey is. You know, the only thing I would add is taking your time while you're writing the survey, recognizing that time is our most precious commodity and it's very valuable. And so take your time when you're making the survey, you're asking your consumers to take their time to make their programs better, your program better.  I don't know if I'm just speaking on just myself, you know, overseeing an organization but just take your time and then that way you benefit their time as well. So everybody gets blessed right there. Thank you so much everyone for joining me and for my wonderful co-host here today. It was great. I got to learn more about all of your organizations too, which is always kind of fun for me anyway. Because sometimes I only see the business side of things or the questions or, you know, how to do a certain thing. So it's fun to get to know you a little bit more. And thank you again, TechSoup for having us. I don't know, Aretha, do you have any last things you need to say? No, I'm gonna say thank you. This was amazing. Emma, Joe, Nancy, Christine, thank you so much. You all have a great day. Thanks everyone. Thanks for taking your time. Bye everybody.