Hello, everyone, and welcome to our regular Insight IBM Research YouTube series. I'm Katja Moskvich, the head of comms at IBM Research Europe. And as usual, I'm here in our Zurich lab in Switzerland. And today we're going to have a bit of a different chat. I know lately we've been quite often talking about quantum computing and quantum is obviously very exciting and I'm still here next to our also model of a quantum computer. But today we're going to be talking about something just as important, it's security. And as we all know, of course, security, you know, doesn't matter if it's offline or online is crucial, right? For our world. So here at IBM, we've got many, many security researchers working in different labs. And before I actually introduce you to our amazing presenters today, I'd like to remind you all to please actively use the chat in YouTube as usual and send us your questions because behind the scenes here, we've got experts in the chat who will answer your questions in real time. Some of the questions we will also answer live during the show. So let's go to our presenters. We've got joining us today, Mark Stocklin who heads security research at IBM Research Europe. We've also got Ian Muloy, department head security research at the IBM TJ Watson Research Center. Then we've got Srinni, and I apologize in advance if I mispronounce your last name, Srinni Tumalapenta. I hope I got it right, CTODE IBM Security Services. And finally, later we will have Yuval Lapidot joining us as well, but you will not see him on screen just yet. So I will introduce him properly a little bit later. Welcome, everyone. Great, great to see you guys. And oh, yes, and we will have another researcher also joining us for the second part of the show because the show is not only on cybersecurity managing threat, but also on protecting yourself, like choosing the best password to protect yourself online. And that's going to be Julia Hesse. She's going to join us a tiny bit later. But for now, let's talk about cyber threats. So, Mark, first question for you. Can you tell us what's actually the role of AI in security in general? Yeah, thank you, Cathy, for the very first question. First of all, good morning, good afternoon, good evening, depending on where you're located in the world. Actually, I want to start saying something that probably all of us know that security is a cat and mouse game. And that means that we as defenders need to be always awake, regardless of where we are in the world, and need to understand how the threat landscape changes, new technologies, new attacks. Cyber criminals are typically the first ones to pick those up and leverage these threats and exploit them. And this is why we at IBM Research Security are really looking always ahead and trying to be vigilant, constantly looking for new threat vectors and how technology can affect us and can affect the security and the posture of organizations, of companies, of individuals. Now, Cathy, I really to connect directly to your question about AI and security, I think it's fair to say that over the last decade, AI technologies have become extremely powerful, also very much available, right? Open source of lots of technologies that are regularly available as libraries for people to use and also very widely adopted by many organizations. And this has a lot of benefits, but also comes with a number of challenges that we need to be aware. Like in the past, when we had new technologies that were introduced, think of web technologies or cloud or mobile, attackers have also leveraged those and have also played some malicious games around this and have used these technologies to attack us. And AI here is really no different. It is really just a technology and it uses our manifold. And I would like to quickly talk about two duels that we see, duels in the sense that attackers, and this is some things we've seen over the last recent years, have been using AI quite a bit to also boost their attacks, right, to use AI to strengthen their attacks, to harden their attacks and automate some of their attacks. On the flip side, right, what we also see, and this is a domain that is very active and we're researching very actively on and we'll talk more about this, we can use AI also to boost and improve our defenses and to automate our defenses, our detectors, et cetera, et cetera. The second duel, and we'll also quickly touch hopefully later on this webinar on this, is the use of AI itself has been very much adopted widely in many industries.  This is what is called adversarial AIs, that attackers can attack AI. And we look into also how we can counter this adversarial AI and strengthen AI itself. So in short, right, I think very promising where we believe is AI technologies to help detection. The first part I was talking about and how we can also respond to cyber threats quickly thanks to AI technologies in threat management. Cool, thank you Mark, I liked the slide. Well, it's gone now, but I hope everybody noticed the little guy here, there in the hat. So I guess that's the typical image of a cyber criminal now. But I don't think all of them would wear hats, but you know, cool. Well, so a question for you, Srinni. You're part of MSS responsible for a platform of our IBM's managed security services business. So in other words, helping many IBM clients to detect and respond to cyber threats as a service, of course. So could you tell us what are the main challenges that the MSS team of security analysts is actually dealing with today like in your daily operations, so to speak? Great, thanks Katya.  We support multi-thousand clients covering the globe and all these are multinational clients having operations in multiple parts of the world. So we provide threat management, monitoring response to all these clients. By virtue of us enabling all the security and monitoring the security technologies, we collect about 60 to 70 billion logs. We have visibility to 60 to 70 billion logs per day. That's the magnitude we have. We have visibility to multi-vendor technologies, be it network, data, applications, identity devices. We have access to multiple vendor technologies. First and foremost, our biggest challenge was to harvest the data, to get the value out of that data. So first and foremost, the collaboration happened from there. So we have a partner with Mark and Ian very closely along with other team members. We got the members from research to get access to that data in a secured fashion, in a secure territory because we got to abide by all the regulations and compliance needs. So the data will not go out, but the members will come into the data in a secured zone and harvest the data. So that's the first thing we had to overcome as a channel. To address your point about what are the challenges that are faced by the analyst community, by the internal team members, it's about alerts. It's about alert volume, okay? In terms of the, if you take the holistic approach of how an analyst processes these things, they look at low value and they look at high value. A low value alert is about something that is not of any importance for us to take a mitigation, a protection on that. It is noise, it is generated out of informational alerts or some internal activity. So, and the client previously have asked us not to escalate on that, but to close on those things. So that's the low value alerts. It's repeatable repeated tasks that the analysts would go back and close it anyway, right? Not escalated to the client. And there are some high value alerts that the analyst has to look into it, spend more time, investigate, and then provide necessary classification, notification to the client and provide recommendations to the client, right? So what has happened, what we heard from the analyst is, it's about how they process the data, how they process these alerts. They spend five minutes to triage an alert, for example. Not every alert takes five minutes, but as an example, right? They have to spend that first five minutes to alert, no, look at the low value alerts as well as the high value alerts. It's not productive use of their time looking at the low value alerts. So what we have done as a means of applying AI ML, now listening to the analysts is to apply the system, the ML AI system as an analyst. It's an analyst by itself. The system now becomes an analyst. Look through that, all those alerts, classify them into low value and high value. Learn from the past behavior of the analysts. They went and looked at these alerts and they closed it. They associated that alert to an existing investigation that is already underway. They ordered to say, let's automatically notify the client on this particular thing. So we looked at that and parsed it and applied AI ML to that and then provided the benefit back to the analyst. Now the analyst doesn't have to look at those low value alerts, they focus on the high value alerts, spend more time on the high value alerts, okay? What is the bottom line for us? The bottom line here is we got to be fast. We got to detect those things faster, respond to those activities faster, provide very, very specific recommendations of mitigations. So that's one from a client outcome standpoint. Second thing from an analyst standpoint is about that alert fatigue. It's about their job satisfaction, okay? The job satisfaction is about our analysts say that there's a good day for them at the office is when they look at something, they look at it as something happening, an attack happening or some activity happening with the client at the client environment, notify the client about that. That's the value they provide to the client. They feel happy when they do that every single day. So about the quality of experience for the analyst, the quality of experience for the client as well. That's where the problems are enhancing and improving the quality. Thereby, that's where we focus on the analyst. Okay, cool. Thanks for that, Trini. So moving on, Ian, I've got a question for you as well. So could you tell us how AI technologies can actually help security operations teams to address difficult, so-called pain points? Sure, easily, Kadi. So there are many different areas in which we can actually apply different AI technologies to some of these challenges we've seen security. One of them is on feature extraction, this continuous machine learning. Having machines that would look at the data that we talked about would automatically be able to adjudicate them, come up with different features that are important discriminatory. Instead of having the analysts come up with custom crafted rules for a specific environment, we can have the machines actually get us, look at outliers, look at anomalies, things along those lines. One of the other big areas that we actually see AI technologies applying to security is on natural language processing. We use this for threat management, for looking at different threat feeds, vulnerability reports, texts on past breaches. So all these different analysts that might be employed who are going through doing these investigations, they'll leave notes. And they're easily notes for another human to understand say, oh, this was not a real alert, or this was not a high priority for the following reasons. And if we can actually have the machine understand that, make men learn and adapt and apply that to new threats as well. One of the other main areas is on this reasoning. So Trini talked a little bit about a threat comes in. What do we wanna do? Do we want to alert on this? Do we wanna suppress it and so on? And again, we can have different machine learning models that would learn and adapt from this, but then also be able to say, well, what was potentially the next step that an attacker might take? What should I look at now? What was potentially the root cause? And really we can start pulling all this information in automatically to start reasoning about the threats, what we have to look at, what we have to be worried about, so that when we actually provide that to an analyst, it's all done, it's there, it's automated, but we can actually come up with some recommendations about what to potentially do. And this kind of hits upon the fourth main topic, which is on automation. Having machines do what machines are really good at, pulling information, doing correlations, automatically executing some mundane routine tasks that the analysts don't actually have to do it. So really it's about making them more efficient throughout the entire pipeline. Right, yeah, thanks for that, Ian. Indeed, I would imagine that this manual work by analysts should be probably really time consuming. So what do you guys think, Mark? Maybe you could tell us what you guys are working on here at IBM to actually help security analysts to cut down a bit on this manual work and speed up their ability to respond to cyber attacks. Yeah, indeed, look, it connects very well to what Srinia was mentioning before about the fatigue and Ian mentioned the same, right? There's a lot of mundane repetitive tasks and we sat down over the last few years with the teams at security services, tried to understand, okay, how do they work? What are their procedures, their playbooks when they investigate a cyber incident? And we learned quite a few things, right? And they can, the findings and their approach can be summarized as follows. So first of all, when they have a new alert that they receive, they are going to investigate basically the threat intelligence. So try to understand what does the world know about this? There are maybe IP addresses or their files are being used or certain behaviors and they do some threat research around this and use different data sources. Secondly, they look into additional local data sources, right? EDR, data lakes, logs, and try to correlate these with the past. Maybe this alert is not just in isolation but maybe something's happened before. So they have some investigation strategies. Threat hunting is another typical word that can be used in that context to investigate the attack more, identify use root causes and maybe understand the impact extent. At the end, the goal is like Trini said, to identify, do I care about it? Do I escalate it? Do I need to remediate or can I close it? Is it perhaps a false alarm? And there are in reality a lot of false alarms that these detectors generate and cause a lot of noise. Now, what we have done, and this is really, I think where we are focusing on improving the situation is to systematize the approach, right? First of all, we researched on different types of tools and methods to automate the investigation and analysis of cyber events, meaning that we're trying to systematize both the threat intelligence. So you have heard Ian said, we're using natural language processing to extract information, to curate the information and to put it into graphs that we can reason over them. And secondly also to systematize the procedures, the approaches, how we reason over incidents and how we connected to worlds to automate and provide recommendations or even to automatically close some of these alerts very quickly because we realize they are similar to some of them that we've seen in the past. And in other cases, we still hand them over to the analysts and say, look, this is an enriched version of what you have normally received, but we have taken or the machine has done some work for you on your behalf. And this now you can decide and you can use your expertise, your human expertise to connect the dots if the machine cannot do so automatically. Right. Cool, okay. And I think if I remember correctly a few days ago, you have released actually a new tool, right, XDR Connect. So would that help with threat management? Srinu, what's your view? Absolutely, no, XDR stands for extended detection and response. So the part that we've talked about till now and Ian was showing that graphic where the automation part is on the right side. So the ML AI and the human analysts augmented, AI ML augmenting the human analysts there, that will help with the detection. So it's about IBM believes in open. The other industry players are more about their products. So they built in this XDR extended detection response with their portfolio, but IBM will bring in all third party vendors, IBM along with IBM products and technologies into this XDR ecosystem, process the data. We apply that AI ML plus our analysts working on the investigations and actually ML also helping with the investigation. That's the detection part of it. And the response part of it is all these things are also helping us to provide that recommendation to go take some action, right? In some cases, we can now enable through our SOAR platform. So a response orchestration platform, we can also enable the automated response. So on a EDR technology, on a network technology or a identity technology, we can go put that policy to protect the environment. So that's the response part of it. So the combination of what we have talked about Ian and Mark talked about till now actually is being incorporated into the XDR strategy. Okay, okay, cool. So thank you for that, Shrini. Ian, a question for you. So we talked a bit about the security of AI itself earlier, right? So in this case, when AI is protecting systems, say, you know, some maybe critical infrastructure, maybe a power plant. So would criminals be able to infiltrate the AI itself and what can be done for that not to happen? Yeah, that's a great question. So Mark previously mentioned this notion of these two duels where we have adversaries both using AI and then attacking AI as well. And that's exactly what we end up seeing going all the way back to some of the initial uses of machine learning for doing, you know, host-based intrusion detection right then and there, people started looking at this and saying, hey, we can attack the AI system itself. So there are effectively four main threats that we actually look at when we talk about an AI system and adversaries can go at any one of them in order to meet their end goals. And it all really depends on what level of access the adversary has to your machine learning models, your infrastructure, your data collection, labeling system and so on. And so one of the first ones we might look at is what we call evasion. And so this is a case where you have a well-trained AI system. They didn't have any control over the training data, the training process or anything on those lines. But what they can do is they can submit queries or they can submit inputs to your system.  And they can craft in such a way that a very, very small and perceptible change would actually cause it to come up with any different classification. So in a visual recognition sense, we like to talk about this as a panda bear versus a given or a cat guacamole and stop sign and speed limit. But the exact same threats apply to any machine learning model and security detectors are no different. So we're making some rather small changes we can completely evade the detection and these detectors that we can rely on would no longer fire a trigger. Now it becomes even more damaging when the attacker actually has control over that training data itself. And this could be done what we call poisoning where they can actually manipulate the training data or the labels. But when you have a system where the adversary has, there's submitting jobs or submitting queries and you have this constant learning of what is normal, for example. They obviously can kind of cause that to the model to drift and cause the model to start to classify their benign behavior as being, they're sorry, their malicious behavior is being completely benign. Another potential threat is what we call extraction. So just simply by submitting queries to an animal and looking at the responses they get back. An adversary can actually completely create their own kind of surrogate model. Sometimes exactly replicating other times this has comparable in the area. And you can use this to steal intellectual property but you can also use this to then have a standing model. The adversary can attack that and then apply it to your own system when we call it a transfer attack. So even if someone doesn't have control over the exact detectors in your end system by kind of observing some of these behaviors as we aside channels, they can then use that knowledge, hone their attacks and then be able to launch them in your system. And then the final one is just what we call an inference attack. And this is where the adversary can learn potentially something about the training data again by observing the input and output. And usually we look at this as having a threat to the privacy of the system itself, learning something about either the population or a particular entity there. So all of these are something that adversaries are realizing. They see AI has a lot of value instead of honing the threats on that. Okay, cool, got it. Yeah, very interesting. Thanks for that, Ian. So Srinu, back to you now. And I'd like to ask you actually about the practice at IBM security and the kind of day-to-day reality when you guys have to manage the security of IBM clients. How do you actually work together with IBM research and how do you use research for services and for products? That's a great question. As the opening, we talked about getting the team access to the data. That was never over. We act as one team among ourselves, right? So the team has access to the data. The research team has helped and is now actually helping and we are incubating a lot of ideas. We talked about that system being an analyst, right? So that's one component that the team is working on where we'll generate hypothesis, we'll generate provenance. Now what has happened in the future? Can this happen in the past? No, what has happened in the past and can this happen in the future? How do you project all things? The alert dispositioning, right? Is this, should this be closed? Should this be alerted? What is the reasoning behind it? So there is activity happening in there. And the second part of it is helping the user, the analyst with certain tools for investigation purposes. That's a threat investigator that is happening where they'll be provided with Mitre-Tac mapping, additional reasoning behind it, all the curation of the telemetry and the information required from Intel sources in one single pane of glass. So they don't have to go search for that information, providing additional information through the hypothesis generation and the provenance tracking, providing that information for hunting purposes. All of those things are part of our triage and investigation methodologies. So that's where the threat investigator comes into picture and research team has been helping on that. And then we also have the hunting language that the team has developed which is now part of our portfolio as well. So all these assets combined are what is helping us to move our security services practice as well as our product portfolio to the next phase. Great. So this threat investigator, a great name by the way. Yeah, so it's recently been made available, which is great. And if I'm not mistaken, it's part of the latest release of CloudPak for security, right? Yes, it is part of CloudPak. And next year, Connect. Okay, great. So I think you guys were gonna show us a demo, right? To kind of demonstrate how it actually works. And I believe this is when it will be great to welcome your colleague here to the show. Let me introduce our security researcher, Yuval Lapidot, Haival. And you're joining us from Haifa. And I believe you have a quick video to show us, right? Thank you very much, Katia. I will show my screen with you. So, hello everyone. I'm Yuval, a security researcher with Haifa Research Labs. And I'm happy to show you today a quick demo of the threat investigator. The threat investigator is an automatic service that leverages AI capabilities to perform security investigations. And it's part of the cases component, which takes a major part in the IBM XDR offering, as mentioned before. Investigator performs an automatic security investigation of cases that are open in XDR. It correlates between various event data sources, such as firewall, EVR, SIEM, for example, IBM's QRIDAR SIEM, network traffic analysis, antiviruses, et cetera. And it enriches the data using different knowledge bases, such as MITRE ATTEC, which is a very well-known framework in security and threat intelligence sources, for example, EXPOS. An investigator connects all the dots together to create a coherent attack story. In today's demo, we will show an investigation of the Fint 7 scenario. In this scenario, an attacker sends a phishing mail with an attachment, the file invoice.doc to an employee. The unsuspected employee opens the phishing attachment with MS Word and runs a macro. The macro extracts an encrypted script from the Word document and sets a scheduled task to execute that script. The script later calls home to the attacker's command and control server. Through an HTTP, it receives further instructions. It then collects information about the victim, such as process dumps and screenshots, and sends them back into the command and control server of the attacker. Let me now go to the Analyst Desktop where you can see the results of the analysis they automatically performed by the threat investigator. So here you can see the general tab, the general view of the analysis, where we can actually see the information about all the information we needed about the analysis. So we can start by saying the case it's connected to, it has an ID and the owner and the date it was created, and also a severity assigned by the investigator itself. Later, we can see some details about the investigation. For example, how many artifacts was in the trigger of the investigations, how many MITRE tactics were found in the investigation, how many findings were found, and how many assets were involved. Later, we can delve deeper into the MITRE tactics that we'll find within the investigation, and you can see their severity in this investigation. Later, we can see a network graph which lists the assets participating in the attack and the connection between them. In the right panel, we can see a timeline, a human readable timeline, that can show all the events and findings that were found as part of the attack. Let's scroll now through this timeline to see what the investigation found for the SPIN7 attack scenario. So the first finding that we can see that was here, a grade, it's a process created event that was created and a threat intelligence system. Let us know that the hash that was opened is malicious. So we can see here the hash, and once we click on it, we can see that the threat investigated that found it is X for exchange, and it classified it as a virus. By further looking into this event, we can see more details about the event. We can see that it's a process created event. We can see on which domain did it happen. We can see more process information, like it's PAD, it's name, and then even about this parent process, we can find some information, and we can see from which data source we received this event. In this case, we received it from a curator data source, which is the IBM SIM. Finally, we can see the raw data of the event, which is represented in this piece. Okay. Just a second. Later, we can see some more events and some more findings that we'll find. In this case, another finding is the finding that was correlated with mitre attack technique. In this case, the technique is a suspicious sketch of dust. And as you remember in the scenario, we know that the virus actually creates a suspicious scheduled task and uses it to run the code. Here, we can also see some more information. We can see that the data source collected this event comes from actually a different data source from MS Defender. And also we can see the raw event. Furthermore, we can see more events and more findings. For example, we can see a firewall accept an event that comes from the firewall, which was enriched with threat investigation. Sorry, threat information, saying that this IP is also suspected or malicious. We can see more and more findings and all is concentrated in one single panel. Next, we can also see the attack graph in a different view. We can see it in a swim lane. And we can also see all the mitre techniques and tactics that were found as part of the attack. Having all this information in one page correlated and enriched and having the ability to view all the events and alerts in an organized timeline can save hours of manual labor. This is why the investigator service is a great example of how AI-based systems can improve the work of security analysts. Okay, wonderful. Thanks, Yuval. That's definitely a great asset for security analysts for sure. Thanks for that. So now that we've learned a lot about threat management and then AI and security, how about we switch topics a bit and I'd like to invite our last speaker to join us. Julia Hesse, security researcher here at IBM Research Europe in Zurich. Hi. Hi, Katja. Hi, so I know you're gonna be talking about passwords and password security. So tell us, what's actually the problem with passwords? I mean, everybody, of course, knows by now they shouldn't put, you know, 000 or their birthday or their, I don't know, pet name as a password. So aren't people already quite aware that they should use a capital letter and a number and a symbol and how can they make their passwords even more secure than that? Yeah, I mean, I hope everybody is aware of that by now. And indeed that would be very secure passwords. But I mean, in my case, the last time I checked I had something like 36, 37 accounts that I was protecting with a password. So for me, it's really, I mean, it's really the question, if I wanted to choose my passwords in the way that you just described, how do I manage to memorize them, right? And I mean, there are certainly some tricks that you can apply for me. What works best, for example, is that I don't remember or memorize the specific characters of the password, but I remember a sentence. And then I use, you know, like the first or first and second letter of each word as a password, right? So a sentence is way easier to memorize than a chain of characters. That's one way to simplify the task of memorizing all passwords. What is also very commonly used is that you kind of use the same passwords for several accounts, right? So there was some statistics of last year or two years ago. The average password on the internet is used four times, right? So in my case, that would reduce my task of memorizing 36 passwords to memorizing only nine passwords. Still quite challenging for me, right? So yeah, so essentially password reuse is a big problem with password security. Right, yeah, no, that's for sure. So, you know, how, like, yeah, actually before, before we even jump to the next question, just something that I would like to ask people here in the broadcasting studio with me. So you guys are all security researchers, right? So I suppose your passwords probably very secure. So maybe Mark, you can tell us, do you have any tricks before Yulia tells us about her research? Thank you for picking on me for this. Well, you know, password is a challenge, obviously. I try to keep one password per one account, not to have shared passwords in the first place, also adding complexity when needed and as much as possible change it. But then, of course, there's the challenging, the challenge out there for managing this. So generally speaking, I use password manager trying to help me assisting me with that, but it is still a challenge to be frank. Okay, well, maybe Yulia will tell us now about her latest research and how to make it even better and how to create even more secure passwords. Yulia, over to you. Yeah, thanks. So let me first show you with a small slide. What the actual problem is with passwords and the internet today. So assume that I want to have a look at my email. So I log in, I wanna log into my email providers to the website of my email provider, right? So the browser will show me a prompt for my password, right? And I type it in, I hit okay. And then essentially what is going to happen is that my password is transferred to my email provider in the clear over this green channel, which essentially means that it's protected from the eyes of any observers, right? So this is called a secure communication channel. I can be assured that only the recipient in this case email provider can read my password, okay? So what is happening at the email provider? Obviously at some point I registered and I chose a password and the email provider stores some form of this password. It's called an encoding of the password or for people who know it, it's a hash of the password. And the provider is going to redo this encoding procedure with the password that I'm sending for login and essentially checks whether this is equal or not to the encoding that is stored and then grants me access or deny success to my emails, right? So this is essentially, it's called password over TLS and it's essentially how the internet works today with passwords. So it's clear text sending of my password to the provider who is supposed to verify the password. And this is okay, this would be no problem if we were not reusing passwords because now what can happen is that my email provider actually impersonates me using this password that it just learned upon login. For example, to access my cloud data for which I use the same password, right? So this is a, well, I mean, usually I would say I trust my email provider but you have to take into account that these are large companies. So by trusting your email provider you're essentially trusting the hardware that your email provider bought. You're trusting the software that your email provider is running, you're trusting every single employee, right? And that's a lot of trust. Okay, so essentially, yeah, the clear text sending or like telling the password in the clear upon login is essentially the big problem that we are facing. Okay, well, yeah, it really doesn't sound too great. And so are you saying that your research is actually about preventing, avoiding such digital identities theft? Exactly, so well, I mean, it's obvious that we cannot really force users to not reuse their passwords. Not everyone is as strict as Mark probably. I have to admit that also I reuse some passwords even though I shouldn't, or I should know better. But so the research question here is really can we somehow have the provider verify that we know the passwords without actually telling them to the provider, right? And in cryptography, we have a very neat technology which is called a zero knowledge proof system. This is pretty, it's not a new thing. It has been around for, well, I would say 30, 40 years. And it's essentially a way to prove that I know a solution to let's say a riddle without actually revealing the solution. So the only thing that I'm revealing is whether I know the solution or not, right? Does it sound magic? Sure. You wanna see an example? Why not? You wanna do a zero knowledge proof? Let's try. Okay, so I have a challenge for you. So this is a worst walley picture, right? I guess everybody knows it. I know the solution, so I know where a walley is. Do you believe me? Okay, sure. You should say no, because otherwise there would be no point in doing a zero knowledge proof now. Okay, no, I don't believe you at all. Thank you. Okay, so the point is what I wanna do now. I wanna reveal to you only essentially that I know the solution. I don't wanna reveal the solution itself. So I don't wanna point at the picture and show you where a walley is. Because for example, I would like to enjoy the experience of searching and finding him yourself, right? So what I could do, so I have here a, this is not going to be a magic trick, by the way, there's no magic happening here. So this is a cardboard, right? And I have cut a kind of walley-shaped hole in the middle here. So now the zero knowledge proof for this riddle works as follows. So I have the cardboard here. I have the picture here. I'm going to put the picture behind the cardboard. Again, no magic happening here. So I'm going to put it in a way that you cannot really see where I put it. And then, can you see him? Sorry, I don't really know. Yes, yes, I think I'm very low. Okay, so, and it was, oopsie, it was actually, so it was that picture, right? Okay, so what did you learn? Not much. But do you believe me now that I knew where he was in the picture? Yeah. Okay, so congratulations. You just verified your first zero knowledge proof. Wow, okay, well, that's really cool. And before you were saying that such proofs also exist for verifying passwords, well, it would be great to use that technique for my accounts for sure. So could you tell us how to do that? Yeah, so the good news is that these zero knowledge proofs, they don't only exist for, let's say, a bit useless things like a versus Wally, but they exist also for password verification. These are also not very new. They also have been around for 10, 20 years. And they're very, very efficient, okay? And so I also wanna use them. The thing is that we cannot really do anything, right? Because our interface to password verification at, for example, or my interface to my email provider is the browser where I get shown a password prompt, right? And it's really the underlying protocol that needs to implement zero knowledge proof instead of this, you know, what I showed you before, this clear text sending, right? So it's really not us who have to, who can change anything. It's really organizations such as the internet engineering task force, which have to first standardize these protocols, right? So that afterwards after the standardization, they can get adopted by the browser and by the service provider. And it's always, you know, both sides have to be able to run the protocol. So you could say, okay, this is kind of like, like for example, in TLS, which we usually use, this is transport layer security, this padlock which shows in the browser, which you usually use for implementing these secure channels that I was showing before. You have this very secure option and then you have like fallbacks to less secure. To less secure options. So optimally, we would have like a first option, the zero knowledge proof. And then as a fallback, we could still do the clear text sending which everybody is doing today on the internet. So currently, we are pretty far along with the standardization process. And by we, I mean, essentially the internet engineering task force, it's not only IBM who drives this, there's many, many other people, but we are pretty active there. And yeah, so afterwards, I really, really hope to see adoption by providers and by also the browsers. Because in essence, this is also useful for providers because they do not risk losing the passwords of their users, which is really something that they don't want to do, right? They don't want to put their users security at stake. So also providers should be very, very interested in deploying these strongly secure password verification protocols. Yeah, for sure. Yeah, thanks for that, Yulia. So I'd like to ask Srinni here, what do you think about what Yulia just described in terms of standardization specifically? So in terms of standardization, it's around people process technologies. Gotta have repeatable patterns, repeatable standards, now based on standards, open standards, that will help us to integrate this with the wider ecosystem of our third party vendors. So IBM has been working and research has been helping us with the open cybersecurity alliance. So defining the standards there and getting that adopted by multiple other partners will help us. So we don't have to go define this multiple times and try to adapt this multiple times. So that's what is the important part. For service providers like us, it's very, very important because we operate like with 30 different vendor technologies when a product. So standardization is very, very important. Cool, great. Thanks for that. So in the last few minutes here, just one more question. What about the massive security breaches that do happen from time to time? We read about them in the news where millions of passwords are suddenly leaked and when servers get hacked, for example, Ian, what do you think? What can we do about those? So did you wanna ask me or Julia about this?  So when it comes to these massive password breaches, some of the times when the things we end up learning are some of the different processes and procedures and protocols that the different endpoints or the different organizations are using. Are they strictly storing passwords in the clear where they can be leaked? Are they storing passwords which have been salted in kind of a hash or digest form where it becomes slightly more difficult for them to be recovered or and then there are certain different layers. You know, what types of ciphers are using, how many rounds they performing and so on. But even when there are some of these big breaches and we hear about them all the time where they aren't storing the passwords in the clear, they are storing them, you know, salted and hashed, technology, you know, GPUs, fast accelerators have actually made it more easy, more convenient for people to actually go about and reverse engineer these, come up with very, very fast and efficient ways of actually recovering what those plain text passwords are simply by guessing. So even there, depending upon if the passwords are still weak, it can actually still expose users to some risk if there is a breach. Yeah, can I follow up on this? Thank you. Yeah, exactly. I mean, Ian, there was really a great answer and it's exactly that. I mean, what I was showing you is how we can avoid this handling of clear text passwords upon comparison, right? Or upon verification. There has been like massive security breaches where providers at some point noticed that while they were essentially logging the clear text passwords upon verification accidentally, right? They did not really know why and how that happened, but they found like millions of passwords somewhere in the storage, right? So this is one thing that we will avoid when using these more secure password verification technologies, simply because the provider will not learn anything about the password. So it has no chance to store the password into your text. The other thing that you also mentioned is while we have these, I showed it before on the slide, we have these encodings of the password that the service provider needs to somehow store because it needs some reference data to verify the password against. So this is really, I mean, it's really like a paradox. How would you verify something without actually requiring to store this data, right? And this is a huge problem, as you said. I mean, even though this might be an encoding, so it might not be clear text, but it might be encodings or salted hash, for example, the less secure passwords that you can reverse engineer so you can kind of, or you can simply guess them and this way find out a lot of passwords still. And this particular issue is actually something that the previous solution that I showed is not solving, but this was something that even the European Union got really interested in and they funded a project in their Horizon 2020 program, which is called Olympus. And IBM, in the Zurich lab at IBM Research, we were actively participating in this project and we kind of looked into how to, well, essentially what to do in this kind of paradox situation. What can we still do? And we came up with a very cool solution, which essentially, like on a very high level, what we did is we kind of split up the encoding in not really, you know, like we did not really do chunks because that would not work. We split it up in like a more intelligent way and then we distributed these pieces of information to different servers, right? And then in the password verification, all the servers need to kind of participate. So the verification procedure is a bit more interactive, but the benefit of this is if you have a security breach at one of these servers, the information that is revealed leaks zero information about the passwords, right? So you can have up to, let's say if you use like five servers, you can have up to four of them essentially breached, but the passwords will still remain secure. And by secure, I mean like 100% secure, right? So this is a very cool solution that we deployed in that project. It was a bit oversimplified. So, I mean, there's more technical challenges in that obviously, but I just like the work that we have done in this project and also the interest of the, for example, of the EU to try to find solutions for these problems. This is really motivating to me as a researcher and I really, really hope to see a lot of adoption of these technologies, in particular of the Olympus technology in the future. Oh, yeah, absolutely. Sounds great. Well, I'm afraid we're out of time. So thanks everyone for participating today and a huge thank you to our experts in the chat as well, answering questions, and of course to everybody who joined us today as well. So we'll see you next time. Thank you. Bye, thank you.


Good morning and welcome to this week's edition of Encompass Live. I am Krista Burns at the Nebraska Library Commission. Encompass Live is Library Commission's weekly online event. We cover commission activities and any topics that may be of interest to Nebraska librarians. We have guest speakers that come in sometimes and we sometimes have Library Commission our own staff as we do today. We do these sessions every Wednesday morning at 10 a.m. central time. They're free. They last for an hour. They are recorded so if you are unable to attend a live session, you can listen to a recording. Or if you have people who you know might have wanted to hear the session, pass them on the info and they can listen to the recording. So I'm going to hand over control now to our speaker today. We have Katherine Brockmeyer from here at the Library Commission who's going to do our second part in a three part series about conducting surveys. Good morning Krista and welcome everyone. Thank you for joining us today. I just want you to know that this is going to be a whirlwind tour of conducting surveys, especially data collection. And hopefully you received the notes page that I sent out yesterday so that you can take some notes along the way. This PowerPoint presentation and all of the information that we go over today will be available afterward once this is archived. And so you can go back and refer to it. I talk pretty fast so I hope you can hang in there. I'll try to keep it tender an hour. It may go over a little bit and if you need to leave, I totally understand. But I'll go through this information as fast as I can and hopefully you can keep up. If you have any questions again, just type it in the question box and Krista will try to stop periodically and Krista will let me know if there are any questions. My name is Katherine Brockmeyer. I'm a research analyst and special projects associate here at the Nebraska Library Commission. Two of my main duties include grant writing and doing evaluations. There's my contact information. If you'd like to contact me with any further questions, I'd be happy to consult and follow up with you on any topics regarding survey research and conducting surveys that you might have. And who might you be? You might be a library director, a trustee or friends and foundation board member. I also see that we may have some others from, I think there was one from a hospital in Nebraska. And if you do know somebody who'd like to know a little bit more about surveys, you could certainly recommend it to them. And what are your credentials? Well, the fact that you want to navigate through conducting a survey and that you have courage. You may have been asked to conduct a survey and don't know anything about it. And so you're just getting your feet wet. I'll try my best to go through this information today in layman's terms. I do have a background in survey research and so sometimes my terminology can get a bit technical, but we will plow through together. And who are you? As you can see, we've got people from Nebraska, Pennsylvania, Wyoming, Washington, Kentucky, Virginia, Vermont, Georgia, Texas, North Carolina, Nevada and Kansas today. So welcome everyone. Good to have you on board. Aren't webinars great? This is the best way to get information when you can't make it to a face-to-face conference. I want to recap a little bit about the first in the series of conducting surveys. I did go over reasons for conducting a survey. Perhaps you're doing an evaluation. You might be getting ready to look into constructing a new library. You might be doing program evaluation or your board may be asking for more information. You may be getting ready for an annual report. That's another reason. Issues in effective questionnaire design, data collection and analysis and reporting. I did go over some of those and then questionnaire design, especially measurement content and structure. We're going to look at a sample survey today, but we're going to concentrate on the formatting of the survey, not the questions themselves. So if you're looking for a question bank, that is available in this first of the three-part series. We're in part two today. That recording is available on the website. I'll give you a link at the end of this presentation. What are we going to go over today? We're going to talk about sampling, how to target your respondents and survey methods. Let's jump right in. What is a sample? A sample is a subset of individuals within a population of interest. Sampling, I'm kind of reading from this, but sampling is the act, process or technique of selecting a suitable sample or representative part of a population for the purpose of determining parameters or characteristics of the whole population. That's a whole mouthful. Basically what you're trying to do is select a smaller group from your entire target audience. What is the goal of sampling? It's to be able to make inferences to the population in question. In other words, if you want to know some things about your patrons and you have your entire population of patrons, you sample a subset and depending on how you sample and how extensive your survey is and your investigation is, you may be able to make some decisions based off of the responses from this smaller subset and generalize to your entire population. I do caution you, however, to take your responses and the data that you collect with a grain of salt considering how rigorous your survey is and your investigation. Why would you sample as opposed to survey everyone? Well, it doesn't cost as much money because you're going with a smaller number of people and you can collect it a lot faster because you're not trying to recruit everyone to do your survey. What's the sampling process? We're going to go through this in detail. You define the population of concern. You specify a sampling frame, which I will define here for you in a second. You specify the sampling method. You determine your sample size. In other words, how many people you're going to try to collect information from. You implement the plan.  First, you define your population. The definition of a population is the entire universe of individuals who have a characteristic of interest. For example, your population might be patrons. Your population might be non-users. Your population might be people who attend programs. Here are some other examples.  In other words, your active library card holders, your non-users, people who come to your library, people who use your computer lab, or people who attend your programs. That's your entire population. You can see it's quite general. The sampling frame has the property that we can identify and we can include in our sample. It's the framework for targeting your respondents. You might use a telephone directory, a city directory, your patron database, attendance sheets, voter registration list, utility bill recipients, newspaper readers, or random digit dialing, which is when you go with a larger firm who can provide you with random phone numbers based off of the prefixes that are in your city. For example, we might be able to get random phone numbers that start with in Lincoln. It's 471-488-486, and then they randomly select the four last digits behind that and you receive a bank of phone numbers. Some of them may not be any good, but you receive an entire list of randomly selected telephone numbers. That would be for a telephone survey. Sampling methods. There are two kinds of sampling methods, probability sampling and non-probability sampling. When you really get into survey research to get the best results, you go with probability sampling. Every individual has an equal chance of being selected. However, it doesn't always work that way for us at the level that we're conducting surveys. We might go with non-probability sampling, which is that some units in the population don't have any chance of being selected, or you can't accurately determine what the probability of them being selected is. What are some sampling methods examples? I did put this in the notes section so that you can refer to it. A simple random sample is where you randomly select several numbers, and your population already has a number assigned to it. It's basically like you're rolling a very large dice, and you have these numbers, and then they correspond with the numbers in whatever your population is. That's also the same with random digit dialing. Systematic sampling is where you have a list and you go every so many persons. I'll give you an example here. You randomly select a number. You start with that one down the list, so your number that you randomly selected was 20. Then you go every 50th person or every 75th person or every 10th person down a list that you have, and that's a systematic way. Stratified is where you know a characteristic about your population. In other words, you know what percentage of your city have library cards. Let's say that 60% of your city residents have library cards. You would randomly select 60% to be users and 40% to be non-users, and you would do that in a random way. Nonprobability, convenience sampling. That's where you grab anybody you can. Like it says, it's a convenience. It's perhaps easier to do. You might have a better chance of getting respondents. Judgment sampling. For example, if you were at Walmart and you decided to randomly sample or not randomly, you decided to sample the people at Walmart to give them a survey. You might only select the people that you know. That would be a judgment sample. Quota sampling is again where you're going for proportions, but you use judgment or convenience to select them and get them into your categories of user and non-user. So they're not randomly selected. And snow wall sampling can also be a very effective way, especially when you're talking about a population that is very hard to reach. So let's say, for example, that you would like to survey Hispanics in your population and you only know a few. Well, you survey them and then you ask them to recommend people that they know. That's what's called, it's done by referral. So the first example of sampling that I have for you is a simple random sample. Your population is all your patrons. Your sampling frame is your patron database. And the method is random number generator. So because you are able to assign, let's say that you have 400 patrons, you're able to assign them in number 1 through 400. And in Excel, I'm going to pull this up real quick. Let's say that you're going to sample 100. 1 and 100. And you have 400 patrons. That's right. Here we go. So it randomly gave you a number between 1 and 400. Here we go. I hope I haven't confused you. We're looking for random numbers, 100 numbers between the numbers 1 and 400. And so you're going to select 100 numbers out of 400 because each of your patrons has been assigned. You can either copy and paste down to 100 or use control D. And then so you can see there's 7, 172, 188, 88. And that is the number that's assigned to your patrons. And then you need to paste special and do values so that it doesn't keep transposing because every time you copy and paste. And then you are able to sort from smallest to largest to make it easy for you. So patron number 6, patron number 7, 8, 9, and then you skip to 34. And this is how you select out of 400. This is how you select 100 of your patrons. Sorry. Let's see. Okay. Example number 2 is systematic. The population that you're looking at are city residents that have a listed landline telephone. Your frame is your telephone directory. And this is very time consuming. You randomly select a number. The number you select that comes up is 20, for example, or smaller, for example, because you're going to be selecting every 20th. And then you start there on the first page. So the number that came up randomly was 6. And so you start there on the first page of the directory. You select the sixth person and then every 20th after that. This is systematic. Now you might come up with a business or some other number every 20th. And you just have to be systematic in how you would decide you go to the next one and then go from 20th from there, for example. That is a systematic way to select. And, you know, be as rigorous as you can. This cannot be scientific. Scientific happens with large corporations and in ivory white towers in academia. Not on the level that we're talking about. You have a comment from the previous one you're doing with the Excel. Just said, always forgot about that function. Thanks for the reminder. You bet. You bet. You bet. Yep. It's a very helpful way to get random numbers. And I suggest that there's also random number generators online that you might be able to find. Absolutely. Just Google random number generator and there are several that will come up. However, they may only give you the one number. And then you'd have to do that 100 times. So it might be easier to do one. Right. It might be easier to do it in Excel. Sampling example number three. This is a convenient sample. You are looking at all the people that walk through the front door of your library. And anyone who approaches or passes by the desk is your sampling frame. And what's your method? You stop as many people as possible. Or you, people that are in your area on the hour every hour for three days. Something like that. As long as you're now, here's where you might be put in some judgment. You might see somebody come in and you know that there are talker. And you don't want to survey them. Because you know that they're going to take half an hour to complete your survey when you know it should only take three minutes. That's where, for example, of judgment sampling where you decide not to ask them. The more rigorous you can be, the better. So please keep that in mind. Another convenient sample would be utilities. Utilities are sometimes very helpful with libraries. You could ask to include a survey in the utility bill. And that's another example of a way to sample. You're going for everybody who has a utility bill. And that's how you sample them. One example of a population is, we talked about it's the entire universe. And so one population, let's say you have 100 people that attend an event. And while they're there, you hand out a survey to every single one of them. And while they're there, you ask them to fill out this questionnaire to provide feedback, for example, about that program. Or perhaps you'd like to ask them other questions about library use and their perceptions of the library. And there's where, if you had 100 people attend and you handed out 100 surveys, you're going for the entire population. Convenient sampling would be that you hand out as many as you can to everybody that walks through the door. So sample size criteria, I'm going to go over this very quickly because it is somewhat technical. The level of precision or your sampling error, it's going to be hard because of how much you know about your population. I'm doing a question about the universe sample. Sure. What return rate should you get on that? Well, your return rate might actually be a little bit better than if you were to actually sample because you have a captive audience. True. If you're conducting... Please, everyone, fill this out. Yes, absolutely. And if you were doing a focus group, for example, and you were doing 10 focus groups and you had eight people in your 10 focus groups, that's 80 people, you have sampled people to participate in your focus groups but your population is your focus group attendees and then you would have a much better chance of having everybody complete the survey because you have a captive audience. So I would expect that your return rate, as long as you have a pen for every person to complete the survey, you might get 80%, you might get 90%. If you want to make the survey short, you don't want there to be reasons for them not to want to complete the survey, but if you keep it short and sweet and they can fill it out right when they're there and hand it back in right when they're there, you should get a pretty good return. There's a little clarification. What point does it become accurate in terms of return rate is what I meant to ask? I'm not sure what you mean by accurate. Accurate in terms of what...? Okay, well, we're going to talk about that in just a second. We'll talk about sample size and that might answer your question. In terms of results. And if that doesn't answer your question, we'll get to sample size in just a second, and I hope that will clear some things up. So when you hear CNN has done a poll or Gallup has done a poll, and especially when it comes around to elections, and people are saying they're going to vote for a certain candidate and 45% say they're going to vote for candidate Smith and 55% say that they're going to vote for candidate Johnson, but they've estimated what their sampling error is going to be. I said 45 and 55. If it's plus or minus 5%, that actually means that the 45% who said they would vote for candidate Smith is actually 40 to 50%, plus 5% on one side and minus 5% on the other side. 55% said they were going to vote for candidate Johnson, plus or minus 5% brings it down to 50% or up to 60%. And so you see there's an overlap. You end up having 50% possibly saying they'll vote for candidate Smith, 50% saying they'll vote for candidate Johnson. Now Gallup conducted a survey in the 20th century of presidents, presidential candidates, and they ended up having a sampling error. They predicted, I think it was, that Wendell Wilkie would win, but they had a problem because they weren't able to survey people who didn't have landline telephones and they ended up voting for his opponent. I think it was Wendell Wilkie. So their sampling error was quite large and they did not predict the candidate that won. Confidence level is if you were to take an infinite number of samples, they would become normally distributed and they would all start to kind of hover around the true mean. And then degree of variability. Again, if you are trying to sample users and non-users, for example, in your population, if you're trying to sample from your entire city residents, they might have two completely different profiles and you would want to go with a larger number of people so that because you have subsets within your population, you have users and you have non-users. Or, for example, if you're going to try to get responses from Caucasians, Vietnamese, and Hispanics, there are large cultural differences perhaps between these three races and so you might want to go for a larger sample size, especially of the Hispanics and the Vietnamese so that you get a better representation of how they truly think as a group. Determining the sample size. Now, I got this off of the Internet. My survey research, some of my classes, I'm a little bit rusty. But if you're, we were talking about confidence levels, these are just suggestions. So if your city is a population of 500 to have a confidence of plus or minus 5% on your responses, in other words, satisfaction with library services, plus or minus 5% to see what your range is, you would want to sample 222 people. That, if your population is 100 and you only sample 10 people, you only have 10 responses, you're going to want to go for a higher sample size. Actually, you want to try to get for 100 respondents because you will get a better representation of what people truly think. Gallup, when they do nationwide polls, they sample 1,000. Past that, there's diminishing returns. So, as you can see, even with population of 100,000. So now you implement the sampling plan. Here's where I stop and recommend that if you live in a larger city and you're trying to do a fair, or even, you know, if you're trying to do a rather rigorous survey, you might consult with your local college, psychology, or sociology department. They may be, maybe they do consulting in town and they may be able to help you implement a sampling plan. They may even help you do a survey from start to finish. Typically, psychology and sociology departments, the professors have conducted surveys in the past. And so that would be one way where you could look into conducting a survey and getting a good sample.   You want to look at your response rate. Once again, if you sample only 20 people and you get a 50% response rate, that's only 10 people. If you sample 100 people and you get a 50% response rate, that's 50 people. As you can see, you're going to glean more information from 50 people than you are from 10. And typically, when you do a mail-out survey, you can expect between 30 and 60%, 30 and 60%, give or take. If you do a telephone survey, you're contacting them directly and you could expect a higher return. I wanted to go back on samples. It is difficult to do telephone surveys anymore and Gallup has run into problems with this, CNN has run into problems with this because everybody's going to cell phones. And even though their numbers are randomly selected, if they're not including cell phone prefixes, they're not going to catch that population, which means you're going to be probably leaving out people who are more technologically savvy, have a higher level of income so that they can afford a cell phone and possibly be younger.  That doesn't have anything to do with response rate, but I did want to throw that in there. Okay, we're done with sampling. Any questions? Nothing new has come up. Okay, all right. That was the quick and dirty, and I wanted to get it done in half an hour and we made it. Yes. All right. We're going to talk about survey methods just a little bit. Now the pros and cons, I discussed the pros and cons of the different survey methods in the first of this three-part series, and so I'm not going to talk very much about it because I go into greater detail in the previous session and we don't have a lot of time. But there are different survey methods that we're going to concentrate on three survey methods today, but I just want to talk about these a little bit. Survey method number one is mail or paper and pencil. So you print out a survey. That's your questionnaire. That's your tool, and you either mail it out or perhaps you hand it out or it's distributed, and people themselves actually fill it out themselves. Group administered, for example, if you're at a program and you hand it out to the people in your program or you're conducting focus groups, you hand it out to the group that happens to be present. There's internet surveys, and I will go over some resources for you to conduct internet surveys. In other words, people go to a website and they take the survey online. There are also ways for you to do it by email. In other words, you send the survey out either in the body of the email or as an attachment, but mostly email is where they email you back and they put X's in front of their responses. And then there's interview. In other words, you have the paper and pencil and you stand there and you ask the questions and you complete the responses for them. So people who come by the desk, you say, could I ask you just five quick questions for conducting a survey? You ask the question aloud. You give them the response categories. They respond. You check it off. Same with telephone.  Could I ask you five questions about our library, blah, blah, blah? And you give them the question and the response categories. They respond and you collect the data. You write it down. That was quick. Okay. Real quick, targeting your respondents. How are you going to recruit your respondents? You might be doing this before you even send out the survey. You might send out a letter saying Happy Police Library is gearing up for... We're talking about the possibility of constructing a new library. We are going to be sending out a survey or we will be emailing out a survey or we would like you to consider coming to this website and participating in this survey. You could send out an email blast. You could post it on your website. With the link to survey, you might send it out in a newsletter or perhaps you would give a press release saying that this is going to be... This is going to be conducted. Any questions? Nope. Okay. We're moving on. So we talked about the method of administration, paper and pencil, which is self-report. If they fill it out themselves, face-to-face, where you ask the questions and you fill it out or telephone, where you ask the questions and you fill it out. There's the electronic kind where you send out an email and they respond in the body of the email. Or you could have the survey attached as a Word document or PDF and they either complete it as a form online and email it back to you or they print it out, use their pencil and complete it and return it to you in some way. And then there's the online or internet survey method. And the pros and cons just a little bit about that. Paper and pencil is fairly inexpensive. If you do self-report or face-to-face, if you do telephone, that's a lot of man hours. And it's quite detailed. Electronic, once again, that's very fast. You're going to probably get a better, you might get a better response rate much more quickly than if you were to do paper and pencil, mail it out and ask them to mail it back. Online and internet, once again, you would probably get a faster return rate. It's very inexpensive, but depending on how you recruit your respondents, you may not get a very good response rate. Okay. Paper and pencil. We're going to get to it here. This is again self-report, face-to-face or telephone. And I'm going to walk you through how to format a paper and pencil survey real quickly. I hope this is what you were expecting from this. If you're wanting the question bank again and how to conduct or how to construct response categories and the types of questions that you can ask, that is in session one. Okay. So you have everything that you want to including your survey, including your questions, your response categories, your introduction. I don't have the, yes, I have the last part on here. Okay. I hope this is large enough for everyone to see. Okay. So you have the title of your survey. You're going to make that in bold and big. Your font size for the rest. You want it to be printed out and take a look at it. You don't want it to be so tiny that they can't read it. So I recommend 12-point font is a pretty good size. Here's your introduction. Happy Place Public Library would like to find out where we are now. You want to put that first to kind of put things into context. Let them know their responses are confidential. If they're going to just be handed in and not adding their name and even if they add their name that you're not going to be using their name attached to their responses without their express permission. Okay. So that is your introduction. And then you have your questions and you put your response categories underneath. We'll number them at the end, but your question, you're going to bold them so that that's what shows up. You have your response categories. I recommend inserting a square of some sort, a checkbox. If you go to Wingdings, there's a really nice checkbox that you can use. It's got a little shadow and it shows them that that's the one that they would X or fill in. And then I do a little space to kind of give it to show, to set it off a little bit. So you would go through each and every one of these. Copy, paste, copy, paste in the right spot. You might also consider your line spacing. You might space it out just a little bit to make it a little bit easier to read. You see how the space helps a little bit? It does. It really does. Now, notice here, there's a skip pattern. If you're going to be asking about library services and they've never visited the library, you need to give them directions. You need to let them know that the next few questions don't pertain to them. And so I would put that in italics because it's a directional entry. And so you would tell them what number to skip to. We'll put the numbers in last. And then here again, you bold the question itself and then you italicize, check all that apply, because that's not part of the question. It's a direction. It's telling them how to respond. Again, there are check marks here for them to do. If there's another category, allow them to check it and then they can fill in. If they don't recall, you might want to give them an out. This is also where you might do a skip pattern if they don't recall and then they can't recall their last visit or what happened with their last visit. Bold and so on and so on. Now, this next question is a matrix. You have your five response categories and you're asking about several dimensions of their satisfaction with the variety of following items. This is where if you have five categories and one, two, three, four, five, six, seven... Oops, I stopped counting. One, two, three, four, five, six, seven, eight, nine, nine dimensions. So our table is going to be six by ten out of you. And I'm sorry if I'm going too fast and you've never... If you're not real comfortable working in Word and you still have some questions about how to do this, you can contact me later. Then watch the recording and put it on pause. Yes, or watch the recording and put it on pause. So your response categories go along the top. Very satisfied, somewhat satisfied. Neither satisfied nor dissatisfied. somewhat dissatisfied. Very dissatisfied. Goes in the fifth one. One, two, three, four, five, six, seven, eight, nine, ten. I only need ten. I'm going to delete a row. I have my dimensions and I need to bold my question. Oops, that did not work. There we go, that worked. Is anybody dizzy yet? Okay, so... Nobody's logged out. Okay, thanks for hanging in there with me. I'm doing a cell alignment centered and down at the bottom to make it look pretty. I'm going to make these a little bit smaller. And then I'm going to make the columns a little bit more narrow because what you're really concentrating on are the dimensions. And you want to make sure it's easy to read.  So something like that. And then they've got where they can check on each dimension. They're very satisfied about the fiction books. They are somewhat satisfied about the nonfiction, etc. And so there is your matrix. Now, down below, if you want to give them a chance to comment, make sure it's close enough to the question so that they know that this relates to the question from above. Select how deep you want your box to be, your text box. And then right here is borders and do an outside border. And there you have a space for them to leave their comments. So you might even say comments about the variety or something of these items. Okay, next is a transitional. A transitional. You might want to leave some space or you might be on the next page. So you're making a transition. Here is your question. You bold it. And then once again, you're inserting your symbol, etc., etc. Here again, if you want to do comments about the library website, this is a border, another transition. Now, if you had a... Here's where we're going to go up and we're going to number the questions. It's always a good idea to number your questions. If you need to move your margin so it looks good, these you may want to move in. Then this can be time consuming, but you want it to look good. So that's how you would format the numbering. This is how I do it. If somebody has a better way, I'd say go for it. Let me see, I guess I did that incorrectly. They want me to move that in. There we go. Leave plenty of white space. You might want to number your pages. You don't want your survey to be too long. If you're turning the page, if it's on the front and the back, you want to put over at the bottom of the page. For example, I would put it over to the right in italics. Right, justified. For example, you may want to number your pages. If it's going to be more than one page, one piece of paper, be sure they're stapled or use 8.5 by... Oops, 11 by 17 is something that you can fold into it. Now, don't forget, the longer your survey, the more survey fatigue, the less likely people might be to complete it. If you can do it on a half sheet and hand it out at the desk and have them complete it right there, just five questions, you might have a better response rate. It's a convenient sample, but there you go. Okay, that is the paper and pencil. I just want to show you, if you're doing a telephone survey, you need to give directions to your interviewer. You give them your script. My name's Jane Smith, blah, blah, blah, blah, blah. Then they say your responses will remain confidential. If they say yes, you check, and then you tell in... I would use all caps to give directions to your interviewer to continue or no, terminate. Say thank you very much for your time. Then number your questions, and then this is where they check off. Your interviewer does the checking off, or if you're doing face-to-face, if you're going to ask the questions at the desk. Here, for example, do you have a library card? Don't give them the... Don't give them the responses. They'll either say yes or no, or they might say don't know. This is for the interviewer. It's in parentheses because you're not going to give that as a category. Here is a skip pattern, and you help... Try to make it easy for your interviewer. I'm going to read from a list of reasons why people might visit the library. You want to put your question into context. If you recall, what was the purpose of your most recent visit to the library? You may include today's visit. Was it to use the computer lab? Yes or no? Browse books? If they get to the point where they're like, I don't remember the reason why I was here.  Don't recall. That's a telephone survey. We're going to go to Survey Monkey here in just a second, and we'll talk about a few other... Where do you go from, current slide? Okay. These are some, and it's not an exhaustive list, but it's the one that I could find, of free... Some of them do have subscriptions, but I think some of these have... They give you limited features, but they're free. Survey Monkey, zoomerang, free online surveys, poll daddy, Lyme Survey, Survey Gizmo, and Twitter Survey. The one that I'm most familiar with is Survey Monkey. Some of them, they will let you have one free survey, or they'll let you have unlimited free surveys, but they only allow 100 responses. Or they give you limited options for how you can format it or analyze your information. So we're doing the walkthrough. Here we are. Okay, we're going to go Survey Monkey. If you have a Google account, it's easy to create your account. If you don't have a Google account, just follow the instructions. Okay, I'm going to be skipping back and forth. Sorry, between Word and Survey Monkey. Let me see here. Okay, so you create a survey, and you name your survey, Customer Satisfaction 2010, something to that effect. You can use survey templates. I haven't really looked at all of this, but that's what you get. And along the left, be sure you kind of scroll through everything just to kind of see what's going on. This is where you go into, this is the tab for your designing your survey.  Analyzing results, that's part three of the three-part session. Okay, so my survey's already started, one.  Absolutely, it's quick, it's easy to use. We use it all the time here at the commission too. We sure do, and for example, I'm going to ask for your feedback at the end of the session, and I'll be sending you a link to my survey, and it is in Survey Monkey, so you'll actually get to see how it works. I've already done the first one here. Let me see if I can... So if you're going to do an intro, but you can name your sections, but I decided not to name my sections, so don't number your pages, but go ahead and number your questions. Okay, so the first thing you're going to do is an introduction, and that's descriptive text. And this is... We're doing the survey. We'd like you to share your information, your responses are going to remain confidential. You can change your question size and the placement, the size of your font, and how wide it is, et cetera, et cetera. Play around, be inquisitive, be curious. Okay. The next thing we're going to do is we're going to add a question afterward, and the type of question that we're going to ask, overall, how satisfied or dissatisfied are you with the services the library provides to the community? And you have five response categories, so you're going to give them a multiple choice where they can only give one answer. In here, you put your question, and for some reason it copied my number. Don't worry about that, because they're automatically numbered. And then your responses are here, and the check boxes are in there, so we're going to have to get rid of those. You can randomize your choices. Unless you're doing 1,000 people and you're talking about satisfaction with services and you want fiction, nonfiction, computer lab, and all these things to come up randomly, I wouldn't worry about that. For this one, you can do a comment field. We'll go to get to one right quick here where you ask them if they want to leave a comment regarding that one question. You can require them to answer the question. They can't proceed unless they do. That can be frustrating for people, so I would leave everything optional if possible. Here it is, overall, and then you can see there are these radio buttons, and they have to choose just one. Add your next question. You can't do skip patterns with a free SurveyMonkey. You can do skip patterns if you have a subscription. For example, this one, you're asking them to skip to a certain question. If you do that in your survey and you ask them to skip to question number six, know what question is number six. That might be a bit difficult for them to scroll down the page and skip to a certain question, and they may end up answering the questions in the middle in between question number three and question number six anyway. Here's this one. If you recall, what was the purpose of your most recent visit to the library? You can include today's visit. Check all that apply. This is Matrix of Choices, only one answer per row. No, I'm sorry. This is also multiple choice, multiple answers. Sorry about that. I hope I'm not confusing you. This is one where you can check all that apply, so multiple answers. Okay, here's where there's an other. Let me show you how to do other where they can fill in. Take that one out there. Let me see how this turns out. There we go. They can check multiple of these. That's the question. Let's see if we can do room for them to do a comment. Add comment field. Yes. Other, please specify. Make the comment field an answer choice. Let's see how that looks. There it is. Other, and then they type in that they were there to make copies. Okay, the next one that I want to show you is this Matrix. How satisfied or dissatisfied are you with these several dimensions on a five point scale? And that one is Matrix of Choices. Here's your question. Here's the row choices, these right here. So your dimensions, and then your column choices are you're very dissatisfied, very satisfied through very dissatisfied. What I like about SurveyMonkey is everything's really clean and looks very professional. And people do SurveyMonkey so often that they're actually just getting used to the look of it. And then if you want to add a comment field, so just say something like please comment. And you can change the size of your text box. And if you want it multiple paragraph of text, three lines big. So now it looks just like mine did. And you've got your categories across the top.  So it helps them separate that out. You can do that also in Word if you want to use a gradient and do every other line in a really light gray. Something that's still easy to read if you print it out. And then in this one you can make the dimensions take up more space and this take up less space if you have really long responses. Let me see. I have one like that. Here we go. The library provides valuable service. Here we go. To what extent do you agree or disagree with the following statements? This is a matrix again. Your row choices are these sentences and they're supposed to agree on a scale. So there are your dimensions. Here are your responses, your response categories. And then down here you can change your question size and placement. So you want your first column. They call it a label. Perhaps you want that to be bigger. Take up more space because you've got a very lengthy response dimension. So if you can see above here, this only takes up one sixth across. This takes up 40%. And then this leaves room here on the right. So that's how you could do that. That is SurveyMonkey. Praying is probably similar. PollDaddy is probably similar. TweetPoll now has a Tweet Survey Twitter. I should say not Tweet. Twitter has a poll and also has a survey option. So you can follow the links from here. Someone did have a question about the SurveyMonkey one. And I didn't notice where it was. What was text validation in the text box? Sure, if you're asking for a telephone number, they have to do it in the format that you're asking. Or they have to put in a response. Let me see here. Let's go back to one that we already have done. Let's see what is tech validation. Okay. So it has to be a specific length. It has to be a whole number. It must be a date in this format or an e-mail address. If you're asking for the e-mail address, basically they'll validate it for you that it has to have an at symbol in it or something to that effect. I'm not sure exactly how that works. But play around. Have fun at SurveyMonkey. We do have someone else who said that they wish they'd seen a seminar when they were learning SurveyMonkey. That you're doing very good job of going through it all. Oh, well thank you very much. I had the play around. You know what? You can't mess it up. This isn't going live. Not yet. And one other thing about SurveyMonkey to collect your responses. You've done the design.  You can ask them to create a link for you, which you would then post everywhere to try to get people to go to that. It's a website address. It's a web address. And it's unique to yours. So here it is. For example, my new survey has this. You can also, if you subscribe to it, you can actually put behind here Happy Place Library or something like that instead of HQD 6, something 5, 5. I feel like I just took an eye exam. Okay. The screens, we've got this on the big screen and I'm just trying to read it from where I'm sitting. All right. That is online surveys, the one that I was wanting to show you. We have comments about online surveys in general, too, that someone mentioned that Google Forms is a quick and dirty way to put out a survey, too. You get your own responses automatically in a Google spreadsheet, which is great, and gives you an embed code to put the survey on any web page. But it's definitely not as much functionality and ability to build logic like with SurveyMonkey. Awesome. It's something but not as complex as SurveyMonkey. Well, thank you. Google Forms. Google Forms. I've taken note of that and I will take a look at that and I appreciate the feedback. Again, this wasn't an exhaustive list, so I certainly appreciate that. And if it's free, you know what? Go for it. Take a look at it, dig around, and see what you can get out of it. Thanks for telling us what the benefits are of them, too. Appreciate that. So here are some resources for you to look at in terms of sampling. If I went over it and you're interested to look into sampling a little bit further. Also, there are some good blogs out there that talk about conducting surveys and how that can help your nonprofit. We even have one of our very own that talks about engaging users with polls. And in that, Susan Nicely, she reviews several, such as PullDaddy, I think, and Twitter poll. So what's your next step? Well, your next step is to have courage. Get out there, try this. I recommend that before you conduct if you're going to conduct a survey, go back and view the first of these three presentations and maybe stick around in the month and view the third of these presentations so that you have an overview of the entire survey process. But if you want to start thinking about your sampling plan, if you're already in the midst of planning your survey and you've got a pretty good handle, you feel on what's going on and you need to think about how you're going to sample and then in what format or method you're going to conduct your survey, what kind of questionnaire you're going to use. I hope that I've been of help to you today and that you can take heart and move forward. Are there any other questions? Nothing about surveying that stuff we'll get to in a second. Okay. And so here are some related topics. Again, there's the conducting surveys part one, the introduction and questionnaire design. That's where the question bank comes in and we talk about response categories, pros and cons of what kind of how to conduct a survey. And then John Felton here at the Library Commission also gave a very good encompass live session on presenting data in meaningful and interesting ways and I will also cover that in my third session of this three-part series. And then there's American Fact Finder Mining, the U.S. Census for Information about your community. That's if you're trying to get a context for how you are going to conduct your survey. That's also very good. I think that was with Get Beth Global. I'm not sure who did that one. Yes, I believe it was. She's in our Government Documents Department here. And upcoming sessions here at the Commission. Michael Sowers does a tech talk about once a month and those are always lots of fun. We've gleaned some very interesting and helpful and useful information from those. Part three of this series will be coming up on July 14th. I hope to see you all there. And July 21st is another encompass live about communication and getting the word out. Once again here are the resources I guess I had that slide twice. Just to reinforce, apparently. And then here we go. Here's my survey.  Let me know how we did today, what you learned, what more you'd like to know. If you have any questions for clarification, this is your opportunity. I certainly appreciate the feedback and the help to make my presentations more effective. And also it's one way for you to test drive a Survey Monkey survey and see how it works. And that is it. And we did it in an hour. We have a thanks great webinar to get me ready for my training needs assessment that they're working on. Excellent. Yes, and Catherine, so thank you very much for attending. I think this was a great session. We're really getting a lot of interest in this series that we're doing. So obviously people need to know what their patrons want and need. You bet. Catherine already mentioned some of the things that will be coming up. Those are just some select sessions she has. If you go to our main page it was shown on the PowerPoint that I was showing originally. Next week our encompass live will be One Book One Lincoln. Our One Book program here in Lincoln where they're actually doing a new thing this year where the community gets to choose what the book will be. Community members always give recommendations but actually there's a voting now. So they have a choice of five. So that will be we'll have people from Lincoln City Libraries next week for that. So hope to join us for that. Our sessions are recorded. And the PowerPoint presentation will be available when we put up the recording of today's session. So you'll have that. Also, someone asked about how long these sessions are archived. As long as we can keep them up there. These are our sessions, our recordings. We have them going back when we first started Encompass Live January of 2009. So these sessions will be up there for a very long time. Unless they become outdated completely we would never take them down. So you should have no problem. Someone's saying they might not need to do this till next spring. No problem. Keep the link or come back and check for all of them. And this is the session, the recording page for the first session. I know people were looking for it. If you just started with today's part two of this series this is the first one and you'll be sent all the links for all of this too. You bet. Okay. I think that is it. We're all set. We're just getting some thank yous. Great webinar. Thanks a lot. Thank you. It was my pleasure. Thank you very much for attending. And hopefully we'll see you next week for part three of this next month. Same today, July 14th. Thanks a lot. Bye-bye. Bye-bye.