Report from the Joint Task Force on Services to the Print Disabled Remarks by Cynthia Archer, Nancy Baker, and John Harwood at the 161st ARL membership meeting. Convened by Mary Case. Welcome to this session, Services to the Print Disabled. I am Mary Case, University Librarian at University of Illinois at Chicago and have been chair of this wonderful Joint Task Force on Services to the Print Disabled. I'm very pleased to be introducing this session. I'm going to take a few minutes to do that, and then we'll introduce our speakers this afternoon. As you know, as librarians, we've been talking almost 20 years about the promise of the digital environment, how it could reduce the cost of scholarly information, increase access and speed delivery, and discovery. But things didn't naturally evolve that way. So we took it upon ourselves to lead conversations about these issues on our campuses to create new capabilities, like the Office of Scholar Communication and Spark, to help educate and advocate for these outcomes. We built coalitions of interested stakeholders, and we confronted, challenged, and collaborated with publishers and vendors to push for change. Not surprisingly, there is another group of individuals and organizations who have also been looking at the promise of the digital environment to significantly improve their lives. They are our students, faculty, and colleagues, our neighbors, our parents, who are print disabled, and have been eagerly awaiting the end of what they have referred to as the book famine. Imagine their excitement anticipating that any digital text would now be readable by their favorite device. The world of knowledge and literature opened to them in a way impossible before. But as in our story, things haven't turned out that way. So they too have begun to fight, and their fight is bringing them to our campuses and to our libraries. Over the past few years, print disabled students and their advocacy organizations have been filing complaints and suits against our universities and libraries, including Berkeley and the University of Montana most recently. The major national organization, the National Federation of the Blind, NFB, who really has been a partner with us in the whole WIPO conversations on accessibility, filed a complaint against Penn State in 2010 and against the Free Library of Philadelphia and Sacramento Public Library in 2012. With all of these complaints increasing, the ARL community, the board, and the two task force who supported this group decided to appoint a task force to help research libraries better understand the technical, legal, and service challenges that we face in providing access to the print disabled. The task force has been working intensively over the last six months. As it happens just a month ago, the NFB issued a major challenge to all of the participants in the joint EDUCAUSE I2 e-book text pilot, declaring that that pilot was in fact inaccessible and therefore illegal, and asking that all of the participants and the pilot be terminated, not the participants. Excuse me. Many of the pilot participants are ARL member universities, and as it turns out, most of our member libraries were not involved in the decision-making related to campus participation in the pilot, and as it turns out, perhaps many of the disability offices weren't either. This is a mission critical moment for libraries. We need to be at the table. We are the experts in content, care about users, have many years of experience as service providers. We know how to facilitate campus conversations and advocate for our users. We know how to challenge vendors and publishers, and this time, the law is actually on at least our user side. And we have a lot in common with the NFB and our print disabled users. We want content to be openly accessible to anyone who needs and wants it, so do they. We want all of our users to be as independent as possible, so do they. We are committed to diversity. They are a part of our diverse community. We talk born digital. They talk born accessible. We both support universal design. At this point, I want to just give you a very brief definition of print disabilities, because I think we often, at least I didn't think of it in the broadest sense, and then give you a little bit of a preview of the findings and recommendations from the task force report. So besides the obvious visual impairments, such as blindness and low vision, print disability includes physical, perceptual, developmental, cognitive, and learning disabilities that render print inaccessible. For example, someone who physically is handicapped and cannot actually hold a print volume. Not something I had necessarily thought about myself. Now very quickly, and I apologize for moving quickly, but some of our key findings from the report, the number of students with disabilities in post-secondary education is growing. In the U.S., there are a number of federal laws that ensure accessibility. In Canada, there is no federal law. Policy varies by province. Also keep in mind that in the U.S., the publishers and vendors who develop the products we purchase or license aren't bound by ADA laws, but we are. The Department of Education and Department of Justice, who share oversight for disability law as it applies to higher education, have issued guidance that states that all programs, including pilot programs, are fully subject to non-discrimination requirements of the ADA, including ensuring access to emerging technologies. Very different strategies are required for retrospective print and prospective electronic resources to achieve accessibility for the print disabled. In the print environment, we have been used to fixing things after the fact, scanning and encoding text to make it readable by software and devices preferred by our print disabled users. The electronic environment retrofitting is even more expensive and won't scale. Integrated accessibility features are far less costly and more sustainable than fixing things after the fact. The Copyright Act has specific exceptions that support library creation of derivative works for the print disabled, but may not allow the copying and modification of some works without permission. Licenses for the increasing amount of digital content we acquire may also limit our ability to make works accessible. And the principle of universal design recognizes that designing for diverse learning styles and abilities reaps benefits for all. Now, briefly the recommendations, and these aren't exactly as they appear in the report, but trying to condense a little bit. One, the growing demand for instructional e-content and burgeoning digital library collections requires greater collaboration among all institutional partners, including academic leadership, research libraries, disability services, and IT services. Two, libraries should collaborate within each institution and actively participate in cross institutional and cross industry efforts to advance universal design standards for digital information resources. Three, universal accessibility should be embedded in future products and services, so special conversion to a usable format will only be required for retrospective works. Libraries should advocate for accessible solutions upfront with accessibility a central decision factor in choosing information products and services. Fourth, licensing must be done deliberately to protect the values and meet the legal requirements of accessibility. In those cases where an inaccessible product must be licensed, libraries should negotiate terms that allow the adaptation of content to meet the needs of patrons. With copyrighted works, research libraries should aggressively assert fair use and support of accessible services for the print disabled. Five, in anticipation of a ruling by the Department of Justice soon regarding the required accessibility of websites, research libraries should institute a plan to make all future websites, pages, and documents accessible while tackling older web resources over time. Six, libraries should have a user-focused policy and procedures for users with disability that is readily available and kept up to date. Accessibility service awareness needs to be a standard part of staff training and staff development. Seven, libraries should designate a library liaison who can provide or coordinate library assistance for their users in partnership with the institution's disability services office. And finally, libraries should identify a portion in the organization to partner with campus assistive technology experts and information technologists and legal services to stay up to date on technical and legal developments. I hope that when you have the task force report in hand, which we hope will be a matter of a few weeks, assuming the board actually approves it, that you will take the time to read the task force in its entire report in its entirety. It is an exceedingly rich analysis of the issues and the current content and prevents, I think, a very strong case for the role of research libraries in actively engaging and in leading these discussions on campus. Good afternoon. So I'm going to do a very high level. I'm not an expert on copyright. I'm not an expert on many things. I'm a leader. I know a little bit about a lot. And I'm hoping what will happen is the Canadian perspective will give you just a little bit of a different lens to look at the issues. So I actually came here to learn and Mary says, well, you have to give back. So here I am. And one of the things I'm interested in, best practices are good. But in this, I think we're looking at emerging opportunities and trying to take full advantage of them. So I'm going to talk very, very briefly on a little difference in our legislation, how it affects us, a little bit on copyright, some good news for WIPO, and a little bit on the model licensing. And I want to make sure I save the last five minutes to talk about a project that's going on a pilot project. A lot of stuff I do is from Ontario because it's easy for me, but I did do some other research in other provinces as well. So the interesting thing I always find, and I always have to remember when I come to ARL, is that education is not federal in Canada, it's a provincial level. So copyright is federal, human rights is federal. And then down at the state, down at the province, that's where our education is. And that's why we have so many different collaborations amongst our regions. The other thing that's at the provincial level are some accessibility. So while we have human rights in Canada, each province has its own code and it has commissions and has set standards and goals for the province. They look at regional needs. Now one of the outcomes of that is there's different opportunities for funding, because they prioritize different things. So that will become more evident. The one that I was involved in was the Accessibilities for Ontarians with Disabilities Act. And it was an act that would actually, the whole purpose of it, was to put in place standards to force people to comply and to try and help the whole province move forward. They wanted it to be more equal. In another one, they had, they tend to have different goals to all equal. The other thing that comes up amongst all of the provinces, I look more closely at Nova Scotia and Saskatchewan and British Columbia and then Ontario, is that when they talk about print disabilities, it's the whole gamut. It's so broad. It's learning disabilities. It's everything. We spend probably more than 50% of our activity helping people with learning disabilities. And then there are those that are visually impaired. So that's, and mental health is creeping up along the sides, huge on our campuses, particularly in Ontario. So another, another thing to keep an eye on. So that gives you a little bit about the lay of the land and how we're slightly different. Lot the same, but slightly different. Now, I don't know anything. I'm not the copyright officer. Thank you, Lord. So all I'm going to do is I'm going to, I've captured three clauses out of the new Copyright Act that was passed in June, 2012. So what's going right now, on right now is we have an act and they're in the process of putting together the regulations, the rules to implement the act. Hopefully we'll have those in two more months. And then of course the Supreme Court of Canada will ensure we all read the act the same way. But that'll be further down the road. Of course, the thing that we really like here is we now have education as one of the reasons to do it. The other one here talks about how we can send things to the U.S. And this is, I was really hoping she would stay and see this slide. I'm not reading it for you. Then the third one, in copyright they actually give a really good definition and in print disability they always mean learning disabilities, always. And they're very explicit in this. Now you know there's a lot been going on in Canada that's been very controversial. We've got lots of Supreme Court decisions that came down the month following June. So in July my favorite one was this one that shows you how much controversy there was in our country about copyright. So we can actually send it to you over the internet now. That isn't a second copy moving down the pipeline. Thank goodness. So that's a little bit on the copyright. The other thing you were interested in was making sure that you can get it in machine readable format from the beginning. And we actually found some language. So the Ontario Council of University Libraries, because we're all paid from the same Ministry of Education, we do a lot of collaboration. And if the Ministry or an Ontario government gives us money, they're touching 21 universities all at the same time. So it's a great way to bring in some money. And one of the things we do is we do consortial negotiations as well as the national, we do the regional. And at this case we have a group of people and they develop model license. So this is one bit of the language that we are using. And I brought another one as well. They haven't tried negotiating with these yet but they now have it in place. These are great people. They've been working with licenses for a long time. And probably if I went to one or two other consortia in Canada I'd find some. But this was the easiest one for me to get a handle on. There were a couple of others. So we can make them accessible in the report too. So that has to do. So right now I've talked a little bit about the difference between us. I've given you a little bit of update on our copyright changes. And I've talked a little bit about model licensing. And then the next thing I wanted to talk about was a pilot project that's going underway with the Ontario Council of University Libraries. So in 2005 they put together the AODA, the Accessibility for Interiors with Disabilities Act. They put a lot of standards in place over the next three years and they put some money on the table under this rubric of enabling change. And most of the time it's used for education. However we were able to get one under innovation. And it's a teeny tiny pilot project. And if it's successful then we'll go back and say this works really well so we would like to do it for all 21. But it's a teeny tiny pilot project. Lots of assessment going on Mackenzie. What they're trying to do is they're going to get three or four university libraries and together. And they will try and identify not textbooks. There's actually another project going on in Ontario called Arrow. And that's a little mini project trying to get vendors to provide textbooks in machine readable format directly from the publisher. And so they're trying to get the vendors to put workflows in place where it's possible. And it's in its third year. The colleges are involved and so are the K through 12.  And what they're finding is that some vendors are better than others. No surprise to us. But they continue to go and each year they allow a few more institutions to join them in that project. Not the one I want to talk about though. So the one I'm talking about is one where it's a textbook. You're in university education. You do research. You don't use a textbook. You have to have access to everything else that's in our collections. And a lot of it is still in print format. And so what we're trying to do is build a central repository on Okol Scholar's Portal Database. And it would be of things that had been scanned for students who have and faculty who have print disabilities stored in there. And then put a, it's like a dark archive if you like. Anyway, put it behind protection. The only people that will be able to search it, some library staff, actually want to give the faculty and students who have registered as students and faculty with print disabilities and agreed to be part of the pilot. So they can go in there and directly search and immediately download anything that's in that database. It's going to start off small. So the failure, the success standard, however you want to put it, the success standard is to get 500 items into the database in the first six months of the project. And then in the next six months, so one term and then second term, another 500 objects. And many of us have things. The theory is that you don't want 21 universities scanning the same thing 21 times and then throwing it away. So you want to do it once, you want to put it in a central place and then just store it. So if it's really successful, then we would have some more people come in. So what they'll do then is they're going to measure the log, see how many things are used once or twice. I know we need a bigger, I know we need a bigger critical mass and a thousand items for us to do good measurement. But if even if we can get just a little bit of a positive trend showing there, then we might be able to get more money from the government and expand it. The other thing they'll look at is the turnaround time. So if students can find something there, faculty can find something there, it's immediate. They don't have to do anything else. The other thing they would try and put in place is if you can't find it, then you can put in a form that you want it and try and expedite that as well. That's one part of the pilot project. The other part of the pilot project is to try and do some intensive training in up to three areas. One would be to empower people in areas such as acquisitions about how they should start to negotiate for electronic access of copies and what does it mean. Another one would be design. How can you do web design? We talk about web design. I'm going to just take 30 seconds here. When I was, the AODA in Ontario has five sets of standards. One of them happens to be for communication which is the web. If the government makes that web standard for everything really high, then they'd have to change all of theirs too and they can't. What they did was they drew a line in the sand and anything new after that point would have to be accessible. Something to keep in your back pocket. So they talk about doing what's good web design, having some webinars we could push out on that. And the third one is I think it happens to be customer services and how people should be able to deal better with students and faculty. So that is kind of, did I keep it under, I always promise to keep it under the 15. That's a really high level look at some of the things that are going on. I know British Columbia Public Library has done some surveys and they're doing some interesting things. We have a lot to share with the public libraries. And with the public libraries, there's surveys that people say, I just want an electronic book and I want to be able to get it from my home. I just want to be able to do it online. And then we have our university press and we kept saying, mark it up. Don't give us a huge PDF. So I'm back at that thing. I'll throw that out there as well. Our university presses could set good examples for having it machine ready. Going back to a more practical way now, get out of the legal and get back on the surface of what is our role here. And just to remind as I start with this, to quote the New Jersey governor, you know, what's our skin in this game? Well, just to remind you, of course, that we make information resources discoverable and accessible and we buy an awful lot of information resources on our campus and we have a common core value to make them accessible to all. You all know that. But one of the things I'm often pointing out on our campus is that when we're talking about software that we really don't necessarily have a role in selecting, at least on the surface, our staff are often dealing with it after the fact that at nine o'clock at night when a student's having problems with their course management system and getting in and trying to get our digital article out of it, which doesn't happen very often, but it does happen. Where did they come? They come to us. And frankly, they come to our library sometimes before they'll call the help desk and where they'll come online to us as a reference question online. And the reason for that is of course we're open late hours and they do know that, but also maybe in the past somebody's been friendly and helpful to them. And at that point they figure that's really where they're going to go. So we do have involvement sometimes in some of these course management systems and other softwares that are coming onto the campus that the development of web pages that aren't necessarily the library's webpage and so forth. And it really helps for us to know what's going on. As was noted, we have over 20 years of experience of converting books into digital format for the use of software that's designed specifically to create synthesized voice. And most of the time we've been pretty successful with that. But if any of you have been involved in these operations, you'll know that sometimes the conversion is really quite messy. Sometimes it's not really very successful. And it's very frequently time consuming. Now, there are new digital products, of course, coming out now that actually do transfer pretty nicely to some of the software that's commonly used by people with print disabilities. But not all of it. And you would have thought when we started to get born digital publications, I certainly would have. And we were beginning to see born digital discovery tools that really the need for an intermediary should be passing. That this should be a time when students who have print disabilities can do on their desktop the very same things that everyone else can do. But as we know, that hasn't happened. That the independence that these students have wanted to gain hasn't happened. And I have to tell you, I'm kind of amazed. Do you talk about a group of patient people? I mean, this has been 20 some years. And we know that the software, as Mary noted, that to go back at afterward and retrofit software to do certain things when it could have been set up in the first place is quite frequently much more labor intensive. We had a recent experience in my library with a very small kind of problem in which what could have taken, I've been told 29 lines of code in the first place, if it had been built in, took us 143 to add on later. Digital fixing is not a real scalable model in the long term. And then finally, when we look at the various accessibility accommodations that we're doing or our student for disability services is doing, that's also really not very cost effective. That when you think about it, it's like kicking the can down the hall and saying, this is your problem now. And this is your expense. Our person from Student Disability Services recently told me what was admittedly one of their worst examples of a problem. They had a nice, as he describes it, beautiful digital text version of a nursing textbook that they were trying to get ready for a student for a class. And because they were trying to get it searchable, and this book happened to have 1400 pages, all kinds of subheadings, illustrations, and other things that all needed to be coded. It took with one of their best students and they have some really good ones over at that office I've discovered. It took them 20 hours a day for eight weeks to get that thing in that form. Now, you know, they always tell these students that they need to have something like four to six weeks of notice just to be sure in case they run into this problem. But have you ever had faculty that know that soon in advance all the time what they're going to be using for their texts? Many of them do, but some really don't. So when I sat and said, what are we what are we to do here? I decided to look at three areas. One of them is getting our own house in order. And what I mean by that, I've talked with a lot of your folks in your libraries that do disability services, and they're a terrific group of people. But one thing I discovered from looking at a lot of your sites, a lot of the policy statements that we have are really designed for providing, you know, getting books from the stacks for people or referring it to somebody to do. And maybe we might want to look at how those are worded and maybe affirm a little more of universal accessibility in them. Furthermore, if you don't have a disability services liaison, and most of you do, as far as I could tell, it's a good time to get one. What I've discovered with our person, it isn't just that they're talking to that office more often. But boy, have they brought up things in our library for me to notice that I'm not sure anyone else would have before that time. The other thing that we may need to do, and again, I know you're all going to say, oh, we've done that, and we've done that until I discovered that, yes, we had done it in our library, but we did it three or four years ago, is to make sure that your staff are actually familiar with the range of print and other disabilities that our students have, particularly the invisible disabilities, the ones that most of us have on our campus, which is students with learning disabilities. And it helps because then it can sensitize our staff on how to deal with folks when they come up. Some people with invisible disabilities don't really want to stand out in a public desk and explain everything that's wrong with them. But very often what they are doing is trying to find out how they can get particular services that they may need. And furthermore, as I kind of as a follow-up, is to make sure all our staff, including our hourly staff, know our policy and know what to do when assistance is needed. I'll recall a little story from one of our directors who won't be named, who recently pointed out what was working on this project that someone had called one of their users in a wheelchair all upset because they had asked at their desk if someone could retrieve books for them. And frankly, they were told we don't do that. Now, I want to point out, of course they did that, but this could have been any one of our libraries and we've all had that experience occasionally where something that we just assume is pretty clear to everybody, including all our student employees, is not. So that's a good, this is a good time to get some of our house in order. The next thing I would want to point out is this, in working with our developers and vendors. There was a statement in one of the, actually, I think it was the EDUCAUSE Internet 2 textbook letter that went out. It's an invitation that I just saw recently that had a last sentence that struck me for a number of reasons. It was basically explaining that while their product was not fully disability accessible, they were making progress and it was getting close, but they wanted to reiterate the institutions, and this was in bold, responsibilities supplement them with reasonable accommodation for visually impaired students. This could be on the bottom of just about every agreement we get from a great deal of our vendors. As I said, many of the products actually transfer pretty nicely to some of the software, but a good deal of it does not. And I think that there is a sort of sentiment in a culture that we're going to fix this after the fact. And it really has come about, I think, from having to take books and actually do something with them. And that's kind of the assumption of how we're going to handle this situation. And we know we shouldn't have to do that. And I think that whether they're legally liable or not, we need to be conveying to our vendors that we expect this to come as part of the product. This shouldn't be in version 10.0. This should be in version 1.0. This shouldn't be an add-on that really we're paying the price after the fact for making it workable. And that responsible product development would require that this be the case. And I am sensitive, I will say, that there are proprietary rights involved with all kinds of people's software and products. And we know ourselves from all of the stuff that we work with that it's nearly impossible at any time to catch up with technology and have all of our materials readable on every device that students can bring to the library or are starting to use. But certainly we would not buy a product that wasn't readable, at least on a computer in the first place. And in a way that someone can hear it. Synthesized voice is not a new development. I remember seeing products 20 years ago in K through 12 that would read a word out loud. I mean, how many internet project products can you go out on now and have it translate and talk to you in Spanish? This is not really difficult. And so I think what, as we look forward on this, we need to be pressuring our vendors that this should be part of it. And then lastly, going back to our campus for a minute, we certainly have to help. I think we're in a position to really help increase awareness on our campus. What I have discovered is that while we sometimes in the libraries are a little frustrated that we're not involved in meetings and people don't think of involving the library because they're not thinking that we do certain things, it's even worse with student disability services. Unless somehow the word disability comes out early in the process, nobody, nobody thinks to pull them in. And as a result, we have a whole network of library liaisons and other people out there who can help spread some awareness and raise the issue of whether this works for disability students. Our own person from student disability services, I said, and what's your most common and worst nightmare that you have to deal with? And he said, faculty members who will, instead of taking that nice digital copy that the library is paying for in licensed form that is born digital and attaching that to their course management page or for whatever other assignment, they reach in the drawer and pull out the old photo copy with the black lines down the side. Then they scan it and by the time it's in third generation it's such a mess we can't do a thing with it. And then we're backing up and trying to find materials online. And I think that as our librarians are out there just calling attention to people that if they're going to link their classes to try to do it from our materials, you know, not retrofit something after the fact is probably going to be more accessible. And frankly, it's going to be a whole lot more readable by everybody in the class. So that's the one thing. The other thing I want to stress, and again I feel like I'm talking to an audience where I'm telling you all things you know, but having discussed this recently with so many people I find out it just doesn't quite work sometimes the way we think it is. I always felt that we had a really good relationship on our campus with IT and with our student disability services. And we do. But it's amazing how many things can come up at times and then they just don't think to tell you or to pull you in until after the fact. And we've had a recent experience that with this textbook situation and I think that I've probably been guilty of it myself with these folks in the past at times. But we really have to be getting a little tighter network here to monitor developments and technology. And I think we can actually help each other in terms of usability testing and sharing information. There's a lot going on. And so in closing, I would say more as a plea than as an explanation. I have a good friend who whenever she couldn't understand why something hasn't happened yet, she would say, you know, if they can create artificial intelligence and send someone successfully to the moon, I don't know why they can't do X, Y, or Z. And I would say the same thing here. If we can create artificial intelligence, do some of the computerized operations that are being done on our campus by researchers and send someone successfully into space, then can't we take born digital text and data and make it universally accessible through access tools provided by our vendors in the first place? And I think clearly it's a question of priorities and commitment. It's not a technological problem in and of itself. And it seems like we're really at a time now where that needs to come to the surface. In the spirit of full disclosure, I'm neither a librarian nor a lawyer. This will have absolutely zero theory in it, but it will be a practical kind of story about someone who strongly believes in lifelong learning learned over the last couple of years. It started, gosh, two years ago when Chronicle of Higher Education reported a story. Now, you have to realize the person who's talking has read a lot of stories about Penn State and the Chronicle of Higher Education. And in the simple days there was one story that said NFB through the Office of Civil Rights sues Penn State for, and then there was a very long litany of things, very long litany of things. And so by 6.30 I'd gotten a note from the president of Penn State saying, what is this all about as though I would automatically know what it's about and how to fix it. And so I didn't have any previous kinds of relations with the National Federation for the Blind. Like many of you, I work at a university that for more than 20 years has done exceptionally good work, a joint project between Information Technology and University Libraries that providing access, really custom solutions for students, some faculty, but mainly students, undergraduate and graduate. So I want to try to summarize here some of the things that Penn State has learned in the last year as we tried to implement a university-wide settlement. You'll all have copies of this. The URLs are available. What I want to do is highlight the key elements. Any kind of settlement is an agreement between two parties. And that is that we will do this, and we will do this by this date, and this is how we will determine success. That's all perfectly reasonable. And it took us about a year of interesting negotiations to get to that point. And at the very end, if I have time, I'll give you my private view of course load. This is the EDUCAUS initiative involving textbooks. That's not really the purview, but it's clearly in the environment and what affects us there, affects us other places as well. A word about Penn State, and I'm just going to list some staggering numbers. And the whole point of that slide is to say that Penn State is a large university. There may be larger ones, but probably not very many of them. 24 campuses, a large world campus, of about 96,000 students, 9,000 faculty, 28,000 staff, lots of research expenditure, 21,000 degrees awarded in 2011 and 2012, and 500,000 alumni. And the Alumni Association has about 170,000 members. The numbers that I love best are the fact that one out of 720 Americans are a Penn State alum and one out of 70 citizens of Pennsylvania are an alum. And the point here is that simply it's a way of saying we are large, complex, distributed, and have lots of moving parts providing different kinds of services to support the teaching and learning enterprise. The agreement has six elements that I'll talk about. One that I won't, not that I'm hiding anything, it had to do with the accessibility of a particular bank that is located in State College, many other towns as well. And the perception was that the bank's website was not accessible. Okay. And last I checked Penn State doesn't run bank websites. So I can tell you what I know about that, but I'm not going to tell you today. First question that I wanted to ask or was asked of me is how many websites do you have at Penn State, John? Like I personally keep track of them. Now, I didn't know and you don't know new websites are spun up every time someone gets a grant. So lots of faculty have ones, you know, to promote their research, recruit graduate students. How many are there? Well, the fact that nobody knows I guess might be an issue. And certainly in the complaint, they looked at some particular pages that were not accessible. And that was certainly very problematic. But it wouldn't make any sense just to pretend like it's a house fire and we bring out the fire company and they put out this fire. We needed to have a much broader view and so the smartest thing that we did right away was to create a committee. And this is the accessible information technology committee. And they agreed to step up and provide leadership. It also involved us licensing a piece of software that would enable us to allow webmasters. And the way that we organized it was to think of every business unit having someone who's responsible for a web presence. A business unit could be a dean, it could be a vice president, okay. And we know who those people are and we say we need to be sure that we're all in compliance. I spared everyone the technical details. You're going to get some mind numbingly technical things. Skip all that. It doesn't matter. There are people on your campus who know what that means. And so we agreed to scan and to do an audit basically and then to figure out what do we do once we get the audit results. Now a word about the web.  Think of 20 year olds on your campus. How are they? How are they doing? Well some are terrific. Some don't have absolutely terrific judgment. That's a pretty fair description of websites. I'll tell a story that my sister who is an associate dean of a medical school, I won't name the place. And one of her jobs was to supervise I guess websites for departments in medical school. And one morning she woke up and found that a department of orthopedic surgery had a new website. They're very proud of it. And they had a dancing skeleton as an animated gift. Probably not the right image really. You know if you put on not your gown and your mask but a thinking cap probably not the best graphical representation. Well anyway she's a pretty firm person and that was removed. But surgeons don't deliberately try to be malicious or subversive. You know the web is new. And so in terms of design in terms of accessibility there's a very let's say skills vary. And so some units we're going to have a very uphill battle. But we have three years to do the remediation. Okay. And the other thing that you'll see is that it's a reasonable settlement. Not all websites are equally important. Okay. There's a lot of web junk out there. Okay. That shouldn't be remedied. It should be removed. You know that project was over 10 years ago. It's of no earthly value whatsoever. But I'm not the person to make that decision. Colleges are, units are. So we developed the concept of triage and that is look at most important. We're going down only two levels. Some websites really is just like Jacques Cousteau was there. I mean they're going to go down 52 levels and even for sighted people try to get back up. Okay. Horrible. Okay. This has nothing to do with accessibility. This has to do with the fact that the web developed over time and we have lots of hands involved and no one really has thought too much about it. So anyway the best thing that we did was to form a university wide committee with folks from the libraries and I will spend more time on libraries simply because of this audience and also because they have the largest and most complex website on campus in my opinion. But finally this comes on individual stories and it provides auditing, training, support and guidance and the expectations that we will remediate. This is simply a review of scanning and identifying your most heavily sighted pages. Fairly soon I became a boy wonder because I knew that the 100 most heavily visited sites at Penn State and that was useful and interesting to know. But I think we discovered that we had thousands of websites, tens of thousands of web pages and so the magnitude was pretty significant. So scan, do a triage fix, scan again. We tried to scan twice a year. You might be wondering John, when will your project be done? It will be done exactly when the web is done. You let me know when you think the web is going to be done and Penn State will date it exactly the same way and not any earlier than that. So we have licensed software and we are working closely with people across Penn State. Second thing is policies. We needed to change policies and policies can drive behaviors and very quickly we adopted a policy on web accessibility. We cleaned up some other policies that sometimes it's like decorating a Christmas tree. We just add additional things on it. We had copyright added into one page but that really didn't have anything to do with copyright. But we needed to have something so we covered ourselves if we mentioned it. Well, we created a policy specifically with the definition of what we meant by accessibility and that's a useful thing to do. That was August of a year ago. After classes started the Penn State Faculty Senate, which is also a group that is not known for moving at lightning speed. It's a deliberative body. They also said we do have a policy that every course has to have a syllabus and certainly that's a breakthrough. Is it a best practice? Well, it's certainly a very good practice. I certainly think you should have one. We modified that to say that we must have a very specific statement about accessibility of course materials and a link back to the accessibility site, information on the Office of Disability Services, many things. Now, again, this is model language. You can adopt it, adapt it, but you must have something there. So that has gone into effect for this year. Those were really the only two policies that would be pretty visible. We have a mothership of information for accessibility, accessibility.psu.edu. It's a word I've actually learned to spell 100% accurately now. At first, I was not so good on it. Some of my colleagues in the Big Ten, I will not mention which one, say, that's an outstanding page. Can we just copy it? I said, thank you for the compliment. The answer is no. Why don't you link to it? And you might want to have some of your own local stuff. Well, that's a good idea, too. So I have no idea how many people I'll link to it, but it's a living site. And all of our communications will refer back to that site. Procurement is very, very important. And this was mentioned also in the discussion of the ARL White Paper. We procure lots of software. We procure lots of other things. And we have not had a formal process for making sure that these are accessible. And we are working with purchasing to develop and to educate them in terms of questions to ask, procedures to look for. And also, I'm going to have to tell you that even at my advanced age, I'm still pretty naive. Things like VPATs, and this is an assertion that it is accessible. And guess what? There are some vendors that are not 100% honest. Say it ain't so. Well, I say it is so. And that's a terrible problem because at the end of the day, if we license it and there's a complaint, guess who gets to answer the email? Well, it's not the vendor. It won't be procurement. So force management system, we were an early adopter of Angel. And one morning I woke up and discovered that it had been bought by Blackboard. And so we thought that we should take a look at Angel and Blackboard announced that it was not going to really put any more investment into Angel. It wanted all of its customers to move to Blackboard. That's a whole separate subject. And there were some inaccessible features of Angel, not a huge number. And some of them were very, very tiny. But so we began a search for replacement CMS. And guess what? Accessibility was explicitly built into our evaluation of four other course management systems. And guess what? The World Campus, our online unit, also was involved. And they had three blind students. God bless them. We always had at least three blind students evaluating every course management system. How about truth telling? You learn a lot if you actually have users from this community who want to learn, who are enrolled in courses, and they can keep a log. And of course, we will provide workarounds for things that are very, very difficult. That was fabulous. And one of the lessons learned that I would say is that if you have trouble explaining to folks at your campus about what accessibility means, bring in a blind student or bring in a blind staff member, a blind faculty member, and have them work with a screen reader and have them look at some websites. There will be no questions after that. There will be no questions about why we need to do this. And people will creep out of the room, ashamed of asking questions before. Okay. And there are blind students. Magnitude at Penn State and why we needed a university-wide effort. These are numbers from last year. I don't quite have them for this year. Last year, we had 50 blind students. Well, think of this as about 90,000 students. 95,000 students, large number. 50 blind students, 50 deaf students, 2,000 students with other learning disabilities. Okay, so we have 2,500. That would be great if it were a small college. But it's a small percentage. But let's think about the magnitude of the impact. Let's say that for our 50 blind students, they're full-time students, they take 10 courses a year. So 10 times, okay. Pretty soon we have interactions now with how many faculty? Lots. You can't do this as a series of one-offs. And in the past, we were all smart enough, and we kind of love doing the one-offs. We can't do one-offs. We have to have process. And that was the spirit in terms of developing workflow around this. No student at Penn State that I'm aware of ever wakes up some morning and says, hey, I'm blind. All the blind students at Penn State have known they were blind for some time, right? So we need to work with them and we need to work with their faculty in terms of what courses they're going to be taking next semester so that we have a decent chance to make sure that they're textbooks. Faculty choose textbooks, not students. I don't choose the textbook for anyone except my courses, so that we have a decent shot of the student having a successful learning experience. And that takes coordination with some people that is easier than it is with others. I hate to tell you that, but these are lessons learned. University Libraries Accessibility. It's the biggest website at Penn State, most complicated, and I think it's probably the best. They have a very, very skillful team of authors and a webmaster second to none. But nevertheless, they're using a commercial product. Adobe owns it. And what is the real killer is that they link to all kinds of external resources that we don't own, we can't control, and that are very variable in terms of how their products actually work. I'm telling you nothing that you don't know. But this has been a huge effort for us to do thorough audit and remediation of the most complicated site at Penn State. The additional complicating factor is that we have faculty authors that write all kinds of things, like course guides and Word documents, PDFs. And unfortunately, our vendors have not made it easy for us to check accessibility, and that needs to be changed. And I hope, again, we will leverage our collective strength as we work with vendors to make it easy. I'm going to say F7 ought to check this sucker for accessibility and flag things and not allow me to save it until it is accessible. Okay. How hard would that be for Microsoft to do? 143 lines of code. We may just outsource it to Canada. And, okay, someone needs to do it. Adobe is better. And I talk with them regularly about getting even better. I like to encourage people to do their best. But this is required a lot of rewriting. And I would not trivialize that effort. So they have a very good website testing procedure. And I'm not endorsing any particular product. But we do have high needs for it. High software is what you see up here. Its full name is something like web sheriff. And in my climate, I said I was not going to advertise anything that had the word Penn State and sheriff in it. And I'll stop there. And they have another version of the model here for the website testing. This is ongoing. And it is a good website now. I would encourage you to visit it. And it continues to improve. But it's not free. And they have a team of people who are dedicated to it, including a blind staff member who is the world's best tester. Some partners. Classrooms. Most of our classrooms at University Park at a number of the campuses also are so-called smart classrooms. I do love language. Smart classrooms. Smart for whom? Smart compared to what? What are the problems with a smart classroom in the sense that a blind faculty member simply cannot find a way to unlock it or to install things or to say I want a DVD? Just think about that. And we have combinations of things with padlocks. And, okay, think about it. This was painful for us. And I said we need to get this right. And we have found workarounds for this. And again, we're working with vendors to say you need to recognize that this needs to be, you know, for today's classroom, Bob and Susan, who are both blind, are going to be able to do a presentation. And we want it to be done as smoothly as it would be for a sighted student. And it's taken some work on that. Clickers. We were abandoning a product. But since the NFB mentioned that that was an issue, we, you know, guess what criterion we use as we're looking for replacement clickers? Well, we've built it in. Bake it in. Bake it in. Bake it in. If you don't have the word accessibility and the criteria for everything that you're doing, you're making a terrible mistake. And it will come back to bite you. And people are actually fairly used to my asking that question as we talk about innovations. Well, here's an innovation. Let's think about textbooks and digital textbooks. And you would think that McGraw Hill would know all about accessibility because they've talked to me about it for years. And yet they have a chance to do something new with 20 of us, not Penn State, by the way. And they were shocked to wake up one morning and have an announcement in the chronic of higher education that the NFB had written them the equivalent of a dear colleague letter. Well, there are all kinds of ways of avoiding that. And I think for higher education and for all of us, there is a tension between innovation and our desire to find new ways to provide services and to deliver key materials, a tension between our desire for innovation and also our obligation, legal if not moral, probably both, to be sure that all of these services and materials are accessible to students or that we can make a reasonable accommodation for these students. Okay, that they're simply not frozen out from the get-go. That's what this is about. And I would simply say that shame on us if we didn't build that criterion in and now they're going to have to retrofit. And I'm not endorsing or I'm not un-endorsing or opposed to this per se, but if it's going to work with a Kindle, if it's going to work with an iPad, if it's going to work with any of these things, it needs to be testing. And the best people to test it will be a combination of IT staff, library folks, and disability services people. And I'm going to stop there because there are probably going to be questions. We do have a few minutes for questions. I can't see you, but if you're out there, please step up. Mary, I have a question. I'm Ed Van Gamer from the University of Wisconsin-Madison. And I think my question is directed to John. John, I'm interested to know what feedback you've received from the NFB on your work at Penn State. We have had some conversations of when we decided to renew our contract with Angel. We called them to explain because otherwise it said that by a certain date we would have announced our selection of new course management system. And when the world changed, we wanted to respond to that. They were very reasonable about it. They are very principled. They're very reasonable. They know the fact that I don't know exactly how many web pages or websites there are at Penn State is not really the criticism of the way Penn State does business. They're not trying to get at that. And they say, yeah, you don't have to go all down 50 levels. But this is reasonable stuff. I would not have any hesitation to work with them. They also have lots of expertise. I'm Pat Burns from Colorado State University. One of the toughest nuts we have to crack is captioning videos. What do you do about that? Not in the library for library stuff. But yeah, I'm sitting here thinking the very same thing. All this on YouTube. And we know that film industries do it. And certainly when I was on the Ontario board looking on that council, trying to look at standards that came up, a lot of films do now provide it. But how to go back? And at one point the standard said you have to go back. We have an archive of film that would have been crippling. But going forward, we should be, as you're saying, preparing. Yeah. Paula Kaufman, University of Illinois at Urbana-Champaign. Many of you know that our campus has been involved in disability services and research for many, many decades now. Things happen sometimes serendipitously. And this morning I received a message from somebody who works with educational technology telling me that we are still not participating and probably won't participate in the EDUCAUSE Internet e-textbook project, but primarily because we have been developing our own accessible textbook software. I'm supposed to get a demo of that in a couple of weeks. And if it looks promising, I'll certainly pass that word along. And John, you're probably aware of this. Yes. Yeah. Thanks. And I'm very, very interested in it. And I would just give a shout out to Illinois. I think they have a very solid research center, and we have learned a lot by working with them. Jim Neal, Columbia. The speakers have focused on accessibility issues as it relates to students and faculty. Could they comment a little bit on the administrative systems at our universities and their ability to enable staff with print disabilities to carry out their work? Jim, we can, but we'd rather not. It remains a process. At Penn State, we have had home-developed largely tools. We have a relatively small number of buying staff. All of them are able to do all of their jobs on our administrative systems. We have smart cards for some things, you know, a fob that are not really, and so we've simply given waivers to people for that second factor authentication. But I've looked pretty hard and have not found anything that is not really workable. However, if you really wanted to come down hard on us and to look at the accessibility of fax machines, I mean, I can barely use the fax machine in my office, and I tremble whenever I have to do it. Well. John, I'm concerned about library management systems. Okay. Don't know anything about that. No good answers, I think, Jim. Tom? But it remains a concern. Yeah. Tom Leonard from Berkeley. I was interested in the Penn State ratio because when we took a snapshot at Berkeley of how many people we needed to serve, it was with a smaller population, almost exactly the same. And that became relevant when we tackled this issue of allowing folks with visual disabilities to roam through our legacy print collections. These are not works that are signed. They're not in the curriculum. They're not on reserve. They're just there. And the inference we drew was not that we were only going to serve a few people, but this was going to be good because we weren't going to be overwhelmed. Yeah. One thing I might share with you on that that I've learned in talking with our disabilities person, he was telling me that one of the things they're a little nervous about is that in the past, the way that, and this is kind of following their professional standards of the professional association, and the way they've determined, you know, which students are sort of disabled on the campus and qualify for their services, has been as a result of sort of some standard testing that they've had in the past. This is speaking mainly in learning disabilities, not someone just coming in and kind of saying, I think I have, you know, and so forth. But that the new standards that they expect may happen, come along, maybe something where it's very much more open and very much more, I think I have a problem and I, you know, it would be easier if I can do this, which in and of itself might not seem like an issue, but I know he said they're really quite concerned about the number of students that they may have to serve under this policy. Not because they don't want to serve people, but because they're not staff to do it in many of these campuses. Please join me in thanking our panelists this afternoon. And I do want to personally thank the members of the task force. It was an incredible amount of work done in a very short time. And my thanks to Pru and Judy, who owe it all very well together. Thank you for listening. Music was provided by Josh Woodward. For more talks from this meeting, please visit www.arl.org.


This paper reviews various enhanced sampling techniques in molecular dynamic simulations, including collective variable biasing and tempering methods, offering historical and contemporary perspectives. The paper discusses mean force biasing using adaptive biasing force algorithm and temperature acceleration as well as bias potentials such as umbrella sampling and metodynamics. Additionally, parallel tempering and replica exchange methods are considered, concluding with a brief presentation of combination methods. This article was authored by Cameron Abrams and Giovanni Bussi.


This holiday season comes Mantik's most terrifying production yet. It came from a deep dark cave.  Just when you thought it was safe to play third edition, Slasher! Also appearing in stores this holiday season, Tentacles ate my baby.  The Phoenix. New Kings of War Titans in store this December.


Hi everyone, my name is Ida Syesh and I am a senior product manager over at Squarespace. So today I'm going to be talking about building love products and bringing them into product management just a little bit about myself. Like I said, I'm a senior PM over at Squarespace. I work on the Unfold team. Unfold is an all-in-one creator mobile app. Prior to Squarespace, I worked at Rent the Runway where I was the PM for growth and acquisition. Prior to that, I worked at American Express where I also did consumer acquisition and on the card applications and prior to that, I was at L'Oreal in marketing. So I'm really excited to talk to you guys today about love products. So really quick overview, we're going to talk about what is a love product, give some examples and then talk about how to really bring them into the product management cycle. So before I jump into the topic, I wanted to take a big step back and ask the question of what is the purpose of a product manager? Obviously, there are so many answers to this because one, PMs do so many different things and two, being a PM is really different from company to company. I'm sure you guys have experienced this. And also in a lot of your coursework, you'll see that the role of a PM is super multifaceted. But just from a general sense, what does it mean to be a PM? So we are here to build winning features. We build features that solve customer problems. We are always working to identify the business need and the customer need. But we're really here to build the products that are important to both users and the business. I want to take another step back here and also talk about, well, what is the purpose of a brand? All of the products that we're building ladder into a specific brand, the company that you work for or the brand that your products are umbrellaed under. So what is the purpose of a brand? There's so many answers to this, similar to being a PM, and it really can differ from brand to brand. You know, they're there to provide continuous products or services. There needs to be brands that they can be identified within the marketplace to build customer trust and understanding, to give meaning to products. But people really need brands to connect and understand the services or the products that are being sold or provided to them. And businesses need a way to kind of differentiate themselves and let consumers know what they really stand for. So going back to my previous work experience, like I said, I worked at L'Oreal and I worked in marketing there. So L'Oreal is a conglomerate and not unlike a lot of other beauty companies, L'Oreal owns so many different brands. Estee Lauder is really similar to this where they own a lot of brands and, you know, Procter & Gamble is another example, Johnson & Johnson. But something that would come up a lot while I was at L'Oreal was just the concept of a brand. What do they represent for users? What do they represent for the marketplace? What do they represent for the business? You'll see here L'Oreal really, you know, subdivides the different brands that they have based on where they kind of sit in the market. And they use that to not only decide what brands you acquire, but to understand like holistically how much of the beauty sector they're covering. But this concept of brands and the meaning of brands was in everything that they did. And it was really, really important to them and to that industry. So something that was really core to that experience in doing marketing or product development, which is creating actual physical products at L'Oreal was this concept of love products. And this really stuck with me throughout my career when I moved into product management. It is something that they would talk about constantly. It was really core to L'Oreal's mission and their strategy. They were not just trying to build brands that could sell a product. They were building brands that wanted to create an experience and a lifestyle that users could love on an ongoing basis. So when they would be acquiring a brand or talking about branding or talking about a new product they were creating, they were really often talking about, well, is this going to be a love product? An example for L'Oreal of a love brand is Maybelline. Maybelline is really like at the core of a love brand. Some of the products that are really, you know, innate to the brand and really famous for our baby lips or gray lash. People absolutely truly loved these products. They, you know, people will use gray lash for years and years and years and are real advocates of it. Or baby lips, people would go to many different stores to buy the different colors. Essie is another love brand for L'Oreal. People go crazy over Essie. They get excited over the different colors and the names of the colors and they're really excited for when a new color comes out or when they have a partnership. Garnier Makeup Removing Wipes. So Garnier invented the Makeup Removing Wipe and it really changed the beauty industry and a lot of people are incredibly loyal to this original product. So this concept of a love brand is something that I thought about a lot as a PM and something to point out is that L'Oreal was never naive about how important a love brand is. You cannot be a love brand for everyone, but it should always be your goal to create a love brand for who you specified are important users to you. So what is a love brand? Love brands are the ones in your life that you are loyal to. The brands that people would use to describe you or you would describe yourself in the things you love with this brand.  You are always telling people about it. You're referring people to this part. It always excites you when they release new things. You really like to be associated with a love brand. I'm sure even as I'm saying this, there's a number of you who can think of the brands in your life that are love brands for you. So here are some examples of love brands. I would say Apple is probably the most famous, especially in tech. They really nail the idea of products people love, especially in hardware. iPhones and AirPods and iPads and iPods, every one of those products are love products and they really change people's day to day life. But there are so many examples of love brands. Nike is one that's only grown over the years. Trader Joe's, Dunkin' Donuts and Starbucks respectively in coffee are love brands. People are real advocates and if you love one, you most likely do not love the other. Target is a great example. There are Instagram accounts dedicated to all the cool things. People find a target because they want to share that with other people. Supreme and Glossier are good examples of brands that have cult followings, especially Supreme. There's always a line around the block for a drop. People are always waiting in line to get into the Glossier store. I think a really good example too is Wawa. I'm from the West Coast and when I moved to New York, all of my friends from Jersey would constantly be talking about this place called Wawa. Oh, have you been to a Wawa? Always so excited about it. And I was like, what is this gas station convenience store that people are raving about? How bizarre that people can be such advocates and so in love with something essentially like a 7-Eleven. And it wasn't until I went that I realized that they've really created a love brand between the product and the experience. They just created something people were really, really passionate about. And there are so many love brands out there, but every brand can't be a love brand for everyone. And that's okay. There's always a question of how many love brands does a single person have? Like you probably have one, maybe two in each segment of your life, but you definitely have those brands that throughout your life you've loved or right now for a passing moment for a few months for a year, you're really into them. So this also begs the question of why do people love a brand? Well, there are so many reasons and it really can differ person to person or where they are at in their life. But for the most part, it's because they connect with it on a personal level. It likely addresses a need in their life or maybe it motivates them and inspires them. It brings them joy. This is especially true of things like beauty and fashion, which maybe they don't have a problem where they need more makeup or more clothing, but it's inspiring. It's exciting. It brings them joy. It's just fun for them. It aligns with their values. More and more we see brands having missions and users, customers really taking that seriously. It could help them achieve their goals. It could help them visualize their goals. There are so many reasons that people love brands and that's really exciting and it just shows the power that brands and products can provide users. So when I left marketing and moved into being a product manager, I constantly would think about this idea of love brands. Why don't we have it in product? And obviously, there are many different kinds of product process and we always talk about building products that spark joy for the user and that can be definitely related to love. But oftentimes we are really specific about the types of things that we are solving for in the product process and you don't really hear people say, oh, because people will love this because it'll be a love product and I always wonder why we were bringing that in more. Obviously there are business goals that are associated with it but where are the love products in tech and in product management? Obviously, there are love products out there in tech. A lot of them tend to be associated with a larger brand, something like Instagram or Apple products being associated with Apple or Google search engine just being part of Google or TikTok. But it can be hard to really when you have so many business goals to bring in the goal of having a love product. So I want to talk about a few really good examples of tech products that are love products and the first one is emojis. This is such a universally used and loved product that it's hard to imagine a time before we had it. I mean, everyone uses this from teenagers to your grandparents to your parents to you. Everyone uses emojis and it really reaches so many different people and I think one of the most interesting things about emoji that really speaks to its success and how much of a universal love product it is is that this was a standalone app that people were downloading themselves years ago, maybe 10 years ago and adding it as an add-on to their Apple keyboards. And Apple acquired them and literally built it into the iOS, built it into the keyboard itself. Not every product can touch people as much as emojis can in reach so many people but this is like a true love product that has really changed the way people communicate with each other. People get excited when there's a new iOS update for there being new emojis. People request new emojis that they think would be really important. We see reactions on Instagram and Facebook and text messages kind of coming out of this idea of emojis and the way it's really allowed people to express themselves in new ways. Another example of a love product today is the Starbucks mobile app or mobile payment app. This is currently the largest mobile payment app. That is truly tremendous. It has more users than Apple Pay and Google Pay which really is an astonishing thing considering you can only use it at Starbucks. Apple Pay and Google Pay you can use anywhere. To begin with, Starbucks was a love brand but they didn't really rest on that. They continued to innovate and build products that were specifically tailored to their customers and the needs of their customers and they were really paying attention to that. It's a really seamless experience and Starbucks is known for people go there every day. So they created a product that helps achieve both business and customer goals. And something that's really unique that both users and the business really benefit from is these auto cash refills.   You've already paid for it might as well choose Starbucks over Dunkin' Donuts or this other coffee shop because you already have money towards them. And they have a rewards program built and you can order in advance and they really took a moment to think about what are the business goals here but how can we create a product that people love when we already know they're coming into our stores every day? How can we enhance that experience for them? So how do we build products that people love? How do we bring that into the product process? The first part is really in the discovery and ideation phase. This is really where it's important to be thinking about love and the role of the customer. So there is a quote from Steve Jobs that I think is really relevant here. He said, people don't know what they want until you show it to them. That's why I never rely on market research. Our task is to read things that are not yet on the page. So obviously it's a really bold statement to say you never rely on market research. I think market research is important but this notion is really powerful that you have to read between the lines. You cannot have the customer tell you everything. That's just not how it works and that's not how you win a winning feature. There's these stories of when Apple was in ideation for the iPhone and then people said that the thing that they hated the most was touchscreens and obviously the first iPhone was a full touchscreen. If they had just listened to that and added a keyboard, history could be different as we know it. But they realized people don't like touchscreens that don't work. So let's build one that works. So customer interviews are critical to understanding the needs of the user and having a direct line of communication to them. This is really your only direct line of communication with your users and you should really take that seriously. You can't rely on the customer telling you what they want. They are not product managers. You are the product manager. If everyone knew what they wanted, they would all be PMs. And we know as PMs how much work it takes to really understand what people want, why they want it and how we should address it. It's really important to take these customer interviews and learn and understand the needs and the problems and the interests of the customer. You need to understand who they are. Sometimes users will say, I want you to build this thing, but you have to take it a step back and really understand why is it that that's what they want? What is the problem for them that we're trying to solve? What is the customer job they're trying to hire us for? These interviews and communications with customers should not just be a one time thing to solve a specific problem. They need to be ongoing. We've all been in situations where we have something that we really want to prioritize on the backlog or leadership wants us to prioritize. However, we go in and we have interviews to validate hopefully our hypotheses or prove something else. And we're only really asking questions for that one feature. And we don't go back to talking to the user until we have the next feature. This is not going to help you in the long run build love products because you're really only talking to users in a vacuum. You really need to get to know them. Trends change. The market changes. Your users and customers are changing. So you need to understand how they're changing on an ongoing basis to be able to really build those high value love products for them. Also talk to customers about other brands that they love. This is a really great way of understanding who they are and where is the success of other brands that they love. Next time you're at a dinner party and someone is raving about a product, engage with them and try to really understand what is it that you love so much about that product. Try to understand what is that stands out to people and think about how you can bring those things into your own products. The next piece here is to look outside of the business. Building products that are loved really requires PMs to look at the marketplace as a whole. You cannot. Obviously it's important to look at competitors in your space but it is just as important to look at other industries and figure out what is going on. Their trends in other industries can downstream impact your feature or just what users now expect from the brands and products that they interact with. Every day your customer is interacting with hundreds of different brands and different products. Your product needs to fit within the larger ecosystem of their lived experience. It will really leave you blind to think about your product just as, okay, when my user uses this product or is trying to discover this product just in that realm. We just think about how does it fit into their larger life and their larger day and the other brands that they interact with and the other features and the other apps and websites that they are on every day. Like I said, look to other love brands. There is so much that you can learn and you might think like, oh, what could I learn from Dunkin Donuts or from Supreme? But you can really look to what is the relationship that they're building with the customer? What are they invoking in them that draws love? Design with empathy. So we talk so much as PMs about user empathy. All PMs have to have user empathy, but what does that really mean? Simply talking to a user is not going to drive user empathy. You have to put yourself in the shoes of the user. Obviously, most of the time you might not be building a product that is for you, but you can think about the product that you love that brings value to you and you use that mindset to kind of click into your prospective customers and clients. Always try to channel those feelings and that is what will help you have user empathy. Love products should do more than just bring value for a moment or solve a problem for a moment. They continue to inspire and excite users on an ongoing basis. And continuously check in to make sure that you're thinking through the lens of the customer. So I mean at every single phase, really check in with yourself and say, did I make this decision as though I was in the shoes of the customer thinking about the problems and the life that they have that I kind of been determined earlier in this process? So continue to really check in. So when you moved out of discovering ideation and now you're in the build phase, working cross-functionally is gonna be a huge part of this. Obviously, as PMs, you know, you can never build a product alone. It takes tons of cross-functional effort to build a product, but that being said, a love product takes even more and getting everyone really bought into this concept of a love product will take a ton of effort as well in cross-functional work. Marketing is key. Your product marketing counterparts are so critical. People can't love a product if they don't know about it, if they don't know how it's gonna fit into the life and how it's gonna improve their lives. Maybe they'll never even download the app without marketing. Maybe they'll open the app and not understand the value of the product just looking at it. This is where marketing comes in. So much of the success of love brands that we see is specifically driven to the way that the marketing is done and the way it really pulls people in and gives them an aesthetic and aspiring and exciting look at the potential that this product can bring to them. Apple does a really good job of this, obviously. You know, the way with iPod to Apple Watch is every product that they release, their marketing is so strong and fun and whimsical and bringing people into not only understanding functionally what it does, but how it would fit into their life. Working with design is also obviously fundamental.  So bringing them in early and making sure that they are able, that you're giving them the opportunity to channel a love product. Oftentimes we work with design and we have such tight timelines that we cut a lot of scope and we take away a lot of really fun and whimsical moments. Make sure that you're thinking about that when you're working with design and that you are empowering them to channel a love product and obviously leverage your engineering teams early. I think often as PMs we think that we are the true holders of what the customer wants. We're the gatekeeper to that. But engineers on a daily basis when they are building things out are putting themselves in the shoes of the user. They're a good engineer will always say to you, I don't think that this is the most ideal experience. It's going to be weird for the user. Bring them in early. Say, hey, we're building a love product. Help me out with this. Think about that as we go through user stories is what is this experience going to feel like for the user? Is it going to get in the way of a love product or is it going to encourage more love behavior? And check in constantly. I know I already said this, but it is so important as PMs will be in build and will feel that we're really far down the line and we're like, don't want to ask any more questions. But it's important that you never lose sight. Continue ask yourself questions like, as a customer, would you love this product? Is this product solving a problem for you? Does this product excite and inspire you? Like I said, obviously not always is a product going to be something that you use. But at this stage, you should already have done that work through customer interviews and user empathy to really put yourself in the shoes of the user and know what it is that they want that enables you to create that love product and put yourself in the shoes. And especially it's like, well, I love this. Does this excite me? Does it inspire me? Things like that. So in summary, there is room to bring love products into product management. It might not be in your product process now. It might not always make sense for every single feature that you build, but there is room for it. And it's important to understand that building a love product will always help you reach your business goals and your OKRs. Because the better the product is, the more love users have for it, the more likely you are to increase retention and conversion, acquisition, keys or actions, things like this that are really important to the business. So don't think about love products or creating what that lends as deterring you away from achieving your business goals. And the second key takeaway here is get to know the customer and think about their overall experience. You can never know your customer too well, but make sure that you are not relying on them to tell you the answer. And if you spend enough time talking and engaging and having empathy for your customer, you will understand what it is that needs to be built. You will not feel as though you have to build exactly what it is that they might be asking for. All right, thank you so much. And I hope that you guys get to build a love product soon in the future.