in such we visited all six student clubs in person to get those students excited about our program and so in order to do this we wanted to define what high achieving was because we were aiming to work with high achieving students and we basically believe that students who are meeting as part of an honor society or a student club and working with topics they're passionate about we're going to be more likely to edit than those who weren't. The photo that you just saw was a student who works with NASA in her spare time as well as in her student club so pretty high achieving. To answer your question will students edit in their spare time? Not really. We worked with 44 students and only two students out of 44 made like good quality contributions but we did see a lot of enthusiasm so before we got to that answer we went and visited all six student clubs in person. We ran two field trips with two of the clubs in particular to get them engaged in Wikipedia outside the classroom and we asked them to take photos of the field trips we went on in Arizona and in California and those two clubs that we worked with did contribute really great content we had two new articles created based off of those field trips but two new articles created we had 21 images uploaded we had 46 articles that were touched by students but at the end of the day that level of enthusiasm didn't really contribute to high quality articles beyond just those two so for us at Wikiyed we're all about scaling programs we want to know how can we make this bigger how can we get this run all across the country and doing a program like this just wasn't worth our time we put in so much staff time so much travel energy a lot of money to visit these clubs and we just didn't see that much content contributed so to learning was that with an incentive yes some students would edit but most of them were just too busy and just didn't quite have it in them to edit outside the classroom but it didn't mean that we didn't have a really great time running the pilot I personally thought it was great the whole report is available on meta if you want to learn more about the pilot program and I'm available to chat if anybody has questions about any of the outreach work that we do all the photos on here were students that we visited with in-person so if you want to ask me questions feel free to follow up at the end thank you so the wiki education foundation bridges academia and wikipedia the primary way we do that as Helene talked about is through the classroom program where we connect or we provide support for instructors and students as they work on wikipedia for a class assignment the purpose of the community engagement program is to make the bridge more bidirectional so in addition to bringing classes to wikipedia we want to bring wikipedia to the resources available in academia the idea is to empower the existing wikipedia community after all the goal is to improve wikipedia why not empower those who are doing so already so the question is how can we get these resources to wikipedia and what resources are available the first major initiative as part of the community engagement program is the wikipedia visiting scholars program which was piloted and developed by the wikipedia library folks and now run by wikiaid in the united states of canada the idea is that we connect experienced content writing wikipedia with academic libraries editors get free remote access to high quality research materials and use them to improve articles and topics of mutual interest libraries increase the impact of their collections and make a contribution to public knowledge in a relevant topic area so far nine institutions have sponsored seven scholars three of them are in the selection process now actually two more have come on board since creating the slide so we started a big push to grow the program if you think that your institution might be interested to sponsor visiting scholar I hope to talk to you at some point this weekend so here are four example articles that visiting scholars wrote a couple of them are new articles a couple of the others were existing either start level or stub level articles so babe ruth cultural competence and health care montana vigilantes they were all created in the last round of visiting scholars here are the three that just started vaibholt at george mason university he was part of the pilot program and were able to renew them for another year he is a featured article writing machine and actually he was part of the 2017 last year as part of this program seattle and bf page are just getting started but already improving many articles and uploading images so if your inexperienced wikipedia editor and want to partner with an educational institution to get access to library holdings this is the page where you can fill out an application there will often be open positions posted but you can also apply if there aren't the idea is that we keep qualified applications on file and when we do outreach to universities we try to make connections where relevant so the visiting scholars program is a big part of the community engagement program the other project has to do with the year of science that we're doing next year other ways that we can empower wikipedians are we talking about this a little bit in the session later along with Ian about the year of science but the idea is to how we can engage our connections leverage our connections that we developed through the classroom program in order to bring resources to wikipedians and what kinds of ways we can do that so we're still researching that at this point thanks I know we've gone over at this point so if you're hungry and you need food immediately you should feel free to go but we would all be also happy to stay here and answer any questions we are funded by grants and donations we are not an affiliated organization with the wikimedia foundation and we do not receive money from the wikimedia foundation at all so one of our major funders is the Stanton foundation they funded the pilot program of the public policy initiative which is where our organization grew out of and they continue to fund us we also have as well anonymous donors who give to our organization to support our programmatic efforts there's a point there expectations that one had there but I think that's really actually quite useful because so many people in the wikimedia movement are going in different directions and people are still doing editons and now years later we realize hey if you have an editon with 20 people maybe one person will result in a long time you mentioned I forgot which one you mentioned about the report that's on MEDA but like if you had some kind of overall report about like these are the things we try and these are the things that in these circumstances don't appear to work that could be a really useful way to direct them there's been a series of reports that we've put together on MEDA so the history of our program for those of you who are unfamiliar with it started as a pilot program at the wikimedia foundation we spun off into an independent nonprofit a year and a half ago, two years ago and when we were at the wikimedia foundation we were part of the the programs team and program evaluation was a big part of that team as well and so we did a lot of work on putting together program evaluation reports on the work of the classroom program so there's a lot of really good information documented on MEDA of what works and what doesn't work in the education sphere that I highly encourage you to check out the two pilot programs the summer seminar and the high achieving students outreach pilot that we've done both of those also have full documentation reports available on MEDA and we are absolutely an organization that believes in not recreating the wheel and in making sure that we are out there documenting what we're doing documenting what works and what doesn't work you know, it's challenging to get up here on a stage right and admit you had two pilot programs and neither of them you're continuing to do because they didn't have good outcomes but we would much rather come up here and tell you, look we tried this it didn't work, don't do it and not continue to waste volunteer time on something and have others try to replicate what we did and only find out that that's a waste of volunteer time, the volunteers in this movement are the most precious resource we have voting them to things that work to actually generate content and generate editors in a massive scale the exceptional students program that was targeted at secondary school high school only university level is there any plan to expand some of this to secondary schools as an organization our mission is focused on being the bridge between higher education in the US and Canada and the Wikipedia world so we're focused exclusively on higher education we've seen some pilot work that's been done in high schools by other groups and I think it's a lot more challenging to locate the good writers, good researchers good critical thinkers at the high school level than it is at the college level it takes a little more effort and we haven't hit the maximum potential yet of what we think the university level can achieve and so we think in terms of staff resources and time and where we can devote our energy we still think there's a lot of untapped potential in higher education that we want to focus on first and that's not to say that high schools aren't possible and high schools aren't good and there's not a lot of potential there too but I think there's still plenty of low hanging fruit in higher education we haven't tapped yet we're looking forward to tapping over the next few years goals are different my goal is converting some groups of vandals into somewhat productive training program a second is perfectly acceptable for the secondary school students I think actually the question that was asked during Andrew Lee's talk about the educational games I think that could be a real high potential collaboration there for turning vandals into vandal fighters in some sort of game based exercise but that's not something that is currently one of the focuses of our organization those are great questions anyone, yeah it seems like there's a bit of a, like have you seen the movie Armageddon with Bruce Willis? I have not but maybe someone else has had that the plot revolves around training like people who drill to become an astronaut in order to save the world and like realistic you do like, wait wouldn't it be you should train astronauts to drill so like I guess what I'm coming from this is it seems like that editing Wikipedia has two different parts of it has a bit of a programming aspect to it and it seems like things would work if you just teamed with researchers who are already writing very elaborate paper citations with another department that does computer programming so one of the things that I wanted people to get from the syllabus I think the best way to treat this is not a sort of some crazy thing you know if you ask a student to write an example in psychopedia let's say it's a grad student working for a discipline specific in psychopedia or ask them to write a particular literature you would structure your course around getting them to map their sort of talents and interests and efforts into the expectation of the medium that they're writing there's no difference in terms of like how that works on Wikipedia and the courses that have we really define goals and like professors that give their students the same kind of feedback in respect to any other kind of assignment like they tend to succeed you know, beyond our expectations right, I mean like this what you all sound like sounds very similar to the same that I had in college was to create a blog and as part of it was learning how to use word press and translate a research paper into a blog so that a lot of people weren't able to I mean, that's a well, as a person who does our outreach to talk to all of our instructors the reason our classroom program I think has been has been that to be thought is they are already a minor student to research but instead of writing it in terms of or having to say they're going to throw the trash or something like that in the end of the semester they're learning this new medium to put that content onto and that's, I think that's why classroom program has been so scalable because we're just tapping into existing infrastructure within the university and that doesn't exist in the other you know, we try to make them do something new you got a question up here I actually wanted to ask about the the extra curricular clubs that you mentioned do you specifically think out clubs that were organized around like what type of club were these I'm just curious like over 200 student clubs across the United States to see and they were all talking about their own specific passionate topics so in the academic, usually like I worked with the geology club I worked with a couple clubs first at the water resource management I worked with a club that was studying art and design so I tried to pick student clubs where potentially they were being affiliated with an academic association that was overseeing their own work so they have a GPA requirement in different course requirements or just any club that's meeting around one specific academic topic so it was really pretty broad in terms of what we decided to engage with, yeah the reason that I ask is because I'm a senior at the University of Chicago and I've been doing a lot of modeling when I don't know if you've ever seen a model you went back on at the college level but yeah, it's one of those things that not very many people look at because why would they but model you went clubs tend to spend the entire year doing research on the topic and the background guide has to be written in an encyclopedia format and so that seems like something I'm going to want to start to forget but that seems like something that would translate really well because one of the biggest problems that model you went clubs has is getting people to do their research on across the year as opposed to just doing it all in one night and so working out a way that did their research earlier and looked for different formats of information and looked for different ways to organize their thought could translate into a thing that show off and be like I did this in addition to doing this other thing one of the questions that we left unanswered was if we could institutionalize the Wikipedia participation of these student clubs from somebody like model you went who said as part of your department to be a member of our club you have to contribute to Wikipedia as a capacity I think that that would be like a great students in the classroom program contribute great content because they're being graded on but if you're working truly extracurably you're not being graded your only true value is what you derive out of yourself and that's just so hard to college students so I would really love to do that because it would seem like the type of organizations that would kind of work well in the middle of that in the drive where there's an actual structure of organization personally within that I have a question I would like to ask if you're being too hard on yourself by saying it pale the reason the background behind this is that you have reached out to them you have told them how Wikipedia works how to create an article that stays with them so did you did you consider that aspect that somebody who's a freshman or maybe even a high schooler who has learned this for the next 20, 40, 50 years and maybe someday when he has some time or motivation that's going to come in I think that's true about our classroom program too and every student who is exposed to what we're working with learn something, what and how they do in a site because they're learning how to read Wikipedia how to participate in it in a really engaging way whether they actually like edit it or not I think it's a lot of investment that they can upload later the question for us is not did the student learn something that is super useful is it worth the staff time like is there an impact on Wikipedia that's directly measurable that is worth the staff time and resources that go in there are so many different programs we could be doing Ian mentioned our dream to do some sort of ask an expert program I think that's what personally I would like to do is kind of our next pilot of try to create a database who are on call for any Wikipedia editor who wants to have an expert take a look at an article you're working on I think Andrew Lee's example of the statue is a really good one in that sense right whichever Wikipedia editor was originally working on that clearly liked the statue and clearly was interested in the statue but didn't know anything about the statue and so introduced errors unknowingly and not I'm sure he or she was not trying to misrepresent the statue but having an expert who was available that you could send an article to and say hey can you give me some feedback on this what's good what's not good which resource should I have used that I haven't cited yet things like that would be I think incredibly useful for a lot of our long-term contributors on Wikipedia and so I think something like that is a better use of Wiki Ed's time to try out a pilot program like that to have the potential to have a lot more impact to the quality of Wikipedia then you know ongoing programs where we know it's making a difference in educating some students about Wikipedia but they're not actually contributing content to Wikipedia so that's how we look at it right it's not that and and if I say you know failed pilot that's you know it's kind of a joking phrase but I just want to be super clear that to us these were successful pilots in the sense that we answered the question we were seeking right like if the question is can we get instructors to edit Wikipedia and the answer is no and so for us that's a successful pilot we did a month long project we tried to figure out we asked a question we answered it and so we aren't going to invest more resources into trying to get the instructors to directly at it you know everybody's really excited there's a lot of motivation there's a lot of enthusiasm that that is true across every new person you're trying to talk to but really excitement and enthusiasm doesn't actually translate into edits on the encyclopedia and at the end of the day what we care about is making sure that the content that gets added to Wikipedia is good quality content and that it's worth the effort that we're putting in okay guys I think I know we've held you back from lunch for about 20 minutes now we are all available all 14 of us on the wiki ed staff are here at this conference and so we're all available and happy to answer any additional questions you might have but thank you guys check check one two three four five six give me a thumbs up okay one two check check check one two three four yeah I'm talking to somebody right behind you in the booth yeah one two three four thank you one two three four check check hello check one two thank you is this on? yes okay hello everyone I hope your lunch was delicious mine certainly was yay food yeah food is our favorite thing as wikipedia so I'm here now to introduce our next speaker Dr. John Howard is the director of the national institute for occupational safety and health which is a subsidiary agency of the CDC he's been a public health administrator for several years I forget how long my apologies Dr. Howard and he's been involved in creating our innovative wikipedia and residents program at NIOSH so please welcome Dr. Howard everybody thank you very much and it's really wonderful to be here it's my first wikipedia meeting although I have been in San Francisco and met with the wikipedia foundation and all the wonderful people there so I just want to tell you a little bit about where I'm from as Emily said I'm from the U.S. Department of Health and Human Services some of you may have heard of that department we are the largest department domestically only the Department of Defense is larger we have a number of brands in our department some of them are very very popular I'm sure you've heard of the Food and Drug Administration the Centers for Medicare and Medicaid Services the Indian Health Service the National Institutes of Health the Centers for Disease Control and Prevention my agency is within one of the centers we are one of the centers of CDC and we thought a number of years ago as we looked at both the inputs to our website which we found was predominantly primarily and in first place wikipedia and we also looked at our outputs and we realized that if that was our primary input then we needed to have more output from our agency to wikipedia we do research in a narrow field of occupational safety and health but our audience is everybody who works in the United States and since we are the largest international institute we have partners in Germany and Finland and England and Africa we chair the World Health Organization's global collaborating centers on occupational health which are 70 centers throughout the world we do capacity building throughout the world to protect workers of the world we have about 2000 scientists that work in our agency and they produce science various outlets like wikipedia so we learned a long time ago that we needed to do more in this area and we were delighted and still delighted to invite Emily and James who is ill today as we know into our little family and they are working is he here? oh my god he's here whoa now there's a little quarantine zone I think around him so don't get too close don't shake hands so we were delighted to bring Emily and James into the institute make them honorary members to teach us how to do wikipedia better and we have seen that already in many of the things we've done not only in the number of people in our audience who have enjoyed some of their reconstruction of some of our sites we have a very large audience all of you whether you are employees contractors or whatever if you work then you are our audience every employer in the united states is our audience every practitioner of safety and health every academician and consultant with employers labor unions employer associations all of these are our audience and we think that wikipedia is the channel for us we engage in a lot of social media and that is a different kind of thing but we are very interested in the encyclopedic ability of wikipedia to coalesce all the information on various occupational safety and to reflect that not just the emerging or the quick what happened last week somebody published a new paper but settled things where students can go in and they can take a topic and they can understand it from the beginning when the romans looked at lead all the way to the various new papers on blood lead levels in children and how their recommendations we think that wikipedia is that kind of channel that would be able to be used by anybody in our audience to look at settled science as well as emerging science as well as science that was just published we think that wikidadia is also a wonderful channel we would like to learn more about that and we would like to put all of our data on that kind of channel so I think that what we have now will start but we want to do a couple things one, we want to make all of our information that we have available on wikipedia so that mobile devices will be able to pick it up Andrew talked this morning about that sort of new frontier and the reason I emphasize it and I encourage you to think about it is that many workers in the United States and throughout the world do not have the access to all of you have to desktop computers but they do have often those people that don't have that access have a mobile device and so we're very very interested in persuading them to look at our data but mobile and that is something that is a frontier and we hope that wikipedia pays close attention to that the second issue is that we would like to co-opt, encourage, trap however you want to use the verb all of our other federal agency science partners into wikipedia and that's something that we are trying to do through a meeting that we're having here in this building November 12th through 14th part of James's vast empire of meetings here in DC but we hope that we can entice other federal agencies to join us and to work together as a collaborative group to be able to work with the community of folks like you to be able to bring science that is paid for by the taxpayers all of you that our scientists do and to bring it into your world so as you know as had been referred to before President Obama issued an executive order saying that this type of thing needs to be done the government needs to be transparent about the information that it generates and puts that information out so it's available to the public we think wikipedia is the major channel for that we've been able to find out looking at our inputs and outputs and page links and all of that sort of thing that for every time somebody comes to our site somebody comes to wikipedia's site for the same information about 10 times greater so certainly we think that the amplification power of wikipedia for federal science is really tremendous and when you look at those of you that may be on the biomedical side those of you that may be on the public health side those of you who may edit on the economic side wherever you look throughout the federal government whether you're in the department of defense and you're looking at science outputs from DARPA or whether you're looking at HHS and biomedical outputs from NIH wherever you're looking economic outputs from the department of commerce there's a tremendous amount of information that is entrapped in federal sites but for you I don't think the federal sites would even be able to pick up that traffic so it's a tremendous ability that you all have to reach all of these folks that are the same people that we want to reach so first of all let me say congratulations to each and every one of you for participating in this wonderful community activity it really is a wonderful thing to see and please think about coming to a federal agency if you work in a federal agency please think about coming to our meeting in November and again do not shake James's hand today thank you very much David we'll talk now so welcome to my house it's nice to have you it's nice to have you with us and those of you who have been here before welcome back and those of you who have heard me before much of a fan I am for the work that you do so let me start by thanking you for the work that you do because it's very important to all of us here at the National Archives because we have discovered that it's the best way to get the word out about us we now have somewhere around between four and five thousand articles entries in Wikipedia that have significant archives content and for the past two years they have received over each year over one billion hits that's absolutely amazing for us and demonstrates just how powerful the work that you are all doing helps us to educate the world about the holdings of the National Archives so thanks for that I've been trying to think about why this passion Wikipedia and it started actually in the early 90's when I was a librarian at MIT and working on a great creative project which I've been trying to track down one of the first internet encyclopedias that depended upon world experts around the world who had subject responsibility for creating entries I don't know whether it was the Interpedia or what it was and I don't even remember what the platform was whether it was Mosaic or something before the World Wide Web it was in the early 90's and it was just before Tim Berners-Lee left CERN and came to MIT Bob hello good to see you New York Public Library so that was the beginning and about the same time the wisdom of the crowds so started thinking about actual people getting involved in helping share their expertise was certainly part of my consciousness and then going from MIT to Duke University Wikipedia was already up and I discovered one day just fooling around at Wikipedia that someone had created an entry about me it was just over the top I wish I had saved it because I'm sure the virgins are saved maybe so if you go back to the very first entry on me it's just absolutely wonderful and then I started watching this wait a minute that's not right that's a little inflated as the community started reacting to this over the top entry never did discover who created it but that was my kind of awareness of the power of Wikipedia so moving from Duke to the New York Public Library and I'm talking because Bob was at the Library of the Performing Arts at New York Public this is the example I left to cite about the power of Wikipedia I tried to encourage the folks at the New York Public and I was responsible for the research libraries at first to use Wikipedia to link our content, New York Public content to the entries in Wikipedia and my favorite example is a curator at the Library for the Performing Arts who had just finished processing the collection, theater collection went into Wikipedia to link to the entry and discovered a massive number of errors in the post and she went in and corrected them and within half an hour she had a flaming email from a faculty member at Northwestern I think who had created the entry how dare you how dare you I was very proud of her because she could flame right back because I'm sitting here with the records so fast forward New York Public to the National Archives the thing that drew me to the National Archives was this administration's commitment to open government and the belief that the National Archives could play a role in supporting the open government initiative and on his first day in office the president told his senior staff that the government doesn't have all the answers and we need to find ways to engage the American people in helping solve our problems and what better way of engaging the American people than engaging the Wiki community to help us get the word out which you have done and which I'm so proud of what you have accomplished so welcome, it's nice to have you here I'm so glad you got to stop by oh, you're there okay okay, it's gonna come up here here we go and I guess this is my clicker alright so we're gonna talk a little bit about what's going on at the National Archives and our work with Wikipedia and the Wikipedia community and I referenced it a little bit earlier this morning but wanna talk a little bit more in depth now did that do it? that isn't the right one where's the one that's is this your right one? oh, there we go make access happen, one of our first strategic goal is to make access happen and some of the information that we have from our strategic plan on make access happen make access happen establishes public access as NARA's core purpose adds strategic context to our daily operations and inspires our workforce to develop new and innovative approaches to delivering government information to the public make access happen signals a significant shift in the strategy and purpose NARA will reach beyond its traditional role of making records available for others to discover and will make access happen by providing flexible tools and accessible resources that promote public participation NARA will be recognized will be a recognized expert and leader in developing information to customers in ways that are most useful and meaningful to them and I think that part out of our strategic plan really gives context to why we're here it's in ways that are most useful and meaningful to the public and Wikipedia and the whole Wikipedia community is part of that so this is a neat graphic I didn't do it but I really appreciate that it was done and it's really cool this shows how we are making access happen and if you look today at the views of our records our public records the vast majority of people looking at our stuff in the digital era is on Wikipedia we have partnerships that we work with too of course Ancestry and others Ancestry being the major one you can see our web including our online public access catalog Facebook and Flickr we're on different sites but look at us with Wikipedia I mean this is really the major way that we provide access today okay so so our work with Wikipedia is changing the way we think according to Archivist David Ferriero and I think it's already done that it has changed the way we think we want to work collaboratively we want to work on third party platforms we want to get out there where the people are and allow them to work and discuss and learn from our records and so we have three of my colleagues who will be talking a little bit more about making access happen Andrew Wilson good afternoon all right so Pam spoke this morning and spoke a little bit now about our overall goals about making access happen I'm going to go into a little bit more specifics about a tool that we have the National Archives Catalog that's one of the principle ways by which we make access happen directly from us however before I get into the specifics about the catalog I'm going to give a little bit of context about the records that we have in our agency I'm still relatively new here I've been here about seven months but one of the things that I've always been impressed by since I've arrived here is just the scale of what we're dealing with and you can see a couple comments there about that the holdings of the archives are so vast they're actually measured in millions of cubic feet which is pretty impressive and it's over 12 billion pages of textual records I forget off the top of my head I think it's maybe a couple million cubic feet for textual pages just textual along electronics artifacts and everything else that we have so so one of the challenges at the same time our resources are as with a lot of government agencies somewhat limited Dr. Howard actually was here talking a little bit earlier about the fact that the size of the health and human services which is roughly 70 or 80,000 people now at the same time it's only about 3,500 people at the same time our budget is probably a fraction of what their budget is so the result is when we're working with the scale of this information it's difficult to fully describe to the level that we really like with a degree of metadata and contextual information to make that information findable and at the same time the reality is that a lot of times the records that are up in the catalog there are only scanned versions there's not much additional metadata that can be taken from that so this being the challenge the archivist talked about this and about the vision of President Obama about engaging with the public around our information and open government and it was clear that the experience and the expertise of the archives was really going to be insufficient to climb this big mountain that we have so that being said historically until recently it's been a little bit of a struggle for the government to collaborate and particularly if they collaborate with organizations it's been new for them to collaborate with individuals now with these new platforms both social media and talk about platforms like Wikipedia the ability to collaborate with individuals is a much different level now it really changes the nature of how we interact with the public a perfect example of this is this is a photo here that was put up around 2009 in Flickr in the archives Flickr account where we posted this image and individuals that recognized this from an article that they had read came back and added some really incredibly great stories about this particular piece and I think that's one of the about the time when we started working with Wikipedia and Spark came for thinking about our catalog as a social catalog so a social catalog there's lots of different ways that this can happen you think about tagging inscribing those are some of the things you can do in the catalog right now we're also looking at things like transcription translation I'm sorry we do tagging and transcription we're looking at commenting and translation so there's many different ways that people can interact online and really what I want to do really quickly is a a live demo of the catalog and so what I did is this is a the president's engagement calendar President Hoover this is live on our catalog right now and I think for me it's a really great example of a couple things we've talked about think about the idea of open government and the idea that you can actually go now today and go back 86 years ago and look on an hour by hour basis doing that specific day so we're looking at the calendar here I went back and I spent a few minutes looking through this real quick found an incredibly powerful story as part of this it's about Madame Curie and the fact that there was a time I might have to reload there was a time where Madame Curie actually came to the United States and was looking for radium for part of her experiments and when she came here actually the President of the United States on this given day came and gave us the radium tour so what you can do is click on the button there that says view add contributions and you can actually add tags and transcriptions directly into the catalog unfortunately we are loading right now so I'm going to switch back so apologies I couldn't do a live demo I actually go to find all this information is on our citizen activist dashboard that's a place where we talk about all the different projects that we have on going both those ones on our site again the tagging and transcription we have tagging challenges on there transcription challenges and at the same time we also link out to other organizations where we do collaboration that's Amara was an organization that helps do translations of video and also with a site called old weather where we do translations of deck logs so talk about some of the remaining hurdles we have Andrew Lee talked this morning a little bit about the challenges with usability it was actually the first thing he brought up we always need to improve their usability it's something incredibly important we need to focus on our end users and what they need at the same time it's really important that we build community that's part of the reason why we're here today we can learn from this community we be a part of this community and by building community around our records we can bring in that contextual information that sometimes we have a difficulty bringing to it and the last thing that we really want to do is really make sure that our content can be found and accessed everywhere possible so being part of Wikipedia and Wikimedia community being part of other communities is really the core of what we want to do my name is Darren Cole I work in the web and social media branch in the Office of Innovation as Andrew and Pam I'm here to talk a little bit about our new innovation hub which I'm sure you've heard mentioned a few times today already generally I work on a project called Today's Document which you might have heard of as well and I'm not ashamed to admit that I use Wikipedia to kind of filter my searches and kind of guide some of the content I choose for that so I'm grateful for the Wikipedia being there for me on the times when we need it we know the records we have but sometimes you help figure out what they mean but I'm not here to talk about Today's Document I'm here to talk about the hub we opened it just this summer so it's kind of brand new so if you have the chance swing around the other side of the building and check it out and really I'm actually speaking on behalf of Deena Herbert who is the hub coordinator she actually just had a baby so she can't be with us but she spent the last several months before birth getting this up and running so this is all really her effort that you're seeing so what's an innovation hub really a term at this point we borrow a few elements from sort of existing concepts that you may have heard of idea labs that you see in the government and in industry maker spaces that you'll see at libraries and also obviously our research room the idea lab they sort of have elements of sharing and collaboration between different units maker spaces are generally more open to the public whereas idea labs generally are not so much and the maker space is a hands-on and a hands-on element and hopefully we brought some of that into the innovation hub usually you don't think hands-on is something you really identify with the archive so much and sometimes it's not but and sometimes in some cases it is so you're getting to in this case we're bringing some of our records into the innovation hub so so what were our goals behind the hub really it draws from the strategic plan which PAM covered in more detail than I will but really it draws down to that single element of digitize everything that's sort of the ultimate goal we have I was going to say 9 billion but PAM actually had a better figure 12 billion records I think so that's going to take a while to digitize but the hub is one way we can make a little dent in that it's an attempt to kind of be more open and engaging with the public then people might usually perceive the archives as being so it's an open space where you can kind of anyone can come in we can connect with our customers and hopefully they can be more vested in that effort the archives you know it really is belongs to the citizens of the United States it's not just a closed off institution other goals you know we're trying to make it low cost the build out right now was very basic just kind of a repurposing of an existing space so we haven't really invested a whole lot of money again because we're trying to be flexible if we need to change it move it around experiment try something different that's what it's there for it's not etched in marble and here's Dana taking the wrappings off on our opening day so she gets to make a mild appearance here even if she can't be here with us but really the hub is sort of a physical manifestation of our innovation efforts and of efforts like the you know our partnership with Wikipedia sort of the main activity going on right now is what we call citizen scanning previously what would happen if the archives research would come in they would request some records they'd go to a research room they'd get to go through those maybe they'd scan a couple pages they'd go home tomorrow the next day someone else might come in request the same thing scan that same page all over again so in this case we're able to capitalize on that effort and get those records into the catalog and sort of you know save for posterity so in this case the researchers will request the records to be sent to the hub they come down to the hub at no cost they can use our equipment we have 12 banks of scanning stations we have 10 different flatbed scanners which will overhead scanners a mix of PCs and Macs that they can use again at no cost to them so they scan their records they get their digital files to take home when we get those same files we compile them we index them with the metadata and then they are ingested to the catalog and hopefully in a few weeks later they're available on the online catalog for anyone to see and maybe you're not a researcher maybe you're just kind of curious maybe you don't have time to scan a full file unit we have a box of the month program that Dina set up you can come in there's a pre-selected box people can kind of take turns scanning a few pages there are as many pages as they want as many pages they have time for it's a good way to kind of get an introduction to the process just kind of get your feet wet see what it's like without really making the full commitment that a standard researcher would have to make so these are some of the early results we have from the citizen scanning efforts early in July we hosted what they call primarily teaching which is an effort from our education branch it's a teacher workshop then they do research in our own original records they create lesson plans using those records and they take them back to the classroom and use them with their students so we had a group of 15 teachers in they were scanning records from a set of Chinese immigration files and in the end we were able to accession 100 new records consisting of over about 400 pages that were ingested into the catalog in the end so that worked out really well as our pilot group and so on July 13th we opened to right now we have just a pre-selected set of records they can't scan anything we're just pulling from military pension files and compiled military service records which are still very popular genealogists so sort of a high traffic set of records so it's sort of a good secondary pilot use and in the period between July and August we've had an average of two researchers a day sometimes we've had a daily high of eight researchers on any one day and we've had a total of over 140 new file units ingested into the catalog just in that month and if you think maybe 140 is not so big a single file unit can run anywhere from 15 pages to 160 or 180 pages in just one file unit this can be someone's pension file from the Civil War so it has a series of documents every time they were requesting new benefits from the pension office so this is a massive undertaking for a single researcher and a big commitment so the fact that they actually stuck through it and scanned all these pages I think it's significant but really very promising that people are kind of sticking with it like this because a lot of times researchers will come in and they'll do one or two pages and leave but people coming into the hub have kind of really stepped up scanning is not the only thing we're doing we also have the other half of the hub which is our meetup area this is sort of a flexible open space where we're going to host editathons wiki meetups other events with the glam and wiki communities hopefully in conjunction with the scanning room have a scan-a-thon we'll be hosting Andrew Lee's wiki space pop-up exhibit after it leaves the conference today so later next week I guess if you come by it should be in place in the hub as well other NARA partnerships the space itself is open and flexible almost everything is on wheels we have wheeled whiteboard partitions tables are on wheels chairs are on wheels obviously but we can move it around and reconfigure as we need it we have free wifi we have a lounge space unlike other areas in the archives food and drink are permitted and even encouraged so if you want to come in and do some tagging and transcription and have a coffee please do and Andrew covered the transcription and tagging efforts but we hope to have a couple citizen archivist kiosks set up so obviously you can do tagging and transcription from the comfort of your home but maybe you come through the hub and you want something else to do there'll be kiosks there for people to use so they can kind of scan it so they'll be there for their use more on the innovation hub you can check out our website you can email us at innovation hub you can follow the archives in of hub hashtag on twitter definitely come by check us out if you can again we're on the other side of the building everyone should have gotten a flyer that looks a little like this it has a map on the back that shows you how to reach it the scanning area is only open you know regular business hours so it won't be open on Saturday or Sunday but the meetup side will be and if you do happen to come by during regular business hours we have at least four very friendly research monitors who are eager to help Rose, Madeline, Kate and Carolyn who may be watching this on the live stream so I'm giving them a little shout out because they're over there right now so we're excited to hear any new ideas you have for how we might use the space and make better use of our partnership and also your thoughts and questions next up is Dominic I think you all know this one hello so I'm going to talk about something a little more techy some of you may know an API is already and you may not so I'm going to hopefully keep this high enough level that you can kind of also understand why it might be important so it's not very easy to illustrate an API because it's data so I'm sorry about that but it put it on a black background with white text but basically what I wanted to just convey also is our catalogue is you can go to a web page and see it if you're a human of data so we have so far before we launched the API not ever had any portal into that data that's easily accessible other than just going and copying and pasting or scraping from the web page but really this is all part of our overall access that Pam mentioned because the goal of the API is to enable all users of the public generally to spread our digital assets and our data the public contributions to our records in all platforms and corners of the web so that's not something that we can accomplish just by having a catalogue on our web page so that's the idea here I wanted to kind of also before I begin because I know a lot of Wikipedians want to just like look at it and see what we're talking about if you want to just go there because there's a lot of computers open these are the places you can go if you just wanted to explore as I'm talking this is the top URL is just the main path to the API itself if you go there without putting in any search parameters or anything you'll still see stuff and there's no key needed API key needed or anything we also have a page we call interactive documentation which is sort of like Wikimedia's API which is like a sandbox page you could go there and if you're not really familiar with APIs there's little drop down boxes in an actual web page that will show you different parameters you can use or how you can construct an API query and then also we have a GitHub repository which is where our actual documentation is so you can check those out for if you just wanted to type all this you could just Google the National Archives official GitHub page and the other two pages are linked in our documentation so having said that what's an API so I was kind of alluding to this a web page is a UI, a user interface API stands for Application Programming Interface what that means is that it's the catalog but for computer scripts, code, apps, web pages so that you can build a tool that can query or ask for data from our catalog and serve it to users on third party platforms or anywhere else and also ours is I think unusual in especially in the cultural sector in that it's a read write API and you can also all of the things that Andrew talked about things that you can do to write data into the catalog when you're adding a transcription or a tag you can do that from the API as well so basically it just allows a way to serve machine readable structured metadata which is what this is if you're looking at this you can kind of start to see what these mean there's there's like names of fields and then the values of fields and you can see our identifier or type of record or things in there but that's just a random snapshot of what it looks like and this is the same as if you were looking at an ad on the web page so to begin with since I'm trying to explain it as really just a way into the catalog itself for computers the beginning of describing what an API is is really just talking about what the data set is that it allows you to access so in this case the data set is really everything that's in our catalog and that's the intention so it means all of the archival metadata for every description or catalog record the digitized records themselves images and their file metadata authorities which is a way of saying when it comes to library and archival metadata the term authorities means people and places and those sorts of canonical things you probably might have come across authority control on Wikipedia NARA's web page is because it is actually the web search for all of our web properties and then of course also the public contributions which is tags and transcriptions so to start off with sort of the principles for this project as we were developing it was that it is a public arrest web API that's read write and there's part of being writable means there's authentication you can log into the same account that you create as a on the web side you can log into that through the API or you can also register through the API and we took inspiration from the DPLA the digital public library of America's API and their philosophy which is that there's a presumption of openness they actually had put out around the time we were first starting to develop this they had already created their API and they had even put out a policy document saying all of their metadata is in creative common zero basically essentially metadata is free in the public domain so following that there's not even an API key to do any type of searches for read actions and all of the metadata is in the public domain that's because by definition any metadata created by the National Archives is a worker of the federal government which makes the public domain so we don't place any restrictions on reuse and I'm just going to kind of this is maybe starting to get scary if you're not techy but I'm just going to kind of walk through quickly some of these things but I also did put like full URL so you could see what it sort of looks like if you understand that but I just wanted to talk about what you can do through the API and essentially it's think of it sort of like advanced search but even more so because you have the option of doing keyword search combined with optionally search on any possible field and just to give you an idea with all the different permutations there's a couple thousand possible fields in our archival metadata so whatever you see on like an advanced search web page is really just a curated list of of fields that the system is letting you search but through the API you could search whatever you like and then it allows you to filter, sort search on ranges and then also specify which fields you want returned rather than the whole document so this is just an example of a type of query you could do where you can see the keyword is Eisenhower the word army is in the title I'm sorting by the identifier so all these different things you could add to get a very specific query back so why, right? What does that mean? I just wanted to throw up an example of what you could do with something like that this is kind of a cool idea of a test that I threw together where if you could do something like that, like query our data in that way then it allows you to do something similar to what Darren does with today's document we curate articles related to the date but I put together a little experiment where just by searching those, the day and month fields through history creating a Twitter bot that every whatever 10 minutes or whatever you like tweets an image with some metadata based on that search so sort of mechanically curate it so that's something that this is just an API read which action, right? But it's something where you can put it to use to actually do something like that and then also just to mention the right functions tagging and transcribing and it's really just following the same sort of theory here where you just but it's in reverse where you specify the thing that you want to tag or transcribe for a transcription you're transcribing which means like the image within the catalog record and then give the text and you just would have had to log in before you could do that so that's that's kind of the general concept here and then also just to to quickly mention there are even more things that you can do through our API I mentioned registration login you can also do bulk exports if you wanted to download a large query bulk ingest if you wanted to like transcribe a thousand things at once because of course we're talking about doing things programmatically so you might have some reason, some tool that has generated those transcriptions and then you can even get to the user data accounts so having said all that when it comes to wikipedia you know wikimedia generally the reason that this is such an important tool for us is we have these big goals one of which is which is actually was in our recent open government plan is that we'd like to be able to upload all of our holdings to wikimedia commons which you know is as you've heard that's a lot of data so it's not something that we have been able to do before we've mentioned already today that we uploaded like a hundred thousand things two years ago which was a very difficult process and that was before the API and we were scraping our web pages and it was not scalable so the idea of the API is that it allows third parties to be able to take our data and reuse it and I'm especially excited about the idea that image metadata on wikimedia commons will be structured at some point in the future so then we can make it a two way street you know if there's a category on wikimedia commons or if there's something we can also suck things back be able to understand the structure of wikimedia commons obviously wiki data is a place that we want to put all of our metadata there's all these, whether it's authority control or having a citation tool on wikipedia, all these reasons that we would want users to be able to create tools but really it's I think the most important thing is that it's about allowing the public to be creative and fulfill its own using our data and that's what wikipedia does best so I'm just going to quickly wrap up for all of us and say just to answer sort of the questions that we started out with why is wikipedia a fit at the National Archives one thing that's interesting to note is for each of the three projects we talked about in a lot of ways wikipedia was a driving force when it comes to transcription in the catalog one of the first projects that we did working with the public in like a systematic way to encourage transcriptions and actually get them in our catalog was when I was still an intern year we had wikisource we produced maybe a couple hundred transcriptions of our records on wikisource and we actually didn't have the ability to add transcribed text to our catalog but we did add links back to wikisource in our catalog where we're good similarly with the innovation hub the first attempts we ever had to do citizen scanning efforts was the scanathons we did with wikipedia and of course the api wikipedia was the use case for the reason we created the api is to enable us to upload all of our holdings to wikipedia or to wikipedia commons so that's why I think it's really important just to drive this home that wikipedia and wikipedia in general are not just platforms for us but they're also it's a partner I think for a lot of cultural institutions it can serve as like a motivator for why to get involved in all sorts of innovative projects and it's important because the glam wiki the kind of the concept or the idea of cultural institutions getting involved with wikipedia doesn't just exist in a vacuum but for the field in general to become more receptive to wikipedia I think that it also depends on all these other sorts of open and innovative projects alongside it because that's kind of what causes change in organizations and opens minds I think we might have time for questions that's all I had if I may be indulged in a brief question here how does the National Archives think about wikisource you are clearly thinking a lot about commons and about wikipedia but what about wikisource does that fit into your picture anywhere I would say it's probably been underutilized but also there's a lot of there's a lot more work involved in engaging with wikisource because of the smaller community so we made early forays into wikisource a couple of years ago just like how we did uploads for wikimedia commons a couple of years ago but then have also we learned lessons and took that to heart and made changes internally but I can see a day in the future where because now we have transcription in our catalog because hopefully wikisource will be getting better tools and becoming more usable and I know there's a wikisource user group in the last couple of years have been developed so we can actually you know be able to export our data to wikisource and suck up things from wikisource it's definitely not a competition I'm not now but okay thank you so at what point and I'm asking for very selfish reasons because it's a process that I have to go through my work but the process from going from physical documentation to the wikifying of the information at the very end do you take time for 508 compliance sure what I can talk about sure let me explain a little bit of 508 compliance 508 compliance is the federal regulations that rule information for individuals with disabilities so for example if an individual is sight impaired they can actually see the information or access the information on the website so one of the things that we definitely absolutely do is make sure that the platforms that we use so for example the catalog or the archive.gov website that the platform itself is compliant so people can use the platform thank you so it was really interesting to me that wikipedia was in some ways like the use case that you built the API for but I'm curious what other uses has the API been put to outside of now one thing I will say is all three of the things we talked about are all very new and have been kind of inactive development so I don't know that we have like big examples of things right now but certainly we have for example the digital public library of America is an aggregator of the metadata from libraries and archives around the country that was another important use case to allow places like that to be able to take our metadata and also just to be able to make our own website outside of the catalog a lot more useful and dynamic also things like data.gov places like that so I think we're looking at what could come up that we could work with going forward so I think there's platforms we haven't even thought of yet that we will be working with as we go forward and I'll add real quick I mean Dominic just touched on this but the idea of us being able to eat our own dog food for somebody to say it we have websites that we own can we pull back from the API and use that information to help better own websites before I ask my question I'll return David Saver and give a shout out to him so I've been following Dominic and National Archives activities for seems like a good couple of months I'm in New York we do have a branch of the National Archives and I'm particularly interested because I know our branch also is responsible for the archives in Puerto Rico a really great opportunity to get what's perceived as a non underrepresented community in Wikipedia how could we who's going to step on the whoever is head of the New York office to innovate some of these stuff are you going to visit New York because one of the few chapters of Wikipedia in the US would be a great collaboration go for it I'll go that's good okay well my question is about the tagging and the opportunity to add stories to these records and it's a phenomenal opportunity to add the president's engagement calendar you can add a story about Marie Curie and it really enriches the cultural context of these records but then I can also go and add a story about how I visited the president and he punched my dog in the face and so what systems do you anticipate to have in place to prevent these sorts of things from happening because as you know Wikipedia has a sort of problem with that so what do you anticipate you're going to have to do in that realm so we have on the back end there is a moderation dashboard that we can use to moderate comments and tags and transcriptions when they come in certainly the most important thing though is that there is a content contributor policy and set before we do anything to any contribution that somebody wants to put in there we're going to make sure that it adheres to that policy so I think it's a combination of having the policy in place and then having the tools on the back end to make some changes if necessary and then I forgot to add my question do you anticipate that taking up a lot of people's time dealing with these sorts of issues no not really I mean today we haven't had any issues at all and frankly in all my experience in federal government that's always a question and always a fear that people have but the reality is that it really very infrequently happens thank you Hi my name is Dan Eldridge username commander CMDR Dan quick question are there identification requirements to gain entry into the scanning facility and like and two can you talk about the cost when it took to develop and implement the API well for the hub the standard it's a standard research room so you need to apply and get a researcher card so you can't walk in off the street you can walk in off the street but you still need to get a researcher card if you want to scan if you just want to go into the room you can go in oh yeah yeah yeah cause it's on the researcher side what forms of ID do you need you need a picture ID and then you get your researcher card and once you've got that then you can come in and scan at any time and getting your card is only a 15 minute procedure you just have to go through a short power point basically it's still a secure federal building so I'm guessing it took more than 15 minutes so to develop the API so can you talk yeah I mean it's you know we've been working on it for maybe two years now but it's also been at least public for about a year now I think so the API was developed after a lot of design with discussions with contractors so there's been a whole process to it and we actually are just don't really even have it's not like we have a huge number of software developers on board but it's just something that we prioritized in house and we continue to iterate on it I mean we want to continue to refine it as we go forward yeah I think the iteration is really key there and one thing I'll say is that just as it's really important to work with end users on the product from the public facing the front end side same thing with the API when we work with as Dominic said we have a lot of a lot of a lot of people in the area of America getting their feedback about their API there's always going to be room for improvement and iteration and have other agencies come to you for best practices or for help in developing their own APIs or to implement some more tools or systems in their agencies whatever not that I know of yet there's some active discussions on APIs in the government but right now I think as a whole the federal government is still just kind of trying to figure things out I just wanted to sound a cautionary note yourselves and also the speaker prior about occupational health and hesitancy Wikidata and Wikipedia as an avenue really to get your information out to the public because it's very widely accessed and endorsing it the way you do in a way you're strengthening its market position out there as a go-to source and one thing that worries me slightly is that we as human culture has fought very hard to be able to have a plurality of opinions and a plurality of sources in daily public discourse and I think we need to think about building robust systems we've had problems in Wikipedia for example with the one case that was in newsweek earlier this year was about an Indian Diplomomil where an abusive admin had been able to manipulate the content of the Wikipedia content on this institute for a number of years using first a sock puppet army then his position as an admin and that led to real harm to Indian families who sort of sacrificed their family savings and went into debt to be able to send one of their kids to that school which in the end gave them diplomas that were worthless and do you have any thoughts about that I mean I understand basically you're looking to get your information out and Wikipedia is there now you can do it so it's a short-term win but I would encourage you to also think about the long-term situation as a democratic society not all of the world is democratic we are aspiring to get there many countries have media systems where the press is not free and all the media that are available say the same thing now if you're building a world where Wikipedia becomes the preferred outlet and it's plugged into Google, it's plugged into the Knowledge Graph panel so that whatever is in Wikipedia and Wikipedia is what people primarily see I have a concern that that system is not very robust everything may be fine now what's it like in two or three two or three generations down the line we should be very mindful that we don't build vulnerabilities into our information dispersal system which while useful in the short-term for the next ten years or so in the long-term might come back to biters one response I would just give that is I also as the Wikipedia in residence at the National Archives I think that's very similar to a lot of times what we hear when you talk to cultural professionals sometimes the concern when it comes to like are we preferencing or vouching for Wikipedia but the one thing I would say is that we don't do anything that's like a Wikipedia exclusive you know we our goal in Wikipedia helps us sometimes in sort of pushing us in this direction is to make everything that we do open and accessible as widely as possible and that's why the slide that Pam showed where Wikipedia is already such an important platform for us that's why that's an important outlet for us to go to and why we would want to invest in it but I wouldn't necessarily say that we want to invest in it intellectually we still allow third party platforms to use our data however they like even if they're in ways that we find distasteful one of our strategic goals beyond make access happen and connecting with customers is maximizing our value to the nation and I really see the ability with the API for other folks out there or anybody out there to more easily get our data and reuse it to however they need it so we see the API as part of that adding value back to the nation and we definitely want to keep it open to anybody so that there is a diversity of sources thank you very much thank you everybody for staying through your break but we're going to let you go now thanks hey everyone we're getting together for the presentation please have a seat please have a seat oh sorry I'd like to present Sydney user flow night wikipedia residents at the Cochrane collab initiative and someone who's helping all the medical articles on wikipedia okay hello I feel like I want to do jumping jacks or something to get everybody moving because it's kind of late in the day but I'm covering quite a lot of material stuff that I would normally cover in about an hour an hour and a half I'm doing in 30 minutes so I'm going to just brush the surface and go quite quickly but I'm going to be here all weekend long so please feel free to catch up with me later and to follow up on anything that catches your fancy that you would like to talk about I'm just a little bit in further introduction of myself I begin my name is Sydney poor with a user named flow night I was a student in 2015 I chose the name flow night because it's short for Florence Nightingale and I feel really good about that because she was very scientifically minded for her time and that's kind of important for the work I'm doing now because I'm collaborating with Cochrane which believes in evidence based medicine based on scientific research so it's an excellent fit and it really brought me full circle back around 10 years later to what I'm doing this presentation is really a call to action I'm not really here just to tell you what we've been doing but really to invite you to get involved in helping edit medical articles and the question is like why are we doing this who are these organizations getting together and it's the reason is because nearly everyone reads Wikipedia and you can see from this slide we're talking about health professionals we know from the surveying we've done that 52% of physicians use Wikipedia 35 to 70 pharmacists and on down the line because of that is really quite important that we have good information available but the issue is only a small number of people regularly edit our Wikipedia articles we did some surveying on this and we know that there is actually a lot of accounts of people who make some edits that the reality is and this was of 2013 only 270 editors made more than 250 edits to medical content in Wikipedia and this included all of the language Wikipedia that we surveyed all of the large languages and half of those were done in English so obviously that's a really small number of people that are working on trying to improve the quality of the content and making sure it stays in good shape so I mean we feel like that this was a pretty big weight to carry so we really needed to look around for some people that could help us and so we looked out and we found Cochrane Collaboration which was an organization that we were using their references quite often in our articles and we came together to work on improving the content of the references to Wikipedia as well as to increase the number of people who are editing Wikipedia articles, the medical articles so who is Cochrane and why is there somebody that we would want to work with they're a global network of 27,000 people and they come from 130 countries around the world they have contributors that are researchers, health professionals patients and consumers so you see in many ways there's some similarities to what Wikipedia is we are very global ourselves and we are made up of not just one type of a person we're not just academics we're not just consumers we're a mix of people that come together to do the work and we have in our global network a reach to around 70,000 people so the thing is when they come to work with us we don't really do a partnership with Wikipedia per se because Wikipedia foundation doesn't do that instead what we are doing is we're doing a loose affiliation with Wikiproject Med Foundation as well as working with all of the other Wikimed projects and I've identified the ones here the Wikimed projects in different languages that are fairly active there are many more of them that exist that at one point in time on their own language have tagged medical content but these are the main ones that we go to there's a pretty good chance we're going to find people editing the biggest new one is the Wikidata one and there is a Wikiproject medicine on Wikidata and it is becoming more and more active and it's a very exciting area for us to work with I would like to give a call out to the German language as well recently I did some work with them and I have a very good understanding what they're doing they're a very autonomous group and they're producing some high quality content so although we're focused more in the United States we do have people out there working on articles that we could collaborate with ourselves so in terms like what is really happening with this partnership one of the things that they have committed to is to having someone come inside their organization to be a liaison for a short period of time to help them figure out how to work with Wikipedia this is not unlike the archives who have done something similar with Dominic because it's pretty similar to that or any other Wikipedia who goes into an organization who is trying to figure out how you fit in that organization how to kind of make it be better than what they were already doing because there are certainly already people at Cochran that was putting some edits on Wikipedia but they weren't necessarily doing a very good job of it it wasn't happening in a very organized manner at all so what they've done is they really are beginning to integrate us into their whole ecosystem so for the past few years they've invited us to come to their international colloquium I just actually was in Vienna this past week and I just flew into DC yesterday they pay for us to have a full day Wikipedia symposium we invited the German editors and other people in Cochran the German Wikipedia editors who were working with Cochran in the area as well as those who were flying into the area to come to that and we had a chance for people to either get introduced to Wikipedia or some of the more experienced users to sit and talk about some of the more complex things for example having to getting Wikipedia and wikidata working together so they provided that platform for us to do that the other thing that Cochran is doing is that they are encouraging all of their little research groups in health topic areas like they may be researching in wound care they may be researching in neonatal health they're encouraging all those research areas to develop dissemination plans for their research for their Cochran reviews to then be put on Wikipedia they are thinking in terms of how to disseminate in press releases typically in the past or in social media but now they're beginning to recognize that they really need a plan built in with that about how to get that on to Wikipedia the thing is when you're doing research you really from the very beginning think with your research question you know who your audience is so now they're beginning to think our audience is Wikipedia so we are very well changing the way that they are doing the research because they know that we are one of the places where the research is going to go in the end the other thing that we are collaborating with them is just really advocating for open science at this conference I was at last week I met a very large contingent of people who are interested in open trials and they're setting up a project to deal with this and they're exploring what software they should use so we were there to help them guide them through either considering Wikidata or looking at the strengths of weaknesses of what Wikidata is perhaps something like that would work for them but they ask us to step up and to kind of work with them because we do have some experience in this area the other thing that we work on regularly it has to do with the translation projects and the Wikipedia library one of the first groups of articles of accounts that got given out with the Wikipedia library was a free Cochran account and the way that works is if you're not familiar with it we approach different publishers and ask them to give access to accounts that are behind paywalls we know from when that was started around 2012 that we've given out all of our accounts and more are available now and we have seen a 15% increase in the number of citations to Cochran since then and this as you can see the graph it was pretty there wasn't an increase all along but it was a much sharper increase once we began giving the accounts out even more importantly though than just us being able to see the new references going in after we've given the accounts go out we know that those people that have those accounts are working on updating the citations that are in Wikipedia the way that Cochran reviews work is that when they do a research review on medical research they updated a few years later automatically their goal is to do it like every like two to four years we often times will have cited an article and they will have done a new update and we won't know it so what we have done recently is started running a bot on Wikipedia English and it automatically generates a list of the articles that need to be updated and also in the article itself it has a little brackets next to it the reference that says that this is out of date so you can click on to it and it will take you to the new reference and this is something that one of the kind of experimental things we're doing and so far it's highlighting the data that comes in a form that looks like this and we're generating like pages of stuff that needs to be updated so when I was at the conference last past week one of the things I did was introduce this to them and show them like literally people whose reviews have been done and how they're not been updated yet and encourage them to actually do something about it like look marking that page so they can find it again and then also to mark their calendars at least monthly so they can go and look to see the new updates as well as to host Cochrane update editing events in their groups so that's one of the the newest things that we have introduced into a collaboration with Cochrane there's three other projects that I'm working on right now very targeted ones one of them is going to be Cochrane WikiProject Women's Health we're going to do something also on disasters and humanitarian emergencies and WikiProject Women's Sciences the general approach when I work with anyone in this organization is to identify topic experts select priority articles to improve and then mentor people in the Cochrane network as to how to interface with Wikipedia and then to hold editathons or regular editing events for Cochrane and Wikipedia the Wikipedia Women's Health is going to be a specific target area in the upcoming group we have identified some of the research topic areas in Wikipedia like pregnancy in childbirth is for example and they're excited about the opportunity to come and work with us so we're going to do the whole idea of it is to have a person like sit with a topic expert become get more information about like what needs to be done to improve the article that have our editors perhaps even some student interns to work on improving the article and then when it was finished there would be some type of peer review that could fit into our good article a future article process where they would essentially say yes this information is accurate and that takes that burden off of our editors looking at that medical content and really wondering you know not if it just isn't our format it looks like it's pretty complete to us but it's somebody essentially saying this is what it needs to have in there and I'm very excited about the other one that we're interested in working with it has to do with disasters and humanitarian emergencies kind of the same type of thing you know the process we would use with Women's Health it kind of takes on a central role because when there is a disaster people are often time scampering to do things that will be helpful but we want what they do to be based on evidence and to be based on good scientific information so by us working in this topic area on Wikipedia we can make this available under a open access license so that people would have access to it and they wouldn't be spending their time trying to think about how to write it down instead they could just take it and reuse it so it's this is a pretty compelling argument to give to organizations who up to this point in time have been having their work copyrighted they could truly see the value of why they would think it would matter that it be have an open license there is an organization called Evidence Aid who is kind of a branched out of the Cochrane and they they have agreed as well as some other organizations to lend their topic experts to work with us kind of a case study that happened last week while I was at the conference was I met a topic expert who had identified something they felt like was missing from Wikipedia and it had to do with migrant health and their first inclination was to want it to add it to the migrant health article which would have not been inappropriate to put it there but it probably would have just been buried somewhere in Wikipedia and nobody would really seen it. So we were brainstorming about what we could do to make sure that the people that were in the situation would get access to it and so we decided the place that was getting the most hits was the European migrant crisis article and we saw by looking at our internal data that it was actually getting a lot of page views and if you looked on the graph you could see that it was out of almost five million articles now it was 1180th on the list of a number of articles that are being looked at. So it was actually a really good number of people that were looking at it. So what was interesting enough about looking at this as well is that last column over the 0.08 has to do with Wikipedia zero. So you can see all the other ones there or like zero or not very much coming through all the articles but that particular article there really was a bump up that people were looking at it through Wikipedia zero which gave me, felt like we were then putting the information where it needed to go in that article people were interested in that area of the world so I felt like that was something that was had real potential for us to get the information that was useful into Wikipedia and so what we did is we took the person wrote it down for us and we then added it to Wikipedia. They agreed to give it away under an open license and we then copied it in so we ended up adding a section on health guidance. So I mean we showed everyone that was working in this topic area what we've done and it made it seem so simple to them how quickly you could get something quite new added and that would be potentially have a lot of impact. In particular what this dealt with was there is really no evidence about doing psychological debriefings for people right after a disaster or emergency and this is something that people have often done in the past and they have found that it's really just a waste of money and it could even be harmful in fact. So we felt like it was something that was quite important for us to share and now they were quite pleased to see that some of their work was in Wikipedia and was making a difference. So I mean I feel like by us working with agencies finding case studies and examples that are meaningful that will show the impact this is part is what will draw more and more people and people felt very compelled by when they hear these type of examples. The last project that I've selected is really near and dear to me personally has to do with Wiki project women scientists it's something that we have recognized as a problem in the past that we were lacking on Wikipedia and we have again working on a project called Wikipedia women in red where the women who should have an article but don't have a red link on Wikipedia and so we know that Cochrane in the past has expressed an interest in having women that they knew were scientists that were missing from Wikipedia being had their articles added so when I explained this project to them they were so excited about the idea that they could come to our Wiki project women's scientists add those names and see those red links and then see them get filled up and they are more than willing to help us develop those lists and also try to find the information we need to make them be you know properly referenced so I mean I feel like that you know those are the three target areas that I'm doing right now this isn't a project that I did last year that I also would like to revive again it was with an organization called students for best evidence it's a global organization that supported out of the UK Cochrane Center we had a Wikipedia week and they do a lot of blogging so they you know they blogged about what Wikipedia is, why would it be meaningful for them and then they had an edit-a-thon inside the Cochrane Center in Oxford they invited Wikipedians to come in as well as the students to come in and edit so we didn't have a huge amount of turnout it was more or less a pilot but the UK Center is a very well organized, well funded and a very enthusiastic location highly supportive of Wikipedia and I believe that you know we'll take another run at this now that they kind of know what it's all about and I believe we'll be much more successful next time around because really in my opinion what this is going to evolve is developing the partnerships between Wikipedia editors and the Cochrane Health Topic experts and some of this can be done remotely through Skype calls and Hangouts and other times you know we'll invite them to come to the table with us and share a meal and exchange health information at the end of the day that we have a lot in common with them and that we will be able to continue to improve the quality of content on Wikipedia and the medical area to be what it should be today and I'm really I'm hoping within the next six months or so to wrap up what I'm doing there that they will have us so baked into their regular processes that I'll be able to leave there and go somewhere else and point to success and do it again somewhere else so I'll be happy to answer any questions or take any comments I thought you said that was English only the obvious point is if you update the English one and then it's translated into Spanish or French should that be on the list that's a really good point I mean we kind of just kind of we only begin using our update on Wikipedia English like two months ago it was very experimental to see if it was going to work you know did it break and whatever so now that we got those bugs worked out I definitely think that we will keep that in mind and also ask the other wikis if they would also like to run the bots on their wikis as well so I mean there's any number of things that we're thinking of trying to automate instead of trying to manually do them one of the things that they're speaking of translation one of the things that they're interested in is finding out which one of their reviews are the most used on Wikipedia and if that if knowing the names of those and they could make it priority to actually translate those figuring that people on Wikipedia English and the French Wikipedia and the German Wikipedia some of the biggest medical content ones were using these the most there's a really good chance that other languages would also like to use those and they would put them priority to translate them because just like us I mean everyone grapples with translation and you just can't do it all at once I mean there's not a system in place Yes this is similar to a translation question but it's within the same language for an information is to providing information to a clinician or something like that abstract straight out of Cochrane are very useful if you're providing information to a patient however you may have to you may have to to make the language more simpler so I'm wondering how much work is being done to make things from English into simple into the simple English Wikipedia we are very committed on you know we're committed at failing to actually you know make our make our content be readable right now the long articles are very complex so what we're trying to do is to get either like three or four paragraphs at the beginning of the article to be concise overview of the whole content and had that be in very you know simple language and then to reference it so that they can easily be translated over and the references will carry over and normally in Wikipedia the referencing is done lower in the article and not in the lead but we really really plead with people to actually reference the lead in the situation because it makes it so much easier when we when we move it over translation wise but that is our goal and so you know people can translate full articles but really what we would like is to improve the lead and then get that translated into many many languages that's that's really the model we're using right now yes so what they can do okay okay two the way that it works in general is that you know most we encourage people to use the referencing tool that makes it so that when you click on it if it's on the internet you can go straight through to where that article is located so with Wikipedia then you can go through to abstracts of Cochrane reviews you cannot go through to the full article that is not at this point in time it's behind the paywall you can't go through it if it's new research if it's a year old then you can go all the way through Cochrane has some of its funders who are asking them to be open access so some of their articles are open access in the area for example of disasters humanitarian stuff a lot of that is done through public health type of the organizations and it is open access but there's many unfortunately there's many other articles that are behind paywalls and also are not open access so that is one of the things that they they're committed by 2020 to being open access and you know I'm there hopefully influencing them to do it quicker and also to make sure that their license they do choose to use is a really good license and not one that's been invented by a publishing company that actually is not what we want it to be so we are we're working on that as well you know there's a lot of things we're working on that I really couldn't cover in the presentation but that's that is definitely one of them Sydney hi thank you for the presentation and thanks to Cochrane for the good work that they do because we know by using your evidence-based reviews it helps us to get our information in Wikipedia but I'm aware that the Cochrane organization just sent out a grant proposal to I think it was to more better disseminate their information make it more known to the world I'm wondering if part of that had anything to do with Wikipedia whether the electronic part of that was there anything that came out of that I think that they like a lot of other people you know they've been around since the 90s and you know a lot of like the federal government has had things on the web for a long time as well but it's really been within what past 5 to 10 years that people have started thinking about what it really really means to disseminate on the internet you know or disseminate with social media so it was kind of around the same time they were thinking about that whenever Wikipedia and them kind of came together so I do think it is part hand in hand but it's not solely about that but they at this point in time do have us build into their dissemination plans I mean being there you know they believe that they know that people don't read their reviews they you know clinicians may read them in guidelines clinicians may read them in Wikipedia articles and virtually no citizens or regular people read them in guidelines but you know right so you know so the place that for people to really get the information is on Wikipedia at this point in time and they're really committed to improve the content not just with the reviews but the people that are in Cochrane are actually usually I mean very small staff of people most everyone is in a university or research center somewhere around the world and they believe in doing good scientific research on health information or there are specialists in their topic area for that reason you know they just want good information to be out there and so that's you know this is why it works together probably need to wrap it up so it worked out about right with no questions is that right Richard okay well thank you so okay so sorry about the delay and getting started but my name is and I am a librarian at Hunter College I am the web and digital initiatives librarian there and today I'm going to talk to you a little bit about how you can use Wikipedia to teach students how to use the library or teach research skills so I got started with Wikipedia back in 2010 when I was part of the public policy initiative at Indiana University and so I've been working with college professors with classes to kind of help students learn more about Wikipedia and how it works and so I've also started to work with it and use it as a way of teaching students more about research so just to do a quick poll how do you all use Wikipedia in a non-wikipedia editing role so what do you do or how do you use it so like high level research what do you mean okay yeah okay so you use the references a lot and so do you think college students should use Wikipedia in their research so why do you think they should use it or how do you think they should use it yeah I think that's a good way any other ideas the idea of Wikipedia but it's really referencing Wikipedia may not be ideal for what policy is worth but certainly the place to start yeah I think that's a great point for you in this case basically it's good at the last thing that someone's saying to the public that you don't know if there is a reference if there is a reference then you can't really go into it but there is a reference there's a reference and I'll let you say what Wikipedia is all those things are different but Wikipedia is a good source that you might not be able to find as quickly so you think students should use it but they just need to evaluate the articles so if you also have a look around what the idea of a source is about if it doesn't have a good benefit then you can just follow it because sometimes someone might not like it but it's one of the few things on sources I would recommend but was there another comment over here I was just going to give you a slide if you really want to take a look if you really allow if you really want to how would it be I have to take the time because I don't want to okay I think the point the time for this question was already passed because our students are going to use Wikipedia whatever we say and so we can't really I don't think we can ask this question more I think the question now is how do we make sure that they use it in the best way possible yeah I agree with you but I think it's a good way because you know students are going to use it and so that actually is a good set way into why and so a lot of you pointed out is where people tend to start and so because it's a good place where students already are familiar with and they're comfortable using it it's a good starting point as they already know it and the content there tends to be less scholarly it's easier for them to read it and it has a lot of detailed information and so they can branch out and see different aspects of a particular topic or realize if they have a broad topic down or different angles or slants that they can take on a particular issue and it can also be a way for them to see how information changes because the information in Wikipedia doesn't always stay the same so someone might come in and change an article or add additional information or they can also see that the information may not always be consistent or one article might be really great in another article maybe it's not so great so it can help teach them to evaluate that information and help them to really as one person pointed out we really want to help them learn how to use Wikipedia effectively because we know that they're going to use it so a lot of information comes from Wikipedia or people have so much information out there that it's a good way for people to be able to also find out or use that information more effectively because there's so much out there they may not really know where to go to get information so being able to direct them to some place like Wikipedia is the starting point and so one assignment is to be able to teach students to actually do background research so oftentimes when students are doing research they don't really think about oh I need to I don't really know much about this topic so I should take a step back and actually educate myself and learn more about whatever topic it is that I'm researching and so that could help me to write a better paper or at least know more about how I want to go about approaching this particular topic and so one thing that you can do is ask students to actually go and pick a topic and read the Wikipedia article and then from that article I'm asking them questions like about the key issues so what are the key issues in that particular topic why is it important what are some important dates or how can they redefine that topic so we were looking at birth control because sometimes when I work with students they have very broad topics and so they're like oh I want to write about blood control or I want to write about birth control they but that's not really a good research topic or that doesn't you can't really write a paper specifically on those you need to narrow it down more and so one way to help them do that is by helping them telling them to read through for example the lead of the article and then looking for that basic information so that they can find out more about that particular topic and then also looking at the subheadings and so we were looking at birth control then they could just look at the subheadings and see that they're all different types of aspects that they can focus on so they can look at the religious views or they wanted to look at it from that point of view they can look at the various methods contraceptions or if they wanted to look at it from more of a the controversy around birth control they could do that as well but a student may not come to birth control or that topic and think about it in that way so just having them look at that article can be one way of educating them about a particular topic and help them make the decision about how they want to narrow their topic or what angle or slant they want to take on it so another option is to have students compare resources and so there's Wikipedia but then there's also traditional library resources like encyclopedia Britannica or McGill virtual reference library those are all library resources and so you can have the students look at both of those two different articles or entries and see what are the differences between the two and so the idea being that to help students learn that they should look at more than just one article or one source and so oftentimes students might come to a particular source and or have a topic and find one that's really good and decide that they're just going to use that one source or that one book article whatever it is and not really look at other sources and so being able to compare the two they can see how the two are different and if one is better than the other or if they maybe they're both the same so we looked at this article on the Wall Street crash in 1929 you have one from encyclopedia Britannica and one from Wikipedia and they of course both tell the same information but Wikipedia includes information that encyclopedia Britannica does not and vice versa and so it doesn't mean that the Wikipedia article is wrong or encyclopedia Britannica article is wrong it's just that sources might give you information different information and they both are good so it's good for you to look at both instead of just focusing in on one particular source another option is to actually have students do a Wikipedia article analysis and so the purpose being to help them kind of what somebody touched on earlier with to actually analyze that article and be able to determine whether or not it's a good article or maybe if it's even appropriate for their particular topic and so in doing that students can you can ask students about those and if there is any if they feel like that particular article has covered the topic well or if there is any bias in a particular article and whether or not the think the information is presented objectively and so and you can also help them look at for example the top page and see if there has been any discussion of neutral point of view or if it's been flagged and have the students kind of discuss that and look at whether or not that article is actually shows any signs of bias one thing you can also do is just have them look for within the article if it makes general statements like many or most or things like that or if it's giving very specific statistics to support whatever is being said in that article so we learned a little bit more about how to actually decide whether or not the article is bias or if it's actually giving good information another possibility is to actually have the students look at the different sources that are used in a given article so a lot of Wikipedia articles of course provide a lot of citations but some may not provide as many and so you can always have the students look at the sources and decide whether or not they are good sources or are sources that they might want to use for their particular research so I was recently working with students in a class where their assignment was to go and research a site of historical significance in so they could pick any site that they wanted but they needed to go and do research on it and explain why it was significant and specifically why it was significant in terms of American politics and so they gave the students an example we looked at the African barrier ground and so one thing that I asked the students was to look at the references and looking at the references to decide if there were any references that they thought would be useful if they were going to go and do some additional research on this topic and so one of the of course the students might look at the first one was a good one to look at and it's not that it's not a good one to look at of course it is but having knowing that their assignment is focused more on the political aspect of it having them look at all the resources and specifically select some and say why would you use that then was able to help them to discuss and see that well if they actually looked at one that was on for example New York burning they might give, it gives them a little bit more information than just the general overview of the African barrier ground and they can look at it from a political aspect and what why it might be actually significant to society and finally another assignment is to have students look at an article and look at the how that article has evolved over time so having them look at the history of the article and compare the current article to an earlier version of that article they get a chance to see that information in Wikipedia of course changes and it's dynamic and you can also have them talk about what information was taken out or put in and looking at the talk pages also looking at that information and seeing what was put in and if they agree or disagree with that or if they think there's something missing or if that discussion was maybe lopsided maybe there was one person that was kind of dominating the conversation and not talking about other aspects of it so it gives them another chance to really understand how Wikipedia works and be more thoughtful or critical about the information that they're getting so instead of just assuming that whatever information is wrong Wikipedia is accurate and that there's not any bias in whatever they might be reading and so now I'm about out of time so I just want to leave a little bit of time for questions I haven't seen that respect but it hasn't come up but yeah definitely can look at those might be some articles to look at and see if there is a particular bias or information that's being kind of slanted or left out I don't have them that documented any place no but I can make them available so that other people can get access it and should be evaluating the article those sorts of specific resources would be valuable to other people who want to try something like this yeah definitely and I have specific questions that would ask them to when they're going through the articles in terms of evaluating them what to look for certain types of information so for example if they're evaluating the article looking at if they are providing general statements or if they're providing details statistics or where their references are coming from if it's mostly online articles or if they are also referencing books or journals or some of the information for example if it's a topic that they're studying in class is the information that they're learning themselves is that also in that article because if it's a key person that's writing on that topic or in that article as well well thank you where's the heck where's the oh okay I thought we had a I thought we had a hookup for a laptop directly here what? I've got one I just don't know literally where it plugs in yeah we did this an hour ago I can put it on a USB probably going to end up asking questions at the end but so we're starting a little bit late it's 4.37 now I'm going to try to keep everyone on time my name is Adam Highland I work for the Wiki Education Foundation and I'm going to talk a little bit about the sort of volume of student editor data and how to think about it one of the reasons why I wanted to talk about this was when you think about like what Wikipedia gets from student editors now from the Wiki Education Foundation our point of view is that Wikipedia gets all of this great content that the student editors produce and that's absolutely true and you can look at the other presentations from the folks of the Wiki Education Foundation you can look at our monthly and annual reports and we'll show you like all this articles added, need content that students have added but when Wikipedians think about what we get from the education program I think we tend to come at it in a pretty idiosyncratic place now my background in this is I joined the education program in 2011 as a volunteer and as a long term Wikipedia editor my thought was that this was going to solve Wikipedia's retention problem this was going to give us an influx of new people who'd be interested in Wikipedia because you know I loved it well why wouldn't everyone else love it they just needed to be exposed to it and I think that what I've seen as I've worked with the program as a volunteer and as I've worked for Wikiped is that we have a lot of people who tend to maybe not officially judge the program on those merits but sort of unofficially talk about it you know they talk about adding student contributions and there's always a yes but those students don't go on to be life long Wikipedia editors and the student edits are not the only place where we see this situation there we go and then edit-a-thons we can sort of consider edit-a-thons like long-term experiments and recruitment to Wikipedia and you know I've always felt this is a sort of silly way to think about bringing new people to Wikipedia and adding new content to Wikipedia because we know that the number of people who edit Wikipedia versus the number of people who read it is just infinitesimal right it's less than you know two I don't have this in the next slide it's of a percent of the folks who edit Wikipedia so it's odd for us to say well even among a self-selected group of people who are going to come and edit-a-thon that we would convert even a meaningful percentage of them if you had a hundred people come to an edit-a-thon and the prior probability of someone wanting to be a Wikipedia editor if you just plucked them at random was a hundredth of a percent you're probably not going to find one even if you run a hundred edit-a-thons and you know student edit-a-thons tend are actually a better experiment for this because the people who go to them tend to want to be Wikipedia editors right one of the you know cardinal advantages slash disadvantages of the wiki education program is the folks who come to edit the articles have to right they need to for a grade so we shouldn't we should expect an even lower conversion rate for students coming out of you know the education program to become long-term Wikipedia editors and I think that it's you know not necessarily reasonable to think of that as a metric but I want to think about student editors as a cohort of people who are acting on Wikipedia during the course of the term and they will go on to you know go do other things they may edit Wikipedia they probably won't you know the next term another cohort of students will come in and edit Wikipedia it won't necessarily have to be the same person and I'm what I hope to do here is convince you that that's a reasonable way to look at the volume of student edits and then take a look at like exactly how big is this cohort what is it doing so one of the things that I've talked about a little bit is I want to you know get get people to think that like editors right are rare right most social sites will follow what's referred to as the Pareto principle or the 99 one rule where 90% of the readers are sort of passive 9% or semi-active and 1% or the the hardcore community Wikipedia follows some ridiculously telescope diversion of that principle you know where you know whereby you cram all of that last 9% and 1% into this tiny tiny fraction of of editors for readers and so when we talk about editors on Wikipedia I wanted to bring this up because I think it's a great graph and I think it's a sort of sentinel point for us to all look at even though this is not the topic of my presentation that we may be seeing an uptick in active Wikipedia editors this when we talk about Wikipedia editors is sort of the graph that we all want to talk about and this is a graph of editors who've made 100 edits within the last month and we've got this is the sort of core Wikipedia community and the or you know one way to think about the core Wikipedia community and even though this is sort of the graph that we visualize when we think about Wikipedia editors these folks are essentially 10% of what you know Wikipedia community generally considers as active editors as we tend to cut things off at highly active editors are those who have edited 100 edits within the last month and sort of active editors are those who have edited 5 edits in the last month and you know for here we're climbing back up into 4,000 for highly active editors and we have about 30-35,000 regular active editors and that's probably a better metric for what the core group of active you know moving editors on Wikipedia is and I think there's like a reason why we should think about this larger group or think about folks who edit slightly less as sort of important members of the community an important cohort to watch the like I mentioned it's like highly active editors are small fraction of the broader community and even among highly active editors when we look at that graph of 4,000 people of the edits that folks are responsible for in that graph right it's a tiny it's a small fraction yet again of that 4,000 people that contribute the bulk of the edits that we have and also life cycles for long term editors and medium term editors are different right so a longer term editor may see a life cycle that's counted in terms of years right as they you know get involved with the project they get heavily involved in article editing then they move over to the project politics side and then they get burned out and they stick tired on their user page a couple of times before it finally sticks and then they go away um this is cynical isn't it but life cycles for shorter users can be entirely different you can have a user who comes in and edits for a few months and makes a few changes to one particular article and then leaves they've satisfied their curiosity and that's a perfectly reasonable life cycle for an editor that comes in and you know and makes maybe say 100 edits or 200 edits total right and then and then leaves and those those folks are sort of the bulk of this this migrating you know slightly active margin what I would consider sort of marginally attached Wikipedia community and they tend to as if I have looking awful lot like student editors so I'm talking about this I want to take a break and talk a little bit about where I got my data sources from um so I I'm looking at editor monthly contributors from uh from Aaron's data set because it is much because I'm only interested in edits per month by editors so it is much more convenient to take you know a two gigabyte TSP file it dig around there rather than going through the whole data set and I'm also picking up student user names and information from the sources that we have at Wikiaid and that you actually can go get if you want to go hit the students API um one caveat for both of these is the data that I'm going to talk about is only for fall 2014 and spring 2015 we don't have the monthly editor data set for fall 2015 because we're in it um and it's updated to June and we also you know less sure of how much the students have contributed you know so far so I want to just give you the information for a full year there so we're thinking about the median student uh they make about 20 edits right this distribution is a little bit skewed um because there are a number of students who make many more edits so the mean is a little closer to 30 um they edit for about one to two months which conveniently is the is the life cycle of their uh of their assignment and they usually focus on about one article now this is not you know the median student is not the modal student right that that you know there are a pro you know a good fraction every semester of about 20 percent of the students who make very few edits they'll make three or four to complete the training and never finish the class and you know that's because you have plenty of students who come to class and then don't finish the rest of the course or come to class continue going through the class and don't finish an assignment in the course and you know we're not going to be able to track that but this is essentially what the median student looks like is they'll make a little more than 20 edits they'll edit for uh you know for a short period of time and they'll work on one particular article um I'm going to take a little break here and uh because I picked up the name from this from Andrew because I like the idea of a comments break uh this is a photo that was uploaded by a student uh it is there's nothing special about this photo it's not going to win featured picture contest uh it's uh it's a picture of a room um inside the Mead Art Museum uh at uh Amherst and uh the thing that's interesting to me about it is this was uh this was an upload that a student made that was uh that was not actually their work it was the work of a photographer who had held copyright on it and someone came by and said you know they wanted to delete it and the student went by uh went back and went through the OTRS process to get the photographer to give a release to Wikipedia and now we have this file on here and I just think that's Dan so like a lot of the students that we have like many wikipedia edit a small number of articles for a few months and never come back you know we can debate perhaps and you know at wikicon and in other conferences and online as to whether or not that's because of the hostility of the wikipedia community or because of the intractability of certain tools or what have you but I would suggest that it's a common enough pattern that maybe that's just the hobbies that folks get into this is a sort of natural progression that folks might want to go through and I you know this to me suggests that the behaviors of a student editor work an awful lot like the behaviors of regular marginally attached wikipedia and so what that what that leads me to believe is that we shouldn't think about student editors as the individual right working through the particular thing but as a group of people who are under a particular assignment for sets of classes over the period of a term and so we should think about a cohort of students moving in working on sets of articles that may be the same that may track with a particular class that moves semester to semester those articles may change we've had courses that go back and update articles that have been previously worked on we've had courses that expand a new material but the way to think about it is that every September you're going to have about and it's growing about 2,500 students coming in and working on a wide variety of articles and they're going to be gone by December and in January you're going to have another 2,500 coming in and working on another set of articles and they're going to be gone by April and the period of editing that we sort of as the community we get to enjoy the fruits of right is what comes in the line of that cohort there are summer semesters but they make up a sort of vanishingly small percentage of the revisions that are committed so this is what I like to call burying the lead if we look at student editing within the periods of the term we get about 4% of that overall active editors so that 35,000 number of folks who have edited 5 times within the last month students represent students that in other words students that edit actually that much represent 4% of those editors now there is a little bit of trickery here and so I really do apologize because if you think about how we measure active editors we think about activity within the previous month and so if I'm checking student edits during the period when the term is running and those students are editing then they are going to by definition have been active during that term so I'm not being 100% honest here that if we were to think about students being active we would average it over the entire year but I'd like to think more about the massive folks who are coming in during that period of time rather than what they represent over the whole year now if you think about averaging over the course of the year students who are active over the year they represent a much smaller percentage because we have more months that students are out of school than they are in school but when they're in school they represent a pretty significant fraction of active editors on Wikipedia and among those active editors they contribute between one and one and a half percent of all of the revisions made by active editors during the period of that term and that number is tiny and feels tiny just as this feels tiny but it should kind of be astonishing or at least it's astonishing for me that we have one particular group that represent that contributes a pretty large fraction of the edits that we get over the period of a term so one of the ways that I want to sort of get us to think about students as a cohort is looking at a semester as a glance now this may not the portions up here are I need to go back here for the the colors up here are for what we are referring to sort of like internal milestones for courses now this is generated using data that we've collected on these courses you could in theory go and generate yourself if you want because all the dashboard data is public and all the course data that we have is pushed also to Wikipedia but we are essentially looking at over the course of the semester from the middle of September to the end of apologize for the colors on the side of the slide here to the end of December where students are working and what they are working in so we get to see students coming in and working through copy editing articles working on the training editing sort of sandbox and then progressively moving more and more to main space over the period of the term and I don't just to me I like this graph it's sort of the heartbeat of a particular semester for a number of students and it's a little difficult to see again I apologize for the the axis ticks on there but this is about 100 classes that you are looking at so the 100 classes going through a milestone in the tallest bar there so one of the other reasons why I want to think about students as a cohort is because we have a really strong advantage this is a set of Wikipedia editors that are coming in that are doing the student training we are looking at the fall 2014 and spring 2015 students we had about a little over half of them doing the training for the fall 2015 we project that we are going to have about 80-85% of the students completing the training by the end of the term which if you subtract those students who don't actually finish the assignment it gets pretty close to 100% for those folks these students also have resources resources the wiki education foundation provides and resources they have through their the reason why we want them to create the great content and they also have support available so this is a distinct cohort of people that's going to grow larger that's got some of these advantages that allows them to produce a lot of this great content on Wikipedia another comments great this is another great file uploaded this is the score to a prologue of an opera that is way way in the public domain published in 1608 scanned and uploaded by user LMR0804 who is a student editor from the fall of 2014 so I want to give a few more caveats after offering that first of all one of the things to remember is we are measuring student edits just as revisions so I'm not looking at who made I'm not filtering any of this for what's considered to be a substantive contribution I'm not even filtering it for what goes into main space because when we count revisions by editors to determine editor activity we don't do any of that and I think it's reasonable that's an entirely different question if we want to ask that the other thing is that long-term when we talk about edit count itis we'll think about sort of like the number of edits that someone produces who edits very frequently and makes iterative changes versus someone who composes in a notepad and who adds everything all at once to a particular page now that is a habit that students are anecdotally much more likely to do than regular editors so it's possible we could be getting a slight underestimate of how many revisions we're getting from the students but that's in the category of not even being wrong because we don't have an actual measurement of how much more students do that so I don't want to correct any of the values by a gut estimate of that and I have to the best of my ability tried to pluck out all of the TAs and online volunteers and professors from the student data because they do show up as students in a lot of these courses I may not have been fully successful and so that will change the specification of this although to be frank the specification will change a lot more if you move the month bands for how much you consider activity but I thought that was a short little tour of a large number of students coming into Wikipedia and anyone have any questions please thank you for reminding me if you want to ask a question go right over there I'm Aaron Hafhager from the Wikimedia foundation I'm really stoked that you found that data set so that's awesome I think I uploaded that like three weeks ago so really fast and solid work it was really fortuitous so I have one quick question so you use the term student editors a lot and I just wanted to make sure that I understood these are Wiki Ed editors that are registered in part of the program right? Correct okay so of