Thank you everyone for coming. I'll just start by introducing myself. My name is Chelsea Raoul. I'm the Digital Initiatives Librarian at Wake Forest University. In that capacity, I oversee digitization of special collections, and then I also liaise with faculty who are interested in pursuing digital humanities and research. So with that combination of responsibilities, you can understand why participating in the DPLA would be so attractive, not only to our institution, but also to me personally, because it helps me to market our digitized special collections as potential corpora for digital humanities research. So fairly early in the process of envisioning the DPLA, Dan Cohen used the metaphor of a pond feeding a lake, feeding an ocean, to characterize the planned technical infrastructure. He wrote in a blog post, one can think of the initial set of materials as content from local ponds, small libraries, archives, museums, and historic sites sent through streams to lakes, state digital libraries, and then through rivers to oceans, the DPLA, and not to extend the metaphor too far, but it certainly influenced Wake Forest's understanding of our niche within a larger ecology. I'm shamelessly using the Ponds Lakes ocean metaphor to structure my presentation, and I'll spend a little bit of time on the ocean and the lake, but for the most part, I'll concentrate on Wake Forest's perspective as a pond or a contributing institution. The DPLA markets itself via its tripartite mission to be a portal, which is the DPLA site, a platform, which is the DPLA API, and then a strong public option. So for me, it's really the second item in that tripartite mission, the portal, which is most resonant, and I actually, just in my history of following the DPLA, I connect it very strongly with Ed Summers, who wrote a blog post that called attention to the phrase the generative platform for unspecified future uses in his description of about what the DPLA could be. So by working with the grain of the web and deeply linking between content and from content to contributing institutions, the DPLA facilitates or can facilitate deep research and knowledge creation, but aside this very serious purpose, one of the primary use qualities of the DPLA, whether it's browsing the DPLA site or interacting with one of the apps that's built on top of the API, is just fun. It's really fun, and I think that this doodle, which was captured during one of the early DPLA plenaries, sort of conveys that sense of fun. So we at Wake Forest want to expose our materials to the world, and we want to do it in a way that aligns with the web and with this concept of a generative platform on top of which people can build things. So moving backwards from the ocean of the DPLA through the rivers to the lake or the service hub, I got some information from Nick Graham and Lisa Gregory, who lead the North Carolina Digital Heritage Center. So I'll speak briefly to their role as a DPLA service hub just in order to contextualize Wake Forest participation as a contributing institution. Chelsea, can I throw in here? So I'm Chris Freeland. I'm an associate university librarian at Washington University, and I'm going to be speaking more on the lake piece on how here locally in Missouri we're building a DPLA response. So the North Carolina Digital Heritage Center is an ongoing LSTA funded program that provides digitization and digital publishing services to cultural heritage institutions across the state of North Carolina. Typically when an institution partners with the Digital Heritage Center to digitize their materials, those materials are also published on the Digital Heritage Center site, digitalnc.org, pictured here. Because of the Digital Heritage Center's strong existing relationships with academic libraries, public libraries, archives, museums, historical societies, and others in North Carolina, it was really a natural fit to act as a DPLA service hub. In in fall 2013 they formally became a service hub part of the I guess the second cohort of service hubs following the DPLA's launch in April 2013, and what that meant for us locally was that in October the Digital Heritage Center invited digital collections managers from across the state to a day-long informational meeting in Greensboro. Emily Gorn, Amy Ruder's door from the DPLA were there, and so the point was for us to talk with one another to learn more about the DPLA metadata model, their ingest process, and ask any questions that might be particular to the peculiarities of our digital collections. We at Wake Forest and really all of these institutions in North Carolina are really lucky because the relationship infrastructure is already in place. I think a lot of times we think that the technical infrastructure is going to be the hard part, but that groundwork has primarily been laid and more sometimes we find that it's the relationship infrastructure that's the hard part, and as it happens it's already strongly established in North Carolina. So institutions can contribute to the DPLA via the North Carolina Digital Heritage Center service hub in one of two ways. The first way is simply for the materials to appear in DigitalNC.org. So this map pictures the locations of institutional partners with the Digital Heritage Center, and it's reflective those those institutions, their materials also appear in DigitalNC.org. As we say in North Carolina, you can see that they stretch from the mountains to the sea, and then also the Digital Heritage Center's partners run the gamut of types of institutions from academic libraries, 68 academic libraries, 52 public libraries or private libraries and archives, and 24 cultural heritage organizations such as museums or historical societies. The second way that institutions can contribute to the DPLA via the North Carolina Digital Heritage Center is to actually contribute OAI feeds of their own digital collections that appear on on their site. And so currently there are 12 institutions contributing to the DPLA this way, including the North Carolina Digital Heritage Center itself, the State Library and State Archives of North Carolina, eight academic libraries, and two public libraries. Of these, most institutions are using content DM, and as it happens Wake Forest is the only institution using dSpace to manage our digital special collections. So to a certain degree, we've just had to blindly feel our way forward and we're proud of the steps we've made so far. So I'm going to very briefly summarize the North Carolina Digital Heritage Center's ingest process, but you should know that this is all just a summary of what Lisa Gregory from the Digital Heritage Center shared with me, and if you have any questions, rather than field them myself, I would be happy to put you in contact with her. Each month the North Carolina Digital Heritage Center provides the DPLA with a single stream of metadata represented in mods. They form the single stream of metadata by aggregating the feeds that are provided by various other institutions such as New Hanover Public Library or Wake Forest. So to incorporate an institution into the single stream, what the Digital Heritage Center asks for is that institutions OAI PMH URL. They take a look at their metadata and make sure that all the elements that the DPLA requires are present. Then they create an XSLT style sheet per institution that transforms Dublin Core elements to mods elements. Once the style sheet is prepared, they use Repox to, which is a Repox is an aggregation software, so they use Repox to combine all of these feeds for each institution providing a feed to indicate which collections that they want to be included into that feed and to assign the style sheet for that institution. Then they do a test ingest. They look at the mods output to make sure that it's not wonky. At that point they're pretty much set and they contribute that single mod stream to the DPLA. Each month, about a week before the DPLA harvests or is scheduled to harvest from the North Carolina Digital Heritage Center, they will re-ingest all of the different institution's feeds in order to incorporate any changes to metadata or any new collections that an institution is requesting be added. At that point they simply wait until the DPLA finishes harvesting. They might do some spot checking to see how materials appear in the DPLA and if there are unexpected outcomes, then they will sort of mediate that process of submitting a ticket, so that's the Digital Heritage Center's ingest process and part of their role of sort of mediating between contributing institutions and the DPLA as a whole. As far as the benefits for the North Carolina Digital Heritage Center of participating in the DPLA, they reported that they get about 100 to 200 visits per month and 300 page views for months directly from the DPLA. And of course the DPLA is always looking for ways to increase traffic to partner websites by promoting content through blogs and Twitter and other social media. But when I spoke with Lisa Gregory of the North Carolina Digital Heritage Center by email, she said, quote, the bigger benefits we've seen so far from participation is the experience we've gained in aggregating metadata on such a broad scale and seeing how it performs as well as being a player in the conversation about data aggregation with others in the DPLA. And I think, I'll talk more about this later, but I think Wake Forest really shares that perspective that driving traffic to our collections is one thing, but there are other benefits in addition to traffic that we derive from our participation in the DPLA. Having shared my understanding of the larger DPLA ecosystem in which Wake Forest is participating, now I'll focus on our experience so far as a contributing institution. From the beginning, our task was purposefully iterative and incremental. Like many institutions, there are imperfections hidden in our digital collections and rather than trying to correct everything all at once, we tried to break things down into smaller chunks that we could achieve each month. And we felt that this approach really fits in well with the DPLA's design process, which also drew inspiration from agile development methodologies. Memorably, for me, was the DPLA beta sprints that garnered submissions that thoughtfully engaged all sorts of design problems from aggregating metadata to preserving an item's context in its original collection and designing interfaces that facilitate serendipitous discovery. So for us, agile development is really a loose, a very loose framework for how we approached contributing to the DPLA. It wasn't necessarily a series of strictly defined four-week sprints, but what was valuable for us was defining a single priority that we wanted to focus on at the expense of all the other possible priorities that we could focus on during a so-called development cycle. So for the rest of the time, I'm going to address our series of problems and sort of discuss how we broke them down into smaller priorities. First, though, a little bit of context. Wakespace is where all of our Wake Forest University digital collections live. Wakespace is an instance of dSpace, and it's the home not only of our digital special collections, but also of our institutional repository and also digital content related to campus events like symposia, recordings of the library lecture series, things like that. So to a certain extent, because of all of our digital content kind of lives in the same dSpace bucket, our first challenge was sort of wrapping our heads around how we would contribute selected collections, and those collections were either selected because, well, we weren't contributing our IR content, but also even within our digital special collections, there were some that we really did want to invest more time in cleaning up, and so we didn't quite want to contribute them to the DPLA yet. And so this was certainly my first experience in exposing collections via OAI and sort of really engaging in that standard. So when I was framing this problem, I was first thinking, like, oh, well, it should be like bundling an RSS feed. We should be able to bundle these OAI PMH feeds for specific collections and then send that bundle to the North Carolina Digital Heritage Center. In the end, it really turned out to be a non-problem because through Repox, they were just able to identify us as a data provider and then identify the sets that we wished to provide. But it was the first challenge that we really tried to engage with. The second challenge was revising our right statement. So when we first began planning on contributing our collections to the DPLA, our right statement didn't exist at the item level, and really it existed more at the collection level. And so we knew that if we were contributing our collections to the DPLA and our items were going to appear in that context, we certainly wanted to include our right statement at the item level. But it also, if we were going to have to populate the right statement at the item level, it was also really an opportunity to revisit what our right statement was and make sure that we weren't falsely communicating a sense of ownership of these materials, but trying to be more explicit about the fuzzy status of many special collections. So it really created an opportunity for me to bring together our scholarly communication librarian and also representatives from special collections faculty, and they worked together and we went through a couple of drafts and got a right statement that everyone was happy with. So before participating in our first harvest and building on our revised right statement, the two things that we identified as changes that we did want to make before our digital changes, we did want to make two our digital collections before contributing was A, populating DC rights and B, populating DC type. So there were lots of other things that we might also want to have revised within our metadata, things like making sure that all of our items had a date created so that they could appear in the DPLA timeline, making sure that all of our things had spatial information so that they would appear in the DPLA map, but both of those would take more time. And so at just the base level, we wanted to make sure we had rights and type, and those were super easy. And we did populate those fields. And then we participated in our first harvest. Oh, right. So here's the value of making sure that DC type was populated. It means that users in the DPLA environment could, for example, select text and make sure that they were viewing items that were textual as opposed to items that were images. Our second big challenge was making sure that our thumbnails appeared in the DPLA. For working with the digital heritage center, this was a little bit of a unique challenge for Wake Forest as opposed to other North Carolina collections who either sort of all of their thumbnails appeared in one space on the server where their digital collections were hosted or there was a predictable naming pattern. Because we're a d-space institution, the URLs of our thumbnails, just like the URLs of our metadata records or digital objects are opaque identifiers. And also, if we're contributing our collections via OAI, the Dublin Core OAI feed, the location of the thumbnail isn't included in that descriptive metadata record. In short, and if you have more detailed questions, I can talk about it later. But essentially, what we did was treat the locations of the thumbnails as identifiers in the Dublin Core record. And there was a handy-dandy attribute that we could turn on to indicate those URLs as thumbnails. And our thumbnails have not yet appeared in the DPLA. The last harvest we did make this change, but that happened to coincide with Code for Lib. So we have an open ticket, and we hope that by the end of the month, we will have our thumbnails appearing in DPLA. So the current challenge that we're grappling with at the moment is that we would really like to move from contributing a Dublin Core OAI feed to either a Mods or a Metz OAI feed. That sort of gives us the control of mapping the qualified Dublin Core elements to Mods elements, rather than relying on the Digital Heritage Center to do it. And this is definitely the area, I would say, where we're feeling our way forward the most blindly. So if you have any feedback about this process, we would welcome it. I think another really tangible benefit of mapping to Mods ourselves is that for collaborative digitization projects in which we've participated in the past, such as this project called Digital Forsythe, which had contributors from different libraries in Forsythe County in Winston-Salem where Wake Forest is located. So we host Digital Forsythe materials in Wake Space at Wake Forest University, but when we contribute those materials, which are fabulous, we want the contributing institution, not to be Wake Forest University, but rather to be Forsythe County Public Library or Winston-Salem University. And it's certainly possible for the Digital Heritage Center to sort of hard code that into their XSLT style sheets, but we would rather build our Mods infrastructure so that we are pushing that out and exposing that information via OAI ourselves. So that's the current task or I guess development cycle in which we see ourselves. Future development cycles, really the possibilities are endless. I mentioned that we certainly would want to make sure that we have data created for all of our items. We would certainly hope to have spatial information for all of our items so that they would appear in the timeline and the map on the DPLA site. Subject, you know, applying subject terms to all of our items is something that we would be interested in doing, but I guess viewing our participation as this ongoing iterative thing meant that we didn't have to do all of those things all at once, but it also helped to define our participation in the DPLA as an ongoing digital project just like all of our other digitization streams so that we could devote a certain amount of time every month to going back and cleaning up our existing collections as well as moving forward with new collections. So a few takeaways at the very end. We see our participation in the DPLA as offering us increased traffic certainly, but really more importantly it's an opportunity to continually evaluate and improve our metadata. For me in my liaison role it's also it also offers us the ability to market our digital collections as data and possibly the ability to engage our public in new ways. At Wake Forest we have a humanities institute and under the auspices of the humanities institute there's a digital humanities initiative. We meet regularly and I pitch the idea of a DPLA API workshop to that group and it was received enthusiastically so I can imagine events like this taking place in the fall or the coming spring and those sorts of events are certainly not things that we could have facilitated ourselves had we not participated in the DPLA. So I think we have time for maybe one or two questions before we move on to Chris's presentation. That was something that we talked about at the beginning. It was one reason we wanted to include the right statement. Within the right statement is a you know contact us if you have questions or sort of an avenue for people to say I don't think you have the rights to this and I don't think you should be displaying it. So because it hasn't taken place yet that we've gotten a takedown request I can't speak exactly to what the time frame would be but at at the longest it would be until the next harvest the next monthly harvest takes place but I imagine if we felt that it was urgent we could probably submit a ticket to the DPLA via the North Carolina Digital Heritage Center and it could probably be sped up. So I'm going to talk my name is Chris Freeland I'm an associate university librarian here at Washington University in St. Louis and I'm going to talk about the the middle piece the building the lake. Chelsea says that that the relationship infrastructure is the hard part. Yes in fact it is and that's what I'm going to talk about today is the challenges that we face here locally the challenges and opportunities that we face locally here in responding to DPLA at in the state of Missouri. Missouri has a similar infrastructure to North Carolina but we're at a different state of development and a different state of having the conversation so that's what I want to walk through today and also a belated welcome to the fair city of St. Louis. I hope you're all enjoying your time. I'm going to talk about these three topics my history with DPLA the local organizing efforts here in Missouri and then sort of bring it all together in the status and next steps. So I started with DPLA in 2011 this is a panel that I participated in at Harvard talking about what could the DPLA be this was before the name DPLA was settled upon this was a what could this thing be this digital public library of America I think the the P public part was inserted in around these conversations so I gave a talk I was the technical director of the Biodiversity Heritage Library at the time and I was asked to give a talk about open access and open data and what it means to work in an open environment and that's what I love to do so I gave a this plea made this plea for keeping data and the infrastructure of DPLA as open as possible and that we should not put any new gatekeepers in place that the that these are open data they should be left as open data and that if we were looking at a way forward with DPLA let's look at the material that has already been digitized that's already mobilized let's pull that together and so I was making this point of digitizing public domain books is non-controversial let's go with that and there was a lot of applause and the a fist pump and a lot of applause in the room um I think maybe because of that fist pump then I was asked to participate in a technical summit that was held in June later that year and this was part of the early work stream discussions with DPLA this again was before the DPLA website had launched and this really was these formative discussions of what could it be how big would this like what could it what could we do with DPLA and we had some some really this was a really good discussion it was a one day session at Library of Congress and I think the the most important one was are the third and fourth points that there was less emphasis on the front door and and more on the services and the entry points that really the power of DPLA was in aggregating these resources and pulling them together under a coherent and consistent interface and being able to build APIs on top of that data so that other people could use the data in the way that they wanted to on one of Chelsea's slides there was a thing there was a little diamond that said the shiny thing we started talking about the DPLA the UI was not the shiny thing the shiny thing was the where the API and the and the data beneath so that was a fun discussion and I and I would say that this again was very early and maybe helped shape the direction of things to come with DPLA I also participated in one of the early hackathons this is on your left is my view I was here in st. Louis the hackathon happened in Cambridge so my view from Skype and then the resulting big brother bringing me into the room in Cambridge I will say this that participating in a hackathon remotely can be challenging but it was also a lot of fun so I think this was the first hackathon organized under the banner or the auspice of DPLA so all of this to say that I was part of the early discussions and was watched very happily as DPLA found funding organized hired staff and became the product of the the platform that it is today and it became this really important tangible thing and that's what brings me here to talk with you today is that the we here in Missouri see this as a thing that we want to participate in we want to make our data available in DPLA so a little bit about the state of the state of Missouri and this is not a words in all kind of conversation there's no dirty laundry to be aired here in early 2000 we had a statewide digitization model similar to the ones in Colorado and elsewhere and it funded a project called virtually Missouri virtually Missouri was both a platform as well as a series of digitization conferences and networking that platform has moved on its own migration path and is now Missouri digital heritage which is sponsored by the state library and the state archives oh I broke the chair sorry the conferences and networking became local so when the statewide efforts pulled back a bit we said well let's keep having these conversations these are really valuable and important to have and they became these casual books and beer kinds of conversations where let's just get together let's let's have a conversation about what we might be able to do together and I think the important thing to think here is that we focus locally because it's easy for us to get you know 30 we were all 30 minutes away from one another so we could get in a room we could get in our cars and and have a face-to-face conversation without a lot of organization so picking up on on this themes we decided to do the social engineering from these existing networks and we I was working at Missouri Botanical Garden at the time and realized that there was a need for bringing technical people together locally and we created this group called TECHO technology exchange for cultural heritage organizations in st. Louis which we thought was terribly clever this wonderful acronym that I now have to explain every time we talk about what TECHO is it is what it is you know who cares what it is the important thing is that it brought people together the the fact was we said yeah we want to work together we want to collaborate and this is cultural heritage organizations and beyond not just libraries and museums but also universities and technology partners even financial institutions so we have a it's a research library at the Federal Reserve Bank of st. Louis but they wanted to be part of the conversation they want to get their content into DPLA they don't see themselves necessarily as a cultural heritage organization yet they have economic data that helps describe the history of st. Louis the history of the united states incredibly important information that should be in a digital public library of america so we think of this in the same way that the Missouri digital heritage does which is all of this content is about Missouri and it should all be made available we also saw this as an opportunity to share experience and collaborate the around emerging topics like linked data and digital platforms and digital exhibitions maybe even going so far as to say maybe in our partner institutions locally here in st. Louis we all don't need to run fedora repositories or we could look at maybe digital preservation at scale locally rather than each of us having to go it alone it all came together then in a meeting that was just six and a half months ago uh september of last year we uh we met at the federal reserve bank of st. Louis and we we asked what can we do with DPLA how can we organize and how can we get our content into DPLA and the number one question was well who can be the service hub in Missouri and the Missouri digital heritage uh the partners were there and they participated in the meeting in full and they said uh the Missouri digital heritage is content DM it's in migration if we want to do anything quickly we can't mobilize Missouri digital heritage so we said okay that's great we understand where where the state is and let's talk about how we can continue moving forward and keep everyone in the loop they were very happy with that they they were very happy to be participating alongside this effort an important thing about timing we uh one of the reasons one of the conversations was well let's let's move quickly and the question was well why so quickly in uh st. Louis this year in 2014 is our 250th anniversary of the formation of the city it's our 250th birthday a jeopardy note that's the sester centennial so if that ever comes up in uh in jeopardy um and also the federal reserve bank of st. Louis was celebrating its centennial its 100th anniversary uh so both of these things came together and we said 2014 is it this is when we want to get our content into dpla an interesting point that came up during the conversations was uh came from our one of our humanists one of our digital humanities workshop folks they they said what questions can't be asked about Missouri in dpla today and if you go to dpla you'll find there is information about Missouri but it doesn't tell the story there isn't an exhibition there isn't all of these sort of formative materials it's a somewhat serendipitous accumulation of materials that have been held in other people's repositories it's not Missouri repositories again because we don't yet have that infrastructure so there are gaps in the Missouri story in dpla we understood that there were lessons to be learned from other hubs that we didn't have to go it alone and we also wondered maybe even most importantly what else could our libraries do together in addition to dpla we all recognize the dpla was important and we wanted to look at it as a goalpost but maybe there are other things that we could be doing and formalizing and organizing as well one of them one of the best conversations was about the scope of content and we got in we had this really good conversation of what's the Missouri-ness of a of a digital object and i think this is an interesting uh an interesting point so you can look at this image it may be a little hard to see with the with the overhead lights but this is the gateway arch it's a relatively identifiable object about Missouri not a whole lot of questions this is a Missouri object a couple of small points um keen observers will realize that this is not a photograph of today's downtown this photograph was from 1966 i think it's a beautiful photograph of uh shortly after the early opening of the of the arch a second point this photograph was taken in illinois that's the the river looking at Missouri so if we were going to post this to our facebook wall or flicker the geotags are going to show this as an illinois object not a missouri object so it is what it is but then you take something like this so this is a piece of the papyri collection at washington university library is this a missouri object it's in missouri it's from our collections um should this go into dpla and again we went with the same had the same conversations that the missouri digital heritage uh same guiding principles which was yes this is a missouri object there's a story to be told as to why that object is in our collections there's an interesting part of the history of science the history of of missouri the the role of st louis in the history of missouri and the history of the united states as to why these unusual assemblage of objects are here in st louis there's a great story to be told about that so we said yes in fact this is a missouri object so with those kinds of really uh you know discussions thrashed about uh we said okay let's let's do this so here's where we are the progress we have a loosely coupled administrative working group uh formed by people from the federal reserve bank of st louis the missouri history museum and washington university we have a technical group who is led that that is led by david henry at the missouri history museum who deserves a special shout out because he's really just a superstar and is pulling all of this together and working with emily gore uh emy rudersdorf and the folks at dpla on making this connection between missouri and and dpla we have participation and support from the missouri digital heritage which we thought was absolutely critical for us to to be able to really go forward in full and we sent out a message just at the end of february to universities public libraries and consortia saying that we were building this and who's with us and we've gotten a really good response back from that so the goal by october is to have our content in dpla through a missouri centered hub 214 days away from today if you can hear the clock ticking i can as well um we're gonna do it i'm i'm confident that we can do this and here's why we're taking a startup approach we're doing uh doing more and talking less uh we're thinking fast and flexible i'm not going to try to throw out all the buzzwords that we had there at those early discussions but really it was how can we make this happen in a quick amount of time the number one way was that we have a gang of the willing everyone who's participating in this conversation wants to see their content in dpla there's we are having every discussion to say how can we make this happen we have as little or as much legal as needed at the right time so we are a loosely formed non-organization at this time we don't have an mo u yet but we are developing one we know that that that we have to have an mo u that we have to have more sustainability and organization around this loose coupling that we have and our guiding principle again at this point has been towards innovation let's innovate and organize and then formalize it's been easier for us to talk to show something then to talk about it in terms of the technical engineering this is also one of the reasons why we were pretty confident we could hit this pretty fast timeline because it's a similar technology profile to what chelsea was describing at wake forest it's re pox and oai and we're using that same technology stack because that's what dpla is is advertising and is helping people build so we're again we're not having to go that alone we're not having to build much in terms of from scratch we're having to make sure that like chelsea was describing our metadata fit and that we have the right on ramps but we don't have to build this all from scratch there are these pieces that we can take and reuse which moves us forward pretty rapidly there are some there were some specifics that we decided upon that also helped in this determining the path of least resistance we agreed right from the start from day one that the aggregator the state-based our aggregator will be a silent utility without any kind of a user interface so it's not as though someone's going to be able to go to like the the north carolina digital heritage center there is not going to be a corresponding user interface anything dot org for this service today it's just going to be the silent utility that's running getting the information from the repositories and putting it into dpla we thought that if we tried to build a ui that that would start stepping on some political toes around Missouri digital heritage that's not really our goal our goal is to get content into dpla let dpla be the user interface to these data another second point was that really to fast track this we had to use oai as the minimum entry today again this is a decision in discussion for getting us getting our content in by October and we understand that that obviously that there will be some collections that may not be oai enabled that will have to build on ramps and that's to come that that that will happen over time and and we think that that's probably largely the role of the Missouri digital heritage is to keep content coming in we see ourselves as that aggregator in that utility and the the engine that that helps move content into dpla but the but it's going to originate from other organizations and and should be done in concert with the Missouri digital heritage so where we are today we are in construction phase we're demonstrating like having this conversation today and talking with other people in the state and region about what could we do together as with dpla so though i said we do more and talk less we're still doing a fair amount of talking it's just the nature of the beast so in terms of our next steps i had to include this this image which i just find is utterly charming it's from 1925 this is a charleston dance contest on the on the steps of st louis city hall i just love this image i don't know why i respond to it so well but i just think it's it's just great um so where we are content into dpla by october fingers crossed um as we get uh more content in adding more members and then having the conversations already happening about sustainability we did not go into this thinking oh we're you know this will just run itself we know there has to be sustainability we're talking about that now but we're not getting it we're not being bogged down by it currently um i think it's this sort of fast and flexible approach and another thing about sustainability is that we're mitigating any of these many of these risks by having a really small footprint you know to be able to run this the reprox service it takes like the smallest corner of a server it's not like as though it's a big onerous task for anyone to run we could move this into any other organization any of the missouri organizations have said sure once it's built we can run this uh going forward so that's good what i want to leave you with what i hope you you take away from uh from my conversation here is that in missouri we have assembled the group is together uh we we brought the band back together we're getting our content ready for dpla and we're looking towards october of having that launch we've gotten the buy-in from the major stakeholders and we started building the low barrier tech utilities that will make all of this happen we look our group looks at dpla as a catalyst it turned our conversations into action it gave us the thing that we wanted to point at and say yes this is what we shouldn't just be having this conversation about doing something together let's do it and let's let's put our attention towards getting content into dpla it's been great to have that to have that goal in mind we also see dpla as a wholly collaborative environment so i mentioned several times we don't feel like we're going it alone we're not having to totally build this all from scratch and in fact the way that chelsea and i were brought together for this talk we both we individually submitted our local perspective from dpla and the c&i organizers said why don't you combine this into a single session and we said of course that makes perfect sense that's the way that dpla operates so everything that that i have done with dpla has uh has demonstrated and shown that it really is this just amazing incredible amazingly incredible collaborative environment given this experience i i say i come back to uh to thinking that casual networks can be some of the most effective uh the this group here and the conversations that you're all having in the hallways are probably more as effective as some of the structured conversations you're having uh back home i'm seeing lots of nods nodding heads throughout the room so yeah this is the reason why we come to these things is to have these conversations and to to build these kinds of networks a couple of final points one you know we said what else can we do besides dpla and we really are thinking about the future so we we fully embraced dpla and we see it as an important repository but we've also then been having conversations well now that we have this group of people together what could what else do our collections have in common could we be going to uh going for any age grants could we be looking for big data projects is there more here that we could uh that we could pull together which is exciting it's it's great to be thinking about the future of our organizations and how we could be working together again in this local statewide environment and finally i don't want to end on a on a down note but instead a reminder of why we're doing this none of the images that you've seen today are available in the digital public library of america we don't have that on ramp to get content from Missouri into dpla that's the reason why we're doing all the work we're doing so that someone could go to to d to dpla and hear this story about Missouri or find these images so that's where we are see you in october and please enjoy the rest of your stay in uh the city of saint louis thank you very much


Well, last week we were looking at setting up a very basic web server using Python, and one thing you couldn't do with that basic setup was have server-side scripts running. So today we're going to be looking at the options that are built into Python. So if you have Python on your system, you should be able to do this, that allow you to not only run a web server, but to run a web server with server-side scripts.   If they're not in there, then they're not going to execute as a server-side script. So you would do make their CGI dash bin. I have already created one of those, and I've put a couple of scripts in there. Now, to start our server up, what we want to do is we want to start up Python, say dash M to load up the module, and we'll say CGI HTTP, and this is case-sensitive, capital S server. And also you want to tell it what port. Again, certain ports will be blocked by certain, by lower permissioned users. So port that your user is allowed to use, I'll just say 8,000 here. I'll hit enter, and you can see server HTTP on 0000 port 80. So it's running on this computer port, I'm sorry, port 8,000. Now if we look at our web browser, we can go to our web server. Again, this is on the local machine, so I can type in local host. I can type in <IP_ADDRESS>. I can type in my local domain using the hostname.local, or I can put in my IP address. Of course, this is running locally. If you're going to be outside of the local network, you'll have to set up port forwarding. But I know that my IP address is <IP_ADDRESS> for this current computer. And as I said, port 80 is what we're using, so colon, sorry, port 8,000, so we're using colon 8,000. So anyone on the local network should be able to type that into their server, and they will get greeted with a directory list, luckily just like the simple HTTP server that Python uses. There is going to be a automatically generated directory list if there's no index file. And of course, there's a CGI bin. Now if I try to click on the CGI bin, it's going to give me an error. It's not going to give you a directory output, a directory list of what's in the CGI bin. This is one of the main security things that are put into place, because you don't necessarily want people viewing all your scripts, because certain scripts may, you basically just want to be careful with server-side scripts, because they're actually running on your machine. They can manipulate things not only as far as the web server goes, but could also manipulate things on your system. So you want to be very careful using server-side scripts. So it's going to require you to directly go to a script you created. Let me open up a new shell here and go into my CGI bin. And by the way, when you run the server, it's using the current directory as the root directory for your web server, unless you tell it otherwise. And again, the CGI bin is where your scripts are going to be. I've got a few in here, a few bash scripts, a couple of Python scripts. It does not matter what type of script or program it is, as long as your computer can run it and you've given it permission to run. So let's go ahead and have a look at this hello.sh. We say that it's a bash script, and we're going to do the output. Again, you're going to have to tell it that the first line has to be content type, and in this case, text slash HTML, and then you need to have an empty line after that before you start your code. And here we're just going to echo out something simple, but you could run any program that you want. So here's a very basic example. Again, if you're running this and for some reason you get this text output and not the actual script output, it's because you might have put an extra line in there. I'll give you an example of that. We have this file called hello.sh. You have to make it executable with changemod. I've already done that, but I'll show you changemod plus x and the name of your script. Running these on your web server is same as running it on your current computer. So if I was to run inside my shell here, hello.sh, you can see the output here. First line is this, then we have a blank line, and then our HTML code, which can be one line, but it can be a whole web page as well. So let's go ahead and run that up here. So to run that up here, all we have to do now that it's in there and made executable is again the address of the server, the port. If it's not port 80, if it's port 80, your browser will default to that. CGI bin and the name of the script. We'll go ahead and hit enter. And you can see we've got hello world. We can look at the code for that, which is just the text output there. I can go into that script again and I can add another line. I can say echo, this is a test. And I can give it more HTML tags if I want. I'm not going to here. I can save that. And if I was to close this and refresh this page, you can see now that text is right there. But again, we can also say give it a line break. Actually, we'll give it a horizontal line break there. And next, I can just run a command. If I wanted to list out the files in the current directory, I can use the list command. Now, do keep in mind that if you want line breaks and things such as that, you're going to have to implement the HTML code for that. So it will output the files in this folder, but not necessarily on a new line for each one. But let's go ahead and refresh that. And you can see the files in this folder. Or actually, it's giving the files in the main folder. I did not expect that. But just keep in mind that things aren't going to be on the new lines. You have to implement that. You'll put this through a while loop if you want. Should be able to do something like while read line, do echo dollar sign line. And we'll do a line break like that. And that should. Now, if I hit F5, put each thing out on a new line. If we look at the code, you can see that they're all on new lines with line breaks. Again, if we didn't have that, and we just had the list, I refresh the code up here, you can see everything's on a new line. But because we're writing our interface in HTML, you need to have those HTML codes for the new line. Very, very basic HTML. If you've worked with HTML before, you understand. This is very simple. I just wanted to point that out in case someone has never worked with HTML. OK, so that's a basic little script. We have a lot of echo lines there. Let's go ahead and look at another script that I called hello2.sh. And again, the extension.sh means absolutely nothing to the computer. Just as it means nothing when you're working in the shell, it means nothing on the web server. The extension means nothing. The most important thing is, of course, to have your shebang line, which is this first line here saying this is a bash script. If you don't have that, it's going to try to use the default shell, which is going to be a problem in some cases with bash scripts. If bash is not your default, but if you're using other programming languages like Python, which will get into a minute, that's definitely going to cost problems. Here's another example. I use this dash e here. So now I can put these new lines here. So this is the same original code that we have before. It's putting out this line of code. And then it's going line break, line break. So it has an empty line. And then it does our HTML code there. So let me go ahead, come up here. I'll hit two. So hello2.sh. And you can see that is the output of this file. So if you want to put all on one line, you can if it's something simple like that. Let's look at another example. Hello3.sh. And here you can see that I am using cat and EOF, although this doesn't have to be EOF. EOF stands for end of file. If you want to have more than one, you can label them different things. Basically what this is saying is everything between this EOF until there is a new line that has nothing but EOF on it. Print that out to the screen. So I'm doing the same thing here, the context type, content type, blank line, hello world. And then I'm also putting in a image here. So definitely if you're going to output a full HTML page, you don't want to echo out each line. You can cat it out like this. So here's an example. We can go three, hit Enter. And there we go. We have hello world and the image, which is the code down here. Now, things to be careful about. If I do this and try running this page again, you can see it outputs the text and doesn't output it as an HTML file because it doesn't realize it's an HTML file because the first line has to be this followed by a blank line for an HTML file. Otherwise, it thinks it's just a text file to display. And so if you put an extra blank line there like that or anything before that, it's going to not work right. So if you're going to be using the cat and a file, it might be a good idea to pretty much always start your files off like this. And that way, if we refresh, you can see it works. No matter what you change inside this main portion of the document, it's not going to screw that up. So that's another pointer for you. So we've worked with some bash scripts here. If I list out here in our CGI file, you can see that I have a couple of Python scripts. Again, the extension is for your knowledge, not the computer. It does not care what the extension is. It cares about the shebang line. And if I do within hello.py, you can see, I have a very basic I have my shebang line, which can be written in different ways. But as long as you have it and it works on your system, this is saying what type of environment to use, basically the interpreter, we're going to be using Python. And again, if you're using a different version of Python, all depends on how your server's set up. And again, same thing. I'm printing out the content type, a blank line, and then my code. So HTML code saying hello world as a header. So if I come in here and I change this to .py, hello.py, that's just again, because that's the name of the file, we'll hit enter, you can see it says hello world. Now I have another Python file in here called Python two, or hello two.py. Just to show you, similar to the cat end of file, we're using three quotation marks here and three quotation marks here. Everything between that is going to be outputted as text. So you don't have to do a different print line for each line of your HTML file, if you're going to be generating HTML file with your server side scripts. So again, I put my prints line for the content type and the blank line. These can be done on one line, but you want to kind of, I would advise doing that separate from the main portion of your webpage, just to prevent, again, if you accidentally put an extra line in there or an extra character or something, it won't run properly. So that's HTML, that's hello two.py, hit enter. And there you go, it's displaying an image, it says hello world. Now again, as I said, it does not matter whether it's a bash file, a Python script, a Perl script, and any type of script or any program that can run on your computer can be used as a server side script. And that's why you can use HTML as a GUI for any program. And I'm going to show you an example of that. If I go back into my main folder here. Oh, I also want to show you, if we go back into our main folder here, I have a Python script here that's executable. If I click that, you can see it displays it as text. It's not inside the CGI file, so it's going to run like that.  Now I also have, as you can see here, a hello world file or a C code, which is a very basic, has a main function, prints out the content type and a new line. You can see I have two backslash ends there, that's a new line and a blank line, and I can print F hello world. So what I can do, I can come down here, I can say GCC, that's the compiler I use, I can say hello dot C, and I can say my output is CGI bin hello. And again, the extension doesn't matter. I'm just going to say dot bin, just so I know that that's a binary file. So I just compiled that. And if I was to actually cat that out to bin, so I cut out that file, you can see it is a binary file. It is a C file, it is compiled. So it doesn't matter whether it's a script or if it's a compiled program, if it's inside the CGI folder and it has executable writes, it can run.  I can say dot exe, doesn't matter, it's on elix system, it's still compiled for Linux. And I'll hit enter and you can see it says hello from C because that's what the code was. And again, also the content type, dot html and that blank line, that's for the output. You don't even need that if you're just running the script. Let's say you're going to have some code that you can want to access through your web server and it does something on the server and you're not going to have any output. And that doesn't even really matter very much, although you pretty much always have some sort of output even if it's just script run complete or whatever. So I hope you enjoyed the tutorial. Please visit filmsbychrist.com. That's Chris with a K. There should be a link in the description. Again, this is using Python as a web server. Let's go ahead and look at that main script. Also you can see there was output here every time we access to file, just like when we ran servers before what files are being accessed. 200 means it was served up properly. And like if it's 404, you know it's not there. 403 is permissions issue, I think. That's what we were getting there. And again, the command is python-m and capital CGI-https-erver. So CGI-httpserver case sensitive and then what port you want to run it on. And when you run that whatever folder you're in, it will be running as that, as the root directory for your web server and all your scripts will go inside a CGI-bin folder. And also you may be asking yourself, let me go inside my CGI-bin folder. Here I'll create a new file. I'll call it who.... I don't even have to put it down. I can just say who again, the extension doesn't matter. And I'll say this is a bin-script and I'll just say uname-a. Save that. Again, I have to make it executable or it will not run. And you also want to consider permissions to write to stuff. You want to make sure that not everyone has permission to write to script files or even that folder because if someone somehow sees a flaw in one of your scripts to where it can redirect it and write a new file to your CGI-bin, well, it can create a new script that has the permissions as you're about to see. CGI-bin, again, we called that who with no extension hit enter. And oh, I forgot something. You can see there's no output even if I control you to look at the source code there because we forgot our content type. Remember if we cat out one of our hello.sh files, you can see, remember, don't forget this. I do that quite often. If you get no output, remember to have those lines. So we'll say content type, text.html, text slash html, then a blank line, and then our command, which can be anything we want. We'll hit F5 up here to refresh. And you can see that all the information about my server here and also actually that's about my server. I want to know about myself. So let me go back into my file here and I'll echo out a line break. And then I'll just say who am I? Save that, refresh it. And you can see it's running as MilX 1000. That's another thing to keep in mind that these scripts are running as whatever user you start the server as. So if you start the server as root, these scripts are gonna run as root. If you start them as a regular user, it's gonna have the permissions of that regular user, meaning those scripts can modify any scripts that user can run. So right now, if I write a script, it can modify anything in my home directory and the temp directory. Hopefully my regular user doesn't have permission outside of my home directory beside the temp directory. So keep that in mind. You may wanna create, especially, there's probably already a user on your system called, well, I don't know if it's by default. I know once you install something like Apache, it's there, but a www-data user that you may wanna start your scripts as that user, limiting their permissions. Unless maybe you're making a program that you want to be able to access and interface with certain user's files. Maybe you're making some sort of media center and you wanna be able to have permissions to start up mPlayer, because a regular web user shouldn't have permission to interface with your GUI. So all depending on what you're doing, just trying to bring that up to keep it in mind, security is important whenever you're doing server-side scripts, and especially, you know, you're opening this up to people on your network. So password protect stuff, be sure you have the right permissions on certain scripts and on certain folders. You don't want someone overwriting a script file with their own information that's going to be doing nasty stuff. That's just a little word of advice. Again, thank you for watching. Please visit FilmsByChris.com. That's Chris Decay. There should be a link in the description. I hope you're enjoying this series on servers and networks. And of course, next week we'll be continuing with a very basic look at another web server using Netcat, which you've probably seen before, but I'm gonna take it a step further than most people will show you. So I hope you look forward to that. And as always, I hope that you have a great day. Okay, this is an introduction to FilmsByChris.com. I'm Chris, that's Chris Decay. That's me right there. My daughter, Ember. And my wife, Jennifer. We pretty much live in the swamps of Florida. I'm a firefighter by day, as well as by night, we work long hours. But that's not why you're here. You're here about the videos I put up on YouTube. These videos are mainly about computers and programming, which means most of my videos look something like this. And if that's what you're interested in, great. If not, that's all right. I do videos on other topics too, such as video editing, special effects, photo editing, 3D design, and music creation.  So I ask that you take the time to go to my Patreon page and look at different levels of rewards you can receive for different levels of backing. There should be a link in the description of this video if you were watching it on YouTube. Otherwise, you can visit patreon.com forward slash metalx1000. And I thank you for your time and your support. Have a great day.