Good afternoon, my name is Fernando Acosta Rodriguez and I'm librarian for Latin American Studies, Iberian and Latino Studies at Princeton University Library. And I'm going to be the first presenter on this session about what is it called, democratizing, democratize, that we titled, democratizing ephemera at Princeton. Princeton began to collect and build an archive of Latin American ephemera and grade, and grade literature in the mid 1970s to document the activities of political and social organizations and movements, as well as the broader political, socioeconomic and cultural developments of the region. Today, the Latin American ephemera collection consists of tens of thousands, I think that four or five of flyers, pamphlets, leaflets, brochures, posters, bulletins and other formats mostly in print. No other research library collects these types of materials from Latin America with the depth and the scope that we do. Over the years, countless Princeton students have utilized the materials from the Latin American ephemera collection as primary sources for their research projects, for their senior thesis. Numerous researchers from all over the world, all over the U.S. and abroad, I shouldn't say all over the world, and many places abroad, including many from Latin America, have traveled to Princeton over the years to use the collections. That's one photo, anyone? Yeah, that's okay. Faculty bring their students to our special collections every semester to view and engage selections from the collections that relate to the courses that they teach. A few years ago, I also co-taught a seminar titled Politics and Social Change in Latin America 1968 to present that revolved completely around materials from the collection. Selected ephemeral materials have also been shown in three different on-campus exhibitions sponsored by the university's program in Latin American Studies and by the library, including one currently in place that is titled Records of Resistance. And it showcases prints and posters created by protesters taking to the streets of Santiago, Chile, as well as Lahore, Pakistan as my South Asian studies colleague, Ellen Ambrosone will tell you about soon. I mentioned all of this to emphasize the research and the teaching value of the collection and of ephemeral materials in general terms. For decades, access to the material was provided by slowly accumulating and organizing thematic sub-collections, creating long and detailed catalog records with numerous title, name, and corporate body entries, or by creating itemized finding aids. And finally, by microfilming the collections that we had curated locally. Reproductions of the microfilm were commercially distributed and resulting royalties were used to fund new acquisitions. We were running almost running a small business selling microfilm in my section of the library. Approximately 350 thematic sub-collections were processed in such a manner over the years. A few representative titles to give you a sense of how the collections were assembled. Where human rights in Argentina, a collection that had two supplements, indigenous peoples, peasants, and ethnic minorities in Bolivia, 1970 to 2005, gay and lesbian issues in Latin America, 1963 to 2001. That's another collection that had a supplement. Environment and ecology in Brazil, 1975 to 2001, and so on. Dozens of libraries, mostly though not exclusively located in the U.S., acquired copies of the microfilm over the years and made it available through interlibrary loan to nonholding libraries. The model just described gradually became unsustainable and microfilming, we stopped microfilming in 2008 completely. The commitment to building, the commitment to building the collection however, continued interrupted and we ended up in only a few years with a growing backlog of an estimated 12,000 mostly unique ephemeral primary sources that for all practical sources were remained inaccessible. We knew that we needed to transition to digital access and to new ways of preserving the original materials under content, but of course these presented numerous challenges. The possible approaches that we considered range from A, giving our ephemera to vendors for digitization and access via commercial distribution, sort of like as we did with the microfilm, to B, completely local production and digitization for free open access distribution with alternative C, various hybrid configurations and possibilities. In the end, we fortunately chose to develop a completely new system for building digital collections locally with free open access for users and communities everywhere. Obtaining the institutional, the administrative support required for doing it was possible because of the proven research and teaching value of the Latin American ephemera collection. It was also possible because the need to find a viable approach for managing such a large and in many ways difficult collection offered the library the opportunity to conceive and develop a scalable and sustainable model that could also be used to handle other collections. In other words, the Latin American ephemera collection would serve or served as a pilot for both project-specific and other broader library needs. That's a photo, well, it's okay. That was just a lot of unprocessed. A few thousand items of unprocessed ephemera in the library. To be able to implement a new processing and access model, the library locally built a new user-friendly and flexible, at least from the point of view of a collection development librarian like myself, repository for managing digital content. This digital repository has evolved over time. In its most recent iteration, we call it FIGI. And I'm going to refer to some of its basic features in the next few minutes, at least those that I have used and know better. And my colleagues will elaborate during their presentations. However, what I do do not want to continue without mentioning that the repository and the public interface where users can access content from the Latin American ephemera collection, it's called the digital archive of Latin American and Caribbean ephemera, was created in large part thanks to a substantial three-year hidden collections seed grant received in 2013 from the Council on Library and Information Resources, CLEAR. A complementary grant from the Latin Americanist research resources project, LARP, funded the digitization of the material during the initial years. The original CLEAR funded project included developing a streamlined data entry system for the creation of item level metadata from the ephemera or from its digital surrogates. A system for ingesting the metadata into a database architecture that would link to the images and the exposure of the digitized ephemera and corresponding metadata through an advanced discovery interface. Once all of the tools were in place, we were able to abandon the previously used archival organizational model where incoming ephemera was sorted and arranged into thematic groupings and subsequently cataloged at the collection level and to replace it with the creation of item level descriptive metadata for every piece of ephemera. Switching to item level description allowed us to eliminate the need to dedicate staff time to sorting physical items and most importantly allowed us to get any selected content online rapidly without having to wait for enough materials on a given topic to be accumulated before initiating the processing. A weight which in the previous collection level model would normally take years. This was quite valuable because it made it possible for example to quickly make available materials needed for a course or materials related to important current issues or political developments of high interest. Regarding a term that I used earlier, sustainability, I want to elaborate a little bit. It was essential of course for the new system to be sustainable in terms of ongoing IT development and support. It was also important to be sustainable in terms of staffing needed for physical processing, description, metadata creation and digitization. In other words, another way of putting it is the new system model would have to be viable without the possibility of adding new staff in those areas. How was this accomplished? For the creation of item level description, simple data entry tools were designed that require minimal technical expertise or training support for staff. They include controlled vocabulary menus that were developed specifically for the Latin American ephemera collection and to a certain extent other collections of this type. And you'll hear more about this when my colleague Ellen talks about this matter. Subject description with the Latin American ephemera collection is provided by first selecting one or more of 19 possible broad thematic categories such as politics and government, arts and culture, human and civil rights, gender and sexuality, education, health, minorities, ethnic and racial groups and so on. And then within each one of these broad categories, whoever is processing can choose from approximately 15 to over 30 available terms that are all based on Library of Congress subject headings. Here, you can see under, what is it, under what? I can't, you can see under the category gender. Read from here. Under, you can see. Oh, gender and sexuality, politics and government. Yeah, you can see many of the LCS sub-divisions that are available to us under the category gender and sexuality. Menus are also available for geographic sub-divisions at the country level. There's no authority control for names of organizations or corporate bodies. If keywords or a better description or more granularity are required, there's a field, it's an open field that is available for that purpose. All of this, what it does is that it allows us to efficiently describe each piece of ephemera ensuring that appropriate standards and quality control are attained without the need for dedicated professional catalogers. To date, the digital archive of Latin American ephemera and Caribbean ephemera contains more than 21,000 fully described items and grows at an average rate of approximately 300 items per month. In a perfect world, we would have longer and more detailed subject lists and authority controls, but for the Latin American ephemera collection, which seeks to rapidly incorporate a very large and continuously growing volume of items, the existing control vocabulary and vocabularies work well. Again, Ellen will soon be discussing this aspect further as it applies to another collection with its own characteristics and needs. Returning to the sustainability requirement, I also want to briefly comment on the digitization, the actual of the ephemera. Because we do not have the capacity to continuously digitize such a high volume of ephemera in our digital studio, we outsource the digitization of the vast majority of the items to an external vendor with the exception of posters of other oversized items and of three-dimensional objects, which are all digitized in-house. The vendor has been able to offer below market, really low rates for the service because Springston is able to guarantee a constant and indefinite supply of materials. The South Asian ephemera project that Ellen oversees has benefit, which is not as large in volume, has benefited from this as other future projects or ongoing projects that are just getting started and future ones may do so too. In some, the current processing model and workflow is vastly more efficient than the previous one in many respects. First, as I described, large amounts of ephemeral materials are processed and made available online much more rapidly. Second, online discovery tools allow us to make them available to an exponentially larger number of users than was possible before, including users in the countries and localities where the materials were originally created. And third, users no longer have to conduct long archival fishing expeditions to access the material. They can now use a very user-friendly interface to browse, search, and sort materials according to their own criteria and selected categories. Now that we're able to get ephemeral content online quickly and sustainably with a proven workflow that includes the ability to efficiently create metadata, Princeton librarians responsible for developing other subject and geographic areas have found it a lot easier and in fact are in courage to build their own ephemeral collections or similar ones. Benefiting from the fissures offered by FIGI and by the experience gained over the years with the Latin American ephemeral collection, they're able to easily reproduce many of its fissures and crucially also help to create new ones which in turn benefits other colleagues developing new digital collections and of course benefits users of ephemeral materials everywhere. I will end by mentioning that the experience that we now have in processing ephemeral materials combined with the fissures offered by FIGI have opened up the possibility of initiating new cooperative projects by making our repository available to other institutions that may have significant ephemeral collections of any extension but not the capacity to make them available, at least not in the way that we do. We currently, for example, have an agreement with the Library of Congress in the office in Rio de Janeiro, Brazil that has allowed us to add hundreds of pieces of fascinating ephemera that they have collected in recent years. In that agreement, LC-RIO does the collecting and we handle the cataloging, the description, and the digitization. We are in conversation with other university libraries in the U.S. that are also interested in cooperating with us and building a collective, building a collective ephemeral collection. The idea is that they will be fully responsible for metadata creation directly into FIGI. The next step, I hope, will be to collaborate with partners in Latin America to build the collection together. Ellen, both Ellen and Kim, have quite a lot to share with all of us about these possibilities. So I'm going to now let them start their presentations. Hi, everyone. My name is Ellen Ambrosoni and I'm the South Asian Studies Librarian at Princeton University. And today, I want to share with you a bit about our growing collection of South Asian ephemera and some of the adjustments that we made to the associated workflows for item level description in order to accommodate the needs of the collection and to encourage collaboration with external partners. The South Asian ephemeral collection walks in the footsteps of the Latin American ephemeral collection at Princeton, or LAE, as we call it. PUL launched SAE in March of this year. The collection contains digitized booklets, pamphlets, leaflets, flyers, posters, and other forms of ephemera from and about South Asia on a variety of subjects and in multiple languages. The collection is a free and open resource that is available to anyone with an internet connection and it is being added to on an ongoing basis. When I joined Princeton in 2019 and began approaching physical processing and description for SAE, I inherited the workflows and metadata schemata that had been put in place for LAE. Yet in the course of working with the materials from South Asia, it became clear that SAE would pose new challenges that would require support from IT to make this material more easily discoverable. The central challenges that we've been thinking about the last couple of years include adding and or adapting fields to accommodate original language scripts and facilitate discovery, creating ways for external collaborators to use FIGI and contribute to our collections, and data ingest and data extraction for internal and third party use. One of the biggest hurdles that we have with materials from South Asia is simply the variety of languages that we work with in the collection in order to represent South Asia in all its diversity. No one person can know all the languages in the collection. So very early on, we needed to find a way to make sure that materials in various languages would be processed and made discoverable. We decided to trial outsourcing metadata creation to a trusted vendor in India who now regularly creates metadata for physical items via digital image. To do so, the IT team created the necessary permissions and enhancements so that our vendor can now log in to work directly in FIGI. This, of course, required a few conversations to sort out how to do this securely so that we could limit their access to a subset of items, but now they can log in as needed to produce the metadata from the digital image. Similarly, we recently collaborated with the American Institute of Lankan Studies, or IELS, in Colombo, Sri Lanka to make a collection of materials produced by radical activists from the 1960s to 1990s available digitally. Our collaborators produced Google Sheets with the related metadata, and the IT team worked to create a way to automatically ingest the metadata into FIGI rather than having to manually enter the data for almost 600 items. As with lots of these projects, it took quite a bit of trial and error, and we learned a lot along the way. But we now have a way to meet collaborators where they are, whether that means working directly in FIGI or ingesting spreadsheets. And IT has worked on a way to verify metadata before ingest to ensure that it will populate correctly in FIGI, thereby reducing the need for so much data entry by hand. Relatedly, I want to mention another feature that IT created that has been an absolute game changer for the South Asian materials, and that is the ability to extract the metadata from the collection into a CSV. This feature was originally developed so that we could share the data from the Sri Lanka collection with the South Asia Open Archives, which is an open access repository of materials from South Asia. They wanted to federate discovery of the collection in their interface, which added yet another valuable layer of discovery to the collection. IT created this functionality, which I have since used, not only to share the metadata with a third party like SAWA, but also for metadata cleanup projects for the rest of the collection. This has been an unintended use for the feature, but also an incredibly valuable one, as we clean up batches of metadata before items go live. Both the Princeton-owned items and our contributed collection from South Asia have required us to think creatively about feature enhancements that allow users to search more intuitively for items from South Asia. For example, items in South Asian scripts are described both by using the original script, original language script, for three fields, the title, the creator, and the publisher, and also in Roman script. Unfortunately, the Library of Congress Romanization Tables are often difficult for non-librarians to internalize and use when they're trying to find items in the catalog or within our digital collections. To try to mitigate this difficulty, I asked IT if they could create a keyword field where we could enter more intuitive ways of romanizing South Asian languages. And here I wanna show you an example of what I mean. So one of the serials that's in the Sri Lanka collection is called Pravahini, and it's produced by the Women's Education and Resource Center in Sri Lanka. And by design, the work, the NGO, wanted to make sure that the publication was available in three different languages, in Sinhala, in Tamil, and in English. And so, but the romanization, as you can see for the Tamil, is sort of counterintuitive. It looks like it says pittavakini instead of the way that it's actually pronounced, which is pravahini. And so if a user, thinking of the way that it's pronounced, searches for pravahini, they will only discover the Sinhala and the English, and they would have missed the Tamil unless we added pravahini to the keyword field itself. So this is a really amazing example of the sort of vision of the NGO and the vision of the curator and the digital library interface actually being in alignment in terms of what the curator was trying to achieve in recognizing that it was published in all three languages. The elastic keyword field has had additional unintended benefits. Outside of mitigating script-related challenges, outside of mitigating script-related challenges. As I mentioned, when I began processing this collection, I inherited not only the workflows from LAE, but also the controlled vocabulary for describing the collection. While most of these subjects can be applied to South Asian materials, there are moments when an LAE subject is not appropriate for use in some South Asian contexts. And here I have in mind, especially items related to indigeneity. So the control vocabulary that we inherited from LAE is indigenous peoples, but oftentimes in say an Indian context, the word tribal is used more often than indigenous peoples. Some communities are using terms like adivasi, some don't accept that term. And so the keyword field allows us to sort of diversely describe an item rather than being married to just one piece of controlled vocabulary. The keyword field liberates the person doing description from their restrictions that we inherited, while also allowing us to retain the controlled vocabulary that should our collections ever be combined would still allow a researcher to find similar items across geographies. Moreover, for contributed collections, our international collaborators have the opportunity to populate the keyword field as they feel appropriate to describe their own collection and in a way that makes sense to them. In the case of the collection from Sri Lanka, our collaborators entered keywords in English when they couldn't find an appropriate subject heading to describe an item. This feature not only gives collaborators more agency over how they describe their own items, but it also serves as a diagnostic that we can audit from time to time to see if we need to add a subject heading to our list of controlled vocabulary. Lastly, from a collection development perspective, the ability to develop figgy in accordance with the needs of the collection has also fostered creativity with regard to collecting born digital ephemera, particularly around protest movements in South Asia. At present, the South Asian ephemera collection includes two born digital sub-collections, one on feminist activism in Pakistan and another on citizenship in India. Recognizing that social activism and practices of dissent are taking place more and more in a digital realm, expanding the file types that can be ingested into figgy has meant that I've been able to collect born digital activist art that captures how individuals are harnessing the power of the internet to advocate for change. A selection of posters from the Orat March or the Women's March in Lahore, Pakistan was recently displayed alongside some of the physical ephemera in Fernando's Latin American ephemera collection for an exhibit at PUL that focused on continuity and change in practices of resistance across different geographies. Without a real investment in the necessary resources to develop the capabilities in figgy, we would not be able to collect this type of material and it would be relegated to the instability of the internet. Instead, we're able to lean into our creativity to provide learning and research opportunities for folks at Princeton and elsewhere. I want to close my part of the talk by saying that I see these feature enhancements for the South Asian ephemera collection as more than just facilitating discovery and access. They are also practical interventions in a library infrastructure that has traditionally centered the needs of the West. More and more libraries and librarians are being called on to examine the ways in which their work may either be complicit in oppression or alternatively in solidarity with the communities represented in their collections. This touches on all of our library practices from the vendors that we purchase from, to the content that we select, to the terminology that we use to describe the items within. The practical interventions that I've just detailed are small moves toward applying insights gleaned from scholarship on decolonization, indigeneity and post-colonial studies to the life cycle of a digital library project. Indeed, recent scholarship by historian Ashil Mbembe encourages us to work collaboratively to co-constitute the future, one in which the West is not the center of the world and one that embraces our inextricable webs of affiliation or even webs of dependency. If we take this call seriously, then we can consider how such modifications to the library infrastructure help to create the conditions for the collaborative construction of knowledge, in this case, in the form of a digital library. So now I'm gonna hand it off to my colleague Kim Lieman to talk a bit more about the technical aspects of this work. Thank you, Ellen. Good afternoon. My name is Kim Lieman and I am an IT project manager at Princeton University Library. I focus primarily on our digital projects and initiatives and help supervise all aspects of our digital project implementation. Now this includes working with key stakeholders developing ephemeral projects like Ellen and Fernando and coordinating with our administration and partners in other departments such as the digital imaging studio, conservation, microforms, cataloging and metadata services and of course special collections. I also serve as a product owner for software development projects undertaken by the digital library services team including our digital repository more commonly known as FIGI which uses the Samvara framework and is built on Valkyrie and digital PUL or D-Poll which is our instance of spotlight in open source extension of blacklight used to create curated collections. Obviously a big part of our library strategy is collaborating with open source communities like Samvara and Blacklight in order to leverage already existing projects enabling us to focus our development efforts on Princeton specific needs. It also allows us to have a larger impact when we share out our contributions with the broader community. Now before moving forward I also believe it's important for me to stress that any successes within my job and indeed these ephemera initiatives are determined by the administrative support, strong departmental connections and collaborations I am a part of within the library and beyond. This model does not work without investment and by that I mean both time and money from all levels and units of the library that it depends on. Now this is not new but it's still worth stating. In my role as a project manager I have found that in many cases when one hears digital collection or digital initiative there tends to be a focus partially if not entirely on the digitization of the physical object. On its surface these processes may seem straightforward either digitize items in house or hand over digital files in some way to IT to ingest or request external access to a selection of repository materials in need of metadata enhancement. Colleagues then ingest the content into FIGI or make specific repository projects available to an external party and then it happily becomes available in one or more of our library's wonderful applications. What is not always apparent but what I hope has been brought to light today is the amount of cross departmental collaborative work it takes both before and after ingesting contributions into any digital repository and the time it takes to create sustainable ways to streamline future requests for similar content. Digital initiatives like the ephemera projects Ellen and Fernando have highlighted today are in fact about everything from the selection, conservation and prioritization of the materials to the metadata created or enhanced by contributors wherever they may be in the world. Value added enhancements like multilingual OCR automated pagination or foliation, logical structure. So think table of contents in your viewers so you can navigate that digital surrogate better cross discoverability in our applications built and are implemented at the library and of course the functionality and continued enhancements of those applications. This all contributes to our digital content in order to make more effective research tools and a better user experience for the growing number of those needing to work with or contribute materials to our applications. Now I'm not naive, I am fully aware of the fact that creating item level metadata for incredibly diverse types of archival collections and enhancing digital surrogates with things like logical structure takes time, language or subject expertise and of course people. For us, thanks to the needs expressed by our stakeholders and the strong support received from our institution for ephemeral projects at PUL, we were empowered to ask ourselves, what if we could develop sustainable and potentially scalable ways in other applications to create item level metadata for posters, leaflets and other types of ephemeral materials and what if we could broaden the scope of those who can create it. Now this is important when thinking about the flexibility and the adaptability of workflows for digitized content. In the past prefiggy days, the amount of time it took from digitized image to available to the public was a bit demoralizing for more than one department including our digital studio. Training was difficult, specialized knowledge was mandatory and Injust was therefore limited to only a handful of internal staff. Now this was not sustainable nor was it scalable. As Fernanda mentioned earlier, in 2013 PUL received support from CLEAR to develop a new model to efficiently create item level metadata for a vast collection of digitized ephemeral content from Latin America. There were two expected outcomes of that three year project that included the development and deployment of a scalable, sustainable and replicable model that could be used for similar ephemeral collections from around the world and ensuring potential for new collaborative initiatives for shared access. Thanks to those foundational use cases and expectations presented by Latin American ephemeral stakeholders and the initial and continued support received from CLEAR and our library administration, we now have FIGI. Injust and item level management can now be executed by a variety of FIGI users internationally and publishing online can be immediate syncing with our digital exhibits and discovery platforms like LAE and DPOL. Okay, so how does it work? Thank you. So just a quick recap, excuse me. FIGI is Princeton University Library's staff-facing repository management application. As our main tool for digital content management, FIGI allows digital files to be easily ingested and completed by a variety of staff from local staging servers so from our digital studio and Firestone Library, for example, or external hard drives from vendors and external collaborators. Okay, so we have our digital images. What about the metadata? Princeton's metadata, excuse me, Princeton's ephemera projects work slightly different than that of standard ingest of content in our digital repository that are typically synced with their associated metadata from ALMA, which is our new ILS, or finding aids by their unique bibliographic identifiers or component IDs. Excuse me. Ephemera items are described directly in FIGI and have some of the most advanced forms of metadata entry in the repository. I'm losing my voice. Excuse me. All right, let's try this again. Including support for controlled vocabularies management, which is managed in attitude by our ephemera stakeholders in collaboration with our colleagues from cataloging and metadata services. In most use cases, the item level metadata may be generated by a variety of internal and external contributors with the subject knowledge or language specialization necessary to responsibly describe the items that have indigitized. Once an item is marked as complete in FIGI, this is the part of the story where it happily becomes discoverable in one or more of our library's wonderful applications thanks to IIF. So you've heard the three of us mentioned collaborators and contributors more than once today. If we want to meet our collaborators internal or external wherever they are, it is imperative that we remain strategically and operationally agile as we develop and adapt our services and resources. And yes, I did take that directly from our North Star statements, which I have shared with you here. These are our new guiding principles that were adopted in place of the strategic plan during the pandemic. Something is really getting to me. Sorry, folks. And we are still referring to them today. Now I mentioned earlier the need for strong departmental connections to help ensure successes with projects such as these. Although we had well-established LAE foundations to build from, it was important that we find a way to organize and develop an emerging community of practice at PUL. Speaking personally, I've tried to connect colleagues and support the concept of a community of ephemera stakeholders at PUL. I have the privilege of hearing about digitization proposals or ideas relatively early on in the process of my institution. I have found that connecting a representative from LAE, along with stakeholders that may have helpful input for a subject specialist developing a new ephemera project is incredibly useful to someone that is new to our ephemera processes. An added benefit. An added benefit includes creating an active network of communication between librarians working with ephemeral collections. And this is something that recently came into play with Ellen in our Near East Studies colleague, Deborah Schlein, who is in the process of gathering born digital ephemera alongside her collections assistant. What is also developed rather organically is the realization of a need for a new workflow for subject additions to the controlled vocabularies. As ephemera projects continue to grow, we've also had to create opportunities for different projects to select favorites that they may be more relevant to their work than others. Going forward, we've developed a plan for ephemera stakeholders to review and discuss controlled vocabularies, subject vocabularies in particular, in partnership so that we can collectively determine additions to the list. This partnership has also been invaluable to me as a product owner for our applications. The combined voices of our ephemera stakeholders and their group's feedback continuously help us in IT prioritize tickets for feature enhancements in FIGI, LAE, and DPOL. While the need for internal collaboration is great, we also wanted to ensure our model would provide potential for a new collaborative initiatives for shared access across continents as well. User-driven feature work for FIGI by members of IT and collaboration with project leads within the library helped create workflows and application enhancements for ephemeral content from external contributors into our repository. A great example of this would be the repository enhancements and workflows that were created to support the work Ellen and her, of Ellen and her international collaborators. Ellen's project needs required the joint collaborative work of IT, our digital imaging studio. Ellen is the library's primary stakeholder and subject specialist and the external international collaborators contributing to the digital South Asian and Sri Lankan ephemeral projects that are either hosted or held by Princeton University Library. Thanks to these use cases, batch ingestion of metadata and images and CSV download of all ephemeral project metadata into or out of FIGI is now possible for current and future external ephemeral project collaborations. We knew we wanted to take the time to create workflows that would handle bulk ingestion in this way for all ephemeral projects when needed. The multi-step process worked on a large part by our colleague Cliff Wolfman, took a great deal of IT and stakeholder time, particularly for review and standardization of submitted metadata and the mapping of controlled vocabularies. In addition to this option, because of enhancements later made to FIGI and DPOL, a separate workflow was created to responsibly give project specific access to external contributors in Indians Sri Lanka for example, who are now able to contribute, work with and enhance digital content directly in both applications. All right, so why the multiple workflows when one obviously sounds a lot easier? Although working directly within our platforms is an ideal option in most instances, establishing alternative workflows that are not dependent on, for example, reliable access to stable internet connections or to allow flexibility if needing to accommodate external metadata creation work already underway, like something Fernando is currently coordinating with another institution. We're important for us to ensure. There are and always will be edge cases that for a variety of reasons may complicate or hinder potential future collaborators' ability to contribute if we are unwilling to be agile in our own developing best practices and continuously building on what we learn from our users and our stakeholders. In my opinion, an ideal way to meet our internal and external collaborators wherever they are is through digital collections and initiatives like that of our ephemeral projects. From the foundations of LAE, we have been able to responsibly build out the collections in partnership to include the collaborative South Asian and Sri Lankan ephemera you've heard about today, Slavic ephemeral projects like print ephemera from the Ukraine crisis and late Soviet era posters, our newly launched LGBTQIA plus pilot whose contents will be shared out with the digital transgender archive and a forthcoming born digital collection that builds off the enhancements and emerging born digital workflows Ellen touched on today, highlighting the current protests in Iran. All of these examples are incredibly important to me and my institution and of course the applications that make it possible in the first place. Continuing to prioritize user driven enhancements and adaptive workflows like those presented today means we'll be able to support future submissions of ephemeral content from all areas and we'll provide the sustainable foundations for best practices for this type of digital content in our repository. To date, we're proud to have been able to respond to our wonderful stakeholders like Ellen and Fernando who asked, can you put this in figgy? Can this work? By saying yes, we can do that and working together to find new sustainable ways forward to do so. We look forward to continuing to collaborate with our colleagues and contributors and to support the growing and diversifying repository needs of the Princeton University community. Thank you. Questions. Hi, thank you. Oh, Jiminy. Jen Stringer, UC Berkeley. I love the guiding principles that you put up previously. If you wanna go to the slide, you can. I thought they were great, I actually took a picture. I was wondering about accessibility though. This is something that I'm just continuing to think about in terms of the usability of our materials with people who have issues accessing our collections in a variety of ways. And so I'm wondering as you were sort of laying out how important making access was sort of from the perspective of systemic racism, the perspective of DEI, I was wondering if the JA part justice and accessibility sort of fits in there somewhere and how you check to make sure that your collections and that your tools are accessible to everyone? As far as accessibility goes, I think it's important to state that, well, as an example of sending out our metadata to SOA in order to so that it could be federated as well as being able to share out our information as well as links to the digital transgender archive. I think it's important to state that anybody who would be able to support a copy of our collection is something that we're very interested in always being able to do. As far as accessibility goes, we try to OCR everything that we can in the different languages that we use. Zinhala was actually our first one and our Sri Lankan collaborators were wonderful. They did a test to see how well it searched because I'm illiterate in such things. And yeah, so one of the things that sort of immediately pops to mind when you say this is just how many folks in South Asia rely on their mobile actually to access these resources versus using an actual computer screen. And so of course it's totally possible to look at these items in mobile format. But I think this work is really ongoing and there are feature enhancements that I think about all the time and imagine as I continue to work on the collection things like even should we start considering image descriptions for each item in terms of the folks who are visually impaired. We just recently got the ability to provide content warnings. So maybe should we do a review of some of that content see if there are things that we need to signal to folks or should we circle back to the folks in Sri Lanka and ask them if they'd like to review their content and make those decisions themselves. And for me, I'm very interested in user feedback for my collection. So that's sort of at the top of my priority list just getting a sense of whether folks have other suggestions for how we could describe items that mean something to them. So these kinds of issues are at the forefront of my mind and thank you for the question. Hi, I'm Derek Devnich from Yousumer said. I have a question more generally about FIGI. So I think this is for Kim. So I'm a technical support person. So the world is full of like various open source components that one could use to build an archival system and you see people making different choices about what to use and then sort of suffering different consequences from that. So I'm just curious if you could give sort of your impression of how the design decisions, like what made you decide to use the system that you went and you decided on and how you decided to make it and like how you sort of feel about the choices retrospectively now that you've worked with those components for a while, like what would you make different choices now? Just look generally what was your experience. Sure, so I am but a humble project manager but I am happy to actually feel a little bit of that question. So FIGI obviously is part of the Sembera community and we factor very strongly in that community. Actually is any of my, is John Stroop in the, he is. Do you want to field that question since that was part of your decision? Yeah, I think it's fair to say. I mean, this work did start almost 10 years ago and the Sembera community is sort of where was the hydro community then and that's where we found ourselves. And sort of the modularity of the components, right? I mean that started as one very small project for sort of getting stuff into Fedora but quickly evolved into a community of practice around different approaches to ingest for example, as they discussed. I don't think we would make a different decision now because the repository architecture we already had in place was sort of highly distributed so they didn't really state it outright but we sort of have a core repository function that pushes out to a lot of different discovery platforms rather than building sort of one new yet another place to find stuff, right? So I think that was the right choice. I mean, I won't deny that the cost of ownership for a big sort of open source repository is high or can be high. There's a lot of not completely bespoke but highly configured or customized software here that certainly something we do continue to take into account but for the kind of workflow we have here, at least with cultural heritage material, it's allowed us to build a really fast moving workflow. So I absent something else out there that by all means we may not know about. I think this was a good choice and we'd probably do this again. Hi. Sorry, everybody's looking in that direction so I didn't mean to get your attention. I was waiting over here. My name's Nathan Gerrotham, Head of Digital Services for the University of Nevada Reno Libraries and was drawn to this presentation in part because of our pretty robust, BASC language protest material collections that we've been working on over the past couple of years but I'm not gonna ask about that. I'm actually gonna kind of riff a little bit on your title and ask about democratization of the processes because that's what occurred to me in the course of your presentation and it's less about just access to materials but it's about shifting the power around in terms of who gets to be involved. And in particular Ellen when you were talking about that, this really occurred to me. So what was it like to get experts on board from all those other units that you listed in the last slide because it involves seeding some power, right? It involves stepping back and saying, okay, we're gonna trust other people to be involved in describing these materials and curating the materials. What does that look like from your perspective as people who are involved in this? Yeah, I mean, I think it requires a lot of self-examination and what I've found in creating these digital projects is that there's no decision like too small or too large like as you're going through them. And just I think reflecting on the fact that like we can't recover a pre-colonial past, right? That's what Ashila Mbembe is talking about. There's no going back in time. There's only going forward in a fractured world and that we have to sort of commit to co-constituting that future. And that means like oftentimes actually getting out of the way and finding ways to sort of lean into the expertise and the knowledge of others. And with the Sri Lanka collection it was just a joy to do that because the curator already had such a strong vision. And so it was sort of about like, okay, where's the framework that we have that we kind of need in place? But then where can we be flexible and where can we relax? And the keyword field was totally amazing in that regard. But also, they had the pandemic actually in that particular case worked in their favor because originally we were just going to ingest everything from the Sri Lanka collection into SAE. Just subsume it and it would be like a sub-collection. It would have its own landing page. But when the pandemic hit, they were actually ready to launch before we were. So we ended up, I ended up working with Kim a lot to sort of build out their own webpage and they had agency over the narrative for that. They created all the text throughout the webpage. And we really like leaned into making decisions collaboratively. Even though I can hop into that at any time and make adjustments, if there were questions, it was always sort of like open communication and what do you think about this? And should we add this logo there and these kinds of things? And I think once you kind of like, sorry, once the coin drops, so to say, you realize you just sort of lean into those collaborative practices more organically. I'll add that in terms of democratization, I think that it operates at very different levels. Internally, the decisions are not made in a centralized way, right? We don't have anybody who says, no, that is how it's gonna be done. We negotiate it and we learn from each other in the process and I think it's a team effort. I mean, of course, it's limited by the technology that's available and the staff that is available, but it's negotiated between curators and support staff. Also support staff that help us with the meta data creation and other aspects of the project have a big voice in the project and in the training process, they really do. In terms of, I mentioned earlier on something about creating the possibility or the vision of having a collective Latin American ephemera collection, one that it's not just Princeton's collection, but it's to put that our infrastructure becomes the right or one of the tools that allows us to make ephemera from not only from US based libraries, but from many other parts of the world, all available in one piece. That's a direction in which I would love to get to take the project that addresses the democratization issue. It's not that easy for different reasons, but one of them being symbolic, that right now the digital archive very much has a Princeton identity, right? And other libraries, if they're going to contribute, they want to get credit too. So we need to develop it in a way that every institution gets the deserved credit. One recent example or recent experience that it's kind of related. We were asked by a library in Peru by a documentation center interested in the collection that we have documenting the decade of the shining path, guerrillas in Peru and the repression that followed. They wanted all of those materials and we shared them with them and they created their own metadata in a way that was much more detailed and a much higher level that we could ever, ever have done. And now we're probably gonna, we may take that back and put it in LAE. They will show it locally in their own site in Lima, Peru and we'll probably present it in our own site.  I think it's very obvious worth mentioning that the project, I think in terms of democratization, there was a big change in that we would collect these materials, we would extract, right? Thousands and thousands of these items and keep them in Princeton and you could only see them in Princeton or in the microfilm. Sometimes we donated, in some cases we donated microfilm sets, but now maybe it's not the perfect solution but we're making easy, accessible in a relatively easy way things that for the most part you cannot find in the countries of origin. And I think that's a significant contribution and that perhaps could be understood in terms of democratization. I'm very close to time but I'll throw this out. I'm Elizabeth List, work with the South Asia Open Archives at the Center for Research Libraries. I just wanna say that I've really enjoyed how much all of your presentations focused on the sort of constant reevaluation of the workflow and the process and getting input and getting feedback and then changing the way that you were doing things and bringing others into the project along with you. I wanted to ask a couple questions about your workflow. First, this sort of digitize and then describe workflow for those of us who are kind of engaging in that type of approach to description and metadata. What lessons have you learned from that workflow and what sort of advice would you give to those of us who are doing projects and might want to move in that direction? And then also just thinking about the collections more broadly, what have you learned about the collections, what's in them and what purpose they serve from the way that people have accessed them as digital collections as opposed to analog collections or microfilm collections, what sort of new understanding do you have of what the collection is from this new way of accessing it? I mean, I think we all have something to say about this. Okay, I'll... Yes, I'm sorry. Regarding the first question, what we learned is that it's almost all of the time. It's much easier, better to describe from the original piece and not from the digital reproduction unless you're dealing with one image, poster or flyer, but then it's basically the same. But if it's something that has 10, 12, 15 pages that you have to go back and forth to understand what you're looking at, then it's a lot better to describe from the object, from the original. But the system gives us the flexibility to do it either way. Yeah, and during the pandemic, it was wonderful. We were able to describe thousands of posters that were waiting for us to do that. You want to? So I'll just pop in here for a public service announcement. Have an MVP for the minimal metadata you're going to have with that digitized object, whatever you establish for your institution. I would say that would be paramount because having that unique identifier that's going to go with your digital image that's also going to go with your metadata is paramount. Something that was, it's not necessarily unexpected but incredibly rich for the potential future of ephemera is the strength of the controlled vocabularies that's being used across the entire ephemera collection. This is not something that you would normally get, say, if you're looking in a catalog record for that one ephemeral item. You'd be able to search a controlled vocabulary subject search across all of the ephemera collections, not just from a geographic area. And I think that that's a real strength of having a digital corpus of material and an application to work with. Just one thing on the workflow, like metadata from the image, I would just say fill in as many fields as you can before it goes, at least for the South Asian stuff. There are things like dimensions that we want to include but if you send it out and then it comes back a couple months later, then you have to pull out the physical item again, then you're counting the pagination and doing the dimensions. Like anything that's low hanging fruit, if that can be done before it goes, that's been better for us. Yeah, regarding the other question, it's hard to say, right? I don't think that I have a complete sense of how the collection is used. I'm sure that what I know is very limited. However, we are more and more frequent, more each time more getting requests from institutions across Latin America to send them the original high resolution images for publications and for different purposes. So we know that there is a large audience and a large audience that values the content. I'm not sure I can give you a fully formulated answer yet either especially since the South Asian materials are so newly online. But if Twitter's any indication, it is really interesting to sort of see when it catches on and what's mentioned when it does catch on. And scholars who are really like amazing religious historians of Islam and things like that, mentioning the booklets on Islam in the same breath that they're talking about, say, the Orat March material and see both as valuable has been really rich and it's been an incredible teaching resource. It's now when I step into any undergraduate course that's doing Islam in South Asia or there's so much to pull out from the collection. So that's been really exciting. And I guess the only other thing I would mention would be having conversations with scholars who have expertise in a part of South Asia that I do not, helps me think about how to shape the collection in really productive ways. So South Asia is always an exercise in sort of leaning into what you don't know. You're always growing in that way. And so that's been a joy to sit down with people who identify things in the collection that you can't see yourself. Thank you.


Good morning everyone. I hope all of you had a very good breakfast and are charged up for the first full talk at Antel today. So my name is Kiran. I work on medical imaging at a company called Predebel Health. And my talk is here going to be on unsuppoised and semi-suppoised deep learning techniques for medical imaging. So before I dive deep into medical imaging, I'd like to give you a brief history and introduction to cancer. So how many of you here know at least one person who has suffered from cancer in the past? It's pretty much everyone I guess. So cancer is the most deadliest disease known to human history at this point. So I have this interesting statistic. So this graph here shows the number of women who have died of cancer in the US from 1930 to 2005. As you can see, there's hardly been any progress made in the number of people who have died. Like, in respect of how many technological advances we have made in the past 100 years, cancer research is still stagnant. So there are a couple of trends here. If you look at this gray line, which is the lung and bronchus cancer, it has actually has been on the rise after 1970, ever since 1970. That is because of the introduction of cigarette smoking. And there's one more interesting trend. If you look at the blue line, which indicates stomach and colon cancer also, it has shown a considerable decline. Can anyone guess what was this major medical innovation which happened in the past 100 years, which resulted in stomach cancer from being completely eradicated from human history? It was the fridge. Yeah. So this picture here is a perfect depiction of how poorly humans have understood cancer. So let's dive a little deeper into cancer. So what is cancer and how does cancer occur? So we all are human beings and we are made of trillions of cells. Each cell has a genetic code, which is like a source code, which instructs how the cell should function. For example, how should the cell divide and how should it should convert food into energy and how it should prevent us from contracting diseases from bacteria, etc. So what happens in cancerous cell is that the genetic code, the instruction that instructs how the cell should divide is corrupt, which results in uncontrollable cell division, which is what you see as a cancerous mass inside the bodies of cancer patients. So how you would treat cancer is that either you would remove this cancerous mass using surgery or you would deliver toxic drugs of chemotherapy, which essentially try to kill off the cancer cells.  And radiation therapy is where you deliver targeted doses of high concentrated X-ray, which try to kill the cancer cells. But if you have a lump of mass and if you want to know if it is cancer or not, the ideal scenario is to take a biopsy of the tissue and analyze it in a pathological slide and then determine whether it is cancer or not.  It has adverse side effects because it can probably impair other brain functions. And this is where medical imaging comes into the picture. Medical imaging allows us to see inside the body. So the picture here shows an MRI scanner. And what happens in an MRI scanner is as the patient goes inside the scanner, this cylindrical structure takes a 3D image of the interiors of the body. And when you visualize this 3D image in your computer screen, you see 2D slice sections from multiple directions. How you would view from the front, from the top and from the side. So how do we use this for spotting brain cancer? So the picture on the left shows the brain scan of a normal brain. The picture of the right shows the MRI scan of a cancerous patient. So this particular cancer is called glioblastoma.   So it's a very worrying statistic. Why is that? So if you were to take a special MRI sequence of this cancer patient, this is what you would get. You can actually see that the cancer like an earthquake has an epicenter and has grown and occupied the entire left hemisphere. So different sequences of MRI show different phenotypes of the cancer. Also during treatment, you need to know how heterogeneous the cancer is. Your cancer is not just a single cell. It's extremely heterogeneous and it doesn't respond to treatment well. So in order to analyze your treatment response, you need to know and map the different regions of the cancer within the brain. So this picture here shows the entire extent of the cancer which is called the edema. And this picture here shows the necrotic core which is the epicenter of the cancer. And this sequence shows two different regions from within the cancer. One is the active tumor which is actively growing. And the central dark region you see is the necrotic cells which are dead cancer cells. And if you were to map all these information together and create a map out of it, this is what you would get. So you have four labels. Each pixel in your MRI has to be labeled as brain or not, as tumor or not. If it is a tumor, what kind of tumor it belongs to? Is it edema or the necrotic core or the active tumor or the non-enhancing tumor? And when a patient is diagnosed with brain cancer, the doctor prescribes the course of treatment. After a few months of treatment, the patient comes for a re-scan and an MRI is taken. And the doctor analyzes how these different regions of the cancer is responding to treatment, and that is extremely crucial for the survival of the patient. So if I were to pose this problem as a machine learning problem, we need to generate labels for every pixel. And this pixel-wise labeling is carried out by doctors, and it is extremely tedious, and it takes about two to four hours of the doctor's time. Once the pixel-wise labeling is done, the doctor visualizes this cancer in a 3D virtual environment and then understands the heterogeneity of the cancer and then plans the treatment accordingly. So this is where we hypothesize that labeling can be done by deep neural networks. But there are multiple problems with glioblastoma segmentation from MRI. It is extremely non-trivial because of the following reasons. We have a shortage of samples. The dataset that was available to us was just 300 scans. And only 2% of the pixels contain tumor. Apart from this, the tumors are extremely heterogeneous. So you have to understand the entire mapping of cancer from just 300 patients. And there's the complexity of the data as well. You are no longer dealing with 2D images, but we are dealing with 3D images and four sequences of these 3D images. So it's essentially a 4D tensor. You have to map the pixels of this 4D tensor to a 3D tensor of labels. And the dense annotations are extremely expensive. So a couple of weeks ago, I had a guy from a neurosurgeon from Silicon Valley who mailed me saying that he's willing to sell his dense labeled annotations for around $2,500 per scan. So that roughly equates to one and a half lakh rupees here. And multiply that with 300 scans, which is definitely not scalable for machine learning problems. So this got us thinking, can we leverage unsupervised learning? You don't have to depend on label data. Can we use unsupervised learning somehow to learn the underlying structure of these brain scans and come up with better diagnosis and segmentations for these problems? So this brings me to the first part of the talk. I'd like to start with deep unsupervised feature extraction. So how do you leverage deep learning? Or how do you train deep networks in an unsupervised manner? So autoencoders are extremely useful for feature extraction. A lot of autoencoders have been used for feature extraction in many deep learning problems. And an autoencoder is the most simplest MLP that you can ever think of. It is just a three-layer neural network, and it has two components. It has an encoder and a decoder. Both the encoder and decoder are simple matrix multiplications followed by a linear activation or sigmodal activation. And the purpose of the autoencoder is nothing but just to reconstruct its input. You feed the input, you ask the autoencoder to reconstruct the input. That's as simple as it gets. Now if you take the MNIST problem, it has a 28 cross 28 patch, and you would rasterize this patch into a one-dimensional array, you would feed to the autoencoder, and you would ask the autoencoder to reconstruct the input. And the loss function is just the mean square of the autoencoder's output and the original input that was provided to the network. But there's a problem here. The autoencoder is susceptible to identity mapping. It learns one-to-one mapping rather than learning the underlying structure. So how do we prevent this? We have a variant called a denoising autoencoder where instead of feeding in the raw input, we feed a noise version of the input and ask the autoencoder to reconstruct the original output, original input. So this ensures that the network is trying to somehow manage and learn the underlying structure rather than doing one-to-one mapping. So how does it exactly learn the underlying structure? Take this example of understanding the curve, learning the curve, and if you were to train an autoencoder for learning this curve, you would take sample patches, sample points from this curve, you would corrupt it, and you would ask the autoencoder to reconstruct the original input. And as you feed more and more samples, the autoencoder is forced to learn the underlying structure and essentially learns the reconstruction function of this curve. And it learns these weights, learns this underlying structure and encodes it into weights. So for the MNIST problem, if you were to visualize the weights and learnt by the autoencoder, this is what it would look like. It has learnt a lot of strokes over here, which are characteristic of the dataset, which is MNIST. So as you can clearly see, I did not use any label data at all. Just with unlabeled data, just by corrupting the input and asking the autoencoder to reconstruct the input, the network has learnt the underlying structure of the data. Okay, so this works for MNIST, but how do we convert this into brain MRI? So we have the dataset over here, which is four sequences of 3D MRI. What we do is, in order to alleviate the data imbalance, we extract ROAs around the tumour and sample patches, small patches from within this ROA. We extract four cross 21 cross 21 patches, four corresponds to the four sequences and we extract 2D 21 cross 21 patches and we stack them on top of each other and you have the image on the right, which is a set of patches which are extracted from this ROA of the brain MRI. So to summarize, you have these four sequences. Now you extract the patch, you noise this patch, you drop around 20% of the pixels and you ask the simple autoencoder to reconstruct the original input. And the mantra for training this is as simple as you extract the patch, you noise it and you reconstruct. You extract noise and reconstruct. You do this until convergence is achieved. But as a typical deep learning researcher, you are not happy with just a single layer autoencoder. We always want to go deeper. Jokes apart, deep layers are extremely useful for learning hierarchy of features. As lower layers learn local features and as you go deeper and deeper, it learns a hierarchy of global features, which are essential for understanding the image. But there's a problem with training deep autoencoders. Because of this reconstruction function, because of the mean square error loss where you would try to reconstruct the original input, it suffers from vanishing and exploding gradients. So this is where you pre-train the autoencoder layer by layer. What I mean by layer by layer is that I take the first autoencoder, I extract patches, noise it and reconstruct it. Once the first autoencoder is trained, I take the encoding middle layer and use it to initialize a new second autoencoder. Now this second autoencoder is training. So when the second autoencoder is trained, I take the middle layer of this second autoencoder and use it to initialize the third autoencoder. And this process goes on until the number of layers I want to achieve. So once these autoencoders are trained, I decouple these encoders and decoders from these autoencoder and I put them together to stack called as a stack denoising autoencoder. So the picture on the right shows a stack deep stack denoising autoencoder. So how do we use this for feature extraction? You take the encoder out alone from this deep stack denoising autoencoder and you feed the patch. During inference, in order to extract features, we don't noise the pixels. We feed it as is and we extract the features out and the learnt representation is a 1D vector which corresponds to the number of neurons present in the encoding layer in the final autoencoder. So this brings me to the second part. Now pre-training is done. We have trained our autoencoders layer by layer and we have prepared a stack of denoising autoencoders and we use it as a feature extractor. How do we go from here to classification? So we have the encoders over here and we have the input patch and we add a logistic layer on top of this encoder and we fine tune it for classification. It's just as simple as that. It's just a multi-layer perceptron of the layers corresponding to the number of autoencoders used plus one layer for the logistic classification layer. And this setup is called as a stack denoising autoencoder.  How do we go about it? We have four sequences of MRI. We extract the patch 4 cross 21 cross 21 patch. We rasterize it and we feed it to this MLP and ask the MLP to classify one among the five classes belonging to brain or what type of tumor it belongs to. Now, I have classified this patch, but how do I start accumulating predictions? So when I extracted this patch I noted the central pixel of this patch and I classified and assigned the class that the MLP says to this central pixel. So the mantra for fine tuning is you extract a patch and you classify you stride by one pixel classify the central pixel, stride by one pixel extract the patch classify and so on. So it's a patch-wise classification network. And the network that we used is a five layer autoencoder with three hidden layers 3,500 neurons in the first layer 2,000 neurons in the second and 1,000 in the third and five neurons corresponding to the number of classes in the problem. So here are how the segmentation results look like for us. The picture on the left is the ground truth. The picture on the right is the classification done by this MLP. This is the second case and you have the third case over here and you have the fourth case. You can see that a very simple MLP a very simple five layer MLP pre-trained with denoising autoencoders has achieved very good excellent segmentation results when compared to the ground truth. So let's analyze the matrix of this semi-supervised learning architecture where you pre-train first on unlabeled data and fine-tune unlabeled data. So the matrix for analysis is called as a DICE metric which is the intersection followed by A plus B and your DICE score is 1 if your segmentation matches the ground truth. So this is our model which is a five layer stack denoising autoencoder a very simple MLP with five layers and this is the state of the art which is a 11 layer deep 3D convolutional fully convolutional and fully supervised network and multi pathway network and the performance of this state of the art the deep medic is that it was trained on 220 scans in a fully supervised manner and had DICE scores of the following for whole tumor, tumor core and the active tumor whereas our network which is a simple file layer MLP which was pre-trained on 135 scans and fine-tuned on 135 scans achieved better performance on tumor core and active tumor than the fully supervised network. It's just a file layer MLP when compared to 11 layer deep and 3D convolutional fully convolutional fully supervised network which is where the current trend in deep learning is going, I presume. So in order to strongly evaluate the semi-super-based setting, we went aggressive on the second experiment. So instead of fine-tuning on 135 scans, we pre-trained our network on 135 scans and fine-tuned it on just 20 scans just 20 scans annotated data and our active tumor was still better than the state of the arts performance. So from here I'd like to propose the next hypothesis which we tried. So far we had this two-step training process where we pre-trained the network in an unsupervised manner and fine-tuned the network in a supervised manner. So can we just do classification in an unsupervised manner with just no label data at all? So to test this hypothesis, we took the denoising auto-encoder we trained it with patches extracted only from normal brain scans we showed this denoising auto-encoder just normal brain scans and during testing we plot the error map of reconstruction. When we plot the error map of reconstruction we see an interesting characteristic here. You can see that the network has started to spike on the regions where the cancer has occurred in the brain scan. So if you were to think of this auto-encoder is like a kid or like a human so we it is detecting abnormalities in a scan just because it was trained by showing just normal scans. Just by virtue of having seen only normal scans the auto-encoder is showing spikes in cancerous regions and we can use the reconstruction function to very beautifully create a heat map of this cancer. And the architecture we chose was just a simple three layer auto-encoder nothing fancy about it and we had a whole tumor dice of 0.8 with just unlabeled data no labeling at all. We had an accuracy of 0.8 when we binarized this error map to create a segmentation map out of it. So how good is this novelty detection? How good is this denoising auto-encoder as a novelty detection? So in order to test this we trained the novelty detector on brain tumor segmentation data set by not showing it any brain tumors at all. Now during testing we switched to a different data set for stroke relations also patients undergo MRI scans and we wanted to detect stroke we wanted to see if the network can detect stroke from the stroke data set. As you can see the first two pictures on the top and the bottom are the raw images the raw MRIs the third picture the picture in the middle is the heat map produced by the reconstruction error of this novelty detector and the green picture shows the binarized segmentation from this reconstruction error and the red is the ground truth. So to test this performance quantitatively we took the dice course again we evaluated it against the state of the art so the state of the art is again the deep medic which is a fully convolutional 3D convolutional network which was trained in a fully supervised manner. It had a dice score of 0.66 whereas our novelty detector which was trained in a completely unsupervised manner with no label data at all achieved a dice score of 0.64 which is as good as the state of the art considering that we did not train it with any label data at all and on a completely different data set. So we thought this novelty detector is really good so can we fuse this with the classification MLP that we trained in the first two steps. So the picture on the left shows the prediction done by the MLP as you can see it has some false positives over there and these false positives can be removed by using the novelty detector's mask and we multiply the mask and retain the connected component which is particular to this cancer and the post-processed image is almost as good as the ground truth and this work has been published and if anyone of you is interested you can follow the link on this slide and read up the paper if you want to know more details about the training methodologies used. Right now until now we had a decoupled training process right so we had a pre-training unsupervised pre-training step followed by a supervised classification step now we wanted to combine this together in one single step process and here we can use hybrid architectures which can train both which we can train both unsupervised and supervised networks in the same fashion. So we take the stack denoising auto encoder again it is the shape has changed but the essential functionality is the same you can see the green boxes correspond to the encoders and the yellow boxes correspond to the decoders and when you have unlabeled data you ask the network to reconstruct the output but when you have labeled data you not only ask the network to reconstruct the output but you also ask it to classify by adding an MLP at the bottom of the encoder and you modify your loss function accordingly you have a reconstruction loss plus alpha times classification loss where alpha is our tuning parameter depending on the amount of labeled data you have to the amount of unlabeled data additionally you would use skip connections to fuse features from earlier layers and aid in reconstruction or classification and this setup is called as a ladder network as you can see it has a ladder structure to it because of the skip connections between the encoder and decoder now if you want to go more into efficient training we can introduce we can replace our fully connected networks with fully convolutional networks we can replace the fully connected layers in our networks with convolutions and max pullings one cross one convolutions can be used instead of fully connected layers as one cross one convolutions result in highly efficient non-overlapping forward inference and backward inference so the encoders are replaced with a series of convolutions and max pulling and your decoders are replaced with deconvolution layers and you have skip connections which fuse features from earlier layers and you have a reconstruction map out additionally you can also use this network to train a segmentation map and you can add a MLP on the bottom of the encoders and use it to classify your rigorous and you would modify your cost function accordingly with reconstruction loss plus alpha times classification plus beta time segmentation again alpha and beta are tuning parameters which you can use depending on the ratios of the data available to you and where this might be useful is that the current brain tumor segmentation 2017 challenge wherein you are not only asked to segment the brain tumor but you are also asked to predict the prognosis of the patient how far does the patient have to survive how far the patient can survive you can use the regression over there to fuse both information from the segmentation networks and use that to aid in this regression so how I envision the future of data efficient learning is that we have all these medical imaging centers all over India we can deploy these unsupervised learning engines auto encoders also GANs because GANs also are trained in an unsupervised learning fashion and we can continually learn the underlying structure of the data as and when data comes into the scans and we don't have to wait for supervisors to come and annotate the data for you to train in a supervised fashion but when supervisors do become available you can clone models out of this unsupervised learning engine and label the data and fine-tune your network for the corresponding task that you are trying to solve be it brain cancer or breast cancer or liver cancer or any kind of medical imaging cancer problem you are trying to solve so I'd like to conclude by saying that I'm really hopeful that with the current deep learning revolution that we are going through and the genomic revolution that we are going through in the biotechnology and biosciences field that we will have technological advances which will bridge the gap between humans and enable us by providing us with weapons which can help us in our war against cancer and I would also like to acknowledge my colleagues at ITMATRA who helped me with the project I'd like to acknowledge Dr. Ganapathy Krishnamoorthy for guiding me in this project throughout the year and Dr. Keshav Das for providing the domain knowledge regarding brain tumor segmentation and Vurgis and Subramaniam are my partners in this project thank you we have time for questions yeah hi I'm Pranitha so I have a question regarding the loss function that you mentioned that you use reconstruction loss plus some parameter alpha times the classification loss so can you please talk about this like if your reconstruction loss is like sum of squared errors or something like that and what is the classification error that you are using and how do you your classification error would be a negative log likelihood like your mean square error would be a different scale and likelihood would be a different scale so yeah you can tune the alpha according to that yeah it seems one of the assumptions in your architecture is that you are taking a 4x21x21 slice so you are just considering only that slice and nothing surrounding it so if I took a normal brain scan and then shuffled all the slices then it would detect no anomaly at all so it doesn't depend on the arrangement of the slices at all so is there a way you can improve it perhaps to understand that this is the correct arrangement also something like that yes instead of extracting 21x21 patch you can probably extract 21x21x21 3d patch cross 4 yeah but what would be the cross 4 in that yeah 4 will be the number of sequences mra is a special sequence you don't have only one image but you have 3 3d images so the 4 is the 4 sequences the flare t1 t2 and t1 so basically your answer is that you will just have to increase the slice size okay thanks yeah it's just curious about your labelled samples like you told you have got few labelled samples so are they all positive or are they mixed positive and negative cases as well so we had around 300 brain scans so each brain scan has about 240 slices and each slice is about 240 again it's a 240x240x240 tensor and you will have around 2% of the pixels 2 pixels of this 240x240x tensor which is labelled as tumor and all of the 300 scans were annotated with had brain scans since for each patch you are predicting only the center pixel it would introduce noise so how are you looking into that it will not introduce noise so you are predicting each pixel based on the order encoders so it might happen that neighborhood which has tumors but one of the pixels might not be predicted correctly so there would be noise in your prediction that would be a peripheral noise but if you can explain further what that noise would be then I might be able to answer let's say I have a band of 5 patches which are any class let's say positive and my prediction says 4 of them are positive 1 of them is not let's say the center pixel of it but when I sort of give this to the doctors I need to give them the whole band 5 of it not 2 and 2 so how do you remove that one mistake there okay so those kind of mistakes can be removed by morphological operations you have dilation and you have dilation kernels dilation erosion kernels those are simple image processing algorithms which you can use to smoothen your predictions but that would further increase your error but the problem that you are trying to solve is not fundamental to the architecture it's not fundamental to the pixel level classification process that has been described here but that noise that you are describing is possible for any kind of segmentation algorithm that is available let's say if I am predicting a whole patch of it then the local neighborhood would come into picture yes so I will predict let's say a patch to be completely class 1 or class 0 but your convolutional neural networks apply convolution across kernels so they have a particular context at which they look into the image and classify whether that is in religion or not so that information is already taken into account by using convolutional neural networks and also as I said in the last few slides your fully connected layers can be implemented as convolutional layers by using one cross one convolution that is same any more questions? alright thank you


Russia's military wives and mothers protest against Putin. In Russia where public dissent is rare, a group of women's blunt message to Vladimir Putin stands out. Bring our men back from Ukraine. The Guardian reported this. We want a total demobilization. Civilians should not be engaged in the fighting, says one of the women at the start of the 9 minute address this month. There are many of us and our numbers will only grow. It is noted that they are the wives and mothers of some of the 300,000 Russian men who were conscripted in September 2022 at a critical period for the Kremlin when it needed to shore up its troop numbers after Ukraine recaptured swathes of territory in the south and north of the country. More than a year later, with their loved ones still on the battlefield, many women are staging public protests and writing open letters, taking to task the official narrative that mobilized troops are required in Russia's war in Ukraine. The Guardian says that Russia has a history of female-led protests during wartime, wives and mothers led an anti-war movement during the First Chechen War in 1994 that helped turn public opinion against the conflict and played a role in the Kremlin's decision to stop the fighting. The women were organized in well-run groups such as the Committee of Soldiers' Mothers of Russia, which had hundreds of regional hubs around the country and, crucially, their message was aired on Russian television at a time when media was not fully subordinate to the state. So far, the authorities have opted against jailing or harassing the women, instead ordering state media to ignore their pleas, while also rejecting their requests for rally permits across the country. Observers say it is too early to assess the impact of the movement of Russian women on a regime that has a long history of successfully quashing opposing voices. But their anger underlines some of the unease, some in the country feel about the conflict and puts a dent in the image portrayed by Putin of a society united behind the war effort.