Good morning everybody. I am really excited to be here today. This is my first DC-DC conference So I'm really excited As Jeff said, I am the director of arts technology and innovation at Arts Council and that is a new role And what my role really is about is thinking about how we use new technologies to create new types of content for our audiences So it's really exploring how do we extend the type of experiences that we give to our audiences using our content and using new digital technologies I sit alongside two other directors who are focused on innovation One who's who's focused on enterprise and business operations and another who is focused on audience and data culture And so the three of us really are kind of trying to put together an agenda that thinks about digital technology from a 360 point of view but what I thought I would do today is Since it's the first first session of today kind of get our minds open thinking out of the box one of the great Joys I have of Arts Councils being able to go across the country and see great artwork And so what I thought today is I would explore some of the challenges and issues around digital technology and making the digital shift through Artistic projects or contemporary art projects that seem to either raise issues about digital technology or try to address them in really creative ways so that's what I'm gonna do in the next couple minutes and I've got a couple videos to show you but I thought what I would do first is take you back to Washington, DC where I'm from This is Gaithersburg library public library in Maryland in the United States Right outside Washington DC. This is my library. I went to this library as a kid It still stands today and every time I have to come back and forth home to my parents house I pass by it and so I was actually in these states last week in San Diego and then Washington And so I was when I was thinking about, you know, what I was gonna say today I started thinking about what did that library mean to me? What was the significance in my life and how has that changed since you know, the kind of emergence of digital technology? And three things came to mind First, it was a sense that library gave me a sense of discovery that library kind of opened up the wider world to me Go going through the different stacks. I'm not I can't say I'm a prolific reader I'm not don't read everything, but I definitely look at a lot of book labels looking at the catalog really kind of shaped What is the outside world outside my neighborhood? And in my region so discovery was a big part of why I was so attracted to the library Second it was a form of empowerment in a lot of ways for a young kid It was a one place my parents would leave me by myself and I could kind of stay in the library by myself So I got my autonomy there. I had a you know library card I was able to borrow books if I wanted to but also I was very much a believer in knowledge equals power and You know, I you know, I would kind of you know use the library to You know write my reports for school and that type of thing and I always thought you know with all access to all this information I'm gonna write the best get the best grades write the best reports that kind of thing So it was a form of empowerment for me But it was also a place where I formed my ambition so I got to see what everybody was doing in the world I you know I've it helped me think about what are the types of careers out there outside of what my family was doing What are those kind of careers and also shape my ambition? I said one day I want to write a book That's in this library. So it was a way. I really shaped my my my ambition So fast forward 20 years and we've got you know emergence of digital technology and we've got now The ability to actually have infinite an infinite amount of information, right? So I thought the library world was big But obviously the digital world is big and it really expanded my horizons being able to access information online And obviously it gave a wider set of perspectives So what I thought was the whole world of perspectives in my local library actually Eliminated or did not include perspectives around a lot of minority issues People from marginalized communities, you know, we're not represented in my local library But online you could get those wider range of perspectives And also provided obviously new distribution channels new ways of Exchanging information and distributing information. You didn't have to become a formally published author in order to get your ideas out there So yes, digital technologies brought us lots of new opportunities And it's really shifted the way that Museums museums libraries and archives are positioned in what their kind of value in USP is But you know, there's obviously a lot of challenges associated with digital technology And so what I'm gonna do now is go through a couple of examples of projects that I've seen that really kind of highlight a Lot of issues we have in terms of making that digital shift Challenge one making sense of information. So we live in a world now where we're inundated with information So how do we make sense of it? The first project I wanted to highlight was a project by a woman named Anna Riddler. She's a contemporary artist and this one's called mosaic virus and what you're seeing here on this image is 10,000 images of tulips during the kind of growth season of tulips So she took all of these tulips all these images of tulips and then she by hand cataloged them And that was part of the presentation of her artwork But what was interesting about this is she took that catalog of tulip images and created videos that was actually The videos were would change and evolve based on an artificial intelligence Program that she created that linked The growth of a tulip and the emergence of the tulip to Bitcoin currency valuing and pricing And so what she was trying to do in this artwork is start to think how can we use kind of the economic theory around that? You know the tulip mania in 17th century Dutch culture that was kind of what we get what some people consider the first kind of economic bubble And how can we link that to the you know the economic bubble of Bitcoin currency? And what I think is really interesting about this is kind of using historic data using his using history drawing on history In order for us to make meaning out of something that's new a new technology that is emerging And I think that's something we should think about As professionals within with collections This second work is by an artist Refik Anadol. He's a Turkish artist and he really is very focused on the idea of using archive Information and again visualizing it. I think we're all now quite comfortable with the idea that of data visualization And that we have to do more to be able to articulate information in meaningful ways This work, which I'm going to show you as a video Works essentially takes meteorological surface data information from the black sea and reconceptualizes that That data as what he's calling a data sculpture of of the black sea So you get a sense of what that what it looks like what does he looks like but that is all formed through actual data So again, I think this idea of a digital sculpture Something about that, you know, can you feel can can can you feel the bounds of data? Can you in a way make meaning from it? Can it become solid instead of so ephemeral? The next project I wanted to talk about is one called resurrecting the sublime and this is really interesting Because we do talk a lot about and Jeff said, you know, we talk about how new immersive technologies can help us get a better sense of Information in the world around us. It kind of engages different senses This is a project that uses technology to engage our sense of smell. So I'm going to show this is a longer video It's about five five minutes, but I just wanted to get you give you a sense of this This project was one of the winners of the lumen prize, which is a digital art prize this year sudden slopes of Mount Haleakala the island of Maui, Hawaii The turn of the 20th century The forests are being lost to colonial cattle farming One tree will be lost forever The Hibiscus Delphus while the Rihanna The Hawaiians called it Maui Hau Kauaihevi the mountain Hibiscus By 1912 it will be extinct The habitat lost the plant lost the relationship between the two Lost could we ever regain a glimpse of what was lost a few years ago a small group of synthetic Biologists at Ginko Bioworks a bioengineering company in Boston set out to resurrect the smell of extinct flower Christina Agapakis Ginko's creative director went to the Harvard University Herbarium in 2016 Where over five million specimens are stored Searching the collections for extinct plants with her colleague Dawn Thompson they found 20 Two would hold enough information to unlock the sense of their flowers each lost due to human destruction of their habitat A third would reveal what may be an even greater loss The first specimen was in the Malvesi cabinet It was the Hibiscus Delphus wild Rihanna's from which she cut tiny tissue samples The second was the Orbexilum stipulatum or falls of the Ohio's scurvy Last seen in Kentucky on Rock Island in the Ohio River in 1881 Before a dam finally raises habitat in the 1920s The third was the Lucidendron ground of Florum the Weinberg cone bush It was last seen in London in a collector's garden in 1806 It's habitat on Weinberg Hill in the shadow of Table Mountain Cape Town already lost to colonial vineyards But this flower may prove to be completely lost The project is bringing to light that Harvard's specimen and possibly specimens in other collections may all be incorrectly labeled An old feud between two 19th century botanists exposed again Back at Ginkgo Bioworks, the scientists could begin working with the DNA Once it had been extracted from the plant tissue and sequenced by paleogeneticists at the University of California at Santa Cruz The Ginkgo team identified gene sequences that might encode fragrance-producing enzymes These sequences of DNA were printed and then they were inserted into yeast Which were cultured to produce the smell molecules The identity of those molecules was then verified using mass spectrometry Giving a list of the smell molecules that each flower may have produced Those lists were sent to Berlin to the smell research and artist Cissel Tollas Cissel began to reconstruct each flower's smell in her lab Using the same molecules from her collection or finding comparative ones to approximate those that aren't available We can use technology to reach back into the past giving us a glimpse of each flower, but we will never know their exact smell Science can tell us which molecules they made, but the amounts of each are also lost Building on that contingency in London, the artist Alexandra Daisy Ginsburg designed a series of immersive experiences to reconnect the lost flowers to their habitats These draw on the idea of the sublime, the sensation of the unknowable, and exposure to nature's immensity that makes us consider our position in it Using genetic engineering so we can once again experience a nature that we have destroyed is both romantic and perhaps terrifying It is sublime Artists tried to express this aesthetic state in 19th century landscape paintings But like these images Even the most advanced technology can only give an incomplete representation In a natural history museum nature's contingency is trapped in time the clock of creation and destruction stopped for us to look at In the installations each landscape is similarly reduced to its geology and the flower's smell The human connects the two and by stepping into this abstract nature they become the specimen on view The smells are diffused and fragments and mix in each installation so every inhalation is slightly different We can never fully experience the flower in the present without its past context This is not de-extinction, but a technological sublime Allowing us a glimpse of a lost flower blooming on a hill On a wild river bank Or on a volcanic slope The interplay of a species and a place that no longer exists As the philosopher Jean-Luc Nancy says The sublime is not so much what we're going back to as where we're coming from But the point is that I think the three examples I just showed just to wrap up that kind of section Is around you know, how can we help people make meaning of information? Is that our new role as museums and libraries and archives? How do we create ways of making data intersect with each other in order for us to be able to Discover new things is that sense of discovery around the intersection of data not just access to data Can we create new ways and and and help people visualize data and information in new ways? And how can we engage different senses? And how does that lead to new types of knowledge when we engage not just our reading, you know Our vision in terms of reading but in terms of site smell and sound So The next challenge is how do we kind of transform information into power? So, you know We now have lots of information at our fingertips But actually how do we help our audiences the people who come to us transform that information into power So that they can use it to change your communities And so I wanted to show you a couple of examples. This is a quick video by the Cleveland Museum of Art Where they this year launched an open access program And I think there's a lot of you know talk about open access And kind of open reserves But I think what's interesting about what cleveland museum of arts doing is they're really trying to push people To not just have access to this to the information and to their collections, but to use them in interesting ways So i'm going to try this video. This is only one minute Really thinking very hard about how they're going to make their open access Program one that's usable by different types of communities and different types of people Now the next example I wanted to show you was actually from the british library So I know you're having a keynote this afternoon from the chief librarian But this is a project called imaginary cities And the artist was micho takeo magruder and what he did was he used and leveraged the Map archive that they have at the british library selecting four or five cities in new york city paris Some major cities and what he tried to do is Use digital technologies to help imagine What those cities could be like or new forms of how that city could work So what he did is he put the digitized versions of these maps online and allowed people to interact with them Focus down on them tag them do do different things with with With the map and digital digital form As well as he mapped kind of the progress of those cities And with the intersection of those pieces of data He created 3d replication in terms of 3d images And immersive experience that would allow you to see what those cities might look like in their new form So I think what that project really does is help us think about okay How can we create again platforms and systems in which people can create the future based on the archive material and collections that we have Now for my next example, I'm going to switch gears into theater and performance This project is called justice syndicate and it was developed by a computational artist Theater professional named joe mccallister and what it does is is an immersive performance In which 12 people act as a jury and they have to make a decision as to whether Somebody is found guilty of a crime Now what's interesting about this is that everybody's given an ipad that has lots of evidence and information on it And they're asked and they're given information at particular periods of time during the performance and asked how it changes their judgment About whether the person should be convicted or not convicted There's an ai system that's driving this behind behind this that actually will give Each audience member a different piece of data based on information It's drawing from them of how they're interacting with the system whether they're making a slow decision whether or not You know the way they voted in certain certain situations I think what joe is is kind of kind of Set upon here in terms of this project is this idea is that we all are inundated with so much information How do we process that information and how do we interact and talk to each other about the information that we are given And I think we live in a world now where we're actually there's tons of misinformation People find it very difficult to navigate Through you know information that is in some published non-published Format and having different Difficulty having discussions about what they're reading And so I think that might be something that we consider as libraries museums and archives How do we help people digest information make decisions about it and have conversations about it? And finally, I think the third challenge I wanted to talk about is kind of How we support new forms of authorship? so As I said before, you know digital technologies allowed us to to be able to Have access to different perspectives and different views and one thing that I was really surprised about When I got to the arts council is the knowledge that many people who have been marginalized in the world of literature and poetry Have used instagram as a way of publishing their poetry But there still is this kind of divide between the real poets and the non real poets the ones who are able to publish Their poetry and ones that are just using instagram And I think a challenge to us is you know, how do we capture? How do we preserve? How do we? How do we capture the type of work that is being produced on the social media platforms and other types of platforms? How do we make sure that that information is accessible to future generations? Also, I mean machine learning is going to present lots of challenges for us In terms of how our collections are used as training data And how we protect copyright IP associated with information that is used in machine learning And I think a project that is being developed by Wayne McGregor Who is a contemporary dance artist kind of throws up a lot of challenges associated with machine learning What Wayne McGregor has done is taken his whole archive of dance videos That he's produced And he's trained a machine to be able to help him choreograph future dance works And so this is a very interesting Kind of project because it brings up a lot of issues around authorship So, I mean what will happen in terms of if Wayne decides to License his machine to somebody else to create dances in the style of Wayne McGregor What will happen what are what are the rights of those dancers in which he's collected an archive of their movements And now he's using it to produce new new work. What are we going to do about that type of thing? I think we all now have the possibility of creating having our own archives and be able to use machine learning to Reproduce ourselves in certain ways and other people will have that ability as well So how are we thinking about that? How are we going to manage that? How are we going to? How are we going to make training data that underlies machine learning accessible so we can understand if there's biases in it and And and to make sure that it's transparent the processes that are being used This is an example of where I think Machine learning can have some problems and where we have to continue to cultivate In our our practice Making sure the voices of marginalized communities get involved in the practice of artificial intelligence and museum and machine learning So this is a picture I took at an exhibition in Paris. It was an exhibition called black models And the whole point of the exhibition was to uncover the identities of black people who are represented in in an old master painting That we see in our in our Museums now I took this picture and I took a picture and there's a woman standing in front of the painting And I put it into the google cloud api around computer vision and as you can see here It identified the woman who's standing in front of the painting and Olympia But not the subject of the exhibition, which was the woman who's in the painting So we can see here while our kind of actions and trying to uncover And develop information about marginalized communities or people who haven't been represented In our museums and our collections can be totally kind of taken back by machine learning if we're not careful We're not guardians In terms of the way that machines are being trained and the data they're using and being very transparent about it So those were just kind of three kind of main issues. I wanted to address through artwork, but you know I'm still on the process of kind of formulating my agenda as head of digital Art technology and innovation. So I'm really interested to be here today and hear from you But I had a couple of closing thoughts about what is the kind of evolving role of museums Archives and libraries And I had a couple thoughts First of all, is our role to be kind of having moved from being people who provide information to being Information laboratories where people can have access to information But also process that information to make it meaningful for themselves Can we start to figure out ways and give tools that that to make information Intersect to create greater meaning and create and opportunities for discovery We all think about our spaces and we think about how to how to organize our physical spaces in a way that respond to community needs And the needs of our audiences What should we be platform innovators in terms of our online tools? So, you know kind of referencing back the Cleveland Museum of Art, you know, should we be partnering with organizations to create better platforms for For providing access to information as well as being able to manipulate them so you can create new works of art or new outputs based on online collections And finally, should we be places should we be sites for activism around data culture? It is the biggest one of the biggest issues that face our country In the world, you know about how we process data in terms of privacy how we're going to train machines What are the data sets used are their bias? Is are we a place in which we can actually help people become activist and be engaged in these issues and help in terms of making sure that all those The way data is used is transparent and that we have a way of understanding underlying data sets Those are just three ideas that I that I have but I'm really really happy to hear from you and have hear from your questions and actually hear what's going to happen during the The conference the next two days So with that I think that's it. I'll end. Thank you


All right, let's summarize all of our ways, or at least some of our ways, of doing division, and let's take a single division problem and see how many ways we can perform the problem. So here's 50 ways of dividing. So let's take the problem 495 divided by 15, and this time we'll show five different methods. So we know many different ways of solving this problem, so let's be a little bit creative here. And one thing we might start off with 15 is a number that we can factor. We can decompose it into two different numbers. So the divisor 15 is 5 times 3. So that means if I want to do the division 495 divided by 15, I could divide by 5 and then divide by 3. So 495 divided by 5, that's 99 divided by 3 is 33. I could also decompose the dividend. So this is 495, and I'm going to break this apart into a bunch of different pieces. And keeping this in mind, because I'm dividing it by 15, what I want to do is I want to break it into pieces that can be divided by 15. So, well, you could think about clock time. What are the 15 minute increments on the clock? 30, 45, 60, which is one hour. So one thing I might notice here is 495. Well, there's a 150 I can break off really easily. That's obviously divisible by 15 plus leftovers. I could break off another 150 and have leftovers again, and one more 150 and then I have 45 leftover. So now when I want to divide by 15, I'll divide each of these pieces by 15, 150 by 15, three times, and then my last one, and that's 10, 10, 10, and 3. And so my quotient is going to be 33. How about repeated subtraction? Remember, that's all the division really is. And so I'm going to take 495 and I'm going to subtract 15 as many times as I can manage to do so. And I might take a block of 10, 15s, 495 minus 150, and I can take another block. This time maybe I'll go for 20, 15s. I have 45 left. And now I can subtract out 3, 15s, and now I have zero. And altogether I subtracted 10, 20, 30, and 3, 33, 15s all together. How about an area model? Again, remember that any method of multiplication can double as a method of division. So let's go ahead and represent this division using an area model. And so that means I'm going to take an area. It's going to be 495. I'm going to make one side equal to 15 and then figure out what the other side has to be. So let's take a chunk. How about a chunk of size 10 that has area 150? Not quite enough. I'll take out another chunk of size 10, also area 150. So far what do we have? 150, 300. Still got area to account for. So I'll take out another chunk of size 10. That's 450. That leaves me 45 left over. And if this area is 45, this side is 15. How do you figure out what this side is? Well, you can do the division 45 by 15 if you remember your timetable. Or, again, we can view 15 and decompose the divisor into 5 and 3. So 45 divided by 5 is 9. 9 divided by 3 is 3. And there's our other length. And so my length of this side, 10, 20, 30, and 3 as my quotient. Again, still an area model. Well, depending on how much we know and how much we want to work with, we can make this a little bit more efficient. So let's take a look at a different area model. Same approach, just using a different version of the same model. So again, I have an area of 495. One side's going to be 15. And maybe I'll start out by taking a big chunk, 30 by 15. That has area 450. And then I have some amount left over, 45 again. And so, again, the question is, if I have an area 45 left over, one side is 15. What's the other side going to be? And I can figure that out. It's got to be 3 once again. And my quotient is the length of the other side, 30 and 3. Okay, so let's take a look at Long Division. Now, it's possible you may have gotten the impression from earlier videos in this sequence that I might not like the Long Division algorithm. Well, I have to say that if you believe that you're actually correct, the standard Long Division algorithm is actually a really badly conceived algorithm that exists for one purpose, which is to save paper. And if paper and ink were so expensive that saving paper was a worthy thing to do, then maybe the standard Long Division algorithm is worth remembering. But paper is cheap, ink is cheap. And so the rationale for the standard Long Division algorithm doesn't really exist. So let's go ahead and see how that might work in a more effective manner. So we'll go ahead and set up our quotient 495 divided by 15. And again, the standard Long Division algorithm assumes that I can make perfect guesses without hesitation every time. So I'll make a perfect guess. 15 into 495 goes around 30 times. And now 30 times 15 gets me 450. I'll subtract that off. And again, I can make a perfect guess without hesitation every time 15 into 45 goes 3 times. I'll write that up there. 3 times 15, 45 subtract and remainder is 0. And my quotient 30 and 3. And again, the standard Long Division algorithm that saves that really expensive paper and horrifically costly ink shifts the parcel quotients over so that they're lined up with the dividend. Again, no rational reason why this has to be. And, well, writing those zeros is so expensive and time-consuming that it's worth dropping them out. And then we collapse the partial quotient into a single line to save a whole one line of space. And there's your standard Long Division algorithm versus an alternative form, which is relying on partial quotients.


All right. Welcome everybody to our first, our submissions working group meeting for 2023. Normally Joe would be the one speaking here but I'm going to be filling in in terms of just running the agenda and kicking things off here. But certainly wish Joe a speedy recovery on what he's been going through and certainly if you're watching this show hopefully after the fact I do a halfway decent job but you always do excellent job of running these but First, before we get too much further we want to introduce our one of our newest attendees to this representing the art consortium. Luke Shan. So Luke, I just say a couple words about yourself if you don't mind. Yeah, well thank you Eric. Hi, my name is Luke if we haven't met before I'm the new program manager for the art consortium. I also work with the open JS Foundation. And, you know, I'm attending all the meetings in the short term but I'm sure I'll be participating and working with you all in different capacities along the way. Awesome. Well welcome aboard and yeah we're excited to have you and join this. Hopefully will be a very productive year if last year is any indication we had a lot of great progress and there's a lot of new ideas and new avenues to pursue this year as well. So I have a bit of I've been looking a little bit before the meeting about some of the audience we touched on in December that we can talk through some of these I think will be more quicker than others but maybe we'll kick things off with maybe an update on some of our attendees from the FDA side with our pilot three shiny up submission I know Paul and others when we last left off. It looks like it did go through successfully into the ECTD transfer, but we just want to do a quick check to make sure there weren't any issues on your side that we needed to talk about or any what's your progress on that. Pilot three or pilot two. I'm sorry pilot two. I was wondering if I was missing something there. Yes, yes. Numbers are hard in January. Yes, sorry for that. So now that's okay. No, actually we were able to follow the directions and implement the shiny app. I did it with both the initial directions and the console. I might suggest some more details kind of outlining those instructions a little bit more. Okay. But that's more of a implementation issue. Okay. So, and I think he sue had some suggestions as well or things to narrate we did. We're still trying to figure it out. I had some weird error messages, which if I ignored worked. Okay, so we're not sure if that's just a result of how things were being pulled from some of the sites at the time, or if that's something that needs to be addressed in the long term so we'll do a little bit more digging on our side. Yeah, very appreciate that feedback, Paul, and certainly I'm happy to consult with on the specifics of what you were seeing as well as, you know, as we think about future pilots in this space, especially they're going to involve shiny or some other, you know, fairly complicated infrastructure and certainly want to make sure that the instructions are clear and making sure they, you know, get you up and running the right way. So I'd be very happy to follow up offline or if you want to put some of your notes on the GitHub repository that we have for the submission. Either way is fine with me whatever would be best working for you. Okay, we appreciate that. So do you want to share some of your observations or experience. I mean, hi everyone. Happy New Year. Same thing, like I was able to follow the EDRG and then successfully launch the R Shiny application. There's no my like major issues. I found some minor issue like error messages and warning messages, but I'm trying multiple. I'm trying to launch the R Shiny app in different machine, like my personal, like scientific laptop and like remote machines, so that I can compare the result in different situation. There's lots of error message like inconsistently so I need more time to see what is happening and what's the, what's, why is it causing the issues. So, I just wanted to give you guys an update on progress that we were able to follow the EDRG and then launch the application. Excellent. Yeah, thanks for that feedback and I've always been. When we did the testing of this we really targeted the windows environment as a quote unquote local install as the primary method that it seemed like you all would be accessing the application but I'm very intrigued when you mentioned the different environments kind of those issues you saw, especially for future development so feel free to share those once you've had a chance to consolidate your findings. Sounds great. Thank you. Maybe we can give you more updates on February meeting. Okay, like offline. Yeah, yeah, happy to anyway like to share those will be happy to hear it. Awesome. Thank you. Great. So yeah, over in that feedback that anyone else have any comments or questions on how pilot two is gone. Since we last discussed. Okay, I'll take that as we can move right along to the real pilot three I should say, and I believe Joel you have sent around request for feedback on the proposal when you give our team an update here on the progress on your side. Yeah, thanks Eric for that for the time. Yeah, so just a quick update on pilot three. And so yeah the proposal was sent out, like early December, I believe, and I unfortunately I couldn't get it on the Arkansas to him get for some reason, my access there wasn't able to But it is it is in our pilot three gets so hopefully everyone does have access there to kind of take a review of our pilot the proposal, essentially the proposal again is just kind of to the FDA to ensure that when we package things up they're able to execute it. And so it kind of outlines all of the details there, pretty much that you know kind of the same approach that pilot one and pilot two took. So we're hoping to follow that and see if there's any other feedback any other alternatives or ways we can get on how to submit pilot three since this is more Adam data related. So if yeah if I am seeing some feedback from our pilot three team but haven't seen too much feedback from other other folks from the consortium so I'll pop it in here the link to that proposal in our chat window. And hopefully everyone can hope yeah hopefully Eric can pop that into the minutes and then yes, absolutely. So yeah looking forward to more feedback on that so then we can update that as as needed and if final then we can you know have a final approach on how we want to submit pilot three. But yeah so with that though, where pilot three is ongoing and progress is looking good. There are the five data set five Adam data sets that we're focusing on to help generate the TLGs for our own. Yeah, I think it was just a table and a graph actually from pilot one. So, having generated those and drafted those were now trying to get the pilot one scripts programs that generated those outputs to see if we can use our Adam source data for input into those those scripts to re output the tables and graph again. We are running into like minor issues with just ensuring that we are kind of using the same define and specifications from like the original seat is pilot data. And I believe there was an update after the seat is pilot data with the test data factory repo. And we're, I'm not too sure which one is truly the, the most up to date specs that we should follow for the Adam production here. So if anyone has any info on that, that'd be great otherwise that the TDF define is the one that we're using now because per their repo seems like they are kind of, they've updated since the seat is pilot repo. And as shared their data sets there. Um, so that's, that's one, two things. And then lastly, since we are, you know, this this consortium is kind of looking into new ways of submitting to the FDA. You know, Thomas is always coming up with other ideas as well and with this submission there. Thomas had just sent me a link this morning to a white paper, I guess that fuse 2017 there's a paper that says transport for the next generation. I know traditionally when we submit data sets to the FDA we, we put it, or we converted to XBT format. And with that I think from this paper it seems like their third thoughts on maybe newer ways to convert the data, other than XBT to submit to the FDA so And one of the ways Thomas was thinking was, I guess C disc is proposing a data set JSON file. And so I kind of actually wanted to know if anyone has any experience or has has seen the this proposal before. There's a wiki link to that. So maybe dip into your mind a little bit Paul to, I believe this, this fuse was in collaboration with the FDA as well. If, if we can also submit in this data set JSON file instead of XBT's. I had heard indirectly that JSON was being proposed. But at this stage, I don't know that it's gone anywhere. Okay. If we look at the, let me pull up the study. The technical conformance guide. Okay. Let's see. Hmm. That's annoying. They changed the link around. Let's see if I can pull it up a link. Let me put this in. Yikes. Let me not put that in. Yeah, Friday to 13 moments are coming true. Yeah. There's something called the study data technical conformance guide. You can Google my the Google results are yielding some strange. They're echoing it with see this. Let me just try a different browser. Just to see what's going on because usually we can pull it as an FDA link. Let me see if I can do that quickly. But the study data technical conformance guide spells out the latest specs. Is it this one? I see one. March 2022. There's actually, I think in October one. Okay. Usually is. And no worries, Paul. I mean, if this is marked, well, okay. Ooh, I have to find it somewhere when I'm showing up as the 2018 one and that's. Out of date. I'll try to see if I can find the most recent one, but the study data technical conformance guide is updated periodically. It's usually every six months. Usually March and October. So the that determines what the is acceptable to go through the gateway. And how should I say this? I have been at FDA for almost 15 years. Since I have arrived, everyone says XP T is obsolete and should be transitioned out. However, 15 years late, almost 15 years later. We still using XP T. All we can say is, if someone wants to. We can use the, I think it's version SAS version eight XP T. And they're not necessarily stuck with the SAS version five XP T, which has been the traditional one that's been used. In other words, you can use it, but it's not required. Understood. And are you guys seeing more version eight. I haven't read up on the differences between five and eight. I think eight will. Oh, here we go. Yes, do you want to post that HESU to the group as a whole. HESU is conveniently found the March. 2022 one. I found October. Oh, you found October. So it's slightly slide update. Oh, is this the right documents that you're looking for? Let me check. Yeah. I opened it on my computer. It does say October 2022. So Lisa does the more. Yeah. So I think so. Yes. That's the most recent one that tells you. All the essentially. There's a revision history, et cetera. But it does say what's actually required. And on page six of this guide. It does say. Version five. It's a file format.  Apparently they can't require. I see. Understood. Thank you Paul for the. The latest guidance then we'll. We'll try to stick with the requirements. But yeah, looking forward to see if we can, you know, maybe. Try some of the newer. The newer ways to transport. Right. I think it would be in the long term, it would be advisable. We can say. And hire. Pardon me. Oh, sorry. I just saw a chat from Doug. Okay. And hire. Okay. So. Yeah, so the systems. So far as I know, there, there have been several pilots attempt. To. Look at an alternatives to XP T. And so far. None of the pilots have been entirely satisfactory. So that. We're still kind of stuck with. The current situation with export files. Yeah, that's, yeah, that's good information to know. I, yeah, I think one of the things that I mean the limitations XP T now is like they. Was it the variable length names are like still stuck at eight labels at 40. And then the 200 limit character. So just wanted to see if. Whether.  I think the later versions. Expand those a little bit, but don't. I'm not 100% positive. So. Yeah, I. It is a. There is ongoing discussion. Let's just put it that way. I don't know that. Anyone has a complete solution that will cover all possible cases. So. Jason's 1 possible source. Essentially what many people do when they first get the data sets is they convert it to a easier to use format. Right on your end is saying when, when you, when you guys receive it. And you'll just convert it to something you're used to, I guess, so to speak, or easier to work with. Um, correct, even X. Even if you want to use it in SAS, you have to really first convert it to a standard SAS data set. True. So. That conversion has to occur. When could, when in some cases some, you can be converted to an R data set. And. I have seen it, although I don't necessarily recommend our CSV. Files. For data conversion. Um, the problem there is that people might try to open it said CSV files. And Excel. Um, and. Um, I noticed Eric is smiling. I've seen the pain that can happen in that, in that case. Yes. So, um, Given that that's the case. There are some issues of interest, but yeah. Appreciate your feedback Paul. Yeah. No worries. I mean, I, I think maybe. For a pilot three, we can, we can. Follow the. Uh, the technical guidance here that you've provided. Thank you for reminding us of that. So yeah, we could, you know, maybe. This could be a pilot four or five. Alternative transports house XBT. But yeah, thanks for just kind of having this discussion. I think it's helpful. Yeah. Right now, until we hear otherwise. We, we have to go with the findings or the guidance provided in the study data. Kind of says. Understood. Yeah, at least from my perspective, just observing this. I feel like we should keep a close eye on the JSON format progress as I think that will. Now again, this is Eric's opinion here. So feel free to challenge, but it is probably the best balance of modern. Yet. Accessible enough that in the different computing languages that we would use to consume these results, we would have fairly a fairly easy time of getting this into the rectangular format that we're used to when we actually look at the. Clinical data. So obviously our has many packages that go from JSON to our data frames. I'm sure SAS has routines that probably could handle it because of some of their modern tooling that they're, you know, marketing in their, in their product pipeline. That's just, again, speculation at this point. But I would hate to reinvent the wheel. If that effort does actually pan out. I definitely hear you're saying, Paul, I think there have been attempts at this. That have been mixed results at best. And, you know, maybe this is the more promising one that can at least be imported successfully in these packages or the software. And frankly, the modern software development world JSON is like the ubiquitous language to go back and forth between client and server side operations and the like, but that's a whole discussion topic for an everyday. This has been, I participated in that exporter or not exporter data set JSON hackathon that COSA the CDIS open source Alliance put on, maybe pilot for could be working with like Sam Hume and the COSA people to do a submission with data side JSON. As like a, or a continuation of pilot three to see if you could push push things along a little bit. Just as an, just as an idea for a pilot for pilot five. Yeah, I think it's not a bad idea. It's just, um, I don't know that the gateway is even set up to allow for JSON submissions. Hmm, that's some point. So, until the gateway is set up to enable those items. We might not want to go down that path. That would be a discussion with the E data team at FDA. And in the past, we've had Ethan Chen and some of his folks participate in this meeting. Yeah, so that could be an action for a future agenda topic to bring, yeah, maybe someone like from their team on exploring that as a possibility I do agree that definitely warrants probably its own pilot instead of trying to merge that with pilot three because I'm afraid that with the discussions involved and the change that will likely need to occur. We wouldn't want to slow down pilot three's progress for for that piece of it ends up being a very iterative thing that's just my opinion but Joe I don't know if you agree with that or if you want to think more about it. Yeah, that makes sense to me I mean, yeah, if Paul is saying the gateway is not open to it yet then, you know, we, it doesn't make sense to work on it at risk right now we can always extend it once we we have word of, you know, have it being, you know, receivable by the FDA. But yeah so yeah well you know once pilot three is done well we'll we'll follow XBT for now. But then, once once it opens up, we'll, we can reopen pilot three to extend it to then convert to Jason and then we submit again. I'm thinking that could probably be the easier instead of a whole new pilot or package. Yeah, that's a similar mindset as I have when I think about the container pilot whatever number that ends up being, we're just going to basically reuse the shiny app we did in pilot two and not try to reinvent too much at once just to isolate the key, the key points and the change so. Sounds good. All right, well thank you Joel for that great update and a great discussion I took a lot of notes for that for the minute so. Yeah, yeah, I'm trying to keep those best I can but as always when I send those out there any corrections it's all in the GitHub repo and you feel free to correct those. I'm glad you're recording it to me I forget. Yeah, you and me both yep. I'd like to move on for the sake of time. Doug had proposed that in this meeting we have a nice introduction to the repositories working group and some of the progress and efforts going on in that space so Doug I can turn it over to if you like to kick that off. Cool. Yeah, for anyone who's not aware we've had we've launched a new working group under the our validation hub that's looking at what a shared source for packages might look like. And so we just wanted to do kind of an intro session just to kind of introduce our team and what we're trying to accomplish. And I think that there's a lot of you know parallels between what our teams are trying to do so just to kind of set that groundwork so that if there are places where we can help each other out. We can kind of be in touch from the start. So I have a couple slides prepared they're pretty like bare bones, but I'll run through those and then we can just kind of have a discussion around, you know what what we might see as you know the next steps for this kind of thing. So let me see if I can share zoom hopefully this works. Can you can someone confirm the guys are seeing my slides. Looking good Doug. Okay cool. All right, so we have a few people on the call representing the team, I think I saw Kevin and Andrew also in the room. So yeah, as I mentioned this is a group starting to look at like what would it, what would it be, what would it, what would it look like to have a regulatory ready repository. And I'm going to kind of leave the repository word I want to harp on that too much because they're still kind of like open ended about exactly what what this product ends up looking like. And I'll get to that in a little bit. So here I just want to start with like what our mission is why we're why we're looking to this problem. And what exactly the problem is. So what we're hoping to do is support a transparent, open dynamic cross industry approach to establishing and maintaining a repository of our packages with accompanying evidence of their quality and assessment criteria. And so in a lot of ways like we have cram already which is like a pretty phenomenal resource that already kind of sets a bar for package quality. But as I'm sure like everyone on this call is aware we have a lot of internal companies specific process that then kind of get bundled on top of that to document this evidence and make it more reproducible and accessible. So you have to kind of like show that that level of quality. And if you've gone through this process in your own company, or if you can get just kind of imagine what that looks like internally, you know, at Acme co over here. They might be taking a look at, you know, the hundreds of packages that you might need to perform a clinical analysis, and everyone might decide that those all look great.  And that's really your only opportunity to get a really direct feedback on like what what whether this level of quality is sufficient. And so if there's like really major concerns about how you're performing analysis or the your choice of packages or things like that that's kind of, you can't get those more at the tail end of your process. And in general, I haven't heard of a lot of situations where that's like been a major concern, but you know it does kind of push a lot of this decision making toward the tail end, which kind of makes a really long feedback loop and leaves a lot of uncertainty on the table throughout the whole analysis process. So what we'd like to do is have this be a more like open dialogue so instead of having this like long delay before you get feedback. This is something where, first of all, all companies have, you know, transparent access to this shared kind of collective body of knowledge about what level of quality, we feel each of these packages has. And then on the regulator side we hope that this kind of opens up a forum for having a more kind of transparent dialogue about which packages. I think hold up to some level of quality or might have like methods that might have kind of like implementations that might not be up to our standards or something like that so that if we get that feedback earlier we can feed it back into like open source community and hopefully improve those packages to the point where people are happy with them. And like I mentioned this, this idea of a repository you might be thinking of something like Cran, but we're not, we're not exactly fixated on that idea. It could be something like Cran where we have some additional like quality assessment considerations baked into it.  We're not exactly sure what this is going to look like and that's largely why we're, we're here is to start thinking about how do we start answering the questions about what's really needed before we jump into a technical solution. So, just to kind of get cut to the chase for anyone representing the regulatory end of things. We want to understand how you observe quality and get a better idea of what we can do upfront to showcase that quality so that it's not a mystery to us whether or not that's going to be something that you're interested in and us using for analysis. And then if you're an industry participant, what we're looking for is support and helping to kind of like draft these things, or intermittent feedback if you just want to kind of look at what we're producing, understand whether that would work for you in your company or whether you just think that, given your experience with interactions with health authorities if that's something that would support that process. So, anywhere where we can like leverage feedback that's really where we're looking for the most input right now. And just kind of paint a little bit of a picture of what we're planning to deliver initially. Like I said we don't want to jump straight to implementing something on a technical scale. You know, invest a lot of time and effort into building something before you really understand what the need is. From the start, it could be something as simple as just like mapping out the various quality heuristics that we could possibly surface in something like a shared resource. So that we all have the same kind of like understanding of the level of quality of the software so if we were thinking if I was framing this in the context of a submission pilot. You know this pilot really just could be a survey of like, if we had this information about download counts or, you know, test coverage. How is that perceived and what else would you be looking for so just starting to have those those conversations would really help us make sure that we're heading off on the right path. From there, we were thinking having something like a mock up of a portal would help to kind of make it a little more tangible so it's not just a checklist but rather something where you're, you're kind of getting a little bit of an impression for what it would look like to inspect that for each package. And if we were then thinking into the further future, if we wanted to put together a pilot where we hope to host these packages in a way that could be consumed by health authority in one in a pilot. We could spin up something like a static package server. If that's a direction that we end up paying in so that you could pull packages from an endpoint similar to Cran but that's more controlled, just so that we could see what that kind of like technical feasibility would look like. So I think that's my last slide, pretty short, but hopefully that kind of gives you a little bit of an idea of where we're at right now and what our goals are. And yeah, like I said I really want to leave the majority of the time for discussion so if people think have any impressions on the path that we're on and think that we could be doing things differently or want to be involved we're very interested in hearing that. So, I can leave it there and open it up the floor. Doug, in terms of how the group is structured like how often are you meeting and what what's the cadence for, you know, the collaboration within your group right now. Yeah, yeah. So right now we meet about once every two to three weeks. It's a little ad hoc at the moment because we just kind of passed this holiday period and we just started off in November. So things are still like kind of settling themselves. We have people split up into three different groups where we have kind of like people that want to be more industry reps that are really here from like an ideation perspective just trying to make sure that we're hitting the right mark. We have people that want to be involved technically, you know, spin something up line on a technical solution and then we have people that are more organizing. So, just trying to coordinate all the different efforts so if anyone feels like passionate about any of those different paths then we can use your help. And certainly on the like health authority side, thinking about what you guys would be receptive to, to giving feedback on would also really help us kind of understand what should be delivering in this early part of this initiative. If there aren't any like, you know, if that didn't prompt any kind of initial impressions that we can always I have one. Okay, I was a chime in I don't want to dominate the conversation but Well, is there overlap or synergies with what's been done in the our validation hub would say the risk metric, you know, effort. Is that going to be like a component of this or how do you think that fits into play here. Yeah, totally. So, I've been biting my tongue trying not to like really bury the lead with risk metric but I think that could be a really fruitful resource for us to start surfacing some of these metrics. But there's, I mean, there's, there's these types of tools all over the our world for doing this kind of like quality assessment so I think on the risk metric side we try to be really exhaustive I was really involved with risk metric before we kick this off so I'm very aware of like the breath of the risks that we can assess on the risk metric side. And it was really built with the kind of like internal use case in mind. So if we, depending on what this looks like for what a shared collection of packages might amount to. If it ends up being something that's like automated, then it might be you might be expecting different things from what we would run with risk metric. And something that's more like how do we take what we would have assessed internally and just make that public, then maybe risk metric is a really good resource for that. So, I mean we'll use the right tool for the for the job and risk metric is certainly among the strong contenders for providing a lot of that great tooling. So maybe there'll be a lot of synergy there. So those are good nuance you share there this is not just about when a company is or a sponsor is assessing which packages are used for a clinical deliverable and then getting that the convenience of risk metric to their input this is also looking at cross industry or cross partnership, cross of hta health authorities so I can see why this is could be bigger than just a risk metric offers but it could still be a key piece to this. Certainly I'm intrigued by where this goes. So, if someone does want to get involved is it just contact you or what's the best way to. Yeah, I'll trap a link we have a repository that where we do all most of our discussion so we have a sign up issue that's pinned there so you can just like message there and we'll get you added to meetings and stuff like that. And from there, we can start. Yeah, primarily want the all as much of the work to be async as possible. So even just jumping in and starting to express opinions on issues is probably the best way to start. So yeah, definitely get involved if there's any interest, and I'll also say that like there's, there's other peripheral elements to this as well so within our consortium there's also a repositories working group that's thinking about how do we improve like the transparency of So if we do end up going down that path and I think there's lots of energy there to start saying you have worked with that team to figure out what are these kind of like core quality measures that we want to see and is that something that might be in the future for just like a agnostic repository. I think that's, that's some pretty pretty big thinking for the early start and I'm not sure that, you know, that's necessarily where things will go but I think that would also be super cool so there's a lot of different avenues that this could take. Definitely I was thinking about that those efforts as you were talking through one of the points you made earlier and your overview is the idea of transparency I think as long as that is at the forefront of like how, how say these metrics or these quality assessments are performed. You know where is the latest updates what's changed maybe in from like a month or previous release of a package to now. I think that as long as you adhere to or the group adheres to those principles, I think this has a lot of promise so certainly feel free to drop a link to the repo and the channel be able to throw that in the minutes. When we send this out for people to get involved but it does sound really promising. And if I can maybe just pose like one targeted question at the FDA folks is the what I was proposing like this, maybe like list of quality measures that we might be able to include as something like a quality report for a package. Is that something that you would be able to get feedback on. Do we have the right people in contact in contact with us to be able to like make that kind of a just kind of first pass doesn't even need to be like a official FDA opinion but you're just getting in some like general ideas around what that would look like are those people attending these meetings or do we have the right kind of like connections to get that type of feedback. Those people don't really exist. Okay, so that's also good feedback. So what would it take then to start just forming some kind of consensus around what, what measures of quality you guys might be looking for. I don't. How should I preface this. There are certain issues that the biggest concerns are probably that FDA it has our around security. Okay, you know where are there any things that will potentially compromise the system. So I one instance that I've seen cited. I don't know to what extent it exists is Haven supposedly can you can run in a buffer overflow issues, but I've used Haven without problems for at least a couple years and not run into those issues so how big of an issue is it. I don't know. So there are some things like that that would be of potential concern. That's more with the security folks. As far as I know, there is no real attempt made to independently gauge the quality of a our package. I mean, basically, what we've been pointing people towards is the our validation hub. Okay, you know, basically saying, look, if you're, if this is a concern, here's, it's not perfect. It's not doing an independent replication of the algorithms, but this is a reasonable stab at the right direction. So to that extent. That's the primary aspect. The fallback position is essentially if you look at the every splash screen of every computer program. It seems essentially you're using as is software. There are no, you know, even software that companies tout as being validated and tested. If you read the splash screen. It's, you're using as is there is no implied guarantee of quality. I mean, that's the cynic of me says nobody is guaranteeing quality at this stage. Yeah, and I don't think I mean guaranteeing quality is definitely a high bar. So trying to reach that I think is going to be. I'm not sure that that's realistic in the, especially in the context of something like our where the landscape is so vast. But what we can do is think about like, what, what qualities can we put out there that at least give us like a reasonable level of confidence and some consistency to, you know, some kind of like heuristic that we use just for measuring package quality that would at least give us like a baseline so there I'm not saying that we're going to like eliminate every situation where heaven might be able to encounter a buffer overflow or, you know, whatever it is, things like that, I think are probably going to be unrealistic for us to try to catch in all cases, or even most cases if just for the potential of those types of issues. So what we can do is form that like central forum for expressing these things so that when you identify that like, you know, there's a buffer overflow issue in haven. We can maybe have a way to record that in a central place so that that becomes a priority for like pharmaceutical companies if they need this package. So contributing that fix becomes a priority for them, because otherwise it's going to be a bottleneck for their submission process so if we can do that then and at least make it more transparent I think that's really the ultimate goal. So maybe we can, what I think would be great is if we, I mean I've already learned a lot about like where your priorities are in terms of assessing these things so if we can put together just like a list of what this might look like. And again, even just like broad strokes feedback about it would really kind of help us set that direction. I have a question to I'm sorry I don't mean to interrupt. Go ahead. Yeah, please. I feel like every conference I go to always has talks about this kind of topic like validation risk assessment. I don't know if there's a way to in what you're doing kind of correlate all those efforts just. I don't know I just hear a lot about auto validator. I think that you guys at Roche have like a GSK we have our own validation process now that I think has been talked about at some conferences it's just like a little. I know we're trying to converge to something that all the pharma companies can can use but I guess I get kind of confused about what what is all the hot topics right now and it seems like there's always these tools out there to show you like the difference between the different packages the risk metrics of the different packages like you know like if things are. If there are tools that are out there and they're like actively being updated to maintain, you know that that that would be good to see in a list as well as like kind of, you know, different topics that have been discussed at different companies using. Validation processes and yeah be helpful for me because when people talk about this it seems like it's always the same. It's similar topic every every time and we're getting there I'm feeling positive about it I just sort of. I get confused about which one is the hot hot one right now. Yeah yeah yeah I I feel that as well. So actually one one things I didn't kind of give this history lean into this just the interest of time but the valid the our validation hub just recently this past year ran a series of case studies where we have we've had like I think eight different companies share their internal validation process so and we put out a white paper after that that tried to find the consistencies between all of those and the differences among them so we do have those resources to try to standardize a little bit, but those are things that are occurring under this assumption that we're all internally doing enough, which is is kind of like this unpleasant like looming fear that there might be something else that we should be doing or maybe we're doing too much and we're just like bogging down a process for no reason. Ultimately this is now trying to like kick off this dialogue with health authorities instead of just being assumptive about what's necessary, and pairing that list back or bolstering it with other things to make sure that it's hitting these notes that health authorities are looking for. It's that white paper it's just on the our validation hub. Let me, I'll share a link with Eric after the call I need to look it up where it was published. Okay. Yeah, no I'd be very, I'd be very interested in that. Yeah, can do. Cool, thanks. Yeah, I think I think that's all the time we need I know we went a little over Eric so I don't know if you had other agenda items but this, this word perfect for me this was kind of the last major item because I don't have a whole lot to share yet on the container pie whatever number that was up being I haven't had a chance to do a lot of digging just yet so not a whole lot on that. But I'll, with the time we have left for a few minutes. Anyone else have any questions or comments about these various efforts or other things going forward. I do have a question on the website for this group. Yes. I noticed I'm helping out Joel with the pilot three and I noticed ours is get isn't updated is it can I do like a PR to get it updated like just with some loose language about what we're trying to do. Certainly, I'd welcome that and I'm going to try and refactor that site and some little easier to maintain in fact I my eyes on doing a quarter site out of it. So, certainly feel free to send me whatever notes you have about the effort description and I'll at least get it updated in the current format, but look for that site to be a little bit different in the coming months because I want to make that bit easier for both Joe and and I to maintain going forward. Okay. Yeah, thank you for that. All right, other questions or comments before we wrap it up. All right. To the new year and yes, absolutely great to see you all for the new year and like I said I think we got a lot of exciting efforts on their way and certainly we'll be keeping in touch regularly so I hope you all have an awesome weekend and we'll see you back in February. Okay. Thanks everybody.


So thank you very much for, can you hear me? Yeah, okay. Yeah, so thanks Mohamed and Victor for inviting me and for letting me present this paper. Actually, this is a project that I have with two colleagues here in Bocconi, Adwid Alfani, who's an economic historian as I am, and Marco Bonetti, who's a bias statistician here in my department. So we're working on this project since I would say one year or so and we are about to finish the first draft. So any comment is really very well welcomed. So, wait, okay. So basically this project, this particular paper that I'm presenting today is part of a bigger project that I have since a couple of years, I would say. And I'm trying to collect for Italian city-state and more generally for Italian north and central Italian cities, data on mortality and especially data on mortality related with the waves, dramatic wave of plagues that started in Western Europe with the black fat in 3047-52 and ended in the mid 17th century. And what I'm trying to work on with this big project and what I'm trying to show you also today is an analysis of the factors that affected the individual risk of death during this major pre-industrial plague. So prevented that we all think that these waves of plagues were to some extent exogenous but the mortality that the chance of surviving of the individuals was related with their social demographic or socioeconomic status. And that's what I'm trying to reconstruct. And what I'm showing to you today is the first step of this analysis that we're doing focusing on a particular case which is the city of Carmaniola. You see it located here in Piedmont for my kingdom, the subaudian state. It was a city of 7,600 inhabitants close to Turin, Turin is here and Turin was 25,000 inhabitants. So it is about one third of Turin, capital of the state in the 1630. What we investigate is a micro demographic database on the city of Carmaniola during the 1630 plague. And what we do is to estimate and reconstruct, sorry, reconstruct theoretically and estimate the risk of contagion and the probability of the individuals to be interned in the plague ward, which I will call Lazareto from now on. That was the name of the ad hoc hospital for the internment and the cure of the, those who got, were infected by the way. And also we will try to reconstruct what was the impact of the health status of hospitalization and the interaction with the individual characteristics on the individuals on each patient on their chance of survival. So more roughly speaking, whether those Lazareto were effective in curing the sick or not. And yeah, I'm gonna go very fast through the literature slide because the literature slide is always very, very boring. What I want you to keep in mind is that we are aware that there is a literature on that. Many, I mean, Guido and I, we have to different extent contributed to this literature. The literature analyzes many factors or many aspects of the impact of pre-industrial plagues. When this literature goes to analyze the micro, the granular level, the micro level of the impact of those plagues, the literature has either tried to reconstruct the patterns of the transmission of the disease or when they have tried to reconstruct how individual factors affected the impact of the disease, they mainly focus on age and gender. Now, what we do here is we try to explore two dimensions that have been neglected also because of data problem in the past. One is how the socioeconomic status affect the chance of survival. And I will try to, I will explain you later what is socioeconomic status in this case and also the impact of the hospitalization. Now, we focus on one plague in particular, which is the last big plague in northern central Italy is the plague of the 1629, 1630. The plague actually originated in 1623 in northern Europe, low countries, northern France and England. And then also thanks to the 30 years war, it's 1618, 1648, he spread in continental Europe, first in Bavaria and Switzerland in 1628, 1929 and then he arrived in northern central Italy. And the overall mortality rate in northern central Italy has been estimated around 30, 35% of population with two millions estimated victims. And we study, as I said before, in particular, how the plague hit the city of Carmaniola. As I mentioned before, the city had about 7,600 inhabitants in 1630, just right before the plague. The city was a wallet city under the subodian state, but with its own government. And the government had a lot of autonomy in dealing with daily organization of the city. Among those prerogatives, the government was governing the cities in times of plague. And in fact, in January, we have seen in the proceedings of the government that the government was already alerted of the possible arrival of the plague from close community who had been infected. And immediately in that month, the government appointed two officials and then the officials became more because this became a semi-permanent health board in the city who were just appointed to try to prevent the contagion. And what did they do? They basically implemented measures that were very well known in the Italian cities at that time. They imposed travel bans to people traveling from non-infected regions or they tried to set up sanitary cordons to avoid people from infected regions or goods from commodities from infected regions to enter into the cities. However, in April, we know that first cases were reported in the city probably also because of the arrival of troops, military troops from Central Europe in the Piedmont region. And in the next month in May, a Lazareto was set up beyond the city walls. This Lazareto will be first a temporary structure made of huts far from the city. And then it will become so big that the city will expropriate a monastery and will put the patients permanently for one year and a half into this former monastery. Now, just to give you an idea of the diffusion of the contagion during the year and a half in which the two years actually which the plague spread through the city. I show you here what is the best measure but still it is a proxy measure of the spread of the plague to the city. So this is the number of admission in the Lazareto. You will understand in a second why this number is a proxy of the plague spread. We don't have better measures. So we don't have any number of that or infected through time. We only know that starting from the first week in which the Lazareto was functioning which is the first week of May 1630. You have a first wave of high number of admission in the Lazareto which is basically dated around August 1630. And during this period, the plague is reported to be a bubonic plague. So a boobos and antisemic death. And then you have a second wave which is around 1630. And here the plague is spreading through fall and it is reported to be a pneumonic plague. Then the plague and the admissions, let's say to the Lazareto declines and you have a last recrudescence in May 1631. Now this, as I said, is the best measure we can reconstruct of the plague spread in the city over time. So just for you to understand better why this is the best measure, I wanna explain with some detail what is the data set that we have. So the data set was firstly put together by Mario Abrate, an economic historian of Piedmont in 1972 who published this paper, this book in which he provided an extensive analysis of the 1630 plague in Carmaniola. We have recovered all the archival sources that Abrate has used and we have checked all the archival sources reconstructing the work of Abrate. And which are those archival sources? First of all, there are two documents that were compiled by the government of the city during and after the plague outbreak. One was the list of all the patients that were admitted to the Lazareto in the year that I've shown you before. So the name of the list is the individuo estatic condotti al Lazareto. So the individuals who were brought to the Lazareto. And in this list, you have the individual level information group at the household level. So if Mattia Fogazzato had been interned in the Lazareto, you know that Mattia Fogazzato belonged to a certain household. And of those individuals, we know the date of admission, the length of stay, the cost of hospitalization and the health status at discharge whether the individuals was alive or he died in the Lazareto. Now, let me anticipate that the cost of hospitalization is basically redundant information. We have discovered that basically there was a fixed amount of money that was spent for each patient every day. This amount of money was unconditional on any status, any characteristics of the individual. And so it was just correlated with the number of days that the individuals were staying in the Lazareto. Another list that we use is the list of all the plague victims in the city. This was compiled after the plague ended and it provides the full list of name and surnames of deaths at home. Now, unfortunately, and this is the first big limitation that we have from this data set, there is no date of death. So we cannot reconstruct in the city any time to event analysis. We can only know that these individuals given the list of name and surnames belonged to probably eventually some households that was interned from the list of admission to the Lazareto. Finally, we have also reconstructed the last part of the work made by Abrade which is to connect or match the individuals in these two lists from the last complete tax sensors for the city of Carmaniola. This was compiled in 6021 and it reported at the household level, the neighborhood of residents of the individuals, the age, the gender of each household members, the occupation and the fiscal group of the household head. Now, this information for us is crucial because what we want to know is how the social economic status affects the probability of entering into the Lazareto and your chance of survival once you are in the Lazareto. Fiscal groups are going from one to five and the richest individuals are in fiscal group one, the poorest are in fiscal group five, the fiscal group six is all the miserable in the city, the very poor that were exempted from taxation. And just to give you a sense of the city, this is the city of Carmaniola, which is the proper city. It's not a rural village, right? You see the wallet boundaries of the city and these are the neighborhoods of the city. Now, what we've done, we have double checked the Abrade's work. Here is just one page or I would say the first page of the list of those admitted to the Lazareto names, surnames, the number of days they stayed in the Lazareto. This is the date of admission. This is another date of admission. And here you have the status at exit. This cross means that they died. This line means that they survived. What are the limitations of the data set? These are important because these dictate basically our statistical work. First of all, we are able to have age, gender and socioeconomic status information only for the individuals that come from the intersection between the three lists. So what we're missing are the immigrants and the children below nine years old because they're not in the 60-21 census. There is no date of that at home. So no time to event analysis is possible. And we have no information of those who caught the disease, remain at home and survive. So they recover it from the disease. We only have information of those who died at home and those who enter into the Lazareto either died or not. Any measure of mortality case fatality is made on a measure of projected total population which is based on the natural marriage and fertility rate that we can reconstruct from the surviving parishes register from one neighborhood of Carmanhola. So we have estimated the structural population at the 1630 Carmanhola just right before the plague by using, by projecting the 1621 population using the natural fertility in marriage and mortality rates from that parish. Overall, and just to give you a sense of what is the situation, what is the outcome of the plague in general, we have about 25 mortality rates in the city, including the victims at home and the victim in the Lazareto. And that compares pretty well with respect to rest of Northern Central Italy, I would say. And we also have a mortality rate within the Lazareto. So this is only among those who were admitted, which also compare pretty well. 28% versus the other measures that have been estimated for other plague words in other city 30 years before. Now, don't compare this measure with this measure concluding that mortality rate was higher in the Lazareto because of course the populations are different between the two groups. Now, recap our research question. We want to understand the socioeconomic factors and the impact of socioeconomic factors on the individual's chance of survival. But given the limitations of the data, what we do is dictated by those problem and gaps in the data, the absence of the date of death at home and the absence of the complete picture of the population in 1630. So what we do and what I will show you in the next 15 minutes, if I can, is the two-fold analysis that we're doing. First, the theoretical model, we construct a theoretical model of the process of admission to the Lazareto and we compare prediction versus realization. And we infer conclusions from the comparison. And then we just, within the population in the Lazareto instead we are able to do a logistic analysis of probability of being discharged alive given the individual demographic and socioeconomic factors. Now, regarding the process of admission to the Lazareto, what we know from the historical accounts and this comes from the government provisions that we have intensively checked in the last year. The city officials were appointed, a group of city officials, usually there were two for each neighborhood and they were regularly visiting the citizen's house and checking their health status. How regularly, every two days, they report conditions of each household. And when a member was found dead by plague, ill or was suspected of being killed, the whole household had to be sent to the Lazareto. So what we conclude is that in principle, what we assume, if the whole household has to be interned when at least one member is infected, the smaller the size of the household, the lower should be the probability of an individual of being interned. To this assumption, which to us seems to be pretty reasonable also because it's historically grounded, we add three other assumptions that to some extent we admit are not that reasonable. One is that the size of the household during the plague outbreak remain constant. The plague outbreak lasted one year, not that unreasonable. The event of the first infection of each individual in the same house was independent, meaning that if I get infected, my wife get infected with the same probability doesn't depend on my probability of being infected. And each individual has a fixed probability P greater than zero of becoming infected in a household with no infected individual. Now, this probability being fixed, this is the most unreasonable assumption that we have to make, means that this probability is independent on individual characteristics of the individual. This member of the household, but not on the characteristics of the household, you will see. Then we have assumed that the size of the members of the household being N equal to the household head plus all the other members, follow up was on distribution being discount data with the expected value being lambda greater than zero. And if P greater than zero is the probability of an individual to be infected, then the probability that all the individuals in a household of size N greater than one are spared by the plague is just one minus P power N. Now, what is the probability that a randomly selected household is spared? It is the probability that it is that the household is spared conditional on the size, the expected size of the household. And these, which is not exactly my cup of tea, but I can guarantee to you that this return, this formula, which just show you that the probability of a household to be spared decreases has the expected size of the household grows. The larger is the household size, the lower is the probability that that household will be spared by the plague. And so the higher is the probability that the household will enter into the lazareto. Now, okay, this is nice, but what can we do with this model? First of all, we can compute theoretically the probability of being admitted as function of any P and the expected size of the household. Or considering any subpopulation, this is important, like for example, a neighborhood or a fiscal group, we can observe that the subpopulation size is equal to the expected size of the household times the number of households. Now, if the population is fixed, then the number of households is decreasing in the size of the households. So the larger the households are, the smaller is the number of households in that subpopulation. What does it mean? It means that the number of households that will enter the lazareto in the subpopulation will decrease in the size of the household. So the larger the households, the less households enter into the lazareto in that subpopulation, but being that household small number in the subpopulation, the proportion of households in that subpopulation will increase in the size of the households. So basically speaking, what you can do is you can estimate the proportion of households admitted in a subpopulation as a function of the size of the households. And why is that important? This is interesting because we can take the number from the 60, 20 census and estimate the actual average household size by neighborhood or by fiscal group. And then we can use our model to estimate the expected proportion of admitted households in subpopulation and compare with the proportions that we observe in our data sets. And this is the main result. So fiscal groups are shown here, six, five, four, one, two, three. So the larger the fiscal group, the larger is the size of the household. And I think Greg Clark will be happy here. And then we have clustered those fiscal groups after extensive selection model across a combination of neighborhoods. I don't have time to explain this, but believe me. So basically the red points here are the predicted proportion of households per group of fiscal group and neighborhoods, all right? Given the mean size computed on the real data. And the blue point are what we observe in the data set. And the first conclusion we have from these analysis is that all the sensitive group entered into the Lazzaretto, according to our model predicts very well how they entered into the Lazzaretto, except the poorest group, which was admitted less than what we would expect. The main conclusion we draw from that is that since the low means size of the households in the poorest group, it was less worthwhile to intern all the members of the Lazzaretto once one members were found healed or died. So the poorest were admitted less than what we would expect. Another thing we can do is to estimate the expected average size of the admitted household. This also comes from the model. And we can compare the predicted household size admitted in the Lazzaretto versus the one we observe. And we can compute the difference or the percentage difference and see that first, our model overestimate the size of the households that were admitted. Obviously our model predicts that all the members were admitted. But the difference between those admitted and those who were left at home is different across fiscal groups. And for instance, the poorest one were interned more or more completely than the richest one. The richest one were resisting to the internment in the Lazzaretto. We have another piece of evidence which showed the average number of days between the first and the last entrance by household members. And you see that while the poor households were actually entering almost altogether into the Lazzaretto once the first member was entering, in the richest household members waited more to enter. So this piece of evidence shows the second part of the story. If we reproduce the process of admission, we see that first the richest entered given to what we expect, the poor were usually left behind. But within the households, the poor when they were entering into the Lazzaretto were entering altogether. While the richest were trying to avoid to enter into the Lazzaretto. Now, I think I only have three minutes, right? Yeah, I would say yes, three to, maybe like up to six minutes or so. Yeah. But so I have to go fast in the second part, which is, well, so what happened when they entered, right? So what we know from the history is that once they were in, the individuals actually received a very decent treatment. Medicines, heating, vitals, their clothes and mattresses were legually clean. They were medicated if ill and they actually received enough space also to isolate the infected from those who were just interned to protect them from infection. There is a part of our work which is concerning with the fact that we want to, so since we basically, we want to estimate a logistic model regression of the probability of being discharged conditionally on the observed demo and socioeconomic individual characteristic, but also the health condition at entrance. I hope you got from my description before the fact that once one individual was entering into the Lazzaretto, all the other members with some distinction across groups were admitted to the Lazzaretto. So we actually not able to observe what is the individual's condition at entrance. And here we have adopted another method and I don't have time to explain it, but basically what we have done is to observe the distribution of the length of stay of the two groups of individuals. So the only things that we observe, the outcome, those who died in the Lazzaretto and those who were discharged, and we were able to make some hypothesis, also given our knowledge of the history progression of the disease per individual on their health at entrance.  So for example, you have here a huge number of individuals who were discharged on the 30th day, which is the last day of quarantine. For all of those individuals, you can assume that they have been entered healthy, they made their whole quarantine within the Lazzaretto and then they were discharged. Now, I only have time to show you the results of our logistic regression. Remember, we test the probability of being discharged alive against the condition at entrance and another certain, a long set of individual characteristics. Gender, the physical group, geographic area, age, and some interaction, those were more, which were more significant. Now, the main results here are, I would say, three. First of all, the chance of survival decreases with age. So the older you were, the higher was, the lower was the probability of being discharged alive. The second result is that physical groups also matter. And the poorest one, who are the reference category here, are those who have the highest probability of being discharged alive. And finally, gender doesn't matter per se, but it matters when it is interacted with the condition at entrance. And the healthy male have a highest probability of surviving than the healthy females. What is the interpretation here is that females were entering healthy. If healthy, they were asked to take care of the unhealthy and they eventually caught the disease. So, and then the probability of being discharged alive declined. So, conclusion, sorry, from the model of process of admission to the lateral, we have shown that the larger and richest households because they're, so the richer households, because they were larger, faced a higher theoretical risk of contagion and internment. And in fact, in proportion with respect to the group, they were interned more. But the richer households, the individuals within the richer households were likely to oppose resistance to the internment. Once we're in, the individuals from very poor households had higher probability of survival. Why? Well, first because the richer household entered in worse condition, exactly precisely because of the fact that they oppose resistance when they were interned, they were sick. And the other reason is that probably, we are pretty confident in that the poor individuals enjoyed better living conditions than at home. And then they could actually increase their chance of survival or maybe isolate themselves from contagion. And that's it. Thank you very much.