Okay. We're now recording and getting started. I wanted to welcome everybody here you can see those of you who who can see this the chat. We have people from a lot of different groups, institutions and so on here today. Quite the diverse spread. And I wanted to welcome everyone. On behalf of myself, I'm Chris Sellers. I'm a professor of environmental history and studies at Stony Brook University and also co founder and project lead at the environmental data and governance initiative. And I'm joined and speaking to you today. This is a workshop by Elena Saxon house who's a senior lawyer on the CR clubs legal team and who led a FOIA coordinating group of the last few years involving many groups, other groups. This is John Chauquan, who is the chief architect of toxic dots, which is a website and digital collection he'll he'll describe a little and who teaches at the Columbia School of Public Health. And we're very pleased today to announce this. The release of this new tool we've been building. It's a collaboration between edgy toxic docs and CR club and other contributing groups to make public. Thousands of FOIA documents received during the Trump years from the federal government. I would like to thank all those who made this possible besides Merlin and Elena, they're sort of foremost in my thanks. But my colleagues at edgy also have supported this project and especially the people at the people's EPA, our website, which is also a portal into the repository. We have Fredrickson and Jess Barnard, also others in the FOIA coordinating group, including those who have already contributed documents and those to come. And the several assistants who worked with Merlin to prepare and mount some of these thousands of documents that are now available. Again, with with a little I mean a story and so an origin story for the, for the repository here briefly, for at least from the perspective of those of those of us at edgy. This idea for this repository first emerged in early 2017. This is when edgy itself was first forming. And we, we got a lot of our early sort of focus in trying to save documents we were documents and data we were worried that Trump administration would would destroy our erase. But we also got wind of this FOIA coordinating group that Elena was organizing and with with a view to, to coordinating more of the many FOIA requests that they anticipated they would be putting into this incoming administration, which seemed, you know, from on the face of it so hostile to much of the environmental agenda that that people had been supporting. So, we, we got in into that group and contributed a little bit along the way, basically, keeping records of the many requests that were being put in. But I think we did initially had the idea that maybe someday we would, we would pull all this together into one big searchable public database. And so for us that was that was kind of the origin. We did as as the years were on and we also the successful FOIA is mounted up. We, we reached out to Merlin and the folks at toxic talks, who had already developed a database Merlin over describe it a little bit, as well as a search engine, as a possible. A possible vehicle for containing all those FOIA documents and the project took off from there. We, we first open it to the public I guess on the 50th anniversary of the EPA in late 2020, but we've now added a slew of other documents and and also will be having most of the Sierra Club collection up there in the next year. So we wanted to take this occasion to let people know that it's there and and let them to experiment a little bit with all that it's going to be able to do. We've gained I think a heightened appreciation in the process here of the need for such a this kind of collection. I mean, FOIA is this powerful tool ever since 1967 72. It's, it's aided the public to know more about what our government is up to. And as wielded by journalists as activists and researchers. It's enabled us to know the extent to which they're acting in the interest of our democracy on on all foreign by the people versus moneyed and private interests. But FOIA itself has some has some issues it works by request. And so rather than automatically people have to request the information take that initiative. It takes time and skill to make it work. And so that automatically means that it's not. It's not that available to many members of the public groups have to devote teams to really making FOIA work requires legal skill as well as time and effort. And I think the main as government has switched to from a print to a digital format, other complications have arisen other, the potential has risen for excessive making this information accessible and usable and searchable. But at the same time, a gap has grown between the spirit of FOIA the freedom of information act and the letter of the law. When you only have to just put up a long URL to release things to the public and nobody knows where it is or has ever seen it and can't find it. And that makes the letter of the law but not really the spirit of the law. And so we're trying to step into that gap with this FOIA repository to do what the federal government has not really done, which is to make it, not just accessible request request in this kind of proprietary fashion, but actually create a database that contains many of these requests, many of these documents and that is keyword searchable, and that you can dive right in and begin to research and find what you're interested in looking for. So, these questions I think that this this the need has become ever more apparent with the Trump administration to where where we've had an administration that is trying to turn the government, many of us saw away from the public interest or the interest of the regulated community. So that I think is accentuated I need to do this are feeling that it's an important project to undertake. And with that I'll turn it over to Elena who will tell us a little bit about about her perspective of getting into this and and what it means for from there from Sierra clubs in. Yeah, hi everyone. So yeah as Chris mentioned we started this FOIA coordination group back in early 2017 when we and other Sierra Club and other organizations were ramping up their FOIA work to try to hold the Trump administration accountable and there was just a lot more interest then and in coming together and coordinating comparing notes both between environmental groups and kind of open government transparency groups. And one of the first things that came up was what are we how are we getting all these documents out to the public. Sierra Club has a lot of you know kind of machinery to get the word out to the public and post documents but not every group does and even for big groups like Sierra Club it's not easy to share documents in a very usable format. So we're going to use discovery software to review it internally, and that works well but that's expensive it requires training it's not something that we can kind of just have any reporter or researcher log into. And so in the past our approach has been to basically share box.com links with you know hundreds of PDFs from a case and each individual one might be searchable but there's no way to kind of search across them. We do often get inquiries from reporters or people working on the hill or academics you know does your trove of documents have you know this talk about this person or this toxic chemical or whatever it is and we've tried to accommodate you know where we're able by like doing the search internally and but that's not really sustainable or something we have a lot of staff capacity for so you know from the beginning everybody saw the need for this kind of public database and I'm very excited that we now are going to have this kind of searchable repository so anybody can dig in. So now let me say a little bit about what's in here and what will be in the in the database soon. And I'm going to talk about Sierra Club documents but I know that there's other groups that have contributed as well. So this certainly doesn't cover kind of everything that is is or will be in the database. So our FOIA requests in the Trump era, focused mostly on external communications between political appointees and their administrative staff and industry and other anti regulation kind of representatives. And you know we kind of focused on the people that we knew had these industry ties or or you know ties to other groups that really opposed any sort of environmental regulation. So we mostly you know we focus first on Scott Pruitt at EPA and Ryan Zinke at Interior but also there were just dozens of other staff whose communications we targeted at those two agencies and then we did have some other more narrow FOIAs to the Department of Security and Exchange Commission and other agencies they just weren't as sort of sweeping and comprehensive as some of the ones to EPA and Interior were. And, you know, so we've gotten just tens of thousands of documents. I think people are pretty aware of all the kind of juicy scandals that came out about Scott Pruitt. But, and, you know, beyond that there wasn't anything quite as you know, kind of the kind of thing that that made headlines but I think, you know, certainly if you're researching a particular policy issue or particular character in the lobbying field, there would be documents that could be very interesting and have not really been dug into yet. And there's other things where it might not have seemed newsworthy at the time, but now more information has come out that may make some of these documents seem more interesting. Like I know that the Republican Attorney General's Association pops up a lot in these external communications. And, you know, I think in much more recently than when we got these documents, it came out that this is one of the these organizations that was supporting the January 6. But so there's things like that that there may be kind of some newfound interest in. And then I guess just some other examples of what we have we have a lot of documents from Nancy Beck who ran the Chemicals Office that I don't think many people have dug into their documents we've been receiving more recently that are communications from Ryan Jackson, Pruitt's chief of staff that nobody's really looked at we've looked at them but I don't think anybody externally really has. Those were sort of the ones where we just had longer timelines in terms of getting them out of EPA and so really getting most of those documents after, you know, after Trump was was actually gone so there's other documents from some of the folks at Interior who were trying to dismantle the Endangered Species Act and a lot on the reorganization of that agency interior to have its headquarters move and and things like that. So there's you know and there's just thousands of documents so I'm speaking just to a small portion of what's there. So we are. We just sent over like a big tranche to to Maryland and and so those are going to be coming up shortly so they might not all be there when you kind of poke around with it today but there will be hopefully a complete set before, before too long. So we are, you know, planning to continue contributing to this project with the additional FOIA work that we're doing we're still doing FOIA work beyond the the Trump administration we have FOIA pending on the Postal Services decision not to electrify its fleet of delivery trucks and you know there's other things that that we're still working on. And yeah so you know we worked very hard to get all these documents I really hope they can be of use to people and not just gather, I guess virtual dust. And so, yeah I'm very excited that this tool makes it more possible for people to access and really find the interesting nuggets in these documents. So I'll pass it over to Merlin now to for the demo. Hi everybody, thank you for attending. I'm in Bangkok, Thailand, visiting my 100 year old grandmother. So I apologize for wearing a t shirt, just kind of ran to pack and didn't pack something a little slightly more formal but here I am. So yeah Chris and Elena gave some excellent expositions I'll recap some of it but then get to the heart of it which is the database and before I do that let me just set a timer I don't want to be that guy. There you go. So, Chris talked about this thing called toxic docs. It's an actual website you can go to it right now. I'll just type the URL here. So toxic docs that stands for toxic documents not toxic doctors, there are toxic doctors out there and there should be documents about them so you can learn about what they've done but that's not what we're about we're about toxic documents about toxic substances and we are I think the largest database of our kind let me just share the screen right now to do that. There we go. Do you all see the screen. That shows up. Awesome.  I think Elena mentioned, use the term discovery. I think most of us probably know what this is, but if you don't. This is where this is the phase in a lawsuit where both sides are basically required to open up their balls. And so forth and I think most famously this occurred in the tobacco trials of the 1990s when Philip Morris and all these other giants had to basically open up their archives and we found out all sorts of things they knew about the addictive properties of, of cigarettes and those documents were part of the success of those cases. Well, tobacco and cigarettes those aren't the only cases where this happens asbestos lead PCBs polyvinyl chloride benzene, you name it. There are lots of active and there's a lot of active ongoing toxic towards litigation. And from the perspective of a journalist researcher and environmental health advocate these cases are amazing. No matter how they end because the discovery process opens up all these materials so in this database we have stuff that nobody has ever seen from Monsanto from British petroleum from Texaco from, you know, you name any any industrial giant we probably got something that we don't know about the owners, etc. The problem is that, even though discovery material is technically public record it's not like there's this beautiful room that anybody can just go to and you say, give me the documents from that Monsanto case and they'll sit you down in a chair and you can look at them comfortably in a in a nice space. No, like you have to depend on the goodness of the one of the two sides in the case either the defendants, or the plaintiffs. And as you can imagine, we don't have much goodwill with the defendants. If we go to Monsanto and we say hey we heard you got sued, would you mind showing us some of the documents that came out in the case.  But we do have some relationships with plaintiff firms. David Rosner and Jerry Markowitz, who teach at Columbia and the City University of New York have written a lot of books on these topics and our expert witnesses and some of these cases and so they have actually access to a lot of the material that comes out during these cases and that's given us an entry point into a much wider network of plaintiff firms, which we've kind of leveraged to get a lot of this material online so even though it's not a physical room where you can sit and look at the stuff it's a virtual room where you can sit and look at that stuff so that's one problem, one thing that toxic docs solve the other thing was a technical challenge though so just because there's lots of documents available and just because a plaintiff firm gives them to you. It doesn't mean it's easy to actually search through it Elena mentioned this before and a lot of the software look through this stuff is really slow and clunky. So the case that a lot of the material when it gets when it comes to us isn't machine readable. So that is the text we have a picture of a document, but you can actually search through it so you have to use this thing called optical character recognition technology where you know it looks through a page and then it realizes oh that's the letter G, and I'll convert it into a letter G so that you know you can copy paste it and search for it and whatever. It used to take a long time so in like the year 2000, even if this project would have been impossible we would have had to rent extremely expensive supercomputers to do optical character recognition on like 2025 million pages, or we could try to do it on like a personal computer, but you know personal computers are really not designed for this amount of volume so we had one of our first batches of documents was four million pages. When I first tried that on Adobe Acrobat Pro just converting it using the optical character recognition I think like my computer passed out at like the 3000 page mark or something like that. The technology has just improved immeasurably in the past 20 years or so and so now we actually don't have to depend on crummy little laptops or super expensive supercomputers. So one of the things we do when we get a batch of documents like the ones that Elena sent to us is in order to do the optical character recognition process we throw it into this giant national computer grid that a lot of academic institutions are a part of and basically what this lets you do is you toss in I don't know a million pages or something and then you parcel the million pages out in these supercomputer clusters that are all over the country and so something that might have taken six months to do 20 years ago takes like six hours to do in the year 2022 so that's one thing that's that's been pretty awesome and we have the thing right here one of things we're really proud of is the speed of the of the interface so you know you type in something like water and you get stuff really quickly my hometown of Los Angeles boom you know 17,000 documents related to Los Angeles and that's something you know we spent six months doing nothing but tests to make sure that people one of the most frustrating things if you do document retrieval is sitting there for 10 seconds with like a spinning wheel or an hourglass that we don't want that so speed has been a priority for for talk to docs. So Chris mentioned we got together at some points the talks that I was humming along we were adding more material etc and edgy was doing it's really important work on preserving the EPA's websites as the Trump administration was trying to destroy the contents. We had this like perfect complementarity, you know we were doing kind of separate things but they were very aligned and so when the FOIA consortium that Elena spearheaded started working with edgy, but they needed a place to store the material. Everything kind of fit together and so that's how that's how we, we got together toxic docs we've always been interested in transparency issues. We sometimes will get threatening letters from like, you know, Monsanto saying you better take this document down and they invoke all kinds of laws to try to do that they have not been successful. Ultimately, but if those kinds of encounters get me thinking a lot about how firms try to stifle the release of material try to keep stuff on paper from ever seeing the light of day so I thought about FOIA as another mechanism of transparency. And there's this project that some of you may know about that I'd always looked to as a potential model. It's called the National Security Archive you could see it on my screen right now this is a project that has filed freedom of information act request to mostly the State Department and the Department of Defense about foreign policy, and they've uncovered so many things about the American foreign policy that we didn't know. So one of the conflicts that I'm very passionate about is the, the American engagement in East Timor the small country, very close to Indonesia, a lot of what we know about atrocities in East Timor comes from the National Security Archive so you know I said yeah wouldn't be cool if those National Security Archive for like environmental stuff. And one of the lawyers who Chris has worked with very closely and who is part of edgy named Sarah Landon who is like a FOIA. So, you know she also had the same idea she wanted to kind of do like something I think she was calling the green FOIA website or something like that so we always had it was just a lot of pieces that kind of conversion came together. So one thing you decided to do is create a separate portal for the edgy material and you can get to it by going to edgy FOIA.toxicdocs.org. It's just like I can't get to the chat if I'm sharing the screen but I'll pop it in later but you should be able to see the URL up there, and we might go grab edgy FOIA.org or something at some point. But this lets you basically just search the edgy documents so that when you type material here you don't get all this other stuff from toxic docs you only get the edgy documents and, you know I'll just type climate change right here let's see what happens. You know 1300 results on climate change and the various things that the Trump administration was doing there, a lot of it was trying to undermine the clean power plan and things that happened in the Obama administration. So you can just kind of type in names when Elena was talking about Nancy Beck I typed in Nancy Beck found 100 documents, mostly emails that Nancy Beck sent out so you know if Nancy Beck is someone you have your eye on here you have 100 emails that you can go you can go read so right now the key way to attack the data set is keyword searches, you know typing phrases you know typing names you might know, and things like that but this has been an issue with toxic docs to you know at some point just typing keywords can become laborious especially if you don't know what the what the keywords are and so this is a separate tool that we're developing. And it will be, you know we've already kind of soft launched it and I think we're ready to actually fully launch it. So go to the site toxic docs dot tools toxic docs dot tools. And what it allows you to do is pick something like the edgy documents, you hit a button, you wait a little bit and then it prize out a list of common people places states and organizations. I'm kind of just scrolling through here right now. So you know, if you have never approached this database before you don't know a lot about the goings on in Trump administration. Boom, now you have a list of people who you can pursue with this tool. There are also developing tools that will look at correspondence networks so if Samantha Dreyvis and Ryan Jackson right a lot to each other, presumably that could be an interesting network connection will have an email extractor. This is actually the first set of documents that we have with a ton of emails in fact I think the majority of these documents might actually be email so we'll allow you to kind of just pry out an email address and then get all the documents associated. With that so that's basically the future but I'm excited to see this continue to grow as Elena said there's another trunk of stuff that's going to be put on very shortly and one thing about FOIA that Chris mentioned is a lot of bottle next in terms of filing these requests and persistence. I filed a few FOIA requests in my day. A lot of times they're denied and I don't try again like I'm dispirited and I'm like I don't have time to do this I just toss it aside and it turns out this is actually the majority experience for most people who file for your request they get denied and the majority of them are never actually appealed so one of the cool things about this project is we we now have people at edgy like Sarah Landon and who can actually help us be more a little more persistent about FOIA requests and how to kind of, you know be the battering Ram when you get denied the first time and things like that so we're going to continue doing that as well so some technical stuff in the future and some non technical stuff. And with that I'll stop sharing my screen now and I'd be really interested in hearing other people's ideas for cool tools and stuff like when we design tools one of the things we do is just sit around the table and say okay what would cool tool be what would be what would be something that would be useful like forget if it's technically feasible let's just come up with the tool. And so we write the tool down and then we try to implement something approximating it so I'd love to hear what people will be interested in that score too. So yeah. Yeah okay we've talked a lot now, but we're really eager to hear your, your comments your questions, your sort of suggestions. And I've also put the links to the there's one portal through a people's EPA, which is sort of a more general website on EPA and then direct link is also there at toxic dogs. So we should be able to get in there and maybe do a little exploring now. And, and we're here to answer any question about that as well. But we're also eager to get more people in groups on board with this. So we're, we're, we're eager to hear what you all might, might want to say add ask, and so on. You can just raise your hand or you can put it in the chat. As you, as you prefer. Chris maybe while we're waiting for people to collect their thoughts I was just going to add and you know just in case anyone joined late. Just to emphasize that I think what's up there. There will probably eventually be like 10 times more, more stuff that that that will be there and so results that you're finding now, you know would not be the, the sum total of what you would find. In a couple weeks. So just want to flag that. And we're also looking, looking to ramp it up still further with with a, we've got a funder interested now in, in hiring some people to really, really get this thing to another level of documents into the millions we hope. I have one question and sorry if you mentioned this already Merlin but I think in a previous demo, was there a way to kind of flag and keep documents or something that kind of sets some aside. One question answers yes and your question. I think it's very similar to Hayden's in the chat. So yes, absolutely so if there's these two buttons that say log in and sign up very easy type your email you make up a password and then boom you got in the that you can use every time and as you go on the site. I'll show you.  So what you do is hit boom bookmark and then that document appears in like a book bag. And so you, as you're going through on a first pass, one of the things I do is yeah I rapidly kind of look through I look for three seconds like okay that could be boom bookmark and then you toss it in the bag and so that's that's definitely a good feature so you don't have to sit there and one by one, you know, download stuff and interrupt your flow, I guess. So yes, definitely. And that's what the thing Hayden is asked for. You don't actually have to sign up or log in to use the site. But if you want to use the bookmark feature, you have to because, you know, if I'm on the site and I haven't signed in it has no way of knowing that my bookmarks are not Chris's bookmarks are not Elena's bookmarks, etc. There's another question here from Allison Cole about plans together document together into collections. Yeah, just elaborate a little bit I understand that's like, I guess like a subject matter, sort of expertise or interests and a lot of heavy lifting but you know I didn't know if you know to help researchers newer to a topic, and you know get a fuller context. So like clean power plant discussions collection and being a key documents that subject matter expert might might pull out on to get folks started. So aside from the keyword searches, having actual document collections by topic. I mean I think you can search it by source. Yeah, as I understand it. That is, this is the Sierra Club collection this is American oversight, and so on, but yeah I think this would take a while and a lot of, like might not be the goal at all but I put an example of a site that's not as. Oh yeah, sure I triple necessarily. And then I think like, has a limited collection but like within that they're just sort of like here of these introductory introductory even docs on the topic, but, but that might not be a goal of the project. So that is definitely a goal project. And I'm going to jot that down as like a very important high priority suggestion. It's something that has, has also been an issue with toxic docs like, you know we have like the world's largest collection on there about of asbestos documents and there are so many components of asbestos different like derivations of asbestos, and so forth and so like looking at this giant dog pile of documents on asbestos can be overwhelming and so one of the things that you know you one would might want to do is exactly say cluster it by more specific subject matter. So there's kind of two solutions to this like one is old school and one is new school so the old school solution is if the documents come to us and somebody says, these are documents about asbestos in. I don't know car breaks or something like that. We can actually flag them as car breaks and then separate them onto a wing of the site so that only people are interested in car breaks can can look at that stuff and we can actually do that. To some extent, Elena can speak more to this but you know the as she noted the FOIA requests are about sometimes very specific subject matter so we can certainly do that in an old school way. The top is if you start making like too many demarcations then you know you could eventually get a list very quickly of like 500 different subsets of documents so it's a balance between having the one giant pile that's too big versus too many subsets but that's an excellent. I think we can find a happy medium between those two. The new school technique is to use some exciting new developments in natural language processing where we can have a computer essentially find clusters of documents that might relate to each other because the same phrase or the same person shows up and we're working with things like that right now is just kind of clusters of documents and so that might be one way around it to so definitely stay tuned but I'm glad you raised that because this is definitely something that would improve user user experience and thank you for this. Sure and I'm yeah I understand what you mean about the new school project I had access to talk through software that could do that but never got to see it work so. Yeah, that's really cool. It's like, I mean sometimes it works really well sometimes like the results are kind of absurd like it'll see like the same word and assume we're the same collection of the same phrases and assume the things are similar but they're not so it's close to prime time but not quite prime time so we're experimenting as well. Thank you. Other questions. Suggestions. Chris maybe you can say more about this but I know one of our ideas to is, even though there's so much Trump stuff to go through. You know we do eventually want to kind of go to other agencies besides the PA and other administrations, even Chris and I are our day jobs were historians and anyone who has been to the National Archives and looked at the EPA record group knows that they're not exactly high on the National Archives priorities and so sometimes, sadly creating your own archive from FOIA is a better archive than the actual EPA archive at the National Archives in terms of really practically being able to find stuff. I know Chris did you think that's in the car in the future like an Obama collection of George. Yeah I think I think so. I think, you know I've looked a little bit at FOIA online, which does accumulate for instance all the EPA, although Department of Interior you have to go separately to Department of Interior, a lot of the EPA as well as some other agencies post their now post their foias and what's supposed to be a central online repository but it's voluntary for the agencies, and that goes back into the Obama years to 2013 I think was the earliest in there for the EPA. So I, we would welcome other groups contributing their foias, all of you from groups that do that.  I think there's been so much stuff. So many requests that have been out there that I know have been have gotten some response that from the last few years that I think we'd like to build out that collection that part of the collection. But yeah I think that will be more sort of our initiative to figure out how to how to how to just go in directly to FOIA online. If we get this grant from the funder will probably be be looking at getting more systematic about FOIA online and getting documents off that, and then also have discussions about, you know, I think it's going to also depend on this question of collections, and you know if there are particular topics that we want, we feel are important to dive into first versus next. I think that's kind of connected to Allison's question about about particular collections that we might want to pursue first versus later I think climate is one of the obvious places to look. And other agencies certainly I mean we already have not just EPA but interior from the Sierra Club. And I think some of the wilderness society is also interior. And there are lots of other federal agencies to that get involved in environmental things so looking at only some of which are on FOIA online. So we're looking to to along all these fronts to expand, especially as we get, we get some staff and a manager and so on to really, to really work on this more full time in the next few months, we're hopeful. I'm sure Elena especially knows a lot of the worst kind of mischief happens at state level with state level regulatory agencies, which have even less of a commitment to electronic transparency so that's another avenue I thought of as well. And Sarah Lambdan has already done some digging into that too is our colleague at edgy Sarah lamb that has had some classes exploring that and gather documents. So, so that's another, I mean that would be a whole nother sort of Merlin would have to figure out the portal system and yeah, vision of the federal versus the state and all that kind of thing. But that's certainly, yeah, a wide open and promising way to expand to. You know, one of the things I found pretty interesting about looking at the material as a whole versus just reading about a set of documents in, you know, like, like some of the high profile stories that that surface around. Edward and the subsequent. EPA director was when you actually see it as a whole you really do sort of see how they use bureaucracy and weaponize it. So there's a number of documents where people in the administration are doing other or surveying other agencies to see if there is any climate change plan in action and what it is exactly. And I think the implication is that they're saying what can be done to put the brakes on such things and it's hard to see that kind of thing unless you see the thing as a whole. So sort of tedious to read because a lot of it just on the surface seems like, you know, very routine row administrative emails. And so it's in some ways a challenging set of documents to use but I think if you read long enough you start to see what exactly a seemingly mundane memo to the Bureau of Land Management is really about. And I would just add some of the. I think we initially thought that some of the, you know, interesting emails would come from the higher level people but it turns out that they're not on their email that much and so you really have to find out who the administrative folks are and we're making the scheduling, you know, appointments and things like that and kind of, you know, one or two levels down from the administrator one or two or three levels down from the administrator. Their communications can be can be really interesting. So the contacts between the politicals and the career people happened, I guess that was where a lot of the friction that we picked up in the Trump years. We also, I mean, as part of this, we're we're we're also looking to create this public environmental archive of which this is a part. It also includes the interview series that we did with EPA folks over the Trump years and and really reflective to the comparison with Obama and Trump and further on back to. We have a whole series of 140 interviews that we did that we're going to be also putting up on that people's EPA website which is, which is has that portal to toxic docs and so on. And we're, oh, we're also, we're also going to be putting up databases there of some runs of EPA documents keyword searchable on environmental justice and that kind of thing, including the to make those more accessible communities are publicly available elsewhere but to have one place to pull a lot of this document documentary material, as well as the four years and so on. So, looks like we have a couple of comments sort of a question looks like EPA is pushing the oh the EPA archive. The online archive. Yeah. Yes, that's true. The edgy was was and Sierra joined us to we have put together this, this big push with a lot of historian groups as well as the green and open government groups to once we heard, we got when that the EPA was getting rid of its online archive.  We pushed back I mean they had not made this is how things work in the digital realm, digital things for the these agencies seem to be entirely disposable. And so they just say oh we're going to get it back to the National Archives but of course that means, you know, they shut down the digital access, the digital archive that means that we have to wait until NARA processes everything. And you know it's kind of lost in a in a sort of in lost to the public really for multiple years on out before you can actually get a hold of it. If they do get rid of this online archive so we got a lot of people groups. Interested in pushing back and we did get the agency to delay. It's, it's termination of the online archive at least so 2023. But we'd actually like to see EPA have that be a model. I mean, we'd actually like to see EPA and the other federal agencies actually do this to create these archives themselves. Because it's about accessibility is about sort of their public their obligation to the public to use the latest technology to really make it more accessible rather than just this literal, you know, baseline level of accessibility that they're stuck on. So but in the absence of that we don't see much sign that's happening. We do want to create some eventually some more pressure to get them to do more of this themselves and the agencies but you know, we don't see that happening anytime soon. And so now is the time actually to formulate a model. This is what we see this initiative as a model that perhaps the government at some point can take up and run with. I mean, when it when it decides that it really wants to make its, its inner workings accessible. So I think that would have shown them how it would be the best person to reach out for your response you want to visit. So I think that would be the contact people would be Merlin and me just just we like to have double contact so things don't get lost but just write us an email. Maybe we can put our emails in here, Merlin. We have, we have separate emails and then we have this one email that goes to both of us. It's intake at toxic docs.org. Oh, that's right. Yeah. That doesn't work. It's because I'm jet lagged and forgot to email but it's something like that. But yeah, just try the other one might actually mail in. You know, I'll give you the generic one. Yeah, so we welcome that I believe Robin is Robin is for justice right. That's great. Yeah, yeah, I'm with her justice that's right. I know you guys have so many of these you put in. That would be wonderful to get your collection in there. Okay, yeah, thank you. I'll keep those all right down those emails. Yeah anyone else. We just just, you know, look us up at these emails and I think you can see the emails and we'll send that a final we'll send that a sort of wrap up email to everybody who's on the participant list. We're inviting people to, to, if you do have for you as you like to contribute to, to let us know. Okay. Any concluding comments you all a liner Merlin. Like thoughts you like to leave us with. No I just really appreciate the work that that you all are doing and, like I said earlier just really happy to be able to get these out of Sierra Club computers and into the world. I'm excited about it. Yeah, I think for you for all the things we've learned about is actually pretty underutilized tool and can be pretty intimidating so. And a lot of the fruits of it, don't actually are not actually widely disseminated beyond the immediate purpose of the FOIA request, whatever it is. So, you know, I hope I hope stuff like this becomes more commonplace. Okay, well thank you all for attending. And also we look forward to to your use and just reach out to us if you use it and you have comments or questions or anything. I mean we're, we're there we're available. You have our emails. We'll also be sending out that final email to everyone who's here that we have. And we look forward to working with you all to really, really building this thing out and, you know, using the fruits of the FOIA potential, bringing it to fruition through this kind of tool. Okay, thanks for everybody. We'll see you soon. I hope.


So hi everyone, welcome. Thanks for having me here and thanks for joining today I will talk about how to ensure your business viability that relies on open source by sharing some of the programs I led at VMware. So let me take you through the agenda Firstly, I will talk about why we ensure the long-term viability of open source projects and what we did in our Ospo's strategic project assessment Then I will share how this assessment is different from the usual project health check It's objective what is in scope and out of scope We will look at four focus areas and criteria with examples of real assessment result I'll talk about the process How we run the whole cycle Collaborating across the teams in our station Finally, I will wrap up with a summary of key points and have time for Q&A So quick intro about me. I studied engineering at uni study Started my career as a Windows software developer at a global technology company in Japan Then I worked in the US and the UK Where I led the engineering teams for Windows Embedded CE, Linux and Android based platform After that I changed my career from telecoms to cloud Leading multiple programs including large-scale business and IT transformations data center deployment and information security process building Moving up to date in the last three years I worked in the Ospo open-source community strategy team at VMware Leading alignment between the community contributions and the business strategies Last month on 22nd of November VMware was acquired Broadcom So the company entities are now part of the Broadcom group So why we should ensure the project's long-term viability? When our business relies on open-source, we must provide a real commercial support for our customers i.e. an enterprise-grade software We have a responsibility the same as for our proprietary software Because they're both part of the product If these projects are no longer maintained or if we don't understand the entire project We will be unable to support our customers Therefore we need to guarantee the quality of the open-source in our products It's continuous availability in the same way as today aligned with our portfolio As open-source projects are maintained by the communities and because we are a part of the ecosystem We have to ensure they will continue to be available and successful For a long term for both for our business and for the communities At VMware we maintained thousands of open-source projects and contributed to thousands more And it's nearly impossible and not cost effective to track the viability of all of them For the projects that are critical used in our key products and to which we actively contribute to Ospo as the center of the gravity of companies open-source operations Should take responsibility to make Make sure they will continue to be viable in line with our business goals So that we can support our customers and keep their trust For these business critical projects, Ospo ran the strategic project assessment And this is in addition to the project health check that was performed by our business units for actually working in the project So there are two main objectives of this Ospo-led assessments The primary objective is to manage the business risk In the project if the project is not owned by a company but by a foundation or other company We don't have full visibility nor full project control This means there might be unexpected direction change, license change or a shortage of maintainers If your organization's engagement is insufficient and relies on the contributions of others You may end up fixing all the code defects and vulnerabilities on a project Which you may not completely understand In our Ospo's assessment, we aim to identify and mitigate the project risks in advance In order to avoid these disruptions that may impact our business The secondary objective is to grow our business opportunity Healthy viable projects know that market and the audiences they want to work with They will attract more people and will help broader adoption along with healthy communities And this is expand this will expand the addressable markets for your future business If the project is owned by your organization, it will effectively represent your company in the ecosystem As such, this project must be maintained to the same high-quality standard as your other products and services So what is the difference between the project health check and this Ospo-led project assessment? Both have the same goal. However, there are key differences. For example, who actually performs the assessment? A project health check is often performed by the internal teams who own and maintain the project within the community If the project is owned by your organization, your Ospo may also help and support it This health check tends to be subjective and carried out from the maintenance or project owner's viewpoint For example, criteria can be how often do we release the code or have we created all the documentation Or are we inclusive enough in our community versus the strategic project assessment are objective Performed by Ospo who are often outside of the project or at least not involved in all of them So they carry it out from a third person's independent viewpoint For example, criteria can be does that project share a clear roadmap so that we can ensure it's in line with our portfolio? Or does that project have open discussion channel so that we will be able to influence the future release? With the strategic project assessment, Ospo plays a role in the future release Ospo plays a role to assess the project externally Feedback the results to the team and to the community so that we can work together towards the improvement So what activities are in scope? The Ospo strategic project assessment is performed based on the publicly available data in the project website or github page A good inclusive project is open and transparent And its data is easily discoverable and accessible by all And it focuses on the factors that may impact the business and also the ecosystem Regarding out of scope Basic documentation requirements as open source are not checked as they're usually covered by the project health check In depth Technical analysis is also out of scope Because they will require many internal technical detail data thus only possible by the teams who are actually working in the project So it is important to Remember that the assessment should not create an extra burden to the teams who are actually working busy day-to-day work We must try to avoid increasing their workload By both parties performing similar assessments Typically Ospo doesn't have sufficient resources to perform multiple detailed analysis for all of the strategic projects The assessment should stay simple so as to be achievable yet still useful It should be the trigger of in-depth analysis for the teams in the business units Our objective is to assess project from different angles And give a heads up to the teams to perform their further analysis So what do we care about in this assessment? Our priority is to assess whether the project is viable for the business And also for the entire ecosystem And we must have a strong level of engagement in order to have an influence to the project So checking the organization's level of engagement in the project is really important in this assessment Having a good grip of the organization's critical projects helps you control your technology roadmap for the long term So we know that the good open source projects understand their target audiences And that they want to work with They openly share their plans and resources Demonstrate healthy governance And the community to attract people to achieve their goals together On that basis we have identified four key areas and respective criteria Project maturity impartiality. I'm sorry project maturity planning transparency activity status and documentation adequacy For example, does your project share a clear roadmap and a plan? Does it provide a sufficient documentation set to enable you to develop and build on the project? Project governance impartiality Is the governance spread across a variety of organizations to represent the ecosystem needs? Has the project defined clear roles and had a decision-making process? Project community openness adopters For example, does the project have an open communication channel and hold the regular meetings? Is the project adopted by many organizations in the ecosystem? Organizations engagement For example, does your team engage enough to understand the entire project and what's going on in the project? Does your organization hold any leadership role in order to be able to influence the project direction? So after further breaking down each of these four areas We have predefined a set of color coded criteria to indicate That status so the team can refer back after they receive the assessment results from us So this is a simplified example of our real assessment result For some of our strategic project This project was initiated and open sourced by our organization Then donated to a neutral foundation two years ago We have many internal maintainers and contributors Working in an organization with a good set of shared documentation and making active releases regularly However, currently the project only have internal contributors and doesn't have any external contributors Therefore not getting much benefit of the community power So our recommendation to the team is to promote the project more to expand the project More to explore contributors from outside of the organization So that the project will better represent the ecosystem needs Share the workload and be more robust and adoptable And this is another example This project doesn't belong to a neutral foundation and is currently owned by a single vendor And they they dominate most governance positions The project itself is very active with many companies contributing However, the roadmap is only shared with the limited people in the community And not publicly shared on the project website nor on the github page And the organization's participation in the project is very small And we don't hold any leadership positions This is a risk for business As the project is used in many key parts of our solutions If we don't fully understand the project architecture And we don't have clear visibility of the project direction It is not good So our recommendation to the team is to increase our involvement in the project Recruit external new talent if needed who already contribute into the project In order to increase our understanding and inference in the project This is necessary to be able to pass away the community to make the project be more open and inclusive So so far I've explained the why and what of the assessment Now we will consider that how During the assessment process we identified a number of actions and recommendations to increase The project viability Then we hold reviews with the teams where the findings are discussed In the company's portal, we also published assessment results with actions so the project users can see internally In the community The team will implement the improvements working with the community With our support where needed And every six months a follow-up meeting is held to review the progress And provide further guidance The OSPOS assessment result might be different from teams view So it's important to get their opinions as well before finalizing Perhaps they have never considered the project viability and business viability So it's important that independent person from the teams perform their project assessment and provides an opportunity to improve This regular follow-up is also beneficial for OSPOS As it helps them to build the relationship with the teams And increase OSPOS knowledge of organizations critical projects In addition, OSPOS is able to raise an early warning for this project if there is any risk To our leadership team So when we work with the teams, we have to keep in mind that this is the assessment This assessment should be the conversation and not like an audit The feedback should be provided in a positive manner and not judging their work or criticism We are here to support and guide them And our focus is to help them find a benefit from our assessment Every project has different goals scale and background There's no one-size-fits-all And no single metrics that works for all projects So it is important for us both to understand the background of each project And focus on the right things as per their priorities Viable projects are transparent, active, impartial and diverse They share the goals clearly They are easily accessible and discoverable by anyone in the ecosystem to attract a wider audience In turn, this will make the project even more powerful and viable And to enable all this, your engagement in the project community is vital Assessing an open source project can be challenging as there's no single indicator that shows the entire project status By assessing a project viability from multiple perspectives Both internal and external to the community The business risk can be reduced By working together with the teams and the communities Your OSPO can ensure your business will continue to be viable, sustainable and successful Well, that's all from me. Thank you for listening. And now I'd like to invite questions And I'd love to hear what your OSPO organization do to ensure the business viability with open source. Thank you Any questions? No questions? Thank you. It's a great question. And yes, there are many, not many, a few projects that I put the red because of my criteria It goes to the red category But this is why it's important to make assessment, share the result and discuss with the team because team has very, very different opinion So all the projects are different, different scale and then some projects are entirely okay just having internal contributors and maintainers Not ideal in general, if you think, because what's the point to donate your own project to the Neutral Foundation? You know, it's under the Neutral Foundation. However, you know, we're not benefiting You know, it can be just owned by your company, but you have to think about many different angles For example, if you just own your open source project and then the external people feel like, oh, this is owned by the company Okay, so maybe I don't feel like I'm participating. You know, I'd like to. I'm interested, but I may not So we tend to donate to the Neutral Foundation to get more benefit, to be open and share our technology, our capability But some don't feel, you know, for example, this project don't have no contributions in the organization So this is project B So, okay, so this is not a good example. I wanted to show the one before Yeah, this one. This one is owned by, used to be owned by the company And now it's donated to Neutral So I don't have any red in this example, but you know, it's ideal to have more external contributors from outside So what's the point being in the Neutral Foundation? But they said, oh, it's entirely okay It's because it's really specific for this product and we are already benefiting, having just internal and we are happy to have Not to have outside contributions and then we, you know, organization strategy is very complicated and it's not shared So not everything is suitable to share and having as many as possible contributors from external Because like the first date keynote, it's really difficult to, as a maintainer, it's really hard to correct Which modification we get and then what is going to the next So for this specific project, I remember that I met the team and they said, oh, it's entirely okay in this way And then I discussed and I understand that their project wrote them up and there was their strategies And I'm happy to leave that as is So yeah, discussion, this is why follow up discussion is really important So it's not always, but I actually don't change the status color code It's because it still should show red and then if it's red and people look at them, oh, what's going on here And then they will also understand because I add the column saying this is red, however, it's okay from the organization's perspective So yeah, with that comment, it is okay But it is important to share all this process and discussion process with the teams and also inside of the organization So it's a great question. Thank you No, it's more like, yeah, we, for example, at VMware, we used to have 20 strategy project because it's really, really key of our products and solutions So we have it and it doesn't, and then if shows yellow or red doesn't mean that our priests don't use it This is why I'm sharing the reason of this yellow and red is this, this, this and that And each business unit, you know, across the different business unit, they want to use the same project That's why we are publishing this information in Osprey's portal Then they look at them, read the reason and if needed, reach out to the team because I'm sharing the point of contact people of the project, you know, main lead And then they discuss and conclude, okay, so it's okay to use And then there are some suggestions they really, really address to But if your involvement in that community is not deep enough And then they can't tell the community because you're not contributing So what are you saying, you know, for example, I am looking at the, I can't raise the project name here But there are some third party project which is really dominated and owned by the company And then, but this is not good. Why don't you suggest, but the team doesn't want to because team is not contributing This is why we have to start from small First of all, we have to have one or two strong contributors in our team because we are relying on that project And then expand understanding and inference Then they start to be able to make opinions to the community leaders And suggest, oh, why don't you have more contributors and why don't you have more leadership roles from other organizations Why don't you donate to the Neutral Foundation But if you're not contributing, you can't do that So I think, first of all, you have to have a strong visibility and inference in the project That is a good start to be able to efficiently use that project in your product and services You're welcome. Thank you Any other questions? Thank you for your presentation And the first basic question that in the Github There's a private mail address and a company credit address And so it's very difficult to just grasp which company you're building beyond the Neutral So how do you just grasp your company's contribution because sometimes it's a private address Do you have to issue or annually ask the team for just the same serial address to the hospital? Yeah, great question. Thank you That's one of the problems I had during my assessment Because oh, this is a private address And I have some ways to find out And sometimes using a different matrix and data I can find out, oh, this person is actually in Google Or this person is in a company like that So I would have a good understanding But for most of our strategy project, we have a very good grip And then there are quite a few contributors and running So actually our company shares our company account, mail account in the project So that we would know how many people contribute into the project And also I have another talk talking about the strategy project table And that one has which group in our organization is leading this project And then if you reach out to that person, they will have a good understanding of Oh, actually maybe 30% of our company is contributing and 20% from Amazon and other companies Okay, so I would see the good understanding So it is not straightforward, but there are some ways And especially for a strategy project I had a very good understanding of the proportion of which company people are contributing Thank you very much Some small tips We also tried to just integrate the mail address to the company one Yes Yeah Our company, yeah, I think we have a role I can't remember now But yeah, we tend to use our companies I think most of them we use our company account Like it happened And second question is that How much coverage for all the post-projects that the employees are involved in Is that the post-covered cover? Or is it covered by the activity format? No, no, so yeah, again, I have another talk about the strategic project table creation So what I did is to my mission when I joined VMI My first mission was to find out what are the handful of important projects for the company Because we contribute so many of them And it's really not ideal and realistic to assess all of them So we pick up the 20, I think it's 15 to 20 normally And we talk with the VU all the time and what really is the critical projects Our open source projects for our product And we have a good idea and we regularly meet with them and then shuffle around Remove some of the project If it's not important for us anymore or adding the new one Then we run this assessment just for that strategic projects And my last question is how much, like a personal effort You're also teaming with Hawaii for making that effort And also what kind of tools, like a chaos or just some tools that can survive Yeah, chaos, did you say? Yeah, yeah, yeah, good question, thank you So yeah, chaos has many, many tools and metrics to measure I had a similar question in the past talk And yes, it is useful, super useful And my counterpart program manager was running using Kale's tool It's because it's relevant for our internal company-owned project But I was mainly responsible for the project which is under the third party Like originated by a company but donated to the neutral foundation So it is entirely not belonging to any company or belonging to the company I.e. it's not owned by us So for that project I didn't use those tools The reason is those tools are... I don't know all the tools in Kale's project owns But I thought that I look at them and I look at the result And then some of them are run by our internal project health check But for this specific assessment our objective is to see the business impact To our business impact relying on the open source So I on purpose not using them and have defined and developed Our own criteria and our own assessment theory to run this And the first goal I shared this with my top management And also the BU leadership where they are quite happy So I just carried on this one But yes, to develop that, further develop I think it's a good idea To assess what else we can do so that we can have more accurate assessment result I think it's a good idea Yeah Reporter, yeah, just myself So this is a simplified version I worked really hard to make it simple But the real one is really have more columns and more letters And so not many people want to read it's because it's too many letters But the project owners are happy to read and then see I'm trying not to be too critical but I just factual stating Based on this criteria your project was assessed like this Not personal, just project is like this And I meet with the team and the team is a little bit cautious when we meet I organize a meeting, we meet and they're not very I feel like some project and not so much but some project Why you're scoring a project like this But I start, I explain the background of this assessment What's the purpose and this is the score I have But what do you think? And then they usually come back and give me such a important useful information What's going on in the team and I just think Oh my god, I didn't notice that And yeah, so I change my, you know, the status won't change But I add that comment in the result and share publicly So that everyone in the organization can see And if it's, you know, then before I publish it I will check with the team, can I share like this And then they sometimes say, oh, don't share this one And then say this instead So yeah, I will shape it and once we both happy then I publish So it takes, you know, I run this assessment once in six months Every six months because it takes so long And also they don't want to be assessed soft And once their action is clear and what's next step is clear And I leave them for six months Then we meet again, hi, how are you? By the way, what's going on this and that And then sometimes, not sometimes Actually in the last assessment I got really, really positive feedback Saying, oh look, you know, we got a leadership role in this project You know, we hired these new talents who knows about this project So we are much, we have much better visibility So it is working, so I'm pleased to hear that So yeah, I think it takes a long time But start small, then it starts to become effective That's what I felt Yeah, it's the inside course Because usually what's for is just a general information And it's just only one section of that We're just using this information in a way Or the wider section of Exactly, exactly Yeah, yeah Yeah, you're welcome, thank you for asking No more questions? Okay, thank you very much for listening again And yeah, enjoy for the rest of the conference Thank you