Okay, so we are right on time and Thank you for organizing this track. Thank you for the opportunity to speak to you This is Marcel my name is Marcos. We are both research associates at the Department of Medical Informatics in the city of Göttingen in Germany and now we switch from the Neurosciences domain to the medical research domain and Our plan is to give you a quick very quick overview of some of the tools that we use in our domain of research and Just to get an overview who has ever been in contact with the domain of medical informatics Quite a few. Well, it's overwhelming actually So you might recognize some of the tools but for all the others we try to introduce a little bit The discipline in general so We thought of how to how to Characterize our domain and we set last like four major fields of research and medical informatics So at the core of it, there's systems for primary care So I think everybody can relate to that when you are sick when you go to the hospital you your case creates data your Your condition is documented and all of this is done in electronic health record systems Maybe there are images taken MRI CT Echocardiography days those are put into some information systems and all of this is primary care domain and then we have a couple of research domains or research heavy domains first of all biomedical research where you study the Systems biology within the body and derive therapy ideas The other route would be to go down here Into take routine data run statistics on them and do use so it's called secondary use because you again Data that was collected in primary care is used for research to develop new therapy options again and Finally because we run something called evidence-based medicine We want statistical scientific evidence that the therapies we run in primary care are actually not harmful and even beneficial for your condition So all of this goes up to the top into what's called clinical trials So basically you do experiments in human To make sure that everything we do in therapy is actually sound We will go through a couple of software tools that are part of some of those fields and One thing I would like to mention is that the primary care is basically the Probably the domain with the least open source software Which is due to the regulations that are in place in primary care. I mean it is you're treating patients Which means that even software will soon have to comply to European medical device regulations and so on so many companies Create proprietary software and sell it because They say our software complies with all the regular regulation So to take you through the software tools that we want to introduce Short story This is Bob Bob suffers from chronic heart insufficiency So this means that Bob regularly has to visit the hospital Have his condition checked up Every time he has to go in for routine checkup they Document the medication he's on They take blood samples and also they make echocardiographic imaging and All of this is stored in the clinical information systems So this is a model hospital because they actually use an open source tool for primary care documentation in this case they use X-Nut which is a picture archiving and documentation system that is open source and That can store the images created and The the structured data that is derived from those images. So the vital parameters that are actually Interesting and further research on the data X-Nut is a very extensible open source tool. You can not only store the images and share them with your colleagues You can also plug in analysis pipelines like in this image J to run analysis on the on the images and data that is stored in there and Yeah, in the end it gives you web-based user interfaces for image upload Also image viewers, etc So since Bob is also not only at the model hospital, but he's a model patient. He has given consent for That his his medical data may be used for research purposes Just like that and this leads us to of course Alice and Alice is a health data engineer At a place called the medical data integration center. So that's basically where we work And her job is to get All the data that is created and documented in primary care systems out of those systems. So you extract this data To mask patient identity because in clinical systems You always have the medical data stored for each patient and you know who this patient is in research We do not want to know who the patient is. So we want to anonymize the data so The data has to be masked then the data has to be transformed according to the formats that you can use in In research and finally put into some kind of research data repository to make it accessible again for researchers The tool that Alice uses is open source and it's talent open studio for data integration. So this is a Graphical user interface data workflow manager. I'll give you a little bit better Image of that so you can create data transformation workflows with a graphical user interface. It's based on Eclipse. It's Also provided as a product by a company, but All in all it's open source tool and you can create these kind of workflows dragging and dropping Like sub-processes that are encapsulated and can be reused in different workflows You can export all of that as a as a jar file and then run it on service and orchestrate this and This is basically what we do to Extract data from all the different clinical information systems and put it into a single report research repository in the end So sub-processes part of these ETL jobs run through talent is for For one the masking of the patient identity The example two we use for that is the Mainzl Lister. It's a German German name for Two mostly created in Germany. So please excuse me that these slides are in German It just says that we pseudonymized the identifying data of a patient. So you see the name on the On the left actually and then you pseudonymize it You just get some random ID number back and you can also use this service this Masking service to create different different secondary IDs that belong to this first ID. So Yeah, there's lots of math behind that and you can actually do lots of stuff so you can do De-identified record linkage. So we have research data from two different systems That have separate IDs. You do not know that those are the same data from the same patient But you can use the service to link those data back again in the end Yeah, so each of those De-identified data packages that we now created in the first step of our workflow Has to be stored and we are working in science. We want to have all the all the data pieces we use in in the scientific progress Archived and stored persistently for this we use an open source tool called cd star This is developed in gutting in its Data storage middleware basically so We mask all the underlying block storage options or that are when running the data center We put a REST API in front of it and we can just use REST calls to store data items together with Access control lists and media data about this item and we also generate persistent identifiers So that each data package that was used in one of those processes Can be identified afterwards permanently Hopefully forever because we use the data center that's In the forever business because they are linked to our library university library So this is the cue to switch So awkwardly switching has been done. I hope the cartoon was long enough to distract all of you um So we've seen that alice has stored her data in cd star and now She has this data in some format. She may be thought of but we would like to have more We would like to have semantic annotation meaning that the data file itself should contain something that tells some other person What the data is about so you may know it from csv files with just data tables where you don't know what it means and Best most of the time table header that says some weird combination of letters that you don't understand So in order to circumvent circumvent that we use open EHR open EHR actually is a foundation So it creates a lot of stuff But the two things we want to focus on would be the specification as well as the clinical modeling part of open EHR So as for the specification the guys at open EHR created a two level modeling Yeah architecture Where the specification states different reference models These are the most basic parts that you can store data in so think about that as Data types in every programming language like character or integer or something like that Using this reference model you can go a level higher and use it in the let's say user space Where different users with clinical knowledge can take these reference model Piece of information and put them together into an archetype an archetype would be a let's say logical Yeah logical compounded Value that you can store in a database afterwards So meaning that instead of just storing a number you want to have the blood pressure You need two numbers and maybe some other units or something like that So an archetype would for example be Bob's blood pressure At the topmost level the templates are even a higher level collection of archetypes meaning that you can model even higher level Constructs like a visit so meaning that a patient comes to a hospital You are able to create how should a visit look like what archetypes what what parameters has to has a study nurse to record in order to Map everything to a common data format If you do that a template may look like that so pretty complex pretty big and it's More or less really hard to just look at the data itself So imagine that will be a really big json x and l file. It's hard to find the data in that So obviously the guys of open hr thought of that and created the archetype query language Which is a kind of hybrid between sql and x part that allows you to traverse the hierarchical data and Get your data that you want to have out of that So very nice. We have now the data in a common format that can be understood by other people So let's introduce another researcher called Carmen And she wants to get the data. She uh, no she first of all she does research on heart insufficiency Heart insufficiency So just the condition that Bob has and she now wants to have some data from the open hr repository show So if he she's capable of doing that She will specify some coin some kind of aql query Get the data out of the database that is set up at the hospital And is then able to use another platform or another tool to analyze the data that is called i2b to transmart Again i2b to transmart actually are two tools, but they are being merged right now together And this tool is a data a clinical data warehouse that allows to do some simple analytics and analytics on data Um, we will focus on transmart for this presentation because it's the tool that we run at the moment But in due time we will switch to the i2b to transmart merged tool Um for that you can see uh in this picture you can see uh, how this can look like so you can look at very basic statistics like the distribution of the age or the uh gender distribution So to kind of get a feel for your data that you have And you can even run more sophisticated analysis using r scripts. You can just create an r script write it Uploaded there and the analytics engine will give you the output that you desired uh, uh, we we use this tool primarily to um, yeah kind of do a data review So in this example you can see that there is a correlation between age and height And you can see that there is a lot of data points on the left But only one at the right. This is because the person that is depicted there has a height of 165 meters Which is let's say unlikely, but it just shows that um Data that you integrate from the heterogeneous it infrastructures usually are erroneous And you have to think about that and you have to keep that in mind um Okay, so let's let's say she uh had a had a research hypothesis Common that specified that did her research and was able to either approve or reject the Hypothesis she writes up a paper and everything is nice. Everything is clean And she submitted this Submits this paper to an open access journal What we like to see is that she opens up her data as well So in the spirit of open research open data, she should uh, yeah publish the data somewhere She could do that in a fathom seek instance located at the hospital run at the hospital Um fathom seek is basically a data repository, which follows the iz isa standard which stands for investigation study and uh assess Say thank you and uh Most importantly, it stores rich metadata with the data files that you can upload there So you can assign a license you can say this data This piece of data was conducted in this study using this investigation and so on and with that you can much better publish the data and uh, you're stored persistently so Even after several years, um, you can provide the same link You can call that link and you can see okay this data was used in this publication and I can maybe open up this publication to find more about the data or vice versa so Yeah, thanks to all the tools and the people that are involved that they were able to use these tools, uh Maybe bob can get healthy and all his fellow patients Can lead longer and happier lives um So for one hospital, this may may even be somewhere the case. I don't know not in germany more certainly, but um, What about sharing data with other hospitals? Um, this is something that is very hard to do For us, especially we're working on that project right now And we can see that there is a lot of data infrastructure on the global level that uh tries to do that to link They get data together so create this internet of data objects um There is a lot of work put into that in different domains, um, and For example in the medical informatics domain On the national scale, we have the medical informatics initiative which uh, yeah has offered us some money Basically to build this up for germany On the international scale odyssey or eden do basically the same with other data formats other technologies But have the same goal. So, uh, it's a very very good movement to see that that they want to link the data together And across domain, there are also developments that most of you people may know Like the research data alliance or the working group from the w3c data on the web Which has specified the doap protocol to Excess digital objects from different domains or from different it infrastructures So in order to create uh in order to use this data over The the boundaries of our hospital we have to think Especially in medicine about the security Um sensor medical data is very sensitive. Um, as we said we we are doing Masking of the patient data, but still this may be not enough if you think of rare diseases It's uh, very likely that's a doctor that knows a lot of rare diseases Uh or works in the field is able to find a specific combination of some diseases and no, okay I know this patient. I've seen that before because it's rare um So this medical data bears both high value for the research, uh, but also potential for misuse so We think that the benefit from linked medical data has to be uh, yeah exploited basically so we we want to have the data We don't want to have it accessible And do research on that to improve healthcare But what we need to do Would be to create secure IT infrastructures for that So not everybody should be able to just like that download everything and have it In a really big nice repository and do some I don't know big data analytics or something like that as he wishes But we do need some regulation and do need some yeah safeguards in place um and for To do so we need accountable and transparent workflows So not only we have to make it secure, but the patient should know where is my data going What is happening with my data where is research may be published for with my data in order to empower the patient um Yeah scandals like the the Cambridge Analytica Uh data leak or the leak what they did with the data doesn't really make it particularly easy to say go up to a Patient and say hey, can we have your data? We want to do research. So they're pretty hesitant on that um This also not all this this not only all this this not only counts for the data But obviously for the for the For the for the tools that we use themselves. So We try to use tools that are off the shelf are available for download But not every tool fits our purpose. So we have someone some sometimes to extend it or even build new tools and from our perspective this has to be open source tools because um This is the only way to really show Get transparency and empower the patient So things like public domain public money public code Show that we are going in the right direction and have a political Let's say a layer to talk about these things So we need that for medical research especially and we should emphasize that Everyone who's was working in this medical field should think about that that making not only the data open Just as is but thinking about the tools that you are using and making these open And the data flows you're creating So Global data infrastructure. We saw them. They are being built. Um, and we should Establish decentralized and free technologies to ensure secure it IT infrastructures Typically the medical information systems are really not Free or open source whatsoever. These are proprietary big blocks that we saw that are very black and nobody touches them But the tools in medical informatics research are frequently used and could be very well rolled out to more of the primary care But we have to advance that we have to politically Like make voice for that and say hey, there are tools you could use and yeah Please do so if you are capable of so we saw that some people from the medical domain here Raise your voice. It would be really a step forward to have more openness in the medical informatics research um, so for the references Obviously thanks for the team in Göttingen. We had a lot of guys that are very open to the whole open source community and we try to Yeah advise it on further use and Like like everyone basically said if you're interested contact us. We have jobs. Yeah, so thank you very much Questions doesn't work if I turn off the microphone. So any questions? Yes So the first comment was that we do not necessarily need open Resource tool or research tools, but especially open formats and open standards, right? So like opening HR presented that there are a lot of other standards fire omob cdm We agree totally so we we do need that to exchange data and enrich it semantically so other can reuse the the same data um, the second comment was uh, what the the question was um Where where is our field of application right where where we where we going with that? So basically as medical informaticists, uh, we're kind of in between everything. So basically we're trying to Help the researchers get the data analyze it and and derive research hypothesis from it We're trying to get to the patient to to to empower His view on his data and we're also trying to Go a little bit into primary care and say hey look at that We have these tools. We use this path to get the data from a to b and wouldn't it be a possibility for you guys in the it Yes Okay, the the question was um, there are similar movements in the Netherlands right now So basically what would we shout it out? We want to have the doing that in Netherlands. So great Um, please come to us after the talk. We would like to talk I guess Um And I I hope I think I don't know which project you are associated with there are some projects that do that internationally But I agree. There's not much communication. So everybody gets this grant and tries to build up something open saucy And then we have 60 different versions of open source standards, which comply very very badly Yeah, so, uh, I guess yeah, the the the answer is let's talk. We have to talk Yeah, sure Using So so your question was regarding the medical information informatics initiative in germany and how the four consortia that are being Funded their act together. So Yes, especially relating to the to the open source tools So, uh, we have uh, like yeah organization that goes above all of it and tries to increase the communication between Uh, but of course, these are these are professors talking about problems that have to be solved Um on a working level, we try to establish those I mean in germany, it's not a huge community. So basically everyone knows everyone somehow and we have meetups and then some things like that. So Um, we have to increase the talking bit in between. We try to do so and we we try to establish Let's say the the the standard for for open source is set I I've I've never seen a researcher in medical informatics who says now open source. We don't need that. Everyone is the same Um, but what we have to do is to deliver that more into primary care and to the people that are not Affiliated with medical informatics itself But maybe more in the medical it which are somewhat more hesitant regarding that


Okay, you can start. All right, so I'll start by showing my awesome Hello World app in red. Why not colors? Because I normally don't use ink. Okay. Okay, so it actually uses middleware or belt. And the app itself is just a pro. And I think one thing that really hurts me is I wanted to do an app that was just a bunch of pro's. But the really annoying thing was that the API for the middleware and for the actual app is slightly different. You can't actually do that because the middleware is automatically instantiated by the builder. You can extend the pro class. That would be one solution. Okay. And because it's all ruby color right, it doesn't actually check that you're passing the end thing. So one thing you can do is you can just pass whatever you want here. And it should still work. I think I need to restart it. It's actually still working here. Maybe just a little. So yeah, I guess the thing about red is that it's really super simple. And it was probably even simpler than I thought it was before I started reading the code. But I guess one of the biggest gotchas is that the middleware can be order dependent. So if you have something like Warden, it will be depending on you having some session middleware before it in like the middleware chain. Yes, I always said that slightly different APIs for apps and middleware seem a bit unwarranted to me. If we're going to go with a super simple API anyway, then I felt like we should have the same API. It was slightly obscure to me at first as well, but if you saw a config.ru file, the contents of that file is actually wrapped in a wrapped builder when the group is executed. And that is everything I had time to prepare. Cool. Anyone else wants to present something before I get to some more advanced stuff? No? See you next time now. Come on. Hello. Hello. Can I go on the table? Yeah. On the table. Okay. Yeah. Hello everybody. I'm new here. It's my first meet-up here in Singapore. I'm also very new to Ruby. I just completed a bootcamp which ended just last Thursday. So, wrap is something that we use, but I didn't know anything about it. So, I put up some present tutorials, read some articles, and this was basically the following tutorial by Ryan Bates, which was quite a long time ago.  What puzzles me is certain things over here. So, just like I read a documentation that you always need to finish your response. And this was what Ryan Bates did as well. But this is slightly puzzling to me over here because there's this. I'm not sure how this is different from this, I guess. Otherwise, I pretty much followed the tutorial and played around with it. I tried to do some buy-butt to find out what the environment was and I learned a little bit about that. And one thing that I tried on this rack app was by using this wheelbarrow, I guess, called Wheelbarrow. So, it allows me to make the changes to the app without me having to restart the server. So, this is something that when I use Sinatra or Rails, it does for me and I don't know what is actually going on behind. So, I learned something new. And then I, well, there was this auth basic which allows me to just do some very simple auth authentication without using what I have to create a user model or anything like that. So, could you increase the font size? Oh, sorry, yeah. So, I guess, how the authentication works is you just go through... Where was the... Are you... Rackup, right? Rackup, yeah. It looks like I've broken my app. Well, okay. Can you require a relative instead? Require a underscore relative. Dot slash lib slash my app. Yeah, that should work. Okay. Right. Is it 9292? Yeah. We go. And also following the tutorial, you can also change the name over here. For some reason, I've already authenticated it before, so it remembers... Incognito window? Sure, it works. It's asking for username and password, which I've set here. Oh, sorry. Not here. So, that's okay. 1002456. Yeah. Yeah, it's okay. That's my record. Pretty much. I can change the name. You can change the name between the display. Okay. Yeah, something like that. Yeah. What does it do that... So, reloader allows you to make changes to what we're starting server, but what is the zero parameter there? That's a cooldown period. So, by default, red reloader... I don't know how it works, but it checks every 10 seconds by default. Oh, okay. So, if you set a zero, then it just keeps doing that. I don't know how much processing power it takes that much. Okay. Yeah. Okay. Awesome. All right. Anyone else? Okay. At home, I have MacBook Air, and it has on the right side. Okay. First, I check this response class... Let's make it bigger. So, this response class, what it does is that... Rack, every app needs to return an array of three elements. So, first element is a status. The second one is the header, and the third one will be the body. And if you just return the response, you're not returning an array. You're returning the object of response class. Okay. So, if you just return the response finish, what it does is just... It executed some block, it did some stuff, and then it returns these three elements, the array that you need. So, that's why you need to call finish, or you don't need to always create the response object. You can just return an array of any three elements. Okay. So, I've got a simple app. Check if I can hold it. Give me a second, let me download this app. So, I've got applications showing that, basically, rack apps are nested applications. So, what middleware does, and actually, middleware behaves just like an app. So, you nest an app, and an app, and an app. And then you can do... Each middleware can do something before calling the next level or after that. So, I do here two things. I use two middlewares. First middleware is a random sleeper, and it will wait a random amount of time between zero and one second, and then it will call the application. So, first it will sleep, and then it will do... It will call the next app. And the second one is time measure. First checks what is the current time. Then it calls the next application. And then it adds the text. As you see, it adds two empty lines. And then some text that shows how much time it took to run this application. So, this code is quite lame. Basically, time now minus... Time now it will return in seconds. So, because this will sleep less than one second, it will be always zero or something. So, I just do... I multiply by thousand to show this time in milliseconds. And now my app, so that... Hello. So, the Hello app, the main application does nothing. It just returns the text. Hello. I haven't managed to see why I need to return error here. I don't know. It just tries to run each method on the third value. So, it needs to be an error, apparently, or a hash. I haven't checked it. So, it will return a status 200. It will return no headers. It will just return this body. And now I build my application using the right builder. So, first, I use time measure. And I need to use time measure first to start measuring time. Because if I do it this way, then first I will sleep, and later I will start measuring time. So, it doesn't make much sense. And then I run my Hello app. So, actually, I build an app that consists of three different applications. And then I run it. So, let me... On my private computer, it runs Puma by default, because I've got Puma installed here. I don't have Puma. The next server is fin. And the next one, in order, is Webrick. So, there is a hard-coded order in the rack that it will always take Puma whenever it's possible. Okay? So, it shows I responded in 400 in a second. And now I refresh it in 450 here. So, it will always sleep between 0 and 1 second, a random number, and then it will respond. Okay, so let me... Yeah, so this app is basically very, very simple. I want to show you now the server and the handler, what it does. Okay, I've got too many views open here. So, let's start with the server, the first file, the first post to read. So, this one, basically what it does is that it parses all the options that you pass to the... when you start your application. It doesn't do much more. It basically takes all the options, it parses them, adjusts them to the handlers format and runs the application. So, the server basically, as you see, it has a bunch of these options. So, you can do Evol, you can do Builder, Warnings, you can do some path additionally, et cetera, et cetera. This is super boring code. And it goes like this. And here, this is the end of that option. No, it's not... yes. Or this is the end of parsing that option. So, this method is like 100 times a code, just to parse different options that you can pass. Now, this one, I haven't really checked, and then we've got the start method. And the start method, you just call rack, the rack server.src, it creates the new instance of the rack server, and then it will... Then what it does, it adds certain configuration, and this is a super interesting thing. So, you can call a global variable starting with a dash, but if you try to call the local variable starting with a dash, it will not work. So, let's do it. Huh! So, it works only with one letter name. I didn't check that. Yep. Okay, I have no idea if it's some special attribute of Ruby, or not. Now, if I do something like this, it says that this global variable is not initialized. Now, if I do something like this, yeah, it just doesn't work. And let's try one more thing. Yeah, this one doesn't work either. So, the only case where you can start the name with a dash is the global variable, and it's only one letter. I guess it's some very, very old legacy code. We have a lot of weird stuff with it, also with numbers as well. Yes. There are some number global variables that stop at a random number. I have no idea. It's something that I learned often when I read the code of some libraries that are, you know, quite old. The code is quite old. I guess that... Yeah, I think that's... Yeah, the first thing... The first thing I reacted to was that everything was a bit messy, and I think everything going with the approach of don't fix what's not broken in the story. Yeah, I think so. I think the first commits are from 2009. Yeah, because this code is... Yeah, some of this could be refactored, could look way nicer, but it's just not changed. So anyway, you parse all these options here once again. So first, that method that I showed you before, it parses options and it generates a hash depending on what options you add in the command line, and then depending on these options they can change some settings. Change some settings. Now, it checks the KID, so it will... If this app is already running, if you provide the process ID and it's already running, it will run you an error, then this is building application. So this is the... I think this is the builder class that we used in this example that will build the whole app and it will return the application that consists of all the middle words that you provide. So this is wrapped up, then it will if you provide it so that it will work as a demo in the background. You reserve the process ID, and then in the end the last code that is the server run. So the server actually here, it's not the... This is not the instance of a server class. The server variable is the instance of handler class, which is... to me it was very confusing because I am in a server class here, so I assume that server variable is an instance of the class where I am, but no, it's a handler. So yeah. So now we get to the handler. And handler is an adapter between the application and the server. So as we know in Ruby we use server applications to run our app. So we can run... The default one is WebRub, but we can run Unicor, which is I guess most popular than Puma, which is one of few multi-fragment options, etc. And each of these servers needs to have a specific handler that will adjust to the rack's requirements. So we have on one hand, we have a framework like Rails or Sinatra that is on one hand of rack, or on one side of rack, and it needs to adjust to the rack's syntax and the way it works. And then on the other hand, we've got a server. So basically this allows us, it's kind of like a bridge pattern. It allows us to run any framework on any server. So Sinatra will run on Puma, Rails will run on Fin or WebRub, etc. Any framework that supports rack will run on any server that supports rack. Let's get to this handler code. So the handler class itself does almost not just determine which server you are using. It doesn't do much more, but it's quite a simple class. Basically if you will provide the name of the server it will try to guess what it is. So it will manipulate the stream, it will change all the letters to small, it will basically if you provide some if your name is in capitalized letter or you don't use the double commands etc. It will try to find what was the handler that you're trying to use. And this class, this module actually, it's not in the class, it's a module, based on that it will try to fetch this server, it will try to initialize that handler that you're looking for. So here we've got default value. If it fails to understand what you meant or if you do not provide the server whichever you want to use it will just try to guess based on the environment variables. So if you don't have any environment variables and I didn't have any of this set, it just picks the first available from the list of kuma, then thing, then the library. So in my case it was thin. I think it is installed with some either with some framework or with some library right now. That's why it picks it because I've never used it actually. So I don't know why it's in my system. Anyway so picking is that there is this function called try with require. So it tries to so it changes the name of as I showed you. It will change the name and then it will require it. But if it can't, then it will not raise an error. It will just go further. So that's why even though I don't have kuma, this try with require rescues the error and it will only if I, I think if I say if I add the warnings it will say that it tried to load kuma but it failed. So then it picks the next one and the next one. And only if the last one fails then it will crash. And then with that function register so registers all the all the handlers that come with wrap itself. So these ones are like CGI these are all, this is all stuff. So basically Rack appeared I think in 2009 or 2008 something like that. Before that there was no Rack so people were running Rails and in general Ruby scripts on CGI then Rack appeared as this module, as this piece between the server and the application or framework and I think the passenger server was the first one to support Rack so then everyone switched to this mode passenger that was actually the mode for Apache and then more servers appeared like Ruby Core and standalone passenger and I don't know, there are a few more like rainbows for example so anyway this module does, but it only meets the handler and then what's more interesting is that what this handler does so each of these servers has a bit different syntax a bit different options a bit different way on starting the application but basically the goal of the handler is to start the server with the application provided there so WebRick is pretty simple here we've got just one method that is called run and we determine the environment, a few options that we need so for example WebRick requires the port option so we check if it exists and if not for WebRick the default port will be 8080 then Rack has an option called host but WebRick the server doesn't recognize it, it uses bind address so we need to change it and basically this is what we do in a handler, we transform the Rack syntax, the Rack options into the syntax that server understands and then what WebRick does is that it basically creates the new options with this server, it mounts the application under the root path and then it starts the server so I will show you one more that is fin because it's a bit different so fin doesn't mount application application is one of the arguments that you start when you create the server so we put the host port application and then options so it's a bit different in that but fin as you see it's like what 630 this is the whole class, there is nothing more 36 lines of code so this is once again very simple and then the last one that I have is Puma and Puma is a bit Puma is a bit longer so Puma has this whole configuration object here so we need to we basically create a configuration object that will, this is not a hash it's a custom class and then this comp goes here so this comp contains everything this comp contains host and the application to see if it has custom methods ah ok it has custom methods that are port etc so yeah ok then we create this launcher and then we have a bit different method we call it launcher run in other cases we have launcher.start any questions so far nope, so actually we run server first before it runs the handler because I was thinking that the handler picks the server and then it runs server.rb so it's the server.rb that calls the handler yes, yeah that's what it does so server here it chooses the handler so server in the end server.run calls handler.run yes that's it regarding the handler then I guess I will show you a few more views so here in this middleware you've got there's no middleware library here so it's hard to let them know what is really middleware in the in the rack in the rack part what you can do is you can go to rails and then in the action pack action pack consists among others of action dispatch and here you can see how different middlewares work not all of them are rack middlewares they are just from here but for example one of the bigger ones is cookies so here you've got the method call and as you see it returns in the end it returns status it returns header returns body so what cookies middleware does in rails first it calls the application and then later only later it updates the it updates the HTTP header and if you see the line line number for 570 something so this a huge there's lots of documentation but it's quite a big application here there's lots of stuff that is happening yeah it has bunch of custom modules and classes etc I think that this itself could be some separate library I've seen libraries that are 50 lines and this is 10 times bigger so you can later check the other stuff remote IP for example this I did not yes so remote IP is simpler it basically provides you IP yes so this is what it does in general this is the core of it so it adds the IP to the environment so that later you can call request.remote IP your application so all of this stuff all 200 lines of code what it does it calculates it transforms the IP to the to the format that we can understand so Rails I think right now the default stack if you do not remove any middleware it runs around 60 of them so there are 60 levels of Rails application before it gets to your controller so it will add this IP it will go through SSL for example at some point the middleware will not call your controller it will just return something up so for example you've got a middleware you can have a middleware that will return something instead of going to the controller just because you want it to be super fast this middleware somewhere between your two middlewares in the Rails app so you want the middlewares that you need to work and then the ones that are not needed you can just skip your middleware will not call the app app.call it will just return everything up I'm not sure if I know any example here the static one no it does it calls it so for example SSL you require SSL and if your request goes for SSL then you will call this application but if not you will try to redirect so if a user makes request and they specify the HTTP protocol it will not get to your controller because it will need to go for all the 60 or something layers so what it will instead 10 layers or so and this layer will stop it and it will try to do something else I use for example one case that I use middleware is translation file so we want to fetch the translation file file that consists of all the keys and phrases for certain language and we want it to be as fast as possible somewhere up in the in the list of middlewares that it doesn't need to go through SSL, middleware it doesn't need to go through a bunch of other stuff but because of that it returns in like 5, 7 milliseconds instead of standard 50 or 100 here is an example of the middleware that does it's not really a middleware oh no it does it also does up.com I think I've seen one that doesn't do it so it seems that not all of these are really behave like middlewares but I might wrong I just thought that I've seen something that doesn't behave this way ok and I've got one more thing to show which is the I didn't mention it in the description I'll take check this later and it appears to be interesting to start. So Builder is a class that allows you to build the whole application that will consist of multiple apps and middlewares this is what I use in this example so this is what I use in this example it's basically it's a small DSL that that composes the application. So as Ted mentioned it's already dependent so it processes line by line and then it runs the application in the end so let's see how it works here this is quite complicated and to be honest I do not understand everything here because it's a DSL so it tries to decognize it basically parses the code that you provide there and okay so first function is use and we've got this use instance variable that will keep all the apps that we it will keep all the applications that we are using all the middlewares that we are using let me try to find where is the first time that it appears so it starts as an empty array and then there are huge more things here so I don't know what this map does here I think we can map it to a certain route Ted idea map is for routing yes but this is you can provide map for use so I think you can map so when you map route you can use custom middlewares for that routing but I'm not sure let me go the beginning so you get map that starts as new and where do we assign it okay yes here is the example so see so we can so is it so that the routes will be available to the middleware no no this middleware will be run only in this in this route yes so for example you can do something like map admin use yeah something like that so this builder allows you to specify middlewares not only on the top level you can do top level as well so this one will be available everywhere and this one will be available only not available it will be run only on this route I don't know if it's possible to nest the mappings because let's say that I want to do admin here and then I want to do something like map to the user and that I want to use another middleware but I'm not sure if it's possible or if I need to do it here and then I need to repeat those stuff so I'm not sure about it anyway we were in this method called use so this method will add this particular middleware or app that we provided with all the arguments to that is so then it will be all will be called in an order that we provided now so this is use and there is map so map only only creates this variable and assigns this block so executes this block in the path that we provide here so this is the second method and then the third one was run so run basically assigns this application the middleware application that we provided as the main one and then and then this is what creates the application in the end so if we have some map we generate it so this generated map will be our main application and if it's not we just have the main app so in my case I can just do something like this because I will not I will not be using be there I can do it without or with the builder but if I don't have any maps like in this application I will just use this application then what we do oh I have no idea what this particle does if someone knows who does then we go I don't know why it reverses it so this use will be consisting of all the middlewares but then we reverse it so it starts from the last one and then we inject ah of course so in a second ok so we need to run we need to go to the last one so that we will have something like nest how to explain it so that the random sleeper will have the hello app as it's up to run so it will be something like this ah right something like this so that it will take ah we will initialize it with this argument and I think we need to do it from the end because if I want to provide the hello app as the new hello app the instance of hello app as a application to be called by a random sleeper I need to create it first right so first I create this object and I provide it as argument here and then because I create this one I can provide it as argument here to the time measure yes exactly yes so each next application is an argument to be run by the previous one yes ok so that's why ah that's why we will burst it and we inject it then starting from the last one and then we've got then this is our complete application and in the end we have the warm up don't know what it is no idea does it ok ok ok that explains a lot ok so I believe that we want to ah before we start serving the request we need to do something for the application so for example imagine that your application wants some data before it serves it to user so maybe this is for warm up or even caching or even caching caching yes exactly yes so this is like a preparation for the app on the start of time before it starts before it starts serving the request and this one ok so this one maybe it checks if the application starts successfully because it creates a mock request and it it just checks I don't know this is just my guess I've never read this option before so anyway we call this and then we return the application and this function to app is called in builder so builder.app will basically create an instance of this builder that will later that will later create all this nested all this nested structure and now let's see ah ok so this is already no I never call builder.app in my code so I don't know where it is ok I do builder.new if I do builder.app will it work is it just an alias no it's not an alias because I do new but then it calls to app but I don't call it manually right? here if I replace with new I do not call to app so I have no idea ah ok yeah so if I provide just new and then I provide block so this block will be evaluated within this instance ok so what I do is ok let's see if maybe run does it no no ok I will just here if someone figures it out can you later let me know write some comment I don't know why it's used up here and what why it works with the new if it's the same with app ok I think this is all I prepared for today any more questions or I'm curious now why my application works because even in Aaron's talk right he said that the body must respond to each but in my app it works even though I just pass a streaming it works you passed a stream but for you it didn't write it didn't for me as well so I wonder if I it's more forgiving or ok or if I move all the middleware I have one middleware ok let's see it doesn't work it doesn't work you see which server did you get which version the newest one all the newest versions I have no idea ok let me just double set 1.6 for 0.3 ok how do I provide a hand here let's see if I do it with web read but what is the option now step also says each was a master for training in Ruby 1.8 I am removing Ruby 1.9 but I'm not using Ruby 1.8 yeah exactly yeah yeah I have seen 1.7 though I don't like it ok let me start with web read yeah because I know Aaron said for sure that the body must hear something for me it's human no same here ok let's let's see if that will help ok 1.7 you see I'm very scared I broke the wrench if I just do yeah it needs to take the environment as an argument as an argument I guess ah what did I do wrong I'll need the curly bracket what you need to cure the brackets for your ok no idea man this is so scary ok what if I do without the bill there what if I do just this way no still the same are you sure you didn't override the string class to respond to each it's still working we fined 1.7 ok no idea you don't like the string too try and see if I do something different ok I suppose it's supposed to work right if in B1.7 it's supposed to come to each master I'm not running I'm not running Ruby 1.7 for sure runs Ruby 2.3 the version of the server is 1.7 as well but it's not Ruby 1.7 yes 2.7 yes still works for some reason but they run Ruby 2.7 Ruby 2.7 I've got 2.15 let's try running 2.3.0 ok ok this one I will run it shouldn't make a difference oh wait wait no no I think change yeah that's positive let's try once again hey it works Ruby 2.3 makes it let's try with Ruby I'm just glad I didn't you didn't go crazy yeah ok that's good this works weird what? now we have another question is it about the basic object being like a range or something the rate of limitation of each but strings should not be infinite each even in Ruby look at them up here each it doesn't no ok so in yes something behaves something behaves weird here we're feeling maybe has some special case like if you're using Ruby I don't know I don't know I will check later you know which question you used before when you did it 2.1.5 yes no here's the same ok we can check in the rack of actually but I didn't know what is that by the way where is that each it's in the body proxy but not here what if you don't just make a special case to address the problem to what this method is a special case to address the problem or we are playing a special case for each of you to save adding too many methods but why was it added what was the name of the problem 434 yeah so the method is yes there is a method missing here or no but the method missing is on the body proxy class not on the stream class ok so it's on the body proxy thing so it shouldn't work without a builder and you returned just straight stream you didn't use the response class so you used the response class correct so somehow it turns the stream into a body proxy yes somewhere but I don't know where the thing is that the body proxy that each calls the body dot each alright yes yeah I think we will not figure it out now just leave it as an exercise ok anyone has any more questions or want to show some piece of code or it's gonna be ok cool thanks for coming then thanks to thinkabox and then for hosting us we will have the issue on the github to ruby sg so that we can decide about the next library so far we've covered sprockets concurrent ruby and today thanks once again thank you


about to begin today's webinar. Please stand by as I connect the line. Ladies and gentlemen, welcome. And thank you for joining today's PIDB virtual public meeting. Before we begin, please ensure that you have opened the WebEx participant and chat panel by using the associated icons located at the bottom of your screen. Please note all audio connections are currently muted. If you require technical assistance, please send a chat to the host. With that, I will turn the meeting over to Mark Bradley, Executive Secretary of the Board. So please go ahead. Thank you very much, Michelle. Good morning, ladies and gentlemen. Welcome to another public meeting of the Public Interest Declassification Board. Of all the public meetings, the board is held over the years. I think you'll find this one to be one of the most compelling and thought-provoking. This meeting is going to focus on a very important collection of historically significant records that trace their origins to a time when our country was locked in a nuclear arms race with the Soviet Union when we were testing atomic weapons in and around the Marshall Islands. Several months ago, Congress asked the board to conduct a feasibility study on how many records still remain classified about these tests, what it will take to declassify them and to make them publicly available. The board's conclusions can be found on the PIDB's website. And these conclusions have been sent to Congress and to the Secretaries of Defense and Energy. What the board will discuss this morning with the help of guest speaker, Walter Pinkas, formerly the Washington Post, and a scholar of the atomic testing on the Marshall Islands and around it is a microcosm of what the board has been saying for years. Namely, that our classification and declassification systems remain in dire need of reform because they are inefficient, too costly, and produce records that are too scattered to access. With that, I now turn this meeting over to Ezra Cohen, the PIDB's chairman. Ezra? Thank you, Mark. As Mark noted, in FY 2022, the National Defense Authorization Act included a section that mandated the public interest declassification board conduct a feasibility study on the declassification of information relating to nuclear weapons, chemical weapons, or ballistic missile tests conducted by the United States in the Marshall Islands, including with respect to cleanup activities and storage of waste relating to such tests. The PIDB conducted a high-level study rather than a record-by-record assessment. This study includes the status of potentially responsive records created by federal agencies, departments, organizations, units, entities, national laboratories that participated in any aspect of nuclear weapons testing, environmental and human radiation testing, containment, storage, and cleanup activities between 1946 through 1989. As part of the study, the PIDB held meetings with stakeholders, including agency staff from the Department of Defense, the Department of Energy. We met with congressional staff, congressional committee staff, State Department officials, representatives from the Republic of the Marshall Islands, historians, researchers, records managers, and archivists at the National Archives and Records Administration. The NDAA required that the Secretary of Defense and Secretary of Energy support the study. DOD and DOE provided information and records that may be responsive to the study. The information provided at the record series level includes responsive component record type, volume, and classification level. This information will be included within the study. Lastly, the study took over about 600 hours of work. The staff of ISU and of the PIDB did an enormous lift here, and they did a fantastic job. And I, in particular, like to recognize the efforts of Sharmila Bhatia and Beth Fiddler, who were instrumental in putting this study together. As always, if you have any questions related to anything you hear today or on matters of the classification system in general, please send your questions to PIDB at narrow.gov. If we can get to them during the meeting, we will. If not, we will respond to them on the blog. And again, the email is PIDB at narrow.gov. I'll give a brief historical background. Well, actually, we're going to have Carter give the historical background. Carter, go ahead. Yeah, thanks, Ezra. As Ezra mentioned, this is a brief background to provide context for this report. Between 1946 and 1958, the United States conducted more than 50 atomic and thermonuclear tests in and around and over the Marshall Islands. And as Mark mentioned, these tests occurred on the heels of the use of the atomic bombs and Nagasaki and Hiroshima against Japan at the conclusion of World War II, and occurred during the nuclear arms race with the Soviets. The tests were conducted on the Bikini Atoll and on or near the Nuitak Atoll. The Castle Bravo test, which was conducted on March 1, 1954 on the Bikini Atoll, was the largest test. And the detonation significantly exceeded the size that the government expected. Bravo was a thermonuclear fusion weapon, the largest weapon detonated by the United States. And as Walter Pinkus writes about in his book, the blast was equivalent to about 15 million tons of TNT, or more than 1,000 times more powerful than the Hiroshima atomic bomb. While the Marshallese were evacuated from the Bikini and Nuitak Atolls, the radioactive fallout impacted the inhabited Atolls and required an emergency evacuation of the residents. The inhabitants were exposed to radioactive fallout and not just the inhabitants. One of my favorite parts of the book was about a fishing trawler that the Japanese somehow wandered into the zone, ironically, called the Lucky Dragon. And I'm looking forward to hearing more about that. But the inhabitants of the Marshall Islands were exposed to radioactive fallout. The Department of Energy administers a medical program that serves those who are exposed to radiation during the Castle Bravo test. And the program provides medical care and treatment. The Marshallese were displaced from their homes and forced to relocate. The contamination of the land and the waters there also limited their ability to rely on agriculture and fishing for their food and livelihood. In the 1970s, the US constructed a containment structure on Runit Island, which is located on the Nuitak Atoll. This island is inhabited. The dome there contains over 100,000 cubic yards of radioactive contaminated soil, metal, concrete, and testing debris, which is encapsulated in a concrete inside an unlined nuclear test crater, which was created by the Cactus nuclear test in 1958. The cleanup activities alone exposed more than 6,000 military service members to radiation. Now the US is required to monitor the containment structure. And this is assigned to the Department of Energy. But the Marshallese are concerned that there is future risk with the rising waters. And as the dome itself begins to deteriorate, that's a bit of background. And I'll now turn it over to Alyssa Starzak to talk about some of the issues about our review. Thanks, Carter. So the PIDB has identified many challenges for a declassification project of records relating to nuclear weapons testing and cleanup activities in the Marshall Islands. So to begin with, multiple agencies have legal custody of potentially responsive records. Both the Departments of Defense and Energy and the National Archives and Records Administration have records. The challenges for any review and declassification project broadly include structural, organizational, and policy issues. Indeed, as Mark Bradley identified at the beginning, the challenges we identified highlight the overall challenges we have with declassification and public availability of records. Just to give you a sense of some of the challenges we witnessed in doing our review, we saw a requirement to search large volumes of records, including non-textual records like film for individual responsive records. We also saw that there was a need for technology and secure systems to process and review records, particularly ones that might be classified. And then there was a need to develop a process for multi-agency declassification review of records that have not been a session to the National Archives or are not included in the National Declassification Center process. To undertake a project like this would require new and dedicated resources. The project would require millions of dollars for technology, secure communications capabilities at various facilities across the nation where the classified records are currently stored, project management tools, redaction software, and web design and hosting. The project would require staff with security clearances to perform searches in digitized responsive records, conduct public access and declassification reviews, and design and manage the public-facing website. And those are just a sample of some of the challenges that we face. I'm turning it back to Ezra. Thank you, Alyssa. As Alyssa mentioned, through this study, we really found very real and present examples of how the classification system and the tools that we have to increase public transparency need to be modernized. With that, we'll turn to Walter Pincus, who joins us today. Thank you, Walter. Walter Pincus is a contributing senior national security columnist for the Cipher Brief. Mr. Pincus was a reporter for The Washington Post for over 40 years and covered nuclear weapons and arms control and politics and congressional investigations. Last November, his book, Blown to Hell, America's Deadly Betrayal of the Marshall Islands, was published. His research began in the mid-1970s when he traveled to the Marshall Islands with Dr. Robert Conard, the long-time head of medical surveillance for the US Navy, and then the Atomic Energy Commission's Brookhaven National Laboratory. Dr. Conard was tasked with monitoring the health and treatment of the Marshallese, who were exposed to the fallout radiation after the Castle Bravo test. Walter? As a risk, thank you very much. I should say that my interest first was brought to the Marshallese back in 1966, when I was covering Congress in a past a bill that ended up giving each exposed person on a wrangler battle $1,000 in cash. And to get the money that had to sign a statement, say they would never again seek money for the US from their exposure. 82 people, men, women, and children. And I hate to admit it, I've covered it ever since, because I was intrigued by that story. My book was the result of two eras of research separated by more than 40 years. First research took place in the 1970s, when I read congressional hearing records, saw some original documents, interviewed many people. And as I said, I was involved in a trip to the Marshalls in 1974 accompanying Dr. Canard and his group on their annual medical examination of the Rongo app survivors of the Bravo test. And we also had a stopover on Bikini. After that, I wrote, I hate to admit, a 60,000-word article for the New Yorker magazine, which had paid for my trip to Rongo app. It went into galleys, but was never published in the magazine. At that point, Mr. Sean, the editor, said it had to be broken up into two articles. And he saw no opportunity for oncoming magazines for that much material. As a result, years later, it became the foundation for my book, Blown to Hell. The second period of research, which went on for almost six years, as I wrote the book, at the same time I was in the final years of writing a column for the post, it took place almost entirely online, although there were some interviews also. Most helpful initially was the collection of energy department documents released by Secretary Hazel O'Leary during the Clinton administration. Early on, I discovered a document which inspired me to really keep going. It was a previously classified transcript of an unusual three-day meeting held in early October 1967 called the Second Interdisciplinary Conference on Selected Effects of General War held in Princeton, New Jersey, and sponsored in part by the Defense Atomic Support Agency. They had held a conference back in January 1967 with a group of distinguished current and former government scientists. At that time, the talk was about the effects of the atomic bombing of Hiroshima and Nagasaki. The conference that I was interested in, again, had a similar group of scientists, and it discussed specifically the fallout from nuclear testing with the focus on the Bravo test, the resultant radioactivity contamination of Marshall Highlanders and Japanese fishermen. Participating in that conference's discussions were familiar names, Dr. Charles Dunham from the 54 Bravo test, Dr. Canard on the effects of the fallout on populations, Dr. Lauren R. Donaldson from the University of Washington on the ecological aspects of weapon testing, and Dr. Meryl Eisenberg of the psycho-social reactions to the bomb. The discussions, as would be expected, when scientists gather together, broaden out to include much more than just fallout and medical aspects of radiation exposure. The transcript, I had to admit it, ran 423 pages. No surprise, the participants at the time went beyond simply reviewing past scientific circumstances and events to include questioning why there is so much public misunderstanding about radiation. They also talked about whether it's caused misunderstandings by government officials who had trouble in difficult situations telling the truth about what had been going on. For example, it was agreed that the government had not told the truth initially about what happened to the Marshallese on Rongelab Hattall and Uderick Hattall. Rongelab 120 miles away from the explosion, Uderick was 300 plus miles. Nor did they assess the impact of the radio activity in the fallout. One exchange began at this meeting. And Dr. Canard complained, quote, when you talk to a public group, it's obviously just don't understand the simplest things about radiation, unquote. As I continued my research, more such documents turned up from amongst the newly released O'Leary materials put much more light even backwards on operation crossroads tests that had never been made public. In my original New Yorker Draft article, crossroads was less than three pages. In my book, it's nearly 90 pages. Most of that material came from DOE Historical Collections. The most important was a site, www.osti.gov. I have to admit, sometime in the past six years, historic documents available through that collection have changed. Fewer seem to be available. I found that out back in 2020 when I was preparing footnotes for my book and found materials I had gathered earlier and quoted from were no longer located on that website. One problem may be the changing nature of search engines. I found over the years, Google can react differently on different days and challenge the fine materials using the very same search words. Things are still changing in terms of what's available. Yesterday, on Wednesday, I should say, while preparing for today's meeting, I decided to test the web and see what a new researcher could find. First of all, I found Wikipedia and other sites use a site and a DOE site called www.wrf.eh.doe.gov to guide people to Project 4.1, the medical follow-up to the exposure of the Mars release. That site no longer exists. However, if you go to www.wrfi.gov, it's a new site from what I use and use their current reports channel in Earth 3M. There are items I never saw listed there in my research, even in past years. Some, too at least, were interesting enough to me on Wednesday to read through them yesterday. And I actually found items that I plan to use in my Cypher brief column that will appear next week. There is no guidance on the index other than the dates that were written to show whether they're previously classified or just part of public reporting of national laboratory work that has not been picked up previously by any of the media. I went through such an instance about writing about Los Alamos and my own history. And this is one of these things about classification I've always felt needed much more work as you people do also. Back in 1977, after I wrote the first stories about the so-called neutron bomb, then Los Alamos director Harold Agnew invited me out to the laboratory, gave me access to the Los Alamos Library, and put a pile of recently publicly released Los Alamos studies on my desk, all of which involve nuclear weapons in one way or another. When I returned to Washington, I wrote a series of stories for the Washington Post based on materials in those Los Alamos studies. One day after the second story was published, Agnew called me and told me he'd been ordered to reclassify all the studies he had shown to me that were previously not class. Let's see what happens next week after I publish what I read in the news studies. There's a wealth of information on nuclear weapons already publicly available and still to be mined. The questions in my mind are whether there are researchers interested enough to search for such material. As important, is there a public out there that cares enough to read what's written? For myself, as I approach 90 years of age, nuclear weapons, their past, present, and future of all subjects I believe are extremely important. For me, the Marshall League is not only represented a means to tell a story that should affect us all. They represented, and still do now, how the US used those Marshall Islanders to error. And therefore, the US has a continuing responsibility to care for them now and in the future. And now if you have any questions that we can try to answer them. Thank you very much, Mr. Pincus. That was fascinating, especially your comments about what's classified and declassified seem to shift over time. I really appreciate you sharing that with us. Ezra, can I interrupt you and tell you one more panic? Absolutely. Go ahead. Part of my odd past is in the 1960s. I twice ran investigations for Senator Fulbright's and the Center for Foreign Relations Committee. And in the first investigation, which was of foreign lobbyists, I had my first experience with classification. And we held all our hearings in closed session. And when time came to make them public, we had to get them clear. And said, I ran the process with the administration representatives to clear the records. And I still remember the first episode, which I had to do, with the Philippine lobbyists. And meeting with the State Department. And at one point, Chairman Fulbright had said during the closed hearing that the then President of the Philippines, Ferdinand Marcos, was a crook. And the State Department wanted to classify them. And if you ever knew Fulbright, he would never allow that to happen. And we ended up publishing it and a lot of other material that we fought over, and the world didn't end. That was my first experience with how arbitrary the whole system is. And as you mentioned in your report, the whole thing ought to be studied. Well, unfortunately, what you just described there is something we see and repeat itself a lot throughout the history of classification and declassification. But Michael, do you want to go ahead now with some questions? Sure, thanks, Ezra. And thank you, Mr. Pincus, again, for your briefing and your comments on the book. Do you see any comparison between the way the US government addresses the Marshallese and the present day with the development of modern weapons systems and the level of transparency to the public? Well, the modern weapon thing, which I also write about, I have to admit I've written about this stuff for so long. There are things that shouldn't be discussed when it comes to weaponry. And part of it is that people don't understand what weapons do and what they can do. And we're dealing now with a great group. I'm amused with all the interest, I'll give you an example, with hypersonics. We're now talking about hypersonics. The Chinese are the Russians ahead of us. We have to rush. Back in 1969, both the US and the Russians developed hypersonic weapons. They're just faster than the weapons we have, except for the forehead of an intercontinental missile. We can't stop them easily. And hypersonics are the same thing, except they travel at a lower level. And back in 1969, we decided, if you couldn't stop the incoming warheads, you couldn't stop hypersonics, we're no more difficult.  But they're really not much different from what we all ready face that we can't stop if there were a massive attack. It's a long way of saying people describe weapons doing more things than they can probably do. In this case, I sort of trust the military more than I do sort of exposing more of what a weapon can do. Thank you. I just have another question. You mentioned that in your remarks that there's still much to be mined. What would you recommend for a researcher who wants to take this up and follow your footsteps? I don't know. One, you have to have an interest in it. And I think I used to have trouble getting the post to print stories around nuclear weapons because I thought in the public's mind, the only thing I care about are numbers. Who's got more, who's got less nuclear weapons? If you grew up back in the 50s and 60s when there was still atmosphere testing, every nuclear test was a front-page story for days. One, when it blew up. And two, when the fallout traveled across the world. Once we went underground, nuclear weapons tests became bigger in some of them. And there were two paragraphs in the back of the paper. Nobody cared about what a weapon could do. They cared only about how many it had. And whoever had more was stronger. I always, ironically, thought the underground, the atmospheric test band was a detriment to people's understanding of how terrible nuclear weapons really are. And the whole lesson of fallout, which people don't understand, even back in the early days, they thought, well, if you stayed in your basement overnight and then came up, things would be fine. People don't understand that you can't eat. You have a danger of living in a wrong lab, even though they've been cleaned up twice because you can't get rid of the fallout. The understanding of nuclear weapons shouldn't be Hiroshima and Nagasaki, where by design, Oppenheimer and the other scientists detonated them 1,800 or 2,000 feet above the ground so the firewall wouldn't hit the ground or you didn't have fallout. You could go back, live in Hiroshima and Nagasaki, rebuild it. What people think about with nuclear weapons is Chernobyl, where nobody's going to live there for 40 years. And that was a minor explosion when it came to a weapon. Were there records that you would have liked to have included in your book that were unavailable due to the classification issues or privacy concerns? Well, it may be classification. It also may be that studies are put out by the labs and by people and don't get publicized so you don't know they exist. My experience Wednesday is a study that came out of Los Alamos in 2018. I don't know when it was posted. But it's like a cliff notes in my book. It reads exactly like my book. They've got a whole bunch of same examples I use. They clearly had the same material and more. I never saw it. Thank you. I have just one question and then we'll ask the other board members. But Mr. Pincus, what was the contemporaneous view of the US government when you were traveling around with Dr. Canard throughout the region? What was the view at that time of the injury and the effects on the Marshallese? I've got to be honest. We went to Bikini and the makeshift event there was we were examining new homes that have been built by a Marshallese company for the return of the Bikini and it's the first return.  And it turned out the Marshallese had been told they had to import the sand used in the cement. And to save money, they used Bikini sand and they had to tear those houses down. And Rongelat, my attention was drawn not just to the examination. First thing is that one of my first memories was there was a Peace Corps volunteer living on Rongelat at the time who'd come from San Francisco. And she spent many hours complaining to me about how Americans were destroying these wonderful natives who should have stayed in their original genre and we brought gambling and liquor to the members. That was bizarre. Since I didn't speak the language, of course, I couldn't talk. The only people I talked to were the Marshallese representatives on the territories group. And they understood Dr. Canard, as people know, was caught in this terrible dilemma of trying to deal both with the arrival of thyroid problems. Every child under 19 had to have their siroids removed over a certain period of time because the radioactive diodine was strong with the siroids. And at the same time, deal with normal health. And there were people, including the Japanese anti-nuclear people, who were claiming that any kind of illness related to the bomb were very anti-American. And so they were using the Marshallese. So you had this feeling that there was anti-Americanism being promoted. And it just begun to infect the politics of the Marshall Island and still does. So I mean, Americans can understand how people can misuse facts for political reasons. And so there are, we have to face that in dealing with the Marshallese. But the fact is that we made an error in blowing up Bravo at the time we blew it up. And is the irony of all the things in the new study I read? Yes, sir. It is an honest acknowledgment in one of the studies I read, not that we misunderstood the wins, which we did, at the highest levels because we hadn't had that experience, but also we had an idea that the radioactivity would be captured in the stratosphere and would never return to Earth. And that was one scientific belief at the time, which was totally wrong. So we have a debt to pay for our mistakes. But our mistakes are geometrically promoted among the Marshallese in order if people can make political gain or I hate to say to get more money. And that's the problem. Thank you. Thank you for answering that. Do any of the other board members have questions that they'd like to ask? John Tierney, go ahead. Well, thank you, Ezra. Thank you, Mr. Pinkes. It's always a pleasure to hear you respond upon your past work and your ongoing activities. Given that the nuclear issue never goes away and certainly has diminished in the attention that a lot of people are giving it over the last couple of decades, I would be curious to know what you think would be a way to re-engage the public so that they're aware of this existential danger and the way that it connects with the environmental and climate change dangers that we have today so that we can get the public to raise its voice once again and get active on this issue. Another part of my ridiculous past line is I worked with CBS and did a documentary at which point we tried to show briefly what would happen if one nuclear weapon hit the United States. And at that time, I chose Omaha, which is where SAC was located. I don't know if you remember, there was an ABC program called The Day After. Sir. It ought to be shown once a year to remind people. I taught a seminar here at Washington for Stanford University and was always amazed how little students knew about nuclear weapons. And Dr. Perry's out there, and they talk about it a lot, but they don't talk about the impact of its use. Everybody talks about tactics and nuclear warfare, which is bizarre. But just to explain what one weapon could do, I mean, if Congress would just focus on that, there was never a study done, by the way, or an investigation of Bravo. Never. And the scientists at that 67th meeting agreed among themselves. They didn't understand why there was never any discussion of the mistakes made or the fallout. And it's probably worth redoing, re-looking at that. Thank you. Thank you, John. Any other questions? OK. Mr. Pinkas, thank you again. We really appreciate your insights. And we hope that this study will help the next generation of researchers really begin to dig into this issue. So what we're going to do now is we're going to go ahead and we're going to go through some of the findings and the recommendations that we make in the feasibility study. I'll just say that, again, as Alyssa mentioned, what we discovered here is that what we've been talking about, what the board has been talking about for many years, that the classification system and then the declassification system are well overdue for major reforms. And some of this can be solved through technology, but there are also major systematic changes that need to be made. With that, we'll start with Ben, who will go over the first challenge and recommendation that we find in the report. Sure. Thank you. First, to set the ground in terms of some of the granular details on the records, the records that may be responsive here are in the legal custody of the Department of Defense and Energy and the National Archives and Records Administration. The records are located in various locations throughout the country. For NARA, the locations would include facilities in Washington DC area, regional facilities, and the presidential libraries. DOD has records located within various components throughout the country. The Department of Energy has records located at its headquarters and the national laboratories, including Brookhaven, Lawrence Livermore, Los Alamos, and Sandia, and the Nevada National Security Site. In addition to these challenges, the records could be contained in larger collections of records that are not necessarily related to weapons tested in the Marshall Islands. I would commend to everyone Appendix D to the report, which goes into some detail in terms of identifying potentially responsive records in greater detail, and gives an idea of the scope of records locations across and within departments. Finally, as noted in greater detail in the report, the cross-agency equity reviews will be a challenge. Many of the records in the custody of agencies require equity reviews, and while Executive Order 13526 provides the authority for agencies to establish declassification centers, agencies have generally chosen not to establish such centers. So it will be, as laid out in the report, a challenge to set up a process for internal or cross-agency equity reviews, and the report discusses some of these past challenges that have occurred in terms of referrals between agencies for these. With that, I think I turn it over to John Tierney to discuss some of the volumes of records. Thanks a lot. I mean, I know this is somewhat boring for people that's not listening to numbers, but it's instructive. When you think of the magnitude of things that are involved here, the Department of Energy estimates 118,000 and 10 records that might be responsive. That's over 66,000 of the classified, 782 that are controlled unclassified information, and 50,656 that are unclassified. These records are in a variety of formats. They're paper, they're digital, audio-visual, and there are different levels of granularity that are reported in the process here. There's a document level, there are reels, there are boxes, so it goes on and on in that sense. The Department of Defense has estimated 16,000 FRC equivalent boxes. So that's totals an estimated 35 to 40 million pages that might be responsive. The classification level ranges from unclassified to classified, and the Department of Defense components provide different levels of granularity on their reporting. Some talk about the document level versus the page level versus the session box level, and on and on and on. So it's a very complicated and huge project to look at. Paul, I think you've got more to say about the cost. Paul Noel. Yes, thank you, John. And also thank you, Mr. Pankas, for your research and your book. We appreciate it. It's a big help. So what is all this gonna cost to do a declassification study and review? The Department of Energy estimated the cost for reviewing projected 3.8 million pages. To do this, they would require additional staff and they based their projections on hiring 18 new contractors and two new federal employees. The Department of Energy estimated it would cost $134 million at about $35 a page and take 23 years to redact 3.8 million pages of classified Marshall Islands documents. In addition, the Department of Defense, DOD, estimated it would cost between $60 and $80 million for Marshall Islands search, declassification review and public release process. These funds would support a staffing increase for program managers, records management personnel, digital archivists, records research analysts and declassification personnel. The DOD would also require technology investments. DOD anticipates classified records would have to be transported to the Washington, DC area for digitalization, review, declassification and public release processing. DOD would also require funding to either improve or build declassification collaboration and workflow systems. This would be a multi-year project which they estimated would take about five years. And finally, there's these clearance costs. Background clearances can take one to two years to adjudicate with the current backlogs. Currently it costs $5,410 for a tier five top secret SCI clearance and an expedited clearance costs $5,845. Unless clearances are ordered at the start of a project waiting for them can delay work from starting out if there is insufficient cleared staff. So this brings the total to a little over $200 million for the project. Thank you. Thanks, Paul Noel. I think I'm gonna cover the recommendation on what declassification and public access review project could look like, which is one of the things that we looked at as part of this feasibility study. And we have a few recommendations in particular. One, we think a project like this would require executive level leadership, direction and support with a senior level official from the Department of Energy to serve as the project leader. There'll also be a need for an interagency agreement on a project plan. So making sure that agencies work together. So declassification guidance for RD which is restricted data and FRD which is formally restricted data, national security and national security restrictions and other non-classification restrictions such as privacy. So the goal should be really to have an agreement for all agencies so that they're doing the same thing with respect to declassification and release. As part of the project, the agency should also consider digitizing records, especially those that contain relevant information and are historically significant. Mr. Pincus, I think you flagged the access questions which I think are really significant. And I think Carter will talk a little bit about as well. But to me, even if you release them, if they're not publicly accessible, they haven't done much good for researchers. Digitization would also improve that public access to records to make them available online instead of only in research rooms or by request. And then finally, one of the things that the PIDB has been recommending for a while is thinking about how to use advanced technologies like artificial intelligence and machine learning to support declassification and public access review. We think there's a lot of potential in that for records in general, but for this study as well. Carter, I'll turn it over to you for the public access piece. Yeah, as Alice was saying, there's two main agencies that have records here. It's the Department of Energy and the Department of Defense, individual components within those agencies. There's no centralized location for researchers to access information. So the Department of Energy has an online resource called OpenNet, which was set up as part of their openness and transparency initiative back in the late 90s, but over the years, records and bibliographic citations have been added to the OpenNet database and includes all the citations from the Nuclear Testing Archive. The Department of Defense and all of its components have their own repositories and there's no centralized resource to access records within these repositories. All of these repositories may have responsive records. For example, the Naval History and Heritage Command reported 13,250 boxes may contain responsive records. Many of these records are classified from top secret down to confidential and include restricted data and formally restricted data. So public access is a big challenge. Thank you, Carter, and thanks to everyone for informing us and going through with the public of the report's findings. I'll just say, in conclusion, that as Paul Newell walked through the numbers that DOE and DOD estimate what the cost will be to do this declassification initiative, should the government decide to do it, it's between $100 million and $200 million and some of the estimates actually go above $200 million. This is truly, I think, a shocking number to everyone. And what I think is really a wake-up call in this is that this is a limited topic. But now think about what it would cost to declassify records or improve public transparency on, let's say, a past presidency or another historical event that is even more sprawling than just this nuclear testing. The costs would be probably exponentially higher. And this really should be a very, very strong wake-up call to Congress, but also those throughout the executive branch that we have to come up with a better understanding of the nuclear system. And we also have to quickly develop and deploy tools. We have to take advantage of all the AI advancements that this country has made to be able to improve public transparency in a more efficient and cost-effective way. So I hope you'll enjoy the report. And again, please feel free to submit your questions at PIDB at narrow.gov. Melissa, anything else you'd like to add? And no, just to agree with you on that point, I think we looked at this process and recognizing the importance of the question of nuclear testing in the Marshall Islands. It does still emphasize how important it is to have a system that enables access to records in general, which we obviously don't always see in this case. And I think Walter Pincus in particular highlighted some of the challenges that he's seen as a researcher. And those are really problematic if we wanna learn from our own history. So I really, I just wanna echo Ezra's point. I also wanna thank Walter Pincus for joining us today. Really fascinating review for us. And also again, the staff for helping us prepare the report. They really did a terrific job in compiling information for us and collecting information. So thank you again and appreciate everyone for joining us today. Okay, well, thanks, Melissa. That concludes this public meeting of the PIDB. And we look forward to seeing everyone in a couple of months. Thank you. That concludes our conference. Thank you for using Event Services. You may now disconnect.