Good afternoon, everyone. Please welcome our next speaker, Joyce Garner, who's coming all the way to Kansas City to just promote this morning. She's going to talk about practical preservation today. Thank you for inviting me on 50 and 50 of the Technical Services Round Table notification talk about practical preservation techniques. I'm an archivist in the National Archives of Kansas City. We are one of 13 regional facilities of the National Archives of Records Administration. And we hold records from, we hold permanent records from US history courts and field offices of the federal agencies in the states of Missouri and Kansas, Iowa, Nebraska, North and South Dakota, and Minnesota. So you have researchers who need those kinds of records. You said that we also, we have lots of Bureau of Indian Affairs records, including the Winnebago agency, which I just let team on arranging and describing those records. I feel we can shift to the graph these days for having that map. But today I'm going to talk about preservation techniques. I am the person in charge of preservation in the National Archives of Kansas City. And being a federal employee, that means I take readings and submit reports and collect statistics and such. But it also means that I'm the person when we run into something that's falling apart, I'm supposed to know what to do with it. And so sometimes they make it up. But there really are some very basic sorts of things that we fall back on. And so a lot of those really are very applicable to libraries in general. I come into the archives profession with an undergrad history of NLS. From an employee state, I have been a school librarian. I've been a congregational librarian and a school retailer. I wrote a column for school library year over years. I just dropped that a couple of months ago. And so I really come into this with a lot of library background. And went back to school into some of the classes about other half-masters that we used to look at in the hospital. Which I've enjoyed very much. It brought me back to a circle to my undergrad. But for me, it's information management. Everything about it, really just in information management. Although I love historical stories that we discover all the time. So we're going to look at some things that will just be very practical. I hope that you can take back and use not only in your libraries, but also in your own personal belongings. We all have family heirlooms that we want to preserve. We also have people who ask us for advice on those sorts of things. So I hope that this hits on several levels. Just an overview of basic preservation for paper-based items today. Please don't ask me about digital or electronic media. I keep saying, but we start getting the email to archive. I'm going to retire at that time because it's not my era so much. But we're going to look at just what constitutes an effective preservation environment. The effects of proper housing and storage. We'll specifically take a little look at pass control and mold prevention, things to crop up in libraries all the time. I'll show you some very inexpensive paper repair techniques, clean communication, and then we'll touch on the depth of preparedness and emergency response. You could do almost a complete graduate program in any one of these areas. So this is going to be overview and here we go. Well, the main thing, what anything we're trying to preserve is that we are fighting with mother nature. Everything that we are trying to preserve, if it's paper-based in particular, is organic.  We cannot completely stop that process. We can do some things that will significantly slow it down. And we can do some things that will preserve information on the paper and medium. But follow the paper's books. That's all organic and it is decomposing. So a lot of it just comes down to provide the right environment for storage. And the four main tenants of a good preservation environment are that it should be cool, dry, and dark, and that the item should be as flat as possible. So when I give workshops or when we teach classes in this, I always say that you don't leave with anything else, remember, cool, dry, dark, and flat, and you will really be ahead of the game. Cool just refers to the temperature of the environment. Dry refers to the relative humidity. And there are particular ranges. Paper-based things like to be on the cool and dry side. And it needs to be a constant, steady temperature and humidity, not an average. If you have, you know, peaks and valleys, that really stresses the paper as it encounters those. So cool and dry, pretty consistent. In the dark, that talks about the effective UV rays. We are getting UV rays in there right now, behind closed curtains, off of the lights, overhead. And so we want to store things so that they are not getting that UV get. We know what it does to our skin. It does bad things to paper too. And if you think about a newspaper that plays in your driveway all day, how loud it is at the end of the day, well, that's highly acidic paper, it's printed on. So it's not meant to last. But you are seeing UV damage over the day as well. So over time, it really accumulates. And then to keep things flat as much as possible. We're not really going into the chemistry behind paper. Paper is made of fibers. And when we fold paper against the grain, we actually are damaging the fibers along that fold. That's so if you want to tear paper, you do it with your finger nail, and then you put it apart, and it's the other little fuzzy thing. That's the paper fibers. Anytime that we fold paper, we're stressing those fibers. Anytime that we roll paper up tightly, we're stressing those fibers. So we want to keep things as flat as possible. Now, if you have a map that is like one I worked on, you know, 18 feet wide and 25 feet long, you know, no map cabinet is going to hold that. But there are ways to keep it flatter than just, you know, fold it up and stuff it into a box or something. So we'll look at those a little bit. The cool dry garden plan, or basic tenant of this preservation environment. We have these little critters called TEM monitors in our stacks around our building. It stands for Preservation Environment Monitor. And these cost a couple hundred dollars a piece. We have them in our stacks, and anywhere that we have original records on building. So in our galleries as well, in our research room. And they take readings of the temperature and humidity every so many minutes. And once a quarter, I put a flash drive in there and pull all that information off. Now put in this software on my computer and make a report, send it off to Washington DC. So this is the highlight of what we're doing or not doing or need to do. But it's very useful information because it really does tell us how we're doing on the temperature and humidity. It's hard data. You may well have an HVAC system that would be able to provide you reports on this kind of environmental control in your building. And I tell people, if you really have no budget at all, at the very least hang a thermometer on the wall and pay attention to it. Now take a daily reading at the same time every day so that you actually know what's going on.  So we really do need to pay attention. Proper storage is really important. We store our records in boxes that are the right size and shape. As you can see in this picture, this is a record storage box. But it is made so that it's the right size for these folders, like legal folders. It has handles, it begins, so it's easy to handle. It has a box that goes over the top of the box.  These are properly stored records. If you're thinking, I don't know what you had in your collections, but this goes to file cabinets as well. If you pull a file drawer open and things go, you know, kind of rumble across the top, it's too full, you need to do something about that. If on the other hand they're really slumping, that's not supporting them well either. So, you know, you need to use proper kind of housing. And I brought in one of the boxes that we use in our stacks for record. This is made out of the asset-free material. The records that we have are old and they're generally on a specific paper, which is slowly deteriorating over time. So we try to put as much asset-free material in physical contact with it. So the folders inside the box are made of asset-free material. The box is made of asset-free material. It slows down the acidic deterioration. When I close the lid on the box, the records inside are in the dark. The box can be sitting out in the room. It takes the UV kit from the record to the side or in the dark. And this is easy to handle, it's a good size. So this is the kind of thing that we use. The memory enforcements really help the box in strength. Thank you for the question. So then what would be too cold for records? The question is, what would be too cold for records? I am in just a moment point to give you the optimal range for things. I hate to quote it off the top of my head because I'll be blown. For things that are oversized, we use math doors. We have lots of big oversize things. We have lots of art and chord engineer records. These people make maths about everything. Maths, charts, diagrams. So we have lots of those things and we like sort of flat. So again, these are in big folders that are made of asset-free material and they're stored in math cabinets when the doors shut and the windows are in the dark. But we have things that are just too big to go in math cabinets. We actually roll them around the outside of an asset-free tube. You can buy these tubes in different diameters. Generally three inches or six inches is what we use. Different lengths. I usually order them in four-foot and six-foot lengths. But we roll them loosely around the outside of the tube.  Spring's open, right? And then you can't get it out.  When we wrap things around the outside of a tube, it doesn't create that tension on the paper. And then we wrap it in mellonex, which is just a polyester film that is not off gas over time. It's completely stable. To keep it clean, we tie it with cotton twill tape. Nothing here that's going to last over time. And then we store them fully-supported on shelves. So it's not truly flat, but it's a lot flatter than it was. Oh, we got it. Because we do get them all curled up as tight as they can be. And it really is the best that we can do. Just looking at some enemies to preservation, I've mentioned acid and lignin are chemicals that are naturally present in paper. And lignin is the chemical in wood that makes a tree trunk stand up erect. It's great in a tree trunk. But in paper, it deteriorates over time. So basically, a sitting paper is deteriorating from the inside out over time. And as I said, we can't completely stop that, but we can slow it down by putting as much acid-free material in physical contact with the paper as possible. So we even sometimes just interleave sheets of acid-free coffee paper, because it's more acid-free material in physical contact. Dirt and rust are very corrosive and can obscure information on the record. Breast comes from paper clips. We find straight pens, stagels, blades, any kind of metal fastener will oxidize over time. It does not have to get wet. And it eats through the paper.  You may also just have information that's obscured. Pest, I'm gonna talk a little bit more about in detail why I talk about. Water is, you know, the humidity in the air. It's water, which is not so good for paper. But also a flood and any kind of emergency that you have is going to involve water of some kind. We'll look it down a little bit more in detail. So just a pretty good pass. The best thing to do is not to let them get started in the first place. So you want to present a good preservation environment. Stable, cool, dry temperature is really not that inviting to clear it. So, and for the past, we're talking about both rosin and insects here. So you want to keep things really clean. Be really careful where you store your food. You should never have food around your records or your books or anything like that because people's sticky fingers get on there and get spilled, it's inadvertent, but it does happen. Weather stripping will help keep them outside where they belong. One of the things that we do if we're getting on landscaping, we don't have anything that's planted directly up against the building. All of our landscaping is set out a couple of feet with a concrete barrier and a sidewalk around there. So it really helps that we don't get the bugs so much that way. Don't use poison bait because if you do without poison bait, whatever critter that eats it, going to not die right when they're where you put the bait, it's going to crawl up in some different corner and die and smell bad and become a food source for other critters. So we use sticky-hug traps. We actually have them in different places around the stacks, around the building. We have a contract with pest control company that they come and check out the traps for us every month. And we keep on log anything that we cite in the building that something else I get emailed because I'm the preservation person at the Lodge. That's so fun, but we keep on log and then the other thing that I can log is see what we've been dealing with. And it's a lot that we keep in the output of some of them. We've had a lot of them know it. We're just really freaking some of our student employees out. I've never one of those wormy things with all the legs, you know, just kind of have to keep dealing with them and then we finally got rid of them. For us, our dumpster is around the backside of the building and our stacks are in the front. So just that physical separation is important. Anything that you have shipped in is likely to bring passengers with it as well. So you really want to be careful about crates, pallets, shipping packing materials because you can easily get insects that come in with them. So we unpack those kinds of things on our dock not in the processing room, not in our stack. There's some things that affect paper on Japanese, just in general, how much acid is in the paper, acid-free paper certainly does not deteriorate like a sitting paper. Not all paper is equally acidic. All these paper clippings are some of the worst. Well, telegrams are really bad. How it's been stored? Has it been flat? Has it been rolled or folded? Just how it's been handled?  She had found a stash of Civil War era letters from her family and because they had been stored in the dark and a wood box, which I'm afraid because it's acidic, but in the dark and in a very cool dry place, they're very flexible, they're in good shape. She just wanted to have them preserved going forward. Exposure to UV rays, pests, all these things that we've talked about really affect how long paper products will last. Also, there are things like what kind of ink was used. You know, there is some kind of ink that actually has iron in it and will oxidize over time. Just if that's its inherent quality. Mountain pen ink tends to fade. It really just depends on what kind of ink was used. Temperature, humidity, air pollution can be very corrosive. So these are all factors that determine our effect, how long the paper item will last. A lot of times we have old books and scrapbooks. I get lots of questions about dealing with old scrapbooks especially. If you have old books, you want to make sure that the covers are supported, so that the book doesn't slump. So, use bookends. It's really a nice, very handy thing that is very effective. If it's really large, it should lay flat on the shelf. A copy table book is meant to lay on a copy table or a shelf is not meant to stand up. It's fine, we'll not support it over time. You know, the way that you remove a book from a shelf makes a lot of difference. It's easy to want to just put your finger on the top and pull down on the top and that will rip that binding. So, if you have things that are special that you really want to preserve, if you keep your books pulled out to the front of the shelf so there's a little room behind them, push the two, the one on each side in, that gives you a room to get a grip on the spine and just pull it straight out from the middle. It's a very simple thing that people don't think about that and it really will help preserve the spine. Scrapbooks really should be stored in a box and people who want to know what to do with their family scrapbooks, that's kind of a problem. You may have scrapbooks that have been donated to you or something like that. Scrapbooks tend to be put together on really acidic paper and they're just kind of crumbling.  So, I really encourage people to go through and take extensive digital photographs of everything in the scrapbook if there are items that open up like an invitation or a program, open it up, take a picture of the inside, thoroughly document that digitally. Then put the whole thing away and then acid-free box. You can have it with some just acid-free tissue paper if you need to. Boxes will live, put it in a place like a bedroom closet and that will really be good for it in the long run. And then, share the digital copies. Don't share the original. You have researchers who want to look at it. They need to look at the digital copies. They don't look at the original. It shouldn't be handled anymore, but it should be retired. Louspapers should be put in acid-free sleeve folders or in plastic sleeves. You always want to avoid anything that says PVC off-gasses has it basically over time. But sleeves made of polyester, polyethylene, there's three polys, and they'll say archival quality on the package. Those don't off-gass, they're very safe when they'll do nothing to harm it. Then you can't handle that letter or whatever it is in the sleeve. You're not touching it. You can read it through there. You can photocopy it. You can photograph it. You can display it that way if you want to, but you're not handling it anymore and you're not getting any oils from your fingers off. You want to avoid folding things more than once. We do usually fold things once. We think they're difficult.  And we can fold and consider very large. Preservation photocopies really are a great idea on acid-free paper. This is especially good for newspaper clippings because that newspaper clippings is going to continue to deteriorate over time. So if you make a preservation photocopy of it, you preserve it at the edge right now. And then you can, again, retie it in original. But they did a sleeve, an acid-free sleeve, but put it away in the dark in an acid-free box and make your photocopies on acid-free paper. You could also make digital copies and save life. But photocopies, you don't have to have any hardware or software to access them.  So here we go, the ideal storage environment for documents and books. A cool, stable temperature in a 68 to 72 degree range. Stable humidity of 40 to 50%. You want to keep it in the dark. So remember that it's not only sunlight, but it's fluorescent lights that can be coming off of them in a properly stored. Photographs are a little bit different. There's a lot of chemistry going on in photographs. And there are so many different kinds of photographs. I actually had an intern with me six weeks this summer specializing in photography preservation. She was in a graduate program, just for that. And I really envied her because it was so cool, you know, the stuff that she was wearing. The voice she could rattle off the particular, the not different kinds of photographs. So each photograph, depending on when it was taken, who took it, how it was developed, how it's been stored, it's going to have individual chemistry in its physical makeup and will determine to some extent how long it will last. But these other things are still important. You want to keep them out of direct sunlight. So it's a little bit exposure to UV rays. How you handle photographs is really important. They should be to sleep or you should be wearing gloves. We don't wear gloves to handle paper documents because we think the gloves actually make you a little bit clumsy, you lose the dexterity. So we say wash your hands, continue to wash your hands because a lot of times our records are old and baggy and our hands get dirty from handling the record. So wash your hands again if you think you're having a problem. But with photographs, you should always wear gloves or they should be sleep, one or the other. Having this stored is important on photographs. They take a little bit more care. The temperature is a little bit cooler for photographs and the humidity is a little bit drier. The emulsion on photographs will be tacky in high humidity and easily damaged at that point. And you'll also get a little mold growth going. So you want to have it a little bit cooler, a little bit drier. Air pollution, again, will be very corrosive to photographs and bugs and rodents like to eat photographs just like anything else that's paper-based. So they're a little bit specialized but a lot of the same factors still apply. So the article storage environment with photos, 65 to 70 degrees, a little bit cooler, 35 to 50 humidity, a little bit drier and again, should be stable and keep them in the dark. Polyethylene, polypropylene, polyester, cultured industry polys, I couldn't remember all of them. You don't want PVC that anything made of polyethylene, polypropylene or polyester will be a good sleeve. There are also paper sleeves made of acid-free material that can be very good for photographs. When we think about preserving photographs against the heat and humidity that can be very damaging to them and makes the emulsion sticky, you should always store photographs flat. We actually find photographs that have been folded to fit the box and anyway, you say that things in the courts sometimes, but that it really ruins wherever they've been folded, the emulsion is cracked and it is ruined at that point. So photographs in particular need to be stored flat and they need to be sleeved or handled. Now some things you should avoid using on your documents and photographs and this in particular, think about your own personal items as well. Anything that you are creating that you think you want to save or someone in your family might think it's an heirloom someday because you could be proactive on this upfront when you create things too. But these are, this is kind of the list of all the supplies that archivists hate because they cause problems with the balloon and transplant tape. We've got a lot in front of that here of a document that somebody is going to tape having on it. And the problem with this is it's a quick fix. It's very pragmatic. And if it's not something that you're going to keep for a long time, it's fine. I put it as something I'm going to keep. Well, I need to know that over time the adhesive will try out and fail and the cellophane will come off. And then what I have is a document that is still torn, still coming apart, and now it's got tape residue on it.  Then we're getting into solvents and archivists don't deal with solvents. That's conservators realm. So we just say don't do that. The same thing is true with white glue over time. It dries out, it fails, rubber cement, yuck, stuff that it leaves behind. Now you can rub it up on your finger. One of the things that we've run into a lot here are those self-stick magnetic photo albums. They really are terrible because there's nothing acid-free about them for the most part. And the clear sheet that lays over the photographs eventually will just kind of become one with the photographs. I mean, it all just sort of runs together into one thing mess. So I tell people that if they're trying to deal with one of these, if you can get the sheet off the top, so that the photograph is exposed, you can take a piece of dental floss and just look it back and forth under the photograph to get it to lift off. Because otherwise, you're trying to get it off because otherwise, you're trying to bend that photograph to peel it off and you may damage it. That dental floss really works pretty well that way. If you can't get them off that way, then really the best thing to do is to take digital photos of the photos, so that you at least have the original and the state that it is at this point and move forward and value them, never use one of those again. PVC, page, and photo protectors are out there. You know, they're very common, they're cheap. Things that are archival quality or acid pretent to be more expensive. So we find these and they generate acids, they bring down and it actually speeds up the deterioration. Any kind of melt-ass hair will rust over time. So one of the things, if you are dealing with things yourself and you're tempted to repair something, think can I undo this without damaging or changing this document at all? And if the answer is no, then don't do it, okay? We can put something in the sleeve, I dig it out of the sleeve, nothing has changed. I put tape on it, I've changed it. So it should not, it should always be reversible. Now if you have torn records, one thing that you can do is encapsulate or sleeve them. So this is what you do if you can't use tape, okay? So we've got a picture here of an exhibit from a court case actually and it was just kind of on old cardboard sort of paper and had completely fallen apart and we didn't know what it was. So one of our archivists took it out to a presentation lab and laid all the pieces out just like a puzzle where there was no picture on the cover and figured out what it was and it was a picture of a boiler. And you know, we find all kinds of things in court cases. It was an exhibit and so it was kind of a cool picture. So now we have this big thing, what do we do with this? So we actually made a very large sleeve that was welded all the way around so that it's encapsulated in that now. And it's just a custom thing that we were able to make. It will always have to be sort of flat. We had a researcher who wanted to look at this. We would probably take a digital phone serve that to them and say, I'm sorry, you can't look at the original because it just is too fragile to actually transport. But do the same thing if you have a letter that has fallen apart of the fold. There's a nice letter size sleeve, reassemble it in there. There's enough static electricity built up inside the sleeve to hold those pieces in place. Then you can handle it. It really works very well. You've not used any tape on it. Now one thing you can do for things that are maybe not a permanent value but very pragmatic things that you do need to repair, you can use heat set tissue. And this is something that actually I put, this PowerPoint is on the conference website, one of the others, and I put pretty explicit directions here on how to do this, thinking that I was not going to be able to actually demonstrate. But heat set tissue is just a material that has a neutral pH tissue. So that's a pretty, mine was in a curling adhesive. It's a lot like if you sew kind of like iron knot interfacing, it's that kind of concept. Only it's more of a tissue. And so what you need is this little packing iron, some a sheet of heat set tissue and then a sheet of silicone release paper.  You wanna have a little bit of overlap around the outside. If you have a scarf tear, you know when something tears and it's kind of like the paper actually split and it's like we got some over here and over here, you wanna make sure that it's like the right way. To line it all up, you kind of tack it down with the heat set tissue, down with the shiny side down because that's the heat set side, tack it into place just to position it. And then you use the silicone release paper, which nothing will stick to, put that down iron over it basically. And then if you want to do the whole process again on the back side for extra strength, you can. If there's any that extended over the side, you just trim it down. This is an easy technique. It takes a little bit of practice. So if you're interested in it and it's really barely inexpensive, you can buy all of these materials from, Gamelord or any library supply company. Takes a little bit of practice, but it's kind of fun. And I brought a little example. This is just the tacking iron. You can see it's the temperature control tissue and silicone release paper. And if you wanna come up and look at these afterwards, you're welcome to. I also have a map here that I did a lot of heat set tissue repair on. So, this was when they came out of my glove box.  And so when I learned to do this, I wanted something that would be useful. My daughter lives in St. Paul and we need this map. So if you wanna look at this afterwards, you can see that every place it's folded, I've reinforced it on both sides of the heat set tissue. It can be re-folded at this point. It really is a very strong repair. Didn't take very long at all. The other thing I said, things need to be reversible. You can take heat set tissue off if you need to just by reheating and taking it off. Eventification is another thing that's really a great simple paper repair. I know I said that humidity is bad for paper, but in a controlled environment, humidity is really very good for paper. If you think about old brittle paper, one of the reasons that it's in a state of tennis because it's become too dry over time. So we can very gently reintroduce humidity to it through the humidification process. These are pictures of humidification chambers in our preservation lab. These are about four by six feet. They are, we're set up there to really deal with these oversized records that we get. But you can see in one of the pictures, I've got some records in there. Something that's been folded up. We get old court records that are actually tri-folded records. And they've been stored, tri-folded for a hundred and whatever years. They're not kind of easily unfolded at this point. And they're very difficult to even scan a photocopy. We can't safely use them until we get them to lay down, relax. So this is the process that we go through. It's putting them through this humidification chamber. Usually it's an overnight process. And then we layer them between sheets of heavy blotter paper with plexiglass plates. This is just a small humidification chamber that I made that we use for smaller things. It's on the very same principle, exactly like the big one. And this is one that you could easily make at home. This is just a rubber band cut. So nothing that will rust. In the bottom, we have a sheet of thick blotter paper that's been cut to death. So what I would do is pour warm water on here. The blotter paper soaks up all the water. So in this case, I don't have any standing blotter at all. But it's really wet. And warm to warm water. This is, we call it a crate. You find it at hardware store and it goes up to us and like it's just plastic. Lay it on top. And then for this one, I added a sheet of, this stuff is called Polytex or Remay. It's a spun polyester. It's a lot like it's very new home. That lay it over the top so that occasionally a very delicate record could be kind of kind of embossed with that crate pad. The records go in here on top and then the lid goes on. And it's an airtight, it's a pretty airtight environment. Very warm and humid in there. The records just, it's very passive though. No hands, no mechanics going on here. I've done this same setup at home with the type of work it takes when it's like paper towels.  Leave them in there for, it depends on what you're putting in, how old it is, how thick it is. But if there are old letters, anything with top and pen, you need to really watch it because the equal run as it starts to be redefined. So that may only take 20 or 30 minutes. Other things may need to be left in overnight. But then the next day, you take them out, layer them between sheets of water paper with a blade on top, let them dry overnight. They come out looking just like they've been ironed almost. And they're really very subtle and much easier to handle at that point when they just relapse. Any damage that was there from a fold is still there. It doesn't make the fold go away. They're just laying flat. Like we give a lot of folks, you know, need any sort of advisory, you know, in better conditions than the ones we have on the show. It probably indicates some mildew or something.  Now, we'll see what we can say about that. Okay, so the question, I'm sorry, I'm asking the question. So the question is about smelly things. I think what you're going to need to do is just keep them segregated. As much as you can. Some of you can sleeve them. You can encapsulate them. You can put them in separate boxes. But if there is that smell, there is something chemical going on that may not be a good thing to have around the rest of your collection. Segregating them is a good thing. Kind of pointing them. I'll take one more question. You don't have to keep that. A cigarette smell from just a circulating book. I don't really have a good answer to that. I don't really have a good answer to that. The best answer might be to replace the book. I've got some examples of some different humidification chambers. I just went out and found these online. Just to give you an idea of how creative you can be in coming. This top on the trash can within the trash can, this is very common. It's really great if you have bold posters or tubes or something like that.  The records go in the small trash can to pour water around the outside. So there's no water touching the records. You put the lid on and get the humidifier in. The two underneath, I've seen these frequently as well. And I've seen a lot of conservation labs where they will set these up in their big oversized stainless steel sinks, actually. Run some water in the bottom. Put a tray or a box of some kind on top of that.  So again, you're creating some kind of enclosed, passive human environment here. You never want water touching your documents, but they have to be separated in some way. But these are just some examples. So you can really think about what you have to work with if you'll find stuff in your kitchen. And then when you stack humidified documents to dry, you can just do all the weights on top or you can do a layer of weights in the middle and another layer of weights on top. Just so long as it's all the same size and thickness and whatever works for the space that you have. If you're doing this at home, like I said, I use just white paper towels. Be careful about anything that's printed on paper towels could run, but just white ones are fine. Mold is a fungus that's composed of long filaments and it eats whatever it's living on. So that's why when you have mold and cheese, you get a nice big margin, I hope, around when you've had off the little blue corner because there are, it's sending boots down into whatever it's living on. So there are inactive molds for all around us in the air all the time. We can't really filter them out. As long as they're inactive, they don't really bother us. It's when we get a spike in humidity that they become active and then our allergies flare up and we get active mold for all of our records. So temperature over 75 degrees, relative humidity over 60%. You have a small window of time which to do something about this situation. 48 to 72 hours and you can expect active mold growth. So what you need to do is if you have things that are damp, get them into a freezer. Actually putting things in a freezer will stop mold growth in its tracks and it buys you time. When you're handling things and have become moldy, you want to be careful of your own health. So wear a face mask, wash your hands frequently and really be careful. As once you get active mold growth going, it will continue to crop up and I will wear the Ruze My Library that moved again to its new facility before the HVAC system was truly regulated and within a month, head of water you're being found is skylights on the collection and they have a continual mold remediation facility set up in the backside of the building now that it's outside and there is constantly looking for things. She uses everything from pans and kibby weather to freezers to unit that, you know, that you need to really be careful of it. I have some really lovely pictures of mold damage here. Once mold becomes inactive, which happens when we get it into a cool and dry environment, those mold scores can be crushed or vacuumed away that they will leave behind permanent stains and so the information can be obscured at that point. These are some records that we had, we opened up a box of court records that apparently had been wet at some point and nobody knew it and it was just a carpet of mold across the top of that box and these are permanent records. The government says we have to keep them. So we had to deal with it and we had people with masks and vacuums going for a while until this was dealt with, that they were binders of court records and you can see where the paper actually has been eaten away by the mold and it just went right across the top of all those pages. So it, you know, hit the top of the edge and just went right down. So we really had some fairly significant loss and it even was on the outside of the binders, we opened the binders up, the paper pulled the card, it was a disaster. So don't let this get started, go back to that temperature and humidity, the thing. If your HVAC system fails, remember you've got a small window of time to act and if you have things that are wet, get them in a freezer. So moving on to our last area of disaster response, when disaster strikes and it will, preparation is the key. So thinking ahead, because I can guarantee you, at some point in your career, you will feel that some kind of a disaster. I've been through a fire and a massive water break at this point.  The disaster recovery, remember that if things actually burn up, if they're actually charred, that damage is still reversible. What we can usually deal with is smoke damage, you know, smoke, pollution like behind can be cleaned off and water will be involved. It's just about anything that you deal with. Having been through a fire, I can tell you, it's a very small fire. I've been through a fire. The fire is very enthusiastic, with how much water they pour down your bed. So we had way more smoke and water damage than we had anything that actually burned.  So it wasn't even in the area that was burning, that just, it was the water that was poured on it. If you have things that get wet in a flood or a leak or whatever, really quick action is important. You can't air dry things, the paper will buckle. It will not be nice when it's gotten nice and smooth. But if you can put them in a freezer, it actually will freeze dry. And we had a water leak in our stacks about a year and a half ago from a bullet that had pierced a scene in our brand new stacks. So people shoot bullets at all buildings around. And it was very comforting to all of us to know that we had torrential downpour, water finds that hole, finds its way down, we had 56 boxes. And everybody got called in to work, it wasn't there, we did a massive refoldering, refoxing and we sent them off to a basically a freezer facility that we have access to as the federal government. Then we brought them out, we left them there for several weeks. They, the boxes, you know, they were tight, fully packed boxes. The paper is really just freeze dry right there. We took them out, cut the boxes in time, laid them out for a lot of paper, let them fall out so that any moisture that remains, we lost absolutely no information. I mean, it was really tremendous. It was months of recovery work on that. So really, getting things into a freezer, you won't get mold growth and it will really start to freeze dry. So it's really a good idea. If you really need something to be fixed, afterwards you need to consult a conservator. It's expensive, but it's an investment. It's something that you really do need to keep. Don't try to deal with it yourself. And Zaster will occur at some point, pipes burst, fires break out, you know, it just happens. So you need an emergency response plan that the staff is trained on and they include your volunteers, your custodians, all of your staff. Our Water League was discovered by a student who happened to take the long way back around at Stacks instead of coming out the more direct way. It could have been a lot worse. She caught up when it opened going up for probably an hour or two. If it had been on a Saturday night, it had gone on all weekend, I'd still be working on it. So preventive measures, fire safety inspections, make sure your exits are accessible. The vital records need to have backups off site so that everything's not in the same place. Your emergency supply should be stored near your Stacks. We now have enough plastic sheeting pre-cut to cover our entire Stacks. It's stored outside the Stacks. So it's not down in the basement, no, where is that? You know, it's right there. It's really well made with shelving. Be careful of what you're storing near the Stacks.  Just like the way the tow trucks show up before the police, when you have a rack, when you have a fire, I can tell you that a recovery contractor will be there along with the fire department. They may not be the best people. So remember, everything is water, rough sleek, so you will have water involved in just any kind of emergency. Anything that you have will require a custom solution. So there are basics that you also need to stop. Consider what you have going on right here. Consider human safety. You don't want, you know, wait into water all over the floor and if electricity's not off, you can get electricity. So you need to think of safety, not heroic. And people don't always do that in the heat of the moment. Throughout the list of this PowerPoint, I know we're out of time. I've just given some real specifics that I would encourage you to go and look at the PowerPoint online. So just ways to immediately respond and stabilize the area. Set up recovery zones by medium so that it will be much more efficient in the way that you deal with things. The way that you pack things out is really important. Everything used to be documented so that you can find every box or every book of whatever it is at any time of note where it's located. And if you have photographs that are wet, they need to be kept wet and then you need to get a conservatory. These are lists of emergencies of why. A number of these things we already have on site, on hand before we leave them. So, because they're really first response things. There's other things you will think about first, you know, renting when the time comes. I have enough masks and goggles for a whole set. I have not bought steel-toed boots for everyone. Just some specifics on recovering things. Paper, as I've said, you wanna freeze it as much as possible. Be really careful, you can get insects and pests going in here too. Photographs are really delicate when they get wet. Do you wanna be very careful with them? If they are wet, keep them wet and keep them cold so you don't get mold growth going on. And these are just some additional websites that are good sources of information on preservation in general and some sources for article supplies. And that's it. I'm sure the government will think that I don't know why. I know that you already thought one of my evaluations was awesome, you are happy with me. And if you have any additional questions, I'm glad to hang around for a little bit and talk to you personally. So thank you very much for having me here.


Welcome to another edition of RCE. Again, I'm your host Brock Palin. I have here Jeff Squires from Cisco Systems and open MPI Jeff Thanks again for helping out with the show Hey Brock looks like we've been going on a Visualization stint recently Yes, actually the topic today. We're gonna talk about involves two former guests from two different packages that are actually part of this Global package. We're gonna talk about today before that You can follow me on Twitter at Brock Palin all one word and then you can also just find that on the RCE website RCE-cast.com you can find all the old back shows there Because I think iTunes only shows like the last five or something like that And there's also an RSS feed where you can subscribe and be able to reading RSS reader Jeff. I also think you have a blog I Do there's a blog pointed off the RCE cast and The big news in that area is that we are finally new blogging platform at Cisco and all of us are very excited about it Because I think I think the package that we use today is The best blogger So so we're excited for the the new launch in a out of model. Whoo Okay, well, let's move into our topic again. I mentioned that two of our guests have been on the show so probably remember Burke Gavichi Who was on about VTK, but he's going to be speaking with us today about pair of you along with Kenneth Marlin? From Sandia who had been on talking about iced tea, which I believe is used inside pair of you And we have a new guest today or catch Aichi he can correct me on destroying his name when he introduces himself But all three of those guys work on pair of you Visualization package the details of which we will let them explain So Burke why don't you go ahead and start off again? Say a little bit about yourself for those who haven't listened to the previous show. Sure. My name is back give a G I Lead the scientific visualization and the informatics teams at Kitware Kitware is For those that don't know is a small company that about 80 people That focuses on visualization informatics Image processing and computer vision. I've been here for about 10 years. I The whole time that I've been developers of developer of VTK and pair of you I have I was the lead programmer for a pair of you for a while, but now Would crush does that? I would crush. I watch it and I'm a technical lead here at Kitware And I've been here with Kitware for over six years now. I I've been working mostly on pair of you as a developer I've been involved with other projects related to a pair of you and VTK and Because that's pretty much it. This is Kenneth mohler from Sandia national laboratories I've been the pair of you lead for at Sandia for several years now My main interest in the past decade has been in large-scale parallel visualization algorithms and systems our research at Sandia Has been driven by advanced scientific simulations and world-class supercomputers during this time We've been using pair of you as a research and deployment platform Okay, so why don't you one of you guys take and give us the 10,000 foot view of what is pair of you? And what does it mean to the average research computing user? I can do that so pair of you is a An application for visualizing Scientific data sets I It was designed originally for Visualizing large data sets, but I think I don't want to necessarily restrict my definition to that at this point So the pair of you project started about It 10 11 years ago for the purpose of Building an end user application around the visualization toolkit the visualization toolkit at the time was already old it was already I believe 10 years old or so and But it did it what it's really a developer tool and it there was no way of Delivering functionality to to our end users and as we started developing large-data visualization functionality in VTK we wanted to be able to Deliver that Therefore we started building a pair of you as essentially a kind of an end user extension To VTK since then obviously our main focus has been large-data data analysis and visualization But pair of you Has also branched into a lot of other things so therefore It is used commonly for scientific data analysis both small and large And it is also a develop development platform for those that want to actually build upon a Toolkit for end user applications and extend it to deliver functionality again In the area of scientific visualization as well as we've been doing more and more lately Pre-processing of scientific simulations and things like that So that's that's a very short summary. Obviously, I'm sure we'll we'll get more into What pay-a-view is and what pay-a-view does Okay. Yeah, so that was that was a good summary there a pair of you and a little bit of VTK there round this out with explain. What's the relationship to ice tea then as well? Ten well, I guess I'll take yes So I'm Ice tea as a parallel rendering library So when you're running pair of you in the large-scale parallel mode You need some sort of mechanism to be able to render images that provide the visual representations So I see Basically provides that capability That ties it all nicely into why you are all working together relationships Between those those pieces of software. So let me let me jump back to something you guys said earlier because She said that you were the technical lead on pair of you at Kitware and Ken you said you were the lead Fool or a pair of you at Sandia. So how do you guys split it up? You have different roles or Are you leads of different parts of pair of you? How does that work? So? pair of you is Is a lot is a lot larger project that is That has many contributors both from funding point of view And but also from you know contribution as as developer contribution Kitware is really the kind of the hub that where all of it comes together. We we started doing the original Initial development of pair of you and we can continue to be essentially to be the gatekeepers for for pair of you and folks like Ken and others Are developers of pair of you and then you know, there are leads at different institutions like Sandia and Los Alamos But Kitware kind of manages the whole project and we're responsible of the releases and and things like that Yes, just to elaborate on that. So San Dia has been a major contributor to pair of you for most of its lifespan We have our own particular interests will have developers at San Dia working on pair of you itself And we'll also have contracts with with Kitware to do works specific for Sandia's interest What the reason why we're so keen on pair of you is that a sandy and other national laboratories Regularly encounter these really large message meshes through advanced simulation and computing program Which typically are much larger than those encounter in academia and industry. So a lot of the other solutions that are just Other people are interested in don't necessarily work for for San Dia's Okay, so in the introduction about what pair of you was you asked about you mentioned that it was kind of like a front-end to VTK Does pair of you actually implement all of VTK's functionality or is that just an impossible task to do because of flexibility or Does even pair of you is all this functionality come from VTK or do you use some other third-party libraries? So VTK is basically a toolkit, right? It's a toolkit which provides you the data model provides you the pipeline model the execution model and all that and pair of you uses almost all of that the infrastructure provided by VTK for a state of processing but since VTK is a toolkit. It doesn't really have the application level logic to Save provide the what happens when the user creates a filter what happens when the user creates a reader all that application logic is typically Not handled by VTK at all because it's a toolkit and that is left to pair of you to for pair of you to manage And that's where a pair of you deals with it And also pair of you as its own extra things for to VTK to do things like parallel rendering or Data distribution and making sure that the data ends up at the right notes for rendering and so on and so forth so Peru has almost it does indeed have the entire VTK directly brought in but not all the filters or Readers are exposed in preview GUI, but that's only because But some of them don't really work well in parallel because of that or other reasons is just because no there's no real We just didn't feel the need for it But it's very easy to add Any of VTK filters into per view using plugins or just writing some simple XML So pair of you is a parallel application. Is it? Single machine parallel like you need one of these big SMP machines to do a large data set or is it used like MPI or some other type of distributed memory architecture? So per view supports both configurations You can run it on a single desktop without SMP or anything just a single process in which case It's not doing anything parallel if there's all the work in single process itself and the other case where you can run it supports running a parallel server using MPI and Then you can connect your private client to this parallel server and then you extend you can do distributed data processing as well as rendering most clusters don't have a Display they don't have acts. They don't have anything like that. How does pair of you actually handle? using a cluster or do you have to move data to like a viz cluster or So this part be independent of the render part or IO part or you tell me So all the things that you said said are possible. So first You can pair you can run in a mode where it delivers the geometry the final geometry not your actual data that you're processing to the client and then it renders on the client So this is very this is feasible when the geometry is that this is typically possible because you're doing things like ISO surfaces or Extracting outer shells or something like that and that's what So if those geometries tend to be much smaller than your data sets you can easily deliver them to the client if that's not possible Then you can run par view in a mode where you have a separate data processing unit and a separate rendering unit Which is so you can set up set it up such that you have a smaller cluster Which has access to X and display cards and all that and then then probably you can deliver the geometry to these nodes for rendering and Another possible that you can compile with West Mesa or which is another So open source library. It's an open source open jail implementation. You can use that That's on your versus without X and it's very well possible to do rendering on the data server Of course, then you're doing software rendering Well, I think that one newer thing that you skipped is is that more recently we have added another library support for another library called Manta, which was Which is developed by university of Utah to be able to do actually ray tracing On on on the data server So it's Mesa and Manta are really two alternatives that we have to just doing it where we don't have access to hardware for graphics acceleration and We're back after a crash that none of you heard due to the magic of recording and post editing and whatnot, but We had some technical difficulties, but now we're back in Brock Brock sounds much better now Yeah, well those with the recording since I did recording. I always sounded absolutely wonderful. The rest of you guys just sounded bad All right. Well now we sound better. Yeah All right, so let's let's pick up where we left off here We were just hearing about the various ways in which rendering occurs in the support libraries and things like that So let me follow up with a question about you mentioned. I think it was Burke Mentioned hardware acceleration for rendering. So this leads to the natural question and Flavored du jour of buzzword and HPC world today about GPUs So are you guys using GPUs the way they're meant to be used to actually render graphics? So, yes for of course our polygonal rendering does indeed go Use open GL which uses GPUs for doing the You're for the rendering part, but at the same time we have special implementations for things like volume rendering or as well as new surface lick algorithm that we added recently to Fairview Which is again using GPU for doing some GP GPU stuff So we have explored in the past Some using the GPU for special solutions like these But I guess we are going forward. We are going to investigate more and more using it For general processing and filtering and things of that So in a lot of systems that have done this they still to do GL rendering You need to have X running or something like that for those of us who have clusters that maybe have Tesla cards on them that we only have the CUDA library, but not the full X running Can you take advantage of that? Can we do any rendering that way? I? Am going to give a non authoritative answer that There are some libraries out there that try to solve that problem Whether or not they work with you know a particular card is you know, it depends a little bit on how much of the Open GL is implemented by the driver and we actually we did get various answers from a media depending on the cards on this But there are things like virtual GL and such that Kind of try to allow you to bypass essentially X to do that in some time so it's sometimes possible and sometimes it's not It's really a lot of times depends on the drivers And in other cases I've also seen where you can only use a single GPU per MPI rank and can you guys support like? Well, no, there'd be a single GPU for the entire MPI job. Can we do? malt a GPU for each MPI process and then Can we even mix that can some of the MPI processes have GPUs and some of them not Yes, that's possible that you would do by actually just like the configuration We were talking about earlier where you have your data processing cluster any visualization cluster in this case Your data processing visualization cluster are actually physically overlapping But you still have a smaller set of nodes that are actually doing the rendering and a larger set of nodes Which do the data processing, but yeah, that's possible You can also in theory There are some complications to this, but you couldn't in theory mix Hardware x-ray to open GL and may say in some cases So but I in our experience it is better to actually have multiple Courses for example, let's say you have a collection of nodes that have Eight cores per node, but one G1 or two GPUs per node It is easier and more efficient to actually have those cores In this case would be four core I shared the GPU Rather than saying one core is going to use the GPU and the other ones are going to do software rendering So you kind of touched on an interesting topic there and this is something a Very into this we argue about in the parallel computing community a lot. How much heterogeneity do you run into?  Customers who actually have a couple of different types of nodes, you know, like this kind of card over here that kind of card over here Or this guy has x this does not have x and I want to use them all together in one, you know rendering job We don't have that Yeah, I mean I have honestly in the last 10 years I have I have almost never run into people that are mixing and matching cards In in their cluster. I guess for the most part because people that do use Clusters to do distributed processing Usually have enough funding to actually get and set up set the whole thing up at once Whereas in the other case people that are Are you using another more general-purpose cluster probably are not using graphics cards in the first place? Let me take a step back then in the beginning is what platforms does pair of you support so as an end to application I'm kind of assuming that you Support all the popular Environments is that a good assumption? That's correct. We support Linux is a 64-bit 32-bit Windows as well as Max And you're you know any Unix not that we're seeing a lot of those anymore, but you know we used to do we used to even release binaries for sons and HPs and Sgi's etc. But we don't really do that as much anymore. We were not seeing a lot And on the case on the server side. We also support Supercomputers such as you know cray and Various IBMs and things like that Okay, good. That was that was exactly gonna be my next questions on the front end. It's desktop be kinds of things on the Back from Supercomputer. You have and have convenient So let me ask a derivative question on that something that's always annoying, but still sometime interesting What is the license for pair of you?  Will probably and we take a both our BSD so And I believe all the components that we have that are that come with the previous source once you check out the depository are Indeed under BSD right one one exception to that is that for Graphical user interface we use the Qt library or Qt library And that is licensed on the various license But we were using the LGPL license of Q So it doesn't bring a Some additional complication if you are a developer that want to build on top of pair of you You have something that's coming be that's BSD and then something else that's LGPL and you have to make sure that you You adhere to both of those licenses So that actually leads me into my next question Hey making pair of you extensible making it support File formats that doesn't support out of the box of course it supports the VTK formats out of box XDMF and a few others I Want to expand it to support my proprietary application and ship it with me the BSD makes it nice that I can write that How hard is it to write a new IOP plug-in to understand my file format both for serial IO and parallel IO? so of The first thing that you need to do to write to bring in your reader into pair of you is write a VTK based reader for it so you need to understand some of the basics of how to How to create a reader in VTK like the different things about the pipeline How does how to satisfy requests and how to provide information about the data that you need to be aware of? But once you have that figured out once you have a VTK VTK based reader that works It's very easy to bring that into pair of you all you do it there You can do that through a plug-in and you just write a bunch of XML that allows you to import that reader into pair of you The same holds for parallel as well. You just thought there are so in VTK When you're running in parallel different requests come down the pipeline asking for different parts of the data And so long as your reader respects that You're good. Your reader is parallel where and that that would just automatically works with pair of you Ultimately if you have If you have a Python library or some Python wrapped library that you can use in which case But we already support something called as a Python programmable source Which is almost like a data source With it allows the user to put in any arbitrary Python script to generate the data So that could be a very easy entry point for people who already have a Python interface for their leader library that they want to use So when I make this reader, do I have to compile it into Paraview or can I load it up as a dynamic object or something like that? Right. Yes, you compile as a separate plugin not into you don't barely compile it into Paraview You compile it as a plug-in that's separate which creates a shared library that then can be loaded into Paraview Okay, so no need to rebuild everything every time I want to do something. That's nice right And plus starting with the most recent release We're also Distributing the development binaries of Paraview so you can't you you don't even have to build a pair of you to build the plugins You can directly so by development binaries. I mean a package with the header files and lives and all that Yeah, one exception to this of course is if you Have to deploy in a platform that does not support shared objects such as the IBM Blue Gene Then you you have to recompile the whole thing Well not recompile actually you have to relink the whole thing with the additional code that you compiled So talking about plugins and whatnot. This leads to a natural question of contributors So you mentioned Sandy and Los Alamos who else is involved in the in the Paraview community attributes code So the the main development comes from mainly Los Alamos, Sandhya and Kitware and we have then a somewhat large number of developers that That are from different organizations. It's it's you know too too many to list here Some examples for example we we we had contributions actually a fair amount of contributions from Edf, which is the French electrical electric company electricity of France We have we have had collaborations with University of Utah at the ski Institute the Vistrails group there We've been working with them for a while and they have developed Modules for Pewe especially for Vistrails integration and then you know, there's there are there's probably It's something in the order of 10-15 developers that you know on and off contribute that are coming from variety of organizations So I couldn't help but notice that you said one of the contributors is the ski Institute in Utah Are these a bunch of ski bomb? What is that? Ski is that stands for scientific computing and imaging Institute at University of Utah. So it's with a C not with a K And at Utah, so they probably like skiing too, but not really directly related Okay, okay. Yeah, I was actually thinking the same thing there Jeff So a pair of you and tile displays some Viz applications work with the tile display directly and Some rely on a third-party tool to actually tile the image across.  Her view actually uses iced tea to do its Rendering power rendering and which includes tile displays as well. Ken. Do you want to elaborate? Right, so the car said a pair of uses iced tea Which provides the scalable sort last rendering on top displays? I'm not going to go into details about iced tea as we just recently did an entire RC podcast on the subject But suffice it to say that tile display support works out of the box with pair of you So to use a tile display you simply run one of the render capable pair of you servers on the computer driving the tile display And then tell pair of you server the number of tiles wide and the number of tiles high that displays then pair of you Take care of the rest I want to add something iced tea rocks Now into discussion with Ken about iced tea I was pretty impressed of just like how much better it performed over some of the other options out there It definitely seems really interesting And actually we're in the process of maybe building a tile display here at Michigan finally after a long time school of information is doing something Easties actually one of the first things I want to get going on it probably via a pair of you So we'll see that goes so to have this whole thing work We need to have X running on there and pair of you and iced tea just kind of fires up in full screen on each of the tiles And there's an XML file that says where each tile is or is it more complicated in that to actually have pair of you display on each one of the tiles correctly That's basically it if you have X running so that you're actually driving the The displays for the tile display then pair of you will automatically Do a full screen view on each one of those instead of the tile display for you? I mean we'll make some assumptions about the layout So you should set up your MPI ranks accordingly, but otherwise it's fairly automatic One technical note that you should be aware of is that in order for iced tea to perform really efficiently in large data sets You should have more compute nodes and you have tiles So pair of you will actually take advantage of the extra compute power you provide so that I can render even faster Okay, let's go follow up on that. So you say you know have more nodes than than tiles Is do you want to have a fixed ratio like a two to one or three to one or something like that? Or can pair of you or iced tea handle? You know a non even rate one and a half to one or or two half or two and a third to one or something like that There's no fixed ratio pair of you will just use as many Any processes that you've specified an MPI job and you've told it how big with how display is so it'll just know Which one to display on but regardless it will use all of your your nodes in order to do the compositing and rendering of the images So how much you actually want that ratio? It really depends on how much geometry you know how big your initial mesh is that you're trying to render If it's something fairly small obviously you don't need very many, but if the larger Your input mesh is the more compute power you're going to need to render it in an efficient manner So a little bit of clarification here not having used pair of you on anything besides my laptop so far There's like these IO servers render servers and then there's like the servers that are running on the display wall Is this all one MPI job or should the The MPI process is running on this on the like render nodes Are they a different MPI run and I tell them where us like a socket is for the MPI run That's running on display wall or does do you just describe all this to pair of you and it just does the right thing? Okay, uh, it's pointing at me to answer the question So the the different modes that Utkars described Can kind of do a combination of what you described so if you are running pair of you in Just a clock just simple client server mode The server then is actually doing the IO Processing as well as the rendering and in that case the server is just one MPI job Having said that what we can was referring to as having a MPI job that has more ranks than there are Displaced Is also valid for this case so you essentially you're just saying that the first six nodes Let's say are driving the tile whereas the other nodes are doing contributing to processing IO and rendering But they are not driving the tiles They do the the sort last composing takes care of where the final images go and then that they get displayed in the data server render server client mode You actually have one MPI job, which is the data server that's responsible of IO in processing And what it will do is that will generate geometry polygonal geometry and then there's another MPI job, which is running on the The display clusters or the viz cluster so to speak That's responsible of getting the geometry And rendering it and the two talk to each other through over to CP IP sockets and of course the client talks to both Both of these server types over sockets as well Okay, so then another thing after that I noticed that there's a config file where you can describe a cave environment for those I don't know what a cave is. It's like an immersive where you've got displays all around you What's pair of you support for cave like? So parents support for cave is still I Would say this infancy we are currently working on it to make it much better. So our current solution is We don't use iced tea for cave rendering What we simply do is the geometry is distributed to all the rendering nodes Everyone gets the full geometry and then every tile just renders the renders its own thing based on this its orientation which is different for every tile in a cave and We just and the client simply synchronizes all these together And client access the driver where you're interacting the user interacts with the scene using the client So we have also been working currently to add support for tracking to this framework. So people have we've been playing with using VR PN, which is what you're a reality Peripheral network virtual reality peripheral network Which is a library which allows you to interface with different devices which are typically used in such VR environments Yeah, I want to clarify that a little bit. So When you're within a cave a lot of times you don't want to necessarily use a mouse To interact with your visualization to rotate it, etc. So they they support a variety of devices That and we you know, we want to support those devices and one of these things that's kind of special is essentially head tracking where There will be some cameras or some system designed to essentially track your head position and orientation and adjust the scene accordingly so We haven't had support for Head tracking and any of these other devices and we're working now in introducing that support to to pair of you Okay, so intact Whoa Where would the con scene pair of you used I think we've all gotten used to seeing these dynamic fly-throughs of the weather on the 6 o'clock Who's and things like that has pair of you been used in any, you know widely seen outside the sign community kind of simulations like a movie commercial or Any commercial kinds of application widespread? pair of you it's is used by a large number of Organizations going, you know from from government to academia to to the industry and yet we do have we We do have partners and customers From the industry that couple pervy obviously with some of the major simulation codes For for the purpose of post processing and another interesting thing is that There is a popular CFD code called open foam that you may have heard of it's it's an open source gpl Parallel CFD code and pair of you is their official adopted Post-processing code. So if you if you actually get open foam you would get pair of you with it So what's coming in a new future versions of pair of you? What are you guys working on right now? So one of the main things that we are really excited about is of every station. So we've been working on adding support for exposing pervious large data visualization capabilities through a browser and So that's we almost have we can see it an alpha now So we have that out and people are excited about it and we are we are now working on integrating it with different web platforms Then another thing is Adding support for collaboration So which entails that you have multiple per view per view sessions running on different Desktops at different locations and they all share and look at the same data set and they communicate with each other too So that you can interactively visualize to as a group Then there's Institute processing and co-processing Ken do you want to talk about that? Sure, I'll talk a little bit about that So the pair of you co-processing library is basically a fairly small library attached to the pair of you services That allows you to run the pair of you analysis subservient to something like a simulation so the idea being is that you can run a large-scale simulation and While it's running and while the the actual data is still in the core memory you can fire off this analysis and visualization to produce Possibly images or or some other extracted information That that generally is a lot smaller than the original data that you started with so you end up you start with a very large three-dimensional mesh you may end up with a fairly small image or a Contour which typically is much smaller than the original message or just some sort of extracted surface or feature and then when you can run it in Much more temporally fine manner, so they don't have to dump everything out the disk and then load it up again cool I'll ask a question. I'd like to ask a lot of other software developers and guess that we have I'm here in RC, which Search code repository do you use and why I just like to hear people's reasons for what they use? use get You're gonna talk about the reasons why We use get us because it's much easier to create forks and work with so we have Some we have parallel developments going on right? We have these different projects driving doing different things on power view at the same time so that has become insanely simplified once we are switched to get and Everyone can have their own forks off of power view Which are doing doing their own feature in enhancements and once they're done with it They can push back to the master repository So so that's really working well. Yeah from a VTK perspective Because VTK is a is a large project Developed by a large community We wanted to leverage the distributed version control capabilities of get and also actually allowed us to even grow our community further Because we don't necessarily have to give everybody Right access to our central repository rather people can Push to things like a mirror for example that may we may have on GitHub or Gatorias or something something like that And just make a pull request to us as it's called Essentially, that's they send an email saying that hey, I have this branch on GitHub For power view. This is something that may be of interest to you Can you please pull pull this change to to your central repository? Of course, we had our challenges working with get also. It's not necessarily the easiest Easiest tool to use so a pair of you was Made for doing very large like Kent said like the national labs tend to work with things that are bigger than the average academic and public researcher What's the largest thing you've ever viewed with pair of you in terms of number of cores required to even fit in memory or Geometry, yeah, I'll go ahead and start so the scientific work at San Diego produces You know some of the largest models that we've visualized a pair of you The really big data sets the stuff that comes from the hero size simulations using the entirety of one of the world's largest Supercomputers they tend to come in sporadic, but important for us. They're driven by scientific need and supercomputing availability Our last such thrust was in 2008 when we analyzed the results of a shock physics code Comprising several billion cells and hundreds of thousands of blocks Now that said Sandhya and Los Alamos National Laboratory are in the process of jointly building a new pedestal computer called Cielo As the system comes online We expect a new wave of the simulation thrusts to occur as scientists are to leverage the new resource So in preparation, we're running several scaling studies For example, we've loaded a super sampled astrophysics data set containing over 40 billion cells And we've also stress tested the system with an internally generally generated data set containing over a trillion cells I'll talk a little bit from my perspective and this is mostly coming from more the our academic partners from the more the NSF community on the In situ we're right now. We're actually Working on doing some scalability studies And we have scaled our code on up to 32,000 cores So this is essentially the simulation running on 32,000 cores, but it is coupled with with pair of you In pair of you doing the processing on the same same as part of the same MPI job essentially When we talk about large data also There's there's there's a little bit kind of differences between doing this sort of in situ or batch processing for example, I asked around before the podcast and There's there's a group at UCSD And they have actually run pair of you on up to 1 trillion particles they have a particle data and We're talking about one just one field within this particle data Being around 100 gigabytes. So when they load something on the order of you know 10 to 20 of these things They are you know going beyond terabyte scale and So that's the kind of more batch processing On the interactive processing side actually the the kind of the numbers that can we're given those are Interactive analysis, so we're really running it on a running it on a vis cluster and We want to be expect to get something on the order of several frames a second to interactively do that and one Number I got from this is Scalability study. We did we did with the together with the tech Scientist the tech Texas computing center And we did a essentially 4k cubed volume. So that's about 64 billion voxels We interactively volume render that on or using our hardware Explorated GPU volume render and that's To look at the scale that's about only one of the fields there is about 256 gigabytes and when you're bringing multiple fields and you have multiple times steps, you're really getting you know on larger terabyte size So those are some of the numbers I mean this obviously these are the stuff that we have done or we have we have known The larger communities, you know doing out there using our tools to do large data And so there's I'm sure there is some hero runs that we haven't heard of yet So for the batch runs a pair of you support some sort of scripting like a python interface or its own Yes, Peru supports python interface So we have a python based API that allows you to create visualizations and configure parameters and do renderings and so on and so forth Okay, guys, thank you very much for your time I'm excited to get pair of you going on display wall if it gets built around here and Where can people find pair of you and is there something like a mailing list of how people get involved? We have Mailing list bug trackers and all that stuff the easiest thing to do is to just go to pair of you org And then follow the links. I think I believe there's a tab there that says help And there's a bunch of entries in there pointing to our wikis and web pages and the documentation and the mailing list That's it there Okay, well, thank you very much for your time guys Alright, thanks for your time Take care


Build Change is a non-profit social enterprise that works with people in developing countries to build houses and schools that don't collapse in earthquakes and typhoons. There are two basic problems that we're trying to solve. The first one is the number of deaths, injuries, and economic losses that often happen in emerging nations when there is an earthquake. It's not the earthquake that kills people, it's the collapse of a poorly built building. So this is a man-made problem and there has to be a man-made solution. The second problem is how the relief industry has traditionally approached post-disaster housing reconstruction. Usually they bring in a bunch of contractors to build large numbers of houses for people at a high cost per house with very little opportunity to build local skills and contribute to the local economy and without involving the homeowner in the process. We've been working within the industry to change the way post-disaster reconstruction is done. The motion tsunami happened in Indonesia in 2004. We started our first program in Aceh to help people rebuild after that disaster. And because I was a bricklayer, I could actually go out and train masons how to lay bricks. So we started small and on the ground. We hired local engineers, local architects. We got our hands dirty trying to figure out how do you actually build an earthquake-resistant house in this context. Once we figured out how to do it ourselves, we started outsourcing our methodology. So we shared our design and construction guidelines with other NGOs. We trained their staff. We supervised their construction. We trained their builders. And we were able to increase our impacts by a factor of 100 by doing this. We've been expanding our model from a direct service approach to a system change approach. We're working with governments from the start right after a disaster to change the way they approach the reconstruction. So we're trying to change the system. In China, after the 2008 earthquake, we actually had more power. We had more ability to enforce building standards by partnering with the government. They gave us these IDs that made us look like government officials, and we would go out and inspect the buildings, train builders on the job, report back to the government. And this was a very, very successful partnership that allowed us to scale up to over a thousand houses in a short time. So now partnering with governments is essential to what we do. With Haiti, we have a partnership with the Ministry of Public Works where we work together with them to write the first-ever guideline on retrofitting or strengthening a building. And we've used that guideline to train government engineers. In Colombia, we also have a partnership with the Colombian government in which we are training the Colombian government vocational institution. We're doing a train the trainers. We're training their team to train other builders in how to build safe houses. We're also partnering with a government subsidy program. Build change doesn't build houses for people. We don't provide the financing for people to build houses. That usually comes from some other source. Because we're working in an environment where building code enforcement isn't common, we need to use that financing as leverage. So we regularly partner with the institutions that provide that financing to make sure that it's contingent upon meeting standards for construction. We're not expecting homeowners to build a house themselves. We're expecting them to hire a local builder who produces their house. So the private sector provides the supply for safe construction while the homeowners have to demand safe construction. We work very closely with the private sector. Everything from the small-scale local builder, the local building material producer, on up to financing institutions, insurance companies and large-scale contractors and material suppliers. Over the years, we have evolved or I guess better defined our theory of change and the barriers to scale for what we do. And it really comes down to money, technology and people. We have to have the right technology. It has to be locally available. The builders have to have the skills and tools to build with it. The architecture has to be appropriate for the climate. And so we've got to get the technology right. On the money side, whatever we're proposing people to do, it has to be affordable. If people don't have enough money, they're not going to build a safe house. And so we are increasingly using financial incentives, insurance incentives to encourage people or to compel people to build safely. On the people side, someone has to want a house or school to be earthquake or disaster resistant. We work with governments to enforce building standards. We work with homeowners so that they understand how to build a safe house and they can demand it. And also use social media and radio and various different messaging campaigns to get the word out to massive numbers of people about safe construction. If you are in social innovation, I recommend that you consider how this money technology people model really applies to what you're doing, how to get the right financing in place, how to make sure that technology is appropriate and how to compel people to really change.