No one knows for how many centuries this wealth has lain here. There are no voices to tell us how it was in those alternate millenniums of sun and rain. The things that strode the jungles and through their challenge to the stars survive now only as outlines dimly red from the rocks. Mountains where the storms gathered are gone. Waters flow where once no rivers ran. But time, though it be slow, is never still. And movement, be it imperceptible, is movement still. The trees fell and were buried, rotting things mingled with the soil that gave them birth. They mixed and blended one with the other, and time did the rest. Deep in the sunless taverns of the earth, it gathered a black sluggish mud-like liquid, waiting it would seem for us. And we are here now to claim our own. We have come of age for us, the legacy of the centuries. The National Broadcasting Company in cooperation with the Alfred P. Sloan Foundation presents Legacy of the Centuries, the story of the petrochemical industry in the South. For nine successive weeks now, we have been following a long trail that has wound through the cities and the crossroads of the American South, reporting what we believe to be one of the most remarkable developments of our time. The renaissance of industry, culture, and the changes in the whole pattern of life has lived in the states below the line of Mason and Dixon. Our guide on this journey is NBC correspondent Henry Cassidy. Today he reports on a remarkable new industry whose very name was unknown a few short years ago, Mr. Cassidy. As a war correspondent, during the crowded years of the rise and unlemented demies of Adolf Hitler, I often pondered why it was that change in living, in thought, or in form of government must so often be accompanied by violence and pain, by sound and fury. And I found myself wondering if there was ever a revolution anywhere in which no one was hurt. Well, I found such a revolution, not in some heavy tome buried in the stacks of a library, but figuratively in my own backyard. For surely the changes that are now taking place in the American South can be called a revolution, a giving way of the old to the new, a creation of something better out of things previously obsolete and inadequate. I saw it in the resurgence of industry in a land that once was thrall to a few crops, crops that destroyed even as they seemed to give. I saw it in new breeds of cattle in the miracle of fruit juices frozen against time itself. I saw wealth that would beggar and alchemists dream and faces turned now from the past to a dawn brave with promise and with hope. And when I looked on these things, I thought my journey was done. My mission completed. But even as a snatch of song can be got you from your homewood way. So it was with me and a chance sentence that came my way. I don't remember the speaker, but his words were intriguing words and they led me down a long and twisting road. The fastest growing industry in the south today and probably in the country is a chemical industry. Here was an area I had not explored, the chemical industry in the south. Surely the man was mistaken. I decided to stay a day or so longer in this lovely and gracious land to prove to my own satisfaction at least that there was some mistake. But there was no mistake. I found a whole new industry, fresh and confident and hand in glove with the forces of tomorrow. I found to a new and somewhat forbidding word, petrochemicals, the many and varied substances that modern alchemy summons up from the bubbling brew of the cauldrons in more practical terms. The useful materials derived from the residue of petroleum. I picked up the beginning of my story down in Houston, Texas in the office of Mr. G. R. Walton, industrial sales manager of the United Gas Pipeline Company. Mr. Walton, I asked, where do we begin? Well, either place to begin on this story, I suppose, is essentially we're going to talk about Texas, although generally the petrochemical empire, as we like to call it, extends pretty well along the Gulf Coast from about in the audience or shall we say the French Quarter on the east down to the King Ranch of Texas on the southwest, a distance of about two hundred and fifty miles. This business, as we now know it, began about the beginning of World War two, caused largely by the pressures generated by World War shortages and demands. People had to do things that they hadn't taken the trouble to do before. We had to have rubber. Our natural rubber supplies were shut off by the Japs in the four east. There had been a few methods knocking around the research laboratories of making rubber from various and sundry raw materials. But all of a sudden it became evident that we had to have rubber and we had to have it quick. After looking over the many possibilities for making rubber artificially or synthetically, shall we say, the best bet seemed to be a process developed for making rubber from petroleum. And that, you might say, is the beginning of the now huge petrochemical business. When our chemists and our scientists found out they could make such things as rubber, they also began to look around to make other things out of petroleum and natural gas. They found that they had all the raw materials needed by this petrochemical business, namely oil, gas, sulfur and salt, all present in abundance and are well distributed throughout this area. This storehouse of all the necessary raw materials together with cheap bulk transportation, ample labor and plenty of room to build and expand is probably the major factor in this development. As a matter of fact, this oil, gas and sulfur have been known for years and have been in some degree of production for most of the 20th century. However, prior to World War Two, its usage or their usage was confined largely to such age-old processes and products as fuel, lubricants, heavy chemicals, acids, alkalizing light. He wouldn't recognize this area now had you been here some 10 or 15 or especially 20 years ago. Thought patterns formed in youth are hard to break and I'm enough of a New Englander to wonder why if this new industry has such promise, it has not developed in my section of the country. Why had it come to the south and not to the north? Where can you find more oil and more readily available? The answer was not long in coming. This is Mr. Ralph D. Patch. Dan, they call him inevitably, I suppose. Mr. Patch is head of the Chemical Products Division of the SO Standard Oil Company in Baton Rouge, Louisiana. He gave me the answer to my question. In fact, too many questions. The raw materials for making chemicals previously had been used entirely in the refineries to make gasoline and heating oil and fuel. In effect, the chemicals that we make had we not made them would have turned into gasoline for your automobile. These hydrocarbons come out of the ground and most of them seem to be centered in this particular section of the country, starting with natural gas, which is chemically known as methane, all the way down to the heaviest crude oil. A great many of the refineries are located in this section of the country so that the streams that are available or suitable for the manufacture of chemicals are readily available by pipeline from your neighbor. A great many of the chemical companies have moved down into this section of the country to build their own plants. And I expect as you travel westward from here along this great golden crescent from here to the tip of Texas, you'll see chemical companies and refining companies all the way along. And there was more to this story. Still other reasons why the south is the favored land for those who work this special brand of miracle. Listen to Mr. William M. Andrews of the engineering firm of Lockwood and Andrews in Houston, Texas. Approximately 20 years ago, the south was just beginning to industrialize. Whereas your east had already built up the heavy industry, as you know. And naturally, the southerners when they first started with their industry copied the easterners who had to put the big masonry buildings around all of their machinery and operations to protect it from the weather. We had to do something to cut down investments and save some dollars. And we found out that we could save the money for the housing due to our weather in Texas. The plant today, which does not have any masonry housing around it is considerably cheaper than is one is completely housed. You will get to see the engineers dream when you see the dial chemical plant, which is largely open air and only those processes which are a necessity have been housed. It had to be that sooner or later I would show my ignorance and ask a very simple and basic question. I again sought out Mr. Patch and framed my question as briefly and simply as I could. What precisely Mr. Patch is a petrochemical? Petrochemical is a synthetic word. We might just as well start off with that when the synthetic business. It's derived of course from petroleum and chemical. It refers to this long line of very new and interesting and almost glamorous chemicals that we're now getting from oil that comes from crude out of the ground. Now the chemists have found that a great many of these synthetic chemicals are combinations of molecules and atoms that contain carbon and hydrogen. Carbon and hydrogen constitute the principle ingredients of crude oil. So by taking the crude from the ground and sorting it out, getting certain raw materials from it that are passed on in turn to the chemical plants, these hydrocarbons as we call them combinations of carbon and hydrogen can be transformed into chemical building blocks, little special molecules that are particularly reactive. They in turn can be transformed into some of the things that we know in everyday life today like synthetic industrial alcohol, like synthetic rubber, a great many of these plastics, these fibers that you talked about nylon, orlone, acryland, some of those type of things. This then was the miracle I had come to find or part of it at least. Still learning they say is a long and sometimes painful process. I set out to find the changes that these things have brought about. I wondered what precisely are the products of this new industry? Are they esoteric things known to scientists or laboratory workers? Or do they affect us all? Would I wear a petrochemical or carry it in my pocket? I found myself in the office of Mr. Lynn Straughn, plant manager of the Jefferson Chemical Company at Port Natchez, Texas. Our main products are ethylene oxide, ethylene dichloride, ethylene glycol, and monodye and triethanolamine. They're common chemicals and they have a wide variety of uses. Most of them come from chemicals and from petroleum. They end up as such products as shirts, suits, in your gasoline and lubricants and products of that type. And what about the future, Mr. Straughn? I think the future of petrochemicals in the Gulf Coast area is very bright. We have the raw materials, water, supply, fuel, and those things are all essential to the promotion of the petrochemical industry. Mr. Straughn was not alone in his awareness of the significance that this will have in the years that lie ahead. It was a recurring theme. Mr. Patch again took a long look into the future and speculated on some wonders not yet born. These petrochemicals are very stable economically. They're they're here with us. They're not subject to crop failures, to locus plagues, to bad weather, that type of thing. These factories run, these plants run day and night. The crude oil is plenty of it ahead. It's here with us all the time. So that any supplier that can acquire his raw materials from a petrochemical manufacturer has got a pretty sure source of supply of his raw materials. Another thing, because it is so stable, because it's being made all the time and constantly, the price is fairly stable. I just don't think there's any limit to what we're going to end up doing with petrochemicals. Of course, the stable price encourages manufacturers to develop new uses new outlets. You can see in the past few years, such things as these synthetic fibers and plastics, the great many of them are coming from petrochemicals. We're constantly getting inquiries from from all kinds of people for new chemicals for samples of experimental chemicals that we have made so that they in turn can try to find out what they might be able to do with those new chemicals. It's an interesting thing. We've got a plant out here that produces a chemical that is pretty widely used. And there's a small stream in there that we have to purge out of that unit because it accumulates in there and acts as a sort of a poison. And we had our chemists take a look at this stream. It's not a very large one to find out what there might be in it. And they tell me that they located in there a chemical that could be used as the raw material for vitamin E. And there was a curious bypass to our story, one that took me to the land of floating mines, not the wartime variety, but an island of machinery in the back washes of the Mississippi, a new world Venice, where the wealth comes not from trade, but from a yellow evil smelling element that means more to those who seek it than all the spices of the East. Sulphur is the magic word in petrochemicals, the catalyst that makes all possible. Port Sulphur, Louisiana, and the office of Mr. George M. Leppert, assistant to the president of the Free Port Sulphur Company. This operation commence in 1933. This mine near here, 10 miles back in the marsh is known as the Grand Decay Mine Free Port Sulphur's Grand Decay Mine. Grand Decay is French slang for big scale, which is also slang for topping as a lake out there, Lake Grand Decay, where the mine is. And 10 miles back over here is the second largest Sulphur mine in the world, produces a million 431,000 tons a year. And this is the point of a huge Sulphur triangle, you might say. So this is sort of a Sulphur headquarters in Louisiana. They built this town. This town is on a hydraulic fill, which means that tremendous amounts of mud were pumped in behind levers and allowed to solidify and the houses were built and gradually it's been added to until you see this thriving little community that we have here now. Here then was my revolution. Here were changes as huge in their effect as they were bold in concept. And the results can be felt across this whole land far beyond the rim of the seeable world. Here was changed without violence. Here was a revolution conceived in quiet and born in peace. There was some reluctance to talk about the social changes that had been brought but the words when they came were strong ones. Listen to Mr. DB Campbell, plant manager of the DuPont Sabine Riverworks in Orange, Texas. Oh, I don't like to talk about what we've done for the community. I hope we've tried to be good neighbors and fit into the community. I think it is a fact, however, that we spent last year $2 million for materials of one sort or another, just within the Orange Beaumont Port Arthur area. From what we've seen in the last 10 years, I I I just happened to crystal ball it would tell me where it might go. If it continues at the rate it is and there is every reason to believe that it will, it certainly will grow to be tremendous. And then I ask Mr. Campbell, has all this changed the community since you've been a part of it? I personally have only been here three years, but when the DuPont camp company came in here about seven years ago, the community was not exactly flourishing. They had had some ups and downs. Unfortunately, they had had some war babies which had gone bad on them when the war was over. And I think I can truthfully say that they were very happy to have the DuPont company come in here. We have employed 2,000 people. We have paid last year, for instance, our payroll, which we turned loose in this community within a 20 mile radius, nine and a quarter million dollars. I think that helps any community. There is quite a building boom on in the city of Orange. I think last year they completed somewhere around four or five hundred new homes. It was morning raised in Orange. Orange was a pretty small town when I was a child. And the bringing on the World War II made a big change. We had quite a few people come here with the shipyard. I think the DuPont plant and other plants saved Orange from a collapse. My working conditions are so much better and my wages is better and the people I work with treat you like you are human being in other words. But that which comes of the earth must return to the earth. This new industry was scarcely a year old before those who till the land were asking if this earth born stuff would be of help to those who coax the growing things out of the seed. The answer was yes, emphatically so, and a new and remarkable job was born. A job that as is fitting was filled by a remarkable man. Mr. Ladd hasted agricultural counselor to the oil industry. And I asked Ladd, just what does the farmer have to do with petrochemicals? Henry, let's get rid of that word right now. I just frankly among farmers we don't use that word. That's something that was dreamed up in the laboratory and sure it's a good word as far as I'm concerned but as much we're talking to the public. Let's just get down and say what it is. Well, first, Ladd, you better tell me then if you don't call it petrochemicals or the farmers don't, what do they call it? Well, you might break it down into two points, Henry. You might say that one point was chemicals that come from petroleum that help to grow things. And the other thing is chemicals that are put in to a petroleum carrier that help to preserve things. In other words, in the first one you might say anhydrous ammonia is a fertilizer that helps to grow things. And in the second group you'd put in the insecticides, fungicides, rodenticides, all of which are big words too. Let's call them by the name farmers, call them that is weed killers, insect killers and rabbit killers or mice killers and, well, fungi. I guess you still have to use that word. That's the one we got. There's no substitute for fungi, is there, Ladd? Ladd, you've tossed me a great big phrase there. I think it sounded like anhydrous ammonia. What's that? There's another way to say it and that's a common way to say it, Henry. NH3 actually is just simply getting nitrogen in a form that is rather new. In the past we always got nitrogen in a granular form, a powder in bags. Now we get it in the gas and it comes in cylinders of very much like the sort of cylinders you see behind suburban homes. You know there's metal things with all the loom and so forth. Well that's the way the stuff comes and it goes into the ground as a gas and immediately becomes a liquid and it provides N, that's nitrogen, directly to the plant in a way that the plant can get it very rapidly and very efficiently. And this thing was sort of born down in the Delaware region of the South, right where we are, rather than just immediately south of us from here to Vicksburg. And it's become almost universal in this region and from this region has spread to 38 states. The thing that pulls it back is the cost of transportation, not of the gas but of the containers which you must use because the stuff is under very high pressure. Well then how about cost? It seems to me that we have a new thing here. Does it cost less than the original forms that we used? No it doesn't but the reason why is because of the cost of the container to ship it back and forth. The gas itself is a pretty economical thing with these containers because it's under pressure and because it's dangerous by the way if they can blow up they've got to be very thick and something is safe in order to go through freight. So the package of the product cost more than the product itself? That's right. We could use it right up in New England for instance and they'd like to use it up there if they get it cheap enough but it costs the dog so much to ship it back and forth. What crop then lad do you apply this in H32? Well you can apply it to practically any crop in the south cotton, tobacco, peanuts, corn, etc. or pasture but the thing I think is most dramatic is that for many many years we've thought of the this region here in the south as being a 12 bushel corn region. Everybody raised corn down here and yet the yield was so low that agriculture's elsewhere couldn't understand why they even bothered to do it. You say 12 bushels an acre? That's above the average for the south. And what's it going to be with your NH3? Well that's the thing that's come up in the very last two or three years. Now we've got all over the south down here a hundred bushels to acre clubs that's a title. There's people that are listed in clubs that guarantee they're going to grow a hundred bushels of corn to the acre where 12 used to grow yesterday as it were and anhydrous ammonia NH3 is the answer. That's a title that's going to become a fact. Yes sir. Lad it sounds to me like your NH3 has already blown up the agricultural business in the south. Well it's been a wonderful thing and it's got a great way to go. We think it's got a tremendous future. Tell me this then Lad I take it that the farmers and people who are actually using these petrochemicals perhaps don't even realize that they are petrochemicals and don't distinguish them from other rabbit killers or weed killers. I think that's right Henry I don't think any farmer worries about where the stuff comes from or what's behind it you might say but what will it do that's what he wants to know and whatever the trade name is that's all right with him just providing it does the job. If he doesn't care where it comes from as long as it does a good job do we need petrochemicals. Oh very definitely very definitely that's the big thing that's come to agriculture in the last 10 years or perhaps I should say 15 just to go back a little bit. You know the potato scandal which exercised all of the public and certainly farmers really came about because of DDT which of course every housewife knows what that is. Well DDT killed the little bugs that cuts down the potato yield and all of a sudden we had an enormous potato yield because of DDT. Some people think it was a political thing actually it wasn't it was just a chemical thing and that's one angle and of course there's a thousand others. But as I understand it we already have an agricultural surplus we have too much found products why should we now develop something that either grows more or preserves more. Well that's the question that has really got all of us worried Henry and the answer to it is this that in as much as agriculture is a biological science rather than a physical science you have to look far ahead you've got to work far ahead. When the Korean War came on all of a sudden we were in shortage and we needed tremendous supplies of everything. Well we don't know what's going to happen tomorrow very truly now we've got too much of everything but we also have a great threat over this as everybody knows if another war would to happen which god forbid we would need immense supplies not only for ourselves but for our allies. So agriculture to be healthy and to do its job for America at large must always run on us at least a small surplus. We need that little bumper there in case nothing goes wrong. Well apart from anything going wrong led in other words if we don't have a war what else then why do we still need to develop these new things and produce more and preserve more. Well Henry that one is a very kindly sort of a thing we're just having the biggest crop in the world of babies here in America between two million and a half and three million a year and all those miles have got to be fed and we can't expand our acreage too much yes with some desert lands can be put under irrigation we can reclaim some worn out lands for odds and ends of things but they're very small and our best bet is to get more from the land we have so as these youngsters come along and we look into tomorrow to our daughters granddaughters sons grandsons we've got to prepare for them and prepare enough so that they can have the same abundance we have now and the petrochemicals is one way to do it because they cut down loss and raise yield these were the things that i saw and the voices that i heard i had found my quiet revolution for in this land of soft ways and gentle accents a change has come no less great i think than many a revolution born of anger and envy of hate and the gnawing pangs of greed no such corruption dulls the brightness of this new day and this new land the promises of tomorrow are written large across this place it enters by a thousand doors new breeds of cattle power plants pushing back the night chimneys that glow by day and night and throw their challenge across the clouds and now the newest miracles conjured up from the residue of time no one knows how long it has been waiting no one knows what centuries crawled by with no one to mark their passing but we have come of age for us the legacy of the centuries you have been listening to legacy of the centuries the ninth in the series of programs heritage over the land describing the changes that have come lately to the american south this series is written and directed by william alan bales and produced by miss lee f paint it is anticipated that future series will deal with changes in other sections of the country


CHAPTER XII THE LAST SATURDAY Every Sunday morning Mr. Merrill gave Lucy half a dollar for the next week of the vacation school, and every Sunday morning Uncle Dan dropped another down Dora's back. She did not like to feel it going down, but since it pleased Uncle Dan to do it, Dora felt obliged not to wiggle more than she could help. But she always shook it out as soon as possible. One day when they reached the Lincoln School, the children found the other girls and the teachers feeling quite disturbed. Some rough big boys had come into the playground and thrown over the tables and the benches and broken the supports to the little slide the little children liked. That day everybody had to sit on the grass, either to sew or to make baskets. Miss Chandler said she would have the seats and tables mended, but it was hard to find anyone to do it. All the carpenters were very busy. That night Dora told Uncle Dan what had happened. At the time he did not say anything, but Dora was not a bit surprised when Uncle Dan came next morning to the vacation school. I heard you wanted a little help, he said to Miss Chandler. My boss has let me off for an hour or so to see what I can do for you.  He did not speak either to her or to Lucy, but went to work repairing the damage the big boys had done. He set up the tables and nailed the benches together, and then he looked at the supports for the slide. It would not do to patch them up because if they should break while the little ones were playing somebody might be hurt. Uncle Dan put in two new posts for the side. I am very grateful to you for coming, said Miss Chandler. I tried everywhere to find somebody to mend the things, but every man I asked said he could not do anything until next week. It means a great deal to the children to have it done so soon. Now just send your bill to the Improvement Society, and I will see that it is paid. There won't be any bill, said Uncle Dan.  Miss Chandler looked quite surprised. Thank you very much, she said, and then she held out her hand. Uncle Dan took it solemnly and lifted his cap as he went away, but as he passed Dora he winked at her. It had been part of the fun that they should pretend not to know each other, but most of the girls knew Uncle Dan and quite envied Lucy and Dora because he had come to make things right at the playground. By this time the cross-stitch spread was finished and it was really beautiful. All the teachers said so. Dora had sewed two of its long seams and done them very nicely indeed. Now she was making a dress for her doll. Lucy's Indian robe was completed, even to the fringe, and she was practicing for the Indian dance her group was to give. Never were six weeks so short before. The girls could hardly believe it when they realized that the vacation school was almost over. All their work was to be on exhibition, and then would follow the dances and play by the older girls. All the fathers and mothers were invited, and Miss Chandler sent a special invitation to Uncle Dan because he had mended the broken tables. But Uncle Dan and Father Merrill did not have time to go to the entertainment. Only Mrs. Merrill. Lucy wore her Indian dress because she was to take part in that dance, and she put on her coral beads. They seemed quite suited to the occasion.  She also wore a blue sash and arcturus. All the girls in the Swedish dance were to be dressed in white and when they reached the school they would be given pretty colored headdresses. Quite a crowd had gathered that last Saturday afternoon. The baskets and the two bed covers in the doll's dresses and baby clothes were spread on the tables for the visitors to look at and admire. Each girl wanted her mother to see what she had made. Some of the children had done modeling in clay, and were feeling very proud of their work. Mary Barrows had made a flat sort of picture of her little sister, and it was so like a baby that Miss Chandler showed it to all the ladies and gentlemen who had come to the exhibition to see what the children could accomplish in the vacation school. Mrs. Merrill looked carefully at the spreads. Dora pointed out the squares she had cross-stitched or outlined, and the long seams she had sewed, and also the clothes Lucy had made for her doll. Mrs. Merrill felt glad she had taught the children to sew. She liked the tiny duct Dora had made from clay. They looked as though at any moment they might quack themselves away. When all the mothers had looked at the sewing and the baskets and clay figures, they sat down on settees to watch the entertainment. First came Lucy's Indian dance with all the girls dressed in their fringed robes and decorated with strings of beads. No one but Lucy wore a coral string, and Dora thought it looked very pretty. After the Indian dance, the older girls gave their play. They had practiced it inside the schoolhouse, and none of the children had yet seen it, so everybody was interested to sit on the grass and watch. It was an outdoor play, and told how a little girl named Hope wandered off and met a bluebird, who was just going to wait at the flowers and plants in the spring. Hope went with him and saw him call the different growing things. First, the useful plants like wheat and corn and sugar and potatoes, and then the different flowers, the violet and forget-me-not and daisy and apple blossom. Each was represented by a little girl who came and talked with Hope into the bluebird, and ended by reciting a poem. The bluebird was very funny in some of his remarks, advising the apple blossoms not to rent any nests to the English sparrows, because they were so dirty and quarrelsome, and thinking he might take one himself since the rent had not gone up since the previous spring, when it was to eat all the kinkarworms and let the bees alone. The potato was comical too, because she was so discouraged with the way inexperienced people planted her too deep, and with all the bugs that sat on her leaves and tasted them. The audience liked the play. It was funny and bright, and yet there were things that made you stop and think, as when the forget-me-not said she was going to cover all the battlefields of Europe with a pretty blue carpet, so that men would remember not to start another war. Next came Dora's dance, and she left her mother and Lucy and ran to the place Ms. Chandler wanted the girls. The dance was very pretty and Dora and Amy Conner danced alone before the others. Dora was pleased that she could do this, for she thought her mother would like to see her. Mrs. Merrill did like to watch her, and she also thought that Dora did it better than any of the other girls, better even than Amy. Dancing seemed to come very easily to Dora. Some of the other people thought so too, and when the girls had drilled and marched and the entertainment was over, they told Mrs. Merrill how well Dora had done. Mrs. Merrill liked them to say so, but she thought it of more importance that Dora could sew nicely. The Village Improvement Society had a surprise for everybody who had come to the last Saturday. Ice cream cones appeared for the visitors as well as for the children. This made a nice ending for the vacation school. After the cones were eaten the people began to go home, but Dora begged her mother to wait a moment. She was extremely anxious to show Arcturus to Ms. Chandler. This was the first time Arcturus had come to the vacation school, and he came only because Dora wore her best white dress and was in one of the dances. Mrs. Merrill said she could not wait because she must go home to get supper, but that she would walk slowly. Mrs. Soames was going her way. Dora might show the bear to Ms. Chandler, and she and Lucy could overtake them. Dora had to wait several minutes for Ms. Chandler, who was talking to some visitors, but finally her chance came. While Dora said Ms. Chandler, turning to her at last, it is good-bye, isn't it? But I hope to come to Westmore again next summer, and dear me, Dora, you'll be nine years old then, and I can't have you in my group. Perhaps you could take the second group next summer, Dora suggested. Perhaps I could, laughed Ms. Chandler. We've had such a good time together, haven't we? Ms. Chandler put an arm about Dora, and then she saw Arcturus. What a darling silver bear, she said. Did somebody bring him to you from Switzerland? She turned Arcturus over and saw his pretty green eyes. Dora was looking down at the bear. I wanted you to see him, she said. Mother let me wear him only once in a while, because he must never be lost. Ms. Perkins says all the silver bears she saw in Switzerland had red eyes. She never saw one before with green eyes. Ms. Chandler did not say anything at all. She looked at Arcturus, and she looked at Dora, and then she looked at Lucy, who had run up to say good-bye, and to tell Dora it was time to go home. End of Chapter 12, Recording by Campbell Shelp Chapter 13 of the Silver Bear. This is a LibriVox recording. All LibriVox recordings are in the public domain. For more information or to volunteer, please visit LibriVox.org, Recording by Campbell Shelp. The Silver Bear by Edna Adelaide Brown. Chapter 13, Dora's Name Who gave you the little bear, Dora? Ms. Chandler asked after quite a long time. I don't know, said Dora. I had it when I came to father and mother. It was on a ring with five other little things. Ms. Chandler looked very much puzzled. I don't understand, she said. Haven't you always lived with father and mother? She came to us in a big red automobile, explained Lucy. Uncle Dan and I pretended it was a chariot of fire, but she is just the same as ours now. Ms. Chandler asked the children to wait for her. She talked with Laura Carroll for a few minutes. Then she came back to Lucy and Dora. Dora, teary, she said. What were the other little things on your ring? A fish, said Dora. Ms. Perkins said it was a salmon and came from Norway, an English soldier, a pink coral heart from Naples, a Dutch sabote, and an Italian vampino from Florence. Ms. Chandler put an arm about Dora and looked at her very hard. Then she kissed Dora much to her surprise and patted Lucy's curls.  Lucy gave Mrs. Merrill the message. This evening? asked Mrs. Merrill quite sharply, but Lucy did not know. John, I want you to be here, said Mrs. Merrill. I'll be here, said her husband, but Molly, I don't see any cause for you to be disturbed. A man wouldn't, said Mrs. Merrill. When Ms. Chandler came, for she did come that very evening just as the shadows were growing long, Mr. Thorn was with her. The children were surprised to see him, but evidently their mother was not. After they had greeted the visitors, she told the children to go down to the fruit store and buy some lemons. You need not hurry back, she said. Lucy and Dora did not hurry. They were not often permitted to go down after supper to where the stores were, and it was exciting to be a part of the Saturday crowd. Almost three quarters of an hour passed before they came back. Mr. Thorn and Ms. Chandler were still sitting in the yard with their parents, and Ms. Chandler and their mother looked as though they had been crying. But both Mr. and Mrs. Merrill looked very happy. Come here, Dora, said Mr. Thorn. Would you like to hear a story? Dora liked nothing better. She settled herself at once on the rector's knee. Lucy sat on the grass at his feet. Once upon a time began Mr. Thorn. There was a little girl who lived with her father and mother in the far west in Colorado. Her father was an engineer who knew about mines, and he was busy helping a mining company get silver out of the rocks. The camp where they lived was very small, and the water they had to drink was not good. By and by most of the people in the camp were sick and many of them died. Among them the little girl's father and mother. There were no other women in the camp, and most of the men were sick. One day when things were at their worst, a man with a big automobile came through the camp because he had lost his way and blundered thereby mistake. The little girl's father had left a note, asking that she be taken to his old aunt who lived in Massachusetts, so the miners begged the man in the automobile, whose name was Brent, to take the child with him. Mr. Brent said he would take her to the nearest city and send somebody east with her. What was her name? asked Lucy. Mr. Thorn hesitated a second. Then he looked straight at Dora. Her name was Theodora Jodwin. He said, Dora looked surprised. Somehow the name seemed one she had heard before. When Mr. Brent reached the nearest town, the rector went on, he sent a doctor and nurses to help the sick people in the camp, but he finally decided that he would take the little girl all the way east with him in the big car. He found a kind woman who very much wanted to visit her sister, living near Boston. She said she would take care of Theodora on the journey. Mr. Jodwin's aunt lived in a town not far from here, but Mr. Brent did not write her that he was bringing Theodora. Perhaps he thought somebody else had done so. Perhaps he did not know just when he could get there, for of course he was a long time on the way. On the very last day of the journey he left the kind woman at her sister's house and started for Brookton alone with Theodora. It was only a short distance he had to go, but while he was driving he felt so ill that he stopped at a druggist's at Mr. Giddings' store across on the corner. There was something the matter with his heart and he did not get well. He never had a chance to tell the people who tried to help him about Theodora. Who do you think she is, Lucy? Our Dora? asked Lucy. Her eyes were big with surprise. You're Dora now, said Mr. Thorn, but she is really Theodora Jodwin. Dora slid off Mr. Thorn's knee. She went straight to Mrs. Merrill and put her arms around her neck. I am Dora Merrill, she said, and then she laid her head down on mother's shoulder. So you are, said Mr. Thorn, so you shall stay, but there is more to the story. Lucy looked at Dora, who turned one ear so she could hear. She was not going to look at Mr. Thorn again. Mr. Jodwin's aunt in Brookton did not know Theodora was coming, and for a long time she did not know what happened to her nephew and niece, and then nobody could tell what had become of Theodora. But the aunt was a very old lady, so it did not worry her as much as it might have done had she been younger and realized fully what a sad thing it would be if Theodora did not find people to love her. By and by the old aunt died, too. Theodora's mother had a dear friend. They grew up together and loved each other very much. This friend lived in Europe for a number of years and she once sent to little Theodora a ring with six tiny charms. Dora lifted her hand. Arcturus, she asked, and the Bambino and the others? Just those, said Mr. Thorn, and so when Miss Chandler found a little girl who looked so much like Gertrude Jodwin and who wore a silver bear with green eyes, like one which Miss Chandler had made purposely because all the other bears were red-eyed, and when this same little girl had the other charms Miss Chandler selected because she thought any little girl would like them, why, you see, it grew very easy to find out what had become of Theodora Jodwin. Mr. Thorn, said Lucy, Dora doesn't have to go away from us, does she? No, indeed, said Mr. Thorn. There is nobody for her to go to. Neither her father nor her mother had any other relatives. The old auntie left a little money for Theodora if she could ever be found. Not very much money, but enough to give Theodora music lessons now if she likes, and mother thinks best, and perhaps to send her to normal school or even to college when she is old enough to go. I am glad, said Lucy, so glad that the chariot of fire didn't stop in some other town on its way to Brookston. So am I, said Theodora, giving mother a great hug. She hugged father, too. Then, because Miss Chandler looked as though she would like to be hugged also, Dora did it. Thank you for Erectorus and the other charms, she said. Miss Chandler kissed her. You must call me Aunt Margaret, she said. Lucy shall too, and we will write to each other. I was very fond of your mother, little Dora, and if you did not have a home where people loved you, I should be glad to take you myself. Mother says you may visit me sometimes. There is something I want to ask you, said Dora. Do you know when my really truly birthday is? Lucy's is in May, but I was too little to remember when mine was, so mother said we would call us the 17th of September, the day when I came here to live. I'll tell you a secret, said Miss Chandler. Your birthday is just the same as mine, and both of them are on September 15th. Mother guessed very nearly right, Dora. When Mrs. Merrill came to take Lucy and Dora into bed that night, she gave Dora an especially loving kiss. Mother, said Dora, the six little charms are really mine now, aren't they? Yes, said Mrs. Merrill. Then may I give Lucy the pink heart? You may lend it to her, said Mrs. Merrill. There is one thing more, Dora went on. For every day I should like to stay Dora, but do you think on Sundays and times when I wear Erectoris I might be called Theodora? It is such an important sounding name. I think you might, said Mrs. Merrill. And now you may write it to Theodora, Jodwin Merrill. You see, we adopted you and gave you our name. Mr. Thorne says that Theodora means the gift of God. Father and I feel that you really are one to us. Perhaps the chariot of fire truly came from heaven, said Dora. Perhaps it did, said Mrs. Merrill. End of Chapter 13, Recording by Campbell Shelp. End of The Silver Bear by Edna Adelaide Brown.


Greetings fellow learners! Now before we get into this dazzling world of data requirements, I have a thought-provoking question for you. When preparing presentations at school or work, how many data-driven facts do you like throwing in? Now for example, in work I prepare small presentations to motivate why we should do project X. And when preparing my conjecture, I add little single facts that pull a lot of weight. And for example, the facts should be big picture and hence easy to understand, and they should also act as a motivator and a conversation starter. So for example, we could make a statement like, we are losing 7% of value per order dot dot dot, and so we propose project X to mitigate this. So turning the question over to you, how would you prepare for these presentations? Now please comment down below and I would love to know your thoughts. Now this video is going to be divided into three passes. One of the first pass, we're just going to go through an overview. The second pass is more details and model comparisons. And the third pass is code. So let's get to it. How much training data does a neural network need? Now this depends on multiple factors. However, four of the main ones are model architecture. The second is model complexity. The third is data quality. And the fourth is data diversity. So let's talk about each. So the first factor is model architecture complexity. Now this is a feedforward network that has parameters that need to be learned. If we increase the number of layers, we can increase the number of parameters that need to be learned. And this means that we need more data for training, generally speaking. Now the second factor is problem complexity. Neural networks, at least from a supervised learning perspective, need to learn how to map inputs to outputs. They learn this mapping using training data. And sometimes this mapping is easy to learn. To illustrate this, let's take a trivially simple example where the network input is the age of a person in years. And the network output should be the age of a person in 10 years. Now this can be solved with a simple function that just adds 10 to the input, of course. But we can also train a neural network to solve this problem. And we probably just need a handful of examples. On the other side, the mapping of the input to the output could be very challenging. So for example, language modeling, we want the network to take some text as input and produce some text as output for the next word. And now here, because the relationship between the input and output is complex, it needs more than a handful of examples for training effectively. Now, the third factor here is the data set size depends on data quality. So we need to ensure that the data used during training is a good representation of data during inference. So if we feed the model clean and crisp pictures during training, during inference, the model is exposed to not so clean and crisp data, then its performance will be low, even with lots of training data. Now the fourth factor is data diversity. So say that we train network that can take an images and classify them as cat dog or monkey. Now a data set with 1500 cats and 1500 dogs and zero monkeys might have a performance of 60%. But a data set with 1000 cats, 1000 dogs and 1000 monkeys may have a performance of say 90%. So even though the data set size is the same in both cases, the second case is more diverse than the first, and hence is able to generalize and perform better. Now there are many more factors that determine the size of the data set required. But these are among the top four. Have you been paying attention? Let's quiz you to find out which of these alterations to a neural network is likely going to decrease the amount of training data required a increasing the number of layers b increasing the number of dimensions of the input c decreasing the number of layers or d decreasing the dimensions of the input. Note that more than one option could be correct. So comment your answer down below and let's have a discussion. And if you think I deserve it at this point, please do consider giving this video a like because it will help me out a lot. Now that's going to do it for quiz time for now and past one of the explanation. But keep paying attention because I will be back to quiz you. Now one way to determine how much data that you need for your problem is to research similar models and similar problems. So let's take a look at some models outlining some factors that we described in past one. And let's start with a computer vision model Lynette five. So this model, it's a convolution neural network with the convolution activation and pooling layers. Now the problem that it is trying to solve is digit recognition where the input is a handwritten digit and the output is a 10 class classification digits zero through nine. Now the number of parameters in its original form is 60,000 parameters for Lynette five. And it was originally trained on a data set of size 60,000 training examples. And the performance of the Lynette five architecture achieved an error rate of 0.95%. So that's Lynette five. Now let's look at another architecture, which is Alex net. So the model here is that it is a deeper convolutional neural network than Lynette. The architecture takes in an image and it performs object classification where it's classified into one of 1000 classes. The number of parameters of this architecture is 60 million. And the data set size is 1.2 million training examples with 1000 images per 1000 classification categories. And this is trained on image nets. Now as far as performance is concerned, it's top one error rate is 37.5%. And top five error rate is 17%. This means that if we take the top one prediction from the Alex net predictions, 37.5% of the time, the true label is not in the top one. But if we take the top five predictions from Alex net, then 17% of the time, the true label is not in the top five. Now let's actually put Lynette five and Alex into a table for comparison purposes. And from this table, we notice a few things from a problem complexity perspective. Alex net is solving more of a complex problem. The number of parameters in the data set to the data set size, that ratio is one to one for Lynette, whereas it's close to like 20 to one for Alex net. Now from a performance standpoint, Alex net is still able to achieve good performance on a relatively complex problem. And this is also likely because of data augmentation, regularization techniques, among many others. So overall, I hope that this table shows how difficult it can be to answer the question, what is the optimal number of training examples? Now you can also observe similar comparisons for NLP models. One way to see this is comparing the glue scores of different models. So glue is a benchmark that evaluates a model on nine language understanding tasks. And each record in this table shows a model and its performance on each of these nine tasks. And their accuracies are then taken as a weighted average to create a glue score, higher the score, better the model. Now looking at these models, the problems they solve and their performance, you can get a sense of how much data that your problem might need. It's that time of video again. Have you been paying attention? Let's quiz you to find out. Increasing the training data by two fold will increase the model performance by a one fold B two fold C four fold, or D by an uncertain amount. Comment your answer down below and let's have a discussion. Now that's going to do it for quiz time for now. But keep paying attention because I will be back to quiz you. In this past three, we're actually going to look at a collaboratory notebook, where we're going to see how the data set size can affect model accuracy. So we first by importing some libraries, NumPy for tensor and array manipulation, torch for creating the neural network, the optimizer, and then scikit learn, we're going to use make classification for generating our synthetic data set, train test split to split the data into a training data set and a test data set. And then we're going to plot some pretty charts with map plot live. Now input dim is going to be the number of dimensions for every single example. In this case, it's going to be a 1000 dimensional input. Number of samples is 50,000. And the number of epochs during training for every single data set size is going to be 10. Next, we define our simple model, which is going to be a very simple feed forward neural network with three feed forward layers along with a relu activation and a sigmoid as well. We then generate this data set where we have 50,000 samples, each sample is 1000 dimensions. And all of those 1000 dimensions are informative. We're then going to split it split it into a test train set. So that would be 40,000 in the training set and 10,000 in the test set. Now we're going to train on the same model about 10 times over here. And each time we're going to only take like some fraction of this data. So for example, of the 40,000 examples in the first iteration, we're just going to take 10% of that data. So it'll be like 4000 training examples. So we train the model with that 4000 examples, we then define a loss criterion, which is going to be the binary cross entropy loss since it's a classification model here, we use the atom optimizer. And then we begin the training of this model with the 4000 examples, where we first zero out the gradients, we will make a prediction for the model, we then calculate the loss by comparing the models outputs along with the actual true label, we then compute the gradients with back propagation. And then optimizer dot step is going to actually perform the update of the parameters of the network and the model effectively learns. And we perform this 10 epochs, that's 10 times. Next, we're going to evaluate the model by taking the output of that model, and determining the binary classification. That is, if it is an output probability greater than 0.5, you have one class, otherwise it is the zero class. And then we just compute an accuracy. Now, if you look at the logs over here, you can see that each iteration, we're going to train the model multiple times. So we have a data set size of 4000, then 8000, then 12000. And for each of those cases, we're going to plot the train and test accuracy. So we get this beautiful chart. So you can see as we increase the data set size, the test accuracy goes on increasing and the train accuracy will go on decreasing. This is actually a good sign because that means that for lower amounts of data, that is when the data set size is small, we have a tendency of overfitting. That means the model is essentially trying to memorize patterns in the data. And so it performs really well on the train set, but it doesn't generalize very well. But as in when the data set size increases, you can see that we are overfitting less and the model will actually generalize better. And so overall observations is that as we increase the data set size, the test performance increases in less than a linear fashion, and we are overfitting less and less. And overall, we can say that while there is a correlation of increased data and better accuracy, this is not always the case. Now this correlation is less than one is to one, especially as we see more and more data. And so to determine how much data you need for your problem, we need to look at other similar problems and models and see how much data they require for training. And using that as a starting point, we can run tests on our own for our specific use case. And so I hope this answers the main question of this video. This is going to be a fun one. If you were to graph the amount of data on the x axis and test accuracy on the y axis, the relationship would look a exponential b quadratic c linear or D logarithmic. Comment your answer down below and let's have a discussion. And if you think I do deserve it at this point, once again, please do consider giving this video a like because it will help me out a lot. And that's going to do it for quiz time and past three of this explanation. But before we go, let's generate a summary. How much training data does a neural network need? Now this depends on four main factors, but there are many others. The first is model architecture complexity, then problem complexity, then data quality and data diversity. We can see how complicated answering this question can be when looking at computer vision models, or even the models on the glue benchmark. Overall, to determine how much data you need for your problem, you need to look at other similar problems and models and see how much data was required for training there. And using that as a starting point, you can run tests on your own for your specific use case. And that's all we have for today. Now the link to the code will be in the description below. But to learn more about training large models with their data requirements, you can check out my playlist on training a transformer neural network from scratch. Thank you all so much for watching. And if you think I deserve it, please do consider giving this video a like on your way out. And I will see you all in the next one. Bye bye.


Good morning, John. Your video about the time when you almost failed a class and then your teacher made you not do that reminded me of the time that exactly that same thing happened to me, which maybe says something. We'll get there. In 11th grade, I took a landscape design course, so I learned about plants and how to plant plants and take care of them and arrange them in people's yards. But that class, weirdly, was at a different campus, so after fifth period, I had to get on a bus and go to a different school. And sometimes I would not do that. Instead, I would go sleep in the band room. I skipped that class because, one, I wasn't taking it very seriously, but, two, my school had a policy that if you got more than ten unexplained absences, you'd fail the class. But before that, nothing happened. And so I kept very close track of my unexplained absences so that I would have exactly ten of them. Because I was that kind of self-important egotistical. Shurned out, into the semester, it wasn't more than ten, it was ten. The teacher told me that I would be failing the class and I got really mad. And I wasn't mad at the teacher, I was mad at myself for thinking I could be so clever and smart and being wrong about that. And I think it helped me realize that I was just sort of taking it for granted that I would always be let off.  And I went into the hallway, I like walked out of the classroom, and I kicked a locker door that was hanging open. I was wearing steel-toed Doc Martens because it was the 90s. And the locker door flew across the hallway. Like it just ripped off and flew away. And now I'm thinking, okay, well I failed the class and I've destroyed school property so I'm in much more trouble. I go back into the classroom, everybody's looking at me, and the teacher says out loud, I'll mark it down as nine. So somehow, John, I went from failing a class, then I destroyed property, now I'm not failing. What happened here? Even in that moment, it felt wrong to me. I didn't question it, because that would have been a bad day, headed home to tell mom about. I actually did a lot of kind of like bad stuff in high school, and I did it knowing that it was like a risk, but it wasn't a big risk. And I remember back then thinking that I was just good at getting away with stuff. Ugh. Why?  Because that's the thing, we're humans, we never stop doing the math. Prejudice and privilege aren't a sign of like some kind of interior rot. They're not a disease that monsters have, they're signs of biases that we all have and deal with.  I think a lot of America is really stuck on this idea that racism is a secret disease that only monsters have. There's a link to something that Gene Demby wrote that helped me think a lot more correctly about this stuff, and that says it better than I could. But a thing I've been thinking about is it seems like the more you have, the more people are wary of taking things from you. And the less you have, or are perceived to have, the more society is okay with exploiting you or punishing you or just giving up on you. I want a sociologist to explain this to me, but it seems real and it seems like both a cause of increasing inequality and the kind of thing that only gets worse with increasing inequality. And yeah, it seems like it's the source of a lot of bad. I'm really grateful to have been raised by parents and had mentors and teachers who helped me realize that I wasn't just some clever kid who was just so good at getting away with stuff because I still see people my age thinking they're that person and it's a bad look. But that instead I just was a person who had a lot, and so people were wary of taking it away. And that, among many things, is a good thing to know. And there are many other things that I think I still need to learn. John, I'll see you on Tuesday.


From Hollywood, it's time now for Johnny dollar How are you this morning? Oh pretty good. Sheriff. How about yourself? Oh, I'm fine Dandy You were over at the city hall this morning. Yeah, that's right I requested the corner to conduct an inquest into the death of George Henderson I know Huh? Yeah came into my office and asked me if I had any reason to conduct an inquest into the death of George Henderson I told him I didn't have any reason but I do it if I was ordered to what happens now Somebody'll have to decide whether there's going to be an in-question. Oh, yeah, I guess Anyhow you start up some action and you'll be getting it. Yeah, where just stay where you are son my guess is it'll come right to you Tonight and every weekday night Bob Bailey in the transcribed adventures of the man with the action-packed expense account America's fabulous freelance insurance investigators yours truly Johnny dollar Expense account submitted by special investigator Johnny dollar to Paramount insurance adjusters Hartford, Connecticut The following is an accounting of expenditures during my investigation of the Henderson matter The death of George Henderson of Culver Montana where I am today a Casual certification announced the death is accidental having been received by a fall from a hotel window No one in Culver seem to be too worried about any of the details, but details are my job That's why I requested the corn his office to conduct an in-quest I Took sheriff Holton suggestion and waited to see what my request flushed up in the dingy colored mountains of Culver An hour later my first bird winged up to my hotel room He was a tall gray haired man and a Stetson earmuffs and the Western version of a Chesterfield his honor mayor Newton Mr. Dollar, I want to talk to you about this request you made for an in-quest into George Henderson's death Yes, sir. Mr. Mayor. You are aware of course that George's death and he was one of my beloved and personal friends for many many years Was a great blow to the entire community. No, I didn't know that mayor Newton Only the smallest part of the community were at his funeral yesterday afternoon his widow and I'd say not more than half it as another people And I understand that your insurance company is not quite satisfied with the certification. Is that correct? More or less. What would they need to be satisfied, sir? An exact knowledge of how mr. Henderson fell out that hotel window I would rather know in quest were held into mr. Henderson's death. Why why sir? George Henderson is dead and buried it should remain that way if an in quest were to be held It would only prove that George fell out of a window. I beg you to consider that mr. Dollar You seem very certain that an investigation would prove that death was accidental It was accidental George fell out the window. Well, I can't just tell that to my insurance company. Can I? We are a small community with a rudimentary police force not equipped in any way for an exhaustive investigation Nor for the financial burden of such an investigation I strongly urge you to reconsider this request for a coroner's inquest you do I do indeed His untimely death was an unfortunate occurrence Outside the pale of any of our poor abilities to foresee A terrible terrible accident. I'd like proof of that Proof an inquest mr. Mayor An inquest All right We will continue with the Henderson matter in a moment Friends, how'd you like to thrill your favorite youngster with some of the most exciting toys of the year? Picture the breathless excitement of any child surrounded by six gaily colored balloon like giant animals up to three feet long And all for the low low price of just one dollar First you get bounce over clown with round pot belly and funny nose Mexico's hoppy the australian kangaroo third. There's rusko the ruler skating bear He's two feet tall and looks almost like real fourth There's whitey the fat indoor snowman and fifth Mortimer the giant mouse 18 inches long and sure to scare the whiskers off any cat That's five different giant animals But now hold your breath for the most sensational toy of all the star of the whole christmas season The jolly giant talking santa claus guaranteed to make everybody's christmas a merrier one He's a big roly poly happy santa. He stands erect on two legs Is actually over three feet tall and 32 inches around best of all he actually talks Just pull the tape and he says merry christmas for all to hear He's the biggest merry as talking as santa ever sure to please your youngsters and spread good cheer Yes, giant santa proves there really is a santa claus That's a total of six giant animals made of brightly colored preforms dirty latex Which the kids can easily inflate and the cost just one dollar not for each Just one dollar for all six of these lovable giants who'll turn your home into a circus parade And here's a surprise mail your order today and you'll also receive absolutely free peter the rabbit Actually over two feet tall with big red ears almost nine inches long But you must send now rush one dollar plus 10 cents for packing and mailing for each set you want to giant animals box 46 grand central station new york city If not delighted with every one of your seven giant animals return them to the super animals company for a full refund But keep the giant talking santa as our gift order now supplies are limited rush one dollar and 10 cents for packing and mailing for Each set in cash check or money order to giant animals box 46 grand central station new york city That's one dollar plus 10 cents with your name and address mail to giant animals box 46 grand central station new york city Giant animals box 46 grand central station new york city My interview with mayor newton had convinced me that i'd get no real help from him in the henderson matter quite the contrary Expense account item three 20 cents coffee myself and sheriff eve holton Well, you got it, huh at the direction of mr. Jackson. That's our corner He deputized me temporarily to conduct an inquest It's going to take place tomorrow morning nine o'clock city hall tomorrow sheriff room 207 Well eve you won't have time to do anything. No, I guess I won't not much anyhow, brother The mayor pitched me pretty hard for not having the inquest New he would any idea why nope You think somebody asked him to stop it. Yeah, who? Don't know johnny honest The next morning I struggled my way against a belligerent north wind to city hall in the inquest if you could call it that I sat in the back of the room and listened while a dr Horace nam assured the six-man jury that george henderson was quite dead when he was called out of his office and examined him on the street Dr. Nam reckoned george had died from a broken neck An ancient bellhop a desk clerk and a chambermaid gave their versions of what had happened the day henderson fell out the window I used to wear to tell the truth the whole truth so i'll help you god I do sheriff I'm the acting coroner today miss cubby Sit down Now When did you see mr. henderson last? Last thursday morning Where was this miss cubby at the butte hotel? You know what time of the morning it was About 10 o'clock. I went in to make his bed and straighten up his room I see I made his bed while he worked on some papers then and I left Did you see him after that? No, sir You didn't see him come downstairs for breakfast or anything? No, sir Do you know if anybody went up to see him? I believe I saw mrs. Henderson in the lobby after that Do you see mrs. Henderson in this room? Yes, sir. I believe that's mrs. Henderson over there. No, that's mrs. Henderson Now do you know if mrs. Henderson visited mr. Henderson in his room? No, sir. I don't know that Miss cubby did you happen to notice if anyone else went up to mr. Henderson's room that morning? No, sir It went on all morning long Sheriff holton acting in the coroner's position questioned person after person All had more or less the same vague knowledge concerning george henderson's death I was most interested in pauline henderson's testimony Now then mrs. Henderson, when did you last see your husband thursday? I went to see him about noon Maybe a little before Where did you see him mrs. Henderson at the butte hotel in his room there the same room he occupied prior to his death Of course the same room from which he fell. Yes Were you alone when you went to see him mrs. Henderson? Yes I must have left before 12 30. I had an appointment with the dentist And that was the last time you saw your husband alive Yes, I was still in the dentist chair when they told me he'd fallen out the window What uh, what did you and your husband talk about mrs. Henderson? Must I answer that? Well, we're trying to determine something here. I'd appreciate it George and I discussed our divorce Could you tell us about that? George and I decided to part about a month ago. He moved out of the house and moved into the hotel Outside of the divorce were you on good terms? Oh, yes, we've always been on good terms Mrs. Henderson, do you think there's a chance that he might have thrown himself out that window? No Mrs. Henderson, do you think he might have thrown himself out that window? No, at least not over us if that's what you mean As far as you knew was your husband in good health? Yes, he was You happen to know when he was examined last? Oh a month or so ago. He was in perfect health. One more thing Did mr. Henderson drink? Yes Did he drink that morning with you? I think he had a couple of drinks Yes, yes, he had a drink or two while we were talking. He could have stumbled to that window The clothes were new york the perfume paris the jewelry tiffany's The look you might expect it on the riviera where everybody tries to act bored with too many good things in life Her dress blank for the occasion of death was cut too well and took carefully for her to pass as a grieving widow She answered the questions without hesitation or emotion 15 minutes later the jury brought in the expected verdict Therefore it is the opinion of this jury that the sad deceased george walter henderson Came to his death as a result of injuries incurred in the fall from the fourth floor of the butte hotel At or about 12 45 p.m. On the 19th day of this month No evidence to the contrary. It is deemed and declared that the manner of death was accidental adjourned As far as culvers people its police and its mayor were concerned Yeah, the mayor Well, mr. Dollar, I hope you're satisfied. It was a pretty good inquest mayor newton I trust the official verdict of the jury will answer any questions your insurance company might have had on their minds and Clear this whole matter of them. I'll forward it to them and tell them the circumstances under which the inquest was conducted Mr. Mayor Sanders factory. I trust no But it served a purpose Now that everybody thinks it was an accident everybody will breathe easier Certainly. Yeah If everybody's relaxing like that Somebody's going to get careless See you, mr. Mayor Johnny dollar will be back in a moment to tell you about tomorrow's episode Friends send for your set of some of the most exciting toys of the year Six giant inflatable toys for only one dollar some up to three feet tall You get bounce all the happy clown hoppy the australian kangaroo Roscoe the two feet long roller skating bear whitey the fat indoor snowman mortem or the giant mouse 18 inches long And last but not least the great giant talking santa a roly-poly giant over three feet tall and 32 inches around the belly That actually says merry christmas out loud when you pull the tape that six sensational giant toys for only one dollar Made of sturdy gaily colored latex that the kids can easily inflate Send one dollar for each set to giant animals box 46 grand central station new york city And if you order right now you get peter the rabbit over two feet tall absolutely free If not delighted with your giant animals your money refunded immediately order today You may never hear this offer again rush one dollar plus 10 cents for packing and mailing and cash check or money order To giant animals box four six grand central station new york city That's one dollar plus 10 cents for each set with your name of the dress To giant animals box 46 grand central station new york city Now here's our star to tell you about tomorrow's episode of the henderson matter people do get careless tomorrow All over calver montana Join us won't you yours truly johnny dollar Yours truly johnny dollar starring bob bailey is transcribed in hollywood written by john dawson It is produced and directed by jack johnstone Be sure and join us tomorrow night same time and station for the next exciting episode of yours truly johnny dollar roy rohan speaking


Okay, good morning everyone. Thank you for coming. This is the HPC big data and data science the room organized by myself Kenneth and Roman who just arrived and Our first speaker for today is Kenneth and he's gonna tell us how to install software for scientists on a multi-user HPC system All right. Thank you Vaz What I'm gonna do this morning first of all, thank you all for being here this early I'm sure you had too much beer last evening. So thanks a lot for coming What I want to do is compare five different tools To get the scientific software installed on Supercomputers so these multi-user high-performance computing systems So first of all I wanted Look at the title of it so installing software for scientists so this can be quite challenging Scientific software has its its quirks. It's very different from normal False software where you can just run configure make make install and has a man page and everything is nicely documented That's not the case with scientific software. It's very very different Good software engineering practices are rather rare in my experience in Scientific software so the installation procedure is very non-standard. You have to figure things out from scratch like for every separate Application almost dependency how happens here, too So these are scientists writing software and a quote. I don't know what I got this quote But if we would if we would know what we are doing it wouldn't be called research That's what scientists do So they mostly care about the science then don't really care that much about the software They just want to get the science done and they need to Write software to get that done so most of these people are not trained software engineers Or system administrators. They just figure things out by themselves mostly and if it somehow works, they're already quite happy and Also on the flip side not everyone not every scientist develops their own software Sometimes they just want to use after that other scientists develop and they want to make it easily accessible to them So they don't want to install or compile it themselves. They barely know what a compiler is. They just want to get the thing working The super computing aspects of the multi-user high-performance computing Systems these are very different from a normal laptop in several ways Both in good and in bad ways, of course the main thing about supercomputers is that you get a lot of performance A lot of parallelism out of these things they can do a lot at the same time But also in bad ways, they are very different. They're not that easy to use especially not for scientists who most most scientists have never seen a terminal up close And then you give them this black window with only letters and you tell them you don't need your mouse anymore Yeah, people get confused There's also a wide variety of users this biologists chemists people doing like history studies of old newspapers and Everything in between So you need a lot of different software Very different software you need also multiple versions because some people want to use open from for some people won't use open from extend And there's lots of variants of the same thing and in ways that are important to them So you have to have everything installed and typically and there's there's other ways of doing this But on a lot of systems once you install software it stays there as long as the system is alive So scientists write papers and two years later They only revisit the paper or they end up writing a PhD that includes that paper and you want to redo some of the runs and Hopefully with the same software or at least as close as possible and of course performance is very important on supercomputers If you buy this big expensive hardware with a really good expensive network, you also want to actually use it properly and get the most out of the system So You don't want for individual scientists. It's good that their Runs are 10% faster with a little bit of effort, but for the administrators of the system 10% is huge because that's 10% more science, right? That's really what you want to aim for So some disclaimers here and I'm doing a comparison between five different tools, but I'm certainly biased So I'm the lead developer of one of these I Tried to make this a very objective comparison. I really tried hard to do that and I hope you'll agree with me that that sort of worked I'm very familiar with easy build the other four not so much But I spent hours in the last couple of weeks playing with these tools reading the documentation Installing software them to try and from earlier familiarize myself with the tools and yeah, that was a lot of time I spent since last Christmas on this There's certainly personal bias still there Retreat to minimize it and I sent my draft presentation to like the people behind these tools like tops Who's giving the next out on SPAC? Give some feedback Ludovic and the gibbick's community. So all these people I got some feedback and basically all tools are covered So I'm not going to tell Any serious lies anywhere unless they missed it in the draft So yeah, that was my attempt to make it very objective So I only have 20 minutes. We have to rush a bit. So I'll do 30 second introductions of each And I'll do them alphabetically So come on is a tool that works on limits macOS and windows with cross-platform Implementer in Python. This is really targeted towards end users towards scientists So it's very easy to install software at conda. Just do comma install something and it goes and does it's magic Basically what it does it pulls in binary packages or pre-built Packages from the anaconda cloud or whatever they call it This used to be Python specific, but they Have now support for C C++ for strength are basically anything out there. So it's no longer Python specific The packages themselves are written in YAML so YAML configuration format together with a build script So it's shell or a bat Depending on the OS And build packages with conda build which is typically not what users do. So this is what the people that prepare the packages do And they're hosted on GitHub and all that stuff. They support about 3500 different software packages and most of these or a lot of these are scientific software easy build It's pretty different compared to conda. It only Mostly works on Linux. So it's very focused on supercomputers It sort of works on Mac OS, but not really. We never tested there It's definitely not a target platform. It also works on Cray supercomputers. These are different enough from Linux. It's It's worth mentioning that it's also implemented in Python. This one is mainly targeted to Support teams of supercomputers so people that get installed requests from scientists have to install the software and then give it back to the Scientist so not people installing software for themselves and that's an important difference Easy build builds everything from source so it compiles everything from source. There's no binary packages that it pulls from somewhere And one of the key things about easy build is trying to get good performance for the installation So it targets the hardware that you're installing the software for and you'll see that's coming back in the talk Let's see it has these easy config files as recipes and it has Python modules Python scripts that have all the logic to install the software What else it generates these module files because that that's Being used on pretty much all supercomputers out there So people know this environment and it makes it relatively easy for the scientists to just access the software We have reasonably good support for site specific customizations to Installations and keeping track of that and today we support about a little bit over two thousand different software packages Then next this is again very different from the other two Cross platform, but at least on Linux and macOS and no windows Implemented in C++ and the next custom DSL they have the tagline it's a purely functional package manager, so it tries to Provide very reproducible builds of software. So it's very focused on that There's an operating system that uses the next package manager as well So don't confuse these two. It's two totally different things. You can use nix on a traditional Linux system You don't need to use nix OS if you don't want to very strong focus on reproducibility of Compilations even down to bitwise reproducibility So getting the exact same binary on two different systems through nix if you just ask it to do it to do the same installation And they do pretty crazy stuff like resetting timestamps to try and get to this bitwise reproducibility So they're very focused on this support atomic atomic upgrades of packages and rollbacks it By default it also downloads binary packages and if they are if it can't fight binary packages It will build from source. So that's not the combo It will just only install binaries while nix will build from source if it has to it has multi-user support As well so it uses the nix DSL for the package recipes and this has over 13,000 different software packages supported and that's not counting the 12,000 Haskell packages. So this is huge Of course a lot of this is not scientific software This is way broader than scientific software nix was not initially targeted at scientific software at all Then there's geeks or geeks HPC. There's a very good blog on Using geeks in an HPC context. That's mostly Ludovic and Ricardo and Piotr Write the stuff very good blog posts. So they're trying to explain how geeks can be used in this context where it has a lot of features that Make it really make sense in this context. This is GNU linux only implemented in a combination of scheme and C++ in my view This is mostly targeted to system administrators and experienced end users. So you I don't think you would give this to Let's say a normal scientist Maybe in the future you could The design is fairly similar to nix. It also has the geeks SD distribution. It works on can you heard if you want to use that? You can very strong focus on reproducibility It only supports free software. That's actually not really true, but it has some focus on that Same upgrades and rollbacks as nix does and over 6,500 packages And then spec is the last one in the row Also sort of cross-platform linux macOS and kray also implemented in python in my view and I know Todd disagrees with this It's mostly targeted towards software developers of big scientific software So they need to juggle lots of dependencies and it has a lot of flexibility for that It's similar to easy build in some ways, but different in in many other ways One of the main big differences is the flexibility. It gives you to Juggle dependencies. You can do very cool stuff like this You can tell spec install MPI leaks a particular version with a particular compiler and this dependency I want it locked to this particular version and then everything else that it needs back will figure out by itself That's certainly quite different from a diesel does All right, so the project comparison I want to fill in this table basically Well, you know compare all these aspects and I have this star rating system and the colors are pretty straightforward So releases I'll speed up a little bit here to avoid going out of time Coma has had many releases since 2012 You can quite easily install it with the shell script. So even a scientist that doesn't know how to use a terminal can do this It has self update support and it basically has very little dependencies It just chips with whatever it needs You don't need to do for installing or using conda, so that's important as well Easy build has been released since 2012 Before being developed in-house for three years It has a bootstrap script to install it or you can just pip install easy build It has self update support and it really relies on having environment modules there It doesn't work without environment modules. So you need this you need to have Python You need setup tools today and to get started. You need a C++ compiler on your system as well Here's also here no pseudo for installation or using of easy build so Normal users can install software without asking admins Nick's has been around for a very long time since 2004 Stable since 2012 so 1.0 has a Install script to install the binary release or if you want to you can build mix from source You have to get this build demon running because this is the thing that actually does the installations And this needs to run as root. So here you do need Sudo to install mix not to run it a normal user can talk to mix and get stuff installed And the installation is just done by the demon so the user is talking to the demon Self update support is there as well and dependencies. Well, not really unless you want to build it from source Then the stuff it needs has to be there It geeks so geeks is more recent on mix. It's no 1.0 yet. So still in data But it does a lot of the stuff that makes does to mix does to Let's just actually not true anymore. It says no installation script available. They fixed that Thanks to seeing the draft of my talk. So that triggered that that's good self-update support and have some minimal dependencies and then spec Handful of releases. Well, yeah since 2014 not not stable yet The installation here is a little bit different You just get clone the github repository and then you can get going straight away So you don't need to really install it. It is why I gave this a lower rating because It feels a bit off certainly if you give this to a scientist that like what this what is this good thing and For yeah, it feels a bit off to me. I know Todd disagrees here To updates packages do get pulled you pull in updates from the github repository and some basic dependencies So documentation comparing the documentation all of these have really good documentation I Gave easy build the lowest rating here because I know where the gaps are in the documentation that have to be filled in I'm sure there are gaps for the others as well, but I can't tell as an outsider. So Try to make this fair here So configuration so once you get the thing installed you need to maybe configure it with condo There's basically no configuration at all. You just and that's it That's very good with easy build you really have to do some configuration out of the box defaults Don't make sense for most people, but we can't come up with a better Default or at least I don't see a way It really really to tell it some things before it can get going Geeks and nicks I gave these a lower rating mainly because of the demon that you have to get running as user as Roots and you need to create build users So realize them to the build users with different permissions And also because both of these are locked to a specific location for the software to a new store and next door You can change it, but you're gonna suffer if you change it. So typically people don't And then SPAC has a very little configuration that you really need to do But if you want to you get a lot of options, so that's very good basic usage Yeah, come as very straightforward you install something you do this source activate of a script And then it's ready to go Yeah, this is creating the environment and this is the actual installation You can easily install other software versions and build packages yourself if you really have to So in terms of basic usage, I'm not making any distinction Everybody's getting three stars because it depends on what you want to do With easy build you search you get this easy config file You can use you do best as robot to make it install the dependencies and then you can load the module Installing other software versions is just changing easy configs and Or maybe writing some Python code yourself depends When Nick's fairly similar you can search for what it has Nick's ends to install things It links it in your profile. It's ready to go or if you really want to you can build your own software or software versions Geeks is pretty much the same just different commands and then SPAC you get all this flexibility of juggling dependencies and all that stuff And you can do SPAC load or you can load the models that it generates To install different software versions. You don't even have to dive into the Python yourself You can just tell it to install a new version and it will go and do what it can to make that happen Time to result. So how long does it take to install software with these tools all the things that do binary packages by default are very fast Seconds well with SPAC and easy build you're building from source So that's a big difference and with easy build you have to build the compiler Typically first before you can get to the software with SPAC can pick up the compiler in your OS So these are seconds to install this FFTW with easy build in SPAC It will take like an hour depending on whether you're going to pick up the system compiler with SPAC or not Performance I'll speed up a little bit here because I'm running out of time and basically want to get to this light So this is performance of an FFTW library with the five tools. This is second So lower is better. I see if you do binary packages if you install these these are generically built So you're not really using the hardware very well in this Set up SPAC and easy build compared from source so these can do a lot better They can target the hardware today SPAC doesn't do this out of the box So which is why it's pretty much at the same level as the binary ones it builds Generically by default, but you can easily tell it to do different easy build really targets the hardware you are all So that's why it's getting better performance Now I'm sorry thought for this, but when I was testing this I ran into an issue with SPAC So SPAC was because of a bug was building with minus or zero so it was quite slow This was fixed in hours my thoughts and it was a quick release They were just not aware of this, but it was something I ran into well while testing So this is definitely not the norm for SPAC don't get me wrong Lot of stuff we didn't cover I won't go into this too much each of these have their key features that are very important So definitely take a look and try to deep dive into all of these before you make a selection This is the summary of the table. So who won well it depends depends on what you want to do It depends on who you are depends on how much experience you have yourself whether you're who of these three you are Really depends on the use case your experience your profile what you actually want to do Do you care about performance or quick installations and so on? and Then one one cool thing was this talk because I sent a draft to a lot of people It told me making waves and stuff happened before I did actually the talk So SPAC had a bug fix release because I ran into the O zero issue that was fixed in hours. So that was really good I was I figured I was doing something wrong and it turned out I found a bug in a different project. So that was cool And it was fixed really quickly There's an easy installation script for geeks now that was it was sort of there But not very clean and they finished it because I was complaining about the installation Which was very manual. So that's good And there's a very good blog post by Ludovic who's giving a talk later today as well on the performance aspect of geeks and Basically the the possibilities they have to get rid of this performance issue Definitely read the blog post. It's very good. It has a lot of technical references to all the things you can do Other build tools, which I'm not gonna cover here I didn't cover most of these because they have a lot less focus on scientific software So I haven't seen people using these on supercomputers to install Software for other people and I didn't cover singularity and new docker. So these container things for HPC, which is now Happening as well. But if you're interested in that, you should definitely take a look at these two projects, especially singularity That's it any questions Yes, oh wait, you should take the mic for the questions. Where's the mic? It's for the recording. So you should take the mic or I can repeat the question. Go ahead The performance lie I'll take the one without the red bar to keep it fair These are not using evx, which is why they're getting bad performance They build generic packages so that you can install them and non AVX hardware It's packed because it doesn't do it by default, but you can tell it to Because here you're compiling from source So you can actually tell it to target your hardware and then it will generate AVX and it will be at the same level as easy build It just doesn't do it out of the box So I was showing you what you get by default. It basically has a warning slide like be careful You can do a lot better with SPAC very easily, but you need to be aware.  They're just not there yet. It's not happening right now, but you can Sorry easy build right. So if there's new like Intel Skylake has AVX 512 We don't really need to change easy build for this because the compiler does it We just tell the compiler build for this thing and if the compiler can do it, then it's fine Yeah, so we easy build is just telling fcw build for AVX By default that's the only difference. You can do the same thing with SPAC here It's harder because you have to rebuild the packages, which is let's say not that common or certainly in the geeks thing But yeah, Ludovic and You should talk offline with Ludovic about this because they really know how to fix this, but it's not there yet today All right So why didn't I cover DBM only supporting certain Linux distributions? That's not really there I mean the things that work work for basically any There's certainly details there. Yeah, but in general it should work on any Linux distro each of these tools you mean CPU I So I know some people using easy build on ARM for example, I don't know about SPAC SPAC has good support on arm and power Yeah Then this was on power Arm so Oak Ridge Oak Ridge has been using SPAC on our network so really well. This is mostly So it's Python so Python runs anywhere as long as you have a compiler that There are some details. Yeah, but as long as your compiler can handle your architecture, you're fine No, well, if you disagree we can talk about this, but on a high level. There's shouldn't be an issue There's certainly detail it if you try easy build of SPAC out of the box on Something sexy like arm or the new power you you may see more issues than on x86. That's true But in principle it should work and it's just yeah All right. Yeah Yeah, last question Yeah Yeah, not necessarily as admin for many of these the user can do it themselves Yeah For for most of these and once also for nicks and geeks once the admin has provided nicks on the system and geeks Users can install their own software Not necessarily it can be for users It's also for admins to install software for the users because they don't know how to do it Yeah, you can do a combination of both He can't yeah And I think on that as a bad fit for providing software to other people you wouldn't use it for that Usable using themselves the other two is like SPAC and easy build are more targeted to work towards Yeah, but they will do it anyway, and then they Don't need permission so you cannot you cannot you cannot avoid them Sure Some people tell them tell their users to use condo. I know yeah There's There's all kinds of details there are also for the generic aspect so with geeks and nicks There is a way out with color. I'm not sure there's a way out for to fix the performance No But with condo users can install anywhere they can store in slash TMP and they can store in their home directory Sorry to interrupt you can yeah, we should take it outside. Thank you. Thanks everybody