Proudly we hail! From New York City, where the American stage begins, here is another program with a cast of outstanding players. Public service time has been made available by this station for your Army and your Air Force to bring you this story as proudly we hail Lieutenant Faye Adams, United States Army. Our story is entitled Army Doctor. This is the story of the first woman doctor ever to be awarded a commission in the regular Army. Our first act curtain will rise in just a moment, but first, if you are a registered nurse, there are outstanding opportunities for you in the Army Nurse Corps. Here is a nursing career with a future, offering opportunities for world travel, specialized training and a good salary. You will begin as a commissioned officer, enjoying the same prestige, pay and benefits as the male officers. You'll work with the best of equipment, have opportunities to develop your professional skills. In addition, you will enjoy a 30-day vacation each year with pay. But most important of all, you'll be doing your country a vital service. Act now. Find out if you are qualified to serve as an Army nurse. Right to the Surgeon General, U.S. Army Nurse Corps, Washington 25, D.C. Right today, your country needs you. And now, your Army and your Air Force present the proudly-we-hailed production Army Doctor. On March 11 of this year in Washington, D.C., an historic event took place, when First Lieutenant Faye Margaret Adams was sworn in as the first woman physician ever to hold a commission in the United States Regular Army. Behind the brief newspaper clipping which announced this event, lies a story. A story that began in a child's questioning mind, and was to grow from question to understanding to determination. Come along, Faye. Okay. Yes? When will Mommy get to come home from the hospital? Only a couple of weeks, maybe, if all goes well. I think in a couple of weeks. Will it be for good this time? Well, I hope so. But you always say that. Well, I always hope it. Why can't the doctor make her well? He's trying at it. He's helped Mother a great deal, but... But what, Daddy? Well, Faye, it's a little hard to explain. You see, Mommy's heart isn't very strong. And every once in a while she has to go to the hospital so she can rest, and so the doctor can treat her. Well, then he ought to make her well. Well, maybe he will this time. I wish I was a doctor. Yes, sort of, I sometimes. I suppose I have to grow up first. You have to grow up first to do anything. But I wish it didn't take so long. Well, it gives you a long time to think about it. Meantime, let's think about catching a bus for home, all right? All right. OK, come along. Yes, Daddy. When you're ten years old, you can wish to be a lot of things. A doctor, a fireman, a movie actress, the president of a bank. One day it's one thing, and the next day it's another with most children. Something made you want to be a fireman, and then that something is forgotten. And you look for an idol somewhere else. But in the case of Faye Adams, there was no chance of the something that made her want to be a doctor to be forgotten. Faye's mother suffered from a rheumatic heart. From the time that Faye was seven until she was 17, her mother was hospitalized frequently. And the doctor came so regularly to the house that he was almost like a member of the family. A childish curiosity about a mother's heart, about people being sick, and why couldn't the doctor make her mother well, grew gradually into an understanding and a desire. Again in the morning, if she isn't any better, then I think we ought to put her back in the hospital. It'll be easier all the way around for everybody. Whatever you think best, Doctor. Dad. Well, Faye, why don't you think you ought to be in bed? I was waiting for you, Doctor Henry's mother, any better? A little. Not as much as I'd like to see, but she's resting comfortably. Will she ever get really well? We hope so, Faye. That's what we're working toward. We're doing all we can. Oh, get my coat for me, will you, Faye, while I wash up? That's a girl. Dad. Yes, honey? What is it? Dad, will I get to go to college next year? Why, I don't know, Puss. You want to? Yes, very much. Why do you want to? Because I've decided... I decided a long time ago. I want to be a doctor. Well, there ought to be a way of working it out, I think. Oh, Dad, I was hoping you'd say that. I'm so glad. Now, wait a minute. Wait a minute, Faye. Isn't just as easy as all that. There's the question of money, you know. I think I can get a scholarship, but scholarships only pay tuition. I'd have to pay my own room and board. That's why I asked you if you thought I could go. Well, now I'll tell you what, honey. You look into this scholarship business and find out how much it's going to cost for a room and board, and then we'll talk some more about it. And I think maybe we'll be able to swing it if you can get your tuition paid for. Oh, wonderful. Oh, Dr. Henry, I have your coat here. Oh, thanks. Thanks, Faye. Well, I gave your father a new prescription to have filled. You'll see to it that your mother takes it along with the other medicine? Yes, of course. Well, that's fine.  You've been a big help to us all. Dr. Henry, I was just talking to Dad about going to college next fall. You're planning to go. That's fine, Faye. I want to be a doctor. Do you think I can? A doctor? Well, I don't see why not. If you're willing to work hard, it's hard work being a doctor, Faye. I know, but I think it's worth it. I've watched you and the other doctors with mother for so long. A long time ago, I decided that if I could spend my life helping sick people get well, I'd be very happy. I guess that sounded a little high flown, but I didn't mean it to. No. No, it didn't sound high flown, Faye. You have to have the desire to cure sickness, to see you through the long years of study, the long years after that. When I was a boy, I said just about what you've said. I wanted to make sick people well. That's what doctors did, so I wanted to be a doctor. I still have that same desire. It's that desire that sees you through the long days, the long nights, the discouragements, the heartbreak, the feeling of inadequacy that hits you time and time again. Good luck, Faye. You go to school and work hard and make your goal a goal to win. In the fall of 1936, Faye Adams entered the University of California as a pre-med student. By the year 1936, the idea of a woman becoming a doctor was not such a radical idea as it had seemed a generation before, but that doesn't mean that it was universally accepted. Oh, come on, Faye, take some time off for a change. Johnny, I can't. I've been taking way too much time off as it is. I told you I had an exam tomorrow, and it's going to be a tough one. All you think about are good grades.  All right, so I'm a grind. I have to get good grades to get into medical school. I can't understand why any girl would want to be a doctor. Why don't you settle for marrying me and raising five or six nice, noisy children? Because I don't want to. I want to be a doctor. All right, so be a doctor. That doesn't say you can't knock off tonight and go to a movie. But I already told you I can't. I've got to study for that exam. OK, that's the way you want it. But I guess you'd better count me out from now on, then. Well, Johnny, I already have a long time ago. And so Faye Adams pursued her studies and aimed for a goal and prayed that there'd be enough money to somehow see her attain that goal and didn't fully realize that there were now forces in the world at work which were going to affect both her and the goal she had set. The year that Faye Adams graduated from her pre-medical course at college was the year 1940. A large part of the free world was at war defending its freedom. America was watching and waiting uneasily for the time and the place to come that would find us defending our freedom too. But the war hadn't really touched us yet. The problems of the rest of the world had not yet become our problem in the fullest sense. And to Faye Adams, who wanted to be a doctor, who had dreamed of it and planned for it and worked and studied for it, there was a final important question to be answered. A question more important to her right then than the problems of a world across the scene. Oh, here's a letter that came for you, Faye. Oh, thanks, Dad. Oh, it's the letter I've been waiting for. Dad, you suppose no such luck. Well, that ends that. Why, what is it, baby? We're sorry to inform you that there are no scholarships available for women for this next term. If you care to apply for entry, we will be happy to consider your application. Very truly yours. Sure, I'd be very happy to apply. But on what? Oh, Faye, I'm sorry. Oh, Dad, it's not your fault. After all, I can't expect you to be able to put out several thousand dollars for medical school just like that. It's just that I've worked so hard for it. I've wanted it so much. I've got the grades it takes. Besides, I guess I'm just not a very good sport. Oh, there they are, honey. Now, don't cry. Something will come along. It'll work out. Maybe you can get a job for a while, and then? And then, when I've saved up enough money by the time I'm 50 and have forgotten everything I ever learned, then I can go to med school. I'm sorry, Dad. Just leave me alone for a while. You all right when I get used to it? On December 7, 1941, the Japanese attacked Pearl Harbor. On the following day, we declared war on Japan and subsequently on Germany as well. Our young men were drafted into the armed services. But too many of them were forced to do work far behind the lines, work that had to be done, but for which we could not spare the manpower. Now, the question of allowing women to enlist in an auxiliary service to the Army and Navy was raised. Congress passed the necessary law, and the first of these organizations, the Women's Army Auxiliary Corps, was formed. To many young women, enlistment in the WAC offered the opportunity to perform a vital and patriotic service to their country in time of war. Hey, what is the matter with you? No, nothing's really all right. Oh, come on. Come on, it must be something. Think I'm going to resign from the camp? Why? Oh, I don't know. The kiddie's been getting under your skin? No, no, it isn't that. It just doesn't seem very important somehow. Oh. Well, what are you going to do? Think I'll join the WACs. That way I can do something to help out my war. For heaven's sake, how do you like that? What's the matter? Is something wrong about it? Nothing at all. Why are you laughing? Because I was just about ready to do exactly the same thing myself. That's what I was starting to tell you. Except you weren't listening. Well, come on then. Let's get going. Fay Adams enlisted in the WAC in 1943 and was sent for her basic training to Fort Oglethorpe, Georgia. After completing basic training, she was to be assigned to a job. Fay, the assignments are out. Did you get yours? Yes, I was waiting for you to come in, Shirley. Well, where are you going? To the Aberdeen Proving Ground in Maryland. I'm to be a technician in the, let's see, what is it, say, the Ballistics Research Laboratory. I guess that's on account of all the physics and chemistry I had in college. I'm glad it's at least related to what I wanted to do. You still want to go to med school? Still? Can't forget in a couple of weeks which you spent practically your whole life aiming for. But you didn't tell me where you're going, Shirley. Washington, to the Pentagon. Well, that's close by. Maybe we'll be able to get together sometimes. We sure will. Fay, are you glad you did it? Did what? You mean join the wax? Yes. It's the best thing that's happened to me in the last three years. Private Adams. Yes, sir. I just wanted to tell you how much I enjoyed the paper that you presented to us. Why, thank you, Dr. Woods. It was an excellent piece of work and very nicely presented, which is more than I can say for a great many scientific papers I've heard given. Most of them induce as much sleep as they do facts. Well, I'm glad you enjoyed it. I certainly did. Are you making a career of chemistry, Private Adams? Well, no, sir, not exactly. I was a premedical student in college and then I wasn't able to go ahead with medical school. I still hope to someday. I see. Doesn't the Army offer medical training? Couldn't you go ahead with it in the service? It's not available for women right now, but they do give training in physical therapy. I've applied for that as, well, it's the next best thing. Well, I hope you get what you want. Thank you, sir. I do enjoy the work at the Ballistics Research Lab, but, well, I guess what it comes down to is that I'd rather work with patients than with chemicals. Which, when you come right down to it, is the way a prospective doctor ought to feel. Well, good luck, my dear, and goodbye. You are listening to the proudly-behaved production Army Doctor. We'll return in just a moment for the second act. Here's a message to young women with college degrees in physical education or the biological sciences. You may be eligible for a commission as a physical therapist in the Women's Medical Specialist Corps. Receive all the benefits of a second lieutenant while you attend the Army Special Physical Therapy course. Write the Surgeon General, Department of the Army, Washington, 25 D.C. You are listening to Proudly We Hail, and now we present the second act of Army Doctor. For the paper she had presented to the group of visiting scientists, and for additional work that she had done at the Ballistics Research Laboratory, Private Fay Adams was given the Army Commendation Rhythm. In October 1944, her application for physical therapy training was accepted, and she was sent to Lawson General Hospital in Atlanta, Georgia. In June 1945, because of her special training, Fay Adams was commissioned a second lieutenant, and after six months of working in Lawson General Hospital, she was sent to the Pacific area. Her assignment was to one of the station hospitals in Okinawa, where she was placed in charge of the physical therapy department. Major Harris, Lieutenant Adams reporting for duty, sir. Well, Lieutenant Adams, come in, come in. I've been expecting you. When did you arrive? This morning, sir. Have you been assigned a quarters yet? Yes, sir. Everything's taken care of. And you're all set to go to work? All set. Well, that's fine. I might as well tell you, Lieutenant Adams, that you've come to me with a very fine recommendation indeed. We're very happy to have you with us. Thank you, sir. I'm looking forward to it. Good. As you probably know, we have a good many patients here at the hospital, cases that we haven't been able to send back to the state yet. Now, your job is to help them get along to the point where they can go home, which I may add is what they themselves want to do more than anything else. I should think so. Well, if you're ready to go to work... I'm ready. And come along. I'll show you the wards. Take you on a cook's tour of the place. Now, the cases you'll be working on are mostly in this ward. We'll just go in and have a quick look around. Go right in, Lieutenant Adams. Oh, yes, I was expecting that call. Will you excuse me, Lieutenant? I'll be back with you in a few minutes. Of course, Major Harris. Have a look around. I'll be back in a few minutes. Hey! Hey, Adams, over here! Hey! Well, you're just about the last person I expected to see in this place. Johnny! Johnny, I had no idea you were even in this part of the world. That's no wonder we broke off communication, remember? Yes, I remember. Well, well, so you finally made it. And I never thought you'd really go through with it. How do you like being a doctor? I'm not a doctor, Johnny. I didn't get to go to med school. But I thought... Oh, what are you, a nurse? No, I'm a physical therapist. It was the nearest thing I could get to being a doctor in the Army. Well, this is certainly a new wrinkle. What about you, Johnny? What have you been doing since we last saw each other? Oh, nothing special. I got on the Army and they sent me out here after I got my commission. Then I got into some action and... Well, this is where I landed. Like you said, here I am. That's what happened. But how are you? Oh, I'll be up around any day now. You can't keep an old counter down for long, you know. Of course, it might take a little help from you, Doc. Johnny, don't say that. I told you I'm not a doctor. But you will be. As long as you've come this far, you'll keep going.  That's right. I've thought a lot about it. You know what I'm going to do? Marry me? No. Make you my number one patient. And that's something I never thought I'd be. You see, Johnny, you never can tell how something's going to turn out until it does. Fay Adams served for 14 months in the Pacific area as a physical therapist. She'd arrived there in December 1945, four months after the end of the war. But though the world was at peace, the business of national defense was going on. And current legislation was pointed toward permanent integration of all of the women's services of the army. For many years, women had been able to choose business and professional careers. Now, still another outlet was opened up to them, the career of national defense. It was a career that seriously interested Lieutenant Fay Adams. Come in. Did you send for me, Major Harris? Yes, Fay, come in, come in. I have some papers here that I thought might just possibly interest you. Transfer. Now, transfer back to the states, then separation from the army. Well, it'll be nice to get home and see everybody again, but... Well, but what? I like the work. I like being in the army. You could be proud of your service, Fay. Thank you, Major. Do you have any idea of what you're going to do when you get out? I know exactly what I'm going to do, if I can. Return to active duty. No, not just yet. I'm going to do something that I wanted to do seven years ago. Ooh, seems a long time ago now. Imagine what I wasn't able to do then and couldn't yet if I hadn't been in the army. I'm going to go on to medical school if I still remember enough so they'll take me. Good girl, I was hoping you'd say that. You'll remember enough. Look at all you've been doing here. You'll be way ahead of the rest of the class. Major Harris, I want you to know how much I appreciate all you've done for me in that line. Letting me go on board rounds and sit in on your conferences and explaining things to me. It's going to be a big help to me. I'm glad to help, Fay. How long have you been in the service now all together?  Well, then the GI bill will see you all the way through med school. Yes, and then? And then what? Then I think I'll try to get back into the army as a doctor. Well, in that case, I won't say goodbye. I'll see you around, Doc. Yes, sir. Following her separation from the army in July 1947, Fay Adams entered the women's medical college of Pennsylvania to continue her medical studies under the GI bill. In order to keep up her contact with the army, she joined the organized reserves and attended weekly meetings where she learned of the latest developments in army medicine. There was time for a little else. Medical school is a full-time job of study with sleep, a luxury. And after graduation from medical school, there's the year of internship. A full-time job of work and sleep are rarity. Yes, I hear it. I've gotten so I hear it before they even say it. What time is it anyway? Oh, 2 a.m., wouldn't you know? Why is it babies always pick the middle of the night to start making their appearance? You'd think they'd have the decency to wait until morning. Calling Dr. Adams. Coming. Calling Dr. Adams. Calling Dr. Adams. I think I'll change my name. I can remember a time not so many years ago when you longed to hear calling Dr. Adams. Remember? Oh, go back to sleep, plutocrats. Calling Dr. Adams. Mrs. Watkins, I'm Dr. Adams. I'll be taking care of you. Oh, Dr. ... I'm sorry to tell you all this way. But I'm so terribly afraid.  Everything will be all right. But the last time when I had my little girl, I don't know if I could face it again. Of course you can. It'll be much easier this time. You'll see. They say you always forget how bad it was. But I haven't. I wish I could have. Last time is all over and done with. This time it won't be nearly so bad.   I'll be right here. Don't worry about anything. Just think of the very nice baby that you'll soon be holding in your arms. I do feel a little better. Maybe it won't be so bad. Of course it won't. And before you know it, it'll be all over. That's what you should think about. Not what has been or what is, but what's going to be. Mrs. Watkins. Yes, Doctor. Would you like to hold your son just for a minute? Oh, we want it, boy. Edward, we're going to call him. Isn't that a nice name? Very nice. Does Fred know? Did you tell him? He's waiting for you down in your room. He'll let the nurse take the baby. Oh. Very nice baby, isn't he? Yes, very nice. And you were a very nice patient. Wasn't so bad now, was it? No. Doctor Adams. Yes, Mrs. Watkins. I want to tell you. You saw the medal that I wear around my neck. Yes. It's a miraculous medal. When I was so frightened before, I prayed. I prayed for a miracle. And the miracle was you. No. Not me. Your faith is my faith. And the help we gave each other. When she completed her internship, Fay Adams volunteered for a year of active duty as a reserve medical officer. And a few months later made application for a commission in the regular army. An application made possible by legislation enacted by the 82nd Congress, authorizing women doctors to hold regular army commissions. On March 11, 1953, a commission was granted. I, Fay Margaret Adams, having been appointed an officer in the Army of the United States in the grade of First Lieutenant, do solemnly swear that I will support and defend the Constitution of the United States against all enemies, foreign and domestic, that I will bear true faith and allegiance to the same, that I take this obligation freely for the purpose of evasion, and that I will well and faithfully discharge the duties of the office upon which I am about to enter. So help me God. And thus, Lieutenant Fay Adams became the first woman doctor in the regular army of the United States. It was an event conceived in the belief that women are equal to men in capability and service. It was an event foreshadowed in two world wars that freedom cannot survive unless it is continually defended. And that the defenders of America's freedom be cared for by the best in medicine and medical personnel. It was also an event fulfilled by the dreams, the struggle, and the achievement of one woman. This has been, in essence, her story, but it could be the story of any woman, and it will soon be the story of many women. With the rapid expansion of the United States Army, more nurses are needed. If you are a registered nurse, volunteer for service with the Army Nurse Corps. There are assignments in this country and overseas. Only qualified graduate nurses can fill them. Write the Surgeon General United States Army, Washington, 25 D.C. Do it now. This has been another program on Proudly We Hail, presented transcribed in cooperation with this station. Proudly We Hail is produced by the Recruiting Publicity Center for the United States Army and United States Air Force Recruiting Service. This is Kenneth Banghart speaking in the same station next week for another interesting story on Proudly We Hail.


The Mickey and Minnie's birthday celebration meet and greet is going away in just, I think, all week. So I wanted to do it. I've never done it before. It's been here for a while and I've never gotten in there and actually seen them in their cute little outfits.  So let's go do this. It is three o'clock on the day of a party and look at how empty it is in here. I mean the park is going to close in just three hours but it's nice to see. Look there's no line over here. Wow! Gideon is just walking around. Look at that. And now they're playing checkers. How crazy is it? Gideon is just sitting over here playing checkers and no one's hanging out. Thank you. You guys are doing great. I'm just like, wow. That is awesome.  A lot of people don't know that and I think they have a new flavor that I really want to try. Right inside of the confectionary. You just go up to the little slushies stand. Oh look! They do have a glacier ice. Oh! So yeah, they give you free samples. How cool is that? I've always wanted to just buy one of these huge Rice Krispies and eat it. And look at that. It is gigantic. Why are they a Jack Skewinton on the back? Beaking of Rice Krispies. Look at this. You can catch them making them and molding them right now. That is awesome. Wow. I want it. I'll take that whole bowl. Wow. That looks so good. This is pretty amazing. How cool is that?  It's delicious. All right. Now it's time. Let's get inside. The wait time is 40 minutes but I do have a fast pass because I plan to come here just for this reason. I love all of the different signs that are in this room. It is so awesome. Casey and the bat. The three little pigs. Oh and of course my shaver one is right over here. The three cobble narrows. The three cobble narrows. And they have it all decorated in here. Look at those streamers. Nice hug. Thank you. Hi Minnie. Hi. Fabulous as always. Thank you. So we're going to do a picture. Sounds good? Yeah. Let's do a picture. All right. Bye Minnie. Thank you. Well that was super fun. They're very nice. I hope and I don't know if it's possible but maybe talking Mickey will come back here. Maybe that'd be fun. I really miss talking Mickey. I think that was a great great job there. The way that he just interacted with the guest. It was awesome. You know since I'm actually in the vicinity over here I want to do something that I actually really enjoy and a lot of people probably should do more often because it's a great experience. And that is to people watch on the patio at Tony's. Because let's be honest it's very few and far between where you're actually able to just sit outside, drink a beer and watch people walk around the hub at Magic Kingdom. And I think that is awesome. Look at this place. I love it in here. And people always knock it down because of the food. But the views are spectacular. Look at the seat I got. Ooh the Dapper Dan's are singing too. Seriously it's just so cool to hang out here and just relax. I got the Peroni. Look at that. It is so cold. I cannot wait. This is going to be delicious. Oh man. Nothing like a cold, cold beer on a hot day. So we also got some delicious bread to snack on. Look at that. Oh man this looks good. This looks like good bread. Add a little bit of that. That is good bread. I like it a lot. I decided to also get some wings because they looked fantastic. Well I don't know if they looked fantastic but the description of the wings sounded fantastic. Garlic Parmesan wings. That sounds delicious. Oh and here they are. Wow look at these things. They look very good.  Holy moly. That's a big wing. They are actually very good wings. I was a little hesitant because they're not buffalo wings. They're basically garlic with the little Parmesan on top. But wow. I don't know what it is. The meat is fantastic. I think they're fried and then baked. The main thing is that there was only five of them for $14 which really seems like a very small portion. But you can really take a plane. I got some free bread with it. And that was actually amazing because it was free and it was delicious. And I just enjoyed myself hanging out here. So overall the experience is worth the money. Another very cool thing about sitting out here during the Halloween party is you get to watch everybody in their costumes. And there is a lot of great costumes I'm saying. Well I think that's going to do it for me. I had a great time hanging out here. And I hope you guys did too. So love the life you live. We'll see you next time. Bye.


CircuitPython6Beta is going on, you should download it and use it because now's your chance to get stuff fixed. A lot of people are using CircuitPython and a lot of people are using it in all sorts of ways we never expected, so get the latest beta, flash your devices and more, and if you're wondering what's going on in the world of Python on hardware, you should go to Adafruit Daily and sign up for the Python on Hardware newsletter. This week, highlights in the newsletter include Halloween projects, or doing Halloween projects. I think one of the things that, and I wanted to, I'll go over a couple things in the newsletter, one of the things I wanted to focus on, Ketney was doing the audio version of the newsletter each week, and I think I can get back to it now too for a segment in the show, and we're focusing it on what things can you actually purchase and then do stuff with Python, and now Halloween, and one of the reasons we wanted to talk about that is, it's really hard to get those spooky sounds you want, that specific pattern for your costume, and it's always been really hard, it was hard to do with microcontrollers, if you have to download an IDE, if you have to do lots of... If you embed it as a header file, it's so complicated. So we have a Halloween countdown display matrix, Creaturized, Monster Mass, Halloween, we have Circuit Playground for Halloween, and Jack Laners, and we have all sorts of things that you can do with Circuit Python, where you don't even need to know how to code, but it gets powerful really fast. So if you want to have a robotic project that moves, and it says a scary sound, and has lights, you could do that with Circuit Python really easily, that's one of the things that was always hard to do with these DIY Halloween projects, and I've been working on this for almost 20 years now, all the way back in the world to make, where it's just like, wow, Halloween electronics is hard to do. It's why a lot of people just buy off-the-shelf stuff, because you can't just... There wasn't anything that you could just drop a sound, drop an image, now you're done. So anyways, there's a lot more Circuit Python music projects this week, some events, Hacktoberfest is every year, it'll be even more virtual, Digital Ocean sponsored this, so if you want to do some open source projects, check that out. PyLadies, we have some news about their latest events coming up and more. Scott's doing a really big deep dive this week on Friday, and he said he's adding this thing in his notes file, so with time codes to the URLs, check that out. We have some back-to-school stuff, I'll actually just say it straight up, like our back-to-school sales are way different this year than they were every other year, because schools aren't going back in the same way, so this is one of those times where if you have a lot of at-home learners, pick up a Circuit Playground Express, you can have them, you can flash it, and now they're using make code, or you can just flash it, and now they're using Circuit Python. So other bits of news around the web, you can see some of the things that we got out, like our Qtify board, there's sensors that you can... The Qtify got out pretty fast, people have already used it. There's an internet calibrated clock using ESP32-S2 using Circuit Python beta. Once again, that's the latest and greatest chip, it's Wi-Fi, it's ESP32, and it runs Circuit Python, so check that out. So Python for Kids Resources, there's a new driver for the SparkFun STU-USB5500, 4500 USB delivery, you can get that on read-a-docs, and you can check out just more stuff that we have in the bigger world of Python, it's all connected, and then we also cover some of the new products we have and more. Check out the number of libraries we have, we're up to 273, and circuitpython.org slash downloads has all the boards well over 400, and then we have Blinka, so check that out. That is our Python on Hardware News for this week, a couple more events. A lot of these are virtual now, and then as always, there's a lot of people that are looking for new opportunities, so we have our jobs board, you check that out as well. Okay, so that's our Python on Hardware News this week.


Hello everyone, I am Dr. Meena Muthaya, working in the department of Obstetrics and Gynecology. Currently I am the head of the department. The treatment of PCOA's mainstay is a lifestyle modification, which includes diet and exercise. Having said the diet, diet would be mostly of a less carbohydrate diet and reducing the sweet intake. And exercise is mainly, it's like a cardio part, most of it, plus stretching the pelvic muscles. That will help for a regularization of the cycles and to solve the infertility purpose.


paid $5 at a yard sale. What a great find. Sold for 311.46 plus shipping. Hey, Bolo buddies, thanks for watching. In this video, we're gonna talk about big money Bolo's items to be on the lookout for, items that you can buy low and sell for a profit. I'm gonna tell you where they got it, what they paid for it and what it sold for. These come from my Facebook group and I invite everybody in the group to share their big money Bolo. These are from December. Most of them sold on eBay and these are gonna knock your socks off you guys. These are items that you should be looking for when you're outsourcing. And I would love for you guys to come over to the Facebook group. It has over 100,000 members and share a Bolo. Best January Bolo is up right now and then next month will be February. The videos come out the month after of course. So come share a Bolo. I also have share your best plush Bolo. Vintage Steely Dan concert T-shirt 1993 single stitch Becker Fagan Faces Tour. This is what it looks like. There's your single stitch and there you go. I don't even know who this is. Every time I say that about a band, you guys always are like, how do you not know who that is? Do you guys all know this one also? It says, hi, court. I'm so behind on commenting, listing and overwhelmed completely. But here are a few of my latest. My eBay store is basket case mama fab finds. Finds on posh Mark, I'm sorry, posh Mark Mercari and whatnot too. Found this cool vintage single stitch T-shirt at my local thrift store. It's a Steely Dan and it was only $4. I grabbed it thinking it would be around 40. It wasn't a band I thought would fetch big bucks, but I was wrong. It sold for $135. That's fantastic. This is a Leap Frog Globe. I would not wanna ship that. I hate finding box for weird shaped items. Actually, this probably wouldn't be too bad. It is a Electronic Educational Eureka Challenge Quantum Leap Geography. So it is probably a great homeschool tool, right? Or for a teacher, a classroom, I don't know. She said, here's another one that I heard about from watching reseller videos. Found this for $5 at the local Goodwill about six months ago. We actually were laminating about, no, lamenting about how it would, ah, how it hadn't sold yet. And we're surprised when that very day it sold during a sale in my store for $67.50. It is Leap Frog Globe that talks and does capital distances, information, population, and even quizzes. So much fun. She said her hubby loved it and thought it was the coolest toy ever. I knew it would sell and it did take a little while, but it could pick up for sure. So five into $67.50. But she said it was long tail. Now I bet during, I know I sold a ton of Leap Frog during lockdown. Probably would have sold quicker then. The Bachman books for four early novels, Stephen King, 1985 hardcover first edition. This was an item from my personal stash. I have been reading Stephen King books since I was 13 when Carrie first came out. We are planning to move next summer. And since I only read books once generally, I decided to sell off my Stephen King novels. Much to my surprise, many are first editions and valuable. This one sold for $450 and the buyer is delighted. So am I. It is actually a collection of short stories he wrote under his pseudonym Richard Bachman. Bachman guys, there you go. And this one also comes from Basket Case Mama Fab Binds. Do you have Bolo's sitting on your bookshelf? Ooh. This one, Rare Turkey Creek, dim dot co, I can't say that word. There was just a plush in another video. So plush also, if you don't know, has that brand. Thanksgiving pumpkin Robin Kelso dinner scene, diorama, I think is how you say it. She said, this is something I never saw before. It sold in November, but I thought you might like it because it would be a good Bolo. This is by dim dot co.  Super cute and remarkably unbroken. Got it for four at the Salvation Army and sold it for 104, definitely one to watch for. So if you guys don't know, I'm looking at my phone and I'm reading exactly what they wrote on Facebook. So if you share in the Facebook group, you're gonna put where you got it, what you paid for it, what it sold for.  It's a great way to get exposure and share a Bolo. How cute is that? Did we look at the pictures? I don't know that we did. There it is. All right, the next one is this vintage Minnie Mouse animated angel tree topper. She said, I bought this adorable Minnie Mouse angel tree topper at a local estate sale over the summer. I did until the holiday season to list it and it sold in just four days for full asking price of 75 plus the buyer paid shipping. I only paid three for her. Thanks so much, Courtney, for all the great videos I've learned so much watching you. My eBay store is creations, et cetera. So it is a light up Minnie Mouse Disney made in China. That's what it looks like. All right, this one says I'm in Australia. In the seventies, there was a TV show Skippy the Bush Kangaroo. Every kid either had the plate, cup and bowl set or desperately wanted it. There's a Bolo here. I was in my favorite OP shop thrift store and saw the saucer for a dollar. I've never seen a saucer and I grabbed it, no comps. So I put it up for auction starting at 35. It sold for 77 and was picked up same day. The guy was thrilled to complete his set. Love watching your show and learn so much. And this is a best summer Skippy Bush Kangaroo saucer only rare. Hard to find and it's in great condition. Look at that. So I guess the learning thing here is this is from a TV show and it is a harder to find item. So you might have walked past it if you didn't know. Now this one we're probably not gonna find in the US but in Australia definitely be on the lookout. I know people from Australia are watching in other countries. So thank you for sharing. I appreciate it. And this gives us kind of an idea of what's selling in other countries. So I love it and appreciate it. Carlton cards, vintage holiday stickers, woodland animals, new old stock. Here we go guys. This is a bread and butter but something that you can probably pick up cheap and sell for a nice little profit. I recall Courtney sharing that the that she picks up vintage, I'm sorry, I am blah, blah, blah, blah, blah, blah. Picks up vintage stickers. So I took a gamble on a bag of random stickers at a rummage sale, church rummage sale that is for a dollar. These two packs of vintage Carlton cards, woodland, holiday animals sold in less than 24 hours. I listed at 15 plus shipping accepted an offer of 12. Still many stickers to go. My store is Jiffy Thrifty LLC on eBay. I did have these listed on Mercari and Poshmark for a couple of weeks, just cross posted to eBay last night. Perhaps I should start there first from now on. So I start my items on eBay and then I cross post to Poshmark and Mercari. Just started back on Etsy but I use list perfectly to cross post and there are different ways to use it. You can use the catalog, you can cross post directly. I have a link down below that shows you how to use list perfectly. I do want you to know that the pro plus plan is now out and it has the auto D list and lots of new features. So you may want to check it out and upgrade. I think you can use my referral code Bolo Buddies for upgrading also. So keep that in mind 30% off your first month. And yes, I do recommend starting on eBay. If you're just gonna start on one and cross post later to the others because eBay I think has the most potential to get a sale quickly. But Mercari and Poshmark, less people are on those platforms, I'm sorry, less items are on those platforms. So it's not as competitive and there may be less competition and different eyes on your items. So definitely consider it. I do have referral links for Poshmark and Mercari down below. Mercari you'll get $10 to shop and $20 to shop when you sell $100 worth of stuff with my referral link. And Poshmark is $10 to shop when you join with my referral links. So check those out. Cross posting, highly, highly recommended. This one comes from Picker Money Flipper. It is a Conchina theme Bolo tie with turquoise and coral sterling silver Western men's. It's a Bolo tie. It's a Bolo Bolo tie. There you go guys. I'm sorry, paid $5 at a yard sale. How on earth do you get sterling silver for $5 at a yard sale? What a great find. Sold for 311.46 plus shipping. Definitely check out Picker Money Flipper on YouTube. They have awesome what sold videos. They are mega Bolo finders. This is a 2005 Hallmark Beauty of Birds number one, Northern Cardinal Ornament, no box. She paid a dollar for it at a thrift store and sold within a few days of listing for 74.99 plus shipping. These sell for crazy money if you have the original box. This one comes from Prairie Picker. And this is one I did not know about. So I am definitely gonna be on the lookout. It even has a imperfection. And no box, no box 74.99. Imagine if you had the box. Here is another one from the Prairie Picker. She said, oops, paid 2.99 at the thrift store for this cast iron stocking holder. It's sold within a day of listing for 54.99 plus shipping. Vintage Tommie or Toomey Deepala stocking holder. And let's see what that looks like on the bottom. Maybe it has, no, I don't see any marking telling that brand. So maybe Google Lens. If you guys don't know how to use Google Lens, it is a free tool. I have a video that tells you how to use Google Lens. Definitely check that out. If you are a reseller, it will be your best friend. It is free. Put Bolo Buddies, Google Lens in the YouTube search bar. It will pull it up. Beat the Market, a scientific stock market system. First edition by Edward O. Thorpe. Bought at an antique market for $8, sold for 129.97. Iron Cabin eBay Store. Probably would have walked right past it. Look at that, you guys. Crazy. The next item is this vintage Navy Blue Leather Coach makeup bag case with mirror. Logo print inside. This is what it looks like a little, I'm gonna say that, yeah, that's vintage. It looks so vintage. How cute is that? Paid $4 at an estate sale back in February for the vintage Navy Blue Leather Coach cosmetic travel case sold for $80 plus shipping, best offer. This one is a vintage Kitty Surprise Hot Pink Mommy Cat with three kittens. So these are something that you buy and it's a surprise how many kittens or I think there's dogs also are inside. So this one, she paid $5 for it and sold it for $59.99 within five months. And that one is Prairie Picker. Prairie, the Prairie Picker also. Another home. Sorry guys. Wow, I need some water. Another Hallmark ornament. Keepsake Optimus Prime. Look at this. It's a Transformer 2014. Look what it sold for. Best offer, $89.99 for this. Not all Hallmark ornaments are equal, but this Transformer one is amazing. Paid $5.99 at a thrift store in November. Did have its original box, the Prairie Thrifter or Prairie Picker, but this says bird crazy. So I don't know, maybe they have two accounts. The next one here, bought at a local Goodwill for 99 cents, sold for my offer of $20.87. Don't ask me what this is used for. I have no idea. Selling as Edge City Traders on eBay and whatnot. Airbrush Templates, Skull Master. So this is an example of selling something that you have no idea what it is and you still make money on it. And it looks like it's new old stock, perhaps. The next item is this sermons of the Rev CH Spurgeon of London, 1858 first series, 13th edition. Look how old this is. How cool is that? Wow, it's got some condition issues. Store name, The Dog Patch Picker. I got this antique book at a yard sale for five and sold it within a few days for $99.93. The next one is got this Jelly Cat, Lovey, for free, sold for a full asking price of $79, plus buyer paid shipping, sold on Poshmark. My closet is a Rittberg, it's right here, guys. Wilbur the Whale, Lovey. If you don't know, Loveys can sell for big money. I have a whole category video on Loveys. You can type in Bolo Buddies, Lovey. The Rebel reseller also has a video on Loveys, check her out. She sells a lot of plush and does plush category videos also. So does Zombie Bargain Hunter, he does category videos if you guys are interested in those types of videos on plush, $79, right there you guys, Poshmark. And thank you so much for being here. Thank you for watching. Be sure to like, comment, subscribe, and share. And let's see if Puppy will come up and say hello. Hey, come here. You wanna come say hi? Come here, come on. She's saying I am too comfortable. Just put the camera on me. Say hello. All right, you guys, thanks so much for being here and thanks for watching.


Welcome to another edition of RCE. Again, this is Brock Palin. You can find us online at rce-cast.com You can also find us on iTunes as well as an RSS feed and get the entire back catalog there. I also have here Jeff Squires from Cisco Systems and one of the authors of OpenMPI. Jeff, thanks again for your time. Hey Brock, it's the middle of the summer. School's out.  I mean that's that's what all you academic researchers do during the summer. Am I right? I'm actually trying to replace our entire cluster by sometime next year, so I got a big big project coming up. No, no rest for the wicked. How terrible. No vacation for me. Okay. What are the deadlines we got coming up? Isn't there's a super computing boff deadline is coming up end of July? I think yeah, I think there's a couple super computing things, but I don't think there's really anything else that I'm aware of coming up So I think that's the only one. Okay. All right. Well, what do we got today? Okay, so today we are going to be covering shifter. This is a often requested topic and we finally got a hold of some people. So we have with us Shane and Doug. Shane, Doug, why don't you take an opportunity to introduce yourselves? All right. Hey, I'm Shane Cannon. So I'm a member here at NERSC at Lawrence Berkeley National Lab. My background is my educational background is in physics. So I've got a PhD in physics from Duke, but all my professional career has been in high performance computing and technical computing. For everything from system administration to being a group lead at Oak Ridge and a group lead here at NERSC. And right now I'm in the data analytics and services group where I work on trying to help users use our systems more effectively, especially on the data side. And then I'm also a member of a project called Keybase, which is a genomics platform that we're trying to build up. And hi, I'm Doug Jacobson. I also work at NERSC at the Lawrence Berkeley National Lab. My educational background is in bioinformatics at the University of Michigan. But I've spent my entire professional career here at NERSC, ranging from user services to systems administration and systems software engineering, which is where the shifter project really comes in. Okay, so where does what is shifter? Maybe I'll take I'll start off and Doug can chime in. It's maybe useful just to think about why we invented this. What we wanted to do is we wanted to we've got a growing number of users that are trying to use systems like Corey, which is our Cray system to do a broader set of things than just modeling and simulation sort of the traditional HPC workload. So we want to enable more users to use the systems more easily. And we saw containers, Docker containers is an interesting way to kind of open that door, but you couldn't really run Docker on our system. So shift was really a response to that problem and trying to think through the issues with Docker and but still trying to enable as much of that capability on our systems.  Anything to add, Doug? Yeah, so this has also been not a recent sort of investigation for NERSC. I mean, NERSC has been in our non capability machines. So so things like our traditional cluster systems we've been doing forms of virtualization for quite a long time starting with Shane's projects. So really we've been dabbling in this kind of thing since 2005. Yeah, for a decade or over a decade. Yeah, but but shifter is just the latest incarnation of that. But it's the first one that allows the user to directly control the every aspect of the environment. I would like to clarify though that shifter is not actually a container system. It's a container compatibility system. Yeah, we don't do anything. So, you know, a container would would imply both sort of the operating environment, which is really what shifters thinking about. But it also thinks about resource limitations. And that's really the job of the resource manager. And so actually one of the design principles of shifter is that it does not at all try to do resource limitations. It leaves that to in our case it's slurm. That's our workload manager. But so we really don't do a whole lot with sea groups with shifter and that's by design. Yeah. All right, so Shane, let me zoom in on something you just said there that was kind of interesting. So this is not a shifter is not a container system in itself and it's in addition to Docker. But you made this interesting comment that you can't run Docker on your systems. Could you give a little insight into why and what some of the problems are? Yeah, sure. And it's kind of multifaceted. So probably the main the biggest reason is sort of security concerns. So if you're trying to run stock Docker, if anybody's played around with Docker and sort of try to do this, you can once you sort of allow a user to use Docker, then you've kind of given them the keys to the kingdom on that that node or that system. Right. And so since we want to allow users to use Docker images, but also be able to access things like shared file systems, it really would be a non starter for us to just put Docker in an altered and just open it up for users to run containers. There's also other problems with in a kind of HPC environment that we're running on the craze. There's no local disk Docker. You can work around this, but Docker really assumes local disk and that would be problematic. We really want to make sure that the solution that we put forward can scale to the full size of our systems. And so there's constraints around that. And then we also wanted to make sure that we could allow we could run it on systems maybe that didn't have the most modern kernels on them. So some of the systems can be a little you guys know the kernels can sometimes lag in the HPC space. And we didn't want to have this dependency like you had to be running the latest version of the kernel to be able to use the system. So, you know, you just told us what shifter is. I mean, why containers in the first place? Why is this a useful technology for science? Yeah, and this is I think this is good because this is something we're really excited about. So first and foremost, I think what's interesting about containers is they really give you a framework for enabling reproducible science. So, you know, one of the challenges in technical computing is you're running your application, but things are changing out from underneath you.  You've got this auditable recipe of like this is how I created that, right? So one, you've got some that's auditable to once you've built it and and you've registered and say a Docker hub.  Others can pull that down and reuse that same image. So you really start to make it easy for someone that you can say here's the analysis I did and then that this other user from the community can go pull that image down and run the exact same thing that you did. And then and also you can do that too. So a month later, a year later, if you want to go back and reproduce those results, you just pull that image down and do that. So I think this is really powerful. And this is why I think it's going to be it's going to grow in adoption within the scientific community is more and more people understand what what this enables. So where does the name shifter come from? I'll let Doug take you because he came up. We were like, what name should we call it? Well, so actually, we had tried a lot of different things. And one of basically, I think that Shane originally posited that we should call it a Proteus, which is a shape shifting. Yeah, or something else thinking of some mythology kind of thing or something like that. And I was having a difficult time moving off from that. And so I decided to sort of latch on to the shape shifting aspect of it. And that's where this shifter comes from. Yeah. And so we talk about like sort of shifting into an image or an environment. And so that's sort of the idea there. So of course, a big question for all these container-ish type systems, whether they're relying on one or implementing their own is do you require root permissions to set this up or use it? Yeah. So the short answer is yes, we do. So what we do with shifter, and I should make this clear that these are very conscious decisions we made along the way. We tried a lot of different things. As Doug mentioned, we've been doing this for... Can we back up for a second and restart the question? Because I think what I'd like... I think... Can we clarify the question? Are you talking about, does the user need root privilege to introduce a container onto the system? Or do you need root privilege? To install the software. To set it up or to install the software? Or let's just clarify what you're... Yeah. Sorry. Actually, I was asking both. Okay. All right. So... Okay. So yeah, shifter does require some level root privileges. So the short answer is yes. We went through a lot of different iterations when we were developing shifter because like Doug mentioned, we've been sort of doing this for almost a decade now. So let me explain where we need privileges in line. So one of the... The main thing is the binary that does the runtime construction for the user, it needs to have root privileges. So it's a set UID binary and that's so that it can do a loopback mount of the file system. So what we do as shifters, we take the image, the Docker image, we unpack it, we repack it and squash it in, like I say, a squash file format. Place it out on the parallel file system. And then at runtime, when the user wants to use that image, they'll... We mount that up to a loopback mount and so that requires privileges. The rest of the operations don't. So we will drop privileges once we've done the things that require it. So at no time in any part where the user has control, do they have privileges? So that's why we can do things like allow them to access their file systems, access the parallel file systems. But there is this caveat that we need a set UID binary just to perform that main operation. Okay, so just to drive that point home. The system gunk on the back end requires some root level permissions to set things up. But from the user's perspective, they just say, use my container and it's running as them and they never any see the root level gunk setup stuff. They're just their user outside the computer and then their... I'm sorry, outside the container and then their user inside the container, no root required for them at all. That's correct. This was really important for us, right? Because we're running a system that's got thousands of users on it. We can't allow a user to potentially access somebody's else's data, corrupt it, whatever. Obviously we want to secure the systems too. So similar to the way, say a resource manager kind of has to do its job. It needs privileges to be able to set up the run and then move into that and switch to that user to start the job up. That's kind of what we're doing here with Shifter as well. But then also the user, but because of some of this and the way that Shifter is designed, this is one of the major design points of Shifter is that at least in the way that we've implemented at NERSC, although their Shifter gives the site a lot of flexibility to configure Shifter however they want, there is no need at the NERSC site at least with the way that we've configured our Shifter installation for a user to do something like file a ticket or something else to get a new container on the system. So long as they put their image out on Docker Hub, they can at any time pull down that image and start running with it under their own control. Yeah, that's right. Now you also mentioned that Shifter gives you some scalability controls. I think we've got the security part down. What kind of scalability advantages do you get? Yeah. Well, so this was actually, it was by design to be sure, but it was not actually, performance was not one of our initial goals with Shifter. It was mostly about productivity. Yeah, scientist productivity. I want to build my environment and then produce it. But the choice of how we pack the image and then put it at least on to our parallel file system and we've done a lot of tuning to get that just right for our installation. But then using this loop device mount, it turns out that that optimizes out one of the most challenging aspects of delivering software in very large HPC environments, which is that, for example, on our system, if you wanted to start up a simple Python MPI application across, say, a thousand nodes, we've seen that take as long as 28 minutes if it's done that naively. But running that same code through Shifter, it only takes seven seconds to launch. And the reason for that is that by distributing the image and locally mounting the image, it doesn't copy the data to the compute node, but it copies the metadata, the file system to the compute node. And so as Python is starting up and walking its hundreds of directories and files and everything in Python paths, so long as it's within the container environment, it gets much faster access and doesn't need to get locks from the central files and all the other stuff that goes along with it. Yeah, you think about normally it'd have to go to something like, in our case, we're running a luster on these systems and we also run GPFS, it'd have to go to those metadata servers to start accessing files. Really, you just have to do one look up to get the image, then you mount it into the kernel namespace of the compute node and now everything's in that sort of local kernel space, so it can be very efficient. We mount the image up read only, so that has some downsides to it, but it does allow it to scale more efficiently. And then there's some other things that we've done that we integrate it with the batch, we can integrate with the batch system and when that's done, that can give you some additional scalability features. In that case, what happens is that all of the processes that are started by the workload manager can basically share the same namespace, so we don't have to repay the overhead of setting up shifter again and again and again. Yeah, which is still low, but it does add something. So how do you handle the file systems and mapping in user names and stuff like that? So you said that you're handling it, you're the same user inside the container as outside, but how is that actually accomplished? So the shifter executable, so the job of the shifter executable is to sort of launch a process inside the container environment. And what it does is it actually collects all of the user information credentials from the external environment. And then it just, when it drops privileges, it assumes all the same identities. So there is no change in privilege outside from inside the environment. One of the things that we do with shifter though is, you know, we said that we bring in things like the shared parallel file systems. Now clearly, the user doesn't have those mount points probably in their image. So a big part of what shifter does is it actually, at runtime, edits the container that the user has presented. So the shifter does not give the user the container that they requested. It gives the user a version of the container that they requested based on site policy. And so the site can set up the shifter configuration file to say, well, this content needs to be copied into the container or these mount points need to exist and these file system mounts need to exist. And then that's really what the shifter executable is sort of legislating is how the container is set up and how it's expressed to the user. So once you're in the container, whatever user was that requested a container, you're that user. And so there's really no additional mapping that needs to be done in that case. So this probably wouldn't happen, but say they're executable in all the environments inside that container and when that container was built, it was built under a UID different than their own and it does not have world permissions on it. Do they basically get a permission denied when they try to invoke it on a machine that's enforcing these policies or does it do something different? Yeah, that's correct. I mean this is and it's probably worth noting there are because of these, some of the things that we do in shifter for security reasons, it can call some incompatibilities with a Docker image if it's not constructed quite the right way. So for example, I've seen cases where people have the software is all installed as root. It's put in the root home directory and then they go to try to run that container and they're like, I can't open access these files. It works when I run it in Docker. So luckily, these are very easy to work around. Generally what we say is, okay, just make sure you make any files you need to be able to access. Just make sure those are read world executable, world readable, and then everything pretty much works. We generally recommend people put the applications in standard paths. You might as well. It's your container. You don't have to put it in some oddball path. So sometimes people need to do a little bit of work in the image to get it to be compatible with shifter, but usually those are pretty straightforward. One of the things that we feel is that a container that's done well for this environment really should not require a complex environment at all. If your path is more exciting than bin colon user bin and your LD library path has to be anything other than empty, it's probably have made it more complicated than needed. So really like a common complaint where people are like, I can just yum install this or set everything up the way I went on my home machine where everything's in user bin, user local bin and everything just works. I should do that inside my container image and then I can have that same experience anywhere. Yeah, that's exactly right. And I think part of the problem is people just aren't used to thinking that way, right? They've always been constrained so they don't think about like, well, I just should make it look the way I wish it would look. Okay, so one of the things that I understand about Docker and please correct me if I'm wrong, is that when you have this Docker specification file, it actually goes and builds your container on the fly to potentially include like what Brock just said, like yum install, whatever, probably open MPI because all right thinking people use open MPI. So does that happen on your system too and does that happen on each compute node? Like is that distributed or is it centralized? How does that happen? Yeah, so maybe it's worth noting here that first off with Docker, there's kind of two stages to the Docker lifecycle, right? One stage is you build an image and that's where the Docker file that you mentioned sort of comes into play. That's a recipe for how this image should look. So it says things like inside this build environment, add these packages, you know, set these environment variables, you know, run these commands. And then if the result of that is an image, right? And then you could push that to a registry like Docker Hub. Now what, and then the second phase of Docker is you run, you know, you run an instance of that image that's a container, right? So what we're doing in shifters, we're trying to leverage as much of that as we can.  Docker provides those. And we don't want to build our own registry because Docker provides that. So what we do is we have a special component that's part of shifter where it will pull down the contents of the image, which are really just a bunch of tar files. It unpacks those, repacks them, and then squashes them into a single squash file system file. And then that's placed onto the global file system for the HPC system. And so you only incur that penalty once when you first pull the image. Now it's packed and ready to go. Whenever you start up a container using the shifter runtime, it just mounts that image in on the compute node. So there's no additional translation or conversions that have to happen. The user is able to cause an image to be pulled. So they get control when they build their image. And they get control to say, I need this particular image on the system. But it's entirely system software and system administrator configured software that is actually handling the image itself at that point. So the user doesn't ever actually have direct control over the image once it starts to come onto our system. Yeah. So we're doing a decent bit of work with Docker here at Michigan for supporting things like persistent databases, data science applications, and others, not on our big shared resources, but persistent services we provide. And a common thing we ran across is friends don't let friends use loopback. And I've noticed you've mentioned that a couple of times. What is different about this sort of environment that you consider loopback to be okay? Obviously, it works on pretty much any Linux system out there. Is it because you're not doing any IO internal to the container? Everything is mapped out to these external file systems? Yeah, I would say that's the primary reason. If we had local disk, perhaps we could copy it on and then mount it up as a native file system or something like that. We think in general, the loopback trick has been great for us. We've used it in a number of ways. We're seeing it even get picked up outside of Docker kind of use cases now. So it's something you have to be cautious with, but I don't think it should be feared. Well, and really the question on this is what is the rate limiting step that you're suffering from? And what we were, the problem that we were having was metadata operations traversing the parallel file system. So by using the loop device, we actually now make the loop device the rate limiting step, which even then it's still more page transfer than anything else to and from to get the data. So the loop device, sure, it causes a penalty of a kind, but the only place it's really causing the penalty is an application startup. And application startup is not where you should be spending. It's not where ideally an HPC application should be so you should be spending most of its time. Any sort of persistent read and write operations should not be targeted at the container. In fact, we prevent that by making it a read-only file system. Instead, those should be obviously directed to more traditional IO mechanisms, MPI or shared memory or even the parallel file system itself, which is tuned for being able to accept large amounts of reads and writes very efficiently, but not perhaps these locking operations of metadata. So you just gave me the keyword there to move us to a slightly different topic here. How do you handle these OS bypass networks and HPC class networks, whatever you want to use the phrase here. How do you handle MPI and all the extra special things that it does inside of a container? Does that present any problems as opposed to bare metal? So I think when we first started developing Shifter, we were really targeting these high throughput type applications, more data intensive applications, but at the same time we wanted to be able to use this for MPI style and traditional simulation type workloads as well. So we started looking at ways to do this and there are a couple of different ways. One, it's important to note, we just passed through the device file system. So inside the container, you still have access to those devices. You're still that user, so you don't have any extra privileges, but you can access them. So the other key thing that you sort of touched on that you have to deal with is, well, you know, a lot of times these are, the hardware is very specific. It might have a version of firmware on it and you need to match that to some degree in your runtime, right? So one model would be, well, you just have to keep your image up to date with the system you're going to run in. So if you need a version of, you know, RDMA libraries to match your OFED release or something, you've got to make sure those are in the image. And that's one model. We have another model that we support in Shifter that Doug really figured out the way to work this where we place the libraries, we take advantage of ABI compatibility, for example, and MPI. We put the libraries that are sort of part of the system and match to the hardware in a location on the system and then we volume out those into the container and set the LD library path so those get picked up at runtime. So that way they're automatically mapped in and used, you know, by the MPI application, for example. Let me make sure I understand what you just said there. So you're saying that, like, I have a container and I'm running, I don't know, Red Hat 7.4 and you're saying that you have, like, an MPI installed for Red Hat 7.4 or 7X just taking advantage of the ABI for Red Hat 7X and that gets mapped into my container so I don't have to do anything for it. Is that what you mean? Almost. What we would do is we have some example containers that already have it sort of set up the right way but I'll be frank with you here, Jeff. We have this working for MPI on our systems. We haven't yet had it work for open MPI but there's no technical barriers other than just some details of the Cray systems and sort of the state of the fabrics. But anyway, what we do is the user in their image would have a build of MPI that has the, it's a sufficient level to have the ABI compatibility and then the only thing that needs to get volume mounted and mapped into that container space is just the libraries, the correct libraries for, say, the Aries Interconnect or the Infiniband Interconnect of the system. So we accomplished this with Shifter's ability to inject content into the image. Yeah. And then it's really just we rely on the linker to do its job with dynamic linking. And so this is sort of, at least at large scale, HPC, this is sort of a novel concept using dynamic linking. Most of the state of the art has been static linking for a long time or statically produced binaries. And so we are actually taking advantage of dynamic linking to provide this compatibility. And this is actually itself enabled by sort of that earlier observation that we can now traverse large amounts of files quickly in application setup with the way that we do things. The thing to keep in mind with the type of system that we're working with relative to perhaps other systems is that we don't have any local disk at all. The entire OS itself is on a remote disk on these compute nodes. The types of pressures on the system are a little different, perhaps, than a locally installed cluster would have. So are GPUs basically handled the same way? Yeah, that's right. And at Narisk, we mainly have Xeon nodes and nice landings nodes, KML nodes, or files. But we are another partner on this project is CSCS. And I think it's important to call them out. They've been a strong proponent and they've contributed to the project. And they have large, as you know, PISDON has a very large GPU system. So they've been doing a lot of work to get GPU support within Shifter. And they've demonstrated that and they have most of the pieces worked out. We're in the process of getting that. They have their own fork of it that they've been doing a lot of that development on. And we're in the process of trying to integrate that work into the upstream branches now. But it uses the basic similar concepts as what we're using for MPI. So let me, again, make sure I understand this is complicated stuff here. So is the idea that your containers are really just mapping the file system? Because there's a bunch of different namespace types in Linux, right? And the word container is actually somewhat amorphous. Most people usually mean Docker containers, but there's all kinds of different flavors of them out there. Are you using all of those namespaces or just the file system ones so that you don't have to do anything special to get devices? Or how could you explain that a little bit? Yeah, so really, Shifter is not a container solution.  Yeah. Or a namespace call.  So in our sort of shared space version of Shifter, where you might have multiple jobs or in the same node, we do use the mount namespace for that version. But for a node-exclusive Shifter where you want the container to be somewhat re-entrant and things like that, we don't even use that. Yeah, there is just the charu. Yeah, Jeff, to your point, we mainly use file system namespaces. We don't use any of the networking-related ones or process-related ones. Or at present, we don't use any of the user namespaces as well. Largely because we don't see a need for most of those in HPC environments. Yeah. So you've mentioned Kray systems a lot and Kray has special runtimes and everything else. And I also know CSE has a lot of Krays. So does this require a Kray runtime environment or can this work on classic X86 clusters? Yeah, I'm glad you asked that because that is a misconception. I think a lot of people, because we partner with Kray in some of this work, they think it only works on Krays. But no, we have examples out there of people. We're running it ourselves on some standard clusters and then there are other sites that are running it on regular clusters as well. Okay. So then my next question is a follow-up kind of getting back to the GPUs and to MPI. What about shared memory applications? When I've seen a lot of these sorts of systems, you still use your job launch system, but they tend to still do, you know, process per core. You know, how do you handle launching one of these things where you want to do a threaded or an open MP application? Yeah. I think, I mean, the brief answer is we're not doing shifters, mostly manipulating the file system namespace and some of the environments set up, you know, like environment variables.  So shared memory space, as long as the user has access to it, they should behave the same way. Yeah. We port through all of Slash Dev. We don't do anything to obscure Slash Dev's Shemem. Unless there is a very exotic implementation and even then the site could configure shifter to bring in, so actually that is the case for a craze system, is that there are a number of paths that need to be brought through to make craze XP Mem stuff work properly and craze sort of argument credentialing system work correctly. And so long as those are exposed within the container file system environment, everything just works. Okay. So still just use your resource manager to launch shifter the same way you would have had at launch a process that you ported directly natively to the platform. That's correct. Yeah, that's right. In fact, when you run a shifter container image, the way it looks is, you know, in your batch job, you would say instead of saying, for example, Arizona's Slarm, you might say, S run, you know, binary name. In this case, you just say S run shifter binary name and that's pretty much the end of it. So a couple of episodes ago, we talked to Greg Kertzer about Singularity, which is another HPC based container project. I wonder, could you compare and contrast shifter and Singularity to explain a little bit of the differences between them? Yeah, sure. Yeah. And it's kind of ironic because both of these, Greg, he's no longer at Berkeley Lab, but when he was developing Singularity was and we're at Berkeley Lab too. So it's a little bit odd that two different solutions came out at the same place. But we actually had started shifter work quite a while back and then they were different. They were more different at one time. They've slowly kind of converged. So I'd say if I had to sort of say, what's the chief difference between them? I think the biggest one is shifter has this concept of an image gateway that we sort of maintain a central repository of any of the images that have been pulled in to be used, right? So any user can pull the images in but they go into a common location and the system maintains control over those images over their lifetime. So we think this is important because we've even heard of cases in some of these others where somebody will accidentally corrupt the image after they pulled it and then they go to run it and it'll break or it might break once they're already running. Since we sort of maintain control of that, we make sure that we never overwrite an image that's already been successfully pulled and registered. So I would say, you know, the fact that we have this central repository and the fact that we maintain control over the images is probably one of the biggest differences. Some of the other things is we've done a lot of work to integrate this with batch systems. So, you know, here we're running Slurm. We've got really good integration with Slurm but people have done it with other batch and resource managers as well. I think singularities got some examples of that now with Slurm as well. But, you know, we've been doing this a little bit longer and there's a few other kind of minor sort of feature differences that I think are intriguing. One of which is we can allow the user to create a file, a per node file that can get mounted into the image on the compute node that's writable. And so this is not visible across all the nodes, you know, that might be using that image. It's just local to that node. But this is a good way to kind of emulate local disk type performance in a system where you might not have any local disk like we do on our craze. So that's a feature that's actually pretty powerful that I think is still pretty unique to Shifter. One of the things you're talking about was the ability of Shifter to be able to take, you know, IOP intensive startup applications like, you know, an MPI for PI applications and things like that and really scale up some of these systems. So what's the largest job that you know of that's been started up under a Shifter environment? So actually just last week, and we got permission from the people that did this to talk a little bit about it, they were able to run on 9600 Intel K&L nodes with Shifter and they were running a Python wrapped application to simulate their next generation. I think it was the cosmic microwave background. And so that was a single job run through Shifter. It was an MPI job that ran with over, I think, 158,000 ranks across the system. That is, I think, the current largest and that did real science to be able to simulate enough data to really sort of plan out their next generation data collection capability, which is, you know, I think they said the Planck satellite had 70 data collectors and their next version is going to have 50,000 data collectors. Yeah, it's like 35x the data output or something on Planck. So we've done, you know, dynamic benchmark style runs ourselves to kind of make sure things are working correctly. But this was probably the biggest real science example that we've come across and so it's pretty cool. That's just in a single example. What Shifter's also done on our system is it's enabled entire new workflows that now just run in production all the time. And so if you were to sum up the compute time dedicated, I would say that the high-energy physics community has probably benefited the most at NERSC functions. Yeah. Because they're able to now package up their applications, have it work, excuse me, other applications. They're a higher framework of software and those images tend to uncompressed when they exceed over a terabyte in size. And they're able to package those up and now make you full use of Cori to do their work. So those are the kinds of things that we really wanted to do with Shifter's. Enable that. Now, by the same token, and this is a question we ask a lot of our guests on here too, is what is the strangest use of, or the most unexpected thing that you've seen Shifter used for? And perhaps this is something outside of NERSC because you're using it a lot in production, but have you gotten a strange user request from someone saying, hey, I'm using Shifter this way and you're like, wow, never thought that would happen. Yeah. I don't know if we've got any things that are just really absurdly strange. I think we've seen ones like Doug mentioned where the LHC folks are using it. And this is kind of what they're doing is if you've ever heard of CVMFS, this is their distributed kind of file system that the LHC folks use. What they've done is they take the whole distribution of CVMFS and create one big mega-squash image from it and they don't push it through Docker because it's just too big. We have mechanisms in Shifter to allow you to directly import that and sort of register it in the system. So that's probably just by sheer counts is one of the stranger ones. But I think as more people use it, we'll probably come across some more oddball uses of it. To be sure, we don't know all the use cases. We can certainly see a list of all the images that people are using and there are many users accessing Shifter on the system today. Mostly they seem to be not filing tickets. Yeah. We're not always aware of how they're using it. Until after the fact sometimes. So what license does Shifter develop under? Yeah. So it is an open-source project. It's up on GitHub at github.com slash nurse slash shifter. And it's under a BSD, a modified BSD license, which is the labs kind of preferred choice. And we think this is a good license choice too because it's easy for if vendors want to kind of pick it up and maybe support it themselves, there's more latitude there. So what are some interesting things that are happening in the Shifter community? What do you have coming up in? What new features do you have coming up in new releases and things? Yeah. We've been trying to get a new release out for a few months now. Doug has been distracted with a small 10,000 node plus system that he has to help support and make operate. But we're hoping to get a release out very soon. Some of the things that that release, we already have these things working, so we're just really trying to get it buttoned up. But that adds private image support. So there are cases where people have images they don't want to make fully public. And so this will add support for that. There's some metrics capabilities and built into it now. Number of runtime sort of enhancements. So that's right on the horizon. Hopefully within a month or so or less. Also, it adds the GPU. It'll be adding the GPU support that CSCS has been really instrumental in. The other thing that further out, what we're looking at is we want to start leveraging layer FS support to do some more clever things. We mentioned that the nodes are, the images are mounted up read only on the nodes. And with the layer FS, we think we can provide some read write capabilities as well, which will make it a little more flexible than it currently is. So those are probably some of the bigger things. Oh, we're also working on more integrated support. I mentioned these cases where we want to import images that are maybe already prepped. So we've got some work underway to try to make that a little more streamlined and user friendly. Okay guys, thanks very much for your time. Where can people find shifter, download it and get involved? Yeah, so as I mentioned, probably the best source is to go to the GitHub site. If you Google, there's some pages on the NERSC website as well. I forget the exact pass, but if you Google it'll show up. And we do have a mailing list that people can subscribe to, as well as if you're really going to get involved in development or something like that, we have a Slack channel that we can invite people to help contribute. But the model is similar to how other open source projects work. So we can fork it, make a PR against that, and we'll review it and hopefully incorporate it. Okay, thank you very much guys. Thanks for your time guys. Oh yeah, thank you for having us.