Thank you for introduction. So today, my presentation will be about the solid solution for UIV point clouds, how raw point clouds can be classified, and possible examples of end products that you can export from that classified point cloud. Why I choose to concentrate on data collected by UIVs. So launch of affordable LiDAR systems, I think, changed market a lot, and LiDAR system become more affordable. As well, spatial data collection before was more like a large-scale project handled by national mapping agencies. But the drone can be deployed easily in any areas. And then, basically, the start of data collection is quite easy. As well, multi-sensor integration is easier. And besides LiDAR data, you can collect images, RGB, thermal, multi-spectral. And as well, Thera-solid did quite a lot development for UIV data processing and simplified the workflow quite a lot. So this is what I want to talk in this presentation. Who knows already Thera-solid software? OK, there is some hands. So Thera-solid is the software for point cloud and images processing. A bit about the company. We are in Finland, the headquarters. And we have users all over the world to have the idea how many tool sets we have. So we calculate that it is for eight industries. And what kind of data you can process. So any point clouds, no matter the source, sensor, and the scale of the project. I will concentrate today on data captured by UIVs. But basically, any data as handheld, collected, or mobile data, as well for the geometric point clouds, also possible. Classified and vectorized data can be used in planning, construction sites, maintenance. And in this example, you can see how different attributes can be used for data classification. For example, dimension, echo returns, then also elevation from ground, intensity value. The paint markings are detected using intensity value. Cars classified using grouping method or segmentation. As well, traffic is removed using moving objects routine. Then slope can be used for rats and potholes detection. The triangulated model can be helpful to vectorize, for example, tram lines. As in this example, classified buildings can be as an input for 3D building models creation. So software can automatically vectorize those. And later, tram lines, buildings, detected trees can be used in further modeling. So basically, you need to classify the raw point cloud in order to use it as a real-world representation in further models. Now, about workflows in TeraSolid. So what are the possibilities for classification? First option is to use wizard. So basically, the input of the data and classification is handled by wizard. It is simplified, so you don't need to set up many things. The second scenario allows you to change the settings, modify the macros according to your data set or project needs. And the workflow stays still automatically. So as a step one wizard data import, there's quite many sensor choices. And by this, software knows what kind of data to expect, so how settings can be adjusted according to that in later classification tasks. As well, this wizard not just import the data but can reproject to local coordinate systems or adjust to geoid model. And then the second step combines multiple classification steps in one dialogue. You can run either everything at once or with the manual inspection between the steps. So first of all, this wizard removes the noise, cut the overlap, improve the matching, then it moves to ground classification, and then above ground features. Here you can see the journey from raw point cloud classification to classified point cloud. So in the beginning, the data is just imported. You can see a lot of overlap between the lines, and data is still quite noisy. In this one, the overlap is cut out, some smoothing as well applied. Ground classification combines multiple steps. So in the beginning, it tries to detect potential ground surface and done from that to collect some points as a final ground classification. Then layering data by height from ground and wizard classification. So automatically wizard detected trees, vegetation, houses, so roofs, walls, and as well cars. From this point, you can go further with some manual classification improvement or export already ready results, as for example, tree locations. Because during the tree classification, each tree was identified. So you can export the location of the tree, and as well some dimensions attributes. Classified roofs can be used as an input for 3D building models. And as I said, so from here, you need to continue either with industry-specific tools or continue with the results. And one of the example is terrain model. So when you have ready the ground surface, either you need to improve that with matching or adjustment to the ground control points, or you can go straight to triangulate that model production. As well, it can be just simple digital elevation model from ground points, or you can add break lines next to the river or roads. And contours production as well as possible. Over, you can calculate the volume of the stockpiles. This is more important maybe for mining business. And then another example is power line maintenance projects, where raw point classification goes further to wires and towers classification. As well, one of the goals is to detect the wires. So then you can use those wires to detect dangerous vegetation buildings in the clearance zone. When vectorization is happening, also the software classifies the points along the wires. And then for the dangerous vegetation, for example, falling tree logic can be used. So this ends my presentation. To sum up, Therosolet has a solution for UAV data processing. And with the classified point cloud, you can go further industry specific with industry specific requirements or already export required results. Thank you for listening.


Item No. SCP-2080 Object Class Euclid Special Containment Procedures SCP-2080 is contained in SITU and has been boarded up and declared condemned as cover. Two personnel are to be posted at observation point 2080 for monitoring. Any civilians found in SCP-2080's property are to be detained and questioned, with amnestics administered if needed. Instances of SCP-2080-1 are not to be removed from SCP-2080. Should an activation event occur, all instances of SCP-2080-1 are to be allowed access to SCP-2080, and all instances of SCP-2080-2 are to be impounded at the conclusion of the event. Two instances of each are held in storage at Site-81 for experimentation. No further action is to be taken to free SCP-2080-1A. Description SCP-2080 is a two-story house in Inkster, MI. It is the focus of a phenomenon involving the reduplication of an individual, Mr. ████████ Herschel Sr., SCP-2080-1, and a Blue 2001 PT Cruiser, SCP-2080-2. Description is verified instances of SCP-2080-1 to be genetically identical, and instances of SCP-2080-2 to exhibit scratches and dens located in precisely the same positions. Instances of SCP-2080-1 cannot communicate, and only repeat 34 distinct phrases. See Document 2080-02A for a full list. Approximately 96% of the available space within SCP-2080 is filled with instances of SCP-2080-1, including the interiors of closets, cabinets, and appliances. Imaging suggests ████% of SCP-2080-1 instances are animate. Activation events occur whenever one or more instances of SCP-2080-1s removed from SCP-2080. Within 10 minutes, an instance of SCP-2080-2 containing four instances of SCP-2080-1 will approach from the south, progress as far up the driveway as possible, and collide with other SCP-2080-2 at present. Instances of SCP-2080-1 will enter SCP-2080, compacting the mass of SCP-2080-1 further into the interior. The activation event will end when the previous number of instances of SCP-2080-1 is reached. This can result in a higher total number of instances, which will become the limit for future activation events. Since containment, there have been ████ activation events, increasing the total number of SCP-2080-1 instances by ██, SCP-2080-1a is thought to be the original instance of SCP-2080-1, and is the only instance capable of communication. It is located in the master bedroom of SCP-2080, pinned between other SCP-2080-1 with its head visible in the window. SCP-2080-1a is expressed pain during activation events. Two attempts have been unsuccessful, and due to enactment of current containment procedures, communication with SCP-2080-1a is no longer possible. Local police alerted Foundation agents in the Detroit, Michigan area to the existence of SCP-2080 after report from ██ Herschel Jackson, Jr., who is believed to have witnessed the initial series of activation events. Transcript of intake interview follows. Herschel Jackson, Jr. Interviewer, Agent Arnstadt, Detroit Field Office. Began log, 1345. Mr. Herschel Jackson, I'm Agent Arnstadt, I'm here to ask you a few questions. Uh, yeah, alright. When was the last time you spoke with your father? This morning, before, you know. What happened? Did anything about him seem different? Well, I wasn't really paying attention to that, on account of we got in a fight. What did you fight about?   And then I, like, left, you know? What happened after you left? Didn't you first notice something was wrong? Sorry, okay?  Not going anywhere, really.  Same license plate, you know? It goes by, I look in the window right, and there he is, smiling and waving like what just happened didn't. And then another one came, and another one after that, and they just kept coming.  They just kept coming. It says in the police report that you ran back to the house. Yeah, and all those cars kept following me, just more and more. I get to the driveway, there's like a ton of cars just in a pile. Tried the front door, nothing, it wouldn't budge, and in the back door… Alright, that's fine, that's enough, thank you. Do you… do you think it's my fault? I don't. We're not sure why this is happening. I doubt it's your fault, really. Don't worry, you'll be taken to your family and we'll see what we can do to help your Dad. Thank you, Ms. Agent. It's just, I really wish I could take it all back, you know? What I said. I'll see what we can do. End log, 1404. Closing Statement After a standard quarantine period, Herschel Jackson Jr. was anesthetized and released to the custody of his grandparents with the cover story of a gas leak causing the death of ██ Herschel Sr. Ashes were provided to the family for dispersion, in accordance with ██ Jackson Sr.'s will. No further contact with the family of SCP-2080-1A is required. Excerpt from Document-2080-02A Look, I'm sorry, don't cry, there's another game next month, and we can go then, I promise. Hey, alright, look at those grades, didn't I say it'd be worth it? Just do this one thing for me, get me this one night out, and we can go wherever you want this weekend. I'm sorry, Ms. Jackson, oh, I'm for real, yeah, I never meant to make your daughter cry. I apologize a trillion times, hahaha.


Troops of the 23 Ahmad Brigade, Yola have arrested a kidnap in Kingpin, Salihu Ibrahim and 24 other suspects are preyting within the northern and southern parts of the country. Addressing the press conference in Yola, Brigadier General Mohammed Jibrein Gamble explained that the suspected Kingpin has been on the force watch list for the past three years, adding that the suspect has been identified as the mastermind behind most of the kidnappings in the area and even in the southern parts of the country. The submachine gun, a submachine gun with five rounds of 9mm ammunition and a barrister pistol were recovered. Millions of Mayra paid as ransom by the victims of abductors were also recovered during the raid. I have maintained an offensive posture, conducting several kinetic operations in the forms of raid operations on criminal hideouts, rescue operations, calling on search, stop and search as well as patrols among others. This has led to several operational successes, such as the arrest of 22 syndicates of kidnappers, some with AK for several rifles, others with locally made weapons, while some good number of them with cash amount ranging to millions of Mayra, with the collector from their victims as ransom. A kidnapping Kingpin named Salihu Ibrahim was arrested with a submachine gun with five rounds of 9mm ammunition and a barrister pistol following a raid operation at Mayra.


The makerspace here at the mix at our main library in Civic Center is very important to teens in San Francisco to teach them skills and activities that they might not otherwise have access to. For example, we have materials and equipment here that a lot of schools don't have, like our maker bots and other equipment that allows teens to explore their creative side, to explore the fields of science, technology, engineering, and math, and to spark their interest and creativity in those subjects. So in the mix at SFPL, teens do a lot of things. They make music in our recording studio, they make films in our film studio, they design things in our makerspace, they make food, we do karaoke, we do video games, we do SAT preparation, we do dancing programs. teens do a lot of different things, activities, engaging with each other, as well as individual projects to help them express their opinions about the world going on around them. I often come to the mix with my friends, because there are books, computers, and other activities for us. I often come here with my friends to study, to study, so I really like the mix. So we have coding programs through Girls Who Code, we have 3D printing workshops, Shrinkydinks, Lego Robotics, Little Bit SynthKits, all of these things that the library has purchased for people to get their hands on and play around with. And we also have people who are experts in a lot of these things, coming in and showing them and teaching them. But we are also learning ourselves as staff members how to use these things if we hadn't known before. The mix has had a tremendous impact on the teens who have come into the space. They've had a chance to take advantage of the many workshops in the space and learn new skills and achieve a sense of accomplishment. Some of these teens don't necessarily do well in school, but here at the mix they're able to learn new skills and feel very proud about what they've achieved here. As a librarian, I am so proud to be able to be in a space where teens have a chance to learn outside of the pressures of the classroom so they can learn what they want, when they want to learn it, and take advantage of all of the tools and the technologies that we have here in the space. The mix and the makerspace are great examples of how libraries can provide tools and resources to their communities so that they can continue to learn throughout their lives. Having access to these kinds of tools and technologies will become as integral as books in the library of the present here at the San Francisco Public Library and libraries of the future around the world.


Hi, I'm Chris Poland, and you're watching the Metal Boys. This is Neil Turbin. We're here at the downtown rehearsal with the one and only Chris Poland, formerly of Megadeth. But in our hearts of music, you know, you're still a Megadeth on the vinyl and on the songs and we can still listen to you and still learn about what you're up to these days and what's going on here and ask you a few questions. Whatever you want to know. Cool. Thank you, Chris. Thank you. It's a pleasure for putting it all together for us here and you know, I rehearsed in your building here. I just didn't, I never got to go into the secret room here. Everybody has. Earlier on, you had a hand injury. Oh yeah. And what exactly happened and how did that affect, you know, your playing and your technique and kind of, because it's like Tony Iommi in a sense, right? Because he had certainly a hand injury, but it didn't stop him and in fact it propelled him in a further direction and it kind of defined heavy music really. It was an accident that happened at school. There was like a nine-pane oak door with frosty glasses and have a safety wire in it. And something happened, you know, in the hallway, all my books got knocked down, the guy ran and like, you know, we did it on purpose so I chased him and he went to run into, I think it was the lab and he slammed the door but the hydraulic was not working on the door. And I put my hands up and this hand hit the wood and this hand hit the glass and it broke and then he opened the door up really quick to see if I was all right and it just shredded my hand. Yikes.  So how does that affect how you approach playing and how you... I can only play the way I play, you know, I play everything like this. Okay. So I do triads like this and my fingers always up and if I start thinking about what my hand looks like when I'm playing, I freak out. So I never, I don't look at my hand while I'm playing in that way. I don't go, hey, your finger's hanging out in the middle, you know. But it does and it's just a weird thing and that's how I get around it. And I don't want to, you know, ask questions that are painful or things that, you know... Oh, I don't care. Disturbing to you but it's just, you know, kind of the... Oh, and there's no fixing it. You know, doctors have told me, you know, if we do anything, you'll never, might not even be able to bend your finger. So I just deal with it. And this is all in school. This is way before Megadeth. Oh, yeah, this is before I even lived in California. Now, were you playing a lot of guitar in those days? Oh, yeah, I just bought a Mahavishna orchestra, Birds of Fire. I was learning stuff off that. I had some blues band. I can't remember what it was. But I was learning all this stuff. And then I came home, you know, from the hospital in a huge cast where my hand was like this with a cast around it. So it was just this huge block. And that was on there for a long time. And then it went smaller and smaller.  And the first thing I did was tried to play guitar. And I realized I couldn't bend my finger. Ouch.  So I was like, well, I'm going to try. And then once I tried, I said, well, I can still play. I just can't play like I did before. And I just learned a different way. So you had to reinvent your style of playing? Yeah, it's weird. And the only thing that's affecting me now because I've gotten older is my thumb has been in a strange position for so long that my knuckle right here every once in a while tells on me. But that's about it. Well, you just got to keep on rocking, man. Yeah, no, yeah, exactly. The more I play, the less it hurts. So as far as other styles of music, other types of influences, I know you had Ohm. And just some of the other. Influences for me are when I was a kid, it was Jeff Beck, Jimmy Page, Leslie West, Hendricks. Then it was Jeff Beck into McLaughlin. And then for a long time, all I listened to was Fusion. But I still would go back and listen to rock stuff. But I wasn't a real heavy metal fan. I was exposed to a lot of it from Dave. Like Dave turned me on to Merciful Fate and stuff like that. And I really liked it. But if I hadn't met Dave, I would have never even. I learned a lot from the stain on guitar-wise, rhythm-wise, and how to do spider chords. And there's a lot that I took from that experience. So I know I'm going off a little bit backwards. But back on Dave Mustaine, I was just wondering in terms of the metallic influence in him, did you find that he was kind of moving away in some aspects? In other words, he brought a lot to Metallica in the first place. So obviously, Metallica is a representation of what he put in, his input. But as we all grow and learn things and maybe change a bit from our influences or our surroundings, would you say that there was definitely a way where he was kind of moving into the Dave Mustaine mode versus the Metallica mode? In other words, they were young when they were doing that. I mean, I think Dave was probably writing probably more than half of the stuff when they were together. So I think they had their idea of how things should be. And Dave had his idea. And that's why the bands are so different.  But they are different animals. Dave's music is a little bit more complicated, but not in difficulty to play, just idea-wise, at least at the beginning. I mean, well, there got to a point where it seemed like Dave was trying to maybe get more radio play.  If I'm radio or no. So that last record, I thought was really good. There was a lot of really good songs on it. And there was a lot of good songs on the system as fail, too. But then there were those songs where you would never hear that on, like, say, piece cells. There's a ballad on that song. It's a beautiful song, but it's really not a Megadeth song. But it is, because Dave wrote it. So there's that point where even Metallica did it. And they started to lean towards playing more of what everybody might like. And there's nothing wrong with that. I have a hard time doing that. I can only write what comes to me. I can't sit down and go, yeah, I'm going to write this song because it might get played on radio. If I pick up a guitar or something happens, that's what I write. I don't go, let me see. How did that song go? I'll write a song. I just like it. I don't do that. Well, so much work and try. I'm not saying those guys do that either. But I just only write what I write, and that's it. That's why I'm kind of an obscure musician. Well, so much more contrived when you're trying to write a hit song or a journey song or something that, hey, we'll get a radio play versus something that's like, this is what I'm feeling. And here's a riff I have or here's some idea or a thought or a vibration or whatever it is. I'm sure my wife and daughter wish that I would do that more. And maybe I will on this record. I don't know. I have a lot of stuff that's been in my head and on my phone. Oh, cool. So by the way, I do the same thing Kurt Hammett does. Every idea I have is on my phone. Oh, that's awesome.  So do you go back to some of these ideas that you had, let's say, six years ago, 18 years ago? I have a song that I wrote long before I was in Megadeth that I'm going to show Carlos and Peg. That's great. With Megadeth, when you guys got your album advance, I guess when, I don't know if that would be the first album that you got an advance for, but maybe one of the future albums. I just wonder, did all the money go for substances or did most of it go for that or the band? First of all, when we got any money, it was in Dave's account. Nobody got a per diem where we got a couple hundred bucks a week to live on. So Dave got the money. Dave got the money. And you guys got payment.  We only had, I think it was six or $8,000 to make a record. And we made it in a couple of weeks. And drugs aren't that expensive. It's like, we weren't even like that far gone then. It was just, we had Cliff Caltieri was up there from, he actually brought amps up there for us. If it wasn't for Cliff Caltieri, I wouldn't have gotten solos on the first record. He asked Dave to give me solos, because I wasn't going to get a bunch of solos. I thank Cliff for that rest in peace. And even if there's no money to spend. Even peace solos, I think our budget was $26,000. But we were in really nice studios. It cost money. And it cost money. And you guys got to eat food while you're in the studio? And plus we were still getting our bills paid. We still had our $200 a week from that we lived on. And it's not like we were going, Dave, I needed $5,000 to say, hey, I need money to go pick up. We would never ask Dave for that. Unless it was during the week, and if you're out of money, hey, Dave, can you front me $20 until payday? I don't even remember ever doing that. So here we are inside of Chris Pollan's studio. Pretty awesome to be here. And there's Chris. This is what we'd get stuff into the computer with. And then we'd use some new stuff over here for the kick and snare. Got some fancy pants stuff and nice sound. It's OK. But this is pretty nice. I had an Owensmart C2, and I did some research on this. And even Carlos agrees. It's as good as that without the crush. But this thing I'm really proud of. I love these little boards, man. SSL, the X-Panda, because you can sum everything, and it gives you that SSL headroom. We're not probably going to mix in here, but we could. That's amazing. We wanted to. But I like to go through when I go to tape. I like to go through this so that I get this vibe of this thing being pushed almost into the red, going into the computer. And that way it gives it a little bit more juice. I was reading about when you left Megadeth and just trying to fathom the similarities between some things that you might have went through and perhaps myself. But the money thing, I didn't have any money from what I was doing until later on. I mean, I just didn't get paid when I was in the band. I was in. I mean, it just didn't exist. $5 to go to Denny's would be. They put the $5. Yeah, $5 in my hand would be like, wow. That's a lot. Speaking of Denny's, I was busing tables from midnight to 8 AM at Denny's after I left Megadeth. And I would be busing tables on a weekend. And some guy would go, dude, you're Chris Poem. And I'd have to go, yeah, I am.  And I would just say, hey, man, I got to eat. Doing what you got to do. I got to pee the rent. There's no shame in working hard. But when you think of it now, it is pretty crazy because at that time, Megadeth was beyond gold. Peace cells are really like a train. And here I am, busing tables in the middle of the night at Denny's. Yeah, but you worked hard. And now you're in the awesome studio. Oh, yeah, I mean, no, I somewhat got it together. I mean, we're still struggling to. But that was a time where it was a struggle, a real struggle. Oh, yeah, it was terrible. And it was also like, of course, Megadeth was a well-known band and doing well. And all the inner workings and all the intricacies and all that, the relationships.  And there'd be like six guys walking down the boardwalk in Venice wearing Megadeth shirts. And at the time, my girlfriend said, hey, you guys, you know who this is? This is Chris Pollan from Megadeth.  So I've had every little version of weirdness from being in that band. Let me tell you. So looking back, do you think you made the right decision when you left Megadeth? And I guess that's the question that's, did you leave or did things happen in such a way that it didn't really give you a choice? It got to the point where it was just time that it wasn't working for anybody anymore. So who knows what would happen if I stayed in the band? It was probably best that I didn't. And I can understand.  And then there's ones that love each other and never argue. Well, I think that wouldn't have been the issue. Probably drugs would have killed somebody or all of us. Who knows? So in the time that you left Megadeth, would you say that the people in the band were still into the drugs and even yourself? Oh, yeah. I mean, that's all documented in books. So it was chaos. Yeah, it was. It's crazy when everybody's not in their right mind. And I mean, things can be happy one day and not unhappy the next day or the next moment. Yeah, that was any given day. And that's not just alcohol, but that's like a bad trip or a bad experience or, oh, don't have something withdrawal? Oh, yeah. Don't have the money? Yeah, well, it's not that you don't have the money.  And what are you going to do? Are you going to ask somebody at 7-Eleven? Right. I know it's a hard topic to even really discuss, but in that respect, do you find that while you guys were in that state of mind or in that state of existence on the road, was there certain unique challenges that you experienced because everyone was, or some of the band were using?  Well, that's, I mean. And then they can't even stand up and they're falling backwards. Yeah, that was the show at the Ritz. That's pretty famous that everybody already knows about. But the main thing was is you had to time your buzz. If you're going to go on stage sick, you had to drink a lot right before you went on and you wouldn't get drunk. You just would be able to stand up and do it. And that was what was terrible. It's like that would go on for a couple of weeks until everybody dried out. So is that something you had to do personally? And is that something that other members had to do in the band? In other words, you're doing it at the same time? You're kind of on the same wavelength with that? Some were worse than others. But at the time, that time, Ellison, he was pretty much OK. I mean, he'd have the sniffles a couple of days and that was it. And what about Gar Samuelson? Yeah, Gar and me, it would take a couple of weeks to get over what we had been doing before we'd go out on the road. Hey, so Neil, I just want to say we were just talking about all the crazy fighting and stuff that went on in Megadeth. It was kind of like when you fight with your brother or something. I mean, there wasn't a lot of grudges held when we would do that because we were not in our right minds. And I never ever was, there were days where I was pissed for a day. But it's not like we were, when we knew we had to do shit, we just got our things done. And things like that didn't happen all the time. But when they did, it was a pretty big deal. You like Bruce Lee too? Oh, yeah. Everybody likes Bruce Lee. Me too. Wow, well, this is such a cool place. And here's your old school Ampex tape. Yeah, we used that machine to make a record called Chasing the Sun. Nice. Well, it wasn't exactly that machine. That machine was way older than this one, but that's the same machine I used. That's why I have that one just in case. If you, it wouldn't believe it. If you take the two track from the computer and run it into that and play it after it's hit the tape, you wouldn't believe what it adds to the two track. And over here is Chasing the Sun, right? That's what this is right here. Yeah, that's the first artwork. Beautiful. It's changed since then. Wow, you got such cool stuff in here. And then over here, I noticed this. That's why I wanted to come into the room as well, because we have these gold wards. This is all just for piece cells though. That's all right. Yeah. I'm pretty sure we should have one for killing by now. It's just so cool to be in here and to hang out with you. The relationship that you have today with Dave Mustaine or Megadeth, do you guys communicate at all? Nope, not since the system has failed. Has anyone in the Megadeth camp, whether it be managers or publicists or anyone ever reached out or nothing like that? I can relate. Yeah. Yeah. But eventually I got reached out to you for some little video thing, but it wasn't like, hey man, let's get the party back together. Would you ever consider going back to Megadeth if they called you up and said, hey Chris, we're doing something and you'd like to? I don't think so. Would you like to throw some money at you? No, I mean, if Dave did a solo thing, maybe, but I'm not, I don't wanna learn all those Megadeth songs. No way. And then how am I gonna do Marty Friedman solos, you know? Yeah, Marty's pretty awesome, but you're pretty awesome too, so I don't think you'd have a problem. I'm sure you're aware that Dave Ellison and Jeff Young, I've recently reunited together outside of Megadeth. Yeah, I'm trying to remember. I just saw actually great people. I just saw Dave, I just did a gig with Dave over at the Ultimate Jam Night at the Whiskey. Oh, cool. And they were doing a tribute to the Big Four, so Dave Ellison and Jeff Young were both playing. And I thought to myself right at that moment when I saw them in the dressing room, the Whiskey, like, why isn't Chris Poland in this room with us? Oh no, I'm doing, I think it's in November. I'm doing a thing with those guys in November in Chicago. Wow, you just answered my question already. So I was gonna ask you if you would consider joining up with Dave Ellison and Jeff Young. So you're already doing that. Yeah, yeah, yeah. That's so cool. Yeah, Dave sent me a text and said, hey, you wanna do this? I said, yeah. From being in Megadeth, would you say that, you know, your writing, your perspective, your approach to music, your life experience? I mean, would you say that? Before and after, you were a completely different person? Made a big difference.  So it was the right thing to do and it lasted just as long as it was supposed to, you know. And I'm really proud of all that, you know, music. All three of the records I played on, so. What's going on today and what music you're working on? Because, you know, we know you've moved on a long time ago from Megadeth, but of course the fans wanna always ask the questions and of course everybody wants to know all the intimate, juicy details. Yeah, yeah, we have Carlos Cruz from Warbringer, is our drummer. Oh wow, great drummer, great guy. Really good. Was just fit right in and he actually, he got us together after Nick passed just to get us out of our funk because it was a long time before we even, you know, felt like playing and one day he just, you know, suggested that we get together and we did and then it started, he came in, he knew like 10 of our songs, like better than we knew him. That's great. And so we've been going from there and he's right now, he's on the road right now for a couple of weeks, but when he gets back we're gonna finish a record that we have a record deal with M Theory Audio. And they released a song called Exit Stage Left which I had forgotten when I named it that there was a rush song called that. But yeah, but it was in my mind, it was a song that we had worked on with Nick and I just felt like I wanna, you know, that song is definitely about Nick for me and anyway, so they released that as a kind of a teaser for the record that's been online, you can look it up and listen to it. But yeah, we have to finish the record when Carlos gets back and we have a lot of material so it should go well. That's great and are you recording that here? Yeah. But what's the name of your recording studio? It's called Glass House. And you're telling me a little story, someone named it? Yeah, I think Donnie Serian named it and I just never named it so that's what it's been called ever since. How cool. Well thank you Chris, it's a real honor for me and a privilege and I'm really, you know, appreciative to finally come and meet up with you in your studio and in your- My hideaway. You're abode here, humble abode in the beautiful downtown Los Angeles and you know, it's just so cool to talk about these times that you experienced and these experiences, because you know, no one else was there that can really explain it this way. I appreciate it.


Here we go. Here we go. All right keyword ultra keyword ultra. We're diving into this topic Actually, a lot of people asked last week if I could cover my thoughts on ultra running and ultra marathon running And I'm happy to do it and that's why I love YouTube and a shout out to all the new subscribers There's a lot of you every single day. And so this is a daily every single day 5 a.m. Mountain time I publish a new video. I'm injured right now. So usually the videos don't start here in the studio And that's what we call my shed that has been converted to my little Recording studio usually I start with filming out in the mountains on the track in the gym Etc. Etc. But right now I am injured and just trying to heal up be smart So anyway, welcome to all the new subscribers. Let's dive into ultra running and ultra marathon running and oh my goodness Okay, first of all, I discovered ultra running about five years ago And yes, I'm gonna give you the five tips that I wish five tips not ten five tips that I wish I would have known Five years ago. First of all, I discovered ultra running through this crazy adventure called Nolan's 14 thanks to my brother go I don't have time to explain what that is now Go check it out upright hand corner for some crazy epic mountain running again upright hand corner But before we dive into my five tips or five points about ultra running What I wish I would have known five years ago. We got to define what is an ultra marathon It's basically it. I shouldn't say basically it is any race that is longer than a marathon 26.2 miles or 42.1 kilometers traditionally ultra races usually start at the 50k distance Which is 31 miles and you don't really talk about I'm gonna go run a 31 mile ultra race They always define it as a 50k and then the other distances that are more traditional in the ultra running scene are so it's 50k 50 mile 100k 100 miles so it's kind of interesting it alternates between k's and miles and I'm just gonna say it right now Okay, no first of all This is a huge topic and I'm probably gonna make more vlogs about this topic in the future But this is just a little precursor to give you a little taste because frankly If you're in high school if you're in college if you're you know, maybe you're 40 years old This might be a 20 year project before you actually ever consider Running an ultra marathon race It's it's a long-term proposition and I'm just planting the seed But I think as runners who are in this sport for the long haul It's really good to know about different disciplines and different opportunities out there within the entire global running space Which includes ultra marathons, but you you you if you're in high school I personally wouldn't recommend jumping into ultra marathon running I think you can save that for at least when you're 25 years old and above I'll explain more about that in a second. Uh, so anyway, okay in a prediction bold prediction right now What is it may 25th 2019? I predict I predict in the next 10 15 No more than 20 years so 20 39 no lot no later than 20 39 We will see an ultra marathon at the olympics I'm calling it the popularity of ultra running continues to pick up speed and momentum on a global scale scale the the the fastest growing location for ultra marathon running right now is china kind of like road racing as well in china But it's it's china. So i'm i'm calling it now and it's i'm also going to predict It's going to be the 100k distance So it's the so instead of 50k or instead of 100 mile it kind of blends the two so it's speed versus strength Combined into the 100k. So that's my bold prediction for 2019 not the question of the day But if you think i'm crazy, let me know down in the comments If you think i might be on to something let me know down in the comments Okay, moving on to my five points for what i wish i would have known Five years ago before dabbling and jumping in to ultra running. All right This is the time to take out your notes take out your phone start taking notes Let's go point number one for what i wish i would have known about ultra running five years ago. Here we go Use your speed while you have it. So this is a shout out to everybody You might be fast in your late 30s But especially the younger generations We're talking folks in their 20s high schoolers and heck if there's middle schoolers watching as well And again, i'm not trying to be controversial But i'm just trying to instill some of my experience with all of you So when you get into ultra running and ultra racing, it's slower paces It's just that you're running in a race You're running usually like eight minute pace for a 50k Especially if you're at altitude over hilly terrain eight minute pace nine minute pace 10 minute pace It's a very common pace for these races And therefore your your fast twitch muscle fibers in your legs are not being as recruited Uh because you're not racing half marathons 10k's even the marathon on the roads If you're really trying to race a half a fast marathon on the roads you are Using your leg speed way more than an ultra marathon Race so all i'm saying is is use your speed while you still have it And i wish now that i know i can run a one hour and 10 minute half marathon at altitude And now i'm fascinated to see how fast i can run a marathon at sea level And i know like these ultra races that i've done over the past two three four years I know they've created some good aerobic uh endurance in my in my breathing in my aerobic system But i just wish maybe i would have focused a little more on shorter races in my late 20s That's all i'm saying that's point number one tip number one is don't be afraid to capitalize now on your speed All right, if you don't agree with me, let me know down in the comments. That is my experience thus far Okay, and point number two for preparing for training for an ultra marathon race Strength over volume going out and running a 50k you might think okay. That means I need to be training. Let's say 10 to 15 percent more in volume Uh compared to my maybe my build up for a marathon race I would say that is actually not necessary Instead focus on leg strength So a lot of people message me on instagram asking for advice for preparing for a 50k or 50 miler And based on again my training over the last three four five years I have found that strength Especially if you can get access to hill training preferably big mountain hill training I know that's not always possible where you might live But that is more important than going out and hitting a 30 mile training run or a 35 mile training run um When you're running an ultra race your legs are going to get tired And that is when again what I have found leg strength pays off more than um Doing long long long aerobic run. So just so instead of running a 25 to 30 mile training run Drop it down to 18 to 24 miles. Whatever you're comfortable with and add Three to 5 000 feet of vertical gain in that run in order to once again build your leg strength All right, that's point number two Strength over volume point number three for what I wish I would have known five years ago for preparing for an ultra marathon race gear Really matters like your running gear really matters when I started honestly I had a pair of running shorts a pair of running shoes and that was it I had no other running gear and you know how I like to keep my running gear simple That's what I love about this sport is that it's simple. We don't need a bike We don't need a pool. We don't need a tennis racket. We don't need hockey skates Or all the hockey gear you just need a pair of running shoes and that's how I train for years and years and years But trust me when I say you need a little bit of extra gear for ultra running simple things like hold on Simple things like a hat like sunglasses when you're out there training for more than two and a half or three hours You need protection from the sun. You need a vest I had no clue that running vest even existed four or five years ago You need a vest to carry your hydration system to carry your food um, and oh You might you might need to invest in some running poles. Yes for those big big mountain days In order to use your upper body strength in some of your training You might need some running poles trekking poles. You might need to buy a light Why because a lot of ultra races happen at night sometimes you will run Especially for the 100k and 100 mile distance You will run through the night or at least part in the night. So it's in it's crazy A lot of this gear is listed down below from amazon in case you have no Idea what to start with or where to begin your purchasing of running gear So anyway, that is point number three running gear really matters in ultra running and point number four for what I wish I would have known five years ago It's a race of attrition at times at times not all the time, but at times. What do I mean? Patience over speed a lot of times in it doesn't matter whether you're a front of the pack Middle of the pack back of the pack runner doesn't really matter those that are patient and Put more emphasis on nutrition and hydration Oftentimes will do much much Much better in an ultra race. It's completely different now. Listen. I haven't raced a marathon on the roads yet Still trying to make that happen But based on all the marathon races I've attended and cheered at or that I've watched on tv The whole nutrition and um and hydration Set up. It's just completely completely different. So Some famous ultra runners. I forget his name, but he says like ultra running is it's a running contest But almost more importantly. It's an eating contest. Therefore. Here's the point You want to dial in your nutrition practice early in your training block and your hydration practice? Why because when you're running for four hours six hours eight hours On 24 hours 24 hours plus you have to be eating Not constantly, but you need to really really be focused on your nutrition Whereas in a fast marathon or half marathon, you know, you might get You might be racing for three hours or four hours or five hours Which of course you need to be you need to be uh cognizant of your nutrition and hydration in a marathon a road marathon race but it's just um It's amplified so so much in an ultra marathon race I cannot emphasize this enough and i'm i'm still learning and making mistakes and trying to dial in What really really works for me many of you remember my situation at run rabbit run 100 last september I tried to run a hundred mile race made it 50 miles dropped out I part of it was because I didn't take salt tablets. I just I just didn't know I just didn't know and that was my fault for not doing enough research ahead of time So anyway dial in nutrition and hydration Early early early in your training by okay. That's point number four Let's move on point number five for what I wish I would have known before diving into ultra running five years ago It has nothing to do with actual running or hydration or nutrition or anything like that It's actually it's more an action item a little homework for you Ultra sign up dot com. That's right. I wish somebody would have told me that ultra sign up is the main online resource for finding ultra races at least across the united states I'm not quite as familiar with how much uh International how many international races are listed there, but that is your go-to resource for and a lot of the elite athletes Their their results are posted there on ultra sign up So anyway, that's a good good resource to know about ultra sign up dot com And I'll just make mention the most popular or sorry the most prestigious Oldest race in the united states is western states 100 which is in california And started in the late 70s. I do believe and it's coming up in june late june I'm thinking about maybe even going out to it. We'll see and then also And then as far as on the international scene outside the united states the most prestigious and biggest ultra marathon race Is is you it's called utmb and it takes place in france, italy Uh, and I think it dips into switzerland ever so slightly. I could be wrong on that Anyway, it starts and finishes in shamanie in france. And so that takes place in august Anyway, I just wanted to mention that ultra sign up dot com It's really good. If you have no clue where to begin doing research. It's good to know about Okay, keyword again is ultra question of the day. Do you do ultra running ultra racing? Uh, I'll be so curious and if you do let us know what your next race is down below And then also second part if you don't do ultra running Do you have any interest in the next five years 10 years? 20 years to maybe consider ultra racing or maybe it's just like you just have no interest You just want to stick to 5 k 10 k half marathon marathon, which is great as well great as well All right, that's it for today. Thanks for being here. Thanks for watching seek beauty work hard And love each other. We're getting there folks Foot's moving along. It's not perfect yet, but it's moving along. See you tomorrow


Well, the most obvious and most certain challenge is the fiscal challenge. Graying means paying. It means paying more for pensions, more for health care, more for social services for the elderly. Most of the developed countries have paves-you-go social insurance systems in which the current working age population is taxed to pay for benefits to the current retired population. Paves-you-go systems are vulnerable to demographic aging. The falling fertility and rising longevity means a declining ratio of workers to retirees and a rising cost rate for paves-you-go social insurance systems. As part of our aging vulnerability project at CSIS, we project that most developed countries will see an extra 10 percent, we'll have to devote an extra 10 percent of GDP to benefits to the elderly over the next 25 to 30 years. So that would bring the total. It varies from country to country, but from an average of about 12 percent to an average of about 24 percent. That's a big increase. An extra 10 percent of GDP is 25 percent of workers' wages. It's also three times what the United States now spends as a share of GDP, or two and a half times what we now spend on national defense. The first line challenge is the fiscal challenge, but the implications reach far beyond that. There's also the challenge of more slowly growing or declining working age populations, which in turn will mean, in all likelihood, slower economic growth. Falling fertility is not only hollowing out the traditional age pyramid at the bottom, it's also ushering in an era of widespread workforce and population decline. In many European countries and in Japan, the workforce will be declining by between one and one and a half percent per year by the 2020s. And this will translate into slower aggregate GDP growth and perhaps also into slower per capita GDP growth. More slowly growing workforces mean relatively less investment and perhaps fewer opportunities for innovation and technological advance, what some economists call learning by doing. The developed countries will have an aging stock of physical capital. They'll also have an aging stock of human capital as the median age of the workforce rises. There's also a social and a political challenge. Will smaller families be able to care for a burgeoning number of frail elders? Although most countries have socialized much of the cost in the developed world of growing old with pension, through pension programs and acute care, health benefits, health benefits, the cost of long-term care in many places. Certainly this is true in the United States. Still largely falls on the extended family. Will the family be able to bear this added burden or will the cost get pushed off onto government with even more dire implications for the long-term fiscal outlook? Will aging electorates become, will the gray lobby become a political titan astride the body politic? What does the aging of the electorate imply for the ability to reform unsustainable old age benefit programs in the future? And most broadly, what about the culture? What about the social mood? Sociology, sociologists have spent much time talking about the individual life cycle and trying to isolate attitudes and behaviors typical to different ages, risk-taking entrepreneurial youth and more risk-averse, cautious old age. Cicero said, young men for action, old men for counsel. Is there an analogy to the society as a whole as the median age rises and what does that imply for the dynamism of the culture and of society in the future? The well-known French demographer, the late French demographer, Alfred Sovi talked about the future of Europe as a future of old people living in old houses ruminating about old ideas. And finally, the geopolitical challenge. The developed world is aging and many parts of it will be shrinking, but much of the developing world will still be young and fast growing. What is this, what are differential rates of population growth and differential age structures imply for the geopolitical balance of power in the future? It's not just a question of manpower shortages or people power shortages. It's also a question of fiscal pressure on defense and foreign affairs budgets and to loop back around to the question of social mood. Will older electorates be less prone to engage issues in the broader geopolitical theater or will they be more prone to protect current consumption? So a whole range of challenges, some of them relatively certain, some more speculative, but all of which need to be considered by policymakers.


This is a message from the SCP Foundation. The SCP Foundation requires the assistance of the public in the containment of a dangerous anomaly. SCP 871 is a group of self-replicating cakes, which produce other self-replicating cakes. If left unchecked, SCP 871 instances may render Earth uninhabitable in just 80 days. It is therefore essential you consume an instance of SCP 871 you encounter.  The SCP Foundation thanks you for your cooperation. Secure. Contain. Protect.


All right, let's get started. So this talk is about value-driven clouds powered by OpenStack and SAF. It's about a journey about how we have seen the value of clouds like that in the current economic market. It's not a very technical talk, but I hope it gives you some ideas and courage to try out OpenStack and SAF-based clouds. My name is Yuri. I'm an infrastructure and platform automations engineer. I'm one of the original engineers for OpenMetal. And I help people understand the value of our platform. OpenMetal is the company I'm with. Our mission is to basically automate and make highly complex open source systems available on demand in a turnkey solution and increase accessibility for the smaller teams. Fun fact, we're a silver member and an infrastructure donor to the Open Infra Foundation. So one of our clouds is actually used for CI-CD for Zool to develop OpenStack. So what makes a cloud a cloud? In 2023, there's bare minimums now. So it's got to be highly available and failure tolerant. You need the distributed storage to be able to mount the data from any location. You need to be able to create and destroy assets like networks and VMs or whole Kubernetes clusters at a whim. You need CLI tooling. And the UI component is important for visibility. Obviously, you also want to support all the infrastructure as code automation. So you need the terraform and ansible and heat and all of those components. But fundamentally, you need to be able to run VMs. You need to be able to run containers, live migrations, support load balancers, firewall as a service, network file systems, a variety of storage, user management, and complex networks. And that is basically a bare minimum. So we want OpenSecondsF to be adopted by organizations of all sizes. So how do we do that? Well, we listen to what they're saying. They want to be able to know what they're signing up for up front. They want to be able to use these clouds for dev, CICD, staging, production. They're looking at alternatives to the public offerings. They're very security aware. They want performance. They want to be able to monitor things like CPU frequencies if they need to. And they want the uncomplicated billing mechanics. So there's quite a few ways to learn OpenStack. The keynote yesterday or on Tuesday, they said about 50% of the people are new. So if you haven't checked out OpenStack, please do. There's definitely self-driven solutions, kind of like the DIY, like DevStack, Microstack. There's definitely official training sources. If your company uses OpenStack, you can find those at the OpenStack marketplace. But people want learning environments that will match the production experience. So they want to be able to test live migration, disaster recovery, development, all those things. Certain interesting things that we found are some tough concepts for cloud native users to actually understand, because they don't have access to what happens under the hood. So things like virtual limits versus actual real usage is not well understood. The ability to provision really big VMs and support burst stability without having to actually use those resources. Instant sizing, everybody is shopping by public cloud instant sizes. When you own your own cloud, you can create your own instant sizings. And the over-allocated resources, the ability to run 400 vCPUs on a 100 CPU box, something like that, is also kind of a trick to explain. So some of the challenges that we've seen with landing an OpenStack deployment is usually, if you're building it out, it has a pretty high upfront cost. You have to probably take a month or two to properly deploy it. So there is time to market delay considerations. And sometimes you need consultation and engineering experience. But you need a safe path to land, figure it out, scale and grow. And you want the access and ownership of your own data. So we've created kind of this solution called the soft landing, which is a private cloud core, kind of a default landing footprint that people can use right out of the box. It's basically to get faster OpenSec and stuff adoption and propel interest in this community. So we want a turnkey solution accessible to mid-size and smaller teams that don't have a big upfront budget. So this is kind of the private cloud core. It consists of three hyperconverged nodes. They basically run all the control plane services for both OpenStack and staff, as well as some of the storage. So we deploy the control plane services and they consume some resources. The control plane usage is fairly static to begin with. And the quantity of the remaining resources depend on the unit size that you selected. If you have three servers and they all have a terabyte of RAM each, you're going to have a lot of leftover things to play with. We also set some fair reserved limits for the control plane so that it operates well. But all of those remaining resources in the white, well, those can be used and they can be used immediately. So the green here represents basically workloads. You can run on that default landing footprint, whether it's VMs or Kubernetes clusters, complex networks, load balancers. The core is full of features that make a cloud a cloud. Now, once you reach a point where you need more, you just add additional servers and grow your cloud on demand whenever you're ready. And they're also converged so they can expand storage and so on. And then you start running workloads on the compute nodes, on the converge compute nodes, and you can slowly start moving away some of those initial workloads from your control plane because it is a critical component of your cloud. So as your cloud grows, you'll want to move those out. And then eventually you reach a mature cloud where most of the compute resources are moved off of the cloud core of the control plane servers. And you're basically running most of your workloads on the compute added nodes. And the nice thing about the extra resources still available on the cloud core is they can be used for as a failure tolerance mechanic. So there's quite a bit of value in that. So if you have an experience over second stuff, we strongly recommend exploring them. You can get most of the features that you probably need from a cloud like that. And there is no cost to try new things on your own cloud. So it reduces the barrier to innovate. You can try new Kubernetes cluster versions, I don't know, new software, and so on. There's a flat predictable billing model based on bare metal servers, since this is basically what we're deploying to. Whether this is your on-prem deployment or utilizing a company like us, the billing is radically different. The other thing is it's becoming fairly easier to deploy these private clouds. So the question is, do you still need the one big cloud monolith, or can you have many smaller clouds VIP clouds, project clouds, department clouds, and so on? And the goal is that you land on OpenStack and stuff safely and then expand. So it looks like I have some time. We're going to kind of show you how we do it. So this is basically our control panel where we can spin up new clouds. I'm going to spin up a little trial, select a location, and select one other cloud cores. Again, very different sizes available. And we'll do that. I tried this before at a trial. And our OpenStack cloud is ready. So we can check out the hardware. We use the smallest unit possible. This is a pretty good environment for learning, so they're fairly small servers. We can go ahead and open Horizon. Now we do deploy a self-signed certificate. Obviously, everything's customizable and configurable with OpenStack. Let's go ahead and get our password. All right, so we're in our OpenStack deployment back by self. We actually do preload all our clouds with some images and flavors of the like. Horizon Cache has to generate a fresh new deployment. So we preload some images. So the feeling is very natural to most cloud users. And we can explore our hypervisors. Everything's ready to go in a highly available configured production-ready format. That's it. That's all I've got for today.


Now, the Minister of Defense, Mohamed Badarou, has pledged that soldiers that killed in Kuhama, Okuama and abducted children in Kurigel will receive justice. Now, Badarou visited troops of the 1 Division of the Nigerian Army in Cardinal to discuss the abduction of 287 children, and the measures taken to rescue them. The act of military sacrifices and extended condolences to the Nigerian Army. The guarantee, what you have been doing to secure our nation, we thank you most sincerely. We know most of you have just returned from the field, and your brothers are there, and soon you will rotate also. We will, by special grace of Allah, we will end this situation in this country, and we will defeat them. We have the power, we have the will. Now, the Minister confirmed that the strategy for rescuing abducted children in Kurigel will yield positive results, while also ensuring the military receives the necessary support to effectively fulfill its constitutional responsibilities. As you have seen, I come here also for the briefing, and I'm briefed fully well. The military in coordination with other security agencies are doing their best, and we are seeing the possibility of releasing them in sight, and I'm sure we'll hear good news. Thank you very much.


Nope. Oh, is your clicker? Where's the clicker? Okay, let's see if that works Nope Yeah, I Just need to get to the next page There we go. I'll just do this Is it working now? And to go back I go up. Okay. So what's the mock? It's a public private Group that got together to say we want to create a public cloud. We have about 400 users some of those users are serving over 10,000 other users, so they're running things on the cluster that support lots of other researchers It's hosted at the mass screen high-performance computing center This is a two-acre data center in Holyoke, Massachusetts. It's platinum lead certified and One of the cool things about it is it has high-speed fiber connecting the entire northeast region our planned upcoming growth over the next six months or so is That the northeastern storage exchange is going to be standing up a 20 petabyte data lake at the MGH PCC And it's going to be integrated with the MOC, so you'll be able to do compute against really large data sets IBM is putting 11 power 9 servers in They're each going to have a terabyte of memory and 4 GPUs There are about 5,000 cores coming over from Harvard over the next six or eight months and As we just discussed in the past couple of days. We're really excited that the Red Hat data hub is going to be standing up on the MOC So you guys probably already knew this most of today's clouds are run by one company they basically don't share their data and That makes it really hard both for us as researchers within the university setting But it also has an impact on folks who want to do optimizations for example all of us working on open source So four or five years ago Peter and or on Krieger Got together with a couple of other researchers and thought There should be a different model possible and so the idea of an open cloud exchange is that You have a mechanism that allows you to take Information or sorry compute from one provider GPUs from another provider storage from yet a different provider and plug them all together so that they plug and play together but it also the ideas behind this also lead to the concept of having a A level playing field so that a big company and a small company or you or me Could create our own services and offer them to anybody and so as we thought about that He said what are the things that would be necessary to create a minimal viable product? We need production cloud services We need to have a method for single sign-on that allows us to use the resources across multiple providers We need to have a billing model We need to have a elastic hardware and we need to have research Federation and The MOC is a production cloud service We offer single sign-on through Keystone We have a billing model We have elastic hardware and we're working on that So why do we actually care about this as essentially part of a group of research universities and private industry? we care because You can't do open-source development and you can't do research if you don't have access to real data access to real users and access to scale and We all know the story of the cathedral and the bazaar If we open things up Then we end up in a situation where we have improved efficiency more competition. It's a better outcome for everybody That's why we care Why should the community care? well There's a lot of reasons but one of the big ones that we keep running into as we break them on is There's no place to run continuous integration and deployment at scale So there's lots of issues that don't get found until they show up at the customer site Or in our case at the MOC And some of those are because of interface changes that get missed between different projects or they get out of sync Some of those are because they only begin to show up at scale Some of those are because the implementers were solving a different set of problems than what we need in running a commercial cloud and Some of it's just because as you get to large quantities of users you start to see as I mentioned before Really different scales and types of problems So We're gonna look at four ways that we've broken the cloud. I'm sure there's more but these seemed like good illustrations We've had major storage failures We've run into problems trying to stand up telemetry for our cloud We've run into problems trying to do charge back and show back and I think we're one of the early folks who's been doing a lot with open shift on open stack and we've run into problems there, too So I want to tell you about the great mock sef upgrade of August 2017. It was a really simple plan Do it in August There aren't a lot of users because we're a research cloud, right? We're trying to essentially triple the size of our staff cluster Nothing could go wrong here, could it Unfortunately Or should I say fortunately nobody was eaten by a velociraptor but seriously It was a disaster in every way possible. Well, I don't know about in every way possible. Nobody died. Nobody got eaten, but Let's get this out of the way up front We lost all the data Like What's the one thing you want your cloud to not do? You don't want it to lose your data and Yes, Peter's pointing out the 170 terabyte was three quarters full So There's some good news. We did a really good job of telling people make sure your bank data is backed up and Pretty much all the users were okay, and that's really good But there's a really embarrassing thing that I need to share with you We believe in eating our own dog food and so our web page Was running on our VMs backed by Seth and not properly backed up So we lost Yeah, we Might as well get it out of the way we were the people primarily affected. So how did this actually happen? So remember this was our plan triple the scale triple the size so This is some really important background our cluster started out as a Fujitsu appliance and It had a lot of placement groups something like ten times as many placement groups as Would be recommended for something that wasn't a Fujitsu appliance and It's also important to note We weren't changing versions We were just adding storage and there's a pretty well-known clear mechanism for taking advantage of this failure recovery You add an OSD it remaps everything It adds the data to the spreads the data out. What we didn't know is that that version of Seth Also stored every bit of history for every change you'd ever made No one had ever tried it with so many placement groups before and There was another thing if you're adding a whole bunch of OSD's You want to actually minimize the amount of time that you're driving out an hour and a half to Holyoke and putting the drives in so we put a bunch of drives in all at the same time and We got a live lock it would come up it would crash it would come up it would crash and What makes it really? particularly painful is that We got ram and we drove out will not we these guys Peter drove out and upgraded the ram But it turned out as we learned later there probably wasn't enough ram in the universe to actually help us in this situation So how did it end? We reached out to red hat in the community for help And the community in red hat said Let us help you which was awesome I'm pretty sure that folks didn't say it to us but said to themselves. Oh my god. Why weren't they using supported bits? Oh my god. What the hell were they thinking but as We were using supported bits at that point Okay As I said, I'm the storyteller here an unsupported config unsupported bits See I'm learning new parts to this story So how did we fix it? I don't actually know how many how long this went on was it weeks Ah Yes, our Rado who's our infrastructure engineer This was planned so that it wouldn't impact his vacation So it was like a month before he was leaving right three weeks a month We've learned a really important lesson Don't schedule anything even remotely near when Rado is going on vacation because every time he goes on vacation bad things happen So the first try involved a script Because there was no way to recover from this other than manually so we there was a script that we tried to create with a lot of help and Unfortunately, there was a bug in the script and the way that it was described to me was the stuff cluster became Swiss cheese four megabyte holes got stuck throughout the master boot record and all over the place So we rebuilt the stuff cluster from scratch now This is a really interesting thing and and there's something really magical about the open-source community because When you can actually reach out to the guy who? Essentially wrote saff and say hey Are we doing this right this time? It's really kind of amazing so Thank You sage So then there was the smaller stuff outage this one will be quicker Data centers are never supposed to lose power right well Peter's going to tell us how this one actually happened because he has a better More details on it than I do Okay, so I think yeah so basically you need some type of fire suppression in a you know in a data center and Really there isn't anything better or especially if you're trying to make code Then sprinklers at the same time if you have 400 volt AC for your distribution to all the racks You really don't want to sprinkle 400 volts so and you also don't want water in the sprinkler pipes normally because Over time some sprinkler heads will fail and you'll sprinkle expensive equipment and 400 volts So they have this fancy setup with a high-tech Smoke alarm that basically sucks air from above the racks into a central thing and the sprinkler pipes are empty and Then the first phase then when the smoke alarm fires It fills the sprinkler pipes and then the sprinkler heads work like normal heat activated However, once the water enters the sprinkler pipes again, we don't want to Sprinkle 400 volts, especially if I'm in one of the racks It triggers an emergency power off No generator no nothing emergency power off and There's also a feature to test valves that trigger the emergency power off that was used when they were building the building There's some little valves that you know, I think they look like the ones for your garden hose That will let water in past each of the valves so that you can test that the power off trips Which they did before there were any computers there Now one of those have been cross-threaded and it started leaking a little bit So some maintenance guy went saw a leaking faucet. You know, this is on the computer floor It's like a hundred decibels or something. You know two acres of racks of computers He cranks the valve harder trying to get it to stop dripping Evidently that caused it to do the opposite the water went through the valve It triggered the emergency power off. So he must have been standing there and the whole floor went quiet So I Can't repeat whatever it is that he said I'm sure but It's not a situation that I would want to be in But it wasn't his fault Yeah, anyway, so that that's one of the more Fascinating outages. I've heard a root cause of it took a while to figure out why that happens Yeah, so basically there was an open conference call from the moment that happened until the moment it was resolved and When the power came back on This is the summary of the bug power out of your data center VMs came up without stuff access Turns out there was a configuration problem that we didn't realize we've made The newest VMs didn't have permissions to break the old locks It was a known issue By the time we figured out that it was a known issue. We'd basically manually unlocked all the images So the cluster in this case was not available Roughly overnight These are some of the folks who helped us Thank You Michael Kidd. Thank You Emma and Janatha and thank you Larsh who works for Hugh and spends a huge amount of time helping us get them off operationally more more more more rigorous So That was our major storage area our storage failures telemetry data so We care a lot about telemetry data and we care about it because we want to do research a lot of the interesting research right now Is not coming out of academia as much as it is out of private companies because they've got the data We want that information to be available to researchers and we want it to be available to the community So you guys can do optimizations on things and so we figured out we should do something about this And so in late 2015 early 2016, we stirred up salameter and was very slow and it led to network slowdowns and It led to situations where jobs wouldn't finish because the network was so slow so We investigated manasca a manasca looked pretty good We were using it pretty pretty reasonably But the thing is we also Remember we talked about wanting to do charge back and show back Show back is when you go to the partner universities and you say see how much compute resources and storage resources You're using we really would like you to help fund it support it all those sorts of discussions Charge back and show back They seem pretty straightforward like AWS when I was there sent me a bill Every month couldn't always understand it, but they sent it to me we didn't realize that we were on the bleeding edge and It wasn't even like what we wanted to do was that hard from our perspective So this is sort of the report we'd like to generate I'm gonna pull out the parts that Matter for the MVP. We wanted a report per project With some pretty standard things the project name sponsoring organization Per VM compute usage per VM use per VM memory usage. I think I left off storage like block storage stuff Pretty simple We wanted a second report projects by institution so that we could go to Harvard we could go to MIT We could go to BU and we could say This is what's going on. This is actually really makes a lot of sense. You should you should put more money into this and For that again the MVP was pretty simple project name project lead name compute usage memory usage I think I also left off storage usage and we we wanted Literally by institution one line with each of those things Seemed pretty simple Originally, we were gonna pull the data together ourselves and generate the reports remember at this point. It was probably late 2017 And then we heard about cloud forms and it looked really cool There was just one thing it needed salameter so We went back and we looked at salameter again and It wasn't easy, but we became convinced that It was actually really performant and in fact, I think it's fair to say it it has been We started running it on a test system and it seemed to work pretty well, but then we tried to generate reports and It was hard because our data was living in different places Some of it was coming from rabbit MQ around CPU usage Some of it was coming right out of salameter. Some of it was coming from another another database file and They weren't all accessible via cloud forms and We were talking to some of the guys who were really deeply involved with cloud forms and they would say oh That's a really cool use case. We should figure out how to do that But it's not gonna be in the next version and so we went back and we said okay, we're gonna generate our own reports for now and This was another thing that went on that was frustrating So Cloud forms did something kind of cool it updated itself but the problem was there was something messed up with our ruby gems configuration and It took us several days to get it working again, but this is really Critical both for research and for doing an open cloud exchange And I want to take a second to dwell on it. That's kind of why I made it in bold If you want to offer services to everybody and You want to be able to cover your costs? There's got to be a basic billing model. There's got to be a mechanism that allows us to charge for usage and It just is really hard to do right now all the folks who are doing this now are writing custom things And so like please let me make a plug if there's anybody in the room who's working in this space Please raise your hand and come and talk to us because we would really like to collaborate on it And it's worth noting we found a couple of Groups within Red Hat and other places that are working on this and we're gonna be working reaching out to them to figure out If there's a way that we can work more closely with them, but What we have now works for an MVP, but what we have now isn't gonna work When each of us want to offer a service and get some remuneration for it, so we've got to we've got to fix that Here's some of the folks who helped us Again, thank you Larsh. Thank you Adam Young Thank you Jason Rittenauer and thank you Sumayn Chen these guys were really really patient and Really really helpful at pointing us in the right directions so We've talked about major storage failures I Want to come back to that for a second though One of the things that's really been fascinating is that the research that we've been doing on the MOC has To a great extent begun moving off of things like Manasca and Solometer and more towards tracing and more to things like open tracing and What's really interesting about that and talking to to mania who spoke about this yesterday And talking to Raja who is I think you're is it accurate? He's your PhD advisor, right? No, Peter's your PhD advisor, so Raja's doing a lot of work on tracing is That the guys who are working on on on the open tracing stuff are already thinking about the fact that hey With open tracing we're gonna be able to do really really Detail level of of charge back and show back and billing and so this is an area that We think there's a lot of interesting overlap both between research and between the larger community and we'd really like to figure out ways to So to work on that more So Open shift on open stack. I don't know that we are The only people doing this in fact, I know we're not but it feels like it a lot of the time And this is just a funny fun shift on stack on split stack because we're gonna be running split stack split stack which is a triple open but What are the some of the things that we've run into that have broken open shift for us One of them was we went to update from three seven to three nine In an open shift on stack environment And at least for us it didn't work and as far as I can tell there's problems with it It's not just us in other words Here's some others and I should call out Rob whose our subject matter expert on this isn't here So if you guys want deeper details, I'm gonna get contact information and we'll we'll have a discussion about it We ran into some fun stuff with Ansible We started out by just using IP addresses In order to get the information over to Kubernetes correctly, but some things don't like that some of the different parts didn't like that So it turned out we really needed to have DNS just for open shift I Think you guys probably already know this but DNS name resolution between open shift and open stack There's stuff going on there. It's really hard Certificates have been another Really huge area of pain for us Some of that's probably because maybe we were using unusual certificates, I don't know but One of the things we ran into was So it turns out that if the first system with the etcd gets corrupted or goes down Then all of your certs fail Probably not where we want to be probably not what we'll be in three months, but these are the things that we've run into Trying to trying to run a production cloud so again Really really critical. I think all the people who've been amazingly helpful about this starting with Dan who Must be one of the most I don't know if Dan is in the room But if you haven't met him he must be one of the most patient and smart guys I've ever come across Raffiella, Savetta, Sam Padgett, Aaron Woodacamp, and Daniel Jeffery they all Some of them in just in the past week or so have become involved to help us solve some of these cert problems And we're really grateful for that So some things we've learned This is a really important one that I wish we didn't do this about as often as we do Anytime we think a project or real feature will work We ask are any current customers using it this way and it turns out that often they are but often they aren't Usually we just ignore the fact that we ask the question which is on us five minutes. Okay If Rado's out of town bad things will happen We learned that too This is a really important one for us because we are very grateful for the close relationship We have with red hat but when we open a ticket if it's critical we ask somebody at red hat to help us sort of Nudged along Larsh Is I don't know if you guys know Larsh, but Larsh is this wonderful developer who's helped us a huge amount And anytime I start to write an email. I literally hear him in the back of my mind going why is this not a Trello cart? and so That's a really powerful thing and it's really important I'm pretty sure I forgot to say thank you to some of the many people who helped us both within the community and within red hat And so please let this thank you go to the people I may have not known about or forgotten And he wasn't supposed to be here. So this next one Is also really important to point out Please if you see you You sitting right there say, thanks you But don't tell them why It'll be pretty amusing if he doesn't know Any questions Anybody who wants to work on some of these cool problems that we're talking about Nobody's raising their hands. It's okay.