All right, good afternoon. So we are doing the lab four primer, which will also help us prepare for quiz three because we'll be messing with processes. So this will essentially be able to write some useful program that will essentially be lab four. Hopefully we can do it by the end today and that should well prepare you for lab four. So this is mostly you guys programming because I'll show you the tools and you guys will come up with a solution and it'll be lots of fun. So here's our task for today. We want to send and receive data to a process. So we want to actually do some IPC. We wanna communicate with some processes.  We're gonna do some actual communication while both processes are alive. So our task for today, we want to create a new process that launches whatever program we give it as a command line argument and then we want to send a string testing with a new line to it and then we want to read any output it gives and that's it. So we want to send data to a process, get data from the process and turn it into a let it execute a different program. So we're gonna need some APIs we'll whirl around through them. Our first new API is called pipe. It is kind of a one-way communication stream. So here int pipe and it takes an array of ints that are just two file descriptors. So it's gonna create two file descriptors. So it's a C wrapper for a system call. So it returns zero on success, minus one on failure, sets error no, kind of what we're used to. So it's a one-way communication channel using file descriptors. So again, I said it gives you two file descriptors. So it gives you file descriptor at index zero, which is the read end of the pipe and a file descriptor with index one, which is the write end of the pipe. So the way this works is any data you write to the right end of the pipe pops out of the read end of the pipe and you use the write and read system calls just on any old file descriptors. So you can think of this pipe as a kernel managed buffer. Again, any data you write to the right end is gonna pop out of the read end and we'll be able to use this to communicate. Yep. So if you want to communicate, they're gonna have to both have access to it and that's what we're gonna solve. So if we fork, it's a copy, right? Yeah, but if you create it and then fork, it's a copy at the fork, right? Oh, it's gonna shake all day because they're jack hammering. Yeah, so anything you write, it doesn't matter what process has it, as long as they have that file descriptor, if you write to the one end of the pipe, you can read that data from the other end of the pipe. Yeah, but the kernel manages the buffer for you so you can use it to communicate between processes and that's what we'll do. Yeah, that's the same question. Okay, we'll see how you use this. Yep, the kernel gets the, the kernel does it, don't worry about it. So it's a kernel managed buffer, there's probably a limit to it, but yeah. Processes, so it's a file, it gives you two files descriptors, you write from one side, pops out the other end. That's all it does. You can use it in a single process, but at that point you are communicating with yourselves, which is pretty lame and we can do something better. Yeah, so one way meaning that if you write into it, it just comes out the other end, but you can't write and go the other way. So if you wanna communicate in both directions, you'd need two pipes. Yep, they could, but if you wanna communicate with them, one should be doing the writes, that's sending data, the other should be doing the reads that is receiving data. Yeah, you can't see the change after the fork though, right? No, they're independent. Yeah, so file descriptors are kinda like pointers, so they're pointers to something that, whatever that file descriptor represents. So when it copies, it's gonna copy all the file descriptors as well in the kernel. Yeah, but the kernel will allocate that for you and manage it. Yeah, that's an array of two ints. Yeah, so that's the argument, but I mean everything's zero index, so if you give it an array of two ints, you have index zero and index one. You don't have index two. So we're gonna do this. So we have a task today. Okay, second one, and then we'll throw it all together and have more questions.  You can use exec lp to make your life a bit easier, so instead of giving a full path that starts with the slash, you just give it the name of a file, and then instead of an array, you just give it a bunch of strings for the arguments and when you're done, you just throw it all at the end. So this behaves the same way as exec v e, except it's a little more convenient under the hood, lib, or yeah, the standard C library is gonna transform it to exec v e, so it's just to make it easier to write for you. So if it's successful, it doesn't return because it transforms this process and starts executing whatever the program is and it's going to search for executables given some path environment variable that's default set for you and this is how your shell would work if you just typed ls. So I'll also find wherever that executable is. So it just gives us a little bit more convenience. All right, next API we're gonna need in our collection is close. You close a file descriptor, so it returns zero on success minus what? Yeah, some aliens are attacking, cool. So close, it's another C function, zero on success, minus one on failure, so that's error no. Close is the file descriptor for the process, so that's no longer usable and the file descriptor is just a number, you don't know what it refers to. So it frees up the file descriptor and it can be reused because the way the links kernel works is whatever it creates a new file descriptor, it's going to use the lowest number possible and we kind of saw that way back in the day. Yeah, we're razor fast.  Yeah, yeah, but your web server's just gonna read any old data, right? So up to you to make sure the data's valid. So this is direct, I'm going direct from one process to another. Yeah, but if I'm reading a file in my system, I have to do an open call and it makes it a file descriptor anyways. So, and then my program, my file descriptor could represent network connection too, it doesn't, it's just file descriptors are anything you can read and write from. Doesn't matter. No, it's just called a file descriptor. File descriptor is just a number that represents something you can read and write bytes to. So we've seen you're used to it being a file, but it can now be a pipe. It could also be a network socket, which you'll probably see-ish, but everything we learn, it's all the same. Yeah, so it's just making our lives easier. So it searches for that file in your path instead of doing like slash user bin LS, you just put LS there and it'll find it for you. So it's just making our lives easier. No, so this is instead of exact VE, remember we had exact VE before and we did slash user slash bin slash LS. So instead of that, with this version, I can just put LS in this argument and it will find wherever that executable is for you. So it's, no, so it searches for everything in this path environment variable, which that's like the default thing here we can see. If we echo path, that's how your shell works and how everything works. So it's gonna search it in that order. Yeah, first folder it finds. So that's why they put local before the system one, so it would take any local changes before. Like your UG whatever system's gonna do the same thing. Yeah, but in my shell if I do that, right, I don't give it the full path. So when I do this, my shell is essentially doing that, searching path to find LS for me. The LS program, like the executable file, then it'll launch it. Okay, so that was closed. All right, so our final API is our dupes. So we can screw around with the file descriptors. So dupes takes an old file descriptor and a new file descriptor and makes both of them point to the same thing. So in dupes case, it just returns a new file descriptor on success, minus one on failure, sets error no. And then since it returns a file descriptor, whatever file descriptor it returns points to the exact same thing as the file descriptor argument you pass as old file descriptor. So and in dupes too, you get to pick what the new file descriptor is. So you get to reuse new file descriptor. So if it's open and valid, it will go ahead and atomically close it and then replace it with, and then make old file descriptor points to the same thing as new file descriptor. And this should, and this is why it's a different system call because it needs to be atomic because it's gonna close and then it has to reuse that same number. So if it's not atomic, you might have another process come swoop in and steal that file descriptor number. So that's what they do. And yeah, any questions about that? It returns a new file descriptor. So both of these return the new file descriptor. And in the case of dupes too, you would, yeah, it makes a duplicate of old. And so in the dupes case, it will return new. Well, in both cases, they return new, but in the first case of dupes, you don't pick what the new number is. And in the case of dupes too, you get to pick what the new number is. So it's gonna close it first if it's valid. Is there a question at the back?  So we're gonna do that now. That's everything we need to know to do our tasks for today. So we'll see. Oh no, okay. Let's get to it. All right, so again, here's our task. We want to create a new process. We want to receive any data it writes. And it's just gonna write data as standard input. And since we can play around with file descriptors, we should have an idea what to do. And then last one, we want to send a string testing to that process. So what would our first step be here? Do we know how to create a new process? Fork and then execute something? Exact feed, okay. Let's start with that and then we'll work with the communication. We'll use LP. So here I have some placeholder code and then I fork. So at the point of the fork, they're exact copies of each other. The only difference being the value of PID. So just to make things more readable, if it is greater than zero, it calls a function called parent and then gives a PID of the child. And if it's the child, it calls the child function and passes in the first command line argument, which should be the program to run. So if we want to just run the program in the child, we can just do exec LP and give that argument to it and it should go ahead and find it for us. So if we go ahead and compile that and do you name, so you name just outputs like the name of the system it is. So that's a good one to test. If I do that, print out Linux, which is exactly what you would expect if you just ran it. So why does it do that? Maybe what's going on? So yeah. So here the child changes to you name right after the fork and the parent does nothing and it just exits, right? We're all good. So why do I see Linux on my terminal? Yeah, your name is running, but so when I start my program here, I have some file descriptors already. That's just by convention. There's three file descriptors whenever a process starts. So anytime you print the three file descriptors, so in the parent, it calls parent, which does nothing exits with the return of zero. And then in the child, it goes ahead and just exec LP's you name without messing with the file descriptors at all. So it'll be the exact same terminal as what we originally started with. So it would just print out the results to standard out and we see it on the same terminal. Yeah. Yeah, but we haven't done anything with them yet. So there's this place holder so I don't have to write code to make this faster. So we haven't done anything with pipes yet. Yeah. So they just refer to something. So they're kind of like pointers. No, just by convention, it's just zero is standard in, one standard out, two is standard error and what they actually are connected to will change depending. No, it just depends what you wanna do with it because we're gonna screw with them right now. Right, so what's working, you name is just outputting its answer to, it's just writing to file descriptor one like we saw in the beginning. So if I wanna capture its output, what should I reach for? So I'm trying to communicate between two processes. So I probably need a pipe to communicate between them. Yeah, wow, we're not awake today. Okay, so what is, so far we just forked. So ignore all the declarations, I just forked. We know what fork does, creates a new process. So now after the fork, I have two processes. One's gonna call parent, one's going to call child. If I am the parent one, it just does nothing and then it would just return zero. It's done so, it exited zero. If I'm the child, it's gonna exec lp the argument which in this case was uname and then that's it, the child just becomes uname. But now I wanna capture the output of the child. And just read it in the parent process that created it. Put into file, yeah, but put, okay, yeah. But first I need to create a pipe. No, I just allocated space for them. There's no call to pipe. Okay, so first, so this is just an array declaration that just zeroes it out. If no one's read this syntax before. But so I have these check error functions that essentially check error just returns if the checks if the value is negative one. If it's not negative one, it means it's fine. And if it is, it's just gonna print error message and then quit out of the process.  So when check error, we're gonna call pipe. So we'll have a pipe that represents the output that we wanna capture. So we just give it the array and after it will set it for us. So we have this. So now after pipe, it's gonna create two more file descriptors if it's successful. So file descriptor, it's gonna use the lowest number. So we have the default file descriptors up here. And then we have, it will create two file descriptors. So we'll have out pipe FD, which will be file descriptor three. And that is the read end. And then we'll have file descriptor four, which is out pipe FD, whoops. Pipe FD, pipe FD. And this is the write end, okay? So what also, yeah. Yeah, and the kernel create an internal buffer. So if you write to one, you can read it out of the other. It's just a number. If you can come up with the numbers yourself. Yeah, so the process could, so standard processes open like this with three file descriptors, but someone might be a jerk and whatever they forked and then exact you, they had like 10 open file descriptors. So I know because I'm launching it through my terminal and I know nothing screw is going on, but you won't always know this. Okay, so that's my pipe. Why don't I just pipe here? Close, close. Anyone want to correct? That was so close. So after the fork, they are the same. Yeah, and then after that, they're independent. They both call pipe. So each new process creates its own pipe and they would get the same file descriptor numbers, but they would be different pipes. So the child would create a pipe, it would get three and four, and then the parent would create a pipe, it would get three and four. Yes, they would be different pipes. So this way I have two pipes that I can't share anymore because I should have done it before the fork. And then if I move it up here, I have one pipe and both processes have references to it. Yeah, mm-hmm. Yeah, it has like a file descriptor table. Yeah, it has a file descriptor table and all of them are essentially, the file descriptor table just has a bunch of pointers to stuff. It's just a table, you- People would just put that. Yeah, yeah, so if I, P and E. How many are you using? So for example, I think I showed this in another lecture, like you can see all the open file descriptors for a process just by looking at this directory. So for example, my shell had, at one point it opened 103 file descriptors. So it messed with a lot of stuff because they're broken. So they're different colors depending on what they actually refer to. So this one is like some file. This one's a socket. These are all shared memory that have to do with Chrome for some reason. And then it has its own status but you can screw around with file descriptors. I think I showed it before, but that's neither here nor there. Yeah, that launches your kernel. You can, but generally, if you like to BIOS updates, it's like a EEPROM chip and it takes a long time and it's generally the BIOS just boots your kernel and then tries to hide itself. But you can interact a bit with it depending but that's much more advanced. Okay, so now we have a pipe. So out pipe FD is correct. So if I want to capture the output for this process, it's writing to file descriptor one. So if I want to capture it, I should probably make it right into the pipe instead of just whatever file descriptor one is. So how would I do that? So here's a hint. We want to use dupe two. Oh, this calls as much more confused than the other one. Okay, here, I will write it and I will proceed if someone explains it to me. Or here, I'll make a pretty number. So what would that do? Which is just a file descriptor that's one. Yes, so it's the right end of the pipe. So I'm trying to, in the child, I'm going to copy the right end of the pipe and replace and make file descriptor one point to the same thing as the right end of the pipe. So now I just overwrote because file descriptor zero originally was copied from the parent and it was to the terminal and now I just replaced it. Well, you only have three file descriptors so we're just replacing it. Well, so standard error still connected to the terminal but in general, you'd have to, you lose it. Yeah, all right, first. So dupe two should return one because that's the new file descriptor it gave. Yeah, which now points to the right end of the pipe after. Yeah, right. Yeah, but I only want three file descriptors when I launch this process just to be nice. No, it replaces. The only way to make a new process is fork. Everything else gets replaced. So like if it's already has like the libc open and it uses libc, that won't get touched and like some other things like that. Yeah, the right end of the pipe. So here I'll go ahead, check for errors quick because we should always check for errors. So if I do that, it makes, it's fine. And now if I run it, what do I expect to see? Nothing, why? Yeah, so I'm not reading from the pipe so the data's going somewhere, I can't see it anymore. Yeah, that's up to the kernel. I think it would just give you an error eventually that you filled up the buffer. So it doesn't lose data. Okay, so everyone good with that? It's getting captured. It's writing to the right end of the pipe. Let's see what happens if we go to the wrong end. So if we connect standard out to the read end of the pipe something very bad should happen. And there you go, bad file descriptor. So be trying to write to the read only end of the pipe which you can't do, that's an error. So we can see that we're actually doing the right thing. So you should only write to file, like write to file descriptor zero and only read from file descriptor. Only read from zero and write to one. Yeah, that's not weird, that's cool, right? This is how things work. Yeah, so it has permissions that the kernel keeps track of. So write only, read only. All right, so let's be good citizens here and close our file descriptors just to be nice and clean. Close this. So I'm just closing, so I'm copying, making essentially, if I give them numbers I'm making four and zero points the same thing. And then I'm gonna close three and then close four because I don't need them anymore. Yeah, in different processes too. Because the parent still has file descriptors open. I've closed a pipe that the parent has but to close a pipe completely every reference to it has to be closed. So the parent still has a reference to it. Okay, so this takes care of the child. It's writing to the read end of the pipe and it's not going to use the, sorry, it's writing to the right end of the pipe. So it's not going to use the read end of the pipe and we already cloned it so it's pointing to what we want it to point to so we can close it too, we don't need it anymore. So whenever we exec LP, there's only three file, there's only like this, there's only the three file descriptors. So there's still only these file descriptors when it launches, except one, we replaced it with what was originally out pipe FD1. And that's why our terminal worked. So we cloned it and then it printed to the terminal because we were sharing the same file descriptor. Okay, so it's writing to the pipe, how do I get data out of the pipe? And I probably want to do it in the parent. So how do I get data out of the pipe? Such a responsible parent. Is read blocking or not? So remember we wrote cat that just read and write? It's just stay there waiting for data if we called read. So we can call read, that's cool. So let's call read. So what does read need? Read needs a file descriptor, a buffer and some bytes. Who wants to tell me how to create a buffer? Come on, I hit my head this morning, I don't know how to program.  Character. Character, create a character array. Buffer, I'll just give you, I'll give you a good size. That size is really good, you'll see why later. So I'll create a buffer this size. Now I can do read, right, I think. So what file descriptor do I want to read from? Okay, so I want to read the data that the child wrote. So the child wrote to file to out pipe fd one. So it should pop out of the zero side. Out pipe fd zero. And then I'll give it my buffer and then size of buffer. So that should come, I'm missing a bracket, thank you. Missing a bracket. So what does read return? Perfect. Yeah, now number bytes at read or negative one if it failed. Bites read equals this. And then I can check error because it's negative one. So I'll check error for my read. All right, cool. So my parent process should have read from that. So if I execute it, what should I see now? Nothing again. But hopefully I read some data. So let's print it out. So this format specifier is gonna be weird. Just trust me, it works. So how do I print off the data? So did I read a C string?  So anyone who's advanced enough to tell me why I should put here? Oh, everyone should know this off the top of their head. There you go. So that means, so this is a string, so there's still percent sign string, but there's a dot star that's saying how many characters to print and it gets the first argument. So bytes read is the number of characters to print and then that's the array. So I don't have to give it, it doesn't have to be null terminated, I'm just printing that many bytes. No one's taught you this? No one knows this off the top of their head? Come on now. Yikes. All right. So if we do that, hey, cool, it outputs. And it outputs because in the print F, we didn't touch our standard out, so we're writing to our original standard out, which means we can see it on the terminal, but we wrote out the data we read from the process. So we communicated, we got data. Yeah, but what about if that process did something cool and you wanted to capture its output and do stuff? So for instance, what if I had to grade your programs? I'm gonna run your program and then maybe I parse some output from it so I could just read output from it and do something interesting. So hey, that's how our grading works in the Python sub-process module. All right, any questions on that? Yeah, yeah, because we're setting up the file descriptors. You can only create a new process from a fork. They're gonna be created from a fork. So it's specifying how many bytes to actually print and it takes the first argument. So instead of just a string, it takes a length and then the string. Yeah, so if I just say, hey, print the first two characters, all right, that should just print li. What? Oh, that, oh, at the bottom. So the terminal will tell me if it doesn't end in a new line. Yeah. So yeah, in the child, I overwrote it so that it went to the pipe instead and in the parent, I never touched it. I mean, it would just, you'd write into that and write data to yourself and- Write it into the buffer. Yeah, which is kind of silly. Okay, let's do the other half.  All right, speed run. So let's create another pipe in pipe. Okay. So now we have two pipes. So we want one that we send data to and we already have one that we get data from. So our child's probably gonna look more or less the same. Oops. So I will just replace out in. How'd I do that? I'm cheating, send it in file. Okay. So now I'm replacing this. So we sped run this. So now I'm replacing the standard in with the read end of the pipe. So now whenever it reads data, it's gonna be reading from the read end of the pipe. So I need to put something into the pipe. So in the parent, the right end of the pipe is going to be in pipe FD one. That's what I should write. So I said I wanted to write a message. So I wanted to write testing new line. So I want to write message Stirland message. So it's just gonna wait for some data. Okay. So now check error by certain totally we're speed running now, right? All right. So that should write some data to that process. So I think that's all we need. Yeah. So if we execute that, if we do you name, we should expect to see the same thing because it doesn't read anything from standard in. All right. Just does that. So what reads something from standard in cat? So cat, if I do that, if I hooked up everything correctly, which maybe I didn't, I should see testing. Hey. So there's the data I wrote into it. Yep. So let's see. Let's quickly be a very good parent. So you told me I should wait. Let's wait three minutes. Jesus. So, so this is how fast you program your labs. Right? Check error. Pid. Oops. All right. So we want to wait on it. Let's just write some asserts. W if exited. New status. Let's just assert. And then in exit status, we should get its exit status because that's a good thing to do. Do you exit status? This is what we had before. And we'll just assert that exit status. What, what's good? Good zero. All right. That's what everyone was going to write. All right. Let's run it now and see what happens. So what's happening? So we're waiting indefinitely which means our cat process isn't exiting. Why would cat not be exiting? We wrote testing. So I wrote testing to it, but it's still waiting. The child's just executing the program. So what's happening? I'll give you another like few seconds. So I wrote to, I wrote to its standard in. I wrote the string testing. It worked before. It was fine. I got it. I printed it out when it was done. That was fine. But now I'm waiting for it to die. It's not dying. So why wouldn't it be dying? Well, so essentially cats waiting to read more input and more input is possible because we even closed the right end of the pipe. So it's still, it's just waiting. It's possible it could get data. It's possible I slept for like half a century and then I'm going to call right again. So what I should actually do is after I write to that pipe, I should be a good citizen. This is why you should be a good citizen because now I'm done with it. So I can close it. And that way cat will know that no other output's possible because no one has that file descriptor open. So you don't. Because it's closed, it means it's done no more output. That's the only way cat, you can't. So I'm waiting for it to die. So I'm waiting for the process to end. How can't communicate with a dead process? All right, so if I do that works properly, continue this on Discord or next class because everyone kind of looks confused. All right, just remember I'm pulling for you. We're on this together.


Hi there, I'm Jeff Watts and welcome to a new playlist on my YouTube channel called Agile Interview Questions. I get a lot of messages from people asking me about what they might get asked in an Agile interview or they've been asked a difficult question and weren't quite sure what they should have answered and I even get questions from people asking me what kind of questions they should ask in an interview and what kind of answers to look for. So I thought I'd give it a go and see what I can do to help you with those Agile interview questions. The first thing I want to get out there is that there's no guarantee that the person interviewing you and certainly not the person who wrote the job out there, have an in-depth knowledge of Agile values and principles and the nuances of the role that you might be interviewing for. If you're in the interview and you get the feeling that this person or these people don't really get it, that's a more of a reflection on them than it is on you. And if you end up not getting the job, I would suggest treating that as a lucky escape. So the first question we're going to look at in this opening episode is one I've been getting a few messages about recently and that is what metrics do you use to assess an Agile team? Well, first of all, this is a tricky one because actually I'm personally not that interested in assessing a team. I'm more interested in whether a team can and wants to assess itself. And secondly, a lot of the aspects of what makes a team a team and makes a team Agile are very easy to measure. Now, there's a famous Einstein quote which says, Not everything that can be counted counts and not everything that counts can be counted. Now, while you may struggle to get that rationale passed an accountant, there is a lot of truth to it. And I think that's true not just from an Agile perspective and a team perspective but in life. Now think, is it easy to measure love? Is it easy to measure happiness? Is it easy to measure friendship? But you know they're all important and you know it when you see it and when you feel it, right? Well, it's quite similar in an Agile context as well. Just because something isn't easy to measure doesn't mean that we should not try and move towards it. We can still try and get more of something or less of something even if it's difficult to actually quantify. So for example, love may be difficult to measure, but hugs might be a decent proxy measure for love. The more hugs we have, the more love we feel, perhaps. So we'll be able to find proxy measures that can help guide us in the direction that we want to go. And you've got to be careful, obviously, because every metric is going to change behaviour and there's a good chance that it will be gained. For example, I used to work at an organisation that took the logical view that retrospectives are Agile. Agile is good, therefore retrospectives are good. The more retrospectives we have, the more Agile we are and the better we are. Now, you can probably see the floor in that already. So we ended up seeing teams have multiple retrospectives per sprint just to hit that metric. And while retrospectives are a good thing, by overdoing it and doing it for the metric's sake, actually saw the reverse. We saw less creativity. We saw less engagement. We saw less improvement. We saw lower morale. And the value from the actual practice went down. Now if we look into this deeper, we can realise that it's not actually the practice of retrospectives that we want. But rather the continuous improvement that that practice can give us. So we're looking for metrics that will help encourage the behaviour rather than institutionalise and encourage a specific practice per se. So what are we looking for? Well, when it comes to the successful Agile teams that I've seen, they tend to have a few common characteristics. The first one is self-improvement. So you see more and more examples of teams getting better by making and taking opportunities to improve. For these teams, staying good enough doesn't seem to be good enough for them. They just want to continuously improve and get better. And they take pride in that. They don't also need to be told to get better either. And they probably won't even wait to the retrospective to find opportunities and put those opportunities into practice. But they'll improve whenever they see an opportunity to. The second thing I tend to notice in great Agile teams is quality. So great Agile teams take quality seriously. They take pride in their work in doing things right and doing it well. Not talking about over-delivering and gold plating here. They do try and minimise waste and over-delivery and focus on the simplest thing that will work. But they do make sure that it does work. One of the Agile principles which happens to be my favourite from the Agile manifesto is continuous attention to technical excellence and good design enhances agility. And this principle really resonates well with great Agile teams. As well as delivering well, they're always taking opportunities to tweak things and make things better and leave things in a better state than they found them in. Those are hallmarks of a great Agile team. The third thing we tend to see in great Agile teams is unity. What I mean by unity is sticking together great Agile teams generally put the team first over the individuals. I don't mean that we don't care about individual development. We do. It's just that the needs of the team will trump the interests of the individual team members when it comes to it. The individual members of a great Agile team don't tend to see that putting the team first as a sacrifice either because it's something that they gladly do, mainly because they see the benefits of being part of a great team. It's something that they enjoy and it's something valuable to them as an individual. And they know that they can expect the same behaviors from their teammates. So look out for some stories where people go outside of their job description or they're swarming or they're taking collective responsibility for things. They're good examples of unity. Another thing I've noticed about great Agile teams is that they're brave and audacious. They take risks. They put themselves out there. Both as individuals and as a team as a whole. And they enjoy pushing themselves. They seek responsibility. They seek autonomy. And they don't really see it as a risk like some people would because they know they've got each other's backs. They know they've got support for when things get tough. And they've also got many examples of success as a team as a sort of backup to draw on. Now, this doesn't mean that they're easily persuaded to take on impossible tasks and burn the midnight oil in order to get them done because they know that would be to the detriment of their physical well-being, mental well-being and the product itself. And it doesn't mean that they value individual heroes either. Instead, these teams, they look at a situation and they see the possibilities rather than the blockers. They assume that it is possible and success is an option. They say, let's give it a go rather than, oh, I don't know. They challenge the status quo and they also challenge their own assumptions. Finally, despite all of those other factors that I've just talked about, great agile teams don't forget about delivering. At the end of the day, they know they're here because they have a job and they need to deliver stuff. They need to work hard and deliver value. And they enjoy delivering value. They enjoy satisfying and, well, really delighting their customers and their product owners and their users. Great agile teams find a way to get stuff done rather than just say, well, I've done my bit or I'm making progress. But bear in mind, it's not about story points or velocity or efficiency or utilization. It's whether our customers are happier. Is our net promoter score going up? Are we becoming more predictable in our delivery? Are there fewer last-minute panics? Agile teams take pride in their deliveries being boring. That might sound strange, but they don't create dramas. And when we factor in their previous focus on quality, we know they're not just delivering shoddy work just to get stuff out of the door to meet a delivery quota. They're delivering good stuff too. Now, if you've been paying attention, and I'm sure you have, you will have noticed that these traits that I've been talking about here have formed an acronym because great teams have squad depth. And I want to come back to something I said near the beginning, which is me assessing a team is never anywhere near as good as a team assessing itself. And great teams want to be great. They want to become greater. They want to stay great. So helping them define what greatness means for them and then giving that team the tools, the environment, the support, the space and the psychological safety to be able to inspect and adapt themselves is the number one thing that we really want to be looking for and looking to create as any kind of leader within an agile organization. I hope you liked this video. If you did, then I've got some metrics that I use to determine my value delivery, whether I'm doing well. And some of them you can help me with. So if you liked it, give it a thumbs up. That's one metric I use to determine which playlist to focus on in the future. You can give some comments. You can subscribe. These are the metrics that I use to determine whether these videos are adding value to you. If you've got another interview question that you'd like me to tackle in a future video, then let me know. Leave a comment and I'll try and cover it. Don't forget to subscribe and click that little bell next to subscribe because then you'll be notified when a new video comes out. You won't have to keep checking. And if you've got an interview coming up, good luck.


sensational news of the war, the surrender of Italy. In New York, Americans of Italian descent are first to celebrate. Their sons, their brothers, help bring peace to the country of their birth. Vino for Vien in the history of the world begins the story of the dismembering of the Axis. Troops landed in force to join General Montgomery's Eighth Army in driving Rommel from North Africa is one of the dramatic chapters of the war, our supreme Allied commander and General Montgomery. Lieutenant General Mark Clark, commanding the combined Fifth Army, Casa Blanca. Here, meeting President Roosevelt and Prime Minister Churchill, gave to Germany, Japan, and Italy two words, unconditional surrender. From a hundred African bases, Allied bombers swarm across the Mediterranean, softening Italian island strongholds with overwhelming air power. Action off the coast of Sicily, producing one of the most spectacular pictures of the war. By the strange tides of a lot of the country. Troops numbering more than 13 full divisions, surrendering to United Nations forces. The occupation of Messina, within sight of the Italian mainland, foreshadows Italy's unconditional surrender. Troops of General Patton VII and General Montgomery's Eighth Armies join in preparation for attack on the mainland itself. Messina's municipal building, the stars and stripes proclaim the Allied occupation. Allied soldiers and civilians meet on new and friendly terms. Her choked with ships, bombed and shelled from air and sea was untenable for the Germans. Forced to leave from the beaches, the Nazis fled across the spades of Messina. The sons of British and American artillerymen began the assault on the mainland of Italy. Battleships, cruisers, destroyers, submarines. The world's fifth largest navy. Now, when President Roosevelt says, the armistice with Italy has been concluded. It is a great victory for the United Nations, but it is also a great victory for the Italian people.  But let us not delude ourselves that this armistice means the end of war in the Mediterranean. We must drive the Germans out of Italy as we have driven them out of Tunisia and Sicily. We must drive them out of France and all other captive countries. And we must strike them on their own soil from all directions.


We have our next participants here. So we have Hilary Lipzig and Christian Hernandez, both from Red Hat and the topic will be Helm and Back Again, an SRE Guide to Choosing. All right guys, you can start. All right, so I'll start again. My name is Christian Hernandez. I am actually a Technical Marketing Manager, Senior Principal Technical Marketing Manager, probably one of the longest titles I think I've ever had. Over at Red Hat, and today we're talking about Helm and Back Again, trying to decipher when to use what, whether to use one over the other. Kind of a little bit of background first. I work in P&T and in product management and engineering. So I work with Red Hat customers who are using our products to keep them happy and to help push their RFEs upstream. And a lot of the times I test Alpha and Beta software before anyone else. So it's broken, hopefully with me first before anyone else. So I spend a lot of time talking with them about operators and Helm and when to use what and giving my perspective on that aspect. So with me is Hilary. So Hilary, if you could introduce yourself. Sure. So my name is Hilary Lipzig. I am the team lead for the Cloud Services SRE team here at Red Hat. We are the SRE group that supports several of OpenShift dedicated managed services. So Red Hat Open API Manager and some others. And actually part of the reason for this talk, this talk is all my fault. Christian sends me his black message one day. He deals with the customers on the sales end and I deal with the customers on the support end in a lot of ways. He says to you one day, he says, I'm trying to decide if I should write a Helm chart or an operator. And I said, well, what is the operator going to give you in exchange for all that overhead? And he says, nothing, I just already know how to do them. I said, great, sounds like you're writing a Helm chart. And that turned into a much longer conversation that turned into the slides that you're about to see. Yeah, yeah. So this is basically a conversation we had turned into a presentation. So this is kind of a little background for everyone. If you ever have a really cool conversation, just turn it into a presentation. And so a little, little over an agenda, right? Some of the things we're going to want to talk about here, we're going to, we have like about 50 minutes. So we're going to use all that time. So we're going to talk about what our operators, right? The pros and cons of operators, what's Helm, right? Like, what is Helm? Kind of a background on that, the pros and cons of Helm. And then pretty much what you probably came here to see, when to choose one over the other. And then talking about how to transition between the different types of deployment options out there between Helm and an operator. So for those of you who know operators, for those of you that know what Helms are, sit tight, right? It'll probably be an overview for you first, before we get into like the deep weeds of, you know, why to choose one over the other. So I believe I'm going first. It is what is an operator, right? And I apologize for the background noise. Again, everyone's remote, right? But we still apologize for the background noise. And so, first of all, what is an operator? So before we get into what an operator is, let's talk about what CRDs are, right? So I'd like to like start there to kind of lay the groundwork. So firstly, CRD stands for Custom Resource Definition. So Kubernetes at its core is really just a collection of controllers that you interact with using an API. And these controllers act on objects and resources that are scoped into its own property domain, right? For example, the pod controller manages pods and the replica set controller manages the scale of that pod. So, et cetera, right? So they have their own particular domain. They manage their own particular domain. So these controllers are declarative in nature. So desired state versus running state, paradigm, right? So like if it looks at the running state and if the running state is different, then it does not... If it's the same, it does nothing. And if it's different, it tries to reconcile that difference. So these primitives allows us to manage containerized workloads. And in layman's terms, it uses Kubernetes really as a resource controller. So what if you wanna manage other resources, right? What if you wanna manage other things beyond what's in the Kubernetes API? So for example, I want Kubernetes to recognize my VMs. I want Kubernetes to manage more abstract things like database cluster, right? So in order to do that, you need to submit a proposal for the Kubernetes code base. You know, that was kind of like the past, but not anymore, right? So now you need to create a CRD, right? So by creating a CRD, you're basically telling Kubernetes that there's a new custom resource type. For example, like here in this example, you see fancy DD, right? So you can pretty much name it whatever you want. And then when you inquire, you know, against that custom resource, Kubernetes understands your language, right? Kubernetes value kind of defined of like, you know, what is Kubernetes, you know, what is fancy DB and Kubernetes understands what you mean when you start speaking that language to Kubernetes. So now when we create a custom resource using the type defined by the CRD, custom resource objects gets created, right? So now you can say, hey, get me everything that is fancy DB. So this is fine for simple CRUD operations, like, you know, create, update, delete, you know, objects inside of Kubernetes, but what if you want more complex things, right? What if you want things that only operators know, you know, how to manage, right? System operators know how to manage, like how to recover from failure, how to back up the object, how to restore it, storage management, right? Like, et cetera. What if you want a little bit more complex things for Kubernetes to manage, right? And this is really the idea behind operators, right? So operators isn't like something that's necessarily new to Kubernetes, right? This is really based on the core idea of CRDs. So operators, it's really to codifying operational knowledge, right? As you see here, you know, you're taking your sysadmin, you're taking your SRE and you're kind of putting them in the box, right? You're codifying them there. And it's really, it's a method of packaging and deploying, managing specific Kubernetes applications, right? You know, so I'm talking about applications instead of controllers. So this has operational knowledge of upgrade, patch, recover from failure, tune application and services, right? You can really go into fully auto autopilot here. Operators, you know, by extending that, that CUBE API goes, you know, beyond that and kind of takes care of the application for you. So you're taking this, the admin knowledge, this operational knowledge, it historically has just been taken care of by the admin and you're kind of encapsulating that in codifying it, essentially, right? You're democratizing the apps, right? So now where either, you know, an SRE or a developer can go and, you know, request to this application, the application now kind of takes care of itself and kind of it becomes a self-servicing, right? So this is a really, really, really long-winded way of saying that I'll replace you with a small shell script. There's that old joke that says, say, you know, I'm gonna replace you with a small shell script. This is kind of that on steroids, I guess, is what the saying goes. What's really cool is that you can get started with things like the operator SDK, right? And so I think for any good operator, a good operator requires a story. So I'm gonna hand this to Hillary because Hillary I think has a really good, this is basically her slide, her idea and I think it's a great description of it. Yeah, Christian once told me he wants to see a good operator requires a story on a t-shirt. So maybe someday we'll actually make that happen. Yeah, so basically what Christian said, right? Operators are super powerful because it basically takes that stateful paradigm and applies it to the application layer. And the most effective operators are operators that are released by the community that develops and maintains the application that the operator deploys. Ideally, this community understands the who, what, where and how of their software. So who are the end users? What are they doing? Where are they running this? How should it be managed? You're gonna use your operator to do more than just install software or even just to upgrade software. It's gonna be responsible for other maintenance tasks that you write into it. And sometimes the operator deploys a software but sometimes the operator actually is the software. So SREs, especially within Red Hat, we develop and maintain operators to automate many tasks, not just deploying and maintaining SaaS products. By and large, what we're about to talk to you about is the SaaS products packaged as operators. A lot of the stories you're gonna hear some of the perspectives, some of the advice is specifically applying to that use case. But there is the additional use case. And if you go to any SRE convention, you'll get to hear a bunch of great talks, some of them presented by friends of mine on using operators to solve problems as standalone software. It is a piece of software. It requires a great story. It requires a good use case, planning it the way you would plan any application. And it has a life cycle of its own. It both operates the full life cycle of the application that it's going to be packaging, deploying, maintaining, and it has its own life cycle. So that's really important to remember when you're looking at operators, when you're using operators. It's not just tooling. It's way more than that. I'll go ahead and next slide, Christian. Find that button. Okay. So yeah, so it's important to know, right? I mentioned the operator SDKs, right? And it's important to know that there are different levels of operators. So as you're hearing us talk about specifically Helm, and in this particular talk, we're comparing really Helm in the Golang one, but there's different levels and then there's different types of SDKs, right? So here, as you can see, you have Helm, you have Ansible, you have Golang. In the operator SDK, being in PNT, I talked to pretty much everyone at Red Hat. They're also thinking about having a Java and a Python one. So it's really, this is just the beginning, right? And don't think you need to know Golang through and through just to get fully operational, right? So you talk about the different levels, right? You see here in the slide, there's essentially goes from, you have the idea of basic install, right? Essentially it's just like this operator just installs the software for me and that's pretty much it. To all the way to what we call autopilot, meaning that once you instantiate an operator, it takes care of itself. And I know it sounds like magic and it kind of almost is, right? There's not a lot of operators that are fully level five. Some of our ISVs at Red Hat, you know, like something like CrunchyDB or CouchbaseDB are actually at fully autopilot meaning that if you ever get a chance to see a demo of CouchDB, you'll see how it's autopilot. Basically, when you scale it, all of a sudden your data's everywhere and it'll balance that database out and all of a sudden you just scaled out automatically which is really cool. But as you see here, there's different levels, right? So you have the ability to do just install, the ability to install and upgrade, lifecycle management, you do deep insights and then really double five is kind of like the dirt of Juana as we've been talking about. It's just kind of self-healing, self-taking care of. Basically that picture of the operator going into that box is essentially the whole thing autopilot. So, Pro, no, go ahead. Just to interrupt briefly, the managed services from Red Hat that I support are typically a level four with some level five capabilities. So these are typically operators that are, I don't want to say enterprise ready but they're also not enterprise ready either. So that deep insights, the metrics, et cetera, that really enables not just for your operator to be very useful and to help the people who are leveraging it to understand what's going on with it when something goes wrong but allows you as the developer and maintainer of the operator to continuously improve. Having those insights, having those metrics, having those alertings, you can do as the reactions like create SLOs and SLIs based on that and continuously improve your software because you have implemented that in your operator. Yeah, and I think, no, it's fine. No, I think you touched on an interesting point where the insights are really important, right? So like, I think that jump from level four to level five is I think has a lot to do with all the insights that you're getting continuously improving. So yeah, that's definitely a good point to bring up where that's very, very important, right? Beta is very important and knowing how to codify something depends on what is happening on the system. Yeah, exactly. It's very much a crawl, walk, run thing in terms of leveling up your operators. In terms of like an SREA supporting it, I don't love supporting below a level four. There are lots of great reasons to have below a level four. There's lots of reasons why you might never go up that high. That's a different talk. I think we've actually kind of touched on that again later in the presentation. But as an SREA, what I get from those insights metrics and alerting, what I can send back to the community. And a lot of these are very much their open source projects. So is an SREA supporting them in a managed way, but they're open source. Everything that I encounter operationally, everything that I suggest pushed back on get changed, that goes up to the entire community. Which is more about how Red Hat operates than operator specifically. But because they started packaging and selling things as an operator way, we're able to put back more operational knowledge into the software and benefiting everybody. Yeah, definitely, definitely. So let's talk about the pros and cons of operators, right? And so, so, you know, it's sunshine and rainbows. It may be not all be sunshine and rainbows, but we'll talk about how, you know, some of the good things that operators give you, I think Hilary touched on a lot of them. So it's, you know, it's a community of freely available and open source software, right? And with the operator, I'm gaining all the advantages of technology without the need to like really deeply understand it, right? So really as a consumer, you know, if I just need a reliable database, right? I go back to the CouchDB example, I just need a reliable database that, you know, I don't need to take care of, right? An operator is, you know, of level five, level four and five above is extremely valuable, especially for those teams that, you know, there's those teams, there's the, you know, we talk about DevOps, there's those teams that do a no ops kind of a thing where it's just kind of like a group of developers, you know, maybe using AWS or something and they need something, they don't have a DB admin, they don't have administrators, but, you know, they'll use something like an operator because all that operational knowledge is codified and you gained the experience of the overall community, right? And as really for me as a, like, if you're on the developer side, right? Or if you're on the ISV side, you know, I'm making my application more accessible to the world, right? So for example, you know, I have an application that automatically takes care of itself, more and more people want to use my application and more and more people are, you know, as it gets more and more reliability, more and more people contribute, right? And this is a whole open source ideology as like, hey, cool, I have a lot of contributions and the software keeps improving, my software keeps improving. And I can drive that adoption to my application. Even internally, right?  And so, so the reconciliation loop, right? In the operator, right, we talked about that declarative approach within my own domain of my operator, you know, if something isn't ready yet, right? So like if, you know, if something isn't, you know, as applications growing complexity, you know, certain dependencies come into play, if it isn't ready yet, well the operator can just take care of it for you, right? And so this is kind of the things where scripting becomes a lot easier, deployments become a lot easier because the operator's taking care of a lot of the stuff that custom scripting would have taken care of. And you know, there's no snowflakes, right? And so, you know, one of the cool things about it is that, you know, everything's the same when operating a system that's built on operators, right? It's just declarative. You have that contract, right? We talk about Kubernetes operating on like promise theory where it's like, okay, you know, I declare that I want this application stack to be there, it'll just be there, right? And you operate that the same, whether, you know, your private cloud, public cloud, when you're an Azure, AWS, right? If you're using Kubernetes as that common fabric and operators as how to operate that, you know, you get that consistency across everything. So there's no snowflakes because it, you know, everything's pretty much managed the same way. So, so I'm gonna pass it all over to Hills here, talk about the cons. Thanks, yeah. So I obviously I'm supporting operators that are, you know, packaging and deploying and delivering SaaS products in the field. So I'm gonna maybe make some enemies amongst the operator community here, but they have some serious downsides. That friendly neighborhood reconciliation loop, you'll notice it's both on our pros and our cons side. The fact that it allows for retries and waits means that you're constantly retrying the same thing over and over again, and it will never fail out, even though a failure would be more appropriate. So there have been several incidents that I've worked where the Kubernetes API server was being flaky and that was reducing customer access to their managed cluster and managed services. So the customer complained that they were having connectivity problems. And so I take a look and it turned out that the issue was a reconcile loop that would never finish. So the operator lifecycle manager or OLM, which is an operator that runs to help basically keep all of the other operators on a cluster on fresh versions up to date. Yeah, meta version, right? Of the operators. Operators for operators is what I like to call it. Yeah, operators for operators. The operators for operators is great, but it's also where things get a little dicey. You have to be careful. So OLM was trying to upgrade a community operator. So this is not a managed service. And to add a little bit of context, if you have a cluster with OpenShift dedicated, you can go and download community operators directly through the UI, they're right there, they're available, they're free, right? And it'll just like automatically get it up and going for you and OLM will install it and help maintain it. So the problem was that the deployment configuration for this community operator was not backwards compatible. So what they had done is they had changed a field that was previously immutable. So not only was it no longer immutable in the new version, but it was also different. Well, the recon style loop is trying to upgrade the exact, that exact value, right? And so you hit this condition where it's like, I'm gonna upgrade this, okay? Well, I need to go through and change these values. Oh, I can't change that value. It's immutable, I failed to upgrade. The recon style loop happens, there's an upgrade. I gotta apply the upgrade, I can't apply the upgrade, fail. And this ended up creating so much pressure on the API server and at CD that it actually created issues where the connectivity was down, the ability to access a cluster went down for a couple of minutes. And it was intermittent, right? Because it happened every time the recon style loop was looping. So what we just did is we just went and deleted the old install plan and then the new one downloaded in and upgraded fine. So that brings me to my next con though. There wasn't any alerting for this, right? This wasn't, there was no, no insights, no alerting, nothing going on to tell us, hey, this is failing. This is why it's failing. It wasn't written in, it didn't exist. So without corresponding alerting and logging, watching the operators and the operands, operands being the things the operators operate. I wonder how many variations of operate I can get into one talk, we'll find out. I'm keeping track. You're not gonna be able to detect failures, right? So with something like Helm, because it does fail, that would have been really readily apparent. It would have been super obvious. It would have been like immediate and you wouldn't have ended up with that pressure on the cluster causing problems. Which makes me to kind of my next con, right? Operators have more overhead to write. So a typical operator is gonna have more logic in it than a Helm chart. And there's power there, but it has trade-offs. So developers writing operators must realize that they're writing software with a software lifecycle, I said that earlier. It should be versioned, tested and released as any application would be. That means unit tests, folks, unit tests. Since it's not just tooling, it's not always going to be the right tool. And something or someone still has to operate the operator. So another operator like OLM, operators for operators like we discussed, a person or both in situations like above where OLM steps on the operator and you have to fix race conditions by hand. And finally, no snowflakes. And you'll notice that this again is on my pros list and my cons list. And I say mine because I did the first version of these slides. So as an SRE for a managed service offering, my fleet is not uniform. These are customer-owned clusters with customer-owned workloads. And it creates an ecosystem, not a monolith. I, SRE, an ecosystem. It's a very unusual version of SRE, right? So they're buying OSD clusters and they're buying AWS instances and they have budgetary and size constraints on resources. So like CPU and memory and so forth, right? So one of the things I see a lot is that customers are pushing the limits on their cluster resources, right? They're trying to get the most value for their money. And I can't blame anybody for that, right? That you're paying it. So the problem is though that in operator land you have installs and upgrades tend to spike the resource usage by the operator, right? It's a temporary spike. But if I have multiple applications running, it's possible that there might not actually be enough resources for that installation or upgrade to complete even though there would be enough for the application that the operator is deploying and managing to just run in normal runtime. So it's easy to get around this, right? You just scale down some workloads temporarily so that, oh, we lost Christian. Anyway, you just scale down some workloads temporarily so that that bulky installer upgrade can finish. But operators are managing operands. So if I have replica sets defined, my operator is going to realize I've scaled something down and scale it right back up. So it's gonna undo my changes before I get the benefits of my change. So that also means that I have to scale down the operator. So first I'm scaling down the operator, then I'm scaling down the workloads that I don't want or don't need temporarily. And that begins to get to an uncomfortable amount of room for human error. So if operator B is also being managed by operator A, operators for operators, like we said, I'm gonna have to start scaling down a lot of pieces before I can have my otherwise fairly simple change stick around, and this is not too great. And if your operator story isn't good, if your operator isn't delivering value, there's not a lot of benefit that accompanies that extra toil. And this can hit just, you know, customers who are administering their own workloads, the same as it can hit me as an SRE. And I actually see it quite a bit. Like I think I said that one of the services that I support is an operator and it actually installs four other operators. And some of those other operators that it installs, excellent, super great operators, like the Prometheus operator gets installed, amazing, couldn't live without that. But some of the other ones are not really doing a lot of those day two, day two being operating the application actions that couldn't be handled by the first operator. And so in this situation, that cascade of operators can be a little bit difficult to work with, whereas some sort of mix and match of like Helm charts with operators would be preferable. And really the takeaway here from having common points between the pros and the cons list is that these are double-edged swords. So you need to be careful, be mindful about how you use them. Next slide, Christian. All right, cool. Now we'll talk about Helm. By the way, that was always great. I always like to listen to Hillary actually talk about what actually goes on with customers, because we like to talk about how everything's all great. There's this old saying, everything goes according to plan until you get punched in the face. That's always a fun thing to hear. So let's talk about Helm. So Helm, one of my favorite topics here. So would it be nice if managing applications on Kubernetes was just like any other framework? So for instance, if you're using like a Red Hat distribution, you're doing like YUM install something, or app get install, brew install. So what if installing and managing applications were just like anything else? And so what Helm gives you and what Helm is really at its core is a package manager, package manager and a templating engine for Kubernetes. So it can be used to repeatedly and reliably deploy cloud and cloud native applications with the correct configurations. So ideally the software you want to run already comes with Helm, with Helm chart supplied by the maintainers. But if not, you can do that yourself. With Helm, you have that ability to configure and deploy software across your fleet of clusters. So just like an operator, just like Kubernetes, where we live in a Kubernetes land, I think I saw a joke somewhere, such post somewhere that someone on the resume put a YAML engineer. This is essentially what we've been coming so far in the cloud native ecosystem with Kubernetes. And so just like Kubernetes, just like with cloud, just like with operators, Helm is declarative. And so you provided YAML or JSON configuration. But yeah, so for a time I used YAML for an API, I don't think anyone thought that would ever catch on. Just like, look at it now, right? Yeah. We were in the days of like, XML was still King, right? Yeah, yeah. First time I saw YAML, right? Especially because you and I worked together in this and then I did several other companies like this where it was like, web to print. XML was like the heart of how that worked and how it worked reliably. And so we start seeing JSON and YAML and we're looking, and of course, one being a subset of the other, it's a different story. But so we're starting to see JSON and we're like, okay, yeah, that's pretty good. And so we see YAML and we're like, you know, I don't know if I like it as much as JSON version of things and we're just kind of, I know my team, we have back and forth like, yeah, well, no, it's not gonna be a thing, right? And it's literally yet another markup language. It's almost like the creators didn't take it very seriously themselves. Yeah, yeah. It is, it now runs my life. So just like- Yeah, now it's, now I can, you know, before I used to have problems now, I think I can indent in my sleep, right? Like now it's just, now I can like see indentation problems like from a mile away. I'm like, oh, that's indented wrong. Yeah. It's how weird, how fluffy. But yeah, like back then it's like, why would you ever use this to post a request? Like there's JSON so much easier, but, you know, look at us now. Yeah. So, so with Helm, Helm comes with like, you know, a few different components, right? So it comes with the CLI, something called the chart, something called templates, values, revisions. These are kind of the nomenclature you'll hear when talking about Helm, right? So you'll hear these over and over again. So the CLI is really just a binary, just like anything else is a goal line binary. And it provides, you know, it's basically your interface into Helm, like the ecosystem. It's basically how you interact with Helm, ecosystem as a whole. So charts. So charts is nothing other than YAMLs, right? So templatize the YAMLs. So it's YAMLs where you get to plug in different parameters.  It's really under the hood, it's goal line templating. So if you're familiar with goal line templating, it essentially uses that, right? You have conditionals, you have loops, you can do all kinds of things. It's a really fancy way of managing your templates. And your values is basically the parameters you pass into a chart, right? And so you have a chart with, you know, different variables and in order to plug in those variables, you provide it the values file, right? Foo equals bar, for example, right? Let's use something simple. That Foo inside the template will be replaced with bar in the chart and it'll render your Kubernetes YAML. So it's basically a way to templatize Kubernetes YAMLs. What it spits out is a raw YAML, essentially. And, you know, when a running instance of that YAML, right? So when you merge the two, when you have your Helm chart, when you have your values and you merge them and then you get your raw YAML for Kubernetes, we call that a release, right? You have a release inside a Kubernetes cluster. And then you have different revisions, right? You know, version one, version two, version three. So these are the things that you'll hear with Helm, right? I kind of don't want to spend too much time on Helm background because there's really not a lot to it. And when I say that, I actually mean that in a way that is positive, right? For me, I'm pretty sure for Hillary, but I don't want to speak for her that I love it when software is simple. I love it when it gets out of your way, right?  But in actuality, you want it to be easy to use and easy to consume. So nothing too majorly complex with Helm. It is basically a package manager. Think of it as like, yum or DNF for Kubernetes. This is basically what it is. So, Hillary, I don't know if you wanted to add to anything to that or if you want to go straight to the pros and cons of it. No, yeah, let's go straight to the pros and cons because it's basically just drilling down on what you said. Yep, cool, all right. Pros and cons, so here, I think Hillary's going to play the bad cop here. No, I'm the good cop. I was bad cop last time. That's right, that's right. You were a bad cop, yeah. I was bad cop last time, yeah, yeah. You need more coffee, sir. It's early over here on the West Coast, so for those who don't know, it's early for us. Yeah, we've just hit 8 a.m., so. So there you go, yeah. There we go, yeah. But as an SRE, I'm used to getting waken up at weird hours and having to be on and smart as sales usually gets to work during normal business hours, so that's the difference. So as we go into the pros of using Helm, I could have actually made this list a lot longer and it's still probably a longer list than you might have expected for a tool that's not as intelligent as operators are, but the title of this talk is Helm and Back Again. So arguably, as Christian said, it's a lot easier, actually I said it too, it's a lot easier to get started with Helm than with operators. Simply put, there's less overhead into generating Helm charts than there is an operator and especially because, as I've said before, you don't need to have your day two operations defined and since it's all YAML, you don't need to know go line. So a really nice feature in Helm is the config YAML file which you can use to dynamically control the particulars of your deployment. This in conjunction with a GitOps tool can be really powerful as it can be leveraged to execute deployments with business logic attached. So from the persona of an application developer, this could allow me to execute AP testing with the ability to rapidly turn it off through GitOps pipeline integrations with Helm. So contrasting that to an operator which could be programmed to do AB testing, but then we need to be repackaged and re-released in order to change and remove the AB test. Operators are just not designed for this type of task, it's not the problem they set out to solve. If you're an Ansible user, you can integrate Helm and Ansible if so desired. So this combines the benefit of Helm charts with the power of Ansible playbooks, but this isn't super dissimilar to writing an operator. And in fact, the operator SDK is Ansible under the hood. It could be a really good intermediary step to writing an Ansible operator which is also YAML driven instead of Golang. And I think I mentioned it later in the talk, but as a note, and you saw it earlier too, the Ansible operators are exactly as full featured as the Golang operators. So if you don't have Golang experience, but you need an operator and you know YAML, easily can learn Ansible from there, just Ansible operator all the way, right? So Helm is better for deploying third party software and this is important. When you don't control the code, you don't control the consistency in the operations of that code. So using or creating a new Helm chart and taking care of your operator, operational considerations separately is a better strategy when you don't own the application. So Helm has strategies for rollbacks. So if an installation or upgrade is failing, the test rules, which is like, you know, I've put through my operation or my application, but I've deployed it or upgraded it and then I've had Helm do some post checks to make sure that things are working. If that's failing, you can rollback. It has that functionality built in. Operators don't have that, right? Operators, you would have to kind of go with the rollback by rolling forward. So you make a fix, you patch it, apply it and roll forward. And that's pretty common within the industry. If your team is mature enough that you're at the point where you're looking at an operator, you're probably mature enough to also do that, but it's something to keep in mind. It's, again, it's possible to convert a Helm chart to an operator. So while the operator will remain more limited, it's gonna be around a level two with a couple of level three functions. These are great for when you need to take, when you need a reconciliation loop. And like I said earlier, as much as I don't love working with operators that are below a level four, they have their time and place. And using it as a crawl, walk, run approach is exactly that time and place, right? And lastly, and this is really uncommon, especially because we're talking about Cloud Native most of the time here, K3S support. So I've used K3S for IoT servers. So that built in Helm compatibility for Kubernetes on bare metal is absolutely game changing in that world. And if you might even have to think about going to that world, Helm is going to be your friend. I'll send it to you, Christian, for bad cop. Yeah, I'll be bad cop. I'll, Hillary's leaving the room. I'm entering the room here in the interrogation room. So, so cons, cons to a Helm chart, right? So trailing CRDs are a real issue with Helm, right? So the cleanup of a Helm, a Helm chart isn't perfect, right? It's, there's still some, there's still some bruises and warts there. And often you need like some clever tooling around that, that shortcoming, right? So there's still some, with Helm, there's still the need for that custom scripting that I talked about where operators kind of kind of shine a little bit. Helm is needs that custom scripting and that kind of brings to the second point where you still kind of need to write a level three or above operator to, you know, basically from scratch, right? That's to say that the work you've done is wasted when you've done a Helm chart, but the Helm operator implementation doesn't extend Helm, right? So it's essentially you're wrapping operator-like functionality around Helm. And so it doesn't actually extend anything that you get with Helm. So to get that as digital functionality, you have to kind of wrap that in a go or an Ansible operator. These limitations are due to the limitations of Helm itself, and then those are done with intent, right? They're what kept Helm simple, right? So kind of our good points or our bad points kind of idea is that Helm is intentionally simple and it's intentionally to solve one specific use case, right? Being a package manager for Kubernetes. So anything beyond that, it's gonna take a little work.  Kind of makes it a little smarter on the installs for the CRD issues. But if your application upgrades come with a database schema upgrade, Helm won't handle that for you, right?  That's, you know, people try to compare Helm and operators like, well, they just install software. What's the difference? Like, well, there's a pretty big difference between installing software and managing that software, right? Like a schema change to a database, that's a big event and there needs to be a lot of operational knowledge of that Helm doesn't really do that for you. You know, if you wanna keep from repeating yourself or have kind of a dry code mentality, you might find that the inherent models for sub charts is less than optimal for first, right? At first. So if you're using sub charts or like an umbrella chart where it's basically a Helm chart that references other Helm charts is what a sub chart is. Typically, variables that are defined for the parent chart must also be redefined in the sub chart, right? It's like, it's the inheritance kind of weird with Helm in that aspect. This again is a piece that creates an uncomfortable amount of human error, right?  You may lint the YAML as much as you want but it won't catch spelling mistakes, right? So anytime you introduce the human elements to something that there could be a human error. And so that's kind of what operators try to solve there. Whereas with Helm, more human interactions is needed. And so kind of quick with the Helm cons because I know we're running short on time because what we really want to know is who wins, right? Who wins this battle, what do you do? I think Hillary has big opinions on this, right? She does this every day for a living. I just talked to customers. So I think I'm gonna hand it off to her to kind of talk about who wins here. I'm curious who wins. Let's see. All right, well I have big opinions on everything. Yes, yes, true. We could spend another hour just on my big opinions. But anyway, I've mentioned this before. Helm is a great starting point in your operator journey, right? Helm provides a way to install a workload on cluster. It's easy, it's low barrier to entry. This installation process has the ability to tap into pre and post hooks like I kind of talked about earlier. And that way you can do some sort of like ordering of rollout, sanity checks, et cetera. And Helm also provides you, it's a templating engine. You said this. So you're not storing the same YAMLs over and over and over again. Which makes it a lot easier to just kind of roll out to many different clusters, right? You write some script that replaces some variables and you're good to go, right? Off you run. And the upgrades are smart-ish, right? Like we said, it can double, it can check some things. It has some limitations, but it's still somewhat intelligent, and this is honestly what most people need and want to begin with. So it's where a lot of people start. It's a great starting point. It answers the question, how do I get this workload onto clusters? And then you have room for refinement later. Do you wanna add anything to that, Christian, before I move on? I think really the only thing I wanted to add is the fact that going back to our initial conversation about when I said, I don't know whether I want an operator or a Helm chart. 99% of the time, if you're asking yourself that question, it's gonna be a Helm chart. And it's true for me, and I think you mentioned it before, crawl, walk, run. Just because you start with Helm doesn't mean you can't just move on. So really, that's really the only thing is that you probably want a Helm chart when you ask yourself that question, when you initially start. Yeah, if you don't know your day two operations yet, start with a Helm chart. If you don't control that software, right? And maybe I talked about on the next slide, which I'll have you move to real quick. If you don't control the software, right? People who do control the software could be implementing things like floating tags, such as latest, which means that your reconciliation loop would just automatically pull the latest image of whatever that software is, and you could get a failed upgrade that you can't debug easily. And I've seen that happen. So really, Helm charts is your crawl. It might even be your run, right? There's a lot of ways to make it your run, other tooling, integrations, et cetera, that you can do. And then when you don't own the software, like what I would recommend stack-wise is GitOps, Helm, and Ansible. Put those three things together and go. That's your safest way, you're never accidentally getting something you don't want and can't debug well. Next slide. So how do you know it's time, right? You do control the software, you're thinking about operators, how do you know it's time, right? So you actually can use the Helm operator SDK to automatically convert your Helm charts to some operators, and that gives you the reconciliation loop. Again, it has limitations of Helm that we talked about, but it's the great next step, right? And it'll allow you to take some time to sort of live with your software as an operator so that you can start pushing the continuous improvements. Like I've mentioned, once you start getting to the operator thing, a big reason for that is it drives continuous improvement really effectively. So when you're ready for a level four or a level five operator, you're gonna need to write a new operator basically from scratch. If you want to use Golan, you'll have to do it completely from scratch. If you want to use an Ansible operator, there's actually some guides for how to do it, somewhat automatically, it's a little bit of some shell scripting and a little bit of some copy paste, but you can get there with some ease, right? It's not world-endingly difficult. Ansible operators use JINJA 2 templates. As a Python developer, I really like JINJA 2. I've been using it since before it was part of the standard library. So that dates me a little bit. I'm really comfortable with it, right? You're like a JINJA hipster, right? I was using it before it was cool. I was using it before it was cool. Yeah, Python 3 brought JINJA 2 into the standard libraries along with what, pandas and some other stuff. That's whole digression. And I was like, oh, wow, all these things I've always used are now part of the standard library. Awesome. But also now I feel kind of old because I talk to newer developers on my team who are like, that didn't used to be part of the standard library. No, my friend, it did not. We're almost over time here. And basically the heart of this is going from helm charts to operator, this crawl, walk, run approach, it allows you to do things iteratively. And I spent 11 years of my life in quality assurance. So I love doing things iteratively. And as you get, when you're reaching your level of maturity with building your application and you're starting to understand your application and the operational pieces and you're able to define when you actually have well-defined day two operations, like what are my service level indicators? How do I know if I'm meeting my service level objectives? When you're to that point, you're ready for your operator, right? That's your time. When you know what you want to measure, that's the time. That's when you're ready to move forward. And so like I said before, you can use operators to install other operators. There you can use operators to start install helm charts. And honestly, what I would always recommend when you're looking at operators, when you're looking at helm charts, when you're looking at an ecosystem, when you're looking at mixing and matching, it's really never fully one over the other. It's really use a helm chart to install an operator. Lots of helm charts do that. Use your operators to leverage some helm charts to do some other pieces and then leave everything in the main operator. There's great use cases for that. You can strike a balance, you can mix and match these. So when does one run over the other, right? Like Christian said, 90% of the time, you're probably gonna want a helm chart, at least to start. When you're ready, when you're mature, when you know your day two operations, when you know what you want to measure, when you know what your objectives are, it's time for an operator and you can get there. That's it, I think we're at time. Yeah, I think we're at time. I think, Michael, I don't know if we have some time for questions. I don't know if we have some questions. I could stick around maybe for a couple of minutes. I don't see any questions from the audience. So I guess no questions. Cool, cool. Well, thank you everyone. Thank you, Hilary, for waking up early. Yeah, thank you. Thank you for waking up early. Thank you everyone. Yeah, if you have any other questions, I'm on Slack, Kubernetes Slack, Christian H814 or Twitter, you can hit me up there. If you're a Red Hatter, I'm on Slack as well, so cool. You cannot find me on Twitter. You cannot find me on Open Slack. You can find me on LinkedIn. I respond to messages. I will answer that. I don't have a lot of public social media though, so LinkedIn's pretty much how you reach me or you ask Christian how to reach me. Yeah, that's me and I'll ask Hilary. All right, thank you guys. Thank you so much. Thank you.


Item number SCP-1415 Object Class Safe Special Containment Procedures SCP-1415 is to be placed in a 35x35x35 cm metal wall safe, to be guarded at all times but no less than two personnel. Access is via a simple iron hinge door, locked with a simple keypad opening mechanism. All of one Level 3 personnel, or of Dr. Turgus, is required for removal, in which case Dr. Turgus is to retrieve the object while keeping the password secret. Description SCP-1415 is a fingerless letter gauntlet designed for use with the left hand. The object has a tendency to expand and contract slightly, giving it the appearance that it is breathing. It has been transported to research site 6 without incident, where Dr. Turgus has received permission to experiment with it. The object's anomalous properties come into effect when the wearer grasps any object in such a way that the palm of the gauntlet rests upon it. It then begins a process that takes place over 1 to 3 seconds. The target is converted into a living organism composed of various human tissues, while destroying tissue in the wearer's body, roughly equivalent to 1500th the mass of the converted object. If the wearer dies while wearing the gauntlet, it may still be used until the carcass is totally destroyed. However, when attempting to convert an object with a dead body as the wearer, the converted object will also be dead. Note, as of Test 6, it has been proven that the gauntlet can fit non-human hands. Addendum Test Log Test 1 Subject D-1415-01 A Hispanic BML Target D-1415-02 A Caucasian Male Result D-1415-02 Skin Tone, Eye Color, and Hair Color change to match D-1415-01 D-1415-02 also develop BML reproductive organs, becoming a hermaphrodite. D-1415-02 also inherited D-1415-01's lactose intolerance. D-1415-01 lost roughly half a pound of body fat, with no harmful effects. Note, D-1415-02 and D-1415-01 were once again proven to be genetically identical. D-1415-02 requested to be terminated. Request was granted. Test 2 Subject D-1415-03 A Caucasian Male Target One Car Bulbul S-60 Result All materials made of living tissue. Headlights are of similar tissue to human eyes, but are capable of glowing. Its wheels became short, infant-like legs. The controls of the car were replaced with a structure of enlarged neurons, allowing for a rider to drive. The seats were composed of a tongue-like tissue, and were described as sickenly moist. D-1415-03's entire right arm, and most of its torso was consumed in the process. Note, the car, now designated R-1415-02, has been relocated to the garage at Storage 48, for use with appropriate permission. It does not appear to require food, and can be refueled via a small mouth with a gas valve used to be. It is omnivorous, and can digest anything a human can digest for fuel. Genetic testing has shown it genetically identical to D-1415-03. D-1415-03 was terminated. Test 3 Subject D-1415-04 A Caucasian Female Target One Queen-Sides Bed Fully Made With Frame Result Target was converted in its entirety successfully. Pillows became tongue-like organs that expand and contract, and the blanket became a sheet of skin attached to the foot. The target's legs were jointed, and it could walk, but would only do so if let around. Headboard became a bony structure containing numerous eye-like organs that moved and blinked independently of each other. D-1415-04's right hand was consumed in the process. Note, D-1415-04 has resisted all instructions to sleep in the bed due to complaints that it talked to her and knew her by name. This effect is likely not anomalous and completely psychosomatic. Test 4 Subject D-1415-04 Target 4x4x5m block of stone Result 99% of the stone was transformed into human tissue. D-1415-04 was consumed in her entirety, leaving behind a pool of blood. However, the target immediately settled, fatally crushing one staff member and injuring two others. Note, after approximately three hours, the target began crawling towards the edges of the containment, leaning on the windows. Each staff have attempted to provide nourishment, but the target has not shown an ability to eat. Target hooked up the massive intravenous drip. Termination pending, regardless. Test 5 Subject D-1415-05 An Indian male who scored highly on fitness tests. Target 1 Artificial Heart However, D-1415-05 lashed out in anger at Dr. ███████ and grabbed her lab coat instead. Result Dr. ███████'s lab coat, transformed into human tissue, was a slick interior. It expanded and contracted and was reported to be a suckling feeling that was also described as being incredibly uncomfortable. Electric shocks caused it to contract. Note, Dr. ███████ is not to remove the coat, as it appears to have imprinted on her. Further analysis is required. Test 6 Subject A fully grown cow Target 1 Blender Result Gauntlet changed shape to accommodate the cow's hoof. Blender successfully converted. Its blending mechanism was replaced by a strong jaw that chews up the food. It has a preference for vegetable material and will only chew up meat it's forced to. Most of the cow's torso was consumed in the process. Note, genetic testing has proven an identical in DNA to the cow. However, conversion was at a greatly reduced efficiency.