All right, sorry about that long transition there. Welcome everyone. I am Chris short technical marketing manager at red hats producer of the twitch channel I am joined today by Jason dobe's and Hear something dobe's is that you in the background? I'm currently scrambling so I had twitch Open to just kind of keep an eye on it and I hadn't needed it So all of a sudden they had like yeah, so yeah I have a Apple TV and small TV over here that I use just for monitoring twitch like with the volume at five to make sure that I can barely hear it Yeah Yeah, so a separate monitor Yeah, so I clearly don't know how to use a computer. Yeah, so, you know, it's today We are going to learn how to Make go operators, right? And if you're not familiar with what a operator is Jason is going to tell you because he wrote a whole freaking book on it And I'm downloading the hundred and fifty-five pages of it right now because I can't find it for some reason on my computer Even though I know I've downloaded it at least half a dozen times. All right God, you know, it's always so weird to like start in on this because you don't know what people tuned into previously I mean you and I was it last week you me and Christian did the we did the answer operator last Tuesday Which was wildly successful made the Kubernetes newsletter, right? Like was super super awesome and fun to do But yeah, like that's a it's a little bit of a different beast than a go operator, right? It is It's just more funny to me than anything I can't believe it was last week like it both feels like yesterday and like a month ago like an attorney ago Yeah, yeah, I think that's just the times we're in right now to be honest with you I mean just to be brutally honest with you Everything feels like it's right happening right now and also taking forever. Yes, exactly. No I mean like let's get off topic already because this is how I end up rolling last Thursday night We're sitting at dinner and my wife said something and I sat back. I'm like, we mean Mother's Day is this Sunday Like I thought we had easily another week. Yeah, so after that we had we had a quick meeting with the kids We're like, hey, we have to pull the other Mother's Day in two days. So that was fun Yeah, I actually had a I think like a two weeks notice. So I got lucky on that one compared to I am You know, it's alright. This is actually kind of tangentially work related But I used to deal with our partners guy named Joe over at black dog awesome guy Right. There was a streak where Chris you probably went to this there was a kube con out in Copenhagen and then the following week was Red Hat Summit in San Francisco, right? So of course I flew home and Basically had what I what I jokingly referred to as a 13 hour layover and at home I kept flying out past that and I saw Joe at Kubecon and he's like we and it's only funny if you were there We had this very detailed deep conversation about like we're gonna keep each other in check We're gonna remember it's Mother's Day. We give each other reminders We're gonna set the kids up and it was successful and we both nailed it But I don't talk to him anymore and I plan on reaching out to him and being like Joe You're not in my life anymore and I almost miss Mother's Day this year and you're killing me right now So yeah, I've got I've got the Calendar notifications set up, but sometimes like when you're really busy like in the middle of launching a twitch channel Sometimes the the calendar reminder isn't quite good enough I was just joking and our My team's Twitter slack that I had a hundred ninety six tabs open and I mentioned that to you and you know At least six of those were generating some kind of notification about this twitch stream right now so You know I Yeah too many tabs But I'm here on time right yes So the calendar did did kind of work for me even though I had like four or five different ones open and I got four or five different Notifications, which was annoying, but I'm here So I've become what what what I've called a calendar driven developer Maybe or you know, I'm not even developer count calendar driven engineer or whatever you want to call me. Yeah I like that I think we should coin that instead of get ops like cow ops or something like how I was so we focused off Yeah, here's your deadlines work back from there It's funny too because when you played the intro video I got like the the email saying red hat twitch stream is going live and I'm like I know I'm on it Right before we drive away our audience. Let me at least mention operators. Yes, because I know we'll get off track again so if you're new to operators and and I Mean, there's so many ways to describe this We if you've listened to any red hat marketing for our open shift for and beyond you've likely heard the term operators Red Hat summit. It's all over the place. It was mentioned on the same slide as rel and open shift and Matt Hicks who's now Head of products and technology inside a red hat had operators on there as kind of this third pillar in the analogy He was making and it's a bit of a you know explanation for that analogy for the point being We really have leaned into the marketing behind them Like I said, he's working on partners and for a while it was like boom get everybody right operators. So You know, I like to do a couple different things. Let's demystify them first It's a pot. Just wrap your head around that to begin with Process is a process. Exactly. It's an application. Yes So it's and I point that out because you know, it seems so mystical the way we talk about all the capabilities One of the cooler parts is it's Managed by Kubernetes slash open shift and I'm gonna use the terms interchangeable over the course of this tree But it at the end of the day, it's just an application running And as you're kind of starting to get from the the topic of the stream the actual underlying language and technology Is not necessarily relevant. So since it's a pod and since it's just doing stuff We talked about how we did one based on Ansible that effectively said hey operating needs to do something It's gonna run Ansible for us under the covers Today, we're gonna do one where go is running under the covers And the extra added power that gets us over Ansible at the cost of having to write go code Which is yeah, a little bit of a way to comment there, but there's also Some reason behind that but yeah Yeah, that's more that I'm not the biggest fan of go So I've got to like temper my shots that I take on it One of these actually I would like to come back and do one on the Python library I haven't played a ton with but k o p f. I'm sure they find a way to say that out loud Yeah, it sounds kind of weird when you say it that's the Python based one that lives outside of the SDK, but Let me talk about that for a second before we Get too too far down so Like I said, this the technology ultimately doesn't matter It is a pod that uses certain techniques built into Kubernetes We're gonna see them over the course of the stream but it We're going to Facilitate a lot of this writing by using the operator SDK Operator SDK is in the process of becoming a CNCF project Originated I suppose technically originated for OS and then we bought them. So we came out of red hat But we've donated it and it's how that works. Yeah, we don't give many we don't give any shout out at all It's just ours now. Yeah, I mean To be fair, we've got eaten by a bigger fish, too So it's not I mean, that's that's true. That's that's very true You know, it would be interesting like yeah, because I'm assuming people would IBM talk about operators as if red hat created them but technically core OS created them and Part of the reason why we bought them was because of that. Yeah Yeah, and they're kind of expertise that they brought to in fact my co-author on the book Josh would Came from Coro West and he was kind of integral to the operator stuff very beginning When it was still being designed and things like that So Oh the framework So we're gonna use a tool called the operator SDK today and that's going to Scaffold out a lot of this go code for us and we can poke around and look at it just to see and get an appreciation for How much it gave us for free? because Yeah, one thought on that so I I was I joined Red Hat in 2018 the summer 2018 and I was on the Ansible team Everyone knew I was smart with Kubernetes and Ansible and you know the Ansible operator thing was just coming about So I kind of joined up with them and kind of work with them on the operator framework and I was Before this call today. I went to see wait a minute. What version of operator SDK do I have? Because it has been a while since like I've worried about writing like a good operator Right like demo operators are one thing, but I was on version 0.4 Yeah, the current release is 0.17 17. Yeah. Yeah, so 0.4. Oh, yeah So if if you need to download the framework you can get it from the releases page compile it yourself and everything or Do what I did. I'm on a Mac Yeah, I just use brew install operator SDK with the hyphen and Off it went and then I had to go find all the stuff in my go directory and clear it out But yeah, it's interesting that it's in brew. That's kind of cool It is very cool that it's in brew And I was like shocked almost when I was like, oh that worked. Oh my god. How far behind I was yeah, so Honestly, you can get out of sync with this stuff and lose some of the capabilities that are in here is what I'm trying to like convey Super quick Even if you were on honestly 16 and this sounds kind of hyperbolic, but it's a bit true Things are still changing because at the end of the day make sure you're very clear about listening to us He said 0.4 and we're now at 0.17 So I can't fault them for changing their API But the flip side is the problem with writing a print book about zero dot whatever software is that it Changes like the second I hit submit on the final chapter. We're like we're done and they're like cool. We're changing the API. Yay Arata Yeah, exactly So there's a couple things that have definitely changed in I think the book was written against 13 or 14 Might even been a bit older than that depending on what is we're doing and functionality was it hasn't changed just the API Calls some of them are kind of hit or miss here in there, but we'll get through all of that today Um, so yeah, so it says DK Chris is gonna drive today. He's never done one thing go. So this is gonna be all sorts. Yeah so Actually, I don't think I've ever produced a Stream and driven that like the terminal at the same time. So it's gonna be interesting. Yeah, so We were having this discussion earlier about Desktops and tabs and so forth so long so Jay is of the mind that we should be using more desktops virtual desktops and fewer Tabs and windows and I'm of the mind obviously with 196 tabs open just open another tab you have the RAM so in your honor Jay, I have decided to open desktop number two and Share desktops the virtual desktop number two as opposed to the two physical desktops that I have to the right and left of me So you are now in Congratulations, my first like real virtual desktop thingy. See how nice and clean it is just it is it is cleaner It's very clean, but All my tabs are gone. Let me let me point out to where where I was playing Chris that um, um, I Always make fun of people when they they log in and they've got enough tabs where you can't see the text anymore You just see the icons.  So I suppose technically it's 24 spaces, but I know where everything is and You know the more and more we go down this conversation the more and more I really think an upcoming stream would be interesting to get like five people and have them show their environments because Like how they work on this stuff Yeah, that would be pretty cool. I want to do that for I think it kind of flood my introduction But I'm a developer advocate which is similar to what Chris does just I speak to developers more right that speaks up and so I tend to be very much Focused on that developer interloop and how painful it is to go from I made a change to let me see that change live And like any developer I'm strongly opinionated that my tools are the best and I Say that I know they're not for everyone But I'm always fascinated to see how people work and that's everything from What your prompt looks like like right here you look like you're running a power line deal Colors you got whatever this frame rate thing is in the top right. Yeah. Yeah, and like that kind of stuff I'm fascinated by that So I think a very cool stream would be we get like five from people to hop on and be like Here's how I use Tmux because I'm when I demo a particular I'm like eight different Tmux setups and I'm like tiny little corner here Keep an eye on this for this and stuff like that So I think you would hate me because I don't use Tmux I just I term to the hell out of everything Well, I guess it's technically like a Tmux because you there is a Tmux integration with it, but yeah Yeah, I and it's just fun to see like to learn new things from that like what people into doing But speaking of terminals, I would see maybe increase the size on this by at least one. I'm looking at it and seeing that Yeah, exactly to the to the comment in chat like you can totally pick up and that's what I love You just find out something exists that you didn't realize and like Just just different techniques like I tend to use tree a lot in demos because it gives people a kind of higher level of you Oh, yeah, absolutely saying I'm some kind of genius refining tree But no, but till you see someone do it.  It's kind of nice to be like, where's that file that I need? Oh, yeah It's under the open shift commons thing as opposed to right like all these assets and so forth so on And in fact the first thing I'm gonna have you do once you generate the project scaffold is run a tree because It's not that big but it's gonna give us tonight a really great way of summarizing the general concepts to what Well, so yeah, like I'm going to start off I am I so my workflow every GitHub repo that ever gets touched goes into a Directory and my home directory called repo Why because I tend to lose them if they end up all over the file system So if if I need that repo in someplace else on the file system it gets him linked and let's for some reason I can't Which we're gonna need to in this case because it's got to be in the go directory because of my first of a few going away It's just yeah So I talking environments though. I have a way around that though Okay I have a sim link for the go directory in my repo directory. Oh, there you go. Okay. Oh, yeah, that actually kind of works out Well, yeah, that'll be fun And it's funny because I do till the slash code and I only recently started using a capital C Which is much as that bothers me with more of an older school Linux background. Okay that the newer installations Yeah, everything's in alphabetical order Yeah, so like you install Fedora and you get downloads of the capital D Documents of the capital D and my CD overrides my old school Linux tendencies So now it's become capital C code that everything goes in but simple a promise. Yeah, I mean, but like think about that We both do the exact same thing just slightly differently exactly as as weird as that is to say So, yeah, all right. So is the I forget Fresh my memory the operator SDK is going to create a directory for me, right? Yes. I don't need to create one myself So operator SDK What happens new I get I get the help There we go. Yeah, you know and just to prove to everyone it is there, you know a good version Sorry, which is to say prove to yourself because you just want to send a check. We're not running. Yeah. Yeah Kubernetes version all that fun stuff. Yeah, so like I'm gonna throw us out to chat to let us know in terms of font size If that comes through on twitch well, it's hard for me because I have the theater view open So it's condensed, but if it looks fine Well, I mean I could I can reduce the screen size like on the max You can do all kinds of fun stuff with screen resolutions and everything. So yeah, if you need a bigger font Let me know because I can go way bigger too All right, so operator SDK new now Name your operator You gotta name the dang thing and then, you know, you you can walk fumble your way through this So we're just gonna call it Uh, we're just gonna call it j. No, we'll not use names. What's our what's our theme? We had star wars theme last week Well, I guess it all comes down to what exactly we having our operator to do are we deploying an application? Are we just going to deploy like a single pod just to see it work? um Well, the point a single pod to see it work would actually have to imply that I have a cluster running somewhere That could help. Yes. I mean we could say you can just do mini cube work. I do have mini cube Yeah, let's just use mini cube. Hello tab completion Mini cube do I not have mini cube installed? That'd be funny. I did just switch laptops For the record like not to not to be like a cop out or anything Old laptops over there waiting, you know, it's you know Three or four weeks before I wipe it clean and new laptop has been in place for a week. So yeah All right, so I imagine anyone watching this stream completely understands that. Yeah, like having to migrate your entire laptop is not fun um Even with even with like the mac Time machine right like there's still a number of settings. You can't do or install or anything like that Oh, mini cube is installed. Okay. Okay. It's just not linked. Ah, I can fix that prulink mini cube Uh, so I'll quickly vamp on what mini cube is. Uh, so we are getting something in chat to increase the font size. Um Um Definitely on this. I mean the mini cube one. We're not saying in that long but holy cat. I can't believe you're not blind enough looking at that um Yeah, so to quickly ramp on what mini cube is. Um Mini cube is a way of running kubernetes locally on your machine Very low footprint runs in a vm Starts up extremely fast. Um, and just gets destroyed really fast So you can just stop it or you can full on delete the vm and have it recreate So for a lot of this operator stuff, um I'm pretty quick to just delete the entire mini cube instance because You're gonna see we add some things at the cluster level and they're not Hard or they're not impossible or even difficult to remove but sometimes it's just simpler to just slash and burn And you'll see how kind of quick this starts up Um chris, I would say go at least one more on the zoom size Going nuts. I'm gonna hit it like three four more. It's too small still Um, let's see. Let's let's see what you chat says. Um Burr's general rule of thumb that he just told me yesterday is that go until it's uncomfortable for you to look at but that's Oh, well then in that case I literally like would love to live at this font size somehow See, I have to because like I I love seeing people's interfaces with like that small font in the amount of detail you get But I find that I squint too much and get wicked headaches before I realize I have the headache and then I'm kind of shocked for the day. So that is a good point Like I I really want all the pixels But I really want to be able to read everything Exactly And that's the funny part. I'm like I could buy a bigger monitor, but all I would do is just still okay font Looks good in according to chat. Um at the end of the day, I still need to be able to read it So, you know, like I could jack my resolution up, but I need to increase the font size and I lose that real estate anyway Well, that's why like So I use the fedora box for streaming But like it it really I appreciate the 200 percent like zoom setting as opposed to the way Mac does it with like the the varying degrees of Sizes right like it doesn't correlate to an actual Like yes, it's 16 by 9, but no, it's not an actual real like dimension kind of thing. It's kind of off Yeah, I know exactly what you're saying. Yeah. Yeah, so it's it's it's super weird Uh in my opinion, but I've done it Live before to where I've downsized and it's worked fine. So if I still need to do that, let me know. I'm happy to cool Yeah, it's funny comparing what we're looking at now to like your top menu and like even it almost full screen on mine I'm like, oh my god, like I can vaguely see something that looks like a clock up there. Yeah, that's about it Yeah So, yeah, so it's downloading the vm right now. Uh, looks like you just use it in the default setting. So two cpu Four gigs that should be plenty for what we're doing. Um, but yeah, I assume my laptop doesn't keel over with all the chrome crap Um, you should just do I need chrome open for this? I need some chrome Not all chrome What else can I close? About 150 of them tabs work. Yeah, I don't need spotify your menu is giving me a seizure too My menu what's wrong with my menu in wildly different worlds. I have I love my hover over thingy Did you know that this feature like the hover over feature actually got this person that made it a job at apple Steve jobs was so impressed like the guy actually stopped Steve jobs in the lobby It was like, hey, can I just show you this and pulled out like this like adobe demo And like higher like Steve jobs hired him on the spot Because of this like glance look over thing. Yeah That's cool. Cause typically story here is you get in like an elevator with Steve's jobs and by the time you get to the bottom You're fired Right Yeah Let's see. I don't need telegram. Actually what I do need to do do this on a different box We need to not do this on this box So let's do this on a different box I haven't got just the box Not the streaming rig either. Oh, no, I don't have just the box I'm gonna do it on the streaming rig. It'll be fine. It's got tons of memory Yes, I named things after looney tunes characters in my house I guess I mean we talked about this last time mine are currently My desktops in the house are all currently named after world of warcraft locations That's fun Yeah, and I've uh, it's but it's especially fun is that my wife actually said while you're setting this up This is a It's interesting issue the story but my wife um, it's been working from home like everybody else and at their job, um, she works with People with disabilities and helping them kind of manage their money in the different programs they can belong to So at their job, it's a very low tech company like their it person Is like, you know, that's that's what it comes down to that kind of small company They needed to get they have a pdf of like 14 pages and they need to have someone sign the 13th page and so their process at Their place is to print it out sign the one page scan the entire print back in into a pdf And she started doing that. I'm like, do you not destroy all of the paper and the ink in the house doing this? This is dumb. Yeah, so she sends me the original pdf and the signed pdf And the first time she sends it to me I just download some command line tools manually munch of them It was like three or four commands very simple And then I'm like, okay, we're done and then a couple minutes later. She's like, I got another one So I'm like, okay now I wrote a little script around it. So she sends me them I download them or run my script. I send her back the combined one And then it becomes obvious after that that she's going to continue to ask me this. So I'm like, yes Throw up a little flask application in front of it and now she can go to a website and Upload it and to my machine and it's going to run it and just to download and to her this was like the greatest thing ever Of course, and this all comes back to my machines are named after warcraft locations So she keeps referring to the dalaran program, which is the name of my desktop I'm like, this is so fun. She's no idea where any of this reference comes from but to her This is just saving us thousands of sheets of paper So yeah, naming schemes are always fun. I use star warships for a while I tend to bounce around a variety of strange things. So I'm certainly not about to judge All right, although you got to zoom this one in again too if you if if there's anything worth us seeing Do what? This particular terminal again might want to zoom in if it's worth.  No, I'm sorry. I was just installing many Cube, I figured it was just kind of mechanical stuff my bad So, yeah, uh, yeah, y'all just watched me install many cube on linux. It was not Super hard. There is a dependency on kvm once this thing is done growing As you can see this Terminal is a little slow because this is live streaming. So does this work? Yes, sweet, okay So many cubes start Yeah, this also reminded me that I don't have many cube running right now either in case I have to take over Oh, yeah, look at you For you, the better question would be what version of minicube. Do you have running? I want to say it's actually slightly old. I feel like it's giving me a warning It says minicube 1.10.1 is available. I'm on 1.7.2 So I mean if it's giving you a warning You know It might be time probably is You know what it was? I had it working for a demo I was doing earlier in the week and it was That closed the wrong window you get old enough and you've broken things half a second before a demo before you like I'm gonna ignore that warning to upgrade right now right Pretty sure I just screwed up and closed the wrong window Maybe not Yes, I did not because my tabs are huge Okay So yeah, it's downloading. What is it doing now?  So this might take a second, but minicube is essentially the Fully fledged version of single desktop bare bones kubernetes We also don't need this up yet For any of the early stuff we're doing at the operators. We can absolutely get moving on creating it This actually brings us all back to what are we doing in terms of our example Application do you have like a simple hello world pot simply? I mean I have an image we can play Yeah, we'll just pull an image from someplace like seriously like I don't have an app in mind. I literally like flying by the seat of my pants on this stuff. So yeah Yeah, it's it's easy enough. We'll pick that when we get to it So for now, let's head back to wherever we're going to write this Okay repo So i'm going to do it on the same box as minicube is on obviously because that would be silly to do it on my mac All right Yeah, especially because the first the way we're going to run it is we're not actually going to originally packages an image There's a development way of setting it up. That's way simpler and that's only going to work if you're on the same box unless you Go through some serious Thorning things that we don't want to be dealing with Right, but you know the problem is going to be on this box No operator sdk Yeah, i'm really prepared for this one. All right, how do we download the operator sdk on fedora? Just go for the binary Might be a dnf to be honest Really? Okay Wow, my computer's really slow. I mean the flip side is i'm fairly set up to go if you just want me to ship. No, it's definitely not in there um Oh, it's not dnf. All right nevermind Let me uh, let me get my computer like working ish again. How about that?  Oh, sorry all right, so Now I think it's important that people understand like how to install this stuff too. That is fair. This is from scratch after all This is very true. Yeah, so I'll give you a quick plug then so next week. Um, yeah, please do i'm doing on developers dot red hat dot com Dev nation has been doing a series of what they call master courses on variety of topics um Next week i'm doing two sessions kubernetes operators one and two where It's much more formal than this kind of hanging out with chris and doing stuff We're going to cover some basics of operators some of the underlying kubernetes tech and then the operators course one Which is monday at 10 a.m. Easter um is going to be The go and the i'm sorry the ansible and the helm based operators And then the second section of it, which is going to be wednesday at the same time is going to cover go operators so I think chris is right that this whole you seeing the the kind of sausage being made is definitely useful for this twitch stream um more Kind of hand wavy like hey, these are the prerequisites and here's a kind of horse type spin on it I'm going to be doing next week Let me make sure I know what the link is Yeah, actually that's perfect. Um, yeah the dn. uh dev is the one i've been passing people Yeah, I did and yeah, exactly dn.dev slash master was the link that got me to the master course. Yeah Um, you'll see in there near the bottom. Uh, the kubernetes operators sdk one and two taught by me um, and then actually the uh ones above that so the administrators and operations one and two Are taught by jafar and eric, uh, both guys on uh christus team So that's upcoming content next week. Again, as a course, it's a little more structured than this kind of twitch Let's hack on stuff and see what happens. Um But they're both free. Uh, definitely come hang out with us there. Uh, they're on youtube and Yeah, it should be cool. Um, but like, you know, I bring that up because chris is right I think this whole let's see all of these extra little pieces makes more sense in this kind of twitch stream Uh, those sessions are an hour each Fairly rigidly time boxed And again more what you'd expect from uh, a conference type talk Uh, so, you know, uh, there's the quick intro agenda and then I actually did a dry run and nailed the first section to almost exactly an hour See if I can do that again, but um, never gonna happen again, but yeah, exactly Absolutely best one I ever did Conference calls. Oh man, uh dev one uh run by dyna trace They do one in austria based on their uh headquarters, but they started doing them in us in detroit And I spoke at two of them now and um The first time I spoke at one I they had like the clock in front of me and I nailed it within I had about 50 seconds left When I finished I was like, wow I'll never nail it that closely again No The second time I did it they forgot to start my timer and I didn't tell them so I got a good extra five minutes on it before they And I'm like it could stop them, but now I know I'm gonna get it off track here nice So I'm still trying to get my terminal to the point where it's like I type and words appear But yeah, we'll get there from here There's a little bit of lag on my own terminal. So that's kind of sad. That is kind of weird Yeah, I mean, it's just the fact that I've got so much open and going on and such or maybe it's because I'm using multiple desktops I remember 12 years ago now probably a bit more recent than 10 years The first time I ever logged into an aws instance in singapore And saw the terminal lag It was this really and this is sad that I got into like that far into my career and realized like wow Geographic actually matters But I'm like, yeah, that was unnoticeable. Like I'm sending this across the planet and there's going to be a little bit of a hiccup Brian tennis on my team mentioned I don't remember what twitch streamer into this mosh. I think it was called mosh is a cool thing Yeah, but I don't I still don't think that would help the fact that my cpu is just crushed right now Yeah, slightly different but mosh if you're unfamiliar with it, um to my understanding and to be fair I've never used it, but it's basically a way of almost caching um an ssh terminal for made few times and you're quickly hopping on and off the internet um so um He uh, like for instance, you've got a conference and your ssh to your home machine And then you do you do a conference slam your laptop shut you walk, you know 20 to the next session you open it up again. It kind of retains it Um is listed twice on that page Yeah, the second one is Uh, you're right. That's an error on the page the second listing of it So the one that occurs after the first operators sdk On May 28th. That's two that's section two. So it is going to be more content. Um, the description Kind of sort of loosely alludes to it, but that's absolutely a typo. That's a different section Okay. Yeah, totally right. That is supposed to be a two I want you to put this Heavy sigh. What am I ssh in two? What is the name of this box? Michigan J frog Michigan J from downloads Wow, this is horribly slow like yeah, let's let you take over and let me figure out what the hell's going on here Because I don't want everybody to have a bad experience because my box is just keeling over And more importantly you drive me a little bit bonkers is like I watch on this delay.  Yeah, sorry painful for me. No, it's fine. My bad. I'm sorry Do you not mean to cause physical pain for people? But yeah, I'm like whatever reason sharing my screen was just the thing that pushed it over the edge You know, I mean I say that but watch my my system actually completely shit the bed right now because of this right Oh, apparently had a full screen Good lord, that's really big. All right. This is where I would like to leave my Font as of right now Rough, I'm gonna stop playing with it. Let me know if this is not readable. I'll tell you right now It's not because chris your overlay is um covering the top of my term anymore. Just fixed it. All right, cool There we go at that lag, but I the youtube lag is worse. That's like seriously like 40 seconds well, you know I think there's in an intentional like seven second lag on or 20 second lag on twitch But youtube it is like I don't even know if they have control Yeah, there's like a normal and low latency mode that didn't seem to have that much of an effect And I get why it's there. It's just kind of funny as I'm looking at like staring at the screen and being like Are you people in the past when you catch up? This is gonna be awesome You know what funny is I have no problems now that I'm not sharing my screen Yeah, zoom is a bit of a hog Yeah All right, so I have mini cube running That's cool. We can do a quick excuse me Check to see that it was just fresh. Um as you can tell by the fact that I set it up about 10 minutes ago I have the sdk installed that's up to date. We established this mini cube. That's slightly old. That's fine And font looks good. All right, cool. Thank you for confirming that okay, so um With the go-based operators um You need to have it in your go path now. I don't really want to spend a ton of time teaching go, but No, please don't modern versions of go solve this problem. So And yeah, just know that go path is a thing Yeah You know, I was giving you all sorts of crap, but I'm definitely seeing a slight bit of lag on my system right now as I do this So, you know zoom just forced everybody to update recently and yeah, I I'm surprised that you're seeing lag because we just did this the other day Yeah, exactly. Like this was this was a non-issue the last time I screen shared on either this or the IBM stream, but Right, that's okay. We can work with it. Um, you just see an extra little lag at how badly uh my typos are, but that's fine Um, so we're gonna run through So, yeah, all right, let's run through um the example I'm going to do for this master course It'll probably be a good practice for me. Um, and let me tweak some things from here But I have this set up to be able to Copy in a lot of the code. So you're not going to see Um, a lot of kind of weird munging in terms of package names or variable names or anything like that Um, so we'll do operator stk. We know we want a new operator I'm gonna call it the visitors operator. In fact, I'll pull up a little quickly notes. So I can get this right Okay, so call it the new visitors operator And it's going to generate some stuff for us. And again, this is One of the benefits of using the stk is it's going to give us a fair amount to start with Mm-hmm So like we're joking about uh tree is going to give us um a nice overview of a lot of what's going on um some things to note here, so Docker file remember that operators are ultimately packaged and delivered as pods. Um, which means we're going to need an image Uh, the stk is going to generate us a base docker file now. You can edit it if you need to Um There's not much you would necessarily need to change in here. Um, you know, it's going to take care of Uh, doing your fast majority of everything. Yeah, exactly the the user wonkiness of Okay, I was about to use the phrase user wonkiness of running on open shift But that kind of detracts from the fact that it's ultimately very good that open shift plays with the user ID as a security feature Um, but it's good. Yeah, it's a best practice For running kubernetes not just open shift, right? Fair point. Yeah Totally totally fair. Um, and it's going to use our universal base image. Um, which again you can change off of that If it makes sense for you, um, I'm certainly not going to because it was given to me and it works Works very well actually. Yes. Yeah, exactly. Uh, especially because I know if I change anything on this stream it's going to break uh, we have this deploy directory Um, and let's take a look at some of these files because this warrants going into a little bit So it creates a service account Nothing much to look at in here, but you wouldn't expect there to be a lot in the service account Um, but realize that your operator is doing stuff to the cluster. So we're going to see when we get in the code Uh, the operator is going to get given an authenticated kubernetes client And that's because we need to do things with us. We need to create stuff. We need to look up stuff. We need to update stuff All of that's done through a service account that the operator runs as so, uh, the sdk generates us a default service account cool Wonderful It's going to generate us a role as well. Uh, and the first thing to notice on here is, um, that's really permissive. Um And it's an interesting design decision because well I go i'm kind of curious your take on this so I mean my take as an operations person as someone who's worked on this stuff in the field is that like It is very clear that developers created this code Right like they want this to work for you. Um, and that's no offense to developers. They are trying to make a thing that works So they're not overly concerned with the security of your specific unique and bespoke environment. So Turn on all the switches. Let it do all the things and it'll work on minicube and any other cluster Now when you go to production And someone sees this They're going to say something so be ready for that Because you do not need this much capability for most operators I love that. I love that mental and i'm actually going to steal that entire phrase of That developers wrote that because it's so true It's it's very much like it's great because it works out of box because You know in very rare cases you'll have to add things to here But all the junk we're going to mess with today. Um should work by default. I think routes are in here. They might actually not be Now in fact, we will need to add things for that are open shift specific But that's fine. We won't use them from any cube right now Um, but he's absolutely right that uh a step and the reason I point this out is before you Uh productize this and before you actually ship it You're going to want to go in here otherwise the moment any Reasonable ops person takes a look at it.  Yeah, exactly But again, I like I said, I like that mentality because you think you're exactly right This is written by developers who wanted to take a this is going to work out of the box approach as compared to Starting with absolutely zero and having you run and then have to add something run and then add something so Regardless of how we got to this point. Know that this is a thing you're going to need to address in some capacity yes, um And then we have a role binding which uh binds the role to the surf's account nothing interesting there Uh, and it's going to give us an operator deployment that you don't have to use but again the SDK the benefit is all of this stuff gets stubbed out for us. Yeah, so And coming from the ansible side, right like where Wrangling yaml is a constant thing Right having any advantage in wrangling of the yaml would be great Yeah Totally totally agree. Um, not having to write even more of it. You know, there's there's the running joke of We feel the kubernetes community that we're all yaml developers and you know, yeah Not wrong I'm a yaml engineer. Yeah, exactly yaml ops.  I'm writing it down right now. Yeah, I do kind of dig that. I want business cards with that Um, how amazing can you imagine handing that out at kubekan? They would like get it too Exactly. I should I should just order a box just for kubekan and just leave them out everywhere Um So it's just a basic run of the mill deployment The one thing you're going to want to change in here is once we build the operator as an image and deploy And push it somewhere play docker hub, whatever You're going to need to replace that because this deployment obviously needs to know where to get the image from Otherwise this can stay as is And why is that? Why is Why can we leave that as is for now? Oh, oh because we're not going to be deploying it as an image right away. Um, the other hang on me closed slack Before that becomes a problem because I'm full desktop sharing Um, exactly because we're not deploying it as an image right now Um, okay, so Now I'm going to point out something odd here that there's nothing else in this directory Um, and we are going to actually see stuff get added as part of the future step So for now acknowledge it's just talking about deployment of the operator itself Um, and then there's nothing else. I really want to dig too too much into here some basic go code Controller if you're at all familiar with kubernetes internals controllers are Function at the control plane And that's ultimately what we're writing. So I said before an operator is a pod or buns is a pod the Part of its secret sauce is it's tied into kubernetes as a controller Which means that it can set up something known as a watch which says notify me when resources of this type get changed And the controller is going to get told. Hey, here's a resource change Do something figure out what needs to be changed So if you're at all familiar with the kubernetes terms, you've seen reconciled you've seen declarative you've seen the words desired state versus actual state All of those things that you've learned in kubernetes apply here as well that we're dealing in terms of resources We're reconciling their state against what's requested in the resource and what actually exists So I could probably run this right now I've never tried it and I don't really have any interest in showing you that. Hey, this does nothing. So let's keep moving So the first thing we need to do is add an actual custom resource So custom resources And I can start this while I talk over it A recently added addition to kubernetes and I say added meaning I say recently added meaning it recently came to ga It was in a data form for a couple of iterations Is the ability to define your own resource types? So you have a resource type called pod and deployment and service and all of these built-in resources types Custom resource definitions. Let me create my own so we can now have a kind called jay's application and awesome database and all these other cool examples This is telling us we want to add one of those And they use the term add api, which I actually kind of like because that's how you interact with the operator on behalf of your application Right. It's that it's that first line and your yaml file, right? Exactly. This this is what we're building off of In fact, so I pointed out specifically that deploy only had those four files Now it's going to give us the crd's directory And that's going to contain two things So let's take a look at the custom resource itself. Mostly because chris just kind of pointed to this It's going to specify a kind visitors app I guess I should quickly pull back and say the app we're deploying is a fairly simple three tier application of a front end A back end and a mysql database You go to the front end and it's going to tell the back end. Hey, we have a visitor Here is the client ip set a time stamp and just store it in the database So very simplistic, but it's multiple pods at least gets things a little more interesting than just deploying a single pod But again, if you're sitting there at home following along on your mini cube, you can't do this You can't create a type or you can't create a resource of type visitors app That's what we're defining means as long as part of the operator The other thing worth pointing out and you might have saw it in that file, but I'm going to point out the command line We have this api version So I I described this in the ansible stream and chris and christian both confirmed that this was a good way of describing it api version is a combination namespacing and versioning construct So the namespace for visitor app to keep it from conflicting with other things called visitors app It's going to be just something I made up my own dns entry And then v1 we're going to call this version one of the custom resource definition Then again kind is the kubernetes word for type. So we've defined a new custom resource type What kind called visitors apps that's going to live in that particular api version So the other piece it gives us it being the stk in that generation process is the custom resource definition itself And we're going to want to come back and edit this a bit Right now, uh, there's a lot going on in this page. So I want you to focus on the spec in the status For a resource the spec is effectively its inbound configuration values And then the status stores information about that resource and this exists for all resources. Um The definition here is saying what can go in this spec and what you'll find in this status And you'll notice the type here is simply object, which doesn't really tell you crap We could just pass in values foo and bar and It'll be considered a valid resource even if they don't need anything So we're going to want to update this to give our users some hint over how to use this And again, if we think of this in terms of an api, excuse me one second We think of this in terms of an api that's just for a good api that it's strongly typed instead of just give me some data and Hope you didn't type it incorrectly Okay Um, I guess I should scroll up never mind The other thing this gave us Is uh, some go code So if I drill into Um, this api's package and then ultimately down to my group and my api version You'll see we get this types file And if we look at it again, there's a lot that comes up So for demo purposes, don't let your eyes bounce around too too much. Just kind of stick with me as I point specific things out Um, first thing edit this file. You'll notice a lot of these say do not edit this Um, because the stk is going to generate you this kind of skeleton code That's going to tie into coob and run certain things and the benefit is I don't need to touch most of it Um, this file in particular we're going to actually edit, uh, and we're going to edit to start with the spec here Uh, the spec again, this is going to be inbound parameters. So when I create a resource of This type, these are the different parameters that I'm going to include Now for this application, um The parameters are kind of dumb, but they're demo parameters. So size is going to be number of back end instances to create type open employment Um, so the size is going to be used in the back end creation Uh, if that's how many different instances of our back end service are going to run And then the title is going to be used in the front end and that's what's going to appear on the web page that we create Um, I point this out because It's going to be the operators call how to use these values and the front the end user is not going to be exposed to Hey, I need you to um Five giga swap. Holy crap um It's going to uh, the operator is going to tell you where it's going to figure out where to put these things I don't need to know that size goes in this deployment title goes in this deployment. This is how they map up Uh, and the reason we put comments in there is that we're going to use this in some generation code So it's going to fill in crd for us. Um Sure, I don't mess up the go format in here Same thing with the status. We're going to flush these flush these out into a Something a little more useful than just plain object So all it's going to do is once the resource is created. It's going to stick inside the resource and say hey Okay, let me take a step back once the resource is created The operator is going to update the kubernetes version of this resource to include information on the back end and the front end images That were deployed as part of that process There's potentially more useful statuses out there, but again demo application I can easily just kind of brush any like kind of like why are you doing this and it's a demo application. Just come with me on that So this is what we vetted in this types file. Um, and then this types file It's uh structs. These are the objects that we're going to be using throughout the rest of the go based operator We'll exit that We're back up to our operator root Now whenever you edit that type, we're going to have two commands that we run The first is this kates generation and this is just going to help regenerate some of that skeleton code for the operator that We relies on the structs And then we're going to generate crds. This one's a lot more interesting because it's going to look inside of The types file we just created rip out that information We just added and stuff it into the crd because again crd is what the end user uses And this is another one of those, you know, I said how between writing the book and now things have changed This used to be called generate open api Um, there's another command that changed So if for some reason you have a copy of the book and you're following along pay attention more to this than the book because They're moving stuff around the right version is Exactly we are in dire need of an errata for that or some kind of second merger, but I'm waiting for it to settle down a bit I'm gonna go back into the crd and now we see That spec that I highlighted earlier is a lot beefier Same thing with the status and you can see that it's pulled in some of the information from the comments So I didn't change this top one But I added in this size parameter and it took the comment before it and said, hey, let's use that as the description Which is really cool because I don't have to start manually messing with the crd I could if I want to mess with these required for instance, or if I wanted to flush it out a little bit more I absolutely could Um, but it's nice having that live in code and then get generated to the crd after the fact Okay So there's probably a reason this is a two-step process that I legitimately Don't have a great answer for but all that did was create us The um custom resource The resource types and the resource definition. It didn't actually generate a controller for us Before I move on though, let's do this I'm that's going to crash actually because I haven't deployed the resource type, but this is fine um So you'll notice I try to do a kubectl get on visitors app and it's telling me Hey, I don't have a resource type called that and that makes sense. That's not a standard kube thing That's something that we've created as part of this operator So let's go and create it So I deploy that um crd file that we were just looking at with the flushed out spec and statuses And I set up that watch again, and now it's actually not pitching back an error for us We have these resources. We just don't have any deployed So as an example, let's deploy one and see what happens And the first thing we're going to see is that our validation is kicked in so we define the spec properly And kubernetes can now tell us. Hey, look, you need something called size and you need something called title I'm not even accepting this as a valid resource, which is very cool. We actually want that to be happening Uh, so for demo purposes, we'll keep the size at one just so we're not putting a lot of things on there We try to apply it again And we see our visitors app resource gets created You see down in the get at the bottom the visitors app resource again gets created And that's it. And that's actually how it should be worked We don't have our operator even created much less running But at the end of the day, it's a custom resource inside of kubernetes So we can create them. I can do anything, but it's 100 valid to create it All right, let's get that out of the way And close this off so we can get back to writing it So again, I started to lead into this. Um, it's generated for us the Custom resource and all of the pieces around that What it doesn't have is the controller and that's separate call to the stk Very very similar except instead of at api now it's add controller And again, there's probably a reason why it's split into two I don't know it to be perfectly honest. I can't envision a scenario where Why is what's put in the two the controller and The controller and the resource itself As I say that I may have kind of contradicted myself. So the kafka operator, um, has multiple resources Defined by it. So there's the kafka cluster. There's a couple of other things and then there's a kafka topic And if I create the topic the operator sees that and adds it as a topic inside the existing cluster But it doesn't do anything to coob itself It's possible that they had one controller handled both and you may want to separate them out It's still I don't know if that's even really the case I'm gonna take all of this out of the demo next week because if I can't answer it Right like it's And it's still to an extent and flux how we want to do this, right? So, you know, half, you know, it's half of this is right like it's all in flight And we're not quite sure yet how it's gonna land into final version kind of thing Yep I will once again reiterate that we are looking at 0.x software. So yes may change that in the future A indeed Yeah, but that's fine. Uh, so for now we've generated we've generated our controller And this is where we're going to spend most of our time actually writing Our operator and this is nice because We've taken a bit to get here But what it ultimately boils down to is effectively two commands and then once we had those two commands Just generating the api and the controller Um, then we're working just straight into our business logic The only things you see me type so far are very business logic specific Additions to the go code. Excuse me. That's where we're going to be going from here So Let's start by looking at the visitor's app and actually We'll do this in visual studio because it's going to be a little bit easier Do it here Zoom in a bit Again, yeah, what's up? I said, thank you for zooming. Oh Yeah, so there's a lot going on in here I'm gonna ask you to just kind of try to ignore the chaos and stick with me on the things I'm highlighting because It's not nearly as bad as it looks, especially if you're not a go coder So blah blah blah blah blah to do user and they do add these nice Little hooks in here that say this is an area you probably want to care about So let's talk about a watch this. Um, I said before that, uh The controller is going to run and it's going to get notified by kubernetes when a resource it Cares about for lack of a better word has some kind of change to it And then it's the operator and the controller's job to figure out. Okay What does that mean for the actual situation? So one key thing to keep in mind is that this reconcile method better be idempotent Because if every call into here it just tries to create the application You're gonna end up with this massive stream of pods if you're not checking because it already exists Do I need to create it? So what does idempotent mean? It means that you write code that has the uh, same expected outcome every time And when it doesn't it fails Right like pretty much. Yeah I mean, I'm sure I could dive into the you know definition Real hard and real, you know deep, but that's kind of the gist of it, right? Like if I could I can safely run this code 100 times and it's not going to break anything Exactly And it will and if something is broken it will potentially fix that for me. Right exactly Um ansible so when we did the ansible operator We didn't have to care about it because the module is an ansible itself takes care of that takes care of all that for you Exactly, which is just awesome when you think about it Just being able to just keep banging on a playbook and it's only going to be like hey, these are the pieces I need to change. Yeah We didn't have to focus on it too heavily in the ansible one here We have to be a little bit smarter, but I'll show you guys how to do that Um, so by default it generates a watch on our actual resource type which makes a ton of sense We're obviously going to care about that But there's potentially other watches we want to create now realize that our visitor's app is going to result in the creation of Three separate deployments one for the front end one for the back end one for the database Three separate services one for each of those um and in our case nothing out There's going to be secrets and things like that um, but we're going to want to um Actually listen for those sub resources So for instance if one of our underlying deployments gets changed We don't want that to happen that deployment is owned by the visitor's app So it needs to come back through this controller. It's gotta yeah It's gotta stay contained within the the code base that this thing is running from we want it packaged and Managed correctly To not blow ourselves up Exactly, um, is it apps? Normally that imports for me um I don't know that I might be missing some libraries, but this is fine. We can easily go type it in And then back to core for service And we need to do the import at the top because it doesn't expect to deal with apps Um for some reason the last times I've ran this and as soon as I hit save in visual studio it was um Um Filling that in I'm guessing it's based on the fact that it's missing some of these go tools. So I will take care of that But this is fine. It just means we have to write a little more code in a room But we're good to go. It's highlighting everything is happy now So we're saying anytime one of the the deployments or the services and the important part is owned by our visitor's app changes So we're not going to feel the requests for every single deployment and every single service running right the operator's namespace Just Just version kind for this one. Yes, exactly um That also means that we need to be careful to set this owner type correctly Uh, and that's for a couple of different reasons, but there is You probably never thought about it before not you chris.  I need that guy to be there. I'm going to create a new one It's back. It's back exactly And that ownership is is how that takes place and it's going to be the same thing here that if we Delete a deployment or delete a pod that owner is going to bubble back up to the visitor's app So this controller is going to get notified and it's going to figure out how to resolve things from there Okay, so I point that out because we're going to have to make sure we set that later The other benefit of this is garbage collection.  I'm kind of okay with it Which is okay, you know, let the let the children live I say but if you want to kill all the children on this twitch That's fine.  Yeah Come out the way. I really wanted it to no. Yeah, I didn't Not exactly where I was going with that quite Okay, so more stub code. Um that I don't want to look at just yet Obviously you're on you can totally look through it up when you're writing one The other big area we need to care about is this reconciling method So this is this is crucial. This is the big guy. This is where the vast majority of our logic is going to live um Now reconcile gets called Repeatedly and frequently. Um any time the resource changes reconcile gets called and note It's a single method. It's not a method that says resource added and resource deleted It's not something that carries a delta of what has potentially changed in the resource It is just saying here is the resource as of today Figure it out and it's not even giving us the resource to see here It we actually have some code in here that pulls the resource Out of kubernetes using this client and says okay now. I need to figure out what's changed in here Now this has a couple of implications This is where the ad impotency comes in because again if reconcile is going to be repeatedly called And it's possible it's called even when there's no changes to trigger it Right. We can't be every single pass through here just blanket say i'm going to create three deployments and three services Because then every single pass through here and then boom boom boom we just blow up and crash on the number of deployments That would be bad We can't sit here and take up a lot of resources or time And i'm going to show you in a little bit technique for how we get around that Um, but it's not good for this to block and take up kubernetes resources Um, since they're cheap and since they're meant to be frequent if you know something's going to take a while Tell kubernetes retrigger another reconcile in x amount of time So for instance, we're going to have code in here that starts our database And we don't want to start the the back end until the database is running So instead of waiting for it, we're just going to say kubernetes Give me another reconcile in five seconds, and then i'm going to check to see if it's running And then if not give me another one in five seconds, and then that way we free up those resources while Um cuba and that whole process of standing up the deployment is doing its thing The the the the ansible equivalent for those out there and ansible land that are learning go for the first time is the wait for Uh command and ansible same premise right like you're just waiting for this thing to come up Once it's up do the next thing It is also so much nicer in the ansible world because you're going to see what our wait for method looks like and it's a little annoying Well, uh, I mean just code From the ops world having to dive through code to fix something is annoying anyway, but you know, that's just me No, it's true, and i'm sure and under the covers the actual stuff that runs python That's probably not the prettiest thing in the world. No, I guarantee it's not right like a I'm I'm sure there's some Hacker e going on in there. That's like not super pretty either, but you know if if if you're writing go code you want to make sure that right like What you have in there is necessary And not muck with what we put in there and intentionally if that makes sense because there's a reason behind it. Yeah Uh now it's generating more code here than we need. Uh, this is actually kind of neat because it is a very stub situation of Fetch the visitors app instance and again visitors app instance. This is the custom resource inside of kubernetes So it's using the types struct that we modified earlier and you can actually see on this hover over And it's going to populate this instance based on the client Remember that it gives us an authenticated client based on the service account that we created So this is where permissions come into play because this client is going to say give me all the Deployments so I can see if my visitor's app is deployed and give me all the routes so I can see if our route is created and so on Uh, give me all the things because I have permission to do all the things Exactly. Yeah, and then this is where it's gonna that's where it's gonna crash where if we had started with a Less developer focused model. I've heard that phrase. You can only check pods Well, then this whole thing just caves in on itself real fast real fast Um, so it generates some stuff here that makes for a decent example where it's got some boilerplate code that sees if the pod creates a pod resource definition in kube and then it actually Creates it and so on and that's cool. It's kind of a tutorial. Um, also not what we're going to do here So I'm just gonna straight up delete it Now what we're going to do is takes a reasonable amount of code So I'm going to cheat and copy it over and then we're going to talk through a few pieces of that makes sense Okay, fine I when you see this you're going to understand why you do not know. I totally I I the fact that five files are there already are already know why right? You don't have to explain and I just already kind of intuitively get it Exactly because like just watching me type code poorly is just not going to be fun or copy. Yeah, like yeah, no So all of what we're looking at is explain that the two different Styles of file we're going to see in here back end front end and my sequel their sole purpose is to create the kubernetes objects for us Um, and then it becomes super obvious once we get into the code themselves It's going to look similar to the yaml that we use when creating the based on a manifest Just in go code, which the benefit is you can't really get a better templating engine than a full-blown programming language um Okay, better is a subjective term there. You know, we're flexible go from does very good Yeah, I mean jinn is also super awesome for this kind of thing You know, it's at the end of the day. It's a programming language that you can just do as much or as little as you want here Um, and again demo purposes. I hard code a fair amount that we probably wouldn't want hard coded Um, but this is fine, but it's fine. It's fine. We're going to pull an image out of my Docker hub account. That's going to be the back end service itself. So that's the application itself Um, and then once we dig in this looks kind of sort of like yaml And what we're doing is creating the go representation of these kubernetes operator of resources So we're creating a deployment and we're giving even if you're not a good developer You can really eyeball what's going on here. We're defining the meta for it the spec the replicas Um, if you if you read enough yaml The coop this should be fairly familiar. The syntax is just different. Absolutely Exactly and and syntax is wonky in a way because where's the one i'm thinking of that There's one of these you have to like mess with the int type and things like that, but Ultimately just like the way you do references and things like that just boil down to just go functionality. So it's it's very easy to Start with the yaml and then convert it into this and it's just a little tedious and mechanical to understand That's where this, you know label selector is in the meta package and Container is in the core package and so But all of this should be Understandable as you just kind of eyeball it and then at this point like I said, it's the the tedium of assembling this Um super super important line here this set controller reference um remember how Yep, this is it remember how I mentioned that owner relationship This is the line that says my visitors app instance, which is what v ultimately is and this hover over is really helpful for the demo This deployment that we just created is the new guy that we're about to create. We're about to return So we want that owner child owner owned Um relationship. I need the actual term there. Um Um Parent child, um, I don't I say I know we've like yeah, I know you don't want to say parent child But this might be the one place you have to it might be and honestly It's better because the industry in general has moved away from master slave Which actually is kind of like neat to see the industry just suddenly wake up I I'm very very happy about that to be honest with you because it was never like growing up in the south As a kid like a learning tech. It was never really like cool to say Yeah, and like it's one of those things like I don't I never thought of it in a negative way until I saw someone Point out that we're explicitly moving away from this and then I was like, oh shit. That is really bad I'm very good with this Yeah, I'm glad we're changing it. So yeah owner owned parent child relationship point being This is that association and this is one of the most crucial lines You're gonna have for all the reasons we outlined earlier where uh, we're gonna get the Garbage collection for free from this. We're gonna get that querying functionality. So that way we're pulling up our things Um, nice diversity change exactly like one of those and again like like we're saying like it's funny that it took the industry so long Um, right like it just now happened just recently happened Um, I mean you could also argue though It took forever for the initiative really start leaning into women in tech too Well, they still have it in my opinion, but whatever Yeah We could do but enough of that That's actually too far. It's cool. Yeah, so okay my favorite story about um changes for women in tech though And you probably know this you're decent enough nerd Um, Jurassic park. Have you ever read the book? No, I haven't read the book but I saw the movie like 20 times when I first came out I love the movie. Um, so in the movie the Sister the oldest sister is the hacker and she has that amazingly epic. It's a unique system scene. Yes. Yeah In the book the genders are reversed. It's the boy is the older child and he's the hacker And from my understanding it was very specifically switched for the movie To have that gender representation of no, we have the girl hacker I mean you think about the fact that that came out in like 93. I think yeah, that was yeah I was young like that 13.  It's really cool Okay, so there's been your diversity tangent for this particular twitch Uh, so that's back end my sql and front end looked very very similar We may refer back to them But realize the point of those files is simply an organizational thing. So that way this one isn't a thousand lines long And their goal is to create go object representations of our kubernetes resources Common is the one that does some different things and these are utility methods That again, i'm going to take a quick shot on go here get a little bit wonky to try to share because they're so the way the typing and the Uh importing in go works that it insists on having the stupid directory information directly in here And and it's don't want to go off in that tangent But it's why I have to structure this in such a way that to make this flow smoothly where i'm not starting to mess with package name and this actual reconcile Object and things like that that there's a very specific flow through this application that completely sounds rehearsed because i've done this it doesn't Point is uh this method insured deployment something I wrote very intentionally named this because again that item potency It's not create deployment. It's effectively a Check if it exists and if not created and you know different languages have different ways of saying that Um create if not exist or something like that. I chosen sure I kind of forget where I got it from at this point, but point is um This code is again fairly Chatty because it's go. Um, although when I do this next week for dev nation, those are primarily Java developers They're used to chatty code. Um We go to the client and say hey, can you find this deployment? Um, if we get an error back and if that error says the deployment is not found We're going to make you assume that that means like we need to create it So we go to the client and we tell it all right fine create this this object and set and dip not meaning dependency here Probably poor variable naming on my part. It's just a reference to the deployment itself Um, and then attempt to create it How would you I've never seen deployment abbreviated any other way Yeah, you know, this goes down to me not being terribly Yeah, this goes back to me just Figuring out how this stuff works as a hacking way at it and not properly going back and cleaning some stuff up But to be in mind of defense though, um Some of this stuff came from the controller itself.  Yeah, I will have my standards are tossed to the wind for my own We got Case in any of this stuff. It's fine. Um, that's you're lucky if I lent this And the only reason it's gonna get lint is because that's how go functions and right exactly If it didn't do it automatically wouldn't be that's been a really bad side effect of this role as a developer at Because I spent so much time writing demo apps to that I very easily be like, oh, I could do this, right? Or I could do this fast Right and that tends to win. What's what's the the venn diagram with the three circles where it's like good fast and cheap cheap. Yep Exactly and only like two of them actually work at any given time Exactly. Yeah, the triangle. It's like where are you gonna land? Um point being we try to get this from the client. Hey, it's not found So we're going to create it inside of kubernetes and then the rest of this is more go wonkiness because of its particular error handling mechanism um One of the takeaway here is this return so The return from the reconcile method itself gives you a return a result and as all things can go an error or actually an error And they have different connotations to what they mean if we return an empty result and a Um empty error Then kubernetes says cool reconcile finished. I'm done if we return the result with an error kubernetes is going to say hey Something went wrong. I'm going to recue the reconcile and then there's the third option which is Not in this file. It's in the mysql file. We jump over there now Um, bear with me as I scroll. Oh, you're fine Oh, I lied. I lied. It's not could be in here. Anyway, we'll see it in a second. There's the third option that says, um Recue this reconcile again, but wait x amount of time and that's what we're going to do with the mysql Um, so types being what they were and again me being not a terribly strong go developer I have affected the same type of method that works on a service instead of a deployment And again the same thing on a secret because that's how we're going to store our mysql credentials So i'm glossing over a lot of this file because they all do very similar things and that flow is Get the uh, get the resource if you can't find it and go create it Other go at uh coders will have different ways of doing it very likely objectively better than my approach But the point being this is that item potency and that makes sure it is running in some capacity Okay, so let's stick with that for now. So all of those utility methods and they don't do anything until Uh, we actually get them in the controller and calling them You know that I think about it. I don't have a busy way of copying this in so let's do Uh, oh Now I've to this is the one part of the flow. It's in all of this stuff is in a repository Um, well, I can actually share with people if they want to follow along So That's right. That's my approach um So again copying in a chunk of code that you don't want to watch me type out But i'm going to walk through the important pieces here and it's going to be a lot of calls out to all the methods We just broke down so, um Let's start with the deployment. So we looked at this earlier. This is give me the deployment for mysql We're going to ensure it's actually created. So create if it's not found Uh, and then based on the results if we actually get and this is real This probably should be error But in any case if we get something back, we're going to want to recue it because this means something went wrong if we got an error Uh, and then we get this is mysql up. So create the secret create the deployment create the service and then We don't want to proceed until mysql is running. Um, and again, this is my technique for this There's other ways of doing it, but this isn't also bad or or gross earning way Um, it's in fact better than my original iteration was to just sleep Because you don't want that reconciled request to sleep So here we see um Is mysql up and we say Okay, okay, uh, this simply returns to us. Hey is mysql running or not or is the deployment replicas Set to one. Do we have something there? We're gonna ask something No, I was just I was just gonna say right like this still looks from an amateurish I'm not even amateurish go right or perspective like you know Very easy to read Yeah, that's the one nice part about go is once you get past all this if error is not equal to nil It's fairly readable. Um, and also, you know conceptually Hey, are there replicas the ready replicas? I want to be we have mysql instance running So if it's not running, this is the important part. Um, I mentioned before that we can tell Kubernetes. Hey, I want you to re queue this Reconciled do it again, but we're gonna have it wait five seconds And that's variable you would probably want to test that and tweak it to kind of yes case scenario Um, and you probably want to back off and eventually say all right, you know what this the shit ain't happening Stop Reconciling. Uh, the mysql database is dead If you need to do something else Again, I'm writing demo code not production code. So I'm like this is gonna work. I'm gonna kill the instance Uh point is you don't want to block that reconciled you want to have it re queue the reconciled request And then that's why again, it becomes so important that this function here is not just blank It create the mysql deployment. It's make sure the thing actually exists Let me do the same thing for the back end and then the front end And if we get down to the bottom, cool Give us back our results no error information. So kubernetes is going to be like awesome Your reconciled has finished. Um, I'm done with you for now Well, it's about time Yeah, exactly joking. No, you're so right So, uh, that is the quote unquote writing of a go operator, uh, which is to say copy and paste glue go operator But I wrote the code originally son of phil so to do good I mean, you know, it's no different than uh stack overflow driven development It's true, which is another ops stack overflow ops In addition to our our calendar ops and our yaml ops. Yeah Um, so let's run it. Um So we have our custom resource definition deployed. Um, give me one second before you run it Oh He's got to turn off some Actually, y'all are lucky that my space heater isn't running right now because this is 64 degrees in my office Um, I'm I have no space heater running for y'all because I want you to have good sound quality Yeah, I uh benefits a twitch streamer. I'm like, all right I just got to go turn my acres off because I'm now freezing in here because right jacked it up entirely too high And you can probably see me get a little twitchy and a little shake and like it's really cold in here Okay, so we want to run this operator. Um now we could build it as an image and push it up to a repo And then use the files we showed earlier to deploy the service account and the role in the image And that is ultimately time consuming And I come from a developer background where that interloop is super critical to us and blowing all of that time To see what is ultimately a typo on my part is super annoying Thankfully the operator sdk, which again, uh, clearly written by developers, which I still love as a concept Gives us the ability to run the operator in a process outside of the cluster But it will attach to the cluster and appear as if it's running within it And I just learned this tuesday and I'm very happy to have learned this because before my process was build the operator Create the image Push the image deploy the image Always always always slowed down the process because no matter what there's going to be some latency between you and the repo Or your cluster and the repo there's no matter what some latency the local flag Uh-uh, we're testing it now. We're doing it live right like go and actually so there's a there's a Um tool called odo, which I believe I signed up to eventually do a twitch stream on Um, I'm also doing a dev nation tech talk on that next week as well Which next week is my absolute hell week as I sit here go through three different Dev nation talks um odo is similar in the sense that not related to operators in the slightest, but it's a way of Skipping that build push deploy step for iterating on changes life changing as a developer to kind of get that out of the equation because Um, that's it just rough to run every time. Yeah Excuse me. So we need to have created our custom resource definition We did that earlier as part of that earlier demo of just showing when get created Otherwise, this is going to attempt to watch in a resource that doesn't exist and it's going to go. I don't do that So our controller is running up there and again another benefit of this Local mode is that it's going to um show us the logs directly here So when I inevitably made some kind of typo here and error starts streaming. It's a little bit easier to see We're going to show the Yeah, we'll just show the pods and services. That's fine. So just to keep an eye on things as they're getting created We'll double check what we can remember from earlier that we edited our custom resource So we have our size and our hello twitch example So to outline everything. It's about to happen operator running in the top panel Again, this detached state, but for all intents and purposes. It is a registered controller in kubernetes except not in kubernetes That middle is just to give us a view of all of the things going on that are going to get created as part of this process And now we're going to go and deploy our custom resource Now i'm going to be a little bit anal about these terms because we I've seen people have used them interchangeably and they're very different things custom resource definitions crd That's what creates the type and then the custom resource itself is an actual instantiation or actual instance of that resource Right. So the the definition is the yaml file essentially And the the the custom resource is the thing that is actually running in the cluster, right like I have I too have had uh been like in conversation with someone that didn't understand That concept and that's what was breaking like their mental model around why this wasn't working for them And the second they were like, oh So that's how I do this thing Like it didn't connect for them for some reason like cr and crd Weren't the same thing, right? You're right. And it's you're right about the mental model breaking I think it boils down to the fact that we the term type is not used And i'm not faulting anyone in kube for doing that that the term resource type is spectacularly Overused just across the industry But that's the model people are used to hear especially programmers like right it but How do you break a model you got to be the first one to do it, right? So so it's not type It's kind and the custom resource definition defines a kind and then a resource is an instance of The type defined by the definition is I even an instance of the definition The instance of the kind defined by the definition not hard concepts I mean once you wrap your head around it, you're not going to mess it up again but that initial Inertia getting over is right like that like reading the docs. It doesn't even Right like you read one doc on cr you read another doc on crd's it's you can like lose Some of that context because it like in the docs.  So Exactly So there's definitely a couple of things that I tend to pick on in particular for Just in these things to kind of cycle back and be like, hey, let's let's not make this mistake again So let me bring a browser over and So actually, you know, I kind of jumped ahead there So we have the application running If I go back up You'll see the log messages That have this Rehealing of the recon style. So my sequel is not running sleeping for five seconds. Remember we hard coded that in and there's all sorts of Production hardening you're going to want to do to exponentially back off and do that cleanly. But the point is You see the recue goes through again. So we have this recon style again Once that finished then we got on to deploying the back end and the front end Those let we're left to go concurrently because they don't have a relationship to each other Again, that'll vary based on your application. If you're looking at a really flushed out, you know 50 microservice application You're going to understand what you can do concurrently and what you can't and that's where The operator thing has really taken off because all of this is custom to your application And then you can tweak this down to oh wait, these five steps can happen at the same time But then we need to wait until after that before we can do something else and you can codify all of that knowledge That's awesome So what's this running? I just set it up as a basic node port Inside of kubernetes and every time I hit refresh you're going to see it's going to stamp out a new Entry saying there's there's another visitor The details, uh, don't worry about them What I typically use this example for though is showing the load balancer scaling when i'm doing a general kube demo Where if I have Multiple backend servers it'll show it flip flopping between all the different instances Point being it's written something to the database and not completely crashed And you'll notice hello twitch that was that title that we specified But the important part to remember is we specified that title in the custom resource itself So we didn't say this belongs to the front end We didn't say it has to go in this deployment Our users got that logical view into our application and the operator took care of pushing those pieces where they need to be This means if you're an op person in charge of your operators, which isn't You know too far from a realm of possibility you can actually you know Change the front page of an app potentially right like it's you know, depending upon how the operator is written it's it's a lot of control over the entire life cycle of the application and its deployment It's a little smaller case or lowercase deployment. Yeah, yes, not kubernetes capital deployment Yeah, which is another fun Thing about don't even get me started Don't even be started Um, I mean the same thing with operators like we there was this mandate. Oh just yeah like having to explain Where was I people versus thing? Right? Yeah. Yeah. So no, I was at uh, it was my last trip before all this Covid-19 stuff like got super serious I was at ibm fast start in los vegas and I got uh like looped around what the guy was saying because he kept saying operator operator operator and using operator the the piece of code and operator the human like interchangeably but not like giving me the context that It has switched Right. So, you know, he actually said so the operator deploys the operator and then what does the operator end up doing? And I like he just punched him a lot. I was just like Which operator? Yeah, exactly. What do you what? Well, your operator person is waiting to see if it comes up probably going to get some coffee Uh, the operator is doing all the work on the back side bringing everything into place how you want it So, yeah, I kind of had to like smart-assedly answer the question just to just to prove the point that like Yes, we need a better way to describe these things Absolutely. This came up in my new book. Um, where we every time I refer to operator the code We use capital O capital O. Yeah, it looks kind of wonky and it does but it helps a lot Yeah, um, a lot of our marketing teams They didn't even listen to me but the marketing teams around the book within red hat when they're like, we're pushing content out They would send you like the abstracts and things and I would go through and change all the capital O's and then they would be like hey, look at this idiot who doesn't know how to capitalize things and re lowercase them and I'm like Don't feel like having this argument again Um, but you mentioned life cycle. So let's talk about that. Uh now our code in there and I forgot to point this out Now that I think about it, um In sure I'll handle front end changes. Okay. Um Where does that live so I can show you? Yeah, so let's do it in front end. Okay. Um, we talked about the item potency in the sense of don't recreate Uh these deployments so every time that reconciled triggers don't drop three more deployments out there Um, but we also have to account for changes So we can't just say does the is the deployment out there without saying has any of the configuration changed Because again remember declarative model. We're not being told why the reconciles being triggered So we really do need to check everything on each run and it's not as painful as it sounds So in this case, um, we are in a reconciled request We've created the front end now. We need to see if anything has changed So we're going to look at the existing read we're going to look at the resource that was updated the current state of it inside of kubernetes We're going to Look at the previous one before the update And we're going to compare the two titles because again, remember that title is one of these configuration these I'm sorry custom resource parameters And the operator knows where it applies it matters in the front end and it knows how to Make that change is necessary. Um, so if the title of the new Object and news not even the right word the updated are the most right recent Most recent. Yeah most recent resources probably the best one Um, if that is different than the actual live resource and we dig through its configuration Then we're going to change it But that's all we're going to do and then we're going to tell obviously kubernetes. Hey update this something has changed Nope, though. We're not going to do anything in terms of saying I want you to recreate the deployment or anything like that kube takes over at that point and understands that a deployment has changed I got to do what it is I do when a configuration value change Which pretty much means destroy the pod and redeploy it with the new config So we're going to see that happen in a second. Um, I really need a better example than just a whole bunch of exclamation points But come with me on this. Why not emojis? Why not about?  Hello twitch. Okay, ask you smiley um With an exclamation because I didn't delete that Uh, so I'm editing the live resource Obviously, this is just a normal resource But I'm going to reiterate the fact that I could have edited the yaml and then reapplied it Yeah And I'm vamping a bit because once I hit save here you're going to see some action happen So keep an eye on the top when the operator gets notified of the new reconcile And can you deny in the middle to see the front end actually get updated? and Quick question. Yes. I've noticed this on a few other live streams. Why the exclamation point when you right quit them? Of muscle memory Okay, um, because it further valid. Well, that was yeah, that was a bit um Muscle memory because if you don't edit something If you edit changes, you don't want to save them. It's q exclamation point And I think that I got into this habit of no, I know I'm I'm I do what I want to do Force exit, but it's probably bad because yes, I missed the w Yeah, I love it. You're like, believe me. I'm going someplace with this You type any other letter any other letter and I just forced myself out. I know it's it's it's a bad habit You know, it's funny. So I was asking christian about um on the last time we streamed about why he does two spaces between um Command parameters. So he would actually oh, yeah And he wrote back to me and I I forgot to reread it because I saw it on my phone I forgot to go back and look at it But he did have a rationale that he came to as to why he did that um And again, and this goes back to our conversation an hour and a half It's all preference. Yeah fun seeing how people work because So I guess I should have more context if you weren't at the previous stream where we were on um Christian uh another guy on chris's team Tends to put two spaces between a command and then it's arguments. So it's most visible when he would do something like that Isn't a real command, right?  I don't care um Jesus take the wheel. I'm assuming I saved it All of that said, uh, it's not Jesus take the wheel is vim take the wheel This is exactly who you want driving things. Exactly. Um So, uh, yeah, and actually if you guys come to the master course next week, you'll see on my bio slide I actually make a comment about exiting vim being difficult You'll see what I mean when she see my slide, but it'll be a funny call back to this whole my poor vim practices Um, so things quickly restarted, but you can tell based on the age that Kubernetes ultimately restarted this front-end pod, which is what we expected to have it. Um, we expected to change Yeah, exactly. I'm glad you pointed that. That's a better way of saying it. It's the only thing we changed So it didn't touch the other two pods Um, and it just restarted what was necessary and again that took some discipline on our part as the operator developer to not um Blanket take the latest set of resources and update them every time and cause a refresh Um, you're gonna want to be a little bit smart about it. I mean like chris said this code is not difficult I'm not any kind of crazy go engineer.  I think I'm actually quite proud of that one Uh, that's gonna be my takeaway from the stream Not not the calendar driven development. Not the it's clearly this was it's clear to me that this is written by developer No, this thing If I don't say that next week, I may be bummed at myself because that is an amazing way of describing it So we see that our little eski emoji has gotten in place in other words the edit has uh landed And then to showcase the garbage collection. I'm gonna actually shut down the operator So you'll see that there's no way for the operator to get told.  We'll do the lazy way of doing it and just delete any visitor's app and you'll see all of this stuff starts to get cascaded down So we deleted the visitor's app which owns the deployment and the services So they get garbage collected once the deployment gets garbage collected It starts to clean up the pod as well Um, and then we're gonna be left with the only service here Which was not owned by it because thankfully I'd rather not kill like a nedy service and destroy the minicube plus review Well, you would have been in better shape than I was like a few I mean, I did start it once we started the stream because minicube is just amazing for that like the amount of times when I'm debugging this up because the other thing mine now is Uh, the custom resource is still the definition is still deployed So if I was changing that or completely cleaning up after the fact I would have to delete that Um, to be honest, I'm sure if I do a backward search on minicube You could see I actually do delete and start every time because it's fairly cheap Um, and I'm just like, uh, I mistyped the namespace. Just gotta trash the cluster and start over because it starts up so fast That's cool Bulk I wanted to cover. Yeah, um We could go through and I'd rather not right now just because I already have something named visitor's operator on my quay account, but um The next steps would be actually packaging this for a formal deployment. So it would be a podman build You gotta build it. You gotta tag it. You gotta push it And that's basic image stuff Right, and if and if and if you weren't familiar with the local tag the local flag I should say prior to now You would have been doing this over and over again every time you iterated on the operator But since you have that local tag You don't have to do this all the time. So like I'm super familiar with building Uh tagging and pushing stuff to quay because until last week. I didn't know anything about the local tag But now I do or local flag and now I'm very happy Um, yeah that local thing. Yeah, it's such a time saver And I should point out you can use something like padman or um, oh, that's that's where I'm running into an issue Uh, or docker. Um, there is the build command built into the operator stk just because the docker file is not um In the standard location Um, and you have to pass in image builder. There it is. I'm pointing. Okay. I'll do it Um, so operator stk. I'm gonna build. Here's the name of the image I'm gonna build and then in my case. I'm running fedora 31 which doesn't run docker unless you jump through a bunch of hoops But that's fine because I'd rather run podman anyway The default for this case because it's way lighter weight than oh, yeah Yeah, you're just holding an image not having a demon is amazing. Yeah Um, but there is a flag for that so I can say I want to use podman as the image builder And then that would build me the image and then I would use podman as normal to push the image someplace And then again, we would take that name which I don't mind doing right now And we would edit this operator deployment that we saw again about an hour and a half ago at this point And I didn't copy that And then we can deploy it using this deployment and then it's a pod living inside of our cluster But again, it's going to get wired up to kubernetes in the exact same way So the minute I create a custom resource If we do a k logs on this pod, we will see the exact same output we saw from the local Copy second guessing that exclamation point now I'm glad Yeah, it's it's the one I'm surprised I hasn't bitten you yet because like I've seen you're not the only person I've seen do it First of all, it's happened multiple times. You're probably only the second person. I've been like, hey, by the way It's it's that's what's fun about this kind of because this is a weird version of pair programming the streaming But I'm having a blast with it because on the ibm developer channel just the slash ibm developer So we're being friends. Yeah same same twitch channel or same twitch site different channel. Yeah They do a variety of things and on friday afternoons 2 p.m. Eastern Me and brian tennis from my team meet up with an ibm developer and we just dork around with open shift stuff He's taught me things that Good behaviors, especially when it comes to community and upstream that I typically didn't do so like There's ways and I don't know the formatting out the top of my head, but effectively you can put in a git push um In a git commit message co-authored by and then some information and github will link that back to the other People so wow, okay. Yeah, and he's he's got also and he's got like a template that he had get configured because um To follow the conventions of my short line my 50 character starting message blank line paragraph describing things co-authored by files edited like He really had this flushed out He used the phrase. Yeah, and I like his phrasing. It would be a good open source citizen, which is not yeah No, like that's I wish I like brian had this brian tannis So jj originated jj. Okay, and then he's taught me and brian about it brian has I think he saved the template because he was the one doing the commit When jj is like no, no, we're gonna do this right and I was like, all right. I did Okay Yeah, and like I said it in a playful way, but like Yeah, getting that kind of you know, I'm like, I'm happy 90 of the time when I just sign off on the commit, right? Like If there's a better way to do this that makes me a better community citizen I want to know jj if you're out there. I have an email address. I know you know what it is Oh, man. Yeah, I'd love to get so at some point I'd love to get jj on the stream to kind of hang out and do that cross-promotion these stuff too Well, and it's not just the cross-promotion stuff.  You know, what can we do as people to become better? You know open source community citizens. What can we do to make the the ecosystem? You know friendly or for people to enter is part of what I do like when volunteering in the kubernetes community itself is They're just guiding people helping the process become a little bit more friendly to human beings, right like uh, a lot of the work that You see and kubernetes, uh github Uh like draft issues is Because of very very thoughtful people coming around and saying right like here's the minimum we need and having that Fresh enough memory to say I remember my first time doing this and how intimidating it was How do I file this pr correctly or how do I file this issue correctly? What information do you need? Because otherwise you would have a lot of people coming in and just being saying like this thing doesn't work. Oh, yeah I see this is not working too If you don't ask them for anything that you're probably not going to get anything and A lot of a lot of work has gone into the kubernetes community to make it more open and friendly And then any other community I've ever seen so that's kind of why I blatched onto it so quickly So deeply I guess So, yeah, jay. Thank you so much for bailing me out Uh, I will I will go close some tabs and and windows and things so that mini cube will actually start on my computer Uh without back to this tabs in the end. Yeah, um Actually, you know every friday is uh tab closing day at at the short household every friday afternoon Because if you notice my calendar like if i'm not on the stream, it's pretty open on friday afternoon So it's it's wrapping up all the research just taking all the notes. It's closing all the tabs this friday. I'm off So getting up on our 3000 mailing list that we've right. Yeah, that's kind of stuff.   Yeah All right, I've got a meeting at four. I'm sure you've got something that you would rather be doing Uh, thank you so much for joining me today j You get a ton of fun as always. Yeah, come back again tomorrow. We have what do we have on tap? Oh, actually john willis and I just scheduled, uh Something for tomorrow. Um, I might have put it in the wrong place in the wrong calendar Nope. I'm looking at the wrong calendar Tomorrow Yeah, so diane has something at noon um diane sorry, we should be clear diane is the Owners not the open shift commons. Open shift commons. The community So the open shift community manager diane has, uh, unleash your your clusters ebpf superpowers with cube cuddle gadget Uh, that's from albin croquet or 2k. I it's a it's a foreign name. I'm not good at it But they're from kinfolk and they're good people and that's the open shift commons at noon tomorrow eastern time, which is 1600 uh utc But at 10 a.m. Tomorrow it'll be myself And the 1400 utc it'll be myself and my my friend and mentor john willis talking about Automating governance risk and compliance and we're just going to talk about john willis's He has a model for this kind of instantiated Automation of of like all those really really hard things that auditors put you with So we're going to talk about tomorrow real quick and like hopefully it'll be an ongoing thing that we can continue talking about But yeah, so stay tuned to our twitch. We've got it on and popping We've got the calendar if you scroll down on the screen here on the twitch channel. You've got the calendar live there Uh, i'm not sure how often it updates. Uh, but it looks let me see it looks like it up Doesn't have you at 10. Oh, yeah, it does have you at 10 o'clock. It does have me at 10 Yeah, so if you had refreshed it should be there, but yeah, uh, so yeah, we've got a lot of stuff going on Um tomorrow, so please come back. Check us out Thank you again j and uh, look forward to seeing you all in the future Thank you everyone in chat and we'll talk soon