Well we're here with Casey Crouch who has got a very interesting technique to train bad habits out of his clients and and his loafers I guess. Tell me what do we call this? That's called the hot quit bottle. We started that at the beginning of this past year 2015 and we it started actually right here at the fertility. I had a horse that I showed for a customer it was actually Paul Hansman trained it and it's a customer of mine and he is and actually they own twisting at the Roxy but Joe and Stacy Robinson we we showed the colt here at the fertility and I hot quit in the first go-around. It was a walking hot quit you know I'll go to my grave I did not hot quit the cow but I mean they they got me for it and anyway before it was all said and done Joe had hot quit twice on this horse here at the fertility so we left here going to Batesville Mississippi and we get to Batesville and I have a pretty good run first day and they have an R on one of my runs and it was for a quit and it was good but anyway my wife got a little excited and she's a big part of this I mean the whole deal. So she got a little excited and we had several several questionable quits and about the third day I hot quit one and when I walked down the arena I could see fire blowing out her ears you know I knew she is a little upset so you know everything settles and we I go to work some horses for my amateurs and whatnot and and everybody's kind of laughing you know ha ha and about it out there in the practice pen and here she comes and she says she walks up everybody gets quiet you know and we're sitting there and she says if any one of you hot quit again I'm gonna explode I mean I'm gonna have it out on you you know so so it becomes a big joke you know so then we decide we're gonna have a hot quit bottle that's how this started if you hot quit you got to put five bucks in the jar we haven't decided what we're gonna do with it yet that'll come today sometime but any idea how much in it nope not really but several people have put in several times myself included and has it helped people you know maybe curb or take off some of the pressure of doing something like that yes it's softened the blow a little bit you know I mean we take it very serious I mean it's but but by the same token you got to enjoy it you know you got to enjoy what you're doing and one time in particular which as it as the year went I mean you know if if somebody even questioned it you know they had come out laughing digging in her pocket I mean you know but we were at Drip and Springs Texas and there was a lady there and and she had a pretty nice run and she hot quit and and she's not she's not in my barn but I mean she you know I help her a lot and whatnot and anyway she hot quit a cow and had a really nice run and this is one time that really stands out and you know and she was very disgusted and you know just really upset with herself and as we're walking out of the pen I ride up beside her and I tell her I see only five bucks and she looks at me and she had heard I mean she knew about it but she looks at me and she's not sure what to say at the moment and then about that time she goes to smiling and she said thank you so much I mean it just softened everything so big question is what are you going to do with the money we don't know yet we got to come up with what we're doing maybe a big party it might be a big one from the looks of the bottle that's right you might start a new trend there you go we're into that thank you


We have to spend 24 hours in the ocean as you read by the title. In three, two, one.  Stop! You're trying to get me to live, but it's not going to happen. I think we've been out here for four hours and 20 minutes. That is crazy. Something just hit my foot. What is good, Mom and Dad? Welcome back to our channel. I am currently standing in an ocean right now. We're actually at Sanibel Island, Florida. And this is the ocean that we're dealing with, but Mariah, we have to start at the challenge. Mariah's actually up here getting ready. What are you doing up here, baby? What are you... Oh, you're eating. She's getting ready because we're... Maybe you've stuck. We have to spend 24 hours in the ocean as you read by the title, or if someone fails and leaves the ocean, maybe her. I'm going to just say that. Then you lose. I just want to say this, guys. When me and Bill first started eating, one thing that we had in common was we were a little bit afraid of the ocean. Like, we'll go in it, but sharks don't let stuff freak us out. I watched Jaws one too many times when I was a kid, and I will never go, what, neck deep? I won't go nipple deep. No, we won the jet skis, okay? Yeah. And they told me to jump out in the middle of the ocean. I don't even know how I did it, but I did it, and I freaked out. We all parked our jet skis, like literally a block out in the ocean. There was 25 feet of water underneath us until it hit the ground, and we literally jumped into the water. Like, there could have been sharks around us, and there could be sharks in here right now. Literally, shark at Mariah. I just want you to know, I don't want to scare you, but do you know that shark attacks? No, most shark attacks happen in less than two feet of water. In Sanibel. Do you hear me? Yes, I heard you. In two feet of water. And what sucks even more is that it happens more in Sanibel. Yeah, I know. Sanibel Island is literally known for more shark attacks, and we just got to be a little bit careful out here. So we're about to start the video right now, guys, literally. Before we announce this video, make sure you guys leave a comment down below who do you think is going to win. Do you think it's going to be me? Oh my God, they don't even need to come down. We already know Team Build's going to come through. I can't promise you guys anything just because this is the ocean. She's way more scared of sharks than I am, or the ocean than I am. So I don't know. I might have an advantage, but you never know how this challenge is going to go. What? I should have brought my unicorn. Oh, you should have brought a floaty. Oh my gosh. See? It's okay. The less stuff that you have, the more chances it is for me to win. So that's all that matters, guys. Make sure to subscribe if you have not already. And like this video, guys. That is how we know that you guys love our videos by liking it. Like this video to get another challenge video in the future, maybe. All right, are you ready, babe? Are you ready? You have everything. We're going to put the GoPro on first. Oh yeah, here. This is also going to have a GoPro because we don't want to ruin our vlog camera. All right, babe, there's no turning back. Ready? In three, two, one.  I am in the ocean. Are you in the ocean? Yes, you are. And by the way, guys. It doesn't matter. Nobody ever said I have to be way in there.  As long as you're in the ocean, but look it, see how the tide goes out and then it comes back in. If you are out of the beach when the tide comes back in, you're done. You're done and I win. And yeah, that's how it is. I wanted to be a little bit more in. I wanted to make it so literally every half hour we had to go another foot out. But Mariah is too scared. She would not agree to doing that again. I mean, I will go in, but like. What? It's just scary.  Come on, that at least is a foot. That's almost in shark attack water. Sting it? Don't worry about sting rays. Maybe, I don't know. Guys, we just moved down to Florida a week ago. I really don't know that the sting rays season are. No, that the end of the stick. Come on. Is that almost two feet of water? Be careful. If you see any fins run away. Come on, babe. There's a little bit of fishies right there. Right, where? Oh, those are minnows. I don't know if you guys can see, but there's a little minnow fish here. All right, guys. I just love that this is our forever home. Like it's actually sinking in, especially because we're getting the house all done. It's sinking in. We're moving into. Get it? But guys, it just feels so crazy because literally we're getting our house done. We're literally getting moving in within like 29 days. Yes, we literally are so close and we only live 40 minutes from here. Like this is our home. We literally live on that island over there. We're on a different island actually right now. Look, it's already blue the water. It's awesome. It's more bluer out there because it's deeper, but over here we're good. And I just want to say one more thing before we really, really get into this video, guys. Make sure to watch the video all the way through so you can see the challenges and the whole hardship it takes to stay in the ocean for 24 hours. My feet are about to get hella soggy. And every hour, we're going to have 30 seconds where we can run up on the beach, grab something and then run back because we need water, we need food and the only food and water that we have is in that little bag over there. That's it guys. Yeah, we were getting a little hungry before we got here and there was no parking.  We found a spot and we're out here. We're filming the video. So let's let a little time go by and we're going to chill out here. Let's wait like an hour. Yeah, an hour or two, yeah. Because we got 24 hours to go. Baby, look at the little birdie over there. Right there. Right there. I wish we could feed him something. That's the coolest bird that I saw all day. He looks like a little snack. Guys, wait, come here. I want to talk to him a little quick. He looks like snappy. He looks like a cute little snack. I need to get up guys. I've been getting soaked. I think it's been. So there's a timer on the GoPro. We don't have our phones on us. They're actually in there. And I literally told Mariah for this challenge we are not allowed to have our phones. Because every other challenge, we have a phone and it's just not fair. And it gives you a little more of temptation of wanting to go touch your phone especially because we've been getting a lot of phone calls and stuff. So it's really hard to not be on our phone. What about someone called us about the house and they're like, hey, you can't get the house unless you give me this information right now. They would not say that. You never know. Actually, I think I got an email that you need to go answer.  Yes I didn't, it's actually about the house. No wait, actually yeah you do. Because that escrow money, you gotta go email that to him. Yeah, they say you have to do it within the next hour. I just forgot about that. Oh my God. I literally just forgot about that.   I forgot about that. Show off the camera. The camera's off. Show me, show me that it's off. The camera's off. It's, I see the red dot on me. Guys, I'm gonna try my hardest to get him out of the water. I am, Team Mariah can't fail this even though I may. Do I know why I did not think that was real guys is because she would have told me this before we got in the water if it was that serious. Mariah loves this house more than anything. Yeah I do. She's like handcuffed to the front door. You love it too. No, I love it. I love every square inch about it. But she just. It is my baby. Because it's my first home. You know I gotta, you know it's very sentimental. What? Bill.  Guys, look at this little birdie. Oh my God. I wish we had food Bill. What? I wish we had food to feed him. Babe, we don't even have a lot of food to feed us. He's cute. Listen to me. It's us before the birds. We take them home. They'll find their food. We take them home. Do you think snapping them would be good friends? Can you actually get out of the water? Grab him, grab him, grab him. I'm not getting out of this deck. Baby, I wanna grab him. You're trying to get me out of the water but here, go get him. Okay, I'll film you. This is what happens when your food deprived. Go get him. Go get him. He's used to people. He's nice. Look at, he looks like snap. He's just chilling. Oh. All right, baby. Read how long it's been. Two hours. It's been officially two hours. This is the two hour mark. I'm getting a little hungry. Mariah said that she needed to run up. She wants to use her first. Actually, we only have five times that we can run up there for 30 seconds. Quick sip of water. So she's gonna use her first one right now. I'm gonna time you. Okay? I'm looking on here. How long do I get? 30 seconds from the moment you step on the beach. Ready? Set, go. Go, go, go, go, go. Oh, is she down to 27? 26? 25? Mariah, why is she? Oh, you're getting water. Come on. 21. What is that? Pringles? Guys, she's literally going as fast again. I've never seen her eat this fast or drink that fast. Like, come on, babe. You're literally down to 15. 12, 11, 10, 18, 19, 19, 20, 20, 21. Up, you're out of time. I win. I win. Wait.  Wait, where's my food? We have a whole Pringles container and you only grabbed a few for you to eat. It's not your turn.  But you could have helped me. Okay. I just feel like I've got some Pringles in the sun. I'll, I'll. I knew that nothing. Can I have, can I have one? Can I borrow one? I just want to see it. I just want to see it. Guys, so it's been 30 minutes later and Bill was really teased by the time when I brought my Pringles out here. Kind of awesome. I've been good. It's two and a half hours. I've been fine, babe. Like, I'm not even telling you. You want to go take your 30 second. But I don't need to. You want, no, no, no, no. I feel like you're- Guys, he's lying because he was literally just stressing to me about how hungry and thirsty he is and how he was really jealous that I was eating Pringles in front of him. So don't let him trick you. I have, it's been two and a half hours, okay? I have five turns. I need to use some of them up. Like, why would I starve myself? Okay, no one's saying anything. I'm more thirsty than anything. Yeah. So are you gonna tie me? And also if you guys are wondering why this towel is here, that's kind of like, let me see how the tie goes all the way up to that right there on the towel. Yeah, we can't go past the towel. Yeah, so we can't go past that if we do, you're out. Okay, here, the timer's on there, ready? Go. 29, 28. You better hurry up. 20, 19, 16, 15, 14, 30, 12, 10, 11, 12. You can't control the time like that, baby. You didn't get me water? I got water. You didn't get me a bee stick either? You only got yourself a bee stick? Here, you can have the- You know what? You can have the first sip. Wow, that was so rude. Oh well. That wasn't even nice. I might even say this for later. Look at this. The Pringles you can't put in your pocket. Dude, you're not nice. Waterproof? That's really mean. Why didn't you grab one? That's really hurtful. Wait. I'm gonna add this challenge right here and just go home. End it. Do it. No. No, do it. No, it's fine. I'm just gonna walk away from you. Do it. Wait. I can't believe you didn't get water without me. Okay, here, you can take a sip. You don't want it? The only way you'll be forgiven is if you step past the towel. You're funny. Mariah's never forgives me for anything anyways. She's so hard headed that even if I step past that, if she actually wasn't. I'm gonna go tan if I step past that. Fine, go over there. I don't care. Babe, we have 21 and a half hours left on this beach. What? Look, some food. I'm not losing. Sorry guys. I'm not losing. No, I'm not. It's too early. Look at Mariah.  I don't know what she's talking about, guys.  Like, come on. Look at her. She's making me like she owns the place. She's getting close. What do you mean? The water's barely touching her feet. Mariah, you're lucky your water's in the feet. All right. Your feet's in the water. As you can see, I took my shirt off. It needed to dry because it's been in the water for a while and I did the irresponsible thing and gave her the GoPro. What time? How long has it been? It's been three hours and 30 minutes. Three, 30, wait, 330 or is it like 320 something? 330. What the? We've been out here for three hours and 30 minutes and as you can see, I'm saving my water. I only took like 10 sips out of it so far so it's still doing good. Mariah still hasn't had any water and it's been an hour and a half since she went up there. Water's looking pretty good, right? Maybe you should grab some. I'm fine. I'm chilling. Yeah. It's been an hour and a half. You had not had one sip of water yet.  Look at this girl. All right, so it's, I don't know how long it's been honestly, she has the GoPro so she knows the time. She says, let me get my tan in. She's been standing out here for a good 10, 15 minutes. I don't know what she's doing but I'm gonna try. I'm gonna try to get her to step past the towel guys. I'm gonna say take a picture with me and I'm gonna have her keep moving over to the left. Mariah, come take a picture with me. Come take a picture with me. For memories, a little selfie picture. Come here. Stand to my left. I wanna take a picture with you. No. Just come here. The tie goes to the, to there. I know exactly what you're trying to do. Oh my God. Do you want to know what's funny? I literally said, I just wanted to take one side of my other picture and you literally, I knew you were gonna throw up there. Come here.  Come here. Why are you making me stand way over here past the... That's fine. Just come here. Come here. If you force me out, that means you lose. Go a little to the left. Go. You just try cheating. I'm not a cheater. Mariah's the better team. Okay. That is definitely an opinion. Not facts. Not facts. Come on to my city. Comment down below who you think's gonna win. Team Mariah or Team Bill? No. Guys. Okay, so I know this is a challenge, but in all honesty, we are out here. We're having fun. That's so much fun. But I'm still competing against her, okay? I'm still gonna win. I can't breathe. Don't listen to her. I'm gonna win, guys. But it's such a beautiful day out here. And thank you guys so much for watching this video. Like, we're having so much fun. But uh, is that a shark?  No. Oh my God, don't get me like that, babe. I'm sorry. Seriously, we saw dolphins earlier. They were so cute. Oh my God, we saw literally dolphins by this boat. It was only, what, 100 feet out? Yeah, it was, they were so... Mariah's like, I want a pet dolphin. Okay, you let me down now. How can you get a pet dolphin, Dave? I want a pet dolphin. All right guys, so Bo is actually over there. I just want to talk to you guys.  And, but really, to him, he's gonna think that I ran out and I lost a challenge. So I'm gonna be like, well, we already lost. Just come out. And I'm gonna try to sucker him out and he really, it actually would lose. I wasn't, I was taking my 30 seconds. So that doesn't mean I lost. So we're gonna try to get him to lose. What are you doing? These fish are so cool, guys. I know you can't see it, but they literally like, they're so tiny. What? I'm trying to look for a fish. I'll be right back. I'll be right back. Huh? Wait, I won? Yeah, you did. Guys, four hours it took for me to win. Are you getting out? What? Are you getting out? Step out. What? Step out. Why? Cause you won. Wait, why are you telling me to step out? I wanna go get food. Come on. Come on. You won. Oh, I won?  Walk back out. Walk back out. No. I wanna see you step out and then I will declare myself the winner. No. Do I know why? Why? Cause you were taking a second 30 second, weren't you? Yes, you were! You were so close to stepping out. The only reason I didn't go past that because I was gonna, I was gonna grab my shirt.  He almost walked out. You stopped. You stopped. You stopped. 21 hours to go and I'm feeling great. I'm gonna get a tan in. Let's fight. I pexed. Let's fight. Oh, you wanna fight? Let's fight. You wanna fight. I will never hit a girl.  What if I could? Square up. Let me just tell you, Mariah would be on this sand right now. So I have a plan, guys. Can you look away? Go tan. Look it, she's peeking over here. Mariah, when you were out earlier, did you even drink any more water? So you literally had gone two hours without drinking water. I think we've been out here for four hours and 20 minutes. It's only been like 15, 20 minutes since Mariah pulled that little stunt. But I got my water here, guys. I am going to try the next best thing, which is sabotage. So I'm gonna take my 30 second break right now. We're gonna sabotage. This is not cheating. It's thinking better than the enemy because she's the enemy only for this video, guys. 30 seconds, I'm taking it. All right, 30 seconds. I still haven't eaten my steak stick either, guys. Look at this. It's still in my pocket where I put it like a couple hours ago, but luckily it's waterproof so I don't have to worry about that. Huh? No! She's drinking my water. Okay, you know what? You know what? I don't care. How many seconds do I have left? 15 seconds? Look it, I got another water bottle. Oh, gosh, this is all that she brought. Oops. Oh, shoot. Oh, God. Oh, no. No! Oh, how many seconds do I have? Stop! How many seconds? Stop! Hey, hey! Okay, okay, okay, okay, okay, okay, okay. I'm coming back. Stay back. I'm coming back. Guys, they went out way slower than I thought. What? Babe, sabotage. I hear you. Sabotage. That's not nice. That's really neat. Okay. After I get out, I'm gonna want water. Exactly. You just chugged my water. Where's my water? You chugged half of my water. I didn't chug it, but I took a sip. Okay, well, here, just to be nice, you can have, wait, wait, wait, wait, wait. Wait, here, just to be nice, you can have this water. You can have this water. Here. Oh, you don't want it? Wait, actually think smart because I just dumped out 90% of your water that was in there. Think smart, bubble butt. Come on. Hold on, I need to pour some liquid on that. Mariah, do it again. Do it again. Hey. What are you most scared of out here? Sharks. Sharks? Okay. I'm getting hungry too. Getting hungry? You know what hour we're on? What hour are we at? We're almost, we're at 4.52. Oh my God, five hours? Well, not five hours yet, but four hours and 52 minutes. Probably 53 now, because I have a timer set on the GoPro, but guys, it's literally insane. I am hot. I'm probably burning. I did not put sunscreen on right here. I totally forgot. I only put it on my arms because I had my shirt on. I had my shirt on when we were putting it on. Do you think I got a good tan? I probably, we're probably darker since we started this video already. I'm very pasty because we haven't had time to go out. We lived in Chicago. What do you mean you're paid? That's how Chicagoans are, you know what I'm saying? All right, so I'm going all the way out here. I'm like 30 feet away from Mariah right now. She's way over there. So she's scared of sharks, but another thing that she's terrified of that she didn't even think of is stingrays. Guys, I'm trying to end this video right now. Stingrays, if you did not know, can actually go up to literally only an inch of water. So some stingrays can go up to your toes. Like if you're only toe deep in water, they can get up there. So I'm going to literally yell out that, oh my God, I see a stingray. I'm going to yell it out. People at the beach are going to be hearing me. Look at the beach, the beach is packed. Like there's people everywhere and I'm going to yell out stingray. I'm going to see if Mariah's going to run. I'm going to run all the way up until the towel, but I'm going to stop. Let's see if this works. Wish me luck guys. Okay, I'm actually really feeling it now. My back is like, hecka burning. Is yours burning? Let me see.  Look at it. It's red. I don't know if you can see in the camera. Look at this. Wait, ready? Look at it. You're actually getting burnt out. That's awesome. I cannot believe how much sunshine she got though. That is crazy. Something just hit my foot.  No, no, no.  It's probably nothing. It's probably nothing, babe.  It's probably nothing. Come back. Come back. Just come back. Come back. Stingray, stingray, babe. Oh my God. Oh my God.  Wait a minute. Wait. Can you step down? Wait, hold up. I stepped out too. Wait, does that, wait. Hold up. We didn't actually work, babe. I literally told them that I was going to say that there was a stingray. For real though, I was going to do this for the video, but there was actually a stingray. I saw one. Was there actually? Yeah, no, there's one right there. Where? You might not want to step in though because they can go up to an inch of water. Wait, you actually saw one? Yeah, I would watch out. Did you actually? Yes, I did. That's why I said stingray. It made me think of like saying stingray. Was it a joke though? No, that's what made me think. Because I was going to try to get you out by saying there's a shark, but then I actually saw a stingray and I'm like, oh my God, that'd be perfect. But it went away. Thank God. We ran out. No, it literally ran away. It was like the pointer thing was like probably a foot long too. Oh my God, yeah, I'm done. Wait babe, that means you lost. You're out, you're done. You lost too though because you stepped out the same time as me. Okay, but I think you did it first. Stingray, stingray babe, oh my God. I was right behind you. Babe, you literally, that's five hours down the drain. Babe, do you not realize five hours we were down here? That was really rude. What, no, I did not think that was going to work. Did you actually see like dad's serious Yes, I did not think that was going to work. But literally when I saw you already saying I want to get out, did you forget that they have stingrays here? Did you forget? Yeah, and I asked you about it. Oh wow, I can't, dude, that's so annoying. She was so focused. Five hours, and I could, I'm so sorry. Wait, hold on, let me get the late.  I want to tell you guys the exact time that we're out here. So we were out here for five hours and 16 minutes on the beach in the ocean. That means we only had 19 actually 18 hours, 43 minutes left. But guys, or 44 minutes left. But we were so close, guys. 19 hours, 44 minutes left. I would have been here all night, babe. I literally had so much stuff. I'm so sorry, Timurai, that I let you down. But it's okay, cause Timurai will come back. I honestly thought that you were gonna get me. She almost got me twice. The one with the email stuff. And then the other, when you like faked that you lost, you were doing good. I honestly was like scared. I need to be on full alert, but I'm so glad that worked out because I am tired. And actually, when I was putting on my shirt, the sunburn that I have on my back, I don't know if it's sunburn, but it hurt putting it on. Yeah, I feel like it's, I'm definitely, I can feel the heart. Guys, I hope you enjoyed this video. Before we leave, we have to do our notification shout out. And this shout out goes to Polar Allusions. Shout out to you, thank you for being a part of our channel. Thank you so much for being a part of our channel, guys. We love you. The way to get a notification shout out is you have to be subscribed with notifications on. So you can be the first to comment down below in the beginning of our videos. And that is how we pick them. Guys, thank you so much for watching. And I hope you guys enjoyed today's video because it was a really fun, relaxing day. We kind of just came back free because it has been so crazy for me and Bill to last couple of days that we need the day like this. And it definitely feels so good. My energy is drained. And I am ready for some real food because all I ate was Slim Jim's. And she ate Pringles. I didn't even get Pringles. I've been out here for five hours. I got Slim Jim's. We should have done a little more prepared. We should have, but I think we limited ourselves to make it harder. So hey, for you guys, I hope you enjoyed this video and we will see you in the next video. Until then, peace out you guys.


Today is the day, at least one of the days, I guess. Today is my pre-op appointment for my breast reduction surgery. If you guys didn't watch my first video, kind of explaining like why I'm getting this surgery, you can click the link in the description because that kind of goes through everything. Today we decide how small I wanna go. So currently I'm a size triple D, double D. I don't know if that's very accurate, so I'm gonna ask them to size me. But I wanna go down to like a C, maybe a small C. I don't know. For people that have gotten breast reductions before, they like don't love their results because they think they're still too big and I don't want that to happen to me. So I feel like going smaller would be better just because I have heard that they do grow back, you know, like maybe if you're, if you get pregnant or you're on your period, I don't know. So I'm just gonna ask him like all the myths that I hear online and see if they're true. Also a couple of people on Instagram DMed me with their questions, so I'll have him answer that. Hopefully, I don't know if like he'll be in a rush or anything, so hopefully I'll be able to film. The place I went with was Westlake Dermatology. If you guys are in Austin, you already know who Westlake Dermatology is. I feel like they are so well known. I was honestly shocked by their pricing because I called around to a bunch of different plastic surgeons here in Austin and some of them could have gone up to like $14,000. Westlake Dermatology was honestly one of the more affordable ones, which was very shocking to me because they're Westlake Dermatology. It is 847 a.m. right now and my appointment is at nine, so I think I'm just gonna go in. I might have to do some paperwork, I don't know. So let's do this, I'm so excited. If you're in the process of losing weight and like getting healthy, should you wait until after you get it done? Like how much will it affect your weight loss? And all that stuff. Yeah, so ideally, as well as with breast reduction, also with lower liposuction cases as well, is that you do wanna be at your ideal weight, like of course weight fluctuation, fluctuates all the time, but ultimately that would be the goal because then your breasts can kind of accommodate to that and kind of be at that ideal shape you want them to be.  Yeah, so I mean of course everything that typically functions with your hormones with your breasts after breast reduction or an augmentation, anything like that will still remain intact. And so ultimately, yes they will fluctuate as well with your period or pregnancy and everything. So does that mean like I should go on the smaller side when getting them done because I've heard a lot of people getting them done and they say that like they wish they went smaller like you know it's still too big. Not necessarily, I mean ideally for example, for you like what is your current process right now? Honestly, would you be on a size me, do you? We don't do that unfortunately. I don't know, I think it's like a triple D, double A, I don't know. I want to be like your dream, like C. Is it more so like you're set on a C or are you saying like you want just something to kind of fit your body? Yeah, yeah. I like that I have like in tops, in certain tops like I like that it fills it up but sometimes it's just like too much it's like spillage or like they don't even fit in the cup so like something that like just fits better, you know? Like I like having boobs but it just like doesn't. It's just a lot of boobs. It's just a lot, it's just a little too much. Yeah, I understand that. So I mean, I wouldn't say to just like go for the smaller just because of normal body habits of period and pregnancy and stuff like that. I think really the key is getting it to fit your shape and your body frame and something that you're comfortable with ultimately. Cause then I see things like with your period, like I mean, they might get bigger but just it's not a lot. Yeah, other things and pregnancy wise that is a bigger fluctuation compared to your period but it's not forever to know. It's really just for that time. Okay, going on for pregnancy then, what about like breastfeeding? That was like my mom's biggest like issue concern. Like just like you have to be able to breastfeed. Nowadays, like, because before there was a lot of, especially with like breast augmentation is getting implant, a lot of patients were super stressed about that kind of thing. But now there's the technique of placing it under the muscle has helped make it where people with augmentations can have breastfeeding. And of course with any surgery, we can't 100% guarantee but we try to make sure that you will have that opportunity. So in regards to breast reduction, yes, you should be able to. But as with, like I said, any surgery, we want you to know that there could be a risk that that might not be. Yeah. And I just found out that apparently not even everyone can breastfeed. So like, I didn't know that. I mean, it's of course everyone, like they want to take that initial step too and then now I said, it might just not be in their cards and ultimately have to do bottle feeding. And I mean, there's nothing wrong with that. Right. And then so like a lot of people have been like, oh, I'm just gonna wait till I have kids to do this. And my issue is like, I don't know when that, I'm gonna have kids. It's just like getting later and later. Right. So do a lot of people with breast reductions like come back like 10 years later to get another one or like, what does that? I mean, everyone's so different with it, truly. And I mean, like you said, like people do want to wait and everyone has that timeframe with it. Like with augmentation, getting implants, there's people that about 100% are like, I'm not getting them until I have kids. People can't come back. You can get an implant change. And I didn't want to wait that long. I didn't. And everyone, that's their personal decision. And really I think you're at kind of the age where it's really the optimal time with recovery and everything. And you want to enjoy them too. You want to like have the time to feel like you're not having like this huge weight on your shoulders and like be able to wear the clothes that you want to because you're in the age of 24 to do those things, not just, but it's everyone's personal decision. And for those that do get it earlier, like, I mean, you might come back like 10, 15 years, but also you might be super content with it and go, don't have to come back. Let's go for that. That's okay for sure.  Will they go away like completely if I do all of that?  So do you know what the scar? The anchor. Yeah, so it's an anchor. And a lot of, I've realized with patients who have kind of more darker skin tone, there's not a risk of joining, which is usually what we recommend or like hyperpigmentation of those scars. And we recommend for you, maybe before, starting what's called even tone correcting serum. We have it here in our clinic, starting at just a moisturizer tone. Even tone. Yeah. It also really has seven like melanin blockers that are going to help prevent that hyperpigmentation. Scar care is really, really important, especially with this procedure, just because it's a little bit more of an invisible scar. It's a lot. It's a lot of YouTube videos. Yeah. And a lot, and there are some patients who just don't want to do this procedure because of that scar. But I've seen a lot of patients who do the scar care that we recommend and really it does wonders on them. If there does come a time, typically it's usually like two to three month mark where you feel like they're still like gray, so they're just not settling because everyone scars differently. I know there are people that have like huge scars that just step away. Some take away after a couple months or so. We do have laser treatments for that. Typically we'll usually recommend what's called Fraxel. It's a great procedure really just for overall skin correcting. What is the cost of that? So, and typically for Dr. Chiki Obi-Vee because it's for a procedure he did on you. It usually costs like $50 a session. But how many sessions? We usually recommend like three sessions of it. Yeah, it's not crazy. Okay, I'm gonna do that. No, it's great. I mean, you might not need it, but if there comes a time where you're just like still like, I just want these to kind of go way faster. Do you not recommend, like do you recommend like waiting like that three months or like when you get done like, wait like two months to at least just to first start the scar care. Yeah, process with that. Then I can take it from there. Cool. I feel like when I was doing like research, there wasn't like everyone was so vague about it. You know, so I want to be like super in detail and like those were like the whole process because I've been thinking about getting the surgery for like a year or so, I'm just really excited.  So like I didn't, I don't know what the difference could be, but I'm just like, since I do videography and photography, like I'm always like standing up, holding a big chunky thing and like then my back's hurting, you know, so much. So now I've like realized like, okay, it could be the boobs, you know. Also, tell me if one boob is bigger than the other because I'm very curious. I feel like it is, but it's normal to have that. If you took like 10 women and looked at the breasts, you'd notice that like nine of them probably would have one side if they're gonna be other and more commonly it's the left side. We're getting the right. So let's see if you're kind of in that. I think this is the case. Yeah, your left breast has more volume, kind of sits a little wider on the right side and there's more volume on the bottom, so. Weird. There was a bit of a trade-off with breast reduction surgery in that your new shape will be awesome but also probably some scars. Yeah, I don't know. And there was a typical scar pattern that I have to create so you know about your research so you know how good. I think that scars can be a bit more of a concern for so-called ethnic skin. So some of that, you know, Asian heritage, Hispanic heritage, African American heritage, those patients, I tend to be very obsessive about their scars because they're a bit higher with some scar issues. Cool. Yeah, the downtime from breast reduction surgery is not bad compared to most plastic surgeries. For the areolas, like, do you decide on the size? I typically will make them roughly 34 millimeters. The real name is areola tone, but who call them quicky cutters? Yeah. So we're talking about quicky cutters, I'm taking a bit of it. Yeah.  So we have these areola markers, areola tones, and this is how I determine areola size during breast surgery. In the past, we would use one that's even bigger, but over time, we realized that that wasn't super aesthetically present. Any questions for me? No, I don't think so. Cool. All right, thank you so much. You're welcome. My pre-op went really well. It was pretty quick, honestly. I got all my questions answered. Now we're going to talk about pricing. I know everyone's very curious, but it's kind of like a more controversial thing to ask, I guess. The whole surgery was about $9,900. And what I found was really interesting was that the lift was the most expensive part. So the breast reduction was only like a thousand more. And they explained that the lift would actually remove a little bit of the tissue. So if you only got the lift, you would go down maybe around a cup size. But they said if you wanted to make them even smaller, go ahead and get the reduction while they're in there. So I obviously did, because I definitely want them to be a good bit smaller than what they are now. My first video where I explained why I want this surgery, I also explained why I'm not going through insurance. If you have ever looked into getting a breast reduction and getting it covered by insurance, you would know that it's such a process and it just didn't work out for me. So unfortunately, I am paying it out of pocket, but I already know I'm not gonna regret spending this money on the surgery. So I made a $200 deposit, and then when I went to my pre-op, I made a payment of $7,350. And then the rest of the money you actually have to pay to Austin Surgical, which is the anesthesia people, which was around $2,000. Some other things to keep in mind that might cost extra. I had to get some blood work done and that was around $80. But my mom just told me that I couldn't put that under my insurance. So I think we're gonna like submit the receipt and get that figured out. Once you go to your one week post-op, they give you the option to purchase scar kits and other things that can help you heal faster, extra pain medications, all that kind of stuff. I also had to purchase my own zip up bras. So you just need a bra that zips up in front or buttons in front. I thought the zip up would be easier. So I went ahead and ordered them on Amazon and I can link those below. You basically have to sponge bath like a week after too. So I bought some disposable like sponges that kind of like get soapy when you wet it a little bit. Another thing to note is that I will be having some drains. I don't know if they're giving me two drains or one drain or I've never had drains in my life. So I don't even know what the heck that is, but I have to buy gauze and hydrogen peroxide for that. It's basically there to just remove any excess fluid. I have the paperwork that they gave me and I'm just gonna read through like some pre-op and post-op instructions. So first, no smoking for two weeks prior or two weeks after in case you're a smoker. And then it says, please do not consume alcohol. Ask brand containing products for at least 10 days prior to your surgery and 14 days after your surgery. I was actually curious about this one because I was like, dang, can I not have like one margarita? So like I Googled it and basically it said that it needs all the body's energy to just like focus on healing your body and not to focus on like your hangover or healing your body from like dehydration from alcohol. That's what I kind of got from it. So I don't know how strict like the 10 days prior thing is, but I definitely probably won't be drinking 14 days after. It also says that it could just increase bruising, which obviously I don't want. So they said to shower with antibacterial soap, so dial and then just do not wear any moisturizers, makeups, perfumes, deodorant before your surgery. That's one thing that got me too. I was like, oh my God, I'm literally going to be so nervous.  So I was like, okay. Wear loose fitting dark clothing to your procedure. They recommend button-up shirts and they also told me to wear sneakers, not any like flip-flops or anything like that. For a post-op, you're supposed to keep the incisions dry for 24 hours or two to three weeks.  It is the end of summer and I love to go swimming. So I'm going to have to get all of my swimming needs out before the surgery. They said to leave any tape covering on the incisions for one week, just like have them take it off at your one week post-op. It's as to wear a soft bra with no underwire for at least four weeks after surgery. Avoid strenuous physical or aerobic activity for two weeks. For four weeks, I'm supposed to avoid heavy lifting greater than 15 pounds and I'm free from all exercise restrictions six weeks post-op. I'm supposed to avoid raising my arms over my shoulders for two weeks and I should avoid laying on my stomach for six weeks. It also says to minimize sun exposure to incision areas for six months to a year, which I feel like they're always covered so I feel like I won't have a problem with that. I'm not sure if I mentioned my doctor's name but I'm going to Dr. Chiki Obi at Westlake Dermatology at the downtown office and at first I thought I would have to get the surgery done at like some random surgery center but they actually have the surgery center in their downtown office, which is kind of nice. I guess the next vlog you'll see will be my actual breast reduction vlog and the recovery and all of that. I'll be sure to make a playlist on my YouTube channel in case you are researching getting it done yourself. Feel free to follow me on Instagram where I'll be posting real time updates on my stories and I will see y'all in my next video. Bye.


Welcome to our talk on enabling autonomous teams through policy enforcement. Before we get started, a quick introduction about ourselves. My name is James Alseth, and I'm a security engineer at Ubico, currently focused on cloud infrastructure security. Presenting with me is John Reese, who is a software engineer at Ubico with a lot of experience in Go and Kubernetes. Before we get started, let's go over what we're gonna be discussing today. First, we're gonna start with a brief history of Kubernetes at Ubico, how we got started on our Kubernetes journey and some of the previous gaps in our Kubernetes tech stack. We're then going to discuss how policy helps to address those gaps and enable more autonomous teams. We're definitely gonna talk about the awesome open-source tooling that enables us to enforce these policies. And we're gonna wrap up with discussing our journey thus far, sort of where we are now and where we see ourselves in the future. And of course, there will be some time with them for questions. So let's chat about how Ubico got started on its Kubernetes adoption journey. For us, that started about two years ago with an initiative led by our infrastructure team to standardize the platform that our services run on. Previous to this, we were mostly running on virtual machines and we didn't have too many containerized workloads yet. Like many organizations, we used a managed Kubernetes service to get up and running as fast as possible and to help avoid some of the pain points of cluster setup and cluster management. For us though, probably the first question that pops into our head is how can we ensure that the Kubernetes workloads are configured securely? And what that's really asking is, how do we control changes that are made to the cluster? Well, we started where I think most organizations do, leaning on three things. Authentication, authorization, as well as consistent peer review. Diving into the first of those, we were able to take advantage of the managed Kubernetes offering within that it allowed us to tie into our existing identity provider very easily. So we got up and running with authentication pretty quick. Additionally, it probably comes to no surprise to those of you who are familiar with Ubico, but we also require strong multi-factor authentication with WebAuthn and Ubiquiz. Additionally, we regularly expire the sessions for those with access to infrastructure, requiring re-authentication with multi-factor authentication frequently. Moving on to the role-based authorization. Thankfully, Kubernetes has role-based access control built in, and it has for quite a few versions now. This allows us to tie users, groups, and service accounts to any role. These roles can either be scoped to a namespace or they can be applied cluster-wide. These roles allow for extremely fine-grained permissions, allowing you to specify the exact verbs an actor can use, such as create, update, or delete. What types of resources they can act on, such as a deployment resource, and it can even go as far as restricting it only to specifically named resources. Again, our managed Kubernetes offering made this easier by tying groups from our IDP into this system. And the final tool in our tool belt was peer review. For us, we enforced this using GitHub branch protection rules, ensuring that all changes happened through pull requests. And on each of these pull requests, we required at least one other person to review. For us, most of this peer review work landed on our infrastructure team, because they had the most experience to Kubernetes and had spent the most time to learn about all of the best practices, security, or otherwise. About peer review, though, it definitely has some drawbacks when it's the only way that you're restricting changes to your clusters. For one, it needs to be consistent in order to be effective, and when you have consistent review, that's a significant time investment for the reviewers. Because of this, it often bottlenecks on the team or individual that has the most experience with the technology, which of course in this case is Kubernetes. And all of this adds up to slowing down the release cycle. And this is important, because when teams hit too much friction, they often start to work around your processes, whether you know it or not. This is of course bad for security because you no longer have control over these configurations, but they're also bad for just general cluster consistency and maintainability. None of this was really a surprise to us. We kind of saw this coming from a mile away, but that didn't make the problem any less real when we had to deal with it. So what we did is we spent some time researching what other organizations were doing and what the Kubernetes community was doing. And for us, the answer became abundantly clear. Policy was the way forward. When people think of policy, it's usually a negative reaction as they imagine having more hoops to jump through in order to get their work done. However, in our case, since all of these changes to Kubernetes happen through the API server, working with structured JSON data, we can automate the enforcement of these policies entirely. But what do we mean when we say that in this context? Well, it allows us to enforce what we actually care about. For example, we don't really care that a services team is deploying a new version of their service. That's a part of their core job function. However, we do care that when they do that, the resources are configured securely. For example, we probably want to ensure that the workloads aren't running as root and that they don't have any extra Linux capabilities attached to them. These policies can also easily extend past security related settings though. For example, we can require each resource in a namespace to have a certain label set that identifies the owner of that resource. That makes it easy. So when you're working on troubleshooting an issue or you just need to know who owns that resource, it's right there in the metadata of the resource. With that, I'd like to turn it over to John to discuss the tooling we've selected to enforce these policies. Thanks, James. So as James mentioned, we knew we wanted to use policy to solve a lot of the problems we were having at Ubico. We looked at a lot of the tools out there that solved this problem, but OPA was the clear winner in this space. We saw a lot of adoption with other tools that we knew we wanted to leverage and it came with its own policy language. But before getting into OPA itself, it's really important to understand what a policy is, what it looks like, and Rego itself. So Rego is the policy language that OPA knows, that understands, and on my screen here, you can see a policy for a Kubernetes manifest that says that it must have a owner label on it, specifically in namespaces must have an owner label. So when a request comes in in the cluster for a namespace creation, this policy will first check to see this input that's coming in, this input document, is it a type namespace? And if it doesn't have an owner's label on it, return a message that says namespaces must have an owner. So the user, the individual trying to deploy this namespace knows how to fix it. And so the important takeaway here is the input keyword. The input keyword denotes a input document for Rego and the input document is just the structured data. In our case, it's a YAML file. So anything beyond the input dot should look really familiar. We see kind, we see metadata, but the input dot and anything after that is just dependent upon the data that you give it. It could be terraform, it could be a Docker file, any sort of structured data. So again, very generic language, and extremely, immensely powerful. And so now that we have this Rego file, we need to wait to actually determine if the document that we give it would be in violation. And there's a few ways to do this, right? We could read the input document, the Rego to James and he can verify on a case-by-case basis whether or not the document is violated or not, but we're all about automation. So as briefly talked about before, we decided to go with the Open Policy Agent and just look at the logo, of course we did. There's no reason not to choose OPA, it's a work of art. But no, really, the community is great. Again, there's so much adoption around OPA. It's been a real joy to leverage OPA working in their Slack channel. Everyone's super friendly, there's always someone there to help you out. It was a really good choice for us. And so how this works, how OPA works as a service, you deploy it somewhere, be it Kubernetes, be it a web server, wherever you want to put it as long as you can get a web request to it. And then you also include your Rego files with that deployment so OPA can know which policies you want it to enforce. And so you submit an input document, again, be it a Kubernetes manifest, be it a Docker file, anything you want, give it to the OPA service, it will validate the document, run through the policies and tell you does this document violate any of the policies that you have loaded into it. And there's also the really nice piece of the fact that it can also take external data. So you can see here in the bottom right, the data document, that's external data. That can come from any number of sources. So if we build on the previous example of the policy where that teams must have a namespace label, like all namespaces must have a label of owner on it, we could also add a policy that says that owners, teams can only own a single namespace. And so in order to do that, we would need some form of external data, in this case, a count of how many namespaces that they've already created. And so when we create namespaces, when we delete namespaces, we can keep track of that number, give it to OPA and then OPA can use that when it's evaluating all of its policies. And so while that was an example of using OPA as a service, you can actually use OPA as a library, which makes it so much easier to enforce policies. A lot of tools out there that will actually take the OPA engine, import it as a dependency, and then run the same checks that OPA itself would. And so we really wanted to leverage this type of functionality in order to shift our policy enforcement to the left, because we quickly realized that when we deployed OPA, while we were able to have this policy enforcement, this policy validation, our engineers didn't really know whether or not the code they were writing, the policies and manifest they were writing would be in violation until they actually deployed it. So we really wanted a solution that we could give them, put on their local machine and they could just run it without having to worry about any of that. It had silly networking stuff. And so we found a tool called Comtess.  Like OPA, you can, but Comtess just makes the experience so much better. So like when I said before, you can give any input document to OPA. It doesn't really care. It's like, it's half true. It still doesn't care, but as long as it's YAML or JSON. So Comtess can actually take a lot of different file formats. You can see here INI, TOML, HCL, and it can convert those file formats into JSON and then shepherd it into OPA so that you can actually use any format with OPA. There was also some nicety about being able to take in files from different folders anywhere on your machine and then print the result again in a user-friendly way. So Comtess is for local policy validation. It's for pipeline validation in all of your manifest and then anything beyond that would be like OPA deployed as a service for continue enforcement. And so you can see here, this is another example of a policy file on the left. This time we're looking at deployments to make sure they're not running as root and that the container has app label for POM selectors. And on the right we have our input document, our deploy.yaml itself. And so if we were actually run this through Comtess using the test command, all you have to do is pass in the policy.rego as well as the deploy.yaml and then it will take that deploy.yaml and see does that input document violate any of the policies that we have set forth. In this case, there are two policies that were violated. And we can take this policy and we can apply it to the OPA as a service and get the exact same behavior. So we're checking both the local environment as well as the production environment. And then again, this can be run local machines, pipelines and you can get that immediate feedback. There's also the benefit with Comtess to be able to share policies across the organization because it's a fair use case, a valid use case that your policies are managed and written by a completely separate team, a security team, much like they are done at Ubico. We'll have a lot of different repositories. We want to make sure that our policies are being enforced and there's really no way to get those policies without trying them out in the cluster. But with Comtess, you can actually push policies, policy bundles to a OCI compliant registry and then pull them down for later use. So you can see here in the example where we're pulling down a bundle of cluster policies and then we're actually running the Comtess test command locally on that bundle that we just pulled on that same deploy.yaml and then the result is the same. So this is a way to be able to push policies out there and then pull them down for cross team usage. This is really huge as well in pipelines or other kind of approaches where you need to bundle together a lot of different policies. And so Artifact Hub is an attempt to be able to expose a lot of different policy bundles because more or less the policies that we really care about, things like containers having resource constraints, containers not running as root, those types of policies should more or less be the same. But we're kind of in a state now where teams, companies, organizations are all writing the same policies. We don't have a good distribution mechanism and Artifact Hub is again the solution for that. It's a currently it's a sandbox project. There are a whole lot of policy bundles out there, I think there's like one or two. But I would definitely keep an eye on this, contribute where you can. If you do have a public bundle that you want to contribute, I recommend pushing it up there and let's try to make this successful though. So we can actually start sharing policies with one another not having to rewrite them over and over and over again for every team who wants to be able to use policy. And so a lot of the talks that we've, a lot of the conversations we've had is all about policy enforcement on the local environments, but we also need to make sure that the policies are enforced in production. And so once you get beyond the local environment, we decided to go with a tool called Gatekeeper. And Gatekeeper lets us enforce these policies continuously inside of a Kubernetes cluster. It's a mission controller. So when you deploy a resource to Kubernetes, it'll go through the admission controller, look at all the policies that you have loaded. And if any of the resources that you are attempting to add to the cluster, it will reject or let the policy go through. And it also has some audit functionality saying it'll continually audit your cluster. Are there any resources that are violating policy if in the case that you're just starting to adopt Gatekeeper and you want to see if I were to actually enforce this policy, how many resources are kind of out of band or just in the case that Gatekeeper went down for a little bit, you know, want to make sure that there wasn't a resource that got in like during that small window. And again, the huge win here is using the same policies. So if you're checking on your local versus in production, we can always continuously use the same policies that doesn't change from environment to environment. But with that said, it's almost the same policies. So we go back to the input document. When you're working with YAML, the input document is going to be a YAML file. So we would expect input.kind, input.metadata, just like we saw before, but when we're in the context of admission control, it's gonna be a little different.  This is the document that is gonna be received from OPA. So again, it's not quite the same. You would actually have to write a policy that had input.review.object in it, but that would only work for Gatekeeper. And if you did just input.kind, that wouldn't work for Gatekeeper. So what we ended up doing was really adopting this idea of rego being the source of truth for your policies because it really is. Rego is generic. It doesn't matter the context that you're in. It's really all about does this input document that you're giving me violate this policy that you have defined? And so in everything that's based off of your rego should adjust the changes of your rego and not the other way around. And so to solve this problem, we actually wrote a tool called constraint and constraint brings to the table three really important factors. First and foremost, it actually does provide a library where you can write policies that work with both CompTest and Gatekeeper. It's really just a wrapper that normalizes a polyfill that will say if you're in the context of Gatekeeper, then spit out your policies for input.review.object. If you're in the context of CompTest, it's just input. And that handles all of that for you. So your policies can be completely unchanged no matter what environment that you're running in. The other benefit is the template and constraint creation and management. So when you saw on the previous screen, the rego there was actually embedded into the constraint template, the YAML. And that's because that's how Gatekeeper is able to load in your policies. It's just done through a YAML file. And so you're gonna have a rego file on disk. And if you were to change that rego file, you would also have to copy and paste your changes into that YAML, which isn't the most ideal situation. And so what constraint will actually do is it will look at all of your rego files and then it will actually generate the template and constraint for you. So you never have to touch YAML. You're only focused purely on your rego file. And then lastly, it actually will generate documentation for your policies. We really wanted to give our engineers the ability to see what policies were being enforced as well as how they can resolve them if they ever run into a policy violation. And so when it comes to the policies themselves, it'll look relatively familiar. The biggest difference here is definitely the comment header. We added some metadata to the policy in the form of a header comment where we say title is the title of the policy. Here we're saying images must not use the latest tag. And then why this policy exists or really any other flavor that you want to give it to give this policy be it a description or anything else. The enforcement type here is saying deny. The alternative is dry run in case you just want to test out this policy in your cluster and not actually do any sort of enforcement. And then which kinds, which Kubernetes resources that this policy will be enforced on be it just ingresses, be it just namespaces, workloads, et cetera. So we define here a list of resources that this policy is enforced on. And then we use this metadata to generate all the other YAMLs that we were previously talking about. And in the violation itself, you see here we're importing two libraries which is the constraint library. The important note here is that pods.container, our pods library will actually look at containers from any possible source because pods can come from Cron jobs, they can come from deployments, staple sets, daemon sets. There's a large list of those and they're all embedded in Kubernetes resources differently. So pods handles that for you. And then we can take the resulting container that comes out of that does that container have an image of latest? And if it does, giving note to the user is saying that images must not use the latest tag. And again, we have two commands for that, one to generate the template and the constraints and then one for the markdown describing our policies. And then this is an example of what the documentation looks like. You can see here the ID P1001. We also embedded something into the tool that you can actually assign an ID to a policy so you can refer back to it instead of just using a title. The severity is coming from the severity of the rule because in Rego you can have warning, you can have deny, you can have violation. And based on that rule, different things will happen. So we pull the severity out of the Rego and put that in the document for you. Same with the resources that it impacts, the description that dictates what this policy does or why we have it. And then this is the Rego itself. If you want that information, get completely configurable. If you change your Rego, again, all the documentation is regenerated. None of this is hand typed. And then you also are given a link to the source and that source can either be a relative URL if the policy lives in the same location of the source, the same repository or if it's a remote repository, the source can also be a URL. So we found a lot of usage out of this tool. We use it for almost every day. It's really ingrained into our workflows when it comes to gatekeeper and policy. So if you're interested, if it sounds something like you'd wanna use, always open to talk about it. Contributions, always welcome. And that's really all I have for the tooling piece of it. James is gonna kind of go into deeper detail of how we actually have leveraged these tools in our policies and in our pipelines and processes. Thanks, John. Now that we have an understanding of how the tooling works together, let's dive into Ubico's policy journey. In the beginning, since we didn't have too many workloads on Kubernetes yet and we were sure that we had consistent peer review happening, we started with a simple plan because we didn't anticipate too many resources would violate these policies. We would start with writing the policies and their tests, after which we would engage with our services teams to add these checks to their CI flows using Conf test. Simultaneous to that, we would be working on deploying gatekeeper to our clusters in audit only mode. And then finally, after we had worked with the teams to remediate all of the identified issues, we would flip the switch and move gatekeeper into enforcement mode. As you might have expected from how I framed the previous slide, this plan ran into some issues. We made it as far as writing and testing the policies, but when we moved to engage with some select teams to add Conf tests to their CI flows, the issues became pretty readily apparent. Well, for one, there were more violations than anticipated, which meant that there was a potentially a very long window between when we identified these resources and when we would actually be able to apply remediation across all of our clusters. This was compounded by the fact that we were all, not only migrating existing workloads to Kubernetes, but we were in a growth period hiring and starting up some new services on Kubernetes as well. Additionally, since the policies were always essentially production, there was no way to safely test new policies or changes to existing policies. This meant that in a CI flow with Conf test, the CI flow would fail immediately if a violation was found. And similarly, Gatekeeper would just reject changes to the cluster if it didn't meet policy. Additionally, adding Conf test to our CI flows wasn't as easy as it could have been. For one, it kind of required our services teams to know the Conf test flags and sort of how it worked. And additionally, these, it took multiple steps to actually get policies from a remote source into the repo and then run the test. But most importantly, the Conf test results weren't surface to the teams working on the resources. This meant that unless there was an actual violation, no one would even know what tests were run or anything like that. This was especially important to us because we do have some policies that we have labeled as just warnings, which aren't blockers for deployment, but they are a way that we want to communicate to our teams, that the way that they have the resources configured might not be best practices, practice. Finally, policy admins such as myself had no visibility into the test results. So we didn't really have any way to track which repositories or which teams had started using the policies or any of the results for which policies were making the test runs fail or which ones for just emitting warnings. With these pain points identified, we determined that there was essentially two things that we needed to do. We needed to build a policy pipeline and that pipeline would ensure that the policies are safe to enforce throughout our clusters and we had to make the policy adoption as easy as possible. If either of these weren't true, it was a pretty good chance that our policies wouldn't be adopted or it would just be a long uphill battle trying to get our teams to adopt them. So first, let's focus on what we did to make that policy adoption as easy as possible. And to do that, we created two GitHub actions. The first is a wrapper around Conf test itself and it addresses some of the pain points from the previous slides. It automatically pulls the latest policies from a remote source. It surfaces the policy violation warnings and violations and warnings into the pull request comments so that the teams working on the resources can see the test results. And it submits the results to a remote server so policy admins can monitor the deployments. The second address is an issue that we learned about later was that some of our teams had started to adopt Flux CD for continuous delivery and we're using its helm operator. And its helm operator has a custom resource that lets you specify the helm chart source and then what values you want to apply to that chart and then it will go and fetch everything needed and just make those changes in your cluster for you. However, since helm templates are just that, they're templates, they aren't covering these resources, we couldn't use Conf test on them directly. And this is because the data structure in the YAML is different. And even if it was the same, with templates, you wouldn't actually have everything you needed until after execution. So what this action does is it parses the helm release resource. It pulls the chart info, the version, all of those things. It automatically sets up a helm repository so that it can pull those templates from the remote repository and then it executes the templates. So with this, we can easily template out these resources before we run Conf test so that it's nice and easy and it's a solid flow. This makes it easy for our developers because they don't have to remember all of the flags for the helm template command or the argument order or anything like that. One thing to note here is that this currently only supports public helm repositories, but we're working on adding support for private repositories too. So with the ease of adoption addressed, let's move on to the policy pipeline itself. Early on in the design of the policy pipeline, we have this one rule, is that we must not ever break production. And for us, that also includes any of the pipelines leading up to production. This meant that for our teams that have an automated deployment from development to staging to production, using custom metrics, that breaking their access to the development cluster is the same as breaking their production pipeline. In order to accomplish this, we use two main methodologies. The first was data-driven policy promotion and the second was a get-offs deployment flow. So diving into our policy promotion strategy, we wanted to tackle one policy at a time, which again, is just reducing the window that new resources are introduced that violate the policy while you're working on remediating the ones that you already know about. Another key component is we use gatekeepers enforcement action property to introduce the policies in dry-run mode where they are not enforced. However, when they are in this mode, we can still use gatekeepers audit functionality to audit our resources in our clusters and see which of them violate the policies. We've made the decision to only switch to enforcement mode after we've identified that all of the offending resources have been remediated. And as a side effect of this, we have avoided sending hard deadlines for our teams to update their resources and make sure that they're remediated. So one thing that we've done to work around that is in the case where a team says they just really can't get this fixed in the next couple of weeks, we can add temporary exceptions to the policy for those specific resources in specific namespaces, but only when necessary. That way we still have good coverage of 99% of the resources in our clusters that are adhering to the policy. And finally, we only promote policies to production when they're linked to a change management ticket, which allows them to be scheduled. This of course ensures that all the potentially impacted parties are aware of this upcoming change. And it lets us schedule around times that we may be adding a feature release, a product launch or something like that. Moving on to our GitOps deployment flow approach. It's a pretty standard approach where we use pull requests to move policies throughout the pipeline. We use branch protection rules to ensure that peer review occurs and that all of our unit tests pass. A couple of things that we're doing there is we require each policy have a unique policy identifier, and we require that each policy have at least two unit tests. One is for the positive path where gatekeeper blocks something that we expect it to. And the second is the negative path where a gatekeeper allows something through that we expect it to not block. We also take advantage of the GitHub code owners features, which ensures that the peer reviews come from policy admins. And that's because the policy admins are the ones here that are most familiar with the regular language and also some of the more intricate details of Kubernetes. And what's probably the most important point is that we use automation to create the gatekeeper and contest resources. This means that reviewers can focus on the policy and not the gatekeeper resources, those giant YAML files. And it also makes it really obvious if someone is attempting to go around the tooling in place because no human should ever be trying to modify the gatekeeper or contest resources directly. With that, let's go through the life of a policy in this policy pipeline. So here we have a high level view of how the policy flows. We start with by introducing a policy into the dev branch through a pull request. And we wanna make sure that the policy always has the enforcement actions set to dry run. After it's merged, Flux will automatically pick it up and start syncing it to the development clusters. Next, we then promote this to staging and production branches, still in driver run mode. And this is to ensure that we have full visibility across all of our clusters. Because try as we might to keep all of these clusters configured identically through development staging and prod. There's almost inevitably some variation in how they're configured. So one thing to note is when we merge into the production branch is when we actually generate the Conf test resources. And one thing that we do there since Conf test doesn't have the concept of an enforcement action is that we parse that enforcement action from the common header and we inline rewrite any violation or deny rules to warnings so that when they're used in the CI flows, the warnings are still surfaced to the teams working on those resources, but it doesn't fail the CI flows yet. So once we have all of these set up and running, we then use the gatekeeper audit data as the source of truth to identify our existing resources that violate policy. And then we open tickets to work with our teams to remediate the issues that we've identified or to add exceptions where necessary. After we've ensured that those are all remediated and the audit data shows this, we switch the development clusters and the development branch to actually enforcement. This is done by changing the enforcement action in the header from drive-on to deny. And once again, after we make this change and merge to dev, we are continuously monitoring the gatekeeper audit data to make sure that there isn't anything that we didn't expect. Finally, once we do that, we follow the same flow to promote to staging and then production. It's worth noting again that when we make this change to production is when the Conf test policies will actually be changed back to either violation or deny. So at this point is when the CI flows will start failing if they have resources to violate the policy. The Conf test metrics here is useful for us to see what's going on across our organization to have a more full picture, especially of newer repositories and newer projects that haven't actually made it into a cluster yet, but they are not the source of truth for when we promote policies. So where is Ubico now on his policy journey? Well, we have gatekeeper deployed to all of our clusters and we're tracking the resources that violate our policies. Additionally, we're making our way through each policy and moving each to enforcement as we go. We've noticed that in a lot of scenarios, these policy violations are actually introduced upstream, whether that be an internal or external project. And whenever we run into that, we try and make our upstream our changes as well. Looking into the future though, there's a few ways that we can work on making this better. The first is to enable our teams to write their own policies for enforcing their own specific best practices. This can be anything from a custom label to requiring that every deployment have a horizontal pod autoscalar attached. Additionally, we're looking into seeking data from outside the cluster into gatekeeper to be used for more important policy decisions. For example, we might want to sync our on-call rotation schedule into gatekeeper so that only the person who's on-call for a given team can make changes to production in the event that a production issue or outage is occurring. Looking even further, we are considering adding mutation controllers as well and those are a different type of controller that the API server can work with, which rather than just rejecting a resource if it doesn't meet policy, it can inline change the defaults as needed. One thing to note there is gatekeeper currently has an open design for their implementation of that. They haven't started on it yet, so if you want to add your thoughts about how that should be shaped or anything like that, go ahead and just search on the opus slack for mutation design and it'll come right up. And that about wraps it up. Thank you everyone for attending and we are going to open it for questions.