Good morning everyone. I'd like to thank all of you for coming this morning before we get started with my talk I'd like to take just a moment to Thank and appreciate all of the folks who have put so much hard work and dedication Into creating dev comps easy this year so that we can all be together. So could we please give them a round of applause? Thank you. So as Roddick mentioned, my name is Leslie Hawthorne. I am a Senior principal technical program manager fancy pants in the office open-source program office in the office of the CTO Most of my day-to-day work focuses on helping our customers understand the business value of open-source and to help them architect their open-source software strategies Today I'm going to talk to you about the process for creating high-performing teams some common barriers to groups that make them much less successful and We're going to leave today learning some lessons that what each of us as individuals be we individual contributors Team leads or just natural leaders or managers can do to create really effective and high-performing teams You have got to be joking Remember when I told you my AV always fails. What magic did you do? Okay, thank you. You're lovely. I appreciate that I'm not going to touch anything except this All right So to frame our remarks today and give you folks a little bit of context for the discussion I'd like to talk to you a little bit about Google's project Aristotle So in 2015 Google decided to launch a study out of their people operations department. That's what they call their HR folks to try to determine What features in the individuals and a team meant that the team would be incredibly successful would ship products that customers loved that would gain new users and The executive leadership and the human resources department had a lot of theories About what would create high-performing teams these ranged from we have to get all the introverts together on a single team We have to get people from elite universities all together on the same team etc etc and They were very surprised when they reported the results of this study Which included in-depth interviews with 200 people and surveys of more than 180 teams To discover that all of those theories were completely wrong The most important Elements to having a high-performing team are those that you see here before you Psychological safety which we'll talk about in depth today The dependability so you know that if your team says someone on your team says they're gonna do something They're gonna do it structure and clarity. I know what is expected of me and I know how to do it meaning so my work personally matters to me and Impact my work matters to the wider world beyond what I do each and every day so the key aspect They found of all of these features was in fact psychological safety And in fact all of the other factors were impossible to create if that team was not operating in an environment of psychological safety So and again, like I said all of the study Makers were extremely surprised by these results I was not surprised by these results because once upon a time I used to work at Google and I certainly didn't feel like I was operating in an environment of psychological safety Although at the time I didn't have those words to talk about it But I definitely felt like I wasn't one of the cool kids, right? Like I had an English literature degree It is not very fancy to have an English literature degree, right? I I went to the wrong university many people at Google go to Stanford and I attended Google's arch rival University the University of California at Berkeley, right? I was a woman working in the engineering department. There weren't a lot of us, right? So there were there are all these ways that I kind of felt like I didn't fit in And it wasn't really safe to be vulnerable, right? Like I felt like every mistake I made Counted a lot more than maybe some of the mistakes that other people made Now what was really strange to me was You know moving on and getting really, you know cool job at places like Red Hat and Then talking to my friends who were still working at Google who were in the engineering department and hearing from them that they had similar experiences that They felt like if they weren't the smartest person in the room, they shouldn't speak up and share their ideas That they weren't that they weren't smart enough to contribute that they often held back Their ideas because they feared that if it wasn't a good enough idea They would get made fun of or people would you know decide that they weren't very intelligent, right? so Again just to sort of frame the idea of psychological safety right operating in an environment where you know if you make mistakes It's okay So to bring it back to all of you folks. How many of you have had jobs where you felt like it wasn't okay to make a mistake? Please keep your hands raised. How many of you found a new job as quickly as you could? Haven't seen and didn't see a single hand go down How many of you have your most brilliant thoughts and Ideas when you're with people you know love and care about you and I'm the same way, right? I I find that I have Very good interactions, and I feel very intelligent, and I have very good ideas when I'm with people who Respect me and who think that I matter and if I'm in a conversation with someone who I think Doesn't have a lot of respect for me. I just can't say the right thing no matter how hard I try, right? so Now that we have a grounding in what psychological safety is and it's important. I'm gonna talk a little bit about common barriers to psychological safety that exist in companies and Again, like I said at the end we'll talk about a few ways that we can overcome these barriers and contribute to our mutual success So I believe if I am pronouncing this correctly my check friends call this animal a crush Did I get it even close to correct close? All right? I win So in English we call this animal a hippopotamus or a hippo. We just shorten it to hippo so in environments that Are not conducive to psychological safety one thing that all happens frequently is hippo wing So the highest paid person's opinion is the only one that matters, right? I've also seen this referred to as the highest prestige Person is the only one that matters. So the rock star engineer in the room is the only one that people will listen to and This is incredibly damaging because even the smartest person in the room Or the highest paid person in the room is not always going to have the right idea and If you're in an environment where everyone is simply waiting for the right person To bless you all with their amazing thoughts. What does that do to the other people in the room? They disengage They don't even try to come up with ideas because their ideas aren't going to matter. They're not going to be heard and They really stop caring about the work that they're doing because they can't feel personally connected to it, right? It's not it's not something that they're participating in. It's something that's happening to them so Just a really quick tip if this is something that's happening in your environment I highly support getting like cool little pins for hippos and just start saying no hippo wing eventually it'll catch on I've done it before Another thing that I freak much more frequently see in startups But it happens also in big organizations is this concept of hero culture How many people have heard the term hero culture? Okay, I see quite a few hands The other way that I hear this referred to is firefighter culture. Is that also familiar? Okay, excellent. So for those who may be less familiar with this terms the idea of hero culture or a firefighting culture is one in which rather than attempting to Fix the root cause of particular problems The solution to problems is simply move faster and throw more human effort at the problem Right. Why take time to address technical debt? If you know that you have a team who will be on call 24 hours a day seven days a week whether they are home or away to fix whatever is broken, right and Hero culture is incredibly destructive to psychological safety for a number of reasons, right? First of all, if you are working 24 hours a day seven days a week And you have far too much to do. There's no way to replenish yourself, right? You're tending towards burnout and obviously someone who is is constantly stressed is not going to feel psychologically safe and then the other place where this erodes psychological safety is In being in an environment where you can talk about how to address problems, right? If the culture of your organization is that we are all very cool and awesome people Because we fix stuff when it's broken all the time being able to bring thoughts like Maybe we should fix it so it doesn't break and that will take us an extra three weeks And it might delay a release, but then we don't have to worry about it as much Those thoughts aren't really welcome, right? They're contrary to the culture that says we are successful When we fix things that are broken and unfortunately People get burned out. They can't keep performing at this optimum level all the time. It's just not sustainable and Unfortunately, you lose people either because they get burned out and they leave Or they get burned out and they stay Which is potentially worse for your organization again something that I see really frequently within startups cultural norms of a company that actually Exclude people and these are often really hard to see, right? So if you read a typical job advertisement for a startup in Silicon Valley It will regale you with a long list of all of the perks the employees get for working there, right? We will serve you breakfast lunch and dinner Every night after work, we will all go out to the pub and have a drink together before we go back to the office On on Thursdays at 4 p.m We have the office foothball tournament and the winner of the office foothball tournament will be I don't know given an extra beer from the keg in the kitchen, right? Now all of these things don't sound so bad, right? I mean I want to drink a beer. I want to have breakfast I'm not a foothball player, but you know this doesn't sound so bad until you kind of start unpacking the layers of What those cultural norms mean, right? A company that serves you breakfast lunch and dinner is Setting the expectation that you're going to be in the office from breakfast until dinner time a Company that has a keg in the office kitchen is setting the expectation that alcohol consumption throughout your workday is normal a company that you know has a foothball tournament every Thursday at 4 o'clock and that's the social focus Of the company and the place where people bond Well, I'm kind of out. I don't play foothball. I guess I like to watch it, but you know, maybe not every Thursday And when these cultural these cultural norms just simply sit within the organization It's very difficult for people to sometimes realize like these sorts of things send a message For example, if you're if you're a parent or you have family commitments, or you even just want to have a social life After work if the company serves dinner and you say I got to go home I got to feed kid lit, right? The message that you are sending in that environment is I am not nearly as dedicated to my job as the rest of you Who are willing to stay here through dinner and late into the night? If you choose not to go to the pub with your colleagues because you don't you don't like to drink alcohol You are sending the message that you are not Cool, you're not part of the team. You're not there to be a friend to your colleagues, right? So again when we're when we're looking at how we set up our organizations and our cultural norms We have to think about what each cultural norm says to To everyone right not just to the people who are like cool. I totally want a beer. So We talked a little bit about a few common barriers just to set the stage So let's talk about how we actually create psychological safety and how we go about the process of building trust So the feeling of safety 100% absolutely starts with trust and you can only really Change yourself and the way that you interact in a given situation. So a lot of this Sage advice is focused on what we as single persons can do To create trust in our environment. So I know red hat is very big on talking about our values of transparency But I cannot stress enough how important it is to to be transparent in your communications with other people and to be really clear about what your needs are and why you need these things and And make and ask people to help you right just be very very clear about it And when you're having those discussions It's not enough to be to be really aware of your own needs and clear about what your needs are If you're going to ask someone for help if you're going to ask someone to collaborate with you You need to understand their needs and you need to help them reach those needs and That's not always going to work But you all and in those cases you just need to be willing to compromise right and you're not always going to get The best outcome for you what you're striving for is the best overall outcome for everyone Some days it's gonna be great for you some days Maybe not so great, but if you want to create a psychologically safe environment that has to be okay with you Last but not least and probably I think the most important Aspect of this is be a real person Like it is possible to be a professional human being and to really be yourself I don't say everything I think when I'm at work, but I'm still me when I'm at work, you know I tell people my honest thoughts and I try to do so lovingly and with respect But I don't you know I don't put on this like robotic professional face where we are all going to talk like this in a monotone So that we don't defend each other This is not a way to create psychological safety people feel like they are in a bad 60s movie So again bring your whole self to work Another way to create an environment of psychological safety and to build trust is through your own vulnerability Besides we needed a cat picture at this point in the presentation. So when I when I talk about Leading through vulnerability. I mean be willing to admit you're wrong Be willing to admit you have a poorly formed idea and you need help making that idea better right don't be afraid of Owning the fact that you're a flawed human being there's not a single person in this room who is perfect no matter How skilled they are as a programmer no matter how long they've been doing their job none of us are perfect And if we can just walk into situations Waiting to be Surprised waiting to be happy because we were wrong and someone else was right and that means we got closer to having a good answer That's really important. It's not just important for us It's important for other people because when we model the behavior That it's okay to make mistakes and it's okay to do something wrong We create that psychological safety for ourselves and for the other people that we're collaborating with because if you can make a mistake I can make a mistake and at the end we're all going to be better So How psychological safety fosters innovation? So one of the things that I promised I would talk about in this presentation was a little bit about how psychological safety is important for creating Healthy environment for diverse teams. So we're going to touch on that quickly So how many folks have read some of the studies that have come out about how having a diverse Team creates better results for the company. I see a few hands This particular study that I'm citing For McKinsey was written up in the Harvard Business Review and I encourage you to read both the article in the study But it has been very clearly demonstrated in a number of studies that the more diverse your team is The higher the financial returns for the company. So diversity isn't simply a Great goal for businesses to have because we want to be nicer to everyone diversity directly impacts the business bottom line, right? so diversity Having lots of different kinds of people contributing to a problem Added to psychological safety Creates inclusion. It is not enough to have different kinds of people on your team So that they can contribute to a better and different result They have to feel like they are included. They have to feel like they matter If there are people in your organization who are not like the dominant type like when I was in human resources It was all like young white ladies between the age of 24 and 27, right? Like we were all the same We had one dude working on our team. He was miserable. He just was like I don't understand why I'm here and you guys all like make me feel like I'm strange because I'm not talking about my shoes We bonded about how I didn't want to talk about my shoes either. So Without psychological safety the diverse participants of your team, they don't feel motivated to stay, right? You've invested so much time and energy into bringing someone into your organization To help them be successful there and if they don't feel like they're safe Because they feel different because they don't feel welcomed because they don't feel like their ideas matter they're going to leave and you're not only going to lose out on the opportunity to have a more innovative product because of that Diverse perspective. You're also losing out on the many dollars that you invested just bringing them into the company So I'm an American and we're emotional and I'm gonna overshare a little bit so I think it's really easy to stand up here and talk about being vulnerable and To talk about like psychological safety and you know what it means and it's all very like theoretical that Psychological safety for me means that I can be me at work every day How did the story be hard to tell so right after I had my daughter I had severe undiagnosed postpartum depression. I had no idea and It took about two years before I couldn't do stuff anymore and I took medical leave red hat was incredibly supportive and when I came back to work. I was so embarrassed Because I'm not the kind of person who says I can't do it Like if you hand me a problem, I will solve it. That's just who I am and I came back to work and I was kind of Closed in on myself a little bit because I was really I just didn't feel good. I thought I'd let the team down and I remember getting an email from a co-worker who's not here today. I wish he was I would thank him in person again When I said to him, you know, I felt really embarrassed. I felt like the team down. I let the team down Taking leave and he said to me Leslie we have missed you so much. We have been waiting for you to come back We have all been waiting for you to come back So that you could be here with us. We have missed what you bring to us We have missed the conversations we have with you. We have missed the way this team operates when you are here Psychological safety means to me that I can come to work every day and be who I am It means that if I say I have to go to daycare right now and pick up my kid.  No one judges me. No one thinks I'm less capable Right, it means to me that if I say I didn't get as much done today as I wanted to get done Because my back went out. No one thinks I'm any less capable, right? I can bring my whole self to work and I'm not afraid and that is so important so We've talked a lot about a lot of theoretical things. We've talked about my own experience So what are the three things each of us can do? Right now to create environments of psychological safety First of all be kind to yourself life is hard We are often our own worst critics if you find that you are Being unkind to your friend who is you because you really do need to be your own best friend I mean you live with you every day, right? then show yourself some compassion and Extend that compassion to others right if someone is having a bad day if someone's made a mistake if someone is You know just not up to par. They're not doing the things that they need to do. Let it slide. It's one day. It's not every day That's not who they are It's one mistake. It's one day And and last but not least we are all as I said before flawed human beings, right? None of us is perfect and I think that the most important thing that we can do together is make way better and Different mistakes every day Thank you


Simon and Schuster Audio presents Memories of the Future A Novel by Siri Hustvedt Read by Catherine Fenton Chapter 1 Years ago I left the wide, flat fields of rural Minnesota for the island of Manhattan to find the hero of my first novel. When I arrived in August of 1978 he was not a character so much as a rhythmic possibility, an embryonic creature of my imagination which I felt as a series of metrical beats that quickened and slowed with my steps as I navigated the streets of the city. I think I was hoping to discover myself in him to prove that he and I were worthy of whatever story came our way. I wasn't looking for happiness or comfort in New York City. I was looking for adventure and I knew the adventurer must suffer before he arrives home after countless trials on land and sea or is finally snuffed out by the gods. I didn't know then what I know now. As I wrote I was also being written. The book had been started long before I left the plains. Multiple drafts of a mystery had already been inscribed in my brain but that didn't mean I knew how it would turn out.  The future. I had given myself exactly 12 months to write the novel. If at the end of the following summer my hero was still born or died in infancy or turned out to be such a dullard that his life deserved no comment. In other words if he was not a hero after all I would leave him and his novel behind me and throw myself into the study of my dead or failed boys' ancestors the denizens of the volumes that fill the phantom cities we call libraries. I had accepted a fellowship in comparative literature at Columbia University and when I asked if I could defer my admission until the following year the invisible authorities had sent me a long winded letter agreeing to my request.  Sample complete. Ready to continue?


Good morning. Good afternoon. Good evening and welcome to another developer experience office hours here on red hat live streaming I am Chris short host with the most and showrunner for this great thing. We call red hat live streaming I'm joined by two of my favorite technical people here in the in the world DeWon Ahmed and Carlos Santana not with the guitar with the Kubernetes clusters DeWon do you want to tell us what we're talking about today Absolutely. Absolutely. So I'm dialed in from the beautiful new Brunswick on a relatively cloudy day That's why I have multiple lights on so today We'll be talking about Kubernetes admission controllers and before I pass into Carlos, I wanted to ask the audience who's using Kubernetes admission controllers. Oh good question And there's a slight delay. So maybe Carlos and or and introduce yourself. Yes, please Folks, let us know if you're using an admission controller. It's a good Yes, a delay Yep, my name is Carlos Santana. I work at IPM as a cloud architect Mostly working with clients on DevOps and cloud native Helping them with OpenShift. So I work a lot with OpenShift Helping them get into the cloud basically and I also do open source. So I'm a technical lead 4k native I run the working group for user experience and docs there So happy to be here. Yeah, we're happy to have you Carlos. This isn't your first time on the channel. Is it Carlos? Line presenting. Yes. Wow. Okay. I can't believe it's taking this long because you've always like been around I feel Yeah, in the chat. Yeah, you're always watching and that's cool. We appreciate that But yeah, it's great to have you on the first and Carlos is actually one of my ex colleagues at IBM Yeah Just to work together in IBM. Yeah. Yeah, you're still in IBM. What do you mean? You never left I see I see that was a trick question So to comment coming back to the admission controller, please that was a trick question because by default I think Kubernetes ships 30 plus admission controllers with a yeah and The the specific controllers will be so you're using it by default even if you don't know it. So today Carlos will be talking about admission webhooks and how He can use these to to manage large set of our go deployments It's it's one thing to deploy a single app on a single cluster But when you have many apps and you're dealing with many cluster What are the challenges that that we're facing before that Carlos? Would you be able to give us some some idea about like what these admission controllers or webhooks are? Just from the very general point of view Yeah, so let's use the official Kubernetes stocks because open shift is Kubernetes, right? And Kubernetes open shift. There's actually a top people stories about Open shift that there's things that started in open shift and then there were like upstream to Kubernetes And some people are confused saying all that's in Kubernetes, but it was in open chip first, right? And then show up in Kubernetes. So they're they're like siblings So let me share my screen and go over a little bit about you see my screen. Yes so we'll go over the admission controllers and admission controllers We're going to talk about the dynamic one. So there's many of them, but there's two specific ones that are very Useful, so one of them is the validation that you can see here a validation webhook and the mutation webhook and I don't know if they have the picture in this in this but essentially is when Something and it could be a one of the admins of the or a developer Working with open shift. It makes a request to the QP API server open chip API server It goes to the different life cycle Checks you can say like authentication. You can see here identification Authorization and then comes validation and mutation webhooks. So if you take the validation webhook Is your opportunity you as a programmer or admin or this is also very very Use in operators when you're writing operators There's a chance the API has Registered you have registered with the API saying Ken is this allowed or not allowed am I allowed to create a namespace? Am I allowed to create a deployment? with a image that is from Docker or or Or the Amazon registry or it must be for quake And you can decide am I going to allow it or not allowed and then and then the API service asking you and asking like All the people that have put validation webhooks. I think that's what I'm saying like I'm using open shift there's already validation webhooks that you can see in there and If one of them say deny forget it, you're not getting to allow access. So this is very different from our back You're familiar with Robics access control So our back even though maybe you are an admin and you can create a namespace Let's say the simple example that it will use a lot if let's say you are an admin straight up this open shift cluster and for some reason I Managing this cluster for for a long time and I want every namespace to have an owner like a name And an email of someone because maybe three months from now I Wonder if this namespace still used right and we can delete it or not. So who do we ask? Usually by that time we cannot find who was the person that created this So maybe I required that that namespace Has a label maybe owner and an email right or an a I Added a link in the chat Carlos if you would like to open that link. I think it might be easier There's the picture which goes from out Yeah, so this is the picture I was looking for. Thank you. Yeah, I would have the other one Yeah, so It was so mean into here It's asking it's asking so evaluation is here, but I was just saying at first as validation and Basically validation is like asking, you know, all these web hooks and web looks at People think that are like very complex type of things. They're just HTTP requests with post. It's a web API Similar similar ones that you use in GitHub when you push something to get help you get called by a web hook Basically doesn't HTTP post request and typically they have application gaysum So what I was saying was if I'm a Atmine and I want to have in my cluster like every namespace might have a Label with the owner I can create a policy. I can create my own validation web hook I create my own code. So I like it. I write a little web web code that says if the namespace to be created Doesn't have a label then deny it. I don't want to I don't want to be allowed Then mutation web hooks it could be something mutating the request. So this one is read-only So you are allowed not allowed. This one can allow it and not allow but on top of that can mutate that request So maybe I'm the lazy developers like And I own the cluster. So I'm not going to like Edit every namespace to add my email address as an owner So maybe I create a mutation web hook and the mutation web hook a skull saying do you want the api server? Ask your web hook saying I And you register like what type of objects do you want to listen? So maybe I only want to listen to namespaces. For example, you don't want to listen to everything And then you say I want to listen to namespaces and if the label owner is not present Maybe I want to add it. So I'm mutating that request So when you create a namespace, I check that it doesn't have an owner And I put like the owner is Carlos because Carlos the only one using his open cluster and I add that Label and owner back to the api saying hey go ahead and patch So the programming model is you're going to patch that resource that typically you see in yaml But it's actually In api world is Jason. You're going to patch it with a json patch saying add the label owner With my name Carlos in there and then let it through so by the time when it hits that's why validation is after So when validation sees that that request it only sees a namespace with the owners are like Very good. The namespace has an owner. So That's a high level kind of a story of like how did it work and Before I learned this I did learn about this recently in terms of like the details behind it Um, I didn't get my my head around this. So how do you for you to you were you knew this or maybe it's it's useful I think I think I didn't have much idea about this before and this five minutes overview got me got me the fundamental So so thank you for that. Yeah. Thank you so much And and crd so I want to add one more thing It's like the reason that is uh using operators is for crd So sometimes you install things like the open she's serverless. For example, the one I'm familiar with which is based on k-native Or tecton. I was working with tecton And the the version of the crd. Let's say from last year was v1. I thought one But now you install the new version of the operator so what What about if I want to upgrade those resources to vbeta or v1 like in k-native where Where v1 and we want every resource to be in v1 So one thing that you can do is if you give me a A yaml, you know, you do keep ctl apply or oc apply with a yaml with a k-native of v1 alpha 1 When you write operators, um, you create a crd conversion Mutation webbook saying I'm going to convert this resource that now has v1 alpha 1 And I'm going to convert it to v1. So when it's written into xcd It has v1. So my webbook can expect the resource saying well this field we change the name from A template to something else now This field needs to be specified. So I'm going to specify default value And and the api version now we can change to v1. So I just upgraded The operator was helpful enough to upgrade my yaml To that version and maybe give you back a message and you collect messages saying, hey, you should like upgrade this this yaml that you happen get to the v1 But we'll try our best to upgrade you on the fly um, but that's the usefulness of the crd and this was It could run it this years on open shift years will say reasons and it's like recent as two years Right, uh, so it's very recent right because I've been alone for a long time before we there was a time that was no crd Um, right if people remember programming those but uh mutation webbook is something that you can specify when you write in your operator and the operator sdk the framework operator framework sdk When you start working with the with that framework, it also helps you You know We we creating these weapons like templating that like the the metadata around it So I just wanted to add the the crd as a as a A notation there because a lot of people ask or see them when they start an operator and they check For validation mutation weapon. They start only see all these Mutation weapons. I wonder like where they're coming from. What are they useful, right? So us a cluster administrator should be aware about these things because they can Break your cluster for example So these validation weapons can have something around The failure mode so we can talk about the failure mode We want to go deeper. Um, the failure mark is like if if let's say the weapon is down Uh, do do I deny the request or do I allow it? Right because this is a thought running in your cluster. So maybe you're doing maintenance or something or it cannot be reached or times out Uh, so you can do fail. Um, I think it would do The failure policies call You can you can fail or you can ignore If it's not here So the failure policy you can say, yeah, um, if I cannot reach that web hook go ahead and Yeah, keep it going, right? It's not that important. Maybe it's a death cluster, right? We're doing testing Uh, and then maybe in production you configure the yaml saying nope, uh, that that means I want that check. Um, I want to fail if if the if the web hook cannot be reached, right? Because it could be a security validation weapons things like uh, opa open policy agents Gatekeeper kiver no type of like they do security policies. They implement validation weapons that you can configure with uh configuration like daml or or regal Um, and you and those sometimes are policies that do need to be compliant So in production that cluster needs to be in compliance. So you don't want to Have it like ignore like if do your best to be in compliance, right? We never heard auditor saying Well, do do your best, but if you can't like that's okay You're either in compliance or not compliance. There's no middle ground. Uh with that Uh any questions from the chat? Uh, no, but we just get a hey from kaleed. So hey kaleed. Nice So these are the documentation in the upstream kubernetes, um about um validation web hook. So um, let's see how we can create one. Um, what do you think? Sounds good and we have a specific use case uh here today. So for those of you who are familiar with argocd You probably know that project for those of you who are not so familiar with argocd. I added a link in the chat So argocd is a github's continuous delivery tool for kubernetes Or when you're deploying your application to your cluster, you probably want to follow a github's model Which basically says you use gith as the single source of truth You make changes in gith rather than making direct changes on the cluster, which is very dangerous You don't do kubernetes apply. You don't do scripts Even worse, you don't do kubernetes apply of a bunch of scripts You rather make changes on gith and then those gets reflected on the cluster and the last thing is All the changes you or your team members are making are very viable and auditable So that's what argocd, which is an open source tool provides in short and Carlos will will explain the specific use case he has When dealing with large volume of argocd apps and how admission webhooks can play a role there Yep, so um, let's take a look at a validation web look The for that you have to create a a yaml file. So if we look at the at the yaml file specification This is this is the one that I talk about read only validation. So You register your webhook And these are the things the rules the rules are the things like I want to only be concerned with these objects If your service That you're going to implement your code basically the the web servers that listens for that http polls request It's going to be sitting in the cluster You you need to specify this the namespace and the name So the api server can conform the URL https The name of the service dot Example namespace dot svc like that service and it will call you you can also have the option instead of putting service in here And and replace this section for URL Um, so I I have a question for you. So if I put a URL, what what can I put in that URL? What do you think I can put in that URL? I think it's it's down here in the specification. So I can put um A a url I can put a url in that section. So what what can I put in there? What do you think I can put there? Do you see here? I mean, it would be the path to Whatever pod you want really, right? And if that's a fully quality, no It can be in my web hook example.com. It can be something outside your cluster. Oh, okay So if you convince your friend To install this yaml file that you put at the bottom bottom bottom on this yaml file Right and you put https and it goes to carlos.com I now have access to I be able to modify or validate Your your request so I could be modifying your image pod on the fly and you will never notice because The deployment says that he has one version And I put in my my hack version. So this also could be used for For malicious use. So that's why you have to be Careful about so the the web hook not necessarily needs to live in a pod in that cluster It could be living in another cluster. It could be a Like a k native and serverless function living in some cloud And it could be useful right because it could be used by many many clusters Or it could be some Exposure that you're like calling something outside the cluster That you shouldn't so that's where like you put things like egress policies You put things like istio to make sure that nothing leaves the cluster you put network policies Um, well for the qp api network policy will not work, but you will put something in the nodes right in the master nodes So you're trying to highlight that because uh, that's some some something that People are not aware that you can put a url and that you'll be calling something Outside your your cluster and your cluster may be broken. You don't know why it's because there's somebody else like messing with you, right? So that was an interesting one. So Fun fun tip there, right always check And that's why validation and mutation what we say cluster scope Resource so usually and that means the one that have access like this. There's it's not namespace So this is an example of a node.js validation a mission controller I want to show a little bit of that example So it's also good to see a specific example. I took this one from a from a blog post. I saw the other day Um, and basically this is the the yamo file. So I have the webhook, right? So I have a webhook validation mutation webhook and I'm going to listen Um, and you can be very specific of like, what do you want to listen? So in case that you're doing development, maybe you don't want to listen to everything in the cluster you want to specific So I I put this one like because I was like experimenting that it was a cluster that my team was using I put like maybe only want to listen to objects that live in a namespace That matched expression CNTK experiments So this is like telling the team like I'm doing an experiment in this this namespace if you want to You know Use my validation webhook go ahead and label Your namespace with experiment equals CNTK and then The kubei-pi server will start sending me events from that namespace. So It's it's very granular what you can listen on right? I don't and then what things why I want to listen I want to listen to pods. So anytime in deployments or We say deployment, but it's a replica set read the bombing case replica set replica set creates a pod When a pod gets created so you have things here like when it gets created deleted Updated like you can listen to anything In this case, I'm listening for pods But it could be config maps. It could be namespaces. It could be any resource In kubei neti that this is why it's very powerful, right? This is What I found out is like this is the the I would say the easiest way to extend kubei netis Right kubei netis giving you a webhook Into here. Yeah, and and then the scope is it's namespace. So these resources say nesca scope and then what I want what I want the The kubei pi to call right is is this pod With a service sitting in the default namespace and then I'm going to allow this registry and the path is slash notice that The kubei pi server always calls with https So my code needs to have Oh, this is something very cool that I found out about open shift is When you're developing this type of things either your framework or the person that is starting this or sometimes even as a bad practice Your same code is creating the certificates the tls certificates So if the the api server is calling an https endpoint and it's sitting inside your cluster Then your program needs to have an https or ssl certificates and then you the The kubei pi server which is serving as a client needs to see a who am I going to trust? So as you can see here, um, I think I saw ca bundle something ca bundle Uh, where was it? I think it was Uh, you look for ca bundle ca bundle the ca bundle It's a base 64 included certificate ca And I was like, uh, do I create a Shell script? I don't know if I have it here somewhere. Um, a a shell script that like looks like this Like everybody have done open ssl Can be familiar with this right? Uh, open ssl credit For the address of like white list register default theme Uh, create a csr Generate the ca don't have the ca now have the my tele-certificate Right and then you have to give this to security and we have to rotate them and all that Um, so the beauty of this for people I appreciate appreciate not doing this is you go into your, um Uh Webhook and you add this annotation at the top I don't know you can see this You say service the beta the open shift the i inject ca bundle So when you send these to kubernetes or open shift Open shift is going to insert in here the ca bundle Of the open shift ca. There's an operator ca in open shift So I don't have to do that piece for the ca The other thing is like, how do I get The the certificate and the key for my app and it could be go python or node. Yes, right? Um, and you go into the deployment Um, and then in the in the deployment you say Huh You create the service of the of my deployment is pointing to my service. But for this service I use a different annotation use another annotation saying service beta open shift Serving cert secret name. Basically, I'm taking open shift. Hey open shift create me a secret In this namespace with this service is with this name And in that secret I should be able to find my service certificate and my key that I'm going to load into my application Once I found out this I'm just testing your open shift like I don't I don't want to deal anymore with open ssl. Um, so by doing this, um, Suddenly a secret would show up in my deployment that I can use so now I can mount it So if you're familiar with deployments, right? You will have a volume that you can mount so you will mount mounted And then you have a value months and certs and in that search folder inside my application. I will see that Um and I just joined this, um, and then Uh, let's see the code. So it's the specific, uh, webhook what it does is, uh, having In no jet. So the thing is this is a webhook So web, uh, especially creating a web server. You can use python node java Apache htpd anything that does the web request with bash cgi um And and basically this is i'm using node jas. Um, this is where i'm loading the the certificates that are created by open shift um, and then i'm inspecting the The the jason that I get from the api service. So he sends me a a review object I can't um, so this is the git or I think I was showing the other one Uh for the node jas, uh, let me see the server. So this is a a little bit more, uh, simple one So this one is basically what i'm doing is i'm going to configure this validation webhook To allowed or not allowed? Through a false very easy And basically what i'm going to say is i'm listening for pods And if the pod starts with a certain registry or it doesn't start with a certain registry I'm not going to allow it So in this case this example is around Uh, let me see if I can find it. It should be here So i'm have environment borrow with my phone saying if the image in the pod Starts with docker.io or gc r.io Good if it starts with I don't know the hack registry.com It's not going to be allowed, right? So this is very useful for air gap environments where customers have an air gap environments They have their own registry usually they they use quay From rehab right and load their own registry and they have it in their like acame.com internal ca And for security reasons, they don't allow any any image to come from anywhere else So they will mirror images from the web They will mirror images from the open ship marketplace and they will have that Image here and they they can implement a validation webbook With this small amount of code, right where i'm checking If it starts with this do not allow the pod to be created or to be created Quick question carlos would that allowed registry url be that mirror registry url then because There will be mirroring from whatever allowed registry to that mirror registry And are they trusting only that mirror registry which they have? Yeah, so quay has a feature called Syncing or mirror so I can have a server called carlos registry You know like registry.carlos.com And I have quay running there in my inside my vpc in the cloud public outcome premise And i'm i'm fetching all those images mirroring those images there and my open ship cluster only takes images from there So that's a very common scenario in enterprise where they want to have And and that's the opportunity to scan images right for cvs constantly That's another thing that you can do nothing that is Very efficient, but it's possible right what happens if in this code I want to take the the registry Of that the the image url of that of that Of that pod and inside my my app. I want to download it And I want to run 3d or I want to run a scanning Right there and if I find something like critical, I say like nope The pod would not be allowed to be created. So that's just an example of like So you can see the code so if we go to the code Basically the these webhooks what they expect back is a A jason a response and they the the document goes into the details. This is document was written Um, I forgot his name, uh, but he's from from google And I told him the other day on twitter, uh, that this is an amazing document and I went back to the I'm a gig I went back to the get history. I did a gameplay And I figure he wrote most of this code I saw other people help but the first version, uh, he wrote this and I said like he explained this very very well So there's a section here called response Uh section and then it goes over, um, You know, what is the response the request and the response of that api? And and he goes around like why am I what information am I getting and what things that I need to reply So I need to reply with something very simple Is a a a jason object that has a mission review and then I'm going to allow it or not allow it but on top of that I can mutate it. Um, so Let's look at the at the art any questions so far. You have any questions? No, you're good. Go ahead. Yeah Or or your free to ask questions folks that are watching.  Um, right. I cannot change the pod But what about if I want to change the pod on the fly? Like, uh, then I will need to create a mutation webhook um, so that goes into like, uh, the example here that I have that that I wrote the other day We say it's a mutation webhook that I want to listen to, um, Argo, um, like a get ups definition of an application and projects.  Um Uh, they have many argo apps so I can show a little bit of a diagram that I'm dealing with So, um, you see my screen? Yeah, with it. So we have, um, typically the clusters. Um, this is not Like, uh, uh, what everybody does right or specific to everybody does But most likely these are the things that we test. Um, so you can start with a single cluster that has the Let's say for example, the namespace is c, i, c, d, dev, staging and prod And then you can go into like, well, I cannot do everything with one cluster I need two clusters because in an enterprise you start like dividing the the resources Um, in in theory, you should be able to have this right with the tenancy and open chip with a big Cluster with a lot of worker nodes in practice a lot of customers want to have their own Own cluster for different purposes so they can have one for production only Uh, or a shared one or they only have one for production and the rest Or they can't even go to the extent if they're a large corporation They can have multiple ones for production meaning like active active or multiple regions And then they have one for staging Dev and c, i, c, d So all these clusters, uh, definitions everything that it gets deployed doesn't get deployed by a human That's kind of like the practice in the enterprise when you go get ups only It's like we call it like no sneaky person. It's niki sre is allowed to go and you run all see apply Um, everything is in git. Um, so then we have multiple git repos and the reason we have multiple git repos is We have different teams In in real world We don't we don't have one only person knowing about everything. We have people that are in charge of managing the Uh, the cluster infrastructure So resources like machine sets machine conflicts in that cluster Because we don't want to bother the development team like, uh namespaces cluster roles Like those are global things that should be done once Um, and maybe managed by by we call it the infrastructure team, right? How many machines you run the development team really don't Care about that. Um, then we have things like services or shared services Like operators So if you're going to have a shared service like let's say service mesh is still maybe you want one control plane And to manage that control plane the short services is the thing is in charge of that or jager or Again in idm some club pack for integration kind of like uh middleware You have one instance and it's managed by the services team and that person are in charge of that And then you have the rest of the you know population of of apps Where they deploy that so you have different git get repos that you need to manage with with argocity. So you will have Different different argo apps In different git repos. So for example Um, I have this example here. So in argo you start with a single let's say you have A very very simple starting point And this is the pattern of Apps of apps. I think you were saying uh asking me before Yeah, you create you create an argo app and you put the yaml for that argo app and This is kind of the seed like Um, you have to apply this manually or you apply this one from cross plane data form ansible Maybe the the whatever automation you have for creating a cluster That automation usually installs argo using like Red hat advanced cluster management or just puts the argo in the cluster And then say and for that argo instance, here's your your starting point. So this is called the bootstrap um, then you have Layers So you have an argo app that defines the argo apps that are only for services The argo app that is for infra and the argo app that is for apps So you're already starting with a parent and child relationship Um, and then you start with well the apps team Um, want to have different environments the ci the dev the stage and prod And then maybe they want to have this in in the github repo and they in their different teams repo They have they want to have another argo app that deals with the ones with the folders inside that repo Uh, for example, the deployables. So let me enable the deployables here And you have the deployables here. So as you can see it gets um and this i'm just describing one cluster So I said before that I have the the ci cd the dev the stage in the prop prop one prop two And a lot of people I even have preprop one preprop two They come with all the certain names. So they have you have many apps, uh, like this And you have the infra also the infra will have um services and infra will have their own argo apps So this one has like the cluster roles for the different So the idea is that The clusters Will reuse the same github repo. You don't want to have different github repo for different clusters. So if you want to follow the 12 factor, you know methodology of having like similar environments if you're using Istio for example, uh 1.6, you don't want istio 1.9 in staging 1.6 in prod You want consistency. So the way you have consistency is by reducing the ripples. So you have to put these ripples in in git So you create git ripples. Uh, so this is an example on how many git ripples you will have Maybe you have the main one that has these argo apps Then you have the the The github repo that has these folders that are shared. They're like libraries this first two are like Like libraries that are shared and then the last one this one is like for the different teams They will have their own Kind of like namespaces assigned and they have their own cycle of using like course customized for their product and staging um So it gets a little bit a little bit complex with so many argo apps everywhere But once you deploy this is static uh to the point that we have argo projects So an argo project is a resource that says these argo apps are only allowed to deploy From this git repo. So you put kind of a governance Right, so if I if if the apps team tried to create a cluster role Um, that is defined over here. Um It will say no no no no like you're not in infra during apps So have some type of control of like in the project. So in the argo project you can do kind of like a Wireless allow list saying I'm allowing Argo apps associated with this project only to deploy to these namespaces and only to deploy from these git repos And the same thing for services like only from the services Library and only these apps. So if you um want to make a change You will be making a change for the whole team across all the clusters So that's where I came with the With the idea of um creating kind of like for development purposes I'm I'm developing this architecture. Um, and it's open source. So this is open source on their project That I work on is called the cloud native toolkit Um, so these I'm creating patterns and best practices on how to do DevOps with git ops And how to structure this in a way that can scale at a large corporations So, um I'm not I'm going I'm using this as a developer of git ops. I don't if that has the role The person that develops uh git ops and and DevOps for other things to use But the idea is like this this is our um, our I'm doing automation Setting them up, but then I want to change maybe I want to tweak or debug something in one Argo apps For example, this one So if I change the the yaml for here to point to a fork of my repo that I have like I'm changing something here um Argo will complain saying this uh Argo app is not allowed to deploy on my fork Because it's not in the Argo project and then um, I want to change Sort of like one thing. I don't want to change the whole thing So I need to change now and then the other thing is like if I try to change this This Argo app would revert it because this is the parent So I need to change this one and this one and when I change this one gets reverted Because the definition of this one is controlled by this Argo app So if I want to do like development on something that I say is is running as as desired in development And I want to make a change just on my fork It's because very difficult of like Argo wants to like the state of truth To be in git so I need to change this I need to change this I need to change the Argo app to add my repo Change this one change this one and then change this one on the cluster And and do it every time that I have a new environment because I bring out clusters like, you know Every day I bring a pretty good cluster start them down Winging that up. So what I did one quick point. Sorry, Carlos is uh, just emphasizing on the point of this is something A use case on development because in production You want to follow that proper procedure of making git changes and going the entire route You wouldn't make direct changes on the cluster This is something what Carlos is mentioning that if you have large set of clusters You want to check something uh using your own personal branch or something on a specific cluster on resources You don't probably have the time or effort of going through the entire change And it's not even intended So how you're using webhooks to make change only for your development uh environment Yep, so um, I had this uh, grand dishes uh, bad idea of creating a mutation webhook and learning how to create one So I said, um, what about if I create a mutation webhook that every time, you know One of these Argo apps gets updated. I overwrite it with my fork The project and the and the Argo app to point to my uh, thing so nothing gets Reverted so I created this mutation webhook That I have in this other people that we can share On the chat, uh, call the git override Uh, if I can find it here And this is one I was explaining that I want to listen to Argo apps and Argo projects Um, and I want to modify them on the fly. So I write a little bit of so I made a configurable So this deployment webhook Will load a config map. So you as a user, um, so Let me see how this Argo apps will look like. Um, and maybe let me do this thing of GitHub now. I know people, uh Tell me lately about this, uh, you feature pressing the dot. Yeah pressing the dot The pressing the dot thing is mind blowing, but yes yeah so For testing I have a testing folder here and it's show kind of like the example what a department that I'm trying to fix So if I have an Argo project That says I want to only allowed, um, you know, um Argo apps to deploy from this git repo So if I try to create my fork or you try to your fork The Argo project will say like you cannot deploy because you you're only allowed to deploy from this git repo So this is uh, some people ask me like what is the usefulness of Argo projects like Argo projects are very useful Uh, a lot of people use the default one and then like well, I got something working. I don't care, but they are very useful Um, they're also they're like you can only deploy to these namespaces right to more governance and then, um, there's a default set of resources you can deploy but Uh, uh, you can add even more But for this example is like I want to override this this line of source repo on the fly Um, and this is the Argo app parent So this this one points to the same repo on this folder called the test child So this is the parent Argo app Uh that deploys this shout app um And I want to override the the shout app to point to my to my own fork I mean and even a different uh in a different branch So to do that, I made it like configurable one one validation webhook configurable for my team For development purposes and I told them as far for automation We deployed this webhook and uh, you based on the your personal fork. So Carlos to increase Um, you will create the config map in your namespace Or in the cluster that you have a all of us after The own cluster by using the same repo and a question on this Who can create this config map because this config map would allow you to deploy your own personal fork So, uh, would you need admin privilege to create that config map? So this this config map would need to live in the same place where this deployment please So you can you can create this deployment in your namespace You don't have to have admin to create a deployment And create the config map what you need admin access is the webhook. Remember like the webhook is like the admin configuration Yeah, uh, right points. It doesn't matter that much. Um, but yeah the the config map So my team will create this as when they get a big cluster like each of us We have so many clusters for testing like we deploy cluster and you're working with your own cluster to do your changes Right to validate that you make the change to debug around Um, and then you have Your that I say like I don't want to use main or maybe you're using the same get repo and you want to use main Uh, you want to use a fork? So I put just fork here. So, uh, the the config map is basically this upstream URL That maybe is used by carlos Um, I want to like change it to my fork Um, and then the branch to my fork branch And I put this get URL Uh, this config mapping there my code Um, not my code, but like some some little example code We'll do the checking saying if if it's an application Uh object, I'm going to check for the the source and and the repo URL And if I have a match, I'm going to reply back. Remember I said like you you give back the api json patches So I'm going to patch the resource mutated Uh with saying no, uh replace the repo URL with my original URL the upstream and replace also the the branch and and if it's a project go ahead and replace it um The the repo source URL right and replace it with my my uh, the original one the original repo URL So that way, uh, you respond back with this uh small um A mission review, but and and that's what we talk about validation. So validation only returned allowed Um and something like this. So this is a mutation weapon I can also pass a patch type and a patch So I'm giving back like hey mutate the argo app and the argo project with my fork URLs Um, and then send it back. Um, so this code can be written in python java Go Different operators written in go that will use a call the go client from Kubernetes So have strong types to the objects that you want to create The objects of the crds But in my case was a simple like json objects that I I'm not doing anything with crds I'm not doing operators and I was able to to do this Uh and patch The the definition that gets into xcd into the cluster But the problem that I then that you will suffer is Uh, what happens if I go notice That that doesn't match the source of truth In the git repo doesn't match the one that is in the cluster. What is the typical thing that argo does? It it reconciles Say it says like no, no, that's not the the the source of truth. I want you to be the the one in git The app goes out of sync Yeah, and it goes out of sync. I was like, ah So, how do I configure argo to say like ignore this small amount of fields, right? Um, so basically what you do is you you configure argo So I have it in the read me here as one of the things that you do um is You configure the argo config map, uh, but in in case of open shift We have an operator called the open shift get ups operator And you will configure a the the instance that operand of argo cd And then use you add this to the spec section in there saying Hey, if if if it's um, I'm going to customize and for applications and projects Please ignore these fields in those like take into account everything but ignore the things that I'm like changing And this way argo will not try to revert you back To the source of truth And and that's that's the way we We deploy we apply this to the argo cd We create the config map to your fork And then we deploy the the deployments, uh, which I have in quake Uh, the image the yaml file Um, and they create the weapon.  Um, uh to activate it That that was like the main namespace selector, uh, deputy Any questions or so far so good so far so good I was told that this was going to be a validation and mission a discussion so I think it's an open discussion because uh, it's developer experience office hour So we're not limited to any particular agenda. It's a free flowing discussion Especially since we have only nine minutes remaining you mentioned about Cicd best practices with using our city and tecton I'm really intrigued to learn more about that and maybe if we can cover it within nine minutes Maybe point us to that cloud native toolkit Uh site where our audience could could learn more Yeah, so um one thing that I did, uh, so let's go and cover one of those two things. Uh, the first one is um best practices on get ups on this type of thing. So the version that I had um One point and the links. I'm going to be in the show notes is this other one that I have here Uh, the cloud native toolkit. This is a template. Um, so if we do This let's do the github. I'm getting addicted to this um Keep without everywhere. Yeah, the old version onto last night, right? I talk about developers like Having a deadline. Um I had this version that was you seen Uh customize. Um, so I had here. I don't know if this is big enough But this is the kind of the framework that I'm using. Um, this is the main repo that I was talking about get ups Um, I have like the different Profiles where you have like a shared cluster and each cluster would have their own Way of deploying. So I have using customize to apply the the services or like a customized to apply like I want I want, um Grafana refactories, you know the service k native All the things that I want to deploy just on common this and we customize I can deploy The problem that we had was we had so many argo apps That all of them had this section here Source, so this is kind of the thing that was fixing right and the next argo app has this thing here And the next argo app has this thing here So we wrote a little script to mutate all the argo apps with the fork that you were doing And it was kind of like leaving the the github 30. So if you wanted to do a pull request back was like all these changes Uh, we also had the project, right? So we had the project in here with these two things only these two things so, um Last night I did a an update and put into master With a better option of like actually using customized properly because i'm I'm learning all this cognitive as we go. So if you use customize properly, uh, I reduce I reduce it to to this. So now if we take the the single cluster the customization Now i'm using patches in customize. So what i'm doing here is The argo the argo projects no longer have The github url i'm patching. So i'm using a feature in customers called patches With a target. So i'm targeting application argo applications with the label Getups and i'm going to add The repo url and the target revision i'm going to target up app projects and i'm going to If they're infra they get the infra url if their services they get service url and they're up So i'm tagging the the resources with the right thing Um, and for example, if you go now to the instances There's no there's no source url. So Actually the editor will tell you that you have a bad argo But the thing is i want i don't want people to start to continue putting URLs in their argo apps. Basically, they're going to go into the customization File in here at the bottom and this is the new section. So now you go to one file To update the git url under branch. So you put your branch in here saying Main and then in here you put um https github.com slash my fork such github's Uh, this is services right this is tagged services And that's it. It will take all the argo apps when you run customized from argo This is not manually like argo supports customized It would apply this and then all the argo apps will have this same information. So you avoid duplication. You avoid error um And there's a little script in here that you run destruction say to run this and basically it will take The difference all these four repos and put them in the right place for the right argo apps in the argo project. So um, that's that's the thing that i was telling you like i have a better way now To do the kind of the git override with you know proper customized and this is kind of the the new version of the framework um For tecton let's see tecton and argo together. So I have an app in here. I have a repo I have this This example running on it on a cluster. Um, so I have my my repos here and I have uh these um This app so the idea is that I'm going to have uh different deployments. Um um So if I modified so if I modified this file, for example, let me do the dot again Go back to the dot So let's say the app thing One of the app things are going to make a change in their The docker file or the application that they have. So this is a customer details app Um, and I make a change to the docker file And I save it say like remove dmv full bar and and I apply it and um Actually it's already applied. So I apply it. I go to argo Which I have here um, oh no, sorry, uh, I go to The pipeline so you can see here. I have a tecton pipeline that kicked off This pipeline what it does is Builds the image. Maybe those docker lanes like things that tecton are good at Uh, check your yaml file do scanning. I remove the ones we in the toolkit We have a long one, but this one is shorter for demo purposes So it's going to build the image and everything but the trick is this is the ce in a space Nothing runs in here just tecton and the the last step. Um, if we see one of the previous runs uh No, if we see that the previous loans from uh Let's see uh promotes Let me see this one. So I think I guess this one. Let's look at this one So basically what this does is builds an image puts it in a registry There's open shift and then the last step it goes into the The repository of the github's repo That defines the dev image and updates the image Then what's happening with that? So if in a github uh repo gets updated The repo tells argl that there's a change or argl checks every four minutes, but um, then argl will Deploy the new version into dev. So you will see the new version in dev Um, so if we these are all the argl apps, so if we go to applications And we go to dev and we go here Argo will deploy the application in dev And then the next thing that we want to do is running another tecton pipeline when the application is running in in in dev And that tecton pipeline will do a functional test We run something like newman we run like integration test So with the way we do that with argl is we add a pulse sync job And that pulse sync job is located here this job that you see here So I think since we're almost few seconds away from the from the from the end line We can we can resume probably in a in a separate session. I think for the audience. We have all the show notes in the chat and I think most of will be more than happy to invite carlos over For another session But I wanted to thank carlos again for coming to to developer experience office hours And thanks chris for hosting the show and for the audience for tuning in Yeah, thank you chris and there for the invitation Yeah, it's great having you carlos and thank you dawan for bringing them on Up next is the one and only open shift commons briefing We'll be talking about cloud native disaster recovery for stateful workloads So stay tuned because very good will happen to you All right folks, if I don't see you again, stay safe out there and uh catch you next time yep


In this episode, we'll be talking about why you need to build service design communities. We'll be talking about how to build service design capacity. And finally, we'll also share the secret to getting organizational leadership on board of service design projects. And here are the guests of this episode. Hello, I'm Hezer. Hello, I'm Mike. And welcome to the Service Design Show. Hi guys, my name is Marc Fontaine and welcome to a brand new episode of the Service Design Show. This show is all about helping you to build more human-centered services. And we know that this is important but also can be a bit of a struggle sometimes. So the way we do it on the show is by listening to some of the world's best service designers and studying their success. We talk about topics ranging from design thinking and customer experience to organizational change and creative leadership. If these are the topics you're interested in, know that we bring a fresh new episode every two weeks on Thursday. If you don't want to miss anything, click that subscribe button. And if you enjoyed this episode, click that like button. And it lets me know that the things we do here on the show are appreciated by people like you. My guests in this episode are two service design pioneers. Hazel White and Mike Press. Hazel and Mike run a design consultancy firm called Open Change which is based in Dundee, Scotland. For the next 30 minutes or so, Hazel and Mike will be talking about three topics. Why you need to build service design communities, how to effectively build service design capacity in organizations and the secret to getting organizational leadership on board of service design projects. If you want to fast forward to one of these topics, check out the episode guide down below in the description or just stick around and enjoy the whole episode. For now, let's jump right in. Welcome to the show, Mike and Hazel. Hi. Hi there. Awesome to have you on the show from Dundee, right, in your new studio. Yes, great to be here. Thanks for inviting us. Yes, fantastic. And yeah, we're here in this new studio right in the centre of the city. So we're kind of excited about that and we're very excited about being on the show. Super excited to have you. So I'm going to ask you the question that I've been asking for over 30 episodes and that is the first time you met with service design. Do you remember that moment? Yes. It's interesting as to whether you call it service design. I used to run a design course, a master's design course. And the world began to change in about 2006. And I read a paper from the Design Council in the UK called Transformation Design by Colin Burns and others. And also the book in the bubble by John Thacker and it presented this whole different way of how design could interact with the world. And so for me, although it wasn't called service design, I didn't know it was service design. That was the beginning of thinking how instead of just designing products, graphics, et cetera, we could use design for good. So yeah, 2006 for me. 1993 for me. And I could almost remember the day that I came across it. A book had come out. I was teaching design management actually in the University of England. And this book had come out called Token Design by Bill and Jill Hollins, managing design in the service sector. And I thought, this is fascinating. You know, I know about design management in consumer electronics or product design and environment, et cetera, et cetera. Service sector. Interesting. So I got the book and then I was fortunate enough to meet Bill about 18 months later at the conference and just found the conversations that we had fascinating. And then that idea lay a little dormant for a few years and then we started doing work on design against crime. And that was looking at how you design around social policy and service design. But especially the thing that really kickstarted it is when Hazel had that encounter with that whole literature around service design, then the two of us started thinking a lot more seriously about it. Interesting. 1993, that goes a long way back. Yeah, yeah. But a really important thing about it was the students who were on the master's course, I was teaching in design, they were getting interested in this. So it was people like Lauren Curry who were beginning to explore themselves into service design that actually began to teach us about that. And I think that's a real privilege actually as a teacher is when you start to learn things and get introduced to new things by your students. That was the first time when I asked a student a question which was, you know, what do you want to do in the future? And Lauren said, well, I want to be a service designer. And I thought, that's a new one, that's a new one. I've not heard that before. Of course, she goes on with Sarah Drummond to start this really exciting consultancy. That was the great thing actually about being teachers is that all the time you're being inspired not only by the books that you read and the ideas around you, but also what the students are doing and achieving and what their dreams and aspirations are. So there was something happening in Dundee from about 2006-2007 when lots of different things came together. So an exciting time. I remember Lauren actually being, I think, here in Utrecht on the Netherlands for her graduation project. So she visited us back then. It must have been 2006-2007, something like that. Awesome. Let's move on to the topics that you shared with me and use the time that we have to talk about them. I'll use them in a random order. So I'll surprise you a bit, right? A big sheet of paper. Let's start with this one. I'll pick the first topic and the first topic is community. Do you have a question starter that goes along with this one? Okay. Why do we need to build communities in service design? A good question, Hager. Why do we need to build communities in service design? Because that's how you get things done. That's how you make connections and that's how you bring a whole group of people together and make change happen. And actually, it can be quite random how that community starts. Let me give you an example. One person who is actually fortunate enough to have as an associate of ours, Rob Manton, is a consultant surgeon, an Ian knows and throat surgeon at the local hospital. He deals with throat cancer. That's his special thing. I remember meeting Rob over a coffee one afternoon and someone had said, you've probably got a lot in common. And we had this fascinating conversation about innovation, about design, about what was happening in healthcare, what was happening in hospitals. And we thought there's something here, there's something that we could do. So we did an experimental event one evening in a local bar because that's always a good place to kick things off. We hired this bar for two hours. We attracted about 40 people, all kinds of people. Some were students, some were doctors, some were business people, et cetera, et cetera. And basically, we were trying to get people to redesign health services.  That went very well. We did a second event. Then Rob said to the two of us, well, why don't you come into the hospital and do a one-day event? Can you share your expertise, your service design expertise with my colleagues? So we had about 35, 40 people, quite a mixed bag. You had consultant surgeons, you had nurses, the hospital administrators and managers, a couple of medical students, et cetera. And we were getting the medics really to walk in the shoes of patients, to walk around the hospital, to go into parts of the hospital that they weren't familiar with. So you get pediatricians to walk into the fracture clinic, for example. And to look at what was going on from patient point of view and to try and understand and analyze that experience and bring them back and introduce them to a range of service design methods to get them to think differently. And since then, we've done a whole range of other things. And we built up, I think, this kind of design, literal, inquiring, critical community of health and social care workers. And just to build on that, one of the people who came along to that event, who's a health service manager, has taken that work forward and has done hospital walk-arounds herself with taking adult learners and shadowing them around the hospital to see where the barriers and what the issues are for them. So people actually building their own community of service design, literate people within the health service. And that's really important so that it begins to get joined up and people are using the same language and it grows. So that's a really important part of having the community there. So are communities in your eyes essential for making service design sustainable? I think absolutely because otherwise it becomes here's some new people come in, a different bunch of consultants with new ideas and they came in and they did something and it seemed quite interesting at the time and then they left and actually we don't really know what happened. I think it's absolutely crucial that you build capacity for people to be able to continue working in that way themselves for it to be sustainable. So one question, Mike, on this topic. If you had to give us advice on one or two things that will help us to build communities, one or two things, what would you say? I think provide opportunities for people to randomly meet up. I mean we're very fortunate in Dundee because we've got people who are absolutely committed to building communities and putting on events and sharing their expertise and so forth. And we've said this in other contexts. You can do big things. You can make big changes in a small city. You can make big changes in a small country. We're a small city in a small country and service design is beginning actually to have quite a significant impact not just in Dundee but also in Scotland. And I think the size is an important thing here. So figuring out how do you pull people together and how do you provide them with opportunities that are engaging that give them things to do but also are not parochial. We don't want to see just change happening in Dundee in isolation from elsewhere. One of the most powerful things that we've done over the last year and it was an experiment. We didn't know whether we'd not never done it before. We'd never been involved in it before. We didn't know it would work. We decided to work with colleagues in developing GovChan. Adam and Marcus's Global GovChan will do it in Dundee. We'll see who turns up. That's our handbook for it and the branding for it. 80 people turned up. 80 people from all over the place from different councils from health boards, educators, business people, people who worked at social enterprise. Big under. The number of things that have spun out of GovChan, the number of things we've been asked to do, the number of autonomous projects that have come out have been remarkable. I think Adam was very surprised when we were the biggest attended GovChan on the planet and, you know, okay, it's not our competition but we are a small city in a small country and it made us very proud that you can actually do things but the important thing was that we were twinned with Istanbul. We made links with Berlin and people could see they were part of something bigger. Immunity is important on a local level but especially as Adam and Marcus have demonstrated it has to be global as well. Right. We're moving on to the second topic because there is so much to discuss. Communities, awesome. I love that one. I think the second topic, I'll make it easy for you. This one probably builds upon the community topic and this is capacity building. We have a question starter that goes along with this one. Okay, the question, I'm going to ask Hazel this question, how can we build capacity? Okay, well that takes me back to thinking about when we ran the Masters course because in the two years that we actually called it a design for services Masters course we had these fantastic students that came from all over the world. And in fact, it was so fantastic it had its own use paper. Awesome. Yeah, Lina who now works in Riyadh. So these students come from all over the world and in two years we had 35 students who came and did the course, one year Masters course and we always did live projects with businesses with local government, with healthcare, with education with social enterprises and they all went off to do really, really interesting things but the organisations that we've been working with we've come back to us and say we need service designers to work with us and these students were all over the globe by now and I realise that actually a more efficient way of helping organisations around us to build capacity was actually to go in and work with people who were in the organisations already so doctors and nurses in the NHS people who are running local government people who are running social enterprises who understood their organisational context they didn't need to have to try and understand how the National Health Service runs or how local government operates, they know that to an extent, they don't know the whole thing but to teach them tools and methods of service design is relatively quick and they can go in and make change happen really, really quickly so that's what I decided to do so I set up Open Change to do exactly that and I would reckon in the first two years of setting it up in contrast to the 35 students I'd had in two years of MasterCourt I think we've probably reached 2,000 people in the next two years, I mean we teach 120 people a year on the Queen's Young Leaders programme that we run through Cambridge University and that's young people from all over the world from Canada to Vanuatu who are using service design techniques on their own local projects so actually, it's flipping the system a bit so that actually you are taking service design to people that need it or service design training to people that need it rather than asking them to come to you for a long period of time so that seems a really important way of building capacity and then lately... Well actually, if we look a little bit harassed and tired it's because next Friday we're doing a session with 400 people which is quite a big gig for us, quite honestly but we've actually figured out there is a way of taking 400 people in one day exposing them to a range of different methods giving them an immersive experience challenging and changing their mindsets well, we'll tell you at the end of next Friday whether it works or not but it's that thing about doing things differently not educating people over a 12 month period in a fairly relaxed way but doing things in a much more intensive and engaging way so for example, one of the recent things that we did which was with, in fact, our City Council here at Dundee they set up something called the City of Design Academy we are the UK's first UNESCO City of Design and they decided that service design should infuse everything that the council does that design is defined in Dundee isn't just about objects or environments important though there's our own design but it's how do you, in a challenging post-industrial city how do you make design work for all the people and that's what the council is trying to do so the first thing we've got to make sure is that all the necessary people in the council understand what design is and begin to use it in their work so we've been doing a series of pilot workshops and projects with people we started off looking at the quite specific challenge of how do schools communicate with parents and you had a range of about 20 people from different council departments, different school departments people who were language support staff in schools all working together, working across departments learning the basic methods of service design and addressing these very specific problems but really importantly going out and actually finding out what the real problems are because the communication between schools and parents it can sometimes be the simplest thing so for example if a school, you know if a school wants to get in contact with a family because their child's not attending school it comes up when the parent's phone has a number withheld you know you can't see who's phoning so they don't know whether it's a marketing agency trying to phone them and actually getting that change so as it says that it's coming from the school means that the parent's more likely to answer the phone because they'll not get the calls back so really simple things to more complex system-wide things but getting the people who are actually going to be delivering those things involved in the solutions as well So one question about this topic if we look back at how you were building capacity five years ago that was maybe getting people in a one-year master programme training them now you're going out to the people to the place where they actually need service design how do you see this progressing in the next three to five years will it be any different or is this something you will continue to do the same way? We've got a cunning plan and this came out of a collaboration with the local further education college Dundee and Angus College who asked us to do the several things in the city council could you come in train our staff in service design we want to improve the student journey improve the quality of the experience of students etc so we did that and then they said the service design thing is fascinating it's great and why don't we as a college begin to offer this to other people so Dundee and Angus College together with Open Change we have a collaborative project called the Service Design Academy which is sda.ac.uk and from this October we will be providing accredited short courses in service design that will be provided online and basically we see this evolving over the course of the next three years to begin with it will be kind of some degree level there's lots and lots of people who maybe just want to have a qualification to go alongside whatever qualification they have to help their promotion at work to help them do their jobs better and that's what we're going to kick off doing then maybe some people actually will postgraduate qualification to build on skills that they already have so we see this is evolving but also it's an opportunity for us to do the events so it's not all going to be online there's going to be a kind of whole mix of different stuff so in a sense you know watch the space we're really excited about how we can work with this college and provide something in service design training and education that has not been provided anywhere else and we're just really excited about that prospect Interesting also in this space you know about the service design academy that you're doing and it just follows along with the idea that I've been having for the service design show university and I think more and more knowledge transfer will be done online or in a blended learning environment so it's really cool to see what the results will be in a year I think those of us across the world who have an interest in this we need to collaborate we need to share content we need to see how we're starting off with something that is very tailored and very specific and I know that there are people in the Netherlands in the United States elsewhere meeting the needs of other markets so we've got to work together and figure out a whole range of diverse, flexible different kinds of experiences and modes of delivering the super wide range of students So essentially we're service designing as we go along but it will adapt to what people actually what people need Topic number three and maybe most interesting one we'll see it's called leadership Okay, so Mike what if leadership was different? What if leadership was different? Part of what we're trying to do in Open Change is to get leadership in organizations to think about that differently We were talking to you earlier about the City of Design Academy we've been running with the City Council and that came out to another event that we did where we were asked to spend a whole day with around 60 to 70 senior managers in the City Council and introduce them to service design and also to get them to think about how they can begin to design the services that they provide to citizens differently So we spent very nice couple of hours with a couple of guest speakers in and we took them through a basic introduction to service design It was a day in June outside it was raining and then we said to everyone okay, you've got an hour we want you to go out there onto the streets of Dundee and talk to people talk to people on the streets about their life, their experiences and their experience in particular of City Council services and you can see all the blood drained from their faces because they're senior policy makers they don't get out much because there's this a desk is a really dangerous place to understand the world and I think all organisations are like this we tend to sit behind desks and we try and look at the world through our laptops and through meetings with people that we're familiar with so most people are not experienced with going out and talking to people so anyway, they all went out we were kind of dreading what it was going to be like when they came back an hour later with smiles on their faces one of the senior directors in the council she left the building she walked across the road to the railway station she saw a homeless man sitting on the pavement on the sidewalk and sat down next to him and got his life story found out what happened to him, where things got wrong why was he there what were his hopes and aspirations for the future what would make his life better what did he think about the city council what did he see in terms of the future for Dundee when she came back, she told his story and that story then fed into other work that they did it's about changing people's mindsets it's about trying to get leaders to think in different ways to get closer to the people that they work with and closer to the people that they serve and that's actually infused itself throughout the council and I'm sitting around the city because one of the things that's actually been a big catalyst in the city as well as the UNESCO City of Design is the Victorian Albert Museum are building a massive new iconic building in Dundee so everybody's kind of thinking about what is design and what can it be so these leaders are thinking about it from a social aspect how can it change so someone else from local government in Dundee asked us to work with them about how do we develop the young workforce what are the barriers and the difficulties for young people who haven't perhaps got academic qualifications or not intending to go to college or further education to get into work with organisations and so we ran a series of events for them where we brought in young people who were on their journey to work or who had just recently got modern apprenticeships and got the council staff to interview them about their experiences, how did they find out about positions what was the selection process like and they found out really quickly that they were using processes and systems that they were using for graduates to interview people who weren't graduates and so they weren't able to demonstrate their strengths, it was off-putting even how they found out about these opportunities often by chance because schools weren't promoting these opportunities and so the young people then worked with the council staff to co-design how that could work better but going back to our previous topic about leadership, one of the really important things was we had the very senior people from the council in the room with these young people and after one of them had interviewed one of the young people and heard the issues and the barriers that he'd had working with his organisation he got out of his chair and he walked out of his room and said I'm going to sort that out now so it's having people that make change happen but is that also the secret sauce that changing minds only happens when people go through experiences yeah I think so I think so because you can hear people's stories, you can watch stories on video, you can see the statistics of something but sitting down with someone who is flesh and blood right opposite you and hearing their story is just a visceral human experience and it makes you want to make change happen awesome, awesome this is your chance to ask the people who are watching or listening this episode and that number is growing I'm really happy with that it's your chance to ask them a question or maybe two I suppose we've mentioned the service design academy that we're getting off the ground we've done in Angus College a bit earlier, as Hazel said it's an experiment and we're designing it as we go so what could the SDA do for you what would you want in terms of education and training in service design what could you be provided with but you're not being provided with at the moment that's my question and my question would be because a lot of what we do is front end the inspirational piece going into organisations and saying this is the value that service design can give you, connecting you with your users I'd like to ask the audience what is the thing that would help you inspire people and help them understand what the value of service design is to their organisations what could we do to help with that so how can we help other people understand the value and the benefits of service design how can we inspire them how can we make them curious, maybe curiosity for me is the right word here our time has flown by it's time to wrap up this episode so thanks for your time Mike and Hazel it was an honor to have you on the show it's been an absolute pleasure thanks for having us thank you, it's been a pleasure so what is your biggest insight based on the things we just discussed with Mike and Hazel share your thoughts and ideas in the comments and remember more people like you are watching these episodes in your comment but just be the thing that helps someone else if you are interested in learning more check out some of the past episodes or head over to learn.servicesignshow.com where we offer courses that dig deeper into the topics we talk about on the show I'll see you in two weeks time with a brand new episode thanks for watching and I'll see you then