good afternoon come on in grab a seat if you sit close to the front we will be nicer to you i like when people are close to me um we're starting a few minutes late because we wanted to give people a chance but here we are welcome to all in the dysfunctional family i'm lee i'm David uh ben was supposed to be here with me and the title is this fall actually i think the title is my fault but that's neither here nor there um so yeah today we're going to be talking about just the functioning of a team and how we can kind of work to make that better in all ways and some of what we're going to be relying on is the experience that i had i've had over the last 40 years um working with teams the entire teams uh leading at my time and especially our first experience working together on a team that existed for four years ago three years ago uh bring that time period and the the art we had and taking that team from being quite dysfunctional to a high performing team and some most of our comments are based on that history hi if we're going to do that slide that that slide from um Patrick and tony uh quote um we've been like a lot on five assumptions of a team this book and his worksheet and you're going to hear a lot of parody of his ideas but this idea to us um everyone we're working with open source software anyone can say they're a group of developer anyone can compete against you and goes anyone's plate off and use prices there competitive advantage um and so when you're not going to use prices competitive advantage you need to come up with something exactly valuable to the client demonstrate and that will give you a competitive advantage over people who are always on people on price otherwise business that's an asset and so this is this was also part this is Ben's part of let's make a business work and my part was let's make a team work and we put them together but we recognize competitive advantage is good for both of us it just occurred to me that we actually don't have anything right um okay so today what we're going to go through our introductions David and I'll give you a little bit more about who we are uh there's a disclaimer that we want to have clarified so that you're aware of a couple of things uh and then we're going to discuss the problem that we were facing and the model that was used to kind of overcome it uh and the experience of having done so and what actually happened when we applied that model to that problem uh and if there's time at the end we'll do a really quick kind of example workshop it'll take maybe at most five minutes probably not even that and then hopefully there's lots of time for a Q&A as well um before we begin and do the introductions can you guys all hear both David and myself just fine okay good I wanted to make sure of that so I'm Lee and as David said I'm filling in for Ben who's our one of our director of partner services managing client services there it is uh and so he comes from a really kind of particular business angle as David was indicating I don't as much uh I have a complicated fun background of mostly doing a lot of team building and working with youth and that kind of thing until I finally moved into the digital world I am not a Drupal developer sorry about that uh but what I am is really really dedicated to team building and to finding uh community and sort of ways to incorporate a much more what I like to call open source but in terms of people way of working together and finding those kind of calming grounds um I worked at a summer camp for 100,000 years I'm aging very well uh and that was obviously a big part of it there when you're working with kids who are coming from all over and trying to get them to do stuff that maybe you're putting up a fight about because the banana popsicle wasn't cold enough or whatever uh you do have to get kind of creative and start to find a way to really connect with people in that way so that was kind of important to a lot of my early career development and now at my planet where we work uh I'm the content and community manager so again my job is really kind of reliant on being able to forge those bonds and be able to connect with the Drupal developers and our designers and let them know that I'm on their team and that we're going to work together to be able to create the content that we can use for all kinds of things going forward so it's a big part of my day to day certainly the team aspects of things and I'm David Colburn um kind of a Drupal cycle since I know 2004 so I've joined my planet until four and a half years ago um when I joined my planet I wasn't brought in to work as a Drupal developer uh they liked the Drupal knowledge I had um but what they were looking for was a strong master my kind of sense of conception has been an agile company I'm just wondering how many people we are actually working in an agile method don't we have one sort or another I'm probably definitely going to be one I'm shocked I'm shocked so back then we had strong masters we were very uh a strict for a time period a very strict um strong process place uh we've since evolved from we like evolution um from that and we know what that is my initial role today I work with Ben who's not there not here today and somebody of me have heard ever to do that speak earlier um I've worked back and forth between a few of those guys um on time services and and still the teams have devoted to those still very strong thousand um and until recently I thought I was the oldest person in the company but a hundred years ago yeah yeah um I'm also a witch so there's a lot of elements to me that yeah maybe just number one I promise not to practice witchcraft on you or maybe I will this is just a very short disclaimer but I think like for those of you who are familiar with agile you know that the retrospective is a safe space so well we'll be making use of our experience and much of that experience came through retrospective discussion if I use a name it's not the name of anybody real if I if I use an event I'm not going to get specific enough that anyone could ever point out a specific person um I want to respect that safe space so sometimes I get a little muddy in my examples that's why okay so what was our problem when Ben and I first got together four years ago you know the problem was we had a dysfunctional theme we were not making our commitments we were running uh two weeks since at that time we really switched to one week as we did in part of our our 18th better name but we were in two weeks since when I first joined we started working with Ben and we were not we were not making any spring commitments and you know that's that's a sign of the theme that's not succeeding one of the reasons we weren't making any is we had a kind of impact on it all sorts of bugs that are leaving and we had to go back and fix it before we could move on um and our team retro and I can say we've paid a lot of a lot of retro uh at my planet um and there's a range of retro discussion in the state some retros with teams will have retros and they discuss what I call rearranging information that they actually don't accomplish much in terms of team growth but a lot of talking happens and a lot of points get raised and no action over occurs and no change or improvement in people's relationship ever the years because you're not talking about the actual problems you're talking about well should we have the room for this this is these are sort of examples should this room be bright brightly between tanning and treat them should we all promise not to be sardines for a lot of shipping and saying to just don't really seriously make a difference if you have a well worthy well worthy and well respectful team uh so retrospectives of that theme were leaving and totally so that was our problem team that wasn't delivering much and it was a long-term problem for a large plant trouble in river city trouble in river city um so I'm sure many of you have encountered similar things with teams uh to the symptoms listed up here a lot of what David was pointing out often express themselves in these ways things like inattention to detail which is how you end up with all these technical debt and bugs and all these problems uh inattention during meetings uh people are this is the one that drives me crazy and I do it even I was doing it to ever earlier typing away on their laptops not paying any attention at all just being a complete distracted person which was it you yeah okay it happens regularly uh unfortunately from all the information I've gathered um people not asking for help or having requests for help go get ignored or worst of all laughed at it's that's a symptom a team that can't actively engage with one another and just say hey buddy like I got a question and I need some help with it if you can't do that you've got a real problem on your hands um complete silence during meetings that one's also one that I really I find it very unnerving when someone asks a question and everyone just like as as though nothing happened it's a very very unsettling response no one asking any questions uh frequent defects mounting technical debt I'm sure there are others if you have other examples this is your moment shout it out tell us your story no no other see this is that silent thing so for you you have to like that but all of these things all of these kind of common symptoms common distractions stuff that just you know you felt it since early childhood when the teacher would call in the class and you know the one kid would say something sarcastic and roll their eyes and everyone else was silent like that's a pretty dysfunctional environment that's not a lot of productive stuff that's going on so all of these things are really really common so those who had a role and I probably would work here on the team um I've been affected by new slang he has been practicing though and three of us then myself in Everett again sat down over a period of days and like look at this list of of things that we recognize in our team in ourselves and and we had to have to become self-aware of our own feelings too as the people who had a role that was supposed to generate results in terms of becoming a better team um Everett and I for example may surprise you listening to me right now to hear that Everett and I have no problem talking and we'll do so for a long time back and forth even in the meeting of people um who also should be participating so we've got to stop talking and listen to the silence and let the silence percolate there's a strong need for self-examination by the people to facilitate the team processes and the team needs and the team growth to see where they were fading the team and the growth and the fine ways to help the teams to see and so there's a little bit personal discovery in order to have team discovery I'm not going to read any of these things up I love your readers I think or not or not in any case so really the point is that no team is perfect and and you have that arc of seeing teams approve that you're working on to experience that but there will always be issues in the team and that's why the process of resurrection and self-examination needs to continue and needs to be based on a strong foundation we were seeking that strong foundation and we think we kind of did let you on these words yeah um I think the most important thing about teams is they are made up of people and people are pretty imperfect as a species we don't quite have it figured out we're getting there though can't wait for AI to control membrane um so this was what Patrick Lindsey only sort of introduced in the book that they relied on and I don't know if any of you who've read the book before if you've seen something similar to this uh but these are the five dysfunctions of a team that he sort of identified and laid out and we found them really kind of productive and worthwhile to build off of so the absolute foundational base mark there is the absence of trust uh and that's the thing that means people aren't going to volunteer information it's the thing that means they'll make a snide remark instead of actually expressing how they feel it's the thing that causes them to disengage entirely so the absence of trust on a team is a really big it's why they don't answer questions or ask questions or offer help or they laugh at people asking for help I mean there's kind of everything spirals out from the absence of trust on a team here can you check on trust a bit and trust there's two types of trust I'm not hiding with the first type of trust you know the first type of trust is is my lending Lee Elunion knowing what I'm gonna back he will not be getting this back but I'm honest this type of trust is knowing that when Lee tells me she's not going to get a back that's indeed what's going on so if there's a deeper type of trust in human interaction it's more than just the social trust of yeah I'll show up that level of trust is simple and it's existence isn't an indication that there's a deep the deeper level of trust that we want to have in order to have good human connections in terms of all the other things that he's now going to keep moving on keeping me through so the next layer is the fear of conflict and unless you trust one another you're not going to be willing to challenge someone on what they're up to I don't believe that David's going to respect my opinion or is pulling in the same direction as me or understands my goals there's no way I'm going to say you know I don't agree with you on this because it's easier just not to disagree with something it's easier to just shut it down and then go to my own thing and damn whatever happens lack of commitment is the next one up so again that if you're not willing to challenge someone because you don't trust each other then you're probably not going to be engaged with what they're caring about and they're not engaged with what you care about and neither of you is committed to any of the goals that your team should have in common something important here to recognize too is the commitment issues and consensus right if you have trust and if you can have healthy productive arguments discussions and disagreements you can recognize that we don't all have to agree 100 percent on the course of action but you will trust that the group has made the best decision it can in that time and place and even if you disagree you will commit to that decision you won't remove yourself in the equation go okay I'm not watching you call it that because you trust and you had your chance to be heard and considered respectful so commitment is not the same as consensus welcome to our johnny and judy come lately's we're glad to have you I stole a little bit from David earlier that's kind of all you've missed avoidance of accountability is the next sort of not pillar layer of the pyramid so again it's a lot of passing the buck and then the final one is inattention to results and that one has to do a lot with kind of caring more about yourself and what you're going to be able to say then about how the team is doing on the whole so I'm more concerned about being able to push this channel to put up the world and write my little medium article and let everyone know I'm the super genius that figured it out as opposed to carrying that actually that future code is solving a problem for our projects and we're going to make a big difference for this project and for our team for having done it so it's a lot of ego and a lot of that kind of issues where that layer really starts to stumble and struggle this this is a big one for me I happen to be out of the school of thought that when you're working in teams as we do compensation and reward should be team-based and there should be no individual reward okay fine I'll give you the loony back please so so in inattention results is is found within the site that you're going after the golden apple and everyone else can call it by the wayside motivation systems in our businesses as we organize them still reward individual behavior and individuals mid max in their behavior for the playing game you got to look at that stuff okay another one of these slides David's not going to read maybe I'll read it remember teamwork begins by building trust and the only way to do that is to overcome our need for invulnerability well that's good that is a good one to read I was a walking in my neighborhood a few years ago I was a massive mural painted on a lot and it said honorable to be vulnerable and I thought that is how I want to live the next decade of my life I would like to embrace that mindset so this really speaks to me so this kind of problem yourself yeah back in the pre-100,000 era uh yeah so it's that again it's that trust that core foundational element opening yourself up and being willing to say I don't understand something I'm wrong about this whatever it is that invulnerability is no good for you you need to be vulnerable so that's kind of a big one so this also speaks to that self-awareness of an earlier slide in reference one of the things that Ben and and ever and myself did when we had no coming back discussion with being aware that the great hard things lack of success and dysfunction would lay in our our lives or how we were behaving and what we were expressing in calling the team and we just didn't keep that to ourselves and the next retrospective we said it about the team we exposed our own mistakes that we've been making our own behaviors of preventing the team forward and I'll talk more about this problem about how we did uh implementing this process because this is the part of it and this part the invulnerability part you can't do it once and the people have to begin are the people of leadership rules you have to show your your vulnerability you have to show your weaknesses you have to own them and you can't just do it once immediately can't just stand up here in this group and say you know sometimes i'm quick to make decisions and ignore what other people are saying um i then have to go out and start listening to people and they have to see me doing that we can we know so for us as we we we went out and bought coffees out there's a workbook goes along with Len she always fly discussions of the team we've got the workbook and it's each of the dysfunctions has a chapter and exercises all good stuff like that and we couldn't leave chapter one trust for five weeks six weeks you were stuck there we couldn't get past it we could never get anyone feeling that we moved anywhere part of that was because trust takes a long time to fill particularly when you've already had dysfunction pain if you're working with people this and brought together hey you guys are a team now introduce yourselves you're in a different situation you haven't built an ingrained dysfunctions we had tons of ingrained dysfunctions and um some of this is really really hard there's going to be a lot of pain we have to actually fire a lot of people um they they simply the more we the rest of us we're getting more open and honest and trusting each other the more thought and the more we thought of the more outrageous behavior and finally it wasn't one of then myself or ever who went to our director of technology and said this guy's got a goal was a team member private this guy's just gone he's got a goal can't be on this team anymore and director of input before I could even realize this guy was being sent home that's it um um it was amazing once that happened and the previous four weeks hard work and us going to our own values and our own limitations that the rest of the team began coming in jelly so we have and move into the next level which is conflict and I can remember our first conversation about conflict with one fellow said to another film two very excellent about this one I don't think you're working hard enough many of you have been in retrospectives how often have you heard someone say a developer say another developer I don't think you're working hard enough I don't think you're working straight enough I think you're holding back your team has really got to progress to have that set out loud and not totally cold and you'll just fight and it did what happened to that was really destructive to the dialogue where the two fellows realized that they were working in different time cycles and one guy was actually taking this work home and I don't like that I don't support working at all but that's what he was doing at the time and between them they managed to get past whatever damage if you were in a dysfunctional union say something about that relationship with history going down um they got past that and continued to be productive collaborators on that team that so without the trust you can't have a really really tough conversation both the interpersonal effort that somebody may be making on pain but also about how are we going to technically solve a problem that's in front of us and one of the signs that we had cracked it was the 10 months later we moved from the same client we moved on to different projects we got everything eight weeks into a technical solution for problems and we were following as there were our biggest time and time we were following their suggestion as to what the technical solution would be we were still the other company uh and we didn't push back enough and eight weeks into it it wasn't then or ever before when myself was said one of the developers the one who was called he had work wasn't very correct and he said you know what I had better idea I know how we should be doing this and he made a suggestion one over the rest of the team went to the client who because of the previous 10 months we had so improved and I'm so consistent and so reliable and delivering value to those trusted hots and they said okay we'll pay you guys to experiment with this new idea for two weeks we experimented with it for two weeks came up with a proof of concept inside for two weeks and they said we like this better we can sell this better this gives us better value throw out the previous eight weeks work we're paying for it add eight weeks to the contract and build this thing this way we couldn't have done that with the team we had a year before it was the same group of people so the payback to us and putting effort into the human relationship side of our teams was the ability to pivot on a project innovate on a project and client's trust was deep enough to pay us for eight weeks of work we were just doing that's the the competitive advantage of well functioning teams yeah that's my quotation David likes to see his name in light there's no name here though so we're about it on that does anybody have any questions up to this point how many tons of time oh jeez right more slowly no nothing so far okay well then we're gonna do the sample trust exercise because we have time okay so you got here just in time so the first questions are you willing to use what what look at those thumbs up i love them what will be involved is in self-selecting groups of four to six to eight people whatever you want but but why aren't you large because there's you put to explain three things about yourself to each other okay into three things you're going to explain are pretty simple it's almost like a stand-up right what did i do yesterday what am i going to do today what thoughts are you done so you don't have to go into anything else uh and you don't this is not a psycho analysis you don't have to call us too much about your mother just a little bit just a little bit but in your groups you're going to tell each other where you grew up you're going to tell each other how many kids were in your family and what was the most difficult or important challenge of your childhood and i'm going to give you a pro tip on if you were the one of the people in a leadership role and i was a facilitator of a retrospector so in my responsibility you go first so if someone wants to stated are we going to go first i'm just going to so i grew up in Ottawa and i came to town straight but that actually was the general contractor of the university uh that spiral uh ram was a balsam model i got into a day before it was coming how many kids five six one nine um what was the most difficult important challenge in my childhood getting over inside and fear of social um situations uh i almost failed kindergarten because of the science um and go for it um like yeah uh so that was the and it haunting all the way through until i went off to university through that so it was a huge barrier to my having an enjoyable life and uh yeah that was my most difficult challenge maybe once or twice um so i grew up in the what was the time of budding the problems now and again no it's still budding uh thank you very much for both very busy and very abandoned at the same time these days uh i'm a four kids which explains a lot about me subscribed to the way children pan out and the most difficult important challenge of my childhood was not overcoming Chinese uh but accepting that we thought that would not always be on me which i struggle it's a day um i had a really hard time accepting that things didn't go 100% my way as a kid and it was a hard thing to do you have 300 somethings you've got in that egg and go you're right it came up a lot so um again the idea is that this exercise doesn't immediately mean that anyone trusts them eating enough to take my word that i wouldn't complete a project say but to begin right and the rest of it has to be the behavior during the work week with each other and it just takes five or six weeks and you can turn that that around and doing these exercises these sort of exercises so if you'd like if you could self-select uh in the groups and just separate enough that you can talk to each other um as if we have this all one at a time we'll be here until oh five or six whatever feels comfortable and we'll we'll walk around but it needs to be sure to come on you sort of you it's pretty and willing to get out of their seats and chat you can do it yourself so a bunch of people who are who seem to be strangers or maybe it's kind of like maybe not all strangers um and that's just one I mean that's kind of your pretty step introductory first example exercise and you don't have to start with that one but it's kind of a good one for a new team but there's lots of really wonderful exercises I was actually speaking with Everett uh our colleague yesterday on the train up here and he was saying he really does know that he's trying to introduce Buck up Fridays into our weekly ops meeting because he thinks it's such a great way to establish I mean that doesn't acknowledgement system ultimately it's just a constant theory he's saying this is happening now this is happening now I don't have to adjust accordingly and if every single week everyone comes up into the room and says here's what I did wrong this week I really used to know this bad boy and someone else says the next thing that's what it gets to you you're like I didn't get that hard at all it's better having that kind of culture where you start to establish not only is it okay to fail but everyone does it sometimes and ultimately didn't we also get to Friday and everyone else still exists and it really starts to build up next month and it's a great way to build trust and in your leadership role you can be the first one to start but you can get that all on it's a big difference another great one from the club that they acknowledged what I thought was really great was having people frankly addressed that they're really good at what they're really bad and your teammates can challenge you on that date if you're playing it's really flexible and and I was like I don't know how to do that but I think I should rate it strange but you all have some other things and you want to have a talk with us what before we go to Q&A there's two other points just like the cover after the trust part we we moved into the conflict part and that's the second hardest part what we discovered was once you can craft those two parts with your team and your relationship the other parts are easy because now you can actually speak openly about problems now because you're working here in the day you can ask for help right so how many people here have experienced these themselves or co-workers staying in the day on a single problem okay I'm going to be dead in the business there now you're wasting our time if something troubles you for an hour you can't find a solution ask someone on your team and if there's no one you can even solve an estimate or that you know but staying in the hour about an entire day on a single problem all of us are going to do that all of us have done that and what it does is slow us down solution is really simple being vulnerable enough to say if I can get this I need help and every single one of us have been in a decision where we have to ask for help in each other every single one of us and when you're on a team where everyone recognizes that and respects that in each other you can ask for help to get it problems get solved faster sometimes better so the trust and the conflict part are the mind and the mind states the two things you need to really concentrate on the work of the five and three and eighty six so on we continue to have regular retrospectors one for spring the crimes of St. Elizabeth we're losing your team for an hour every week and we're going you're getting a better team and you know it's part of the deal um sometimes we have to find a program but we get other things to keep track of and fuck with Fridays is one idea but we also with that specific team we pay similar here uh red voter who ever used or experienced a happiness index so happiness is going to be really complicated it'll be really simple it's like estimating and estimating the scandal doesn't matter you're either happy you're not happy but you're very happy you tell me who you are and I don't know who's in your case right and say what very happy he's but all I need to know is you're very happy or you're not happy you ask this question every very specific every week where were you last week I was happy very this week not happy oh let's find out what's going on here right it's just an indicator it doesn't matter if my happy is the same if you're not happy what matters is change in your state so that's the happiness index and we started using that as a very quick way to pick up on changes in our central okay and those changes are important because you left unhappiness and truth and slide you down your team becomes less cohesive less less prepared to get the hard work done so those are some of the things we just started this week with this system let's create a team you work on what can be instructing to an ex-team anyways straight into the air program guys you prepare the coffee drink you can even ask about another air program and then unfortunately business requires a difficult team up in all these people and I've seen people in other teams but for a while they're writing a good message straight into the message and it's really this is the other thing I like to shut up right after this one here when your organization is totally out of the house so it's eating and straight people working in that job that they can't get at that time spreading people who learn good practices around that job and many of those aren't just the process but the importance of a human relationship once in a while so we spread people throughout the organization who have the skills and those experiences they would but they're the ones who won't pull it up for a bad adjunct experience on the team they're the ones who provide that team so uh hard sense is it Johnny at the seat the good seeds yeah okay too aggressive too much think you might have mentioned this but end to end how long the whole process take from starting to finish to really feel like your team was so the team had existed for it was like four or five months wasn't it before you have started to use an applied technique so the team had been brought together in December then I started working together on the team in April in in June or July to recognize what we've been trying to do wasn't working enough there was no decision to change and we started changing so July after the end of September and then and pick up in every way we and we're lucky to use internal team measurements like velocity as a measurement of productivity but they pick up in every way we can see we you can hear a happy team happy team laughs a non-happy team has headphones on that's just one in the theater doesn't always need the teams on a happy team but if you put together with everything else no one laughs they're all wearing headphones um there the number of teams that's in sprint to sprint is more than the number of tickets that were in the sprint any sort of measurement of the channel too are all indications of a team that's not happy kind of team is not doing well and my partner tells me she doesn't she makes ads out it's a little it's the word cultish I mean it's a little too much about happy and then using happiness to be productive but my darkness is what is waste I used to be the prettiest I wasn't always a developer or a site builder or or in the software world because of 19 same station I started working on my first job that well it did exist like that was one but it wasn't a big thing I was in training history for 25 years what was waste to us was if we made a mistake on the on the credit grass paper people find what's placed in the in the digital he that's he that's a waste right you gotta redo it anything and so having a team that collaborates well helps check each other's work reduces the number we say it reduces the number he has it allows him to have a client you know also the major key to quickly tap the defense and if you're finishing your sprints your strength and defense you've got time to be able to sprint you go oh these defense boom you go after it right before you start to have strength there are all sorts of things you gain head of the band you gain when you have a team that functions well and in my mind one of the things to gain a lot of internal topics and one of the ender job a lot of time to work every day many might years money has been retired they hate their jobs if they're still working today they hate their jobs if they're retired they hate their jobs I don't mind though I'm not going to stop working until I hate my job maybe a little bit but still I love my job I love and doing this job and to me that's what's very important from then inside what was what did then all about you stopped wasting time I'm sorry I was being productive and answering the client's needs and satisfying project requirements and getting something out of time and we weren't burning through the budget for no reason and we all bought more projects so I came back time again please can you give me your money that's that big story story oh my friend tells me that he says hey do you want my money and I say yes how did I get the money that's pretty good I just want to touch a little bit more on your question the because I think it's really important to identify with different teams operating this is very dysfunctional and ultimately they require all the members of the team to no longer be on that team and part of that as they discover through this process was he didn't actually want to be there he wasn't doing anything he wanted to be doing this was not his area because in terms of software development he didn't want to be like none of it so he's never going to be happy because it's a little bit hard to do software development I would be very happy uh so we've got to take different stuff for different teams and you'll find a way through that as you start to work on these processes and start to bring them in some teams to sort of you know good retrospectives like those who know what they're doing in this environment and then you're going to start to start off with the functionality right away and other teams you could be out for two months and eventually one or two people have to move on before you start to see the result so it does very much better did you feel like there's any regression like would you have to like carry the momentum out do you have to if you have trouble with this you can sit around the retrospectives and not let them turn into furniture just and it's not that every retrospective has to be deep gazing at your soul but we just it's around also being honest with me we had a group of fellows in our office this past several weeks learning we got excited and so and uh but I had to work with them on the first day doing some introductory games and getting to know you kind of things so they kept coming me with questions all these kind of things I did not think it was easy for them uh but they one of them had this formula for love that he was sharing with the group at one point and it obviously there is no formula for love really the formula looks like every variable has to align with his existence but what is the basic example I can use true for love for friendship for any relationship to be in your coworker space uh if you remember that's a very strange word anyway it's about spending time if you just spend a little bit of time and then you spend a bit more time and then you spend a bit more time you start to just bond kind of innate with people and that's why it is like having the headphones in and being in your little corner all the time it might be easier to do and I was like really concentrated putting that way it's a lot harder to build a connected relationship between us to collaborate on things to build that sense of trust so there is an element of just having those retrospectives and spending that hour hour and a half together once a week means you're spending time with your partner actually just acknowledging each other's existence in a way that kind of has to be doing in order to continue to see the results the book recommends like plot sites once a quarter twice a quarter whatever it is but that idea just has kind of taken that group and letting them be a group for a shot because they're really important to access other questions how do you feel about the little exercise it's just like you're moving the day or of course did you see any potential in your own last question yeah and it's not the kind of thing you can necessarily do because you know David and we're working with on day one so not going to be honest about it it's going to be a bristly thing it's hard to say it's hard to say i'm not it's something it's actually much harder i find to say i'm really excellent at this and being able to pinpoint and articulate how great you are at one thing if that feeling a bit like showboating in a way it's not nice so being able to do both of those like hard probably it's something you can do with a true baby information team but probably through that process it's a really good one um one other exercise we did um you know the first thing was to all take the Myers-Briggs thing and Myers-Briggs lost but let's put that right out there there's a ton of lost in Myers-Briggs but it's the exercise it's important not to resolve your Myers-Briggs thing we did the Myers-Briggs thing and each of us came up with our result and then we talked about it and you know every result was whatever it was and someone in the team said that's you they could have just been following you around for a month and written down how you became and that would be what you are on Myers-Briggs um and that's not you and oh you're on north and so we were expressing to each other what our strengths and weaknesses were so normally did when i was transferred i just went there but the team would tell me what they saw me actually do that and what they saw me so you was that 360 review it and uh me explaining me um tells us that suggest to us that forget top-down reviews those top-down 360 quarter views are useless and i agree uh i don't know i don't have experience so many of them that i can't tell you how terrible they are they're useful but we get to do peers and they shouldn't be quarterly they should be whatever you need it hello and those of things are important to those of things that they move who likes to have i have a question around what i did wrong this week what is you know x y z and then someone has an opinion on how i can do that better than i have i was in awe of just staying strictly getting to an area where your conversation has been hijacked for 25 minutes and you're waiting for friday so how do you keep the lean consistent and valuable so before you get into conversations about what they're wrong you have to get to the first point where the understanding is that any discussion of what they're wrong isn't about someone's personal it's about the team's collective desire to succeed and all we're looking for is the way to all of you succeed that's part of the team so the conversation isn't about all you should have done in a dcb the conversation should be about what do you think you should have done instead it does take and given that i'm not a trained facilitator i believe it doesn't it does take someone willing to be a facilitator it takes someone who that's ever been someone who does some training and is taken in and is willing to when you've got this strong answer to a good role to do this is willing to help nudge conversation back inside a forum to each one so i think it needs to be a facilitator if the team isn't at the point where they have that sort of conversation yeah i think probably for those specific examples i mean lay out right very clearly the whole group in advance all we're doing is stating the thing we're not going to get into the solution right now we just want to hear the things when we kind of have this be a place to hear it and then if that does start up having a point to facilitate or facilitate or a lot of conversations rotating right so facilitators so that it's not ever just one person who takes on the role of like dead mother like me everybody obeying my rules kind of thing and that that person who said i think i did i love good ideas maybe i should talk about it i thought i saw that now i'm great and we'll just now we're going to go on to mark say yeah we are so how do we get back from that from two tiny not public summer workout you figure it out i don't even know because he's like hey i'm the chattelist every once in a while he's like what we've met in the end big grandmothers i think it's the end of those hours from 12 to 2 we can chattel a lot of times we're in and out that's great we're going to be 10 to 12 per sign and 10 to 4 per sign yeah it's having us we're not going to be 10 to 4 hours in 4 hours anything else again i'm still in 10 to 4 hours 45 minutes until you can chattel anyway thank you guys so much so if we have any questions or we're on the further we're going to be in first spot so in the green room


Continuing to accumulate mass, its gravitational collapse causes the protostar's diameter to shrink significantly, and its core temperature rises to 5 million Kelvin. Some astronomers point out that this is hot enough for some hydrogen fusion in its core. This makes it a true star, but because it is still growing by accreting large amounts of material from its surroundings, it is not yet stable. Astronomers call these stars pre-main-sequence stars, or young stellar objects. During this phase, a strong solar wind forms pushing back on the gas and dust surrounding it. A typical transition for a star with three times the mass of our sun or less is through a T-Tari phase named after the star, T-Tari. The orange star at the center of this photograph is T-Tari. It's the prototype for the T-Tari class of variables. Here we see it surrounded by a dusty yellow cloud named the HINZ variable nebula. A typical characteristic of T-Tari stars are jets of high speed gas and dust streaming from both poles as strong magnetic fields guide matter from the star's circumstellar disk into the core. We can see these jets in star birth nebula. Here's a striking example from the Corayna nebula. You can see the jets at the top of this mystic mountain. Here's a view of a multiple star system called XZ-Tari. It's neighbor HL-Tari and V-1213-Tari, just 450 light years away. These young stellar objects are illuminating the entire region. XZ-Tari is actually a binary star system. It is expelling hot bubbles of gas into the surrounding space. Gas from an unseen disk around one or both of the stars is channeling through magnetic fields surrounding the binary system and forced out into space at nearly 540,000 kilometers per hour or 300,000 miles per hour. This outflow, which is only about 30 years old, extends nearly 96 billion kilometers or 60 billion miles from the star. Stars can remain in the T-Tari phase for as long as 100 million years and reach 10 million Kelvin at their cores. As their solar winds pick up, they disperse the remaining gas and dust around them back into the interstellar medium. This ends mass accumulation, and the star settles into hydrostatic equilibrium. At the mass of our sun, the core temperature reaches 15 million degrees Kelvin. And it has 99.8% of all the collapsing molecular cloud fragments matter, leaving only 0.2% left over for planets, moons, asteroids, and comets. It is now a main sequence star.


Hello everybody, so I guess we can start. Thanks for coming. I will talk about Tressloop on Inspector Gadget. First, my name is Albon, I work in KidFolk as a CTO and one of the co-founder on working on a lot of Kubernetes and BPF things. So Tressloop on Inspector Gadget, that's two open source projects on GitHub. Tressloop est un système de tracement qui s'appelle un peu similaire à S-Trace, mais dans une façon différente, il utilise un C-Grooves, un BPF et un ring buffer. Je vais parler plus tard. En Inspector Gadget, il n'est pas un single node, mais au niveau de la Kubernetes. C'est une collection de gadgets de différents types pour tracer différents aspects comme l'opinion de nouveau file ou l'entraînement de nouveau processus, etc. C'est une collection de gadgets pour les développeurs de Kubernetes pour les aider à débarquer l'application. Je vais parler un peu du contexte de pourquoi ces deux existent. En KinFolk, nous avons un stack avec une distribution called Flatcar Linux, qui est un O.S. optimisé pour les contenus. En haut de cela, nous avons un locomotive qui est une distribution de Kubernetes. Ce type de tools, Inspector Gadget, sont des tools qui sont en haut de cela, qui offert des tools à plus de niveau pour les développeurs de Kubernetes. Tout cela est un open source. En Flatcar, nous avons une nouvelle distribution, nous avons un nouveau canal pour les releases. Nous avons l'alpha-beta-stable traditionnel. Quand quelque chose est nouveau, il y a l'alpha-stable, et après un while, il y a l'alpha-beta-stable et quand l'alpha-stable, il y a l'alpha-stable. Mais récemment, nous avons ajouté un nouveau canal de distribution called Edge. C'est pour l'expérimentation, je pense, donc il n'y a pas de garantie. Mais nous pouvons l'expérimenter en ajoutant quelques nouvelles features de BPF ou des features de groupes de groupes ou quelque chose d'autre que nous voulons expérimenter. Donc le contexte est de travailler sur cela. Mais, à travers l'expecteur Gadget, en ce moment, on n'a qu'à utiliser les features normales de Linux, donc il n'a pas besoin d'acheter sur Flatcar ou Locomotive. Il peut également s'adapter à un système de Linux. En inspecteur Gadget, il n'a pas besoin d'acheter sur Kubernetes, mais il n'a pas besoin d'acheter sur une certaine distribution de Kubernetes. Ainsi, à ce moment, c'est beaucoup plus facile de commencer avec Locomotive, parce que c'est un peu à l'âge, donc il y a besoin de nouvelles features dans différents places. Donc avant d'aller plus dans les détails sur ces deux outils, je vais donner une introduction à d'autres tools de BPF pour que l'on soit plus à l'âge de l'expecteur Gadget et qu'il s'agit de l'envers du travail. Tout d'abord, la PCC. Alors, je vais juste vérifier combien de gens ont eu l'air d'utiliser la PCC. Ok, c'est tout le monde. C'est bien. Donc vous savez déjà que c'est une collection de BPF programmes qui servent à deux propositions. C'est une collection de tools qui sont vraiment utiles pour l'utiliser dans votre système Linux. Mais c'est utile pour l'apprentissage de BPF parce qu'il nous donne beaucoup d'exemples. Et il vous montre comment vous pouvez faire des choses dans différents terrains, dans la chaine, dans l'utilisation, etc. Je vais rapidement vous montrer le site, si je peux. Juste pour que vous veuillez voir les listes de tools qui sont là. Il y a beaucoup d'exemples. C'est ce que vous avez vu avant. Il y a beaucoup de tools où vous pouvez voir la source code dans BPF, sur l'exemple de comment faire des choses. Donc vous pouvez voir comment ça marche. Mais ceci est quelque chose qui est pour... C'est sur un single node. C'est généralement des tools de commande. Ça peut être dans un contenu. C'est ce que je vous montre ici. Ici, dans un contenu docker, il y a des tools de BPF. Il y a un autre tool qui est séparé de BPF, qui est indépendant de BPF Trace. Avec BPF Trace, vous pouvez mettre beaucoup de codes de code en ligne. Il n'utilise pas les mêmes langues de C pour mettre votre code de BPF en BCC. Mais c'est une nouvelle langue avec l'inspiration d'Ork and Sea. Je vous montre le site. Il y a beaucoup d'exemples. Ici, vous pouvez voir quelques lineaux. Dans un single line de code, vous pouvez ajouter un point de trace sur l'exécution de BPF programme qui ne pourra pas avoir beaucoup de codes de C. Tout le code de C, vous verrez le site de Coulos, qui se trouve à 3 000 tangents. Il y a d'autres exemples avec d'autres lineaux et beaucoup de tools basés sur BPF Trace. C'est un autre moyen d'ajouter des programmes de BPF qui sont utiles pour tracer. Ici, c'est le même screenshot. Ceux deux tools de BPF Trace, les tools de BCC, sont tous un single line. Mais ils seront utiles pour le niveau de la cubanité. Qu'est-ce qu'on doit faire pour adapter ces tools à la cubanité ? Normalement, la granulité de la tracation doit être différente. Dans la cubanité, nous carenons des portes. Nous ne carenons pas de tracer ce processus, ce PID. Mais nous sommes plus de haut niveau et nous voulons tracer un poudre. Normalement, nous ne voulons pas de tracer ce poudre. Nous voulons avoir une interface où nous voulons tracer ce poudre et le système va faire ce qu'il faut. Nous devons adresser le poudre à l'aide de la cubanité. C'est un moyen très utile. Si vous savez que la cubanité est utile, vous savez que c'est plus pratique que d'always vérifier le nom du poudre. Ce que je veux, c'est que la cubanité est utile comme l'expérience de l'usage où les développeurs ne devraient pas s'assurer des nodes. Mais d'ailleurs, nous avons utilisé l'interface traditionnelle de la cubanité. Il y a un projet qui s'appelle la tracation de la cubanité. Donc, ce qui fait, c'est qu'il tue les programmes de tracation de la cubanité et on les schedule sur un nôtre de votre clé de communauté. Donc, il y a beaucoup d'agoles qui ont été listés avant, mais il y a un couple de choses qui sont un peu fausse que je vais vous montrer après. La Tracation de la cubanité fonctionne de cette façon. Sur votre laptop, vous typez la commande de la Tracation de la cubanité. C'est à dire, c'est un plug-in dans le tool CLI de la clé de la cubanité. Et quand vous typez ça, ça exécutera la tracation de la cubanité. Et puis, à partir de là, il ne sera qu'à parler de la service API de la cubanité. Il ne sera pas SSH pour un nôtre, mais qu'il soit utilisé dans la façon normale de la cubanité de créer des portes, créer des maps configs, des configurations, etc. Donc, quand vous demandez de tracer quelque chose, le service API va commencer un port sur la cubanité, appelé Tracerunner, qui va exécuter votre programme de BPF et ensuite installer le programme de BPF dans le canal Linux. Et puis, il va pouvoir obtenir les logs de ça et retourner à l'UE. Donc, c'est un couple de choses un peu faibles dans le traitement de la cubanité à l'époque. Ce n'est pas si facile de filtrer le traitement sur un point spécifique. Parce que, à l'époque, les BPF traits étaient plus en train de tourner sur un système où vous pouvez filtrer sur un PID. Mais, sur le niveau de la cubanité, nous n'avons pas l'interface de l'usage où l'usage peut vérifier quel PID est en train. Donc, il y a un couple d'issues pour filtrer ça. En tout cas, il y a un demo que j'ai essayé quelques mois auparavant où vous pouvez filtrer le traitement de la cubanité sur le programme BPF qui est ici sur la ligne de commande. Et ici, je vais pouvoir filtrer sur un point spécifique par dire que j'ai envie de filtrer tout dans ce groupe. Ce n'est pas si pratique parce que dans ce groupe, vous devez spécifier l'idée de ce groupe. Et ici, vous devez trouver quel groupe vous avez filtré et ce n'est pas parfait de l'interface. Oui. Donc, à ce moment-là, ce que j'ai parlé de la base, j'ai parlé de la base BPF que vous pouvez filtrer sur un ordre sur le traitement de la cubanité que vous pouvez filtrer sur le niveau de la cubanité sur la base BCC. Donc, la prochaine partie de la base BCC c'est ce que nous pouvons faire avec la base BCC pour filtrer ça sur la cubanité. Donc, je pense que vous avez l'auparavant, ce que je vais parler de cet inspecteur gadget. L'inspecteur gadget est une structure similaire pour la base BCC. C'est aussi une base BCC aussi. Donc, je vais vous montrer un démon qui est ici. Vous pouvez voir ce qu'il y a sur la chine ou est-ce que c'est très noir? Ok. Est-ce que c'est mieux? Ok. Je suis sûr. Ok. Ok. Je suis désolé de ça. C'est bon? Donc ici, je vous montre un support gazeuse pour l'ins charming About Sorry! Quoi? Oui Oui. répétitively. If I check the logs of that, you see it doesn't work because it doesn't have the white capability that it doesn't configure it correctly. So now I will show you a tool in Inspector Gadget that could help to debug that. So Inspector Gadget can be called on the command line like this, where you have a list of gadgets. One of them is called Capabilities. What it does is you can select a pod and it will show you in real time what are the capabilities requested by that pod. So there is a different way to select the pod. You can give the pod name or you can select by label. So for now I will just give the name and hopefully it works ok. It requires a lot of capabilities here. Oh sorry, I didn't set the pod name correctly so it was showing all the process instead of filtering correctly on the pod I asked for. So now as you see it request only Capsis Nice, which is what you expect when you can run the command nice. And it does that every 5 seconds because in the script I had a Sleep 5. I have other gadgets to show as well. For example, ExactSnoop, it will show you every time a new process is created, what is the command of that process. And I can filter by label and then I can say I want all the label, my app equals something. Now it should only filter the process in that pod with that label and every time this process, this pod create a new process, it's displayed that here. So that's it for ExactPod. There is a few other gadgets, some of them are not implemented yet. Some of them are more or less complete. But there is some to trace network TCP connections or every time a file is open and so on. All of this is not created from scratch, what I did I just picked the BCC scripts that already exist with a small modification to adapt them and then InspectorGadget run them. So all of this so far relies on what is already written in BCC. So what are the challenges of doing that of writing InspectorGadget? The first one is how to select the pod. Because when you install a BPF program with Kpro about response, that's something installed on a Linux kernel and the Linux kernel doesn't care about containers, it only see a lot of process. So the BPF program will run for all the process on all the containers. There are a few BPF helper functions that can be useful for selecting, filtering what you want to see or not see to get the name of the process but that's not really useful because name of process can be changed that doesn't tell you which container it is. The PID, but we want to trace several PID, not only one. What is more useful for me is filtering on a C-group ID because every container should run normally in a different C-group so that's a good way to filter the pod. For this, I need Linux 4.18, so a fairly recent one. Another thing, here I talk about C-group ID but there are actually several C-group subsystems on Linux. There is a C-group version 1 with several ones and C-group version 2. This BPF helper function only returns the C-group version 2 ID. So how it works on SystemD, you could have a directory like that with C-group version 2 hierarchy and then on this directory, you have the system called name2handleArt, which returns the signal, I will try to reply, ok, it's back, cool. So this system called name2handleArt returns the C-group ID. The problem is on Kubernetes, Kubernetes normally only uses C-group version 1 so far. So on here, I need C-group version 2, so what can be done? SystemD helps us a lot here because in SystemD, you can configure it to enable both C-group version 1 and C-group version 2 and then all the other components that create C-groups, we need to take care to configure them to let SystemD set up the C-group system and not let them create C-groups by themselves. So there is a lot of configuration but in Flakar Linux on the Edge of Channel, it's already done for you. Then in the BPF program, you need to filter on a specific C-group ID. So here I have some pseudo BPF code, so it's a bit more complex than but it's just to illustrate where when you ask on the command line to filter only on one specific label. What I have is BPF maps, which are a kind of global variable, which are prefilled with all the Kubernetes labels for the pods that exist on the system. And the BPF program will look up the C-group ID and look for this C-group, what are the BPF, sorry, the Kubernetes labels, and filter on them. So at the moment, that's a proof of concept that works like that with some string manipulation. It's back. But there are work to be done to be more efficient working with ID instead of strings. How do we prepare these BPF maps with all the Kubernetes labels? What I do is I use OCI hooks, so when Kubernetes, every time a container is started, it will start RunC by default, and run OCI hooks, well, I did a script there to fetch, every time a container is started, fetch the Kubernetes label and fill those BPF maps. So if you want to try inspect a gadget, it's easier to do it with FlatCar Linux on locomotive so far, but you can look at the instruction for installing that. You can also do it yourself on the other system, it's a bit more difficult, but you can get help from the instruction on GitHub, so it's what you need to do there. Okay, so now I go to a more interesting gadget, an inspector gadget, which is called a trace loop. So the use case is, as a developer, I like to use a trace to get to know what syscalls are executed when I want to debug an application. But a trace is a bit problematic, it's slow, so you cannot just enable a trace for every process on all your cluster, that will not work. On some time, I want to debug something that has just crashed, and it's too late to add a trace there, so I cannot really add a trace retroactively if the program just crashed, and it doesn't crash reliably. So the idea is to have a flight recorder system that records all the syscalls all the time for all your ports, but in an efficient way, which doesn't have the disadvantage of trace, which will not work for enabling everywhere. So that's something trace loop does with BPF. Everything is recorded all the time, but it's not saved anywhere, it's only saved in memory in a ring buffer, and only when needed, I will get this information. So if I compare stress on trace loop, it has several differences. Stress capture the syscalls using ptrace on Linux, trace loop use BPF instead, trace, you have to specify which process you want to trace, so the granularity is the process, trace loop instead does that on the syscalls level. So all the process in that syscalls will be traced. Stress is slow, trace loop is fast, and that's because trace is reliable, so it's working in a synchronous way, you get the events as they happen, and it cannot lose events. In comparison, trace loop is working in a synchronous way, so it writes events in a ring buffer. So if the ring buffer is full, then events will be overwritten, and you, theoretically, it's possible to lose events this way. But still that makes it very useful for debugging. How it works, that's a bit different than other gadgets. I install BPF program on the trace point syscenter, so every time a syscall is executed for all process on the system, it will execute this BPF program, and then it will check if the currency group of that process need to be traced or not. So for all ports, it will redirect the execution to a different BPF program, and then each pod will have its own ring buffer, where the syscalls are stored. So the syscalls are permanently stored in that ring buffer, in memory, but then it's only sent to the user space, only when the user request it, so normally it's not saved continuously because it will be too slow, but only undamaged. Ok, so I will show you a demo of trace loop, first as a standalone tool. Ok. So what I will do here, I run a trace loop, but on the specific syscalls, the one for SSH. So now it's starting, it's recording everything that happened on the syscalls, I will put the process in the background, and then let me just start a new terminal and do SSH, so something should happen here. Ok, and actually I didn't need to put it in the background, so now it's recording everything but nothing is visible because nothing is sent to user space, and when I stop this, here it dumps the last syscalls of SSH that were in the ring buffer. So I see differences calls with the parameters, like this. And then I can show you how it works on Kubernetes, so not only on the command line here, but using kubctl, so if I do kubctl get pod, I can, sorry, I will start a new pod. So this is doing some operation in the scripts, a multiplication, sending that to the BC program, saving that to a file, and then attempting to print that file. So if I do that, what will happen? As you might guess, it will not work correctly, so here I don't know if I have internet working or not, but because this file is different than this file. So I did a mistake in my script, I don't use the correct humanities cluster, sorry. So here there is an error. So what I can do now is with kubctl gadget, use trace loop to see what I can do here. So here I can list the existing trace. So even through my program finished and terminated, I still have a trace that has kept a little while, and I can show this trace using trace loop, and now it lists the last same calls of the program. So if I go, sorry, does it work? Cool. I think I have a problem in my demo. I didn't trace the right thing, but you get the idea. So here this is the kind of trace you can see. Here you have a white system call. For some of the argument, I print the buffer, and some is not implemented yet. How does it work behind the scene? It uses linux perf ring buffers, that if you want to see the documentation, it's in a manual page for perf event open, and here I can put a different kind of message. They can be of different size, but because of the way it's implemented, it needs to be a power of two size. So what does trace loop do? It adds a message in the ring buffer, every time there is a new syscall on syscall enter, and for every parameter that is traced, we add an additional message, and then when the system call returns, another message, and all of this message are reconstructed to reconstruct the system call in user space on demand. So all the system calls on linux are different with different parameters, so trace loop pass, the description in ccfs, which contain text format, what are the different parameters, and it pass data start-up to know the different format. This is a method which is inspired from trace left, which is another ebpf syscall framework that we work on before. And then, after having passed this other syscall format, this is send to a bpf map in memory so that the bpf program can read that and know for each system call what to do. So from that, it knows that the system call close doesn't have any string to the reference, the system call open as a path name, and the system call write as a buffer with a size specified in a different argument, and so on. So lastly, I would like to show what, how trace loop can be integrated with systemd. So that's more a wish list than something which is well implemented, but I will show you what the idea is. I would like if systemd run had an exact start pre command, so I could say here start to record whatever happened on this systemd unit, and then run the program. So this option doesn't exist yet, but I think that will not be too difficult to add. And then what it will do in the pre command, it will inform trace loop, which cgroup should be monitored. So since that doesn't exist yet, I'll try to do that with systemd unit, so that's what I have here. That's why you can maybe see here, I will show you in a different screen. Here, I have systemd unit, which run some program here, just a script. And I have a pre command which inform trace loop over its HTTP API to add a specific cgroup, and then a post command to ask it to dump the last syscalls in a wing buffer. So the demo, let's try to do that. Yes. So here it is, I started trace loop in a demon form. And then we start to try to start this thing. And then here, I get all the output from that systemd unit. Because of the post command, I'm expecting to see the last system calls at the bottom. So here, I'm filtering on some things where I see the BC program called the white system call and so on. Okay. So that's the same thing here. I would like to integrate that with systemd status, so we can have this kind of information directly in the systemd tools. And that's about it from my talk. There is lots of work pending on GitHub that was necessary for this work to go BPF in BCC and in Kubernetes. That's it. Thank you. Is there any question? Well, as you can see from your wish list for systemd run, the trouble is, you're starting the trace loop in advance for a process that crashes early. Right? That's the same problem still exists on Kubernetes. That if you have a pod which basically crashes during startup, that you can't really capture things because you no longer, you don't have a pod yet to manually specify hey. Yes. That's possible to do because on, so the normal way to deploy things on Kubernetes is you deploy your deployment and then that will create pods but you don't know in advance the name of the pod. So you cannot just say kubectl gadget trace this pod name because you don't know the name of the pod yet if it is a early crash. But you can filter things by label. So if you have a deployment, you know the label in advance. So you can, you kubectl gadget execs noop if you want to trace execution or open snoop or any gadgets and use the label filter. And that works even if the label doesn't yet exist? Yes. So how it works at the moment is with string comparison in BPF which is not ideal but that's a proof of concept. And what I want to change it to is using an ID system which it will still work with this use case where you want to trace from the very beginning of the pod. So it works because I have the OCI hook right before the pod is started, RunC will execute that script and get the information about the C-groups on Kubernetes labels. Okay, thanks. Okay. Do you have already some overhead numbers? Can I just run trace loop in a production process that keeps on crashing randomly? Or is it still too heavy or too much overhead? I've not done any check number on overhead. I think one of my colleagues did, but I don't know. I will not expect this much because when you don't ask for the information, then it's trace loop will just write in a ring buffer which is normally very fast, so it doesn't actually does any context switch or anything like that, but I don't have number, sorry. Otherwise it's not really ready for production because of other problems it needs to be more tested on debug. You mentioned that the ring buffer captures events. How big is that ring buffer? Is it configurable and is it possible to have something else as a backing store for that so that you can make your ring buffer big enough to capture a very long series of events? At the moment it's not configurable. I think I set it to 8 megabytes, but I just choose that randomly. And it's a different ring buffer for each CPU because otherwise if you have several whiter to the same ring buffer that will not work correctly, so there is each CPU as its own ring buffer and each application as well, so different C-groups have different ring buffer, different set of ring buffer for each CPU. So I'm not quite sure which size it should be, but per ring buffer is only something in memory. It's something which can be backed on disk, but one of my colleagues did something where it's regularly fetched the information from the ring buffer in memory and keep it. But yeah, my idea was just to have something to get the last events not necessary, everything. I think that will not be scalable to trace all the system calls for everything all the time and dump them somewhere. So I want to ask, how do you garbage collect the buffers because they outlive the C-group that they monitor if I understand correctly? And then the second question, how reliable is pointer dereferencing? Is it always reliable because I can imagine that your BPF program may be scheduled after the application that you are monitoring and then the application already may have changed the contents of the memory that Cisco saw, I mean, the kernel saw and you entered the Cisco, right? Right. So for the first question, so far each ring buffer has a reference from the trace loop demon. So if you kill the demon, then the reference disappear on the memory is freed. But so far I don't have anything to release it after a little while. My idea was after some minutes or hours, I don't really know, after it has crashed and removed it. But as a proof of concept, I just keep them forever until the user specifically type close this. For the second question about what happened if we tried to start the tracing system after pod has already been started, in this case, you're right, it will not trace it because I start monitoring a specific group in the OCI hook, which if it's not run, if you have not yet installed it, then it will not catch things. So yeah, that's a limitation, but the idea would be to have a trace loop always installed on the system. So it runs as a demon set on Kubernetes to have it all the time. Other questions? Then thank you. I will be here during the conference if you want to talk to me about BPF or trace loop or something.


How's it going everyone? Caleb here with Layton Sports Cards. Next up tonight we've got a case of Absolute Baseball 2022 here, Random Division style. Thank you again so much for joining everyone. I've got the results for you right here one more time. Good luck everyone. Should be a fun time. Break number four. Here we go, 10 box full case. Just get one sec here. Let's get to it. Absolute Baseball. Ten boxes of its. Thanks again everyone. Appreciate you getting on in the break. Let's see what we can find for your Random Division style here. Case break number four. Kaboom hunting. Let's see what else we can do. Slide that back a little bit empty of course. Let's take a pack inside each of these. Supposed to be an auto and a mem sometimes we've seen some funky stuff though. Multiple autos or whatever it is. Let's see how they are tonight. Gonna have a few base cards at the back here. It's gonna be a Mookie Betz there Dodgers. Excuse me, N.O. West to 99. Statistically speaking, Matt Olson, rookie class redeptmers, Boba Shet and Shohei. Base Rafael Devers and then it's gonna be a uh to 149 there, N.L. Central, Joey Vado, Paul Worthy, Kabooming coming. We'll find one here eventually by the time we get to the end of the break. Uh that's gonna be Matt Veerling that time. Phillies, there you go. N.L. East, Colton Welker, rookie force this time. Rockies, N.L. West to 299 and rookie baseball material booklet signatures, number to 60 for the Phillies. Luke Williams. There you go. Nice looking card there. Phillies once again N.L. East. There you go N.L. East. Solid. Sam said this is his favorite product to top of. Just because of all the books. All the books, uh well when I have it, like when I'm prepared it's not that bad but. Books are sick, it's just. If I'm not prepared they are sick. Rocks are stressful. It is, it is. You gotta be careful with the books. Bellinger there, N.L. West to 149, Paul Worthy, N.L. Ryan, rookie class, Kyle Mueller, Higolonzo and Mike Piazza, Grace Harper base, and Mariano Rivera base. It's going to be redemption this time. Jonathan Papelban, Marks of Fame. Uh we'll check this one out here. Give us a second. That could. The Red Sox or Phillies? Yeah Red Sox or Phillies, it will be uh 2022 absolute baseball. That's an interesting one. Red Sox wasn't. See who Papelban is in the product here once that gets. Cardboard connection probably. Papelban is going to be Phillies. There you go Phillies. Nice that is going to be N.L. East. Yeah that was not my guess either. N.L. East. There you go Phillies. Jure Durand that time that's Red Sox. There you go. Solid there for the A.L. East. Jure Durand RPA that's to 199. Nice one there Red Sox to 199. Good stuff for you A.L. East and T.J. Freidel. That's going to be a double patch to or triple patch excuse me to 25. N.L. Central. One of 25. All right we had a solid Durand in that box. Onto the next one. Papelban. All right here we go. Next two boxes. It's 199. Luis Robert. A.L. Central. Thank you class. Joe Ryan. Historical Duos. That is going to be a double Houston card there. Pettit and Roger Clemens. A.L. Central. Freddie Freeman. Rafael Devers. Going to be a Texas auto this time. There you go Texas. Yoel Pozo. A.L. Central. Excuse me A.L. West. Texas. Jose Ciri that is oh that's awesome. On the barrel there that is going to be number to 50. Signatures. Nice one Houston. A.L. Central. There you go. Excuse me A.L. West and Corey Ray. Corey Ray is a patch to 99. You go Milwaukee and L. Central. Next one up. All right keep hunting everyone. Good luck here to 199 on the Jazz Chisel. We go there on the Jazz and Elise. Extreme team Luis Robert. A.L. Central. Statistically speaking Seeger. It's going to be a Brian Reynolds. And Frank Chance. Chicago. Extreme team there to 149. Jacob DeGron. Metz. We go Metz and Elise. Barry Larkin on the patch this time. A.L. Central. Nice one. All the fame memorabilia. Barry Larkin. You go Reds. And your Don Elvarez. To 99. Baseball material. Signatures. Nice one. Take that. To 99 for Houston. A.L. West. Very nice. I love that that book. Yeah it's a nice your Don. We'll take that all day. Gotta be honest Mike. When you win like every game it's just not as exciting. Oh no. But day to day basis just just you know it takes the love away from it when you beat everybody in blood. Oh no. Sam's going to be really sad when that lock runs on the playoffs. When we make it when we lose and A.L.C.S. to the A's. Yeah. To the A's. Okay here we go. Let's continue everyone. A.L. West Astros to 199. That's a double Brooklyn card there. Dodgers there you go. N.L. Wests. Shoeless Joe Jackson in there. Sammy Sosa. Ozielbies. Yeah an extreme team. Kyle Lewis for the Mariners. Eight of 25. Nice one A.L. Wests. Billy Williams for Oakland to 25 on the patch. You go A.L. Wests. And Angel Zeppra that time for the Royals. Going to the A.L. Central. Beautiful patches. Wow. Holy cow. I was not expecting that. Two of 10. Very nice. Awesome patch there congrats on that one. Very, very nice. Be happy with that one A.L. Central. All right so let's do it. Here at Cold Air for the Yankees. A.L. East to 99. Vado Central. Ano Central. Sturgeon Duos there. Cleveland. Babe Ruth. Trey Manson. You got Aaron Judge and J.D. Martinez base. Zach Rex. It's going to be the 199 there. Zach Rex. What a name. I was going to say that's an awesome one. Here you go A.L. West. That's pretty cool. It's going to be a rookie patch. Autograph this time to 199. Tony Santalon. N.L. Central. It's an awesome name. It does kind of sound like a 12 year old Xbox line. Right. It's going to be Joe Ryan patch there. 81 of 99. There you go. That'll be A.L. Central. Let's keep going. Zach Rex. Zach. A couple more boxes. Bon Hunting. All right starting out here. N.L. Central to 199 on the Cabrion Hayes. Sherser Metz. Soto. Pete Rose. I've got Salvador Perez and Jose Altuve. Redemption this time is going to be a Yohan Adon. That is a tools of the trade. Two swatch signatures. Yohan Adon. Let me check that out quick one sec. N.L. East. I don't remember Adon at all. Nats. It should be Nats against everything. Yeah there you go. That's going over to N.L. East. N.L. East. There you go. Tools of the trade. Nice one. Rookie patch autograph to 199. O'Neill Cruz. Congrats N.L. Central. Very nice. Awesome O'Neill Cruz rookie patch autograph. There you go. Central. N.L. Central. And Riley Adams. Patch there. Going to be also going to the N.L. East. To 199. Here we go. Continue. All right. Sandra Bogart's there. A.L. East. That's to 25. Chris Bryant. Otto Lopez. Marcus Semeon and a Vladigrero Jr. Redemption is going to be a Edward Cabrera Rookie Baseball Material Booklist signatures. Booklit for you Edward Cabrera. I think so. Just want to make sure. Get Marlins. Go N.L. East. It's going to be Brandon Marsh. Rookie patch autograph. Very nice. Gorgeous patch. Number to 50. There you go. That is a sick looking hit there. A.L. West. Congrats. Nice one to 50. And then very nice here. It's going to be a Vidal Bruhan. Rookie patch to 299. There you go. Very nice. Congrats A.L. East. Just keep going. A couple more boxes. No Kaboom yet. Flawless Baseball after this. That's going to be fun. Random teams. Flawless Baseball. We got football coming up later. Here we go. Give me Bryce Harper there. Here we go N.L. East. It's going to be number to 149. Doliz Garcia. Joey Vado. Ichiro. Marcus Simeon. N.L. East. Nick Fortez. Number to 199. Out of there. Tools of the trade is a Bobby Abreu. Rookie patch autograph for the Phillies. Nice one.  Or excuse me Bobby Abreu. Little throwback to 25. Yeah Bobby does have a nice one. Long time player there. Made it around the Yankees. Phillies. Spencer Strider. That's going to be N.L. East. It's going to be to 299. Bobby Abreu. Pretty cool there. Congrats again. Yeah absolutely. It's got a pretty deep checklist. Yeah as you can see here. Top and pass the ball. Halfway through. Yeah halfway through. Quite the checklist. On the back. We'll save that for the back. Ichiro there. Nice start. A.L. West. It's to 199. Trade turner extreme team. L.K. Line. Randy Rosarino. Ronald Acunha Jr. Rookie class there. That is going to be A.L. Central to 149. Or excuse me A.L. West. You're saying Central and Houston. It's because they're in the N.L. Central. Yeah it throws me off so much. A.L. West there. Astros. Eleberto Mondes. That's going to be a patch there. Hands of City to 99. A.L. Central. Redemption on the back is a Wander Franco. Rookie baseball material. Booklet signatures. Red. Boom. That is big.  Awesome card for the A.L. East. Oh man. Red. Booklet signatures. That's to 99. Very nice. Grats on the Wander. That'll be gorgeous when it arrives. Awesome hit A.L. East. Good stuff. Good stuff. Let's go to 99 on the red. Red. Next one up. Continuing here. This is the Kaboom box. Sam's calling the Kaboom here. Grats again. That's awesome. That is awesome. Brandon Lowe to 199. Here we go A.L. East again. Ryan Belade. Jose Tube. German Munson and Jose Bray. We got Steve Carlton. Cabrion Hayes and Hallworthy. Joey Vado. N.L. Central number to 25. 25 there. Stan Musial. N.L. Central as well. Number 12 of 25. Throwback patch. Very nice. Stay in the man. And it is going to be A.L. West. Cal Raleigh to 99. Booklet Auto there to 99. Here we go A.L. West. Next one up. Continuing here. We got to 149. Chai France. A.L. West. Icons Null and Aronato. P. Delonzo by storm. Ozzie Smith and Mickey Mantle. Ichiro Gladi Jr. Dennis Santana. Rookie Patch autograph there to 75. A.L. West. Gorgeous looking card. Reverse and rare only. But if you did it for one it's no problem. Nice Dennis Santana. You've got Rodolfo Castro. Beautiful. Once again on the autograph there to 50. Nice one there. N.L. Central. Let's take that. We'll take that. Dodger's not interest what I say. Oh sorry yeah I think I said Dodger's. Nice Rodolfo. N.L. Central. And Glen Otto there to 99. That's an H my head killer. Yeah what am I doing? Keep going. Glen Otto. Let's A.L. West. All right keep going. I actually said the last box was not good. Oh I see I see. Changing your answer. Yeah. All right good luck all. Auntie. I'm gonna call the Kaboom when I see it I think. It's a good strategy. Yeah pretty undefeated I gotta be honest. It's a decent strategy I'll tell you that. Okay here we go. Let's continue. Good luck all. Ozzie Elby's that time. We go N.L. East. Jonathan you need to buy Storm.  N.L. Central cards there at the back. We've got Gil Hodges with Merrifield. It's going to be a shoelace. Joe Jackson there. It's the 149. What was he Cubs or what? White Sox. That's going to say Sox. That's A.L. Central. Corey Seeger there. That is going to be Texas once again. Going over the A.L. West. Not or excuse me to 149. Put the number in a different spot that time. So 149. And it's going to be the Booklets to 99. Matt Veerling. There you go nice looking card there. Booklet Auto. Congrats on that. N.L. East. Booklet for you there. Continuing on. J.D. Martinez to 149. A.L. East. Story. Shohei by Storm. And Eloi Jimenez. Back to base cards here. That jazz chisel. Matt Chapman. It's going to be Shane Bos there to 999 on the rookie card. Going to 999. A.L. East. Alejandro Kirk. Rookie patch autograph that time to 75. A.L. East. To 75. Alejandro Kirk. Tyler McGill. Rookie patch autograph that time. There you go Mets. And A.L. East. And a Garrett Hampson. Dual patch there to 199. N.L. West. Another one up. Flawless baseball to follow. Little bit more here on the absolutes. No kaboom yet. Where's it at? Where's it at? Hotel Marte. A.L. West to 149. Shane Bos. Barry Larkin. Rod Crue. And Trevor Story. Joey Vado. Bartolo Cologne. It's going to be really stargill there to 99. N.L. Central. We have San Francisco dual player patch. Logan Webb. Joey Bartz to 99. You go N.L. West. And T.J. Freidel. 57 of 99 for the N.L. Central. Book wit patch autograph. You go N.L. Central. Continuing on. All right to 199. Paul Goldschmidt. N.L. Central. Mark McGuire. We got Freddie Freeman. It's going to be Raphael Devers. And O'Neill Cruz there. N.L. Central to 999. Rookie there. You go N.L. Central. David Peterson that time. There you go N.L. East. It's going to be to 199. Rookie patch autograph. Vet patch auto there. And it's going to be Rookie patch autograph. Videl Bruhan to 199. There you go. Nice hit. A.L. East. Congrats. Very nice. And then triple patch here to 149. N.L. East Braves. A.J. Minter. Here we go next one up. Here we go. Slice and slice. Good luck everyone. Starting out A.L. West. Adolesz Garcia to 199. Another Adolesz there. Giovato. You got a Ryan Sandberg base. Wilson Chairs base. Statistically speaking to 149. It's going to be A.L. East. Trevor's story. Brandon Rogers there. Triple patch. It's going to be for Colorado. A.L. West. Or excuse me N.L. West. Brendan Rogers. And a Lucas Gil there. That is going to be numbered to 199. Lucas Gil there. Congrats on that. Solid for you there. Yankees. That's going to be N.L. East. Or excuse me A.L. East. All right one more. One more and then one more box. We got to find that kaboom. We're getting down to it. Where's it at? Wallis baseball. Random teams is up next tonight. Here we go. Sammy Sosa there. N.L. Central the 199. Duos N.L. Central. Chance Tatis Jr. Lindor. Kaboom is also Francisco Lindor. There you go. Very cool. It's going over to the N.L. East. There you go. Congrats on the kaboom. Very sick cards. N.L. East grabbing it tonight. Very, very sick. That will get magged up for you. Congrats N.L. East. Nice card for you there. Rookie to 999. Brandon Marsh, A.L. West. You got Joe Ryan that time. Rookie patch, autograph there. A.L. Central to 199. Lars Newt Barr, Rookie patch, autograph. N.L. Central to 199. Larby A's and a kaboom in that box. That's fine. And Carson Kelly dual patch there. N.L. West. One more. Paul. What's going on Paul? Go Ravens baby. Big Ravens man. Oh man. The one guy I have on my side in this chat. Can be me. Here we go. Although the Ravens are the team I can tolerate the most in the division. Yeah, I'd say the Browns are the team I tolerate the most. Hey, nice. We've come to an even ground. There we go. Sam and I have come to an even ground after Truce. Our third day of fourth day of working together. To 99 there on the Kershaw. N.L. West. Starkel duo's there. Guardians. Fabers, Bragman and Soto. Yeah, Billy Martin. Louis Struber. Give me number to 25 there. N.L. West. Ketel Marte. To 25. Also wow. Sick patch there. The button patch. Seven of ten. Seth Beer. Nice. That's a pretty cool card. Sick button patch there. He grants on that one. N.L. West. Nice. And it's going to be a Kevin Smith. 32 of 99 there. Patch on a grab. Nice one. Here we go. A.L. West. Last box here and then a little recap. Be right on the flawless baseball. You get Pete Rose to 99. Wilson Contreras. Pete Rose, N.L. Central. Mookie Betts. Wow. Another Kaboom. Erin Judge. Kaboom. Very nice. There you go. Congrats on that one. A.L. East. Awesome card. Congrats again there. There's that Lucas Jill as well. A.L. East. Esteban Floriel this time. Rookie patch autograph. A.L. East. Your box here. To 199. You have N.L. West this time. Andre Jackson. Rookie patch autograph to 75. And Austin Hayes on the dual relic to 199. A.L. East. Awesome. Thanks again so much everyone. Let's get a recap going for you. One time is always with some absolute there. Here we go. Start out with some patches. We've got Carson Kelly. Brendan Rogers. A.J. Minter. The dual Logan Webb and Joey Bartz. Garrett Hampson. Corey Seeger. Glenn Otto. Stan Muziel. Alberto Mandisi. Spencer Strider. Vidal Bruhan. We've got Riley Adams. Joe Ryan. Billy Williams. Barry Larkin. Corey Ray. We've got T.J. Freidel. Colton Welker. Matt Veerling. And the Sepp beer. Button. Okay. Patch autographs here. Joe Ryan. Lars Knuthbar. Vidal Bruhan. David Peterson. Taylor Mengele. Alejandro Kirk. Rodolfo Castro. Dennis Santana. Bobby Abreu. Nick Fortes. Going to be Brandon Marsh. Edward Cabrera. O'Neill Cruz. Joanne Adon. We've got Tony Santian. Zach Rex. It's going to be Jose Ciri. Yoel Pozo. Jonathan Papobon. We've got Gérd de Reine. Frankie Lindor Caboom. Aaron Judge Caboom. And the red book of Wander Franco. That's going to be the 99. Al East. You did very, very well. A couple of cards that just missed the recap. Austin Hayes, Esteban Florial, and Andre Jackson. Thanks again so much everyone. We'll do the books here as well quick. Matt Veerling. We've got Cal Riley. Angel Zepra. Jordana Alvarez. Luke Williams. Kevin Smith. Lucas Gill. One-day TJ Freidel. Appreciate it. Hope you guys did enjoy some absolute baseball tonight. Random Division's case there. Number four.


life. It seemed like his lies had finally caught up to him. In 1984, Super Password premiered on NBC. Everyone wanted to be a contestant on this show for the simple fact that they could be on TV and potentially earn tons of cash. That's exactly what one particular bearded contestant had in mind, too. In 1988, a man by the name of Patrick Quinn appeared on the show as a contestant, and he captivated audiences with his confidence and ability to answer anything that was thrown at him. But he was hiding a dangerous secret. Quinn was quite a charmer, which is why the host, studio audience, TV viewers, and even the other contestants connected with him right away. He was also pretty good at playing Super Password, too. So, what happened next wasn't really much of a surprise to anyone. Quinn had obviously watched the show since it started airing because he was a pro at winning the various challenges. In fact, he was so good that he made it to the final round. But getting the grand prize became the least of his worries. A fellow contestant started describing different words that began with a certain letter, and it was Quinn's job to figure out what they were. He only had a small amount of time to guess correctly, which he did. But would he be satisfied with the grand prize? As soon as the countdown ended, all sorts of whistles mixed with the sound of a cheering audience were heard. And viewers at home saw $55,000 flashed on the bottom of the screen. This was a pivotal point in Quinn's life, but it wasn't necessarily for the better. Quinn returned to the studio a month after the episode aired, albeit sooner than the producers expected. But he quickly explained to them that he had to arrive earlier, because he had been tasked to go on a government mission to Turkey and had to leave soon, and this baffled the show's producers. According to the show's host, once the episode wrapped up, Quinn had told him that he was in the CIA. And yet, the host had no idea whether Quinn's backstory was legit or a work of fiction. But something was definitely off about the contestant. Quinn alleged that he was once stationed in the North Pole so he and other CIA operatives could intercept Russian transmission for information that they could use. He also claimed he was happy to have earned himself a little extra money for himself. But his story came back to bite him a few weeks later. When Quinn returned to the NBC studios, he was floored when he knocked on the producer's door. When it opened completely, he had been expecting to see the studio's staff members, but there were others waiting for him and they were not happy. Inside the office were men in suits and they had guns and badges. The show's network executives glanced at Quinn and they saw fear in his eyes. It seemed that his story was nothing more than a con and the FBI wasn't about to let him go with a slap on the wrist. Quinn knew this day would come, which is why he had originally asked to get his prize money early. He was hoping that he could make a run for it all the way to Turkey, but the FBI agents had caught him and cuffed him. But how did they know he was the guy they had been looking for? Despite using an alias, a viewer from Alaska had been watching Quinn's super password episode, and they knew right away who he was and that he was wanted by the law. In fact, Quinn's name was actually Kerry D. Ketchum. Once the viewer noticed Ketchum on the TV screen, he called the cops. Then, authorities set up an operation that would allow them to catch their fugitive in the producer's office. But who was Ketchum and what had he done? Although he impersonated a CIA agent, he wasn't actually a part of the CIA. He used to be a deputy at the Montgomery County Sheriff's Department in Dayton, Ohio. Unfortunately for him, his life and law enforcement ended when he went on a crime spree that made him famous for all the wrong reasons. While Ketchum served as a deputy, he didn't hold on to his rank for very long because he was charged with theft. According to authorities, he stole $200,000 worth of military equipment and ended up behind bars for 18 months. But that wasn't the end of his crime spree. Ketchum was also indicted in Indiana for illegally acquiring a $15,000 bank loan to get a car and then forging $15,000 worth of checks. But these crimes were mild in comparison to something else he had done. Ketchum was once again caught and sent to prison for five years after filing a death benefits claim for his ex-wife with the Fireman's Fund American Life Insurance Company. The amount was for $100,000 and investigators had easily seen through his lies because of an obvious oversight. Ketchum had claimed that his ex-wife has been killed in a car accident. But the Air Force enlistee who was reportedly killed was very much alive and completely unaware that her ex-husband had filed a life insurance claim. But Ketchum still felt that people weren't seeing him for who he truly was. Ketchum felt people had misjudged him. Because I'm friendly and easy to get along with, people think I'm conning them, he was quoted saying, That's one of the reasons I went on the game show to use my own intellect for something other than bad. But US District Judge Matthew Byrne Jr. disagreed.  That's what I'm sentencing here, someone who uses his wits and his intelligence to become just a common thief. Prosecutors accused Ketchum of unleashing a virtual tornado of deception before he was taken into custody. And in addition to serving time, he also had to pay the insurance company for his attempt at fraud. But dismayed viewers still had one question on their minds. Did Patrick Quinn really exist or did he create an alias out of thin air? It turns out that there was a real Quinn out there somewhere. It was the name of Ketchum's old college professor, which he borrowed in order to win $55,000 on super password. The Los Angeles Times quoted Assistant US Attorney John F. Walsh III saying, He's used his engaging manner and his cleverness to flout systematically all the responsibility the law has placed on him. But what led him down this dark path?