So Carbon 5, that's where I work. I work in San Francisco. We also have an office here in Santa Monica with a couple of other offices around the US. We are a strategic digital product consultancy, there's lots of words. We work with our clients really collaboratively to build software applications basically. We have teams that are product managers, designers and developers who partner with our clients really closely to sort of develop their product ideas. The important thing, I guess the most relevant thing for you all tonight is that as a consultant PM, it's different from being at a product company where you own this product in a really robust and ongoing way. But the thing that I do when I give away these products each time I do a project, but I also have to build a team every time I do a project. So often I'm leading projects and I'm on the hook for making that teamwork. So hands up here if you lead a team currently. Okay, so some of you lead a team. What about influencing teams? I don't believe you for a second. Okay, so some leaders, lots of influencers including Alexa. And if you're a product manager or even if you're a product minded designer, we have a couple of those in this room, it's your responsibility in part to make sure that your team can build a great product. And that can be really difficult when you're not also the person who is in charge of that team. And in our practice, in our product practice, we actually don't include very much conscious design of team dynamics. And this is a thing that I started to feel was kind of lacking, like what if you all were able to take what you know about product and apply it to the team that's making your product? What would that look like? So usually when you think about teams and team development, we're thinking about one of two things. We're thinking about individuals and their development, and we're thinking about outcomes. So individual development is really important. I don't know anyone who would argue that professional development doesn't matter. So I'm not saying we should do away with that. But I do think that it is insufficient to think about teams in a kind of raise all boats way. Like we'll make these people better, therefore their teams will be better. That's what we need to do. So I think there's more to be done there. And then the outcomes part, teams have goals. That is really important. It's vital that teams have clear goals. We roll those goals up to organizational goals, roll it down to individuals. That's all great. But an outcomes only focus doesn't, again, doesn't really tell us that much about how the team is doing as a team. It tells us about how the team is producing something, the outcomes of the product that it's making. Maybe we have some stats about the team's work. But it doesn't tell us a lot about the team as an entity. So I don't know why that happened. You can look at that for a second. Or you can look at something that way. Or you can not look at anything. Slides are really unimportant. So you need to start. I believe we need to start thinking about how we develop teams as teams. Not just as groups of individuals, not just as people who produce outputs, but as a thing in and of itself. So the good news for product people is that you already have tools that enable you to do this. Tools from your product tool set that you can apply to teams. I'm going to just take a little moment and see if I can make this. So tonight I'm going to talk about a couple of specific tools that you probably already have. One is Eric Reese's Lean Startup, the build-measure-learn cycle. And the other is product slash team road mapping. Has everyone here used build-measure-learn or read Eric Reese's book? Lots of them. No, that is okay. I just want to get a sense. And what about roadmaps? Does everyone use roadmaps for their products? A little bit. Okay. That is also okay. So build-measure-learn is Eric Reese's Lean Startup, which I totally recommend reading. He takes the scientific method and tries to... He's actually in town, like Tom, where he lives. Is he really? Yeah, across campus. Right here. He's on a little bit of a book tour right now. He's got a new book coming out. It came out this week. Oh, nice. Okay, Brad, I can't wait to read it. And Lean Startup Week happens in San Francisco, which is his big conference. So that's happening next week. So he's all over the place talking. I'm going to read it. Okay. Okay. So that's happening next week. So he's all over the place talking. And yeah, so he wrote this book, The Lean Startup, where he attempted to take the scientific method and apply it to how we build products in a startup environment. So he, in a really brief way to talk about his cycle, he talks about how you build, you have a hypothesis that you can test, and you do this kind of evidence-based, experiment-based product development cycle. So I started thinking about how we might use that cycle. In fact, how we probably are without really thinking about it. We kind of apply that cycle to how we build teams. The clicker works again. So we are not going to start with build, though. And the reason we're not going to start with build is that unlike a startup who is making their first ever product, you very rarely get to start your team from scratch. Although if you are a startup making your very first product, that is the exception. When you are working with a team for the first time, that team probably exists in some form for most people. And you need to sort of learn some stuff about them. So this is my main departure from that lean startup model. So I'm going to just pause and sit with this kind of science analogy that he uses for one more second. Because at this point where we don't have a team, we're not building a team from scratch, we probably don't want to go in with a hypothesis. Because we're going to miss a bunch of stuff. We don't know anything about this team, let's say. Or we know a lot about it because we're already in it, but we're not really sure what we know. So when I think about the first step with a team, I'm not thinking I have a hypothesis. Now let me move forward with that and figure out what I'm going to do to test it. I'm thinking more like, it's more like a zoological approach. It's more observational. So if you're going to put it in those kind of science terms. So you're doing observational research really about your team. And your measure at this point is really open-ended. And it's really exploratory. It's much more from, I guess, from a UX. It's more from a UX skill set in that respect. So your UX team, also there in your friends. And if this is not stuff that you have done, it might be something to tap your UX team for. So we measure because we want to know where to focus. We can't focus on everything at once. And in a minute when we look at the product app or tool, it has 12 dimensions. That is a lot of stuff to work on if you're a team. It's really overwhelming. So you want to find a way to focus. There are lots of tools that you can use for measuring at this early stage. So some of you may have read Google's Project Aristotle Great Teams project. They did a lot of research. They have the benefit of having thousands and thousands of teams that have worked together over a long period of time. They did a really fantastic piece of research that identified a lot of things that were important for great teams. But the number one thing that they discovered was that psychological safety is incredibly important for successful teams. So, and there are evidence-based surveys that you can use to measure psychological safety. So tools like that exist, your HR software. What does that mean? Psychological safety. I think a working definition is like people feel safe enough in the team that they're able to kind of ask questions or challenge or enter into debate and things like that. They don't feel like they're going to get shut down or in some way like punished or forbid within that kind of professional environment. Absolutely. Yeah. But it's definitely worth digging out the Project Aristotle to work through a few good articles about it. Yeah, HR software, if you're at a place that's big enough to have a suite of HR tools, they often have surveys and things like that that can be really useful for learning about how your team is feeling and how they're going. Really all the way at the other end, does anyone use team temperature as a practice? No. So cool to be able to introduce team temperature. It's the simplest tool in the world. When you're either having a meeting, reflection, a lot of teams will do it on a regular cadence. You can ask the team a really simple question like how is everyone feeling today out of five and people will hold up their fingers like I'm feeling a three or a five. Or you can just do thumbs up, thumbs down, sort of a Roman vote on how's everyone doing. You know right away, it doesn't matter if you're not measuring this in a scientific way, it doesn't matter if they end up here or here, if they're knowing this, it's different from this. So team temperature is really great and simple. And product upward is the piece of paper that I handed you, we also have an application. This is a team assessment tool that we built at Carbon 5. And the reason that we wanted to build this is because we specifically wanted to look at product teams. So in researching this tool, I found that lots of people have written about how to make great products. There's tons of amazing research going back decades about effective teams and how to make your teams better. There was not very much, so a couple of things, but not a lot of stuff that actually got at the overlap between the great product and the skills and habits you need to make a great product and the effectiveness of the team building that product. So I wanted to make something that actually captured both of these things in a single assessment tool. So when we measure a team, we talked at the start of the sort of measure section about the fact that we wanted to find a focus, an area of focus by measuring. Did anybody, looking at your dartboards in front of you, does an area of focus jump out that we think, oh, we should probably think about this? Yeah, does anyone want to share what the area might be?  That one's like a, yeah, that's a nice idea. Right. Yeah, and a super important product dimension. Right. Like if the team is all able to, when I was thinking about these product dimensions, it's not just about like the metric or the product, it's about the team's relationship with that. So the team's understanding of what they're measuring and why and therefore their ability to make sure that they're actually driving towards that goal. It's really important. Anyone else want to share an area of focus? Sure. Yeah. I'm on a product that is delivering very, very quickly. By most sort of product metrics we're doing very well. But I feel all, I am really sort of burned out and tired and going through this, the clearest area of focus was clear responsibilities. So I'm realizing that even though we really understand what our product has to do and by when, knowing who's responsible for it and who ultimately sort of takes ownership and how fast they need to have done by is a real sort of point spot for them. That's really interesting. Thank you. Anyone else? Yeah, I would just echo that. Yeah, I think that and also like shared process because I think once you have like a good established process with like defined goals and responsibilities it can really help clear out like a problem is to occur. You can kind of find the root cause within the process what you need to kind of clean up. But yeah, most of the projects that I work on has been, a lot of the confusion has been about roles and responsibilities and then you get to this finger pointing scenario and then it's like alright, we gotta turn the other way and once that happens it creates kind of a friction between what I'm current and so on. Absolutely. Yeah, and that can be really hard and I think that what you two both said about the clear responsibilities and the shared process it kind of touches on how some of these areas we might not think of as being about psychological safety but both of those things really affect psychological safety. If people don't really understand their roles it can lead to these kinds of behaviors that make people feel like they're not safe to debate or challenge things or ask questions if they don't quite know what their roles are they don't know whose toes they're stepping on or somebody maybe is going to point their finger at them. That's not a psychologically safe team and you mentioned the shared process as well like it's the same kind of thing like we don't know what's expected of us we don't really know how to resolve this we don't have ways to resolve this it doesn't give people the safety of like a structure around them so all of these things kind of play into that larger topic of psychological safety but this doesn't measure directly. So I'm going to show you a real team now I have a little case study that we can walk through together so this team like those two teams we just heard about I didn't plan on this way the lowest score here so this is an aggregated score this is a screenshot from our app all of you have obviously just done one response from the team this is six people on the team and this is their aggregated dartboard so clear responsibilities scored the lowest for them in this case and then they have a few things hanging out in that kind of two's sphere, not much of three's so for this team though and these are all the dimensions we don't have to talk a lot about this we know that was their lowest the other thing that teams brought up a lot when we were researching this tool was that they didn't just care about the thing that had the lowest score they were really really interested in the variance so they wanted to know which things had a large kind of deviation in their answers for this team you couldn't see it on that screenshot but we have another chart that shows that their biggest variation was in clear shared vision so obviously you have more variation it's going to score a little higher probably everyone agreed their clear responsibilities was a problem for this team and for this team they were exhibiting the kinds of behaviors you described so there was there was kind of blame they also weren't sure about like who owned a thing and therefore they sometimes felt like other people were undermining them and taking responsibility or taking credit for their work because it wasn't really clear who was supposed to be doing what so this is not a pleasant environment to be in for sure but they all agreed that that was a problem clear shared vision some people gave it some 4s and 5s some people gave it some 1s and 2s and for this team they were like well we all know this is a thing but what is going on with this because it seemed to them more troubling that they disagreed about something as important as this and also that seems kind of weird right like if the vision is clear and shared then wouldn't we all say that it was so this team actually decided not to focus on their lowest scoring dimension they decided to focus in on clear shared vision this is where we go from measure into loan and you you're all product people so you know why this is the same as when we get metrics on our products but they don't tell us why and it's the same with your team so they know that they all agree that they don't have clear responsibilities they also know this weird thing right some people think we have a very clear vision some people don't they have no idea from doing this this is just a dumb assessment tool it's not smart, can't tell you anything so they have no idea why and some say no worries these are not soft skills this is a thing that I like to start arguments about frankly all the time does anybody else hear people describing this kind of like team building or leadership as soft skills yes lots of nodding does it annoy anyone else we've got a big debate about it sometimes I feel so alone these so this is the small beginnings of my fight to get rid of that word but these are not soft skills I don't think sure some of these you may use interpersonal skills and other warm fuzzy ways of talking to people communication and things like that we may or may not want to label those with a potentially pejorative word like soft skills but when you're thinking about moving from where you've done a measurement about your team and you're learning about how to change it you are really using a robust and rigorous process there's nothing soft about this at all and it's not particularly warm and fuzzy in fact what we're doing here is treating your team like a product you might not want to use those words with your team you might that might be fine but I mean I'm in consulting so our team literally is our product that is what we are selling I don't necessarily say that to them sorry Nicky I'm used to being soft so this is really where the work comes in the measuring part is kind of easy but the real work comes in when you start to dig into that and figure out okay why what is happening with this I can almost guarantee that this room is familiar with all of the things you already described one of them so I'm going to skip over five minutes and come back to it we use a lot of different activities for learning this is just a handful you could pull almost anything that you would use for learning about your product maybe that's not true but many of the things that you would use you see your metrics and go what the hell is going on there you can apply those to your team as well and things you're already doing with your team may also be helping with this so if you're doing regular retros hopefully you are because they're rad that's a really good way of understanding what is actually going on how might I match up the things that people are talking about in retro that they want to change with these areas that we've identified a couple of things again from UX I think that some of these UX skills are kind of underutilized when we think about teams team interviews especially in a team where there's not a lot of psychological safety or there's a lot of trust it can be really good to just do some kind of empathy interviews almost like a customer interview with your team to understand what's going on for people how they're feeling and just really get a picture of what's going on for the team as a whole observational research that might sound super weird the case study example though I will give an example of how we actually did this very informally but you might just be watching like how your team is working together and it doesn't have to be a formal study you don't have to time people and things like that you might want to but it could be just observing when do people go to lunch what do they do when they come back are they talking to people where they're sitting just noticing how your team is behaving physically in space is really useful mentoring I think is a really interesting one because we usually think of it as one way right like I'm mentoring you I am giving things to you it's kind of broadcast but that isn't actually true mentoring is two way and you're hearing things from the person you're mentoring they have questions they have concerns they have areas they want to develop you can learn a ton of stuff from mentoring people and 5WISE we'll come back to you because I'm going to use that with my example team does everybody know what 5WISE is to have you use it some people are shaking their heads ok so 5WISE is a technique that comes from lean manufacturing and the purpose of it is to do root cause analysis so the idea I think it's yeah it's from Toyota right and it's in the Six Sigma process really it's you behave like a three year old and you ask why a bunch of times that's what you do and it sounds like you use that in I work for Toyota that's why so you have done this I work for Toyota so yes I'm very familiar with the 5WISE it gets beaten into you day one that's awesome and now kind of intimidating because now I'm like I probably don't do it but really when I use 5WISE with clients it's just really about digging into the things and you ask why the idea is that you ask why but which sounds really simple but you have to figure out at each step like what's the next why I'm gonna ask like what am I gonna focus in on and really you're looking for a kind of a revelatory moment a thing where you're like oh that is really the thing that's the root cause of our problem um sorry I have time so we're gonna I'm just gonna run through this 5WISE you don't have to read all of the things but this is more or less the 5WISE that this team went through the first thing we have to do with 5WISE is not actually ask why the first thing is to understand what our problem statement is so for this team the problem wasn't oh we scored low on this dimension um or we you know didn't like what we saw on this assessment or we think we don't have a clear shared vision none of those things was actually the thing they wanted to know about the thing they had a problem with was that they didn't agree on whether they had a clear shared vision or not that was their problem so why why don't they agree about this this crazy team so it turned out when they asked that question of the group this is where they had a reasonable amount of trust on their team um it turned out that the PM's so their product managers had communicated the vision and they documented it and they'd done a pretty good job and they felt pretty happy with it and they felt like people were reflecting it back to them accurately they're like we don't see a problem developers were seeing an issue not with the work that the PM's had done um so there was no disagreement there uh but they saw in their day to day work that there seemed to be kind of different interpretations of that vision coming out as they as they went about their work um and when they saw features you know coming down the pipeline they're like oh well somehow isn't matching up we don't seem to share this vision even though the PM's did good work so why why was there this disconnect for these guys um their developers were working in silos and that meant that the vision was being made a reality in lots of different ways by people who were not really communicating day to day and I want to pause at this point and say that what this team did and the the results that they found and the things that they did to fix it uh by no means the right answer for every team so there are plenty of things that happened with this team that they identified as a problem that would be completely fine on a different team in a different context as you will see why are they so siloed because people were working on one part of the code base and not on anything else now this is I don't know if anyone comes from a dev background this can be completely fine and even necessary especially you know you have giant we work with enterprise clients with giant code bases there's no way that everyone is startup style working in every piece of the application so this is not wrong but they identified it as a reason that they had these silos and they had these these experts were being created so they said well why do we work that way why are we putting people in these silos this is awesome because we were thinking about okay it's not necessarily wrong that we work that way but why do we work that way well it turned out they had inherited a whole team um it was a robot team actually um and they knew one part of the code because they had built that part of the application and they reshuffled the teams they joined this team and they knew that part of the product so they didn't change that they were like well you guys know that part you keep on building over there but why do they continue right like why did they not ever question that and just went okay well this is our silo we're gonna work like that they just it just never never occurred to them to work in a different way and these are the kinds of things where teams often you know you fall into habits you have a way of working especially at enterprise but not only at enterprise level and when you don't deliberately make these decisions you can fall into a place where perhaps it's not ideal that we're working in these silos is this actually what we need to do to build that product so as they're going through this process they're thinking that the learn stage includes some ideation right like what are we what can we do about that um what do we need to do to change the way we're working we agree that this is the problem we think this is the root cause um how might we change it so this team actually decided that the way they were going to change that was to pair more um now in our did you just say pay more pair is it like pay more money oh they're engineers need more money no maybe right but that's never gonna happen um no uh pairing in uh in engineering teams um can be really really valuable so this team was not pairing at all and they said well we are in silos we don't like it we want to break them down so we think that the mechanism for that is going to be to pair then we're in build right now we talked already about the fact that we don't get to hire and fire on this team so what actions can we take that uh that can build this team by building its effectiveness and building its capabilities not by building its um headcount for this team uh we built a team roadmap so I think maybe half of you said that a product roadmap was a thing that you had done in the past um I started making these team roadmaps mostly because I didn't quite know what to do I knew that I wanted this team to get to a place I knew that that couldn't happen immediately and for me being a product person I was like okay that feels like a roadmap let me make it look like a roadmap so I made a roadmap uh so I'm going to stick with my example team and their lack of clear shared vision and their need to do some pairing so what we actually wanted for this team um wasn't for them to pair 100% of the time um it was for them to become an autonomous team that's awesome um how do we get to autonomy has anyone ever worked with a team of engineers and said be autonomous just go be autonomous has anyone ever tried that not that great this team had actually kind of tried that there was in their organization there was a lot of focus on the idea of autonomy like you should be able to be these little autonomous units owning your part of the product and we had a bunch of cowboys being like I will make the decisions I don't need to talk to anybody they were super well meaning and pretty good engineers but some weird stuff was getting built so they still wanted to be autonomous even though they just identified that as being their problem yes they wanted to get to a point where they were truly an autonomous team not necessarily autonomous individuals and certainly not random people going off and building weird things in the way that they thought which was kind of how they were interpreting autonomy but they wanted to be a truly autonomous team i.e. they could build their product and work on their product and release on regular cadence and not have to kind of be waiting on other teams and figuring out a whole bunch of dependencies and moving in lockstep with a giant road map that never moved forward and basically ending up with a product that wasn't getting released on a regular cycle so they wanted that kind of autonomy as a team we wanted them to get there right but if the team is not aligned then autonomy is really dangerous because then randomness happens so what they wanted to do first actually was not jump to autonomous let me move and it wasn't even jump to aligned like try and hammer it in more the PMs have done this great job of sharing their vision and yet were still not aligned what they actually wanted to do was be collaborative and weirdly once we started road mapping clear shared vision kind of happens here in the middle right so we've now discovered a whole big milestone a big chunk of work that has to happen that clear shared vision is sort of dependent on so milestones we broke that out into some milestones like how are we going to know when we're getting towards these like getting towards these types of things and they came up with milestones again these are not milestones that you should copy down and use in your team they're very specific to the team but they one that was really interesting to me was this one this idea that people weren't doing releases they had one team that did releases no one else knew how to do a release like you could say go release the product and they wouldn't know how so they identified that as a thing like when everyone on the team has done a release we'll know we're getting somewhere but first they were like we should be working in all parts of the code base we're going to pair them up so that they're learning the different parts of the code base that's what they're going to do which leads us to the last part which is the tactical actions so how do we get there yes was this still a group of six people that you showed up with the initial yes so they did that initial assessment there and they kind of worked through all of this with their management and us as well we weren't included in the six at that point sorry the previous slide why is it not first aligned and then collaborative because it seems like people are going to collaborate but then they don't have an aligned or shared vision right yeah that's a really good point and again like it depends on the team exactly how you would order these things for this team they had identified that they kind of knew the vision they thought they knew the vision but the work that was coming out didn't actually reflect the vision so when they were delivering stories they were just a little off and they didn't quite you know one feature would be this way because it's one interpretation of what they thought the PM had said and another story would be more like this and then they end up with rework right because then the PM is like oh no I man because they've gone off in their silos to actually do the work so that they think they know what they're doing and then what that comes out is not quite right so they felt like the reason they weren't aligned was actually not that they needed to talk more about getting aligned they thought that actually the thing that would change that was the collaboration and that if engineers were working together and talking and then collaborating more closely day to day with the product people that that is how they would get to alignment so that's why for this team but yes this is not like an order of operations that is necessarily right for any other functions tactical actions the tactical actions so the big tactical thing that they wanted to do was pairing and they wanted to figure out how to achieve that so one reason that this team perhaps didn't pair was that they didn't really know how pairing is a skill and it has to be learned and not every developer wants to pair and not every developer is good at pairing but this team had they came up with these actions themselves so they were not opposed to pairing and yet they weren't doing it so that led management to sort of think more why, why aren't they so many wise, so many wise why are they pairing if it seems like it would be helpful and they don't hate it and this is where the kind of observational research came in I don't even remember who it was but somebody on that team looked around somebody in a management position important, they had money somebody who could spend money looked around and said this really isn't set up this space is not set up for pairing little rows of desks sort of like this big and they had a little not very good monitor little crappy monitors on each desk on an arm and they sat near each other so everyone's all in the one space it's hard to like have conversations when you're right next to somebody else and it just wasn't space there really wasn't space to pair to be at like a shared monitor and be comfortable pairing and I don't know if you've seen developers sharing a lot but you know they really need that big monitor they type, the two of them will both type, they'll have a keyboard each often and they'll have a single monitor maybe a monitor and a laptop and they're working together on the same piece of code, they're talking about it they're problem solving together and it's a super effective way of working through difficult problems it's also an incredibly effective way to share knowledge so for this team it was more about the knowledge sharing aspect that we wanted to do it so this genius manager looked at this issue and said you know what we need pairing stations we need to make a pairing station so they had money to order a big monitor they ordered a couple of big monitors some extra wireless keyboards some mice laptop stand, put your laptop on and created two pairing stations kind of at the end of a row like a really large desk and that gave them gave their team the space and the conditions that they needed to be able to do pairing I like to tell this story because it always makes me kind of stop and think like would I ever have thought that when I saw that my team needed help with clear shared vision I should buy them new monitors and keyboards that is not how I would have gone about solving that problem if I hadn't gone through these steps with this team and seen it happen I never would have thought that would that would help and it's important to note that this little tiny tactical action is also not the thing that's going to be the game changer this is not magically going to lead to the vision being clear and shared but it's a step along the way it's a baby step a measurable step so this is a circle and at the beginning we talked about the fact that you keep going after doing this more observational measurement that is not tied to a hypothesis now we've actually I didn't pause to talk about them we've kind of got two hypotheses running here though we have a hypothesis that pairing is going to bring us closer together and bring us into alignment around our vision that's a hypothesis we don't know if that's true it is measurable though and the other one is that people will pair more if we provide them with these pairing stations and we think that ultimately these things are going to shift the needle so once we're at this point we no longer we might want to repeat the product dartboard or some survey maybe on a quarterly basis that's completely valid and we should do that and see how we're doing overall but now we're in experiment land where we need to be measuring the results of the experiments that we're running so for this team they can measure whether people are pairing more that's actually super easy in their backlog tool they can see how many tickets or how many stories have two developers' names on instead of one and they can look at one iteration to the next and see if that number increases and they probably maybe they want to say it increases by 10% or 50% their hypothesis might include the number that they expect it to grow by so that's that hypothesis pretty easy to test and for the other one about like we think that pairing is going to lead to more of this knowledge sharing slightly tricky you might have to think a little harder about that but for this team for a lot of teams you can actually look at metrics around whether things got accepted so we were talking about that churn that the thing coming out the other end wasn't quite right so stuff would get rejected or there would have to be rework from the product manager as long as you're actually using a tool that tracks it got rejected I had to do rework you can actually measure that like did I see fewer rejections did I see more acceptance the first time round of this work that was happening when they were paired when they were paired yeah exactly and even you might look just overall you know in this iteration did we see more acceptance more quicker than the other one than the previous iteration or iterations and you would expect to see that changing over time yeah so some of those tools that we often tell teams not to to use we often tell teams not to be metrics vision driven don't you know count how many points you did in iteration and show the teams progress that way but you can actually dig into those tools to answer really specific questions and they're incredibly valuable so thank goodness that they give us all those charts that is all of the stuff I have to talk at you about tonight but hopefully the DARPA improved helpful to you and absolutely happy to take questions or if you want to chat more about the stuff that you found right there thank you for me I'm looking to start an app so I'm not a product manager but I'm trying to learn the ins and outs of developing a team and all the parts that I need it seems I don't know if your company provides services for people in my position we do have the capital ready to go it's just the building team are you a new client? yes we do you work with larger enterprise clients we're not that by any means we work with about this is not a sales pitch but we work with about 50% startup clients and then the rest is a mix of enterprise growth companies some non-profits so I think it's about 25 to 30% growth companies same for enterprise so yeah we absolutely work with startups and so you are the lucky person in the position of being able to build a team from scratch and build a culture from scratch so for us we're kind of making a decision between trying an agency to get the minimal viable viable product hiring a CTO to manage that process so whenever it launches and we want to bring it in house what was built and what was designed so we're thinking about that solution or we would love to build a team internally but we understand that takes time as well too so that's kind of where we're can you share what kind of app it is we started thinking it was going to be in dating but then it kind of we really looked at the solutions that it solved in the features it would be more adult men's entertainment adult men interesting yeah I think we often find with startup teams that we're working with because we do like to be collaborative and we don't tend to build a product and hand it over so we're building a team at the same time as we're building a product so a lot of what we do is focused on how do we get this team to a place where we can hand this over to them and they can keep carrying it forward because otherwise you can build a thing but if you build it kind of like this team if you build it in your silo then the company that paid you to build it actually doesn't have what they need in terms of their skills and in terms of the team they've built to take it forward so you help source teams too we do help with hiring yeah we help with interviewing and then we repair that's the main thing we do we pair both functionally and we'll pair developers and we'll pair product to people to skill up that new client team do startups usually work in like a single office or they work remotely or with our startup teams often they'll come actually work co-located with us in our office I think especially with new teams it's super valuable if you can actually be in the room together it builds a lot of trust and that psychological safety aspect that I talked about earlier so if it's at all possible it's awesome if that can happen and a lot of our clients, startup clients in particular will come work in our office where's your office? we have an office here in Santa Monica as it happens Joe and Nicky we can take soft lines I'm from Orange County so thanks everyone for listening does anyone have any public questions? yeah how do we use this for nonprofits? how do we use it for nonprofits? I'm trying to think I feel like we must have used it with one of our nonprofit clients we have based it at a couple of nonprofits I personally haven't worked on a nonprofit project for a little while but other people have used it I was at a nonprofit I was at the National Museum with you yes not so good on scientific method not awesome on measuring things very often but I think it can help any team or not even necessarily this tool although you are welcome to use it but any of these types of tools can really help yeah yeah so I know that we just went through maybe a portion of some of the for your team study that you kind of identified to kind of review here with us but when they I guess can you tell us a little bit about long term what were they planning to do once they've tackled that one with clear shared vision what was up next in the queue in terms of making sure their team is optimizing on all the other maybe weak points that they have within the team because then that problems can shift into a different direction if maybe some are addressed and then also there's so many dimensions here yeah for a team I guess I want to get your thoughts on like stepping back a bit and looking at values with within the organization or within the startup or whatever nonprofit you're looking at do they try to maybe align with those values to also kind of make sure that they're keeping that in mind too so I think the answer to the second part first I think always has to be yes like whatever it is that you're doing whatever tools you're using to measure and figure out what to do differently with your team your team always has to grow in a way that is aligned with the organizational values although not always in a way that's aligned with the existing organizational culture because maybe you're part of trying to change that and those two things often don't match so yes I think that they would want to align with the values and hopefully those values are not too out of depth with these kinds of things I think this you know some cultures and even values can be very oppositional for example the research that we use when we put this together tells us that that is not a great way to make a successful team so if you were actually trying to build an oppositional culture you would because you felt like that was going to I mean they do tend to those kinds of teams tend to grow like this so they tend to have a lot of success early on those kind of very very oppositional very competitive sort of a very achievement driven cultures can have a lot of success really fast but then it tends to taper off it's not very sustainable so there is good reason to go for that if you want that early fast growth this is going to tell you that you're weak in some areas that's what you're trying to do that was a long answer to your question but I wanted to go back to the other part just coming from where I used to work I just know that I've been presenting so many different types of tools like this there's like thousands of them out there that you can look at and they're all fantastic but I guess my question is what it sounds like you've worked kind of a full gamut of like start-up more you know, stapled organizations to non-profit which is completely different have you found that the tool has provided you know, a good general kind of foundation to align with all those different cultures because they're going to be completely different so far we have on the teams that we work with at Carbon 5 so that's some caveats but we I think that we tend to work with teams that also align with our values so that maybe biases these results for sure I can't remember the number off the top of my head we've used it with around 30 teams so far I think and that's spread across across all the types of organizations that we work with let's see yeah I believe and this is this is biased of course because I'm a human but I believe that the evidence that it's based on should apply across teams and organizations regardless of their culture like that you are going to get better results ultimately if you you know, if you build psychological safety in your team and and there are some other big bias assumptions especially on that product side and this is where I get pushed back from Enterprise a little bit I think you should build an MVP so I think having the smallest hole and getting it out there like I drank that Kool-Aid some people don't agree with that especially in Enterprise I feel like there's sufficient evidence that I wanted to include it in here that that is actually a better way of releasing products but you know that's an opinion it's an opinion with some evidence to back it up that you could find evidence the other way yeah what if you mentioned psychological safety then what and where does this fit there were like five tools I think there were some H there was like HR tools there was the psychological safety surveys and I can't remember maybe two other things on team temperature how do you use those in conjunction with this you know you could use really it depends on what you have your fingertips the size of your team and your organization and I guess what you're comfortable with and what your focus is so we use this a lot of the time because we're always working with product teams and because this gets at both the product and the team effectiveness part that's really important to us so we want to make sure that we're actually including that if you do a psychological safety survey it's not going to it's job is not to understand the product side so that tool is not going to measure that but it will do a really good job and you know a really evidence based job of measuring psychological safety so if that's a big concern on your team then that might be the kind of thing you can Google them definitely there are psychological safety surveys online I forget if you have to pay for them yeah that's fine do you feel like that's a prerequisite for getting into this discussion like if you didn't have psychological safety could you bring out the dark board or you're just sort of you got to fix the safety element first I think I don't think it's a prerequisite we've certainly had some teams some very conflict written teams where I think that there was probably pretty low psychological safety and they actually used this we had a project lead who used this really deliberately because the team had very low trust and there was a lot of blaming going on and he was when he was running regular retros which I think were happening weekly he was finding that he could not get that it was just there was lots of conflict he couldn't get the team to be honest about things he couldn't get them to talk about the stuff that they would talk about one on one so they couldn't come together as a team and problem solve and it was pretty harrowing and so he actually grabbed this we didn't have an app then he was using a paper version and he started to use this in place of a regular retro and the thing that worked for that team was that this gave them very unemotional categories to respond to so they they didn't have to come up with their own things which were like leaving with emotion and blame and this person said this and I didn't like it they were like oh well I feel like we're doing this on this this abstract thing like just abstract it at one level that helped that team so I don't think there are hard and fast rules about that I can imagine I actually had a team recently at Carbon Fire not a team I worked on that refused to use it because of that exact reason they were like no this team is so low trust and if we do anything that might rock the boat or expose they were really afraid of exposing things that would make the team feel upset so they jumped the other way and they said we don't want to go there and ask questions that are sensitive we don't want to talk about them they went what way? they just didn't do anything you're coaching now they chipped away at the challenges that they had for sure but they they mostly did it one on one with people so they worked one on one with the client team members they weren't just like the culture to save face that allowed for problems to be abstract enough where people could talk about them exactly there was an area of disagreement especially on the team where some people on the team fundamentally didn't believe in the product and that was what they didn't want to surface they were like this is supposed to surface product issues and we don't want to do that so for them they were like we are going to focus on different areas to ignore this they're fine they're fine now