It was pretty excited to come talk here for a couple of reasons. One is there are twice as many engineers that graduate in India than they do in the US, graduate about a million engineers a year. We graduate about 500,000 in the US, and you'll hear that this talks really focused on engineers. It's one of my passions. I am also pretty excited about what happened today. The talks that happened in this room today are really good predecessors to what I'm about to talk to. I think you'll see the thread that runs through all of this from today's talks. I'm the CTO and Founder at Rally. I run a group of folks into my office that work on advanced services, and you'll see some of the stuff that we're working on as we go through this talk. We have a variety of products and services that you may or may not be familiar with, and the Rally product that's been around for 10 years. We'll talk about some of the data that comes out of that solution, and some of the insights we've got from that, as well as Idle's End and FloDoc, which is the newest product that we've got for team collaboration. Definitely, we're checking out. Totally changed the way our teams work and how fast they work. How many of you in the room are engineers? I'm guessing we're going to be way over 75 percent. That's what I was hoping for. The question to you is, what do you want to become? Do you know the answer to that question? Happy. That's a good answer. I wonder what's in the room. I wonder, is that one of the answers? I want to become the CEO. That's a standard answer. I think this one comes up more often. I want to be a great engineer. That's why you got into it. The question is, what is a great engineer? What's that look like? A lot of cases, it looks like this, that the chief engineer, the technical founder on a young startup, or even a larger startup. I'm going to put a different notion into your head, and we'll talk a little bit about this one, and that is a global or a citizen engineer. This is a friend of mine, Dave Douglas, who was with Greg Papadopoulos, who were the Co-CTOs at Sun Microsystems. Dave's book talks about the notion and the concept of engineering that is technically responsible, as well as environmentally responsible, as well as socially responsible. That notion of an engineer that can find the sweet spot between solutions that have all those attributes to me is something amazing to strive for. Maybe I know what you're thinking. Is this something my boss is going to let me do? Where do I find the time for this one? Can I afford to break out and try to do this as a startup? That's always a good question. It hits everybody right in the gut. Can I figure out how to solve the wicked problems that are found or that are wrestled with at the center of that intersection of those three circles? Can I really drive innovation? Do I know what that looks like? Those are tough questions, right? Those are things that Sheryl Sandbard might say you need to lean in on. You don't know until you try, and until you do try, you're never going to know. So I'm going to borrow on Christopher Avery's notion around responsibility. If you haven't seen his framework, it's at ChristopherAvery.com. Some really interesting, simple models to work with akin to some of the notions in five dysfunctions of a team. But this is the opposite side of that of how do you take responsibility? It's a pretty simple three-step process, and it really comes to play a lot. That you have some intention that I actually want to lean in, and I actually think this is something I want to do. When you have that intention, you tend to move to a second step of awareness, and that is now you start to understand what it is that's necessary for you to become that thing or that person. And finally, there's that crossing the threshold notion of I'm ready to confront this issue. I'm actually ready to say I'm going to do this. I'm going to become the technical founder and startup. I'm going to work on solutions that actually figure out how to make not only a great technical solution, but one that actually makes the world more sustainable and the society that we're working in better. But that's the confrontation I'm going to ask you to think about, but we're mostly going to talk about intention and awareness. Well, let's explore this problem of what it would be like to be a citizen engineer, and what it's like to work in that domain in the center, right? And it goes back to what where a lot of us were taught, right? As engineers, we've been through kind of a process, and this is my rock geology professor from 1985, a gentleman by the name of Bernard Amade, and certainly I did plenty of problems on this kind of paper. In the U.S., we call this E2 paper. I don't know if you all in India use the same templates as we do in the U.S., but this is what we use. It's green on one side, on the other side there's a graph, so it's convenient for being able to not only lay out your problems in your given statements, but to find your solution. So what do we start with on the top of this page, right? Write down what the problem is. Usually that's given to us. Then we work through an assumptions, a free body diagram, a little bit of an understanding of what's in the context that we're working in. We put some frame on this notion, and then we work to an answer, a numerical answer, right? We come to an answer. And as a good engineer in a four-year program agreed, you're gonna work through some amount of these.  If you're lucky to be an engineering physics major, you're gonna do 5,000 of these.  To start with a given problem and get yourself to a numerical answer inside of boundaries, conditions that you describe and set up very effectively. That's a great set of skills. In fact, I would argue that is critical for being able to solve some of the toughest problems on the planet. You have to have the discipline to be able to work through those kinds of things. But there's some amazing assumptions that we make when we go through that process. And most of it has to do with this notion of we work inside of a set of boundaries and we think about the world as normally distributed. In this picture, if you were to stand up a set of, put a small crowd together and we were to measure the average height of all the men in the room, we'd find that somewhere in a very Gaussian distribution, heading around five foot nine. But what surprises us in this world is that the whole world doesn't look like that. If you draw some from some of the concepts and some of the work that the folks at Cognitive Edge have been doing and others in complexity space, you find that the world has a Gaussian distribution in the center, but then there's a big Pareto tail off the end. And when you find those answers out on the far end of the tails, they aren't very probable, but they're plausible. You might pick on the BP oil spill in the United States. You might pick on Fukushima nuclear power plant disaster in Japan. You might pick on the last amazing hurricane, or typhoon in the Philippines, right? Those disasters though, though plausible are very impactful because they are way out on that distribution, right? And that's a little bit of what we're having to deal with here is we have to deal with the notion of things that are way outside this little safe boundary that we as engineers drew in that little free body diagram that says, hey, that house will stand up inside of these contexts. But that's not always the context we're dealt with as engineers. And so those low probability events, they have really high impacts. And how do we deal with that? How do we process that? I mean, as engineers, we're pretty much trained down the path of, hey, let's bring the expert in the room like Todd was talking about. And if you put a bunch of experts in the room on some of these problems, in a lot of cases they'll lead you off into the wrong path. So what is it that's plausible? Here's another quick example of that. If we were to take and stand the tallest man in the world and the shortest man in the world, we'd have a 3.3 difference. There's a Gaussian distribution. It's not that big a difference, right? We can engineer inside of that kind of a domain. But if we were to have somebody special walk into this room like Bill Gates, and we were to talk about income equality inside this room, there'd be a 650 times difference between the income in here. That's plausible. I could imagine Bill Gates, I'm sure he's been to India numerous times. I could imagine him showing up at an Agile conference. It would actually skew the room a little bit, right? So that's the world in which designing with complexity. You're designing with Pareto curves as opposed to designing with Gaussian curves means. It means having to deal with an engineer in a space that is uncertain. If you've seen any of the work from Cognitive Edges is a very simple motion of how to lay out a complex space is a continuum, right? From simple concepts to complicated, complex and chaotic. The question is how do we engineer with that? What do we deal with to figure out those pieces? A lot of what we've been dealing with in engineering for through its educational system and through the hundreds of years that we've been architecting physical structures is a pretty known complicated space. Where really supply was the limiter, not demand. Now things are changing on us, right? So if we think about this and we split this right down the middle, we can talk a little bit about what these two sides of the world are like. In one notion you can think about mechanical systems and physically built systems on the right hand side. The civil engineers, the engineering physics folks working with materials. And on the left hand side, folks working with the natural systems or social systems. And I would say us as software engineers working in our domain, right? We talked a lot about today and this thread was brain science, right? So the number of neurons in the brain roughly equal the number of transistors in the web right now. The number of synapses in the brain roughly equal to the number of links. We're engineering in a complex space. We're not engineering in a complicated space anymore. We've got to bring other forms of work and other ways of approaching it. And if you started at Ash's talk this morning and moved through Jess's talk and then into how to bring creativity and really better ways of working into your team with Kara's talk, you're starting to see some of the patterns here about what we're really programming is a different way of working. It's a different way of thinking. And those are the things that to me are necessary to build citizen engineers. So let me give you an example from the engineers of that boarder space. It turns out that when I started working on Rally in 2002, this was kind of our release board and our early team. And kind of every time we dropped a release every two months, we got a sticker. We've since moved to continuous delivery and so stickers don't make any more sense. But when I started working on Rally, at the same time, my rock geology professor, Bernard Amade, held the first engineers of the boarder's conference in the US. That group now is up to, I think it's up to about 50 professional organizations and 200 university organizations in the United States, almost 300,000 members in the last 10 years. It's been an absolute explosion from Bernard and he calls it his good work. As a result of his good work, we also saw the folks in Canada form engineers of our boarder's teams. And they've done a very spectacular job of cataloging failure. And there's a, if you look it up on the website, or if you just search it on Google, you'll find the catalog that EWB Canada has done on failure. And we'll talk about this TEDx video. But this is a kid here from Uganda, his name's Enoch. He's really happy today. What he's happy about is he's got fresh running water. And that was, as a result of an engineers without boarder's project, the team came in, did an assessment, built their self a sand filtration water system for his village in Uganda. And then through the process, they step back from what was going on and they do an after-action review. They come back six months later. And what do you know, the sand filtration system is working or not? Not. In fact, it's not working. And neither is the US one that was built two years earlier that's sitting actually a quarter mile away that the Canadians never noticed. Why'd that happen, right? So the Canadians were smart enough, I think in this case, to go ahead and do enough and figure out what was going on here. What happened in these cases? Why was this, why did this not sustain itself? When they do root cause on it, they'll go to a couple of things. One is it was installed by somebody else. There was no ownership in the village. There was no notion of why the village needed it. In fact, the reason the sand filtration system worked is because there was actually a faucet on the end of it. And what happened was the kids thought they'd never seen a faucet before. So they turned the faucet on and they used the sand filtration system and put a hose on the end of it and they played in it. And the sand filtration system was built and designed for about 40 gallons of water to filter through the system today for the small village. Well, they blew through that very rapidly as a toy as opposed to in use of actually drinking water with totally unintended consequences of actually then killing off a second water system and had to go back to walking again. That's the notion of engineers designing right inside the Gaussian distribution, building an amazingly effective sand filtration system for a village of 40 people and enough water for them to not have to carry it at all. And then not having the wherewithal to understand the real social dynamics that are going on inside the village and to really didn't saw something that was sustainable or that actually made the village better. That's the world of engineering with uncertainty, right? Here comes a set of engineers with a great plan. They delivered an amazing solution, but they really failed to repair. They didn't do the work. They didn't sit with the users. They didn't understand what was going on. They didn't even notice that the Americans had done something two years earlier in the same village that had failed. It's lack of preparation. Something to do with planning. They did some amazing analysis. They knew exactly how much water they'd need for the village. They knew how much sand to build. They knew exactly how to size it. They knew where to draw from the aquifer, but they didn't have any way of actually working through it in incremental and iterative fashion, right? They came in, they were there for a summer. They flew in on their summer break. They had designed it from an earlier trip in the spring break. They installed it. They patted everybody in the back. The kids thought it was the greatest thing in the world when they left. That's not engineering with uncertainty. They executed fantastically. Too bad they didn't know how to explore. To me, that's one of the big things that we're missing in the engineering discipline and almost gets worked out of us in an engineering education is that we've got the discipline to execute on a plan like nobody's business, but we've forgotten the ability to step back and sit with users, the ability to actually let the thing emerge, the incremental delivery, and to be prepared. To be prepared to get lucky, to be prepared for the uncertain, the plausible to even notice what could actually happen. We get so wrapped up in this model of annual budgets, comprehensive analysis, detailed plans, and waterfall delivery because we are experts and that's where we're trained in. Instead, if you assume a world of uncertainty exists and you look through what Todd was talking about and others today of maximizing the actual value delivery and working with that soft fuzzy end, you get a very different model of how this might look, right? Of something based on incremental delivery, something based in a world of empirical learning with disciplines around exploration and the ability to adaptively steer, you maybe get yourself a different programming model or different operating model for a business. So maybe there's a six step process here. I'm not gonna say this is linear. I'm gonna deliver it to you in a linear fashion, but I do say that the first one's pretty darn necessary and if you don't get started with agile execution, you don't get the mindset of incremental delivery. You move your way past, you get yourself even in, if you pick up design thinking or systems thinking, you tend to pick those up, you tend to come to a design and you implement that design and move on. Obviously the Scrum machine that everybody's taught as a pattern is a really effective way to get yourself into the system. I picked this up out of the Safe Framework as an SBC and I think what most of us are coming to in the world is that it's actually, we always said Scrum 1, Scrum 1 from a brand standpoint for sure, but it's XP practices that really delivered the engineering expertise and led to things like Jez was talking about today of continuous delivery and continuous integration that these two are actually inextricably linked if you're gonna build a successful team and that's really what your job is, right? The engine only runs because the team's got the capacity to creatively think and understand what's going on and work through that stuff. We've done some analysis on teams because we have a product that's been in the space for 10 years. We actually have about a 70 million objects under management in our SaaS environment. We have a variety of on-prem customers that we don't see their data, but we built a framework called the Software Development Process Index and we did some analysis on those 70 million objects and those 80,000 teams that exist in our data set and we looked at a bunch of different process types and this relates to Todd's talk from a minute ago. We see really four different types of models exposing themselves in our data set. One is people who work only on task-based hours, people who do no estimates, people who do full scrum, story points and task hours and then people who do lightweight scrum and find their way into that place that Todd was talking about of where's the right place of how much estimate to do and I think it's right for most people to get started with full scrum. It gives you, you'll see in a second the results of this but you'll see why a lot of people move to lightweight scrum. So that software development process index has four criteria to it or four categories of a balanced scorecard and that is responsiveness, how fast you respond to changes, the quality that you deliver, the productivity and the predictability and so there's a stacked bar chart of those four processes and their delivery capacities and we actually see that overall lightweight scrum does the best. There's only 6,000 teams reporting in that data set. There's 48,000 teams reporting in the full scrum data set but you can't tell on this picture but the green bar in the middle over there, that guy is actually tallest. So quality's highest in full scrum. So not surprisingly that you'd get that if you're doing more estimating and really understanding exit criteria much better than you would if you're moving a little bit lighter but actually we see a little bit higher performance in a team that knows how to balance how much value estimating and cost estimating to do. So that's driven us to some conclusions that just come out of the data, not from this is truly heuristic data over time.  Kind of makes intuitive sense. If you've got a big team and you're running really effectively above seven plus or minus two, that there's really no evidence that you need to change that. That comes from just analyzing the data. I think this one's more interesting. This is the one of, you can actually cut your time in half in your process if you aggressively control WIP. So if your WIP limits are really low you will deliver much faster. Little's Law playing out perfectly in this setting, right? You'll have a quarter as many defects too but you'll be 34% less productive. What do you want, right? That's your trade-off decision that you're trying to make. This is what happens when you look at this data and you incrementally improve you build a great scrum engine, right? If you wanna learn more about that there's all of this is diagnosed or kind of laid out in a bunch of findings that we did in a white paper called Under Agile Metrics. It's being refreshed right now with a whole bunch more, especially in the area of engineering practices. So I think where we're heading with this is if you again take the approach that was challenged a little bit earlier, I think it was yesterday, one of the folks said, I wonder if everybody really wants flow. If everybody really wants to find themselves in a place where they're really driving cycle time down, increasing life cycle profits, trying to bring variability in the system through innovation and trying to drive up speed. That's kind of the argument in Don Rennerson's book of, this is if you're running a product development process and you are trying to discover what is the best thing to build and what users like, this is the world you're gonna run. This is the world that Ash was showing us this morning, right? This is the assumption of the operating model that you'll be working underneath. If you do that, step two of this is find and protect some of your slack. All right, it's so easy to give your slack back to the system in the form of more throughput, but if you don't preserve some of that slack, whether it goes into 20% time or other forms of improvement, you don't find yourself, I think, getting better and actually being able to explore. To me, that's the next key piece. We've seen the lean startup pictures a couple of times here around build measure learn. If you don't have that slack in the system, you don't have any time to run multiple AB tests. You don't have time to be able to build two or three of the same thing. And so that slack to me is a critical next step. Then, now that you say, hey, I really wanna explore a little bit more, I wanna balance the discipline of continuous delivery with the ability to test a number of things as they go through the system or back up like Ash did this morning and say, we're gonna not ship that feature because it doesn't have the pickup rate that we need. So to have the ability to throw away stuff like that, we have to add the discipline that's necessary for the scientists have. And when I think about scientists, I'm not thinking about rocket scientists, I'm thinking about scientists like this 11-year-old girl in my son's class. I'm assuming this happens in all your public education also. This is called science fair. These science boards are what get picked up and you can see this gal, she's got it laid out perfectly. She's got her hypothesis laid out or results and her conclusions. It started the same way we did in chemistry, right? Got out our lab books. We wrote up in the top right corner. Here's what we think's gonna happen. Here's the experiment we're gonna run and here's our results. We're gonna share that with our peers and we're gonna potentially repeat that process. That thinking got left in the dust in a lot of cases as we moved in as engineers and continued to execute towards a specific number. So if we go back to the world of managing on this continuum of chaotic complex, I don't want to end at this spectrum if you're truly in a system down standpoint, your operations environment's going crazy. Yeah, you do need a small firefighter team. They've gotta go in and they've gotta do something because they've gotta figure out what's happening. They can't just sit there and analyze the answer. You gotta do something and see how the system responds back to you. On the other end of the spectrum, it's pretty easy, we give you a recipe. Scrum is a recipe, right? When you're early on in something, following the recipe is a great way to get started. If you're engineers, obviously, we'll get out our E2 paper when I analyze the solution and if we're actually working in the domain of trying to explore at the same time, we need to get our lab books out. This is the world of being able to balance exploration and execution. This is the world of firefighters, bureaucrats, engineers, and entrepreneurs and you need to be able to be both and that's what I see in citizen engineers is the ability to both act like an entrepreneur and discover, but also act like an engineer and be able to deliver because if you're gonna affect and dent the world, you've gotta be able to scale these solutions. So the step after this is adding the empirical learning cycle.  So the notion is if we start with hypothesis, we need to figure out how to turn those into knowledge. We need to use the exploration process as a way to understand the system as a whole, right? That's what we're doing. We're working in a complex domain. We have to do probes since respond is what David Snowden and the folks at Cognitive Edge would describe as a method to figure out what's going on and that's what these experiments are, right? We've laid this out in a notion here that has on the front of it a formation process. That formation process we think starts with a canvas usually in the form of new startups. That's a lean canvas, but you can also imagine an initiative canvas if you're working in a known domain. We enumerate the assumptions, we identify those key assumptions, almost like a story map, and then underneath those key assumptions, we enumerate experiments from which to be able to then solve that and run them through a Kanban board just like Ash was showing this morning. That begins to build the process of actually understanding and learning. If you're experimenting the world of business, the lean canvas on the top that Ash uses is absolutely fantastic place to start. Way better than the business model canvas that Alex Osterwald started with because that assumes a known solution. In Ash's world, he's working and starting with problem and then having solution on the top allows for much more experimentation. Because of what we're doing in our space and because of our background, we took that same canvas and built it to experiment on initiatives to try to figure out relative value of initiatives very quickly so that we could rank our backlog. So same kind of canvas approach, same way to enumerate all your assumptions, but then to be able to pull them off and be able to work your way iteratively and incrementally through those assumptions to figure out is this technically feasible? Is this business effective? Is this actually long-term sustainable and is it desirable for our customers? So let's talk a little bit about a different example in the business side from an Indian company that I've worked with. I'm a mentor at a group called the Unreasonable Institute in Boulder, Colorado where they take 26 entrepreneurs from around the world who are social entrepreneurs and bring them and put them in a place with about 40 mentors for six weeks and try to give them wings so that they can affect a million people on the planet. It's my safe to fail or safe to run experiment place. It's where we've been playing around with a lot of these concepts so that we could work at the whole business level, not just at a single development team level. And so let me talk to you a little and let Mishkin talk to you a little bit about his phone. Let me, oops, gotta back this up. Pause here. Hey, do you guys have a AV hookup? I forgot to do that. Is there one? Oh yeah, sorry about that. It's probably not going through the sound system. Okay, can you try that again? That'd be great if we could just back up for a second. Was delivering a baby? And he came out, he was pretty ashen-faced, the pregnant woman, and her child had died. Apparently that had been caused by anemia. Even I knew that this was completely treatable. You took iron tablets, these are widely available there, sub-seed iron there, cheap. I was shocked, I mean, this woman and child were no longer there. I didn't understand this at all. And the problem that he had learned about at that point was it's not that we need a solution for anemia, but what we need a solution for is for being able to diagnose anemia in the field. And so he took a number of engineers in India and they built a blood oxygen-like sensor. Put your finger in this probe. There's no prick, it works on light. Something called photoplasmography. We measure the signal, we diagnose anemia, very simple. This is what we do, it's a simple tool which can be taken like a mobile phone to the point of care. So Mishkin did really effectively in that cycle is he framed the problem real well, right? He wasn't an engineer trying to work on solving anemia. He was an engineer working on trying to have remote diagnostic in the field for villages in India, right? He was able to understand and empathize because he'd obviously been at a situation that was obviously terrible and affecting for him, but it allowed him to write down the hypothesis and document it and move forward. One of Einstein's favorite quotes and I think Ash hit on this this morning, right? Formulation of the problem is often more essential than the solution. It takes us back to our lab book, right? One of the steps that we missed in the lab book that people often skip because they go to the hypothesis step is, what's the question we're trying to, what are we playing with? We spent time actually pondering and understanding and doing actual observations in the field so we have some empathy associated with what's going on here. So step four in this kind of stick step process is take time to empathize. So first question is, do engineers have empathy? Sometimes my wife would definitely argue that I do not have empathy, but we'll do a little example here. So while you were watching that, what was going on in your head was a lot of what was going on in the skateboarder's head, right? The same synapses that were firing in their brain, we now know from brain science a lot of things that were going on are the same synapses that you were firing. It's actually born into us, a sympathetic nerve and the ability to empathize with right out of the womb. So this is a natural process, even though we got drilled into ourselves, how we're gonna answer these great problems on E2 paper, we still do have it and I remind my wife of that on numerous occasions and I'm getting better at actually expressing it. So we just need to reconnect with it. I'm gonna show you another video about an engineer and a designer who's really had to go to a whole another level to understand this and it really paints the picture of, well, this is Doug Dietz, he's the designer of the MRI. I'll start with this, which is kind of the, a little bit of the thread or the kind of the heartbeat of my story today and you've heard a lot about the design thinking and the design process that a lot of these folks have gone through, very powerful tools and I just wanna emphasize that empathy at the beginning for me is something that I think is really what is the heartbeat of the project when you move forward and a lot of times when you move forward into the iteration and prototyping and some of the design phases that you go through, you need to actually refocus back and see what the empathy was that actually got you started at the beginning. So I thought I would just share a quick story with you guys and this is a little tough story to tell but I'm gonna give it a shot.  How many people here just have an MR scan? A big MRI scan, oh my gosh, a lot of you guys so don't charge the stage.  I think I had worked on this thing for probably about two years and doing, I'm an industrial designer and I was doing the enclosures and the controls and displays and the coils and patient transfer, that kind of stuff and I was so excited to see this, the first product that we installed so I went to see it in its actual environment in the hospital. So I was just running through this hospital to check out my new product that I had just finished. I'm really excited and I kind of almost like barged in, took my wallet and all the metal stuff out of my pockets and ran in and was with my baby and I was just a proud papa basically and I remember the technologist coming in and saying we have a patient that's gonna be coming through so could you step out for a little bit? I said sure, no problem, I'll come back in, we can talk later and I remember walking out and I was standing in the hallway and this is kind of what I saw down the hallway. I saw this young family coming down the hallway and I'm just standing in the hallway and they're coming toward and I can tell as they get closer the little girl is weeping and as they even get closer to me I noticed the father just leans down and just goes remember when we've talked about this you can be brave. And they turn to walk into the MR suite and I kind of follow them in and I still remember it like it was today guys.  I was just in there, in this environment just doing my happy dance and she just freezes and looking at her angle and just seeing in that same environment where I was just standing I realized that this is something totally different. On the wall is that horrible warning sticker and it's got the magnet and the big exclamation point. It's got that yellow and black tape like on the thing, almost like an accident scene. Everything looks really kind of like beige. It's got this kind of weird colored flooring and just kind of everything's bleached out and then the room itself is kind of dark and it kind of has those flickering fluorescent lights.  And then of course an MRI for you folks that have had one, it is just a terrible noise. So the little girl just starts to really cry. I mean, she's just breaking down and I can tell you guys, I'm standing behind and I see these parents and for you guys that are parents, they're looking at each other and they don't have to say a word because they don't know how they're gonna get their child through this. So that was a huge awakening for me. So the challenge that we have here is you've got that little guy and he needs to go through a scan in that. And what does that look like to you guys? What does that machine look like? Big clam, looks like a stapler.   And guys, this is a color picture. This is a color picture.  It looks like an anxiety curve. And that really starts when they first understand that they need a scan or they're in their car coming to the hospital. And it just starts to ramp up as they get to the hospital and you know how it is, you get on like nine elevators and you follow the blue line on the floor and you get to the site and when they come in contact with this piece of equipment, you know the tears start to come way up in there and actually at the bottom of that arc it really doesn't even flatten out that much. And just working with these young families and these parents, I spent some time with this one father and I remember they had to sedate his son and in this modality, nuclear medicine, they have to sedate the children about 80% of the time so they can scan them. And he was carrying his little boy out to the parking lot. Because he had sedation and I'm walking out with him just to thank him and all of a sudden he just stops. And he's got this kind of weird look on his face and I'm like, what's up? And he goes, you know, I forgot where I parked my car. Why do you think he forgot where he parked his car? Absolutely.  Is insurance going to pay for this?  All those things that I thought he would be worried about and the family would be worried about, their number one thing was, how am I going to get my child through this? So they went through a design thinking process associated with now really understanding the problem. That is 80% of their patients had to be sedated to go through that device. The redesign on that thing didn't change the outside of the machine at all, but it built basically a carnival ride out of the whole concept. And the result is they end up sending you a backpack before you go. And in the backpack's a coloring piece and it's in one of two different design categories. This one happens to be in this pirate ship motif. And that's the redesigned atmosphere and experience. And so that, we'll hear now Doug talk a little bit about the end result of that redesigned concept when they frame the problem correctly. And probably the last thing I just want to leave you guys with here is really that when you design for meaning, good things will happen. And sometimes if you go the other way around, which a lot of times we do where you're designing for money or you're designing for some of these other things and hoping that meaning comes, it doesn't work that way. And though we got the sedation to be reduced significantly here, and I think their patient satisfaction in this hospital went up 92%. And their patient volume also went up so they're able to get more children in in a day, which makes the the actual waiting list and some of the time that you have to wait for a scan a lot less. So a lot of people ask me things like, are you really proud of those things, which I think I really am. And they ask me if I kind of measure the success of this on those things. And I'll be honest with you, I don't. What I really think I would measure myself in this kind of a project is have I had influence on that conversation in the car on the ride home for that family. And if I've had that opportunity to make that a different kind of conversation for this family, and you've seen some things already in this great some great talks on how you interact with healthcare, if it is just a totally different conversation in that car, and it's maybe one of those things where they just embrace the experience a little bit better, because a lot of times in healthcare, you know, this is like a speed bump in the road for these folks. And if we can just make that a little bit smaller speed bump, if we can turn it into a little bit more of an experience, I think it's going to be a real positive thing. Thank you very much. So how many users have you talked to in the last year? What's the requirement on your engineering team to have them in the field? How much do they work with their UX design team? You got to start by interviewing folks. You got to keep revisiting and going back to the place of empathy. The tools that we use are simple paper tools to be able to build and understand empathy from watching, saying, listening, observing what's going on with people in the context. We then work through the process of seeking insights from that and really understanding what the need is. You heard Ash this morning use a phrase, the job to be done. What's the job we're trying to get done? You heard Doug redefine the job of trying to make the speed bump smaller. When you can take that understanding in the circumstances and the jobs to be done, you can use that to frame the experiments that allow you to validate or invalidate your assumptions that can then show you the demand before you build the software. If you do it really well, you'll find yourself at the middle of this curve and we'll talk a little bit about that and that is maximize learning when your experiments produce when you're operating at the 50-50 place where what you think is going to happen happens about 50% of the time. If you operate on the one curve where what you think is going to happen 100% of the time, obviously you're wasting everybody's time. You're not leaning in. You don't understand. You don't feel safe. In an environment inside of an existing product environment, a lot of times that's what's getting the way. You run set up an experimentation sandbox that you feel safe to run inside of so you run experiments that deliver results that you know the answers to. On the other end of the spectrum, if you don't have the empathy to really understand the problem, you end up getting results every time that you didn't understand.  You're certainly learning but you didn't do the preparation work necessary to get yourself into the right space to really understand what's going on. So you're also wasting 100 tons of time. So this is the space in the middle that comes from framing the problem effectively. What we expect to happen versus what we actually happen, we start to understand what is the whole system here. That's what these experiments are about to really understand the dynamics of what's really going on here. So steps five to me is taking this empirical learning skills and using it in the world of the Lean Startup process. We add a step on the front of the Lean Startup process around the formation of the team, building the sandbox that provides the safety, providing the vision and assembling the team necessary to run through that process. But then it's running through the process that you're really used to and that is the first step of this in the Lean Startup process is discover. So what's the problem? What's the problem that's going to do that's going to justify this and then programmed in the process as we would in any good agile process is the stop point. And that stop point in the case of an experimentation cycle is the pivot persevere or pause. That happens a lot on internal projects as a pause, right? Not now because the technical platform is not there yet. Not now because it's not the highest priority. So that gets added right? You then go into the space of really can the founders or the entrepreneurs sell this thing? What's the playbook? What's the steps necessary to get it into the hands of the customers and successful? And what's the funnel that's necessary to go to market? Do we have enough noses out there to actually sell this solution to? That brings you to the next pivot persevere or pause statement. And then in the existing entity or large organization, it brings you to a step called transition. And that is where the new product development team or the experimentation team now needs to hand it off to the rest of the business. You've got a time period here where you're going to bring other departments onto your team and transition this new product or this new solution into the existing kind of operating or functioning engine of the business. And that makes a larger picture and you can get... This is a clickable diagram with videos of all these steps on our website under rallydev.com slash ELS for Enterprise Lean Startup. And it brings me back to talking about and putting a wrap on what we saw from Mishkin, right? In Mishkin's sense in his biosense story, he was really effective at going out to the point of delivery and actually experiencing what was going on with his friend in Paro in the midst of Pune. We took him through an exploration process that was Lean Canvas based exploration process to really try to understand what their business would look like. And in the case of a lot of these social entrepreneurs we see two-sided business models where they have an impact side of their business where they're affecting a population that may be below the poverty line or that cannot afford solutions and they're usually getting money from a side of the business which is very complex business models but teasing those apart is kind of part of what we've learned in that cycle and then taking it to execution is the third step in the cycle empathy, exploration, and execution. Finally you need to get your engineering solution to scale, right? For me the safe models around what is the best visual articulation and what that means it means being able to bring a whole set of teams together to deliver not just one team's worth of features in a time period but to be able to deliver a whole product in a set time period. To me it's an articulation of that whole model of how do we run an entire business and actually adaptively steer it. It's a picture towards business agility. We took a stab at what that was like on a book that we wrote with a lot of small vignettes. It's available on PDF on Kindle right now and we're trying to understand what does this look like if the whole operating system of the business is completely different. It's a lot easier to see in these social venture startups than it is to see in large global 2,000 organizations because you can't change the operating system there but what I see I really like and it brings me back to our social mission. Our social mission at Raleigh is around creating and mobilizing citizen engineers. With our IPO last year we funded our foundation and we'll be taking seven teams, a team with each of our development offices and adopting different social entrepreneurs and citizen engineers to give them the tools, to give them the time and to give them the training necessary to run through this kind of a process as a way to both help share what we know about running an agile business but also to educate and engage our own employees in understanding this entire process. That's what we're doing on this side. It's highly motivated and highly patterned after obviously the work that ThoughtWorks has done in this area around social impact. It's our best effort in this path and it's something that really keeps me going. When I look at what we saw before from Kara and talking about the idea process of evaluating solutions, these overlapping Venn diagrams to me are what we're trying to do for engineers Yeah, we spend a lot of time in technical feasibility and business effectiveness questions but now we need to add the question on here of is this a sustainable solution? Ultimately we're going to be asking the question, does this create biodiversity? Does this create clean water? Does this create cleaner air? What does it do to make the planet the one spaceship we've got right now until SpaceX figures out how to get us off this planet? How does it make it better? Because that's really the question with 7 billion of us on the planet. The third, the fourth piece is the desirability or social impact. When you find the overlap of these circles the desirability question causes the solution to get pulled out of your hand. This is the way you affect millions of people quickly is when you have the solution that's that desirable that it really goes. Working and trying to find the intersection of that space I think is the job of citizen engineers. I spoke at the Engineers Without Borders conference in Canada last year and ran into actually a gentleman who only lives about 20 miles from me named Juan Lucero. He ran an experiment while we were up there at Engineers Without Borders Canada. Most of the folks that come to EWB conferences are in their 20s still in college and what they're working on is how to bring social justice into the question and this is the desirability question. So the design problem they gave us was you have oil sands in the middle of Alberta and you need to get them to the world. So right now the only way we know how to do that given the US's lack of interest in a pipeline is to take that west and that means take it to the Pacific Ocean and to get it to the Pacific Ocean you have to go through at least two if not three native peoples lands. So how are you going to work and have a discussion with the indigenous people who live on the western coast of Canada and have your pipeline actually make their community better? That's a hard question. With a bunch of 20 year olds it was an interesting afternoon spending ideas, prototyping and role playing ways to have that dialogue and that's what citizen engineers do. They have that dialogue, they get obviously we didn't have any indigenous well it may not be obvious we didn't have any indigenous people so it was really hard to get empathy but the role play practice prepared some of those folks who are actually going to go out and do that job. They haven't succeeded yet but that's the role. So to me what do you want to be? Chief Engineer, technical founder? I think you want to move from thinking about it as a T-shaped pretty wide to really E-shaped people. I think the way we get there could be this linear step of get discipline at agile execution find and protect your slack so that you can experiment. Remember how to run like a scientist and run hypothesis that are where you have the safety and the empathy to be able to explore effectively and quickly. Take time to empathize become that discipline explorer where you run a discipline process of being able to evaluate your darlings which is what they really become as to whether you should pivot persevere or pause them and then figure out how to scale that and bring that to the big side of your business so that you can affect the 7 billion people on the planet. To me the E stands for executing, exploring and empathizing not just executing. That's a global citizen engineer and global citizen engineers do one thing and that is they dent the world. That's why I'm here. That's why I came to India. It's my first trip. I was thrilled to be able to do it and I was thrilled to be able to follow a day like today and thrilled to be able to talk to a group of people where there's a million engineers created a year. So we know from a lot of the things that we talked about on the first night with Ray talking about injustice to women that the solutions to big problems like poverty and injustice to women don't get solved on a global level. They get solved on a local level. So we know from the social entrepreneurs that we work with that you have to have empathy on a local setting to be able to actually crack these problems and that we need to enable these skills and capabilities on a worldwide level. It's not something that somebody from some other country is going to bring to you or you'll end up like the EWB teams did with the water system just not working three months later. It has to be actually brought to all of us and that's what I think our role as engineers on the planet is to be able to bring those skills to others. So I stood on the shoulders of a lot of folks in this talk. There's some of their pictures and I'm going to say thank you. I don't know if anybody has questions. We got a couple minutes but I think all that's waiting for us is dinner. But be glad to answer but I can understand if you don't have any either. The slides are on the web obviously under that bit.ly location and that's where you can find all those URLs for the stuff that we produce in my group as kind of early prototypes of services. Thanks.