Myself and also Chris Matz and Olaf Nassan. I come from an oil and gas background. Oil and gas have a lot of uncertainty. Chris and Olaf have a lot of experience in the financial sector and they've done a lot of work in understanding uncertainty in those areas. So what we're going to talk about today is what can we learn in the software world, some of the things that software experiences that are similar to what we see in other businesses. So first let's start with risk and uncertainty and some of the definitions that at various times you'll see relative to risk and uncertainty. Here are three different variants of definitions. One was from Frank Knight who was an academic and distinguished between called risk immeasurable and uncertainty was quantifiable. You see this in academic publications quite often. I don't quite understand how it came about or how it came to be used but I don't particularly like it because I don't see it consistent with the English language. So I'm going to try to not go that direction. The PMI generally has risk, understands that risk is there and says it can be the positive or negative risk which is again counter to the English language. So when I talk about risk and uncertainty in general I'm going to be looking at risk as a situation that involves exposure to danger like getting run by a bull or uncertainty is the state of being uncertain, not known or established or questionable. So that's sort of the basis that we're starting from. So since this is about risk I called it risky business. Risky business is a take off on a film that was made a few I think in the 1980s with Tom Cruise. I just want to show this little short clip here. We don't have sound. Can we check sound on the computer? The first thing about risk management you got to know is when you got too much risk. We're about to go there. The next step, I live in Houston, Texas. Houston, Texas is known for having real challenges with hurricanes and so this is an actual plot of the results of projections that I think 16 or 18 different weather services had regarding a path of Hurricane Rita. Hurricane Rita had come just after Hurricane Katrina had made a significant impact in New Orleans. It was a huge, huge devastation in New Orleans so Hurricane Rita was a Category 5 hurricane at the time. So it was a serious thing because had it come through and hit Houston as a Category 5 would be a big issue. So the interesting thing, what do we know about every single one of these projections? They're going to hit? Anyone else have anything to say about every one of the projections? The profile changes over time. Each one of them does change over time. Anything else about each one of the projections? Confidence levels for each? Okay, possibly. So the answer I'm going for, which is an interesting thing because I did this for high school kids and I got a different answer and they were smarter than me. My answer was that all of them are wrong because I'm an old guy and I want to be negative about it. The high schooler said every one of them is a possibility. That's the same answer but a much more fun way to say it. Everyone's a possibility, I say everyone's wrong, but that's the whole thing. In one sense, everything's wrong because if we were to hold accountable to the single projection that you had, it would actually be wrong. There's no way to be possibly right. The one thing we do know is something we can measure and that's the plus down there because that's pretty accurate to be measured. And it turned out that the actual path of the hurricane wasn't like any of them. But what we also know is that the collection of estimates was actually useful because it gave us a band, something we call the cone of uncertainty. It gave us a band of where we might be concerned about hurricane hitting. Again, then looking at that cone of uncertainty, by looking at this over a sequence of time, we can actually see and make decisions based upon where we need to evacuate perhaps and over time eventually it comes through. This is the same type of management and bounding of uncertainty and learning through things that you can measure. Some other examples of uncertainty. I love this one, a war example. So this is a quote. They couldn't hit an elephant at this dist. It was the last words of General John B. Sedwick, the Union Army Civil War officer that he uttered during the Battle of Spotsylvania in 1864. He was bragging that they couldn't hit him. And they did. I played this the other day. I'm going to play it again here. I think it's really a brilliant description of the importance of feedback. So this is another war example here of Gordon the guided missile. Gordon the guided missile sets off in pursuit of its target. It immediately sends out signals to discover if it's on course to hit that target and the signals come back. Now you are not on course, so change it up a bit and slightly to the left. And Gordon changes course as instructed and then rational little creature that he is, he sends out another signal. Am I on course now? And back comes the answer. No.  So he adjusts his course again and sends out another request for information. And back comes the answer. No, Gordon, you're still very wrong. You must come down a bit and a foot to the right. And the guided missile, its rationality and persistence, a listen to us all, goes on and on making mistakes and on and on listening to the feedback and on and on correcting its behavior in the light of that feedback until it blows up the nasty enemy thing. Then we applaud the missile for its skill. And now if some critic says, why it made a lot of mistakes on the way, we reply, yes, but that didn't matter. Did it? It got there in the end. All its mistakes were little ones in the sense that they could be immediately corrected. And as a result of making hundreds of mistakes, eventually, the missile succeeded in avoiding the one mistake which would really have mattered, missing the time. So the movie industry, all movies they aimed to be just as great as the Titanic. The Titanic was until recently the number one grossing film of all time, supplanted by Avatar. Unfortunately, most movies end up like the Titanic. 78% of films lose money. And only 6% of films contribute to making 80% of the profit. Huge area of uncertainty. I think this is an industry that doesn't necessarily knew how to manage uncertainty because it's so driven by egos and everything else because I don't think it's necessarily the most profitable industry overall. But there are other industries that are able to manage risk and be effectively very profitable. Book industry is a good example. It's my book. In 2004, out of 300,000 books that were printed or that were available for sale, less than 25% sold more than 100 copies. Isn't that amazing? Poker. Any poker players here? Texas hold them? All right. So just go with me here. These are three different possible hands. Which of these three hands do you think would be the best Texas hold them whole hand? Would you think it would be the ace four? The pair of threes? Or the six-nine? How many for A? You don't have to know a lot here. A couple of A's. How many for B? Would be more for B. How many for C? A couple more for C. If it was just those two cards and you were going to stop, obviously the pair would be the highest. Let's see what it has. 29.6% chance of winning. The ace four has a 33.5% chance of winning. And what looks like really crappy hand, the six-nine, actually has a 36.5% chance of winning. What's going on here? What's going on? In this case with poker, that your strength of your hand going into this is largely determined by your weakest card. Your weakest second card. So the stronger the weak card is, the better your chance of winning. We don't think that way.  How much of your team's success is driven by your weakest link? We tend to focus on strengths. We tend to forget about what we need to do to bring our weakest links up. Sometimes you can bring your weakest link up and make a huge difference. Again on poker, I'm thinking of creating a metric for poker called percent of hands won. Think percent of hands won would be a good metric for poker? Success? How many think it's a good metric for poker success? Percent of hands won. What's that? I might not tell. It's a good point, very good point there. I was going to say though, it turns out that this is pretty good metric. It's a pretty good metric for determining if you're going to lose. Because the more hands you won, the more likely you're going to lose. Why is that? The more money you're likely to lose. Why do you think that is? You're playing too much. In order to win hands, what do you have to do? In order to win hands a lot, it would normally be the same wads when it stays in the hands all the time. But in order to win, win at a higher percentage, it means you have to stay in the game longer. Really, really good poker players know one thing. Win to get out. Think about that with your software. Do you know when to get out?  Something to think about. Are you looking for the right metrics? I mean, I compare this one often to on-time delivery. Does on-time delivery really matter? Or does it really matter how much value you deliver? What's your focus? Oil and gas exploration. Something that's near and dear to my heart. What do you think the success rate is for new frontier oil and gas exploration? High success? Any ideas? Less than 1%, 2%, 10%. Yeah, so it's about 90% to 80% failure, 10% to 20% success rate. 90% failure if you're using competitors' software and data. 80% if you're using IHS data. We make a big difference and products we produce. Oil and gas also has a lot of safety issues. It was my beginning to think it wasn't such a good idea to turn off those unit tests. You've all heard about the big blowout that BP had in the Gulf of Mexico and certainly we heard about it and I think probably everyone heard around the world. It was huge. Actually, the company I was with, Halliburton, was involved with it. I'm not speaking from any inside information but basically, it was effectively a decision by the engineers to turn off some unit tests that resulted in the big explosion. They did some tests. They didn't quite get the results they wanted. They decided they weren't going to run the second test and they weren't going to run the follow-up test that would have actually caught the issue and they would have known about it and they would have been able to shut down so that they didn't have the explosion. It's in software, it's an issue, sometimes everywhere we go. It's great and lintening to those tests is important. One of the things we do in Welling Gas is use something called a tornado plot. In other industries you see this. I'm going to introduce this idea of a tornado plot and what you're really looking at here is a plot that shows what are the uncertainties that are really important. This is net present value that we're showing and the base case is the vertical line there. It's 1,350 million is the base case and then what we plot is this tornado that shows where is the uncertainty. So the big uncertainty in this is in the amount of reserves in the ground which could take our valuation way down to possibly only 300 million or it could take us up into a much larger amount. Next down we have Welling Gas prices and then there's uncertainty and then down at the final at the bottom there is drilling cost. Now this is one of those things where we often have the measurement inversion. What do we do? Well we focus on cost sometimes. It's not that important. What really matters is what's the big picture. So what would an intelligent company do with this? They say well this is my size and my uncertainty. What could I do to understand my uncertainty better? So like with reserves maybe I could get a little bit more data about it. Maybe I could do some more seismics or maybe I could do some preliminary drilling to find out and quantify a little bit better what reserves are so that I can better understand the amount of uncertainty and make informed decisions based on that. So now I'm going to go into some risk management tools associated with that in particular the one particular flavor of this that I like and I've worked with Chris Madsen Olaf Masson on real options. And the idea behind real options is that it's the right but not the obligation to undertake a certain action prior to an expiry date. Very simple concept. You have the right to do something but you don't actually have to do it and at some point that right will expire and you no longer can do it. One very simple example of a real option is an airline ticket. I buy an airline ticket doesn't actually don't even think about whether it's a refundable ticket or a nonrefundable ticket because that gets into a secondary issue. Let's just say I'm going to buy a ticket and given I've got that ticket I now have a right to board the plane. They can't take that away from me because I've got that right to board the plane. Now there's other issues associated with if they're not going to fly and things like that but basically I have a right to board the plane. They can't make me get on that plane. If I don't want to get on the plane I don't have to. So I've got a right to do it but I don't have to do it and at some point the plane is going to fly away and my option to get on that plane has expired. Very simple, simple concept. Most tickets, almost all tickets operate as a form of a real option and if you want to look at it from the perspective of a refundable, nonrefundable ticket what you actually have is an option on an option. You have the option to get for the ticket you also have the option to refund it and so you have the option on the option. Now there are a couple of tickets that aren't real options. Anyone have an idea of a ticket that's not a real option? What's that? Yeah? Speeding ticket? Parking ticket? Not necessarily options. Alright. So real options are the right but not the obligation to take some action prior to an expiry date. Chris and Olaf have simplified this into three bite-sized pieces that I'm going to expose here. First of all, options have value. There's a value to having that option. Options will expire at some point and key one, you never commit early unless you know why. So it's all about there's a decision you can make or you can commit and the commitment actually expires the option. So you don't want to do that unless you know why. But if you know why, you may very well want to commit early. I'm going to take a different spin on this but first we're going to look at options from the perspective of options, decisions, and commitments. So options are the options that are available to you. From that you can make decisions. Decisions don't necessarily expire the option. You can make decisions but the decision is generally the difference between decision and commitment is decisions are reversible. So you still have the option it's just you happen to go down a particular decision path and you actually want to be able to do that. What you want to make sure you don't do is commit early unless you know why. Because once you make the commitment you've expired the option the value of that option has disappeared so you've made that commitment. The commitment might be the right answer but you want to know why you've made that commitment if you're going to do it. So we can look at this as an example here someone's going to run and jump off the cliff. At the beginning they have an option. They have the option of jumping off the cliff or not. Then they've made a decision, a decision to start forward. At some point they've committed and in this case it's probably committed pretty early. Okay? But this is the idea of the commitment of which point you can't go back. So given that framework of real options there's four particular areas that I want to call out and put it in different context. First of all is the value of uncertainty. Second value of information. Third is the value of flexibility and the fourth is the cost of delay. So the first point is the value of uncertainty. This guy Patrick Leach an orthopathic leach why can't you just give me a number? I love that title. Why can't you just give me a number? Because it's something we face all the time. As a very provocative question what's the source of all value? He teaches a lot. He's asked of all of his classes. He never comes around to people giving him the answer that he's looking for. It's not really surprising that he doesn't get it. He gets the standard answers to all of the questions and all of those things which are very, very truly do contribute to value. What he's really trying to get to what's the source of all value though is the fact uncertainty has to be present for value to be created. There has to be uncertainty because if there's not uncertainty if you have a competitive advantage and you're doing something and you're building value it'll be arbitraged away and eventually it'll be risk-free. So the fact that there's uncertainty is the source of value. And this is actually shown out also in Black Scholes. Black Scholes turns out to be not a very good way of looking at real options. It turns out to be a pretty good way for pricing financial options. But one of the key things from it is there's a volatility number a V number, a volatility. Volatility is an indication of how great the uncertainty is and the uncertainty the higher the price. So the higher the value. The amount of uncertainty has a direct input indicator of the amount of value. This is also shown here as a risk-reward, risk-return chart here. And typically this is the best you can do. That efficient frontier is the best you can do. At a high risk you're going to get a higher return. You can have something under the efficient frontier which means you've got a high risk and not getting as much return as you possibly could be getting. But you generally can't go to the left of that curve. The thing I want to pull out of here is the value of certainty is that if you're going to reduce the risk you're typically also going to reduce the return. So I'm going to go through an example here using a tornado example and just saying we're looking at this and we've seen this chart here and we say so our example here product acceptance uncertainty how well we're going to do in the market that's our huge uncertainty. We've got some schedule uncertainty we've got some general market uncertainty and maybe we've got some cost uncertainty. And we look at that and there's some negative numbers there and we say you know what we'd really like to do some things to reduce our uncertainty and we're going to eliminate those red bars okay so very simply management okay let's remove those red bars so we no longer have low end well what's really happening in the system we can probably do that but the result is going to be we're going to chop away all the high end so yes we can reduce uncertainty but reducing uncertainty has actually reduced


I have a good idea. I've got reason to be. That's why I came to see you about. Me? Now what have I got to do with it? No, don't play innocent. It's about Carmen and you know all about it. Oh, I don't know what you're talking about, Major. No, like fun, you don't. You didn't put on that act for me.  No, you can't talk to me like this. Oh, can I, though? You haven't any proof. Ah, so you admit it, huh? I certainly don't admit anything. You don't admit anything?  Oh, so you do love her. Well, I admire her very much. That's beside the point.  I never did.  What?  What are you talking about? Listen, bearded lady or no bearded lady, you can't pull a wool over my eyes. The minute I mentioned Carmen, you acted gilded. But don't think you can laugh yourself out of this. But I really am. I may be small, but I'm far from dumb. And something else, my fine bearded friend. What about these notes you've been sending to the Spanish Senorita? What? Yes. So, I guess that makes you stop for a minute. I don't know what you're talking about. No. Supposing you explain how some of that fine rice paper viewers happen to find its way into Carmen's wagon, some of that lovely rice paper you are looking for some of that lovely rice paper you are so foolishly fond of. What rice paper?  I suppose it wasn't good enough to use for Carmen. All must up like that. But I never threatened her, really. Ah, so! How did you know she was threatened?  You said she was threatened. I'm sure you did. Oh, no. I only said she was getting notes. But you know what was in them far better than I do. You just admitted it. Oh, you cheat. I'll get even with you for this. I'm not afraid of a beard, no matter what's behind it. And let me tell you something, my fine lady. You're going to get some more of that fine rice paper and you're going to write another note to Carmen. No, I won't write it. Yes, you will. Now hurry up with it. So, you're not even smart enough to hide the paper on which you write such dangerous notes. Here it is. Come on. Here's a pen. I won't. Do you want me to go to Mr. Randall with this information? Oh, no, no, no. Please, please, Major. I'm sorry. All righty now. All right. Ms. Carmen Bandini. The notes you have been receiving... Not since I've been receiving. ...have been sent in jest, period. How do you spell jest? J-E-S-T. There is no danger. I guess that covers it pretty well. Now, add this. You are being protected by a brave man, period. ...protected by a brave man. P-E-R-R-O. Oh, don't spell out period, silly. Just put a dot. No, I do everything wrong. That is what I would do. Now look, you're crying all over the note. Here, take it and see that Carmen gets it. The idea. Frightening a poor innocent girl. And as for you and the bouquet, I'll get even with you if that's the last thing I do. No, but I didn't. Not the bouquet.    Ah, these women. Well, hello there, Rags. What do you want? What do you suppose Rags wants? Oh, it's you too, Jerry. Yes, Jerry? Have you found out for sure yet whether the bearded lady had anything to do with those notes written to Carmen? Now listen here, Jerry.  I'm taking charge of this case from now on. But I've got to find out if there's something I can do to protect Carmen. I've got to do it. I know, I know, but from now on, I'll be making it my business to revenge the beautiful little Spanish lady. But if something should happen... It won't. I promise you.  And if it's not settled by then, well, we'll have to do something about it. But I just got a hunch. What about? That's some pretty unusual things that's going to be happening on this lot before tomorrow. Gee, it kind of makes me curious. Well, it's sometimes just as well for boys not to be too curious. Now listen, Jerry, I want you to do something for me. What is it?  What for? Now, don't ask any questions. I need them. And, you know, I don't get along with Clara very well. I go over and borrow them myself. Well, sure, Major. I'll be glad to. I'm supposed to go over for a fitting or something anyhow. Come on, Rags. Let's go. A fitting? What for? Gee, I thought you knew. Yes, I've been telling most everybody on the lot. I'm going to work El Mundo. Oh, yes, of course. I forgot all about that. Sure. And I'm going to have a big shiny hat and leather boots and a velvet coat and everything. Oh, say, that's just fine, Jerry. You ought to look pretty good in all that finery. Well, I don't know about that, but you're going to be fun to get up and dress and everything and go out into the ring with everyone applauding and put El Mundo through his trick. Say, I have to learn Spanish and everything. Oh, of course. So the owl will follow and understand you, huh? Yes. Well, here's a wardrobe top. I'd better go in. Say, sure, you don't want to come along with me? And talk to Clara? I should say not. That woman riles me something awful. Okay, I won't be long. I guess she's only got to take my measurements. Come on, Rags. Here you go. I'll be waiting right out here in the backyard. But remember, Jerry, don't you tell her what you want those scissors for? How can I? I don't know myself. No, no, I mean, don't tell her there for me. She's so mean she wouldn't give them to you then. You and your fight. Rags, you're quick. Hi there, Clara. Well, now if it ain't Jerry Dugan, it's about time you got over here at the wardrobe top.  Could you do that? Well, I guess I could, but it mightn't fit so good. I'm sure glad I got here in time, then. What size shoes do you wear, Jerry? Oh, about six and a half, I guess. Guess? Well, the last time I got shoes, I got sixes. Of course, I've grown some since then. Well, then, just think of that. Well, guess we'll order sevens, then. Can't have you growing out first thing of a pair of those shiny black riding boots, not when they're so expensive. Riding boots? Oh, gee whiz. Here, Jerry, for goodness sakes, hold still.  Oh, I'm sorry. Then, I've got your arm measurement. Now, your waist measurement. Say, Clara, you know what was just as easy riding on that elephant's back? He sure jounces up and down, though. It's not as safe as he can be any worse than you are. Gee, I'm sorry, Clara. Honest I am.   No, I mean stand still, not get so excited. There, I guess that's it. I'm sure glad that's over with. You've got nothing on me. I'll just soon measure a knee or an angle worm. I must say I'm not looking forward to fitting you. Well, I'll be better next time, Clara. Oh, say, would you mind lending me those scissors of yours for a little while? I certainly would. I use these myself. They're my favorite pair. Oh, you got any others I could borrow? Well, what do you want them for? Well, I just need them. If boys aren't the ones now, what were their secrets and all? I guess I got no pair you can have. Well, just where did I put them? Oh, gee, thanks, Clara. You're so well. I hear they are right under my eyes. Seems like I'm getting more abs and minded every day. I'll be careful of them, Clara. Mind you bring them back. Sure, I'll remember, Clara. Now, where are the dickens as a major gone to, Rags? Oh, there you are, Major. Get him. Of course I did. Oh, say, that's real nice of you, Cherry. I'll get him right back to you. You better, Art, Clara, I'll sure raise the roof. Hey, where are you heading for now? Well, I guess I'll be going back to my wagon. Are thereabouts? We'll go part way with you. Got to go buy the office wagon and see Mr. Randall a minute anyway. Yeah, good. Then I got to get back to rehearsal. Pronto? Sure. I'm practicing my Spanish, too. Oh. You know what? I find myself just naturally saying the words. It's lots of fun. I think I'll learn to really talk Spanish. I wish I could talk Spanish. It'd be pretty nice to be able to talk to the lovely Senorita Common in her own language. Yeah. And, you know, you never can tell when it'll come in handy, either. Oh, you're a pretty lucky boy to have her show you some of the routine she used to do with that new elephant. I'll say. She can't be there all the time, but she's sure nice to do it. Goodness, what's that? Hmm, it sounds like... Just what I thought. Why, it's coming from the bearded ladies' wagon. She's left her door open. She certainly has. Well, I guess I better go up and close it. She'll be disturbing the whole circus lot. I shouldn't wonder. Gee, she certainly can't snore, can't she? Sounds like a trumpet or something. Well, we got to go this way. Come on, Ike. I'll see you later. Both of you. I'll see you. Don't forget to bring back Clara's scissors when you're through. No, I won't. Well, I got to get some important business with these scissors done myself first. Isn't that just like her to leave the door wide open, hide herself to sleep? That's what she did. Hmm. Well, can't imagine anything more convenient than that. Not if I managed it myself. Lucky, she's making so much noise. Dog gone needs wooden floors. She just won't wake up till I give her the trim and that she needs. I'll get even with her for once in my life. Oh. Hmm. There. I guess that does it. Hmm. I sure hope she likes herself without that beard of hers. Guess I got rid of that. All right, all right. Now if I can just get out and close that door. Oh, gee, I... Phew. That did it. I didn't even have time to pick up those scissors. That sure is bad luck. I just got to get those scissors. I won't say she to work.  I better get out of here and quit.


During lunch and the question came up which is harder to be the last presentation before lunch or the first presentation after lunch I'm not I'm not going to tell you But we did have another interesting conversation and I just wanted to take a moment and kind of pull our Participants here just from my own enlightenment How many of you are here representing an established nuclear power plant? anyone The Koreans are okay How many are here representing an established research reactor? How about an R&D organization? R&D, okay Who here is from the operator side? Who here is from the regular who here is from the regulatory side? Okay, who who here is what you would consider? Developing or a newcomer country Who's just getting started in this? Newcomer Developing yeah, yeah, just just getting started to develop your first reactor get your first power program in place That any country that pushes you back on the new nuclear power project will have to fulfill the power approval or authorization to proceed And this includes how would you first of all have an appropriate legal framework in place How would the regulatory framework legislate the requirements in terms of the manpower development for professionals As well as Okay, okay, who did not hold their hand up What do you where are you from what what do you represent? Okay, thank you, okay, so I better get my presentation before David starts Whatever is going to be So so here we are late on the fourth day. We've had Knowledge management nuclear knowledge management discussions all this time Everyone's informed They're Energized they're inspired they're excited. They're they're very much committed to knowledge management, right? So you're going to go back and you're going to you're going to crusade knowledge management We had a discussion at the end of my presentation yesterday where the question came up and how how do you start? Developing a nuclear knowledge management program at the level that you're at. How do you do that in a large? Organization, how do you get the top level of agreement and the budget and all the things that you need to start that program? How do you do it from from here?  How do you start? but There are different challenges to it So we're going to talk a little bit about that and what I want to do is just give you some of the the things that I've seen That I think are useful to kind of give you some value that you can take back and Start to and start to develop a nuclear knowledge management Infrastructure at your level some things that you can do so countries with small Programs emerging programs developer cut programs, whatever they don't have the resources available that some of these large World Bank, you know, I mean those are great things to know But it's hard for us to go back and do something like that, you know from our position Those are great. Those are great ideas and they give great perspective and they show what can be done But whenever you're small when you're first getting started, you don't have so I need just give me ten million dollars a year And in five years, I'm going to show you something You know, it's like we're organization hasn't even existed for five years. We're two years old So it's a different kind of a challenge Some of the disadvantage we talked about that in our group is a lack of history You don't have all of this Momentum going forward with a large organization and a lot of resources and a lot of talent that you can move on You just don't have that at that point If you're just getting started what I've seen in my experience and I mentioned earlier I looked at the participants list There are represented from six countries here in this room that I visited Mostly they're regulatory organizations to try to help get some of these programs started and heart start to meet some of these Obligations to the agency and so forth and they're usually young people or dynamic people and we had this discussion a little bit ago We were talking about how do people My age you 20 more years of work versus people like Hannah Who's 25? How do they view 20 more years of work? Do they want to be in the same office? 35 years from now And I thought I was going to spend three years in Oak Ridge and I was going to be gone and that will be 40 years June 1st You know, so there's usually a lot of young people that are involved in these programs Resources are minimal time Money people talent are limited and there's oh so much to do. There's a high turnover rate also I mentioned that earlier in one of my other or my other presentation is you've got opportunities That's exactly what we talked about with the cyber security person. There are places to go. It's a growing organization Positions are going to be created that didn't exist before And it's just opportunity for growth. It's a great place to be if you're young Dynamic and growing things are going to happen Relationships are just getting started. You're just learning people We've all learned met people this week that we can work with now for the rest of our careers For how long they may be But we've got insight we've got contacts now with the agency that we didn't have before got contact With all these consultants and you just really don't have that whenever you're first getting started So those are some of the disadvantages some of the advantages are also lack of history So you think of history as momentum. Well, we've got all of this moving forward But you can also think of it as inertia, you know, how do you how do you turn a? Battleship, you know very slowly, you know, it's hard to make any progress So you can take advantage of that. You said we don't have the inertia. We don't have the slow moving Bureaucratic we have to get things done so it can be an advantage to you as well Along those same lines, you're not entrenched. Well, this is the way we do it because this is the way We've always done it, you know You have to find new ways to do things and you can work from a clean slate and and in my experience That's a rare opportunity So many times when I want to introduce a new idea I have so many legacy things that I have to deal with and have to consider that it just becomes two or three times Harder than it would be if I didn't have anything if you've ever remodeled the house and I have it's easier to build it Then it is to remodel it. That's been my experience professionally and personally as well You people young people are open to change your opening new ideas You're not stuck in the muds like me, you know or some of the people that I work with you say, well, you know That's brand-new. I'm already getting lectures from Hannah that I don't know how to message And she's slowly introducing me to how to message how I am. She's very proud of me. I sent her an emoji yesterday You know, but you've got all these new ideas and you're open to these new things and you're not stuck like we are and Of course, there are all kinds of assist organizations the IAEA if you're part of a region you've got regional Organizations that can support you other training outreach organizations as well There's just a wealth of research out there a wealth of people that are willing interested to help you get started You could take advantage of those So I kind of want to say a little bit about KM programs in general And this is this has been my perspective has been my experience We talked about this a little bit earlier this week The best ones are fully integrated into the system. I I've seen lots of examples in developing software I thought we had learned better, but apparently we have not where someone comes up with an idea and says We're going to build this great software tool top level managers are going to be able to use this And they're going to be able to push a button and when they do they're going to get this report And it's going to have all this information and it's going to just really make their job a lot easier Well, where's that information going to come from? Well, this guy down here is going to enter it for me So this guy down here This is not really part of his job. He's doing his job But then he has to stop and do something else that he's not evaluated on and then he comes back and does his job So in cases like that if it's not integrated if it's something that goes outside of the organization that goes outside of your job It's not going to happen. My supervisor Told me one time he said was after the NKM symposium. He said this is really great He said maybe when you come back you can show us something we can do with respect to knowledge management We can start doing within the group. I said Michael. We're doing spreadsheets. We're doing calendars We're doing project trip reports. I said we're doing briefings on different activities I said we're doing knowledge management all over the place. Let me give you a list of what we're doing at the group level Here's a group a list of what we're doing at the division level There's a group list of what we're doing at the order at that laboratory level all of these things are integrated and they work It's the ones that are outside of the organization that become a burden that become something. It's that's Not fully integrated that are going to fail. I can tell you that from four years of developing IT systems It should improve the effectiveness of the process Efficiency of the individual. It's like we have this thing that we can do I don't have to use it, but if I use it it makes my job easier. It makes me better It makes me look good. Okay a good KM program has got to be something that people want to use that people get something out of On a personal level and if they can do that they can also get something out of it at an organizational level And here's one that are really pushed for That knowledge needs to be captured as close as possible to the time it's created. I was talking to a regional lead a while back And we were trying to come up with some ideas that we could engage a certain country with We talked about additional protocol. We talked about information management. We talked about several things. He said what else is there? I said well, what about knowledge management? And he said well, what's knowledge management? I said well, that's a long discussion But in general it's capturing the experience and insight of individuals in a way that can be Retrieved and used in the future to support the organization And he said well Ron, I don't I don't think that's going to work for these guys Because they're just getting started and they don't have any knowledge yet Can I said the time to capture knowledge is when it's created not at the retirement party? Well, okay We'll talk about it. Anyway, I don't think you ever did anything on it But it's really important to capture it when it's there How many times have you said well, I need to do this? I've got these other things to do I'll come back and do it later And how often does that happen not as not as many times as you plan for it to right? So what I'd like to do in this next little next few slides is just give you a list of some things That I think you can go back and do you can do these things You can make a case for them and you can get started Anatoly said something earlier today small steps are better than no steps And I think that pretty much captures what this is all about We've talked about mental protege programs. I think they're great. I think they're fantastic Who here has had a mentor when you were getting started? One anybody else Who here has been a mentor? You've been a mentor to Well, I can't tell you the benefit. I mean you're you come into a company country country you come into a company You've got a new job. You're inspired. You know you you're okay. I'm going to change the world You know and they say here's your desk you know, here's your pencil and Here's where the restrooms are and here's where the cafeteria is here's where the water fountain is and you sort of Wait for something to happen and and all that energy can just kind of come out of you But take advantage of that go the best programs that I've seen Part of the hiring process is that somebody gets assigned to a new employee. So this is the person This is the person you go talk to They may not know the answers But they've been here long enough that they can point to the person that's got the right answers And you can learn how they do their work and they can take you to meetings And they can show you how decisions are made around here They can show you where the online resources are and how the where the good ones are where the ones you don't want to waste your time with our And that's all free. I mean, that's that's just something that you can do just to say I you know, I mean one of the one of the things that I personally am enjoying about my job is being able to help young people Get started like young like someone helped me whenever I was getting started way back But anyway, so mentor project programs. I think are just a great a great way to get started one of the well, here's a here's a slide I got from the American productivity and quality Center and you can see here This is this is an evaluation of how effective mentors are and they're saying essentially this is what? 6659% are saying it's either very effective or effective 3% says not effective. I Kind of wonder about that. I would like to meet those people But anyway, the people that are involved say that mentoring is a very good thing to do Another thing that I think is essential and it's something that I've seen Consistently that needs to be done. I know it happens a lot in Power plants and things that its proceduralization is very important but some of these other organizations they don't do as much of that and The example that I have here is we work with the country and there was a lady that she She was a dynamo. She was on the go all the time And she was she was involved in everything and she was known all over the world for what she did And she built the program up from nothing She had a lot of turnover. She had a lot of young people in there I think they all could have been her children and In a certain age, she's supposed to retire and she's approaching that age and We kept talking about knowledge management we kept talking about the need to write some of these procedures down her comment was I Know I need to do this. I just don't have the time to do it Well back that up with the mental protege program. She didn't need to be the one to do it Assign some new person say we have this process here I want you to write down the procedures of how we do this You'll have to talk to these people you write it down. We'll get together. We'll review it We'll adjust it as we need to and we'll go on from there. So you get the procedure Okay, and somebody learns something they get some experience That they can use to kind of be a backup What we wound up doing was we got her group in and we were talking about procedures and we did this little exercise And we asked her to write down the five most important things that she needed to communicate her staff before she left And then we asked her staff to write down the five most important things that they thought they needed to know from her before she left Anyone anybody want to guess how much of an overlap there was None none. She was really surprised. Okay, that made the whole trip worth it So anyway, anyway, I think procedures are very very important Task rotation here's another another thing that you can do task rotation is Taking someone out of the job that they're doing and say You know ideally for a year But six months three months something like that It's got to be enough of the time to get some exposure to say we're going to take you out of this organization and Move you over here. We're going to take you out of this organization and move you over here and you're going to work here Okay, and this is going to be your job and You need to talk to the people that you need to talk to Do the things that you need to do to be able to be proficient at this job Then then we move you back. Okay So then all of a sudden you don't have single points of failure anymore You don't have to worry about bringing somebody in from the outside to replace somebody if the succession planning need comes up You've got somebody's had some sort of exposure to this already content management this came up earlier today in Anatolia's discussion about about information management and I'll kind of want to make a distinction between Document management and content management Document management is just a File okay, it's a file on your computer. It's just where the documents go and you go look them up and Our folder and you go look them up and you you know kind of dig through them to find out what you need content management systems are much more They can do like templates. They can do workflow Okay, they can do a search and retrieval a much more sophisticated thing you can look at look it up on the internet SharePoint is considered a content management system It also involves collaboration involves calendars and so forth But it's also a place where you can store and retrieve your documents content management. I think is essential to have in place and going back to The previous examples, it's got to be integrated into your workflow Okay, in our organization. We have two calendars. We have a technical calendar and we have a staff calendar And when I'm here, I'll let my supervisor know that I'm going to Be gone for a week And then I'll look at our technical calendar and say what what's our group going to be doing this week? What are they going to be doing next week? Who's going to be where? that's where we keep our our Our staff meeting notes is where we keep our financial information. It's not something that's outside of what I do Okay, it helps me do my job better Okay, so I think a content management is something that needs to be put in place early on It's like is this worth keeping? Yes. Well, let's put it here. Let's set up a mechanism where If my hard drive goes down That I haven't lost my career Okay, okay, we had an example with the Department of Energy where we were doing this reporting and eventually went to the agency and Part of what they had to include in their reporting for the different DOE laboratories were maps So we started investigating where are these maps? Well, we're not sure We think they're on Melissa's computer But she's on her honeymoon And she's not answering her phone and nobody knows the password to her account and we're not sure that's where they are Anyway, so thank you for no complimentary access during that period because we couldn't have found our route maps to get you there But things like that if they're in place if there was a good place to put those which there is now And then that's where they would put them. So I think it's important to be able have a content management system early Look at what you need if you don't Pick the right content management system. It's not going to work for you It's going to be a burden and that goes I think to another slide that I was talking about Capturing knowledge near the point of creation. Okay, that's That's essential. I think and you can do that. You can do that as part of your day-to-day work You can do that in trip reports. You can do that in inspection reports and do that in lessons learned any sort of documentation let's say for example you've got some sort of procedure that you've done before you've got a written procedure and You go through this process and you write up the results of that procedure Why not include something say? Suggestions on how to improve the procedure or this is what worked or this is what didn't work make that part of it While the person is there doing that. Let's say every couple of years Every couple of years you're going to go in and look at that procedure to see if it needs to be revised Well, I've got two years worth of written procedures and if I don't have that information then I have to Figure out exactly what needs to be revised what worked what didn't work But if that's just simply included as part of the reporting process, I can look in there and say well Everybody thinks this needs to be corrected. Everybody thinks this works pretty well So that's capturing information as close as you can to the point of origin Collaborative problem-solving. Okay, they're called knowledge cafes group thing Whatever they're different different types of topics for you and I just tell you it really works for me to be able to say I've got a problem, you know, and I don't know how to solve it, but maybe you guys can help me Let's get together. Let's talk about this. It doesn't only help solve the problem But all of a sudden the people that are involved now have exposure to what's going on in the organization. It's knowledge sharing okay, and It creates a broader base to address the issue than you would have had before I think it's a good idea to include some some junior people that can sort of see how the process goes and see how that How that takes place Succession planning. I think I've already talked about that to death. So I'm not going to do it again and experience list and this this used to be what they call a expertise list or Something along those lines and and what that was Was something says are you provisioned in this are you proficient in word? Are you proficient in Excel? Are you proficient in PowerPoint? Have you you know all these things? But the problem was in my experience These things never went anywhere. Okay, they they tended to just be the sort of thing that somebody would say we're going to do subject matter expert list and and What are you an expert in well, I'm an expert in all of these things here and then we would go through this and nobody would ever refer back to it Because it was it was too subjective But an experience list is a little bit different. It's rather say are you an expert? Have you done this?  Gathered reports in this area. We've done an inspection in this area. Have you trained in this area? Okay, and what that does is it's a little bit easier to keep up for one thing. It's a little bit more objective for another But it's something that if I wanted to know well, gee, I've never done this before I wonder who in our group has you know, I can go look that You're claiming to be an expert. I wouldn't talk to you, you know Doesn't seem to be the case, but you say you've had some experience here Then I know these people can give me a little bit better information and whatever they came to be an expert in so that's the list that's what I wanted to talk to you about this afternoon and It's very simple very basic things that you can do But what I want you to be able to do is go home on Monday and say this class was worth it I understand from a high level how things work. I understand the the concepts of knowledge management I understand how they work in large organizations. I understand how they're implemented. I understand how to measure success factors These are some things at low cost to no cost that we can do today I got something out of this class. I can we can do it just just give me a little room I'm not even asking for a budget I'm just asking for a little bit of time to show you that what you sent me to is worth it And I can bring something back to the company that shows it was a value and it connects with everything The agency is doing with things other countries are doing with things that Experts in the field are doing that you can take something home and and take this inspiration and do something with it before you get Flooded with whatever comes on Tuesday. That's it