The National Broadcasting Company and its affiliated stations present the Pacific story. This is the story of the Pacific, the drama of the millions of people who live around this greatest sea where the United States is now committed to a long-term policy of keeping the peace. This is the background story of the events in the Pacific and their meaning to us and to the generations to come. Rihou, Keystone of North China. Probably no place has ever seen such splendor nor such intrigue as this mountain fortress of Rihou. That island out there in the Palace Lake is where the Manchew Emperor Kwong Su was kept prisoner by the Dowager Empress before he was found strangled to death. I had nothing to do with the strangling of Kwong Su. Oh, you are the Dowager Empress. I am. But I thought you died the day after the Emperor, the day after Emperor Kwong Su died. Oh, that. The important thing is, Rihou has always been a place of evil omen. Well, ever since I came here after Japan collapsed, I wondered about this land of evil omen. Today, the Japanese are being moved out and the provinces are torn between the Chinese nationalists and the Chinese communists. In the light of what has happened before, I can't help but wonder what's going to happen now. The Manchew emperors established the city of Rihou and the province of Rihou as their summer capital. It was I who built up the glory of Rihou. You? I am Qianlong. Oh, yes, yes. Emperor Qianlong. 1736 to 1796. Isn't that right? Yes. Mine was the last glorious period of the Empire of China. Were you the one who founded the summer capital here at Rihou? The summer palace was begun here before my time. But it was I who enlarged and beautified it and made it the envy of all the world. Oh, of course. It was I who made the Lama Temple of Putala shine with splendor and paved the monks to tend it with meticulous care. It was I who brought the most priceless collections of objects of art to the palace and the rare old books and the priceless manuscripts and illustrations and maps. And you see those artificial lakes? Yes, I've been looking at them. I made them and even made those artificial islands. I've heard a great deal about those islands, especially that one out there. What happened after I died is no concern of mine. But remember, it was I who made Rihou the... Dynamic character, that Qianlong. He was the boss of the Empire of China for 60 years. And nearly every summer he'd leave Peking and move up to the summer capital at Rihou. 130 miles northeast of Peking. And he didn't move up to Rihou just for the change in climate either. I will tell you why Qianlong came up here to Rihou every summer. Who are you? I was courtier in the court at Rihou. Oh, yes. Not only Qianlong, but all the other Manchu emperors before and after him came up here ostensibly to hunt the tiger and the antelope and the deer and to present brilliant military maneuvers. But what he really came up here for was to impress the Mongol princes, to receive them in dazzling audience, to give them fabulous gifts in order to keep them under his eye. He didn't trust them, eh? He understood them. Well, thanks. It was in 1793 that the British sent a mission to Rihou to fix up a trade treaty. Qianlong sent junks to bring Lord McCartney and his party up to pay hope. You see that golden roof, Captain? Yes, sir. Look at it shine. That's the temple of Putala. Oh, yes. That's where the emperor worships. Right. He's made Rihou here the pearl of China. I've always wanted to see this place. Uh, look at the Chinese lined up on the dock to meet us, Lord McCartney. Where? The emperor sent a reception committee. Gracious of him. Well, it's probably curiosity to see us white men as much as anything. He's probably flattered that you're bringing greetings from King of England, boy. Quite. Oh, they're cheering. Yes. Wave your hat to them, Captain. Really, sir. Well, a week later, Lord McCartney and his captain still had not had an audience with the emperor. One morning they were walking in the palace park. Look, sir. Uh, yes. Could that be the emperor and his monkeys coming toward us? It could be. Uh, wait. It is the emperor. I'll wager. It is the emperor. Uh, you have the greetings from the King, Captain? Yes, sir. Right here. Oh, this is most fortunate. Uh, let me have them. Yes, sir. Yes, sir. Thank you. Your gracious majesty. Ken, what does this person wish with me? You are foreigners? Yes, I am Lord McCartney, and this is my aide, Captain Spandrock. Would you give your emperor these greetings from his majesty, the King of England, and thank him for his graciousness in sending a junk down the river to bring us up? One moment. Your imperial majesty, the two foreigners represent the country of England, and they bring greetings from their King. Tell them, as they do not adore the same gods as I, that I would not desire to talk with them. Well, Chunlong died three years later, but he did not die at the re-hole summer palace. His successes ruled as he'd done, spent most of the year at P. King, and came to re-hole for the summer. That bolt of lightning killed me at re-hole in 1820. Well, let's see, who are you? I was emperor at Jiajing, and not one member of the imperial family came close to the palace here at re-hole for 40 years after I was killed. Re-hole Palace, with its golden pavilion, its magnificent temple of Putala, with its precious collection of objects of art, rare old books and precious manuscripts, was abandoned for 40 years. Then the Taiping rebellion broke out, and when the British and French troops took P. King, the emperor Tsing-Feng and his wife fled to re-hole. No one had lived in the palace for 40 years when we came. There was a curse on the palace, but we did not believe it. In less than a year, the emperor, my husband, died. Then we knew this place, this curse. And so you went back to P. King? I did. And the next time P. King was invested by foreigners in 1900, we went to Xianfu. We would never go back to re-hole. But the Emperor Kuang Tzu was a prisoner on that island out there in the Palace Lake, wasn't he? The first I heard of it was the morning of November 14, 1908. Your Imperial Majesty, your Imperial Majesty. Stop that yammering. Open the door, please. Your Imperial Majesty. Yes, yes. The Emperor Kuang Tzu has addressed me. Oh, yes, Your Majesty. Motherly, hospicious, orthodox, heaven-sent... Prosperous. Prosperous or nourishing, brightly manifest... Ram. Come, sedate, perfect, long-lived, respectful... Reverent. Reverent, um... Worshipful, worshipful, illustrious, exalted Empress Dowager. Now, what is it? His Majesty the Emperor Kuang Tzu is dead. Yes. He was found strangled. How unfortunate. What shall we do? What is there to do? The same night that Kuang Tzu was strangled to death, the Empress Dowager had Henry Puyi, who was only two years old, brought through a sandstorm to the Palace of Peking and enthroned him as the Emperor. It wasn't the interests of the Empire. Dowager Empress, I've always wanted to ask you this. It's been said all these years that in order to keep your power when something unfortunate happened to the rulers, the Chinese on the throne and then dominated them. Is this true? I established the succession, the best interests of the Empire. Henry Puyi was the third Emperor you enthroned this way, wasn't he? He was.  Who can father the ways of the dark? Well, after that, the Manchu rulers of China would not come near Rihou. Three years later, in 1911 and 1912, when the Republican Revolution broke out, and again Peking was in danger, they would not think of coming to Rihou. The days of the Manchus were done, but a whole new and equally spectacular period started. Rihou was a prize, and not only did Chinese warlords fight, one against the other for control of it, but the Japanese also had their eyes on it. Rihou is sometimes considered part of Manchuria, sometimes considered part of in Mongolia, because it's the key or the root between them. Well, when the Japanese moved in and took Manchuria, a fat, self-satisfied Chinese warlord was the boss here in Rihou. He was living luxuriously here in the Imperial Palace that the Manchus had forsaken. He smoked opium and gorged himself on luscious Chinese food until his stomach was a porch, and his jaw was sad. His name was Dong Yulim, which means Jade Unicorn. The Japanese proclaimed Manchuria, including the province of Rihou, as Manjukuo. It is a matter of no consequence. Except that the Japanese are demanding that you, General Dowell, adhere to their new regime of Manjukuo. I am not opposing them. Besides, I take no orders from the Chinese government in Nanking, and I send no revenues to Nanking. But the Japanese require that you do take orders and send revenues to the Manjukuo regime. That is not practical. The question is, General Dang, if what you are doing is practical. I must retain relations with Chiang Huei Yang at Peking. He controls our supply and our ammunition. Yes, but... And I cannot retain cordial relations with Chiang if I permit the Japanese, actually, to move in and occupy Rihou. Let them think they control Rihou from Mukden, yes. But actual occupation, that is another thing. Chiang knows as well as the Japanese that Rihou is the key to North China. And if the Japanese control Rihou, they will hold a sword over North China. Of the Japanese also know that Rihou is the key to Manchuria. And all that is necessary is to turn the guns in the other direction through our mountain passes. If the Japanese try to move into Rihou, I will resist. Well, meantime, over in Sinking and Manchuria, the Japanese commander called a bunch of us newspaper men together and made an important statement. One of our officers, Captain Gonchiro Ishimoto, has been kidnapped by General Dang Yulin. Captain Ishimoto is now being held in prison in Rihou. It will therefore be necessary for us to send a military expedition into Rihou. General Muto, where was this captain of yours kidnapped? Or precisely where is not known. But he was on a mission to Bebao. Mission? Why should he be on a mission inside Rihou? For matters of security that cannot be answered. A detachment of our troops tried to rescue Captain Ishimoto, but were repulsed by the superior numbers of Dang's troops. It was therefore necessary for us to occupy a town on the border of Rihou. Dang denied the kidnapping of the Japanese captain. There is no such person as Captain Ishimoto. He is only a fiction. The whole thing is a hoax and a provocation. This makes the whole north a powder magazine, General Dang. All it needs is a spark to set it off. And the invasion of Rihou by the Japanese will be that spark. Tell the Japanese I will resist them to the last man. The Japanese dropped the story about the kidnapping, but meantime some shooting was going on in the pacified state of Manjukuo. Several Chinese generals suddenly attacked Japanese outposts, grabbed sections of railway and seized a good many Japanese hostages. General Dang mobilized them. These are the troops of the North Eastern Army, sent by Chang. How many are there? About 40,000, General Dang. With my 50,000 and the 30,000 of the Guominchon People's Army, that's about 120,000 men. Yes, sir. And with the volunteers we have between 150,000 and 170,000. Good. I will post these men in the mountain defiles and then let General Muto try to take Rihou. Now the Japanese concentrated their troops outside the Great Wall at Shanghai Guan, China's first gate of the realm, where the Great Wall ends at the sea. Muto then sent an ultimatum to Dang. We therefore demand that all Chinese troops be evacuated from the province of Rihou. The Chinese countered with the defiles. General Dang Yu Lin, chairman of Rihou province, is the High Provincial Military Authority of the Chinese government. It is his duty to defend Rihou. The allegations made by the Japanese government concerning General Dang must be regarded as the deliberate of plot. The Chinese government hereby lodges its protest. General Muto has lodged two drives into Rihou, General Dang. Two drives? Yes, sir. Look at this map. One is moving directly southward from Gailu up here. Yes. And the other is heading directly westward from Qinchao. Both of the drives will converge here on Rihou city. This is Muto's objective. Wait until his forces get into the mountain passes. I have mined them with dynamite and manned the walls of the defiles with my best troops. Besides that, the passes are filled with snow.  When he was finished, he called for his opium pipe and sat inhaling the sweet smoke when the word came. General Dong, General Dong. The Japanese have blown up our ammunition dump. Go away. Go away. Don't bother me. Do you not understand? Japanese spies marked our ammunition dump so they could be seen from the sky and Japanese planes have come and blown them up. Blow up what? Our ammunition dump. All our ammunition. The Japanese have blown them up. They have? Well, stop them. Stop them. Now go away. What can we do without ammunition? From north and from the east, both at the same time, the Japanese forces came. The Japanese bombers blasted the snow-covered mountain passes, blew up the dynamite mines and slaughtered the thousands of troops, the mules, the ponies and the camels trapped in the defiles. General Dong. They have taken Bay Bowl, Chaoyang and Ning Yuan and now they are heading directly here for real old cities. Stop them. Stop them. They are coming in tanks and armored cars. Order the commanders to hold. Yes, sir. Yes, sir. On the approaches to this city, the poorly led and poorly equipped Chinese troops place themselves squarely in the path of the onrushing tanks. They have cannon as well as machine guns. Stay down. Stay down until they are close. Then maybe our bullets will go through them. They're going to run us down. Wait. Do not move. Now. Fire. Fire. They're still coming. They're still coming. Most of the Chinese who died beneath the Japanese tanks had never before seen an armored unit. Nor had they seen airplanes. Suddenly everywhere, airplanes came out of the sky from nowhere and dived and strayed. Ten days after the ultimatum, the Japanese rolled in army trucks into the heart of the city of Rihou. Not a shot was fired to stop their entry into the city. But Dong, the fat opium-smoking Jade Unicorn, had long since left. Shall we continue directly westward, General Dong? What other way can we go? Whip this animal up. Yes. Come. Come. Get up there. Where did you get this flea-bitten-ass? It was the only animal left in town. Well, whip it up. The beast should be forgiven if... If what? If it at least is doing its duty. In ten days, the Japanese had gained the key to North China and the Corridor II in a Mongolian. They had gained an area of 60 to 70,000 square miles, a springboard for further aggression and a base for economic conquest. Oh, that is an inaccurate term. Economic conquest. Well, General Muto, what would you Japanese call it? Actually, it was more a cooperative arrangement with the people of Rihou. The Japanese took over the province of Rihou lock, stock, and vow. They became the bankers, the industrialists, the merchants, and the Imperial Palace, which Jin Lung had made one of the most fabulous places on earth, and where the Emperor Dowager had spun so many webs of intrigue, the Imperial Palace had become a Japanese headquarters. Yes, they were efficient, those Japanese. That they were so tired, sir. Oh, the Dowager Empress. You came back? I used to sit there in the palace and watch them and listen to them. They regulated everything. The land policy, the labor policy. All policies must favor the Japanese. We must encourage Japanese to immigrate from Japan. Who is that ruffian? I've often wondered if the curse of the Manchus was also on the Japanese in the palace at Rihou. Our policy in Rihou was very sound. Rihou, as we read it, not only provided us with rich resources, which we needed, but it was itself a buffer state against Russia. And ironically, it was the Russians who sprung the trap on the Japanese in Rihou. For all the time the Japanese were in Rihou, the Russians were watching them as the Japanese, before they moved in, had watched dark. The Russians bombed a machine gun, Calgon, a hundred miles and more to the west. The Japanese fled out of Calgon and the Chinese communists, who had been lodging their hills around the city for eight years, moved in. The Russians took Rihou, disarmed the Japanese and then left. From Calgon, the Chinese communists moved into Rihou and swept down and occupied the Imperial Palace. This I did not think I should ever see. Communists in the Imperial Palace of the Manchu Emperor. Rihou was once more to become a scene of conflict. The Chinese communists were in Rihou. The Chinese nationalist force wanted themselves to be there. We communists liberated Rihou. We will therefore stay here until there is an overall settlement. General Chao. Yes. Intelligence has just reached us that the United States transports are moving the 13th and the 52nd nationalist armist from South China up here to Manchuria. American transports moving nationalist troops against us? Yes. And the nationalist troops have American equipment. That is the report. The Russians will not permit them to land at Dairon. They will land at Andung, Yinkau and Hulu Dao. Then they will march along the Manchuria coast and into Rihou. Yes, General Chao. We will stop them. The plains and the hills and the mountain passes of the province of Rihou was alive with Chinese communist troops deployed to stop the approaching nationalists. The streets of the city of Rihou sealed with the business and the austerity of a war capital. Peace had come between the Japanese and the Chinese but not between the Chinese nationalists and the Chinese communists. Rihou was again the center of conflict. How little wisdom men have. What was that? Look at them, Chinese against Chinese and Rihou the battleground. How much better off all China would be if we still ruled. The people toppled you, Manchu, from the throne because you ruled for yourself, not for them. They were better off under us. Look! The implications of the battle for Manchuria were broad but Rihou became a testing ground. General George C. Marshall got representatives from both sides together and arranged a ceasefire order until the deeper issues could be considered. But for days it was a hair-trigger truce. The nationalist troops are still attacking us in Rihou. But the nationalists countered this with another claim. You communists, instead of ceasing your military operations are enlarging them and are even burning down villages. We're here to pilot. Yeah, go ahead. There's the palace of Rihou there below us.  That's it. We'll take a bearing from here. We should be over the target in a couple of minutes. Right. Sergeant? Yes, sir. Are those pamphlets ready? Yes, sir. Now what's the matter with those guys down there, Captain? We have to deliver a personal invitation to each one to quit fighting? A lot of the Chinese down there don't know what truce has been signed. The thing I'm worrying about is can they read those pamphlets? Well, enough of them, Captain. We're coming over the target. Right. Sergeant? Yes, sir. Stand by with those pamphlets. Already, sir. Give us a word, navigator. Yes, sir. Ready? Okay, Sergeant. Jump out. Yes, sir. Read. Look at them flutter down. Is that all of them? That ain't half of them. We're dropping enough to cover that country down there like there was a snowfall. The fighting stopped, but Rihou was still a power magazine. The sparks were still flying between the Nationalists and the Communists. We must occupy Rihou. We will not permit Rihou to be occupied by you Nationalists until an overall settlement has been made. Rihou is the key to the overland communications between Manchuria and North China. And we must occupy it. At the present stage of negotiations, we will not agree to this. Good day to you gentlemen. The Nationalists were playing for keeps. So were the Communists. General Marshall conferred long and intently with Generalissimo Chiangai-shek. The next day their decision was announced. But Nationalist government feel that they are strong enough to yield on the issue of Rihou. Where the Manchu emperors had ruled in the splendor of their glittering courts, now they were rough and ready Chinese Communists. Where Qianlong had for 60 years strolled his leisure in the fabulous gardens and where he had not dained to walk with the agent of the British King, now Communist troops walk with grim faces and rifles loaded for action. And where the puppet Emperor Quang Tzu had been kept prisoner on the artificial island in the palace lake by the Empress Dowager. Now the waters of the artificial lake lap on the island as if it is no different from any other insignificant island on earth. It is only fair that your radio audience know that I had nothing to do with the strangling of Quang Tzu. Yeah, of course. Rihou is cursed. An evil woman has hung over it from the date that Xing was killed by the bolt of lightning. It was the same curse that killed my husband. And Quang Tzu? Yes. One follows another here, but the same ill fortune comes to all. Would you excuse me please? I must go and see what is going to happen now. Rihou is such a fascinating slave. Listening to the Pacific story presented by the national broadcasting company and affiliated independent stations to clarify events in the Pacific and to make understandable the cross currents of life in the Pacific basin. For a reprint of this Pacific story program send 10 cents in stamps or coin to University of California Press, Berkeley, California. May I repeat? For a reprint of this Pacific story program send 10 cents in stamps or coin to University of California Press, Berkeley, California. The story is written and directed by Arnold Marquess, the original musical score was composed and conducted by Thomas Paluso. The principal voice is that of Eddie Marr. This program came to you from Hollywood. This is NBC, the national broadcasting company.


I nowadays give away all my presentations. So if you know somebody back at your workplace who could have come to the conference, and yet you think there's some valuable information in here about retrospectives that you'd like to share with the people that you work with, just send me some email, or if you have a USB stick with you, I'll be happy to download everything. I've done that for people who are in my tutorials for the last two days. I'll be happy to do that for you as well so you can have the presentation. If you don't see me here, and I'll be home on the next week sometime, send me some email. When I get home, I'll be happy to email the presentation to you, and you can get it, you can change it, you can sell it on the internet. You can do whatever you want, okay? So that way, you don't have to write down everything, all the quotes and all the URLs, and all of that good information. So, well, how many of you are doing Agile's something? Everybody, almost. How many of you are doing retrospectives? I'm like, well, why are you here? I'm like, hey, better, I'm like, yeah. That's what it's all about, isn't it? I mean, that's what retrospectives, in fact, that's what Agile is all about. And the way Agile works is to do a retrospective because as we'll see, if you don't, then you're not gonna learn. You have to take that time out periodically to stop, think about what's working, what should be done differently. Now, this session has two parts. Not really, it's one continuous session, but there's gonna be break in the middle. And I think we should try to stay in sync with the other sessions so that if you wanna peer one of those other sessions in the second half, this is law two feet again, or if in the middle of the first session you wanna go hear the other sessions, we'll try to make it easy for you by taking the break at the same time, and also if others wanna come in, we'll allow that space for them. So if you're worried about how that's all gonna work out, we will try to, I've got people I think, who are watching the time, yes, who will let me know, time, and then we'll all take our break together. Okay, so you don't have to worry about that. Okay, so this is strange, isn't it? Look at that, I can look at myself, and I can look at the slides. Can you see that? Can you see this? No, can we do just the slides? Is it possible, is it possible just to have, I know you probably worked hard to get that all set up for us, and now we're saying we don't like it. So he's gonna, oh no, not that bad. The other half, so we can, all right, yeah. Oh, there we go, okay. So there's my contact information you've got. All right, good, you've got that. Well, let's see if this works. Okay, so how many have been out to the Agile Manifesto site where you know about the Agile Manifesto? Let's ask a different question. Who has never heard of the Agile Manifesto? Not good. Or you're afraid to raise your hand and say that you don't know about the Agile Manifesto. So we're not sure about the answer to that question, but that's why we're here.  Together to think about better ways of developing software.  That's kind of unusual for software. Josh gave a great keynote this morning about some things that we value. So for the first time, and I've been in this field for decades, for the first time we started talking about values and principles and practices. So the one we're gonna look at, or our focus today, is this one, at regular intervals. Now that's the first thing that's different, right there at regular intervals. So we were used to doing something like a look back on projects. We were used to doing that, but we only did it once. We did it at the end of the project, and now they're saying, no, you need to do this often and you need to do it regularly. There needs to be, you know, some people describe the Agile Project as having a heartbeat. It's a living thing, so there's a regularity to it. You don't wait until the end, although you can still do a retrospective at the end, and you should, and it's different. It's a different kind of practice. But this says we have to do this often. We have to do it at regular intervals. The team reflects. What does that word mean? Reflect, what does that mean? Look back, yeah, I think that's a part of it. It's got another big part. What does it mean? It does have the idea of a mirror, doesn't it? Reflection, and what does it mean when you reflect? Yeah. You need to look at yourself. You need to look at yourself. And how are you gonna do that? And we have a mirror in the team room, so we all line up and say, oh, yeah, I need to, and my haircut, is that how we do it? What enables us to look at ourselves? What's the behavioral characteristic that enables us to do that? Is what? Yeah, and how do we do that? What's involved? I know he knows because he sat through a class with me the day before yesterday, and we gave some tips. Do you remember those tips? What were the tips about? What were we talking about? Yeah, we did talk about influence, didn't we? Yeah, and change, we did. Did we also talk about thinking? This is about thinking. I once had a manager ask me, Linda, what's this peer programming?  He said, yes, yes, I get that, I get that. One of them is typing. Well, what's the other one doing? I said, I think the other one is thinking. And this was in the United States, so he pounded on the table. He said, thinking, we don't have time for that. We have deadlines. We're gonna get the product out the door. Thinking is a waste of time. I wanna see code. Working code, isn't that what it's all about? Working code, it does take time. This is not for free. You do have to stop and think. And that's effortful. So if you think that you're gonna be able to do this quickly without any investment, no, no, that's the first justification, is why is it worth the time and we have to take the time to how to be more effective. We want it to be better. How do we do that? So I'm a musician and when I see this word, tune. Tune. Any musicians in the crowd, what do you play? Oh, she's a singer. Do you have to tune? You're so loud. Yeah, I do have to do a little bit of tuning. Any other musicians? Anybody play a stringed instrument? Or maybe saxophone? I suppose you can tune it. Can't you a little bit? You don't want to tune too much. Stringed instruments, especially, they have to be tuned. And if they're not, we can't play together. But tuning is not a major upheaval. It's not a big adjustment. It's a small adjustment. See, I say music and music here. Isn't that wonderful? Let me say lots of money. It seemed to work quite as well, did it? So tuning is about small adjustments and adjusts. Just small changes. So I'm a believer in the power of small change.  And small changes can have enormous impact. So the Agile Manifesto says if we're going to make those small changes in order to know what to do, we've got to stop at regular intervals and think about it. So that's why we're here. And if you haven't read the Agile Manifesto, you can go there and you can have my slides anyway, so you don't need to write down that URL. Look at this question a lot from people who have been in software for a while. Is that a post-mortem? What does that word mean? It's Latin. What does it mean? I don't know. It's what does it mean? What does it mean? Post-after-mortem-death. After-death. That's what we used to do. At the end, when the project was over, so what comes to mind when you think of after-death? It's a body, isn't it? A body lying on a slab, cold and dead. And no matter what we learn, it's not going to help this guy. It's too late. And that's the problem with what we used to do with a post-mortem because we did stop at the end and we did look at what happened and we did try to capture those lessons learned and we did try to save those. But it was too late. There's nothing we can do when the project is dead. So if we're going to carry forward with that medical metaphor, wouldn't it be better if we said, hey, let's not wait until we're dead? So I'm incredibly old and I attuned to that. I said, yeah, I'm not waiting until I'm dead. Let's learn now. Let's go in and see the doctor and we'll have a check-up and he'll say, Linda, your blood pressure's a little high. You've got a lot of stress in your life. Wait, do you have a lot of stress in your mind? And he says, you've got to stop that traveling around the world. You need to do some more exercise. You need to watch your diet. Have you ever had that experience?  How many of you do that? A few or you're lying. So that's what we're trying to do with our projects. We're trying to have a little check-up. We're trying to say, you should take some more vitamins. Let's have some vitamin C or some vitamin D or let's make some small changes while we are still alive. With the idea that that will improve our lives, our projects while they're still running, we're not going to do this. Because if we wait, it's too late. Now, of course, you should also do a post mortem at the end of every project. I already said that and I already said it's different. So what Angela wrote to the table was learning while the project is still alive. That's what we're talking about. So it's very different from a post mortem. There aren't too many books about this subject. This is the best. It's a classic. It was written by my good friend Norm Kirst and I advise you, if you haven't read it, to go out there somehow and find it and read it. It is the best for justifying the practices. It gives you the history. It gives you the psychology. It gives you the rationale, the underpinnings, the understanding about what retrospectives can do for you. It's called project retrospectives and he was talking about post mortems. But everything in there applies to all flavors of retrospectives. So that's my first recommendation. And then, of course, my good friends, Diane Larson and Esther Gerbe said, we're going to build. We're not replacing. We're going to build on Norm's book and we're going to say what things are different about agile projects and what things are different about agile retrospectives. So in order to really take advantage of this, you need to have already read. So there's a prerequisite, I guess, for this book, Norm's book first and then Esther and Diane's. This is excellent because it says, now we want to do some things differently. So you need both of these. And there's a new book that's just come out. This one is available digitally. It's by my friend Patrick Koua and he said, I know that Diane and Esther's book is good, but I'm going to add to that. So in this community, at least, they have said this is good. I'm going to add on to that. I'm going to add on to that. So these books depend upon one another. They don't disparage the work of the others and they are all good and they'll all help you do a better job of leading your retrospectives. So we've got some three very good books there. Oh, and this is another of my favorites. Do you know this one, Winnie the Pooh? Does he appear in India? Yeah, actually Winnie the Pooh has a different name in every country. So that's his English name, Winnie the Pooh. He's a little bear. Christopher Robin is his owner. It's the little boy who takes him everywhere. And here's a picture of Christopher Robin coming down the stairs now and he has Winnie the Pooh behind him and it's uncomfortable. Winnie the Pooh says, oh, this is how I go down the stairs. Bump, bump, bump on the back of my head. Oh, I know. I know there must be a better way. If only I could stop for a minute. If only I could stop bumping and think of a better way. Does that sound like any software projects, you know? If we could just stop bumping our heads? If you haven't read the book, you should. There's also a movie, if you want to watch the movie. And the penguins are coming, so that's also good. Winnie the Pooh and the Penguins. How many of you believe this? By going through some experience, I learn automatically. And I'm not going to ask this gentleman here because he spent two days with me and we know how the brain works now. So I'm going to ask him, but what about the rest of you? I go through an experience and I learn automatically. That's what I do. My brain learns. How many of you believe that? Okay, so let me pick on you. Have you ever made the same mistake twice? What happened there? Okay, I have a friend who's been married five times. Every time I ask her, I say, Susan, why are you getting married again? She says, oh, this time. This time is different. This time everything will be wonderful. I know this time I won't make the mistakes I did in the past. So far it hasn't worked out. She still keeps getting married. Does that remind you of any software projects you know? We'll talk about some research. We'll talk about some experiments that show when you run a little scenario and people make mistakes and then you let them do it again. They keep making the same mistakes over and over and over. That's really not true. Now there are some kinds of mistakes that we do tend to learn from, especially as children, if we put our hands on a hot stove. We learn and we carry that with us. So we don't want to do that again. So those visceral experiences, we do seem to learn from. But the kinds of experiences we're having in marriage or in software projects are, for the most part, not visceral. They're mental. We made a mistake in judgment. We made an error in coding or testing. We don't seem to do a good job. Yes, we have accumulated a lot of this. Experience provides data. And in fact, that's what is the driver in retrospect is the data. What happened? We do have to accumulate that information. Looking at that data, then we have to do something else. We have to process that. We have to examine it. And then we have to say, what are we going to do as a result of having had that experience and not doing that, not stopping and examining that and picking it up and saying, what are we going to do differently? Will lead us to just make the same mistakes over and over. That's how our brains work, unfortunately. So this is the data. And again, I will not only send you the presentation, but if you'd like to read the paper. Now, this is a publication by Harvard Business Review. So do not tell the publisher that I'm sending you the paper. But if you want to read about the experiment, what they did was, wait, how many managers in the room? Well, we know managers are very intelligent. Right? So managers were given a problem to solve. In that scenario, they were put under a lot of pressure and things started to go wrong. And the managers, I know it's hard to believe, the managers began to make mistakes. They asked the managers to look back and the managers said, oh, I guess, yes, I can see that I did make some mistakes. I know better. I should have. So they know better ways. They know what they should have done. They all agreed that. And then they ran another scenario, another really complicated problem. And then they put on a lot of stress and things began to go wrong. And what happened? They made the same mistakes. Even though they had just admitted that there was a better way, they made the same mistakes over again. And that's managers.  So if managers are susceptible to this flaw, what about the rest of us ordinary people? We had the same fallibility. So it doesn't work to even know a better way. It's under stress. When things start to go wrong, we make those mistakes over. So I'm going to tell you a story about my first retrospective and what I learned that I hope will help you. And while I'm telling the story, I want to invite you a table at a time to come up and look at these funny cards I had on the stage. We'll talk about what the cards are and how you can use this exercise in your retrospective. But I want you all to have a chance to read them. And I know I can't invite you all up. So we're going to start here. You come up and when they have finished, then let's take the next table. And so you can self-organize based on that little algorithm. So you come up and while we're doing that, I'll tell you the story. So I was working in a medium-sized telecom company and my job was research. I was trying to look at patterns. And I was introducing the gang of four patterns. I was writing some patterns. I was actually writing a book about patterns. The whole organization was so excited about this that they said, well, we use patterns for everything. This was a company that produced a giant switch. Switches make telephone calls or at least they did in the good old days. And this one makes long-distance phone calls. It was about 8 million lines of code. And the company knew that it was working perfectly well but that the lifetime of this switch was well coming to an end. They knew that the number of users was diminishing. They knew that their company was in danger of going out of business so they thought we have to come up with some new ideas.  Just small experiments. And what they found was that most of them were not doing very well. The project that was supposed to save the company struggled to produce anything. And I don't know if you've ever seen this in a company or not but we are loss averse. We find losing very painful and that includes once we've made a decision about something we are reluctant to give that up. So in the staff meetings the executives would say well you know this project isn't doing very well but we've already spent a million dollars on it so we can't stop now. Six months later, well we've already spent two million dollars we can't stop now. Six months later we'll spend three million dollars we can't stop now. Six months later we've already spent four million dollars and that was at a time when the dollar was really worth something. Finally they did cancel the project and they canceled it in January. They canceled it in January which comes after December and in December in the U.S. most of the celebrate Christmas. So what do you think the people on that project were doing over Christmas? Were they taking a lot of time off to spend with their families? What do you think? No, they were not. They were working overtime. In fact they had been working overtime for two years. So they not only missed Christmas but they missed all their vacation time and all other holidays. I'm sure that would never happen in India. It happens in the U.S. all the time. So here people were in January. They had worked for two years and especially they had just worked over Christmas. So how did they feel? So my boss came to me after the project was canceled and he said, Linda this is such a bad experience. I want to make sure that this never happens again. I'll bet there are patterns. Could we find some patterns so that we learn from this? I said, ah, well sure, we'll do that. I had no idea how to begin. But I had a friend and his name was Norm Kurth and I knew that he was writing a book called Project Retrospectives and it had something to do with looking back and learning and so I called up Norm and I said, Norm help me, help me Norm. I don't know what to do. And so he gave me some ideas from his book even though the book wasn't out. He was sharing those exercises. He said, you know Linda, people don't just get in a room and talk. You don't learn that way. He said, you have to really trick their brains. You have to get them to do something so that all that data is captured somehow. And he said, I'll tell you a good way to begin. It's called a timeline. You make a space on the wall or we've got a stage here. You can do it on the floor, you can do it on the table and you make a space and you say here on the left is the beginning of the project and then we mark off say every month or every six months and we mark up the time since the project began until the project was canceled. And then we give people some cards or stickies and we say, what happened on this project? Write that down and put it in the appropriate place on the timeline and the way I do the timeline now, this was not Norm's idea, I put it on a color and the color means something. The color means how I felt when that event happened and so when the timeline is done you can stand back and look at it and you can see a lot just by looking at the colors without actually reading the cards. There's a lot of information. It is my favorite retrospective exercise. I encourage you to do it and I'm going to tell you today a way to do that both for project, for inter-project retrospective for an agile retrospective and for what I now call continuous retrospectives. How many of you are moving toward continuous integration, continuous deployment, continuous delivery? That's our goal. So that means we have to have a continuous retrospective and the tool that will enable you to do that is this line, the timeline. Something we've been using now for decades we can adapt it. It's a very adaptable tool. So my team came in and we built our timeline. It was on the wall. It had lots of cards because after all the project had been running for two years it had cards about all the things that had happened and how people felt about them and I sent to the team, all right now the timeline is completed. I want everybody to do what you're doing. Walk that timeline. Read all the cards. Seems pretty simple. That's what people are doing here. They come up. They read the cards. They sit down. Easy. They stood up. They went to the wall. They started reading the cards and then they began to gather in the groups. They'd stand together and they'd point at a card. They'd touch the card and they'd read it together. They were remembering some event. Then they'd move to another card and they would gather in a different little group and they'd touch that card. Remembering. So my war, the war from my history is Vietnam.  It's a lot like this. There's a wall in Washington DC. It doesn't have events on it. It has names. It has names of people who died in the Vietnam War. It's right in the middle of the city. A big noisy city. But when you get close to that wall, people are quiet. And they do the same thing that my team was doing. They go up to the wall and they touch a name. And they stand there in little groups and they remember. They remember what happened. I didn't know what to do. I didn't know what to do. I just saw them standing and moving and touching cards and talking to each other and gathering in little groups and remembering what happened. And at least I had the grace to take a chair and sit in the corner. And I was just waiting. And after a long time, they finally sat down. But I realized something very important about retrospectives that maybe you don't take advantage of. And when I saw this research, I thought, well, yes, this explains it. Oops, there we go. Lots of bad things happen in organizations. And when those bad things happen, we pretend that we are not affected. We pretend that we can go through that bad thing and we can just keep on going. Maybe this is just something in the United States. I don't know. Maybe it doesn't happen in India. We don't talk about it. We don't talk about those bad things. Maybe no bad things happen in India. Well, we have plenty of bad things. And we don't talk about it. So imagine what those guys are going through. You know, having worked for two years and they knew the project has been canceled, were they excited about going ahead and starting another project? Were they all saying, oh, yippee! I get to start another project. Let me add it. So the research shows that if you don't have some way of talking about it, if you don't have some way of, well, they say that going through something like that is like carrying a burden, a physical burden on your back. And then if you don't have a chance to put that down, that you take that burden with you on the next project and the next project and the next. So it says, it keeps them from moving into the new setting. So on the new project, it keeps them from moving forward with enthusiasm. They just are weighed down. So that's a very good thing that could happen in retrospectives. This is a time when we could talk about it. And maybe we can't talk about it because it's too painful, but maybe we could put it on the timeline. So one of the things I do as an independent consultant is I go into companies and I will lead a retrospective over a project and what do you think those projects look like? Do you suppose some CEO sends me an email and says, Linda, we have had such success. We want you to come in and look at this project because we know there are wonderful things that happen, so we want to know all the good things that went right on this project. Do you suppose that's the kind of email I get? No, in fact, I have never gotten any email like that. Usually the email says, we've got this giant disaster. I want you to come in here and do a retrospective. And is that executive interested in learning? Does he want to make things better? Is that why he wants me to come in? What do you think? Maybe he might, but what might he also... and it's always a he? What might he have in mind? Why does he want to do a retrospective on this massive failure? Yeah, he wants somebody to blame, doesn't he? He wants to say who's responsible for this failure and he doesn't know exactly how to do it, so he thinks I'll bring somebody in from the outside and they will have to find somebody to fire. That's not what it's about. And so I explained retrospective is about learning. It's not about finding somebody to blame. And it is about, especially for those failed dysfunctional projects, it is about finding a way to bring closure to all of those horrible things. This is a practice that has really nothing to do with software. It comes from military. Military organizations around the world have always done retrospectives because lives are on the line and they want to make sure that doesn't happen. So again, I'll be happy to send you any of this information, including that other Hardware Business Review article. It's about how all those other military organizations do retrospectives and have been doing it for thousands of years. This is an interesting book. I know you don't have time to read it, but it's the only company I know that hired a monk. They were having so many problems and so many bad things. They said we need to have an on-site monk and he will help us deal with all of these horrific issues. And the timing for that was wonderful because this is a company that sits right across from Manhattan. And if you stand at the windows of this building, you can look across and see what was the World Trade Center. And people were doing exactly that on 9-Eleven when terrorist planes flew into the World Trade Center and destroyed that building. They were watching. Oh, it was a good thing the monk was there that day. No, you're probably not going to want to hire a monk though, are you? No, you never know. So what I always tell that executive is my rule for retrospectives is pretty clear. And this should be your rule as well. You should constantly remind yourself of what Norm Curth said in his book. We have to believe when we come together to look back and see what happened, whether we're going to write a timeline or whether we're going to have a discussion about what should be done differently, is that we believe that everybody on the team was doing the best job he or she could. Of course, we're better now. We know more now. But we believe that at the time given their skills and abilities and what they faced, they were all doing the best job they could. And that means we don't ever point a finger at anybody. Norm's rule is no naming, no blaming. That is not allowed in the room. You cannot say, I was okay, but it was that other guy who was working on the database. He's the one who messed things up. Now, of course, you can talk about problems if you want to. You can say, yes, we had problems with the database. That's all right. You just can't put a name.  And you can't name another team, either. Well, the developers, we were great. Man, those testers. Were those marketing guys? Boy, amen. So that should be your rule as well. No naming, no blaming. I hear all the time from people who say, Linda, our retrospectives are so boring. All we do is get in a room so easy to do. And I said, well, do you start with a crime directive? And you say, well, naming, no blaming? And they said, we forgot. So put that back in place. You can talk about all the problems you want, but you cannot attach the name of a person or the name of another team. Of course, if you want to say something good, that's all right. And that goes not only for what you say, but for a card on the timeline or for a note on a flip chart or any other part of any exercise you might do. No naming. No pointing of fingers. It's all about assuming that everyone did the best job they could, at least for the sake of their retrospectives. What we know is something interesting that I just heard. The cognitive scientists tell us a little area of your brain it sits over your right here. And that's the part of your brain that, at least at the moment, we believe is involved in innovation and creativity.   When you get in a room with people and you listen to what they have to say and you think about what they are thinking. So by gathering and looking back and talking about what happened and listening to other points of view and hearing what other people think, you're exercising the part of your brain that's involved in creativity and innovation. And that means you're now sort of practicing thinking about new ways. So there's another good reason for doing retrospectives. Exercise the part of your brain that will help you be more creative by thinking about what other people are thinking. There are lots of times to do retrospectives. We can do them at the end of a project, we can do them at the end of an interval, we already talked about the heart beat. We also know that a lot of times companies are going through an agile transformation. Did any of you do that or are you starting to do that? A very good way to begin and again I will be happy to send it to you. There's been a giant study that shows retrospectives are a good way to lead agile transformation. Because if you gather and think about, well what are we doing now? What's really working well for us? What should we think about doing differently? That will give us some ideas for new experiments in our agile transformation. It will inform our decisions about what kinds of changes we can make in our organizations as we move to agile. Instead of just having a meeting with people who are so far removed from the work that they have no understanding of what's actually going on, you actually look. And how are we working now? And what are the things that we're doing well? Maybe we don't want to touch those right now. Look at areas where we could use improvement. Maybe that's where we should focus our attention as we move to agile, whether it's initiating new practice, TDD, pairing, sandouts, whatever, that maybe we won't want to do that wholesale. Maybe we want to decide where to focus our attention by doing a retrospective. You're being really quiet. I haven't got a question yet. Do you suppose they are okay? I didn't get through any of my material on the last two days because so many people ask questions. So here are the questions you ask. And I think a lot of times what I see is you don't quite have it right. I think you get the first one right. What's working well that we don't want to forget? It's just every single good thing, but these startling things, the surprising things, the innovative things, we say wow, that really worked well. Let's not forget that. Now the second question is where we go astray. The second question most teens ask is what's not working well? Where are our problems? What are the negative things? That's not the right question. If that's the question you're asking, then no wonder you're drowning in complaining and whining because you just ask for it. You say, where are you having problems? What's going wrong? What are the troublesome areas? Well I can tell you plenty of those. That second question says what should we do differently? Whether it's at the end of the project thinking about future projects, whether it's at the end of the interval thinking about the next interval, what should we do differently going forward? It is not about focusing on the negative, focusing on the problems. Yes, of course that is informed by some of those problems, but maybe not. So here's another story. I do a lot of traveling and I'm often in a different time zone. I arrive in the middle of the night getting old and I can't remember how to set my alarm clock. Of course I had my phone. I can use my phone but I'm making mistakes with that as well.  I had it set a.m. and p.m. I had it backwards. I forgot to turn it on. I guess I should just give up. I stay home. Forget it. Do you ever do that? So my husband is a wonderful guy and he always does the same thing. He throws everything that I say in presentations back at me. He said, Linda, why don't you take your own advice? What are you going to do differently? Stop whining and complaining about your iPhone and your alarm clock. What are you going to do differently? Who would be happy to be married to a guy like that? I don't know. I feel like I go through the steps. I try to remember how to set the time and I turn on the alarm and I kind of think it's something differently. I got it. I'll get another clock and so then I'll have the phone and I'll have two clocks. Surely the odds are that I'll get one of those right. He said, fine. It'll be an experiment. I went to the store, I got another clock and what I noticed was because now with the extra clock and so I'm setting the phone and I'm setting the trip. What I found was that all of them were now right. Now, I cannot explain that and it doesn't matter and that's exactly what happens on teams. You have a lot of problems and some of them there's really nothing you can do anything about and so you should just forget about it. Your focus should be not on the problem because maybe you can't solve it. Just ask yourself what are we going to do differently going forward? It's a much more exciting discussion and what you'll see is the same magic. You will see that somehow doing something different changes everything and you won't understand why it works and you don't need to. We used to do an exercise in retrospectives I had a class called Five Wise. Well, here's the problem why do we have that? Well, it's because it will why do we have that? We spend an enormous amount of time in retrospectives going through that and always we get to the bottom and we say the problem is our manager what are we going to do about that? That was always the case when you dug deep enough you were going to get so far away from anything we had any control over so what good was it? So just shift your focus and say all right we have a lot of problems I know but let's think about some things we can do differently going forward and they do not have to be big they should just be very very small experiments and then you can ask yourself what have we learned from that? We learned that somehow we can set all the clocks correctly and what still puzzles us there are always going to be puzzles and you should write them down and anything that comes out of your retrospective you should make public and you should post it somewhere where everybody can see it often when I go into a company and I do a timeline or a lot of exercises I see at the end of the day that the executives all come in and they love that timeline and they look at all the flip charts I once saw a CEO of a large company get down on his knees because he wanted to see a card that was posted on a really deep timeline he wanted to know the truth he wanted to hear it and maybe realize that executives well wait a minute how do executives get information about projects? how do they know what's really going on? how do they? they're direct reports and do the direct reports say oh I don't know what are those over there? what do those direct reports usually say? everything is fine oh yeah are we going to deliver on time? oh yeah are we going to have any problems? oh no will the customer be happy? oh yeah he and it is usually a he he wants to know and he has no way of getting that information and share the results of your retrospect these are all anonymous of course he will read what you say and if you have a puzzle and you post it he will be happy to read it it's your way of communicating right up the ladder use that that's powerful so here is the difference what happens at the end of a project you look back on that it's like examining the body of course that's a good thing to do and it's your intent to say yes we have this wonderful project here are the things that we learn here are some suggestions for doing things differently you go through the regular set of exercises that's good that's a very good thing but what should Angel projects be doing? unfortunately where we've gotten into trouble these are trying to do exactly that and that is wrong so the practice we want for Angel project is not let's look back on everything that happened let's ask what went well let's look at what should be done differently no every small interval how long are your intervals how many are doing 30 days a couple of people how many are doing 2 weeks most people anybody doing 1 week anything smaller than a week there are a lot of companies now that are hours release every 2 or 3 hours that's trending for continuous delivery so for whatever your interval is what you should always be doing is some small experiment I have one company I'm working with now where they do 2 week intervals and they on average do 6 experiments so it's a different kind of retrospective at the end of the interval what you're asking is how did that experiment work how did those 6 experiments work how did those 3 experiments work so when we say what worked well we mean what worked well with those experiments when we say what should be done differently we mean what new experiments should we do going forward when we say what puzzles us and what did we learn we mean what did we get extract that information from those experiments and by experiment and I'm going to talk about this tomorrow morning I mean a real scientific experiment I mean a real hypothesis I mean a real plan I mean real measurement I mean real analysis so that when you get to the end of the interval you can say we learn from our experiment something that will help us going forward I have a question I can't believe it be still my heart yes so the thing about action items is that somebody must own those and I know the idea is well the team owns the action items and the problem is it's like my friend has been married 5 times do you remember her? she has good intentions and things are going to be different this time and then some stressful event happens and those action items they don't get done maybe your team is different maybe you're very effective I don't know but I'm not a believer in action I used to do that so I started doing strong in 1994 and in the beginning I was a believer in the 5 lies and action items what happened over time nobody did those actions and after a while people said how are we doing this? it could be yes an action item could be an experiment yes absolutely absolutely yes absolutely that's an important issue that whatever it is that you're trying to do in your organization choose your words carefully and in some organizations just saying the word experiment is something you would not want to do because it implies that you don't know what you're doing well we have no whether this will work or not so we're just going to do a little experiment and if some manager at some level hears that you're doing experiments that you're wasting time on things that might not work out it could be very upset so sure if you want to call it action items that's fine with me but they are they really are experiments the other thing I hear from teens is our retrospectives are boring all we do is get in a room and answer the same thing and people complain and we do the same exercises and less boring well of course an experiment now that's exciting and I get to think in a scientific manner I'm going to talk about this tomorrow morning so I don't want to get too much away but we're not often given the chance to be scientists it's how we make progress it's not a debate so the strongest person when it's about a hypothesis and testing that hypothesis with a real experiment and then doing some analysis and learning so that you can build on that and go forward and you're not endlessly talking about the same thing over and over and over we're still having the debate about whether we should do code reviews or pair testing we've been debating code reviews for decades we should just do some experiments so I'm okay with calling it an action item because that's very deep language language always matters yes another question you know could be I'd be I'm open to that I just have never seen it be very productive time but if it works for you so I know I've said this before but I'm old I'm incredibly old and I'm pragmatic and I know that we have very little proof for almost everything we do the software business is notorious for following fads getting on the latest bandwagon following the silver bullets we love silver bullets so if you say it works for you I say do it as long as it's working as long as it's useful what you're doing is as useful as what I am seeing now and I say wonderful do it yes oh yes good, look at these questions you were just waiting for somebody weren't you? before coming to this session I had an impression and I read the blogs and the statements where they say a retrospective always result in an action nonsense well wait a minute we're calling experiments actions that was his idea that is an action and the team decides to choose the action which gives the most benefit to the operation of the project team and that's her effort so they choose the side which are the two action items they would want to follow in the future generations I'm sure you didn't hear what she had she's very soft spoken but she's saying a retrospective should result in actions and we're going to call experiments actions so I'm good with that and then she said there could be a prioritization because we want to make sure that our effort is spent with given the effort that we have to exert and certainly we know that teams have limited time and that we should we could examine all the possibilities for experiments going forward and saying you know this one is really cheap and easy and we can get a lot of information from that so certainly that could be part of it if you have too many things my experience is that the team puts things out there and they just happen but you're absolutely right just like the gentleman over there should call them actions and there might be a sense of prioritization and we might have examined some problems along the way in order to determine that yes, yes and yes there are agile coaches who teach that what happens in a retro stays in a retro and the previous slide you mentioned that you should probably go ahead so I mean there are coaches who actually teach this that it stays within the team because you have honest opinions you have things so as you said we may not be able to do something about some problems but at least we uncovered those how do you relate the two so maybe there might be environments where teams don't feel they are safe and of course we know that one of the guidelines for retrospectives all comments should be anonymous you can say anything well actually you cannot no naming no blaming so you can't say anything that you want but that it's all anonymous if you had a chance has everybody been out to look at the timeline there is no name on any of the cards yes we could look at the handwriting some people do that and if there are colors some people writing color pens might be able to see so if that's describing your environment you might want to think about that absolutely all of these are recommendations that fit in a context but the other thing I would say is that you might be surprised that you're not as unsafe as you might believe and that there might be benefits in sharing that would be worth an experiment you could try some kind of sharing because I believe that once you start doing experiments and other teams hear that you're doing experiments and they say I want to know what happened with your experiment so I once did some research in knowledge management do you know what that is we have a knowledge management system so you have a big database and it has knowledge lots of people like that because they want to put knowledge in but nobody ever wants to take knowledge out so those enormous projects were failures and I then learned something interesting I said it's just imagine that some group of people in your company had a secret reading high level executives perhaps had a secret reading and maybe they were going to decide something momentous like whether people were going to be fired or not does that ever happen and when the meeting is over people in the room are told do not tell anybody what happened in this meeting here's the question how long does it take before the information from that meeting is spread across the entire company how long before everybody knows the secret how long does it take a day it depends on the organization depends on the organization but it happens pretty quickly doesn't it so how does that happen did they all go to the knowledge database did they look at the knowledge they talk to each other people talk to each other I looked at Josh's app today for rewarding people we have to have an app for rewarding why can't I just go up to you and say hey thank you for doing why can't we just say thank you person to person that's where all the power of communication and anything is is my talking to you and that's exactly what should happen with retrospectives if you don't talk about it if you don't share the results of your experiments if you don't somehow make public what your team is learning isn't that what it's about learning if we don't share it, if we put a wall around it well then all that learning is lost to the rest of the organization so I'm not looking at the time are you looking at the time you are we haven't had a break yet oh we're just we didn't have a break you didn't have a break I promised them they had a break and now you're making me look like a liar so is the break over do you miss the break alright no what we're gonna do is we're gonna take a break right now we're gonna take a very short break let's take seven and a half minutes so go outside go to the restroom or do whatever come back and I'll stand right here if you wanna come up and talk about anything and then we'll come back and talk about our experiments yeah wait let me take this off I can see other people in the room now who were in my tutorials for the last few days I try to advocate not sitting and that sitting is very bad for us and in fact there is a study that shows the longer you sit every day the sooner you will die I pay attention to research like that so I stand up I try to stand up now I know some people are not comfortable with standing very long but see he learned look at that he was standing so we should allow that we should allow people who say that's enough I'm tired of sitting I want to go stand up that's absolutely a good thing to do so now we need to pick up and we're not gonna miss lunch yes okay we're not gonna miss lunch so we have talked about maybe I'll have to change the name now these action items are really experiments and so that's the purpose for agile retrospectives is to do some small experiment in the next iteration now the question is where do those ideas come from and often I have seen that in agile teams during the daily stand up what held you up yesterday hey that gives me an idea for an experiment well how can we capture those good ideas and the answer is the timeline which is an exercise that can be done in a project retrospective at the end of a project and it's put on the wall or on the floor or on a table it's usually built during the retrospective how many views the timeline and all how you do it during the retrospective you build the timeline yes that's an exercise during the retrospective but now I coach my agile teams to build the timeline as it happens so you don't wait until the end of the let's use two weeks as an example you don't wait until the end of the two weeks before you start building the timeline you build it as it happens so as events occur over the course of the two weeks people are picking up cards and writing them and putting them on the timeline it's real time it's a real time timeline it's being built as the events happen and that's not only good because then you don't forget because even you all you young energetic people even you forget don't you maybe sometimes what happened two weeks ago so it avoids that problem it also gives us a chance to capture those ideas they come up during stand ups they come up when we're stuck on a problem and then we say I have an idea for an experiment let me go put that on the timeline so when we're ready to do the retrospective we just quickly look at the timeline which has already been built and we pull out ideas that are there for experiments that might give us some ideas for other experiments so the best way to start doing experiments on your agile projects is to build the timeline in real time so you need to find a place I don't know where it's going to be a wall, a conference room some place where you can have this timeline going all the time there are those cards to it now the wonderful thing about it is if you have a safe environment other people could walk by and they could see something as it happens let's perhaps open the possibility that maybe we could even do something about it so when you came up did you notice that there were different colors of cards and across the top did you see what those colors mean those are just my choices you can use any colors you want but when people look at the colors of the cards what cards stood out for you what cards did you tend to focus on or did you notice what color was that pink the pink or the red, why is that why do we go for the dirt that's an anger that's a frustration and not only that we thought yeah I know how that is did you see the one oh here it is I quit so these cards come from a lot of classes in retrospectives where I say pretend that you're very retrospective on your last project so this is not from a company if I go into a company I have to sign a non-disclosure agreement I can't walk out with their cards so these are from classes but I say imagine, remember that project so they are from real people who are thinking about what happened on real projects and it's amazing isn't it that it looks like it could have come from a team working together on a project or maybe it could have come from your team because most of these comments are things that we do here from everybody so you can use any color you want I choose red or pink this means anger or frustration blue means happy green, challenge yellow, surprise but you can use different colors you can use different feelings or emotions but even when you stand back you get a lot of information just by looking at the colors so you should always start your retrospective with brown rules here are some and of course that would include normal curse prime directive, no naming no blaming and we talked about this earlier there were some people who came up to ask a question over the break who facilitates and the answer is it cannot be somebody on the team now you don't have to bring in somebody like me from the outside but you do have to bring somebody from outside the team and often what I see is two teams that kind of work together and a member of one team will come in and facilitate a retrospective from the first team and vice versa so you have a little reciprocal arrangement there and you should all take turns facilitating a retrospective is an interesting experiment and also it's a good way of sharing information so if team A sends a facilitator over to team B well they're going to see that retrospective and they're going to take advantage of all the information that comes up they're going to learn and in fact if you start spreading that role around not just with two teams but maybe three or four teams then that's a good way of sharing information because I facilitated your retrospective to learn what you did with your experiments or your action items I could take that back to my team in fact I could take that on to the next retrospective I facilitate it's a learning experience taking on that role but don't expect us from master or a member of your team to do both facilitation and participation it is impossible the facilitator of the retrospective has to follow the agenda creates the agenda make sure the time is followed so that we don't miss our breaks for lunch I'm sorry I shouldn't pick on you so here are some exercises we've talked about the timeline and I'm a firm believer in feeding people so you should always have food at your retrospectives and even if your company says no we don't have a budget for that we can't take turns take turns bringing in some it doesn't have to be elaborate or expensive it could just be some nice cookies or I don't know whatever would work for you because as we've learned in the last couple of days food is a powerful influence and there's plenty of research that shows when we share food together we are more open we're more trusting we do a better job of listening so for the price of a few cookies you can get a much more effective retrospective so I don't know what it would be here in the United States it would be the big chocolate chip cookies with the big chocolate chips so the lesson I learned and again we talked about this yesterday do not bring in healthy food do not bring in healthy food do not bring in healthy food if you show up with whole wheat fig news everybody is going to say why is this why whole wheat it has to be something that everybody likes and so it's a festive occasion sharing food it's more enjoyable and it's deep in all our cultures that having some nice food means that we're happier and we know we're happier we're more productive absolutely have food so we're going to look at the timeline here's a picture of the timeline that was given to me by a manager of a team in Houston we did this retrospective and then he said I was entering all the data from the cars and he said I thought I'd share this with you it's a little snapshot it's an Excel spreadsheet I thought this was a little too much managing but nonetheless it made him happy it just shows you what you can see from a project without reading anything on the cars what do you notice what do you notice about the timeline yeah challenges become anger and well wait how does it start yes everybody's happy yeah and also not too much hardly any cars look when this started this was 2001 and we did this retrospective in 2004 nobody remembers this is not just a two week interval this is a long project so wait until the end has the penalty nobody's going to remember what the heck happened but we do know we were happy those of us who do remember anything remember those little happy moments and we were so excited when we got going and then uh oh some challenges and often the same event can be a challenge and also a source of anger or frustration it just depends so we have a few challenges red cards uh oh more challenges oh god lots of red there to the end let's do the retrospective for another team in Houston and we finished and I always ask people to stand back a little and look at the timeline and tell me what you see so for this particular team everybody finished the timeline and they stood back and they looked and I said do you see on the timeline because it was all red every single card was anger or frustration what a project that must have been so they stood, they looked at the timeline finally one young gentleman in the front row went up he picked up a blue card and he wrote and posted it at the very end and he said thank god it's over you can see a lot from a retrospective if you just look up so I want you to think about experiments I want you to be able to start doing experiments and it might take a little bit of practice so we're going to have a look at how experiments are run did we have a question about the timeline yes we did you can always do a retrospective and look back and learn we don't have to wait till the end it would have been better for this team it would have been better for the team with all the red cards if they hadn't waited until the end well it is and what we know retrospective has a flow and if you're going to do a retrospective after three years there are other things that you would do besides the timeline if you look at Norm's book he has lots of tricks and exercises that will help you remember one of them is called the artifacts game and you ask the team to clean out the desk and bring in a real object that reminds them of something from the experience they had and they'll bring in the original requirements document they'll bring in the t-shirts they gave at the kickoff project they'll bring in a board if there's any hardware they'll bring in the first version of the prototype of the hardware they'll bring in all kinds of things around those objects and they'll say oh yeah I remember now and then they'll put a card on the timeline so there are other exercises that can go on at the same time to make sure that we do remember as much as possible is that okay? alright so here's how we do an experiment just in case so we're going to start by asking a question what can we do differently about this I wonder about that we might do a little research what are other companies doing maybe there's research to show whether or not this is a good idea and then you have a hypothesis you want to test that hypothesis in your experiment how would we test it how would we show whether or not this hypothesis is true and then do your experiment collect the data and analyze it sometimes that can be a simple process sometimes very complicated and then share the results with others so I'll give you an example so I do a lot of interaction with, I don't consult there but I'm very good friends with the CEO of Menlo Innovations have any of you heard of Menlo it is the most agile company on the planet you should go to MenloInnovations.com to see how an agile company is run and they do experiments all the time so I was talking to Rich Sheridan in Singapore we were both doing talks there and he said Linda I'll give you a good example of an experiment he said you talk a lot about standing and I do in the class that I gave a couple of days ago I said it's not just me now you can go out and buy furniture that will allow you to stand up some of that furniture will allow you to walk you can have a treadmill desk or a standing desk now the problem is those are really expensive so at Menlo someone had been reading the research at Menlo someone had said you know this could be a good idea but I don't know so there's the question what would this work for us so we said I don't want to spend a lot of money buying a treadmill desk or a standing desk let's find a way to test this hypothesis that it would be a good idea to stand up let's find a way to test that without spending a lot of money in fact when you look at your experiments we had somebody call us on that it should not involve a lot of investment and it should have a big return at Menlo they say they should be cheap how can we do a cheap experiment so the team said I know what we can do we'll go get some cardboard boxes we'll bring them in and when we feel like standing we'll just pick up our laptops we'll put the box on our desk we'll put the laptop on the box and we'll stand and we'll see how we like that how does that work for us how do we feel at the end of the day the kind of data we could collect would be do you think you are more energetic less tired do you like standing up do you feel like you're doing something instead of just sitting sitting sitting all day long it's subjective data but data nonetheless and then volunteers who wants to try the experiment so about half the team did so they had six people who brought in boxes and over the course of the next iteration the next two weeks they had boxes on the desk and they were standing up and they were trying standing sitting standing sitting and at the end they surveyed those six people and they liked it a lot and they felt more energetic and they thought maybe their back pain was going away and that maybe they felt at the end of the day they weren't so tired that they actually because they were standing felt more energetic so they said okay let's take the next step and let's buy some of that furniture and they grew it little steps little steps other teams experimented other teams did things moved over to standing desk and what Rich Sheridan who's the CEO told me said that goes on all the time we always had cheap experiments running and you should too that's what agile is all about and it's not just enough for your organizations I want to talk about your life you should be doing that in life just like I did with my alarm clock what can you do better and you should ask yourself that periodically at the end of every day at the end of every week at regular intervals you should stop and say how can I be more effective how can I tune and adjust my life so that I can be more effective he's giving me my signals now my advice they should be cheap they should be small don't try to solve all the problems in the world we call that boil the ocean no little tiny things and sometimes I see in retrospect is somebody will put a card on the timeline and all of a sudden the problem goes away magic you should do something with it that depends on your culture depends on your organization that depends on how popular your team feels how safe we heard a wonderful team today that's something that's very important how safe do you feel but realize that if you're not sharing what you've learned you're keeping that information which is valuable from the rest of your organization find a way around that we're almost out of time this is something you'll have to ask for my slides but it's from a new book by Robert Sutton and Huggy Rout it's called pre-mortem and it's a rather elaborate technique for let's not wait until the end of the project to do a post-mortem or an examination of the project let's do it before we run it and maybe that would help us decide whether we even want to do the project so it's a technique for looking at the scenarios that you would run to say you've worked well on this project even before we've done it where did things go wrong with the idea that you could either anticipate those or say, you know, there's enough here that we have to worry about that I'm not sure we should go down that path so pre-mortem or I've also heard it called future perspective look into the future and anticipate by asking the same questions and doing the same exercises we're on the timeline what happened first and then what do you think will happen six months out and what will happen before the end of the project anticipate those events he's yelling at me now okay so we know that these are important and if you need convincing one of those things you can do is you can buy a wonderful book that I wrote called fearless change you want to introduce anything once you've got these experiments and you have great ideas and you say well now let's get that idea going across the organization there are patterns for doing that in fearless change so I recommend that book as well we talked about facilitation most developers or technical people that I know know nothing about facilitation so I've collected a lot of resources many of these are free online places you can go to learn about facilitation techniques and the absolute best book is the one at the bottom Gene Tobacco's book called collaboration explaining a retrospective is a meeting and often we waste an enormous amount of time in meetings we need to do a better job of that so I recommend it highly not just for retrospectives but for facilitating any meeting at all did you have a comment? yeah oh wow that's a whole other topic but I definitely believe you can run success for retrospectives for teens you can do everything that we've done here using an enormous set of tools but one thing that's often left out retrospectives for distributed teens is the food we have food so what I do is I have the teens saying India has a recipe for some really nice Indian dish and they send it to us and we all go to the grocery store the Indian grocery store and we buy those spices and we make that dish and we are all eating it at the same time and we say oh this is good I'm not sure that I have enough of some kind of spice I wonder how your mother made it and how your wife makes it and now we feel a little closer to you and a little more understanding of what your culture is like and we learn about each other and that's a big part of retrospectives and it should be there for distributed teens as well so find a way to share in food he's really nailing that now so buy Nora's book Esther's book Patrick's book if you go to my website I've written a lot of stuff about retrospectives you can download all that for free and there is a yahoo group retrospectives at yahoo.com and the people who hang out there are the people who have written the books and the people who are really interested in retrospectives and retrospective facilitation and it's a great place to go for suggestions because you'll hear from people all over the world who mostly have either been through what you're going through or would love to discuss it with you have advice for you suggestions for you so go hang out there you'll learn a lot and I hope that maybe you've gotten a few ideas you're going to start trying some experiments will you try some experiments? good okay so if you will do that tell me how that works for you tell me what interesting and exciting experiments you're doing so that I can share those with others because sometimes people get stuck so I always have to talk about menlo innovations but I would love to talk about what you're doing so thank you for staying with me and skipping your break yeah it's been fun thank you so much