Hi and welcome everybody and to our afternoon seminar and I'd like to begin by acknowledging the traditional owners of the lands on which we meet the Ngunnawal people and pay my respects to the Elders past, present and emerging. My name is Ruth Nichols and I'm one of the members of the AES Committee, local Canberra Committee and today we're really pleased to be able to welcome one of our committee members as Scott Bailey to give us presentation on why programs fail. Before we get into the presentation this afternoon, just a couple of housekeeping things. So first of all, I think most of you are on mute already and if you're not, if you could please put yourself on mute. We will have opportunities to have some discussion as we go through the seminar today and that'll be a point at which you can come off mute. You can also, if you'd like to put some questions in chat as we go along. And when we get to those points of having our discussion, we'll be having a look at what's come up in the chat box and use that as an opportunity to support our learning. So the other thing I wanted to let you all know is that our session is being recorded today. So that's just something to keep in mind. The AES has a YouTube channel where we publish recordings and seminars. So I think without further adieu, I'd like to welcome Scott Bailey for his presentation. Thank you, Scott. Thank you, Ruth. Hello, everyone. I'm doing double duty here trying to admit people into the session while I'm paying attention to what I'm going to say. So anyway, we'll work it out. I want to talk today about the symptoms and causes of underperformance and government programs and what that might mean. And I'm waiting for this PowerPoint slide to move forward. There we go. So I want to have a little bit of discussion about examples of failure and I'm going to share some of mine and I'm going to ask if you'll be willing to share some of yours. I'm going to talk about the common symptoms of program underperformance, what they might mean. What the main causes of underperformance might be in government programs. And the reasons why an impact evaluation might conclude that a program is ineffective. And then if we have any time left, I'll talk a bit about key challenges for evaluators and assessing social programs. And I'm just going to pause for a second and let some people in. Thanks for doing that, Scott. Okay. This is very slow, this internet. So here's some examples from my own experience about different cases of underperformance. The exact details are through the agencies to protect the guilty, so to speak. But these are all real world examples. Myself and some others did a study on mental health services for people in crisis. And we pulled 900 patient files. And each of those files was meant to have a plan for the patient, the client. And we only found about 40% of them did have a plan that was required. Now the department would say, well, everyone had a plan, we just didn't put them on the file. That may or may not have been true. I never really knew. But one thing was for certain, their quality control certainly fell down. That's without doubt. Another piece I did at work once somewhere else was looking at services to child protection allegations. And these would get prioritized for high, medium and low. What we found was over a 12 month period, only 30% of the high priority allegations got a response within the department's own timeframes. So that's a lot of kids who didn't get investigated or notionally at high risk. Another example of mine was I looked at a national grants program. As part of this grants process. A community agency would get a grant, they'd spend the money and then they were supposed to have an audit report report back to the funder. And then the grant would be acquitted that is to say signed off what we found over a three year period, only about 50% of these grants were actually ever acquitted. And this was a program with hundreds of millions of dollars every year. Those are sort of illustrative examples from my own experience. I was wondering though if people would be willing to share some examples from their own experience of what looked like underperformance in a government program. I'd invite you not to identify who the agency or the clients were but just give a rough flavor of what was the nature of this underperformance. Would anyone like to offer an example or two. I know it can be tricky sometimes to think of examples, but I can see a hand from Harry. So go for it, Harry. Good day, Scott and Ruth, can you hear me? Yes, we can hear you. Yep. So look, just to get things kicked off, I can talk about one that we've published. So we're not too embarrassed about it. We worked on an app to support university students who might get discouraged early in their studies to try to stick it out at university. It was something we did with funding from Department of Social Service as part of their try test and learn fund. And we spent a lot of time on developing the app. Beta the behavioral economics team based in Prime Minister and Cabinet spent a lot of time developing the app thinking a lot about what would help students. We did a lot of user testing. And we did a pilot roll out into the eventual roll out and in terms of student retention, which is our sort of the main thing we're looking at. We didn't see any improvement. And we also wanted to have a look at grades and we couldn't see much improvement there either. And it looked like, although we'd spent a lot of effort in trying to make the app sticky so that people would keep using it. After that initially signed up that a lot of the problem was that there was a big drop off in usage after the first week or two of downloading the app. So there was a much prospective impacting on retention or results if they if they weren't using it much. So that's, but that hurts because there was a lot of time and effort and imagination that was poured into that for seemingly little return. That's our example. Thanks. Oh, thank you, Harry. That's a great example. Any of us be willing to share one of theirs. You're welcome to put your hand up if you want to like Harry did. I know Scott, in my experience doing independent evaluations, I've certainly seen some programs being implemented in a very short period of time and expecting to see impact results, you know, within like, you know, less than a year. And while, you know, the program still really just trying to get itself organized and yeah, that's been that's been an interesting, you know, phenomenon I've seen on a couple of occasions. Kim. Yes, hi. Not quite sure how my, my camera is going but hello. Kim Gray here. I was musing over an example that I'm not going to name and I think the big challenge with this one was that was a big difference between the programs expectations, the settings and the way it was supposed to work. What it actually was like, and what the participants felt was of value to them. So their experience of the program was not very satisfying. And therefore, data that could have been used to talk about its performance wasn't as informative as it was thought to be so I put that down to a very big difference between the way the policy designers thought the program worked and the way the participants experienced the program. Okay, so we've heard some examples about implementation about design, but strategies not working. Do you have your hand up Julie. Yeah, I can't find the little reaction buttons on using my little hand. I've got one that you might even remember it's affectionately known as the roads program, and it was about improving governance. So it was, you know, it's a real issue of how you define success, but basically, or failure, but basically, you know, you're putting a simple solution to a very complex problem. And then it's likely going to fail and this one I have in mind was a sort of an anti corruption process and I can't remember the details of the middle bit but the, there was, if we build roads, we will improve governance and reduce corruption. And that was not success, I think that would be a failure. Does that ring any bells, Scott. I'm laughing because that's one of my favorite examples. It's going to work. What a wacky idea. Hey, that's my best one. Excellent. All right, let's move on a little bit. If the IT will cooperate. Have lots of IT problems here. I'm going to talk a bit about symptoms. Some common symptoms include things like quality problems, client complaints, staff turnover, insufficient outputs, wastages, inefficiencies. Ineffective programs. Programs are not responsive to their client group. Limited external political support for the program. Adverse publicity and adequate reporting, excessive waiting times, criticize criticisms from from external watchdogs like parliamentary committees, or the auditor general's office. All series of things that can happen. And these are important because without the perception that there's a problem performance improvement has no starting point. That is to say program managers and public officials need to feel some kind of performance pain. The question is, where does it hurt and performance pain in public programs. Can I mention several interrelated sources, including inadequate production of goods and services program just doesn't produce enough. The programs good and goods and services are of insufficient quality. Many resources are being consumed by the program. The program is ineffective. It fails to fulfill its intended purposes. For example, unemployment for youth isn't being reduced. The issue of client dissatisfaction and complaints. And there may be an image of that of course is staff dissatisfaction and turnover conflicts with other related organizations coordination problems. That's quite common between federal state agencies. There's no adequate inadequate adaption or innovation in the program a failure to respond to the changing client X needs or external circumstances. Performance reporting can be a problem. I mentioned lack of political support and burst publicity criticism from watchdog agencies. But all these problems are actually symptoms of underlying causes. So figuring out where it hurts sets the scene for performance improvement, perhaps in three ways. It determines whether there is a perceived need or change to improve performance. It helps to identify the sources of performance pain. And public sector managers also benefit from understanding who cares about a performance problem, and who's willing and able to do something about it. These potential underlying causes include the program's mandate. That is to say the mandate could be unclear the program lacks authority. There was a confusion of roles across agencies. It's the issue of strategy. That is to say the program's theory of change is faulty or the assumptions are untenable. The strategy does not fit the program's environment. It's another common one. Perhaps structure that is to say the program structure does not fit its environment or its strategy this inconsistencies attention. One of my favorites is the topic of performance leadership. That is to say the leadership style is incongruent with the program, whether it's inadequate governance accountability or net a limited focus on using performance feedback to drive continuous improvement. Culture can be an issue if the culture and incentives do not support a focus on continuous improvement and achieving results. The organization systems can be an issue in terms of policies systems and processes failing to support effective program management and service delivery. And finally the topic of resources. If the level of resources financial physical people technology and operational capacity is inappropriate for the program's design and systems. So what I'm trying to do is argue that we have some very common symptoms and most of us is bumped into one time or another. But these symptoms are actually symptoms of underlying causes that are more fundamental drivers of the problem. So I'd be interested in hearing from yourselves at this point how does my list of symptoms and underlying causes fit with your own experience. Does this make sense. Is it logical. Is it something you've observed in your employment. What's your reaction to this. Scott it's Peter graves here if I can't quite put up my hand if you can hear me. Yes Peter thank you. I'd like to support you but literally on the lack of political support. Of why government programs do fail and also their reforms. And some of my comments are quite relevant to what's going on with the current government bringing back program evaluation. Because I always thought about climate change that Prime Minister thought was the greatest challenge of our time. We had a climate change department formed in 2000 and whatever seven.  So having senior leadership that is actually continuous is extremely important. And in that regard. Some of my recent research about the management reform of managing for results. It never went on long enough. While it went on for 13 years. It stopped because the support the top changed. And it is now being brought back. So there is a time there is a fact of extended time in a lot of what you're also talking about. So Peter this lack of political supporters are. Perhaps that relates in some way to changing governments from one political party to another. No, I think it also involves changes of ministers. And I think of Martin Bolz when he was head of immigration trying to bring in an evaluation culture there. And he brought in Wendy Southern at a very senior deputy secretary level. He moved over to health and brought Wendy with him to bring in a similar evaluation culture. And when he had a falling out with the health minister of the time and left. So did all of those attempts at bringing in an evaluation culture in health fell over because there wasn't one to begin with. So yes unfortunately very senior people literally have too much influence on the value of APS programs. I've certainly observed an emphasis on evaluation wax and wane in the Commonwealth over the years. And in the shorter term depending upon the emphasis of the leadership team of a particular department. Without mentioning particular departments I can think of 304 that have either focused on evaluation secretary chant and then it went off the boil or the exact opposite has happened. And it sometimes disturbs me that I focus on managing for results depends upon the personalities at the top, and that it's not more systemically grounded. And our systems and practices that's just my issue a little bit. Anyone else like to offer some comments on my list of symptoms and causes and their own experience. I just want to let you know we've got a comment from Lou who doesn't have a microphone. And Lou says we've experienced a range of these with the programs we oversee a lack of consideration of evaluation in the establishment phase has been very problematic for us. I can think of a number of programs that were established run for a number of years funded. No baseline data collected no M&E frameworks put in place. Then I can think of some that were 15 years into the running them like oh maybe we should do an evaluation of this to see how we're going. It intrigues me how things can go on for that long. I'm wondering about under what circumstances is a problem amenable to a solution that's called a program so programmatic solution. If you like that, because maybe it's not it's unrealistic to think that programs can address or melee melee all sorts of social circumstances that are less than desirable. And so programs overreach what they're really capable of doing in the real world where in government, the real world is that there's an election cycle but there's ministers come and go that they each have their favorite priorities that secretaries come and go. They each have their own biases and prejudice because that's how people work. What are the limits to what a programmatic solutions. I'm attracted to your idea in a couple of different ways on the one hand, I think it's fair to say we've got some great examples of important policy success. The reduction of teenage smoking in Australia, I think that's nothing short of amazing what's been accomplished. But then I could probably try and think of another two couple of examples where, alright, the Australian aid program in Papua New Guinea with investments designed to reduce corruption. And G that's a real stretch we have small sums of money. What's our political influence what's our understanding of the country what's our linkage with key decision makers, our ability to influence incentives and pop in again officials that that whole area makes me think that's a huge stretch on our part. I'd be interested in other people's views. Scott it's Peter Graves again just supporting you on that PNG comment. He was in a place to know told me that the PNG people don't work out the results of their own programs when governments change, because the new government doesn't have an incentive to know the results of what the previous government achieved. And it's all a different course of events. I'm thinking closer to home I think the topic of youth unemployment has been a foreign in government side for. Oh Lord, at least since the 1980s that I can think that far back. What other people think Kathy. I used to work in jobs Victoria in the Victorian government and it's quite interesting how you pointed out you find employment was like a challenge as well and I remember us trying to set our goals at the initial program design as well too. And it, you can set those goals but it's also like there's other factors at my core that might be other social. It comes to it's kind of similar to like that wicked policy problem, where you might define the pro like the problem and be able to identify the goals in the initial program phase, but when you actually work and have to implement, you're dealing with other complex challenges as well. And so that's where evaluation can be quite challenging when you're working towards implementing that program but then other programs like other problems arise as well. Thank you. That's a good point. Other comments or observations. Kim here. I'm using over high Scott. Yes, I'm using over the slide you have up with symptoms and causes. And, and I like the way you've talked about needing to feel pain need to actually have a reason to explore something. So some signs that there's a problem. But they're not the cause and then you've got a smaller list of causes. So I just wondered what you'd say about the relationship between symptoms and causes. Do you see any relationship or is this a much more complex question. Yeah. I've mowed that one over and I think it's fantastically complicated and probably context dependent. For example, the particular symptom could be related to one or more different causes, pick something stuff dissatisfaction turnover. Maybe that's because they're overworked. That is to say there's they have inadequate resources so maybe that's why they're dissatisfied and they turn over. Maybe they feel like they don't have the mandate authority to do their work and so they're being held accountable for things outside of their control and that's the source of their dissatisfaction and turn over. Or maybe they're unhappy with the leadership of the agency and that's the source of their dissatisfaction turnover. So I can't imagine there's a direct necessarily one to one correlation between all the symptom is always related to these causes. Perhaps more in a diagnostic said well I've got these cluster of symptoms and now I'm going to have to try and work backwards to try and identify with the underlying causes. But I'd be interested in your thoughts but my initial reaction is not going to be a straight one to one relationship. Yes, I'm sure I'm sure that's the case sorry I was just responding briefly. I'm sure that's the case Scott and. And I would sort of also argue that you need to explore more than the symptoms that you've been able to observe. I think there's a lot more to unpacking the causal drivers than what you can necessarily observe. There may be things that you have you need to do more research into something to understand what could be going on. I think looking for symptoms is a great way of thinking about what you're doing when you're linking monitoring data and monitoring indicators through to unpacking an evaluative question. I thought that was quite a nice way to to think about what you're offering here that finding out where it hurts is a question more like monitoring. Does it hurt anywhere let's just check in getting it doesn't mean that it's necessarily going well you may not just you just may not have enough information yet. Yeah, and I had the poor judgment wants to apply some diagnostic work to symptoms and organization I was working in that I won't name, but we had some real quality problems. And so I did some five wise analysis on their quality problem and I also did a problem tree analysis, and that raised some very sensitive issues very quickly. And it wasn't received real well. In senior management sort of the SES band two level and upwards. And I guess I was pointing at things that were a incredibly problematic be not necessarily amenable to easy problem solving and see didn't help particular individuals and their reputations. And so, whilst I thought my work was reasonably insightful it didn't go anywhere because politically there was no support for it. And so that was one of my comments a moment ago it's one thing to identify symptoms but another is who's interested in the symptom who's willing to work on it. And I think those are perhaps the really important diagnostic questions rather than a more mechanical symptoms might be a reflection of that cause that's just an initial reaction by me. I just wanted to mention that Raul had posted a couple of observations in the chat box as well. I've seen those. He raises the idea about the brain drain to the public sector impairing public sector capability on various fronts. Yes, it's interesting that our current government has spoken about the need to rebuild public sector capacity. And that was something raised in the 30 review of 2018. And then the Commission of audit and I forgot what year that was 2012 maybe raised this issue about public sector capacity. I says, I agree that it's an issue but I honestly don't know at least the Commonwealth level what the government's going to do about that in such a tight budget environment. One of the things governments are conscious of is the number of public sector FTE positions and the associated salary budget there's actually incentives for common governments to employ consultants, because it treats they're treated differently on the balance sheet. And so it's been an issue for a number of governments they can say well the total number of public servants has been flat in the Commonwealth for the last number of years and that's almost a point of pride while they spend more money on consultants. But of course at the same time the capacity has been and capacities being outsourced as well as corporate knowledge is that being outsourced. So yeah rule makes a great point but I honestly don't know what the solution to that is, especially in a really tight budget environment. But having said that. Maybe I'll contradict myself this was the argument that Bob hot Paul Keating used in 1986, when the budget was a bit tight. And they said you know what, we've got to do better at spending our money where it counts and they pushed evaluation policies and made it mandatory and Paul Keating was trying to identify programs that worked and didn't work. So either had to be fixed or they'd be canceled or wound back. And that way he could generate additional money to spend on other programs that work better, which makes a great sense and personally I'm really supportive of that was only one problem though. They got the incentives a bit messed up. And so programs realized fairly quickly, if they did an impact evaluation of their own program and they criticize that next time around they got a budget cut. And there were one and a half rounds of that before every department did a self evaluation said hey we're great everything's good. So the incentives got distorted and other countries or chilies an example. Those big major impact evaluations aren't under their departments on control the equivalent of Prime Minister and cabinet leads them. So they let line agencies do implementation studies but the big strategic policy high disability impact evaluations are actually managed by Prime Minister's office usually done by consultants. Other comments or questions. Scott if I could come in on that again bringing in an international example that is ahead of Australia and it is in America. Through a piece of law actually signed by Donald Trump. The foundations for evidence based policy act black letter law in America requires every agency to have a chief evaluation officer and to have an annual evaluation plan. That is a good start. What happens to the reports afterwards is a separate issue. But in America it is now mandatory obligatory and legally required to do program evaluation. Yes, the AES input into the thought review recommended the that all departments have a chief evaluation officer chief performance officer equivalent to finance and HR trying to locate responsibility for that function. Not sure there's a much support for that although we do have the evaluator general argument that's been topical for the last couple of years here but I'm not fully sure how that's going to play out in our space just yet. Julie's there, Julie's there. Hi Scott, I'm just wondering if where you would include sort of like a lack of stakeholder involvement in in the framing of the, the situation of interest, if that would be an early symptom, if they weren't involved in defining in framing the, the situation of interest and defining, you know what success and failure looks like. Oh, I think you may be a pre symptom or do you have another word for that. Well I think that's certainly an issue whether it's a symptom or a cause I'm not sure off the top of my head but I think you raise a brilliant issue. Let me think of it's old school but I've always like Joe Holy's five steps to program management. He was the chief evaluator in the federal US Health Department some years ago, and he'd say step one. Engage with your stakeholders and trying to develop a consensus about what the problem is, and what needs to be achieved. Step to design programs that potentially might work to address this problem. Step three, you implement this program, while you evaluate it from a whole range of different value perspectives. Step four was to use your performance feedback to drive continuous improvement. And step five was to communicate back to your stakeholders your funders government in the public about what you've achieved, and that was his idea of what the continuous improvement cycle was certainly in my work in government. So examples of step one develop a consensus stakeholders about what the problem is and what a good outcome look like that always would come back to bite you if you don't get that right up front and I understand why sometimes we don't because there's pressure to act and I get that and spend money and be seen to be doing things for political reasons. But that is a problem, never goes away if you don't sort it out, eventually whether it's five 1020 years, it will come back to you again. And I see Kim holding up her hand. Yes, Kim. I was thinking maybe this idea of a lack of stakeholders involved at the start it's something that we see in working indigenous affairs where we're thinking about other right people involved. We think a lot harder about that now that the closing the partnership is requiring more work on engagement and partnership. So I would, I would argue maybe that it's a risk indicator, rather than a symptom. So maybe there's a set of risk indicators that could be correlated with signs of potential program failure. I hadn't thought about that like that before but I kind of like your idea I think that's there's something good in that. Yes, Julie. I just found my hand to raise so thanks. But I was going to say that's an interesting idea Kim risk indicators. Signs that something's wrong. And I think that might might kind of help us decide what is amenable to a program solution and what needs much more systemic change. And I think one of the issues that might might come up as program failures, we're always going to be fight failure as a program, because they're long term and as you said Scott they're always going to pop up later on we might sign this kind of squash them down. But if the systemic conditions that are actually holding them in place are still there, then nothing really is fundamentally going to change. Other comments observations, experiences you're willing to share. Yeah, I was just going to say Scott there was a there was a comment from Lou I just want to acknowledge Lou would put in there about the section being established through an MPP specifically. And that was to enhance governance program governance conduct evaluations and develop internal capability. Interesting trifecta there. It does intrigue me that not all new policy proposals will have a budget allocation for evaluation for example.  Yes Julie. Well, Scott you're assuming that that that's the intention to have success and change something. The intention might be just to be seen to be doing something. Don't pretend to be naive here Scott. Nothing because you're quite right and what's his name. Duncan Fraser wrote about a series of articles in the evaluation news and comments back in the old Lord. It was fairly 1990s I think it was, and he argued the programs were different types, and one type was a I'm here to solve a problem type. There was another one that was, I'm just here to give the illusion that we're doing something because it's basically uncontrollable. You had a couple more types and I just can't remember what they are off the top of my head but yeah somewhere functional problem solving programs and others were more political and symbolic in their intention so yeah you make a good point. Another one I just thought of it could be that I'm just biting time until I come up with the solution you know. Oh yeah, a whole kind of a mix of those two that you suggested. I'm sure that would be a really fun thing to do to write that list. Oh, and another one of his I've just remembered is to divert resources towards your political supporters. It wasn't that we're actually trying to solve very much but we're diverting grant funds towards an area of the economy where we like those people so we'll make sure that they get a bit of Jerry. Never seen that one in practice now. Comments. If not, I'll move on it willing or move on. I'd like to suggest this is a related topic but I'd like to suggest. If you employ someone to do an impact evaluation, there are four and only four explanations why impact evaluation might conclude that the program is ineffective. And I mean it to my life thinking there's only four. The first one is strategy. If you're teaching for I'll use a silly example but you'll get the point if you're teaching farmers to bin to burn incense to improve their crop yields and thereby raise their income. What doesn't really matter how much incense nor how well they burn it it's just not going to work. So sometimes the fundamental strategy is just wrong. And it doesn't matter how much you do it or how well you carry it up is just not going to work. For one reason your impact evaluation can find your program doesn't work. The second one is your strategy is fine, but you're not actually able to implement it with an integrity that is to say the intended services aren't getting to the right people. There was a really amusing evaluation I read some years ago in the US. And these people were supposed to go door to door in urban inner city urban areas in Chicago, and teach people how to maintain their homes basically repair them. The program was funded. It was evaluated. It ran for 10 years. There was only one problem. The problem, the program as designed was never implemented. They never actually did the things the program said they were going to do. The program staff decided the original plan was silly. So they did other things instead. Rather than teach home handyman skills to these inner city residents on how to fix their doors repair window. They went into an advocacy mode. And so they taught these people how to be community activists and lobby government officials. In that sense, I'll come back to you in a sec Julie. So in that sense the program is designed was never implemented, although it did get evaluated interestingly enough. The program and its design and its implementation are quite reasonable, but something in its external environment changed. So what used to work doesn't work anymore. An example I've seen an international development is where you have a partner you're partnering with someone and overseas government to do something in their country. They're highly supportive and Australia's doing certain things and the overseas government's doing complimentary things, but then their priorities change and they stop doing their part of the agreement. So what used to work doesn't work anymore. And the fourth reason on my list is that actually the program works just fine. It's the evaluation itself that it's faulting. Here about someone saying an impact evaluation found that the program is ineffective. I mean, I'm always looking for these as potential reasons the strategies want implementation doesn't work. Something in the environment has changed, or the evaluation itself is wrong. We never like to admit that but it's a possibility. Julie, you had your hand up a moment ago. Yes, Scott, just a little comment. I like you for your four ideas around this but the example that you gave. I thought around people door knocking to help people maintain their homes. I thought that was so blatantly a theory failure or a strategy failure that everybody working on it could see it. And that's why they implemented something else. Well, before I just my bias perhaps before I'd argue about strategy hadn't worked. I'd want to know the program was implemented first. It might be sauce up front but I'd still before I declared a clear strategy failure I'd want to know something about implementation, but I could think of other examples some. Immunization. We know immunization works the science of that as well established we don't have to test that all we have to figure be clear about is the implementation. I've seen examples of some overseas countries that I want to name where Australia was delivering drugs for immunizations to community health centers that were never ever implemented that were never used on children. In that sense the strategy of immunization was okay but implementation there was a delivery failure. Maybe that's a better example than my silly incense burning one. What has been your experience everyone I'd be interested in examples if you think this makes sense or maybe I'm missing something. Got we have an example from Lillian in the chat box about timing. Knowing when to conduct the evaluation too early and it may not show much in terms of outcomes that certainly my experience to Lillian and understanding the complexities of evaluation work, which sounds like it speaks a bit to methods appropriateness to. Yeah, I like that when it's a great. It's a really good one something I like that. I do a better consulting work with Commonwealth Health. And one thing they do with their program logic models they talk about short term outcomes medium term outcomes long term and impact. They make the program areas put timeframes on those. So, program area X in what time frame after the delivery of your outputs are you expecting to see the short term outcomes is it one to three years five to seven whatever the same again for the medium terms and the same again for the long terms. And that can be really helpful particularly for two reasons that forces you to think about what the trajectory of outcomes is. Okay, if you're an evaluator and you see this, it's a way of flagging what expectations are at different points of time so thank you Lillian I think that's a grand idea. Any other comments or observations. Kim here. I just back up that comment. Noting that a lot of the time you see program logical theory of change saying that short term is one year and medium term is up to three years and long term is longer. I really like the way you flag that areas would need to work out what is their expectation, which I'm not sure people can really do very accurately, frankly but I guess that's better than an arbitrary approach. Yeah, I think you make a great point. I'm not sure we often really do know what the likely trajectory is. But yeah, it's a good thing to wrestle with and facilitate a good discussion about that. Over in the chat, Hannah's made some contributions. Hannah says I work in the international environment and often see stakeholders in the countries we work in that have variable priorities. These might be determined by their government of the day or other external factors. It can make it interesting managing expectations both when designing and evaluating programs to add to that a time stakeholders may not have the high level insights to decide what is a priority and why and this can impact what they tell us. Yes, I agree with all of that. At the same time, programs are political creations. And ideally, they should be reflecting just to what extent it's possible. A bit of a consensus amongst that political process and the stakeholders I mean sometimes governments just do things because they have to urgently and they don't have time responding to COVID could be an example, but it's not always like that. And we certainly have programs that have been running for decades. And maybe the community hasn't really had much opportunity to have a say about this program and how it's being implemented or what it's trying to achieve. And maybe they have but we just have to have the same discussion again. Lots of friends who are teachers. I don't have a background in teaching but I've got friends who do and the role of public sector teachers comes up every five years or something and it has done forever and I suppose it will continue to do so. The extent to which the public education system has responsibility for the narrow instructional aspect of students versus more broader life skills. There's some little on issues like sexuality and this sort of thing. So, where their demarcation and responsibilities lie in the public education system seems to be a topic that's open for discussion on every five years. This was my impression. All right, I might move on a little bit. This is my home stretch here. I'd like to make some comments about the key challenge challenges for evaluators seeking to assess social programs. When I got a lot of this from the former boss of mine, Pam Williams, back in that around 2000 when we worked together, and also from some of my own experiences in a range of different sectors, I think there are some common challenges. One of these being, how are you going to decide on the evaluations area focus? Options include the changing nature of government policies over time, funding levels, desired outcomes, strategies in a particular sectoral area. They can change massively. So what are you going to focus on? You want to assess the volume of outputs delivered, service coverage, adequacy of access, things like waiting list, you can do that. Service quality, sure. This is Pam's impact on stakeholders where it's the immediate client, the family, the broader community, both intended and unintended effects, interactions with other programs. These are all options. Personally, I think when I include myself in this, I think we've been very, we've underdone the area of unintended effects. I really don't think we've given that great justice. Interestingly, you know, I was working with a fellow in the ACT government, one of Raul's colleagues, and I won't name the program, but this person developed a negative program theory. That is to say he had a positive theory how the program would work based on various assumptions. And then he went, what if every one of my assumptions didn't hold any exact opposite outcome happened? There was a negative program theory. I've never seen one before. I thought of a shared genius, though, myself. It was a horror story if it all happened, but it was a great thing that he was aware of this. Something else that comes up a lot is the equity of funding allocations across geographic regions or across different client groups. And then performance monitoring, evaluation and accountability system. These are all topics that an evaluation could potentially focus on. And there's no immediate right or wrong to it. But by Jib, there's a whole range of issues there more than you could possibly help to cover and run evaluation, which means you can have to make choices. That's really the essence of my message. And then the systemic challenges. But I see this a lot. The department hasn't developed a program logic model or theory of intervention about how a program is supposed to work. And any theories of action are really implicit rather than explicit. If you interview people that can explain it to you and they'll have a rationale but it's very much in one or two people's minds. And it's never really been codified, which means it hasn't ever really been tested either. A common systemic challenge I see a lot is that the target group cannot be serviced at a great standards within the current resourcing envelope. And that means the department creates waiting lists, cues. They unofficially will raise the eligibility requirements or narrow the intended target group. And that's just their way of trying to manage with not enough money to service everyone in need. And all this issue to a free service of having infinite to demand, often not having a clear target to measure against and demand management, which is sort of government. Weasel words for demand reduction becomes management's focus not meeting demand that becomes the new unofficial goal. And sometimes we don't have clear measure of successful outcomes or appropriate service division, like that is to say we have unclear criterion standards for service delivery. So you as an evaluator do you want to open that can of words even potentially suggest some or try and work with your stakeholders to try and develop criteria and then evaluate against that you could do. We also find that workforce issues are common staff shortages the need for training high turnover low morale. If you ever worked in a program area where 25% of the staff turned over every year I have that means in four years 100% of your staff have turned over. And that has huge implications for learning corporate memory. You can often find to that you can find large amounts of administrative program data collected, but the data is not very good quality limited ad hoc analysis undertaken, what the data actually means the client files can be incomplete. That's not uncommon. And decentralized service provision coupled with unclear roles and responsibilities between the central office and regional office that makes some really difficult evaluation work. More and more will see cont service delivery being contracted out with met perhaps limited contract and quality oversight. And all that wasn't enough, particularly in areas like health and education, each level of government has limited control but overlapping roles. Federal government has policy and funding state has responsibility for ensuring good outcomes. At the same time you've got alternative service providers are also operating in the sector. What you find in some of these national sectoral areas like health education is a government, government's role often becomes not efficient and effective delivery of a service, but one of plugging gaps and available services without creating perverse incentives. That's often what happens in practice. That's some interesting implications for what that means for doing an evaluation. And if all that wasn't enough I'd raise the issue of methodological problems. Limitations and available data it systems often not user friendly, you can have privacy ethical considerations trying to identify all the relevant outcomes for a client group. Evaluate is often how to meet for independent expert advice and substantive matters. For example. And that can raise issues with the value as being accused of second guessing clinical judgments I've been accused of by judges of trying to interfere in legal matters. When I was doing an assessment of their management of deceased estates. I was accused by doctors and emergency departments in one state of trying to second guess their clinical judgments in their services to suicidal youth. And I got around that by Australia back then had clinical standards for the treatment and emergency departments of suicidal youth. So I had the two doctors who wrote the standards and got them to join my evaluation team. And then I paid them to go to 11 hospitals with me, and they pulled patient files and system against the standards they themselves wrote.  So there are ways of working around this issue of you don't have the expertise or you're not one of us. There's also this issue of decentralized service provision. The practical consequence of that means very time consuming and expensive evaluations with large samples. Do you want to represent it of samples from 26 different sites around the country. I've any valuations like that in the health system that gets quite expensive fairly quickly. Do you want to pick five of the 26 sites and just study them, but then how are you going to generalize from the five back to the 26, or, or some of the sites sort of typical you could try and do a typical case qualitative samples Michael pattern would say. The other issue I would raise is that when service delivery has been contracted out and that's increasingly common evaluators need the authority to examine the contracted operations that is to say the authority, the ability to follow the money to see where all that goes. I'm conscious I've been talking a lot, and I'd like to stop there and. find a whole series of challenges under three headings where evaluators looking to assess social programs and please, how to invite you for any comments or suggestions or feel free to disagree or point out something I might have missed. Thank you. And Scott, I just noticed that we're just coming up to the end of our time so I'm not sure if people might need to go soon but does anyone want to ask any questions. I've Scott before we wrap up. We'll make any comments. It's great to say thank you Scott for such real world comments on the practice of the public service in Australia. Sorry Harry I think I talked over you. No I was just saying thanks to Scott. Enjoy the seminar. Thanks. Thanks Scott. Everyone in the Canberra committee would like to thank you very much for presenting for us today. It was such a such an interesting seminar. I really enjoyed being able to hear about the examples, especially the, the incense sticks I'll take that one. I'll take that as a note for program design. Thank you everyone as well for coming and joining us today and we look forward to seeing you again.


Life is good all this month at Appalachian Wireless. Get the LG G5 for just one penny with sign-up or renewal of a two-year service agreement. That's almost $100 off the regular price. Better service, bigger savings. That's today's Appalachian Wireless. Programming on WMT-Wattsburg is made possible in part by the National Endowment for the Arts. For nearly half a century, Appleshop, nestled in downtown Wattsburg in Letcher County, has been a cultural center of the Appalachian region. Filmmaking and preservation, culture and arts education, broadcasting through WMMTFM, they're all part of Appalshop's work in Central Appalachia, according to Appalshop's Ada Smith. We would not be able to do that without the National Endowment for the Arts and Humanities. They provide an amazing amount of seed and matching funding. And now, just short of its 50th birthday, Appalshop stands to be one of the victims of President Donald Trump's proposed federal budget. Appalshop would not be here if it wasn't for federal money. Trump's budget calls for the elimination of the National Endowment for the Arts, National Endowment for the Humanities, and the Corporation for Public Broadcasting, all of which contribute to Appalshop's operating budget. The elimination of those programs would be devastating to the non-profit Appalshop and other arts-based organizations that receive federal funding. I think when I think about the NEA and the NEH being under attack, I think about why don't they care about our culture? Why don't they care about our music? Why don't they care about the arts and crafts we make here? And to me it is very anti-community for the government to do, and I believe that arts and humanities projects are super valuable for our country and something that we should be investing in. To zero out the only arts and humanities funding that this country provides seems very short-sighted to me. Last year, Appalshop received about $270,000 in funding from the National Endowments for the Arts and Humanities, and the Corporation for Public Broadcasting, which is also on the chopping block in Trump's proposed budget. Smith said attacks on the arts, humanities, and public broadcasting by way of federal and state budgets are nothing new. When I think about the threats, what I think about is what a waste of talent and contributions of the people here to not be able to access funds like these. The fact that we have amazing ideas, smart people, good communities that want to put these resources to use and submit very competitive proposals to do so, and then those opportunities be taken away. And now, Appalshop waits, reporting in Weizberg for EKB News. I'm Chris Anderson.


Hi, I'm Ty Davis today. We're gonna talk get lab value stream management and the security insights that you have available was security insights Similarly to how you know a common theme here is that you're gonna see that we have dashboards that Are available at a project level and at a group level So it's giving us specific information around projects or it's giving us specific information around a group which can be a team or a vertical that's part of that and What we have here in the security compliance dashboard is it gives us the ability to see a group view of All the security vulnerabilities that make up all the projects within that group So you can see how many vulnerabilities are have been introduced over time. There's a report card of of Security statuses within a project and you can also sort by severity as well as report type and With these you can click into them see specific variables if I need to go in and Get more information around that I could do that I can dismiss the vulnerability. I can create an issue if I'm maybe I'm security Manager and I want to you know, if there's something that's a little more secure. I'm not secure more severe than a medium There's a critical. I can click in that. Oh, no critical. We're doing all right All right medium it is So I can Dive deeper into those vulnerabilities and address those with my team here, you know In security status, it's gonna break that down by the projects and so I can see which teams have a critical, you know Here this team is a concern compared to everyone else. They have 288 If we go at a more granular level Within a project what we can also see is there's a couple different things around security compliance the project security dashboard it's a good place to get an overview of the vulnerabilities that are in that project and It's similar to what you saw at the group view, but I can be more specific in that group a Dependency list allows you to see your projects dependencies key details about them including known vulnerabilities. This information It's sometimes referred to software bill of materials, but it's going to be giving us detailed information around dependencies Alliance license compliance It's gonna give you an overview of just the Alliance and if they follow the compliance of that project Last one I'll briefly mention and I don't I don't have data because this is So new and fresh on their most recent release and this is kind of a it's a threat monitoring So you can see data on both total and block traffic Which allows you to more easily determine how you can configure a tune or evaluate that web application Firewall Be sure to check out our next videos and continue our conversation


Are you willing to share your doorknock script still today? Yeah, for sure. So I work every type of lead. I am involved in advanced markets, mortgage protection, filing expense, Medicare supplements, all of it. So which doorknock script do you want? Oh, let's do the final expense one. Okay, I gotta come to the door. All right, I'm opening. Hello, who are you? Cody? Yes. Hey, Cody. It's Leslie. I've been trying to get in touch with you. You sent in this information and I know how important it was for you. I don't know when you got it, but due to COVID, they've just now got me out here. Do you have a few minutes? Yeah. Okay, great. Perfect. Do you want me to take off my shoes or can I keep them on? You can keep them on. Perfect. And then from there, I just go straight to, I always sit right at the table. And then at the table, I don't go. So why'd you fill this out? Yeah, I love that. No, it's just, so I'm assuming you sent this in because you're worried about your burial.  You know what I love? Here's the five things I noticed, by the way. Absolutely incredible. Make sure you guys give Lesley some love and comments to share that live on video. Like you can tell the chick has got it. Five things I noticed. Number one, you said their name. No, number two, you were. First name. Yeah. First name only. Oh, that's cute because they'll correct them. It's Mrs. Adams. Okay, Mrs. Adams. Yeah. Exactly. Number two, you were loud. You didn't like get quieter and lose confidence. You had some confidence, which is great, right? You're talking to old people through a screen, right? You mentioned the word of how important it was to them. You made about them instead of about you, right? Which is huge. Four, you were extremely assumptive every step of the way, and you always finished with a question. Yeah. And not an open-ended question either. It's usually like, yeah. So impressive. So impressive. And the reason why I can't, we can't talk about this right now. Yes or no. Instead of when do you want me to? Now, I do. I'm very good at booking appointments at the door. If I can't get in, an appointment is being made, and I always say to the team, it's not a no unless it's a no to your face. So I'm here now. If you're going to say no, do it now. Yeah. And that's when I let the lead go, sort of. You know what's the most impressive part about you, and I notice this with a lot of top producers like yourself, is their mindset, their headstrong. No one is screwing up their day. Like they are in complete control of what happens throughout the day and the accept responsibility for what happens. Like if you don't get an appointment, you don't blame the prospect. You blame yourself. Oh my gosh. We just talked about that in the morning. Yes. And you can pinpoint exactly what you did and why you didn't get the appointment. I might want to know exactly how to screw that up. I can try to save it, but I know I'm just going to go over there and try to meet her in person. Yep. So impressive. So impressive. As you are building out a team and helping others, what's some of the things that you're noticing from some of the people that are doing well versus some of the people that are really struggling? Oh gosh, that's so easy to answer. It's just it's it's they've allowed. I'm guilty of it. Like especially you got to remember a lot of women talk to me. So with women in general, they just don't show up. Like something comes up and they just don't show up. And I'm like, this style session has been going on forever since we've had the same. We have had the same weekly schedule since we started and you get a notification and an email and a text reminding you to get on and you just didn't log in. Like you you're going to have to I can't make you log into your computer. So I think the part of the problem is just showing up and just log on because once you log on, when I see you, if you're in another state, if I see you logged in, I am directly talking to you for some reason, I'm going to pull you out of whatever you're in and my agents will tell you like they call me on purpose to get pulled out of whatever crud they're in. And like today we went over time blocking something so simple as time blocking. Got agent. I got an agent with five appointments, eight appointments, 11 appointments, all because we sat and just got out of our funk first and got it all out of the way. Yeah, I love that. I said all the time is literally step number one to success. It's just showing up. It's just it's so easy to not show up. It drives me insane. I freaking love that you said that. That's so on point. Did you earn six figures your first year? Yeah, I did. How rare is that? Oh yeah, I did. I nailed it. How awesome is that? What was the exact number? Help me out. It was, I need to look, it was somewhere between 362,382,000. Two carrier trips back to back. No, three. You were there in one of them. It was a couple carrier trips back to, I mean, shoot, I was on vacation for like the whole summer of the next year. Geez. This was your, this was, so it was 360,380, whatever it was, thousand and premium your first year. Yeah. That's unbelievable. Oh yeah. I learned so much about leads I learned about.  Like you will learn how to manage your money fast. Yes, that's right. I bought leads from day one. I never had a lead given to me.  Oh yeah, let's talk about that. We can avoid that if you want. I don't care. I was told it wasn't terrible. I could have paid it off in full, but I took my time paying it off because who wants to pay me on your hands? Yeah, exactly. And they take payment plans. So I just set up an auto pay and let it, let it do its thing, but it never happened again. Yes. Yes, that's right. Yeah.   Go get a CPA right now. Yes. Oh, that's so good. Yeah. I had the same thing happen to where I was told by a CPA, you owe the IRS $20,000 at least. And I'm like, oh, mine was higher. I'm like, I'm 20 years old. What are we talking about here?  I'm like, I blew it. I can spend it all. Where do you mean? Where did it go? Why did you do your job?  And I got it down to eight grand, paid it and then I hired a CPA. I used H&R block, got the tax problem, and went and hired a professional. That's awesome. You did the smarter way. Please you guys get a regular CPA. And what you're talking about is running a business, right? Like they are, everyone's a business owner. I don't think people grasp that. Like they're like, okay, so I'm independent. I can just show up and work nine to five, like a regular person. No. Oh, I can't tell you. Maybe through, okay. So I'll be, I'm going to put some transparency out there for you. Last year was my absolute worst, and I just mean worst paid worst income in the business since I've been in the business.  This, this did not happen for me. And it's shame on me. And I sat in it. And I, this is just at six to 10 p.m. Monday through Friday. I'll show it to you. It's what I love. Yeah. It's the purple are the appointments. This is the week when I have my kid. Did you see all those appointments? Yeah, that's a lot. That's awesome. So, but time blocking and time management didn't happen during COVID. I allowed that to tear apart my business. I got emotional. We're women. Okay. We're going to get emotional, but also like I am, I am doing what mom and dad both do.  I am both a broker and a mom with men. You guys just don't make the money and we spend it. The agent, they'll like talk to Lauren. Like she's, she does a lot, right? Yes. But a lot of guys go out and produce and their women are at home having babies and like try doing all of that and being the producer too. When you're not in a supportive role. That's a lot. I respect a time blocker. You're just dead in the water and that happened to me and I'm embarrassed about it and I promise you it'll never happen again. I respect the heck out of that, by the way. Can they email you for the time blocker? Oh, yeah. I'll send it over. I'll post it on the, can I post it in your Facebook group or something as a file? Sure. Yeah. We have a group. It's called the insurance ladies. I co-host it. It's actually Joanna's group. I co-hosted with her and Amanda and Malia and some awesome other chicks in the business and it's, it's no, it's no recruiting. It's totally women-friendly, any agency, any, anything. And it's in the file section there, but I will give it to you. Cool. I love that. Okay. You guys have to grab that at super impressive. It looks like it was, you know, one of the most complicated things in the world to build. It's just that you have it. You can make it. It's just a block. Yeah. Just blocks, six a.m. to 10 a.m. Sunday through Saturday. Pretty simple. I'll give you my schedule because you're going to want that. How can I, how can we, how can we, you guys too, like just because I'm a chick, like you guys are not working like this. That's true. Yeah. I'll just give it to you because I go to the gym. I got a personal trainer. Like, see? I'm in there. I see you guys. That's the pride of all of them out. Yeah. I'm doing that too. Hey, if you enjoyed this, I got another one you're going to love. It's right there. Click on it. See you in there.


From Hollywood, it's time now for Johnny dollar Yes, I'm returning the call you made to mr. Ellis Rasmussen If you will state your business, I shall be glad to transmit it to him You tell mr. Rasmussen I'm an insurance investigator from Hartford and the matter involves a member of his own family. Oh Young mr. Rasmussen. Yes Could you hold on a moment, sir? I could Mr. Rasmussen will send a car for you at six o'clock. Look I can take a cab it Oh, well Bob Bailey in the exciting adventures of the man with the action-packed expense account America's fabulous freelance insurance investigator. Yes truly Johnny dollar Expense account submitted by special investigator Johnny dollar to the Universal Adjustment Bureau Hartford, Connecticut The following is an account of expenses incurred during my investigation of the Rasmussen matter Expense account item one two hundred and four dollars and thirty-five cents airfare from Hartford to San Francisco to Los Angeles Trying to compile the details of the Rasmussen case I've been on it three days when I was Stonewall in Los Angeles with a home be Hills address and the phone number of Ellis Rasmussen At six o'clock a liberate chauffeur in a maculet uniform stepped up to me at the desk Mr. Dollar. Yes, my name is stopper sir. I have mr. Rasmussen's car outside Well gee whiz stopper Ain't it the truth? A few minutes later when we turned into the lush green home be Hills section I had a suspicion I was about to deal with a bona fide millionaire When we parked in front of the big two-story colonial home and a man with graying hair and swallowtail coat stepped out of the door Well, I knew I was gonna meet the real article Party sir. Hello, Hardy and mr. Rasmussen is waiting for you this way, please We stopped in front of a huge panel door Hardy tapped on it once then pulled on the knob As we entered a tall man with a shock of pure white hair rose and miss chair and turned toward us This is mr. Dollar mr. Rasmussen. I want about four fingers of sour mash. What do you want? You took the words right out of my mouth mr. Rasmussen very good, sir He's a pretty nice fellow. We're all pretty nice fellows around here mr. Dollar sit down Thanks, could you hand me that lighter? Oh sure here. Thank you. What are you doing in Los Angeles? Why federal underwriters of Hartford wrote a blanket policy for all Imperial rubber company employees? Your son was an executive with Imperial when he was killed in Melea last spring Federal owes his widow $25,000. I don't know where she is mr. Dollar I see. I doubt if you do Let me put it this way. I Never met the young lady Fred married a one-night in Elk on Nevada Two days later.  Dollar? Oh, I wouldn't want to trouble the heart. He said a place for mr. Dollar. It's very good, sir Well Well, I thought she might phone me when she got back with states Never did never a letter nothing I'm old and sick, but I still want to see the girl my son married Not an easy thing to lose a son mr. Dollar and I lost a good one. I lost the best son a man ever had I'm sure you did sir to your son Fred During dinner and afterward over coffee in the cares I listened to the story of Ellis Rasmussen's life It came from the lips of an old man who was dying But in whose eyes I could see reflected the memories of a brawling bustling life that started in an Oklahoma oil field and moved to Alaska and Arabia and Africa More and more during the talk I began to know his lost son For an everything the old man had to say about himself. I could sense an unmistakable reflection of his son Finally, I thanked him and left If I may say so I do hope you'll call soon again, sir Mr. Rasmussen and George all visit very much I haven't seen him so much like his own self since we received the terrible news of young mr. Rasmussen's death He must have been quite a man hearty young Fred Rasmussen. He was all of us missing dreadfully None of us ever met Mrs. Rasmussen and we were most anxious to receive her especially after young mr. Fred's death Imagine so the car is already mr. Dollar. Good night. Good night, honey. Fine night mr. Dollar Yeah Stomper. Yes, sir. I Didn't want to press the point with mr. Rasmussen, but now maybe you can straighten me out Did he approve of his son's marriage? Let's put it this way mr. Dollar Mr.. Rasmussen approved of mr. Fred and if mr. Fred got himself married then mr. Rasmussen approved of the girl Between them two they had that kind of understanding real people Expensive on item two one dollar ninety-eight cents telegram to personnel division imperial rubber company requesting a copy of all information They might have on Laura Olson Rasmussen Item three six dollars one long-distance phone call to the universal agent working the case in san francisco Get a list of all the passengers who are on that plane Get someone checking the hotels in the bay area. She might have checked into one when she hit Frisco Listen, we're looking for a woman whose husband was brutally murdered about two weeks before she got back to the States If she's anything like I think she was probably about at the end of a rope now start asking questions at places where people like that go On Wednesday morning, I rented a car that's item four twenty five dollars and made the rounds first stop Los Angeles Board of Education By four o'clock in the afternoon. I had found 35 Laura Olson's who had attended public school in Los Angeles And were more or less in the proper age bracket The next day the folder arrived from Imperial rubber company Among other things it contained a passport picture and a complete description of Laura Olson Rasmussen She was a blonde girl with a pouting silvery kind of mouth and wide dark eyes Yes, what do you want? I'm looking for Mrs. Francis Olson. Are you Mrs. Olson? Do you have a daughter named Laura Olson? No, I'm an insurance investigator. I'm trying to locate Laura Olson Rasmussen How'd you get this address? What's that? A picture of her. I see That's my Laura. What about her? I've been trying to locate her for some time. Is she here? No, she ain't here. She haven't been here for five years. Do you have any idea where I can find her? Friends maybe? Other relatives? You say her name's Rasmussen now? Yes, she married a man named Fred Rasmussen. Married? Well, ain't that just something? You didn't know your daughter had been married, Mrs. Olson? How would I know? How would I know anything about her? Saturday at noon a registered letter arrived from the agent in San Francisco containing the list of passengers who have been on the plane with her Three of the names were in the Los Angeles area including a Mr. Oberlin who lived in Pasadena Sure, I remember her real pretty. We sat together all the way from Honolulu What's up? We're trying to locate her Mr. Oberlin Did she have it I mentioned her plans when she returned to the States lands, you know What hotel she might be staying at in San Francisco or if she was going on to another city? You say that very emphatically. Yeah, I guess I do You didn't have to show me a picture. You know a guy always prays will meet someone like her on a plane You know what I mean? Live it up. Yesterday's gone. She said tomorrow ain't here and the only thing we got is today Yeah, we had a swell time You're sure about this. We were pretty jumpy ball if you want the truth of it Mr. Roblin did she mention anything about being in Malaya before she boarded that plane? Then she didn't tell you that her husband had been killed a week before Yeah Oh, he was murdered by guerrillas in Malaya. No, she didn't mention that she didn't mention it at all, mr. Johnny dollar How are you hearty? How's mr. Rasmussen? He's not so well, sir. That's why I called could you possibly find time to visit him tonight? And may I send a car right away? Is it serious? I knew why he wanted to see me Have you located my daughter-in-law yet? No, I haven't located her mr. Rasmussen, but I know something about her I Know she drank whiskey and flirted with a fat salesman on an airplane all the way from Honolulu to San Francisco I know her mother's a drunk. I Know she didn't think enough of you or your son to contact you or anybody else when she got back Mr.. Rasmussen It looks to me like your daughter-in-law is a first-class bum Act two of yours truly Johnny dollar in just a moment Our daily lives are sharply affected by world news and for a complete roundup of the news every single week Good evening. Just keep your dial on CBS radio for the news broadcasts of our famous CBS news men Edward Armorrow and Lowell Thomas here up to the minute news with Edward Armorrow and Lowell Thomas on CBS radio Now act two of yours truly Johnny dollar and the Rasmussen matter There was a blank coupe with MD on the license parked in the driveway of the Rasmussen house when we pulled up In the bedroom a silvery-haired man in a black suit was sitting beside the bed that held Ellis Rasmussen He was introduced as dr. Butler Then I shook the hand of mr. Rasmussen someone suggested that dr. Butler might like to use the library for his calls And I was alone with the old man If you want some whiskey, I keep it in the sideboard over there not now. Thanks. I Yeah, I wish I had some news for you mr.  I don't work for you mr. Rasmussen I'm an insurance investigator trying to locate a woman and pay off a claim if I don't find it The case will just have to sit unless you or someone else concerned makes a report to missing persons Then the cops can take over and maybe they should right now My son was a fine man. I Can look back on all the years. I had with him and be proud of every year and every day He married a girl named Laura Olson I don't know where she came from or who she was But I know my son wouldn't have married her unless he loved her unless she loved him in return and was worthy of his love You know a lot mr. Rasmussen Perhaps I should go to the police. No No, don't do that We'll find her mr. Rasmussen. We're getting it narrowed down Well, I'd better leave now as you say Mr. Dollar yes, sir. I want to see her Yes, sir I'm Sorry, I talked to you the way I did phone call for you mr. Dollar. Would you like to take it in there? Oh, yeah, sure Keep an eye on an idea trust me, sir Johnny dollar Oh, yeah, you the insurance guy looking for a Laura Olson Rasmussen. Yeah, have you got anything? We got it Huh? She's here with the rest of the girls in the drunk tank. A drunk tank will always smell of disinfection This one was no different. There are no bunks. No chairs. No blankets. No nothing So you stand to sit on a concrete floor and wait for something to happen The legal period is 24 hours. You get rebooked or you get released All depends. What's the story on her dolly? I've got a check for $25,000 for Gee insurance money. Yeah, two girls better learn to get along. Which one back there sitting on the floor What's the situation if somebody comes up with bail they can have her Expense account item seven one hundred dollars bail While I was waiting around officer daily broke open a file on her a dozen aliases a dozen charges And one conviction for shoplifting a career of petty thievery that began at the age of 16 and ran up into a 22nd year Expense account item a thirty five dollars telegrams. I sent wires to all parties concerned all parties except Ellis Rasmussen Ordering a stop on their activity since Laura Olson Rasmussen had been found my name is Johnny dollar Thanks for getting me out Why I Did it for a friend I didn't know I had any Expense account item nine twenty cents two cups of coffee. We had it in a diner across from the women's section of the main chair. I Looked at Laura Olson Rasmussen while she drank the coffee Looked at the blonde hair and the wide eyes in the potting mouth Looked at the woman who had once been the wife of Fred Rasmussen. What's the catch mister?  No, you don't Where do you live? I've been staying at the Piedmont Hotel. You know where it is. No Not many people do Especially people with clean shirts What are you been doing since you got back from Malaya? I've been getting along You got something to do with Fred You know about Malaya. I know about a lot of things. I've been looking for you for a month What why didn't you contact your father-in-law when you got back? Why should I? What would he care about me?  I wish I was kidding you. I wish to heaven. I was kidding you What now I Want you to come over to my hotel? Oh now to sign some papers. I Have a check for $25,000 for you Who's that your husband was insured? You're his beneficiary all you have to do is fill out an application I'll give you the check. I Don't believe it. It's true. Come on Expense account item $10 $2 cab fare to my hotel. I took her upstairs with me stood over her while she filled out the necessary papers Outside of that we didn't say a word Johnny dollar Yes Tell him no luck yet stuffer How is how is the old man? I'll talk to you later. Yeah Okay, thanks Here's your check Anything else? Nope, that's it Okay, we're on some time Sure. Oh me about a man named stuffer who worked for his old man for years Was that him on the phone just now Yeah, yeah, I Didn't know what I was going to say to the old man But I did know I was hoping that if Rasmussen had to die that he'd die before anybody told him the kind of daughter-in-law I'd turned up. I didn't want to be in on that Expense account item 11 $83 hotel bill. I checked out at 5 30 picked up my airline tickets at the desk That's item 12 then sat around the lobby for five minutes. I'm 13 two drinks for myself Yeah, I read in the paper that Alice Rasmussen is dying. It's that true. That's true Man by sedan So just What are you drinking? Nothing. I know why you didn't tell him you found me and I don't blame you Fred's dad is anything like Fred was and I know how you felt Finding me the way you did. Let's forget it. Mrs. Rasmussen. Shall we?  The human thing would have been to see him when you came back But not a line not a word That old man in that house knows his son was really a man and on that basis He believes without seeing you that his son married a real woman He had love and sympathy and help and devotion and all the things you don't seem to have any use for waiting for you in that house he So on everyone I loved Fred Loved him from the first minute I saw him You know what I was doing when I saw him Was serving cocktails in a place like this? He didn't ask me what kind of a family I came from Where I was good or bad Just put one of those big arms around me one night and set your mind He said that to me He said it because he loved me no one ever loved me, but he did Don't even who I was where I came from and all he said was You're with me now We went to Malaya together and I never knew In all of my life What I knew then how it was to be wanted by someone who was decent Kind Then he was killed. He told me one afternoon when I was in the chair I took a boat and Then I took a plane back here. I want to see Fred's father.  Couldn't you see me walking in there and saying I'm me? Couldn't you see that mother of mine moving in? What would that have done to the old man? It would have crashed out of his whole memory Fred Don't think Mr. Dollar. I haven't got my memory, too. I didn't drink that away. I was loved by a man And I loved him back. I'm going out there pretty soon. Would you like to meet him? I think so I Think so very much. Oh, Mr. Dollar. I've been waiting in the lobby. I thought you might be here.  I didn't say much. She didn't say much But in the half hour it took to get out to home be hills something happened to her again That's something that must have happened when the big arm went around her shoulders the first time good evening Mrs. Rasmussen hardy. How do you do Mrs. Rasmussen? We're very happy to see you. Thank you hardy Red spoke of you often that was kind of Fred This way please. I think you can introduce Mrs. Rasmussen. Mr. Dollar ring if you need me, sir Laura if there ever was a man for you not to be scared of it's that man in there How can I tell him about myself? I've been in jail. I watch Well, mr. Dollar come in I did I've brought someone for you to meet mr. Rasmussen. You'd be my son's Laura Yes, you're Laura.  No Nothing you have to tell me Now feel it Mm-hmm. You're my daughter Understand that yes, then that's all the explanations we need between us. Yes Hardy hardy Yes, mr. Rasmussen bring bring mrs. Rasmussen some brandy, I think and I'll have some sour mash mr. Dollar Sure, make mine sour mash too hardy. Very good, sir Expense account item 14 40 bucks miscellaneous item 15 35 dollars denigrafing Expense account total one thousand nine hundred and sixty five dollars remarks The old man's got a few weeks more Laura's moving into the house with him to take care of She won't be telling him a lot of things about herself. She doesn't have to You should have stood there like I did and seen that big arm go around her shoulder when he said You're my daughter Yeah Yours truly Johnny dollar our star will return in just a moment Folks abroad want to know more about us Americans how we live how we eat what we do in our leisure time You know something you can help promote international goodwill by corresponding with someone abroad for the name of a correspondent Right to letters abroad 45 e 65th Street, New York. That's letters abroad 45 e 65th Street, New York Stay tuned for five minutes of CBS news to be followed over most of these same stations by the FBI in peace and war Now here is our star to tell you about next week story next week one cute tiny little mouse That's right. Mouse almost scares a big insurance company out of business Join us. Won't you yours truly Johnny dollar yours truly Johnny dollars starring Bob Bailey Originates in Hollywood written by John Dawson. It is produced and directed by Jack Johnstone Herd in tonight's cast were Virginia Greg Jean Tatum Eric Snowden Roy Glenn will write Frank Nelson and Jack Crucian Musical supervision is by Amarigo Marino Be sure to join us next week same time and station for another exciting story of yours truly Johnny dollars Dan coverley speaking