with Adam Swallow of OUP who is with us here today. I was involved in a book project that a number of people in the room contributed to on development ideas, something that was discussed this morning quite a bit. And one of our conclusions was that while economics and economic policy are extremely important, so are societies and societal preferences. And the differences amongst societies explain, of course, the very different outcomes that policies can experience in a range of countries. And perhaps this afternoon we'll touch on that even more than this morning, but I thought the panel did an admirable job and plenary this morning of dealing with that. We're going, I'm going to ask our speakers to speak in the order they did this morning. So we'll hear first from Montek, Singh Alwalia. Montek and I have several things in common. The first is a great love of India. The second is an obscure affiliation in Singapore. And the third is a college at Oxford. I'm a great fan of his and was living in India during part of his time in government and hugely admired the energy he brought to very important responsibilities in the planning commission. We also have with us Stefan Durkan who has done a great deal of field work in Ethiopia, I think another fascinating country, another laboratory for development and who joined DFID as chief economist a little while ago. And finally we have Steve O'Connell who is able to remain affiliated with his university, Swarthmore, while he is the chief economist at USAID. So we'll hear from each of them for about 20 minutes and I'm sure they'll all be fascinating. And then we hope to have plenty of time for your views very briefly and questions for the panelists. So Montek, I don't know if you want to speak from here or from the podium, what do you prefer? The podium. Well friends, first of all let me say it's a great pleasure. Is this too much amplified? Is it okay? All right, well great pleasure to be here and thank you Finn and David for inviting me. Yeah, you know by making me speak from the podium you've undermined one key strategy I had which was David was gonna hit me on the head when I overshot my time. Now he'll have to wave some flags visibly but if I do overshoot it's because he hasn't lived up to his promise. You know I was asked, I'm very conscious of the fact that you all spent a whole day as indeed I have listening to some very fascinating research results. So I don't wanna give you a whole bunch of research results. It's meant to be a little lighter in tone and David and Finn also told me that some of it should be addressed to younger people who are hoping to have a career in government as economists. So I'll try and combine these two things. You know, I thought about it and I think the role of economists, you could think of three roles that economists have in the making of economic policy. The first of course is as an inside advisor to governments. That is what most people think of the economists anyway. The second which is actually very important is as producers of research potentially putting forward what are sort of valid inferences that can be gained from experience and what are not valid inferences. Once governments are in government, once economists are in government, they never feel the need to be briefed on the constantly developing nature of the subject. And yet I think it's very important that not all economists want to be in government. Some actually work on producing relevant research. The third which is becoming more important as our societies are becoming more open with much greater participation by different stakeholders is the role of the economists as an advisor to different stakeholders. Because we don't just want economists to think or to be thought of as some sort of providers of high level non-criticizable technical work and the rest of the stuff happens elsewhere. Politicians actually are constantly responding in any open society to what stakeholders either say or believe. And it's very important if we want economic ideas to surface, then the stakeholders who approach the politicians are well informed. So there's three very different roles and I can well imagine that the same economists could play all three of them. And actually that would be ideal. That they do these things at different times in their lives. Now let me paint to you two little amusing pictures of these roles, one which is extremely positive and the other and not well known and the other which is actually incredibly negative and is quite well known. The positive image is I believe this is a story that I got from a book which Meghna Desai has recently published called Hubris. And it describes William Pitt the Younger as Prime Minister of Britain, late 18th century, inviting Adam Smith to address the cabinet and asking the cabinet to stand while Smith sat and lectured them on whatever he thought was good economic policy. This I'm sure all economists dream of has, but I must tell you, those of you that become advisors, this is not gonna happen to you. Just put it down to the fact that you're probably not Adam Smith. But I think the world has also changed. Much more likely is the role of economists that Keynes had in mind in his much quoted, extremely well known quote which I shall try to reproduce, where he said, practical men who think themselves quite exempt from intellectual influence are usually the slaves of some defunct economist. And they went on to say, mad men in authority hearing voices in the air are distilling their frenzy from some academic scribbler of a few years ago. So this is not a very flattering description of economists. I mean, they're described as defunct or an academic scribbler of a few years ago. It's not a very good description of policy makers either because they're either in a snobbish way described as practical men which given Keynes' background was definitely a put down or they're described as mad men in authority. But the fact of the matter is that in real life you do probably the role of economists is closer to what Keynes had in mind. Now Keynes, of course, if you read that quote you get the impression that maybe what Keynes thought is that the problem arose because the economists were of yesteryear. But actually that's not true at all. Today, if you look at the big issues facing the industrialized countries, forget about the developing world because we're supposed to be different and complex and so on. The industrialized countries, I mean, and the most important issue today is should you have a more active fiscal policy to regenerate growth? And now you can have a bunch of Nobel prize winning economists on one side, arguing yes, that's what you should do and another bunch of Nobel prize winning economists on the other side, saying that would be an unmitigated disaster.  You will realize that politicians do not regard consultation with economists to be, say, comparable to consulting your cardiologist or cardiac surgeon. I mean, you know that there are differences but by and large you work very hard to get the right fellow and then you do what he wants. They view it, I think, a little more, like at least in India, our ancient kings used to view the choice of astrologers. I mean, they belong to a profession and sanctified a point of view and it was important for other people to know that, you know, if the king is going to war it's because the astrologer says a good thing. But they also knew very well that if you wanted to go to war the astrologer would find it possible for you to do so. And at the most what he would say is, look, it's not a good idea to go to war but if you do want to go to war, do it on a Wednesday. So, I mean, very rarely do politicians appreciate economists whose advice constrains them in a serious way. So I think this is a very important thing about which economists should know and I think you should work it into one's strategy for how to operate. Now, why does that happen? I think we had in the morning plenary a good idea of the sorts of problems that arise.  I mean, they pretend that they're like engineers because they set up systems similar to what engineers do, a set of interacting interrelations with many different variables which technically enables you to judge that if you change this, this is what'll happen. The only difference is that engineers agree hugely on what the equations that describe the system are. So you can have an enormous amount of, give me some difference at the margin. In the case of economists, there's only an agreement that you must have a model in your mind and it should be more than one equation. But exactly what goes into each equation nobody really knows is very difficult to quantify and in any case, you have to eliminate a lot of that in order to make the analysis simple. So you've got a commitment to conceptualize the problem as a complex interaction of human and other relationships but no real agreement on what matters. And this enables you to extract from your economic analysis whatever rule of thumb you like. I mentioned the issue of fiscal deficit. I mean, each one of these gentlemen can write down the equation that they have in mind and why what they're saying is true. Give you another example, devaluation. I mean, anyone goes through his or her undergraduate courses on macroeconomics will say that devaluation increases aggregate demand. So you would expect that it will have a stimulating effect other things being the same. But then somebody else says that's not true if you take into account balance sheet effects. So if you have a balance sheet that is impaired with a lot of foreign debt, then the devaluation could really screw up aggregate demand. So there's a small change leading to a completely different conclusion. I think economics actually full of these and that's the nice thing about it. You can probably describe the behavior of political people as choosing a set of rules of thumb. Political people never sort of come in and say, well, you know that model what would happen if we do this or that? I mean, that's all sort of intermediate product. They have a rule of thumb and the rule of thumb would go something like there's too much inflation, what are we doing about monetary policy or there's too much something else, what are we doing about this? So these rules of thumb are really extracted from some simplified model. What the economist has to know is that small changes in the model can lead to very different rules of thumb and the debate should really be on that. Are you quite sure that this is the rule of thumb because you know actually while it is generally true, I mean with a small difference it could be something else and that is what lies at the root of this enormous and valid suspicion in my view of the sort of one size fits all type criticism. It is said that that's what the IMF and the World Bank do or did and if they did, that's definitely wrong. Whenever you ask them, they say, no that's not what we do, we fully realize but the poor fellows have never got over that charge. But I think this is a very important part of a lot of the debates that actually go on. Now part of the problem is that I think given that we have this determination to follow rules of thumb, what should the economist actually be concentrating on? Well, you can divide it up into sort of macro and sector. Macro is the area where in some way the economists have dominated. I mean the politicians are there but macro is really decided between the finance ministry, macroeconomists and the central bank macroeconomists and the macroeconomists in the international, in the financial ratings agencies and where the international institutions are active, those guys. And I think a lot of what I've said applies there and that debate goes on and a good interaction is one where not only the government but the stakeholders are also aware that the debate between rules of thumb is really a debate between what is it that actually drives the economy. One can't judge what that should be except by reference to the country itself and in principle there must be a correct model for each country. But in my view, a lot of progress is made if people actually recognize that you are to find out what the correct model is because that opens up the discussion. And when you come to sectoral policy that's actually much more interesting in some ways because economists do not dominate sector policy. Sector policy requires the economists to interact with stakeholders and requires all of them to interact with sector specialists. And that is a very different ballgame because whereas the macroeconomic debate is all conducted on the basis of certain commonalities of understanding, the sector debate moves all over the place and it is extremely difficult for economists, the job of the economists is to explain to the policymaker that there are many different objectives that you may be following here and each one of them will upset somebody or the other and you have to find a balance and more importantly, you have to explain to the people the rationale of what you're doing and hope that they understand. This is another role that economists can play and actually if you want anecdotes, since people always remember anecdotes rather than arguments, I don't know if this is true but it is said that I think it was some Montague Norman who was the governor of the Bank of England in the interwar years who was first fellow to hire a young economist as his economic advisor. So this young man comes along from Cambridge University full of himself that he's going to be the first economic advisor to the governor of the Bank of England and on his very first meeting, he's brought down to earth because the governor says, young man, let's get this perfectly clear. Your job is not to advise me, it is to advise the people of the correctness of what I'm doing. That is also by the way an important role for economists to play as long as other economists can take them on. So I feel that there's a big debate and economists pitch into it everywhere. Now, I sort of thought, well, this doesn't seem very satisfactory for a research conference. I mean, research conference, you're meant to zero in and prove something and all I've said so far is that it's very, very complicated and one really doesn't know what's the right thing to do. So I was thinking to myself that I was a member of this group called the Growth Commission set up by the World Bank many years ago, chaired by Mike Spence, a good friend and a very distinguished economist and actually who shares with us an affinity with the University of Oxford. So Mike agreed that along with all the usual preaching about what might be sensible things to do, we would put in a list of 12 really bad ideas and I was arguing that it's generally clearer what are bad ideas than what are good ideas. So we should be able to put down a list. So we did put down a list and Mike on several occasions told me that many people really liked that list. He didn't tell me that they liked the list of good ideas but they all seemed to like the list of bad ideas. We were very careful in the Growth Commission report by saying, just as there's no one size fits all, you can never be sure whether these bad ideas are always bad ideas. You know, there may be circumstances where this particular idea for a particular period of time might be necessary but by and large you can be pretty sure that they're bad ideas. Well, I thought that was pretty good but when I looked at that list before coming here, I thought to myself that one of them almost certainly would stand the test of several years and that is to governments today, do not nationalize industries. But then I find two days ago in the Financial Times, the newly elected leader of the Labour Party has declared that he's in favour of nationalizing several industries. So I don't want to comment on British politics, it's quite possible that the criteria we had laid down that there are certain circumstances where some of these bad ideas may be valid but it's just an indication to you of how much in economics one can differ and have differences between reasonable people and still not be sure of what's the right thing to do. So I conclude from all this that, you know, in an open society, economics is not like physics, generally if you were to set up a space commission and say we want to go to Mars or wherever, you would have a pretty huge consensus on 90% of what needs to be done. Economics is not like that but what do you need? Much more than in those sorts of technical things. You need a very active debate and I just want to close by making a reference to the sustainable development goals which have been mentioned or invoked different times. I think sometime this month, 17 broad goals have been laid down. The difference is that these goals have been laid down with a lot of participation by developing countries and I think as was pointed out in the morning, they are symmetrical whereas the MDGs were sort of telling the developing countries what to do. These things are goals which both developed and developing countries have to follow but they are extremely general. I mean for example, one of them says things like, I forget the list but one of them has things like reduce inequality between and among countries. Now that is an, I mean there could be a huge range of differences and by the way, it says that along with ensure rapid growth. So to my mind, wider could contribute hugely to making sure that these two objectives are sort of kept closely together. It's not totally clear to me by the way that if those objectives are both to be observed, for example, my guess is that to observe those objectives, inequality between nations can come down, it already is coming down. But you know whether inequality within nations is also going to come down or even whether you should be that hung up about it growth is rapid enough is an open question. Yet what I'm saying now contradicts the social development goals as stated. Now what I believe is going to happen a year from now, they already have broken down these goals into a whole lot of, no sorry, these are goals and then there are going to be targets and that's list down. Then there are going to be 169 indicators. Now the indicators are what can be quantified. So we don't know for example, if the measure of interstate or intercountry inequality is going to be the global tile index or the global genie coefficient or the global Palmer ratio or whatever. But by the time those 169 indicators are put together, that will become clear and then we can monitor for the world as a whole and for each country. Now economists to my mind have a very big role to play in making sure that sensible indicators are chosen because the data set in most countries does not contain reliable data which would respond to many of these indicators. That doesn't matter, over time they'll develop it. But this is really where economists have a large role to play. But you know, if you take a pure economist approach you would say that if I've got 169 indicators, then I will almost certainly have a situation where there'll be progress on some of them and not enough on others and even regress in some of them. So what do we say? So a typical economist view would be, pull it all together by giving weights and have a sort of a single objective function. Stakeholders will be totally opposed to it because stakeholders are concerned about each of these indicators and none of them is willing to accept a trade-off because that would seem to depart from basic commitments. So we're gonna end up with 169 indicators. We'll monitor 169 of them. And I just hope that economists will have a big role both in their own countries and globally and why there could play a leading role in this to make this particular decision a more informed decision. Thank you very much. That was wonderful, Montek. And I'm sure lots in the room will want to have at you a bit later. Stefan, over to you. Right. Well, good afternoon. As David mentioned, I work these days for the UK Department for International Development. So, you know, the UK's Development Ministry spending development aid, contributed to development policy and so on. I should also say that I'm not quite sure where I'm talking as David. I probably don't here. Still have my hat on as a professor in Oxford. And I just also thought of plugging that, you know, so from 1st October I'll be returning most of my time back to Oxford, but I'll go to some new institution that's been founded a few years ago, the Belfatnik School of Government. So Oxford will actually have something new, which is actually a public policy school as well, where some of these things, maybe some of you will one way or another sooner or later pass through contributing to global policy making in the world. So I want to talk a little bit about my experiences as a civil servant, having all my life been an academic, coming into the civil service and having great ambitions on saying, you know, as an academic, how can I bring some thinking from the academic world, thinking, putting things into policy, influencing, making a difference, and how would you do this as an economist? So I kind of, four reflections I'll make and then come to a few conclusions. And so the question is a little bit, you know, economist, what do we experience in our world of international development policy making? So some of the things we experience there.  I find it a kind of amazing thing to be able to do this as an academic, I have never been accountable to anyone.  And to basically knowing that there are accountability processes and you know they have the right, even if I intellectually totally disagree with even the premise of the question, I think that's quite something. And that's the first thing I definitely want to say, coming as being inside government as a loyal servant, it is actually something quite an interesting thing. But I came with a lot of ambitions. This is a very interesting man. I learned about him once in a while. I don't know whether anyone recognizes him. This is Thomas Alverdeson, you know, the man from the light bulb. The man had a very interesting approach to everything. He was an engineer, Montec alluding to it. And in fact, the point I will end up making comes down to a little bit what Montec has alluded to. But he had an interesting relationship with Salt and Pepper. It was as follows. He would always have a hiring process, actually quite similar to Oxford, but with a different objective. You know, as you know, it used to be in Oxford, if you wanted to get a job, you were taken out to dinner and they would look at whether you could use your knife and fork and hold the conversation and you would get a job in Oxford at high table. Actually, Thomas Alverdeson did something slightly differently. He did observe the guest even before anything was set when the food was put in front of the table. And he would observe the guest very carefully and the candidate, he would observe it in such a way that basically he kept on looking at what they would do. If the candidate put Salt and Pepper on their food before trying it, he would not hire this person. This is not an evidence-based way of deciding anything. So, of course, he's an engineer, but it's quite an interesting thing. And actually, he basically said, no, this person has preconceptions of the world. I can't deal with this. Everything has to be evidence-based. So anyway, so that was interesting. I definitely would have failed that test, but I'm not an engineer. But the people we deal with are actually quite interesting. You may, maybe as an academic coming into government, thinking about, look, I would like to make sure that evidence is all it is. But what kind of people do we meet? You know, I have met in my job very senior people in the civil service that told us, just trust your gut. That's all what you need to do. It was an extremely senior person in civil service, actually. And it really terrified us. Just trust your gut. Your gut feeling is all what you have to do it. Intuition is all you need. Of course, we also meet politicians who basically fundamentally say, I know I'm right. And in fact, the job of an academic gets reduced too. Can you please prove that I'm right? Which is essentially policy-based evidence-making. Can you please find a piece of evidence that supports the policy? Look, I think it's pretty good in the UK, but it happens still all the time. It happens everywhere. Because you will ask Montecollude to, you will always find somehow an economist or a research paper that you can cite that is published in a journal that will prove every possible point of policy-making ever invented in the world. You will find it. We are the kind of profession like that. And one of the things is that one of the risks you run as the economist in government is that you actually become fundamentally the prostitute of government. That you can deliver anything you want and any kind of service in that kind of sense in a very strange, short of way. So it's a really interesting. But there's another side that makes it very hard. If you want to be, you also know something about the way policies often being done. They have an idea in their heads and they want the example of a research paper that proves what it says. In fact, exemplification is one of the main criteria, main ways that actually policy makers work. If you read the politician's speech, it will always be a general point, for example. In fact, that's what we often are told. Can you give me an example to prove this general point? It's a message I will have had occasionally. Let me not say, give more details, what examples. But anyway, but it's basically, and it's a bit of a bit that also policy documents, even World Bank documents, for example. World Development Reports, if you analyze them, they're typically basically a piece of text, little footnote, such citing one paper that somehow proves this point. Exemplification. It's the technique in every policy document that seems to be produced. And it's a really tricky thing because what we do, we do actually the following. This is the danger it is. This is really one of the dangers for us being in a world of the profession we are that every point can be kind of proven. Because we create the impression that the example in the distribution of success or failure is actually a typical example of, say, for example, we say, let's do a skills program. There are skills programs in the world. You'll have to look very hard, but there are skills programs in the world that work. A few here and there. But we create the impression by quoting them as if they work. As if it's a typical example, the median example. And the way the politician uses it in their speech, the for example is to create the impression that this is a median example of it. But we know, of course, it could be actually that there are plenty of examples of massive success and plenty of examples of failure. And if they want to make a speech to say, skills programs are rubbish, I can deliver you the example that skills programs are rubbish, I can provide you the example that skills programs are brilliant, and I can give you that. I create all the time the impression that is a typical example. So it's one of these things that we do and we see it all the time. So it's something there as economists that we seem to be able to, as Montecar alluded to, always find an economist that will prove your point. And that's actually really something that I definitely observed, making the idea of an economist in government really hard. Because in the civil service, there will always be an economist that at that moment feels like, oh well, I find you the example that actually this works. And so that's a really tricky one. It's my first one. There's another side to, is also a little bit how the profession evolves. And often how the profession forgets somehow some of the key things, the environment that we're actually working. And this is actually international development really, really tricky. So a second point is a bit like, we tend to be, we are the solution providers. However, an international development and the eight narrative that gets presented creates already some problems. Again, Montecar alluded to the SDGs, which, you know, they are general. There's all kinds of reasons why they look what they are. And you know, maybe we can work with. But when it comes to the end process, when suddenly you get 193 world leaders and going to sign them, actually it's probably 192 because Burkina just had a coup, somewhere, all kinds of characters that are lots of shady ones as well that sign this, it's not altogether clear what it means. But we signed somehow, we create an impression of some kind of dream that we have in development, which is, okay, it's reasonable. Because what is the dream typically? You know, anyone working in development, what is the dream is actually trying to create Sweden? And that's basically what we do. The SDGs definitely, that is Sweden. Let's try to create Sweden or at least what we think Sweden looks like. Okay, it could be Finland too. But it should be somewhere Scandinavia, Nordic country that we're trying to create. Now, why do I say this? Because even me in the UK, we are doing things as if we want to build a society that looks a dream with a very inclusive society that is fair, that is welcoming to refugees, politically moderate, with a strong welfare state. Actually, for those who know a little bit about the UK, it's the only country that actually has a golden threat on it, which is what our Prime Minister in the UK talks about, which basically has all the beautiful institutions there. But basically, it's this kind of idea of creating this kind of state that is politically not really meddling into world affairs, but actually represents some really good values. Whether we USAID or whether we DFID, that's what we typically do. Whether we UNICEF, whatever it is, that seems to be what we always kind of stand for. Now, I'm not saying these are wrong ideals. These are pretty good things. But let's be clear that the reality is more like a Jackson Pollock picture. And that actually, as economists, we are simply asked with a given the task, can you please turn that little mess into that kind of dreamland? Let's create an inclusive growth. Let's indeed get inequality down within countries, across countries while we're growing, as if this is all just, look, you'll get us a little recipe. So there's a lot of that there. So John Berryman, the American poet, probably correctly says towards development economists, I'm sure Swedes don't exist, Scandinavians in general don't exist, take it from there. You know, look, we can't just start as if the world will be just easily turned into Sweden. Now, I'm not saying it's wrong ideal, but what we do, and this is something again, and that's actually, as economists working in international development, sometimes fail to understand. Is that actually these realities are very different in different places? Or put it slightly differently, the states and the politics that we have to work in these places is extremely heterogeneous. Let's say take actually the kind of countries that if it works in. Oops. The, sorry, I got a slide in the wrong place here. Yeah. The kind of state that we work in. So we often are basically one of the tasks in development, and if I'm honest with you, just in this morning alluded to it, let's turn the world into Asian tigers. That's the kind of task. Now, look, it's not a bad ambition.  They've grown, they've reduced poverty, they've done an absorption of labor, all kinds of things have happened. You know, there's maybe other things we don't like about the state, but some say kind of from a developmental point of view, there's something there to be said about it. For example, as Diffit, as economists in Diffit working in the world that we actually work in, you know, we rarely meet Asian tigers. In fact, take it from me, there are no tigers in Africa. You don't meet these kind of countries. We meet all kinds of other places. Even in Asia, if you think of the animals that represent an awful lot of the South Asian states, they always seem to be picking a peacock or a beautiful pheasant. Now, it's great. There's beautiful features to that bird, but it's a bird with little muscle and not very powerful either. So it's a very different state we meet in these places than actually the Chinese state or the Vietnamese state. If I go to Africa, I don't meet that many tigers, I meet a lot of tiger fish. Like the fish in the Congo River, what does the tiger fish do? I don't think it has any other purpose in evolution than eat little fish. It's basically states that we meet where we're trying to do things, but where fundamentally we're dealing with elites that fundamentally just want to eat little fish. Or we're having countries that maybe look much more like barking hyenas. Where you actually become scavenger states, where you think of unfortunately South Sudan at the moment, where it basically is all a fight of all the scraps and all the pieces and all the kind of things. Where the elites are basically turning against each other to getting more the scraps. Now, it's very hard for us as economists to actually think through what would be the ideas that we have in these one places, how do you turn this into it? In fact, I do think we meet a lot of hippos in Africa as well. Hippo states, which is an interesting animal. It's very big states, very big animals. I'm not always sure what they do all day, but they are very big. They are, you know, they do something, not entirely clear. You know, you can work with them, you can make progress. They often are, you know, lots of patrimonial states for pattern networks are really important. But it's again things where we need to understand what's going on. And if we meet these kind of states, we should also never forget that a hippo is the most dangerous animal in Africa. That more people get killed through hippos than to any other animal. And that we should also not forget about it. In fact, you know, there's not that many this kind of tiger kind of economies that we often meet as economists working on development. And we just need to be conscious of it. Now, why do I emphasize this? Because our ideas that we want to bring over and bring across, we better understand who we actually are dealing with and where we're trying to apply them. And it's actually very hard because we are very easily the kind of discipline that quickly wants to say, how do we plant it to some other? And the one thing I definitely learned as a civil servant, work in international development, that I had to start reading politics again and understand what actually happens in politics. And just simply saying the models can be transposed, I'm not sure. For example, I think that if we want to do East Asian policies into Africa, we probably need to think about it that the state we meet in Africa is not like the East Asian state and actually take it from there and actually start thinking what we could do, how we could work and so on. And that's important for an economist because we are always craving for passing his own. It's a fundamental challenge. The politics of the places that we're working is a fundamental challenge to the external validity of policy prescriptions we make and their likely impact. It makes it very hard because it is a real tricky thing then to do and so on. My advice that I now tell to my economists is that don't talk politics only at night over pizza, but talk it where you sit in the office when you're actually trying to think through what are the economic policies we would like to support in this country and how we work through it. Don't do it at night where you work in these countries and at night everybody goes on and complaining about the politics but actually find ways of bringing it in. And sorry, I had something, my slides jumped here. There is another thing that we also should know we're very conscious of. We are dealing with particular types of countries we work in. We should be very wary of what we're always very willing to do as economists. No, I'm not a relativist. I do believe there's something like truth. I'm not sure how you get to it but I still believe there's something like truth. No, not everybody is right. So I put it very simply, not all of you are right, okay? Because more or less in the room you're representing all kinds of views and all kinds of opinions, we're not all right. The truth is out there. I did like it at the time as well as a good TV series but it is still there. We should be the agents, Scully and Mulder of research trying to figure out what actually is happening in the world. But the problem is that while we are trying to do this and debating, we rush so quickly to give policy prescriptions. I am tired and sick of tired of policy briefs. I don't read them anymore as different because they typically summarize one little piece of research that somehow tells this one piece of little research has a policy conclusion. Usually it hasn't. We keep on trying to give somehow as if there's a solution out of every little piece that we have. We also try to always say there is a silver bullet there as well. These are the three things you need to do. I know I just want policy makers want but we want to do. Let me be a bit clearer. It doesn't mean that research doesn't have policy implications but we should be much more clear and much more careful. Research is to do with the bodies of evidence with a lot of things that are happening in research. A policy brief typically present one piece of research and tries to present it again as if it's representative of all the evidence that exists around this. I can't judge and I have to tell my different colleagues please don't use anything on the basis of one piece of evidence. Don't try to say as if that it's all there. And furthermore don't ever believe it when someone says and this is going to be you really have to do this you'll see it works. There is a silver bullet there as well. As economists we have over time not often done that and we always come to do it. I would actually almost appeal to economists to write sometimes a bit less about policy until you're really sure that that's what the balance of evidence tells you. And there would be a little bit more humble and a little bit less always thinking that every point has to have some policy implication everything has to be done. And that every piece of research should have a policy brief because often good research doesn't have really an implication that someone should really take up. It's a little piece that actually adds to what we have and actually builds more to it. And so finally I want to say something here about another thing that economists should be conscious of when they work within government as civil servants. We should be conscious that even we are biased humans. I'll just report and I don't have two minutes but I'll just very quickly on a bit of work that has happened in DFIT and also in the World Bank. Just some little behavioral economic survey work and just briefly testing a few very simple basic things. So we basically presented a lot of economists also a lot of other people in DFIT also in the World Bank, lots of economists and other people in DFIT but the results were the same for economists for all the rest, some bit of evidence. This from an impact evaluation that many of you will know very quickly to read I'm not going to go through it but it has the scale of does something have an impact or not a particular treatment doesn't have an impact doesn't have no impact. This is the number of people that got better from it the number of people that got the same and there's some kind of control and we said it's a beautiful design scientific experiment and that's the evidence. Now tell us what you now see in this that's what we asked people. But we offered two framings one said this was about the treatment for skin cream and the other we said this is about the impact of the minimum wage in a particular country in a particular setting of introducing it, two framings and this now tells us how many people got it wrong or in fact it tells us how many people got it right, sorry. If we ask it framed simply by saying this is about skin cream 65% of the people got it right 35% clearly couldn't quite read the table. Okay, that's fine. People make errors, it's too quick and whatever. 65% got it right. When the only thing that changed was not the numbers but a few words, only less than 50% got it right. This is what we call confirmation bias in behavioral economics, behavioral psychology that people actually implicitly want to get their own points proven and they see it, they see the data and they'll interpret it differently. The World Bank was actually worse than us. We had to have 48% got it wrong with the frame of minimum wage, the World Bank 34% got it right, so almost 70% got it wrong when the framing was and we could see that it was correlated to somewhere else in the survey we asked people's political views. The errors were correlated with political views. Let me not go further into it but basically we did it perfectly, beautifully designed all kinds of things. Basically left wing people will make more errors if the experiment didn't show an impact of minimum wage, right wing people make more errors if the experiment showed an impact of a minimum wage on poor people, a positive impact. Anyway, so I want to conclude, we should be very careful and humble. We need much more scientific humility and integrity. Look, we can't be all right here. We can't be all right all the time and let's not simply turn it always in a bit of a, anyone is always available to actually make a big point. And we should just be conscious to be working highly politicized environment where people that trust ideology, gut feelings, we work in developing countries in really messy environments where the elites don't really want much change and we should just be very conscious on it. We should actually always encourage synthesis work. Good, careful, honest, synthesis work. Social scientists don't value synthesis works. It's extremely hard to get the top researchers to actually bring ideas from other people together. If you ask the top social scientists what you think about it, not so sure. Either say, I have a paper on this or if you give me a lot of money, I'll get you a paper on this. They'll never tell you what often a scientist will say. Actually, I know about research has been done there and there and there and that seems to be the kind of thing we get. And honestly, that's all the time. We don't really dare to bring social scientists to our ministers because they always give this kind of answer. Give me a lot of money and I'll tell you in three years. People, top researchers are not interested enough and I must say that's why there is actually very good. And now finally, we should help young economists working in government civil service and all the countries they work in to actually get a good understanding of the politics and empower them to navigate how do you deal with this? What Montec's advice, having worked so much in these things, we need to get out of it, getting advice to people. How do you get this? Because fundamentally, we can be very powerful. If we try to seek the truth and to be very careful what the truth is, truth to power is possible. And I'll leave it there. Thank you. Good afternoon. I wanna start by thanking Finn for inviting me. I worked with Lance Taylor in 1985 when I was still a graduate student. So I remember you a new wider when it began but this is my first time visiting. So it's been 30 years of my career, 30 years of wider and I'm really pleased to be here. Well, let me start with a disclaimer. So I am the chief economist at USAID but I will be speaking here as myself. I'm going to very much contextualize my presentation as a look inside USAID. I'm a macroeconomist. I've spent my career doing research-based consulting in Sub-Saharan Africa on macro issues and I've done zero of that really at USAID. That's not where USAID is placed. So for me it's been a move into a new environment as an economist with an all-purpose toolkit with a specialization that's not terribly relevant day-to-day but where the toolkit and the habits of thought and the standards of evidence have been relevant and I want to give you a sense for how that's played out in one particular big debate going on within USAID. So I actually, Montec has a lovely paper that touches those bullets. I'm not going to go through them but I'm going to, what I say will touch on some of the themes that he raised. The context I want to put this in is USAID's new mission statement which arrived on the day that I began my job two years ago and it states it's there in the blue. We partner to end extreme poverty and promote resilient democratic societies while advancing our security and prosperity. And we partner in two key senses here. So one of them is this is a real statement of solidarity with the 2030 agenda which is going to be promulgated a week from now. And that's a sea change for USAID. We were not key players in the first Millennium Development Goals campaign. We contributed to the whole range of efforts but by our own lights, our own strategies, we weren't a driving force at all and the Obama administration has really embraced the 2030 agenda. The second one is really important too. It's something I value having spent my life working with African governments and African colleagues on the ground. We partner with domestic actors with the understanding that what they do matters most for the development outcomes. Those values will frame sort of what I'll have to say. Let me give you our definition of extreme poverty because we're gonna talk about $1.25 a day which is a target or an indicator that we're gonna pay a lot of attention to but our concept of extreme poverty and the statement we partner to end extreme poverty is much broader. So I'm not gonna read this to you except for the first line. Extreme poverty is the inability to meet basic consumption needs on a sustainable basis. So basic consumption needs, there's a shout out to standard measures of poverty but on a sustainable basis opens the whole game because a sustainable change in poverty requires that the household acquire human and non-human assets, requires that the household be in a resilient setting, requires that the household be able to be clear of the vulnerability to poverty as well as the actual snapshot of the household's current situation. I won't also try to give you a sense of that complicated diagram on the bottom but that is our view of what the multiple dimensions of poverty are at the very top of there is the interaction between assets and incomes but then below a set of correlated outcomes that are not perfectly correlated with consumption poverty, strongly correlated but we think of as independent elements in a broad setting. Let me sort of just underscore against the view at some level that we got from the previous two speakers that economists are for hire and we don't really know what we're doing.  I think these guys both do know what they're doing but somehow we, not economists but policymakers in developing countries are doing something seriously right over the last few decades. This picture is the declining red line is the consumption poverty headcount against the base of the 1990 value, proportional decreases. The other six or so indicators are other dimensions in our definition. So broad-based progress and then the blue mountain there is changes in income. Here's another look at tectonic changes in the world economy that suggests that things are moving inexorably and dramatically in the developing world. That red line there is the share of the emerging market economies who are in the top 20 economies in their contribution at decades intervals in their contribution to global economic growth at PPP exchange rates. So that's inexorably rising. It reaches above 50% projected out by the wheel growth rates in 2010 to 2020. The blue line is essentially the OECD basically. I've excluded Mexico and Turkey and put them in the emerging markets group. So there's big, big stories going on here. These are in the top 20. These are big economies but by the time you get into 2010, 2020 Japan is not even on that list. Nigeria is on that list, for example. So a lot of things are happening. And success is behind the doubling down on the extreme poverty goal that the World Bank has done, that USAID has done with the impetus from President Obama in his 2013 State of the Union address and that the development partners are gonna do jointly at the younger meetings in a week. Here are the key stylized facts that we are grappling with in thinking about our own commitment. First, the geography of extreme poverty as a result of the tectonics you saw on the previous slides is shifting very hard and fast into sub-Saharan Africa and that is projected to continue. And it's also the overlap of extreme poverty incidents with conflict and fragility. Institutional fragility and conflict is getting much, much closer. So when you look forward to the task of completing the job with some confidence looking back at the success the last couple of decades, the world is very different. The task is a very different looking task, looking forward. A set of trends here that we talk about, I'm drawing heavily from our so-called vision for ending extreme poverty. This is a document that I worked on a lot with colleagues throughout the agency, bringing the entire agency together to try to position ourselves with respect to this commitment. What's it gonna mean for our programming? And that will be coming out right before the younger meeting. We look at a set of forward-looking trends here, including ones that are clear threats to progress, including the changing security situation in the world and climate change, a set of changes that could go positively or negative. They are definitely opportunities. That's the rapid pace of urbanization in Sub-Saharan Africa, South Asia. The youth bulge in even now in Sub-Saharan Africa that we can look forward to over the next 15 years, a real opportunity and then a set of other considerations here, ending with technology, which certainly in my career of 30 years working in Sub-Saharan Africa, the mobile phone has been the single most profound technology, technological transfer outside of public health into Sub-Saharan Africa. And why is that? It's because it obliterates distances and distances are the key, are arguably one of the real key drivers of slow development progress in Sub-Saharan Africa. So let me try to get you a little bit into the weeds that we've had to try to negotiate in my agency as we've tried to position ourselves on this commitment. So first of all, let's suppose we abstract from all the complex multi-dimensionality and just focus on $1.25 a day, okay? So that's a target. And we're given that target by a political process. And so as Montek describes in his paper, our task as economists is to contribute to reaching that target, technical advice on how to get there. Well, big first order question for us and big first order question is, where do you place the balance between delivering consumption benefits to the poor, building the assets of the poor, and transforming the economic opportunities of the poor? That is one of the oldest questions in development economics. It was the theme of the 1974 book, Redistribution with Growth, which came out at a moment that looks rather like today in some ways, where there was a pivot towards basic human needs and towards the welfare of the household as a focal point for development assistance. Second question, what's the appropriate balance between external partners and host governments in financing or delivering benefits to the poor? That's not even a question you would have perhaps asked a few decades ago, but infused with the political economy and the institutional complexities of development, this is about who owns this goal? How does the state constitute itself as a legitimate deliverer of public services? How can donors partner with the state as opposed to displacing that process? Last question, what concept of inclusion should donors be thinking about in countries where the median voter is not poor? So if you look at countries that have the highest headcounts in the world, those are almost exclusively in sub-Saharan Africa, plus Haiti, but there are many countries where there are a lot of poor and yet the headcount ratio is low, let's say 15% or below. So the median voter is far from being extremely poor. So whose goal is it? We partnered to end extreme poverty. Governments have goals and we have goals and some of the deepest learning about aid effectiveness has been that when conflicts of interest are sharp, the effectiveness of aid is undermined. But if the median voter is not extremely poor, the domestic politics of any extreme poverty may be much less urgent for incumbent governments than they are for external players. And one natural answer is okay, we can go in there and get it done. But that then seems likely possibly to perpetuate the divorce of these very governments from their own constituencies, from their own poor. In that kind of environment, how have I construed my role and the role of the economists at the agency as we try to navigate these questions? I think one key role is that we are trained to think in general equilibrium and dynamic terms. So this is partly my macro background maybe, but I think it's part and parcel of how we think just in this presentation this morning, for example. And we're trained to think, to put the evidence high on the list and keep it there. So those considerations have driven us in our positioning towards acknowledging just how crucial broad-based growth is going to be to accomplishing the extreme poverty goals that remain on the table. It's driven us to acknowledge how deeply important institutions are going to be to making those accomplishments sustainable. And so our theory of change, which I'm gonna make the mistake of showing you a picture of, and I really was not comfortable with the concept of a theory of change as if you could have a single theory that would fit more than one place in the world. So this is a very encompassing diagram. On the top I have that picture of the household. It's multi-dimensional situation, but the household is situated within a system and the two fundamental pieces of that system are those two horizontal bars, inclusive and sustainable economic growth driven by effective governance and accountable institutions. So we have, in the end of the day, tried to position our agency, vis-a-vis this extreme poverty goal, in this way, making those two things absolutely central and then across those pillars, you can see a set of pillars or constraints that might be operative in different environments. Let me tell you a little bit about how economists have shaped our positioning in terms of programming on economic growth in recent years. So I've listed our programs here from top to bottom by their size. We are first and foremost a public health agency. We spend $7 billion a year on PEPFAR, the HIV AIDS initiative, and on other initiatives in the health area. Our growth initiatives are Feed the Future, which is a agricultural productivity program coupled with a food security dimension. This is our one program that actually targets a $1.25 a day poverty. And the structure of the program is designed to allow a reasonably disciplined structure of evaluation. So the Feed the Future teams choose a zone of influence, might be a million or two million people. It could be a rather large area. And they program to farmers, small landholders. So these are not the extreme poor actually. Most of the program goes to small holders who are above the extreme poverty line. But the design is structured to allow you to construct a counterfactual zones where you're not operating and have what economists call a kind of intent to treat assessment. In other words, how has the whole zone done in terms of its poverty outcomes, where the channels might be through the labor market. Not directly through those landed farmers who you're working with, but through their impact on the labor market, through their impact on local urbanization, for example. Education, in the education area we've made a very, very sharp pivot over the last five years or so towards learning and knowledge, towards measuring direct outcomes and away from measuring people sitting in classrooms. And there we've relied very heavily on the work of Lent Pritchett and others. The evidence is just very strong. And that is built into our education programs, letting girls learn and early grade reading. I'll pass through global climate change. The Millennium Challenge Corporation is not part of USAID, but is a big part of the US government's approach to growth and it raises one very sharp conundrum for USAID. The Millennium Challenge Corporation was introduced following a lot of the work by Dollar and Cray at the World Bank, by Paul Collier and others on selectivity and on the effectiveness of aid. And the notion that if the policy environment was weak, the domestic policy environment was weak, the effectiveness of aid was going to be low and in that environment, the traction of conditionality was also going to be limited. So it was an argument against conditionality and for selectivity. Go to countries where the environment is strong enough that you can have low conditionality aid and high ownership ex ante. So that's the model of the Millennium Challenge Corporation and the DNA of that institution really reflects thinking of economists. It's in some way like central bank independence reflects the thinking of economists about institutional design. It has the impact for us, USAID. I mentioned earlier that the geography of poverty is increasingly crossing itself with the geography of fragility and conflict. So the countries in which the poor increasingly live are not countries who are scoring high on the MCC scorecard. So the model that we have that is best suited to gold dependence and instrument independence. In other words, you get a politically driven goal and then you have the independence to choose the right way to go about it. That's the MCC model. That model is not really available to us as USAID where we're essentially constrained to a few kinds of growth program. I'll pass through Power Africa which is a response to growth diagnostic work. This is a technique that came out of Harvard. Many of you have probably even used it, which we applied in a number of countries in Sub-Saharan Africa and Power bubbled up as a constraint with regularity. So we've set up a whole program to try to focus on that. Okay, I'm gonna end, I have one more thought. One more thought to try to convey. Why all this work? Why all this sort of thinking about trying to open the game back up to broad-based economic growth, back to an institutional focus? And the reason is that we're not policy makers in the developing countries like Montec is. We're donors, we answer to our own domestic politics and we face ferocious pressures to target in a very granular way. And I'm just gonna quickly list those for you to give you a sense. First of it, first, most of the best evidence that we have these days coming from randomized controlled trials in the field and so on is not about the system properties, although we know it's the system properties that matter most of all. So the evidence in some funny way draws us towards small bore issues and away from the big ones. Secondly, I'll pass through that one. Other pressures. Accountability, this is a lie to my previous point. Very much harder to claim and attribute impact in these broader areas. Public service delivery, quite straightforward. Capacity building, policy reform, much, much harder. Two others I'll mention. There is a natural tendency for country programs to be spatially concentrated. You don't run your agricultural program even in the entire rural area of a country. So a country strategy is always gonna face a stage at which they make a choice about where in the country to locate. Well, the extreme poverty agenda pushes that choice very powerfully towards where the poor are. Well, no problem, but think about, for example, Feed the Future, our agricultural growth program. To date, we have chosen the zones of influence as the high potential areas with the view of having a systemic impact. Another alternative would be to choose the areas where the poor are concentrated. How do we decide in our placement? I'm going to end there. I have one last thought for you. Maybe we'll have a chance to discuss. And that is that the simplest way of thinking, the most natural intuitive way of thinking about solving the $1.25 a day poverty problem is to go out and make sure everyone has a $1.25 in their pocket every day. How much would that cost? Well, calculate the global poverty gap as a billion people, less than that now, but a billion people in 2011 who are below that line. That's PPP adjusted though, right? So those dollars are way cheaper to buy in actual US dollars. It'll cost you about 70 billion, half of the annual aid, just to get everyone right up to $1.25. That's a challenge, I think, and a metric that we're going to have to be testing ourselves against as an aid agency over the coming years. Thanks very much. I think our speakers of the afternoon have done a wonderful job of touching on various dimensions of policy dilemma. How can policy make a difference? And how do economists feed into that process? All three have been at grips with the question of how economists can make a difference. So please raise your hands. When the microphone comes to you, we hope it'll be working. Identify yourselves and please be really brief with your question or comment. And we'll take two or three and come back to the panel rather than play ping pong. So I see a gentleman here in the fourth row close to the center aisle. Thank you, Rolf van der Hoeven from Institute of Social Studies and also having worked as an international servant for the ILO. Very quick question to Stephen Durkin about the policy brief. I completely agree with him but I want to ask he is also on both sides of the fence. Politicians always ask for a policy brief. So how do you square that circle? Because if you don't provide the policy brief, you don't get funding for your projects and your projects are put aside because you're not policy relevant. So since you also have both sides of the fence, I quite like to have some practical advice to that. Thank you. Thank you very much. I saw a gentleman on the aisle here around the eighth row. Could you keep your hand up please so the microphone can find you? Thank you. Yes, hello. I am John Key from Mauritius. And first of all, I fully realize that you know, economists in developing countries really have a hard time. They have a hard time in policy making especially when you have to deal with accountant, when you have a financial secretary who is an accountant and secondly when you have a minister who is also an economist. It becomes very difficult if you are an economist and you're going to policy making to really to deal with two types of people. One, accountants who like to balance accounts and secondly you have to convince a minister when he was in opposition, he was listening to the economist, then you were a good economist but when he becomes a politician, when he becomes a minister, it becomes very difficult to convince him. And I myself have been in policy making. First of all, in academia, in all universities you know that there's a big discussion between two departments. One is the management department and the other one is the economics department. Economists and management people are always at daggers. Then secondly is that you... Could you abbreviate please? Yes, I'll just, my question would be basically how to deal given your experience and your wisdom, how should young economists in policy making to deal with accountant first of all, they want to balance the accounts and secondly with a minister who is a minister of finance. Thank you very much. Great, thank you. I see a hand at the very back there, close to the staircase. Could you keep your hand up please, sir? Thank you. I'm Alan Winters, I'm actually one of Stefan's predecessors. A comment actually stimulated by something that Stefan said, but I'd be interested in other people's comments. I mean, it's a little bit like Ralph's. You said, Stefan, that we too readily drop into giving policy conclusions. Two questions follow from that. One is the sort of question Ralph asked. So what are we as academics supposed to do when you plead and make it a condition of grants that we have impact? How are we actually gonna deal with that while recognizing the importance of the points you've made? The second consequence of your statement is what really constitutes sufficient evidence for us to make any policy statement at all? Is it the case that every single difficult question that comes up, we have to say, sorry, minister, I can't comment because I don't have 95% certainty, you decide. Is it not the case that what we need is not so much to back out of policy debates, but to be really clear as it were about what we know, what we half know, what we hope, and so on? And if so, how do we communicate that without essentially just becoming derided by the people who really believe that they know as you started off with? Great, thank you very much. We'll come back to the panel now, but while we're doing that, it would be great if some of the younger people in the room could think about things they would like to raise and hear from the panel on. So over to the panel. Shall we go in reverse order, perhaps even first, then Stefan, then Montek? Yes, so just on the first question about dealing in the policy arena with non-economists or accountants, maybe for that matter, my sense is that, let me quote Luol Deng, who was a minister of finance in the Sudan. He said, the first obligation of the economic advisor is clarity. And I have found that to be an opportunity as an advisor that we are essentially teachers. Look at Justin's presentation this morning, for example. A complex system was described, but you didn't need a PhD to understand it. That is a standard I think we can aspire to. So partly in my work, for example, during the Ebola crisis, where there was a real fog of war feeling. There was no information on what was going on. There were projections coming out of the Center for Disease Control that were just taking an exponential and extrapolating it out in terms of infection rates that simply trying to communicate back to policymakers, look, the dynamics of this economic effects, they come down to exit by a set of individuals, social distancing by the ones that remain. And then finally, the set of instruments we have control over which is improving the health system, essentially improving the handling of the epidemic. And that kind of clarity, I think, is something that even if we're not taking a position on a policy issue, can allow policy makers to move forward with a little more confidence on what's important and where to focus effort. I think that very briefly on what standard of truth to Allen is really required, one angle on that is that rejecting falsehood is something we ought to try to do confidently. So this is along the lines of Montex 10 things to avoid. And there are lots of opportunities for that, I find. Just lots of consensus bad ideas. So great, and it's quite following on from what Steven says. So actually what I found most powerful is that actually politicians get very worried if you tell them the thing you propose, all the evidence says it won't work, in a negative role of evidence. And it's rejecting the falsehoods, the kind of thing. There's a really important negative role of evidence in the sense of actually stopping them. I would know definitely in the UK system is that there's very few ministers would dare to do this if they say all research will say that actually this is the wrong thing to do. Because with public opinion, there will definitely be then many of them in newspapers and whatever the process is. But that is definitely a role. It's sometimes harder, and as economists, we find we're very good at explaining the past and we're terrible at the future. So we don't really have that much often to offer and to say this is definitely the next sequence of things you need to do. But then it's again what you refer to and actually what Alan is talking about, being very, very clear about what we know and what we don't know. So we shouldn't be too readily behave as if we know. Sometimes you get economists who will present us truth, something that only exists in the theoretical model without any evidence ever. You know, we have to be so much more careful and so much more humility about it. But there is still a bit of the elephant in the room there in terms of what shall we then do and so on. So I tend to think now that our social scientists say, and that's in answer to the question on the policy brief and the kind of things. I'll start from where I ended in my talk is to say the importance of good senses work and to actually learn to value this again, not getting your most junior person to write actually the summary piece, not let your comms people write your briefs, but actually be all over it and thinking, okay, what do we know here now properly together and actually see whether with other researchers with, you know, reputation is strong, metallurgical song, young, old, whatever, actually see, you know, is this a good definition of what we kind of know about this and what we don't know. And actually always starts from there. So synthesis work is important. So the policy brief, the one that I would read is the one that is extremely clear in to say this is how it fits in the literature in terms of the other evidence. This is what contradicts these things or it's consistent with that. And doesn't try to say, look, I found an example where, you know, Quality Journal of Economics will publish it because it is quirky and it has something new and something funny. It could be totally irrelevant. A policy brief should be actually, can we tell it how it actually fits in the rest of the research and is it enough to form an opinion on? So those who are involved, say, in RCTs, those come more from science backgrounds. They use systematic reviews, meta-analysis. I'm not going to say that it's the only truth. I know as different we invest a lot in these, but I think in general in social science and in more of the economics maybe that's present here, we need to find ways of actually bringing these things properly together. Having protocols for writing survey papers, having protocols to write synthesis work, so there's a discipline about it that actually can be questioned and critically assessed as well. Economics Journal, Journal of Economic Literature is typically an article, you know, one person's grant idea about a topic. Journal of Economic Perspectives is at least on us. It's just an economic perspective that is quickly written and you get it published if you know the editor. And so on, so on. So I would say, look, there's things that we can do. It has to be trying to be more systematic understanding what do we know and don't know and presenting it and communicate it. Very quickly on Accountants and Economists are your bosses. My Secretary of State, my boss, is an accountant and an economist. I feel with you, I understand your troubles. My head of the Department of Permanent Secretaries and Accountants and Economists as well. So anyway, look, you, what I would say to young economists and I say to in our organization as well, don't try to, you know, try to show off but also try to think why does the politician ask that question and try to actually take them along and then do it. You know, they are, I find in my dealings with them, you need to understand why they ask the question. That's the hardest thing. And if you get a sense of why they ask the question, you can start actually to get some answers to construct it. It does mean understanding organizations, understanding the politics. Let's not be the technocrats that think that Varian 101 will offer the solution to the question that they're asking. Thank you. That was very, very good and very interesting questions. You know, first of all, on the issue of whether a minister is an economist or an accountant or a lawyer, I really wouldn't bother about that. At least in my experience, what you need to be sure about is the guys open-minded and willing to listen.  I mean, actually are willing to listen. But you have to think of three different situations where you're giving advice and it may or may not be accepted. One is where there's a predetermined position. I mean, the guys just come in, he's won a huge mandate on the program saying, I'm gonna do X and somehow you think X is a bad idea. That is a very tough one because in the logic of democracy, if he's actually won an election, promising to do something and he has the moral and the political right to do it. And I think what you can do is tell them, well, listen, why don't we do some damage limitation and let's find out how you can meet your promise and yet not do too much damage and most of all, leave yourself a line of retreat open.  So I think that's one kind of situation. The other kind of situation is, which is not connected with a predetermined promise but you suddenly have a crisis and that can often happen. Now, you know, I had a boss who's neither an economist nor an accountant and he had a wonderful plaque on his table which I think brought out some of the dilemmas that you face in that situation because the plaque said, this is a crisis. We must do something. This is something. Therefore, let us do it. Very often, the most important thing for a politician faced with a crisis is people, the TV comes along and says, what are you gonna do? It's simply not possible for the person to say, well, this is a very complex issue and I'm gonna think about it or I've appointed a committee. I mean, he's got a response. I think the best thing that the economists can do is to suggest a response that indicates that they care about some of the basic stuff but leave them a certain amount of room so they can think about what to do. And you know, if I had my, I mean, I wouldn't care about, if I had promise, I mean, my wishes, I wouldn't be wishing, oh, I hope that the, whatever, the finance minister or the planning minister or the prime minister is an economist or a lawyer or an accountant. I mean, the only thing, we obviously want the person to be intelligent and open-minded and so on. I wish they all knew the law of unintended consequences. It is single most important and it isn't actually found in any textbook. Because you know, if you go to any minister and you say, they say, well look, there's a shortage of whatever medicine or this particular new medicine has come in and it's overpriced. So let's impose price control to make it affordable. I mean, they just have to know that the consequences of that are one, two, three, four, five. So this can be applied across the board. Most bad ideas arise because people do not take into account the law of unintended consequences. Now, you know, it may well be the case that sometimes a politician is aware of it but nevertheless feels he wants to do it. Well, then I think the economist is exonerated from blame, I mean, you can't stop politicians who've come into power from doing what they actually wanna do. But your job is to make sure that you've told them what the consequences are. The last category, which is actually the most important, there isn't a predetermined position. There's a tendency to want to do something and then you just have to orchestrate a consensus within the cabinet, within the government, within the outside world and so forth. And you know, in my view, that's what I meant that a very important role for economists is to be creating an informed debate. I mean, if politicians knew that whatever idea they're gonna come up with is gonna be subjected to a lot of scrutiny by other informed people who may not be on their political side but have a viewed with respect in society, then they would want to know that listen, this is what I wanna do, but what's X or what's Y gonna say? And I mean, you can maybe do some balancing. I really don't think in the case of economic policy, I gave the example, it's not a chap choosing a cardiologist because he's not prescribing for himself, he's really prescribing for the public at large. So it's a very different thing, but the bottom line is a huge amount of weight has to go on the ability to persuade and maybe a little bit on the ability to obfuscate when there's a really bad idea. There's no harm in throwing some ink or smoke or something around it. But it's ultimately persuasion, I think that matters. Thank you, it's hard to follow these three on that issue and I'm not speaking as an economist because I'm not one, but I worked for three prime ministers and a number of ministers. The first thing I'd say and we've heard it already is try to make them feel you're interested in them and their view, that it's not just your view that matters to you, it's harder to do than you might think because they don't always want to speak to their advisors, but if you encourage them to tell you, you can often figure out a safe way for them to do something rather than the risky way they're thinking of undertaking an initiative. The second thing, and I thought it was very important, is to the extent possible, do your own work. And if it's not your own work that you're founding your advice on, make it your own by thinking it through in your own terms.  So those would be my two words of advice there. And finally, just to speak of somebody and many of you know whose boy Friday I was for six years, Sylvia Ostry, who used to be on the board of Wider, having worked for her for six years, I never knew her to utter a word she hadn't written herself. Now, our last round, and we'll go to this corner of the room, sir on the aisle behind the black shirt. This John Rand University of Copenhagen. I found a little bit inconsistency between what Stephen Durkin and Steve O'Connell said, and I'm referring to the discussion about a defectiveness. Stephen, you emphasize that it's important to have these synthesis reports covering the whole literature, trying to get an informed opinion. At the same time, I hear Steve O'Connell referring to the Millennium Challenge Cooperation and focusing on very specific, very specific literature within the defectiveness literature. So focusing on the dollar burn side work, showing that it's good governance, but we have a whole other field. We have a lot of well-published articles, even a lot of synthesis work in recent journals that actually emphasize that it may not be as effective. So is there a disconnect between your two views on this topic, for example? Thank you, there's another hand back there. Thank you, yes? Yes, thanks, my question is also to Stephen. I think it is more about demystifying the classification of certain African countries as a few types of animals. I mean, the loafing hippos, the hyenas and the tiger fish, I think I'm just more curious on what are these countries doing wrong to deserve those titles? And if you can assist a little bit in, it's just at least giving us some color on what would be necessary or what needs to be done to at least graduate them from this loafing hippos into the Asian type tigers, thank you. Great, one last person, I'm looking at the back because it often doesn't get much of a chance. Sir, there, I saw your hand up behind the gentleman who was speaking just now. Thanks, Sean Muller, Parliamentary Budget Office of South Africa. Could you speak more loudly? Yes, Sean Muller, Parliamentary Budget Office of South Africa. I just, I found that my own experience correlates rather well with the statements made by the panel, I was a sort of young academic economist, I did my PhD on the limitations of drawing policy conclusions from randomized trials and then I went to work in the Parliamentary Budget Office and what we do is try to advise politicians in a non-partisan manner on public finances and economics. And I felt that there were three things that prepared me quite well for that. The one was sort of a healthy skepticism about what economics can achieve based on some reading of the methodology literature that typically, as young economists, we don't really get taught. The other was following local politics very closely. And the third was good advice from a senior politician who was retired at the time that we should not be too grand and just think of ourselves as tutors. But just the one sort of question I want to post to the panel, linked to some points that have been made. The incentives for young economists, we're creating incentives for young economists to oversell their work. We see this in the award of, for example, the Bates-Clarke Medal. So on the one hand, we hear from people with experience a good way of approaching these things, but on the other hand, the incentives for young economic economists are really to oversell their work in potentially quite an irresponsible way to policymakers. This is my view and I'd be interested to hear the panel's comment on that. Thanks. Great, thank you very much. I saw Jerry Halliner's hand up so if you could be brief, Jerry, we'd love to hear from you. It's really a related point. I wonder if the panel could comment on how wider or other institutions could more effectively train young economists, build capacity in a way that's rather different from the way in which our current doctoral programs and economics function. Thank you. So why don't this time we start with Montek, Goat Stephan, and then Steve. Montek, please. So the questions were addressed to others, right? That's okay. No, let me just address the last question because the other two questions were specifically addressed to the panelists. You know, I mean, I already said in the case of wider that, for example, some of these controversial issues that will swirl around like the sustainable development goals, et cetera, I mean, they won't necessarily be of much interest in the academic world. For example, I made the point that if you have 169 indicators, the usual economists approach would be you must be able to give some weight to each of these and produce an objective welfare function of some kind. And we know also that politically completely unacceptable. So what do you do? Well, I think you need a non-rigorous but public understanding of, you know, how to handle this, which indicators really good, which indicators are not so useful. And that's not the kind of thing that a graduate, a department of graduate studies would actually produce. But maybe something like the UNU could take a little bit more interest in it, but I don't know. I mean, that's certainly one thing. Second, I was mentioning to David that, you know, I think wider could do a lot more sort of conferences and so on in universities or in think tanks around the world, not based only in Helsinki, but provide an opportunity where both professionals, academics, practitioners, et cetera, discuss some of these issues. And if these things are made more publicly understood, which really means using the media, then I think we would end up maybe making our policymakers a little bit more aware. The last is a gratuitous throwing in because it reflects, I think, some of the concerns that people have. I mean, young economists go into something and obviously they want to make an impact. I think the worst thing about governments is that they're all organized in silos. And each government by department or ministry by virtue of its name defines a turf area. And yet if you look at it as an economist, I mean, the instruments that have to be deployed to achieve that objective are not in the control of that ministry itself. And I think this is one of the biggest problems. Let me just give one example because there was a lot of concern in India that child malnutrition, I mean that's one dimension where we are not doing well. Though fortunately the latest data suggests there's been a big improvement in the last 10 years. But ignoring that, we're all concerned about it. And we made a presentation to some of the key political leaders, I did, that if you look at what it is that ensures that children are going to be well nourished, it relates to a very large number of different ministries and sociocultural practices. I mean, kids are gonna have low birth weight if the girls are married off too young and have babies too young.  I mean, there's no policy that's gonna change that. You've gotta somehow change the mindset of society. That's point number one. Point number two is of course when income is inadequate and that there are a lot of policies that aimed at improving income. Point number three is the state of health of the kid which is affected by vaccination, which is affected by the cleanliness of drinking water, which is affected by the availability of sanitation. These are all things outside the Ministry of Women and Child Development which actually has the nutrition programs which are widely seen to be the way to combat malnutrition. Then you've got issues of educating the women in how to bring up children. Early breastfeeding, exclusive breastfeeding for six months, taking the child to the breast within two hours. Now these are things that they are not known and are very frequently contrary to the long established practices. Again something that's not done by the Ministry that does supplementary nutrition. And yet in the government, whenever they thought about why do we have malnutrition, the answer would be let's have an evaluation of the Integrated Child Development Program which was essentially a program for supplementary nutrition. So you know, I mean an economist would go mad and this can be, this is replicated throughout not in as extreme a way as this one. So really if you want a sensible, mutually supportive policy, you really need to get a large number of ministries to act together. And as anyone who's worked in government will tell you, getting ministries to act together is actually most of them regard as an interference in their constitutional independence. So I think economists somehow have to rely on external public pressure to make sure that government takes that kind of a consolidated view. Great. Thank you very much on tech, Stéphane. Right. Well if you came my way, just let's first just to actually give an answer to the Tiger's question that you said. And just let's be very clear, you know, I wasn't, what is the point I partly tried to make is that you know, if we, sometimes we do things like SDGs and MDGs, we sometimes create the impression that just throwing money at them will fix it. We create an impression as if silver bullets will be there, very quick fixes and so on. So I want to just make sure that people working in international development business to be conscious that these countries are very different. Now having said that, I don't tell you how you can turn into an Asian Tiger, but let me at least say one thing I definitely learned working international development in DFID is that it's proven very useful for all our people on the ground, whether it's the economist or people from other disciplines, to actually reflect carefully and talk to political scientists, talk to people who know the country well and so on, to ask very simple questions such as to what extent do we have the sense that the elite in this country is actually interested, aligned, with trying to get growth and poverty reduction. And it's been proved a very helpful question because in all the countries we work in, after the 28 that we focus on, we could probably say two, probably where we say, look, this seems to be there's a fundamental alignment with what the elite seems to want in terms of growth and poverty reduction. I'm not saying that it's not to align their pockets, it actually turns out to be countries where we think, they have strong incentives, they want to get richer, they better grow this country and reduce poverty as well. But we have too many of the others. So I simply want to say here, and I don't know how you turn into an attacker, but it's probably a little bit what Steve was alluding to, that bottom block that we have, there are institutional barriers that mean that simple translating one into the other is going to be very simple. And these institutional barriers create probably the political elites, incentives and so on and so on. And so we're talking there. But with it simply, I want to tell my economists, be aware of the politics and the elite interests in your country you work on. I'm not saying it's all bad and so on, but I privately would be very happy to give views in terms of what you could do, but I don't think I have the time. I just want to say, look, let's be aware of it, that it's not straightforward and it probably needs to do, as far as I'm concerned, I think it's a lot to do with institution or settings, which are very hard to change. But we can, and we can talk about it privately. Just quickly then, there's still a little bit that I feel like also earlier, I didn't really give Alan a good question in terms of, of course, we want wider and research to have an impact. And Alan raised this, you want us to have impact. In a way what you're also asking the question, what can wider do more is probably also saying, how can we make sure that the people that work with wider or the young people that we do in PhD programs can actually have a better and be more effective in policy environment. So I would still say the first thing, it's definitely not by being less technically skilled. I do think that we have something to offer there that actually a good economist that can think through trade-offs that can be on top of technical issues and weighing the trade-offs and the incentives and understand these things, great. But there are more things that I think we can do in PhD programs. I mean, the minimum amount is, make sure people are open-minded. I find it striking when I hire people in different, I always have an interview question, maybe I shouldn't give it away, but one interview question is at the end of the interview, can you tell me the last time you changed your mind because of a piece of evidence? And the silence that you get from the other side of the table is killing me all the time, actually amusing me as well. I've maybe once or twice got a good answer, actually with people that trained in medical science who have more tradition of actually changing their mind. Economist, social scientist, they're dreadful at it. We are just dreadful and we are often in our ideas and I don't think, think about it now. When did you last change your mind because of an article you read by another researcher? Try to get students to be open-minded in that sense. Know about different things of doing and be open-minded in general. That is a few things. We've been selling Oxford a bit too much. The typical thing that Oxford always says that's looking for its students, people that can think on their feet. We want people to be able to think on their feet to be articulate. I wish in PhD programs we paid more attention on writing articulately about your science, about what you're doing. And then the synthesis work, make sure you learn and teach people to do this rigorous synthesis work. When did you last have a course in university on rigorously writing a synthesis paper? I don't. Steve. Sorry. Last word. Okay, so first to respond to John Wren's concern about whether there's a disconnect here. I think I can put you at ease there. So the point about synthesis being the principle of what economists can bring to the table on a complex issue I completely agree with. And that was a lot of the fights we had in developing our extreme poverty vision were with a bureau, let's say the Democracy and Governance Bureau, where we're writing the section of this document where we're assessing what's the evidence on democracy and growth. And so those fights were fairly deep and they were very fruitful by the way. The fights that take place within the civil service as opposed to with the politicians can be very honest and can get you quite a ways towards a synthesis kind of attitude. So there's not a disconnect on the MCC. I did cite Dollar and Cray, but the MCC was designed by the Bush Jr. administration back in the early 2000s. So that was right at the center of the Dollar-Cray dominance of the literature, right at the height of the disaffection with conditionality. So if I hadn't been behind in my presentation I would have given my plan shout out to wider's work on aid effectiveness, which I fully agree has had a synthetic kind of character and has moved the debate more towards the center. Two more things. One on the notion of whether young economists have a serious incentive to oversell their work. I don't find that too convincing myself actually. I don't think economists are particularly egregiously positioned on this. Lately we've seen really shocking looking results from psychology, for example, the natural sciences inability to replicate and so on. The antidotes are known, the antidotes from the public policy side are the sympathists towards synthesis, but in terms of the publication situation, pressure to attribute sources of funding to me is a much deeper concern. And at least in the American Economics Association we've made some progress in raising our standards in that respect, and then transparency of data and availability of data. My view is that the lift you have to do to get a paper published anymore, let's say in the Journal of Development Economics is horrible. I mean the scrutiny is way deeper than it was in the past. I think it's gone way over the top in some ways. So final thing to Jerry, Jerry's question suggestions for wider. I think that's a great, great question. Here's an old fashioned kind of answer. One is that economic history has never really managed to mainstream itself in the modern era in top economics departments, but I think that the good policy economists are imbued with it. I'll give you an example that when the financial crisis broke in the US we had Ben Bernanke at the Fed who wrote the classic article on the disintegration of the banking system during the Great Depression and a bunch of other work. We had Christina Romer at the Council of Economic Advisors who wrote, written some classic work on volatility of output going all the way back to the early part of the 20th century. These were people who were interested and imbued with reality in a way that grad school sometimes don't make happen. The other thought I have is that I have accumulated a fantastic admiration over time for the ODI fellowships. And when I was in graduate school at MIT I took a year off and that was the last time I was in government was with the Carter administration working for the Council of Economic Advisors. So if wider could put in place a kind of fellowship placement program that would encourage a grad student to take a year out there in the field confronting this discussion we've had I think it would be spectacular. Great, well I think you'll all agree with me. It's been a wonderful panel. We've also had very interesting questions but also comments from the floor. We could have taken many more except that we're out of time and we all need a bit of time before dinner to catch up with the outside world. Thank you all very much, Finn, to you and your colleagues, congratulations.