Washington, DC
DEPUTY ADMINISTRATOR ISOBEL COLEMAN: Thank you, Dean, for that nice introduction. And it is really a pleasure and a thrill to be here with all of you today, to kick off Learning and Evidence Month, couldn't probably come up with a more wonky title for something that I think is so incredibly important and exciting. I would say it is not a stretch to admit that part of the reason I am here at USAID, in fact, a large part of the reason I'm here at USAID, is to bring more evidence into programming. So, this month and this kickoff, to me, is super important.
And I'm thrilled to be here with Esther Duflo. I will also admit, I'm a bit of a fan girl. I have been following your work for a couple of decades. And in fact, the first piece I wrote in Foreign Affairs, almost 20 years ago, was citing work that you have been doing in India. And maybe we'll start there, just looking back over 20 years plus of the work that you've been doing on really bringing evidence to bear on what works. You know, you wrote the book, Poor Economics, you really helped so many practitioners and policymakers and others think about what works in global development, how do we make our programmatic dollars have the biggest impact? How do we do the most good with the resources we have? How do we stop doing some of the bad that has been actually proven now not to move the needle? It is part of the reason I went to GiveDirectly. I think we, as a world, are coming to the realization that much of the complicated programming that we do doesn't actually beat the cash benchmark. And so, how do you, when you look at the span of this evidence, the work that you've – this body of work that you've really helped drive, in this movement you've helped create? What do you think has changed? And how do you see the changes in action?
ESTHER DUFLO: Thank you very much. It's great to be here. Great to see my student making himself useful and about speaking in this great organization that is USAID. I think the landscape in terms of development policy, not just international institutions, but developing country governments as well has dramatically changed in the last 70 years or so. And it's a confluence of many factors. Some of them we – we, the global we – implicating both, all the organization is within what I call the movement or the ecosystem. So, that is Japan APN, USAID and DFID and now FDCO have played some role in and some that is broader. And I think that the sea change that we see is that there is much more willingness among practitioners and policymakers to be very pragmatic in the problems that they are trying to address. And therefore, once you accept the need to be pragmatic, then the ability to be evidence-driven follows almost automatically. So, to kind of get you in a sense, we switched from a world where the pressure implicit or explicit was very much a sort of Washington consensus type of view where what countries needed to be after was some amount of macroeconomic stabilities, budget, discipline and growth as the one other final objective and the way in which we measured the success or the failures of country.
And there was a number of issues with this way of seeing things, and our big one is that we have no very concrete advice to give to countries for how to generate quotes. We're pretty good at knowing what should not be done – hyperinflation, vigilant state policies, you know, we know it's not a great idea. But once you've had two fundamental rights, we don't know what else to say. And that had led to countries that left the different countries in somewhat of a bind, in the sense that they didn't get much guidance, nor did they have much of a view of what they could do. Progressively, the discourse has changed in emphasizing what I think makes much more sense, which is, the final objective should be human welfare. And of course, it can be courses welcome to produce human welfare, but it's a means to an end. And there are other means to that same end. For example, a good use of public dollars of all of these countries.
And so, this is what was driven in part by the Millennium Development Goals, followed by the sustainable development goals, and generally your shift in the international community and in developing countries towards these much more well-defined objectives. And once your objective is something as concrete as reducing maternal mortality, then, number one, you might have solutions that have been tried in other countries that you're willing to try. Number two, if you don't know what works, you have a way to find out because now the problem is well defined. And therefore, the way to acquire evidence for how to solve this problem is also well defined. And this has led to, really, I think it's this major shift in the way that many developing country governments think about their mission and how to carry out and think about spending their hard earned tax dollars, which is either, there is already evidence, and we are trying and going for that evidence, or we are trying new things, and then we should be open-minded about whether it works or not.
So, this thing has opened really a boulevard for well-meaning institutions like USAID for example, to become a partner in this effort. And USAID has very much led by example, in particular with the creation and the success of DIV, the Development Impact Ventures Fund, which it was clearly an idea of how to work in partnerships between local actors between researchers to produce evidence about what might be helpful. And then what now, what, in a sense, we need to do next is to how can the whole institution, not just DIV, be helpful in relaying what has been learned? Put a deep effort of tools to finally the effort of anybody who is funding or undertaking this type of project. This has become, I think, a much easier job now, because of this greater willingness that we see it just about everywhere, from policymakers, at the very local level, or at the national level, to engage with evidence, either as a producer, or as a consumer.
DEPUTY ADMINISTRATOR COLEMAN: Thank you for that. And you say it's become easier. But from where I sit, it's still a big challenge to get evidence into our programming. And it's something Dean and I are talking a lot about, you know, how do we make sure that our Missions are either using it, or we're creating it on the evidence front, and making the tools and the accessibility to evidence much easier. So, because there is a lot of evidence out there already, on what works, and yet, I feel like we still have a pretty big challenge here on bringing more of that to bear in what we are actually – how we are deploying our dollars. And I don't know if you have any recommendations for what more USAID could be doing. I mean, DIV is a great example, but it is a small part of our overall budget, which a lot of it is going to work that is tried and tested. But you know, a lot of our humanitarian response is in cash, and it's moved to cash because of the evidence, but I think they're big parts of our budget that are not yet rooted in evidence and how we get there. I just wonder if you have any thoughts from looking across what was formerly known as DFID and other organizations and how they've been doing it?
MS. DUFLO: That's a great question. So, a few years ago, I was fortunate enough to be a member of President Obama Commission for, I don't remember its exact name, but it was something about rethinking international cooperation. And, in this context, I got very acquainted with the work of USAID and how many people at USAID conceive of their role.
One thing that struck me at the time is that the first thing I tried to do was to get a list of projects that have been funded by USAID, across the entire agency. And this list was not available. So, I don't know whether it is now, it is not that they wouldn't give it to me, it's just that it wasn't there. So, my objective in getting this list was to go project by project and classify whether this particular project has evidence behind it, what kind of evidence, whether it's in the USAID bucket, or whether if it has no evidence, it is producing it in any form of a way, or – and then, you know, what fraction of the USAID budget is actually goes to things that we simply have no idea. So, I never got the answer to these questions because that list wasn't available. And it was explained to me at the time that USAID doesn't think in these terms of actually having projects. So, I don't know whether this has changed. And I'm not saying that it is a fundamental flaw necessarily of the organization since there are many objectives that an organization like USAID could have but it's certainly a barrier for reaching that objective of either you produce evidence or you use it If nobody really knows what is being done at every single level.
And I think that might relate to also, again, from the benefits of this sort of deep dive in USAID in the context of this commission. I think this may have to do with the way that we still conceive of the role of USAID. And maybe the way that we could think of shifting that role. From its inception, USAID is really about like doing things. So, you know, you travel in the woods of countries where USAID have a big presence and then you see a hospital and it says “From The American People.” And this is kind of what USAID is about, is to kind of help out or something like that, in ways large and small. And I think this was a fine way to operate when USAID was created and at a time where some developing countries really had a huge shortfall of funds to just operate their basic day to day life. But things have changed a lot since then. There has been economic growth in those countries, fortunately. Even the poorest countries in Africa are not as reliant on cooperation from the entire world as they used to be. There is also competition for USAID or FTDU or Agence Française de Développement in the kind of the newly emerging countries; China, of course, first of all.
So, the role of – I think it's really important to think of now to rethink what should be the role of USAID? What should be? How do we maximize the impact of our tax dollar? And a first answer could be well, we make sure it's used on useful stuff. That's great. That's the first good thing. But I think even better is to think well, what is it that USAID can do that nobody else can do? If I put money into a great project that has good evidence, that's fine. But could the country have done it? Is it something where I'm replacing the action of someone else or not? And in some cases, the answer might be yes, nobody could have done it, for example, because the country really doesn't have the resources. Or in some cases, it could be well, you know, maybe we could have convinced them to invest their own money in doing that, because, you know, represents such a small part of the budget anyway.
So, in that case, if you think about where do I get the maximum leverage of the taxpayer money, it's actually you want to go one step further, it's not just doing things that are helpful and not wasting money, but it's also using these dollars to leverage more dollars from the countries in going into the right place. So, if we think about it that way, and everybody in the organization gets to rethink their mission that way, then the whole mode of operation becomes different. Because then you're thinking, okay, what if I spend $1 now, does it change something beyond the dollar, even beyond the people who are helped? There are various ways to do that. One, of course, is humanitarian aid, where basically USAID plays the role of the insurer of last resort. There is a crisis. We need to act very quickly, in order to help people out. This is something no one else can do because no one else has access to, you know, the ability to quickly expand the budget in response to an emergency.
So, if we take out the COVID crisis, if, you know, if it had been me in charge of the world, I would have dramatically expanded the amount of money spent not just by USAID but by other countries, simply because we were spending 27 percent of our budget on fiscal support. At the same time, the poorest countries were spending 2 percent of their budget on fiscal support, because they have no ability to grow and we have ability to grow – this is a place where we have unique ability.
And then of course, once you've raised that money, if you raise it, then you use evidence to spend it the most effectively possible. Cash transfer being a great example. That's the first one. The second one is generating evidence. Because generating evidence creates public goods that other people can rely on, so go for these things that are untested but test them. And the third thing is lead by example. So, demonstrate how – and so this is how the using the evidence in the context of this USAID mission – it's not just good because the dollars will be well used, it is good because you can demonstrate to the partners’ country how this can be done. So, I think if we manage to rethink the role of USAID in this way, if everybody sees themselves as this kind of mission of maximizing the leverage, then the inside work might be easier.
DEPUTY ADMINISTRATOR COLEMAN: Thank you. You've mentioned DIV a couple of times and I know that you were instrumental in starting up the French Development Agency's version of DIV, the Fund for Innovative Development. When you look at DIV and FID and think about some of the successes that they've had, what lessons should we learn from that? And you've also talked about trying, you know, of course, trying new things, and really putting money into initiatives that others wouldn't necessarily fund, which is really what's behind DIV and FID. And we have a big push here at the agency to do more local work too, to really focus on localization. And I'm sure there's an opportunity for us to be testing out more innovations that are created and run locally; you know, that’s they – people on the ground who were closest to it can see what really needs to happen. And, you know, our ability to put money into those initiatives to test and I know that you have been very engaged with Pratham over the years, I think we were talking earlier, in the earlier – Dean was in the earlier session about teaching at the right level. And the things like this that are simple interventions – nothing's really simple, but lower-cost interventions to drive to better outcomes. Hopefully, DIV can show the way. But how do we scale it? And what lessons should we learn? And how should we think about localization? Is that an area that's ripe for much more investment in innovation so that we can really help drive better outcomes at the local level?
MS. DUFLO: Yeah, thank you very much. So, DIV is really a great example of what I was saying about maximizing the leverage of the dollars for many reasons. So, first of all, the DIV team and Michael Kramer did a calculation of the social rate of return of the DIV portfolio based on just the first few years of what DIV invested in, and then the subsequent scale up of those interventions.
And what they found is that a reasonable fraction of this project, not huge fraction, but a reasonable fraction of this project, went to be adopted very widely and scale up very widely to a variety of models, that is, by governments, by the private sectors, funded by international cooperation or not. Some of them, several of them, reaching eventually millions of lives. And then when you value these investments even in a very conservative way, this is much greater than the entire portfolio of DIV. That's the first direct way in which DIV had high – super high leverage.
The other is that it piloted and showed the proof of concept of a radically different way of doing things, which was not staff-driven but locally-driven, where have people from the local area come up with their innovation, which was insisting on – which was not insisting on proving you have something that works. But you have to choose between either you have something that works that you want to scale up, or you have something that you don't know that you want to evaluate. And the ability to work with a vast variety of completely different actors that had been absent from the USAID portfolio, including local organizations.
This model was then applied by others. First the Global Investment Fund – Global Innovation Fund, sorry, out of London and then DIV that literally in France – I’m sorry, the FID in France that followed the same model, but basically made it available suddenly to part of the world in particular in Francophone Africa that had been left out not by design, but simply because everything is in English. And therefore, it was difficult for people in Francophone Africa to apply.
What we have seen in FID – for full disclosure, I'm the chairman of FID, so I'm not an objective leader here, but what we have seen in FID – is that we immediately got deluged by applications from countries which had never really been in the innovation ecosystem before. In particular, in some of the poorest regions in the world, the Sahel region in Africa or regions which are, you know, in very difficult situation, either due to climate shocks or due to conflict. And we got great applications from local actors in a way that was completely new for the [inaudible]. So, this model is just – the DIV, then FID model is not just about, you know, being rigorous about evaluation, but also being open about local actors and being very supportive.
I think we can do more there. And in particular we can do more to work on helping out local researchers, either in universities or in think tanks, NGOs, et cetera, and a member of, really the civil society at-large to take ownership of this movement themselves. Something we've done a lot in FID is to do summer schools and supporting researchers, giving tiny, tiny grants, which we call stage zero grants, to make sure that there actually is a pipeline of entrepreneurs, researchers, bureaucrats, activists, who are kind of engaged in the movement. This is kind of the other part of localization is unique, once the money is available in a way that is open and flexible, which I think has really been done very successfully by DIV and then FID, is to make sure that the bodies are available but the bodies are available, they are great, talented people everywhere, but we need to find them and support them. And this has not necessarily been a priority before and needs to be in the future.
DEPUTY ADMINISTRATOR COLEMAN: We have only a few minutes left before we go to Q&A. So, I'd like to just ask you one last question as the Chairman of FID. Maybe you could reflect on an example of something very innovative coming out of the field. You said Francophone Africa that has really kind of been off the beaten path, right, for a whole variety of reasons. What's something that has come to FID that is really exciting to you as an example of innovation?
MS. DUFLO: Yes, absolutely. So, there are several that I would love to highlight. Let me take the opportunity to say one thing to sort of emphasize one thing that you've mentioned before, which is the simplicity of the interventions. So, that's not necessarily – one of the issues that we have, in many development cooperation is the temptation to do what I call the cake that is 100 times good.
DEPUTY ADMINISTRATOR COLEMAN: I call it the Christmas tree; everybody wants to hang ornaments all over it.
MS. DUFLO: Exactly. So, there is this little story for kids in France called a kid that is 100 times good, where some kids are trying to make a cake, and they're putting everything they like in it from, you know, pizza to chocolate and so on. And the result, of course, would have been very bad for them. But some horses coming and eating the 100 good cake and are sick as a result of it. So, that's how the story ends. And then the mom makes a very simple apple tart or something like that. So, I think there is a temptation for the 100 times good cake. And that makes evaluation of course very difficult because you don't know what causes what. It also makes the implementation very difficult. And the tracking of implementation is very difficult.
So, again, going back to sort of the basics, and what is it we're trying to achieve at any given time? What's the outcome? What are we trying to do? What's the best way to get there? And the part time intervention and generally the whole pattern philosophy of okay, it's very simple. Kids should learn how to read and count by the time they finish grade five. Wait, can we focus on this one thing? I think it has really been, it seems, like, obvious but it's a huge shift in focus for many developing countries that are trying to achieve so much more in schools and achieve almost nothing as a result.
So, in terms of – so jumping on that for a sort of example of simple but powerful things that came in FID. At a very, very early-stage beautiful program out of Kenya is using hairdressers for mental health counseling for women. So, women spend a lot of time in hairdresser to have the time to be prepared. And then if the hairdressers are trained in providing mental health counseling, that can be a way in which this kind of a fast triage solution for women with mental health problems. We are in the process of trying these out and evaluating them at a larger scale. But that's like an example that's really exciting.
Another one is in the agricultural sector, is a startup out of Morocco, a commercial startup that is trying to put solar powered fridges in the ground to preserve the cassava harvest. Because cassava actually gets degraded over time. So, that's a very, again, very simple problem with a very – with a simple, pretty robust solution that they are evaluating in several countries, including in Togo.
Another one, I guess again in this area is the idea of putting back land in – very, very degraded learning cultivation via creating, it’s called half moons so it's creating this kind of seawater that doesn't cost any money. Farmers were trained, it has a huge takeup after the training. So, that's another great example.
At a completely different scale, and since we have Robert [Osei] from Ghana here, we are working with DIV to help scale up work, an intervention called Lively Minds, that helps parents engage with young children from the very beginning of their life. Again, very simple idea that you need to talk to kids earlier that parents might not know that has been evaluated now repeatedly, including in Ghana is ready to be scaled up. The government is very excited in taking it on. But what they needed was a transition period where people could be trained where pilot can be run at a much larger scale, and so on, and FID and DIV are working together and getting Lively Mind adopted in the whole of Ghana. So, there it's collectively small budgets that are going to be able to leverage tremendous government resources behind this program.
DEPUTY ADMINISTRATOR COLEMAN: So, I'm sure you could go on because there's so many great examples. But we're going to turn it back to Dean for some Q&A from others who are participating. Dean, back to you.
DEAN KARLAN: The question that came in the audience was actually about firefighters. Now, it could be that I don't know about a different thing you said about firefighters. But for whatever it's worth, I think that actually does create a nice dichotomy. So, I'm going to ask about both plumbers and firefighters and whether your answer changes if it's a firefighter versus a plumber. Could you elaborate on what you meant by the reference to plumbers, and then how that might differ if we're firefighters.
MS. DUFLO: So, what I meant by plumber is that the policy work involved is too often conceived of as a big conceptual exercise where you have to get the right idea in order to get something done. And sure enough, we need some ideas, but that's not sufficient. These ideas need to be implemented on the ground. And then you mentioned earlier the behavioral economics idea, both due to behavioral economics idea, both due to the fact that we can as human beings and the details of the implementation of the programs often end up making a huge amount of difference. And it's kind of nobody's job to pay attention to those details. The simple things like how difficult or easy it is for someone to apply to a program. The more boring things like what is the fund flow for programs and the authorization which you can kind of tough one would be like a design of the tap idea for a plumber and the second one is more like the sewer issues of people how to the money goes through the cables. So, this is what I call plumbing is those design ideas where we really don't know until we try. So, we have to try something out, see what it gave us, try to understand why or why not it work without the benefit of a very clear, you know, theory behind this or even what we should be looking at. That's kind of what makes it a plumbing job, in a lot of cases.
Firefighting is maybe relates to – that's not my – to my knowledge, it's not my image, but I'm happy to elaborate on what I – how I understand it. It correlates to this idea of maybe acting as insurance in case of disaster, which is you have to act support very, very quickly. You do your best. And again, I think firefighting would benefit a lot from the previous work of rebels, that would give us an idea of how the thing needs to be run.
I'll give you an example. In the case of cash transfer. One thing we didn't have when the COVID crisis hit is in a lot of countries, a set of channels by which we can deliver money easily to people. And if we had that, which is really is a plumbing job, because it's literally getting, setting up the pipeline that reach every single person to a potential transfer when the transfer is available. Then when the crisis happened, the firefighter ideally would be able to push a button and the money would fall. So, something which I really would hope that collectively we can work on. And that's something like potential loss and damage fund for climate. It would be used for is to create pipelines for everybody to be within reach of a cash transfer, which clearly is a plumbing job. But then the firefighters could use it and say, well, you know, there is a drought in Niger, in this one, upper corner of Niger. Why don't we launch cash transfer for people who live there? We are ready, we know how to identify people, we know how to target them, we know how to move forward. So, this is how the two work together, we scrambled thinking about how I can make the job easier and faster for the firefighters, and so they don't have to think about reinventing the wheel in the middle of a disaster. And the firefighter is ready to use those channels when the need arises.
MR. KARLAN: Great. And you didn't mention it, but one of your papers from Morocco, I think also speaks very well to this where just labeling cash transfers without doing any conditionality actually generated some of the same effects you would get from conditionality in terms of the way the money gets used.
MS. DUFLO: Yeah, so that's an example of simplicity, I think it's the conditional cash transfer where a little bit of missing good cake. And they were evaluated that way so they kind of spread that way. And it took a while to understand whether or not everything part of the cake was needed and in what form. And it is possible that something much simpler, but [inaudible] is sufficient, which of course would make it more practical to use in an emergency situation. In other settings like the, you know, the work that you've done, on the [inaudible], we do need the whole shebang, it is not sufficient to give people an asset, you give people an asset and the means to use them the most effectively possible. So, it's not always an answer that you need to go for simplicity that depends on the program, but it often is the case.
MR. KARLAN: So, I think we have time for one more question, powers that be, or no?
DEPUTY ADMINISTRATOR ISOBEL COLEMAN: I think we have time for one more. And with a pretty short answer. I think we've got another one.
MR. KARLAN: Okay. Well, here's the question I'll compose. And I don't know if it's a short answer, but I have confidence Esther will do. So, when you're talking to a minister of anything, how do you decide which foot to put forward? Here is strong evidence to use for the issue you're facing, or gosh, you could really benefit from producing some evidence yourself on this, the evidence from elsewhere is lacking. Or, you know, how do you decide where to go on that spectrum?
MS. DUFLO: Well, it's completely driven by what the minister that I’m talking to comes with as a question. It's really, I don't believe in substituting expertise for the democratic process. And I don't believe in substituting my expertise for whatever it is a minister has to tell me. So, that's kind of a combination of – that it depends. And also, I would say a one-time conversation with the minister can be of some use. But what is of much greater use is collaboration.
An example we have is very deep and long collaboration with the state of Tamil Nādu, in the south of India, where it's really over time that we've accumulated evidence, they've produced it, they've used it, et cetera. I have a very close working relationship with the Minister of Finance there. And it's really made of those two aspects. In some cases, if he asks me a question, I'm able to tell him something that I already know, or sometimes he doesn't know, and we work together on setting up a project.
So, for example, they were asking about how – whether we knew anything about how they could raise the tax revenues for money that is owed to them, but they haven't collected. And there was some amount of information and some amount that needed to be discovered. And so, we developed together a project, which evaluates, kind of, the better way to get there.
MR. KARLAN: Thank you.
DEPUTY ADMINISTRATOR COLEMAN: I think we are just about out of time now. I will say, from my perspective, thank you, Esther Duflo, so much for joining us here and Dean for this conversation. It really is, I think one of the most important things that we can do in our time here is to try to bring more evidence into our programmatic decision-making. And I love the conversation about plumbers and firefighters. Just before I left GiveDirectly, we tried a very interesting initiative on the plumbing, which was to use, you know, an anonymized cell phone data to identify, for example, in your example of Niger those living in a drought affected area, you can find them just based on cell phone tower data, and then opt in via SMS, to create that plumbing. And then the firefighters actually used this system and deployed it in Togo during COVID. So, you know, I think this is the next frontier of making sure that the plumbing pipes exist, and then the best way to use them to deliver whatever it may be that the goal is.
And I do take away also the ongoing challenge that we have here to bring more evidence to bear. I think it's a – I think there's a lot of internal work – cultural shift, training, accessibility. I know Dean is working on a dashboard to be able to bring more evidence more quickly to people on, you know, use the example of maternal mortality. This is what you want to do, here is send a tried and true tested interventions, whatever it may be that we can get to the point where we know what programs actually have evidence behind them that question that you have asked in the Obama years, but I don't think we can still answer today, which is a challenge. But we're working on it, and we'll get there.
So, with that, I just want to thank you.