The Developmental Evaluation Pilot Activity (DEPA-MERL) will test the effectiveness of a developmental evaluation (DE) approach, as well as accompanying monitoring and evaluation tools and flexible contracting mechanisms, in achieving effective programming for innovative interventions, untested theories of change, and/or implementation in complex contexts.

The Challenge

For complex interventions or new innovations, traditional midterm and end line evaluations can occur too late to aid in programmatic fine tuning; and often help interventions reach their pre-defined outcomes, rather than work on systems change that may require the redefinition of outputs and outcomes.  

The Innovation

Traditional approaches to monitoring and evaluation focus on making existing processes, structures and systems more effective. DE seeks to improve not only program design, but takes into account the entire complex dynamic system in which the program, project or activity is taking place. DE provides an approach to evaluation that is quick, ongoing, and takes an iterative approach to data collection, analysis and feedback that contributes to timely changes throughout the project cycle and allows for system changes as well as changes in targeted outcomes.

The Approach

[[nid:375591]]

Developmental Evaluators are “embedded” within the program, project, or activity to contribute to modifications in program design and targeted outcomes throughout implementation. However, Developmental Evaluators remain independent from implementing staff, providing a third-party perspective to support an adaptive management process. DE does not prescribe a single methodological design, tool, or framework – the evaluation approach taken is based on emerging needs. Methods might include network and outcome mapping, contribution analysis, or other approaches based on information needs. DE is a highly flexible approach and is well-suited for projects under flexible contracting mechanisms in which implementation is likely to change in response to emerging conditions on the ground. 

DEPA-MERL Pilots

DEPA-MERL has piloted four DEs to date to test the effectiveness of the approach. Learn about each pilot and see examples of deliverables and learning products at the links below:

  • Sustained Uptake (2017-2018): This DE worked with seven teams across the U.S. Global Development Lab to catalyze the next generation of breakthrough innovations and their uptake within and beyond USAID. Over 22 months, the DE provided extensive evaluative and adaptive management support to the Lab, providing them with evidence on effective and efficient models for both internal and external sustained uptake. The DE further improved teams’ capabilities of achieving ecosystem-level outcomes, and provided tools to continue this work moving forward. 
  • Bureau for Food Security (2018-2019): As USAID’s Bureau for Food Security (BFS) navigated a structural transformation, BFS identified a need to use learning from current practices to inform the Bureau’s future structures and functions. BFS aimed to use DE to better understand how successful the Bureau’s units were in capturing learning and dealing with challenges related to knowledge management and knowledge sharing. The DE was not fully implemented due to external challenges  and ultimately focused on a narrow set of questions and products.  
  • USAID's Agency-wide Digital Strategy DE (2020-2022): In early 2020, USAID launched its first-ever Digital Strategy in order to align the Agency’s vision for development assistance with the world’s evolving digital landscape. The Technology Division within the Bureau for Development, Democracy, and Innovation is responsible for leading the strategy implementation across the Agency. DEPA-MERL launched a DE in mid-2020 to support four of the Strategy’s 17 implementation initiatives to learn and collaborate better within and across teams. For the first time at USAID, the Developmental Evaluator was embedded remotely into the initiatives due to the covid-19 pandemic. The DE was extended twice, lasting for two years.

 

Additional Resources from DEPA-MERL

  • Developmental Evaluation Case Study Series - This series of case studies gives examples of implemented DEs around the globe, both inside and outside DEPA-MERL. Each case study provides background on the programming, the DE design and implementation approach, the suitability of DE, and lessons learned for implementation. 

    • Boresha Afya (USAID) -  USAID supports the government of Tanzania to strengthen integrated health services through the Boresha Afya program. This DE, conducted between 2017 and 2021, helps USAID generate real-time evidence and support adaptive management practices to improve the quality, efficiency, utilization, and scalability of integrated health services in Tanzania. 

    • Jilinde (Gates Foundation) - Jilinde’s learning laboratory approach integrates DE at multiple levels to help partners scale up and retain a pre-exposure prophylaxis (PrEP) service with its most vulnerable clients. The DE identifies enablers and challenges throughout its 5-year implementation (2017-2022).

    • CMP (U.S. Department of State) - Beginning in 2015, this DE provides ongoing learning for the project, which uses memory to support peacebuilding and reconciliation in post-war Sri Lankan communities. 

    • Jalin (USAID) - Between 2018 and 2022, this DE supports  Indonesia’s Ministry of Health and DAI Global in their efforts to reduce maternal and neonatal mortality from preventable death through co-created, local solutions. 

    • FCF (DEPA-MERL) - From 2016 to 2018, this DE improved coordination between stakeholders from nearly 50 organizations as they implemented a collective impact model to increase the number of children in Cambodia living in safe, nurturing family-based care.

    • Uptake (DEPA-MERL) - In 2017 and 2018, this DE brought together diverse partners to catalyze the next generation of breakthrough innovations to advance USAID’s mission to support partner countries on their journey to self-reliance

Quick Facts

  • Tools: Embedded evaluators, emerging needs tailoring, network mapping, outcome mapping, contribution analysis
  • Funding mechanism: Contract (buy-in option for USAID operating units)
  • Partners: Social Impact (prime), Search for Common Ground, The William Davidson Institute at the University of Michigan
  • Period of Performance: 10/01/2015 – 9/30/2024

Contact the MERLIN team at merlin@usaid.gov 

Image
Developmental Evaluation Pilot Activity (DEPA-MERL)
GCFSI PHOTO/LINLIN LIANG
Tags
DEPA-MERL