|
|
|
|
|
|
|
|
|
|
  |
 |
  |
Evaluation Overview and Methodology
>> Table of Contents >> Evaluation Overview and Methodology Limitations of the Evaluation | Evaluation Team | Structure of the Report
Evaluation Overview and Methodology
At the request of the U.S. Agency for International Development (USAID), PricewaterhouseCoopers LLP conducted an evaluation of the Agency's Cuba Program. The evaluation was conducted under the aegis of the Support for Economic Growth and Institutional Reform (SEGIR) Contract, specifically under the General Business, Trade and Investment component. The evaluation was carried out from February to May 2000.USAID contracted with PricewaterhouseCoopers LLP to:
- assess the effectiveness of the current Cuba Program,
- make recommendations for improving the effectiveness of the Program, and
- assess the compliance of grantees with the conditions of their grants.
A three-member evaluation team conducted in-depth interviews of USAID, State Department and other U.S. executive branch officials who are familiar with the Program, as well as informed congressional staff and experts on Cuba. In addition, the team conducted structured, in-depth interviews of the USAID Cuba Program grantees, examined their work products, and reviewed the extensive information related to their grants on file at USAID. The team also reviewed a variety of press reports and secondary sources related to the Program. Finally, the team interviewed several representatives of think tanks and academic institutions not currently associated with the Program. A list of persons interviewed can be found in Appendix B.
The final version of this report was delivered to the USAID Bureau for Latin America and the Caribbean on July 21, 2000.
Limitations of the Evaluation
While any evaluation is limited by the constraints of time and available resources, the evaluation team faced three hurdles in determining the effectiveness of the Cuba Program:
- Time-delayed Program Impact. Because many Cuba Program activities are designed to lay the groundwork for a transition in Cuba, much of the Program's impact cannot be measured at such an early stage. As Thomas Carothers argues in his recent book, "many of the most important results of democracy programs are psychological, moral, subjective, indirect and time-delayed." 1
- Lack of Cooperation from the Cuban Government. Promoting democracy on its face would appear to be something that should be pursued out in the open. Nevertheless, the repressive environment maintained by the Government of Cuba imposes the necessity of extremely discreet operational methods, which in turn frustrate evaluation of Program activities. The Government of Cuba's antagonism toward USAID's Cuba Program and Program grantees hampers the efforts of the Agency, its partners, and this evaluation team to gather important information pertaining to Program effectiveness. Further, the team believes that Program clients would be placed in jeopardy if it were to engage in client-satisfaction interviews, surveys, end-use reviews, or similar evaluation techniques within Cuba itself. All of the team's work was performed within the United States.
- No Access to Classified U.S. Government Documents. The evaluation team did not have access to classified U.S. Government documents relating to the Cuba Program.
- Varying Location of Activities. The closer a partner's work gets to or inside of Cuba, the more difficult, risky, and costly it is to carry out that work. Similarly, the closer a partner's work gets to or inside of Cuba, the more difficult and more costly it is to evaluate with respect to either efficacy or compliance. The evaluation team has found much of the work of Program partners well beyond the scope of our assessment resources primarily due to lack of access to clients and limited contact with Cuba. Off-island activities (e.g. work in third countries, building or cooperating with U.S.-based NGOs, and information gathering and analysis) are generally easier to carry out, monitor, and evaluate than on-island work. However, off-island work probably has a less direct and less evident impact upon Cuban society and politics as well as a longer lag time for effects to be detected.
Despite these obstacles, the evaluation team was able to carry out its work through the appraisal of Cuba Program work products; in-depth face-to-face interviews with Program administrators, grantees, and other key informants; telephone interviews, examination of Cuba Program grantee files; and a review of relevant secondary sources.
Evaluation Team
The PricewaterhouseCoopers LLP evaluation team was composed of John Booth, Juan Rial, and Chris Siddall. Siddall served as team leader.
- John Booth specializes in political participation, democracy, and democratization in Latin America. For nearly 25 years, he has conducted in-depth research on these issues and written several books and articles focused on democratization in Latin America. Currently, Booth is a professor of political science at the University of North Texas. He has also taught graduate-level courses on democracy and democratization, Latin American politics, revolution and political violence, and models of democracy.
- Juan Rial has 14 years of experience in providing expertise on democracy and governance issues in emerging democracies. His primary areas of focus include democratization, development of civil society, civil-military relations, trade unions, strengthening of judicial and legislative institutions, anti-corruption measures, and civic education. Rial has extensive experience in Latin America, having worked in El Salvador, Guatemala, Honduras, Nicaragua, Panama, the Dominican Republic, Argentina, Brazil, Colombia, Ecuador, Paraguay, Peru, and Uruguay. He served as an advisor to Uruguayan President Julio Maria Sanguinetti on civil-military relations and constitutional reform from 1985 to 1989 and from 1995 to 1997. He has also served as an advisor in Bosnia-Herzegovina, the West Bank and Gaza, Malawi, Mozambique, Angola, Burundi, Equatorial Guinea, Rwanda, and East Timor. Rial is a Permanent Resident of the United States and a citizen of Uruguay and Spain.
- Chris Siddall has 10 years of experience with projects designed to strengthen judicial, legal, and regulatory frameworks for democratic institutions and market-based economies. He is a member of the PricewaterhouseCoopers Integrated Managing for Results team that is working to advance strategic planning, performance measurement, and evaluation methodology throughout the U.S. Agency for International Development.
Structure of the Report
This report includes three sections and two appendices. The first section provides an overview of the USAID Cuba Program, grantee activities, and key Program actors. The second section examines overall Program effectiveness and the compliance of grantees with their grant agreements. The final section of the report provides recommendations that the evaluation team believes would enhance the effectiveness of the Cuba Program. Appendix A briefly describes the activities of individual grantees and highlights their successes as well as obstacles that they have faced during Program implementation. Appendix B lists persons interviewed during the course of the evaluation.
1 Aiding Democracy Abroad: Carnegie Endowment for International Peace: Washington, D.C. 1999, p. 340.
[Return to text]
 |
|||
Last Updated on: April 12, 2004 |
