|
|
|
|
|
|
|
|
| >> This is USAID >> Intersectoral Partnerships >> ISP Handbooks |
|
A number of organizations have developed or are developing work that will further inform the process of assessing the results of ISPs. This section introduces some of these efforts, all of which are works in progress. Grassroots Development Framework - Inter-American Foundation The Inter-American Foundation (IAF) developed one of the first sets of indicators that measures the results of partnerships. IAF has worked with local NGOs to strengthen community development in Latin America for decades. Rarely, however, were the results of this work recorded. To begin systematically tracking the results of their efforts, IAF began in 1992 to develop a grassroots development framework (GDF) to "collect and assess results data that are not always reflected in more traditional analysis." 21 The GDF attempts to capture both the tangible and intangible results of grassroots development. IAF identified three levels of impact: personal, organizational and societal. Each level is divided into the tangible and intangible aspects of the work that NGOs carry out on those three levels. These six potential areas of impact are portrayed in an inverted cone shape (see Figure 5) that donors and NGOs can fill in. At the personal level standard of living and personal capacity are assessed. At the organizational level, organizational capability and culture are assessed. Finally, at the societal level, the policy environment and community norms are assessed. Each of these six sections contains three to five variables that can be measured by IAF's menu of 45 indicators. For example, when looking at standard of living, four variables are measured: satisfaction of basic needs, knowledge and skill levels, employment and income generation, and acquisition of assets and capacity to save. Indicators are provided for each variable. IAF has also developed a list of questions that can be used when gathering data to document the indicator. The GDF assists development practitioners in measuring the impact of all NGO projects. Thus, in some situations, the framework could apply to projects carried out through ISPs. At the same time, it is important to keep in mind that the framework contains certain strengths and weaknesses. Figure 5: The Inter-American Foundation's Grassroots Development Framework
While individual projects have been monitored based on the GDF, the IAF has yet to operationalize the framework in a way that would permit the organization to analyze the results of its combined efforts in Latin America. In other words, the GDF cannot combine the sum of the individual parts of IAF's work to promote grassroots development. Presently, the IAF is working towards this goal (Rodriguez 1999). DOSA: Measuring Organizational Capacity -- PACT PACT's Discussion-Oriented Organizational Self-Assessment tool (DOSA) is an effort to measure organizational capacity in civil society. With assistance from USAID's Office of Private and Voluntary Cooperation (BHR/PVC) and private voluntary organization (PVO) colleagues, PACT developed this tool to assist PVOs in strengthening their organizational capacity. The tool has been adapted to assess organizational capacity of smaller NGOs in developing countries as well. Specifically, DOSA measures "organizational capacities and consensus levels in six critical areas, and assesses the impact over time of these activities on organizational capacity (benchmarking)."22 Presently, the tool serves not only to assess organizational capacity, but also as a springboard for organizational change. The six critical areas are:
DOSA and similar organizational capacity tools developed by PACT employ self-assessment techniques through facilitated discussion. The DOSA tool measures organizational capacity by forming an internal assessment team (usually comprised of the organization's staff) to discuss a series of questions (outlined in the tool). Each individual scores the discussion questions. PACT has adapted this same methodology to use with individual organizations. The methodology could be applied to measure the effectiveness of an ISP. DOSA was designed as an easy-to-use assessment process that efficiently and effectively assists PVOs and their NGO partners in achieving meaningful, lasting change. PACT has found the tool to be quite effective. The methodology itself has a number of strengths due to the internal evaluation process. Because the organization's staff assesses the organizational capacity, the evaluation is usually more grounded in reality. There is also a higher internal commitment to organizational change. However, for the methodology and tool to work most effectively, participants in the exercise must feel that the process is safe and transparent. SCAT - The World Bank Through the Social Capital Assessment Tool (SCAT), the World Bank is investing resources in the measurement of social capital.23 Measuring social capital is important in measuring the effectiveness of ISPs because increased social capital can be a significant outcome of ISPs. The World Bank's Social Capital Initiative intends to strengthen the methodological and empirical underpinnings for measuring social capital.24 The conceptual framework for the SCAT divides social capital into two levels. The micro-level refers to the potential contribution that horizontal organizations and social networks make to development. The macro-level refers to the institutional context that shapes these organizations and includes formal relationships and structures, such as rules of law, legal frameworks, political regimes, level of decentralization and the level of participation in the policy formulation process. (See Figure 6.) Figure 6: SCAT Conceptual Framework: Levels and Types of Social Capital 25
The SCAT focuses on the micro-level, particularly the way that cognitive and structural social capital interact at the community, household, and institutional levels. Cognitive social capital refers to values (i.e., trust, solidarity, and reciprocity), beliefs, attitudes, behavior and social norms. Structural social capital includes the organizational structure and practices of an organization and is built through horizontal organizations and networks that have collective and transparent decision-making processes, accountable leaders, and practices of collective action and mutual responsibility. The SCAT provides validated indicators that can measure social capital and its relationship to other development indicators in the areas of poverty alleviation, inequality reduction, and economic growth. The World Bank notes that the SCAT represents the start of the creation of a tool to measure capital rather than a definitive methodology. Pilot testing of the SCAT continues to refine the tool. Valuing Partnerships Toolbox -- Prince of Wales Business Leaders Forum's Another work in progress that directly relates to measuring the results of ISPs is the Prince of Wales Business Leaders Forum's (PWBLF) research on valuing partnerships. When complete, this work will be one of the first global and systematic efforts to develop tools to assess the impact of ISPs as a method to promote sustainable development. In order to offer tools for partnership practitioners to measure the results of their ISPs, the PWBLF spent eighteen months holding a series of workshops to explore the topic. They began the process with a large international workshop, followed by a series of regional workshops. In April 1999, a final workshop was held, during which experts from around the world developed learning points and ideas for key tools to be developed and shared. PWBLF is developing a tool box of indicators that measure the impact of ISPs. This tool box should be available in 2000. Civil Society Index -- CIVICUS CIVICUS: World Alliance for Citizen Participation is developing a civil society index to assist in the definition of a strong, healthy civil society. The index will provide an assessment of the health of civil society at the country level and its impact as an effective public actor nationally, regionally, and globally. CIVICUS proposes the following products related to the index:
Knowledge Resource Group -- The World Bank The Knowledge Resource Group's objective is to collect, analyze, link, and disseminate the lessons learned about partnerships involving business, government and civil society. The KRG draws from the existing base of knowledge on partnership building and World Bank experience in developing tools and programs that can hasten the learning process and can increase the sustainability of the partnerships because lessons are being learned and applied. Sharing the lessons of partnership will also facilitate new partnerships. KRG's role is to make information about effective practices in tri-sector partnerships available, through the networks of those involved to assist their efforts to improve their partnerships. The thrust of KRG is outreach and engaging each sector as a source of knowledge and experience as well as a dissemination target. In this way KRG will be able to make a long-term contribution to the general area of partnership building as a vehicle for achieving international development goals.27 New Partnerships Initiative Resource Guide -- USAID Performance indicators, as discussed in the NPI Resource Guide, are used to measure the impact of NPI-related activities in USAID's field missions. NPI pertains to the three sectors of business, civil society, and democratic institutions of governance. The indicators used for measuring partnerships in the NPI framework center around the enabling environment, strategic partnerships, and capacity building. According to the resource guide, it was found that the indicators used by missions to measure NPI focused more on capacity building than on the development of the enabling environment or the establishment of partnerships and linkages. In regard to capacity building, the indicators being used track changes in the mobilization of financial resources and specific categories of an organization's development but not the changes in an organization's diversity or the dissemination of models of organizational development. The objective of NPI is to garner more effective responses at the local level by civil society, business, and the public sector in collaboratively addressing development challenges. In order to gauge whether this objective has been achieved, NPI identifies the following intermediate results (IRs). See the NPI Resource Guide for specific indicators to measure each IR.
The resource guide discusses how it is conceptually difficult to form indicators that directly measure the NPI objective itself. While the indicators being used can track whether the desired results are being achieved with regard to the development issue (because these results are stated in the Strategic Objectives (SOs) of particular missions), they cannot track whether the three sectors of business, civil society, and the state are collaboratively addressing development issues. In order to best measure the NPI objective itself, indicators need to be developed that track the sustainability of the development impact achieved through the approach, the cost-effectiveness of the approach, and the relative pace at which the desired level of impact is reached. Also, in order to determine if these indicators are most appropriate for an NPI approach, they should be used in a non-NPI context as well for comparison. Handbook of Democracy and Governance Program Indicators - USAID While the NPI Resource Guide focuses particular attention on inter-sectoral partnerships, the Handbook of Democracy and Governance Program Indicators focuses on the measurement of USAID's four democracy and governance objectives:
Although the handbook says very little about partnerships and efforts to foster inter-sectoral partnerships, the last two objectives, relating to civil society and the public sector, are relevant to assessing ISPs. The handbook offers a list of "candidate" indicators that can be utilized by USAID missions or that can help them develop other indicators to achieve the Agency's four democracy and governance objectives. It also discusses how to select indicators, how to statistically analyze indicators, and some of the costs involved for each method of data collection. |
|
FOOTNOTES 21 - From Inter-American Foundation web page at http://www.iaf.gov/brouch/brouch.htm. 22 - See http://www.edc.org/INT/CapDev/dosafile/dosintr.htm. 23 - Defined by the World Bank as "institutions, relationships, and norms that affect the quality and quantity of a society's social interactions." See http://www.worldbank.org/poverty/scapital/whatsc.htm. 24 - See http://www.inform.umd.edu/iris/soccap2.html. 25 - This diagram is reprinted with permission from Krishna and Shrader. 26 - See http://www.civicus.org/cindex.html. 27 - See http://www.civicus.org/krg.html. |
 |
|||
|
