|
|
|
|
|
|
|
|
|
|
  |
 |
  |
Skip Sourcebook Navigation Planning Programs: Monitoring and Evaluating Performance and Reporting, Using, and Sharing Performance Information
USAID Sourcebook
Results Define
Measure
Core Values Sources FAQ Glossary USAID ADS Updated 10/10/02
Planning to monitor results involves determining:
- Tools (e.g., performance indicators, evaluations, special studies) that will be used to measure progress over time;
- Current situation (performance baseline) and the change in the situation that will signal success (performance target);
- Data collection methods that will be used, the frequency of data collection and the responsibility for data compilation and analysis or interpretation; and
- How data will be used in the appropriate level of decision making, resource allocation, and communicating the results.
1. Performance indicators and their definitions
Define each performance indicator carefully. Indicators may be either quantitative or qualitative. When the activities or anticipated results are expected to affect men and women differently and the gender difference is potentially significant for managing toward sustainable impact, the SO Team must include gender-sensitive indicators.
Be precise about all technical elements of the indicator statement. Consider this indicator: the number of small enterprises receiving loans from the private banking system. How are small enterprises defined -- all enterprises with 20 or fewer employees, or 50 or 100?
What types of institutions are considered part of the private banking sector -- credit unions, government-private sector joint-venture financial institutions?
Include in the definition the unit of measurement. Provide enough detail to ensure that different people at different times, given the task of collecting data for a given indicator, will collect identical types of data. Consider an indicator for the value of exports. Will the value be measured in current or constant terms? In U.S. dollars or local currency?
2. Data source
Identify the data source for each performance indicator. Data sources may include government departments, international organizations, other donors, NGOs, private firms, USAID offices, contractors, or activity implementing agencies. Be as specific about the source as possible, so that the same source can be used routinely. This will ensure consistency and avoid misinterpretations. Consider this: What will happen if a switch is made from estimates of infant mortality rates based on national sample surveys to estimates based on hospital registration statistics? It can lead to false impressions of change.
Plans may refer to needs and means for strengthening the capacity of a particular data source to collect needed data on a regular basis, or for building special data collection efforts into USAID activities.
3. Method of data collection
Specify the method or approach to data collection for each indicator. Provide sufficient detail on the data collection or calculation method to enable it to be replicated. To the extent possible, the data collected should be sex-disaggregated. Consider the trade-offs between primary data collection and reliance on existing secondary data.
- For primary data collection, consider:
- the unit of analysis (individuals, families, communities, clinics, wells);
- data disaggregation needs (by gender, age, ethnic groups, location);
- sampling techniques for selecting cases (random sampling, purposive sampling);
- and techniques or instruments for acquiring data on these selected cases, such as structured questionnaires, direct observation forms, scales to weigh infants).
For secondary data from existing sources, consider that quality may not be as reliable as primary data cuts costs and efforts.
4. Frequency and schedule of data collection
Gather comparable data periodically to measure progress. How frequent the collection should be depends on the data being gathered. For example, collect fertility rate data from sample surveys every few years because of the expense and because changes are slow; collect data on contraceptive distributions and sales from clinics' or other outlets' record systems quarterly.
5. Responsibilities for acquiring data
Assign responsibility to a particular office, team or individual for the timely acquisition of data from one or more sources for each performance indicator.
6. Data analysis plans
Good analytic tools are available. Over time, analysis can be more sophisticated.
- Plan the analysis of performance data for individual indicators or groups of related indicators.
- Identify data analysis techniques and data presentation formats to be used.
- Disaggregated data (by gender, race, age, location, etc.)
- Comparison of actual performance data with:
- a) past performance,
- b) planned or targeted performance, or
- c) other relevant benchmarks.
- Analysis of relationships among performance indicators. For example: How will a set of indicators (if there are more than one) for a particular Strategic Objective (SO) or Intermediate Result (IR) be analyzed to reveal progress? What if only some of the indicators reveal progress? How will cause-effect relationships among SOs and IRs within a results framework be analyzed? How will USAID activities be linked to achieving IRs and SOs?
7. Plans for complementary evaluations
Plan what evaluation efforts, if any, will be needed to complement information from the performance monitoring system. If certain performance targets are not being met, and if the reasons why are unclear, then plan an evaluation.
8. Plans for reporting, using, and sharing performance information
- Plan to use performance information to improve the performance, effectiveness, and design of existing development assistance activities.
- Revise Agency or operating unit strategies where necessary.
- Plan new strategic objectives and/or activities.
- Inform decisions whether to abandon Agency program strategies, strategic objectives or activities that are not achieving intended results.
- Document findings on the impact of development assistance.
- Plan what evaluation efforts, if any, will be needed to complement information from the performance monitoring system.
- Plan, schedule, and assign responsibilities for internal and external reviews, briefings, and reports.
- Clarify what, how and when management decisions will consider performance information.
- Specifically, plan for the following:
- Operating unit performance reviews.
- Assess progress toward achieving SOs and IRs.
- Assess if activities' inputs, outputs, and processes are supporting achievement of IRs and SOs.
- Help plan Annual Report preparation by scheduling tasks and making assignments.
- Plan for reporting and disseminating performance information to key stakeholders, such as host government counterparts, collaborating NGOs, other partners, donors, and customer groups. Use a combination of communication techniques -- reports, oral briefings, videotapes, memos, and newspaper articles.
- Understand which decision-making processes -- budget discussions, programming decisions, evaluation designs/scopes of work, office retreats, management contracts, and personnel appraisals -- benefit from performance information.
To view PDF files, download
the accessible version of Adobe Acrobat Reader.
 |
|||
Last Updated on: October 10, 2002 |
