Data Sources and Collection Methods

The description of data sources in a PMP should be operationally specific enough to convey where data for each indicator will come from.

Data sources tell us where to look for performance data. The range of sources used in a PMP is often broad and may range from records kept by other organizations to data obtained directly from firms and farms, or from individuals. Knowing the source of a particular type of performance data will help a Mission to identify an appropriate data collection method when preparing a Project MEL plan, Activity MEL Plan, or evaluation Statement of Work (SOW).

When considering sources of information for indicator both the accessibility of a source and the cost of obtaining data from that source are important. In some cases, partner governments may collect data that would be useful for monitoring progress in an area served by a USAID activity, but regulations governing administrative data may preclude its being shared with USAID. In other cases, the cost of obtaining data multiple times over the life of an activity, using the same tools and procedures, may be prohibitive. For this reason, it may be prudent to identify more than one possible source of data for indicators included in a PMP where there is uncertainty about the feasibility of using specific data sources.

• Instrument of various sorts — scales, thermometers, distance measurement devices and other existing continuum, including “number of” and similar quantitative descriptors
• Survey instruments that may include either closed or open-ended questions, but can be forced to remain unchanged over repeat uses, including customer satisfaction surveys and citizen score-cards. USAID's Conducting Mini Surveys in Developing Countries volume is useful aid for this method as is the Epi Info software, which is well suited for surveys of a moderate size.
• Structured observations forms that have the same characteristics as survey instruments, on which USAID's TIPS on Direct Observation provides guidance
• Scales and rating systems in which every position on a continuum is described in a precise manner on which two observers would be highly likely to agree.

An example of this type of scale, which is included in the USAID TIPS on Measuring Institutional Capacity illustrates the general approach USAID uses to monitor changes over time in its annual CSO Sustainability Index report.

For the very reasons that some data collection methods are highly appropriate for gathering performance information in a consistent way over several years, other data collection methods are less suitable, including:

• Unstructured or semi-structured survey and direct observation instruments
• Key informant interviews that rely on rough interview guides and are rarely replicable
• Group interviews where the process or dynamics could affect the data

In addition, while some existing data series remain constant over time, any secondary sources from which USAID is considering collecting performance data, including other international development organizations, warrants a review from this perspective. Record keeping systems used by firms and government agencies, over which USAID lacks control, sometimes change and could result in a broken performance data series.