Conducting Data Quality Assessments

Data Quality Assessments (DQAs) are one of two important ways in which USAID checks on the quality of the performance monitoring information it collects and publically reports. The second way in which USAID staff learn about performance data quality issues are through project audits conducted by the Office of USAID’s Inspector General which sometimes include a review of project performance monitoring. Occasionally a USAID evaluation will also touch on performance data quality, but this occurs less frequently.

Planning for Data Quality Assessments (DQAs) begins with the development of a PMP and continues, at the project level, in a project MEL Plan, as outlined in earlier kit pages on Data Quality and Limitations and Data Quality Assessment Plan. During implementation the focus shifts to the practical steps involved in conducting a DQA.

Who Conducts DQAs?

USAID's November 2012 update of ADS 201 indicates that Missions should not hire an outside expert to assess the quality of their data. Mission staff, usually the technical offices, Monitoring and Evaluation staff, or project/activity implementers, as part of their award, can conduct the assessment, provided that mission staff review and verify DQAs conducted by implementing partners. This may entail site visits to physically inspect records maintained by implementing partners or other partners. This revision of ADS 201 is somewhat more explicit than earlier versions in defining DQAs as an in-house USAID and Implementing Partner responsibility, rather than one that would normally be contracted out to a third party.

Tools for Conducting a DQA

USAID's TIPS on Conducting Data Quality Assessments outlines an eight step process for carrying out a DQA as summarized below. In its November 2012 revision of ADS 201, USAID incorporated a new template for writing up the results of a DQA which can be filled in on line. This USAID Data Quality Assessment Checklist can be accessed on the next page of this kit and completed on your computer, saved or printed out. This checklist helps user assess performance data in light of USAID's five quality standards—validity, integrity, precision, reliability and timeliness. At the end of USAID's checklist are several recommendations for conducting an effective DQA. These recommendations are reproduced below.

Missions benefit from a written report on a DQA that includes specific findings about data quality issues and recommend improvements that might be made. A USAID/Macedonia DQA report, while not new, nevertheless includes a useful structure for presenting DQA issues and recommendations as the section below on the Mission’s indicator for exports illustrates.