Definition
Meta-evaluation is the systematic assessment of an evaluation's quality, essentially, evaluating the evaluation. It examines whether the evaluation process adhered to professional standards (such as the OECD-DAC criteria or UNEG quality standards) and whether its findings are credible, useful, and fit for purpose. Rather than judging the programme being evaluated, a meta-evaluation judges the evaluation itself.
Meta-evaluation serves as a quality assurance mechanism, providing confidence that evaluation findings can be trusted for decision-making. It can be conducted internally by quality assurance staff, externally by independent evaluators, or through peer review processes.
Why It Matters
Meta-evaluation addresses a critical gap in M&E systems: without quality assessment, organisations may make important decisions based on flawed or biased evaluation findings. A poorly conducted evaluation can waste resources, mislead stakeholders, and even cause harm if decisions are based on incorrect conclusions.
For donor compliance, many funders (including USAID and the Global Fund) require evidence that evaluations meet minimum quality standards before accepting findings for accountability purposes. Meta-evaluation provides this assurance systematically rather than through ad hoc judgment.
Beyond compliance, meta-evaluation generates organizational learning about evaluation practices. By identifying recurring quality issues, such as weak methodology, insufficient stakeholder engagement, or untimely reporting, organisations can improve their evaluation procurement, oversight, and use practices over time.
In Practice
Meta-evaluation typically assesses an evaluation against established quality criteria. The OECD-DAC criteria (relevance, coherence, effectiveness, efficiency, impact, sustainability) form the most common framework, though UNEG's four quality standards (utility, feasibility, propriety, accuracy) are also widely used.
Common approaches include:
-
Post-completion review: A designated quality reviewer assesses the final evaluation report and terms of reference against quality criteria, often using a standardized scoring rubric. This is the most common form, typically completed within 1-3 days.
-
External quality assessment: An independent evaluator who did not conduct the original evaluation provides an external judgment, often required for high-stakes evaluations or donor compliance.
-
Peer review: Other evaluators or M&E practitioners review the evaluation, bringing fresh perspective and domain expertise.
-
Real-time quality checks: During multi-phase evaluations, quality checkpoints assess methodology and early findings before the evaluation concludes, allowing for course correction.
Meta-evaluation outputs typically include a quality rating (pass/fail or scored), specific findings about strengths and weaknesses, and recommendations for improving future evaluations. For high-stakes decisions, organisations may require evaluations to meet a minimum quality threshold (e.g., 80% on all criteria) before using findings for resource allocation or programme redesign.
Related Topics
- Evaluation Criteria (DAC), The five criteria used to assess evaluation quality
- Evaluation Quality, Standards and approaches for ensuring credible evaluations
- Utilization-Focused Evaluation, An approach that emphasizes making evaluations useful for intended users
Further Reading
- UNEG Quality Standards for Evaluation, The international professional body's four quality standards.
- OECD-DAC Evaluation Criteria, The five criteria most commonly used for meta-evaluation.
- Chambers, R. (2012). "Do Evaluations Work?", Critical perspective on evaluation quality and use.