Meta-Evaluation

What Is Meta-Evaluation?

Meta-evaluation is the systematic assessment of an evaluation's quality - essentially, evaluating the evaluation. It examines whether the evaluation process adhered to professional standards (such as the JCSEE Program Evaluation Standards or relevant UNEG norms and standards) and whether its findings are credible, useful, and fit for purpose. Rather than judging the program being evaluated, a meta-evaluation judges the evaluation itself.

Meta-evaluation serves as a quality assurance mechanism, providing confidence that evaluation findings can be trusted for decision-making. It can be conducted internally by quality assurance staff, externally by independent evaluators, or through peer review processes.

Why Does Meta-Evaluation Matter?

Meta-evaluation addresses a critical gap in M&E systems: without quality assessment, organizations may make important decisions based on flawed or biased evaluation findings. A poorly conducted evaluation can waste resources, mislead stakeholders, and even cause harm if decisions are based on incorrect conclusions.

Agree the evaluation-quality review process, evidence, and any acceptance conditions in the terms of reference or applicable funding agreement. A quality score alone is not authorization to rely on unsupported findings.

Beyond compliance, meta-evaluation generates organizational learning about evaluation practices. By identifying recurring quality issues - such as weak methodology, insufficient stakeholder engagement, or untimely reporting - organizations can improve their evaluation procurement, oversight, and use practices over time.

How Is a Meta-Evaluation Conducted?

Select a named evaluation-quality framework and record its edition and relevant sections. The JCSEE Program Evaluation Standards, third edition (2010), cover utility, feasibility, propriety, accuracy, and evaluation accountability. UNEG's Norms and Standards for Evaluation (2016) is a distinct framework addressing matters such as utility, credibility, independence, impartiality, ethics, and evaluation processes. The six OECD DAC intervention criteria concern the merit of the intervention; they are not a substitute for judging the quality of the evaluation itself.

Common approaches include:

• Post-completion review: A designated quality reviewer assesses the final evaluation report and terms of reference against quality criteria, often using a standardized scoring rubric. This is the most common form, typically completed within 1-3 days.

• External quality assessment: An independent evaluator who did not conduct the original evaluation provides an external judgment, often required for high-stakes evaluations or donor compliance.

• Peer review: Other evaluators or M&E practitioners review the evaluation, bringing fresh perspective and domain expertise.

• Real-time quality checks: During multi-phase evaluations, quality checkpoints assess methodology and early findings before the evaluation concludes, allowing for course correction.

Report evidence-based quality findings and limitations alongside any agreed rating. Separate not applicable, not evidenced, and inadequate practice. An illustrative percentage threshold is not a professional standard, and a high total must not offset a critical ethics or evidence failure.

Method Source

JCSEE, Program Evaluation Standards, third edition (2010). This is methodological guidance, not a funding requirement.