Theory-Based Evaluation

Theory-based evaluation tests whether the program's theory of change held up in practice. It uses evidence-at-each-step reasoning rather than relying solely on a counterfactual comparison.

What Does Theory-Based Evaluation Do?

A theory-based evaluation walks the results chain from activity to outcome and tests each link with evidence. For every "if X then Y" claim in the theory of change, the evaluation asks three questions:

• Did X actually happen, at the intended quality and intensity?
• Was Y observed among the people the program reached?
• Is there evidence the X-to-Y link operated, rather than the two events simply co-occurring?

Alternative explanations are then surfaced and systematically considered. Did another intervention also target the same outcome? Were participants already on an improving trajectory? Could the result reflect a measurement artifact rather than real change? A credible theory-based evaluation assesses these explanations against the evidence and states which remain plausible and what uncertainty remains.

The output is a chain of reasoning, not a single effect estimate. Each link in the theory of change is rated on the strength of evidence supporting it, and the overall verdict is built up from those ratings.

Common Variants

Four approaches used in theory-based inquiry are:

• Contribution analysis. Assesses how much the program contributed to outcomes that typically have multiple causes. Built for messy, multi-actor contexts.
• Realist evaluation. Pawson and Tilley's context-mechanism-outcome framework. Asks "what works, for whom, in what circumstances, and why?" Useful when a program is expected to behave differently across settings.
• Process tracing. Small-n, in-depth case analysis that tests specific causal claims with evidence tests (hoop tests, smoking-gun tests, doubly-decisive tests). Strong for single-case or few-case attribution questions.
• Outcome harvesting. Starts from observed outcomes and traces back to establish program contribution. Useful when outcomes were not fully specified in advance.

When Does Theory-Based Evaluation Work Best?

Theory-based evaluation is the right fit when:

• The program is complex and outcomes have multiple plausible causes.
• A credible counterfactual is unavailable for the evaluation question. Assess the feasible comparison designs; a single site alone does not establish this limitation.
• Sample sizes are small and statistical power is unachievable.
• The theory of change is strong, but quantitative outcome measurement is limited.
• The evaluation question is about mechanism and "how did this work," not just effect size.

Governance, advocacy, policy-change, and systems-strengthening programs may raise these questions. For a pilot at a single site, choose the approach according to the evaluation question and available evidence.

How Does Theory-Based Evaluation Fit into a Proposal?

Choose theory-based evaluation when the questions concern mechanisms, contribution, implementation and context. It may complement an experimental or quasi-experimental design, or support a carefully bounded contribution argument when an identified counterfactual is unavailable. Explain the method, evidence and rival explanations for the actual commissioning purpose; do not infer acceptance or a prescribed method from the funding source.

Two proposal moves strengthen the case. First, name the specific method (contribution analysis, realist evaluation, process tracing) rather than writing "theory-based" as a generic label. Second, explain why a counterfactual is infeasible or inappropriate before introducing the alternative. Build the budget from the scope, evidence requirements and fieldwork rather than assuming a typical cost or savings relative to a quasi-experimental design.

Common Mistakes

• Weak theory of change going in. Theory-based evaluation can only be as good as the theory it tests. If the logic model is a vague diagram with unstated assumptions, the evaluation will be vague too. Invest in the theory of change first.
• Assuming theory-based evaluation automatically estimates net effects. Theory-based inquiry may form part of an impact evaluation and explain causal mechanisms and contribution. It does not automatically estimate a net effect; that requires a design and evidence suited to the causal question. Choose based on the decision you need to support, not the budget line.