Causal Inference

What Is Causal Inference?

Causal inference is the process of determining whether an intervention caused observed outcomes by establishing a credible counterfactual and ruling out alternative explanations. It answers the question: "Did our program make a difference, or would these outcomes have occurred anyway?"

Causal inference goes beyond correlation or association. It requires constructing or approximating what would have happened in the absence of the intervention (the counterfactual), then comparing actual outcomes against this alternative reality. The strength of causal claims depends on the credibility of the counterfactual and the extent to which alternative explanations have been systematically ruled out.

Why It Matters

Causal inference is essential when stakeholders need defensible evidence that a program achieved its intended effects. Without it, you can only describe what happened, not what your program achieved.

When a decision requires an estimate of causal effects, choose a design that supports the intended claim and states its assumptions. Correlation alone is insufficient because observed changes may result from:

• Secular trends: broader economic, political, or social forces affecting outcomes independently of your program
• Selection bias: systematic differences between program participants and non-participants that affect outcomes
• External shocks: events like market changes, climate events, or policy shifts that affect all groups
• Maturation: natural changes that occur over time regardless of intervention

When the question is whether your program "made a difference" or "caused improvement," causal inference provides the methodological foundation for answering that question credibly.

In Practice

Causal inference appears across multiple evaluation approaches, each with different strengths and feasibility constraints:

Randomised Controlled Trials (RCTs) create the strongest causal claims through random assignment, ensuring treatment and control groups are statistically equivalent at baseline. Any post-intervention differences can be attributed to the program with high confidence.

Quasi-experimental designs approximate causal inference when randomisation is not feasible. Methods include:

• Propensity score matching - comparing participants with non-participants who have similar observable characteristics
• Regression discontinuity - exploiting arbitrary eligibility cutoffs to create comparable groups
• Difference-in-differences - comparing changes over time between treatment and control groups

Contribution analysis offers an alternative when counterfactual-based methods are impractical. It builds a causal story by gathering evidence that alternative explanations have been ruled out, rather than through direct comparison groups.

Process tracing examines the internal causal mechanisms - whether the expected pathway from activities to outcomes actually occurred as theorised.

The choice of method depends on feasibility, ethics, resources, and the strength of attribution required. Stronger causal claims require more resources but provide greater confidence in program effectiveness.