Systematic Review

What Is a Systematic Review?

A systematic review is a rigorous, structured approach to identifying, appraising, and synthesizing all available evidence on a specific evaluation question using explicit, reproducible methods. Unlike traditional narrative reviews, systematic reviews follow a pre-specified protocol that minimizes bias through comprehensive literature searches, explicit inclusion/exclusion criteria, and systematic quality assessment of included studies.

The approach is particularly valuable when you need to answer questions like "What interventions have been shown to work for X outcome?" or "What does the evidence say about the effectiveness of approach Y?" Systematic reviews are often conducted by research institutions, evaluation departments, or donor organizations seeking to inform program design and policy decisions based on the best available evidence rather than anecdote or selective citation.

Why It Matters

Systematic reviews transform scattered research findings into actionable evidence for M&E practitioners. They provide a defensible basis for program decisions by answering what has actually worked in similar contexts, rather than relying on what might work based on theory or limited examples. This evidence aggregation reduces the risk of replicating failed approaches and helps identify which intervention components are most critical for success.

For donor organizations and large programs, systematic reviews support knowledge management by creating a curated, up-to-date understanding of what the evidence base contains - and more importantly, what it doesn't. This gap identification informs where new evaluation investment is most needed and prevents redundant research efforts.

In Practice

A well-executed systematic review follows these key steps:

1. Develop a protocol specifying the review question, search strategy, inclusion/exclusion criteria, and quality assessment methods before beginning the search.

2. Conduct comprehensive searches across multiple databases (e.g., Google Scholar, PubMed, World Bank Open Knowledge Repository, evaluation repositories) using predefined search terms and strategies.

3. Screen and select studies based on explicit criteria, typically involving two independent reviewers to minimize selection bias.

4. Appraise study quality using standardized tools appropriate to the study designs included (e.g., risk of bias assessments for randomized trials, quality checklists for qualitative studies).

5. Extract and synthesize data from included studies, either narratively or through meta-analysis when studies are sufficiently similar in design and outcomes.

6. Assess certainty of evidence using frameworks like GRADE to communicate confidence in the findings.

Systematic reviews are resource-intensive and require specialized skills in research methodology. They are most appropriate when making high-stakes decisions about program approach, when a topic has accumulated substantial research, or when an organization needs to establish an evidence base for advocacy or policy work.