Side-by-Side Comparison
| Factor | Outcome Harvesting (OH) | Most Significant Change (MSC) |
|---|---|---|
| Developed by | Ricardo Wilson-Grau (2015) | Rick Davies and Jessica Dart (2005) |
| Starting point | Observed outcomes, traced back to attribution | Stories of change collected from participants |
| Driven by | Evaluator/facilitator, with informant input | Participants and selection panel members |
| Output format | Structured outcome statements: description, significance, contribution, evidence | Narrative stories ranked by stakeholder panels |
| Typical timeline | 4-8 weeks for the evaluation cycle | 3-6 months for a full cycle with selection panels |
| Rigor comes from | Substantiation process: verifying each outcome with multiple sources | Selection panels: multi-level filtering by stakeholders who justify their choices |
| Participant voice | Moderate (informant interviews, not participant-authored) | High (participants author stories in their own words) |
| Accountability reporting fit | Strong (structured outputs work for donor reports) | Moderate (stories need structuring for donor outputs) |
| Best for | Complex/emergent programs, advocacy, systems change, innovation | Participatory programs, empowerment, community-led, unanticipated-change contexts |
| Facilitator skill required | High (substantiation design, evidence triangulation) | High (selection panel design, facilitation across levels) |
| Cost | Moderate: 30-60% of a standard summative evaluation | Moderate to high: 40-80% of a standard summative evaluation |
For the formal method definitions, see outcome harvesting and most significant change.
Shared Foundations
Both methods share three foundational features that distinguish them from counterfactual-based evaluation approaches.
Both are theory-based, not counterfactual-based. Neither tries to construct a "what would have happened without the program" comparison. Instead, both ask "what change is observable and how is the program implicated?" This makes both appropriate when counterfactual design is infeasible (small-n, ethical issues, emergent outcomes) or inappropriate (complex programs where no control group is meaningful). See counterfactual and theory-based evaluation.
Both are qualitative-primary. Both work in the language of narrative and description rather than statistics and effect sizes. Both produce outputs that can be supplemented by quantitative data but do not depend on quantitative measurement for their primary findings.
Both emphasize learning alongside accountability. Neither method is a pure compliance tool. Both are designed to produce findings the program can actually use, including unanticipated outcomes that conventional M&E would miss.
These shared foundations mean both methods fit naturally with adaptive management, learning agendas, and theory of change reasoning. See theory of change for the broader framework.
When Outcome Harvesting Wins
Outcome Harvesting is the better choice when the following apply.
The program is complex or emergent. Advocacy programs, governance reform work, systems-change initiatives, and innovation projects produce outcomes that were not fully predictable at design. OH handles this well: it starts with what actually happened rather than what was planned, and traces back to program contribution. Programs with highly specific theories of change (service delivery, targeted behavior change) may find OH less distinctive than MSC or conventional methods.
Timeline is tight. A typical OH cycle is 4-8 weeks: identify outcomes, draft statements, substantiate with informants, finalize. MSC cycles are 3-6 months because of selection panels. If the evaluation is feeding a donor report due in 3 months, OH is feasible; MSC probably is not.
Multiple actors contributed to outcomes. OH's contribution-analysis framing is well-suited to programs where the program shares credit with others (advocacy coalitions, policy reform work, multi-stakeholder initiatives). The outcome statement format (description, contribution, evidence) forces honesty about the program's specific share of contribution.
Accountability output is the primary deliverable. OH produces structured outcome statements that can be tabulated, counted, and aggregated. A donor report with "28 documented outcomes, 18 with strong substantiation" works. MSC's story format is harder to tabulate without additional synthesis work.
When Most Significant Change Wins
Most Significant Change is the better choice when the following apply.
Participant voice is central to the program's theory of change. Empowerment programs, localization-led programs, community-driven development, and participatory programs have explicit commitments to centering participant perspectives. MSC operationalizes this through participant-authored stories and participant-led selection. OH can include participant voice but does not require it structurally.
Unanticipated outcomes are expected. MSC's open-ended story prompt (typically "In the past X months, what was the most significant change you observed in Y?") invites participants to name outcomes the evaluator did not anticipate. OH is better at structuring known outcomes; MSC is better at surfacing unknown ones.
The evaluation has time for selection panels. MSC's distinguishing rigor feature is the multi-level selection panel process: stories collected, then field-level panels select, then management-level panels select, then stakeholder/board panels select. Each level must justify its selection. This takes 3-6 months minimum and is core to the method. Skipping or compressing selection undermines MSC.
Learning is the primary purpose. MSC stories move people. They work well in internal learning events, external communications, and advocacy with audiences who respond to narrative. If the evaluation output is meant to change how internal staff think about the program or how external audiences perceive it, MSC stories do this better than OH's structured statements.
Using Them Together
Some programs combine OH and MSC. Common combined workflow:
- Story collection (MSC phase): solicit change stories from participants and frontline staff using MSC prompts. Collect 40-100 stories.
- Selection (abbreviated MSC phase): field-level and program-level panels select 20-30 most significant stories. Single-level instead of multi-level to save time.
- Outcome harvesting from stories (OH phase): rather than going to informants cold, use the selected MSC stories as the starting point. Draft outcome statements from each story (description, significance, contribution, evidence).
- Substantiation (OH phase): verify each outcome statement with at least one additional source beyond the original story author.
- Synthesis: organize outcome statements into thematic clusters, produce report.
Combined cycle: 3-4 months. Cost: roughly 1.3-1.6x either method alone. Best when you need both rich participant voice (MSC strength) and structured accountability output (OH strength). Overkill when either alone would suffice.
Outcome Harvesting Process
The standard OH process has six steps, developed by Wilson-Grau.
- Design (1-2 weeks): define the harvest question (what kinds of outcomes matter), the time frame, the informants to contact. Typical harvest question: "What changes in the practices or policies of key actors has the program contributed to in the past 24 months?"
- Gather outcome descriptions (2 weeks): review program documents, interview informants, collect candidate outcomes. Each candidate is a draft outcome statement.
- Formulate outcome statements (1 week): refine each outcome into the standard format: description (what changed, when, where, who), significance (why this matters), contribution (how the program is implicated).
- Substantiate (1-2 weeks): verify each outcome statement with at least one source beyond the original informant. This is where OH rigor lives; unsubstantiated outcomes are flagged or dropped.
- Analyze (1 week): cluster outcomes by theme, identify patterns, draft findings.
- Support use (1 week): present findings, support the program team in drawing decisions from them.
Total typical cycle: 6-9 weeks. Skilled facilitator-led. Budget for substantiation is critical; an OH without substantiation is a list of claims.
Most Significant Change Process
The standard MSC process has eight steps, developed by Davies and Dart.
- Establish domains of change (2 weeks): identify 3-5 domains the program is trying to influence. Domains are broad categories (e.g., "changes in how women participate in community decisions"), not specific indicators.
- Collect stories (4-6 weeks): ask participants and stakeholders for stories of the most significant change they have observed in each domain. Standard prompt: "Looking back over the last X months, what do you think was the most significant change in [domain]?"
- Field-level selection (1 week): field teams select the most significant story per domain, with written justification for the selection.
- Program-level selection (1 week): program-level panel reviews field selections, selects most significant story per domain, again with justification.
- Stakeholder-level selection (1 week): higher-level panel (board, management, donor if appropriate) selects most significant story.
- Feedback loops (2 weeks): share selected stories and selection reasoning with participants, field staff, and program teams. This is where MSC's participatory learning happens.
- Verification (optional, 2 weeks): verify key claims in selected stories with secondary sources.
- Synthesis: report on selected stories, selection reasoning, and thematic patterns.
Total typical cycle: 3-6 months. Selection panel design is the core craft; bad selection design produces MSC that feels like "favorite stories" rather than rigorous qualitative evaluation.
Sector Examples
Health: Advocacy coalition on tobacco control, Southern Africa
A regional coalition working on tobacco-control policy advocacy used Outcome Harvesting for its 3-year evaluation. OH was the right fit because: outcomes were policy-level changes contributed to by multiple coalitions and government actors; pre-specified indicators would have missed the policy wins that happened in unexpected domains; timeline for donor reporting was 8 weeks. The harvest produced 34 outcome statements across 4 categories (national policy, sub-national regulation, industry compliance, public awareness). 22 were substantiated with government documents, news reports, or partner interviews. The donor accepted OH as appropriate for advocacy evaluation; the coalition used the output to plan the next phase.
Education: Girls' education empowerment program, South Asia
A 4-year girls' education program used Most Significant Change because participant voice was central to the program's empowerment theory of change. Story collection spanned 600 girls and 300 family members across 60 communities. Field-level selection panels (community committees) chose 20 stories per community; program panel narrowed to 60; stakeholder panel (board plus external reviewers) selected 12 as most significant. The process took 5 months. Selected stories revealed an unanticipated outcome: mothers reporting their own changed behavior in supporting older daughters' education. This outcome had not been in the original theory of change and prompted a program design adjustment for year 5.
Governance: Public accountability program, West Africa
A 3-year governance program combined OH and MSC. MSC collected 140 stories from citizens, frontline officials, and civil society partners. Field and program panels selected 45; one stakeholder panel did final selection. Then the evaluation team converted the 45 selected stories into 38 OH outcome statements (some stories clustered into single outcomes), substantiated 30 through secondary sources, and produced a synthesis report. Combined cycle was 4.5 months. The combined approach gave the donor accountability-ready outputs and gave the program participatory learning material for its next phase.
Livelihoods: Pastoralist livelihoods program, Sahel
A livelihoods program chose Outcome Harvesting over MSC because its theory of change was relatively specific (livelihood diversification, savings group adoption, livestock vaccination uptake) and the program management needed fast evaluation turnaround for a donor review. OH produced 24 outcome statements across 3 program components in 7 weeks. 19 were substantiated through program records, community interviews, and market data. The evaluation team flagged 5 outcomes where substantiation was weak and recommended further monitoring. OH was the right call for this program; MSC would have produced richer stories but not fit the timeline.
Common Mistakes
Mistake 1: Choosing OH or MSC as a shortcut for impact evaluation. Both methods are distinct tools for different questions, not cheap alternatives to RCT or quasi-experimental design. Programs that pick OH or MSC because "impact evaluation is too expensive" often end up with evaluations that cannot answer the impact question they were trying to avoid. See RCT vs quasi-experimental for when counterfactual is actually required.
Mistake 2: Underbudgeting facilitation. Both methods depend on skilled facilitator discipline. OH without rigorous substantiation is a list of claims; MSC without rigorous selection panel design is favorite-stories. Experienced facilitators cost more; the methods do not work with junior facilitators who have not done this before.
Mistake 3: Skipping or compressing selection panels in MSC. The multi-level selection process is what distinguishes MSC from ordinary story collection. Cutting selection levels, or running selection panels without adequate justification discipline, removes the method's core rigor.
Mistake 4: Claiming rigor without substantiation in OH. Every outcome statement in an OH report should have substantiation evidence. Unsubstantiated outcomes should be flagged or dropped. Reports that present all outcomes as equally verified undermine credibility.
Mistake 5: Treating OH and MSC as interchangeable. They answer different questions. OH asks "what changed and how did the program contribute?" MSC asks "what mattered most to participants about what changed?" These are related but distinct questions, and picking the wrong method for the question produces unsatisfying output.
Mistake 6: Failing to frame donor-facing purpose. OH and MSC are not accepted equally by all donors. Proposals using either method should explicitly frame why counterfactual design is not suitable (complexity, ethics, small-n, emergent outcomes), how rigor is achieved (substantiation, selection panels, triangulation), and what the expected output structure is. Unframed, either method can come across as insufficient to traditional donors.
OH vs MSC Selection Checklist
Run through this during evaluation planning to decide which method fits.
Program characteristics:
- Program type: service delivery / advocacy / systems change / empowerment / innovation (flag applicable)
- Theory of change: specific and pre-defined or emergent and unpredictable
- Participant voice centrality: core to theory of change or supplementary
- Multi-actor contribution: single-program or multi-stakeholder
Evaluation constraints:
- Timeline available: 4-8 weeks (OH feasible) vs 3-6 months (MSC feasible)
- Budget for facilitation: experienced OH or MSC facilitator secured
- Donor acceptance: method pre-cleared with donor if required
Output requirements:
- Primary output format needed: structured outcome statements (OH) or narrative stories (MSC)
- Accountability vs learning: which is primary purpose
- Audience for findings: donor report, internal learning, external communication
Mixed-method decision:
- If OH + MSC combined: budget supports 1.3-1.6x cost vs single method
- Combined timeline of 3-4 months available
For the broader methodology selection, see how to choose evaluation methodology and qualitative vs quantitative vs mixed methods. For the theory-based evaluation framework both methods sit within, see theory-based evaluation. For an AI-assisted step-by-step workflow, see the Evaluation Plan playbook.